h2o Flow UI not working - python-2.7

I recently upgraded to H2O-3.11.0.3820. My web based flow ui is not working.
When I go to the link, i get a light blue colored screen with all the options that I used to get earlier, missing.
Unable to find anything relevant on stackoverflow. Any one else facing similar issue? Any help will be much appreciated!
Checking whether there is an H2O instance running at http://localhost:54321..... not found.
Attempting to start a local H2O server...
; OpenJDK 64-Bit Server VM (Zulu 8.17.0.3-win64) (build 25.102-b14, mixed mode)
Starting server from C:\Users\shekh\Anaconda2\lib\site-packages\h2o\backend\bin\h2o.jar
Ice root: c:\users\shekh\appdata\local\temp\tmpvrxqvd
JVM stdout: c:\users\shekh\appdata\local\temp\tmpvrxqvd\h2o_shekh_started_from_python.out
JVM stderr: c:\users\shekh\appdata\local\temp\tmpvrxqvd\h2o_shekh_started_from_python.err
Server is running at http://127.0.0.1:54321
Connecting to H2O server at http://127.0.0.1:54321... successful.
-------------------------- ------------------------------
H2O cluster uptime: 03 secs
H2O cluster version: 3.11.0.3820
H2O cluster version age: 5 days
H2O cluster name: H2O_from_python_shekh_vdbwfl
H2O cluster total nodes: 1
H2O cluster free memory: 51.56 Gb
H2O cluster total cores: 0
H2O cluster allowed cores: 0
H2O cluster status: accepting new members, healthy
H2O connection url: http://127.0.0.1:54321
H2O connection proxy:
H2O internal security: False
Python version: 2.7.12 final
-------------------------- ------------------------------

As you discovered, install a stable release of H2O-3.
Nightly builds are number 3.ODD.y.z
Stable builds are numbered 3.EVEN.y.z
(So 3.11.0.3820 is a random bleeding edge build, not a stable build.)

Related

openstack neutron-gateway - error: "could not add network device eno2 to ofproto (Device or resource busy)"

I performed a cluster node installation using this guide [OpenStack Charms Deployment Guide].(https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/install-maas.html), where the type of network is a Flat network and the components used are:
Maas
Juju
Openstack
My lab is composed by following devices:
1 IBM System 3540 M4 Maas (500GB HDD - 8GB RAM - 1 Nic)
1 IBM System 3540 M4 Juju (500GB HDD - 8GB RAM -1 Nic)
4 IBM System 3540 M4 Openstack (500GBx2 HDD - 16GB RAM - 2 Nic)
1 Palo Alto Network Firewall
Public Network 10.20.81.0/24 - Private Network 10.0.0.0/24
Maas: 10.20.81.1
Juju: 10.20.81.2
Openstack 10.20.81.21-24
Gateway 10.20.81.254
Instance: 10.0.0.9 - 10.20.81.215 (floating)
network plan
10.20.81.0/24
+-------------+
Firewall
10.20.81.254
+-------------+
|
+-------------------------------------------------------------+
Switch
vlan81 vlan81 vlan81
+-------------------------------------------------------------+
| | || | | |
+--------------+ +------------+ +------------------+
|Maas+Juju |Juju Gui| |Openstack
|10.20.81.1 |10.20.81.2 |10.20.81.21-24
+--------------+ +-------------+ +------------------+
|
+--------------------------------------------+
Private Subnet-1 Public Subnet-2
10.0.0.0/24 10.20.81.0/24
+---+----+--+ +----+------+
| | +----+ |
| | | | |
| +--------+ VR +-------------+
| | |
+--+-+ +----+
| |
| VM |
| .9 |
| |
+----+
On my lab, the nodes for Openstack present two eth interface, the first one (eno2) the single external network used as floating IP, then the other one (eno3) for the private network.
On Juju gui I've that:
neutron-gateway:
bridge-mappings: physnet1:br-ex
data-port: br-ex:eno2
neutron-api:
flat-network-providers: physnet1
I've opened this post https://ask.openstack.org/en/question/119783/no-route-to-instance-ssh-and-ping-no-route-to-host/ to resolve the problem about the Ping and Ssh connection to my instance, but during same check I've seen this issue on neutron-gateway:
error: "could not add network device eno2 to ofproto (Device or resource busy)"
Maybe that is the cause of my first issue, but I don't understand how I can fix it.
$:juju ssh neutron-gateway/0
Welcome to Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-46-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Tue Mar 19 16:07:19 UTC 2019
System load: 0.64 Processes: 409
Usage of /: 5.7% of 273.00GB Users logged in: 0
Memory usage: 13% IP address for lxdbr0: 10.122.135.1
Swap usage: 0% IP address for br-eno2: 10.20.81.21
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
3 packages can be updated.
0 updates are security updates.
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ovs-vsctl show output
ubuntu#os-compute01:~$ sudo ovs-vsctl show
6f8542aa-45d7-409d-8787-8983f3c643eb
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "eno2"
Interface "eno2"
error: "could not add network device eno2 to ofproto (Device or resource busy)"
Port br-ex
Interface br-ex
type: internal
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Port "gre-0a145118"
Interface "gre-0a145118"
type: gre
options: {df_default="true", in_key=flow, local_ip="10.20.81.21", out_key=flow, remote_ip="10.20.81.24"}
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port br-int
Interface br-int
type: internal
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port "tapb0b04b07-8f"
tag: 2
Interface "tapb0b04b07-8f"
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port "tap2354468c-88"
tag: 4
Interface "tap2354468c-88"
Port "tap6d2b2fe0-47"
tag: 4
Interface "tap6d2b2fe0-47"
ovs_version: "2.10.0"
juju status
$:juju status
Model Controller Cloud/Region Version SLA Timestamp
openstack maas-cloud-controller maas-cloud 2.5.1 unsupported 22:10:17Z
App Version Status Scale Charm Store Rev OS Notes
ceph-mon 13.2.4+dfsg1 active 3 ceph-mon jujucharms 31 ubuntu
ceph-osd 13.2.4+dfsg1 active 3 ceph-osd jujucharms 273 ubuntu
ceph-radosgw 13.2.4+dfsg1 active 1 ceph-radosgw jujucharms 262 ubuntu
cinder 13.0.2 active 1 cinder jujucharms 276 ubuntu
cinder-ceph 13.0.2 active 1 cinder-ceph jujucharms 238 ubuntu
glance 17.0.0 active 1 glance jujucharms 271 ubuntu
keystone 14.0.1 active 1 keystone jujucharms 288 ubuntu
mysql 5.7.20-29.24 active 1 percona-cluster jujucharms 272 ubuntu
neutron-api 13.0.2 active 1 neutron-api jujucharms 266 ubuntu
neutron-gateway 13.0.2 active 1 neutron-gateway jujucharms 256 ubuntu
neutron-openvswitch 13.0.2 active 3 neutron-openvswitch jujucharms 255 ubuntu
nova-cloud-controller 18.0.3 active 1 nova-cloud-controller jujucharms 316 ubuntu
nova-compute 18.0.3 active 3 nova-compute jujucharms 290 ubuntu
ntp 3.2 active 4 ntp jujucharms 31 ubuntu
openstack-dashboard 14.0.1 active 1 openstack-dashboard jujucharms 271 ubuntu
rabbitmq-server 3.6.10 active 1 rabbitmq-server jujucharms 82 ubuntu
Unit Workload Agent Machine Public address Ports Message
ceph-mon/0 active idle 1/lxd/0 10.20.81.4 Unit is ready and clustered
ceph-mon/1 active idle 2/lxd/0 10.20.81.8 Unit is ready and clustered
ceph-mon/2* active idle 3/lxd/0 10.20.81.5 Unit is ready and clustered
ceph-osd/0 active idle 1 10.20.81.23 Unit is ready (1 OSD)
ceph-osd/1 active idle 2 10.20.81.22 Unit is ready (1 OSD)
ceph-osd/2* active idle 3 10.20.81.24 Unit is ready (1 OSD)
ceph-radosgw/0* active idle 0/lxd/0 10.20.81.15 80/tcp Unit is ready
cinder/0* active idle 1/lxd/1 10.20.81.18 8776/tcp Unit is ready
cinder-ceph/0* active idle 10.20.81.18 Unit is ready
glance/0* active idle 2/lxd/1 10.20.81.6 9292/tcp Unit is ready
keystone/0* active idle 3/lxd/1 10.20.81.20 5000/tcp Unit is ready
mysql/0* active idle 0/lxd/1 10.20.81.17 3306/tcp Unit is ready
neutron-api/0* active idle 1/lxd/2 10.20.81.7 9696/tcp Unit is ready
neutron-gateway/0* active idle 0 10.20.81.21 Unit is ready
ntp/0* active idle 10.20.81.21 123/udp chrony: Ready
nova-cloud-controller/0* active idle 2/lxd/2 10.20.81.3 8774/tcp,8775/tcp,8778/tcp Unit is ready
nova-compute/0 active idle 1 10.20.81.23 Unit is ready
neutron-openvswitch/1 active idle 10.20.81.23 Unit is ready
ntp/2 active idle 10.20.81.23 123/udp chrony: Ready
nova-compute/1 active idle 2 10.20.81.22 Unit is ready
neutron-openvswitch/2 active idle 10.20.81.22 Unit is ready
ntp/3 active idle 10.20.81.22 123/udp chrony: Ready
nova-compute/2* active idle 3 10.20.81.24 Unit is ready
neutron-openvswitch/0* active idle 10.20.81.24 Unit is ready
ntp/1 active idle 10.20.81.24 123/udp chrony: Ready
openstack-dashboard/0* active idle 3/lxd/2 10.20.81.19 80/tcp,443/tcp Unit is ready
rabbitmq-server/0* active idle 0/lxd/2 10.20.81.16 5672/tcp Unit is ready
Machine State DNS Inst id Series AZ Message
0 started 10.20.81.21 nbe8q3 bionic Openstack Deployed
0/lxd/0 started 10.20.81.15 juju-26461e-0-lxd-0 bionic Openstack Container started
0/lxd/1 started 10.20.81.17 juju-26461e-0-lxd-1 bionic Openstack Container started
0/lxd/2 started 10.20.81.16 juju-26461e-0-lxd-2 bionic Openstack Container started
1 started 10.20.81.23 pdnc7c bionic Openstack Deployed
1/lxd/0 started 10.20.81.4 juju-26461e-1-lxd-0 bionic Openstack Container started
1/lxd/1 started 10.20.81.18 juju-26461e-1-lxd-1 bionic Openstack Container started
1/lxd/2 started 10.20.81.7 juju-26461e-1-lxd-2 bionic Openstack Container started
2 started 10.20.81.22 yxkyet bionic Openstack Deployed
2/lxd/0 started 10.20.81.8 juju-26461e-2-lxd-0 bionic Openstack Container started
2/lxd/1 started 10.20.81.6 juju-26461e-2-lxd-1 bionic Openstack Container started
2/lxd/2 started 10.20.81.3 juju-26461e-2-lxd-2 bionic Openstack Container started
3 started 10.20.81.24 bgqsdy bionic Openstack Deployed
3/lxd/0 started 10.20.81.5 juju-26461e-3-lxd-0 bionic Openstack Container started
3/lxd/1 started 10.20.81.20 juju-26461e-3-lxd-1 bionic Openstack Container started
3/lxd/2 started 10.20.81.19 juju-26461e-3-lxd-2 bionic Openstack Container started
iptables
Any suggestions please. I am still unable to solve the problem. Thanks.
update 26/03/19:
On Juju gui I've that:
neutron-gateway:
bridge-mappings: physnet1:br-ex
data-port: br-ex:eno2
neutron-api:
flat-network-providers: physnet1
Before to make the deploy of Openstack I've changed data-port from br-ex:eno2 to br-ex:eno3
neutron-gateway:
bridge-mappings: physnet1:br-ex
data-port: br-ex:eno3
The issue on eno2 is been resolved but the ping to instance is still present.

Intermittent DNS issues while pulling docker image from ECR repository

Has anyone facing this issue with docker pull. we recently upgraded docker to 18.03.1-ce from then we are seeing the issue. Although we are not exactly sure if this is related to docker, but just want to know if anyone faced this problem.
We have done some troubleshooting using tcp dump the DNS queries being made were under the permissible limit of 1024 packet. which is a limit on EC2, We also tried working around the issue by modifying the /etc/resolv.conf file to use a higher retry \ timeout value, but that didn't seem to help.
we did a packet capture line by line and found something. we found some responses to be negative. If you use Wireshark, you can use 'udp.stream eq 12' as a filter to view one of the negative answers. we can see the resolver sending an answer "No such name". All these requests that get a negative response use the following name in the request:
354XXXXX.dkr.ecr.us-east-1.amazonaws.com.ec2.internal
Would anyone of you happen to know why ec2.internal is being adding to the end of the DNS? If run a dig against this name it fails. So it appears that a wrong name is being sent to the server which responds with 'no such host'. Is docker is sending a wrong dns name for resolution.
We see this issue happening intermittently. looking forward for help. Thanks in advance.
Expected behaviour
5.0.25_61: Pulling from rrg
Digest: sha256:50bbce4af6749e9a976f0533c3b50a0badb54855b73d8a3743473f1487fd223e
Status: Downloaded newer image forXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/rrg:5.0.25_61
Actual behaviour
docker-compose up -d rrg-node-1
Creating rrg-node-1
ERROR: for rrg-node-1 Cannot create container for service rrg-node-1: Error response from daemon: Get https:/XXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/v2/: dial tcp: lookup XXXXXXXX.dkr.ecr.us-east-1.amazonaws.com on 10.5.0.2:53: no such host
Steps to reproduce the issue
docker pull XXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/rrg:5.0.25_61
Output of docker version:
(Docker version 18.03.1-ce, build 3dfb8343b139d6342acfd9975d7f1068b5b1c3d3)
Output of docker info:
([ec2-user#ip-10-5-3-45 ~]$ docker info
Containers: 37
Running: 36
Paused: 0
Stopped: 1
Images: 60
Server Version: swarm/1.2.5
Role: replica
Primary: 10.5.4.172:3375
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint
Nodes: 12
Plugins:
Volume:
Network:
Log:
Swarm:
NodeID:
Is Manager: false
Node Address:
Kernel Version: 4.14.51-60.38.amzn1.x86_64
Operating System: linux
Architecture: amd64
CPUs: 22
Total Memory: 80.85GiB
Name: mgr1
Docker Root Dir:
Debug Mode (client): false
Debug Mode (server): false
Experimental: false
Live Restore Enabled: false
WARNING: No kernel memory limit support)

Access to PCF app from another machine

I'm new with the PCF and trying to deploy a simple web-app. I've installed the cf CLI, pcfdev and pushed my app to pcf:
cf push test-ui -b staticfile_buildpack
...
name: test-ui
requested state: started
instances: 1/1
usage: 256M x 1 instances
routes: test-ui.local.pcfdev.io
last uploaded: Thu 23 Aug 13:09:04 +03 2018
stack: cflinuxfs2
buildpack: staticfile_buildpack
start command: $HOME/boot.sh
state since cpu memory disk details
#0 running 2018-08-23T10:09:17Z 0.0% 5.3M of 256M 25M of 512M
So, now, I can access my test app by link test-ui.local.pcfdev.io from the same machine where I've started my pcf instance. But I don't know how to access to this app from another device in the same network.
Could someone tell me what I should do to open my test app from another device in the same network as my local machine?
Since PCFDev was installed in your local machine. I believe you cannot access the cloud foundry apps outside of your machine unless some networking stuff should be done that provide access to other machines.
I've found a solution: using the reverse proxy to redirect request from my local port app's url:
(using nginx as example):
server {
listen 8090;
server_name pcf-rp;
location / {
proxy_pass http://test-ui.local.pcfdev.io;
}
}

H2O + HDFS (Cloudera)

We have a Cloudera cluster up and running with an h2o instance although it appears to be running off h2o.jar (which as I understand it--please correct me if incorrect) is the stand-alone h2o. I can connect, but it will not load any files from our HDFS. (all of this i can see via 'ps' on edge node.
So I started an instance with h2odriver.jar
java -jar /path/to/h2odriver.jar -nodes 2 -mapperXmx 5g -output /my/hdfs/dir
I get several output/callback addresses:
[Possible callback IP address: 10.96.243.46:33728]
[Possible callback IP address: 127.0.0.1]
Using mapper->driver callback IP address and port: 10.96.243.46:33728
So I fire up python and try and connect (same thing happens if I use 10.96.243.46):
>>>h2o.connection(ip='127.0.0.1', port='33728')
and get
'Connecting to H2O server at http://127.0.0.1:33728..... failed.
H2OConnectionError: COuld not estalich link to the H2O cloud http://127.0.0.1:33728 after 5 retries
...
Failed to establish a new connection:[Errno 111] Connection refused',))`
Thing is on my screen with the H2O jar/java job I can see:
`MapperToDriverMessage: Read invalid type (G) from socket, ignoring...
MapperToDriverMessage: read: Unknown Type `
I cannot figure out how to launch h2o in cluster mode and have it access our hdfs system or even connect. I can connect to the h2o.jar version, but that sees no hdfs (it can see the filesystem of the edgenode). What is the proper way to launch H2O so that it can see the attached HDFS system (We are running Cloudera 5.7 in a enterprise environment, Python is 3.6, H2O is 3.10.0.6 and I know we have a ton of firewalls/security-- i beleive we are setup through LDAP
You are correct that h2o.jar is meant to be the standalone version of H2O which is not meant for connecting to HDFS.
Using the appropriate h2odriver.jar for your particular hadoop distribution is the way to go.
The correct beginner instructions can be found here:
go to http://www.h2o.ai/download/
choose H2O "Latest Stable Release"
choose tab "Install on Hadoop"
It says to run the following command:
hadoop jar h2odriver.jar -nodes 1 -mapperXmx 6g -output hdfsOutputDirName
[ Note this is "hadoop jar", not "java -jar" as written in the question. ]
You should see output like this:
Determining driver host interface for mapper->driver callback...
[Possible callback IP address: 172.16.2.181]
[Possible callback IP address: 127.0.0.1]
...
Waiting for H2O cluster to come up...
H2O node 172.16.2.188:54321 requested flatfile
Sending flatfiles to nodes...
[Sending flatfile to node 172.16.2.188:54321]
H2O node 172.16.2.188:54321 reports H2O cluster size 1
H2O cluster (1 nodes) is up
(Note: Use the -disown option to exit the driver after cluster formation)
Open H2O Flow in your web browser: http://172.16.2.188:54321
(Press Ctrl-C to kill the cluster)
Blocking until the H2O cluster shuts down...
Then point your web browser to the place where it says to "Open H2O Flow in your web browser".
(The other addresses in the output are diagnostics, and not for end users.)
In this case, the python connection command would be:
h2o.connect(ip = '172.16.2.188', port = 54321)
I recommend going to Flow in a web browser, start importing a file by typing in "hdfs://", and seeing if autocompletion works. If it does, your HDFS connection is working.

SSH Connection disconnected

I'm a student from korea
first, i'm sorry about my low level english :)
I'm make a web service using AWS + nginx + django
I connect to AWS instance(ubuntu) using SSH protocol
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-74-generic x86_64)
* Documentation: https://help.ubuntu.com/
System information as of Sat Apr 30 07:03:51 UTC 2016
System load: 0.0 Processes: 105
Usage of /: 23.8% of 7.74GB Users logged in: 0
Memory usage: 14% IP address for eth0: 172.31.17.137
Swap usage: 0%
Graph this data and manage this system at:
https://landscape.canonical.com/
Get cloud support with Ubuntu Advantage Cloud Guest:
http://www.ubuntu.com/business/services/cloud
21 packages can be updated.
17 updates are security updates.
Last login: Sat Apr 30 07:03:52 2016 from 210.103.124.253
pyenv-virtualenv: no virtualenv has been activated.
and
manage.py runserver --settings=abc.settings.production
So everyone can access my web service!
but.... after 30miniute
the SSL connection is broken itself....
export this message
packet_write_wait: Connection to 52.69.xxx.xxx: Broken pipe
and nobody can't access my web service...
so... my web site can't access when my computer was power off, none SSL connection...
I want everyone can access my web service 24/7
please give me a method thank you :)
When you want to run a command that continues after your current shell terminates, you should use the nohup command to launch it.
That causes the process to be detached from its initial parent shell so it is not killed when the parent terminates.