I am using GKE for Jupyterhub deployment. I followed the step given in Documentation but I am facing kernel connection issue.
I am using Helm Chart version 1.2.
Jupyterhub version 1.5
I saw a similar issue posted here: https://github.com/jupyter/notebook/issues/2664, I changed the tornado version but issue still persist.
In the console when I checked It was showing Websocket connection Failure
Related
I am trying to establish a LDAP connection (<cfldap>) from within a Docker image of Coldfusion 2021. It would be hard to post any relavent code here simply because it would expose our AD tree, however, the same code I am trying to run works just fine from an installed copy of CF2021 on a linux server.
The reason for using a Docker image (vs. install) in this instance, is an attempt to setup a local development (MACOS) environment. So far, everything seems to be working great with the exception of LDAP calls.
Note: I have successfully run a ldapsearch call from a bash shell within the container.
The error I'm getting:
An error has occurred while trying to execute query :Could not resolve
a valid ldap host
The Docker image repo I pulled from:
https://hub.docker.com/r/adobecoldfusion/coldfusion
Update: I've just noticed CF version differences between the server that isn't having the problems:
Linux Version: 2021,0,01,325996 (installed a few weeks ago non-Docker)
Local MACOS: 2021,0,02,328618 (Docker)
Update 2: We've installed a fresh ColdFusion 2021 Docker image on a Linux box directly connected to our network and we are still seeing this issue. This narrows the issue down to Adobe Cold Fusion 2021's interaction with Docker and it's ability to do <CFLDAP>.
Update 3: 10-13-2021 - It would appear the CF team is aware of this, has confirmed the bug and is looking into it.
Update 4: 11-12-2021 - This bug is related to the version of Java running within the Docker image. "Adobe CF support suggested updating to JAVA SE 11.0.13 (LTS) inside the docker container" which has worked when tested. Expect CF to solve the problem in future Docker CF2021 releases.
Seeing error below :
"upstream connect error or disconnect/reset before headers. reset reason: connection failure" when tried with below steps:
have created AKS cluster and deployed ASP.NET core docker container. Have installed Istio as per instructions below:
https://learn.microsoft.com/en-us/azure/aks/servicemesh-istio-install?pivots=client-operating-system-windows
Have deployed ASP.NET core docker container in default namespace with istio-injection enabled. Here below is the virtual service and gateway yaml files.
After all this when trying to access http://52.224.26.59/Home/View where 52.224.26.59 is external ip of istio-ingressgateway getting error as "upstream connect error or disconnect/reset before headers. reset reason: connection failure".
I have resolved this issue.
It's because of an issue with an older version of .NET Core version.
Once I updated with .NET Core 3.1, I was able to deploy and access .NET Core WebAPI.
Regarding other issues, there are more challenges and road is not clear with Istio and .NET core combination.
Ambari version: 2.2.2.18
HDP stack: 2.4.3
OS: centos 7.3
Issue description:
Ambari-server can't communicate with Ambari agent. I can see below error in the ambari-agent logs:
ERROR 2017-09-18 06:35:34,684 NetUtil.py:84 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)
ERROR 2017-09-18 06:35:34,684 NetUtil.py:85 - SSLError: Failed to connect. Please check openssl library versions.
I am facing this issue recently and it appears this can be replicated consistently after the instances are restarted. (I am using EC2 instances).
I am able to register agent nodes successfully, install HDP cluster, run yarn jobs etc.. no problem at all. Once i restart my instances, I see this problem.
There are some solutions already posted for this problem like:
Downgrade the Python from 2.7 to lower. This is a known problem of
Ambari with Python 2.7
Control the certificate verification by disabling it.
Set "verify = disable"; under /etc/python/cert-verification.cfg
I don't want to play with Python as it can disrupt lot many things like Cassandra, yum package manager etc...
Second work around is very much easy and it works well!
Now comes my question :- Is it safe to disable the certificate verification in Python ? i.e. by setting property verify = disable
Generally, it's a bad idea. If somebody has access to port on server that is used for agent-server communication (8443 if I'm not mistaken), he can register as agent and get all your cluster configs&passwords. Or classic man-in-the-middle attack would allow to do the same by reading your unencrypted traffic. A bit more difficult attack would allow to send commands to agents (probably with root permissions).
Your issue sounds like you reprovisioned your ambari-server host, and left old ambari-agent instances running, or maybe your certificates became outdated? At first connection to ambari-server, agents generate certificates and send to server. Server signs these certificates with it's own key, so now server-agent connection is encrypted. Did you try to remove old certificates and restart server&agents as suggested here?
How did we investigate this issue and What solution we adopted:
Investigation Details:
Downgrading to Python 2.6 is not feasible as there are OS dependencies and as per Suggestion from 'Dmitriusan' in the previous comment, it's not a good idea to disable certificate verification in Python.
We use AWS EC2
With Python 2.7, JDK 1.8 and Cent OS 7.2 there is no issue. Everything is smooth.
With Python 2.7, JDK 1.8 and Cent OS 7.3 and Centos 7.4 we are seeing this issue.
Issue which I have reported here, is with respect to Centos 7.3 and with Centos 7.4 Issue is slightly different. Certificate verification fails while adding nodes to the cluster itself.
Downgrading from centos 7.3 to 7.2 is not straight forward. And AWS EC2 market place provides Centos 7.0 Image and when we create instance from this image, it applies security and patch updates resulting in Centos 7.3.
We can create our own Image of Centos 7.2 from existing servers but, It's always good to be with the latest update for the OS for security reasons.
To describe it shortly, we had workarounds but not a solution.
Solution which we adopted:
After series of tests, we decided to upgrade to Centos 7.4, HDP-2.6.3.0, and Ambari 2.6.0.0
With Centos 7.4 and Ambari Version 2.6.0.0, we don't see this issue even though I have 'Python 2.7.5' installed.
So this looks to be an Issue with Ambari
Older version of Ambari (2.4.2) does not recognize the force TLS configuration. We upgraded Ambari to 2.6.2 and heart beat started working.
I'm working on a project that I haven't touched in about 4 months. Before everything on the deploy was working fine, but now I'm getting an error when trying to deploy an update.
Failed to pull Docker image amazon/aws-eb-python:3.4.2-onbuild-3.5.1: Pulling repository amazon/aws-eb-python time="2016-01-17T01:40:45Z" level="fatal" msg="Could not reach any registry endpoint" . Check snapshot logs for details. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03build.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
In the eb-activity log, it further states [CMD-AppDeploy/AppDeployStage0/AppDeployPreHook/03build.sh] : Activity execution failed, because: Pulling repository amazon/aws-eb-python before repeating what was shown in the UI.
The original was using a Preconfigured Docker 64bit Debian jessie v1.3.1 running Python 3.4. I've tried upgrading to the latest, which is version 2.0.6, but it never completes (don't need to get into specifics of that error, separate issue and I'd like to stay on 1.3.1 if possible). I've also tried upgrading to the latest 1.x but it has the same result of upgrading to 2.0.6.
Any ideas, or anything else I should be looking for clues?
Docker Hub has deprecated pulls from Docker clients on 1.5 and earlier. Make sure that your docker client version is at least above 1.5. See https://blog.docker.com/2015/10/docker-hub-deprecation-1-5/ for more information.
I am trying view logs of my running application on Bluemix using : "cf logs my-cool-app" command (CF version 6.11) .
If fails with :
FAILED
Loggregator endpoint missing from config file
Anyone seen this issue?
The problem appears to stem from the use of the 6.11 codebase for CF CLI and the current version of CloudFoundry that Bluemix is running. Good news is that an upcoming upgrade will alleviate the problem. We're investigating potential workarounds.
This is just an issue with the CF CLI version 6.11.