tensorboard with AWS s3 bucket - amazon-web-services

On linux, with tensorflow/tensorboard 2.9.1, I'm trying to run tensorboard with a folder on a S3 bucket. I'm well authentified: aws s3 ls is working. I'm working behind a proxy.
But when I'm running:
AWS_LOG_LEVEL=1 AWS_DEFAULT_PROFILE=myprofile AWS_REGION=eu-west-1 tensorboard --logdir=s3://mybucket/tensorboard-output/
I'm having the following issue:
2022-07-11 10:01:18.596111: E tensorflow/c/logging.cc:40] Curl returned error code 77 - Problem with the SSL CA cert (path? access rights?)
2022-07-11 10:01:18.596228: E tensorflow/c/logging.cc:40] HTTP response code: -1
Resolved remote host IP address:
Request ID:
Exception name:
Error message: curlCode: 77, Problem with the SSL CA cert (path? access rights?)
0 response headers:
2022-07-11 10:01:18.596256: W tensorflow/c/logging.cc:37] If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
2022-07-11 10:01:18.596267: W tensorflow/c/logging.cc:37] Request failed, now waiting 0 ms before attempting again.
It seems to be an issue with Curl and certificate issue. I already tried to specify the environment variable CURL_CA_BUNDLE= to the right ca.crt, but that's not helping, it seems it's not checking this environment variable...
Any idea ? Thank you in advance!

Related

SSL certificate verification failed when trying to build GraalVM compiler with mx

I'm trying to build the GraalVM compiler using the mx build tool. I've Python 3.10.4 and Java 17.0.2 in my PATH. However, when I run mx I get the following message:
Downloading COMMONS_MATH3_3_2 from ['https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.2/commons-math3-3.2.jar', 'https://search.maven.org/remotecontent?filepath=org/apache/commons/commons-math3/3.2/commons-math3-3.2.jar']
Error downloading from https://repo1.maven.org/maven2/jline/jline/2.14.6/jline-2.14.6.jar to /Users/cesarsv/.mx/cache/JLINE_c3aeac59c022bdc497c8c48ed86fa50450e4896a/jline.jar: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
WARNING: ** If behind a firewall without direct internet access, use the http_proxy environment variable (e.g. "env http_proxy=proxy.company.com:80 mx ...") or download manually with a web browser.
Error downloading from https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.2/scala-reflect-2.12.2.jar to /Users/cesarsv/.mx/cache/SCALA_REFLECT_12_fa13c13351566738ff156ef8a56b869868f4b77e/scala-reflect-12.jar: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
...
The error seems to be related to an SSL certificate validation when downloading the dependencies necessary for the GraalVM build with mx. There is no clear solution for this issue in the GitHub repo of the used tools.
It seems that mx uses python to fetch the artifacts that it needs for the build from external repositories. Python 3.7 and above don't have any SSL certificates activated by default. Therefore, the scripts used by mx can't validate any SSL connections.
So I activate SSL in Python manually by creating and running the file install_certificates.command and it solved my problem.

SSL Certificate error: [SSL: CERTIFICATE_VERIFY_FAILED] when using aws client in windows 10

Recently I am gettin an error when, for instance, listing data from Amazon S3:
aws s3 ls
SSL validation failed for https://s3.eu-west-1.amazonaws.com/ [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)
I have noticed that the company I work for has added a ZScaler Client Connector. It seems that this client is causing the error. I wonder if someone could give a hint about how to solve this issue.
Upps It seems that I found the solution myself.
Check this site:
https://www.shellhacks.com/aws-cli-ssl-validation-failed-solved/
I downloaded ZScaler certificate and then pointed from config:
$ cat ~/aws/.config
[default]
ca_bundle = /data/ca-certs/whatevername.pem
I was getting crazy, I hope it helps someone else.
Solved it this way:
Locate certifi bundle (cacert.pem) location python -m certifi
export AWS_CA_BUNDLE="[full path to cacert.pem]" from step 1
verify

sslv3 alert handshake failure when accessing AWS Lambda Function using Open edX and API Gateway

So I have a running image of Bitnami's Open edX Ficus 3.1 release in AWS EC2 with ubuntu 14.04. This platform is coded in python and uses openssl and requests library. The problem at hand is when I'm making an HTTP request through the platform to AWS Lambda function (it's HTTPS secured) I get the following when logging the python error:
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:661)
How can I resolve it? I tried updating requests (0.14.2) and openssl (OpenSSL 1.0.2k 26 Jan 2017) library but it's still the same error.

Ejabbed with letsencrypt certificate

I'm trying to get Ejabberd to work with letsencrypt certificates on centos7.
I keep getting errors about the certificates not being signed by a known CA.
I have created the certificates by certbot, and I joined the privkey and fullchain files to single file.
All c2s connections work fine, but s2s connections don't.
When starting Ejabberd I see the following relevant log entries:
[warning] <0.606.0>#ejabberd_pkix:check_ca_dir:386 CA directory /etc/ssl/certs doesn't contain hashed certificate files; configuring 'ca_path' option might help
[warning] <0.606.0>#ejabberd_pkix:mk_cert_state:240 certificate from /opt/ejabberd/conf/xxxx.pem is invalid: certificate is signed by unknown CA
Connections to for example draugr.de generate the following entries:
[info] <0.793.0>#ejabberd_s2s_in:handle_auth_failure:206 (tls|<0.792.0>) Failed inbound s2s EXTERNAL authentication draugr.de -> XXXXX.net (::FFFF:89.163.212.45): unable to get local issuer certificate
I hope someone can help me out, thanks!
[EDIT 2020 may]
It looks like ejabberd now has automatic acme support (meaning it can request the certificate on its own from letsencrypt). So what you read below is obsolete.
As of 2018 november,
Merely installing letsencrypt using certbot is enough click here to see how. Ejabberd uses the provided certificates.
Note that you may need to register multiple subdomains for some strict jabber clients to work properly.
conference.yourjabberdomain.com
pubsub.yourjabberdomain.com
upload.yourjabberdomain.com
yourjabberdomain.com
or install a wildcard certificate from letsencrypt
sudo certbot certonly --server https://acme-v02.api.letsencrypt.org/directory --manual --preferred-challenges dns -d *.yourjabberdomain.com
I think there is a rule in the ejabberd config file ejabberd.yml that allows this to happen
certfiles:
- "/etc/letsencrypt/live/*/*.pem"
I was able to solve it myself finally but i am out of office the next few days and can't get you the exact configuration to solve it.
But if I recall correctly I downloaded the CA bundle here https://curl.haxx.se/docs/caextract.html and there was some configuration parameter for ejabberd to use this CA bundle in stead of the default one.
Hope it helps you.
If it is working for c2s and not working for s2s then it looks like the s2s block in configuration file is not updated with certfile. I believe you have something like this for c2s:
port: 5222
ip: "::"
module: ejabberd_c2s
starttls: true
certfile: 'CERTFILE'
protocol_options: 'TLSOPTS'
Similarly your s2s block should have:
port: 5269
ip: "::"
module: ejabberd_s2s_in
starttls: true
certfile: 'CERTFILE'
protocol_options: 'TLSOPTS'
max_stanza_size: 131072
shaper: s2s_shaper

How to deploy to Cloud Foundry with Docker Image from Private Docker Registry and Self-Signed SSL certificate?

How to add self-signed certificate to Cloud Foundry (PCFDev), so I would be able to deploy with Docker Image from private Docker Registry?
For this example I'm using PCFDev:
user#work:(0):~/Documents/$ cf push app-ui -o nexus-dev/app/app-ui:latest
Creating app app-ui in org pcfdev-org / space pcfdev-space as user...
OK
Creating route app-ui.local.pcfdev.io...
OK
Binding app-ui.local.pcfdev.io to app-ui...
OK
Starting app app-ui in org pcfdev-org / space pcfdev-space as user...
Creating container
Successfully created container
Staging...
Staging process started ...
Failed to talk to docker registry: Get https://nexus-dev/v2/: x509: certificate signed by unknown authority
Failed getting docker image by tag: Error parsing HTTP response: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>\r\n<body bgcolor=\"whit
e\">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>The plain HTTP request was sent to HTTPS port</center>\r\n<hr><center>nginx/1.10.0 (Ubuntu)</center>\r\n</body>\r\n</html>\r\n"
Staging process failed: Exit trace for group:
builder exited with error: failed to fetch metadata from [app/app-ui] with tag [latest] and insecure registries [] due to Error parsing HTTP response: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>
400 The plain HTTP request was sent to HTTPS port</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>The plain HTTP request was sent to HTTPS port</center>\r\n<hr><center>nginx/1.10.0
(Ubuntu)</center>\r\n</body>\r\n</html>\r\n"
Exit status 2
Staging Failed: Exited with status 2
Destroying container
Successfully destroyed container
FAILED
Error restarting application: StagingError
TIP: use 'cf logs app-ui --recent' for more information
You can start pcfdev with -r option,
e.g.
cf dev start -r host.pcfdev.io:5000
from Insecure Docker Registries