Certificate issue on sam deploy - amazon-web-services

I am fairly new to AWS Lambda. I am playing around with a project that I am trying to deploy using the AWS SAM CLI. Below is the command I use:
sam deploy --s3-bucket com.nilay.bucket \
--stack-name HelloWorldLambdaJava \
--capabilities CAPABILITY_IAM
This initially failed for certificate verification issue with ssl related to cloudformation.us-east-1.amazonaws.com certificate. After some googling I circumvented this by exporting the certificate to my mac, converted it to .pem format and created a variable AWS_CA_BUNDLE. Now the deploy fails for another url (s3.amazonaws.com?) for the same certificate issue. How can I add this certificate to the certifcate bundle. It seems like the variableAWS_CA_BUNDLE` should really take a truststore as the value, but all the documentation that I see for this has a .pem file listed in it.
The sam deploy command doesn't allow --no-verify-ssl flag as the AWS CLI command does.

I did two things:
A) The first problem was solved for me from the following link. It was an issue using PIP and accessing AWS services.
SSL CERTIFICATE_VERIFY_FAILED in aws cli
Unfortunately python requests do not use any operating system's CA trust store. https://github.com/requests/requests/issues/2966
You have to set REQUESTS_CA_BUNDLE and AWS_CA_BUNDLE environment variables
https://github.com/bloomreach/s4cmd/issues/111#issuecomment-406839514
I'm accessing AWS from my corporate network. I have no issues when connecting from home on my own computer.
The solution for me is to get the root certificate used when making connections and to save it locally somewhere as a .PEM file.
Then create two local environment variables and point it to the .PEM file. Run these commands to set the environment variables (or do it manually):
setx AWS_CA_BUNDLE "C:\Users\UserX\Documents\RootCert.pem"
setx REQUESTS_CA_BUNDLE "C:\Users\UserX\Documents\RootCert.pem"
B)
The other thing I did was to update the Python certifi package. I then appended the cacert.pem file with the contents of the RootCert.pem that I downloaded.
C:\Python\Python38\Lib\site-packages\certifi\cacert.pem

Just to explain how can you generate the required file (tipically for your corporate network).
On your PC with git installed, using git shell with command (also work from VSCode Git bash terminal). Git also installs openssl so no wories ....
in terminal (git bash) type
echo | openssl s_client -showcerts -servername s3.eu-central-1.amazonaws.com:443 -connect s3.eu-central-1.amazonaws.com:443 2>/dev/null
then grab all parts
-----BEGIN CERTIFICATE-----
xxx
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
xxx
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
xxx
-----END CERTIFICATE-----
including that header and footer to have a whole certificate chain and save to ca-bundle.pem file
After that modify your aws config file
C:\Users\YOURNAME_HERE.aws\config
[default]
region = eu-central-1
output = yaml
ca_bundle = C:/aws/ca-bundle.pem

Related

After upgrading AWSCLIV2, aws-cli broke [duplicate]

I installed AWS CLI on the Windows server 2007 32bit.
aws --version
aws-cli/1.8.8 Python/2.7.9 Windows/2008Server
I configure aws cli using keys
Once I run below command to test AWS S3, I get this SSL error:
aws s3 ls
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
Please help to get rid of this basic error.
If you want to use SSL and not have to specify the --no-verify-ssl option, then you need to set the AWS_CA_BUNDLE environment variable. e.g from PowerShell:
setx AWS_CA_BUNDLE "C:\Users\UserX\Documents\RootCert.pem"
The PEM file is a saved copy of the root certificate for the AWS endpoint you are trying to connect to. To generate it, first export the certificate in DER format (For details on how to do this, see here). Then run the following command to convert to the PEM format:
openssl x509 -inform der -in "C:\Users\UserX\Documents\RootCert.der" -out RootCert.pem
If you are using Powershell and not bash, then you will need to first install openssl.
For a full list of environment variables supported by the AWS CLI, see here
use this option with your cmd
"--no-verify-ssl"
Not sure if it's related to to the OP's issue, however, one of our devs had this issue this morning, turned out he was using Fiddler (on Windows), to debug other issues. After stopping Fiddler (which was intercepting https traffic), the issue was resolved.
I had the same issue on Windows 10. It happens to be due to the aws cli not reading the internet proxy setting from the Windows registry. Fixed same error by setting the environment variables HTTP_PROXY and HTTPS_PROXY to the corporate internet proxy. Hope it helps somebody!
Mine was resolved with:
pip install awscli --force-reinstall --upgrade
I ran into a similar issue on Mac OSX in the company/corporate network.
If you don't know the proxy URL Get it from your company's network administrator and configure with the following commands.
Linux, macOS, or Unix
$ export HTTP_PROXY=http://proxy.example.com:1234
$ export HTTPS_PROXY=https://proxy.example.com:1234
Windows
$ set HTTP_PROXY=http://proxy.example.com:1234
$ set HTTPS_PROXY=https://proxy.example.com:1234
More information
I added the certificate to C:\Program Files\Amazon\AWSCLIV2\awscli\botocore\cacert.pem and it resolved the problem.
My issue was our company's VPN. It worked after I disconnected from VPN
AWS already posted a clean solution for this, here it is:
Instead of hacking your system now the CLI supports you passing it a .pem file with the CA chain for it to communicate with your proxy:
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.html#tshoot-certificate-verify-failed
To fix this, instruct the AWS CLI where to find your companies .pem file using the ca_bundle configuration file setting, --ca-bundle command line option, or the AWS_CA_Bundle environment variable.
Problem most likely caused by corporate proxy. In my case I was running the commands on AWS CLI behind proxy server and was getting certificate error.
So to get around this I added --no-verify-ssl flag. Though this is a bad idea, I used this as a temporary solution to get the job done until it is resolved by the network team.
I believe this option would have been tried already but just putting it here for everyones reference:
when you have proxy added to your ec2 machines and it is in private subnet with a S3 vpc-endpoint attached. I was getting the same error.
Bypassing the proxy using no_proxy for the bucket as per : https://aws.amazon.com/premiumsupport/knowledge-center/connect-s3-vpc-endpoint/
didn't help me and was still failing with the same error.
the only catch here was we need to add endpoint url which is s3.ap-southeast-2.amazonaws.com as below and it worked for me:
export NO_PROXY=169.254.169.254,s3.ap-southeast-2.amazonaws.com
169.254.169.254 is used to access instance role credentials in my case.
I had a similar issue and solved it by setting the proxy as follows:
$ set HTTP_PROXY=http://proxy.example.com:1234
$ set HTTPS_PROXY=https://proxy.example.com:1234
Linux:
$ export AWS_CA_BUNDLE="/data/ca-certs/ca-bundle.pem"
Windows:
PS C:\> setx AWS_CA_BUNDLE C:\data\ca-certs\ca-bundle.pem
$ aws s3 ls --ca-bundle "/data/ca-certs/ca-bundle.pem"
For me ec2 instance date was incorrect, after changing the date and time, fixed the problem.
Simply rebooted the ec2 instance
When you use a AWS CLI command, you receive a "[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed" error message. This is caused by the AWS CLI not trusting your proxy's certificate due to factors such as your proxy's certificate being self-signed, with your company set as the Certification Authority (CA). This prevents the AWS CLI from finding your companies CA root certificate in the local CA registry.
To fix this, instruct the AWS CLI where to find your companies .pem file using the ca_bundle configuration file setting, --ca-bundle command line option, or the AWS_CA_Bundle environment variable.
Please refer https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.html#tshoot-certificate-verify-failed
aws configure set default.ca_bundle <your CA file>
I agree with above answers, do the following
1- Remove your cli and install latest cli
2- check the certificate exist: C:\Program Files\Amazon\AWSCLIV2\botocore\cacert.pem
3- if it doesn't exist remove the cli and go to: C:\Program Files\ and remove Amazon
4- Install cli latest version it should work.
5- Try testing with your VPN connected
use the following option to overcome the ssl certification issue.
aws s3 ls --no-verify-ssl

AWS CLI - [SSL : CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1056)

Am trying to use AWS-CLI to retrieve aws elasticbeanstalk details, but am getting the following error.
Error message:
C:\abdul>aws elasticbeanstalk describe-environments --environment-name myenvname
SSL validation failed for https://elasticbeanstalk.us-east-1.amazonaws.com/ [SSL
: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate
in certificate chain (_ssl.c:1056)
Note:
I can work without any issues when I try to retrieve my EC2 details,
C:\abdul>aws ec2 describe-instances --instance-ids 'i-xxxxxxxxxxxxxx'
Above command works without any issues, I get the above error only when I try "elasticbeanstalk" commands.
Note:
I have all the necessary certificates required in place.
Thanks in advance.
I found my way to this post while Googling. In my case, the error message I received was:
SSL validation failed for https://ec2.us-west-2.amazonaws.com/ [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091)
I found this blog which told me to add an Environment Variable called AWS_CA_BUNDLE whose value was a path pointing to the CA Cert file (which I had saved on my local machine after requesting it from our corporate network team). Once I added that environment variable, I was able to run my AWS CLI commands successfully!
I had the same issue. This is how I resolved it.
Run below command first
$export REQUESTS_CA_BUNDLE=/path/to/company/certificate.crt
And then run AWS cli command
aws elasticbeanstalk describe-environments --environment-name myenvname
Steps to get this working in macOS/Linux
Download the Corporate Self-Signed Certificates using OpenSSL
openssl s_client -showcerts -verify 5 -servername ec2.us-west-2.amazonaws.com -connect ec2.us-west-2.amazonaws.com:443 < /dev/null | awk '/BEGIN/,/END/{ if(/BEGIN/){a++}; out="cert"a".crt"; print >out}' && for cert in *.crt; do newname=$(openssl x509 -noout -subject -in $cert | sed -n 's/^.*CN=\(.*\)$/\1/; s/[ ,.*]/_/g; s/__/_/g; s/^_//g;p').pem; mv $cert $newname; done
Create a bundle.pem by concatenating all the files fetched from the first command.
cat ec2_us-west-2_amazonaws_com.pem company_intermediate.pem company_root.pem >bundle.pem
Make it available in AWS_CA_BUNDLE environment variable.
export AWS_CA_BUNDLE=/Users/velayutham/work/corp-cert/bundle.pem
aws ec2 describe-instances --region us-west-2 ==> This should work fine now.

How to deploy maven project in aws ec2 instance without pem file

In my previous question I solved a problem of deployin a Maven project on AWS EC2 instance with Gitlab CI/CD by using SSH with PEM file, but I have read on Internet that it is not a best practice to commit the .pem file in a Git repository. So how do I have to change to deploy my application on aws without using pem file.
I'm trying to follow this tutorial but here the application is written with node.js while my app is built with maven so what do I need to change?
It does not matter what language is used to write an application. The tutorial is correct: you should use GitLab CI/CD environment variables to store secrets such as keys.
Variables are exposed as environment variables at the build time. You can use them like:
production:
stage: deploy
image: alpine/latest
variables:
GIT_STRATEGY: none
before_script:
- eval $(ssh-agent -s)
- echo "$DEPLOY_KEY" | tr -d '\r' | ssh-add - > /dev/null
script:
- ./deploy # This script uses SSH to deploy things
- ssh-agent -k

Configuring aws cli use fakes3

Keen to setup fake s3, have it working via docker setup. Running on port 4569. I cannot figure out how to test using aws cli (version 1.10.6). specifically change the port for the access.
i.e. want to do a command like
$ aws s3 cp test.txt s3://mybucket/test2.txt
i need to specify the port, i've tried
--port settings on command line: i.e. AWS_ACCESS_KEY_ID=ignored AWS_SECRET_ACCESS_KEY=ignored aws s3 --profile fakes3 cp test.txt s3://mybucket/test2.txt (says not valid parameter)
adding a profile and including end_point="localhost:4569 in config in ~/.aws`. gives error about AUTH Key
running fakes3 on 443 but that then clashes with my local machine
Has anyone got aws cli working with fakes3?
$ aws s3 --version
aws-cli/1.10.6 Python/2.7.11 Darwin/15.2.0 botocore/1.3.28
Use the --endpoint-url argument. If fakes3 is listening on port 4569, try this:
aws --endpoint-url=http://localhost:4569 s3 ls

SSL CERTIFICATE_VERIFY_FAILED in aws cli

I installed AWS CLI on the Windows server 2007 32bit.
aws --version
aws-cli/1.8.8 Python/2.7.9 Windows/2008Server
I configure aws cli using keys
Once I run below command to test AWS S3, I get this SSL error:
aws s3 ls
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
Please help to get rid of this basic error.
If you want to use SSL and not have to specify the --no-verify-ssl option, then you need to set the AWS_CA_BUNDLE environment variable. e.g from PowerShell:
setx AWS_CA_BUNDLE "C:\Users\UserX\Documents\RootCert.pem"
The PEM file is a saved copy of the root certificate for the AWS endpoint you are trying to connect to. To generate it, first export the certificate in DER format (For details on how to do this, see here). Then run the following command to convert to the PEM format:
openssl x509 -inform der -in "C:\Users\UserX\Documents\RootCert.der" -out RootCert.pem
If you are using Powershell and not bash, then you will need to first install openssl.
For a full list of environment variables supported by the AWS CLI, see here
use this option with your cmd
"--no-verify-ssl"
Not sure if it's related to to the OP's issue, however, one of our devs had this issue this morning, turned out he was using Fiddler (on Windows), to debug other issues. After stopping Fiddler (which was intercepting https traffic), the issue was resolved.
I had the same issue on Windows 10. It happens to be due to the aws cli not reading the internet proxy setting from the Windows registry. Fixed same error by setting the environment variables HTTP_PROXY and HTTPS_PROXY to the corporate internet proxy. Hope it helps somebody!
Mine was resolved with:
pip install awscli --force-reinstall --upgrade
I ran into a similar issue on Mac OSX in the company/corporate network.
If you don't know the proxy URL Get it from your company's network administrator and configure with the following commands.
Linux, macOS, or Unix
$ export HTTP_PROXY=http://proxy.example.com:1234
$ export HTTPS_PROXY=https://proxy.example.com:1234
Windows
$ set HTTP_PROXY=http://proxy.example.com:1234
$ set HTTPS_PROXY=https://proxy.example.com:1234
More information
I added the certificate to C:\Program Files\Amazon\AWSCLIV2\awscli\botocore\cacert.pem and it resolved the problem.
My issue was our company's VPN. It worked after I disconnected from VPN
AWS already posted a clean solution for this, here it is:
Instead of hacking your system now the CLI supports you passing it a .pem file with the CA chain for it to communicate with your proxy:
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.html#tshoot-certificate-verify-failed
To fix this, instruct the AWS CLI where to find your companies .pem file using the ca_bundle configuration file setting, --ca-bundle command line option, or the AWS_CA_Bundle environment variable.
Problem most likely caused by corporate proxy. In my case I was running the commands on AWS CLI behind proxy server and was getting certificate error.
So to get around this I added --no-verify-ssl flag. Though this is a bad idea, I used this as a temporary solution to get the job done until it is resolved by the network team.
I believe this option would have been tried already but just putting it here for everyones reference:
when you have proxy added to your ec2 machines and it is in private subnet with a S3 vpc-endpoint attached. I was getting the same error.
Bypassing the proxy using no_proxy for the bucket as per : https://aws.amazon.com/premiumsupport/knowledge-center/connect-s3-vpc-endpoint/
didn't help me and was still failing with the same error.
the only catch here was we need to add endpoint url which is s3.ap-southeast-2.amazonaws.com as below and it worked for me:
export NO_PROXY=169.254.169.254,s3.ap-southeast-2.amazonaws.com
169.254.169.254 is used to access instance role credentials in my case.
I had a similar issue and solved it by setting the proxy as follows:
$ set HTTP_PROXY=http://proxy.example.com:1234
$ set HTTPS_PROXY=https://proxy.example.com:1234
Linux:
$ export AWS_CA_BUNDLE="/data/ca-certs/ca-bundle.pem"
Windows:
PS C:\> setx AWS_CA_BUNDLE C:\data\ca-certs\ca-bundle.pem
$ aws s3 ls --ca-bundle "/data/ca-certs/ca-bundle.pem"
For me ec2 instance date was incorrect, after changing the date and time, fixed the problem.
Simply rebooted the ec2 instance
When you use a AWS CLI command, you receive a "[SSL: CERTIFICATE_ VERIFY_FAILED] certificate verify failed" error message. This is caused by the AWS CLI not trusting your proxy's certificate due to factors such as your proxy's certificate being self-signed, with your company set as the Certification Authority (CA). This prevents the AWS CLI from finding your companies CA root certificate in the local CA registry.
To fix this, instruct the AWS CLI where to find your companies .pem file using the ca_bundle configuration file setting, --ca-bundle command line option, or the AWS_CA_Bundle environment variable.
Please refer https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-troubleshooting.html#tshoot-certificate-verify-failed
aws configure set default.ca_bundle <your CA file>
I agree with above answers, do the following
1- Remove your cli and install latest cli
2- check the certificate exist: C:\Program Files\Amazon\AWSCLIV2\botocore\cacert.pem
3- if it doesn't exist remove the cli and go to: C:\Program Files\ and remove Amazon
4- Install cli latest version it should work.
5- Try testing with your VPN connected
use the following option to overcome the ssl certification issue.
aws s3 ls --no-verify-ssl