I created an EMR cluster using the AWS UI but did not bootstrap to install boto3. Now, I am getting to execute pyspark scripts which use boto3. So, I SSH to the master node and am attempting to install boto3. Below is screen shot I gathered to show as if I am installing it again (so the messages are saying that it is already installed), but you get the point. Then I run Python3 interactively to test boto3 and it can't find it. What am I doing wrong? Also, will I need to install boto3 on the slave nodes as well?
Thanks
You install boto3 for Python2, so you don't see it from Python3.
Try to install boto3 with pip3:
pip3 install boto3 --user
Related
I have a api gateway set up as follows:
Now I need to update the above set up to to vpc link and the same url so at the end I want to convert it to:
As you see I can do it through console.Can I do the above through cli as well?
I found the following I cannot get a working example.
https://docs.aws.amazon.com/cli/latest/reference/apigateway/update-method.html
https://docs.aws.amazon.com/cli/latest/reference/apigateway/update-rest-api.html
Can anyone help? is it supported at all?
Its is supported. You are looking for update-integration command
https://docs.aws.amazon.com/cli/latest/reference/apigateway/update-integration.html
with connectionType is VPC_LINK
I had the same problem and I managed to solve it by upgrading to the latest aws cli:
pip install awscli --upgrade --user
Afterwards, restart your terminal to make sure you load your latest cli
I'm trying to use this very simple command:
import boto3
client = boto3.client('sagemaker-runtime')
listed in the documentation
but i'm getting this error:
UnknownServiceError: Unknown service: 'sagemaker-runtime'. Valid service names are: acm, etc..
My goal is to be able to invoke the endpoint that I've created in Amazon SageMaker.
I'm doing this from a Jupyter notebook in Sagemaker, so I feel like this should work no problem. How do I get it to run here, and outside of the Sagemaker environment?
Amazon SageMaker is a very new service (December 2017).
You will need to update your boto library to use it:
sudo pip install boto --upgrade
sudo pip install boto3 --upgrade
sudo pip install awscli --upgrade
The documentation is incorrect. This is how you get the client with the SageMaker Python SDK.
import boto3
client = boto3.client('runtime.sagemaker')
I've done this successfully. And, as John said, be sure to update your versions of boto3 and awscli.
In case you use Jupyter Notebook, create cell at the top and execute below.
!pip install boto3 --upgrade
In my case, to upgrade boto3 in terminal didn't work.
I am using a python script from github to run on AWS lambda.
I realized that the import boto statements are not working while running inside lambda.
Is there way to instruct lambda to load boto2 as well?
OK, all I had to do was install boto using "pip install module-name -t /path/to/project-dir" per the docs:
http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html
Then zipped the files needed for lambda and uploaded.
I am working in the US-East-1 (N.Virginia) and have even configured the Default Region Name to us-east-1 using the command aws configure.
But I am not able to access Batch using CLI. Batch is not even listed as one of the Available Services in aws help.
Any ideas how to enable Batch in AWS Cli? I have administrative access in IAM console so permissions don't seem to be the issue.
The batch service is relatively new, so its commands only exist in fairly new versions of the aws CLI.
Commands for batch in the latest cli documentation: http://docs.aws.amazon.com/cli/latest/reference/batch/index.html?highlight=batch
If you are running Windows, simply download the updated installer. https://aws.amazon.com/cli/
If you are using OSX or Linux use pip. pip install --upgrade awscli
I installed AWS CLI and Boto3 but can't find Shared Credentials File to put user access key.
I tried to install it on Windows 7 and Ubuntu and I have got the same issue, simply can't find the Shared Credentials File in the default location ~/.aws/credentials
I was using this official guide of boto3
Thanks,
In fact, you can create your ~/.aws folder yourself then manually key in and save the config into ~/.aws/credentials and ~/.aws/config
If you think it is troublesome, after you install awscli usign apt , you can run aws configure to use the shell interface to create those folder.
sudo apt install awscli
aws configure
The aws shell script will ask you for access key and create ~/.aws/credential and ~/.aws/config