I make a spot request using the below:
req = conn_spot.request_spot_instances(price=self.spot_price,instance_type=self.instance_type, ebs_optimized=self.ebs_optimized,image_id=self.ami,placement=self.zone,key_name=self.keypair,security_groups=[self.security_group])
I am able to get the spot request ID using the below:
request_id = req[0].id
I can check on the status of my request id usng the below:
reqs = conn_spot.get_all_spot_instance_requests()
Now...given the request_id I need to cancel the order e.g. if taking too long. How do I do that using boto given the request id?
cancel_spot_instance_requests(request_ids, dry_run=False)
See: boto documentation
Related
Does boto3 have any method which allows one to get the text of the error if the query failed? get_query_execution returns a status of the query only.
You can get the error message from 'StateChangeReason' field of your response['Status'].
As per get_query_execution documentation:
StateChangeReason (string) --
Further detail about the status of the query.
import boto3
client = boto3.client('athena')
failed_query_id = '08adbf00-5f14-4d54-9311-fd55e2024781'
response = client.get_query_execution(QueryExecutionId=failed_query_id)
print(response['Status']['StateChangeReason'])
There are a few examples for the way to pre-sign the URL of an S3 request, but I couldn't find any working example to pre-sign other services in AWS.
I'm trying to write an item to DynamoDB using the Python SDK botos. The SDK included the option to generate the pre-signed URL here. I'm trying to make it work and I'm getting a URL, but the URL is responding with 404 and the Item is not appearing in the DynamoDB table.
import json
ddb_client = boto3.client('dynamodb')
response = ddb_client.put_item(
TableName='mutes',
Item={
'email': {'S':'g#g.c'},
'until': {'N': '123'}
}
)
print("PutItem succeeded:")
print(json.dumps(response, indent=4))
This code is working directly. But when I try to presign it:
ddb_client = boto3.client('dynamodb')
params = {
'TableName':'mutes',
'Item':
{
'email': {'S':'g#g.c'},
'until' : {'N': '1234'}
}
}
response = ddb_client.generate_presigned_url('put_item', Params = params)
and check the URL:
import requests
r = requests.post(response)
r
I'm getting: Response [404]
Any hint on how to get it working? I checked the IAM permissions, and they are giving full access to DynamoDB.
Please note that you can sign a request to DynamoDB using python, as you can see here: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html#sig-v4-examples-post . But for some reasons, the implementation in the boto3 library doesn't do that. Using the boto3 library is much easier than the code above, as I don't need to provide the credentials for the function.
You send an empty post request. You should add the data to the request:
import requests
r = requests.post(response, data = params)
I think you are having this issue, that's why you are recieving a 404.
They recommend using Cognito for authentication instead of IAM for this cases.
Using bellow code i am getting an OpenID token for an IdentityId.
import boto3
cognito_client = boto3.client('cognito-identity')
data = {'IdentityPoolId': identity_pool_id,
'Logins': logins,
'TokenDuration': 24 * 60 * 60,
'IdentityId': identity_id}
cognito_client.get_open_id_token_for_developer_identity(
**data)
But i have two IdentityPoolId, so i need to identify proper IdentityPoolId for a given IdentityId.
Using boto3 library how can i identify a IdentityId belongs to a IdentityPoolId or not?
If all you have available is just the IdentityId value you would need to use list_identities() function of the Cognito-Identity client to get all of the IDs from a pool and then determine if your ID is part of the pool.
https://boto3.readthedocs.io/en/latest/reference/services/cognito-identity.html#CognitoIdentity.Client.list_identities
import boto3
def getPoolID(boto3Client,PoolIDs,idVal):
For PoolID in PoolIDs:
response = boto3Client.list_identities(IdentityPoolId=PoolID)
for ID in response['Identities']:
if idVal == ID['IdentityId']:
return PoolID
cognitoClient = boto3.client('cognito-identity')
PoolIDList=['PoolID1','PoolID2']
IdentityIdToCheck='Value'
FinalPoolID = getPoolID(cognitoClient,PoolIDList,IdentityIdToCheck)
print FinalPoolID
You might need to adjust how you get the response based on the number of IDs in your ID pools, as you may need to paginate through the records
I want to have a lambda calling a Sagemaker instance in another region. If both are in the same region, everything works fine. If they are not, I get the following error:
The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'POST
/endpoints/foo-endpoint/invocations
host:runtime.sagemaker.us-east-1.amazonaws.com
x-amz-date:20180406T082536Z
host;x-amz-date
1234567890foobarfoobarfoobarboofoobarfoobarfoobarfoobarfoobarfoo'
The String-to-Sign should have been
'AWS4-HMAC-SHA256
20180406T082536Z
20180406/us-east-1/sagemaker/aws4_request
987654321abcdeffoobarfoobarfoobarfoobarfoobarfoobarfoobarfoobarf'
I use aws-requests-auth (0.4.1) with boto3 (1.5.15 - updating to 1.7.1 didn't change anything, changelog) like this:
import requests
from aws_requests_auth.aws_auth import AWSRequestsAuth
auth = AWSRequestsAuth(aws_access_key=config['AWS']['ACCESS_KEY'],
aws_secret_access_key=(
config['AWS']['SECRET_ACCESS_KEY']),
aws_host=config['AWS']['HOST'],
aws_region=config['AWS']['REGION'],
aws_service=config['AWS']['SERVICE'])
payload = {'foo': 'bar'}
response = requests.post(post_url,
data=json.dumps(payload),
headers={'content-type': 'application/json'},
auth=auth)
printing auth only gives <aws_requests_auth.aws_auth.AWSRequestsAuth object at 0x7f9d00c98390>.
Is there a way to print the "Canonical String" mentioned in the error message?
(Any other ideas how to fix this are appreciated as well)
A work-around for the asked question:
req = requests.request('POST', 'http://httpbin.org/get')
req.body = b''
req.method = ''
print(auth.get_aws_request_headers(req,
aws_access_key=auth.aws_access_key,
aws_secret_access_key=auth.aws_secret_access_key,
aws_token=auth.aws_token))
The problem is not solved, though. And now I wonder what the first argument of auth.get_aws_request_headers is.
I have to fetch the retweets for the tweets and create the JSON file with retweets,user id etc using the python script. Kindly help me to sort it our this issues.
Thanks in advance!!
This task require some fields of knowledge, and since you ask in a general way, I reckon you need a script to run immediately, but setting up this process requires sometime
This part to get connect to twitter API
from twython import Twython, TwythonError
APP_KEY = 'YOUR_APP_KEY'
APP_SECRET = 'YOUR_APP_SECRET'
twitter = Twython(APP_KEY, APP_SECRET)
Use Twitter API call from Twython,
you can find a list here https://twython.readthedocs.io/en/latest/api.html, the param is the same as twitter API
response = twitter.get_retweets(id, 100)
Pagnation
each call to API have limit of returns, in example for engine.get_friends_ids was limited to 5000 (https://dev.twitter.com/rest/reference/get/friends/ids), if you want to get more than 5000, you have to use the cursor in the returned result (if cur = 0 in json returned means no more results), following is example of how to handling cursor
#Set a temp to loop
cur = -1
#Stop when no more result
while cur !=0:
response = twitter.get_friends_ids(user_id=user_id, cursor=cur)
#Some code to handle the response
cur = response["next_cursor"]
API key
Key expires after some calls (https://dev.twitter.com/rest/public/rate-limits), so you need to set some code to auto change your key, or wait for some period (key reached limit return error code 429)
Response
The response from API was in JSON format, which was easy to use, you can access data by selecting base on response[key], in example
reponse["ids"] or response["next_cursor"]