thanks for greate packages!
I have problem when i create development with localstack using S3 service to create presignedurl post.
I have run localstack with SERVICES=s3 DEBUG=1 S3_SKIP_SIGNATURE_VALIDATION=1 localstack start
I have settings AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_DEFAULT_REGION=us-east-1 AWS_ENDPOINT_URL=http://localhost:4566 S3_Bucket=my-bucket
I make sure have the bucket
> awslocal s3api list-buckets
{
"Buckets": [
{
"Name": "my-bucket",
"CreationDate": "2021-11-16T08:43:23+00:00"
}
],
"Owner": {
"DisplayName": "webfile",
"ID": "bcaf1ffd86f41161ca5fb16fd081034f"
}
}
I try create presigned url, and running in console with this
s3_client_sync.create_presigned_post(bucket_name=settings.S3_Bucket, object_name="application/test.png", fields={"Content-Type": "image/png"}, conditions=[["Expires", 3600]])
and have return like this
{'url': 'http://localhost:4566/kredivo-thailand',
'fields': {'Content-Type': 'image/png',
'key': 'application/test.png',
'AWSAccessKeyId': 'test',
'policy': 'eyJleHBpcmF0aW9uIjogIjIwMjEtMTEtMTZUMTE6Mzk6MjNaIiwgImNvbmRpdGlvbnMiOiBbWyJFeHBpcmVzIiwgMzYwMF0sIHsiYnVja2V0IjogImtyZWRpdm8tdGhhaWxhbmQifSwgeyJrZXkiOiAiYXBwbGljYXRpb24vdGVzdC5wbmcifV19',
'signature': 'LfFelidjG+aaTOMxHL3fRPCw/xM='}}
And i test using insomnia
and i have read log in localstack
2021-11-16T10:54:04:DEBUG:localstack.services.s3.s3_utils: Received presign S3 URL: http://localhost:4566/my-bucket/application/test.png?AWSAccessKeyId=test&Policy=eyJleHBpcmF0aW9uIjogIjIwMjEtMTEtMTZUMTE6Mzk6MjNaIiwgImNvbmRpdGlvbnMiOiBbWyJFeHBpcmVzIiwgMzYwMF0sIHsiYnVja2V0IjogImtyZWRpdm8tdGhhaWxhbmQifSwgeyJrZXkiOiAiYXBwbGljYXRpb24vdGVzdC5wbmcifV19&Signature=LfFelidjG%2BaaTOMxHL3fRPCw%2FxM%3D&Expires=3600
2021-11-16T10:54:04:WARNING:localstack.services.s3.s3_utils: Signatures do not match, but not raising an error, as S3_SKIP_SIGNATURE_VALIDATION=1
2021-11-16T10:54:04:INFO:localstack.services.s3.s3_utils: Presign signature calculation failed: <Response [403]>
what i missing, so i cannot create the presignedurl post ?
The problem is with your AWS configuration -
AWS_ACCESS_KEY_ID=test // Should be an Actual access Key for the IAM user
AWS_SECRET_ACCESS_KEY=test // Should be an Actual Secret Key for the IAM user
AWS_DEFAULT_REGION=us-east-1
AWS_ENDPOINT_URL=http://localhost:4566 // Endpoint seems wrong
S3_Bucket=my-bucket // Actual Bucket Name in AWS S3 console
For more information, try to read here and setup your environment with correct AWS credentials - Setup AWS Credentials
Related
When executing this command,I get this error:
C:\WINDOWS\system32>eksctl create cluster --name eksctl-demo --profile myAdmin2
Error: checking AWS STS access – cannot get role ARN for current session: operation error STS: GetCallerIdentity, failed to sign request: failed to retrieve credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, request send failed, Get "http://169.254.169.254/latest/meta-data/iam/security-credentials/": dial tcp 169.254.169.254:80: i/o timeout
myAdmin2 IAM users credientials are set up as follows:
Credentials file:
[myAdmin2]
aws_access_key_id = ******************
aws_secret_access_key = ********************
config file:
[profile myAdmin2]
region = us-east-2
output = json
myAdmin2 has access to the console:
C:\WINDOWS\system32>aws iam list-users --profile myAdmin2
{
"Users": [
{
"Path": "/",
"UserName": "myAdmin",
"UserId": "AIDAYYPFV776ELVEJ5ZVQ",
"Arn": "arn:aws:iam::602313981948:user/myAdmin",
"CreateDate": "2022-09-30T19:08:08+00:00"
},
{
"Path": "/",
"UserName": "myAdmin2",
"UserId": "AIDAYYPFV776LEDK2PCCI",
"Arn": "arn:aws:iam::602313981948:user/myAdmin2",
"CreateDate": "2022-09-30T21:39:33+00:00"
}
]
}
I had problems working with myAdmin that's why I created a new IAM user called myAdmin2.
myAdmin2 is granted AdministratorAccess permission:
As shown in this image
aws cli version installed:
C:\WINDOWS\system32>aws --version
aws-cli/2.7.35 Python/3.9.11 Windows/10 exe/AMD64 prompt/off
My Env variables:
C:\WINDOWS\system32>set
AWS_ACCESS_KEY_ID= ***********the same as I have in credentials file
AWS_CONFIG_FILE=~/.aws/config
AWS_DEFAULT_PROFILE=myAdmin2
AWS_DEFAULT_REGION=us-east-2
AWS_PROFILE=myAdmin2
AWS_SECRET_ACCESS_KEY=****************the same as I have in credentials file
AWS_SHARED_CREDENTIALS_FILE=~/.aws/credentials
I think those are all the necessary things I have to mention. If someone can help, please. I can't move on with this error!!
It worked finally! everything was well configured, I just had to reboot my laptop and it resolved the issue!
I have made a few API Gateways in AWS.
At this point , I would like curl to the endpoint of each API Gateway in order to start testing. But, I cannot find the endpoint URL:
For example, via CLI, I ran:
aws --profile dev apigateway get-rest-apis --output json
I get: (no endpoint URL)
"apiKeySource": "HEADER",
"description": "API one",
"endpointConfiguration": {
"types": [
"REGIONAL"
]
},
"createdDate": 1570655986,
"policy": "{\\\"Version\\\":\\\"2012-10-17\\\",\\\"Statement\\\":[{\\\"Effect\\\":\\\"Allow\\\",\\\"Principal\\\":\\\"*\\\",\\\"Action\\\":\\\"execute-api:Invoke\\\",\\\"Resource\\\":\\\"arn:aws:FOOBAR\\/*\\/*\\/*\\\",\\\"Condition\\\":{\\\"IpAddress\\\":{\\\"aws:SourceIp\\\":[\\\"0.0.0.0\\/0\\\""]}}}]}",
"id": "foobar",
"name": "foobar"
}
you have to deploy the APIs.
you find then the https address in the deployed stage.
or you can use a custom domain name
I have an AWS ElasticSearch domain in eu-west-1 region and have taken a snapshot to an S3 bucket sub folder also in the same region.
I have also deployed a second AWS ElasticSearch domain in another aws region - eu-west-2.
Added an S3 bucket replication between the buckets but when I try to register the repository on the eu-west-2 AWS ES domain, I get the following error:
500
{"error":{"root_cause":[{"type":"blob_store_exception","reason":"Failed to check if blob [master.dat] exists"}],"type":"blob_store_exception","reason":"Failed to check if blob [master.dat] exists","caused_by":{"type":"amazon_s3_exception","reason":"Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 14F0571DFF522922; S3 Extended Request ID: U1OnlKPOkfCNFzoV9HC5WBHJ+kfhAZDMOG0j0DzY5+jwaRFJvHkyzBacilA4FdIqDHDYWPCrywU=)"}},"status":500}
this is the code i am using to register the repository on the new cluster (taken from https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains-snapshots.html#es-managedomains-snapshot-restore):
import boto3
import requests
from requests_aws4auth import AWS4Auth
host = 'https://search-**es-elk-prod**.eu-west-2.es.amazonaws.com/' # include https:// and trailing /
region = 'eu-west-2' # e.g. us-west-1
service = 'es'
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key, region, service, session_token=credentials.token)
# Register repository
path = '_snapshot/es-elk-prod' # the Elasticsearch API endpoint
url = host + path
payload = {
"type": "s3",
"settings": {
"bucket": "es-prod-eu-west-2",
"region": "eu-west-2",
"role_arn": "arn:aws:iam::1234567:role/EsProd-***-snapshotS3role-***"
}
}
headers = {"Content-Type": "application/json"}
r = requests.put(url, auth=awsauth, json=payload, headers=headers)
print(r.status_code)
print(r.text)
from the logs, i get:
curl -X GET 'https://search-**es-elk-prod**.eu-west-2.es.amazonaws.com/_snapshot/es-mw-elk-prod/_all?pretty'
{
"error" : {
"root_cause" : [
{
"type" : "amazon_s3_exception",
"reason" : "Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 72A59132E2830D81; S3 Extended Request ID: o0XalToNp19HDJKSOVxmna71hx3LkwoSFEobm3HQGH1HEzxOrAtYHg+asnKxJ03iGSDDhUz5GUI=)"
}
],
"type" : "amazon_s3_exception",
"reason" : "Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 72A59132E2830D81; S3 Extended Request ID: o0XalToNp19HDJKSOVxmna71hx3LkwoSFEobm3HQGH1HEzxOrAtYHg+asnKxJ03iGSDDhUz5GUI=)"
},
"status" : 500
}
the ARN is able to access the S3 bucket as is the same ARN i use to snapshot the eu-west-2 domain to S3 as the eu-west-1 snapshot is stored in a sub-folder on the S3 bucket, I added a path to the code, such that:
payload = {
"type": "s3",
"settings": {
"bucket": "es-prod-eu-west-2",
"path": "es-elk-prod",
"region": "eu-west-2",
"role_arn": "arn:aws:iam::1234567:role/EsProd-***-snapshotS3role-***"
}
}
but this didn't work either.
What is the correct way to restore snapshot created in one aws region into another aws region?
Any advice is much appreciated.
I've had similar but not identical error messages about The bucket is in this region: eu-west-1. Please use this region to retry the request when moving from eu-west-1 to us-west-2.
According to Amazon's documentation (under "Migrating data to a different domain") you will need to specify an endpoint rather than a region:
If you encounter this error, try replacing "region": "us-east-2" with "endpoint": "s3.amazonaws.com" in the PUT statement and retry the request.
I' using Cognito user pool for securing my API gateway . Now I would like to make requests to my API using postman but I need to pass in Authorization token as the API is secured. Is there any AWS CLI command or REST API to generate auth tokens(by passing username/password)? I have searched documentation but couldn't find any examples. Thanks for your help.
You can do this using the following CLI commands:
Register a user
aws cognito-idp sign-up --region {your-aws-region} --client-id {your-client-id} --username admin#example.com --password password123
Confirm user registration
aws cognito-idp admin-confirm-sign-up --region {your-aws-region} --user-pool-id {your-user-pool-id} --username admin#example.com
Authenticate (get tokens)
aws cognito-idp admin-initiate-auth --region {your-aws-region} --cli-input-json file://auth.json
Where auth.json is:
{
"UserPoolId": "{your-user-pool-id}",
"ClientId": "{your-client-id}",
"AuthFlow": "ADMIN_NO_SRP_AUTH",
"AuthParameters": {
"USERNAME": "admin#example.com",
"PASSWORD": "password123"
}
}
You should get a response like this if everything is set up correctly:
{
"AuthenticationResult": {
"ExpiresIn": 3600,
"IdToken": "{your-idtoken}",
"RefreshToken": "{your-refresh-token}",
"TokenType": "Bearer",
"AccessToken": "{your-access-token}"
},
"ChallengeParameters": {}
}
Use the following command to generate the auth tokens, fill in the xxxx appropriately based on your cognito configuration,
aws cognito-idp initiate-auth --auth-flow USER_PASSWORD_AUTH --client-id xxxx --auth-parameters USERNAME=xx#xx.com,PASSWORD=xxxx
Note: You can use any one username or password under applicable cognito user pool.
The client can be found under general settings--> app client
The AccessKeyId and SecretAccessKey is not required as it already defined while setting up the aws cli. If not done use the following link to set that up first
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html
There is an AWS CLI command to generate Auth Tokens. You can use InitiateAuth CLI Command for this.
Note: Make sure you have done the UserPool configuration matching the expected tokens.
Consider the example:
aws cognito-idp admin-update-user-attributes --user-pool-id myUserPollId
--username myUser
--user-attributes [{"Name": "custom:roles","Value": "ROLE1,ROLE2"}] --region us-east-1
This gets me error:
Invalid JSON:
[{Name:
You can always try using shorthand syntax:
--user-attributes Name="custom:roles",Value="ROLE1,ROLE2"
If you really want to use the JSON syntax, try this:
--user-attributes '[{"Name" : "custom:roles","Value" : "ROLE1,ROLE2"}]'
Ensure that the user-attributes list is enclosed in single quotes
--user-attributes '[{"Name": "phone_number", "Value": "+123434532"},
{"Name": "name", "Value":"name_your"}]'
In case someone get stuck in the same problem again, below are the tested steps to have user attributes updated via aws cli with json file.
Step 0: Setup AWS CLI in case you haven't already. Mac users can run:
brew install awscli
Step 1: Have a valid json handy with you, saved in a file. Sample json with valid format:
{
"UserAttributes": [{
"Name": "custom:additional-attribute1",
"Value": "Value for additional attribute 1"
},
{
"Name": "custom:additional-attribute2",
"Value": "Value for additional attribute 2"
}
]
}
Step 2: Run the following in your console:
aws cognito-idp admin-update-user-attributes --user-pool-id XX-XXXX-X_XXXXXXXXX --username XXXXX#XXXXX.com --cli-input-json file:///Users/YOUR_PATH_TO_THE_FILE/user-attributes.json
Parameters:
--user-pool-id :: Your user pool ID.
--username :: The user you want to udpate.
--cli-input-json :: This is the command that loads json file and parses it.
That's it. If your json is valid and aws cli authorises, the given user record should be updated instantly.