AWS DynamoDB resource not found exception - amazon-web-services

I have a problem with connection to DynamoDB. I get this exception:
com.amazonaws.services.dynamodb.model.ResourceNotFoundException:
Requested resource not found (Service: AmazonDynamoDB; Status Code:
400; Error Code: ResourceNotFoundException; Request ID: ..
But I have a table and region is correct.

From the docs it's either you don't have a Table with that name or it is in CREATING status.
I would double check to verify that the table does in fact exist, in the correct region, and you're using an access key that can reach it

My problem was stupid but maybe someone has the same... I changed recently the default credentials of aws (~/.aws/credentials), I was testing in another account and forgot to rollback the values to the regular account.

I spent 1 day researching the problem in my project and now I should repay a debt to humanity and reduce the entropy of the universe a little.
Usually, this message says that your client can't reach a table in your DB.
You should check the next things:
1. Your database is running
2. Your accessKey and secretKey are valid for the database
3. Your DB endpoint is valid and contains correct protocol ("http://" or "https://"), and correct hostname, and correct port
4. Your table was created in the database.
5. Your table was created in the database in the same region that you set as a parameter in credentials. Optional, because some
database environments (e.g. Testcontainers Dynalite) don't have an incorrect value for the region. And any nonempty region value will be correct
In my case problem was that I couldn't save and load data from a table in tests with DynamoDB substituted by Testcontainers and Dynalite. I found out that in our project tables creates by Spring component marked with #Component annotation. And in tests, we are using a global setting for lazy loading components to test, so our component didn't load by default because no one call it in the test explicitly. ¯_(ツ)_/¯

If DynamoDB table is in a different region, make sure to set it before initialising the DynamoDB by
AWS.config.update({region: "your-dynamoDB-region" });
This works for me:)

Always ensure that you do one of the following:
The right default region is set up in the AWS CLI configuration files on all the servers, development machines that you are working on.
The best choice is to always specify these constants explicitly in a separate class/config in your project. Always import this in code and use it in the boto3 calls. This will provide flexibility if you were to add or change based on the enterprise requirements.

If your resources are like mine and all over the place, you can define the region_name when you're creating the resource.
I do this for all my instantiations as it forces me to think about what I'm putting/calling where.
boto3.resource("dynamodb", region_name='us-east-2')

I was getting this issue in my .NetCore Application.
Following fixed the issue for me in Startup class --> ConfigureServices method
services.AddDefaultAWSOptions(
new AWSOptions
{
Region = RegionEndpoint.GetBySystemName("eu-west-2")
});

I got Error warning Lambda : lifecycleIteration=0 lambda handler returned an error: ResourceNotFoundException: Requested resource not found
I spent 1 week to fix the issue.
And so its root cause and steps to find issue is mentioned in below Git Issue thread and fixed it.
https://github.com/soto-project/soto/issues/595

Related

AWS AppSync searchItems type return data while table is empty

I deleted all the items in the DataTemplate table but when I query them again with the searchDataTemplates endpoint on the app or in AppSync it returns the old data, but when I use the listDataTemplates it returns nothing which is correct. Needed to repopulate the data in the table.
data template table
search endpoint
list endpoint
when I updated items individually it worked just fine but when i deleted all the items from the console (around 700 items) the search endpoint stopped working. Just the search
UPDATE:
I repopulated the data hoping it'd reset but now the listDataTemplates shows the new data and the search still shows the old data, is there some cache that needs to be reset?
SECOND UPDATE:
I removed the table and the appsync functions are gone however when i recreated the table (with no data) the testing out the function still returns the old data. I'm guessing the opensearch stuff hasn't been updated?
If you are using AppSync with Amplify CLI, #searchable will automatically create the followings:
An OpenSearch Domain
A Lambda Function that will be attached to the DynamoDB Streams and push the changes (create/update/delete) over to your OpenSearch Domain.
And the problem that you're facing is most likely due to the Lambda Function created failed to push the changes from DynamoDB Streams to OpenSearch. A quick suggestion is to check on the created Lambda Function first.
Reference: #searchable
This issue can only happen if caching is enabled in your application.
I am not sure what's the infrastructure you are using, so i would go ahead with some educated guess. Please feel free to correct me if i overstepped.
From your description of question, you have an AppSync as API layer and DynamoDb as primary database.
If these are the only two resources you have, please check the AppSync cache configuration.
Open AppSync console
from left panel select APIs -> your api -> caching
Validate Caching behavior is set to None
In case if you have AWS OpenSearch enabled for search query (i could be wrong, however picking up from previous comment). Then validate the cluster configuration.
Open AWS Open Search Service console
From left panel select Domains and click on the openserch domain that you are using
scroll to the bottom right and look for Advanced cluster settings and ensure the attribute Fielddata cache allocation is set to 0
If Fielddata cache allocation is not 0, update the cluster configuration and modify the advanced cluster setting to set the Fielddata cache allocation field to 0.
Wait for a few minutes (I would suggest 5 minutes) and then retry your use-case.
I hope this would help resolve your issue.

InvalidSignatureException when using boto3 for dynamoDB on aws

Im facing some sort of credentials issue when trying to connect to my dynamoDB on aws. Locally it all works fine and I can connect using env variables for AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION and then
dynamoConnection = boto3.resource('dynamodb', endpoint_url='http://localhost:8000')
When changing to live creds in the env variables and setting the endpoint_url to the dynamoDB on aws this fails with:
"botocore.exceptions.ClientError: An error occurred (InvalidSignatureException) when calling the Query operation: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."
The creds are valid as they are used in a different app which talks to the same dynamoDB. Ive also tried not using env variables but rather directly in the method but the error persisted. Furthermore, to avoid any issues with trailing spaces Ive even used the credentials directly in the code. Im using Python v3.4.4.
Is there maybe a header that also should be set that Im not aware of? Any hints would be apprecihated.
EDIT
Ive now also created new credentials (to make sure there are only alphanumerical signs) but still no dice.
You shouldn't use the endpoint_url when you are connecting to the real DynamoDB service. That's really only for connecting to local services or non-standard endpoints. Instead, just specify the region you want:
dynamoConnection = boto3.resource('dynamodb', region_name='us-west-2')
It sign that your time zone is different. Maybe you can check your:
1. Time zone
2. Time settings.
If there are some automatic settings, you should fix your time settings.
"sudo hwclock --hctosys" should do the trick.
Just wanted to point out that accessing DynamoDB from a C# environment (using AWS.NET SDK) I ran into this error and the way I solved it was to create a new pair of AWS access/secret keys.
Worked immediately after I changed those keys in the code.

AWS DynamoDB Requested resource not found

I am trying to connect my app to DynamoDB. I have set everything up the way Amazon recommends. But i still keep getting the same error over and over again:
7-21 11:02:29.856 10027-10081/com.amazonaws.cognito.sync.demo E/AndroidRuntime﹕ FATAL EXCEPTION: AsyncTask #1
Process: com.amazonaws.cognito.sync.demo, PID: 10027
java.lang.RuntimeException: An error occured while executing doInBackground()
at android.os.AsyncTask$3.done(AsyncTask.java:304)
at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
at java.util.concurrent.FutureTask.run(FutureTask.java:242)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)
Caused by: com.amazonaws.services.dynamodbv2.model.ResourceNotFoundException: Requested resource not found (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ResourceNotFoundException; Request ID: GIONOKT7E3AMTC4PO19CPLON93VV4KQNSO5AEMVJF66Q9ASUAAJG)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:710)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:385)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:196)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:2930)
at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.updateItem(AmazonDynamoDBClient.java:930)
at com.amazonaws.mobileconnectors.dynamodbv2.dynamodbmapper.DynamoDBMapper$SaveObjectHandler.doUpdateItem(DynamoDBMapper.java:1173)
at com.amazonaws.mobileconnectors.dynamodbv2.dynamodbmapper.DynamoDBMapper$2.executeLowLevelRequest(DynamoDBMapper.java:873)
at com.amazonaws.mobileconnectors.dynamodbv2.dynamodbmapper.DynamoDBMapper$SaveObjectHandler.execute(DynamoDBMapper.java:1056)
at com.amazonaws.mobileconnectors.dynamodbv2.dynamodbmapper.DynamoDBMapper.save(DynamoDBMapper.java:904)
at com.amazonaws.mobileconnectors.dynamodbv2.dynamodbmapper.DynamoDBMapper.save(DynamoDBMapper.java:688)
at com.amazonaws.cognito.sync.Utils.FriendsSyncManager.initalize_credentialprovider(FriendsSyncManager.java:43)
at com.amazonaws.cognito.sync.ValU.FriendListActivity$SyncFriends.doInBackground(FriendListActivity.java:168)
at com.amazonaws.cognito.sync.ValU.FriendListActivity$SyncFriends.doInBackground(FriendListActivity.java:160)
at android.os.AsyncTask$2.call(AsyncTask.java:292)
 
         
What could be the solution?   
Okey it seems you need to add:
ddbClient.setRegion(Region.getRegion(Regions.EU_WEST_1));
// Add correct Region. In my case its EU_WEST_1
after the following line:
AmazonDynamoDBClient ddbClient = new AmazonDynamoDBClient(credentialsProvider);
Now it works. The table was successfully created.
Have a nice day and thanks!
It seems that the table you are trying to connect to doesn't exist. Verify the table name in your code agains the name of the table you created.
Please note that table name is case sensative.
You need to check a few things:
Check your credentials in your code:
private static String awsSecretKey = "your_secret_key"; //get it in AWS web UI
private static String awsAccessKey = "your_access_key"; //get it in AWS web UI
Check your Region code and set correct value:
client.setRegion(Region.getRegion(Regions.US_EAST_1));
You can get this value from your AWS Web Console. Details
Check does you have already created DynamoDB table & indexes under your Region.
If no, check your code
#DynamoDBTable(tableName = "Event")
public class Event implements Serializable {
public static final String CITY_INDEX = "City-Index";
public static final String AWAY_TEAM_INDEX = "AwayTeam-Index";
And create manualy from AWS Console or somehow else your table (Event in my case) and indexes (City-Index, AwayTeam-Index in my case). Please note - table and index name in case sensative.
Good sample - https://github.com/aws-samples/lambda-java8-dynamodb
From the docs it's either you don't have a Table with that name or it is in CREATING status.
I would double check to verify that the table does in fact exist, in the correct region, and you're using an access key that can reach it.
Or you might select the wrong region.
Along with #Yuliia Ashomok's answer
AWS C++ SDK 1.7.25
Aws::Client::ClientConfiguration clientConfig;
clientConfig.region = Aws::Region::US_WEST_2;
If using Spring boot, you can configure the region via application properties:
in src/main/resources/application.yaml
cloud:
aws:
region:
static: eu-west-1
As this problem is some what platform agnostic. For anyone coming at the same problem from .NET/C# ...
You can instantiate your client with the Endpoint in the constructor:
AmazonDynamoDBClient client = new AmazonDynamoDBClient(credentials, Amazon.RegionEndpoint.USEast1 );
I would have assumed that this would have been brought in from you AWS Profile but seems not although you could do something like this, where profile is imported from your SharedCredentialsFile:
new AmazonDynamoDBClient(credentials, profile.Region );
If you are sure that you have already created the table in DynamoDB but still getting this error. That means you may chances that your region is not correct. Just look at the top right corner of your AWS portal, with your profile dropdown. One another dropdown will give you the option to select your region. And now follow the process again with the right region.
Hope this works. This works for me.

Is it possible to get a time for state transition for an Amazon EC2 instance?

I'm accessing EC2 with the aws-sdk for Ruby. I have an array of instances from describe_instances().
This provides me with the state of the instances and even a state transition reason. But how can I get a time for the state transition?
Edit
So I have:
client=Aws::EC2::Client()
resp =client.describe_instances({ filters })
and I would need
resp.reservations[0].instances[0].state_transition_time #=> Time
similar to
resp.reservations[0].instances[0].state_transition_reason #=> String
This information is not available via the Amazon EC2 API at this time. The aws-sdk gem returns all of the information available from the DescribeInstances operation as documented here: http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html
The State Transition Reason is not always populated with a date and time and may not even be populated at all per the documentation. I have not found any hints in the documentation that specify the conditions in which you DO get a date/time, but in my experience, the date/time are present in the State Transition Reason for between 30 and 90 days. After that, the reason seems to persist, but the date is dropped from the string.
All of the documentation that I can find is listed here:
Attribute Definition
EC2 API - Ruby

Apache Camel with AWS DynamoDB (aws-ddb) URI parameters

I'm trying to route messages from AWS SQS to AWS Dynamo DB using Apache Camel using the following route definition:
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="aws-sqs://my_queue?accessKey=${aKey}&secretKey=RAW(${sKey})&maxMessagesPerPoll=10&deleteAfterRead=true"/>
<unmarshal>
<camel:json library="Jackson"/>
</unmarshal>
<to uri="aws-ddb:my_table?accessKey=${aKey}&secretKey=RAW(${sKey})&readCapacity=15&writeCapacity=100&operation=PutItem"/>
</route>
</camelContext>
But on execution Camel complains that the Dynamo Db URI is missing some required parameters:
org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: aws-ddb://table?amazonDDBClient=%23ddbClient&amazonDdbEndpoint=ap-southeast-2&readCapacity=10&writeCapacity=10 due to: Status Code: 400, AWS Service: AmazonDynamoDB, AWS Request ID: KHA79STK78SHC2BG2R8HLPF7RJVV4KQNSO5AEMVJF66Q9ASUAAJG, AWS Error Code: ValidationException, AWS Error Message: 2 validation errors detected: Value null at 'keySchema.hashKeyElement.attributeName' failed to satisfy constraint: Member must not be null; Value null at 'keySchema.hashKeyElement.attributeType' failed to satisfy constraint: Member must not be null
The fun part is, these 2 parameters are not described anywhere in Camel DDB doc. I spent some time browsing the Camel source and found 2 undocumented URI parameters: keyAttributeName & keyAttributeType, which worked perfectly for me. (I hope this find of mine helps someone as well).
Now even more fun is that these should not be requested for inserting items into Dynamo DB, but I can no longer reproduce this error when I remove the 2 undocumented params from the URI.
So my questions are:
Why did AWS request hash key data for PutItem requests?
Why can't I reproduce this behaviour any longer?
I could not find any hints either in Camel or AWS documentation, googling only uncovers a handful of irrelevant results.
Could it be that you didn't specify the region your Dynamo DB table belongs too?
I had the same problem and found that my data was stored in the US East region instead of the region I had my table in.
And to answer your two questions then it would be:
Because the table didn't exist, Camel AWS tries to create it for you as can be seen in the DdbEndpoint class. Personally I don't like this side effect but it is there :/
This means that those attributes suddenly got required.
You cannot reproduce it after the first try because now it is created :)
Delete the table and you will be able to reproduce it.
So my solution was to specify my own AmazonDynamoDBClient with the region set and then put it in the registry so it could be found from the route with the parameter: amazonDDBClient
Hope it helps!