So I got access to SimilarWeb ranking API from AWS(https://aws.amazon.com/marketplace/pp/prodview-clsj5k4afj4ma?sr=0-1&ref_=beagle&applicationId=AWSMPContessa).
I'm not able to figure out how to pass the authentication or how to give a request to retrieve the ranks for domains.
For ex. how will you pass the request for this URL in python?
URL: https://api-fulfill.dataexchange.us-east-1.amazonaws.com/v1/v1/similar-rank/amazon.com/rank
This particular product does not seem to be available any longer. Generally speaking, an AWS IAM principal with correct IAM permissions, can make API calls against AWS Data Exchange for APIs endpoints. The payload of the API call needs to adhere to the OpenAPI spec defined within the DataSet of the product used. The specific API call is 'SendApiAsset'. The easiest way to think about is to read the boto3 documentation for it, here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dataexchange.html#DataExchange.Client.send_api_asset
Other AWS SDKs have the same call, idiomatic to the specific language.
The managed policy that describes the IAM permissions needed is named AWSDataExchangeSubscriberFullAccess, the dataexchange specific permission needed is 'dataexchange:SendApiAsset'.
The awscli way of making the call is described here: https://docs.aws.amazon.com/cli/latest/reference/dataexchange/send-api-asset.html
The required parameters are: asset-id, data-set-id, revision-id. You will likely also need to provide values for: method and body (and perhaps others also depending on the specific API you are calling.
The content of the 'body' parameter needs to adhere to the OpenAPI spec of the actual dataset provided as part of the product.
You can obtain the values for asset-id, data-set-id and revision-id from the AWS Data Exchange service web console describing the product/dataset.
Related
I am trying to use the AWS SSO SCIM API in order to retrieve the AWS SSO groups for a particular user.
Looking at the documentation https://docs.aws.amazon.com/singlesignon/latest/developerguide/listgroups.html
It mentions the following
To see group info for a certain member, call ListGroup with a member filter
If you're using the member filter, you have to use the id filter (refer to supported filter combinations).
It appears that the id filter, is the group ID, because of this note: Note that the use of id as an individual filter, while valid, should be avoided as there is already a getGroup endpoint available.
Also, at https://docs.aws.amazon.com/singlesignon/latest/developerguide/limitations.html next to the members attribute it mentions that it is supported, but cannot be read in a response.
I have been playing with the API but could not get any group info for a user regardless of the combination of the parameters.
Based on the above, I conclude that the API does not support retrieving group data for a user. Would you agree?
I don't have access to an AWS SSO SCIM endpoint, but from looking at the documentation I suspect one of the two approaches may be possible:
GET /users/id and look at the value for "groups" - this one seems questionable as the limitations page says they support the user resource's "groups" attribute, but there's also a note about not supporting multi-valued user attributes. You may need to explicitly add the ?attributes=groups parameter to your GET to ask for that attribute to be returned, if it is in fact supported.
GET /groups?filter=members[value eq "userId"] or some variation of that depending on what syntax they allow. This isn't listed as supported, but if it works, I expect that you would get a list of groups in response that did not contain the "members" attribute - but that each group returned would match the query.
There's some amount of uncertainty with both of these, but with the information available those seem like the two most likely paths to successfully accomplish what you are aiming for. If neither of those work, it may not be possible - but a support case with AWS would probably be helpful to confirm there.
As #ZollnerdMSFT recommended, I raised an AWS support request. AWS support responded that the AWS SSO SCIM API does not support retrieving the groups associated with a user. They have submitted this as a feature request internally, however, cannot provide an estimate as to when it will be implemented.
In the scenario of listing all versions of an object using its key as a prefix:
import boto3
bucket = 'bucket name'
key = 'key'
s3 = boto3.resource('s3')
versions = s3.Bucket(bucket).object_versions.filter(Prefix=key)
for version in versions:
obj = version.get()
print(obj.get('VersionId'), obj.get('ContentLength'), obj.get('LastModified'))
Do I get charged only for listing the objects that are matching the prefix?
If so, is each object/version listed treated as a separate list request?
No, each object/version listed is not treated as a separate list request. You're only paying for the API requests to S3 (at something like $0.005 per 1000 API requests). A single API request will return many (up to 1000) objects/versions that match the indicated prefix. The prefix filtering itself happens server-side in S3.
The way to get a handle on this is to understand that AWS SDK calls ultimately result in API requests to AWS service endpoints e.g. S3 APIs. What you need to do is work out how your SDK client requests map to the underlying API requests to determine what is likely happening.
If your request is a simple 'list objects in my bucket' case, the boto3 SDK is going to make one or more ListObjectsV2 API calls. I say "or more" because the SDK may need to make more than one API request because API requests typically yield a maximum number of results (e.g. 1000 objects in a ListObjectsV2 response). If there are 2500 objects in the bucket, for example, then three ListObjectsV2 requests would need to be made to the S3 API.
If your request is 'list objects in my bucket with a given prefix', then you need to know what capabilities are present on the ListObjectsV2 API call. Importantly, prefix is one of the parameters. This is how you know that S3 itself is doing the filtering on your supplied prefix (where you have indicated .filter(Prefix=key) in your code). If this were not a feature of the underlying S3 API, then your SDK (boto3 etc.) would be the one doing the filtering on prefix and that would be a much more expensive and vastly slower operation, because the SDK would have to list all objects, potentially resulting in many more LIST requests, and filter them client-side. Note: the ListObjectVersions API is similar to ListObjectsV2 in this regard and both support prefix.
Also, note that VersionId, Size, and LastModifed are all attributes that appear in the ListObjectVersions response, so no further API requests are needed to fetch this information.
So, in your case, assuming that there are fewer than 1000 object versions that match your indicated prefix, I believe that this equates to one S3 API request to ListObjectVersions (and this is considered a LIST request rather than a GET request for billing afaik, even though it is a GET HTTP request to https://mybucket.s3.amazonaws.com/?versions under the covers).
I a newbie to AWS. My requirement is to add field-level, document-level, and collection-level permissions for reads and writes for AWS DocumentDB. One collection should accessible only from one microservice and the document should be modified only by the owner of the document(i,e user document can be modified only by that user)
I have done enough research and found, adding a restriction for accessing a DB can be done using Role-Based-Access-Control if we want to allow only for one tenant, but didn't get a clear idea of my problem statement i.e managing collection-level, document-level, and field-level permissions
Is there any other way to achieve this..?
Any help will be appreciated
I have a requirement to build a basic "3 failed login attempts and your account gets locked" functionality. The project uses AWS Cognito for Authentication, and the Cognito PreAuth and PostAuth triggers to run a Lambda function look like they will help here.
So the basic flow is to increment a counter in the PreAuth lambda, check it and block login there, or reset the counter in the PostAuth lambda (so successful logins dont end up locking the user out). Essentially it boils down to:
PreAuth Lambda
if failed-login-count > LIMIT:
block login
else:
increment failed-login-count
PostAuth Lambda
reset failed-login-count to zero
Now at the moment I am using a dedicated DynamoDB table to store the failed-login-count for a given user. This seems to work fine for now.
Then I figured it'd be neater to use a custom attribute in Cognito (using CognitoIdentityServiceProvider.adminUpdateUserAttributes) so I could throw away the DynamoDB table.
However reading https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-dg.pdf the section titled "Configuring User Pool Attributes" states:
Attributes are pieces of information that help you identify individual users, such as name, email, and phone number. Not all information about your users should be stored in attributes. For example, user data that changes frequently, such as usage statistics or game scores, should be kept in a separate data store, such as Amazon Cognito Sync or Amazon DynamoDB.
Given that the counter will change on every single login attempt, the docs would seem to indicate I shouldn't do this...
But can anyone tell me why? Or if there would be some negative consequence of doing so?
As far as I can see, Cognito billing is purely based on storage (i.e. number of users), and not operations, whereas Dynamo charges for read/write/storage.
Could it simply be AWS not wanting people to abuse Cognito as a storage mechanism? Or am I being daft?
We are dealing with similar problem and main reason why we have decided to store extra attributes in DB is that Cognito has quotas for all the actions and "AdminUpdateUserAttributes" is limited to 25 per second.
More information here:
https://docs.aws.amazon.com/cognito/latest/developerguide/limits.html
So if you have a pool with 100k or more it can create a bottle neck if wanted to update a Cognito user records with every login etc.
Cognito UserAttributes are meant to store information about the users. This information can then be read from the client using the AWS Cognito SDK, or just by decoding the idToken on the client-side. Every custom attribute you add will be visible on the client-side.
Another downside of custom attributes is that:
You only have 25 values to set
They cannot be removed or changed once added to the user pool.
I have personally used custom attributes and the interface to manipulate them is not excellent. But that is just a personal thought.
If you want to store this information, and not depend on DynamoDB, you can use Amazon Cognito Sync. Besides the service, it offers a client with great features that you can incorporate to your app.
AWS DynamoDb appears to be your best option, it is commonly used for such use cases. Some of the benefits of using it:
You can store separate record for each login attempt with as much info as you want such as ip address, location, user-agent etc. You can also add datetime that can be used by pre-auth Lambda to query by time range for example failed attempt within last 30 minutes
You don't need to manage table because you can set TTL for DynamoDb record so that record will be deleted automatically after specified time.
You can also archive items in S3
Simple question, but I suspect it doesn't have a simple or easy answer. Still, worth asking.
We're creating an implementation for push notifications using AWS with our Web Server running on EC2, sending messages to a queue on SQS, which is dealt with using Lambda, which is sent finally to SNS to be delivered to the iOS/Android apps.
The question I have is this: is there a way to query SNS endpoints based on the custom user data that you can provide on creation? The only way I see to do this so far is to list all the endpoints in a given platform application, and then search through that list for the user data I'm looking for... however, a more direct approach would be far better.
Why I want to do this is simple: if I could attach a User Identifier to these Device Endpoints, and query based on that, I could avoid completely having to save the ARN to our DynamoDB database. It would save a lot of implementation time and complexity.
Let me know what you guys think, even if what you think is that this idea is impractical and stupid, or if searching through all of them is the best way to go about this!
Cheers!
There isn't the ability to have a "where" clause in ListTopics. I see two possibilities:
Create a new SNS topic per user that has some identifiable id in it. So, for example, the ARN would be something like "arn:aws:sns:us-east-1:123456789:know-prefix-user-id". The obvious downside is that you have the potential for a boat load of SNS topics.
Use a service designed for this type of usage like PubNub. Disclaimer - I don't work for PubNub or own stock but have successfully used it in multiple projects. You'll be able to target one or many users this way.
According the the [AWS documentation][1] if you try and create a new Platform Endpoint with the same User Data you should get a response with an exception including the ARN associated with the existing PlatformEndpoint.
It's definitely not ideal, but it would be a round about way of querying the User Data Endpoint attributes via exception.
//Query CustomUserData by exception
CreatePlatformEndpointRequest cpeReq = new CreatePlatformEndpointRequest().withPlatformApplicationArn(applicationArn).withToken("dummyToken").withCustomUserData("username");
CreatePlatformEndpointResult cpeRes = client.createPlatformEndpoint(cpeReq);
You should get an exception with the ARN if an endpoint with the same withCustomUserData exists.
Then you just use that ARN and away you go.