I am new to groovy scripting and I have to test AWS Rest API for which Authorization (consist of access_key and signature) is required. Request if anyone have working code available. It is PUT call on AWS S3.
I have tried searching and unable to find the same.
If you can afford it, the latest versions of soapUI Pro have AWS signing built it.
Otherwise, you'll have to write your own following the steps at Examples: Signature Calculations in AWS Signature Version 4. There's an example at Signature4 signing for elastic search in aws in Java, but it's easy enough to call from Groovy.
Related
I've been exploring OneLogin and I think I have a basic understanding of how to use it; I was successful at configuring the Node+Passport sample application to authenticate with OneLogin and I understand the code.
I want to try it with AWS API Gateway. I've got an API already defined in AWS. On a high level I know I need to configure an Authorizer in AWS for OneLogin and that there is going to be some bit of information that I will need to pass in a header when I make my calls to the AWS Api Gateway (I assume a Bearer token that I get from authenticating with OneLogin before I interact with AWS, but I don't know).
Can someone give me a description of what I need to do? The most recent question that I see here on the topic was posted in 2016 back when the answer was "You can't do that." That answer has changed in the past 4 years. The OneLogin documentation says it can be done, but I have found not information on how to do it.
GENERAL PATTERN
The role of an authorizer function is to perform the following actions:
Validate access token
Optionally collect additional claims
Cache the claims for subsequent requests with the same token
Provide claims to business logic
In fact this is a general pattern that can be used anywhere, as referenced in my blog post.
TOKEN VALIDATION
I would use the One Login Introspection Endpoint.
Some NodeJS code of mine that calls an introspection endpoint is here.
AWS SPECIFICS
Since you are using OneLogin you will need to write some custom code. Your lambda authorizer will need to do standard OAuth 2.0 work, then return an AWS policy document. Some resources of mine may help you to understand how this works:
Lambda Authorizer Blog Post
Associated NodeJS Code
PAIN POINTS
The AWS plumbing is a bit painful, and it is possible you are using some different technology choices to me - I am using the Serverless Framework.
Hopefully though, this gives you something to run locally and compare against. Happy to answer any follow up questions ...
I am trying to get the list of userpools in aws cognito this error code arrises
Host URL: https://cognito-idp.us-east-1.amazonaws.com/ListUserPools?Version=2016-04-18
I am using postman it calculates aws signature correctly so no problem in that...
AWS ListUserPools : https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ListUserPools.html
Where to place that request syntax on the documentation link ????
You have to either use the AWS CLI or any of the AWS SDKs to make this call. You can't just do it using postman. Scroll down to the documentation link you have posted. Now select any of the SDKs that you want to develop using. Then follow the documentation for that language.
Google Cloud gsutil iam get gs://testBucket command should return bucket policy, but instead received "Failure: GetBucketIamPolicy must be overloaded"
Verified storage.buckets.GetIamPolicy and storage.buckets.setIamPolicy are in placed.
Any help or suggestion is appreciated.
That functionality only exists in the JSON API; it sounds like you've somehow managed to get gsutil to try using the XML API to make this call.
Here's the base API client class:
https://github.com/GoogleCloudPlatform/gsutil/blob/0e4bdc80f90f42edd86c3da772c22087e63b21be/gslib/cloud_api.py#L84
And here are the subclasses that implement functionality for the JSON and XML API (note that GetBucketIamPolicy is only implemented in the JSON API's client class):
https://github.com/GoogleCloudPlatform/gsutil/blob/0e4bdc80f90f42edd86c3da772c22087e63b21be/gslib/gcs_json_api.py#L334
https://github.com/GoogleCloudPlatform/gsutil/blob/0e4bdc80f90f42edd86c3da772c22087e63b21be/gslib/boto_translation.py#L160
My best guess is that you have HMAC credentials configured in your boto file, rather than OAuth2 credentials. This will force gsutil to use the XML API (since HMAC credentials only work for that API), regardless of whether the command is supposed to support the XML API. The iam command is supposed to only support the JSON API, but it looks like we didn't add a test for the edge case where only HMAC credentials were configured.
I've filed https://github.com/GoogleCloudPlatform/gsutil/issues/846 to track this bug in gsutil.
"Failure: GetBucketIamPolicy must be overloaded"
This error means that the function GetBucketIamPolicy is not implemented in the gsutil program.
This indicates that the Google Cloud SDK is not installed correctly, Python is not set up correctly, or you have external libraries with name conflicts with the Google libraries.
Note: I am not confirmed this yet: yesterday there was an internal issue mentioned about gsutil. If you are using the latest version, try going back to a release from two weeks ago.
Previous versions
Previous versions of Cloud SDK are available in the download archive in Google Cloud Storage.
#mhouglum, #John Hanley I was able to replicate the issue on a different machine and the solution is to issue "gcloud config set pass_credentials_to_gsutil true" command
Like #mhouglum said, gsutil will try to read the OAuth credentials first from "gcloud auth login" but since the pass_credentials_to_gsutil" is set to false, it will read the HMAC credentials from the .boto file which doesn't support the XML API.
Thank you both for your time and efforts.
I'm trying to invoke an api request (service: execute-api) and a Signature v4 is required. I've been going through the documentation and I see clearly this:
Alternatively, you can use the AWS CLI or one of the AWS SDKs to
handle request signing for you.
I don't own the API and originally just thought I could use CURL but obviously IAM is configured. I'm wondering what the best way of making this request signed is?
Note:
Looks like there is an AWS4Signer class that may be what I'm looking for to generate the signature non-manually
I'm sorry, but from your question it's not clear whether you're just experimenting with the API or if you want to write a client that calls it (as in production code).
If you're just testing, you can use Postman to call the API (it supports SigV4). Details here.
If you are writing a client, the way to go is generating the SDK from API gateway, as noted in the comments. Should that not be possible, the next best option is to use one of the language-specific SDK signers to generate the SigV4 signature. AWS4Signer, like you said, is the way to go. It should be straightforward to integrate with it, but if you can share more details of your specific use case (platform, language, where do you get the AWS credentials from, etc), people can give you a better answer.
Last, if you want to generate the signature yourself, here's how the canonical generation of signatures work.
I've gotten the necessary access key/signature from my client and I can interact with the API through the Ruby SDK right now. Thing is, the Ruby SDK doesn't have any kind of high-level API methods to request a spot instance. So, I need to do this manually via raw REST API requests.
Basically, the authentication information that I am using is correct (as it works via the Ruby SDK), but I can't get raw requests to work... I either get back the spot request wizard webpage as the response, or an error: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
Here's the URL I'm using:
https://ec2.amazonaws.com/?Action=RequestSpotInstances
&SpotPrice.1=0.05
&AvailabilityZoneGroup.1=us-east-1c
&LaunchSpecification.ImageId.1=THE_AMI_ID
&LaunchSpecification.KeyName.1=THE_KEYPAIR
&LaunchSpecification.InstanceType.1=m1.medium
&AWSAccessKeyId=THE_ACCESS_KEY
&Signature=THE_ACCESS_SIGNATURE
&Version=>2013-10-01
&Expires=>THE_EXPIRATION_TIME_36000_SECONDS_LATER_THAN_NOW
&SignatureVersion=2
&SignatureMethod=HmacSHA256
Any ideas on why this won't work? I've tried exploring the Ruby SDK code to see how they are doing it, but it's so complex, I can't figure out where this action actually takes place. Thanks!
How do you calculate signature? First at all check that you use correct signing process version. AWS api actually supports versions v2 and v4. Some aws resources supports both versions, some just v2 or v4. Base on this I would recommend to do following:
Check what version of the signing did you implement. More on versions:
http://docs.aws.amazon.com/general/latest/gr/signing_aws_api_requests.html
Check is your implementation match with algorithm described here: http://docs.aws.amazon.com/general/latest/gr/signature-version-2.html