I have an Event Hub and I want to restrict who can and cannot publish and consume events. The publishing of events works as described here: https://learn.microsoft.com/en-us/rest/api/eventhub/get-azure-active-directory-token
I am able to publish events by supplying a JWT to my Event Hub:
https://${Hub-NameSpace}.servicebus.windows.net/${EventHub}/messages
Example:
curl --location --request POST 'https://${Hub-NameSpace}.servicebus.windows.net/${EventHub}/messages' \
--header 'Authorization: Bearer ${TOKEN}' \
--header 'Content-Type: application/atom+xml;type=entry;charset=utf-8' \
--data-raw '{"Data":"Data"}'
However, when I try and consume using the same URL (sending a GET instead of a POST), I receive the following:
<Error>
<Code>400</Code>
<Detail>The requested HTTP operation is not supported in an EventHub. TrackingId: SystemTracker:EventHub:, Timestamp:2022-07-15T00:04:56</Detail>
</Error>
Example:
curl --location --request GET 'https://${Hub-NameSpace}.servicebus.windows.net/${EventHub}/messages' \
--header 'Authorization: Bearer ${TOKEN}' \
--header 'Content-Type: application/atom+xml;type=entry;charset=utf-8'
I am aware that I can consume using the Shared Access Policy endpoints, but I was wondering if I could also do it using the Authentication Token from AAD.
I have a feeling that I should only need to provide the Consumer Group, but I cannot find any documentation on doing this.
In the previously linked article it states:
This article gives you an example of getting an Azure Active Directory (Azure AD) token that you can use to send events to and receive events from a Service Bus namespace.
However it shows no examples on receiving events.
Receiving events is not supported using the Event Hubs REST API.
In order to consume events, you'll need to either use the AMQP or Kafka protocols. The easiest path to do so is using one of the official SDKs. More information can be found in the Event Hubs Getting Started Guide.
Related
I was hoping to see some examples on how to implement Dynamic client registration that generates client credentials for an OAuth2 flow on AWS.
Any references?
DCR generally works like this, but can vary quite a bit depending on the client scenario. Hopefully this gives you the general flavour of things:
STEP 1: GET A DCR ACCESS TOKEN TO USE FOR REGISTRATION
Typically this involves an initial OAuth flow with a dcr scope, to get a DCR access token, needed to register:
curl -X POST https://login.example.com/oauth/v2/token \
-H: "Content-Type: application/x-www-form-urlencoded" \
-H: "Authorization: some-credential" \
-d client_id=initial_client
-d scope=dcr
STEP 2: REGISTER AND GET A UNIQUE CLIENT
This involves a POST with a JSON request and response, simplified below:
curl -X POST https://login.example.com/oauth/v2/registration \
-H: "Authorization: dcr-access-token" \
-H: "Content-Type: application/json" \
-d grant_types=[client_credentials] \
-d scope='openid profile transactions'
The response then typically has quite a large payload, but in particular a unique client ID and secret are returned:
client_id: cj348034t534tu90
client_secret: h7890g723470fghfgh42rt
other-fields: ...
STEP 3: USE THE UNIQUE CLIENT
From that point onwards the client just uses the client ID and secret to get application level tokens:
curl -X POST https://login.example.com/oauth/v2/token \
-u "cj348034t534tu90:h7890g723470fghfgh42rt" \
-H: "Content-Type: application/x-www-form-urlencoded" \
-d scope='openid profile transactions'
DYNAMIC BUSINESS
DCR is quite a powerful concept and one scenario that showcases it is Open Banking, where approved merchants can automatically onboard and start calling bank APIs. In this case the initial DCR access token is retrieved based on Mutual TLS trust and other regulatory checks.
PROVIDERS
DCR features are provided by more advanced OAuth providers and I doubt it is supported by Amazon. Curity has plenty of resources on the design patterns though, so if you want more info, see these resources:
Curity Website - with a search for DCR
Introductory Video
I have successfully configured the API Manager, ID and IS according to the documentation: https://docs.wso2.com/display/AM260/JWT+Grant#JWTGrant-UsingtheJWTgrant.
I invoke the WSO2 token endpoint to exchange an external JWT for a WSO2 access token:
curl -i -X POST -k -d 'grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&assertion=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6WyJ3c28yIl0sImVudmlyb25tZW50Ijp7InByb2ZpbGVzIjpbImRldiJdfSwiYXV0aCI6eyJyb2xlcyI6WyJVU0VSIiwiQURNSU4iXX0sInVzZXJfbmFtZSI6IjAwMDAwMDk5Iiwic2NvcGUiOlsib3BlcmF0ZSJdLCJpc3MiOiJQQVMiLCJleHAiOjE1NDUzNDgyODcsImdlbmVyYXRlZEJ5IjoiUEFTIiwianRpIjoiOWQ4ZWU3ZTgtNDBlZS00MTZjLTlkYjgtYjU2NDZhYTZhN2JmIiwiY2xpZW50X2lkIjoiZnJvbnQtcG9saXphcyJ9.Ccs1OxjteRsvHTump-ZTawEsqlTrIeO0LJUzt5Ita8udvMOa_tB1rHOtI8GAa2mDCPMD_Z_jtZ2SlXPs10GvsYlF4jS_wcCVAPtHsoigzuNtg5t7CVfeCI2Bzhak721LdYBcjB9s0Jn24G9eb2jqx8NF0RPlKgmhbxwdY0b8XeigLp-kGCsFKY_fDIjFUM0oifzCWOmtaCRMtMx3CKVZOWq9dBIokheCi2foL8YkBCz57yo4vb782AYWXdiHj38TPPe4IguARuoc9FSymyiL1gWHJmyMZFvAeAJkDnHHEnnezqPmcWQweC1ylLwUYGNVLM8YSfuBDtcGBWSO0F-WKw' -H 'Content-Type: application/x-www-form-urlencoded' https://localhost:9443/oauth2/token -d 'client_id=w_paekjnDDY8zcCfCRgj_81g2eYa'
This answers successfully with an access token, a refresh token etc etc.
I have created an application in the WSO2 APIM store. In the production/sandbox tabs, the only checked Grant Type item is JWT.
The point is, I use the previously gotten access token (which is itself an JWT token) to invoke an API subscribed with the above application:
curl -k -X GET "https://192.168.179.129:8243/myapp/api/v1/customers" -H "accept: application/json;charset=UTF-8" -H "Authorization: Bearer eyJ4NXQiOiJOVEF4Wm1NeE5ETXlaRGczTVRVMVpHTTBNekV6T0RKaFpXSTRORE5sWkRVMU9HRmtOakZpTVEiLCJraWQiOiJOVEF4Wm1NeE5ETXlaRGczTVRVMVpHTTBNekV6T0RKaFpXSTRORE5sWkRVMU9HRmtOakZpTVEiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6IndfcGFla2puRERZOHpjQ2ZDUmdqXzgxZzJlWWEiLCJuYmYiOjE1NDUzNDgxOTgsImF6cCI6IndfcGFla2puRERZOHpjQ2ZDUmdqXzgxZzJlWWEiLCJzY29wZSI6ImRlZmF1bHQiLCJpc3MiOiJodHRwczpcL1wvbG9jYWxob3N0Ojk0NDNcL29hdXRoMlwvdG9rZW4iLCJleHAiOjE1NDUzNDgyODcsImlhdCI6MTU0NTM0ODE5OCwianRpIjoiNjRlM2I5N2UtOTNlNC00YzQ2LThlNmQtMzlmZjQzOWQxM2Y0In0.UBLOsCCD3t4Wf8nXBnDkkGXxefYySelDzEcs1F_IrbshMJXohxcL92Av1nmcpdNdjin7GdC8Y305rrkBt9T1L_cMAHLYYcI5cI1J7wmAgEd1CEv9gI7IUYfAdbga2AeV4kIlNsgiV6PKnU34WnY7rEVqXD908eEHY5UvaNXc0Bz6C8d-p39-SqKUblGHPh9vdkpcCGcK0CgGKjtiU2lai_JkRALdgEgonT37R5eqmuxPxUouWNz9TCJgTuonKPA-9bYOsMvbzGlm--0m0j9gdxnv-3N1Kv_2JqSCR4pToDClhSKgFCE1L025LIICM-sLd_PDU5pwYge_iKseiIDZfA" -d 'client_id=w_paekjnDDY8zcCfCRgj_81g2eYa'
I get the following error (900908) - Resource forbidden:
<ams:fault xmlns:ams="http://wso2.org/apimanager/security"><ams:code>900908</ams:code><ams:message>Resource forbidden </ams:message>
<ams:description>Access failure for API: /myapp/api/v1, version: v1 status: (900908) - Resource forbidden </ams:description></ams:fault>%
I must be missing the final step which is how to allow those access tokens gotten in the JWT grant to be used to access an API subscribed by an application.
The error code 900908 means the API is not subscribed by the application. Please double check.
Technically Bee was right in his answer, but I would like to point specifically at what I was doing wrong in case it happens to others:
The problem was that the client_id/client_secret I was using when exchanging the JWT to get the access token were the ones from the Service Provider I had created. WRONG!
The ones was that need to be sent are those from the subscribed application. With that the resource forbidden error doesn't show up anymore.
I tried to access Google's Datastore through their REST Api. It says that they allow authentication through the API-key. However it doesn't seems that I can get it to work any where. I copied the snippet generated from their Try this API page.
curl --request POST \
'https://datastore.googleapis.com/v1/projects/PROJECT_ID:runQuery?key=[YOUR_API_KEY]' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data '{"query":{"filter":{"compositeFilter":{"op":"AND","filters":[{"propertyFilter":{"property":{"name":"id"},"op":"EQUAL","value":{"stringValue":"ID"}}}]}},"kind":[{"name":"NAME"}]},"partitionId":{"namespaceId":"NAMESPACE_ID","projectId":"PROJECT_ID"}}' \
--compressed
But it keeps returning me an 401 error.
{
"error": {
"code": 401,
"message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
"status": "UNAUTHENTICATED"
}
}
It looks like it require me to use OAuth instead, which is not what their documentation says. Anyone experienced something similar?
You are using an API key which is incorrect.
This link details which services support API Keys. Cloud Datastore is not one of them.
Using API Keys
You want to use an Access Token which is derived from Service Account credentials.
Review this document.
Using OAuth 2.0
The steps to generate an Access Token:
Load the service account credentials json file.
Extract the client_email, private_key and private_key_id.
Create a json payload.
Call the authorization URL: https://www.googleapis.com/oauth2/v4/token
This returns a json object. Extract the access_token.
Use the access_token instead of an API Key.
There are examples on the Internet in various languages. The link will get you started. The process appears complicated, and it is, but once you understand it, generating Access Tokens is easy and they can be reused until they expire (typically 60 minutes which you control).
This document on Google Cloud Storage authentication is the same for Cloud Datastore. The key is understanding "Authorization: Bearer" which is a header you need to include with your curl request.
Authentication
We are developing an application and we are using google API shorter URL to reduce very long URL.
This was working fine until a week ago. The API started to return 403 Unauthorized.
The problem appeared when we switched API key. It was working fine when we were testing but failed in production.
We checked that the API keys are correct, and they are. We are testing request with postman. Credit card is also correct but we are still under the free plan, so nothing wrong with payment.
API key is not restricted, and registered for the for the Google Shorter API.
The dashboard is showing our requests but all are in error.
The request we made :
curl -X POST \
'https://www.googleapis.com/urlshortener/v1/url?key=TheApIKeyStaNDshEre' \
-H 'Accept: application/json' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{
"longUrl": "http://example.com/foo/bar/log/url/with?params=true"
}'
Someting the team hasn't see is that the API key has been regerated AFTER the limit.
From google blog post :
Starting May 30, 2018, only projects that have accessed URL Shortener APIs before today can create short links.
We missed this part when they decided to regenerate the key, and now we are locked out.
So we'll go for an hot fix to switch to firebase.
Don't regenerate your keys or you will be locked out forever.
(sorry #Ricky, we missed the part, you were right)
Google is shutting down the service. This could be the reason if your code worked properly before.
https://developers.googleblog.com/2018/03/transitioning-google-url-shortener.html
I'm attempting to use Postman, the chrome extension api client Postman to access Dynamodb via http. This is actually an approach I read about in the Book Dynamodb Applied Design Patterns. It appears that I am doing something wrong because I am unable to sucessfully authenticate and gain access to the web service.
The url and headers I'm using in the Postman client with my most recent request parameters and error messages follow:
url: dynamodb.us-east-1.amazonaws.com
x-amz-date: 20150701T162011Z
x-amz-targe: DynamoDB_20120810.ListTables
authorizaton: AWS4-HMAC-SHA256 Credential=AMyAccessCode/20150701/us-east-1/dynamodb/aws4_request, SignedHeaders=host;x-amz-date;x-amz-target, Signature=8ngTnF8WH//njvBdY5bY5dSp5CAKi8qTXNFuv5Ws+30=
content-type: application/x-amz-json-1.0
The Body of my request is:
{
"ExclusiveStartTableName": "Owner",
"Limit": 3
}
I'm getting the following error message:
{"__type":"com.amazon.coral.service#InvalidSignatureException","message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'POST\n/\n\nhost:dynamodb.us-east-1.amazonaws.com\nx-amz-date:20150701T162011Z\nx-amz-target:DynamoDB_20120810.ListTables\n\nhost;x-amz-date;x-amz-target\nb9e264461dcb0e94e69652f8b2d17c737a29506863d6f09c0f9fc98e9d560e5c'\n\nThe String-to-Sign should have been\n'AWS4-HMAC-SHA256\n20150701T162011Z\n20150701/us-east-1/dynamodb/aws4_request\n7a6da3d9e8ed6317e0cb9217e9ea1174d01e86871a159f339f5f6969283264d5'\n"}
Question/Oddity 1. When I issue a request I usually get response back indicating that my request has expired. The error message will include "valid" times that are four hours ahead of the actual time tha I used. When I use the time provided by the error message in the request, I no longer receive the exprired request error message.
Question 2. This message seems to indicate I have to calculate the hash of the canonical request. The documentation on amazon is a bit unclear on this. Do I need to calcuate the has of the canonical request and if so do I include that as a header" If so, what is the header name?
I have been able to calculate the signature and replicate the signature in the Amazon documentation.
Thanks for your input.
I was able to invoke using Postman 3.
CURL Exmaple:
curl -X POST \
https://dynamodb.eu-west-1.amazonaws.com/ \
-H 'authorization: AWS4-HMAC-SHA256 Credential=ABCDDEFSDFDDFDAG/20170725/eu-west-1/dynamodb/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date;x-amz-target, Signature=7402a0163c267385b7270f5c6ce49748518cac4f4d7e03addbd9ddedf9c9970' \
-H 'cache-control: no-cache' \
-H 'content-type: application/json' \
-H 'host: dynamodb.eu-west-1.amazonaws.com' \
-H 'postman-token: 0152b36c-2947-2a6f-91b9-87884b38fb50' \
-H 'x-amz-date: 20170725T181135Z' \
-H 'x-amz-target: DynamoDB_20120810.GetItem' \
-d '{
"TableName": "TableName",
"Key": {
"id": {"S": "KeyId"}
}
}'
Insert Body
Enter Credentials and Click on Update Request.You might need to click on Update Request more often to update x-amz-date
Update Content-Type Header to application/json
You're on the right path. As you imply, you're running into issues with the API authentication scheme that Amazon uses. See http://docs.aws.amazon.com/general/latest/gr/signature-version-4.html.
The way this works is that Amazon wants you to take a few pieces of information including the service you're calling and current timestamp and compute a hash that uses this data along with your AWS Secret Access Key (actually, they derive a key from your Secret Access Key, but don't worry about that for now).
One of the key design principles going on here is that Amazon wants to prevent Replay Attacks, where someone just sends a request you already sent, even if they can't read it. Amazon expects you to report the current timestamp in the request header, you'll include that exact same timestamp in the hash, and when Amazon receives your request, they will try to re-compute your hash using all the info they've got.
If the hash matches AND the reported timestamp is within 15 minutes of the current time, you're accepted. I suspect the 15-minutes issue is one of your problems.
As far as how to simulate these calls. I think you'll find it's probably easier to just write a very basic RESTful API of your own that does these calls using an SDK published by Amazon. Then you don't have to deal with any of the crazy signature computation and you can call your APIs however you want.
Copy-paste from https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-use-postman-to-call-api.html
Launch Postman.
Enter the endpoint URL of a request in the address bar and choose the appropriate HTTP method from the drop-down list to the left of the address bar.
3.If required, choose the Authorization tab. Choose AWS Signature for the authorization Type. Enter your AWS IAM user's access key ID in the AccessKey input field. Enter your IAM user secret key in SecretKey. Specify an appropriate AWS region that matches the region specified in the invocation URL. Enter execute-api in Service Name.
4.Choose the Headers tab. Optionally, delete any existing headers. This can clear any stale settings that may cause errors. Add any required custom headers. For example, if API keys are enabled, you can set the x-api-key:{api_key} name/value pair here.
5.Choose Send to submit the request and receive a response.