I'm trying to post a request using curl to my es cluster in AWS using my accessKey and secretKey. I have successfully done this through postman (details here) where you can specify AWS credentials but I would like to make this work with curl. Postman can auto-generate your curl request for you but all I get are errors.
This is the generated curl request along with the response
curl -X GET \
https://search-00000000000001.eu-west-1.es.amazonaws.com/_cat/indices \
-H 'Authorization: AWS4-HMAC-SHA256 Credential=11111111111111111111/20181119/eu-west-1/es/aws4_request, SignedHeaders=cache-control;content-type;host;postman-token;x-amz-date, Signature=11111111116401882398f46011f14fdb9d55e012a4fb912706d67c1111111111' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Host: search-00000000000001.eu-west-1.es.amazonaws.com' \
-H 'Postman-Token: 00000000-0000-4001-8006-9291e208a000' \
-H 'X-Amz-Date: 20181119T220000Z' \
-H 'cache-control: no-cache'
{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}%
IDs have been changed to protect the innocent.
I have checked all my keys and region, and like i said this works through postman. Is it possible to access this AWS service using my keys through curl?
This is quite a long rabbit hole. Thanks to Adam for the comment that sent me in the correct direction. The link https://docs.aws.amazon.com/apigateway/api-reference/signing-requests/ really helps you understand what you need to do.
I've since found a script that follows the signing requests method outlined above. It runs in bash and whilst it is not written for use with elasticsearch requests it can be used for them.
https://github.com/riboseinc/aws-authenticating-secgroup-scripts many thanks to https://www.ribose.com for putting this on github.
If your host contains ':443' remove it and try again.
This worked for me.
"My initial problem: If I access it with Postman using the same url, I get the same error, but removing the ‘:443/’, it works fine, so it’s nothing wrong with the key and secret I’m using."
Related
when I execute
curl --request GET "https://${ES_DOMAIN_ENDPOINT}/my_index_pattern-*/my_type/_mapping" \
--user $AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY \
--aws-sigv4 "aws:amz:ap-southeast-2:es"
where $ES_DOMAIN_ENDPOINT is my AWS Elasticsearch endpoint, I'm getting the following response:
{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}
I'm confident that my $AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY are correct.
However, when I send the same postman request with the AWS Authentication and the parameters above, the response is coming through. I compared the verbose output of both requests and they have very minor differences, such as timestamps and signature.
I'm wondering, what is wrong with the --aws-sigv4 config?
This issue happens due to the* character in the path. There is a bug report in curl repository to fix this issue https://github.com/curl/curl/issues/7559.
Meanwhile, to mitigate the error you should either remove a * from the path or build curl from the branch https://github.com/outscale-mgo/curl-appimage/tree/http_aws_sigv4_encoding.
I would like to send email with Mailgun via Postman. I write my Private API key in all different way in the Authorization section in Postman. But it always refuse me, got a HTTP 401.
What is wrong?
I tried send via CURL as tutorial suggest, but is fails also:
https://documentation.mailgun.com/en/latest/quickstart-sending.html#send-via-api
kukodajanos#Kukoda-MacBook-Pro-2 ~ % curl -s --user 'api:b3c5...' \
https://api.mailgun.net/v3/mg.tiket.hu/messages \
-F from='janosontech#gmail.com' \
-F to=kukodajanos#icloud.com \
-F subject='Hello' \
-F text='Testing some Mailgun awesomeness!'
Forbidden%
Basic authentication (as in your first screenshot) should work. (Also make sure, you don't have any spaces, newlines or other wrong characters included in your token)
Are you using your primary API key or a sending key specifically for that domain? If the first, try creating a sending key for you domain. If the latter, try recreating the key.
I have the origin http://app.mydomain.com/v2/check (no https)
I have configured my cloudfront with ssl cert *.mydomain.com
In Route53 I have created A record with alias app2.mydomain.com pointing to Cloudfront xyz.cloudfront.net
Now how to I set behavior such that I call (POST) https://app2.mydomain.com/v2/check it call http://app.mydomain.com:8081/v2/check.
curl command
Working
curl --location --request POST 'http://app.mydomain.com/v2/check' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'disabledRules=WHITESPACE_RULE' \
--data-urlencode 'text="The quick read focks jumped over the lazy broon dogg."' \
--data-urlencode 'language=en-US'
not working getting http 403 error
curl --location --request POST 'https://app2.mydomain.com/v2/check' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'disabledRules=WHITESPACE_RULE' \
--data-urlencode 'text="The quick read focks jumped over the lazy broon dogg."' \
--data-urlencode 'language=en-US'
This can be done by by modifying the origin.
In the console:
Select the Origin and Origin Groups tab.
Select the Origin and click Edit
Once you click edit there is an option for Origin Protocol Policy. Select the value of HTTP Only as shown below in the screenshot.
Click Yes/Edit
This can be performed in the CLI by performing the update-distribution function, or in CloudFormation by updating the OriginProtocolPolicy value to http-only.
As you're using POST requests ensure that it is enabled in the Allowed HTTP Methods within the Behaviour for the origin. This returns a 403 if the method is not enabled.
.
Other than this ensure the host that you're trying to connect to through either the browser or programmatically as an API call, is added as an alternative domain name in the CloudFront Distribution configuration.
In the Alternate Domain Names CNAME section I had to put app2.domain.com in the list. I had only domain.com do to which it was not working.
I have added an attestation on google cloud platform to use for image signing and attestation by the attestor and want to remove the added attestation but I can't seem to find any documentation on how this is done or even if this is how is should be done.
I have seen the one for removal of the attestor but none on removing of deleting the attestation. I had added it using the following command:
Official documentation version:
gcloud container binauthz attestations create \
--project=$PROJECT_ID \
--artifact-url="${CONTAINER_PATH}#${DIGEST}" \
--attestor=${ATTESTOR} \
--signature-file=./signature.pgp \
--public-key-id="$KEY_FINGERPRINT"
Online tutorial version:
gcloud beta container binauthz attestations create \
--artifact-url="CONTAINER_PATH#DIGEST" \
--attestor=ATTESTOT_ID \
--attestor-project=PROJECT_ID \
--signature-file=./signature.pgp \
--pgp-key-fingerprint="KEY_FINGERPRINT"
but from a more recent documentation the --attestor should include the following --attestor="projects/${ATTESTOR_PROJECT_ID}/attestors/${ATTESTOR}" and unfortunately the tutorial I am following didn't use it this way and only added the attestor_id or name. So I want to remove this version and add new one but I am getting a conflict error
Resource in project [xxxx] is the subject of a conflict: occurrence ID "f5981e62-7b42-4f57-8486-b0d9518509fa" already exists in project
So how is it to be removed.
Update: documentation used to compare to online course: https://cloud.google.com/binary-authorization/docs/making-attestations
Looks like somekind of underlying resources(the error messages indicates that) are still need to be deleted.
Found some documentation on binary authorization where they have explained complete tear down and clean-up. It looks like apart from the attestor, we need to delete some other connected resources as well.
According to the official Google Cloud Documentation on Creating attestations and project occurrences DELETE REST API method documentation, I derived the curl command to delete a specific attestations:
curl "https://containeranalysis.googleapis.com/v1beta1/projects/${ATTESTATION_PROJECT_ID}/occurrences/${OCCURRENCES_GUID}" \
--request DELETE --header "Content-Type: application/json" \
--header "Authorization: Bearer $(gcloud auth print-access-token)"
Assuming the executing user has containeranalysis.occurrences.delete permission as included in roles/containeranalysis.occurrences.editor, the response would be 200 with an empty json. I am not sure if the following header is required but at the time of my testing it wasn't.
-H "X-Goog-User-Project: ${ATTESTATION_PROJECT_ID}"
I have provided feedback to Google Cloud documentation to include my curl command above at the official Google Cloud Documentation on Creating attestations
just started with wso2-am. I published the example PizzaShackAPI: /pizzashack/1.0.0 API and registered a default application against it. I can (re)generate keys for the default application and with this, I can call e.g. the GET/menu item in the API console. This gives me the expected list as long as the access token is valid.
However in the API console the equivalent curl command is also given as :
curl -X GET --header 'Accept: application/json' --header 'Authorization: Bearer 00cb3832-f73f-3536-b287-4330a47ef4bd' 'https://192.168.1.9:8243/pizzashack/1.0.0/menu'
I run this under windows (curl-7.54.0-win64-mingw) but this doesn't work. Tried replacing some quotes with double quotes etc. but all to no avail. Furthermore, I assume that I am using one-way SSL to the service running in wso2-am itself (on port 8243) and I don't need to supply any certificate myself. I also realize that the server side uses a self signed certificate and not sure whether this has any implications in this situation.
Hope someone can help me out on this.
As an addition - just noticed that in the sys$output of the wso2-am server the following is listed:
[2017-05-13 00:38:38,858] ERROR - SourceHandler I/O error: Received fatal alert:
unknown_ca
javax.net.ssl.SSLException: Received fatal alert: unknown_ca
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1639)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1607)
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1776)
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1068)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:890)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:764)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at org.apache.http.nio.reactor.ssl.SSLIOSession.doUnwrap(SSLIOSession.ja
va:245)
I think that this is caused by the curl command hitting the API service .
Thanks Peter
You are accessing HTTPS url of the API, You have to either use a keystore or suppress the SSL in cURL with -k option
curl -X GET --header 'Accept: application/json' --header 'Authorization: Bearer 00cb3832-f73f-3536-b287-4330a47ef4bd' 'https://192.168.1.9:8243/pizzashack/1.0.0/menu' -k
You also can use below HTTP URL of the API, note the port difference.
curl -X GET --header 'Accept: application/json' --header 'Authorization: Bearer 00cb3832-f73f-3536-b287-4330a47ef4bd' 'http://192.168.1.9:8280/pizzashack/1.0.0/menu'