I'm using the official GCP PubSub emulator to test integration locally.
I'd like to send messages via classic curl/postman tools but it is getting complicated because this emulator requires encryption of incoming messages.
For instance, if we send it like this:
curl --location --request POST 'http://localhost:8091/v1/projects/my-project/topics/transactions:publish' \
--header 'Content-Type: application/json' \
--data-raw '{"messages":[{"data":"{\"foo\":\"baz\"}","attributes":{}}]}'
Then, I'm getting 400:
{
"error": {
"code": 400,
"message": "Payload isn't valid for request.",
"status": "INVALID_ARGUMENT"
}
}
due to invalid incoming messages. It requires encryption and if I sniff the encrypted body it works.
But it is overwhelming to encrypt messages running it locally.
In order to disable encryption in GCP I can follow this guide but it is not applicable to local emulators run - there is no GCP environment or I don't know how to do it.
Are there any options to disable emulator decryption? If not where to report it, there is no GitHub project for this.
Ok, it took time for me to understand, but I think you mixed 2 things: encryption and encoding.
The data value in PubSub isn't provided encrypted but encoded in base64. There isn't encryption required here. Base64 encoding is a raw encoding to prevent data loss, encoding type, special characters, binary data and boring compatibility things.
Note: On your local computer with pubsub emulator, the data aren't encrypted at rest and in transit. On Google Cloud, with PubSub service, the data are encrypted in transit and at rest
With curl you can use this command (with linux OS)
curl --location --request POST 'http://localhost:8091/v1/projects/my-project/topics/transactions:publish' \
--header 'Content-Type: application/json' \
--data-raw "{\"messages\":[{\"data\":\"$(echo "{\"foo\":\"baz\"}" | base64 -)\",\"attributes\":{}}]}"
Yes, backslash are boring...
I don't know how to do with postman
Related
Normally, for testing lambda locally, I use
sam local invoke WebhookFunction -e test.json
in test.json
{"body":"test"}
This valiable is passed to event.
def lambda_handler(event, context):
Now I want to do equivalent thing by curl
I tried this.
curl -X POST -H "Content-Type: application/json" -d '{"body":"test"}'
however I think {"body":"test"} is not correctly passed to event.
I guess I need to set something more.
Can anyone help me?
This won't work, unless you have a lambda RIE (Runtime Interface Emulator) running as a proxy for the Lambda Runtime API locally.
Depending on the language you've written your lambda in, you need to build a docker image and run it locally.
Finally, you can do this:
$ curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
This command invokes the Lambda function running in the container image and returns a response.
You can use one of the AWS base images for Lambda to build the container image for your function code.
Choose your lambda language and follow the instructions here.
Finally, test your lambda container locally with RIE.
There's a really nice blog post that walks you through the entire process here.
Foreword
I don't know that much about https and SSL, so I may use the wrong wordage for certain things but bear with me I've done as much research as I have the energy for
Goal:
I am trying to use the geolocate feature of the google maps API on my esp32 wroom-1, but this API requires an https connection, I have seen in other online examples using this wifi library with this http library and passing their SSL certificate as a plaintext string as the second parameter to the http.begin() function. The example that I seen used a plaintext SSL certificate in this format:
const char* root_ca= \
"-----BEGIN CERTIFICATE-----\n" \
"MIICiTCCAg+gAwIBAgIQH0evqmIAcFBUTAGem2OZKjAKBggqhkjOPQQDAzCBhTEL\n" \
"MAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UE\n" \
"BxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxKzApBgNVBAMT\n" \
"IkNPTU9ETyBFQ0MgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMDgwMzA2MDAw\n" \
"MDAwWhcNMzgwMTE4MjM1OTU5WjCBhTELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdy\n" \
"ZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09N\n" \
"T0RPIENBIExpbWl0ZWQxKzApBgNVBAMTIkNPTU9ETyBFQ0MgQ2VydGlmaWNhdGlv\n" \
"biBBdXRob3JpdHkwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAAQDR3svdcmCFYX7deSR\n" \
"FtSrYpn1PlILBs5BAH+X4QokPB0BBO490o0JlwzgdeT6+3eKKvUDYEs2ixYjFq0J\n" \
"cfRK9ChQtP6IHG4/bC8vCVlbpVsLM5niwz2J+Wos77LTBumjQjBAMB0GA1UdDgQW\n" \
"BBR1cacZSBm8nZ3qQUfflMRId5nTeTAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/\n" \
"BAUwAwEB/zAKBggqhkjOPQQDAwNoADBlAjEA7wNbeqy3eApyt4jf/7VGFAkK+qDm\n" \
"fQjGGoe9GKhzvSbKYAydzpmfz1wPMOG+FDHqAjAU9JM8SaczepBGR7NjfRObTrdv\n" \
"GDeAU/7dIOA1mjbRxwG55tzd8/8dLDoWV9mSOdY=\n" \
"-----END CERTIFICATE-----\n";
And then passed it to the constructor as such: http.begin(url, root_ca);
The questions:
I cannot find any tutorials on how to export an SSL certificate in this plaintext fashion on windows 10 anywhere online, I have been able to export a certificate that I made on windows into private files but they are not the plaintext as seen in the desired result. Is this easily doable with windows 10? If so are there any tutorials or instructions readily available?
Every place where I could create an SSL certificate asked for a domain, I know this is a realm I could research on my own but could someone who knows more about it explain it to me like I'm 5 with my use case? (i.e. I am using an esp32 arduino, the API I want to contact is google maps, so where should the domain be in this case?)
Thanks in advance,
Ozzie
I am performing a batch of asynchronous long_running_recognize transcriptions using Google's Cloud Speech-to-Text, and it appears that some of my requests are timing out, and/or not returning anything. How may I access the stored results of my API calls? I'm using Python 3.7.
I realize that the API call returns results to the function that made the call. What I'm asking is, does Google store the results of my API calls somewhere? And how do I access them?
You should probably call the asynchronous method when submitting larger audio files. Specifically this calls the LongRunningRecognize method. This should submit a Long Running Operation and should return an immediate response, for example:
{
"name": "operation_name",
"metadata": {
"#type": "type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeMetadata"
"progressPercent": 34,
"startTime": "2016-08-30T23:26:29.579144Z",
"lastUpdateTime": "2016-08-30T23:26:29.826903Z"
}
}
With this response you can poll for the result given the operation_name:
curl -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
"https://speech.googleapis.com/v1/operations/your-operation-name"
Note: when you are not receiving any return values using this method, I would suggest increasing the timeout and retry of the client. This can be done with something like:
long_running_recognize(retry=10, timeout=300)
Source
So I am considering HyperLedger Fabric to use with an application I have written in C++. From my understanding, the interactions i.e. posting retrieving data is all done in chaincode, in all of the examples I have seen this is invoked by using the CLI interface docker container.
I simply want to be able to store data produced by my application on a blockchain.
My question is how do I invoke the chaincode externally, surely this is something that is able to be done. I saw that there was a REST SDK but this is no longer supported so I don't want to go near it, to be honest. What other options are available??
Thanks!
There are two official SDKs you can try out.
Fabric Java SDK
Node JS SDK
As correctly mentioned by #Ajaya Mandal, you can use SDKs to automate the invoking process. For example, you can start the node app as written in app.js of balance transfer example and you can hit the API like it is shown in ./testAPI.sh file.
echo "POST invoke chaincode on peers of Org1 and Org2"
echo
VALUES=$(curl -s -X POST \
http://localhost:4000/channels/mychannel/chaincodes/mycc \
-H "authorization: Bearer $ORG1_TOKEN" \
-H "content-type: application/json" \
-d "{
\"peers\": [\"peer0.org1.example.com\",\"peer0.org2.example.com\"],
\"fcn\":\"move\",
\"args\":[\"a\",\"b\",\"10\"]
}")
Here you can add your arguments and pass it as you wish. You can use this thread to see how you can pass an HTTP request from C++.
I'm trying to send email attachments using the SES CLI, but every time the mail arrives and I open the attachment I get an error in Adobe:
could not open the file because it is either not a supported file type or because the file has been damaged.
The command I'm using is:
aws ses send-raw-email --raw-message file:///root/AWS/INSPECTOR/message.json
And the contents of that file is:
{
"Data": "From: sender#exmple.com\nTo: recipient#example.com\nSubject: Test email sent using the AWS CLI (contains an attachment)\nMIME-Version: 1.0\nContent-type: Multipart/Mixed; boundary=\"NextPart\"\n\n--NextPart\nContent-Type: text/plain\n\nThis is the message body.\n\n--NextPart\nContent-Type: application/pdf;\nContent-Disposition: attachment; filename=\"report.pdf\";\npath=\"\/tmp\/report.pdf\"\n\n--NextPart--"
}
I've seen the page at http://docs.aws.amazon.com/cli/latest/reference/ses/send-raw-email.html but I can't quite get the syntax correct, so any help would be appreciated....
The attachment should be passed in Base64 encoding with specifying Content-Transfer-Encoding: base64 in the MIME.
Here is the link of previous thread where I answered:
Sending aws cli SES as a file attachmennt
I was able to write some code for a college to solve the same issue for plain/text. I did try this with a PDF type but unfortunately I wasn't able to get that working correctly, the received file seemed to be corrupt. I think for other file types you have to encode it in base64 but not sure on the exact structure to be used with the cli.
echo '{"Data": "From: from#domain.com\nTo: to#domain.com\nSubject:
[Subject]\nMIME-Version: 1.0\nContent-type: Multipart/Mixed;
boundary=\"NextPart\"\n\n--NextPart\nContent-Type:
text/plain\n\n[Body]\n\n--NextPart\nContent-Type:
text/plain;\nContent-Disposition: attachment;
filename=\"test.txt\"\n\n'$(cat ./input.txt)'\n--NextPart--"}' >
message.json & aws ses send-raw-email --region eu-west-1 --raw-message
file://./message.json
Essentially the cat command in the middle writes the text into the message.json so that it can be dynamic. Hope this helps someone.
EDIT
Thanks to #James Dean:
The following is an example with a PDF attachment:
echo '{"Data": "From: from#domain.com\nTo: to#domain.com\nSubject:
[Subject]\nMIME-Version: 1.0\nContent-type: Multipart/Mixed;
boundary=\"NextPart\"\n\n--NextPart\nContent-Type:
text/plain\n\n[Body]\n\n--NextPart\nContent-Type:
application/pdf;\nContent-Disposition:
attachment;\nContent-Transfer-Encoding: base64;
filename=\"test.pdf\"\n\n'$(base64 test.pdf)'\n--NextPart--"}' > message.json & aws ses send-raw-email
--region eu-west-1 --raw-message file://./message.json
Cheers,
Alexei Blue.
The sample you tried to adapt adds plain text and embeds it to the email. You are trying to add a pdf, however you are only adding the header to the mail, but you aren't adding the pdfs content.
You need to embed the pdf base64 encoded as well.
Doing a quick search this answer to the slightly different question "How to embed images in email" might help you with the embedding. Instead of an image you want to embedded a pdf in this case.
If properly prepare your json and it should work with the aws-cli.
Using AWS CLI v2 to send a zip file:
echo '{"Data": "From: test#test.com\nTo: test#test.com\nSubject: Test email sent using the AWS CLI (contains an attachment)\nMIME-Version: 1.0\nContent-type: Multipart/Mixed; boundary="NextPart"\n\n--NextPart\nContent-Type: text/plain\n\nThis is the message body.\n\n--NextPart\nContent-Type: application/zip;\nContent-Disposition: attachment; filename="file.zip"\nContent-Transfer-Encoding: base64\n\n'$(base64 file.zip)'\n\n--NextPart--"}' > message2.json; /usr/local/bin/aws ses send-raw-email --cli-binary-format raw-in-base64-out --raw-message file://message2.json
This way you have encoded the file in base64, the HEADER specifies that, and then you send the rest of the data in raw format, and the CLI will encode that for you.