Related
I have an issue in the past 3 days with 3% of the requests to our lambdas.
They fail due to connection timeout to other aws services. see stack trace in the same lambda init
2021-10-30T16:37:33.310Z 7954e15a-8ae7-491e-880b-f5b532bde961 INFO TypeError: Unable to generate certificate due to
RequestError: Error: connect ETIMEDOUT 52.4.211.23:443
at /var/task/node_modules/cognito-express/lib/strategy.js:42:23
at bound (domain.js:416:15)
at runBound (domain.js:427:12)
at tryCatcher (/var/task/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/var/task/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/var/task/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/var/task/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/var/task/node_modules/bluebird/js/release/promise.js:725:18)
at _drainQueueStep (/var/task/node_modules/bluebird/js/release/async.js:93:12)
at _drainQueue (/var/task/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/var/task/node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (/var/task/node_modules/bluebird/js/release/async.js:15:14)
at processImmediate (internal/timers.js:464:21)
at process.topLevelDomainCallback (domain.js:147:15)
at process.callbackTrampoline (internal/async_hooks.js:129:24
2021-10-30T16:44:18.380Z 25392661-b635-4b73-9aed-67e655f13364 ERROR Unhandled Promise Rejection
{
"errorType": "Runtime.UnhandledPromiseRejection",
"errorMessage": "SequelizeConnectionError: connect ETIMEDOUT",
"reason": {
"errorType": "SequelizeConnectionError",
"errorMessage": "connect ETIMEDOUT",
"name": "SequelizeConnectionError",
"parent": {
"errorType": "Error",
"errorMessage": "connect ETIMEDOUT",
"code": "ETIMEDOUT",
"errorno": "ETIMEDOUT",
"syscall": "connect",
"fatal": true,
"stack": [
"Error: connect ETIMEDOUT",
" at Connection._handleTimeoutError (/var/task/node_modules/mysql2/lib/connection.js:189:17)",
" at listOnTimeout (internal/timers.js:557:17)",
" at processTimers (internal/timers.js:500:7)"
]
},
"original": {
"errorType": "Error",
"errorMessage": "connect ETIMEDOUT",
"code": "ETIMEDOUT",
"errorno": "ETIMEDOUT",
"syscall": "connect",
"fatal": true,
"stack": [
"Error: connect ETIMEDOUT",
" at Connection._handleTimeoutError (/var/task/node_modules/mysql2/lib/connection.js:189:17)",
" at listOnTimeout (internal/timers.js:557:17)",
" at processTimers (internal/timers.js:500:7)"
]
},
"stack": [
"SequelizeConnectionError: connect ETIMEDOUT",
" at ConnectionManager.connect (/var/task/node_modules/sequelize/lib/dialects/mysql/connection-manager.js:126:17)",
" at processTicksAndRejections (internal/process/task_queues.js:95:5)",
" at async ConnectionManager._connect (/var/task/node_modules/sequelize/lib/dialects/abstract/connection-manager.js:318:24)",
" at async /var/task/node_modules/sequelize/lib/dialects/abstract/connection-manager.js:250:32",
" at async ConnectionManager.getConnection (/var/task/node_modules/sequelize/lib/dialects/abstract/connection-manager.js:280:7)",
" at async /var/task/node_modules/sequelize/lib/sequelize.js:613:26",
" at async MySQLQueryInterface.select (/var/task/node_modules/sequelize/lib/dialects/abstract/query-interface.js:953:12)",
" at async Function.findAll (/var/task/node_modules/sequelize/lib/model.js:1753:21)",
" at async /var/task/src/routes/root/index_routes.js:20:18"
]
},
"promise": {},
"stack": [
"Runtime.UnhandledPromiseRejection: SequelizeConnectionError: connect ETIMEDOUT",
" at process.<anonymous> (/var/runtime/index.js:35:15)",
" at process.emit (events.js:412:35)",
" at process.emit (domain.js:470:12)",
" at processPromiseRejections (internal/process/promises.js:245:33)",
" at processTicksAndRejections (internal/process/task_queues.js:96:32)"
]
}
here is the mysql init code
if (global.sequelize != null) {
console.count('\x1b[32mRESERCH: connection exported from globals instead of creation\x1b[0m');
module.exports = global.sequelize;
} else {
console.count('\x1b[31mRESERCH: new connection created\x1b[0m');
global.sequelize = new Sequelize(
s.sqlDbName,
s.sqlUsername,
s.sqlPassword, {
host: s.sqlDbHost,
dialect: 'mysql',
// to print out the query + it's time
// check if causes performance issues
benchmark: true,
pool: {
max: 5,
min: 0,
idle: 10000
}
});
It's only some of the requests but it's causing a lot of errors for our users.
couldn't detect the root cause.
seems to be solved by changing the lambda's VPC slightly
it had 2 subnets, public and private.
removed the public
Not sure why it worked. maybe it forces the lambda to connect to db and cognito from the internal ip
Normally my lambda runs fine, today I am starting to receive this error:
ERROR Invoke Error
{
"errorType": "TimeoutError",
"errorMessage": "Connection timed out after 120000ms",
"code": "TimeoutError",
"message": "Connection timed out after 120000ms",
"time": "2020-11-27T10:34:39.272Z",
"region": "eu-west-1",
"hostname": "lambda.eu-west-1.amazonaws.com",
"retryable": true,
"stack": [
"TimeoutError: Connection timed out after 120000ms",
" at ClientRequest.<anonymous> (/var/task/node_modules/aws-sdk/lib/http/node.js:83:34)",
" at Object.onceWrapper (events.js:286:20)",
" at ClientRequest.emit (events.js:198:13)",
" at ClientRequest.EventEmitter.emit (domain.js:448:20)",
" at TLSSocket.emitRequestTimeout (_http_client.js:673:40)",
" at Object.onceWrapper (events.js:286:20)",
" at TLSSocket.emit (events.js:198:13)",
" at TLSSocket.EventEmitter.emit (domain.js:448:20)",
" at TLSSocket.Socket._onTimeout (net.js:443:8)",
" at ontimeout (timers.js:436:11)"
]
}
I don't have any clue what might be causing this, I know that this lambda invokes several lambdas and awaits on all of them to conclude, I checked the logs on cloudwatch for each one and all of them had completed with success.
Here is a snippet of the code I use to trigger all those child lambdas and wait for them:
function invokeAll(){
const baseLambdaPayload = {
arg1: args.myarg1,
arg2: args.myarg2,
arg3: args.myarg3,
arg4: args.myarg4
};
lambdaInvocations = [
...lambdaInvocations,
triggerLamdba(
'lambda1', {
...baseLambdaPayload,
morePayload: await getMorePayload()
}),
triggerLamdba(
'lambda2', baseLambdaPayload),
triggerLamdba(
'lambda3', baseLambdaPayload),
triggerLamdba(
'lambda4', baseLambdaPayload)
];
return await Promise.all(lambdaInvocations);
}
const triggerLamdba = (name, payload) => {
const params = {
FunctionName: name,
InvocationType: 'RequestResponse',
Payload: JSON.stringify(payload)
};
return lambda.invoke(params).promise();
}
Any tips to where I should look into? Thanks.
I have a database table with approx 4,000 records (currently). A response to an API call (POST, JSON) gives me data of this table for a maximum of 1,000 records per API call. A parameter ‘PageNo’ defines which of the 4,000 records are selected (e.g. PageNo = 1 gives me record 1-1000). The header data of the response includes a ‘PageCount’, in my example 4. I am able to retrieve that ‘PageCount’ and the test below loops through the PageNo (result in Postman Console = 1 2 3 4).
How I can call the same request repeatedly in a loop and use the values of the PageNo (i) as a parameter for that request like so:
{{baseUrl}}/v1/Units/Search?PageNo={{i}}
In my example I would expect the request to run 4 times with PageNo2 = 1, 2, 3, 4.
I am aware that I can use a CSV file and loop through the request in Collection Runner but PageCount changes (i.e. the number of records in the table change) and I need to run this loop frequently so creating a new CSV file for each loop is not really an option.
Postman Test:
pm.environment.set('Headers2', JSON.stringify(pm.response.headers));
var Headers2 = JSON.stringify(pm.response.headers);
pm.environment.set('PageCount2', JSON.parse(Headers2)[10].value);
var i;
for (i = 1; [i] <= pm.environment.get('PageCount2'); i++) {
console.log(i);
postman.setNextRequest('custom fields | json Copy');
}
Postman Request:
{
"Location":"{{TestingLocation}}",
"Fields":[
"StockNo",
"BrandDesc"
],
"Filters": {
"StatusCode":"{{TestingUnitSearchStatusCode}}"
},
"PageSize":1000,
"PageNo" : "{{i}}"
}
With postman.setNextRequest() you can set the calling request as the same request. But you need an exit strategy, otherwise that request would be called infinite times. This only works with the collection runner.
On your first(!) request, store the amount of pages and create a counter.
Increment the counter. If it is smaller than the amount of pages, set the current request as the next request.
Else, do not set a next request. The collection runner will continue with its normal flow. Optionally remove the pages and counter variables.
You can let the request detect that it is its first iteration by not having initialized the amount of pages and counter variables.
Performed a POC for your use-case using postman-echo API.
Pre-req script --> Takes care of the initial request to the endpoint to retrieve the PageSize and set it as an env var. Also initializes the iterationCount to 1 (as env var)
Test Script --> Checks for the current iteration number and performs tests.
Here's a working postman collection.
If you're familiar with newman, you can just run:
newman run <collection-name>
You can also import this collection in Postman app and use collection runner.
{
"info": {
"_postman_id": "1d7669a6-a1a1-4d01-a162-f8675e01d1c7",
"name": "Loop Req Collection",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
},
"item": [
{
"name": "Run_Req",
"event": [
{
"listen": "test",
"script": {
"id": "dac5e805-548b-4bdc-a26e-f56500414208",
"exec": [
"var iterationCount = pm.environment.get(\"iterationCount\");",
"if (pm.environment.get(\"iterationCount\") <= pm.environment.get(\"pageSize\")) {",
" console.log(\"Iteration Number: \" + iterationCount);",
"",
" pm.test(\"Status code is 200\", function() {",
" pm.response.to.have.status(200);",
" });",
"",
" pm.test(\"Check value of pageNo in body is equal to iterationCount\", function() {",
" var jsonData = pm.response.json();",
" return jsonData.json.PageNo == iterationCount;",
"",
" });",
" iterationCount++;",
" pm.environment.set(\"iterationCount\", iterationCount);",
" postman.setNextRequest('Run_Req');",
"}",
"if (pm.environment.get(\"iterationCount\") > pm.environment.get(\"pageSize\")) {",
" console.log(\"Cleanup\");",
" pm.environment.unset(\"pageSize\");",
" pm.environment.set(\"iterationCount\", 1);",
" postman.setNextRequest('');",
" pm.test(\"Cleanup Success\", function() {",
" if (pm.environment.get(\"pageSize\") == null && pm.environment.get(\"iterationCount\") == null) {",
" return true;",
" }",
" });",
"}"
],
"type": "text/javascript"
}
},
{
"listen": "prerequest",
"script": {
"id": "3d43c6c7-4a9b-46cf-bd86-c1823af5a68e",
"exec": [
"if (pm.environment.get(\"pageSize\") == null) {",
" pm.sendRequest(\"https://postman-echo.com/response-headers?pageSize=4\", function(err, response) {",
" var pageSize = response.headers.get('pageSize');",
" var iterationCount = 1;",
" console.log(\"pre-req:pageSize= \" + pageSize);",
" console.log(\"pre-req:iterationCount= \" + iterationCount);",
" pm.environment.set(\"pageSize\", pageSize);",
" pm.environment.set(\"iterationCount\", iterationCount);",
" });",
"",
"}"
],
"type": "text/javascript"
}
}
],
"request": {
"method": "POST",
"header": [
{
"key": "Content-Type",
"name": "Content-Type",
"value": "application/json",
"type": "text"
}
],
"body": {
"mode": "raw",
"raw": "{\n \"Location\":\"{{TestingLocation}}\",\n \"Fields\":[\n \"StockNo\",\n \"BrandDesc\"\n ],\n \"Filters\": {\n \"StatusCode\":\"{{TestingUnitSearchStatusCode}}\"\n },\n \"PageSize\":1000,\n \"PageNo\" : \"{{iterationCount}}\"\n}"
},
"url": {
"raw": "https://postman-echo.com/post",
"protocol": "https",
"host": [
"postman-echo",
"com"
],
"path": [
"post"
]
}
},
"response": []
}
]
}
How can we insert the following data to aws dynamodb tale.
{
"userId":"4",
"gpId": "44",
"uname": "username",
"position":"Cloud Solution Engineer",
"companyName" : "xyz Technlologies",
"skills": [{"linux":"1","windows":"2","Docker": "3"}]
}
I have tried using the following code in node.js
var userId=event.userId;
var gpId=event.gpId;
var fbId=event.fbId;
var uname=event.uname;
var position=event.position;
var role=event.role;
var companyName=event.companyName;
var skills=event.skills;
dynamodb.putItem({
"TableName": tableName,
"Item" : {
"userId": {"N": userId},
"gpId": {"N": gpId},
"uname" : {"S":uname},
"position" : {"S":position},
"role" : {"S":role},
"companyName" : {"S":companyName},
"skills" : {"SS":skills}
}
I am trying the below code it is giving below error to me.
{
"errorMessage": "Expected params.Item['skills'].S to be a string",
"errorType": "InvalidParameterType",
"stackTrace": [
"ParamValidator.fail (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:50:37)",
"ParamValidator.validateType (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:218:10)",
"ParamValidator.validateString (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:150:14)",
"ParamValidator.validateScalar (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:130:21)",
"ParamValidator.validateMember (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:94:21)",
"ParamValidator.validateStructure (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:75:14)",
"ParamValidator.validateMember (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:88:21)",
"ParamValidator.validateMap (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:117:14)",
"ParamValidator.validateMember (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:92:21)",
"ParamValidator.validateStructure (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:75:14)"
]
}
I tried to change {"S": skills} with skills.
it is giving me following error:
{
"errorMessage": "Unexpected key '0' found in params.Item['skills']",
"errorType": "UnexpectedParameter",
"stackTrace": [
"ParamValidator.fail (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:50:37)",
"ParamValidator.validateStructure (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:77:14)",
"ParamValidator.validateMember (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:88:21)",
"ParamValidator.validateMap (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:117:14)",
"ParamValidator.validateMember (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:92:21)",
"ParamValidator.validateStructure (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:75:14)",
"ParamValidator.validateMember (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:88:21)",
"ParamValidator.validate (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:34:10)",
"Request.VALIDATE_PARAMETERS (/var/runtime/node_modules/aws-sdk/lib/event_listeners.js:109:42)",
"Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)"
]
}
I have tried applying map as suggested in an answer but it gives me the following error:
{
"errorMessage": "There were 3 validation errors:\n* UnexpectedParameter: Unexpected key 'linux' found in params.Item['skills'].M['0']\n* UnexpectedParameter: Unexpected key 'windows' found in params.Item['skills'].M['0']\n* UnexpectedParameter: Unexpected key 'Docker' found in params.Item['skills'].M['0']",
"errorType": "MultipleValidationErrors",
"stackTrace": [
"* UnexpectedParameter: Unexpected key 'linux' found in params.Item['skills'].M['0']",
"* UnexpectedParameter: Unexpected key 'windows' found in params.Item['skills'].M['0']",
"* UnexpectedParameter: Unexpected key 'Docker' found in params.Item['skills'].M['0']",
"ParamValidator.validate (/var/runtime/node_modules/aws-sdk/lib/param_validator.js:40:28)",
"Request.VALIDATE_PARAMETERS (/var/runtime/node_modules/aws-sdk/lib/event_listeners.js:109:42)",
"Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)",
"callNextListener (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:95:12)",
"/var/runtime/node_modules/aws-sdk/lib/event_listeners.js:75:9",
"finish (/var/runtime/node_modules/aws-sdk/lib/config.js:308:7)",
"/var/runtime/node_modules/aws-sdk/lib/config.js:324:9",
"EnvironmentCredentials.get (/var/runtime/node_modules/aws-sdk/lib/credentials.js:126:7)",
"getAsyncCredentials (/var/runtime/node_modules/aws-sdk/lib/config.js:318:24)",
"Config.getCredentials (/var/runtime/node_modules/aws-sdk/lib/config.js:338:9)"
]
}
skills is not a string set (SS), it should have a Map attribute type (M) after the following 0-index assignment:
var skills = event.skills[0];
http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_AttributeValue.html
I seem to be failing at the first hurdle in my quest for Amazon S3 Browser-Based Uploads.
Using Amazon's example shown here : http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html
Using Groovy/Java, I'm unable to encode the example post policy and produce the same encoding as Amazon.
{ "expiration": "2015-12-30T12:00:00.000Z",
"conditions": [
{"bucket": "sigv4examplebucket"},
["starts-with", "$key", "user/user1/"],
{"acl": "public-read"},
{"success_action_redirect": "http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html"},
["starts-with", "$Content-Type", "image/"],
{"x-amz-meta-uuid": "14365123651274"},
{"x-amz-server-side-encryption": "AES256"},
["starts-with", "$x-amz-meta-tag", ""],
{"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},
{"x-amz-algorithm": "AWS4-HMAC-SHA256"},
{"x-amz-date": "20151229T000000Z" }
]
}
I have this defined as a Groovy multi line string:
String policy_document = '''
{ "expiration": "2015-12-30T12:00:00.000Z",
"conditions": [
{"bucket": "sigv4examplebucket"},
["starts-with", "$key", "user/user1/"],
{"acl": "public-read"},
{"success_action_redirect": "http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html"},
["starts-with", "$Content-Type", "image/"],
{"x-amz-meta-uuid": "14365123651274"},
{"x-amz-server-side-encryption": "AES256"},
["starts-with", "$x-amz-meta-tag", ""],
{"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},
{"x-amz-algorithm": "AWS4-HMAC-SHA256"},
{"x-amz-date": "20151229T000000Z" }
]
}
'''
I'm then encoding and verifying with an assert which fails.
String base64Policy = (new BASE64Encoder()).encode(policy_document.replaceAll("\n", "").replaceAll("\r", "").getBytes("UTF-8"))
assert 'eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLA0KICAiY29uZGl0aW9ucyI6IFsNCiAgICB7ImJ1Y2tldCI6ICJzaWd2NGV4YW1wbGVidWNrZXQifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwNCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LA0KICAgIHsic3VjY2Vzc19hY3Rpb25fcmVkaXJlY3QiOiAiaHR0cDovL3NpZ3Y0ZXhhbXBsZWJ1Y2tldC5zMy5hbWF6b25hd3MuY29tL3N1Y2Nlc3NmdWxfdXBsb2FkLmh0bWwifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sDQogICAgeyJ4LWFtei1tZXRhLXV1aWQiOiAiMTQzNjUxMjM2NTEyNzQifSwNCiAgICB7IngtYW16LXNlcnZlci1zaWRlLWVuY3J5cHRpb24iOiAiQUVTMjU2In0sDQogICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sDQoNCiAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUlPU0ZPRE5ON0VYQU1QTEUvMjAxNTEyMjkvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LA0KICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwNCiAgICB7IngtYW16LWRhdGUiOiAiMjAxNTEyMjlUMDAwMDAwWiIgfQ0KICBdDQp9' == base64Policy
Just to help with readability, when I encode I get :
eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLCAgImNvbmRpdGlvbnMiOiBbICAgIHsiYnVja2V0IjogInNpZ3Y0ZXhhbXBsZWJ1Y2tldCJ9LCAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwgICAgeyJhY2wiOiAicHVibGljLXJlYWQifSwgICAgeyJzdWNjZXNzX2FjdGlvbl9yZWRpcmVjdCI6ICJodHRwOi8vc2lndjRleGFtcGxlYnVja2V0LnMzLmFtYXpvbmF3cy5jb20vc3VjY2Vzc2Z1bF91cGxvYWQuaHRtbCJ9LCAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sICAgIHsieC1hbXotbWV0YS11dWlkIjogIjE0MzY1MTIzNjUxMjc0In0sICAgIHsieC1hbXotc2VydmVyLXNpZGUtZW5jcnlwdGlvbiI6ICJBRVMyNTYifSwgICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sICAgIHsieC1hbXotY3JlZGVudGlhbCI6ICJBS0lBSU9TRk9ETk43RVhBTVBMRS8yMDE1MTIyOS91cy1lYXN0LTEvczMvYXdzNF9yZXF1ZXN0In0sICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwgICAgeyJ4LWFtei1kYXRlIjogIjIwMTUxMjI5VDAwMDAwMFoiIH0gIF19
But I should be getting :
eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLA0KICAiY29uZGl0aW9ucyI6IFsNCiAgICB7ImJ1Y2tldCI6ICJzaWd2NGV4YW1wbGVidWNrZXQifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwNCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LA0KICAgIHsic3VjY2Vzc19hY3Rpb25fcmVkaXJlY3QiOiAiaHR0cDovL3NpZ3Y0ZXhhbXBsZWJ1Y2tldC5zMy5hbWF6b25hd3MuY29tL3N1Y2Nlc3NmdWxfdXBsb2FkLmh0bWwifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sDQogICAgeyJ4LWFtei1tZXRhLXV1aWQiOiAiMTQzNjUxMjM2NTEyNzQifSwNCiAgICB7IngtYW16LXNlcnZlci1zaWRlLWVuY3J5cHRpb24iOiAiQUVTMjU2In0sDQogICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sDQoNCiAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUlPU0ZPRE5ON0VYQU1QTEUvMjAxNTEyMjkvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LA0KICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwNCiAgICB7IngtYW16LWRhdGUiOiAiMjAxNTEyMjlUMDAwMDAwWiIgfQ0KICBdDQp9
I'm hoping a fresh set of eyes will point me in the direction. Many thanks for any advice that can be offered.
The following worked for me in Java. Leave the carriage return and line feeds in place.
String policy_document = "{ \"expiration\": \"2015-12-30T12:00:00.000Z\",\n" +
" \"conditions\": [\n" +
" {\"bucket\": \"sigv4examplebucket\"},\n" +
" [\"starts-with\", \"$key\", \"user/user1/\"],\n" +
" {\"acl\": \"public-read\"},\n" +
" {\"success_action_redirect\": \"http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html\"},\n" +
" [\"starts-with\", \"$Content-Type\", \"image/\"],\n" +
" {\"x-amz-meta-uuid\": \"14365123651274\"},\n" +
" {\"x-amz-server-side-encryption\": \"AES256\"},\n" +
" [\"starts-with\", \"$x-amz-meta-tag\", \"\"],\n" +
"\n" +
" {\"x-amz-credential\": \"AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request\"},\n" +
" {\"x-amz-algorithm\": \"AWS4-HMAC-SHA256\"},\n" +
" {\"x-amz-date\": \"20151229T000000Z\" }\n" +
" ]\n" +
"}";
String base64Policy = (new BASE64Encoder()).encode(policy_document.getBytes());
Thanks for pointing me in the right direction.
The final solution was to add \r\n in that order at the end of each line.
String policy_document = "{ \"expiration\": \"2015-12-30T12:00:00.000Z\",\r\n" +
" \"conditions\": [\r\n" +
" {\"bucket\": \"sigv4examplebucket\"},\r\n" +
" [\"starts-with\", \"$key\", \"user/user1/\"],\r\n" +
" {\"acl\": \"public-read\"},\r\n" +
" {\"success_action_redirect\": \"http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html\"},\r\n" +
" [\"starts-with\", \"$Content-Type\", \"image/\"],\r\n" +
" {\"x-amz-meta-uuid\": \"14365123651274\"},\r\n" +
" {\"x-amz-server-side-encryption\": \"AES256\"},\r\n" +
" [\"starts-with\", \"$x-amz-meta-tag\", \"\"],\r\n" +
"\r\n" +
" {\"x-amz-credential\": \"AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request\"},\r\n" +
" {\"x-amz-algorithm\": \"AWS4-HMAC-SHA256\"},\r\n" +
" {\"x-amz-date\": \"20151229T000000Z\" }\r\n" +
" ]\r\n" +
"}";
It's unfortunate that it needs to be implemented so rigidly and that using a Groovy multi line string cannot accomplish this to keep the markup cleaner.