Base64Encode for AWS not matching Amazon Example - amazon-web-services

Question

I seem to be failing at the first hurdle in my quest for Amazon S3 Browser-Based Uploads.
Using Amazon's example shown here : http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html
Using Groovy/Java, I'm unable to encode the example post policy and produce the same encoding as Amazon.
{ "expiration": "2015-12-30T12:00:00.000Z",
"conditions": [
{"bucket": "sigv4examplebucket"},
["starts-with", "$key", "user/user1/"],
{"acl": "public-read"},
{"success_action_redirect": "http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html"},
["starts-with", "$Content-Type", "image/"],
{"x-amz-meta-uuid": "14365123651274"},
{"x-amz-server-side-encryption": "AES256"},
["starts-with", "$x-amz-meta-tag", ""],
{"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},
{"x-amz-algorithm": "AWS4-HMAC-SHA256"},
{"x-amz-date": "20151229T000000Z" }
]
}
I have this defined as a Groovy multi line string:
String policy_document = '''
{ "expiration": "2015-12-30T12:00:00.000Z",
"conditions": [
{"bucket": "sigv4examplebucket"},
["starts-with", "$key", "user/user1/"],
{"acl": "public-read"},
{"success_action_redirect": "http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html"},
["starts-with", "$Content-Type", "image/"],
{"x-amz-meta-uuid": "14365123651274"},
{"x-amz-server-side-encryption": "AES256"},
["starts-with", "$x-amz-meta-tag", ""],
{"x-amz-credential": "AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request"},
{"x-amz-algorithm": "AWS4-HMAC-SHA256"},
{"x-amz-date": "20151229T000000Z" }
]
}
'''
I'm then encoding and verifying with an assert which fails.
String base64Policy = (new BASE64Encoder()).encode(policy_document.replaceAll("\n", "").replaceAll("\r", "").getBytes("UTF-8"))
assert 'eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLA0KICAiY29uZGl0aW9ucyI6IFsNCiAgICB7ImJ1Y2tldCI6ICJzaWd2NGV4YW1wbGVidWNrZXQifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwNCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LA0KICAgIHsic3VjY2Vzc19hY3Rpb25fcmVkaXJlY3QiOiAiaHR0cDovL3NpZ3Y0ZXhhbXBsZWJ1Y2tldC5zMy5hbWF6b25hd3MuY29tL3N1Y2Nlc3NmdWxfdXBsb2FkLmh0bWwifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sDQogICAgeyJ4LWFtei1tZXRhLXV1aWQiOiAiMTQzNjUxMjM2NTEyNzQifSwNCiAgICB7IngtYW16LXNlcnZlci1zaWRlLWVuY3J5cHRpb24iOiAiQUVTMjU2In0sDQogICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sDQoNCiAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUlPU0ZPRE5ON0VYQU1QTEUvMjAxNTEyMjkvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LA0KICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwNCiAgICB7IngtYW16LWRhdGUiOiAiMjAxNTEyMjlUMDAwMDAwWiIgfQ0KICBdDQp9' == base64Policy
Just to help with readability, when I encode I get :
eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLCAgImNvbmRpdGlvbnMiOiBbICAgIHsiYnVja2V0IjogInNpZ3Y0ZXhhbXBsZWJ1Y2tldCJ9LCAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwgICAgeyJhY2wiOiAicHVibGljLXJlYWQifSwgICAgeyJzdWNjZXNzX2FjdGlvbl9yZWRpcmVjdCI6ICJodHRwOi8vc2lndjRleGFtcGxlYnVja2V0LnMzLmFtYXpvbmF3cy5jb20vc3VjY2Vzc2Z1bF91cGxvYWQuaHRtbCJ9LCAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sICAgIHsieC1hbXotbWV0YS11dWlkIjogIjE0MzY1MTIzNjUxMjc0In0sICAgIHsieC1hbXotc2VydmVyLXNpZGUtZW5jcnlwdGlvbiI6ICJBRVMyNTYifSwgICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sICAgIHsieC1hbXotY3JlZGVudGlhbCI6ICJBS0lBSU9TRk9ETk43RVhBTVBMRS8yMDE1MTIyOS91cy1lYXN0LTEvczMvYXdzNF9yZXF1ZXN0In0sICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwgICAgeyJ4LWFtei1kYXRlIjogIjIwMTUxMjI5VDAwMDAwMFoiIH0gIF19
But I should be getting :
eyAiZXhwaXJhdGlvbiI6ICIyMDE1LTEyLTMwVDEyOjAwOjAwLjAwMFoiLA0KICAiY29uZGl0aW9ucyI6IFsNCiAgICB7ImJ1Y2tldCI6ICJzaWd2NGV4YW1wbGVidWNrZXQifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRrZXkiLCAidXNlci91c2VyMS8iXSwNCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LA0KICAgIHsic3VjY2Vzc19hY3Rpb25fcmVkaXJlY3QiOiAiaHR0cDovL3NpZ3Y0ZXhhbXBsZWJ1Y2tldC5zMy5hbWF6b25hd3MuY29tL3N1Y2Nlc3NmdWxfdXBsb2FkLmh0bWwifSwNCiAgICBbInN0YXJ0cy13aXRoIiwgIiRDb250ZW50LVR5cGUiLCAiaW1hZ2UvIl0sDQogICAgeyJ4LWFtei1tZXRhLXV1aWQiOiAiMTQzNjUxMjM2NTEyNzQifSwNCiAgICB7IngtYW16LXNlcnZlci1zaWRlLWVuY3J5cHRpb24iOiAiQUVTMjU2In0sDQogICAgWyJzdGFydHMtd2l0aCIsICIkeC1hbXotbWV0YS10YWciLCAiIl0sDQoNCiAgICB7IngtYW16LWNyZWRlbnRpYWwiOiAiQUtJQUlPU0ZPRE5ON0VYQU1QTEUvMjAxNTEyMjkvdXMtZWFzdC0xL3MzL2F3czRfcmVxdWVzdCJ9LA0KICAgIHsieC1hbXotYWxnb3JpdGhtIjogIkFXUzQtSE1BQy1TSEEyNTYifSwNCiAgICB7IngtYW16LWRhdGUiOiAiMjAxNTEyMjlUMDAwMDAwWiIgfQ0KICBdDQp9
I'm hoping a fresh set of eyes will point me in the direction. Many thanks for any advice that can be offered.

Answer 1

The following worked for me in Java. Leave the carriage return and line feeds in place.
String policy_document = "{ \"expiration\": \"2015-12-30T12:00:00.000Z\",\n" +
" \"conditions\": [\n" +
" {\"bucket\": \"sigv4examplebucket\"},\n" +
" [\"starts-with\", \"$key\", \"user/user1/\"],\n" +
" {\"acl\": \"public-read\"},\n" +
" {\"success_action_redirect\": \"http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html\"},\n" +
" [\"starts-with\", \"$Content-Type\", \"image/\"],\n" +
" {\"x-amz-meta-uuid\": \"14365123651274\"},\n" +
" {\"x-amz-server-side-encryption\": \"AES256\"},\n" +
" [\"starts-with\", \"$x-amz-meta-tag\", \"\"],\n" +
"\n" +
" {\"x-amz-credential\": \"AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request\"},\n" +
" {\"x-amz-algorithm\": \"AWS4-HMAC-SHA256\"},\n" +
" {\"x-amz-date\": \"20151229T000000Z\" }\n" +
" ]\n" +
"}";
String base64Policy = (new BASE64Encoder()).encode(policy_document.getBytes());

Answer 2

Thanks for pointing me in the right direction.
The final solution was to add \r\n in that order at the end of each line.
String policy_document = "{ \"expiration\": \"2015-12-30T12:00:00.000Z\",\r\n" +
" \"conditions\": [\r\n" +
" {\"bucket\": \"sigv4examplebucket\"},\r\n" +
" [\"starts-with\", \"$key\", \"user/user1/\"],\r\n" +
" {\"acl\": \"public-read\"},\r\n" +
" {\"success_action_redirect\": \"http://sigv4examplebucket.s3.amazonaws.com/successful_upload.html\"},\r\n" +
" [\"starts-with\", \"$Content-Type\", \"image/\"],\r\n" +
" {\"x-amz-meta-uuid\": \"14365123651274\"},\r\n" +
" {\"x-amz-server-side-encryption\": \"AES256\"},\r\n" +
" [\"starts-with\", \"$x-amz-meta-tag\", \"\"],\r\n" +
"\r\n" +
" {\"x-amz-credential\": \"AKIAIOSFODNN7EXAMPLE/20151229/us-east-1/s3/aws4_request\"},\r\n" +
" {\"x-amz-algorithm\": \"AWS4-HMAC-SHA256\"},\r\n" +
" {\"x-amz-date\": \"20151229T000000Z\" }\r\n" +
" ]\r\n" +
"}";
It's unfortunate that it needs to be implemented so rigidly and that using a Groovy multi line string cannot accomplish this to keep the markup cleaner.