String distributionDomain = "d21geuebylb7j1.cloudfront.net";
String privateKeyFilePath = "/Users/Desktop/rsa-private-key.der";
String s3ObjectKey = "small.mp4";
String policyResourcePath = "http://" + distributionDomain + "/" + s3ObjectKey;
System.out.println(privateKeyFilePath);
byte[] derPrivateKey = null;
I am trying to make signed URL for my cloudfront distribution but I am getting invalid key error. I am getting issue with my rsa-private-key.der file. I have made this file from pem file as mentioned in Cloudfront documentation.
Below is my error logs:
Exception in thread "main" org.jets3t.service.CloudFrontServiceException: java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException: invalid key format
at org.jets3t.service.CloudFrontService.signUrlCanned(CloudFrontService.java:2148)
at test.SignedURL.main(SignedURL.java:74)
Caused by: java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException: invalid key format
at java.base/sun.security.rsa.RSAKeyFactory.engineGeneratePrivate(RSAKeyFactory.java:216)
at java.base/java.security.KeyFactory.generatePrivate(KeyFactory.java:390)
at org.jets3t.service.security.EncryptionUtil.signWithRsaSha1(EncryptionUtil.java:526)
at org.jets3t.service.CloudFrontService.signUrlCanned(CloudFrontService.java:2133)
... 1 more
Caused by: java.security.InvalidKeyException: invalid key format
at java.base/sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:330)
at java.base/sun.security.pkcs.PKCS8Key.decode(PKCS8Key.java:356)
at java.base/sun.security.rsa.RSAPrivateCrtKeyImpl.<init>(RSAPrivateCrtKeyImpl.java:91)
at java.base/sun.security.rsa.RSAPrivateCrtKeyImpl.newKey(RSAPrivateCrtKeyImpl.java:75)
at java.base/sun.security.rsa.RSAKeyFactory.generatePrivate(RSAKeyFactory.java:315)
at java.base/sun.security.rsa.RSAKeyFactory.engineGeneratePrivate(RSAKeyFactory.java:212)
... 4 more
I had same issue this solved my issue.
You can try this:
public enum CloudFrontUrlSigner
extends Enum<CloudFrontUrlSigner>
Utility class for generating pre-signed URLs for serving private CloudFront content. All dates must be in UTC. Use Calendar to set the timezone specifically before converting to a Date object, or else use DateUtils to turn a UTC date String into a Date object.
Protocol protocol = Protocol.http;
String distributionDomain = "d1b2c3a4g5h6.cloudfront.net";
File privateKeyFile = new File("/path/to/cfcurlCloud/rsa-private-key.pem");
String s3ObjectKey = "a/b/images.jpeg";
String keyPairId = "APKAJCEOKRHC3XIVU5NA";
Date dateLessThan = DateUtils.parseISO8601Date("2012-11-14T22:20:00.000Z");
Date dateGreaterThan = DateUtils.parseISO8601Date("2011-11-14T22:20:00.000Z");
String ipRange = "192.168.0.1/24";
String url1 = CloudFrontUrlSigner.getSignedURLWithCannedPolicy(
protocol, distributionDomain, privateKeyFile,
s3ObjectKey, keyPairId, dateLessThan);
String url2 = CloudFrontUrlSigner.getSignedURLWithCustomPolicy(
protocol, distributionDomain, privateKeyFile,
s3ObjectKey, keyPairId, dateLessThan,
dateGreaterThan, ipRange);
here is the link of AWS Documentation: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/cloudfront/CloudFrontUrlSigner.html
Related
I can successfully encrypt the value using the following code:
final static Charset ENCODING = StandardCharsets.ISO_8859_1;
var awsCreds = AwsBasicCredentials.create(KEY, SECRET_KEY);
kmsClient = KmsClient.builder().credentialsProvider(StaticCredentialsProvider.create(awsCreds)).region(Region.US_EAST_1).build();
var sdkBytesString = SdkBytes.fromString(stringToEncrypt, ENCODING);
var encryptRequest = EncryptRequest.builder().keyId(KEY_ARN).plaintext(sdkBytesString).build();
var encryptResponse = this.kmsClient.encrypt(encryptRequest);
var result = encryptResponse.ciphertextBlob().asString(ENCODING);
In the result I can see encrypted value.
BUT The problem is that I need this value in UTF8 not ISO_8859_1. When trying to get ciphertextBlob in UTF8 - getting conversion error:
Blockquote
java.io.UncheckedIOException: Cannot encode string.
I need to save the string in UTF-8 DB and to send this encrypted string to another service that accepts UTF-8 strings\
Could you please advise how to get UTF-8 string after encryption?
Actually Base64 encrypting solves the problem:
https://github.com/amazon-archives/realworld-serverless-application/blob/master/backend/src/main/java/software/amazon/serverless/apprepo/api/impl/pagination/EncryptedTokenSerializer.java#L51
I need some help for uploading large files into s3 bucket from salesforce apex server side.
I need to be able to split a blob and upload it to aws s3 bucket using Http PUT operation. I am able to do that upto 12 MB file in a single upload because that is the PUT request body size limit in Apex .
So i need to be able to upload using multipart operation. I noticed s3 allows to upload in parts and gives back a uploadId. wondering if anyone has already done this before in salesforce apex code. it would be greatly appreciated.
Thanks in advance
Parbati Bose.
Here is the code
public with sharing class AWSS3Service {
private static Http http;
#auraEnabled
public static void uploadToAWSS3( String fileToUpload , String filenm , String doctype){
fileToUpload = EncodingUtil.urlDecode(fileToUpload, 'UTF-8');
filenm = EncodingUtil.urlEncode(filenm , 'UTF-8'); // encode the filename in case there are special characters in the name
String filename = 'Storage' + '/' + filenm ;
String attachmentBody = fileToUpload;
String formattedDateString = DateTime.now().formatGMT('EEE, dd MMM yyyy HH:mm:ss z');
// s3 bucket!
String key = '**********' ;
String secret = '********' ;
String bucketname = 'testbucket' ;
String region = 's3-us-west-2' ;
String host = region + '.' + 'amazonaws.com' ; //aws server base url
try{
HttpRequest req = new HttpRequest();
http = new Http() ;
req.setMethod('PUT');
req.setEndpoint('https://' + bucketname + '.' + host + '/' + filename );
req.setHeader('Host', bucketname + '.' + host);
req.setHeader('Content-Encoding', 'UTF-8');
req.setHeader('Content-Type' , doctype);
req.setHeader('Connection', 'keep-alive');
req.setHeader('Date', formattedDateString);
req.setHeader('ACL', 'public-read-write');
String stringToSign = 'PUT\n\n' +
doctype + '\n' +
formattedDateString + '\n' +
'/' + bucketname + '/' + filename;
Blob mac = Crypto.generateMac('HMACSHA1', blob.valueof(stringToSign),blob.valueof(secret));
String signed = EncodingUtil.base64Encode(mac);
String authHeader = 'AWS' + ' ' + key + ':' + signed;
req.setHeader('Authorization',authHeader);
req.setBodyAsBlob(EncodingUtil.base64Decode(fileToUpload)) ;
HttpResponse response = http.send(req);
Log.debug('response from aws s3 is ' + response.getStatusCode() + ' and ' + response.getBody());
}catch(Exception e){
Log.debug('error in connecting to s3 ' + e.getMessage());
throw e ;
}
}
I was looking at this same issue in the last couple of days, unfortunately you are going to be better performing this transfer from outside of Salesforce due to the APEX Heap size limit of 12MB.
https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_gov_limits.htm
Whilst it would be possible to write files using multipart, it seems you cannot get them out of the database for splitting them into chunks you can send. A similar question was asked on the stackexchange -
https://salesforce.stackexchange.com/questions/264015/how-to-retrieve-file-content-from-content-document-in-chunks-using-soql
The AWS SDK for Java exposes a high-level API, called TransferManager,
that simplifies multipart uploads (see Uploading Objects Using
Multipart Upload API). You can upload data from a file or a stream.
You can also set advanced options, such as the part size you want to
use for the multipart upload, or the number of concurrent threads you
want to use when uploading the parts. You can also set optional object
properties, the storage class, or the ACL. You use the
PutObjectRequest and the TransferManagerConfiguration classes to set
these advanced options.
Here is the sample code from https://docs.aws.amazon.com/AmazonS3/latest/dev/HLuploadFileJava.html.
You can adapt to your Salesforce Apex code:
import com.amazonaws.SdkClientException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.transfer.TransferManager;
import com.amazonaws.services.s3.transfer.TransferManagerBuilder;
import com.amazonaws.services.s3.transfer.Upload;
import java.io.File;
public class HighLevelMultipartUpload {
public static void main(String[] args) throws Exception {
Regions clientRegion = Regions.DEFAULT_REGION;
String bucketName = "*** Bucket name ***";
String keyName = "*** Object key ***";
String filePath = "*** Path for file to upload ***";
try {
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withRegion(clientRegion)
.withCredentials(new ProfileCredentialsProvider())
.build();
TransferManager tm = TransferManagerBuilder.standard()
.withS3Client(s3Client)
.build();
// TransferManager processes all transfers asynchronously,
// so this call returns immediately.
Upload upload = tm.upload(bucketName, keyName, new File(filePath));
System.out.println("Object upload started");
// Optionally, wait for the upload to finish before continuing.
upload.waitForCompletion();
System.out.println("Object upload complete");
} catch (AmazonServiceException e) {
// The call was transmitted successfully, but Amazon S3 couldn't process
// it, so it returned an error response.
e.printStackTrace();
} catch (SdkClientException e) {
// Amazon S3 couldn't be contacted for a response, or the client
// couldn't parse the response from Amazon S3.
e.printStackTrace();
}
}
}
Here is my code in the JSR223 PreProcessor.
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import java.security.InvalidKeyException
import java.security.MessageDigest
import groovy.json.JsonSlurper
import java.text.SimpleDateFormat
static byte[] HmacSHA256(String data, byte[] key) throws Exception {
String algorithm="HmacSHA256";
Mac mac = Mac.getInstance(algorithm);
mac.init(new SecretKeySpec(key, algorithm));
return mac.doFinal(data.getBytes("UTF8"));
}
static byte[] getSignatureKey(String key, String dateStamp, String regionName, String serviceName) throws Exception {
byte[] kSecret = ("AWS4" + key).getBytes("UTF8");
byte[] kDate = HmacSHA256(dateStamp, kSecret);
byte[] kRegion = HmacSHA256(regionName, kDate);
byte[] kService = HmacSHA256(serviceName, kRegion);
byte[] kSigning = HmacSHA256("aws4_request", kService);
return kSigning;
}
Example parameter values to the function getSignatureKey are (these I am passing to the Parameters section of the JSR223 PreProcessor as variables)
key = eC6hEyRSTXMzsG6+juOObz8LbXb36iEYW7PPN1MJ
dateStamp = 20190123T083434Z
regionName = us-west-2
serviceName = test-mlp-us-west-2-4023179c-7708-4c5e-a831-28259b8a8872.s3.us-west-2.amazonaws.com
This code is not working and not generating the AWS signature.
Here is the sample Signature value I need to get
Signature=2a6092ec4ff49dc9j3b92d436635a57f312753kcc9f553ce1718b9b1594c4362
1.What is wrong with this code?
2.How can I assign the AWS signature to a variable and use in the JMeter?
If "This code is not working" the first thing you should do is to look into jmeter.log file - in case of Groovy script failure it will contain information regarding what exactly failed and why and some hints to fix.
I also think you should generate the signature basing on request details (URL, parameters, headers, etc.)
StringToSign =
Algorithm + \n +
RequestDateTime + \n +
CredentialScope + \n +
HashedCanonicalRequest
See Create a String to Sign for Signature Version 4
The whole process with examples is described in the How to Handle Dynamic AWS SigV4 in JMeter for API Testing and the test plan you can use as the reference is available in the GitHub repo.
I am new to AWSLex. I was trying to fetch the SlotType information through ModelBuilding GetSlotType API.
The request URL: GET /slottypes/name/versions/version HTTP/1.1
To fetch the latest version as per document is Pattern: \$LATEST|[0-9]+
Through Postman I am able to execute with version number.But while fetching the latest version $LATEST, the URL I used is
https://models.lex.us-east-1.amazonaws.com/slottypes/serviceType/versions/%24LATEST
which worked perfectly through Postman.
The same url when I had used to create the signature didn't work. I get following error
"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."
The canonical request what I had generated is below
GET
/slottypes/serviceType/versions/%24LATEST
content-type:application/json
host:models.lex.us-east-1.amazonaws.com
x-amz-date:20180809T052553Z
content-type;host;x-amz-date
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Any help on how to fix the issue is highly appreciated.
String payloadHash = CommonUtil.hexEncode(sha256Hash(payload));
ZonedDateTime utcNow = Instant.now().atZone(ZoneOffset.UTC);// Date
for headers and the credential string
String amzDate =
utcNow.format(DateTimeFormatter.ofPattern("yyyyMMdd'T'HHmmss'Z'"));
String dateStamp =
utcNow.format(DateTimeFormatter.ofPattern("yyyyMMdd"));
String canonicalHeaders = String.format("content-type:%s\nhost:%s\nx-
amz-date:%s\n", contentType, host, amzDate);
String canonicalRequest = String.format("%s\n%s\n%s\n%s\n%s\n%s",
method, canonicalUri, canonicalQueryString, canonicalHeaders,
signedHeaders, payloadHash);
String credentialScope = String.format("%s/%s/%s/aws4_request",
dateStamp, region, service);
String canonicalRequestHash =
CommonUtil.hexEncode(sha256Hash(canonicalRequest));
String stringToSign = String.format("%s\n%s\n%s\n%s", algorithm,
amzDate, credentialScope, canonicalRequestHash);
byte[] signatureKey = CommonUtil.getSignatureKey(secretKey,
dateStamp, region, service);
String signature = CommonUtil.hexEncode(CommonUtil.HmacSHA256(URLEncoder.encode(stringToSign,"utf-8"), signatureKey));
String authorizationHeader = String.format("%s Credential=%s/%s, SignedHeaders=%s, Signature=%s", algorithm, accessKey, credentialScope, signedHeaders, signature);
MultivaluedMap<String, String> headersMap = new MultivaluedHashMap<>();
headersMap.add("Content-Type", contentType);
headersMap.add("X-Amz-Date", amzDate);
headersMap.add("Authorization", authorizationHeader);
This is how I am generating the signature.The same code works fine with version as number. The issue lies when I encode the url with $LATEST as value and pass it.
I am attempting to use a GET request to use the Amazon Mechanical Turk GetFileUploadURL function. However, I get this error code when attempting it.
AWS.NotAuthorized The identity contained in the request is not authorized to use this AWSAccessKeyId
This is the code I'm using to create the request.
now = DateTime.now
#For creating the signature hash
data = "AWSMechanicalTurkRequesterGetFileUploadURL" + now.to_s
sha256 = OpenSSL::Digest::SHA256.new
sig = OpenSSL::HMAC.digest(sha256, Rails.configuration.secret_key, data)
signature = Base64.encode64(sig)
puts "https://mechanicalturk.amazonaws.com", "/?Service=AWSMechanicalTurkRequester&AWSAccessKeyId=#{Rails.configuration.aws_key}&Version=2014-08-15&Operation=GetFileUploadURL&Signature=#{sig}&Timestamp=#{now}&AssignmentId=#{mturk_results[0][:AssignmentId]}&QuestionIdentifier=file1"
puts "\n\n"
uri = URI('https://mechanicalturk.amazonaws.com')
params = {:Service=>"AWSMechanicalTurkRequester", :AWSAccessKeyId=>Rails.configuration.aws_key, :Version=>"2014-08-15", :Operation=>"GetFileUPloadURL", :Signature=>sig, :Timestamp=>now, :AssignmentId=>mturk_results[0][:AssignmentId], :QuestionIdentifier=>"file1"}
uri.query = URI.encode_www_form(params)
res = 0
Net::HTTP.start(uri.host, uri.port,
:use_ssl => uri.scheme == 'https') do |http|
request = Net::HTTP::Get.new uri
res = http.request request # Net::HTTPResponse object
end
Any ideas as to what I'm doing wrong?
You'll need to format your TimeStamp correctly. The time stamp must be in UTC and in the following ISO 8601 format:
YYYYMMDD'T'HHMMSS'Z'. For example, 20150830T123600Z is a valid time stamp. Do not include milliseconds in the time stamp.
It needs to be of the form:
Timestamp=2016-04-23T08:00:05Z
While Ruby's DateTime.now method returns them of the form:
2016-08-05T10:43:27-07:00
You can read more about Timestamp and AWS signatures here:
http://docs.aws.amazon.com/general/latest/gr/sigv4-date-handling.html