how to set hash in Postman Pre-Request Script for Marvel API - postman-pre-request-script

I have a pre-request script that I gathered from another post on StackOverflow, but I'm still getting invalid credentials.
Attempted to do this just with str_1 but it's not working. Not sure what request.data is supposed to do as it keeps returning NaN. I think that the problem might be there, but still at a loss. I've attempted converting all variables to a string, but that still returned the same error.
URL = https://gateway.marvel.com/v1/public/characters?ts={{timeStamp}}&apikey={{apiKey}}&hash={{hash}}
// Access your env variables like this
var ts = new Date();
ts = ts.getUTCMilliseconds();
var str_1 = ts + environment.apiKey + environment.privateKey;
// Or get your request parameters
var str_2 = request.data["timeStamp"] + request.data["apiKey"];
console.log('str_2 = ' + str_2);
// Use the CryptoJS
var hash = CryptoJS.MD5(str_1).toString();
// Set the new environment variable
pm.environment.set('timeStamp', ts);
pm.environment.set('hash', hash);
{
"code": "InvalidCredentials",
"message": "That hash, timestamp and key combination is invalid."
}

If someone can comment on why this is the solution, I would appreciate it. Here is what the issue was. The order of the hash actually matters. So had to flip the order of pvtkey + pubkey to pubkey + pvtkey. Why is this?
INCORRECT
var message = ts+pubkey+pvtkey;
var a = CryptoJS.MD5(message);
pm.environment.set("hash", a.toString());
CORRECT
var message = ts+pvtkey+pubkey;
var a = CryptoJS.MD5(message);
pm.environment.set("hash", a.toString());

I created in Android Studio, a new java class named MD5Hash, following the steps of https://javarevisited.blogspot.com/2013/03/generate-md5-hash-in-java-string-byte-array-example-tutorial.html
I just simplified his (her) code, only to use it with Java utility MessageDigest
public class MD5Hash {
public static void main(String args[]) {
String publickey = "abcdef"; //your api key
String privatekey = "123456"; //your private key
Calendar calendar=Calendar.getInstance();
String stringToHash = calendar
.getTimeInMillis()+ privatekey + publickey;
System.out.println("hash : " + md5Java(stringToHash));
System.out.println("ts : "+ calendar.getTimeInMillis());
}
public static String md5Java(String message){
String digest = null;
try {
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] hash = md.digest(message.getBytes("UTF-8"));
//converting byte array to Hexadecimal String
StringBuilder sb = new StringBuilder(2*hash.length);
for(byte b : hash){
sb.append(String.format("%02x", b&0xff));
}
digest = sb.toString();
} catch (UnsupportedEncodingException ex) {
} catch (NoSuchAlgorithmException ex) {
}
return digest;
}
}
As you can see, if you copy paste this code, it has a green arrow on the left side of the class declaration, clicking it you can run MD5Hash.main() and you'll have printed in your Run Screen the values for the time (ts) and for the hash.
Then go to verify directly into the internet :
https://gateway.marvel.com/v1/public/characters?limit=20&ts=1574945782067&apikey=abcdef&hash=4bbb5dtf899th5132hjj66

Related

AWS SDK2 java s3 select example - how to get result bytes

I am trying to use aws sdk2 java for s3 select operations but not able to get extract the final data. Looking for an example if someone has implemented it. I got some idea from [this post][1] but not able to figure out how to get and read the full data .
Fetching specific fields from an S3 document
Basically, equivalent of v1 sdk:
``` InputStream resultInputStream = result.getPayload().getRecordsInputStream(
new SelectObjectContentEventVisitor() {
#Override
public void visit(SelectObjectContentEvent.StatsEvent event)
{
System.out.println(
"Received Stats, Bytes Scanned: " + event.getDetails().getBytesScanned()
+ " Bytes Processed: " + event.getDetails().getBytesProcessed());
}
/*
* An End Event informs that the request has finished successfully.
*/
#Override
public void visit(SelectObjectContentEvent.EndEvent event)
{
isResultComplete.set(true);
System.out.println("Received End Event. Result is complete.");
}
}
);```
///IN AWS SDK2, how do get ResultOutputStream ?
```public byte[] getQueryResults() {
logger.info("V2 query");
S3AsyncClient s3Client = null;
s3Client = S3AsyncClient.builder()
.region(Region.US_WEST_2)
.build();
String fileObjKeyName = "upload/" + filePath;
try{
logger.info("Filepath: " + fileObjKeyName);
ListObjectsV2Request listObjects = ListObjectsV2Request
.builder()
.bucket(Constants.bucketName)
.build();
......
InputSerialization inputSerialization = InputSerialization.builder().
json(JSONInput.builder().type(JSONType.LINES).build()).build()
OutputSerialization outputSerialization = null;
outputSerialization = OutputSerialization.builder().
json(JSONOutput.builder()
.build()
).build();
SelectObjectContentRequest selectObjectContentRequest = SelectObjectContentRequest.builder()
.bucket(Constants.bucketName)
.key(partFilename)
.expression(query)
.expressionType(ExpressionType.SQL)
.inputSerialization(inputSerialization)
.outputSerialization(outputSerialization)
.scanRange(ScanRange.builder().start(0L).end(Constants.limitBytes).build())
.build();
final DataHandler handler = new DataHandler();
CompletableFuture future = s3Client.selectObjectContent(selectObjectContentRequest, handler);
//hold it till we get a end event
EndEvent endEvent = (EndEvent) handler.receivedEvents.stream()
.filter(e -> e.sdkEventType() == SelectObjectContentEventStream.EventType.END)
.findFirst()
.orElse(null);```
//Now, from here how do I get the response bytes ?
///////---> ISSUE: How do I get ResultStream bytes ????
return <bytes>
}```
// handler
private static class DataHandler implements SelectObjectContentResponseHandler {
private SelectObjectContentResponse response;
private List receivedEvents = new ArrayList<>();
private Throwable exception;
#Override
public void responseReceived(SelectObjectContentResponse response) {
this.response = response;
}
#Override
public void onEventStream(SdkPublisher<SelectObjectContentEventStream> publisher) {
publisher.subscribe(receivedEvents::add);
}
#Override
public void exceptionOccurred(Throwable throwable) {
exception = throwable;
}
#Override
public void complete() {
}
} ```
[1]: https://stackoverflow.com/questions/67315601/fetching-specific-fields-from-an-s3-document
i came to your post since I was working on the same issue as to avoid V1.
After hours of searching i ended up with finding the answer at. https://github.com/aws/aws-sdk-java-v2/pull/2943/files
The answer is located at SelectObjectContentIntegrationTest.java File
services/s3/src/it/java/software/amazon/awssdk/services/SelectObjectContentIntegrationTest.java
The way to get the bytes is by using the RecordsEvent class, please note for my use case I used CSV, not sure if this would be different for a different file type.
in the complete method you have access to the receivedEvents. this is where you get the first index to get the filtered returned results and casting it to the RecordsEvent class. then this class provides the payload as bytes
#Override
public void complete() {
RecordsEvent records = (RecordsEvent) this.receivedEvents.get(0)
String result = records.payload().asUtf8String();
}

AWS Signature Creation. Confused on how to covert the SigningKey and SingingString into Signature. AWS Example Seems To Not Yield Expected Result

I'm trying to write my own AWS4 signer, and I've gotten about 2/3 of the way there. Source code here :
public class Test
{
private static String region = "us-east-1";
static byte[] HmacSHA256(String data, byte[] key) throws Exception {
String algorithm="HmacSHA256";
Mac mac = Mac.getInstance(algorithm);
mac.init(new SecretKeySpec(key, algorithm));
return mac.doFinal(data.getBytes("UTF8"));
}
public static byte[] justSha256(String data) throws NoSuchAlgorithmException, UnsupportedEncodingException {
MessageDigest digest = MessageDigest.getInstance("SHA-256");
byte[] hash = digest.digest(data.getBytes("UTF8"));
return hash;
}
static byte[] getSigningKey(String key, String dateStamp, String regionName, String serviceName) throws Exception {
byte[] kSecret = ("AWS4" + key).getBytes("UTF8");
byte[] kDate = HmacSHA256(dateStamp, kSecret);
byte[] kRegion = HmacSHA256(regionName, kDate);
byte[] kService = HmacSHA256(serviceName, kRegion);
byte[] kSigning = HmacSHA256("aws4_request", kService);
return kSigning;
}
public static String getSimpleDate()
{
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("YYYYMMdd");
return LocalDate.now().format(formatter);
}
public static String getAMZDate()
{
/*DateTimeFormatter formatter = DateTimeFormatter.ofPattern("YYYYMMDDHHMMSS");
String timeStamp = new SimpleDateFormat("YYMMDD'T'HHMMSS'Z'").format(Calendar.getInstance().getTime());*/
TimeZone tz = TimeZone.getTimeZone("UTC");
DateFormat df = new SimpleDateFormat("yyyyMMdd'T'HHmmss'Z'"); // Quoted "Z" to indicate UTC, no timezone offset
df.setTimeZone(tz);
String timeStamp = df.format(new Date());
return timeStamp;
}
public static String createSigningString(String timeStamp, String simpleDate,String serviceName) throws UnsupportedEncodingException, NoSuchAlgorithmException {
/*AWS4-HMAC-SHA256
20150830T123600Z
20150830/us-east-1/iam/aws4_request
f536975d06c0309214f805bb90ccff089219ecd68b2577efef23edd43b7e1a59*/
String algorithm = "AWS4-HMAC-SHA256\n";
String amzDate = timeStamp+"\n";
String simpleDateRegionServiceRequest = simpleDate+"/"+region+"/"+serviceName+"/"+"aws4_request\n";
String canonicalHash = getCanonicalHash(getCanonicalString("GET","/","Action=ListUsers&Version=2010-05-08","20150830T123600Z",""));
String signingString = algorithm+amzDate+simpleDateRegionServiceRequest+canonicalHash;
return signingString;
}
public static String getCanonicalString(String method, String absolutePath, String queryString, String timeStamp, String payload) throws UnsupportedEncodingException, NoSuchAlgorithmException {
String contentType = "Content-Type:application/x-www-form-urlencoded; charset=utf-8\n".toLowerCase();
String hostUrl = "host:iam.amazonaws.com\n";
String date = "x-amz-date:"+timeStamp+"\n";
String signedHeader = "content-type;host;x-amz-date\n";
String hashedPayload = Hex.encodeHexString(justSha256(payload)).toLowerCase();
String canonicalString = method+"\n"+absolutePath+"\n"+queryString+"\n"+contentType+hostUrl+date+"\n"+signedHeader+hashedPayload;
return canonicalString;
}
public static String getCanonicalHash(String canonicalString) throws UnsupportedEncodingException, NoSuchAlgorithmException {
return Hex.encodeHexString(justSha256(canonicalString)).toLowerCase();
}
public static void main(String[] args) throws Exception {
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("YYYYMMDDHHMMSS");
String timeStamp = new SimpleDateFormat("YYYYMMDD'T'HHMMSS'Z'").format(Calendar.getInstance().getTime());
String canonString;
System.out.println( canonString = getCanonicalHash(getCanonicalString("GET","/","Action=ListUsers&Version=2010-05-08","20150830T123600Z","")));
String signingString = createSigningString("20150830T123600Z","20150830","iam");
String key = "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY";
String dateStamp = "20120215";
String regionName = "us-east-1";
String serviceName = "iam";
String signingKey = Hex.encodeHexString(getSigningKey(key,dateStamp,regionName,serviceName));
SoftAssertions softly = new SoftAssertions();
softly.assertThat(canonString).isEqualToIgnoringCase("f536975d06c0309214f805bb90ccff089219ecd68b2577efef23edd43b7e1a59");
String copiedSigningString = "AWS4-HMAC-SHA256\n" +
"20150830T123600Z\n" +
"20150830/us-east-1/iam/aws4_request\n" +
"f536975d06c0309214f805bb90ccff089219ecd68b2577efef23edd43b7e1a59";
softly.assertThat(signingString).isEqualTo(copiedSigningString);
softly.assertThat(signingKey).isEqualToIgnoringCase("f4780e2d9f65fa895f9c67b32ce1baf0b0d8a43505a000a1a9e090d414db404d");
dateStamp ="20150830";
signingKey = Hex.encodeHexString(getSigningKey(key,dateStamp,regionName,serviceName));
softly.assertThat(signingKey).isEqualToIgnoringCase("c4afb1cc5771d871763a393e44b703571b55cc28424d1a5e86da6ed3c154a4b9");
System.out.println("COPIED STRING : "+copiedSigningString);
System.out.println("SIGNING KEY : "+signingKey);
String signature = Hex.encodeHexString(HmacSHA256(signingKey.trim(),justSha256(copiedSigningString)));
System.out.println("Signature : "+signature);
softly.assertThat(signature).isEqualToIgnoringCase("5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7");
softly.assertAll();
}
}
For some reason I'm failing to create the correct Signature. Which is created by using the SigningKey and SingingString as input into an HMACSha256 function, specifically the function highlighted on this page https://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-java.
But for some reason, I cannot produce the signature that AWS says will be created here on this page.: https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html
(5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7)
Despite using their function, their Strings as input, and double checking my work. What am I missing?
Are you supposed to use a different function for these Strings or byte[]?
I feel like I'm missing something, and I don't know where to reconcle this confusion because from my code and what I see on these pages I seem to be doing the right thing.
I'm especially confused because when I pasted the String from their site into my code to see if it would convert properly it did not, despite using their same Hmac function (which did work for creating every other hash string posted on their site).
The hex representation of the date, region, service, and signing keys is shown for illustration, because the keys contain bytes that do not represent printable characters.
But you appear to be hex-encoding your signing key before using it to sign the request. Don't do that. You will want to hex-encode it only for viewing/debugging. The actual signing key should be retained and used in its original binary/byte form.
Use the digest (binary format) for the key derivation. Most languages have functions to compute either a binary format hash, commonly called a digest, or a hex-encoded hash, called a hexdigest. The key derivation requires that you use a binary-formatted digest.
https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html
The canonical request hash in the string-to-sign is used in hex-encoded form, as is the final signature. The key derivation is all binary.

aws s3 - browser upload using post - gwt & java

I am working on a wgt webapp and would like to upload files to s3 from the browser.
Since my credentials are on the server, I need to create the signature on the server and send it to the client to be able to ulpoad.
here is the code I am using:
-dateStamp is in the right format - yyyyMMdd
-policy is base64 encoded - I double checked that
public static String getSignatureForS3Upload(final String dateStamp, final String policy) {
byte[] signingKey = null;
byte[] signature = null;
String strSignature = null;
try {
signingKey = AwsUtill.getSignatureKey(AppConfig.getS3SecretKey(), dateStamp,
AppConfigShared.getMyAwsS3RegionName(), "s3");
signature = HmacSHA256(policy, signingKey);
strSignature = bytesToHex(signature);
}
catch (Exception e) {
// log
}
ServerDBLogger.log(Level.INFO, byteArrayToHex(signature));
ServerDBLogger.log(Level.INFO, bytesToHex(signature));
return strSignature;
private static byte[] HmacSHA256(final String data, final byte[] key) throws Exception {
String algorithm = "HmacSHA256";
Mac mac = Mac.getInstance(algorithm);
mac.init(new SecretKeySpec(key, algorithm));
return mac.doFinal(data.getBytes("UTF-8"));
}
private static byte[] getSignatureKey(final String key, final String dateStamp, final String regionName,
final String serviceName) throws Exception {
byte[] kSecret = ("AWS4" + key).getBytes("UTF-8");
byte[] kDate = HmacSHA256(dateStamp, kSecret);
byte[] kRegion = HmacSHA256(regionName, kDate);
byte[] kService = HmacSHA256(serviceName, kRegion);
byte[] kSigning = HmacSHA256("aws4_request", kService);
return kSigning;
}
final protected static char[] hexArray = "0123456789ABCDEF".toCharArray();
public static String bytesToHex(final byte[] bytes) {
char[] hexChars = new char[bytes.length * 2];
for (int j = 0; j < bytes.length; j++) {
int v = bytes[j] & 0xFF;
hexChars[j * 2] = hexArray[v >>> 4];
hexChars[j * 2 + 1] = hexArray[v & 0x0F];
}
return new String(hexChars);
}
With the genarated signature I get an error message:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
What am I doing wrong?
Is there any good tutorial or sample code to do this?
Thank you!
There have been changes to the AWS signature process - the version at this time (May 2016) is AWS Signature version 4 (http://docs.aws.amazon.com/general/latest/gr/signature-version-4.html). You need to be careful when looking at examples and Stackoverflow etc that the information relates to the same signature version you are using as they don't mix well.
I started using the AWS SDK for an angular/node file upload but eventually found it easier to generate the policy on the server (node.js) side without the SDK. There is a good example (albeit node based which may not be what you are looking for) here: https://github.com/danialfarid/ng-file-upload/wiki/Direct-S3-upload-and-Node-signing-example (but note the issue with the S3 bucket name here: AngularJs Image upload to S3 ).
One key thing to watch is that you correctly include the file content type in the policy generation and that this content type properly matches the content type of the file you are actually uploading.

How do I provide ObjectContent that is a string

I'm writing a unit test which tests the scenario where a body is sent in the request which is a plain string, i.e. not parseable as JSON.
In this test, I'm setting the HttpRequestMessage something like this:
var ojectContent = new ObjectContent(typeof(string)
, "aaaaa"
, new JsonMediaTypeFormatter());
httpRequestMessage.Content = objectContent;
The problem is, when I debug the code, the request body has been set to "aaaaa" (note the additional quotes) which is enough to cause the deserialisation code to treat the request body differently, meaning I can't test what I mean to test. I need the request body to be aaaaa.
Can anyone advise how I can set up the test so that the request body does not contain these quotes?
Edit: I have also tried new ObjectContent(typeof(object)... and it gives the same result.
Another way is to bypass the MediaTypeFormatter by using StringContent instead of ObjectContent:
var content = new StringContent("aaaaa");
httpRequestMessage.Content = content;
Okay, so I needed to create a media type formatter that didn't interfere with the input in any way. I used this:
private class DoNothingTypeFormatter : MediaTypeFormatter
{
public override bool CanReadType(Type type)
{
return false;
}
public override bool CanWriteType(Type type)
{
if (type == typeof(string))
{
return true;
}
return false;
}
public override Task WriteToStreamAsync(Type type, object value, System.IO.Stream writeStream, HttpContent content, TransportContext transportContext)
{
var myString = value as string;
if (myString == null)
{
throw new Exception("Everything is supposed to be a string here.");
}
var length = myString.Length;
var bytes = System.Text.Encoding.UTF8.GetBytes(myString);
return Task.Factory.StartNew(() => writeStream.Write(bytes, 0, length));
}
}
Then, when I want to generate the body of the `HttpRequestMessage', I do so like this:
objectContent = new ObjectContent(typeof(string)
, "not json"
, new DoNothingTypeFormatter());

java.lang.classcastexception - SOAP Fault - Using KSOAP2 in Android

Searched quite a bit. The problem with these errors is that while the text might appear the same, the problem is always different.
My service takes ONE string value and returns a string response. Here is my code:
private class UploadStats extends AsyncTask<String, Void, String>
{
private static final String WSDL_TARGET_NAMESPACE = "http://tempuri.org/";
private static final String SOAP_ADDRESS = "http://192.168.1.101/rss/RSS_Service.asmx?WSDL";
private static final String INSERTURLACTION = "http://192.168.1.101/rss/RSS_Service.asmx/InsertURL";
private static final String INSERTURLMETHOD = "InsertURL";
#Override
protected String doInBackground(String... url)
{
String status = "";
SoapObject request = new SoapObject(WSDL_TARGET_NAMESPACE,
INSERTURLMETHOD);
request.addProperty("url", "www.yahoo.com");
SoapSerializationEnvelope envelope = new SoapSerializationEnvelope(SoapEnvelope.VER11);
envelope.dotNet = true;
envelope.setOutputSoapObject(request);
HttpTransportSE httpTransport = new HttpTransportSE(SOAP_ADDRESS);
try
{
httpTransport.call(INSERTURLACTION, envelope);
SoapObject response = (SoapObject) envelope.bodyIn;
status = response.getProperty(0).toString();
}
catch (Exception exception)
{
Log.d("Error", exception.toString());
}
if(status.equals("1"))
{
Log.d("InsertURL", "New URL Inserterd");
}
else if(status.equals("0"))
{
Log.d("InsertURL", "URL Exists. Count incremented");
}
else
{
Log.d("InsertURL", "Err... No");
}
return status;
}
}
I get the error:
java.lang.classcastexception org.ksoap2.SoapFault
What am I doing wrong? If any more details are needed, I can add them.
The error was related to the webservice.
An incorrect namespace on the service side can cause this error (as can a lot of other problems).
Best way to check is to run the webservice on the local machine (where the service is hosted).