I am trying to make a request.post(someURL) where the host name is not the same as the url.
The host I am trying to reach is in a private cluster but is accessible from a public URL. The problem is when I make a normal post request I get a 404 response because the host is expecting its internal name (hostName).
Is there a way to do something like response = request.post(someURL, hostname = hostName, data = data) so the request will go to someURL but give hostName as the host name instead of someURL.
"hosts" are almost always passed in the header. try something like this
#add/remove whatever other headers you need
headers={
'Accept':'application/json, text/plain, */*',
'Accept-Encoding':'gzip, deflate, br',
'Accept-Language':'en-US,en;q=0.9',
'Host':'hostName'
}
response = request.post(someURL, headers=headers, data = data)
Note you may want to check the request as some sites require a Host, Origin, Referer, etc.
Related
this code snippet is taken from Postman. cURL taken from the postman works fine and java code generated from postman gives a 200 response for the particular call. but the response body is not there.
what should be the user agent header?
Do I need to use this postman token in my java code as well?
Do I need to add additional headers?
My Goal is to fetch some data from this GET call.
OkHttpClient client = new OkHttpClient();
Request request = new Request.Builder()
.url("http://blahblah=60041441&attributes=blah,blah,blah")
.get()
.addHeader("User-Agent", "PostmanRuntime/7.13.0")
.addHeader("Accept", "*/*")
.addHeader("Cache-Control", "no-cache")
.addHeader("Postman-Token", "7af03a15-blah,364c160f-92d7-459f-b261-4993801944a7")
.addHeader("Host", "blahblah.na.blah.net:9081")
.addHeader("cookie", "someURL=1800; com.ibm.isim.lastActivity=blahblahToekn; JSESSIONID=blahblahblah:1ajblahi8; LtpaToken2=blahblahbalah")
.addHeader("accept-encoding", "gzip, deflate")
.addHeader("Connection", "keep-alive")
.addHeader("cache-control", "no-cache")
.addHeader("User-Agent", "postman")
.build();
okhttp3.Response response= client.newCall(request).execute();
System.out.println(response.body().toString());
Suppose for simple get request following will do just fine, all other details can be omitted:
Request request = new Request.Builder()
.url("http://blahblah=60041441&attributes=blah,blah,blah")
.get()
.build();
Most of the headers (like user-agent, accept-encoding etc) will be automatically added by OkHttp client, so you can safely remove those from request:
.addHeader("User-Agent", "PostmanRuntime/7.13.0")
.addHeader("Host", "blahblah.na.blah.net:9081")
.addHeader("accept-encoding", "gzip, deflate")
.addHeader("Cache-Control", "no-cache")
.addHeader("Connection", "keep-alive")
.addHeader("cache-control", "no-cache")
.addHeader("User-Agent", "postman")
Since / is a wildcard, suppose you can skip it as well.
.addHeader("Accept", "*/*")
If you endpoint requires authentication, suppose before sending this particular Get request you need to send authentication request first. To automatically handle authentication cookies you can try to add CookieJar to your client, so those can be omitted as well (assume headers names were altered somehow, btw?):
.addHeader("Postman-Token", "7af03a15-blah,364c160f-92d7-459f-b261-4993801944a7")
.addHeader("cookie", "someURL=1800; com.ibm.isim.lastActivity=blahblahToekn; JSESSIONID=blahblahblah:1ajblahi8; LtpaToken2=blahblahbalah")
You can also check answers for that question about the ways to add CookieJar.
I am attempting to use a pre-signed URL to upload as described in the docs (https://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObject.html) I can retrieve the pre-signed URL but when I attempt to do a PUT in Postman, I receive the following error:
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
Obviously, the way my put call is structured doesn't match with the way AWS is calculating the signature. I can't find a lot of information on what this put call requires.
I've attempted to modify the header for Content-Type to multipart/form-data and application/octet-stream. I've also tried to untick the headers section in postman and rely on the body type for both form-data and binary settings where I select the file. The form-data setting results in the following added to the call:
Content-Disposition: form-data; name="thefiletosend.txt"; filename="thefiletosend.txt
In addition, I noticed that postman is including what it calls "temporary headers" as follows:
Host: s3.amazonaws.com
Content-Type: text/plain
User-Agent: PostmanRuntime/7.13.0
Accept: /
Cache-Control: no-cache
Postman-Token: e11d1ef0-8156-4ca7-9317-9f4d22daf6c5,2135bc0e-1285-4438-bb8e-b21d31dc36db
Host: s3.amazonaws.com
accept-encoding: gzip, deflate
content-length: 14
Connection: keep-alive
cache-control: no-cache
The Content-Type header may be one of the issues, but I'm not certain how to exclude these "temporary headers" in postman.
I am generating the pre-signed URL in a lambda as follows:
public string FunctionHandler(Input input, ILambdaContext context)
{
_logger = context.Logger;
_key = input.key;
_bucketname = input.bucketname;
string signedURL = _s3Client.GetPreSignedURL(new GetPreSignedUrlRequest()
{
Verb = HttpVerb.PUT ,
Protocol = Protocol.HTTPS,
BucketName = _bucketname,
Key = _key,
Expires = DateTime.Now.AddMinutes(5)
});
returnObj returnVal = new returnObj() { url = signedURL };
return JsonConvert.SerializeObject(returnVal);
}
Your pre-signed url should be like https://bucket-name.s3.region.amazonaws.com/folder/filename.jpg?AWSAccessKeyId=XXX&Content-Type=image%2Fjpeg&Expires=XXX&Signature=XXX
You can upload to S3 with postman by
Set above url as endpoint
Select PUT request,
Body -> binary -> Select file
I was able to get this working in Postman using a POST request. Here are the details of what worked for me. When I call my lambda to get a presigned URL here is the json that comes back (after I masked sensitive and app-specific information):
{
"attachmentName": "MySecondAttachment.docx",
"url": "https://my-s3-bucket.s3.amazonaws.com/",
"fields": {
"acl": "public-read",
"Content-Type": "multipart/form-data",
"key": "attachment-upload/R271645/65397746_MySecondAttachment.docx",
"x-amz-algorithm": "AWS4-HMAC-SHA256",
"x-amz-credential": "WWWWWWWW/20200318/us-east-1/s3/aws4_request",
"x-amz-date": "20200318T133309Z",
"x-amz-security-token": "XXXXXXXX",
"policy": "YYYYYYYY",
"x-amz-signature": "ZZZZZZZZ"
}
}
In Postman, create a POST request, and use “form-data” to enter in all the fields you got back, with exactly the same field names you got back in the signedURL shown above. Do not set the content type, however. Then add one more key named “file”:
To the right of the word file if you click the drop-down you can browse to your file and attach it:
In case it helps, I’m using a lambda written in python to generate a presigned URL so a user can upload an attachment. The code looks like this:
signedURL = self.s3.generate_presigned_post(
Bucket= "my-s3-bucket",
Key=putkey,
Fields = {"acl": "public-read", "Content-Type": "multipart/form-data"},
ExpiresIn = 15,
Conditions = [
{"acl": "public-read"},
["content-length-range", 1, 5120000]
]
)
Hope this helps.
Your pre-signed url should be like https://bucket-name.s3.region.amazonaws.com/folder/filename.jpg?AWSAccessKeyId=XXX&Content-Type=image%2Fjpeg&Expires=XXX&Signature=XXX
You can upload to S3 with postman by
Set above url as endpoint
Select PUT request,
Body -> binary -> Select file
I was facing the same problem and below is how it worked for me.
Note, I am making signed URL by using AWS S3 Java SDK as my backend is in Java. I gave content type as "application/octet-stream" while creating this signed Url so that any type of content can be uploaded. Below is my java code generating signed url.
public String createS3SignedURLUpload(String bucketName, String objectKey) {
try {
PutObjectRequest objectRequest = PutObjectRequest.builder().bucket(bucketName).key(objectKey)
.contentType("**application/octet-stream**").build();
S3Presigner presigner = S3Presigner.builder().region(s3bucketRegions.get(bucketName))
.credentialsProvider(StaticCredentialsProvider.create(awsBasicCredentials)).build();
PutObjectPresignRequest presignRequest = PutObjectPresignRequest.builder()
.signatureDuration(Duration.ofMinutes(presignedURLTimeoutInMins)).putObjectRequest(objectRequest)
.build();
PresignedPutObjectRequest presignedRequest = presigner.presignPutObject(presignRequest);
return presignedRequest.url().toString();
} catch (Exception e) {
throw new CustomRuntimeException(e.getMessage());
}
}
## Now to upload file using Postman
Set the generated url as endpoint
Select PUT request,
Body -> binary -> Select file
Set header Content-Type as application/octet-stream (This point I was missing earlier)
It's actually depends in how you generated URL,
If you generated using JAVA,
Set the generated url as endpoint
Select PUT request,
Body -> binary -> Select file
If you generated using PYTHON,
Create a POST request, and use form-data to enter in all the fields you got back, with exactly the same field names you got back in the signedURL shown above.
Do not set the content type, however. Then add one more key named “file”:
Refer the accepted answer picture
I'm trying to send https requests with python "requests" lib
payLoad = {'session_id': str(webSessionId), 'Merchant': 'Company'}
resp = requests.get(url, params = payLoad, cookies = cookiesSession, headers = headers)
print resp.url
But I'm getting server error 400. At the same time if to check url from sent request can see following:
https://url/error
Passed parameters are missing.
If to remove cookies following url is returned:
https://url?Merchant=Company&session_id=d1160eda748a5ede015bbe1abe35606138086fc8
But Server returns 403 error(authorization)
Cookies and headers are valid as were successfully used in previous two https request. But here is some strange situation. How can I send "get" request with params and cookies?
I am trying to send mails with attachment by using Amazon SES
HttpRequest httpReq = new HttpRequest();
httpReq.setMethod('POST');
httpReq.setEndpoint('https://email.us-east-1.amazonaws.com');
Blob bsig = Crypto.generateMac('HmacSHA256', Blob.valueOf(awsFormattedNow), Blob.valueOf(secret));
httpReq.setHeader('X-Amzn-Authorization','AWS3-HTTPS AWSAccessKeyId='+key+', Algorithm=HmacSHA256, Signature='+EncodingUtil.base64Encode(bsig));
httpReq.setHeader('Date',awsFormattedNow);
httpReq.setHeader('From','sample#gmail.com');
httpReq.setHeader('To','sample#gmail.com');
httpReq.setHeader('Subject','Hello');
httpReq.setHeader('Accept-Language','en-US');
httpReq.setHeader('Content-Language','en-US');
httpReq.setHeader('Content-Type','multipart/mixed;boundary=\"_003_97DCB304C5294779BEBCFC8357FCC4D2\"');
httpReq.setHeader('MIME-Version','1.0');
// httpReq.setHeader('Action','SendRawEmail');
String email = 'Action=SendRawEmail';
email += '--_003_97DCB304C5294779BEBCFC8357FCC4D2 \n Content-Type: text/plain; charset="us-ascii" \n Content-Transfer-Encoding: quoted-printable \n';
email +='Hi Andrew. Here are the customer service names and telephone numbers I promised you.';
httpReq.setBody(email);
System.debug(httpReq.getBody());
Http http = new Http();
HttpResponse response = http.send(httpReq);
I am getting error like
<AccessDeniedException>
<Message>Unable to determine service/operation name to be authorized</Message>
</AccessDeniedException>
Kindly please help me where i am doing wrong .Thanks in advance
Take another look at the documentation. There are several issues with your code.
SES expects an HTTP POST with all of the parameters strung together consistent with application/x-www-form-urlencoded POST requests.
Your HTTP request needs to be Content-type: application/x-www-form-urlencoded, not multipart/mixed... -- that's part of the raw message you're trying to send.
You are mixing up things that should be in the body, and setting HTTP request headers, instead. For example, these are also incorrect:
httpReq.setHeader('From','sample#gmail.com');
httpReq.setHeader('To','sample#gmail.com');
httpReq.setHeader('Subject','Hello');
These should go in the request body, not in the HTTP request headers. Also, the values are urlencoded. From the example code:
Action=SendEmail
&Source=user%40example.com
&Destination.ToAddresses.member.1=allan%40example.com
The line breaks were added for clarity.
Your interests might be best served by trying to successfully send a simple e-mail, first, and then later attempting to modify your code to support attachments, because you have numerous errors that will need to be corrected before this code will work properly.
http://docs.aws.amazon.com/ses/latest/DeveloperGuide/query-interface-requests.html
http://docs.aws.amazon.com/ses/latest/APIReference/API_SendRawEmail.html
I'm new to internet programming and got a problem when sending a post request to a server (I received an unspecific error message). The server I want to connect is Microsoft HealthVault PreProductionEnvironment. The Specification say the this:
"HealthVault Service communications occur via schematized XML in the body of an HTTP request over a TLS Connection [RFC2818]. The HTTP requests MUST be submitted using the POST HTTP verb."
I tried to check the post request by sending it to "http://posttestserver.com/post.php?dir=ms_hv" (result displayed on "http://posttestserver.com/data/2011/12/05/ms_hv/"). This is the data I got:
Time: Mon, 05 Dec 11 05:26:53 -0800
Source ip: xxx.xxx.xxx.xxx
Headers (Some may be inserted by server)
UNIQUE_ID = TtzGna3sqvkAABZHetMAAAAH
CONTENT_LENGTH = 1157
HTTP_CONNECTION = close
HTTP_ACCEPT_ENCODING = gzip
HTTP_ACCEPT_LANGUAGE = en-US,*
HTTP_USER_AGENT = Mozilla/5.0
HTTP_HOST = posttestserver.com
CONTENT_TYPE = application/x-www-form-urlencoded
REMOTE_ADDR = 188.98.76.99
REMOTE_PORT = 21675
GATEWAY_INTERFACE = CGI/1.1
REQUEST_METHOD = POST
QUERY_STRING = dir=ms_hv
REQUEST_URI = /post.php?dir=ms_hv
REQUEST_TIME = 1323091613
Post Params:
key: '<?xml_version' value: '"1.0" encoding="utf-8"?><wc-request:request xmlns:wc-request="urn:com.microsoft.wc.request"><header><method>CreateAuthenticatedSessionToken</method><method-version>2</method-version><app-id>bc1fdbef-f814-4ee3-b414-1e9f6956c7ab</app-id><language>de</language><country>DE</country><msg-time>2011-12-05T14:26:51.31Z</msg-time><msg-ttl>1800</msg-ttl><version>2.0.0.0</version></header><info><auth-info><app-id>05a059c9-c309-46af-9b86-b06d42510550</app-id><credential><appserver2><sig digestMethod="SHA1" sigMethod="RSA-SHA1" thumbprint="50054D4FAEE16F69AAF66B7596ED30F9922B949E">fZaPzgI3x2ngzKnI2AU0leWgR7ycj7GcABPxktIKRd/u5eWk4dMGLiHv7x9oLKDAGC9BtAOGqe1YX80OpRFQC3VCioc9SGyAtBf0dTiM2tNcDTIboNqq0 m/d8FaI/MjIt8SlvXbV/LmCnEnjToPaiYnXqesJLflNUBRgotH2MK6YljcdB3G2JxghBlaqAYXKfx7c 8WQ9hA nnw75kTaJWXdgTAtdb0yUT5poYvm/ahG1PEl9BQCrR CmdYGECet4nlT1pFozYizI1lpZ4DQEv5eDz9YHGsqGO59oSOTX3iamRK wsaVvhKUBqS6 g7Q34wRa pv9mTocenMmCFOQ==</sig><content><app-id>bc1fdbef-f814-4ee3-b414-1e9f6956c7ab</app-id><hmac>HMACSHA256</hmac><signing-time>2011-12-05T14:26:51.3193190Z</signing-time></content></appserver2></credential></auth-info></info></wc-request:request>'
== Begin post body ==
== End post body ==
The issue I don't understand is the post body beeing empty. All the post-data is shown in "Post Params". Is this correct? Could this be the fault?
Please tell me if you need more detailed description.
Solution (probably):
The post body is empty but in the specification it says "XML in the body of an HTTP request", this probably means "in the POST body" (but mine is empty). I'll check out this posibility.
By using a "?" in your URL, you use the HTTP GET method. You can mix GET and POST in one request (which can lead to some obscure problems, especially when register_globals is switched on in your php-installation), but you shouldn't:
As I interpret your error-message, your server does not accept submitting the informations via HTTP GET. Instead you have to use the HTTP POST method. An example for a HTTP POST using QT you can find here.