I am trying to simulate this in Postman. When you click the button it is opened in another window so I cant see the request in the developer network toolbar in Crome. However I can see it in Fiddler. The request look like this:
POST http://www.q88.com:8081/ws/API.asmx/GetHVPQQuestions_DataSet HTTP/1.1
Host: www.q88.com:8081
Proxy-Connection: keep-alive
Content-Length: 52
Cache-Control: max-age=0
Origin: http://www.q88.com
Upgrade-Insecure-Requests: 1
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Referer: http://www.q88.com/ws/API.asmx?op=GetHVPQQuestions_DataSet
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9,da-DK;q=0.8,da;q=0.7,und;q=0.6
Cookie: CompanyID=-1
AuthorizationString=XXXXXXXXXXXXXXXXXXXXXXXXXXXX
Now I would like that to be exported to Postman. However as I cannot get the curl from Crome, how do I add this raw request to Postman easily?
To import headers
Fiddler: File > Export Sessions > Selected Sessions... > cURL Script > Save as *.bat > Copy content of file
Postman > Import > Raw Text > Paste copied text
To import body
Fiddler > Inspectors > Raw > Copy content of last line
Postman > Body > raw > Paste line
I tried the above method and failed with Fiddler 4 and Postman v9.1.3. It cause Error while importing Curl: URI malformed
Change to
Download FiddlerExportToPostman
Unzip the two file to Fiddler root folder's subfolder, with the name of ImportExport
Restart Fiddler
Fiddler：Select what you want to export → File → Export Sessions → Selected Sessions... → Postman → Next → Save
Postman：Import → File → Choose the file you export
Related
Starting from few days ago, I've started to receive BadRequestException with Unexpected end of JSON input
This is the trace from NestJS:
mapExternalException(err) {
switch (true) {
// SyntaxError is thrown by Express body-parser when given invalid JSON (#422, #430)
// URIError is thrown by Express when given a path parameter with an invalid percentage
// encoding, e.g. '%FF' (#8915)
case err instanceof SyntaxError || err instanceof URIError:
return new common_1.BadRequestException(err.message); // <--- throws here
default:
return err;
}
}
Application is hosted on AWS and what's weird to me is the url from the request that you can see from sentry log:
{
data: {},
headers: {
accept-encoding: gzip,
content-length: 1024,
content-type: application/json,
host: IP_THAT_SEEMS_FROM_AMAZON,
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36,
x-amzn-trace-id: Root=1-some-trace-id,
x-forwarded-for: SOME_IP,
x-forwarded-port: 443,
x-forwarded-proto: https
},
method: POST,
query_string: {},
url: https://IP_THAT_SEEMS_FROM_AMAZON/api/report
}
Is it possible that AWS is doing internal requests to /api/report endpoint, and if e.g. HTML is returned application throws Unexpected end of JSON input (because that's usual reason for that error when making requests)?
If anyone is familiar with what's happening, any help would be appreciated!
I have trouble with setting default file for subdirectory.
http://d2770ni5llmjzr.cloudfront.net/songinfo/index.html works properly, but http://d2770ni5llmjzr.cloudfront.net/songinfo ends up with downloading blank file.
I already set static domain of my bucket as origin of distribution.
Following is log of curl -v http://d2770ni5llmjzr.cloudfront.net/songinfo
* Trying 13.225.105.93...
* TCP_NODELAY set
* Connected to d2770ni5llmjzr.cloudfront.net (13.225.105.93) port 80 (#0)
> GET /songinfo HTTP/1.1
> Host: d2770ni5llmjzr.cloudfront.net
> User-Agent: curl/7.55.1
> Accept: */*
>
HTTP/1.1 302 Moved Temporarily
Content-Type: text/html; charset=utf-8
Content-Length: 313 < Connection: keep-alive
x-amz-error-code: Found
x-amz-error-message: Resource Found
Location: /songinfo/
Date: Wed, 31 Jul 2019 20:21:42 GMT
Server: AmazonS3
Age: 9262
X-Cache: Hit from cloudfront
Via: 1.1 2f061b9f7c9e6f3de68229cddaa32e46.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: ICN54-C1
X-Amz-Cf-Id: KgKM9xl5ggcwOWvoUng9rWiVE12UI0q0oPvCUJ2IvJxmyPlJtvmMRg==
<html> <head><title>302 Moved Temporarily</title></head> <body> <h1>302 Moved Temporarily</h1> <ul> <li>Code: Found</li> <li>Message: Resource Found</li> <li>RequestId: 454F29ADAC061274</li> <li>HostId: FsYd6DZeb9nuWA/wQazkp+doZmsriSoakDrcsyQy7jkFQFw/y8ngKW0lR1yT1sYprBJ6CtjK3KU=</li> </ul> <hr/> </body> </html>
* Connection #0 to host d2770ni5llmjzr.cloudfront.net left intact
Web files named index (ex. index.html, index.php) are the default files for their directory. The index.html file loaded automatically on safari, chrome, and firefox for me.
>HTTP/1.1 302 Moved Temporarily
>Location: /songinfo/
This is an HTTP redirect, and it is exactly what S3 is supposed to do when you ask for a "directory" that will serve an index document, but you don't supply the trailing slash in the request.
Your site is working correctly, but curl doesn't follow these redirects by default. You need to tell it to, with the --location (long form) or -L (short form) option.
curl -v -L https://...
You should then find that curl does what you expect.
Let me start this with, I do not know python, I've had maybe 1 day going through the python tutorials. The situation is this. I have an angular app that has a python app hosted with Apache on a vm in an iframe. I didn't write the python app, but another developer wrote me an endpoint where I am supposed to be able to post from my angular app.
The developer who made the python endpoint is saying that there is something wrong with my request but I am fairly certain there isn't anything wrong. I am almost 100% certain that the problem is that there are no CORS headers in the response and/or the response is not set up to respond to the OPTIONS method. Below is the entirety of the python endpoint:
import os, site, inspect
site.addsitedir(os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))+"/../")
import json
from datetime import datetime
import pymongo
from Config import Config
def application(environ, start_response):
response = environ['wsgi.input'].read(int(environ['CONTENT_LENGTH']))
if response:
json_response = json.loads(response)
document = {
'payment_id': json_response['payment_id'],
'log': json_response['log'],
'login_id': json_response['login_id'],
'browser': environ.get('HTTP_USER_AGENT', None),
'ip_address': environ.get('REMOTE_ADDR', None),
'created_at': datetime.utcnow(),
}
client = pymongo.MongoClient(Config.getValue('MongoServer'))
db = client.updatepromise
db.PaymentLogs.insert(document)
start_response('200 OK', [('Content-Type', 'application/json')
return '{"success": true}'
start_response('400 Bad Request', [('Content-Type', 'application/json')])
return '{"success": false}'
I have attempted the following to make this work: I added to both start_response functions more headers so the code looks like this now:
start_response('201 OK', [('Content-Type', 'application/json',
('Access-Control-Allow-Headers','authorization'),
('Access-Control-Allow-Methods','HEAD, GET, POST, PUT, PATCH, DELETE'),
('Access-Control-Allow-Origin','*'),
('Access-Control-Max-Age','600'))])
Not: I did this both with the 200 and the 400 response at first, and saw no change at all in the response, then just for the heck of it, I decided to change the 200 to a 201, this also did not come through on the response so I suspect this code isn't even getting run for some reason.
Please help, python newb here.
Addendum, i figured this would help, here is what the Headers look like in the response:
General:
Request URL: http://rpc.local/api/payment_log_api.py
Request Method: OPTIONS
Status Code: 200 OK
Remote Address: 10.1.20.233:80
Referrer Policy: no-referrer-when-downgrade
Response Headers:
Allow: GET,HEAD,POST,OPTIONS
Connection: Keep-Alive
Content-Length: 0
Content-Type: text/x-python
Date: Fri, 27 Apr 2018 15:18:55 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.18 (Ubuntu)
Request Headers:
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
Access-Control-Request-Headers: authorization,content-type
Access-Control-Request-Method: POST
Connection: keep-alive
Host: rpc.local
Origin: http://10.1.20.61:4200
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36
Here it is. Just add this to the application right at the beginning:
def application(environ, start_response):
if environ['REQUEST_METHOD'] == 'OPTIONS':
start_response(
'200 OK',
[
('Content-Type', 'application/json'),
('Access-Control-Allow-Origin', '*'),
('Access-Control-Allow-Headers', 'Authorization, Content-Type'),
('Access-Control-Allow-Methods', 'POST'),
]
)
return ''
For Python with CGI, I found this to work:
print '''Access-Control-Allow-Origin: *\r\n''',
print '''Content-Type: text/html\r\n'''
Don't forget to enable CORS on the other side as well, e.g., JavaScript jQuery:
$.ajax({ url: URL,
type: "GET",
crossDomain: true,
dataType: "text", etc, etc
I am new to Load runner , Am facing am issue while play back of the script
LR 12.50
O.S Windows 7 SP2
Protocol is Mobile HTTP/HTML
Recording mode is Proxy
Let me explain my scenario
While executing following function:
web_custom_request("authenticate",
"URL=https://ws-xx.xxx.com/tcs/rest/authenticate?include=user,company",
"Method=POST",
"Resource=0",
"RecContentType=application/json",
"Referer=",
"Snapshot=t1.inf",
"Mode=HTTP",
"EncType=application/json",
"Body={\"password\":\"xxx\",\"username\":\"xxx\",\"version\":\"1.0.40\"}",
LAST);
For the above POST method , am getting response as below
HTTP/1.1 200 OK\r\n
Date: Tue, 13 Oct 2015 19:19:21 GMT\r\n
Server: Apache-Coyote/1.1\r\n
Content-Type: application/json\r\n
Set-Cookie: dtCookie=DBE9311E44E5C47902702DC762030583|TXlBcHB8MQ; Path=/;
Domain=.xxx.com\r\n
Connection: close\r\n
Transfer-Encoding: chunked\r\n
Which is fine ,Now the second custom request is shown below
web_custom_request("profiles",
"URL=https://ws-test.xxx.com/tcs/rest/profiles",
"Method=GET",
"Resource=1",
"RecContentType=application/json",
"Referer=",
"Snapshot=t2.inf",
LAST);
For the above GET requests in the replay logs am getting:
401 unauthorized error.
GET /tcs/rest/profiles HTTP/1.1\r\n
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT)\r\n
Accept: */*\r\n
Connection: Keep-Alive\r\n
Host: ws-test.xxx.com\r\n
Cookie: dtCookie=DBE9311E44E5C47902702DC762030583|TXlBcHB8MQ\r\n
\r\n
t=5921ms: 172-byte response headers for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
HTTP/1.1 401 Unauthorized\r\n
Date: Tue, 13 Oct 2015 19:19:22 GMT\r\n
Server: Apache-Coyote/1.1\r\n
Content-Type: application/json\r\n
Connection: close\r\n
Transfer-Encoding: chunked\r\n
\r\n
t=5922ms: 4-byte chunked response overhead for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
8b\r\n
t=5923ms: 139-byte chunked response body for "https://ws-test.xxx.com/tcs/rest/profiles" (RelFrameId=1, Internal ID=2)
{"errors":[{"message":"Authentication required to access endpoint","status":"401","code":"
NotAuthenticated","header":"Not Authenticated"}]}
I refereed this link.
My understanding from the above custom request , login is success but the next
subsequent requests are getting failed.
I have used web_cleanup_cookies() function but didn't solve the issue .
I tried to capture the Cookie ID using the below function
web_reg_save_param("COOKIE_ID",
"LR= Cookie: dtCookie=" ,
"RB= |TXlBcHB8MQ\r\n",
"Ord=All",
"RelFrameId=1",
"Search=All",
LAST);
web_add_header("Cookie",lr_eval_string("{COOKIE_ID}"));
Now question is where to place parameter "COOKIE_ID" in my script while there is
no value in script for COOKIE_ID?
How to handle this issue ? Can anybody please help me .
Please add below headers to the script
web_set_sockets_option("SSL_VERSION","TLS");
web_set_user("username", "password", "domain:portno" );
web_set_sockets_option("INITIAL_BASIC_AUTH","1");
In Vugen, Select snapshot view and compare both record and replay requests, suspecting there might be a missing of header in replay request.
If cookie is the only thing changing you can add it by using web_add_cookie function.
I just started storing user uploaded images on Amazon's S3. It's pretty nice because it took care of my storage problem. However, I am struggling when it comes to having the browser cache the images.
I am using django-storages. In their docs they specify that you can put things on the request header for an image by setting the AWS_HEADER var in your settings. I am doing that and getting no results.
Basically when the app requests the image(s), I get a 200 EVERY TIME. ARG... when I take the browser straight to the image (copy and paste the link into a new window) I get a 200 then a 304 every time after that.
It's very frustrating because it re downloads the image every time. Some pages have up to 25 small thumbnails on them and it's redownloading everything every time the page is reloaded.
I am serving my static files using djangos staticfiles and they are working properly. I get a 200, then 304 after the file is cached.
here are my AWS settings in settings.py
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = '***'
AWS_SECRET_ACCESS_KEY = '***'
AWS_STORAGE_BUCKET_NAME = 'foobar_uploads'
AWS_HEADERS = {
'Expires': 'Thu, 15 Apr 2020 20:00:00 GMT',
'Cache-Control': 'max-age=86400',
}
AWS_CALLING_FORMAT = CallingFormat.SUBDOMAIN
here are the request and response headers for when the app requests the image: (i've replaced what i feel might be sensitive information with '*')
##request##
GET /user_uploads/*****/2012/3/17/14/46/thumb_a_28_DSC_0472.jpg?Signature=FVR6T%2BXFwHMmdQ9K3n7Ppp7QxoY%3D&Expires=1332023525&AWSAccessKeyId=***** HTTP/1.1
Host: *****_user_uploads_sandbox.s3.amazonaws.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.79 Safari/535.11
Accept: */*
Referer: http://localhost:8000/m/my-photos/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
##response##
HTTP/1.1 200 OK
x-amz-id-2: Hn3S+3gmeLHIjKCpz+2ocE6aPsLCVHh56jJYTsPHwxU98y89x+9X1Ml202evBUHT
x-amz-request-id: 528CEB880CA89AD3
Date: Sat, 17 Mar 2012 21:32:06 GMT
Cache-Control: max-age=86400
Expires: Thu, 15 Apr 2020 20:00:00 GMT
Last-Modified: Sat, 17 Mar 2012 20:46:29 GMT
ETag: "a3bc70e0c3fc0deb974edf95668e9030"
Accept-Ranges: bytes
Content-Type: image/jpeg
Content-Length: 8608
Server: AmazonS3
here are the request/response headers for when i manually request the image by copy and pasting link to the image:
##request##
GET /user_uploads/*****/2012/3/17/14/46/thumb_a_28_DSC_0472.jpg?Signature=FVR6T%2BXFwHMmdQ9K3n7Ppp7QxoY%3D&Expires=1332023525&AWSAccessKeyId=***** HTTP/1.1
Host: porlio_user_uploads_sandbox.s3.amazonaws.com
Connection: keep-alive
Cache-Control: max-age=0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.79 Safari/535.11
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
If-None-Match: "a3bc70e0c3fc0deb974edf95668e9030"
If-Modified-Since: Sat, 17 Mar 2012 20:46:29 GMT
##response##
HTTP/1.1 304 Not Modified
x-amz-id-2: FZH0imrbNxziMznhl5zAoo38CaM7Z+TFnd8R6HtTYB3eTmVpCih+1IniKaliRo18
x-amz-request-id: 3CACF77FBB39D088
Date: Sat, 17 Mar 2012 21:33:22 GMT
Last-Modified: Sat, 17 Mar 2012 20:46:29 GMT
ETag: "a3bc70e0c3fc0deb974edf95668e9030"
Server: AmazonS3
I see there are a few differences such as the "If-None-Match:" or the "If-Modified-Since:" . I think that if I were to set those, then it should work like I'd like.
Is there an easy way to do this?
Thanks for any help!
EDIT 1: I read this article and couldn't translate it very well.. http://coder.cl/2012/01/django-and-amazon-s3/comment-page-1/
If you don't want django-storages to add the auth query to your static media, add the following to your settings.py:
AWS_QUERYSTRING_AUTH = False
This is caused by the S3 Boto backend, which will automatically append the query string unless instructed not to, via django-storages. If you look at the source code for the application, you'll notice that it looks for some extra undocumented settings in your settings.py file (around line 34).
I responded to this similar issue raised on the Github for django-compressor here.
It doesn't look like you're sending a "If-None-Match" or "If-Modified-Since" in your app's request, so S3 has no way to send you back a 304 since it has no idea what you have. As you can see your browser has the file cached, so it is sending both of those headers and getting a proper 304 response.
If you're keeping local copies of files, you'll need to store the Last-Modified and/or ETag and send them along with your request for the image.
As a related aside, I'd consider using just the max-age header and skipping Expires. According the to the spec, Expires should not be more then a year in the future. Using both Expires and max-age is redundant, and max-age can be set to a relative amount (as you have done).
I highly recommend this article on caching and his related REDbot tool for checking your configuration.