Convert SoapUI Request into CFHTTP - coldfusion

I am looking at a request in SoapUI that is sending header info to a specific endpoint but I am having a hard time recreating it in ColdFusion.
Below is what the RAW request looks like in SoapUI:
>> "GET https://test-01.mywebsite.com/data_api//1.0/service/requests HTTP/1.1[\r][\n]"
>> "Accept-Encoding: gzip,deflate[\r][\n]"
>> "Authorization: Bearer A1BEC30F7E0273059E775A6A2645E273[\r][\n]"
>> "Host: test-01.mywebsite.com[\r][\n]"
>> "Connection: Keep-Alive[\r][\n]"
>> "User-Agent: Apache-HttpClient/4.1.1 (java 1.5)[\r][\n]"
>> "[\r][\n]"
<< "HTTP/1.1 200 OK[\r][\n]"
<< "Pragma: No-cache[\r][\n]"
<< "Cache-Control: no-cache[\r][\n]"
<< "Expires: Wed, 31 Dec 1969 16:00:00 PST[\r][\n]"
<< "Content-Type: application/json;charset=UTF-8[\r][\n]"
<< "Content-Length: 6796[\r][\n]"
<< "Date: Fri, 13 May 2016 15:40:08 GMT[\r][\n]"
<< "Server: hws[\r][\n]"
<< "Set-Cookie: X-HR-ClientSessionId=2_10.85.12.121_1463154008475;Secure; path=/; HttpOnly[\r][\n]"
<< "Content-Encoding: deflate[\r][\n]”
I am not sure if I am not formatting the Authorization header correctly or what but any help would be great.
EDIT I got a RAW HTML output from the client which I have updated above. I am still trying to recreate that header in ColdFusion.
My New question(s): Do the "New Line" characters make a difference in the header values? Should I also add a parameter for the content type?
I did try the following:
<cfset NL="Bearer BD4DF031B24180C9338F0D9F060556A7" & Chr(10) & Chr(13)/>
<cfhttp method="get" url="https://test-01.mywebsite.com/data_api//1.0/service/requests" result="orderList">
<cfhttpparam type="HEADER" name="Authorization" value="#NL#">
<cfhttpparam type="Header" name="Accept-Encoding" value="gzip,deflate">
</cfhttp>
<cfset CurrentOrders = deserializeJSON(orderList.filecontent)>
<cfdump var="#CurrentOrders#">
When I dump everything from the cfhttp call I get:
struct
Charset UTF-8
ErrorDetail [empty string]
Filecontent Connection Failure
Header HTTP/1.1 200 OK Connection: close Expires: Wed, 31 Dec 1969 16:00:00 PST Date: Tue, 17 May 2016 19:23:36 GMT Server: hws Pragma: No-cache Cache-Control: no-cache Set-Cookie: X-HR-ClientSessionId=3_12.161.115.226_1463513016026;Secure; path=/; HttpOnly Content-Type: application/json;charset=UTF-8
Mimetype application/json
Responseheader
struct
Cache-Control no-cache
Connection close
Content-Type application/json;charset=UTF-8
Date Tue, 17 May 2016 19:23:36 GMT
Expires Wed, 31 Dec 1969 16:00:00 PST
Explanation OK
Http_Version HTTP/1.1
Pragma No-cache
Server hws
Set-Cookie X-HR-ClientSessionId=3_12.161.115.226_1463513016026;Secure; path=/; HttpOnly
Status_Code 200
Statuscode 200 OK
Text NO
I am getting a 200 OK status code but still getting a Connection Failure.

It looks like you're double encrypting your security token.
I modified your code so I could capture the request with Fiddler as per Leighs Answer. To get ColdFusion to send the traffic through Fiddler I modified Dmitri Pisarenko answer for http and added it to my JVM Arguments.
<cfhttp method="get" url="http://localhost/data_api/1.0/service/requests" result="orderList">
<cfhttpparam type="HEADER" name="Authorization" value="Basic #ToBase64("Bearer 6EDC52118E164AE659EA2C772F3B9804")#">
<cfhttpparam type="Header" name="Accept-Encoding" value="gzip,deflate">
</cfhttp>
The head I get leaving the cfhttp request is:
GET http://localhost/data_api/1.0/service/requests HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip,deflate
Connection: close
Authorization: Basic QmVhcmVyIDZFREM1MjExOEUxNjRBRTY1OUVBMkM3NzJGM0I5ODA0
Host: localhost
Connection: Keep-Alive
As you can see, the Authorization header isn't the same as what SoapUI created.
I modified the value of the Authorization param to : "Bearer 6EDC52118E164AE659EA2C772F3B9804" and I get a header with an authentication header that matches the raw header from SoapUI:
GET http://localhost/data_api/1.0/service/requests HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip,deflate
Connection: close
Authorization: Bearer 6EDC52118E164AE659EA2C772F3B9804
Host: localhost
Connection: Keep-Alive

Related

httpOnly cookies in cross-domain requests not being sent

I'm really really sorry for asking yet another CORS question but I'm stuck and I can't understand why.
I have any API that, on login, responds with a Set-Cookie header which sets a cookie with my user info. This is what that response looks like:
Response from api.somesite.com
Access-Control-Allow-Credentials: true
Connection: keep-alive
Content-Length: 183
Content-Type: text/html; charset=utf-8
Date: Mon, 19 Apr 2021 19:55:02 GMT
Set-Cookie: socialAccesstoken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6Ijk5MDk3NzQzNjkxNjU1MTY4MCIsInVzZXJuYW1lIjoiY3JlYXRpdmlpaSIsImRpc3BsYXlOYW1lIjoiTGVvIDjvuI_ig6MiLCJ1cmwiOiJodHRwczovL3QuY28vWjNBM01xaFBDbiIsInBob3RvIjoiaHR0cHM6Ly9wYnMudHdpbWcuY29tL3Byb2ZpbGVfaW1hZ2VzLzEzNzExMzQ3MjY0NzI0MTMxODYvZ0Q5Sk5lVU1fbm9ybWFsLmpwZyIsInByb3ZpZGVyIjoidHdpdHRlciIsImlhdCI6MTYxODg2MjEwMiwiZXhwIjoxNzA1MjYyMTAyfQ.Q7yWf-ywoZ-rWOhYseKTt_0V2_AlEMQ-cCL2rlRNm_U; Max-Age=86400; Path=/; Expires=Tue, 20 Apr 2021 19:55:02 GMT; HttpOnly
Vary: Origin
X-Powered-By: Express
The Cookie is set successfully and I can view it in the Application tab for api.somesite.com.
I then try to make a request from a page on the same domain:
Request from somesite.com to api.somesite.com
Response
HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Origin: http://somesite.com:8000
Vary: Origin
Access-Control-Allow-Credentials: true
Token-Expired: true
Content-Type: application/json; charset=utf-8
Content-Length: 196
ETag: W/"c4-HQkmHNLuibgvd4gvz0JqjTLKjFU"
Date: Mon, 19 Apr 2021 19:59:18 GMT
Connection: keep-alive
Request
Host: api.somesite.com:3000
Connection: keep-alive
Content-Length: 143
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.128 Safari/537.36
content-type: application/json
Accept: */*
Origin: http://somesite.com:8000
Referer: http://somesite.com:8000/
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: socialAccesstoken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6Ijk5MDk3NzQzNjkxNjU1MTY4MCIsInVzZXJuYW1lIjoiY3JlYXRpdmlpaSIsImRpc3BsYXlOYW1lIjoiTGVvIDjvuI_ig6MiLCJ1cmwiOiJodHRwczovL3QuY28vWjNBM01xaFBDbiIsInBob3RvIjoiaHR0cHM6Ly9wYnMudHdpbWcuY29tL3Byb2ZpbGVfaW1hZ2VzLzEzNzExMzQ3MjY0NzI0MTMxODYvZ0Q5Sk5lVU1fbm9ybWFsLmpwZyIsInByb3ZpZGVyIjoidHdpdHRlciIsImlhdCI6MTYxODg2MTQ2MiwiZXhwIjoxNzA1MjYxNDYyfQ.BpmQzWWYuBHLQQh6VpatflBsQUv2A0ahLqt1UgYOq8Q
As you can see the cookie is sent correctly.
But then if I switch my api to api.differentsite.com and make the same requests:
Response from api.differentsite.com
HTTP/1.1 200 OK
X-Powered-By: Express
Vary: Origin
Access-Control-Allow-Credentials: true
Set-Cookie: socialAccesstoken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6Ijk5MDk3NzQzNjkxNjU1MTY4MCIsInVzZXJuYW1lIjoiY3JlYXRpdmlpaSIsImRpc3BsYXlOYW1lIjoiTGVvIDjvuI_ig6MiLCJ1cmwiOiJodHRwczovL3QuY28vWjNBM01xaFBDbiIsInBob3RvIjoiaHR0cHM6Ly9wYnMudHdpbWcuY29tL3Byb2ZpbGVfaW1hZ2VzLzEzNzExMzQ3MjY0NzI0MTMxODYvZ0Q5Sk5lVU1fbm9ybWFsLmpwZyIsInByb3ZpZGVyIjoidHdpdHRlciIsImlhdCI6MTYxODg2MjYzMSwiZXhwIjoxNzA1MjYyNjMxfQ.MvOLarBw_Mz-h26WvOrOqyE0IaDQEnIqp2xLUiFqAZQ; Max-Age=86400; Path=/; Expires=Tue, 20 Apr 2021 20:03:51 GMT; HttpOnly
Content-Type: text/html; charset=utf-8
Content-Length: 183
Date: Mon, 19 Apr 2021 20:03:51 GMT
Connection: keep-alive
(the cookie is set correctly at api.differentsite.com:3000 according to the Application tab in Chrome)
And then I try to make a request from somesite.com:
Response
HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Origin: http://somesite.com:8000
Vary: Origin
Access-Control-Allow-Credentials: true
Content-Type: application/json; charset=utf-8
Content-Length: 147
ETag: W/"93-2hkHY0W/lJkUQBK+JwrKp/OTCro"
Date: Mon, 19 Apr 2021 20:03:26 GMT
Connection: keep-alive
Request
POST /api HTTP/1.1
Host: api.differentsite.com:3000
Connection: keep-alive
Content-Length: 143
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.128 Safari/537.36
content-type: application/json
Accept: */*
Origin: http://somesite.com:8000
Referer: http://somesite.com:8000/
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
There's no cookie sent, so authorisation fails.
Why is this happening? This should work, no?
My request looks like this:
const headers = new Headers();
headers.append("Content-Type", "application/json");
const result = await fetch(endpoint!, {
method: "POST",
headers,
body,
redirect: "follow",
credentials: "include",
}).then((response) => response.json());
I've checked over and over again and I'm certain these settings are correct.
I've followed the advice on this question but they didn't work. I'm unsure what Access-Control-Allow-Headers I should add on my request.
Looks like a night's sleep was all I needed.
The problem was caused by my cookies missing the correct settings. For httpOnly cookies to be sent in cross-domain requests they need to have:
same-site set to none
secure set to true
So I made sure these two values were set on cookies when in production:
sameSite: process.env.NODE_ENV === 'production' ? 'none' : 'lax',
secure: process.env.NODE_ENV === 'production' ? true : false,
And it started working.
EDIT:
While this is the correct answer, it seems that Safari now blocks third party cookies by default, meaning no cookies will be sent if the client is safari. This can be deactivated in the settings, but it's on by default.

iCloud Calendar empty response

I have sent below request to iCloud calendar service to get all calendar list. But response is always empty. Can anyone help me?
Request:
GET /10232836851/calendars/F41F7478-4345-4A4A-8CD5-548122EF2C22/ HTTP/1.1
HOST: pxx-caldav.icloud.com
user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2
authorization: Basic XXXXXXXXXXXXXXXXXXXXXXXX==
depth: 1
content-type: text/xml
Response Header:
Server: AppleHttpServer/a6f3179
Date: Fri, 25 Nov 2016 07:35:53 GMT
Content-Length: 0
Connection: keep-alive
Last-Modified: Wed, 23 Nov 2016 19:32:45 GMT
Dav: 1, access-control, calendar-access, calendar-schedule, calendar-auto-schedule, calendar-managed-attachments, calendarserver-sharing, calendarserver-subscribed, calendarserver-home-sync
Accept-Ranges: bytes
X-Responding-Server: pv41p47ic-tydg09053001 23 a63660a6f7d1a25b5a7ed66dab0da843:44702101
X-Transaction-Id: 55b238a2-548b-41fc-acc0-b697adb4ab84
Cache-Control: private, max-age=0, no-cache
Strict-Transport-Security: max-age=31536000; includeSubDomains
Via: icloudedge:hk02p00ic-ztde010805:7401:16G8:Hong Kong
X-Apple-Request-Uuid: 55b238a2-548b-41fc-acc0-b697adb4ab84
Access-Control-Expose-Headers: X-Apple-Request-UUID; Via
To get the list of calendars on the server using the CalDAV protocol you use the PROPFIND method, not a GET. This is pretty well described in the SabreDAV Building a CalDAV Client web page.
Something like this:
PROPFIND /calendars/johndoe/ HTTP/1.1
Depth: 1
Content-Type: application/xml; charset=utf-8
Host: ...
Authorization: ...
<propfind xmlns="DAV:">
<prop>
<displayname />
</prop>
</propfind>

Trying to send a GET request, getting 301 to the same address

I'm trying to send an HTTP GET request in C++ using sockets, and I'm getting in response a 301 Moved permently, but to the same address I've asked for!
Here is my GET request :
GET /watch?v=1cQh1ccqu8M HTTP/1.1
Host: www.youtube.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Firefox/29.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language:en-US;q=0.6,en;q=0.4
Connection: keep-alive
All the \r\n are perfectly in place, because this GET request used to work for me not long ago, and I have not touched it since...
The response I'm getting from youtube :
HTTP/1.1 301 Moved Permanently
Date: Mon, 08 Dec 2014 11:04:10 GMT
Server: gwiseguy/2.0
Content-Type: text/html; charset=utf-8
Expires: Tue, 27 Apr 1971 19:44:06 EST
Location: https://www.youtube.com/watch?v=1cQh1ccqu8M
X-XSS-Protection: 1; mode=block; report=https://www.google.com/appserve/security-bugs/log/youtube
Cache-Control: no-cache
Content-Length: 0
P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/bin/answer.py?answer=151657&hl=en for more info."
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic,p=0.002
According to the page they tell me to check http://support.google.com/accounts/bin/answer.py?answer=151657&hl=en, it says I need to add some kind of cookies now?
I've allways send this request without sending any cookies, so I am a bit confused...
me: Was your original request really done with https and not plain http...
#Amit: No, I was connecting to `www.youtube.com', then I've sent the GET request
Then you should look more closely at the redirect:
Location: https://www.youtube.com/watch?v=1cQh1ccqu8M
As you can see, this does redirect you to the same host, same page, but different protocol: you must use https instead of http.

HTTP Set-cookie Headers not taken into account by browser

I have a web server that is serving files. Here is a request I make and the answer:
GET / HTTP/1.1
Host: 127.0.0.1:8004
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131030 Firefox/17.0 Iceweasel/17.0.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cookie: djdt=hide; sessionid=1ksbrjhfunsjtywlk2q4p2rl7zrsyvza; messages="Nice and all"
Cache-Control: max-age=0
HTTP/1.0 200 OK
Set-Cookie: csrftoken=Xkg4CnLEahplJPHEJGLrmGvPjebMqyh9; expires=Mon, 09-Feb-2015 21:15:51 GMT; Max-Age=31449600; Path=/, sessionid=qy5guifmpkra78ubkztxxlmk3tu5vr7s; expires=Mon, 24-Feb-2014 21:15:51 GMT; httponly; Max-Age=1209600; Path=/, messages=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/
Vary: Cookie
Server: gunicorn/18.0
Date: Mon, 10 Feb 2014 21:15:51 GMT
x-frame-options: SAMEORIGIN
Content-Type: text/html; charset=utf-8
Up to that point everything is fine. Things get weird on the very next request:
GET /static/css/bootstrap.css HTTP/1.1
Host: 127.0.0.1:8004
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20131030 Firefox/17.0 Iceweasel/17.0.10
Accept: text/css,*/*;q=0.1
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Referer: http://127.0.0.1:8004/
Cookie: djdt=hide; sessionid=1ksbrjhfunsjtywlk2q4p2rl7zrsyvza; messages="Nice and all"
If-Modified-Since: Sun, 13 Oct 2013 20:23:36 GMT
Cache-Control: max-age=0
HTTP/1.0 304 NOT MODIFIED
Date: Mon, 10 Feb 2014 21:15:54 GMT
Server: WSGIServer/0.1 Python/2.7.3
Content-Length: 0
Yeah, the browser (Iceweasel) just sent the previous cookies, instead of sending the new one. Actually, it does not take the set-cookie HTTP header into account. That I don't understand. Same behavior on Chromium. On both browsers, nothing appears in the console. When using the developers tools from Iceweasel to the see the request, the previous cookie is parsed in the 'sent cookie', but the received cookie isn't parsed as it should be, such as it is in this (otherwise unrelevant) capture:
You may have notice the Server HTTP response header is different when serving the static file. Indeed, the local Django app serves as a proxy when requesting pages, but serves static content himself. Both contents are on the same domain, same port, so this is supposed to be the same from the browser perspective.

Why am I getting 400 error from FB's Graph API?

I have a FB app, which uses FB.api() from its JS SDK to post a message on user's feed. The issue is, I get 400 error instead.
(Please note many data have been modified for privacy)
The request was sent to URL:-
https://graph.facebook.com/me/feed/?access_token=178486488887736|2.AQCDILuiyM-wBx8.3600.1313269200.1-1000|_z6EB7Ebp6Bxgf-ss
Request header:-
Host: graph.facebook.com
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
DNT: 1
Connection: keep-alive
Cookie: datr=IezG1St6azJsvDL; lu=wgWUTDTuiTLJXgPI8A; locale=en_US; app_id=178486488887736; s=Aa7YfcM3PxNBg; L=2; act=1313261419%2F1; presence=EM31p_5f1B01179550304F5122K0H0U0OQ0EsF0CEblFDacF0EutF0PCC; c_user=100001304; sct=1313128632; xs=60%3Aa1c1753e6a6abd0fe7b23bfe4; p=167; _e_zzrj_3=%5B%22zzrj%22%2C1313261446%2C%22act%222Fyourmemorablestatus%2F%22%2C%7B%22ft%22%3A%7B%7D%2C%22gt%22%3A%7B%22tti_ms%22%3A5464%2C%22app_id%22%3A178486488887736%2C%22is_early_flush%22%3Afalse%2C%22browser%22%3A%22Firefox%205%22%7D%7D%2C0%2C16%5D
The request used POST method. Post data was:-
Referer: http://connect.facebook.net/rsrc.php/v1/yK/r/RIxWozDt5Qq.swf
Content-type: application/x-www-form-urlencoded
Content-length: 1993
access_token=1784864.1-1000011Ebp6Bxgf-ssENEpia7iPE&message=My%20memorable%20status%20messages%20%3A-)%0A%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%EF%BF%BD%0A%0A%EF%BF%BD%20blog.applegrew.com%20and%20cink.applegrew.com%20both%20are%20now%20down.%20Moved%20them%20today%20to%20a%20new%20%22cloud%22%20based%20system%2C%20and%20now%20they%20are%20up%20in%20the%20air%20somewhere.%20Raised%20tickets%20against%20my%20service%20provider.%0A%0A%EF%BF%BD%20Check%20out%20the%20google%20home%20page%20today%2C%20and%20pluck%20the%20strings!%20%3A)%0A%0A%EF%BF%BD%20wow%20mallu%20girls%20r%20beautiful.%0A%0A%EF%BF%BD%20My%20Ctrl%2C%20C%2C%20V%20keys%20have%20started%20to%20wear%20out.%0A%0A%EF%BF%BD%20Developers%20are%20users%20too.%20People%20tend%20to%20forget%20that.%20%3A(%0A%0A%EF%BF%BD%20Once%20a%20wise%20one%20said%3A%20Bugs%20can%20neither%20be%20created%20nor%20destroyed%2C%20it%20has%20always%20existed%20and%20will%20come%20around%20in%20different%20manifestations.%0A%0A%EF%BF%BD%20unzip%2C%20strip%2C%20touch%2C%20finger%2C%20grep%2C%20mount%2C%20FSCK%2C%20more%2C%20yes%2C%20unmount%2C%20sleep%20.....%0A%0AThese%20are%20all%20Linux%20commands.%0A%0A%EF%BF%BD%20%EF%BF%BD%20Tonight%20going%20to%20Bryan%20Adam%20concert!%0A%0A&pretty=0&sdk=joey
Response header:-
Cache-Control: no-store
Content-Type: text/javascript; charset=UTF-8
Expires: Sat, 01 Jan 2000 00:00:00 GMT
P3P: CP="Facebook does not have a P3P policy. Learn why here: http://fb.me/p3p"
Pragma: no-cache
WWW-Authenticate: OAuth "Facebook Platform" "invalid_request" "(#1) An unknown error occurred"
X-FB-Rev: 422152
Set-Cookie: _e_zzrj_3=deleted; expires=Thu, 01-Jan-1970 00:00:01 GMT; path=/; domain=.facebook.com; httponly
X-FB-Server: 10.62.1.41
X-Cnection: close
Date: Sat, 13 Aug 2011 19:43:26 GMT
Content-Length: 78
Well I found the problem. In FB message can be max 420 characters long (including spce etc.). I wish FB responded with a more humane error.