I am trying to get the following URL with CFHTTP but I am not getting the page. Any ideas?
http://www.google.com/flights/#search;f=JNB;t=MRU;d=2016-12-19;sel=JNBMRU0MK854;s=0
<cfhttp url="https://www.google.com/flights/##search;f=JNB;t=MRU;d=2016-12-19;sel=JNBMRU0MK852;s=0" method="GET" resolveurl="true" useragent="Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.87 Safari/537.36">
<cfhttpparam type="header" name="HTTP_REFERER" value="http://example.com/feed/" >
<cfhttpparam type="header" name="Accept-Encoding" value="gzip,deflate,sdch" >
<cfhttpparam type="header" name="Proxy-Connection" value="keep-alive" >
<cfhttpparam type="header" name="Accept" value="application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5">
<cfhttpparam type="header" name="Accept-Language" value="en-US,en;q=0.8">
<cfhttpparam type="header" name="Accept-Charset" value="ISO-8859-1,utf-8;q=0.7,*;q=0.3">
<cfhttpparam type="cookie" name="some-cookie" value="1">
</cfhttp>
<cfoutput>#cfhttp.filecontent#</cfoutput>
Using http instead of https worked for me.
If you want to test it, use the method suggested by #bkbk.
<cfdump var="#cfhttp.filecontent#">
But do note that, outputing the filecontent will not work similar to the original site. This is because the site uses some security header like
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-xss-protection:1; mode=block
Which will not allow loading of few of the original site scripts and css.
It is very likely that the response is coming in as HTML, but your browser isn't displaying the tags. To display a result you can see, use
<cfoutput>#xmlformat(cfhttp.filecontent)#</cfoutput>
or
<cfdump var="#cfhttp#">
As an aside, what about just running
<cfhttp url="https://www.google.com/flights/##search;f=JNB;t=MRU;d=2016-12-19;sel=JNBMRU0MK852;s=0" method="GET" resolveurl="true" useragent="Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.87 Safari/537.36" />
Related
I am posting few inputs into a url using cfhttp and expecting to download some data in a xls file. I am trying to do get data using cffile ="write" which doesn't work. Can any one suggest how can we go with this. Here is the code below
<cfhttp url="#Baseurl#" method="post" result="ExportToExcelresult" redirect="no" resolveurl="true">
<cfhttpparam type="header" name="REFERER" value="#Baseurl#" >
<cfhttpparam type="header" name="Cache-Control" value="no-cache">
<cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded">
<cfhttpparam type="header" name="Connection" value="keep-alive" >
<cfhttpparam type="header" name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36">
<cfhttpparam type="header" name="cookie" value="TestCookie=;" encoded="yes">
<cfloop collection="#CookieList#" item="i">
<cfhttpparam type="header" name="cookie" value="#CookieList[i]#" encoded="yes">
</cfloop>
<cfloop collection="#PostCookieList#" item="i">
<cfhttpparam type="header" name="cookie" value="#PostCookieList[i]#" encoded="yes">
</cfloop>
<cfloop collection="#PostDefaultCookieList#" item="i">
<cfhttpparam type="header" name="cookie" value="#PostDefaultCookieList[i]#" encoded="yes">
</cfloop>
<cfhttpparam name="ToolkitScriptManager1_HiddenField" value="" type="formfield">
<cfhttpparam name="LeftNav1_LoginView1_treeView1_ExpandState" value="#EXPANDSTATE#" type="formfield">
<cfhttpparam name="LeftNav1_LoginView1_treeView1_SelectedNode" value="#SELECTNODE#" type="formfield">
<cfhttpparam name="__EVENTTARGET" value="" type="formfield">
<cfhttpparam name="__EVENTARGUMENT" value="" type="formfield">
<cfhttpparam name="LeftNav1_LoginView1_treeView1_PopulateLog" value="" type="formfield">
<cfhttpparam name="__VIEWSTATE" value="#VIEWSTATE#" type="formfield">
<cfhttpparam name="__VIEWSTATEGENERATOR" value="#VIEWSTATEGENERATOR#" type="formfield">
<cfhttpparam name="__EVENTVALIDATION" value="#EVENTVALIDATION#" type="formfield">
<cfhttpparam name="ctl00$MainContent$repMUData$ctl00$btnExport" value="Export to Excel" type="formfield">
</cfhttp>
When I do cfdump here is the the result
When I do cfdump that it is giving some binary data. For sure I am getting some data but not sure how to extract the data into xls file
To write the binary data to file just use the function filewrite. In the following snippet I write to a temporary file, but you would write to were you want to permanently store the file. I then read file back into spreadsheet object to verify that the write worked as intended.
<cfhttp url="http://example.com/test.cfm" result="ExportToExcelresult">
</cfhttp>
<cfscript>
//Replace with the file path where you want to permanently store the file
yourFileLocation = getTempFile(getTempDirectory() ,"xls");
//Save to file system
filewrite(yourFileLocation, ExportToExcelresult.filecontent.toByteArray());
//Not needed. Only verifying there is a spreadsheet written to the file location
writeOutput("Is spreadsheet: " & isSpreadsheetFile(yourFileLocation));
//You will not be working with a temp file. Do not delete it.
fileDelete(yourFileLocation);
</cfscript>
I'll keep this short. I have OAuth working with the Google Photos API. I am able to return albums and media items but not album specific media items.
Below I've done three requests. The first two work while the third does not.
Note: In the third request I've taken the albumID from the results returned in the first request.
<!--- This works --->
<cfhttp method="GET" charset="utf-8" url="https://photoslibrary.googleapis.com/v1/albums" result="res">
<cfhttpparam name="Authorization" type="header" value="OAuth #application.oauth2.GetAccess_token()#" />
</cfhttp>
<cfdump var="#deserializeJSON(res.filecontent)#">
<!--- This works --->
<cfhttp method="GET" charset="utf-8" url="https://photoslibrary.googleapis.com/v1/mediaItems" result="res">
<cfhttpparam name="Authorization" type="header" value="OAuth #application.oauth2.GetAccess_token()#" />
</cfhttp>
<cfdump var="#deserializeJSON(res.filecontent)#">
<!--- This doesn't. I get a 404 for this --->
<cfhttp method="POST" charset="utf-8" url="https://photoslibrary.googleapis.com/v1/mediaItems:search" result="res">
<cfhttpparam name="Authorization" type="header" value="OAuth #application.oauth2.GetAccess_token()#" />
<cfhttpparam name="albumId" type="formfield" value="AHeArAbj3EfLJqs23FHD9SWuReCaaPA0VBZxw6HNwL752bJr5X882_ZJnitk7kjZJwtzXvrw6hp2" />
</cfhttp>
<cfdump var="#res#">
This is the result I get for the last request;
What is wrong with the last request I am making? I am trying to use this API call: https://developers.google.com/photos/library/reference/rest/v1/mediaItems/search
I am passing few credentials along with __EVENTARGUMENT, __VIEWSTATE. But I was not able see the variables and data in the console or in the fiddler, Am I missing anything. I tried with url, formfield and body with no success. By the way I am using ColdFusion 9.
<cfset authenticationRequestBody = "__LASTFOCUS=#LASTFOCUS#&__EVENTTARGET=#EVENTTARGET#&__EVENTARGUMENT=#EVENTARGUMENT#&__VIEWSTATE=#EncodeViewState#&__VIEWSTATEGENERATOR=#EncodeViewGenerator#&__EVENTVALIDATION=#EncodeEventValidation#&#encodeForURL(UNameString)#=#UserName#&#encodeForURL(PwdString)#=#encodeForURL(Password)#&#encodeForURL(ButtonString)#=Submit">
<cfset stsUrl = "https://somesite.com/yyy/login.aspx" >
<cfhttp url="#stsUrl#" method="post" resolveurl="no" >
<cfhttpparam type="header" name="Accept" value="application/xhtml+xml,text/html">
<cfhttpparam type="header" name="REFERER" value="#BaseUrl#" >
<cfhttpparam type="header" name="Accept-Language" value="en-US">
<cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded">
<cfhttpparam type="header" name="Connection" value="keep-alive" >
<cfhttpparam type="header" name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36" >
<cfloop collection="#cookies#" item="i">
<cfhttpparam type = "cookie" name="#i#" value="#cookies[i]#">
</cfloop>
<cfhttpparam type="body" name="PostData" value="#authenticationRequestBody#">
<cfoutput>
<cfdump var="#GetHTTPRequestData()#">
</cfoutput>
This is Not a Problem related to the configuration Because I checked the JVM version and TLS version at the Site using SSL test server. There is something that I am missing here in the code..
Coldfusion 11 (Update 12)
JVM : 1.8
TLS : 1.2
I was ablbe to get till the Login Screen. Even after passing the Username and Password in the body, it doesn't validate. When I access the URL directly with the same credentials it logs me in successfully.
Try doing a
<cfdump var="#cfhttp#">
OR
<cfhttp url="#stsUrl#" method="post" resolveurl="no" result="result" >
...
</cfhttp>
<cfdump var="#result#">
Problem is not with the configuration or compatibility version.. The issue is with the cookies we are passing from the start.. When we navigate through a other page using cfhttp we need to carry the old cookies that we got from the past cfhttp calls.. Also in my case I need to initialize the cookie in the first call.. Below is the example for two calls..
<cfhttp url='#BaseUrl#' method="get" redirect="no">
<cfhttpparam type="header" name="Connection" value="keep-alive" >
<cfhttpparam type="header" name="Cache-Control" value="no-cache">
<cfhttpparam type="header" name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko">
<cfhttpparam type="header" name="cookie" value="TestCookie=;" >
</cfhttp>
<cfhttp url="#stsUrl#" method="post" redirect="no" resolveurl="yes" result="postResult" >
<cfhttpparam type="header" name="REFERER" value="#BaseUrl#" >
<cfhttpparam type="header" name="Cache-Control" value="no-cache">
<cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded">
<cfhttpparam type="header" name="Connection" value="keep-alive" >
<cfhttpparam type="header" name="User-Agent" value="Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36" >
<cfhttpparam type="header" name="cookie" value="TestCookie=;" encoded="yes">
<cfloop collection="#CookieList#" item="i">
<cfhttpparam type="header" name="cookie" value="#CookieList[i]#" encoded="yes">
</cfloop>
<cfhttpparam name="__LASTFOCUS" value="" type="formfield">
<cfhttpparam name="__EVENTTARGET" value="" type="formfield">
<cfhttpparam name="__EVENTARGUMENT" value="" type="formfield">
<cfhttpparam name="__VIEWSTATE" value="#VIEWSTATE#" type="formfield">
<cfhttpparam name="__VIEWSTATEGENERATOR" value="#VIEWSTATEGENERATOR#" type="formfield">
<cfhttpparam name="__EVENTVALIDATION" value="#EVENTVALIDATION#" type="formfield">
<cfhttpparam name="ctl00$MainContent$LoginCtrl$UserName" value="#UserName#" type="formfield">
<cfhttpparam name="ctl00$MainContent$LoginCtrl$Password" value="#Password#" type="formfield">
<cfhttpparam name="ctl00$MainContent$LoginCtrl$LoginButton" value="Submit" type="formfield">
</cfhttp>
I am trying to get temp_token for credly API's badge builder SDK. I am using ColdFusion as server side language. My code look like,
<cfhttp method=”post” result=”objGet”
url=”https://developers.credly.com/badge-builder/code” >
<cfhttpparam type=”formfield” name=”access_token” value=”my_access_token” />
</cfhttp>
As well as, I have tried
<cfhttp method=”post” result=”objGet”
url=”https://developers.credly.com/badge-builder/code” >
<cfhttpparam type="header" name="Authorization" value="access_token=(my_access_token)” />
</cfhttp>
<cfhttp method=”post” result=”objGet”
url=”https://developers.credly.com/badge-builder/code” >
<cfhttpparam type="header" name="access_token" value="(my_access_token)” />
</cfhttp>
<cfhttp method=”post” result=”objGet”
url=”https://developers.credly.com/badge-builder/code” >
<cfhttpparam type="header" name="access_token" value="my_access_token” />
</cfhttp>
All of them are giving me response as,
{
“success”:false,
”error”:”You must pass an access_token to retrieve a badge builder embed code”
}
Am I missing something?
Update:
I have checked their official wordpress plugin, they are using type as body. I have tried that too, but gave the same result.
I have the following CFHTTP function which should return 3 cookies (XSRT-TOKEN, hl, and EASFC-WEB-SESSION) but instead only returns a JSessionID cookie.
<cfhttp url="http://www.easports.com/uk/fifa/football-club/ultimate-team" method="GET" result="stage1">
<cfhttpparam type="header" name="Accept" value="text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" />
<cfhttpparam type="header" name="Accept-Encoding" value="gzip, deflate" />
<cfhttpparam type="header" name="Accept-Language" value="en-US, en;q=0.5" />
<cfhttpparam type="header" name="Connection" value="keep-alive" />
<cfhttpparam type="header" name="Host" value="www.easports.com" />
<cfhttpparam type="header" name="User-Agent" value="Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36" />
</cfhttp>
Yet when I make a request to the page using an XMLHttpRequest function and look at the repsonse it has returned the 3 cookies as expected. Also when I load the URL in a browser and have a look in Fiddler the page again is returning the expected 3 cookies.
Anyone have any ideas what might be going on at all?
Are you running Fiddler on the same server that is doing CFHTTP? On the actual web server?
Also - try using userAgent param of the CFHTTP - not sending it as a separate header.