I have a URL which when run in the browser, displays JSON data, since I am new to coldfusion, I am wondering, what would be a good way to
grab the data from the web browser? Later on I will be storing the individial JSON data into MySQL database, but I need to figure out step 1
which is grabbing the data.
Please advise.
Thanks
You'll want to do a cfhttp request to load the external content.
Then you can use deserializeJSON to convert the JSON object into the appropriate cfml struct.
See the example Adobe gives in the deserializeJSON documentation.
Here is quick example:
<!--- Set the URL address. --->
<cfset urlAddress="http://ip.jsontest.com/">
<!--- Generate http request from cf --->
<cfhttp url="#urlAddress#" method="GET" resolveurl="Yes" throwOnError="Yes"/>
<!--- handle the response from the server --->
<cfoutput>
This is just a string:<br />
#CFHTTP.FileContent#<br />
</cfoutput>
<cfset cfData=DeserializeJSON(CFHTTP.FileContent)>
This is object:<br />
<cfdump var="#cfData#">
Now you can do something like this:<br />
<cfoutput>#cfData.ip#</cfoutput>
Execute this source here http://cflive.net/
Related
I am working on a old server which is running CF9 and is still a part of the development of the company. I have to make a <cfhttp> call to get a response from a website. If I run it in a browser, I get the expected response in XML. However, when I make a <cfhttp> call, it just throws an error.
Code:
<cfset objSecurity = createObject("java", "java.security.Security") />
<cfset storeProvider = objSecurity.getProvider("JsafeJCE") />
<cfset objSecurity.removeProvider("JsafeJCE") />
<cfhttp url="https://apiwebsite/api/integration/v1/events/a200363c-6b8f-4876-8cd0-105d41bb2fdf?oauth_version=1.0&oauth_nonce=73aa7a7d2bfdff27fd04e3967e313d5d&oauth_timestamp=1495298341&oauth_consumer_key=consumer_key&oauth_signature_method=HMAC-SHA1&oauth_signature=btkwYPlWGuquUOpwUNPFcLA17q8%3D"/>
<cfset objSecurity.insertProviderAt(storeProvider, 1) />
<cfdump var="#cfhttp#">
Error from cfdump:
ErrorDetail I/O Exception: peer not authenticated
Filecontent Connection Failure
Header [empty string]
Mimetype Unable to determine MIME type of file.
It is so frustrating as I do not know how to fix it. I am open to looking around at custom tags if needed.
I am using cfhttp (Lucee Server) to scrape page contents from a url in the following manner:
<cfhttp url="#libs.originalAdPage#" method="GET" />
I then place this content in a div on my page.
This code has been working for a long time.
I have a need to report on the url's that have been scraped for their content and that information is placed into another website form that is not in my control. I decided to convert the url's to shortened bitly url's. I built the process into the page to create a bitly link and return that url to replace the existing url.
If i use the page with a shortened url from linkedin the page is scraped and displayed correctly in the div.
<cfhttp url="http://bit.ly/1NPhPgc" method="GET" />
But if I do an identical cfhttp call to a Indeed.com page shorted to a bitly URL I get a connection failure error.
<cfhttp url="http://bit.ly/1RQvlim" method="GET" />[![cfdump of connection failure][1]][1]
If I open this URL directly in the browser the page is displayed correctly.
Any ideas would be greatly appreciated.
Thanks,
Michael
I don't have access to a Lucee server to test with, however cfhttp on a ColdFusion server works fine for me for both of those bitly URLs. cfhttp follows the redirect and the FileContent contains the indeed.com page as would be expected.
Have you verified what happens with the Bitly Indeed URL if you prevent cfhttp from automatically following the redirects so that you can debug and follow the redirects manually? ie
<cftry>
<cfhttp url="http://bit.ly/1RQvlim" method="GET" redirect="no" />
<cfdump var="#cfhttp.responseHeader#" />
<cfhttp url="#cfhttp.responseHeader.Location#" method="GET" />
<cfdump var="#cfhttp#" label="cfhttp2" />
<cfcatch>
<cfdump var="#cfcatch#" label="cfcatch" />
</cfcatch>
</cftry>
Indeed.com do pay attention to crawlers and user agents - just see their robots.txt for evidence of this.
Do you have access to a different server to test with in case there is something specific to Lucee's cfhttp implementation or to your IP address (eg blacklisted due to all the scraping)?
Have you tried tweaking the cfhttp useragent and/or any other headers as per How to emulate a real http request via cfhttp?
I am trying to run a webservice using coldfusion. I can run the wsdl in the browser fine. When I try to run it via coldfusion I get:
Unable to parse WSDL as an XML document.
Parsing error: Fatal Error: URI=null Line=-1: Premature end of file.
It is recommended that you use a web browser to retrieve and examine the requested WSDL document to ensure it is correct.
I have tried multiple methods:
wsargs.login='******';
wsargs.password='******';
ws = CreateObject("webservice", "https://correcturl.com/dswsbobje/qaawsservices/biws?WSDL=1&cuid-******", wsargs);
req = getSOAPRequest(ws);
</cfscript>
<cfdump var="#req#">
<cfset wsargs = structNew()>
<cfset wsargs["login"]="******">
<cfset wsargs["password"]="******">
<cfinvoke webservice="https://correcturl.com/dswsbobje/qaawsservices/biws?WSDL=1&cuid=******"
method="runQueryAsAService"
returnvariable="results"
argumentCollection="#wsargs#">
</cfinvoke>
<cfinvoke webservice="https://correcturl.com/dswsbobje/qaawsservices/biws?WSDL=1&cuid=******"
method="runQueryAsAService"
returnvariable="results">
<cfinvokeargument name="login" value="******"/>
<cfinvokeargument name="password" value="******"/>
</cfinvoke>
But all give me this error. I have see other related errors and have tried the solutions in them, such as clearing out the Application.cfc/cfm and adding refreshwsdl='true' to the cfinvoke, none of which have done anything. Can anyone help me with this?
Thanks.
I guess I didnt have a full understanding of how this works. The url I was trying to use was to what I guess was the wsdl definition. I ran url an via wizdler ran the method. That then gave me a soap request that I then saved in a cfcsave content tag. My final code that worked looks like:
<cfset strURL = "https://correcturl.com/dswsbobje/qaawsservices/biws?WSDL=1&cuid=******">
<cfsavecontent variable="strXML">
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
<Header>
<QaaWSHeader xmlns="VendorInfo">
<sessionID>[string?]</sessionID>
<serializedSession>[string?]</serializedSession>
<ClientType>[string?]</ClientType>
<AuditingObjectID>[string?]</AuditingObjectID>
<AuditingObjectName>[string?]</AuditingObjectName>
</QaaWSHeader>
</Header>
<Body>
<runQueryAsAService xmlns="VendorInfoLR">
<login>******</login>
<password>******</password>
</runQueryAsAService>
</Body>
</Envelope>
</cfsavecontent>
<cfhttp url="#strURL#" method="post" useragent="#CGI.http_user_agent#" result="objGet">
<cfhttpparam type="XML" value="#strXML.Trim()#" />
</cfhttp>
Idea from :http://www.experts-exchange.com/Software/Server_Software/Web_Servers/Q_24311762.html
This soap stuff is new to me, and I have more research to do to fully understand it. :)
I'm running Coldfusion8 and am uploading files to Amazon S3.
When displaying images, I want to check whether an image is available from S3 and if not show a fallback image. My problem is, don't know how to check for existing images.
If I list the link to an image, it's something like this:
http://s3.amazonaws.com/bucket/l_138a.jpg?AWSAccessKeyId=_key_&Expires=_exp_&Signature=_signature_
I'm trying to check for existing files like this:
<cfif fileExists("http://s3.amazonaws.com/bucket/s_" & items.filename)>
<cfdump output="e:\website\test\dump.txt" label="catch" var="found!!!">
</cfif>
Question:
Do I always have to provide accesskey, expires and signature when checking for an image? If I enter the image path without credentials in the browser, the image is loaded, so I don't understand why my fileExist is not working. Any idea?
You could use cfhttp if you have a site-wide page not found message set up.
<cfhttp url="http://a.espncdn.com/photo/2012/0813/nfl_u_flynn1x_203.jpg" method="head">
<cfdump var="#cfhttp.filecontent#">
returns object of java.io.ByteArrayOutputStream
<cfhttp url="http://a.espncdn.com/photo/20notanimage3.jpg" method="head">
<cfdump var="#cfhttp.filecontent#">
returns <html> <body> <h1>Error Processing Request</h1> </body> </html>
Can also check the statuscode returned by the server
<cfhttp url="http://a.file.exists.gif" method="head">
<cfdump var="#val(cfhttp.statuscode)#">
200 is ok, 404 is not found, etc
I've used the getObjectInfo method in the S3.cfc to see if an object exists:
<cffunction name="getObjectInfo" access="public" output="false" returntype="string"
description="Creates a bucket.">
<cfargument name="bucketName" type="string" required="yes">
<cfargument name="filekey" type="string" required="true" hint="" />
<cfset var data = "">
<cfset var content = "">
<cfset var contents = "">
<cfset var thisContent = "">
<cfset var allContents = "">
<cfset var dateTimeString = GetHTTPTimeString(Now())>
<!--- Create a canonical string to send --->
<cfset var cs = "HEAD\n\n\n#dateTimeString#\n/#arguments.bucketName#/#Arguments.filekey#">
<!--- Create a proper signature --->
<cfset var signature = createSignature(cs)>
<!--- get the bucket via REST --->
<cfhttp method="HEAD" url="http://s3.amazonaws.com/#arguments.bucketName#/#Arguments.filekey#">
<cfhttpparam type="header" name="Date" value="#dateTimeString#">
<cfhttpparam type="header" name="Authorization" value="AWS #variables.accessKeyId#:#signature#">
</cfhttp>
<cfreturn cfhttp.StatusCode />
</cffunction>
If I get a 200 status back, then I know the object exists.
I haven't used Coldfusion for a long time, but I did a quick lookup and the fileExists method seems to be for filesystem lookups, not remote URLs.
There are other Coldfusion methods for requesting URLs. One forum discussion on the subject I just quickly found is here: http://forums.adobe.com/thread/765614
But, assuming you're generating HTML to be consumed by a web browser I would suggest doing an image check / fallback in HTML/CSS/JS rather than server side. You could do this with CSS background-image tricks, or directly load and check images with JS. One question dealing with this that I found is here (there are probably a bunch of similar questions on this stuff): Inputting a default image in case the src attribute of an html <img> is not valid?
CF9 +
<cfscript>
FileExists('s3://#accessKey#:#secretKey##[your bucket]/[your file]');
</cfscript>
Does anyone have an example of how Coldfusion's GetHttpRequestData() works? I'm looking to use this func to save data from the AJAX Upload script: http://valums.com/ajax-upload/
The script works in FireFox but not Safari, Chrome, etc...
Ideas?
What error do you get?
Maybe these links will help:
http://www.coldfusionjedi.com/index.cfm/2007/7/1/Undocumented-change-to-GetHTTPRequestData-in-ColdFusion-8
http://www.bennadel.com/blog/1602-GetHTTPRequestData-Breaks-The-SOAP-Request-Response-Cycle-In-ColdFusion.htm
You might also want to read this recent thread about that script. As valums suggested, you should be able to extract the binary data from getHttpRequestData().content (when needed).
In my very limited tests, it seemed to work okay with IE8/FF/Chrome/Opera. However, I had no luck with Safari (windows). It seemed like the request data was getting mangled (or possibly misinterpreted by CF?). So the final content-type header reported by CF was incorrect, causing an http 500 error. Granted, I did not test this extensively.
Here is my quick and dirty test script (lame by design...)
<cfset uploadError = "" />
<cfif structKeyExists(FORM, "qqFile")>
<!--- upload as normal --->
<cffile action="upload" filefield="qqFile" destination="c:/temp" />
<cfelseif structKeyExists(URL, "qqFile")>
<!--- save raw content. DON'T do this on a prod server! --->
<!--- add security checks, etc... --->
<cfset FileWrite( "c:/temp/"& url.qqFile, getHttpRequestData().content) />
<cfelse>
<!--- something is missing ...--->
<cfset uploadError = "no file detected" />
</cfif>
<!--- return status old fashioned way (for compatibility) --->
<cfif not len(uploadError)>
{"success": true}
<cfelse>
<cfoutput>{error": "#uploadError#"}</cfoutput>
</cfif>
You want to look into using cffile with action="upload" to upload the file: http://cfdocs.org/cffile
GetHttpRequestData() is intended for decoding protocols like SOAP, XML-RPC, and some of the more complex REST-ful protocols. HTTP file uploads are normally done as POSTs using the multipart/form-data MIME type. Looking at http://www.cfquickdocs.com/it doesn't appear that GetHttpRequestData() has any special support for multipart data, which means you'd have to split and decode the parts yourself. Not my idea of fun, and completely unnecessary if you're just doing file uploading.
<cffile action="upload"> or <cffile action="uploadAll"> (new for CF9) should be quite sufficient for processing file uploads, even for those done via an AJAX upload script.