Just to give you a brief intro on what am I trying to achieve.
I need to scrape a value for each month from a public website.
As an example, for a particular status in the month of May, 2014; the TOTAL value is 224,481 (at the bottom right part of the page). I need to capture this value and store in the database for each of the month.
The following URL will need to be run by changing the parameters for each month.
VEEC Website with params
To achieve this, I am trying to get the HTML using CFHTTP as follows and then using JSOUP will scrape the intended value from the returned HTML before storing into the database.
<cfset f_url = "https://www.veet.vic.gov.au/Public/PublicRegister/Search.aspx">
<cfhttp method="GET" url="#f_url#">
<cfhttpparam type="Header" name="Accept-Encoding" value="*">
<cfhttpparam type="Header" name="TE" value="deflate;q=0">
<cfhttpparam name="CreatedFrom" type="URL" value="#StartDate#">
<cfhttpparam name="CreatedTo" type="URL" value="#EndDate#">
<cfhttpparam name="Status" type="URL" value="PRP,PRV">
</cfhttp>
<cfdump var="#cfhttp#">
I'm getting the connection failure error when I try to run the code snippet.
Please advise if there is an alternate way of doing the same.
This has to be implemented just to gather data on the development side and not meant for production use.
Edit (Didn't want to Delete the question): I had to use the internet proxy to access the website in the CFHTTP tag to make it work. My bad for not checking it earlier. Hopefully, it might help someone in future. Thanks everyone.
Sounds like the classic secure cert thing.
CF has limited trust of secure certs. If the SSL cert at the destination you are calling is not one the CF likes, you will get a connection failure message. What you have to do is import the SSL cert in to the Java Keystore that CF is using.
Here is an old version of how to do it:
http://mkruger.cfwebtools.com/index.cfm?mode=entry&entry=8E44925A-B73D-E3AD-709D4E02FD6D4588
Related
I tried creating a web service in ColdFusion, for that I just created a component with a function inside it having access="remote"
<cfcomponent displayname="testPost" hint="testing.." output="yes">
<cfsetting enablecfoutputonly="true" showdebugoutput="true">
<cffunction access="remote" output="true" name="testPostReq" displayname="testPostReq" description="testing" returntype="any">
<cfset p=9>
<cfreturn p>
</cffunction>
</cfcomponent>
Now I wanted to call this webservice and I am trying to hit this in my browser, https://sampleapp.xyz.com/DirectoryToCfc/CfcName.cfc?wsdl but in browser I am getting "Unsupported Operation. Check application log for more details." when I checked application logs I saw this log there:
"Warning","ajp-bio-8013-exec-4","09/27/21","12:57:55","COMPONENTUTILS","To use Component Browser, enable RDS Service using Administrator. Note: RDS is intended for development use only."
How can I deal with this, any idea? Is it possible to do without enabling RDS?
Please let me know if you know something about this, I will be really grateful.
You can create a webservice for WSDL. Here I'm going to taken an some real time demo url http://www.dneonline.com/calculator.asmx?wsdl & Coldfusion 2018. Please follow the below steps.
Step 1 : Create webservice in CFAdmin - > Data & Service - > Webservices
Step 2 : Give the Webservice name & WSDL URL value. Refer my image here (TestService.png) I've created 'TestService' for the url http://www.dneonline.com/calculator.asmx?wsdl.
Step 3 : Once you added it successfully. You can able to see that in "Active Coldfusion WebService Options"
Step 4 : With the help of createObject method & webservice value you can call your all methods in your WSDL url. For example,
<cfset myObj = createObject('webservice','TestService')>
<cfdump var='#myObj#' abort="true">
Step 5 : You can see my below dump about demo webservice url Methods & Classes. Refer image dumpService.png
I hope this will help you to know about simple Webservice.
This morning our Amazon EC2 Windows server became nearly unresponsive, throwing 500 errors intermittently. In the CF log I see a bunch of these errors:
The request has exceeded the allowable time limit Tag: CFQUERY
After rebooting the server, everything is back to normal.
While I hunt down the root cause for this, is it possible to configure CF Admin to send an email when a certain Log File error begins to appear several times in a row, like this one? We have already configured Health Status Checks for outages on EC2, but they never triggered because the server was still running, just incredibly slowly. I would like to be the first responder to situations like this but didn't know because I wasn't actively monitoring the server when it occurred - a user had to alert me to the problem a couple of hours later, which is embarrassing for me.
Thanks in advance!
Don't know if this is best practice, but what I do is utilize the OnError function in the application.cfc for reporting of unhandled errors.
<cffunction name="onError" returnType="void" hint="Runs when an uncaught exception occurs in the application.">
<cfargument name="Exception" required=true/>
<cfargument name="EventName" type="String" required=true/>
<cfmail to="myemail#mydomain.edu" from="myemail#mydomain.edu" subject="Error Alert" type="html">
<cfdump var="#session#">
<cfdump var="#CGI#">
<cfdump var="#Exception#">
</cfmail>
</cffunction>
You could obviously filter down to the types of errors you want reported.
If you just want to check specifically in the function (where the time out occurred), then you should use:
<cffunction name="yourfunction">
<cftry>
...
<cfcatch>
<cfmail to="myemail#mydomain.edu" from="myemail#mydomain.edu" subject="Error:yourfunction">
<cfdump var="#session#">
<cfdump var="#CGI#">
<cfdump var="#cfcatch#">
</cfcatch>
</cftry>
<cffunction>
Query timeouts are not the only source of overwhelmed servers. You might want something more global.
What you can do on the CF Administrator app is to use Alert Configuration. Not only does this allow you to notify yourself if certain conditions are met, but you can also kill the problematic threads.
What we have done in addition to that, is to schedule a script on our development server to attempt an http request to the production server with a one second timeout. If the attempt fails, mail gets sent to people, pagers, and cell phones.
Is there a way to output the raw html of a CFHTTP call? I am trying to see how some of the header authentication information is coming across.
I am open to browser plugins or code updates whichever helps me see what is going on during the cfhttp call.
So for example:
<cfhttp method="get" url="https://test-ows01.mywebsite.com/criminal_api//1.0/service/requests" result="orderList">
<cfhttpparam type="HEADER" name="Authorization" value="Basic #ToBase64("bearer:4EC8B09D3F911764B1DCD3EFA38DFB31")#">
</cfhttp>
what does the above call look like when it happens.
If I am understanding correctly, it sounds more like you want to view the http request sent to the remote server, rather than what is received. Installing a tool like Fiddler will provide very robust debugging, allowing you to view http requests as they happen. (See also the documentation for Enable HTTPS traffic decryption).
Tip for quick debugging, a low-tech hack is to switch the target URL to a separate .cfm script on your server. Inside the script, dump GetHTTPRequestData(), to display the request headers and body sent to the script.
test.cfm
<cfhttp method="get" url="http://localhost/receivingPage.cfm" result="orderList">
<cfhttpparam type="HEADER" name="Authorization"
value="Basic #ToBase64("bearer:4EC8B09D3F911764B1DCD3EFA38DFB31")#">
</cfhttp>
receivingPage.cfm
<cfdump var="#GetHTTPRequestData()#">
You can use requestcatcher.com for this.
It let you make a personal subdomain and then you can send your request to that URL. Very handy. Helped me alot for a complex SOAP integration.
I'm looking to make heavy consumption of a RESTful API from a ColdFusion application.
I'm not a CF expert but I'm anticipating the repeated cfhttp calls will become a bottleneck as I believe that each results in a connection being established, request sent, response received and connection torn down.
I'm curious - is there a way to maintain a connection pool that requests could be sent through to avoid the repeated establish/tear down?
Does the ColdFusion server provide such a facility that I just don't know about (we're using CF 8) or can I write a java custom tag that could maintain the pool?
Certainly someone else has encountered this.
Unfortunately I think the answer is, "no", specifically because of your requirements. That's just not how REST works; and the limitation is the API side, not a ColdFusion issue.
You could do something similar, assuming you had control over the API end of things too, but it wouldn't be REST.
I think you might actually be able to do this by using the "Keep-Alive" request header with your cfhttp calls. For example:
<cfloop from="1" to="50" index="i">
<cfhttp url="http://mysite.com/getPage.cfm?i=#i#" method="get">
<cfif i LT 50>
<CFHTTPPARAM type="HEADER" name="Connection" value="Keep-Alive">
<cfelse>
<CFHTTPPARAM type="HEADER" name="Connection" value="close">
</cfif>
</cfhttp>
<cfdump var="#cfhttp.filecontent#">
</cfloop>
I haven't tested this, but in theory it should keep the connection to the back end open while you make each of those requests (assuming the backend allows for this and the delay between connections hasn't triggered a time-out). You should be sure that your API response includes a "Content-length" header so that the client (your cfhttp code) will know when each request has completed. You will want to issue an explicit "close" as I've shown to prevent unneeded open connections to the backend.
On CF 9.01, Windows Web Server 2008 R2, a web service has stopped running here, so I'm trying to figure out the problem. I think the problem may have started when I applied the 9.01 hotfix.
I added a test.cfc to the server
<cfcomponent output="false">
<cffunction
name = "echoString"
output = "no"
access = "remote" returntype="any">
<cfargument required="true" name="inputString" type="string"/>
<cfreturn arguments.inputString>
</cffunction>
</cfcomponent>
and call it with
<cfinvoke webservice="https://nww.somedomain.nhs.uk/cfcs/test.cfc?wsdl"
method="echoString" returnvariable="returnedString">
<cfinvokeargument name="inputString" value="test string 2">
</cfinvoke>
and get the string response, but in the cf administrator the web service is not being added
when I invoke the web service for the first time as I think it should be.
Other than reinstalling CF or trying to roll back manually, any clues as to the problem or how I can test please?
I'm on 9,0,1,274733 . The https is not the problem I think. Firstly it was working fine then with no change to code it stopped. Secondly my test invocation of test.cfc works as expected over https.
If I hit https://nww.somedomain.nhs.uk/cfcs/test.cfc?wsdl (which works) or my problem cfc https://nww.somedomain.nhs.uk/cfcs/providerapi.cfc?wsdl directly in the browser they both correctly show the XML details.
However, invoking either of the CFCs does not add them to the active web services list in administrator.
I pasted the "echostring" method to my problem CFC (providerapi.cfc) and tried calling the method but I get a "webservice cannot be found" error. It's like the providerapi.cfc is cached , but I can't see it in administrator to delete it.
I tried this verbatim on 9,0,1,274733 and it worked 100% fine. I am using Win 7 and I just tried it on my dev environment. The web service was added to the list in CF admin so you are correct to expect that.
Can you ensure that you can hit the URL direct (i.e. https://nww.somedomain.nhs.uk/cfcs/test.cfc?wsdl) in your browser and view source to see the WSDL XML?
One thing that jumps out is the use of https:// in your web service invocation. Can you try it with just http:// or even better against your localhost?