When I click the search button on this page, it sends a post request. I want to do the post via cli-http. How can I do that?
(def default-http-opts
{:socket-timeout 10000
:conn-timeout 10000
:insecure? true
:throw-entire-message? false})
(clj-http/post initial-url default-http-opts)
can post a request but the problem is that I want to pass in some parameters. These parameters(the buttons selected) are default on the page.
They are:
AdvancedSearchForm:CourseOrSubjectSelection=ALL_ALL
AdvancedSearchForm:GraduateLevelSelection=ALL
AdvancedSearchForm:allStudyAreas=t
AdvancedSearchForm:departmentList=
AdvancedSearchForm:facultyList=
AdvancedSearchForm:keywords=
AdvancedSearchForm:level=ALL
AdvancedSearchForm:semester=ALL
oracle.adf.faces.FORM=AdvancedSearchForm
oracle.adf.faces.STATE_TOKEN=_id21519:_id21520
source=AdvancedSearchForm:searchButton
The key AdvancedSearchForm:semester contains ':', so I use string as a key like this "AdvancedSearchForm:semester", is it OK in clj-http?
I do it like this:
(spit (file "/tmp" "ts.html")
(:body (http/post initial-url
{:form-params {"AdvancedSearchForm:CourseOrSubjectSelection" "ALL_ALL", "AdvancedSearchForm:GraduateLevelSelection" "ALL"}})))`
Actually the page it returns is indeed "Results" but no courses are listed. only the template. I want to get all the course links which are only shown by manually click. Any help?
is the image I screenshot from Tamper Data. It shows what happens after I click the Search button. Seems like client is redirected to searchresult.jsp. I use curl to imitate that. I do it like this
curl -D "form data..." https://handbook.unimelb.edu.au/faces/htdocs/user/search/AdvancedSearch.jsp
Then quickly run
curl https://handbook.unimelb.edu.au/faces/htdocs/user/search/SearchResults.jsp
No results contents are shown though the page is downloaded.
It looks like the server doesn't understand the parameters you send to it.
The escaping in use is the percent-encoding. Try to check out if it get in use by using the debug functionality avail in clj-https README.md:
;; print request info to *out*, including request body:
(client/post "http://example.org" {:debug true :debug-body true :body "..."})
or try to manually run the requests either with the curl command in a terminal or with the convenient Firefox restclient add-on.
From their GitHub page (https://github.com/dakrone/clj-http):
;; Send form params as a urlencoded body (POST or PUT)
(client/post "http//site.com" {:form-params {:foo "bar"}})
Related
I have 2 requests
1st Request
After did my first request, I get the response where I can parse for a taskId
In my test tab, I will then parse and store it like this
let taskId = pm.response.json().body.result[0].data.task
console.log(taskId)
I can see taskId printing in my console as 938
2nd Request
I require making a GET with this dynamic URL with the taskId that I got from the first one
http://localhost:3000/fortinet/monitor/{{taskId}}
So I set the above URL , set the HTTP verb to GET
in my Pre-request Script tab, I did this
let taskId = pm.globals.get("taskId")
Result
ReferenceError: taskId is not defined
Image Result
How can I debug this further?
The most suggested way is to use :key as in
http://localhost:3000/fortinet/monitor/:taskId
See the colon before taskId. The reason being, URI values sometimes many not be environment dependent. So, based on the usecase, you can use like I said or {{taskId}}
You have to set variable, but you are doing it wrong.
try this:
pm.globals.set("taskID", pm.response.json().body.result[0].data.task)
more you can read here:
https://learning.postman.com/docs/postman/variables-and-environments/variables/
Please note, that URL which ends with resource identified like https://example.com/:pathVariable.xml or https://example.com/:pathVariable.json will not work.
You can go with https://example.com/:pathVariable with Accept: application/json header.
For passing dynamic value, first you have to set it in environment or global variable in Tests tab because tests runs after request and you will get response value after request sent, but because you get response in json you have to first parse it, so what you can write in Tests tab is as follows:
var jsonData = JSON.parse(responseBody);
postman.setEnvironmentVariable("taskId", jsonData.token); // OR
postman.setGlobalVariable("taskId", jsonData.token);
Then you can use taskId as {{taskId}} wherever you want in url parameters or in request body or form data wherever.
If you want to know in detail how to extract data from response and chain it to request then you can go to this postman's official blog post which is written by Abhinav Asthana CEO and Co Founder of Postman Company.
I am writing a web app using Ring and Compojure, and Friend library for authorization. Recently I've made some changes to the project, and now I am not able to fetch request parameters any more.
What I used to do is this:
; Routes definition
(defroutes app-routes
(POST "/*.do" request
(insert-data (:params request))))
; Middlewares
(def secured-routes
(handler/site
(-> app-routes
wrap-params
(friend/authenticate friend-params-map))))
and form/URL parameters would be parsed into a Clojure map. Right now this does not seem to work, and (:params request) contains maps of the form
{:* <request-url>}
with a single :* key. If I try to (println request), a get a Clojure map with lots of key-value pairs, among which there are
:body #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x6ef9a9e1 HttpInputOverHTTP#6ef9a9e1]
that seem to contain request data, correct? But how do I fetch those?
Previously I was just using wrap-params middleware as described above, and it worked.
One approach that works is invoking (body-string request) in each handler, that will return a stringified version of request body. But it would be great to get an automatically parsed Clojure map for each url handler. Obviously, I'm missing something.
Can someone please advise? I would also be happy to find out more how to debug those kinds of handler issues myself - meaning, debug middleware chains and find out why wrap-params does not get invoked or is invoked improperly.
Thanks!
UPDATE: per #nberger's comment, i've tried changing secured-routes definition to this:
(def secured-routes
(-> app-routes
(wrap-defaults site-defaults)
(friend/authenticate friend-params-map)))
but now the login page does not work at all (authentication stopped working). After moving wrap-defaults after friend/authenticate as suggested in the comment, Friend complains in big font about Invalid anti-forgery token (???)
I've found the issue. In the front-end code, POSTs to *.do have been modified to send application/json content-type. Below is the jQuery's ajax call that was used:
$.ajax({
url: '/saveInput.do',
type: 'POST',
contentType: 'application/json; charset=UTF-8',
data: JSON.stringify(formValues),
});
I've modified it to
$.ajax({
url: '/saveInput.do',
type: 'POST',
data: formValues,
});
and it works now. Moreover, original Ajax call with JSON content-type can be preserved by including wrap-json-params middleware from ring-json, so that routes definition finally becomes:
(def secured-routes
(-> app-routes
(friend/authenticate friend-params-map)
(wrap-defaults (assoc-in site-defaults [:security :anti-forgery] false))
wrap-json-params))
I've opted for the latter.
#nberger - thank you for help!
I'd like to get OpenID connect working in my little luminus project. I'm a little new to the workflow in luminus/ring/compojure (coming from django, flask, and servlets mostly). I've successfully redirected to Google so I get the "code" back from Google, but then I need to make one more request to Google before logging in the user and this call requires another callback the user is not involved in, so I need to put the user's request on hold like a promise, but I'm not sure how that part works in compojure.
; this is my code that redirects them to Google, where they accept
(defn login [params]
(let [google-oauth2-client-id (System/getenv "GOOGLE_OAUTH2_CLIENT_ID")
base-url "https://accounts.google.com/o/oauth2/auth"
args {"client_id" google-oauth2-client-id
"response_type" "code"
"scope" "openid email"
"redirect_uri" "http://localhost:3000/oauth2Callback"
"state" "anti-forgery here"}]
(assert google-oauth2-client-id "can't find GOOGLE_OAUTH2_CLIENT_ID in environment")
(redirect (str base-url "?" (make-query-string args)))
)
)
; this is my code handling Google's first response
(defn oauth2-callback [params]
; params has the code to send to Google
; here I should send another request to google that comes back to another callback like oauth2-token-callback that processes the request to the user in the current context
(redirect "/youreloggedin")
)
By the end of this method I should be sending the user a message saying they're logged in, but I need to wait until the request comes back. How is this workflow handled in luminus?
Solved. I didn't realize I could just ignore the callback parameter.
(client/post "https://www.googleapis.com/oauth2/v3/token"
{:headers {"X-Api-Version" "2"}
:content-type :application/x-www-form-urlencoded
:form-params {:code (params :code)
:client_id (System/getenv "GOOGLE_OAUTH2_CLIENT_ID")
:client_secret (System/getenv "GOOGLE_OAUTH2_CLIENT_SECRET")
:redirect_uri "http://localhost:3000/oauth2Callback" ; ignored
:grant_type "authorization_code"
}
:as :auto ; decode the body straight to hash (if possible)
})
Based on the documentation for Google's OAuth2 for web servers here, the flow consists of the following steps:
Your application redirects a browser to a Google URL; the URL includes query parameters that indicate the type of access being requested.
The result is an authorization code, which Google returns to your application in a query string.
After receiving the authorization code, your application can exchange the code (along with a client ID and client secret) for an access token and, in some cases, a refresh token.
If I understood your question correctly, step 3 does not necessarily involve a callback to your server, you can just perform the request to Google with an HTTP client. I recently implemented OAuth2 for GitHub in this project, step 3 is implemented in this function:
(defn callback
"Handles the callback from GitHub OAuth flow."
[code]
(let [params {:form-params {:client_id client-id
:client_secret client-secret
:code code}}
{:keys [body]} (client/post access-token-url params) ;; This is doing the POST
;; request to GitHub.
token ((qs-map body) "access_token")] ;; Getting the token from
;; the response here.
{:status 302
:headers {"location" "/repos"
"set-cookie" (str "token=" token ";Path=/")}}))
I used clj-http as the HTTP client but any other will do.
As of today, almost all of my app's calls to share links on user's feeds are failing with the following error:
{
"message": "(#1500) The url you supplied is invalid",
"type": "OAuthException",
"code": 1500
}
Uisng PHP cURL, I'm posting to https://graph.facebook.com/{user_id}/feed and submitting a link parameter pointing to a valid, working URL (plus message params) etc.
Strangely, I can issue a command line cURL request and the request seems to work correctly (at least I haven't had an error yet).
I don't want to file a bug report yet incase I've missed something in a breaking migration. Any ideas as to what may be causing this?
http://developers.facebook.com/bugs/476666205677592
I have this problem too. it happened randomly. I'm sure the url I supplied is valid and can not reproduce it. So reported bug here..
Erro While FB Publish: {contents = "(#1500) The url you supplied is invalid"}
Solution:
1. Go to: https://www.facebook.com/
2. Click on Setting > manage app
3. Select "Edit App" button
4. Select "Permission" under the "Setting" panel at left side
5. Make Auth Token Parameter: To "URI Fragment(#access_token=…)"
6. Click "Save Changes" button.
Its fixed now!!!
The solution for me was:
1) I uploaded photo to my facebook user account
$fb->setFileUploadSupport(true);
$fb->setAccessToken('access token of my user (just to post an image)');
var_dump($fb->api('/me/photos', 'POST', ['image' =>'#F:\\fb\\fb_2.jpg','msg' =>'sss']));
2) After that set access for this picture to "For everyone" on my facebook page.
3) Then took var_dumped id to url https://www.facebook.com/photo.php?fbid={var_dumped id}
4) For every user, authorized my app, the following code started work normally $fb->api('/' . $fbuserid . '/feed', 'POST', [ 'link' => 'https://www.facebook.com/photo.php?fbid={var_dumped id}', 'message' => 'my post', 'type'=>'photo']);
I've just seen this library https://github.com/technomancy/clojure-http-client with this snippet of code on the README page which is what I'm looking to do
(res/with-cookies {}
(res/post "http://localhost:3000/login" {} {"user" user "password" password})
(res/get "http://localhost:3000/my-secret-page))
However it appears that the lib is deprecated and it advises you to use the clj-http library instead. I'm just wondering if anyone knows how to replicate this sort of behaviour using that library?
At the moment I just do
(post "<site i want to login to>" {:form-params {:username "<my username>" :password "<my password>"}})
Which returns a cookie that has a http 302 redirect to the authenticated page, but I have no idea how to make the client follow this redirect using the authenticated cookie
Any help would be appreciated.
FYI I resolved this,
(defn login [login-url user pass]
(let [result (client/post "http://my-site.com/login" {:form-params {:username user :password pass}})]
(when (= (:status result) 302)
(:cookies result))))
Should login be successful it will return a cookie map, this can then be used in subsequent requests when visiting pages that require you to be logged in, e.g.
(when-let [cookies (login "http://my-site.com" "my-user" "my-pass")]
(client/get "http://my-site.com/user-page" { :cookies cookies }))
=> <html><head><title>Hello my-user!</titl.......
I guess you need to explicitly use follow-redirect function from the library.