I'm calling facebook authentication API
https://www.facebook.com/dialog/oauth?client_id=3768637750&redirect_uri=http://localhost:8080/login.html?loginType=facebook
I get back following response.
=">http://localhost:8080/login.html?loginType=facebook&code=AQA84TrJjJNQaib2qvhGSdrPIIwJoIlfs9ZABjhBO6H9vt2wSZuRedigWjgV8SMg7QafCq-0xgbvi5k1e-RtFNA1pbrcfSsIqBL__-YWWhVFFepTGUuJeXWPW8Z3orRl-fWJUqb2mnmAJ995VFVX3O9N4iDj_3mhgQLC0DVwegprezqV6fU1tElMpH5Gj0#=
I'm using Spring3.0, and when is use some method like getCode() to read query parameter in Controller, I get truncated value for query paramter "code"
instead of getting,
AQA84TrJjJNQaib2qvhGSdrPIIwJoIlfs9ZABjhBO6H9vt2wSZuRedigWjgV8SMg7QafCq-0xgbvi5k1e-RtFNA1pbrcfSsIqBL__-YWWhVFFepTGUuJeXWPW8Z3orRl-fWJUqb2mnmAJ995VFVX3O9N4iDj_3mhgQLC0DVwegprezqV6fU1tElMpH5Gj0#=
I get
AQA84TrJjJNQaib2qvhGSdrPIIwJoIlfs9ZABjhBO6H9vt2wSZuRedigWjgV8SMg7QafCq-0xgbvi5k1e-RtFNA1pbrcfSsIqBL__-YWWhVFFepTGUuJeXWPW8Z3orRl-fWJUqb2mnmAJ995VFVX3O9N4iDj_3mhgQLC0DVwegprezqV6fU1tElMpH5Gj0
I am using UTF-8 encoding, any help on this please?
I am using UTF-8 encoding
That doesn’t matter, since this is not a character encoding issue.
A hash # in an URL marks the start of the “fragment identifier”, and that does not get passed to the server.
I get truncated value for query paramter "code"
There is no truncation, because the value of the code parameter ends before the #.
Related
I'm trying to read an email-id value using request.POST.get(). If the post data contains email-id with a '+' symbol, like "example+something#gmail.com", it gets read as
"example something#gmail.com".
I know this is happening because the + symbol is getting decoded as space, but how do I prevent this from happening in this scenario?
Solved it! The post data was not being urlencoded before being sent. For my scenario, since I was using Javascript XMLHttpRequest to post data, I simply replaced the "+" symbol with "%2B" using the js replace() function.
Another way according to https://stackoverflow.com/a/14830517/5918981 is to use encodeURIComponent()
In cryptography, starting with:
'http://www.server.com/?tag=xx&uid=99' (for example)
a hash length extension attack attempts to append to the url with
'http://www.server.com/?tag=xx&uid=99%80%00&deposit=100'
where the the '%80%00' represents the percent encoding of the 'padding' characters used to extend the message.
I'm using HTTPConnection.request('Get', url) in python to send the request but I'm getting an invalid uid message. Does this mean that the server is treating the %80%00 as part of the uid value? Or am I encoding the url incorreectly?
Could someone tell me what the problem is and how to fix it?
Thanks.
It depends on the backend code. One possible solution is to append &uid=99 to the url being used for the attack to overwrite the invalid uid.
Ex. http://www.server.com/?tag=xx&uid=99%80%00&deposit=100&uid=99
The server then replaces the invalid uid with the valid one.
I use Postman for REST API testing and parametrize tests with global variables.
I should put a phone number into GET request: /path/get?phone={{phone}} but leading + sign in the phone number is interpreted as a space.
What is the syntax to URL encode global variables in Postman? Is it possible to run JS encodeURIComponent() on variable in URL?
I am late but still worth it:
Just highlight and right click the part of url you want to encode. Select encodeURIComponent
That's it.
Use the Pre-request scripts (it's next to body) for this:
var encoded = encodeURIComponent({{phone number}});
or
var encoded = encodeURIComponent(pm.environment.get("phone number"));
and to proceed, use:
pm.environment.set("encoded phone number", encoded);
And set your URL to /path/get?phone={{encoded phone number}}
Just a shortcut to Mohhamad Hasham' answer.
You can encode and decode direct in the Params Value field:
The trick is to get your environment variable in the pre-request script and then set it after encoding it
var encoded = encodeURIComponent(pm.environment.get("phone"));
pm.environment.set("encoded phone number", encoded);
This will work as well:
var encoded = encodeURIComponent(pm.request.url.query.get("phone"));
pm.request.url.query.remove("phone");
pm.request.url.query.insert("phone", encoded);
I came across this question looking for an answer to a similar question. For me, the variable was a JSON object. The endpoint I needed to hit was expecting an object list as a query parameter and I have no way to change that to be the request body.
As much as some of the answers helped, I ended up coming up with a combined solution. Also, some of the code given in other answers is outdated as Postman has updated their API over the years, so this uses methods that work on 7.22.1.
pm.environment.set("basicJSON", '[{"key1":"value1","key2":"value2"},{"key1":"value1","key2":"value2"}]')
var encoded = encodedURIComponent(pm.environment.get("basicJSON"))
pm.environment.set("encodedJSON", encoded)
This solution requires that both basicJSON and encodedJSON exist as environment variables. But what was important for me was the ease of editing the object. I didn't want to have to decode/encode constantly to change values, and I didn't want to have to open the environment variables dialogue. Also, it's important to note the single-quotes around the object. Excluding them or using double-quotes would cause Postman to send something like "[object Object]" which is useless to an endpoint expecting actual JSON.
I had similar problem with braces { and } in query parameter.
By turning off the following setting it started working for me.
For the postman version 9.28.4 ==>
You can use 2 methods:
By selecting the part of the url in url bar -> right click -> EncodeURLComponent. (screenshot attached)
You can also use "pre-request script" tab of postman and write the script for the variable manually. (screenshot attached)
The problem with right-click => Encode URI Component is that it destroys the raw value of that parameter. You can use the following pre-request script to overcome this (which also works for cases where you have disabled that param):
// queryParam is of type https://www.postmanlabs.com/postman-collection/QueryParam.html
if ((queryParam = pm.request.url.query.one("name_of_your_query_param")) !== undefined
&& queryParam.disabled !== true) {
queryParam.value = encodeURIComponent(queryParam.value);
}
Click the Params button to open the data editor for URL parameters. When you add key-value pairs, Postman combines everything in the query string above. If your URL already has parameters - for example, if you are pasting a URL from some other source. Postman splits the URL into pairs automatically.
https://www.getpostman.com/docs/v6/postman/sending_api_requests/requests
POSTMAN's documentation on building requests in the section "sending parameters" is helpful here. You can encode path data by simply encoding the URL with a colon, listing the key name of the encoded element, and then a new section will appear below the query parameters allowing you to customize values and add a description, just as we do with query params. Here's an example of encoding the URL for a GET request:
https://somesite-api-endpoint/:record/get
And here's what the display looks like after you add path data. Any value you add in the path variables section will automagically update the URL with your data.
This is probably a silly question, but I been scratching my head over this for far too long.
I am trying to request the photo information from the facebook GraphAPI using Facepy/social-auth in django.
My view has the following code, but how do i turn the resulting json into python objects?
instance = UserSocialAuth.objects.filter(user=request.user).filter(provider='facebook')
graph = GraphAPI(instance[0].extra_data['access_token'])
p=graph.get('me/photos')
Facepy seems very good, but the documentation is poor at best, is there a better python facebook sdk that plays nice with social-auth?
Thanks for all suggestions.
Facepy returns native Python objects, not JSON.
response = graph.get('me/photos')
for photo in response['data']:
print photo['source']
You can use simplejson's loads function
from django.utils import simplejson
simplejson.loads(args)
Deserialize s (a str or unicode instance containing a JSON
document) to a Python object.
If ``s`` is a ``str`` instance and is encoded with an ASCII based encoding
other than utf-8 (e.g. latin-1) then an appropriate ``encoding`` name
must be specified. Encodings that are not ASCII based (such as UCS-2)
are not allowed and should be decoded to ``unicode`` first.
``object_hook`` is an optional function that will be called with the
result of any object literal decode (a ``dict``). The return value of
``object_hook`` will be used instead of the ``dict``. This feature
can be used to implement custom decoders (e.g. JSON-RPC class hinting).
``parse_float``, if specified, will be called with the string
of every JSON float to be decoded. By default this is equivalent to
float(num_str). This can be used to use another datatype or parser
for JSON floats (e.g. decimal.Decimal).
``parse_int``, if specified, will be called with the string
of every JSON int to be decoded. By default this is equivalent to
int(num_str). This can be used to use another datatype or parser
for JSON integers (e.g. float).
``parse_constant``, if specified, will be called with one of the
following strings: -Infinity, Infinity, NaN, null, true, false.
This can be used to raise an exception if invalid JSON numbers
are encountered.
To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
kwarg.
I'm going to write a program can post and read messages from SQS with authentication and I've read the document from here
Link: Query Request Authentication
I have successfully written the process which post a message to specified queue follow by the document. But I always get 403 error when I try to receive message from queue. And I found the signature string rules are different for POST and GET methods.
the signature string is:
GET\n
sqs.us-east-1.amazonaws.com\n
/<My Account Id>/<Queue Name>\n
AWSAccessKeyId=<My Access Key>
&Action=ReceiveMessage
&MaxNumberOfMessages=10
&VisibilityTimeout=600
&AttributeName=All
&Expires=2012-04-01T11%3A29%3A24Z
&SignatureMethod=HmacSHA1
&SignatureVersion=2
&Version=2011-10-01
and the url is
https://sqs.us-east-1.amazonaws.com/<My Account Id>/<Queue Name>?
Action=ReceiveMessage
&MaxNumberOfMessages=10
&VisibilityTimeout=600&AttributeName=All
&Version=2011-10-01
&Expires=2012-04-01T11%3A29%3A24Z
&Signature=<BASE64 encoded HmacSHA1 digist with signature string and my security key>
&SignatureVersion=2
&SignatureMethod=HmacSHA1
&AWSAccessKeyId=<My Access Key>
And I always get the 403 forbidden error:
<ErrorResponse xmlns="http://queue.amazonaws.com/doc/2011-10-01/">
<Error>
<Type>Sender</Type>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
</Message>
<Detail/>
</Error>
<RequestId>16f6e910-62e6-4259-8c09-0358b84cbe60</RequestId>
</ErrorResponse>
Is there anyone can tell me how can I deal with it? Thanks a lot
The error message tells you that the signature is being calculated wrong. This is really tough to debug. I spent hours on it the first time I tried it. There's an example signed SQS request at http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/MakingRequests_MakingQueryRequestsArticle.html. You should put those parameters in your program, calculate the signature, and try finding bugs into your program creates the same signature.
Specific problems I had, and fixes for them included:
Sorting the query parameters correctly. They must be in ascending order when creating the string to sign. Your example URL does not show them in order. Did you sort them differently when creating the string to sign?
URI encoding properly. Every parameter must be URI encoded in the string to sign. Your sample URL does have URI encoding, so this probably isn't your issue. But make sure you're not double-encoding.
Padding the base64 signature. At least some AWS services insist that the signature be a multiple of four characters long. Two-thirds of the time a base64 encoding will be too short, and need one or two equal signs appended to it. Most base64 encoding libraries do that for you, but not all.
Of course, the easiest thing is to use somebody else's library to make the requests, but what's the fun in that? Good luck debugging this.
It's most likely the parameter order: when assembling the signature version 2 string, at the last step the Amazon documentation specifies:
Add the query string components (the name-value pairs, not including
the initial question mark (?) as UTF-8 characters which are URL
encoded per RFC 3986 (hexadecimal characters must be uppercased) and
sorted using lexicographic byte ordering. Lexicographic byte ordering
is case sensitive.
I've spent two days debugging this same "SignatureDoesNotMatch" issue by checking my HMAC, BASE64 and URL encoding procedures and it was just a problem of parameter order.
The documentation should emphasize this issue more; if you use unordered parameter strings (e.g. the same one in the request URL, like those found in the documentation examples), you're going to get this non-intuitive error from the server.