I've been using Google's documentation to attempt to build/test server-side events for GA4 using Postman. I've found the event to work when using their event builder (I see the event in real-time events), but when I transfer it to Postman, it doesn't work (does not show in real-time events). I keep receiving a 403 Forbidden message. I'm 100% sure I have it set up correctly, but I'm not sure if there's some sort of setting in Google to allow requests from a server instead of a browser (I'm using an out-of-the-box implementation of a GA4 data stream). I've tried sending a User-Agent from my browser in the header and that doesn't seem to make a difference. I'm sending the following:
METHOD:
POST
URL:
www.google-analytics.com/mp/collect?measurement_id=<<MY ID IS HERE>>&api_secret=<<MY API SECRET HERE>>
BODY (RAW TEXT):
{
"clientId": "783221853.1614059609",
"events": [{
"name": "add_to_cart",
"params": {
"items": [{
"item_id": "Robot Toy",
"item_name": "12345678"
}]
}
}]
}
HEADERS:
Content-Type: Text/Plain
Content-Length: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.192 Safari/537.36
Any ideas on what I might be doing wrong, or if it is an issue with Google and the fact that GA4 is still buggy? Or maybe it thinks I'm a bot and has blocked the traffic?
We had the same 403 error and discovered that https:// was missing in front of the URL and the request was denied because of using http only.
Body is correct. But if you send json body you should have:
headers = {
"Content-Type": "application/json"
}
Related
I have been tearing my hair out trying to get API Gateway to be the front door and trigger for a Lambda function.
I set up a Lambda, called "ContactFormLambda" that accepts a POST, with the details:
{
"subject": "Enquiry from ZenithWebFoundry",
"name": "contact.name",
"number": "contact.number",
"email": "*******#gmail.com",
"comment": "this is a test"
}
Now, in the API Gateway, I created a new API called "contact" and created a new rescource called "contactformlambda", set it up as a proxy to the lambda and defined a POST method (because that's all I'd be doing.
I tested this through the gateway and it works.
This is the API Gateway settings under the Lambda (from the above work):
I then wanted to be able to POST to the gateway from a form, so I realised that I'd need CORS, so I enabled CORS on the resource. This created an OPTIONS method with the three headers on it:
access-control-allow-headers
access-control-allow-methods
access-control-allow-origin
Then I tried to submit the form, but the options pre-fetch fails with a 403. I have followed the instructions here to do all of this. I have tried lots of variations and nothing seems to work.
Having said that, one of the times, it did work, but it let everything through not just the domains that I had specified in the Access-Control-Allow-Origin field. I can't see what I did differently here. It feels like each time I set it up, it does something different. I'm obviously inept at this, can anyone clear this up?
I got this somewhere towards working. Basically, Ignore the "Enable CORS" Action from the Actions menu in API Gateway: it doesn't work, instead follow this guide.
Also, the persistent 403 for me was caused by the url to the resource being case sensitive. I was trying to hit:
https://ovo5xmxf7e.execute-api.ap-southeast-2.amazonaws.com/prod/ContactFormLambda
but the actual url was:
https://ovo5xmxf7e.execute-api.ap-southeast-2.amazonaws.com/prod/contactformlambda
crazy huh?
I said, "somewhere toward working", because basically CORS works a little too well. I specified:
Access-Control-Allow-Origin: 'http://zenithwebfoundry.com'
And despite this coming back in the POST response:
General
Request URL:https://ovo5xmxf7e.execute-api.ap-southeast-2.amazonaws.com/prod/contactformlambda
Request Method:POST
Status Code:200
Remote Address:54.230.243.227:443
Referrer Policy:no-referrer-when-downgrade
Response Headers
access-control-allow-headers:Content-Type,X-Amz-Date,Authorization,X-Api-Key,x-requested-with
access-control-allow-methods:POST,OPTIONS
access-control-allow-origin:http://zenithwebfoundry.com
content-length:0
content-type:application/json
date:Mon, 12 Feb 2018 10:36:17 GMT
status:200
via:1.1 fda24eb25587b7e25d4fa1c50a862ba0.cloudfront.net (CloudFront)
x-amz-cf-id:b7DscV6b6qWl_mHUsEi77nRjICECZD3bd2iXiFv89A1d_xrDvfBQ0Q==
x-amzn-requestid:8d5578fc-0fe0-11e8-9e16-65a7dc0a72e4
x-amzn-trace-id:sampled=0;root=1-5a816e1f-aeeb63ef8590ed3868fe4149
x-cache:Miss from cloudfront
x-requested-with:*
Request Headers
:authority:ovo5xmxf7e.execute-api.ap-southeast-2.amazonaws.com
:method:POST
:path:/prod/contactformlambda
:scheme:https
accept:application/json, text/plain, */*
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cache-control:no-cache
content-length:147
content-type:text/plain
origin:http://localhost:4200
pragma:no-cache
referer:http://localhost:4200/contact
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36
It still completed the action even though my origin was clearly not http://zenithwebfoundry.com. So why is the
access-control-allow-origin:http://zenithwebfoundry.com
setting ignored?
Also, another observation, I have noticed that the pre-fetch is no longer happening. I have tracked it down to the fact that I am no longer sending any CORS disallowed headers (I was sending Access-Control-Allow-Origin in a mistaken belief that I had to send it in the request). I have now started sending "x-requested-with:*", which now forces the preflight check to see if "x-requested-with" is allowed to be sent.
I was hoping the pre-flight check would also prevent execution of the post from localhost, but it doesn't.
Specifically, I'm trying to scrape this entire page, but am only getting a portion of it. If I use:
r = requests.get('http://store.nike.com/us/en_us/pw/mens-shoes/7puZoi3?ipp=120')
it only gets the "visible" part of the page, since more items load as you scroll downwards.
I know there are some solutions in PyQT such as this, but is there a way to have python requests continuously scroll to the bottom of a webpage until all items load?
You could monitor page network activity with browser development console (F12 - Network in Chrome) to see what request does the page do when you scroll down, use that data and reproduce the request with requests. As an alternative, you can use selenium to control a browser programmatically to scroll down until page is ended, then save its HTML.
I guess I found the right request
Request URL:http://store.nike.com/html-services/gridwallData?country=US&lang_locale=en_US&gridwallPath=mens-shoes/7puZoi3&pn=3
Request Method:GET
Status Code:200 OK
Remote Address:87.245.221.98:80
Request Headers
Provisional headers are shown
Accept:application/json, text/javascript, */*; q=0.01
Referer:http://store.nike.com/us/en_us/pw/mens-shoes/7puZoi3?ipp=120
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
X-NewRelic-ID:VQYGVF5SCBAJVlFaAQIH
X-Requested-With:XMLHttpRequest
Seems that query parameter pn means the current "subpage". But you still need to understand the response correctly.
I’m developing an Android App and a Web Service that communicate. My Web Service is in WEB API 2 with token bearer authentication.
My problem is that when I send too many requests (~20 request in 15 seconds) to my Web Service from my Android App, the WS response with
“401” : “Authorization has been denied for this request”
This happen ONLY on the production server (Amen hoster) AND from the Android Device. For example, if I try with Postman, everything works fine. So it’s related to my production server and/or my android app request.
The code for access to the Web Service
URL obj = new URL(SERVEUR_URL + url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("Authorization", "Bearer " + token);
con.setRequestProperty("Content-Type", "application/json");
int responseCode = con.getResponseCode();
String responseMessage = con.getResponseMessage();
The authentication provider on my Web Service is the default one. No modifications.
The request from my Android App (not work every time)
GET http://api.xxxx.com/api/Weesps/GetAvailableWeesps HTTP/1.1
Authorization: Bearer XXXX
Content-Type: application/json
User-Agent: Dalvik/2.1.0 (Linux; U; Android 6.0; Google Nexus 5X - 6.0.0 - API 23 - 1080x1920 Build/MRA58K)
Host: api.xxxx.com
Connection: Keep-Alive
Accept-Encoding: gzip
The request from Postman (work every time)
GET http://api.xxxx.com/api/Weesps/GetAvailableWeesps HTTP/1.1
Host: api.xxxx.com
Connection: keep-alive
Authorization: Bearer XXXX
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36
Postman-Token: bca55154-775d-9709-7a8b-4793393890ad
Accept: */*
Accept-Encoding: gzip, deflate, sdch
Accept-Language: fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4
Cookie: dadaproaffinity=14ff51cc869a14d3552485cb4ceee1faa1be7165cc5d4b0e2b19370f11afcbea
What I have tried:
Reproduce this error in local : it works fine on local server (web and SQL servers) from android app or from Postman
I check that the token was sent correctly in every requests
The request from Android is the same every time
Tried to add missing header to my android app request
I spend two days on this problem and read many stackoverflow posts but no one helps me.
Thanks for your help.
UPDATE 1 :
With Fiddler I saw that in GET request from Postman, they were a Cookie header. This cookie is sent when we ask for a bearer token.
Example of token response from the server
HTTP/1.1 200 OK
Cache-Control: no-cache
Pragma: no-cache
Content-Length: 691
Content-Type: application/json;charset=UTF-8
Expires: -1
Server: Microsoft-IIS/8.5
Set-Cookie: .AspNet.Cookies=XXXX; path=/; HttpOnly
X-Powered-By: ASP.NET
X-Powered-By: ARR/2.5
Date: Tue, 31 May 2016 16:55:39 GMT
{"access_token":"XXXX","token_type":"bearer","expires_in":1209599,"userName":"Foo",".issued":"Tue, 31 May 2016 16:55:40 GMT",".expires":"Tue, 14 Jun 2016 16:55:40 GMT"}
Fiddler and Postman saved this cookie and they automatically put it in requests to API (example on the “The request from Postman” code block). When I remove the cookie from the Postman GET request, it doesn’t work (just like my android app).
Now, the question is: why WEB API 2 send a cookie instead of only using the token ? And why the token work great in the first requests and don’t work properly for the following requests ?
According to ASP.NET WebAPI2 flow you can see on the bottom of that page, it seems your requests are always authenticated but sometimes fail to get authorized.
So imo, the AuthorizationFilter[Authorize] rejects some of your requests for an unknown reason. What I would suggest is to dump the request your API receives as well as the claims identity attached to the token. Try to see if there is differences between them when you have a successful response and when you have a 401.
That way, you may be able to determine either it is your request that got malformed, if it is the claims identity that is not good or if it is the AuthorizationFilter that refuses you for another reason (like too much queries or else).
Good luck !
UPDATE 1
According to your new input, I think that your Web API is configured to use both token and cookie authentication.
What I see here is you have two solutions :
1°/ Store the returned cookie in your Android application and use it for next calls. Simplest and fastest way to solve your problem without changing all your API, but you store an authorization cookie : it can leads to security problem (CSRF attacks).
2°/ You can check how your authentication and authorization filters are set to disable cookie authentication and only rely on token authentication : it will hence forces all the requests and your API to only use token and will prevents you from suffering CSRF attacks. More complex because you have to dig into your web API configuration.
Check the following links (sorry, as I don't have enough reputation yet to post more than 2 links per post, you'll find them as text at the end of my answer) :
ASP.net Secure a Web API 2.2[2] : From the chapter "Configuring the Authorization Server" at the bottom
MSDN article on Web API security[3] : More general and technical information about web api security, how to secure it and CRSF attacks
StackOverflow .NET cookie and token authentication[4] : Check David Banister's answer, I think it is exactly what you want to do : Only use token for all your API calls.
StackOverflow Authorize filter and authentication[5] : More information about such mechanisms for your API
And finally
Cookie authentication with web API and 401 codes[6] : Sounds like your actual problem, isn't it ?
I hope it helps you, good luck !
// Links
2: www.asp.net/web-api/overview/security/individual-accounts-in-web-api
3: msdn.microsoft.com/en-us/magazine/dn201748.aspx
4: stackoverflow.com/questions/22568409/mvc-net-cookie-authenticated-system-acessing-a-web-api-with-token-authenticatio
5: stackoverflow.com/questions/21231751/authorize-filter-and-authentication
6: brockallen.com/2013/10/27/using-cookie-authentication-middleware-with-web-api-and-401-response-codes/
Finally, I got my answer:
My Web Service send a Cookie named “dadaproaffinity” the first time I ask for a request. This Cookie was automatically put on the following request by Postman but not by Android HttpUrlConnection. So, I just take this Cookie and now I just add this Cookie on every requests with the Token.
But : This cookie is send by IIS, not by my Web Service ! That’s why it works on local but not on the production server. I googled this cookie and there are very few responses about that. The only one that I find in English is :
Technical Cookie of IIS Server hosting the site.
Need to route to the correct server session, in order to keep it active
Does anyone have more information about this IIS Cookie ?
I am trying to make a GetOrders call on eBay Trading API, and am getting CORS error:
XMLHttpRequest cannot load https://api.ebay.com/ws/api.dll.
No 'Access-Control-Allow-Origin' header is present on the requested
resource. Origin '' is therefore not allowed
access.
These are the headers :
Request URL:https://api.ebay.com/ws/api.dll, Request Method:OPTIONS
Status Code:200 OK Request Headersview source Accept:*/*
Accept-Encoding:gzip,deflate,sdch Accept-Language:en-US,en;q=0.8
Access-Control-Request-Headers:x-ebay-api-siteid, x-ebay-api-dev-name, content-type, x-ebay-api-cert-name, accept, x-ebay-api-request-encoding, x-ebay-api-call-name, x-ebay-api-app-name, x-ebay-api-compatibility-level
Access-Control-Request-Method:POST Connection:keep-alive
Host:api.ebay.com Origin:<mydomain.com>
Referer:<mydomain.com> User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36
Response Headers:
Allow:GET, HEAD, POST, TRACE, OPTIONS
Connection:keep-alive
Content-Length:0
Date:Sun, 19 Jan 2014 18:15:10 GMT
Server:Apache-Coyote/1.1
X-EBAY-ESB-GUID:urn:uuid:UUIDDDODSDFLSDKLSDKFJKS
X-EBAY-ESB-SITEID:0
Does eBay need to explicitly "allow" my domain for CORS ? I have searched internet, and eBay API documentation, and there is no indication of this. If I use curl, the API call is successfully run, though the eBay trading API does not return Access-Control-Allow-Credentials: true.
In practice, all browsers enforce the Cross-Origin Resource Sharing restrictions described in the CORS standard which includes sending the preflight OPTIONS request like you're seeing. The check is not mandatory, though and non-browser user agents such as curl, Android/iOS HTTP clients, etc. generally do not enforce this check.
There is no eBay developer process for allowing particular domains to make cross-domain resource requests. The CORS support is somewhat patchy across the different APIs. For example, the finding API call findItemsByKeywords works fine cross-domain (see here), while other services don't allow it (see a very old but still unanswered request here).
This is not a great answer, but you can always use a backend proxy to make your calls. In this way, your backend can make the calls with curl or the like, and you also get the benefit of being able to hide your appID, user tokens and other secrets from local inspection of your javascript.
You may also want to file a request with eBay, but some of those tickets have been languishing for some time.
I have been wrestling with this issue for several hours:
I have a Single-Page Application written in Angular which communicates with a DjangoREST backend. I am trying to implement an auth fonction with session Cookies. The way I see it is:
1/ Show any unlogged visitor a login page
2/ Make a POST to url/login with the credentials
3/ Obtain a "sessionid" cookie and writing in a service that the user is logged
4/ Redirect vistor towards reserved content and used get & post to access contents with the cookie
The login endpoint is already set and works. When I make a post, I receive a HTTP 200 response with user info and a Set-Cookie, but subsequent calls do not contain the Cookie:
Request URL: ...
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:application/json, text/plain, */*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4
Connection:keep-alive
Content-Length:38
Content-Type:application/x-www-form-urlencoded; charset=UTF-8
Host:devinify1.herokuapp..
Origin:http://mobilevinify.herokuapp...
Referer:http://mobilevinify.herokuapp...
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36
Form Dataview sourceview URL encoded
username:felix#vinify.co
password:test
Response Headersview source
Access-Control-Allow-Origin:*
Connection:keep-alive
Content-Length:189
Content-Type:application/json
Date:Sat, 14 Dec 2013 20:45:14 GMT
Server:gunicorn/18.0
Set-Cookie:sessionid=ijz27zy655qn0cwmlnvr66609hsyvdub; expires=Sat, 28-Dec-2013 20:45:14 GMT; Max-Age=1209600; Path=/
Vary:Cookie
My code is a very simple adaptation of the angular-app example:
https://github.com/FelixLC/MobileWebApp/blob/master/app/scripts/security/security.js
I have tried this on localhost et on heroku. The server and the client are on different domains, CORS are allowed.
When I try to make calls, I receive an error from Django
TypeError at /vinibarwines/
int() argument must be a string or a number, not 'AnonymousUser'
Should I try to get this cookie and put it in the headers with angularJS?
You can try to login at http://mobilevinify.herokuapp.com/#/login with felix#vinify.co & test. Then Click on Vinibar, there is a 500 internal error on the GET request
Any help much appreciated
Felix
Here is the full layout of how I actually do my authentication. Django/Angular Authentication. It's a pretty extensive response, I'm more than happy to answer further questions you might have.