I have noticed that for every request, request.scheme is http. I can't find any official source why, but I have been told by my peers this is because of Cloudflare acting as reverse proxy and a tls terminator, causing my server's hosting provider to see http instead of https.
One part of my app uses the Google Classroom API, and I have configured a callback to a secure endpoint of my server. Upon attempting to fetch a token from the callback's absolute uri, oauthlib raises oauthlib.oauth2.rfc6749.errors.InsecureTransportError: (insecure_transport) OAuth 2 MUST utilize https. because it thinks the request is http and insecure. I have researched and found out I can set os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1' to mitigate this issue, but I am hesitant to do so because I am unsure if that compromises security.
My thought process is to manually change the request url to https when I'm absolutely sure it is secure. Inspecting request.META to find which headers are set in the case of a reverse proxied http request, the code looks like this:
authorization_response = request.build_absolute_uri()
if (
authorization_response.startswith("http://") and
request.META["HTTP_X_FORWARDED_PROTO"] == "https" and
request.META["HTTP_ORIGIN"].startswith("https") and
json.loads(request.META["HTTP_CF_VISITOR"])['scheme'] == "https"
):
authorization_response = "https://" + authorization_response[7:]
... fetch the token passing authorization_response and etc
which seems to work.
Considering how little information I could find this online, I was wondering if this is a best practice and if there's a better way of letting oauthlib know the request is secure. Or, since my domain has HSTS preload, is this all unnecessary and I can get away with setting os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'?
Related
Infra of system
Expected:
I want to block requests, which is not from Server FE (domain.com)
Ex: Users make request from another apps such as Postman -> it will response 403, message access denied.
I used the rules of ALB, it works but users can cheat on Postman
Also I use AWS WAF to detect request. But it's not work.
Is there any way to block request from Postman or another apps?
We can generate secret_key and check between Server FE and Server BE. But users can see it on Headers and simulator the headers on Postman and call API success.
Current Solution:
I use Rule of Application Load Balancer to check Host and Origin. But users can add these params on Postman and request success.
Rule ALB
When I add Origin matching value (set on ALB) -> We can request successful
Postman success
Postman denied
Users can cheat and call API success.
Thanks for reading. Please help me give any solution for this one. Thanks a lot.
No. HTTP servers have no way to know what client is being used to make any HTTP request. Any HTTP client (Browsers, PostMan, curl, whatever) is capable of making exactly the same requests as each other.
The user-agent header is a superficial way to do this, but it's easy enough for PostMan or any other HTTP client to spoof the user-agent header to one that makes the request look like it is coming from a web browser agent.
You can only make it more challenging to do so. Some examples to thwart this behavior includes using tools like Google captcha or CloudFlare browser integrity check, but they're not bulletproof and ultimately aren't 100% effective at stopping people from using tools/automation to access your site in unintended ways. At the end of the day, you're limited to what can be done with HTTP, and PostMan can do everything at the HTTP layer.
Edit:
After investigating this further, it seems cookies are sent correctly on most API requests. However something happens in the specific request that checks if the user is logged in and it always returns null. When refreshing the browser a successful preflight request is sent and nothing else, even though there is a session and a valid session cookie.
Original question:
I have a NextJS frontend authenticating against a Keystone backend.
When running on localhost, I can log in and then refresh the browser without getting logged out, i.e. the browser reads the cookie correctly.
When the application is deployed on an external server, I can still log in, but when refreshing the browser it seems no cookie is found and it is as if I'm logged out. However if I then go to the Keystone admin UI, I am still logged in.
In the browser settings, I can see that for localhost there is a "keystonejs-session" cookie being created. This is not the case for the external server.
Here are the session settings from the Keystone config file.
The value of process.env.DOMAIN on the external server would be for example example.com when Keystone is deployed to admin.example.com. I have also tried .example.com, with a leading dot, with the same result. (I believe the leading dot is ignored in newer specifications.)
const sessionConfig = {
maxAge: 60 * 60 * 24 * 30,
secret: process.env.COOKIE_SECRET,
sameSite: 'lax',
secure: true,
domain: process.env.DOMAIN,
path: "/",
};
const session = statelessSessions(sessionConfig);
(The session object is then passed to the config function from #keystone-6/core.)
Current workaround:
I'm currently using a workaround which involves routing all API requests to '/api/graphql' and rewriting that request to the real URL using Next's own rewrites. Someone recommended this might work and it does, sort of. When refreshing the browser window the application is still in a logged-out state, but after a second or two the session is validated.
To use this workaround, add the following rewrite directive to next.config.js
rewrites: () => [
{
source: '/api/graphql',
destination:
process.env.NODE_ENV === 'development'
? `http://localhost:3000/api/graphql`
: process.env.NEXT_PUBLIC_BACKEND_ENDPOINT,
},
],
Then make sure you use this URL for queries. In my case that's the URL I feed to createUploadLink().
This workaround still means constant error messages in the logs since relative URLs are not supposed to work. I would love to see a proper solution!
It's hard to know what's happening for sure without knowing more about your setup. Inspecting the requests and responses your browser is making may help figure this out. Look in the "network" tab in your browser dev tools. When you make make the request to sign in, you should see the cookie being set in the headers of the response.
Some educated guesses:
Are you accessing your external server over HTTPS?
They Keystone docs for the session API mention that, when setting secure to true...
[...] the cookie is only sent to the server when a request is made with the https: scheme (except on localhost)
So, if you're running your deployed env over plain HTTP, the cookie is never set, creating the behaviour you're describing. Somewhat confusingly, in development the flag is ignored, allowing it to work.
A similar thing can happen if you're deploying behind a proxy, like nginx:
In this scenario, a lot of people choose to have the proxy terminate the TLS connection, so requests are forwarded to the backend over HTTP (but on a private network, so still relatively secure). In that case, you need to do two things:
Ensure the proxy is configured to forward the X-Forwarded-Proto header, which informs the backend which protocol was used originally request
Tell express to trust what the proxy is saying by configuring the trust proxy setting
I did a write up of this proxy issue a while back. It's for Keystone 5 (so some of the details are off) but, if you're using a reverse proxy, most of it's still relevant.
Update
From Simons comment, the above guesses missed the mark 😠but I'll leave them here in case they help others.
Since posting about this issue a month ago I was actually able to work around it by routing API requests via a relative path like '/api/graphql' and then forwarding that request to the real API on a separate subdomain. For some mysterious reason it works this way.
This is starting to sound like a CORS or issue
If you want to serve your front end from a different origin (domain) than the API, the API needs to return a specific header to allow this. Read up on CORS and the Access-Control-Allow-Origin header. You can configure this setting the cors option in the Keystone server config which Keystone uses to configure the cors package.
Alternatively, the solution of proxying API requests via the Next app should also work. It's not obvious to me why your proxying "workaround" is experiencing problems.
I have a custom domain example.com that is redirecting to my API gateway api-example.com, but it doesn't seem to pass the user-agent field, all my user-agent values are AmazonAPIGateway_5rfp2g9h9b.
If I call directly the api-example.com then it works fine, but if I call example.com, doesn't work.
Any idea on how I could pass the correct user-agent HTTP Header?
Thanks
It’s not clear what you mean by redirect or the domains you have listed, so you have two custom domains ? And if so how did you do that, Cloudfront with a custom origin? And what type of integration request do you have? Is this a REST or HTTP API? Probably why you are getting down voted because you don’t have any detail and the domains don’t make sense.
Either way in your API make sure you have the user-angent field defined where it is applicable:
Request Part of your API, and make sure your integration request is forwarding this header
Likewise make sure Cloudfront forwards the ‘user-agent’ header, that it is also whitelisted if you are using Cloudfront
Note this header comes from your Web browser or SDK being used sometimes sets this too. So if you don’t set this header for whatever reason that could be a problem, I don’t know if for example when you say from this domain that means you are using a hosted website, and another means making a request from Postman, etc.
Short answer: Validate the contents of your header
Ref AWS user-agent redirect here.. as listed below.
Redirects and HTTP user-agents:
..Programs that use the Amazon S3 REST API should handle redirects either at the application layer or the HTTP layer. Many HTTP client libraries and user agents can be configured to correctly handle redirects automatically; however, many others have incorrect or incomplete redirect implementations.
Before you rely on a library to fulfill the redirect requirement, test the following cases:
Verify all HTTP request headers are correctly included in the redirected request (the second request after receiving a redirect) including HTTP standards such as Authorization and Date.
Verify non-GET redirects, such as PUT and DELETE, work correctly.
Verify large PUT requests follow redirects correctly.
Verify PUT requests follow redirects correctly if the 100-continue response takes a long time to arrive.
HTTP user-agents that strictly conform to RFC 2616 might require explicit confirmation before following a redirect when the HTTP request method is not GET or HEAD. It is generally safe to follow redirects generated by Amazon S3 automatically, as the system will issue redirects only to hosts within the amazonaws.com domain and the effect of the redirected request will be the same as that of the original request...
Optional/Additional help, I was trying to understand your description, if you're going across domains, thats CORS.
Please consider CORS which you seem to be missing, please see configuration
here.
Also very important you Enabling CORS support for a resource and its methods does not recursively enable it for child resources and their methods.
If you want to setup your custom header for
user-agent
Setup CORS in Console
How to setup from console under the resources enable the CORS.
Setup your Headers
As a last step you have to REdeploy to a stage, for the settings to take effect!
i'm making a web app in a restful way.
For my client side i'm using angularJs, the same is hosted at - lets say -
https://domain.com
My backend is built on spring boot and i call all the resources from a subdomain - lets say -
https://xyz.domain.com
Now when a user logs in , the backend sends an http only cookie to the client.
I can see the cookie in response header but its not being set in the browsers cookie.
After a bit of research, i have tried sending cookie with domain = .domain.com
but that didnt work either.
Is there a way i can set cookie coming from xyz.domain.com for my client side at domain.com
(Note - i'm not using www.domain.com )
Any help or clue would be great.
Thank you for going through my question.
The problem you're describing is related to cross domain cookie policies. I don't know your exact use-case, but looking at CORS and P3P headers should give you a good start. As an option, you can try setting your cookie manually via Javascript.
Making CORS working isn't enough, you also need to enable withCredentials in angular.
https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/withCredentials
Example:
angular.module('example', []).config(function ($httpProvider) {
$httpProvider.defaults.withCredentials = true;
});
I was hoping to get some recommendations on how to approach redirecting users from HTTP to HTTPS using an ember initializer with ember-simple-auth.
`import ENV from 'cio/config/environment'`
SSLInitializer =
name: 'ssl'
before: 'simple-auth-cookie-store'
initialize: (container, application) ->
application.deferReadiness()
# Redirect if hitting HTTP and SSL is enabled
if ENV.SSL and window.location.protocol is "http:"
window.location.href = "https:" + window.location.href.substring(window.location.protocol.length)
return false
application.advanceReadiness()
`export default SSLInitializer`
But it seems that the cookie gets invalidated even when the if statement evaluates to true. I've tried several things, including:
before: 'simple-auth'
before: 'store'
application.destroy() within the if statement, before the window.location.href is set
From what I can tell, after debugging. The app does redirect to HTTPS, but then the cookieName is not found in document.cookie. (https://github.com/simplabs/ember-simple-auth/blob/master/packages/ember-simple-auth-cookie-store/lib/simple-auth-cookie-store/stores/cookie.js#L154)
Before this method worked because we had simple snippet in the index.html, but w/ CSP we'd like to keep it in an initializer. Any recommendations?
Thanks!
You really should be forcing a redirect from HTTP to HTTPS from the server as doing it from the client does not add any real security.
Think about it, the user has downloaded the application to their browser from an insecure endpoint and from then on nothing can be trusted. Even a server based redirect is problematic since it relies on the redirect advice from an untrusted endpoint. Users should really be accessing things from an initial trusted starting point otherwise all bets are off. This is known as the secure referral problem and will likely never be solved because of the business model behind SSL certificates.
You also shouldn't really trust cookies from the untrusted HTTP domain in the trusted HTTPS domain unless you have a way to authenticate those cookies on the client. Sharing of cookies between HTTP/HTTPS is covered in RFC 2109 (Section 4.2.2 Set-Cookie Syntax).
This means:
A cookie set with "Secure" will be available only on HTTPS
A cookie set without "Secure" will be available on either HTTP or HTTPS.