How to debug Django app that runs on top of TLS/SSL? - django

What is the best way to debug a Django app that runs on top of TLS/SSL?
Background:
I have a Django web app that uses X.509 client side certificates for authentication. When running under Apache, my app can only be reached via HTTPS. Clients that connect to the app provide a client side certificate which Apache validates and then forwards to the app in an environment variable. The app parses the certificate and provides access controlled content.
So far, I have only been able to debug the app under regular HTTP, with "./manage.py runserver". I have simulated an HTTPS connection by using a custom view handler middleware that kicks in, in debug mode. The view handler adds information to the request, similar to the information that would be parsed out of an actual client side certificate when run under HTTPS.
It would make debugging much easier for me if I could debug with the actual client side certificates that clients provide when connecting via HTTPS.

We use nginx in front of Django, with client certificate checking. NGINX does the SSL termination, client cert validation, and checking against revocation list. The client cert fields are passed in header variables up to the django app.
So then our django app doesn't receive the cert, it just looks at the header variables. I think the same mechanism applies in Apache.
For clients accessing the development server (e.g. './manage.py runserver'), we simply have a special case in the client. Example of a python client:
if (proto == "https"):
conn = http.client.HTTPSConnection( "cert."+webhost+":"+port,
key_file = certfile, cert_file = certfile)
headers = {}
else:
# fake client for local connections. pass cert info in headers, as it would come
# out of nginx
conn = http.client.HTTPConnection( webhost+":"+port)
headers = { 'X_SSL_CLIENT_S_DN':'/C=US/ST=California/O=yyyy/CN=zzzz',
'X_SSL_CLIENT_I_DN':'/C=US/ST=California/O=xxxx/CN=wwww',
'X_SSL_CLIENT_SERIAL':hex(serialnum),
'USER_AGENT':"test client user agent",}
For unit tests, we do the same thing using the Django test client:
from django.test.client import Client
self.client = Client()
response = self.client.get(url, data,
**{
'HTTP_X_SSL_CLIENT_S_DN':'/C=US/ST=California/O=yyyy/CN=zzzz',
'HTTP_X_SSL_CLIENT_I_DN':'/C=US/ST=California/O=xxxx/CN=wwww',
'HTTP_X_SSL_CLIENT_SERIAL':hex(serialnum),
'HTTP_USER_AGENT':"test client user agent",
})

I've come up with a workaround that works fairly well for me. I still debug with HTTP, but I pass the client side certificate in via an HTTP header. So, when I debug the web app with HTTP, I have the clients copy the client side certificate into an HTTP header. Before entering the views, the web app copies the certificate from the header and into the regular location in which it would be passed by Apache when using HTTPS.
The client side certificates are PEM formatted so, to be able to pass them in HTTP headers, the only thing that needs to be done is to remove the newlines on the client and reinsert them on the server.
If using this approach, note that Apache's default limit for the size of a single HTTP header field is 8190 bytes, configured with the LimitRequestFieldSize directive. For certificates that are larger than that, the configuration must be changed or the certificate must be split up and passed in multiple headers.

Related

How to configure a Daphne server to invite clients to add the Certificate Authority that issued my certificate if they hadn't done it yet?

If I make a request to my Daphne/Django server in Postman or the Android app we're developing, Daphne serves the certificate, but it's rejected. If I first make a simple get request to https://letsencrypt.org/ and then make a request to my server, the certificate is accepted.
How can I make sure a client will trust my certificate, even if it's the first time this client is seeing a certificate issued by this CA?
Everything bellow can serve as a history of how I studied the problem.
Original title: SSL Certificate works in browser but can't be verified by Postman
I have an AWS EC2 instance running Ubuntu 18.04, with python 3, Django, a bunch of project dependencies, Daphne running with ASGI, with a certificate by Let's Encrypt. Daphne is using port 8000 for HTTP and por 4430 for HTTPS, iptables is configured to redirect requests from port 80 to 8000 and from port 443 to 4430. Django is configured to enforce secure connections with SECURE_SSL_REDIRECT=True in the settings.py file.
There's a "Site in Construction" temporary page being served, and it's properly accessible from every browser and every device I tested so far. If I explicitly type http, I get redirected to https and the certificate is accepted. Every browser I tested (Firefox, Brave, Chrome, Chrome for Android) says cert is good.
Curl outputs the HTML content returned from the server. I don't know if it accepts the certificate or ignores it.
The Problem
Postman, however, says "Error: unable to verify the first certificate". Only works when I disable "SSL certificate verification", which doesn't answer my question: why Postman is unable to verify my Let's Encrypt certificate?
I'm building an API that runs on the same server, using the same domain, and it's meant to be consumed by a mobile app. Currently, the Android app is throwing a "TypeError: Network request failed", which I suspect could be caused by the same thing Postman is complaining about.
When I spin the server locally and configure 1) the app to use http://localhost:8000 and 2) the server not to enforce SSL, it works in browsers, Postman and in the Android app.
I've being looking for answers in many places for days, so any clue will be very welcome.
EDIT
Interesting clue:
If I make a request to my Daphne/Django server, it servers the certificate, which is rejected. But if I first make a request to https://letsencrypt.org/ and then make a request to my server, it works!
This pattern holds true in both Postman and our Android app.
It also happens when I first make a request to https://alloy.city (instead of letsencrypt.org), which is served by a Node.js app, and uses a certificate also issued by the Let's Encrypt CA.
So maybe the question should be: how to configure my server to politely invite clients to add the CA that issued my certificate if they hadn't done it yet?.
Apparently, that's what my Node.js server does.
Yes, in settings, tap ssl verification off
File > Settings > General > SSL Certificate Verification > off

Unable to read XSRF-TOKEN cookie in Angular JS 1.5.5

TO protect app from CSRF attack we set a cookie named XSRF-TOKEN from server side. So from client side code we are able to set-cookie and send across to server, But to validate CSRF in the server side we need to send header while firing 'POST' service call. As per angular document automatically $http sets header X-XSRF-TOKEN by reading the cookie ( Please refer link), but Javascript code is unable to read the cookie though we have deployed our application on same domain.
Server side cookie generation code and service deployment details are as below,
final Cookie newCookie = new Cookie(
"XSRF-TOKEN",
csrfValue);
newCookie.setPath("/");
httpResponse.addCookie(newCookie);
UI is deployed in 8080 port and service is deployed in port 8084 inside same VM
Port 8080 and 8084 are different origins, so you can't read cookies from one on the other, the same as you can't access the cookies of any other website in javascript running on yours.
How does the service authenticate the user? If it's token based, and the token is sent as a request header, you don't even need further protection from csrf.

can proxy server set cookie?

can the proxy server intercept my https request and set cookies before actually sending the request?
I'm going a GET on an url from chrome browser. In the development tools, under "Network", I noticed that the first request, the one that I made, has cookies set. but I did not set any cookies.
any thoughts?
No it can't. To proxy HTTPS requests your browser issues HTTP CONNECT command (https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT). Proxy then creates a tunnel between the browser and a target server.
A conventional proxy can neither view nor manipulate a TLS-encrypted data stream, so a CONNECT request simply asks the proxy to open a pipe between the client and server. The proxy here is just a facilitator - it blindly forwards data in both directions without knowing anything about the contents. The negotiation of the TLS connection happens over this pipe, and the subsequent flow of requests and responses are completely opaque to the proxy.
It cannot modify or see what is being transferred as it is protected by TLS encryption.
The only way to modify HTTPS conenctions on the fly is if you install some external CA certificates on your computer. This is known as MITM Attack.

How to use HTTPS for webservice and android app?

Im working on some JSON-based web service that is supposed to work with Android application.
I would like to encrypt data transport between client (android) and server (virtual server in datacenter).
I don't have to make sure that my server is my server, just data encryption.
I have no idea how to use HTTPS.
Do I just put my PHP files in private_html and use https://example.com url?
To use HTTPS, you don't have to do anything in the coding of your web service - it's all in your hosting. Here the are steps you can follow. The specific instructions differ in your hosting (IIS, Apache, AWS/Azure, etc), but you can google specifics on how to accomplish any of these steps for whatever host and application framework you decide.
Buy an SSL certificate (there are many different vendors, but expect between $75-$200 for the certificate) based on the vendor, reputation, and level of security you need.
Generate a certificate signing request (CSR) from the server you'll be hosting.
Upload the CSR to the SSL vendor who will validate and provide the certificate for your use.
Import the SSL certificate into your application server, and configure the site to use the certificate. For instance, if you're hosting Microsoft IIS, you'd import the SSL certificate and then add HTTPS bindings on 443 to the specific website hosting your web service.
Another point of security. Since you are deploying SSL, you don't have to do any application level encryption (assuming you are not putting sensitive information in query strings - use POST if you think you need to). You probably would want to implement some security to restrict access to your web service so only your app can access it. Best practice is some level of OAuth, but at a minimum some type of pre-shared key in the header of the request is a lot better than nothing.
Here are some additional sites for more information:
https://www.digicert.com/ssl-certificate-installation.htm
https://support.godaddy.com/help/category/742/ssl-certificates-installing-ssl-certificates?prog_id=GoDaddy
If you don't want to pay for a certificate, you can use certificate signet by your own CA and add the root certificates into your application using HTTPClient and keystores
Here there's some guides
http://datacenteroverlords.com/2012/03/01/creating-your-own-ssl-certificate-authority/
http://developer.android.com/reference/org/apache/http/client/HttpClient.html
KeyStore, HttpClient, and HTTPS: Can someone explain this code to me?
http://blog.antoine.li/2010/10/22/android-trusting-ssl-certificates/
You can limit users to use JUST and only HTTPS in apache, IIS or whatever do you use. If your client connects to your server, his communications will be likely to encrypted, because he is already using HTTPS. And for responsing in HTTPS you virtually cannot send HTTPS responses, as far as I know, unless that other side isn't also a website (for example, if you have your website, you could send such a response e.g. to Google). You should be okay to send data like http status codes (OK, NotModified, PageNotFound, ...), or if you want something more, or if it is a requirement, then there you still have JSON and you could encode it as well, with some encoding algorithms, or use binary JSON format.
Check if your hosting company provides a free public shared https address. Most of them do.
If you want to understand how to do it right, follow this thread
Warning: Don't stick with the solution below for production.
If you plan o use an https endpoint without a certificate you have to make sure to disable peer verification, check this answer

How to use HTTPS with HttpReceiveHttpRequest()?

I'm using the Windows HTTP API to process web service requests in C++ (not .NET) and everything works just fine for HTTP requests. When I change the URLs I'm expecting with HttpAddUrl to https://example.com:443/foo/bar my tests from Internet Explorer no longer connect. My code does not get called at all and the calls to HttpReceiveHttpRequest don't complete when an HTTPS request comes in.
I created a certificate authority for myself and it is visible inside IE but I can't figure out what to do next.
What do I need to configure to make HTTP.SYS call my code when an HTTPS request comes in?
You'll need to install the SSL cert in the machine store (mmc.exe, add Certificates snap-in, manage the Computer account, import the cert). Then have a go with httpconfig- it's a GUI version of httpcfg/netsh http that's much easier. I have this tool on every server I maintain that has SSL certs. Once that's configured, your SSL server registration should route correctly.