Serving 206 Byte-Range through Nginx, Django - django

I have Nginx serving my static Django files which is being run on Gunicorn. I am trying to serve MP3 files and get them to have the head 206 so that they will be accepted by Apple for podcasting. At the moment the audio files are in my static directory and are served straight through Nginx. This is the response i get:
HTTP/1.1 200 OK
Server: nginx/1.2.1
Date: Wed, 30 Jan 2013 07:12:36 GMT
Content-Type: audio/mpeg
Content-Length: 22094968
Connection: keep-alive
Last-Modified: Wed, 30 Jan 2013 05:43:57 GMT
Can someone help with the correct way to serve mp3 files so that byte-ranges will be accepted.
Update: This is the code in my view that serves the file through Django
response = HttpResponse(file.read(), mimetype=mimetype)
response["Content-Disposition"]= "filename=%s" % os.path.split(s)[1]
response["Accept-Ranges"]="bytes"
response.status_code = 206
return response

If you want to do this only in nginx then in your location directive which is responsible for serving static .mp3 files add those directives:
# here you add response header "Content-Disposition"
# with value of "filename=" + name of file (in variable $request_uri),
# so for url example.com/static/audio/blahblah.mp3
# it will be /static/audio/blahblah.mp3
# ----
set $sent_http_content_disposition filename=$request_uri;
# or
add_header content_disposition filename=$request_uri;
# here you add header "Accept-Ranges"
set $sent_http_accept_ranges bytes;
# or
add_header accept_ranges bytes;
# tell nginx that final HTTP Status Code should be 206 not 200
return 206;

There is something in your config that prevent nginx from supporting range requests for these static files. When using standard nginx modules, this may be one of the following filters (these filters modify responses, and byte-range handling is disabled if modification may happen during request body handling):
gzip
gunzip
addition filter
ssi
All these modules have directives to control MIME types they work with (gzip_types, gunzip_types, addition_types, ssi_types). By default, they are set to restrictive sets of MIME types, and range requests works fine for most static files even if these modules are enabled. But placing something like
ssi on;
ssi_types *;
into a configuration will disable byte-range support for all static files affected.
Check your nginx configuration and remove offending lines, and/or make sure to switch off modules in question for a location you serve your mp3 files from.

You can define your own status code:
response = HttpResponse('this is my response data')
response.status_code = 206
return response
If you are using Django 1.5 you may want to have a look at the new StreamingHttpResponse:
https://docs.djangoproject.com/en/dev/ref/request-response/#streaminghttpresponse-objects
This can be very helpful for big files.

Related

Remove HTTP headers from Prometheus in Zabbix

I have a server that has a Nginx VTS module installed on it, which outputs metrics in prometheus format.
When I try to actively check web.page.get via Zabbix I get the HTTP header and then the data in the format below:
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 24 Sep 2020 09:16:20 GMT
Content-Type: text/plain
Content-Length: 33769
Connection: close
Vary: Accept-Encoding
# HELP nginx_vts_info Nginx info
# TYPE nginx_vts_info gauge
nginx_vts_info{hostname="example",version="1.18.0"} 1
# HELP nginx_vts_start_time_seconds Nginx start time
# TYPE nginx_vts_start_time_seconds gauge
nginx_vts_start_time_seconds 1600367492.145
# snip output...
I wrote a regular expression that removes the header but only outputs the first line:
# \n\s?\n(.*)
# HELP nginx_vts_info Nginx info
How do I rewrite the expression so that the header is removed and the rest of the data is available?
Please try below regex
\n\s?\n([\s\S]*)
in regex . wont check newlines unless specific flags set. hence in your example, only the first line was returned. so rewriting it to include newlines as well will help.

Django+Angular CORS not working with POST

My Angular4 app (running on http://127.0.0.1:4200 development server) is supposed to access a django REST backend on the web. The backend is under my control and is available only via HTTPS (running Apache that tunnels the request to a gunicorn server running on an internal port). Let's say that this is https://example.com/. For historical reasons, logging the user in is done using sessions, because I want the users to be able to also use Django's admin interface after they logged in. The workflow is as follows:
Users opens http://127.0.0.1:4200, I perform a GET request to https://example.com/REST/is_logged_in which returns a 403 when the user isn't logged in via sessions yet, 200 otherwise. In the former case, the user is redirected to https://example.com/login/, rendered by Django's template engine, allowing the user to log in. Once logged in, the user is redirected to http://127.0.0.1:4200
When clicking on some button in my Angular UI, a POST request is performed. This post request fails with 403, even though the preflight OPTIONS request explicitly lists POST as allowed actions.
Here is my CORS configuration in Django:
NG_APP_ABSOLUTE_URL = 'http://127.0.0.1:4200'
# adapt Django's to Angular's presumed XSRF cookie/header names
CSRF_COOKIE_NAME = "XSRF-TOKEN"
CSRF_HEADER_NAME = "HTTP_X_XSRF_TOKEN"
CORS_ORIGIN_WHITELIST = (
urlparse(NG_APP_ABSOLUTE_URL).netloc
)
CSRF_TRUSTED_ORIGINS = (
urlparse(NG_APP_ABSOLUTE_URL).netloc
)
CORS_ALLOW_HEADERS = default_headers + (
'x-xsrf-token',
)
CORS_ALLOW_CREDENTIALS = True
This is what Chrome reports for the (successful, 200) first REST GET request to check whether the user is logged in (after he successfully did) in the response:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Origin:http://127.0.0.1:4200
Allow:GET, HEAD, OPTIONS
Connection:close
Content-Type:application/json
Date:Wed, 26 Apr 2017 15:09:26 GMT
Server:gunicorn/19.6.0
Set-Cookie:XSRF-TOKEN=...; expires=Wed, 25-Apr-2018 15:09:26 GMT; Max-Age=31449600; Path=/
Transfer-Encoding:chunked
Vary:Accept,Cookie,Origin
X-Frame-Options:SAMEORIGIN
The corresponding request had this:
Cookie:sessionid=...; XSRF-TOKEN=...
Host:example.com
Origin:http://127.0.0.1:4200
Referer:http://127.0.0.1:4200/
Now, to the actual problem:
Preflight request:
Request URL:https://example.com/REST/change_user_data/
Request Method:OPTIONS
Status Code:200 OK
Access-Control-Request-Headers:content-type
Access-Control-Request-Method:POST
Connection:keep-alive
Host:example.com
Origin:http://127.0.0.1:4200
Referer:http://127.0.0.1:4200/dashboard/account
Preflight response:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Headers:accept, accept-encoding, authorization, content-type, dnt, origin, user-agent, x-csrftoken, x-requested-with, x-xsrf-token
Access-Control-Allow-Methods:DELETE, GET, OPTIONS, PATCH, POST, PUT
Access-Control-Allow-Origin:http://127.0.0.1:4200
Access-Control-Max-Age:86400
Connection:close
Content-Length:0
Content-Type:text/html; charset=utf-8
Date:Wed, 26 Apr 2017 15:36:56 GMT
Server:gunicorn/19.6.0
Vary:Origin
X-Frame-Options:SAMEORIGIN
Now my failing (403) POST request:
Accept:application/json
Accept-Encoding:gzip, deflate, br
Accept-Language:de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Connection:keep-alive
Content-Length:60
Content-Type:application/json
Cookie:sessionid=...; XSRF-TOKEN=...
Host:example.com
Origin:http://127.0.0.1:4200
Referer:http://127.0.0.1:4200/dashboard/account
The response headers:
HTTP/1.1 403 Forbidden
Date: Wed, 26 Apr 2017 15:36:56 GMT
Server: gunicorn/19.6.0
Vary: Accept,Cookie,Origin
X-Frame-Options: SAMEORIGIN
Content-Type: application/json
Access-Control-Allow-Credentials: true
Allow: POST, OPTIONS
Access-Control-Allow-Origin: http://127.0.0.1:4200
Set-Cookie: XSRF-TOKEN=...; expires=Wed, 25-Apr-2018 15:36:56 GMT; Max-Age=31449600; Path=/
Connection: close
Transfer-Encoding: chunked
Why wouldn't this request work? It makes little sense to me!
Best regards!
I had the same problem, trying to send a POST request to Django (port 8000) from my Angular CLI (port 4200). I thought it was a problem of Django so I installed cors package however the "problem" is with the browser (actually is not a problem, it is a security issue, see here). Anyway, I solved the problem adding a proxy rule for my Angular CLI, as follows:
First, instead of sending my requests to http://localhost:8000/api/... is send them to /api/ (i.e. to my ng server running at port 4200).
Then I added a file in my Angular project called "proxy.conf.json" with the following content:
{
"/api": {
"target": "http://localhost:8000",
"secure": false
}
}
Finally, run your ng server with the flag "--proxy-config":
ng serve --watch --proxy-config proxy.conf.json
All API requests will be sent to the port 4200 and Angular will internally redirect them to Django, avoiding the CORS problem.
Note that this is only valid for development and won't be used when you build your app code and add it as the static code of your Django server.
Finally, with this solution I didn't need anymore the python module for cors so you could remove it.

How can I use SECURE_SSL_REDIRECT with a Cache-Control header?

I've enabled SECURE_SSL_REDIRECT in the settings of my django deployment, so now these headers are getting sent to the client:
< HTTP/1.1 301 MOVED PERMANENTLY
< Date: Fri, 19 Feb 2016 15:57:50 GMT
< Server: Apache/2.2.15 (Red Hat)
< Location: https://www.example.com/
< Content-Length: 0
< Content-Type: text/html; charset=utf-8
The main disadvantage with 301 redirects is that they tend to be cached for a very long time by browsers, so I would quite like to add a Cache-Control: max-age=604800, must-revalidate header to this. Preferably, I would like a way that doesn't involve re-implementing SECURE_SSL_REDIRECT.
You can try overriding Django's SecurityMiddleware to add the http headers you need. Below is a full implementation of the middleware:
class CustomSecurityMiddleware(SecurityMiddleware):
def process_request(self, request):
response = super(CustomSecurityMiddleware, self).process_request(request)
# SecurityMiddleware returns an HttpResponsePermanentRedirect only if
# the request should be redirected
if response is not None:
response['Cache-Control'] = 'max-age=604800, must-revalidate'
return response
This implementation retains everything that Django's SecurityMiddleware does already, while adding the custom http headers you need.
The custom middleware should replace SecurityMiddleware in settings.MIDDLEWARE_CLASSES.

nginx : meaning of "expires 30d"

[Update]
expires 30d : Static file cache expires after 30 days on client's browser
etag on : This attribute is only available after version 1.3.3. Each static file has 'etag hash value'. Client will make a request for server if the static file is changed (Even though not expired yet).
===================================================================
Here's a sample of nginx.conf file for django project
server {
listen 80;
server_name hostname.com;
...
location /static/ { # STATIC_URL
alias /path/to/static/; # STATIC_ROOT
expires 30d;
}
location /media/ { # MEDIA_URL
alias /path/to/media/; # MEDIA_ROOT
expires 30d;
}
...
}
In this code, what is the meaning of "expires 30d" ?
(1) static, media file would be deleted after 30 days, and manage.py will regenerate them automatically.
(2) static, media file would be deleted after 30 days, and I should type manage.py collectstatic manually.
Similarly, I wonder the meaning of "expires max"
This adds two HTTP headers to the responses (Expires and Cache-Control). Those headers are used by the browsers to cache content, avoid doing the same requests for static content each time a page is loaded.
expires 30d means that all content in static and media folders will be cached by browsers during 30 days, but nothing will be deleted and you won't need to regenerate anything in the server.
expires max set the Expires header to the value "Thu, 31 Dec 2037 23:55:55 GMT", and the Cache-Control to 10 years.
See the nginx documentation for more details: http://nginx.org/en/docs/http/ngx_http_headers_module.html
For more info about HTTP caching see http://www.mobify.com/blog/beginners-guide-to-http-cache-headers/

Django Cache-control header: Why isn't Chrome caching this resource?

I have decorated a Django view with cache_control as follows:
#cache_control(
private=True,
max_age=5 * 60, # 5 minutes
)
def my_view(req):
…
When I try it with the local test server, it works as expected: subsequent page views in Chrome use the cached resource and don't make a request. When deployed in production, though, Chrome seems to ignore the Cache-control header and makes a new request every time I hit that page.
Here's the full list of headers that the production server responds with:
Cache-Control:private, max-age=300
Connection:close
Content-Encoding:gzip
Content-Length:13135
Content-Type:text/html; charset=utf-8
Date:Wed, 22 Jan 2014 20:39:29 GMT
P3P:CP="IDC CURa ADMa OUR IND PHY ONL COM STA"
Server:nginx/1.4.1
Set-Cookie:csrftoken=87y26bT5uPmyA9wt51N7m4blyqBH5nSo; expires=Wed, 21-Jan-2015 20:39:29 GMT; Max-Age=31449600; Path=/
Vary:Cookie,Accept-Encoding
What could be going wrong? Any ideas? Thanks in advance!
Got it: it was a combination of Google Analytics' cookie and the Vary:Cookie header (set by Django's SessionMiddleware). Analytics' cookie changes with each request, but since ga.js doesn't load when working on localhost, the problem only showed up in production.