We want to serve protected media from django, using something similar to the django nginx x-accel-redirect setup.
The only problem is that the static files are not located on the public facing django/nginx machine, but in a internal machine that streams the file via http/rest api.
Currently we download the file on the nginx machine and serve it via nginx x-accel-redirect, but we want to optimize this part and looking for options. x-accel-redirect has known problems with files that are streamed from another source.
We are contemplating using django itself as a quasi buffer, but are open to other options as integrating something like whizzer/twisted, or maybe even having another service altogether.
What would be the best option for serving those static files and preserving security?
Use: http://www.allbuttonspressed.com/projects/django-filetransfers
Make your own Django storage backend for the internal machine's http/rest api, that returns
a File object, and pass that object to filetransfer's server_file function.
That's how I do it in Mayan EDMS https://github.com/rosarior/mayan/blob/master/apps/documents/views.py#L300
django-storages' backends could help you get started.
https://bitbucket.org/david/django-storages/wiki/Home
Update:
Django_resto appears to have an http based storage class
https://github.com/aaugustin/django-resto/blob/master/django_resto/storage.py#L62
I had success doing something similar using django-http-proxy. This assumes that the image server is at least as reliable as the django server.
Then in my urls, I simply mapped the url to the http proxy view, something like:
(r'^protected/.*$', 'httpproxy.views.proxy'),
Then configured PROXY_FORMAT accordingly.
Implement a simple one-shot signature system in the media machine, using any very thin (django is OK, as it does not need to get to the database) code layer, and x-accel-redirect in nginx.
In the auth machines, generate the correct signature only when the user is allowed to get the resource, and return a 302 to the signed media.
The signature could be time-based, expiring in a fraction of a second, so a sniffer can't use the URL again.
You could use lighttpd to handle the streaming. It has a nice module to protect resources with signatures: http://redmine.lighttpd.net/wiki/1/Docs:ModSecDownload
So I'm thinking you could have nginx just proxy to the streaming server (that's lighttpd).
It's pretty easy to cook up the signature, here's a python example: init.py#cl-27">https://bitbucket.org/ionelmc/django-secdownload-storage/src/be9b18701015/secdownload_storage/init.py#cl-27
Related
I'm using Django's request.get_host() in the view code to differentiate between a dynamic number of domains.
For example, if a request comes from www.domaina.com, that domain is looked up in a table and content related to it is returned.
I'm running certbot programmatically to generate the LetsEncrypt certificate (including acme challenge via Django). I store the cert files as base64 strings in PostgreSQL.
That works perfectly fine, but I can't figure out how to 'apply' the certificate on a dynamic per-domain basis.
I know that this is normally done using TLS termination, nginx or even in gunicorn. But that's not dynamic enough for my use-case.
The same goes for wildcard or SAN certificates (not dynamic enough)
So the question is:
Given I have valid LetsEncrypt certs, can I use them to secure Django views at runtime?
Django works as a wsgi server. Django gets an http request and does some work of its own. Then it hands it over to middleware and then to your views.
I'm fairly certain that the generic work django does at the start, that it already requires a regular http request, not a "binary blob of unreadable encrypted stuff".
Perhaps gunicorn can handle https termination, but I'm not sure.
Normally, nginx or haproxy is used. Also because it is something that needs to be really secure.
I'm using haproxy now, which has a handy feature that you can just point it at a directory full of *.pem certificate files and it will read them and use them. So if you could write the certs to such a dir and make sure haproxy is reloaded every time a certificate gets changed, you could be pretty close to a dynamic way of working.
I have a two-layer backend architecture:
a "front" server, which serves web clients. This server's codebase is shared with a 3rd party developer
a "back" server, which holds top-secret-proprietary-kick-ass-algorithms, and has a single endpoint to do its calculation
When a client sends a request to a specific endpoint in the "front" server, the server should pass the request to the "back" server. The back server then crunches some numbers, and returns the result.
One way of achieving it is to use the requests library. A simpler way would be to have the "front" server simply redirect the request to the "back" server. I'm using DRF throughout both servers.
Is redirecting an ajax request possible using DRF?
You don't even need the DRF to add a redirection to urlconf. All you need to redirect is a simple rule:
urlconf = [
url("^secret-computation/$",
RedirectView.as_view(url=settings.BACKEND_SECRET_COMPUTATION_URL))),
url("^", include(your_drf_router.urls)),
]
Of course, you may extend this to a proper DRF view, register it with the DRF's router (instead of directly adding url to urlconf), etc etc - but there isn't much sense in doing so to just return a redirect response.
However, the code above would only work for GET requests. You may subclass HttpResponseRedirect to return HTTP 307 (replacing RedirectView with your own simple view class or function), and depending on your clients, things may or may not work. If your clients are web browsers and those may include IE9 (or worse) then 307 won't help.
So, unless your clients are known to be all well-behaving (and on non-hostile networks without any weird way-too-smart proxies - you'll never believe what kinds of insanity those may do to HTTP requests), I'd suggest to actually proxy the request.
Proxying can be done either in Django - write a GenericViewSet subclass that uses requests library - or by using something in front of it, e.g. nginx or Caddy (or any other HTTP server/load balancer that you know best).
For production purposes, as you probably have a fronting webserver, I suggest to use that. This would save implementation time and also a little bit of server resources, as your "front" Django project won't even have to handle the request and keep the worker busy as it waits for the response.
For development purposes, your options may vary. If you use bare runserver then a proxy view may be your best option. If you use e.g. Docker, you may just throw in an HTTP server container in front of your Django container.
For example, I currently have a two-project setup (legacy Django 1.6 project and newer Django 1.11 project, sharing the same database) and a Caddy server in front of those, routing on per-URL basis. With a simple 9-line Caddyfile things just work:
:80
tls off
log / stdout "{common}"
proxy /foo project1:8000 {
transparent
}
proxy / project2:8000 {
transparent
}
(This is a development-mode config.) If you can have something similar, then, I guess, that would be the simplest option.
Does Casablanca (Microsoft's C++ REST SDK) be used to serve web assets (html, images, js, css, etc)?
I built a REST server with Casablanca, and for the specific REST stuff it works great. But how do I serve a full site?
Do I have to run nginx/apache in parallel to my REST server to serve assets, or build some simple file webserver with boost.asio, etc?
I am targeting an embedded system and thus want to keep a low resources demand, so ideally Casablanca it self could serve my assets as well as my routes.
You probably won't want to do that with Casablanca. It's intended for use as a REST endpoint server. It's possible, but it makes more sense to use an instance of Apache/Tomcat to serve a website.
That being said, it is possible. You can give it a try by doing something like the following in your GET Handler:
void HandleGet( http_request Message) {
utility::string_t body =
"<html><body>"
"<h1>Hello World!</h1>"
"</body></html>";
Message.reply( status_codes::OK, body, L"text/html");
}
I am relatively new to javascript, and I got an uploader tool called fineuploader that I was considering to use. However locally (development machine) I got it to work (vb.net), but when I put it on my external server, I noticed that there is a post done in http and directly to the server's domain name (e/g/ mydomain.com), instead of mydomain.com/testproject. The site only allows for https traffic.
Is there an easy way to change this? (so it should point at https://mydomain.com/testproject/FileUpload.aspx
The code used by fineuploader shows a parameter called 'endpoint: '/FileUpload.aspx'
Do I have to make changes in the settings of IIS for this webservice?
If you need to enable CORS (cross-domain requests), Fine Uploader supports this. You should read my blog post on how CORS support is implemented in Fine Uploader and how you can support such requests in your server-side code.
Can I have the soap:address location in a WSDL relative to the WSDL location, or at least relative to the server?
For instance I want to write:
<soap:address location="https://exampleserver.com/axis2/services/ExampleService" />
as:
<soap:address location="/axis2/services/ExampleService" />
This would enable faster deployment to multiple servers, like test servers. Also, in the case of axis2c if I want my service to be used both from HTTP or HTTPS life becomes harder for developers using my service as they can't simply import the WSDL from it's default location "?WSDL".
The WSDL describes to clients the message formats, types, parameters etc needed to interact with the web service. This is then used by tools like WSDL2C to generate the code needed for the interaction.
But even if you expose your service on HTTP or HTTPS, the client stub code will be the same. You don't regenerate your client stubs for each endpoint address. The client stays the same, it's the access point that changes.
This address should not be hardcoded in the generated client code, it must be a configurable URL inside the client application.
Sure, you have an URL specified inside the WSDL and it's a nuisance when you deploy your web service in the dev server, and then to staging and next into production. The endpoints will be different in each environment (maybe multiplied by 2 for HTTP + HTTPS) but at this point your developers are not affected because you don't regenerate the code.
When it comes to access the web service, you would still have different addresses (for dev, staging and prod servers) even if it would be relative to something or absolute. So I don't see how it is helpful to have relative address inside the WSDL since you still have to manage the access points into the client configuration.
There are two ways of getting the WSDL.
One where a hard-coded wsdl is served, for example:
https://hostname/contextname/services/myAPIService/myAPI.wsdl
and another one where a generated wsdl is served, for example:
https://hostname/contextname/services/myAPIService?wsdl
If you use the dynamic option it will use this code:
req.getRequestURL().toString();
to get the URL that will be used in the generated WSDL. This code is in the class ListingAgent (in the package org.apache.axis2.transport.http).
From what you mentioned in your question if you want to have relative location it must be because you want to use it in multiple servers, so you would need to use the dynamic option.
One problem I found with the dynamic options is that if in the original WSDL the location is using HTTP, then in the generated one it will still use HTTP even if you have used HTTPS to access it. (This happens in version 1.5 which is the one my project is using)
Another problem is if you are using a load balancer, because the generated WSDL will be generated with the location of the final server instead of the balancer. An option for this would be to extend the classes AxisServlet and ListingAgent to replace the code mentioned above.
After a long search I'm almost sure that soap:address's location attribute has to be an absolute URL. This gets things more complicated if you work with different environments, such as development, test and production.
Maybe a workaround would be to read, on the client side, the first part of the URL from a config file (e.g. https://exampleserver.com) and the final part from the WSDL (e.g. /axis2/services/ExampleService) and combine them to build an absolute path. The former will allow you to switch among environments.