I would like to be able to compress the data sent from a client to a django server.
The data I need to compress is mainly binary, but some endpoints receive also large jsons with binary data encoded to base64.
I've found lots of info regarding the opposite way (server returning compressed response) for example https://docs.djangoproject.com/en/1.2/ref/middleware/#module-django.middleware.gzip, however this is not what I need.
What I'm basically asking is:
Is there a standard/recommended way for compressing client requests? gzip, LZW, Huffman...
How do I support this in Django?
Responses:
1 - You could compress the data on the client and then send it to the server. I've seen Javascript libraries to compress to ZIP. The data can be uncompressed on the server.
2 - Uncompress the request parameter using Python zip library.
Related
I enabled brotli compression on server side in vertex by added a compressor with StandardCompressionOptions.brotli() and enabled setCompressionSupported(true) on server option before creating server.
But In response I am getting compressed data appended to uncompressed data in response.
This is happening for brotli compression only.
How can get rid of uncompressed data coming with compressed data
I was asked this question durning an interview: How to send large data to a phone?
I've heard words like "long pulling" or "streaming" but I don't know how to do it. Finally I said you can split the data to smaller chunks and send it over HTTP. The interviewer said "Well that was interesting...". I guess that was not a good sign.:(
Anyway, what is the best practice for sending large data file to a mobile device?
I've used the following approach in recent project:
Mobile device requests data from server via web service and starts to send requests with time interval to check if server is ready with data
Server prepares large response, packs it to ZIP file (compression is pretty good for text files with raw data - 5MB goes to 100Kb) and places this file to a specified folder
Mobile device gets a response from server that data is ready. This response contains a link to file.
Mobile device downloads the file, unpacks it - you got your data on mobile!
The same approach is used to send data from mobile to server.
Wonder what the community says about the most efficient (in terms of I/O and speed) solution for delivering multiple files back from a single request to a webservice would be. The client is not a web browser.
The options I see so far:
creating a zip archive and streaming it back to the client.
base64 encoding files an returning array of strings that would need to be decoded by the client.
Using Mime multipart/related and sending Mime headers for each file in iteration, also potentially streamed back to the client.
Maybe there are others I haven't considered?
CLARIFICATION:
Let's assume the files may be in the 10s of Megabytes, and that memory is around 4G but there are likely other processes and/or simultaneous requests.
I think you need to consider the bindings (streaming) and transports protocols (SOAP, REST). How large is the average file?
We have a web service that sends the video content in the response as binary (in different formats asx, asf, ram, mpeg, mpg, mpe, qt, mov, avi, movie, wmv, smil, mp4, mxf, gxf, flv, 3gp, f4v, mj2, omf, dv, vob).
Do you see any issue with performance, if I have an intermediate application which makes a request to web service to retrieve video content and render in browser?
Thanks
As long as the web service returns binary data directly, then there will be no performance hit. If this is an XML or SOAP web service that wraps the whole thing in a SOAP envelope and bae64 encodes it to make it all text, then you will not be able to play it directly and it will have a big impact on bandwidth, cpu, and memory.
Also note that by serving the video directly instead of using a true streaming protocol the user will only be able to seek within the portion downloaded so far. A streaming protocol like RTSP, RTMP, or the many varieties of HTTP Streaming allow seeking to any part of the file and only downloading the part seeked to.
Is it possible to POST request data from browser to server in compressed format?
If yes, How can we do that?
Compressing data sent from the browser to the server is not natively supported in the browsers.
You'll have to find a workaround, using a clientside language (maybe a JavaScript GZip implementation, or a Java Applet, or ...). Be sure to visually display to the user what the browser is doing and why it is taking some time.
I don't know the scope of your application, but on company websites you could just restrict input to compressed files. Ask your users to upload .zip/.7z/.rar/... files.
The server->client responses can be gzip compressed automagically by the server.
Compressing the client->server messages is not standard, so will require some work by you. Take your very large POST data and compress it client-side, using JavaScript. Then decompress it manually on the server side.
This will usually not be a beneficial thing to do unless your bandwidth usage is a major bottleneck. Compression requires both time and CPU usage to perform.