I am kinda newby in python thus the question.
I am trying to create a simple http web server that can receive chunked data from a post request.
I have realized later that once a request sends a headers with chunked data, the Content-length headers will be set to zero, thus reading the sent data with 'request.get_data()' will fail.
Is there another way of reading the chunked data?
The request I receive does give me the data length in the 'X-Data-Length' headers.
Did you write both of the js upload file code and the flask in backend to handle upload request? If not then you will need some help with js to upload it.
One way to achieve chucked data upload is:
Chucked that file in the frontend with js. Give it some headers in the request for the total size, number of the chunk, chunk size... and send each chuck in a separate POST request (You can use dropzone.js for example, they will do the job for you, just need to config the params)
In the backend, create an upload API which will read the request headers and merge the file chunks back together
Related
I have a problem with sending files to some API. When I do it via my server some pdf files send correctly but some not. But when I send the same pdf via POSTMAN everything works just fine. I am not sure what is the reason (probably POSTMAN encodes it into base64 but from my server, I send it in binary - just a guess). Is there a way to get the sent request file content from postman so that I can imitate it and compare with the request on my server?
Hello I'm working on an app where user can upload their own video (yes like Tiktok). I going with the direct upload with pre-signed using PUT route for maximizing upload speed.
My question is how do I really limit the file size that my user can upload for a video?
A solution I came up with is to use lambda to check for content-length of the object that got upload. But as far as I know that content-length properties was being set in the header of the client. What if the content-length in the header and the actual file size was upload to s3 did not match up? Is it even possible that someone can do that with PUT upload?
A solution I came up with is to use lambda to check for content-length
of the object that got upload.
An alternative is to set content-length-range matching POST policy.
What if the content-length in the header and the actual file size was
upload to s3 did not match up
Not sure why this could happen, but other than using content-length property, multipart upload is worth considering too. It is just a series of regular requests. You initiate a multipart upload, send one or more requests to upload parts, and then complete the multipart upload process. You sign each request individually.
A Node.js plugin that supports S3-streaming-upload.
How can I attach a file to the environment in order to launch the collection in the Postman collection runner? The file is attached to the form-data, and manually the request runs ok, but as soon as I run it in the collection runner, the runner doesn't see the file and the 500 Internal server error occurs. How can this issue be fixed?
Request BodySee the full detail in the link
While constructing requests, you’ll work frequently with the request body editor. Postman lets you send almost any kind of HTTP request. The body editor is divided into 4 areas and has different controls, depending on the body type.
Note about Headers: When you are sending requests through the HTTP protocol, your server might expect a Content-Type header. The Content-Type header allows the server to parse the body properly. For form-data and urlencoded body types, Postman automatically attaches the correct Content-Type header so you don’t have to set it. The raw mode header is set when you select the formatting type. If you manually use a Content-Type header, that value takes precedence over what Postman sets. Postman does not set any header type for the binary body type.
Form-data
form-data
multipart/form-data is the default encoding a web form uses to transfer data. This simulates filling a form on a website, and submitting it. The form-data editor lets you set key-value pairs (using the data editor for your data. You can attach files to a key as well. Note: due to restrictions of the HTML 5 spec, files are not stored in history or collections. You will need to select the file again the next time you send the request.
Uploading multiple files each with their own Content-Type is not supported yet.See the image here in the link
I've managed to make a file downloader in C++ (using winsock). It downloads every simple link with a file like: www.page.com/image.png
I want to make it download all of the images from an entire page, such as all the images from a 4chan thread, but I don't know what I should send in the http request to get the page's source. How can I request the source of a webpage?
You don't send anything in the http request, in the manner you're thinking.
An http request sends a single request, for a single document, and returns a single document from the server.
To download an entire page, you will have to parse the downloaded HTML document, extract all the relative links from the HTML source, then issue a separate http request for every image, css, js, etc... referenced from the main document.
This is how tools like wget's --recursive option download entire pages.
If the page is located at the root of the http://www.page.com server, you would send a GET request to the www.page.com server asking for the / resource:
GET / HTTP/1.1
Host: www.page.com
Let's say the page was actually located at http://www.page.com/thepage.html. You would send a GET request asking for /thepage.html instead:
GET /thepage.html HTTP/1.1
Host: www.page.com
Either way, you would then have to parse the resulting HTML to get the individual URLs of all the <img> tags that are on the page.
Hi I want to refuse incoming requests with too large body or header in my Jetty.I suppose that I have to set some filter, but I haven't found any solution. Do you have any suggestions? Thanks.
Easy enough to build a Servlet Filter or Jetty Handler that pays attention to the request's Content-Length header and then rejects (responds with an http error status code) the request.
As for the header size limit, that's controlled by the HttpConfiguration.setRequestHeaderSize(int)
However, there are a class of requests, that uses Chunked Transfer-Encoding, with these kinds of requests, there is no Content-Length and you will just have to reject the request when reading from the HttpServletRequest.getInputStream() after it hits a certain size.
There is also the complication of Mime multi-part request body content and how you determine the request content is too large.
One other note, unfortunately, due to how HTTP connection handling must be performed, even if a client sends you too large of a request body content, the server still has to read that entire body content and throw it away. This is the half-closed scenario found in the spec, its up to the client to see the early rejected http response and close/terminate the connection.