I am working in socket programming with help of C++.And I have to write a code to download file.
My Http Header for this is :
char header[] = "HTTP/1.1 200 OK\r\nContent-Type:application/vnd.ms-excel;Content-Disposition:attachment;filename:\"abc.xls\";Content-Length:14; \r\n\r\n";
but file get downloaded as "download" it does not have extension too.
I also tried this with
char header[] = "HTTP/1.1 200 OK\r\nContent-Type:application/octet-stream;Content-Disposition:attachment;filename:\"abc.xls\";Content-Length:14; \r\n\r\n";
but its not working.
Can anyone help me regarding this.?
The HTTP response header you've tried looks like this:
HTTP/1.1 200 OK
Content-Type:application/vnd.ms-excel;Content-Disposition:attachment;filename:"abc.xls";Content-Length:14;
instead it should look like this
HTTP/1.1 200 OK
Content-Type:application/vnd.ms-excel
Content-Disposition:attachment;filename="abc.xls"
Content-Length:14
I recommend you study the relevant standards before trying to implement a protocol. In this case this is the HTTP standard (RFC 7230 and following RFC) and "Use of the Content-Disposition Header Field in the
Hypertext Transfer Protocol (HTTP)" (RFC 6266).
Related
I am working on a C++ project where i listen on sockets and generate HTTP responses based on the requests i get from my clients on my fds, in short i use my browser to send a request, i end up getting the raw request, i parse it and generate the corresponding http response.
However in the case of large POST requests, usually what happens is that i get partial requests, so in the first part i will usually only find the first line (version/method/uri), some headers but no body, and i guess am supposed to get the rest of the body somehow, however i am unable to figure out two things,
first of all how do i know if the request i am getting is partial or completed from just the first part ? i am not getting any information relating to range, here's the first part i get when my client sends me a POST request.
POST / HTTP/1.1
Host: localhost:8081
Connection: keep-alive
Content-Length: 8535833
Cache-Control: max-age=0
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="96", "Google Chrome";v="96"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Origin: http://127.0.0.1:8081
Upgrade-Insecure-Requests: 1
DNT: 1
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryOs6fsdbaegBIumqh
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: navigate
Sec-Fetch-User: ?1
Sec-Fetch-Dest: document
Referer: http://127.0.0.1:8081/
Accept-Encoding: gzip, deflate, br
Accept-Language: fr,en-US;q=0.9,en;q=0.8
how can i figure out just from this whether or not am getting a partial request or just a faulty request (I need to generate a 400 error in the case of a request that says it has X content-length but the body size is different)
second question is, suppose i already know whether or not its partial, how do i proceed with storing the entire request in a buffer before sending it to my parser and generating a response ? here's my reception function (i already know the client's fd, so i just recv on it
void Client::receive_request(void)
{
char buffer[2024];
int ret;
ret = recv(_fd, buffer, 2024, 0);
buffer[ret] = 0;
_received_request += buffer;
_bytes_request += ret;
std::cout << "Raw Request:\n" << _received_request << std::endl;
if (buffer[ret-1] == '\n')
{
_ready_request = true;
_request.parse(_received_request, _server->get_config());
}
}
and here's the code that checks whether or not a client is attempting to send a request, parse and generate a response
int Connections::check_clients() {
int fd;
for (std::vector<Client*>::iterator client = clients.begin();
client != clients.end() && ready_fd != 0 ; client++)
{
fd = (*client)->get_fd();
if (FD_ISSET(fd, &ready_rset))
{
ready_fd--;
(*client)->receive_request();
if ((*client)->request_is_ready())
{
(*client)->wait_response();
close(fd);
FD_CLR(fd, &active_set);
fd_list.remove(fd);
max_fd = *std::max_element(fd_list.begin(), fd_list.end());
free(*client);
client = clients.erase(client);
}
}
}
return 0;
}
as you can see am coding everything in C++ (98) and would rather not get answers that just dismiss my questions and refer me to different technologies or libraries, unless it will help me understand what am doing wrong and how to handle partial requests.
for info, am only handling HTTP 1.1(GET/POST/DELETE only) and i usually only get this issue when am getting a large chunked file or a file upload that has a very large body. thank you
PS : if needed i can link up the github repo of the current project if you wanna look further into the code
how can i figure out just from this whether or not am getting a partial request or just a faulty request (I need to generate a 400 error in the case of a request that says it has X content-length but the body size is different)
The body size is, by definition, the size of the Content-Length field. Any bytes that you receive afterwards belong to the next HTTP request (see HTTP pipelining). If you do not receive Content-Length bytes within a reasonable time period, then you can make the server issue a 408 Request Timeout error.
second question is, suppose i already know whether or not its partial, how do i proceed with storing the entire request in a buffer before sending it to my parser and generating a response ? here's my reception function (i already know the client's fd, so i just recv on it
Your posted code has at least the following problems:
You should check the return value of recv to determine whether the function succeeded or failed, and if it failed, you should handle the error appropriately. In your current code, if recv fails with the return value -1, then you will write to the array buffer out of bounds, causing undefined behavior.
It does not seem appropriate to use the line if (buffer[ret-1] == '\n'). The HTTP request header will be over when you encounter a "\r\n\r\n", and the HTTP request body will be over when you have read Content-Length bytes of the body. The ends of the header and body will not necessarily occur at the end of the data read by recv, but can also occur in the middle. If you want to support HTTP pipelining, then the additional data should be handled by the handler for the next HTTP request. If you don't want to support HTTP pipelining, then you can simply discard the additional data and use Connection: close in the HTTP response header.
You seem to be using a null terminating character to mark the end of the data read by recv. However, this will not work if a byte with the value 0 is part of the HTTP request. It is probably safe to assume that such a byte should not be part of the HTTP request header, but it is probably not safe to assume that such a byte won't be part of the HTTP request body (for example when using POST with binary data).
EDIT5:
I eventually fixed this issue by more or less throwing away half my code. Rather than sending data to a ruby server using HTTP, I'm now using MQTT to a broker to a NodeJS server. The NodeJS part here isn't important but to anyone else having this issue I STRONGLY recommend sending all IoT data using MQTT, and that's what solved my issue.
I'm currently trying to send data collected from sensors on an Arduino WiFi rev2, to my rails server hosted on Heroku. I do this by sending my data in a JSON format. My problem is that while my methods seem to work initially, with the first few POST requests being received and processed fine, after 2-3 requests the arduino hangs, and I receive status code: -2. I'm using the ArduinoHttpClient library.
I've tried using a local server, which has the same problem, aswell as sending the POST request via both curl and postman. Both curl and postman seem to work as expected, so I imagine the issue is with the arduino code although I can't be sure.
client.beginRequest();
client.post("/input");
client.sendHeader("Content-Type", "application/json");
client.sendHeader("Content-Length", postData.length());
client.beginBody();
client.println(postData);
client.endRequest();
LED(0,128,0);
Serial.println("Gone");
int statusCode = client.responseStatusCode();
String response = client.responseBody();
Serial.print("Status code: ");
Serial.println(statusCode);
Serial.print("Response: ");
Serial.println(response);
When this code fails, the arduino will hang for about 20-40 seconds and I will receive 'status code -3' from the Serial. However I have also received status code -2 and -4 in the past.
When it does succeed I receive the following: "Status code: 204" which is what I would expect.
EDIT:
I've since tried posting to requestcatcher.com, and the problem persisted. I'm therefore fairly confident this is an arduino problem, I also received the following output:
POST /input HTTP/1.1
Host: arduino.requestcatcher.com
Connection: close
Connection: close
Content-Length: 88
Content-Type: application/json
User-Agent: Arduino/2.2.0
{"inputs":[{"input_id":"1","value":1.778002}{"input_id":"2","value":18.037}],"id":"13"}
EDIT 2:
I accidentally discovered that the POST requests go through fine if the "Content-Length:" Header is omitted. Obviously no JSON actually gets sent so this does not fix my issue but it is likely that this header or the JSON itself is the issue.
EDIT 3:
Regardless of server I receive either status code -4, or -3, even on request catcher.
EDIT 4:
After various adjustments, code now looks as below. This seems to have helped a little and it fails less often but still does fail. I'm beginning to wonder if this is a problem with ArduinoHttpClient.
String postData = "";
serializeJson(doc, postData);
serializeJson(doc, Serial);
Serial.println(postData)
client.post("/input", "application/json", postData.c_str());
LED(0,128,0);
Serial.println("Gone");
int statusCode = client.responseStatusCode();
Serial.print("Status code: ");
Serial.println(statusCode);
client.stop();
doc.clear();
lastCycle = millis();
Try replacing
client.beginRequest();
client.post("/input");
client.sendHeader("Content-Type", "application/json");
client.sendHeader("Content-Length", postData.length());
client.beginBody();
client.println(postData);
client.endRequest();
with just
String contentType = "application/json";
client.post("/input", contentType, postData);
or
client.post("/input", "application/json", postData.c_str());
You don't need to explicitly specify the request headers - or call beginRequest(), etc. - when using the post() method(s) in that library.
I have an app that appears to enable gzip encoding by default while sending data to the server.
We tried disabling the gzip compression by explicitly using:
IXMLHttpRequest2::SetRequestHeader(L"Accept-Encoding", L"") (on the HTTP Request Object, of course)
This still doesn't seem to help. Is there anyway to disable GZIP being enabled in the HTTP-Request headers from the C++ App?
Thanks!
To ask a server to do not use a specific encoding you should provide a list of Accept-Encoding values. From section 14.11 of RFC2616 (HTTP/1.1) you see that it has one of forms (values are examples):
Accept-Encoding: compress, gzip
Accept-Encoding:
Accept-Encoding: *
Accept-Encoding: compress;q=0.5, gzip;q=1.0
Accept-Encoding: gzip;q=1.0, identity; q=0.5, *;q=0
If the content-coding is one of the content-codings listed in Accept-Encoding field, then it is acceptable, unless it is by a qvalue of 0. (As defined in section 3.9, a of 0 means "not acceptable.")
Then to ask the server to do not use gzip compression you should provide, instead of an empty string, this value for Accept-Encoding:
gzip;q=0
This will require the server to do not use it and but you have to provide another encoding. See section 3.5 for available encodings. Use the quality q parameter to inform the server about your preferences (do not forget that if it can't provide that encoding for your request it'll reply with error 406).
identity;q=1.0, gzip=0.5
In this way you ask to use identity encoding and, in case it's not available, you can accept a gzip encoding too (this will prevent the server to reply with an error if it, for any reason, can't use any other encoding for your request). You may try performance of other encodings too (compress and deflate, for example).
Code
Then, finally, you have to use IXMLHttpRequest2::SetRequestHeader(L"Accept-Encoding", L"identity;q=1.0, gzip=0.5"). In SetRequestHeader you see that it's an append to headers sent by default so if you specify an empty string actually the value won't be changed (actually how it is interpreted may depends on the server, I didn't find any proper specification about this, you may inspect both your HTTP request and response to check what is actually sent/received).
Old value: Accept-Encoding: compress
Call: IXMLHttpRequest2::SetRequestHeader(L"Accept-Encoding", L"")
New value: Accept-Encoding: compress
I am trying to make a simple http server with C++. I've followed the beej's guide of network programming in C++.
When I ran the server in some port (8080, 2127, etc.) it successfully send response to browser (Firefox) when it accessed via address bar with: localhost:PORT_NUMBER except in port 80.
This is the code i wrote:
printf("Server: Got connection from %s\n", this->client_ip);
if(!fork()) // This is the child process, fork() -> Copy and run process
{
close(this->server_socket); // Child doesn't need listener socket
// Try to send message to client
char message[] = "\r\nHTTP/1.1 \r\nContent-Type: text/html; charset=ISO-8859-4 \r\n<h1>Hello, client! Welcome to the Virtual Machine Web..</h1>";
int length = strlen(message); // Plus 1 for null terminator
int send_res = send(this->connection, message, length, 0); // Flag = 0
if(send_res == -1)
{
perror("send");
}
close(this->connection);
exit(0);
}
close(this->connection); // Parent doesn't need this;
The problem is, even I have added the header on very early of the response string, why does the browser not showing the HTML properly instead shows only plain text? It shows something like this:
Content-Type: text/html; charset=ISO-8859-4
<h1>Hello, client! Welcome to the Virtual Machine Web..</h1>
Not a big "Hello, client!.." string like a normally h1 tagged string. What is the problem? Am I missing something in the header?
Another question is, why is the server won't running in port 80? The error log in server says:
server: bind: Permission denied
server: bind: Permission denied
Server failed to bind
libc++abi.dylib: terminate called throwing an exception
Please help. Thank you. Edit: I'dont have any process on Port 80.
You need to terminate the HTTP response header with \r\n\r\n, rather than just \r\n. It should also start with something more like HTTP/1.1 200 OK\r\n, without the leading \r\n.
For your port problem, if you have nothing else running on the port in question, you may find that the socket created by the last run of your program is still sticking around. To work around this, you can use setsockopt to set the SO_REUSEADDR flag on the socket. (This is not recommended for general use, I believe because you may receive data not intended for your program, but for development it's extremely handy.)
Your request starts with \r\n when it shouldn't also it did not specify a status code and you need a blank line after all the headers.
char message[] = "HTTP/1.1 200 Okay\r\nContent-Type: text/html; charset=ISO-8859-4 \r\n\r\n<h1>Hello, client! Welcome to the Virtual Machine Web..</h1>";
As for your port 80 issue, some other application maybe bound to it.
you need to add "Content-length: ", and the length is your HTML code, just like this:
char msg[] = "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nContent-length: 20\r\n\r\n<h1>Hello World</h1>";
Symptom
I think, I messed up something, because both Mozilla Firefox and Google Chrome produce the same error: they don't receive the whole response the webserver sends them. CURL never misses, the last line of the quick-scrolling response is always "</html>".
Reason
The reason is, that I send response in more part:
sendHeaders(); // is calls sendResponse with a fix header
sendResponse(html_opening_part);
for ( ...scan some data... ) {
sendResponse(the_data);
} // for
sendResponse(html_closing_part)
The browsers stop receiving data between sendResponse() calls. Also, the webserver does not close() the socket, just at the end.
(Why I'm doing this way: the program I write is designed for non-linux system, it will run on an embedded computer. It has not too much memory, which is mostly occupied by lwIP stack. So, avoid collecting the - relativelly - huge webpage, I send it in parts. Browsers like it, no broken HTML occurred as under Linux.)
Environment
The platform is GNU/Linux (Ubuntu 32-bit with 3.0 kernel). My small webserver sends the stuff back to the client standard way:
int sendResponse(char* data,int length) {
int x = send(fd,data,length,MSG_NOSIGNAL);
if (x == -1) {
perror("this message never printed, so there's no error \n");
if (errno == EPIPE) return 0;
if (errno == ECONNRESET) return 0;
... panic() ... (never happened) ...
} // if send()
} // sendResponse()
And here's the fixed header I am using:
sendResponse(
"HTTP/1.0 200 OK\n"
"Server: MyTinyWebServer\n"
"Content-Type: text/html; charset=UTF-8\n"
"Cache-Control: no-store, no-cache\n"
"Pragma: no-cache\n"
"Connection: close\n"
"\n"
);
Question
Is this normal? Do I have to send the whole response with a single send()? (Which I'm working on now, until a quick solution arrives.)
If you read RFC 2616, you'll see that you should be using CR+LF for the ends of lines.
Aside from that, open the browser developer tools to see the exact requests they are making. Use a tool like Netcat to duplicate the requests, then eliminate each header in turn until it starts working.
Gotcha!
As #Jim adviced, I've tried sending same headers with CURL, as Mozilla does: fail, broken pipe, etc. I've deleted half of headers: okay. I've added back one by one: fail. Deleted another half of headers: okay... So, there is error, only if header is too long. Bingo.
As I've said, there're very small amount of memory in the embedded device. So, I don't read the whole request header, only 256 bytes of them. I need only the GET params and "Host" header (even I don't need it really, just to perform redirects with the same "Host" instead of IP address).
So, if I don't recv() the whole request header, I can not send() back the whole response.
Thanks for your advices, dudes!