How do you upload a file using Emscripten in C++? - c++

I'm trying to upload a file to a server. I have been successful in downloading data using Emscripten's Fetch API with a GET request, but so far have been unsuccessful with POST requests.
Here is my current implementation: (the file is being opened and read as expected, but the server is not receiving the file)
void uploadSucceeded(emscripten_fetch_t* fetch)
{
printf("Successful upload of %llu bytes to %s.\n", fetch->numBytes, fetch->url);
// The data is now available at fetch->data[0] through fetch->data[fetch->numBytes-1];
emscripten_fetch_close(fetch); // Free data associated with the fetch.
}
void uploadFailed(emscripten_fetch_t* fetch)
{
printf("Failed upload to %s - HTTP failure status code: %d.\n", fetch->url, fetch->status);
emscripten_fetch_close(fetch); // Also free data on failure.
}
bool UploadFile(const std::string& url, const std::string& file_name)
{
emscripten_fetch_attr_t attr;
emscripten_fetch_attr_init(&attr);
strcpy(attr.requestMethod, "POST");
attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY;
attr.onsuccess = uploadSucceeded;
attr.onerror = uploadFailed;
// Set headers:
const char* headers[] = { "Content-Type", "application/x-www-form-urlencoded", 0 };
attr.requestHeaders = headers;
// Read file data:
std::ifstream in_file(file_name.c_str(), std::ios::binary);
//
in_file.seekg(0, std::ios::end);
int file_size = in_file.tellg();
//
in_file.seekg(0, std::ios::beg);
std::stringstream buffer;
buffer << in_file.rdbuf();
//
char *cstr = new char[buffer.str().length() + 1];
strcpy(cstr, buffer.str().c_str());
//
attr.requestData = cstr;
attr.requestDataSize = file_size;
// Send HTTP request:
emscripten_fetch(&attr, url.c_str());
return true;
}

You need to make sure that the request header has:
"Content-Type", "multipart/form-data; boundary=[custom-boundary]\r\n"
...where [custom-boundary] is a string of your choice.
Then in the request data, you start with that custom boundary, followed by "\r\n", then you have another header, such as:
"Content-Disposition: form-data; name=\"myFile\" filename=\"G0000U00000R01.html\"\r\n"
"Content-Transfer-Encoding: binary\r\n"
"Content-Type: text/html\r\n\r\n"
...followed by the file contents, followed by "\r\n" again, and finally followed by the same custom boundary as before.

Related

Tried to parse chunked transfer encoding,it's not working though, the file which I decoded is totally unreadable

I tried to parse the data which was generated by chunked transfer encoding in a Rest API ,I did see the data has value when I tried to print the value in a string and I thought it should be working,but when I tried to assign the value to the file, the file is totally unreadable, the code below I used boost library and I gonna elaborate my thoughts in the code , we gonna get started from the response portion of my code, I have no idea what wrong I have done
// Send the request.
boost::asio::write(socket, request);
// Read the response status line. The response streambuf will automatically
// grow to accommodate the entire line. The growth may be limited by passing
// a maximum size to the streambuf constructor.
boost::asio::streambuf response;
boost::asio::read_until(socket, response, "\r\n");
// Check that response is OK.
std::istream response_stream(&response);
std::string http_version;
response_stream >> http_version;
unsigned int status_code;
response_stream >> status_code;
std::string status_message;
std::getline(response_stream, status_message);
if (!response_stream || http_version.substr(0, 5) != "HTTP/")
{
//std::cout << "Invalid response\n";
return 9002;
}
if (status_code != 200)
{
//std::cout << "Response returned with status code " << status_code << "\n";
return 9003;
}
// Read the response headers, which are terminated by a blank line.
boost::asio::read_until(socket, response, "\r\n\r\n");
// Process the response headers.
//this portion of code I tried to parse the file name in the header of response which the file name is in the content-disposition of header
std::string header;
std::string fullHeader = "";
string zipfilename="", txtfilename="";
bool foundfilename = false;
while (std::getline(response_stream, header) && header != "\r")
{
fullHeader.append(header).append("\n");
std::transform(header.begin(), header.end(), header.begin(),
[](unsigned char c){ return std::tolower(c); });
string containstr = "content-disposition";
string containstr2 = "filename";
string quotestr = "\"";
if (header.find(containstr) != std::string::npos && header.find(containstr2) != std::string::npos)
{
int countquotes = 0;
bool foundquote = true;
std::size_t startpos = 0, beginpos, endpos;
while (foundquote)
{
std::size_t myfound = header.find(quotestr, startpos);
if (myfound != std::string::npos)
{
if (countquotes % 2 == 0)
beginpos = myfound;
else
{
endpos = myfound;
foundfilename = true;
}
startpos = myfound + 1;
}
else
foundquote = false;
countquotes++;
}
if (endpos > beginpos && foundfilename)
{
size_t zipfileleng = endpos - beginpos;
zipfilename = header.substr(beginpos+1, zipfileleng-1);
txtfilename = header.substr(beginpos+1, zipfileleng-5);
}
else
return 9004;
}
}
if (foundfilename == false || zipfilename.length() == 0 || txtfilename.length() == 0)
return 9005;
//when the zipfilename has been found, we gonna get the data from the body of response, due to the response was chunked transfer encoding, I tried to parse it,it's not complicated due to I saw it on the Wikipedia, it just first line was length of data,the next line was data,and it's the loop which over and over again ,all I tried to do was spliting all the data from the body of response by "\r\n" into a vector<string>, and I gonna read the data from that vector
// Write whatever content we already have to output.
std::string fullResponse = "";
if (response.size() > 0)
{
std::stringstream ss;
ss << &response;
fullResponse = ss.str();
}
//tried split the entire body of response into a vector<string>
vector<string> allresponsedata;
split_regex(allresponsedata, fullResponse, boost::regex("(\r\n)+"));
//tried to merge the data of response
string zipfiledata;
int myindex = 0;
for (auto &x : allresponsedata) {
std::cout << "Split: " << x << std::endl;// I tried to print the data, I did see the value in the variable of x
if (myindex % 2 != 0)
{
zipfiledata = zipfiledata + x;//tried to accumulate the datas
}
myindex++;
}
//tried to write the data into a file
std::ofstream zipfilestream(zipfilename, ios::out | ios::binary);
zipfilestream.write(zipfiledata.c_str(), zipfiledata.length());
zipfilestream.close();
//afterward, the zipfile was built, but it's unreadable which it's not able to open,the zip utlities software says it's a damaged zip file though
I even tried something else ways like this slow http client based on boost::asio - (Chunked Transfer) ,but this way is not working as well ,VS says
1 IntelliSense: no instance of overloaded function "boost::asio::read" matches the argument list
argument types are: (boost::asio::ip::tcp::socket, boost::asio::streambuf, boost::asio::detail::transfer_exactly_t, std::error_code)
it just NOT able to compile in the line which is
size_t n = asio::read(socket, response, asio::transfer_exactly(chunk_bytes_to_read), error);
even I have read the example of asio::transfer_exactly, there's no exactly example like this though https://www.boost.org/doc/libs/1_57_0/doc/html/boost_asio/reference/transfer_exactly.html
any idea?
I don't see you read the format correctly: https://en.wikipedia.org/wiki/Chunked_transfer_encoding#Format
You need to read the chunk length (in hex) and any optional chunk extensions before accumulating the full response body.
It needs to be done before, because the sequence \r\n that you split on can easily appear inside the chunk data.
Again, I recommend to just use Beast's support, making it all a simple
http::response<http::string_body> response;
boost::asio::streambuf buf;
http::read(socket, buf, response);
And you will have the headers fully parsed, interpreted (including Trailer headers!) and the content in response.body() as a std::string.
It will do the right thing even if the server doesn't use chunked encoding or combines with different encoding options.
There's simply no reason to reinvent the wheel.
Full Demo
This demonstrates with the Chunked Encoding test url from https://jigsaw.w3.org/HTTP/:
#include <boost/process.hpp>
#include <boost/beast.hpp>
#include <iostream>
namespace http = boost::beast::http;
using boost::asio::ip::tcp;
int main() {
http::response<http::string_body> response;
boost::asio::io_context ctx;
tcp::socket socket(ctx);
connect(socket, tcp::resolver{ctx}.resolve("jigsaw.w3.org", "http"));
http::write(
socket,
http::request<http::empty_body>(
http::verb::get, "/HTTP/ChunkedScript", 11));
boost::asio::streambuf buf;
http::read(socket, buf, response);
std::cout << response.body() << "\n";
std::cout << "Effective headers are:" << response.base() << "\n";
}
Printing
This output will be chunked encoded by the server, if your client is HTTP/1.1
Below this line, is 1000 repeated lines of 0-9.
-------------------------------------------------------------------------
01234567890123456789012345678901234567890123456789012345678901234567890
01234567890123456789012345678901234567890123456789012345678901234567890
...996 lines removed ...
01234567890123456789012345678901234567890123456789012345678901234567890
01234567890123456789012345678901234567890123456789012345678901234567890
Effective headers are:HTTP/1.1 200 OK
cache-control: max-age=0
date: Wed, 31 Mar 2021 20:09:50 GMT
transfer-encoding: chunked
content-type: text/plain
etag: "1j3k6u8:tikt981g"
expires: Wed, 31 Mar 2021 20:09:49 GMT
last-modified: Mon, 18 Mar 2002 14:28:02 GMT
server: Jigsaw/2.3.0-beta3

hello world example for a mongoose webserver with SSL in C

I am trying to set a mongoose web server v3.3 with a self-signed SSL certificate. I know how to do it without SSL but I want to implement HTTPS.
I have implemented something like this:
void *event_handler(enum mg_event event,
struct mg_connection *conn) {
const struct mg_request_info *request_info = mg_get_request_info(conn);
static void* done = "done";
if (event == MG_NEW_REQUEST) {
if (strcmp(request_info->uri, "/hello") == 0) {
// handle c[renderer] request
if(strcmp(request_info->request_method, "GET") != 0) {
// send error (we only care about HTTP GET)
mg_printf(conn, "HTTP/1.1 %d Error (%s)\r\n\r\n%s",
500,
"we only care about HTTP GET",
"we only care about HTTP GET");
// return not null means we handled the request
return done;
}
// handle your GET request to /hello
char* content = "Hello World!";
char* mimeType = "text/plain";
int contentLength = strlen(content);
mg_printf(conn,
"HTTP/1.1 200 OK\r\n"
"Cache: no-cache\r\n"
"Content-Type: %s\r\n"
"Content-Length: %d\r\n"
"\r\n",
mimeType,
contentLength);
mg_write(conn, content, contentLength);
return done;
}
}
// in this example i only handle /hello
mg_printf(conn, "HTTP/1.1 %d Error (%s)\r\n\r\n%s",
500, /* This the error code you want to send back*/
"Invalid Request.",
"Invalid Request.");
return done;
}
// No suitable handler found, mark as not processed. Mongoose will
// try to serve the request.
return NULL;
}
int main(int argc, char **argv) {
const char *options[] = {
"ssl_certificate", "cert.pem",
"listening_ports", "443s",
"num_threads", "10",
NULL
};
static struct mg_context *ctx;
ctx = mg_start(&event_handler, options);
if(ctx == NULL) {
exit(EXIT_FAILURE);
}
puts("Server running, press enter to exit\n");
getchar();
mg_stop(ctx);
return EXIT_SUCCESS;
}
The problem is I am not able to access my server from the web browser. I think the problem is that the first event my callback receives is MG_INIT_SSL but I do not know how to handle or process it. Could anybody please help?
Firstly, I believe you should not have to handle other events than MG_NEW_REQUEST in your event handler.
I would also debug using openssl:
openssl s_client -connect <hostname:port>
to see that the SSL connection gets set up properly.
In any case, Cesanta does provide a complete working example for you to use:
https://github.com/cesanta/mongoose/tree/master/examples/simplest_web_server_ssl

How to use libcurl for storing gzip response from REST API?

We have one REST API which returns response in gzip form. Whenever I download that same response from postman and save it as .zip it gets stored correctly. However, when I write the response to .zip file using libcurl methods then file gets corrupted by invalid data. No differences found on comparing the responses from postman and libcurl methods.
FILE * wfp1 = SYSS_fopen(responseFile, "w+"); //File pointer to write the REST response
if (!wfp1)
{
perror("Write File Open:");
}
request.SetWriteDataObject(wfp1);
request.SetWriteDataFn([&](void* contents, size_t size, size_t nmemb, void *stream)
{
char* fchar = nullptr;
fchar = static_cast<char*>(contents);
char* buffer = (char*)SM_alloc(nmemb + 1, sizeof(char));
snprintf(buffer, nmemb + 1, "%s", fchar);
fileStream.append(buffer);
SM_free(buffer);
return fwrite(contents, size, nmemb, (FILE*)stream);
});

http server response using sockets

My sockets server is receiving a GET request for an image, the image is 2MB so it doesn't fit in a single send(), this is what I am sending in the first send():
std::stringstream wsss;
wsss << "HTTP/1.1 200 OK\r\n"
<< "Connection: keep-alive\r\n"
<< "Content-Type: image/x-icon\r\n"
<< "Content-Length: " << imageSize << "\r\n"
<< "\r\n";
wsss.write(imageData, imageSize);
Does every subsequent send() of this image needs the header fields?
I am sending a .ico image, are the header fields correct?
the image is 2MB so it doesn't fit in a single send()
send() is not guaranteed to send as many bytes as you ask it to send. It can send fewer bytes. Its return value tells you how many bytes it actually accepted for sending. So you should call send() in a loop until all bytes have been accepted. If you move this loop into its own reusable function, that will also allow you to send the icon data without having to first copy it into the std::stringstream.
Try something like this:
int sendData(int sckt, void *data, int datalen)
{
unsigned char *pdata = (unsigned char *) data;
int numSent;
// send() can send fewer bytes than requested,
// so call it in a loop until the specified data
// has been sent in full...
while (datalen > 0) {
numSent = send(sckt, pdata, datalen, 0);
if (numSent == -1) return -1;
pdata += numSent;
datalen -= numSent;
}
return 0;
}
std::stringstream wsss;
wsss << "HTTP/1.1 200 OK\r\n"
<< "Connection: keep-alive\r\n"
<< "Content-Type: image/x-icon\r\n"
<< "Content-Length: " << imageSize << "\r\n"
<< "\r\n";
// do not append the image data to the stringstream...
//wsss.write(imageData, imageSize);
// send the headers first...
std::string headers = wsss.str();
int res = sendData(TheSocket, headers.c_str(), headers.size());
if (res == -1) ...
// now send the image data...
res = sendData(TheSocket, imageData, imageSize);
if (res == -1) ...
Does every subsequent send() of this image needs the header fields?
Every HTTP response to every HTTP request for the same image needs to send the same headers1. But every send() for any particular response does not need to repeat the headers, they only need to be sent once. Just keep sending whatever bytes have not been sent yet. That is why you have to pay attention to the return value of send() so you know how many bytes have been sent so far and how many bytes are still need to be sent.
I am sending a .ico image, are the header fields correct?
In general, yes1.
1: assuming that either:
the client sent an HTTP 1.1 request without a Connection: close request header.
the client sent an HTTP 1.0 request with a Connection: keep-alive request header.
Otherwise, your Connection: keep-alive header would be erroneous, you should be sending a Connection: close header instead, and then close the socket after sending the complete response.

Simple Http Get Response

I have been writing a simple web server (http 1.0) for a class, but whenever I try to get a file (wget 127.0.0.1 /filename) is is short a few bytes. The confusing thing is when I sum the number of sent bytes it matches the file size, but not the amount wget receives.
Why is wget not getting all of the data I write to the socket?
some wget output
wget:
--2012-10-27 19:02:00-- (try: 4) http://127.0.0.1:5555/
Connecting to 127.0.0.1:5555... connected.
HTTP request sent, awaiting response... 200 Document follows
Length: 5777 (5.6K) [text/html]
Saving to: `index.html.6'
99% [=====================================> ] 5,776 --.-K/s in 0s
2012-10-27 19:02:00 (322 MB/s) - Read error at byte 5776/5777 (Connection reset by peer). Retrying.
--2012-10-27 19:03:52-- (try: 4) http://127.0.0.1:5555/ZoEY8.jpg
Connecting to 127.0.0.1:5555... connected.
HTTP request sent, awaiting response... 200 Document follows
Length: 163972 (160K) [image/jpeg]
Saving to: `ZoEY8.jpg.4'
91% [==================================> ] 149,449 --.-K/s in 0.001s
2012-10-27 19:03:52 (98.8 MB/s) - Read error at byte 163917/163972 (Connection reset by peer). Retrying.
Get method:
void *
processGetRequest(requestParser request)
{
string resp= "HTTP/1.0 200 Document follows\r\nServer: lab5 \r\nContent-Length: ";
string path="";
path =request.path;
//find file
int page= open (path.c_str(),O_RDONLY);
FILE * pageF= fdopen(page,"rb");
//get size
fseek(pageF, 0L, SEEK_END);
int sz = ftell(pageF);
fseek(pageF, 0L, SEEK_SET);
//form content length
stringstream ss;
ss<<resp<<sz<<"\r\n";
resp=ss.str();
//make response
if(page<0){
cout<<"404 \n";
resp = "HTTP/1.0 404 File Not Found\r\nServer: lab5 \r\nContent-type: text/html \r\n \r\n";
write( request.fd, resp.c_str(), resp.length());
return 0;
}
if(path.find(".gif")!=string::npos)
resp += "Content-type: image/gif\r\n \r\n";
else if(path.find(".png")!=string::npos)
resp += "Content-type: image/png\r\n \r\n";
else if(path.find(".jpg")!=string::npos)
resp += "Content-type: image/jpeg\r\n \r\n";
else
resp += "Content-type: text/html \r\n \r\n";
//write response
write( request.fd, resp.c_str(), resp.length());
int total=0;
char buff[1024];
int readBytes = 0;
int er;
//send file
do{
readBytes= read(page, buff, 1024);
cout<<"read bytes "<<readBytes<<"\n";
if(readBytes<0){
perror("read");
break;
}
total+=readBytes;
er= send( request.fd, buff, readBytes,0 );
cout<<"sent bytes "<<er<<"\n";
if (er==-1){
perror("send");
}
else if( er != readBytes){
cout<<"Read write miss match\n";
}
}while(readBytes>0);
close(page);
return 0;
}
Edit:
I have been working at this while and I'm wondering if Im doing my sockets wrong
// Set the IP address and port for this server
struct sockaddr_in serverIPAddress;
memset( &serverIPAddress, 0, sizeof(serverIPAddress) );
serverIPAddress.sin_family = AF_INET;
serverIPAddress.sin_addr.s_addr = INADDR_ANY;
serverIPAddress.sin_port = htons((u_short) port);
// Allocate a socket
int masterSocket = socket(PF_INET, SOCK_STREAM, 0);
if ( masterSocket < 0) {
perror("socket");
exit( -1 );
}
while ( 1 ) {
// Accept incoming connections
struct sockaddr_in clientIPAddress;
int alen = sizeof( clientIPAddress );
int slaveSocket = accept( masterSocket,
(struct sockaddr *)&clientIPAddress,
(socklen_t*)&alen);
// send slaveSocket to get method
}
My first answer is below, but i just noticed something..
"Content-type: text/html \r\n \r\n";
The headers must be separated from the content with two CR/LF. It looks like you have space in there
you can try this:
"Content-type: text/html\r\n\r\n";
Is the output buffer being correctly flushed and closed after the last write? Try changing the size of your 1024 byte read buffer to something larger than your gif file. This isnt a fix, but you may get different results, and this may help track down the cause of the problem. Maybe also put some logging into the read write loop. See if the size of the last buffer write equals the number of bytes the response is missing.