i am working on a web server
and i have created a function which sends the file and the http headers
when accessing the http://127.0.0.1:8080 page
the problem is that its not copyig the file correctly.. its throwing some grabage at the end.. (i am trying to pass a binary files thats why im using std::ios::binary)
example.bin
My Example bin file
:)
Downloaded file:
My Example bin file
:)ýýýýÝÝÝÝÝÝÝhjß/ÝÝ
my code:
// download file http headers
message_ = "HTTP/1.0 200 OK\r\n"
"Cache-Control: public\r\n"
"Content-Description: File Transfer\r\n"
"Content-Disposition: attachment; filename=example.bin\r\n"
"Content-Type: application/zip\r\n"
"Content-Transfer-Encoding: binary\r\n\r\n";
std::filebuf *pbuf;
std::ifstream sourcestr;
long size;
char * buffer;
sourcestr.open("example.bin",std::ios::in | std::ios::binary);
pbuf=sourcestr.rdbuf();
size=pbuf->pubseekoff (0,std::ios::end,std::ios::in);
pbuf->pubseekpos (0,std::ios::in);
buffer=new char[size];
// get file data
pbuf->sgetn (buffer,size);
message_ += buffer;
I think the problem is that buffer is not null-terminated after the 'pbuf->sgetn()' call. Try:
// EDIT: this won't work for reason stated by #Joachim in his answer.
buffer = new char[size + 1];
pbuf->sgetn(buffer, size);
*(buffer + size) = 0;
message_ += buffer;
If message_ is a std::string an alternative without NULL termination would be:
message_.append(buffer, size);
Hope that helps.
The big problem is that you are appending the contents of a binary buffer to a string. A string in C++ is terminated by the character '\0', which means that if the binary file contains that character in the middle, the "string" will be terminated. And if the binary file does not contain any '\0' character, appending it to a string will append memory until the terminator is found, which is the reason you get extra garbage. The line message_ += buffer simply means to add the memory pointed to by buffer until string terminator is found.
You must send a file in two parts: First the header, then the file data.
Related
I am writing a HTTP server in C++, and serving static files is mostly OK, however when reading .PNG files or other binary's, every method I have tried fails. My main problem is when I open up Dev tools, reading a example image would give a transferred size of 29.56kb, and a size of 29.50 kb for my current method. The sizes given also do not match up with what du-sh give, which is 32kb.
My first method was to push the contents of a file onto a string, and call a function to serve that. However, this would also server ~6kb if memory serves correctly.
My current method is to read the file using std::ifstream in binary mode. I am getting the size of the file using C++17's filesystem header and using std::filesystem::file_size. I read the contents into a buffer and then call a function to send the buffer contents 1 byte at a time
void WebServer::sendContents(std::string contents) {
if (send(this->newFd, contents.c_str(), strlen(contents.c_str()), 0) == -1) {
throw std::runtime_error("Server accept: " + std::string(strerror(errno)));
}
}
void WebServer::sendFile(std::string path) {
path = "./" + path;
std::string fileCont; //File contents
std::string mimeType; //Mime type of the file
std::string contLength;
std::string::size_type idx = path.rfind('.');
if (idx != std::string::npos) mimeType = this->getMimeType(path.substr(idx + 1));
else mimeType = "text/html";
std::filesystem::path reqPath = std::filesystem::path("./" + path).make_preferred();
std::filesystem::path parentPath = std::filesystem::path("./");
std::filesystem::path actualPath = std::filesystem::canonical(parentPath / reqPath);
if (!this->isSubDir(actualPath, parentPath)) { this->sendRoute("404"); return; }
std::ifstream ifs;
ifs.open(actualPath, std::ios::binary);
if (ifs.is_open()) {
//Get the size of the static file being server
std::filesystem::path staticPath{path};
std::size_t length = std::filesystem::file_size(staticPath);
char* buffer = new char[length];
*buffer = { 0 }; //Initalize the buffer that will send the static file
ifs.read(buffer, sizeof(char) * length); //Read the buffer
std::string resp = "HTTP/1.0 200 OK\r\n"
"Server: webserver-c\r\n"
"Content-Length" + std::to_string(length) + "\r\n"
"Content-type: " + mimeType + "\r\n\r\n";
if (!ifs) std::cout << "Error! Only " << std::string(ifs.gcount()) << " could be read!" << std::endl;
this->sendContents(resp); //Send the headers
for (size_t i=0; i < length; i++) {
std::string byte = std::string(1, buffer[i]);
this->sendContents(byte);
}
delete buffer; //We do not need megs of memory stack up, that shit will grow quick
buffer = nullptr;
} else {
this->sendContents("HTTP/1.1 500 Error\r\nContent-Length: 0\r\nConnection: keep-alive\r\n\r\n"); return;
}
ifs.close();
}
It should be noted that this->newFd is a socket descriptor
It should also be noted that I have tried to take a look at this question here, however the same problem still occurs for me
if (send(this->newFd, contents.c_str(), strlen(contents.c_str()), 0) == -1) {
There are two bugs for the price of one, here.
This is used to send the contents of the binary file. One byte at a time. sendContents gets used, apparently, to send one byte at a time, here. This is horribly inefficient, but it's not the bug. The first bug is as follows.
Your binary file has plenty of bytes that are 00.
In that case, contents will proudly contain this 00 byte, here. c_str() returns a pointer to it. strlen() then reaches the conclusion that it is receiving an empty string, for input, and make a grandiose announcement that the string contains 0 characters.
In the end, send's third parameter will be 0.
No bytes will get sent, at all, instead of the famous 00 byte.
The second bug will come into play once the inefficient algorithm gets fixed, and sendContents gets used to send more than one byte at a time.
send() holds a secret: this system call may return other values, other than -1 to indicate the failure. Such as the actual number of bytes that were sent. So, if send() was called to send, say, 100 bytes, it may decide so send only 30 bytes, return 30, and leaving you holding the bag with the remaining 70 unsent bytes.
This is actually, already, an existing bug in the shown code. sendContents() also gets used to send the entire resp string. Which is, eh, in the neighborhood of a 100 bytes. Give or take a dozen.
You are relying on this house of cards: of send() always doing its job complete job, in this particular case, not slacking off, and actually sending the entire HTTP/1.0 response string.
But, send() is a famous slacker, and you have no guarantees, whatsoever, that this will actually happen. And I have it on good authority that an upcoming Friday the 13th your send() will decide to slack off, all of a sudden.
So, to fix the shown code:
Implement the appropriate logic to handle the return value from send().
Do not use c_str(), followed by strlen(), because: A) it's broken, for strings containing binary data, B) this elaborate routine simply reinvents a wheel called size(). You will be happy to know that size() does exactly what it's name claims to be.
One other bug:
char* buffer = new char[length];
It is possible for an exception to get thrown from the subsequent code. This memory get leaked, because delete does not get called.
C++ gurus know a weird trick: they rarely use new, but instead use containers, like std::vector, and they don't have to worry about leaking memory, because of that.
I am using a multiparser to send http POST request and I need to send a big file (4Go).
However when I test my code it give me a bad_alloc exception and crashes. I tried it with smaller files and it worked for files smaller than 500Mo, but the bigger the files get the more it crashes randomly.
Can you help me?
Here is the code where the crash occurs. It is when it buids the body of the request:
std::ifstream ifile(file.second, std::ios::binary | std::ios::ate); //the file I am trying to send
std::streamsize size = ifile.tellg();
ifile.seekg(0, std::ios::beg);
char *buff = new char[size];
ifile.read(buff, size);
ifile.close();
std::string ret(buff, size); //this make the bad_alloc exception
delete[] buff;
return ret; //return the body of the file in a string
Thanks
std::string ret(buff, size); creates a copy of buff. So you essentially double the memory consumption
https://www.cplusplus.com/reference/string/string/string/
(5) from buffer
Copies the first n characters from the array of characters pointed by s.
Then the question becomes how much you actually have, respectively how much your OS allows you to get (e.g. ulimit on linux)
As the comments say you should chunk your read and send single chunks with multiple POST requests.
You can loop over the ifstream check if ifile.eof() is reached as exit criteria for your loop:
std::streamsize size = 10*1024*1024 // read max 10MB
while (!ifile.eof())
{
std::streamsize readBytes = ifile.readsome(buff, size);
// do your sending of buff here.
}
You need to consider error handling and such to not leak buff or leave ifile open.
i'm trying to send email via SMTP, i send multipart mail, with text part and application/octet-stream part. when i try to send not *.txt" files, for example .jpg or .docx it is got corrupted and some bytes are lost. For example, when i try to send file 123.docx, size of this file is 166 020 bytes. i recieve file in my email, but it has only 166 006 and i can't open it. Variable "total" shows correct number of bytes is sent.
My code is bellow:
#include <windows.h>
#include <winsock.h>
#include <stdio.h>
#include <iostream>
#include <fstream>
using namespace std;
int fileSize(char fileName[])
{
std::streampos fsize = 0;
std::ifstream myfile (fileName, ios::in); // File is of type const char*
fsize = myfile.tellg(); // The file pointer is currently at the beginning
myfile.seekg(0, ios::end); // Place the file pointer at the end of file
fsize = myfile.tellg() - fsize;
myfile.close();
cout << "size is: " << fsize << " bytes.\n";
return fsize;
}
int main() {
char text[2048];
//connecting to smtp server
//sending DATA
strcpy(text,"DATA\n");
send(s,text,strlen(text),0);
recv(s,text,sizeof(text),0);
cout<<"recv - "<<text<<endl;
//FROM field
strcpy(text,"FROM: laboratory4_mm#rambler.ru\n");
send(s,text,strlen(text),0);
//TO field
strcpy(text,"TO: ");
strcat(text,reciever);
strcat(text,"\n");
send(s,text,strlen(text),0);
// SUBJECT field
char subject[2048];
cout<<"Enter the theme of the letter"<<endl;
cin.getline(subject,2048);
strcpy(text,"SUBJECT: ");
strcat(text,subject);
strcat(text,"\n");
send(s,text,strlen(text),0);
// delimeter of multipart message
strcpy(text,"Content-Type: multipart/mixed; boundary=\"---nsabnqeaSA43ds2\"\n");
send(s,text,strlen(text),0);
//Text part
strcpy(text,"-----nsabnqeaSA43ds2\nContent-Type: text/plain; charset=utf8\nContent-Transfer-Encoding: 8bit\n\n");
send(s,text,strlen(text),0);
cout<<"Enter the text:"<<endl;
cin.getline(text,2048);
send(s,text,strlen(text),0);
//File part
char fileName[256];
cout<<"Enter file name: ";
cin.getline(fileName,255);
int size = fileSize(fileName);
char fileLength[1024];
itoa(size,fileLength,10);
cout<<fileLength<<endl;
strcpy(text,"\n-----nsabnqeaSA43ds2 \nContent-Type: application/octet-stream\nContent-Length: ");
strcat(text,fileLength);
strcat(text,"\nContent-Transfer-Encoding: binary\nContent-Disposition: attachment;\n\n");
send(s,text,strlen(text),0);
ifstream fin;
fin.open(fileName,ios::binary);
char *buf = new char[1024];
int readBytes;
int total =0;
while((readBytes = fin.read(buf,1024).gcount())>0) {
int sent= send(s,buf,readBytes,0);
total+=sent;
delete buf;
buf = new char[1024];
}
fin.close();
delete buf;
cout<< total<<endl;
strcpy(text,"\n-----nsabnqeaSA43ds2--\n");
send(s,text,strlen(text),0);
// telling that DATA is over
strcpy(text,"\n.\n");
send(s,text,strlen(text),0);
fout<<text;
recv(s,text,sizeof(text),0);
cout<<"recv - "<<text<<endl;
//Disconnecting from server
return 0;
}
There are several fundamental bugs in the MIME encoding, here:
First, a blank line is supposed to separate headers from content. The blank line is missing:
strcpy(text,"Content-Type: multipart/mixed; boundary=\"---nsabnqeaSA43ds2\"\n");
One newline generated here.
strcpy(text,"-----nsabnqeaSA43ds2\nContent-Type: text/plain; charset=utf8\nContent-Transfer-Encoding: 8bit\n\n");
Mail content begins here, without a preceding blank line.
Additionally, the newline that precedes a boundary delimiter is a logical part of the boundary delimiter, so that newline is missing as well. See MIME documentation for more information.
strcpy(text,"\n-----nsabnqeaSA43ds2 \nContent-Type: application/octet-stream\nContent-Length: ");
Note that here you are explicitly sending a newline before the boundary delimiter, so you must be aware of this requirement.
Secondly:
while((readBytes = fin.read(buf,1024).gcount())>0) {
int sent= send(s,buf,readBytes,0);
Sending the contents of the binary file, as is? That's not going to fly. Even though it may or may not be correct MIME encoding, SMTP is still a plain text-based transmission protocol. There is an extension for transmission of binary data, but the shown code does not use it.
As is, this is the SMTP version of undefined behavior. No guaranteed results. If you need to attach this file reliably, you must base64-encode it.
I have a C++ linux server, with basic server sockets. Here's what i'm using to server png images:
string ms = "HTTP/1.0 200 OK\r\nContent-type: image/png\r\n\r\n";
ifstream myfile("xampl.png", ios::in|ios::binary|ios::ate);
string line;
char* memblock;
streampos size;
if(myfile.is_open()){
size = myfile.tellg();
memblock = new char [size];
myfile.seekg (0, ios::beg);
myfile.read (memblock, size);
myfile.close();
}
ms.append(string(memblock));
cout << "\"" << ms << "\"" << endl;
char* msg = new char[ms.size()+1];
copy(ms.begin(), ms.end(), msg);
msg[ms.size()] = '\0';
int len;
ssize_t bytes_sent;
len = strlen(msg);
bytes_sent = send(new_sd, msg, len, 0);
I know i'm trying to read the png file as a binary file, but i have no idea what else to do. When i telnet to this server, i get a response with weird characters which make me believe that i have served the file, but when i check it out in my browser, i get the image not found icon, in all browsers. Please help...
OK, let's start with the equivalent PHP code to get rid of the mess it is to do the same thing in C++
// send required headers as plain text
header("Content-type: image/png");
// read the image as a binary block
$img_data = file_get_contents("xample.png");
// send it
echo $img_data;
C++ equivalent:
string headers = "HTTP/1.0 200 OK\r\nContent-type: image/png\r\n\r\n";
send (new_sd, headers.data(), headers.length(), 0);
ifstream f("xampl.png", ios::in|ios::binary|ios::ate);
if(!f.is_open()) error ("bloody file is nowhere to be found. Call the cops");
streampos size = f.tellg();
char* image = new char [size];
f.seekg (0, ios::beg);
f.read (image, size);
f.close();
send (new_sd, image, size, 0);
Converting memblock to a string is not going to work if it has embedded null characters, which it almost certainly does, and you don't pass the length. I'm not sure why you're doing all that faffing about with char*s and std::strings, but once you have your memblock and size, use them for send. If you want to prefix the response string, just send it first.
I want to write a server side code. It should work with popular browsers and wget. My server check that file exists or not, if exists then browser can download it. But I have some problems.
Honestly, I read lots of question-answer (for example: Send binary file in HTTP response using C sockets) but I didn't find out. My browser (Chrome) can get text. But I cannot send any binary data or images etc. I am changing header according to downloading files. But I cannot send a downloadable files yet.
I have some questions.
void *clientWorker(void * acceptSocket) {
int newSocket = (int) acceptSocket;
char okStatus[] = "HTTP/1.1 200 OK\r\n"
"Content-Type: text/html\r\n"
"Connection: close\r\n"
"Content-Length: 20\r\n"
"\r\n"
"s";
writeLn(newSocket, okStatus);
const char * fileName = "/home/tyra/Desktop/example.txt";
sendF(newSocket, fileName);
}
1- If I wouldn't write "s" or something else inokStatus, my message cannot send. I don't understand anything of this.
This is writeLn function :
void writeLn(int acceptSocket, const char * buffer) {
int n = write(acceptSocket, buffer, strlen(buffer) - 1);
if (n < 0) {
error("Error while writing");
}
}
This is sendF function :
string buffer;
string line;
ifstream myfile(fileName);
struct stat filestatus;
stat(fileName, &filestatus);
int fsize = filestatus.st_size;
if (myfile.is_open()) {
while (myfile.good()) {
getline(myfile, line);
buffer.append(line);
}
cout << buffer << endl;
}
writeLn(acceptSocket, buffer.c_str());
cout << fsize << " bytes\n";
A little messy. I haven't used file size yet. If I send a file, then I rearrange these things.
2- I can send text and browser demonstrates it but browser didn't understand new lines.
If text file contains (123\n456\n789), browser demonstrates (123456789). I think I should change Content-Type header, but I didn't find out.
I don't want that browser demonstrates text files. Browser should download it. How can I send downloadable files?
Sorry, I explain everything pretty complicated.
As to your first question, you should find out the exact size of the file and specify it in your "Content-Length: xxxx\r\n" header. And, of course, you should ensure that the data is sent completely out.
Indeed, in your writeF function you use a std::string as a buffer:
string buffer;
this is not appropriate for binary data. You should allocate a raw char array of the right size:
int fsize = file.tellg();
char* buffer = new char[fsize];
file.seekg (0, ios::beg);
file.read (buffer, size);
file.close();
As to the second point, when your data is not HTML, specify as Content-Type: text/plain;
otherwise, your carriage return should be represented by <br> instead of "\r\n".
In case of binary downloads, to have the data download as a file (and not shown in the browser), you should specify
Content-Type: application/octet-stream
The issue is strlen here. strlen terminates when it gets a '\0' character. In binary file you will have a number of '\0' characters.
While reading the file you should find out the file size. This size should be used in int n = write(acceptSocket, buffer, strlen(buffer) - 1); in place of strlen
Change the writeLn(acceptSocket, buffer.c_str()); to writeLn(acceptSocket, buffer.c_str(), buffer.size()); and try...
For the case of 123\n456\n789 you need to send <PRE>123\n456\n789</PRE> as browser will parse this text as html and not like the OS parses and shows the output. The other way you can do is replace all \n with <BR> ...
Regarding question 1 - if you don't want to send any content back then remove the s from the end of okStatus and specify Content-Length: 0\r\n in the header