I'm creating an FTP client.
I'm getting a gif from the server, but after that the gif is corrupted.
When I change the file extension to look at the diff, I see that the
CR/LF characters are gone.
How could this be? I made sure to use image mode.
Here's my read code in TCP socket.
string TCPSocket::long_read()
{
pollfd ufds;
ufds.fd = sd;
ufds.events = POLLIN;
ufds.revents = 0;
ssize_t bytesRead = 0;
string result;
char* buf = new char[LONGBUFLEN];
do {
bzero(buf, LONGBUFLEN);
bytesRead = ::read(sd, buf, LONGBUFLEN);
if (bytesRead == 0) {
break;
}
if (bytesRead > 0) {
result = result + string(buf, bytesRead);
}
} while (poll(&ufds, 1, 1000) > 0);
return result;
}
Here my get code in main.cpp
else if (command == command::GET) {
string filename;
cin >> filename;
string dataHost;
int dataPort;
if (enterPassiveMode(dataHost, dataPort)) {
dataSocket = new TCPSocket(dataHost.c_str(), dataPort);
if (fork() == 0) {
string result = dataSocket->long_read();
size_t length = result.size();
char* resultArr = new char[length];
memcpy(resultArr, result.data(), length);
// mode_t mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH;
FILE* file = fopen(filename.c_str(), "w+b");
if (file) {
fwrite(resultArr, length, 1, file);
fclose(file);
}
else {
cout << "open failed";
}
break;
}
else {
writeAndImmediateRead(rfc959::TYPE_I);
controlSocket->write(rfc959::RETRIVE(filename));
string result = controlSocket->read();
cout << result;
int reply = Parser::firstDigit(result);
// I'll remove incomplete local file if request fails
if (reply != rfc959::POSITIVE_PRELIMINARY_REPLY) {
remove(filename.c_str());
continue;
}
wait(NULL);
cout << controlSocket->long_read();
}
}
}
EDIT
I did make sure to use Binary mode. And when I transferred a text file(though of a smaller size), it doesn't have this problem. Here's the output:
EDIT 2
Output from Wireshark showing Request: TYPE I and Response: Opening BINARY mode
By default, FTP servers and clients perform data transfers as "ASCII mode", which means that any CRLF sequence is translated on-the-fly to the host's ASCII line ending (e.g. just bare LF on Unix mmachines). This behavior is mandated by RFC 959; see Section 3.1.1.1.
To transfer your data as binary, and avoid the ASCII mode translation, your FTP client will want to send the TYPE command first, e.g.:
TYPE I
Your .gif file should then be transferred as is, with no replacements/transformations on any CRLF sequences.
Hope this helps!
Related
I'm trying to create a webserver to learn how HTTP functions. I am trying to send a png file to the browser, however the image successfully makes it.
Here is my png sending code:
std::ifstream in("P:/server"+location, std::ios::binary);
if(!in.is_open()) {
std::cout << "failed to open file" << std::endl;
in.close();
}
in.seekg(0, std::ios::end);
int length = in.tellg();
in.seekg(0, std::ios::beg);
char *data = new char[length];
in.read(data, length);
in.close();
std::string headers = "HTTP/1.1 200 OK\n\rContent-Length: " + std::to_string(length) + "\n\rConnection: keep-alive\n\rContent-Type: image/png\n\r\n\r";
int totalLength = headers.length() + length;
char *allData = new char[totalLength];
std::strcpy(allData, headers.data());
std::strcat(allData, data);
int bytes = send(socket, data, totalLength, NULL);
Once the server should have sent the image, it shows up as the missing image icon.
I have checked to make sure that all the bytes are being sent, and that the image is being loaded.
Any help would be very much appreciated!
There are quite a few mistakes in your code.
You are still trying to process the file if is_open() returns false.
You are using \n\r when you need to use \r\n instead.
You are using strcat() to append data to allData. When appending binary data, like a PNG, strcat() will truncate on the first 0x00 byte encountered. You need to use memcpy() (or equivalent) instead.
Your call to send() is sending data when it should be sending allData instead. So you are you not sending the HTTP headers at all, and you are sending data using the wrong length.
You are assuming send() will send all of the data you give it in a single operation. That is almost never the case, especially fo large amounts of data. send() returns the number of bytes it actually accepted for sending, so you need to call it in a loop until all of the data has been accepted.
You are not sending an error message to the client if something goes wrong while preparing the file.
That being said, allData is actually unnecessary, and is a waste of memory. TCP is a byte stream, so you can send headers and data individually one after the other. How many times you call send() doesn't affect how the other party receives the data.
You might also consider minimizing memory usage further by changing data to be a fixed-sized buffer so you can send() the content of in while reading from it in smaller chunks.
Try something more like this instead:
int sendData(int sckt, const void *data, int datalen)
{
const char *ptr = static_cast<const char*>(data);
while (datalen > 0) {
int bytes = send(sckt, ptr, datalen, 0);
if (bytes <=0) return -1;
ptr += bytes;
datalen -= bytes;
}
return 0;
}
int sendStr(int sckt, const std::string &s)
{
return sendData(sckt, s.c_str(), s.size());
}
...
std::string filename = "P:/server"+location;
if (!fileExists(filename)) // <- you need to implement this
{
if (sendStr(socket, "HTTP/1.1 404 Not Found\r\nContent-Length: 0\r\nConnection: keep-alive\r\n\r\n") == -1) {
close(socket);
}
}
else
{
std::ifstream in(filename, std::ios::binary);
if (!in.is_open())
{
std::cout << "failed to open file" << std::endl;
if (sendStr(socket, "HTTP/1.1 500 Error\r\nContent-Length: 0\r\nConnection: keep-alive\r\n\r\n") == -1) {
close(socket);
}
}
else
{
in.seekg(0, std::ios::end);
std::size_t length = in.tellg();
in.seekg(0, std::ios::beg);
if (in.fail())
{
std::cout << "failed to get size of file" << std::endl;
if (sendStr(socket, "HTTP/1.1 500 Error\r\nContent-Length: 0\r\nConnection: keep-alive\r\n\r\n") == -1) {
close(socket);
}
}
else if (sendStr(socket, "HTTP/1.1 200 OK\r\nContent-Length: " + std::to_string(length) + "\r\nConnection: keep-alive\r\nContent-Type: image/png\r\n\r\n") == -1)
{
close(socket);
}
else if (length > 0)
{
char data[1024];
do
{
if (!in.read(data, std::min(length, sizeof(data))))
{
close (socket);
break;
}
int bytes = in.gcount();
if (sendData(socket, data, bytes) == -1)
{
close(socket);
break;
}
length -= bytes;
}
while (length > 0);
}
}
}
I found this code snippet for downloading file:
wxURL url(wxT("http://www.example.com/file.txt"));
if(url.GetError()==wxURL_NOERR)
{
wxString htmldata;
wxInputStream *in = url.GetInputStream();
if(in && in->IsOk())
{
wxStringOutputStream html_stream(&htmldata);
in->Read(html_stream);
wxLogMessage(htmldata);
}
delete in;
}
But fistly it just logs content of file and only for text-files. But I need to download *.exe file to execute it later. So I need to adapt this code to work with binary data, and save this data to file on the disk. Too many Streams used here for my understanding what's going on here. Please help.
I have written below code previously...
This will work fine to download binary files in any platforms.
/** START */
// ex) ht tp://mysite.com/mypath.jpg
wxString path = wxT("/mypath.jpg");
wxString server = wxT("mysite.com");
wxHTTP http;
http.SetHeader(_T("Content-type"), contentType);
http.SetTimeout(10);
// wxString imageFilePath = wxT("/tmp/image.jpg");
wxFileOutputStream output(imageFilePath);
wxDataOutputStream store(output);
if (http.Connect(server, 80))
{
wxInputStream *stream;
stream = http.GetInputStream(path);
if (stream == NULL)
{
output.Close();
}
else
{
unsigned char buffer[1024];
int byteRead;
// receive stream
while (!stream->Eof())
{
stream->Read(buffer, sizeof(buffer));
store.Write8(buffer, sizeof(buffer));
byteRead = stream->LastRead();
if (byteRead <= 0)
{
break;
}
}
output.Close();
}
}
else
{
output.Close();
}
I am a beginner programmer trying to inflate text stream from pdfs. I have adopted and slightly altered some open source code which uses zlib, and generally it works very well. However, I have been testing on some different pdfs lately and some of the inflated streams are returning blank. Could anybody advise me as to why?
I have come across this question below which seems to address the same problem but does not really give a definitive answer
zLib inflate has empty result in some cases
#include <iostream>
#include <fstream>
#include <string>
#include "zlib.h"
int main()
{
//Discard existing output:
//Open the PDF source file:
std::ifstream filei("C:\\Users\\dpbowe\\Desktop\\PIDSearch\\P&ID.PDF", std::ios::in|std::ios::binary|std::ios::ate);
if (!filei) std::cout << "Error Opening Input File" << std::endl;
//decoded output
std::ofstream fileo;
fileo.open("C:\\Users\\dpbowe\\Desktop\\Decoded.txt", std::ios::binary | std::ofstream::out);
if (!fileother) std::cout << "Error opening output file" << std::endl;
if (filei && fileo)
{
//Get the file length:
long filelen = filei.tellg(); //fseek==0 if ok
filei.seekg(0, std::ios::beg);
//Read the entire file into memory (!):
char* buffer = new char [filelen];
if (buffer == NULL) {fputs("Memory error", stderr); exit(EXIT_FAILURE);}
filei.read(buffer,filelen);
if (buffer == '\0') {fputs("Reading error", stderr); exit(EXIT_FAILURE);}
bool morestreams = true;
//Now search the buffer repeated for streams of data
while (morestreams)
{
//Search for stream, endstream. Should check the filter of the object to make sure it if FlateDecode, but skip that for now!
size_t streamstart = FindStringInBuffer (buffer, "stream", filelen); //This is my own search function
size_t streamend = FindStringInBuffer (buffer, "endstream", filelen); //This is my own search function
if (streamstart>0 && streamend>streamstart)
{
//Skip to beginning and end of the data stream:
streamstart += 6;
if (buffer[streamstart]==0x0d && buffer[streamstart+1]==0x0a) streamstart+=2;
else if (buffer[streamstart]==0x0a) streamstart++;
if (buffer[streamend-2]==0x0d && buffer[streamend-1]==0x0a) streamend-=2;
else if (buffer[streamend-1]==0x0a) streamend--;
//Assume output will fit into 10 times input buffer:
size_t outsize = (streamend - streamstart)*10;
char* output = new char [outsize]; ZeroMemory(output, outsize);
//Now use zlib to inflate:
z_stream zstrm; ZeroMemory(&zstrm, sizeof(zstrm));
zstrm.avail_in = streamend - streamstart + 1;
zstrm.avail_out = outsize;
zstrm.next_in = (Bytef*)(buffer + streamstart);
zstrm.next_out = (Bytef*)output;
int rsti = inflateInit(&zstrm);
if (rsti == Z_OK)
{
int rst2 = inflate (&zstrm, Z_FINISH);
if (rst2 >= 0)
{
size_t totout = zstrm.total_out;
//Write inflated output to file "Decoded.txt"
fileother<<output;
fileother<<"\r\nStream End\r\n\r\n";
}
else std::cout<<"output uncompressed stream is blank"<<std::endl;
}
delete[] output; output=0;
buffer+= streamend + 7;
filelen = filelen - (streamend+7);
}
else
{
morestreams = false;
std::cout<<"End of File"<<std::endl;
}
}
filei.close();
}
else
{
std::cout << "File Could Not Be Accessed\n";
}
if (fileo) fileo.close();
}
I am working on Chrome Extension with Native Host messaging. I am not able to use message text in my host application. Everything working fine from the establishing connection to get response in extension.
I need to use Message text in my application for further use/execution in simple text datatype (string/char). I know message is in UTF8 encoded form i tried to decode but still getting problem, can any one help me out?
When i decode message chrome extension console show Error: "Error when communicating with the native messaging host." and if i use that message text after "cout" same error is there "Error when communicating with the native messaging host". Direct sending and receiving message works fine for me.
Code is something like this :
std::string mycode(std::string data){
data= data+"abc"; //changing text to any thing.
cout<< data;
anotherFunction(data);//killing processes using string data
}
int main(int argc, char* argv[])
{
std::cout.setf( std::ios_base::unitbuf );
while (true)
{
unsigned int ch, inMsgLen = 0, outMsgLen = 0;
std::string input = "", response = "";
std::cin.read((char*)&inMsgLen, 4);
if (inMsgLen == 0)
{
break;
}
else
{
for (int i=0; i < inMsgLen; i++)
{
ch = getchar();
input += ch;
}
}
response.append("{\"echo\":").append(input).append("}");
outMsgLen = response.length();
std::cout.write((char*)&outMsgLen, 4);
std::cout << response;
cout<< input;
//using "input" variable for further user
mycode(input);
}
return 0;
}
That's wrong, did you read the docs ?
Try something like this...
_setmode( _fileno( stdin ), _O_BINARY );
_setmode( _fileno( stdout ), _O_BINARY );
char cBuffer[65536] = {0};
while(true)
{
unsigned int uiSize = 0;
std::cin.read((char*)&uiSize, sizeof(unsigned int));
if(uiSize != 0 && uiSize < 65536)
{
memset(cBuffer, 0, 65536);
std::cin.read(cBuffer, uiSize);
std::string strIn(cBuffer);
std::string strOut = "{\"result\":\"This is a Test\"}";
uiSize = strOut.length();
std::cout << char(((uiSize>>0) & 0xFF));
std::cout << char(((uiSize>>8) & 0xFF));
std::cout << char(((uiSize>>16) & 0xFF));
std::cout << char(((uiSize>>24) & 0xFF));
std::cout << strOut.c_str();
}
else
break;
}
You need to set the io to binary, otherwise things like this could happen...
if a byte with the value 00011010 (CTRL ALT Z = 26) is present it will be treated as a EOF and end the communication. :)
I'm currently using libzip in a C++11 program to extract the contents of a compressed file and store them into a data structure that will also hold metadata related to the file.
I'm using the current method to explode the zip file and get the content of each file in it:
void explodeArchive(const string& path, vector<ZipFileModel>& files) {
int error = 0;
zip *zip = zip_open(path.c_str(), 0, &error);
if (zip == nullptr) {
throw logic_error("Could not extract content of file " + path);
}
const zip_int64_t n_entries = zip_get_num_entries(zip, ZIP_FL_UNCHANGED);
for (zip_int64_t i = 0; i < n_entries; i++) {
const char *file_name = zip_get_name(zip, i, ZIP_FL_ENC_GUESS);
struct zip_stat st;
zip_stat_init(&st);
zip_stat(zip, file_name, ZIP_FL_NOCASE, &st);
char *content = new char[st.size];
std::cerr << file_name << std::endl;
zip_file *file = zip_fopen(zip, file_name, ZIP_FL_NOCASE);
const zip_int64_t did_read = zip_fread(file, content, st.size);
if (did_read <= 0) {
continue;
}
if (strlen(content) < st.size) {
LOG(WARNING)<< "File " << file_name << " is truncated.";
}
if (strlen(content) > st.size) {
content[st.size] = '\0';
}
ZipFileModel model;
model.name = string(file_name);
model.content = string(content);
model.order = -1;
files.push_back(model);
zip_fclose(file);
delete[] content;
}
zip_close(zip);
}
My problem is that I get random segmentation faults with gdb pointing to zip_fclose(file);:
Program received signal SIGSEGV, Segmentation fault.
0x00000001001ef8a0 in zip_source_close (src=0x105001b00) at /Users/xxx/Projects/xxx/xxx/src/libzip/zip_source_close.c:48
48 (void)src->cb.l(src->src, src->ud, NULL, 0, ZIP_SOURCE_CLOSE);
What's the best way to debug this? As I said it happens intermittently so it's hard to pin down the exact cause.
You aren't closing the zip_file when there's nothing to read.
First you open the file inside:
zip_file *file = zip_fopen(zip, file_name, ZIP_FL_NOCASE);
Then try to read something:
const zip_int64_t did_read = zip_fread(file, content, st.size);
and if there's nothing to read you continue and the file is never closed.
if (did_read <= 0) {
continue;
}
So, just add:
if (did_read <= 0) {
zip_fclose(file);
continue;
}