I'm stuck and I need help.
I'm trying to write the correct code for sending back an image file so the web browser can render it. It can send back text/html just fine, but image/* is not working.
You can see the code and the URL is shown below.
https://github.com/MagnusTiberius/iocphttpd/blob/master/iocphttpl/SocketCompletionPortServer.cpp
What the browser is receiving is just a few bytes of image data.
I tried vector, std::string and const char* to set the values of WSABUF, but still the same few bytes are sent over.
Please let know what is the missing piece to make this one work.
Thanks in advance.
Here's your problem:
PerIoData->LPBuffer = _strdup(str.c_str());
The _strdup function only copies up until the first null, so it cannot be used to copy binary data. Consider using malloc and memcpy if you don't want to use the C++ library.
The alternate implementation (in the false branch) is also incorrect, because it saves the data in an object (vc) that goes out of scope before the I/O is completed. You could instead do something like
vector<char> * vc = new vector<char>;
Related
I have generated a GeoTiff dataset in-memory using GDALTranslate() with a /vsimem/ filepath. I need access to the buffer for the actual GeoTiff file to put it in a stream for an external API. My understanding is that this should be possible with VSIGetMemFileBuffer(), however I can't seem to get this to return anything other than nullptr.
My code is essentially as follows:
//^^ GDALDataset* srcDataset created somewhere up here ^^
//psOptions struct has "-b 4" and "-of GTiff" settings.
const char* filep = "/vsimem/foo.tif";
GDALDataset* gtiffData = GDALTranslate(filep, srcDataset, psOptions, nullptr);
vsi_l_offset size = 0;
GByte* buf = VSIGetMemFileBuffer(filep, &size, true); //<-- returns nullptr
gtiffData seems to be a real dataset on inspection, it has all the appropriate properties (number of bands, raster size, etc). When I provide a real filesystem location to GDALTranslate() rather than the /vsimem/ path and load it up in QGIS it renders correctly too.
Looking a the source for VSIGetMemFileBuffer(), this should really only be returning nullptr if the file can't be found. This suggests i'm using it incorrectly. Does anyone know what the correct usage is?
Bonus points: Is there a better way to do this (stream the file out)?
Thanks!
I don't know anything about the C++ API. But in Python, the snippet below is what I sometimes use to get the contents of an in-mem file. In my case mainly VRT's but it shouldn't be any different for other formats.
But as said, I don't know if the VSI-api translate 1-on-1 to C++.
from osgeo import gdal
filep = "/vsimem/foo.tif"
# get the file size
stat = gdal.VSIStatL(filep, gdal.VSI_STAT_SIZE_FLAG)
# open file
vsifile = gdal.VSIFOpenL(filep, 'r')
# read entire contents
vsimem_content = gdal.VSIFReadL(1, stat.size, vsifile)
In the case of a VRT the content would be text, shown with something like print(vsimem_content.decode()). For a tiff it would of course be binary data.
I came back to this after putting in a workaround, and upon swapping things back over it seems to work fine. #mmomtchev suggested looking at the CPL_DEBUG output, which showed nothing unusual (and was silent during the actual VSIGetMemFileBuffer call).
In particular, for other reasons I had to put a GDALWarp call in between calling GDALTranslate and accessing the buffer, and it seems that this is what makes the difference. My guess is that GDALWarp is calling VSIFOpenL internally - although I can't find this in the source - and this does some kind of initialisation for VSIGetMemFileBuffer. Something to try for anyone else who encounters this.
I have a server (python) that sends bytes from a file to a client in c++. I am using libcurl to make requests to the python server and flask to do all of the "hard" work for me in python. After i get the file bytes from the server, i want to write it to a zip file on the client side. Initially, i was going to use libcurl to do it for me, but i decided i didn't want to do that as it would require an extra function in my wrapper which is not necessary.
FILE* zip_file = fopen(zip_name, "wb");
//make request and store the bytes from the server in a string
fwrite(response_information.first.c_str(), sizeof(char), sizeof(response_information.first.c_str()), zip_file);
//response_information is a pair . First = std::string, Second = curl response code
I do plan on switching to fopen_s (safe version of fopen), but i want to get a working program first. This is part of a bigger project so i can't provide code that can be run. Some things to note that i think can be causing this: storing response as string then attempting to get the c string version and write it to the file. When storing the return value/code of fwrite, i get "8" which means "*" bytes written apparently. Also, when im on windows, it says that the file was modified after i run my program, but nothing is in the zip file itself. How can i write the response bytes to a file?
The third parameter in fwrite is a count of items to write. So sizeof doesn't seem to be the thing you need. response_information.first.c_str() is a pointer, so sizeof(response_information.first.c_str()) returns a pointer size. Here it should be:
fwrite(response_information.first.c_str(), sizeof(char), strlen(response_information.first.c_str()), zip_file);
or
fwrite(response_information.first.c_str(), sizeof(char), response_information.first.length(), zip_file);
I am having troubles filtering data that is passed to PR_Write. It is the Mozilla function that is used in passing all sorts of data sent to the server. I managed to hook it using a DLL( extremely basic ) using the code from Wikipedia on hooking.
The following is the declaration of the PR_Write function referenced from the Mozilla website.
PRInt32 PR_Write(PRFileDesc *fd, const void *buf, PRInt32 amount);
The second parameter buf is what I am logging by casting it to a const char*, it works fine but I don't know how I can filter the data since it logs everything from start to end.
The code below is what I tried but it is too heavy a loop and crashes Mozilla.
char *p=(char*)buf; // get pointer to beginning of the buffer
while (*p!='\00')
{
// do some data filtering
*p++;
}
The idea was from Grayhat Python book to iterate through the buffer and filter data as needed but the loop is too much since the buffer is always extremely large.
Overall, I need a way to filter the data that is passed to the second parameter.
Thanks for any suggestions in advance :)
In my project I want to send possibly large and frequent XML data to a custom server written in c++. I don't want to use Apache and CGI because the data is too frequent to be starting a CGI process for every request. I would prefer if the data was recieved directly in the c++ code that will process the data and send a reply.
I started out by using libmicrohttpd for the c++ server but now I believe it won't be possible because it doesn't give access to the raw POST data. I started looking for another library but I can't seem to find a c++ library that does this. Can anyone suggest a c++ http server library that has access to the raw post data?
Here is the code I intended to start with. It is one of the example files provided in the source code of libmicrohttpd. Post Example from libmicrohttpd library
Edit:
A little more context.
From what I understand to access the post data in libmicrohttpd you create MHD_PostProcessor function that gets called incrementally as the post data is received in chunks. But in the example below it only shows how to get post data in the form of key value pairs. But I can't see how to get the data from a post.
The example implements the MHD_PostProcessor as post_iterator. See the definition of
static int post_iterator(void *cls,
enum MHD_ValueKind kind,
const char *key,
const char *filename,
const char *content_type,
const char *transfer_encoding,
const char *data, uint64_t off, size_t size) {
...
in the example posted above. You will see it only shows how to iterate the key value pairs.
MHD does give you access to the raw POST data, just grab it from "upload_data" directly instead of passing it to the MHD_PostProcessor. MHD will give you the uploaded POST stream incrementally by calling your main request processing callback repeatedly with more and more POST data being given to you raw, unprocessed in "upload_data".
I am using Curlpp to send requests to various webservices to send and receive data.
So far this has worked fine since i have only used it for sending/receiving JSON data.
Now i have a situation where a webservice returns a zip file in binary form. This is where i encountered a problem where the data received is not complete.
I first had Curl set to write any data to a ostringstream by using the option WriteStream, but this proved not to be the correct approach since the data contained null characters, and thus the data stopped at the first null char.
After that, instead of using WriteStream i used WriteFunction with a callback function.
The problem in this case is that this function is always called 2 or 3 times, regardless of the amount of data.
This results in always having a few chunks of data that don't seem to be the first part of the file, although the data always contains PK as the first 2 characters, indicating a zip file.
I used several tools to verify that the data is entirely being sent to my application so this is not a problem of the webservice.
Here the code. Do note that the options like hostname, port, headers and postfields are set elsewhere.
string requestData;
size_t WriteStringCallback(char* ptr, size_t size, size_t nmemb)
{
requestData += ptr;
int totalSize= size*nmemb;
return totalSize;
}
const string CurlRequest::Perform()
{
curlpp::options::WriteFunction wf(WriteStringCallback);
this->request.setOpt( wf );
this->request.perform();
return requestData;
}
I hope anyone can help me out with this issue because i've run dry of any leads on how to fix this, also because curlpp is poorly documented(and even worse since the curlpp website disappeared).
The problem with the code is that the data is put into a std::string, despite having the data in binary (ZIP) format. I'd recommend to put the data into a stream (or a binary array).
You can also register a callback to retrieve the response headers and act in the WriteCallback according to the "Content-type".
curlpp::options::HeaderFunction to register a callback to retrieve response-headers.
std::string is not a problem, but the concatenation is:
requestData += ptr;
C string (ptr) is terminated with zero, if the input contains any zero bytes, the input will be truncated. You should wrap it into a string which knows the length of its data:
requestData += std::string(ptr, size*nmemb);