I'm attempting to use TinyXML to read and save from memory, instead of only reading and saving files to disk.
It seems that the documnent's parse function can load a char *. But then I need to save the document to a char * when I'm done with it. Does anyone know about this?
Edit: The printing & streaming functions aren't what I'm looking for. They output in a viewable format, I need the actual xml content.
Edit: Printing is cool.
Here's some sample code I am using, adapted from the TiXMLPrinter documentation:
TiXmlDocument doc;
// populate document here ...
TiXmlPrinter printer;
printer.SetIndent( " " );
doc.Accept( &printer );
std::string xmltext = printer.CStr();
A simple and elegant solution in TinyXml for printing a TiXmlDocument to a std::string.
I have made this little example
// Create a TiXmlDocument
TiXmlDocument *pDoc =new TiXmlDocument("my_doc_name");
// Add some content to the document, you might fill in something else ;-)
TiXmlComment* comment = new TiXmlComment("hello world" );
pDoc->LinkEndChild( comment );
// Declare a printer
TiXmlPrinter printer;
// attach it to the document you want to convert in to a std::string
pDoc->Accept(&printer);
// Create a std::string and copy your document data in to the string
std::string str = printer.CStr();
I'm not familiar with TinyXML, but from the documentation it seems that by using operator << to a C++ stream (so you can use C++ string streams) or a TiXMLPrinter class you can get an STL string without using a file. See TinyXML documentation (look for the "Printing" section)
Don't quite get what you are saying; your question is not clear. I'm guessing you are wanting to load a file into memory so that you can pass it to the document parse function. In that case, the following code should work.
#include <stdio.h>
The following code reads a file into memory and stores it in a buffer
FILE* fd = fopen("filename.xml", "rb"); // Read-only mode
int fsize = fseek(fd, 0, SEEK_END); // Get file size
rewind(fd);
char* buffer = (char*)calloc(fsize + 1, sizeof(char));
fread(buffer, fsize, 1, fd);
fclose(fd);
The file is now in the variable "buffer" and can be passed to whatever function required you to provide a char* buffer of the file to it.
Related
i have a big xml file( osm map data file to parse). the initial code to process is like this:
FILE* file = fopen(fileName.c_str(), "r");
size_t BUF_SIZE = 10 * 1024 * 1024;
char* buf = new char[BUF_SIZE];
string contents;
while (!feof(file))
{
int ret = fread(buf, BUF_SIZE, 1, file);
assert(ret != -1);
contents.append(buf);
}
size_t pos = 0;
while (true)
{
pos = contents.find('<', pos);
if (pos == string::npos) break;
// Case: found new node.
if (contents.substr(pos, 5) == "<node")
{
do something;
}
// Case: found new way.
else if (contents.substr(pos, 4) == "<way")
{
do something;
}
}
then here people tell me i should use memory mapping file to process those "big data file",
detail is here:
how to read to huge file into buffer,
i mean when it is a fixed size and not very large, may i could load one time into memory and append the content to a string object, then i could apply find(), method and other string method to extract the node content of a xml file. ( the code in the beginning of my question use this method and i test that will produce right result). Then if the file is very large, how apply those methods (not using xml library such as libxml)?
in one word, for small xml file, i could load the whole content to a std::string and apply the find(), substr() operation and got wanted information in the xml file. when the xml file is very large, when i need use the memory mapping file to cope with. then could append the whole content to a std::string, how could i parse the file not using exsit xml library?
hope i have clearly express my question.
If you're using std::string members to get the data you need, you're almost certainly not parsing the XML in the traditional sense of parsing XML. (That is, you're very probably not making any use of XML's hierarchical structure. Although you are extracting data from XML, "parsing XML" means something much more specific to most people.)
That said, the C equivalents of the std::string members you seem to be OK with, such as memcmp and the GNU extension memmem, just take pointers and lengths. Read their documentation and use them in place of their std:;string-member equivalents.
I need to parse a file, get some data and write them in another file using RapidJson.
Right now I could retrieve values and put them in a document. My only problem is to insert that document in file:
FILE * pFile = fopen ("read.json" , "r");
FILE * wFile = fopen ("Test.json" , "w");
if (pFile != NULL)
{
rapidjson::FileStream is(pFile);
rapidjson::Document document;
document.ParseStream<0>(is);
string mMeshID = a.GetString();
//how to add that document to wfile
fclose (pFile);
}
Is there any way to write a RapidJson::Document in file ?
EDIT: the only way I found is:
// Convert JSON document to string
GenericStringBuffer< UTF8<> > buffer;
Writer<GenericStringBuffer< UTF8<> > > writer(buffer);
doc.Accept(writer);
const char* str = buffer.GetString();
fprintf(wFile, "%s", str);
fclose(wFile);
There is better documentation about FileWriteStream after this question was asked.
Using FileWriteStream instead of StringBuffer can reduce memory usage. FileWriteStream uses a fixed size of buffer (can be stored in the stack), while StringBuffer needs to store the whole JSON in (heap) memory. It becomes a big difference for big JSON.
You better use
fwrite (buffer.GetString(), buffer.GetSize(), 1, wFile);
it's safer (in case the buffer is not null-terminated) and faster (no strlen).
Other than that and the lack of error checking in your code, it's fine and should write the JSON to the file NP.
I am trying to do something like this using rapidxml using c++
xml_document<> doc;
ifstream myfile("map.osm");
doc.parse<0>(myfile);
and receive the following error
Multiple markers at this line - Invalid arguments ' Candidates are: void parse(char *) ' - Symbol 'parse' could not be resolved
file size can be up to a few mega bytes.
please help
You have to load the file into a null terminated char buffer first as specified here in the official documentation.
http://rapidxml.sourceforge.net/manual.html#classrapidxml_1_1xml__document_8338ce6042e7b04d5a42144fb446b69c_18338ce6042e7b04d5a42144fb446b69c
Just read the contents of your file into a char array and use this array to pass to the xml_document::parse() function.
If you are using ifstream, you can use something like the following to read the entire file contents to a buffer
ifstream file ("test.xml");
if (file.is_open())
{
file.seekg(0,ios::end);
int size = file.tellg();
file.seekg(0,ios::beg);
char* buffer = new char [size];
file.read (buffer, size);
file.close();
// your file should now be in the char buffer -
// use this to parse your xml
delete[] buffer;
}
Please note I have not compiled the above code, just writing it from memory, but this is the rough idea. Look at the documentation for ifstream for exact details. This should help you get started anyway.
I'm pretty close to losing my head here ;)
I'm developing a service that uses gsoap. I would like to return a mime response.
I have everything working, but when reading binary files, all kind of files like jpeg, pdf, etc... contains the \0 char several times over the data (if opened with notepad can see a lot of NUL).
So any code for reading a raw file fails miserably once it finds the end-of-file char. I have tried to replace the \0 but the file becomes incorrect to display.
I have also tried several methods including the example that comes with gsoap.
So resuming,
fstream generic code doesn't work.
for (i = 0; i < MAX_FILE_SIZE; i++)
{ if ((c = fgetc(fd)) == EOF)
break;
image.__ptr[i] = c;
}
doesn't work also
QFile::ReadAll works but when converting QString to char* the array is trimmed in the first NUL.
So, which is the best aproach to read an entire binary file? Its crazy how sometimes C++ at the basic.
Thanks in advance.
I have tried this as retnick suggested below
UrlToPdf urlToPdf;
urlToPdf.getUrl(&input, &result);
QByteArray raw = urlToPdf.getPdf(QString(result.data.c_str()));
int size = raw.toBase64().size();
char* arraydata = new char[size];
strcpy(arraydata, raw.toBase64().data());
soap_set_mime(this, "MIME_boundary", NULL);
if(soap_set_mime_attachment(this, arraydata, size, SOAP_MIME_BASE64, "application/pdf", NULL, NULL, NULL))
{
soap_clr_mime(this);
soapMessage = this->error;
}
but no luck... the mime response is bigger than the actual file...
David G Ortega
to read binary files use fread()
Once you read it treat it as an array of bytes not as a string. No string functions allowed.
EDIT: The gSOAP documentation section 14.1 explains how to send MIME attachments. I only refer to the relevant function (please read it all).
int soap_set_mime_attachment(struct soap *soap, char *buf_ptr, size_t buf_size,
enum soap_mime_encoding encoding,
const char *type, const char *id,
const char *location, const char *description);
char *buf_ptr is your buffer.
size_t buf_size is the length of your buffer.
So just do your QFile::ReadAll.
this gives you back a QByteArray. The QByteArray has the method
QByteArray QByteArray::toBase64 () const
this will return a
QByteArray base64image = QByteArray::toBase64(rawImage);
so now just do
soap_set_mime(soap, "MIME_boundary", "<boundary.xml#just-testing.com>");
/* add a base64 encoded image (base64image points to base64 data) */
soap_set_mime_attachment(soap,
base64image.data(), base64image.size(),
SOAP_MIME_BASE64, "image/jpeg",
"<boundary.jpeg#just-testing.com>", NULL, NULL);
I have not tested this but should be close to finished.
QFile::ReadAll works but when converting QString to char* the array is trimmed in the first NUL.
Are you sure it's actually trimmed or you just can't print/view the array in the debugger [since C-style strings are 0 terminated]?
If the QString itself is not enough for your needs you may want to convert it to a std::vector or similar using the range constructor or range assign, you'll have lots less grief towards the how much data the container holds.
EDIT:
Here's some sample code for fstream reading from a binary file:
std::ifstream image( <image_file_name>, std::ios_base::in | std::ios_base::binary );
std::istream_iterator< char > image_begin( image ), image_end;
std::vector< char > vctImage( image_begin, image_end );
The std::ios_base::binary is the most important part of the thing (similar to fopen/fread ["rb"] & probably QFile has something similar)
Also posting some sample code usually helps in getting the right answer.
HIH
I have the solution for this... As renick suggested I tried his idea but it failed without undestanding it so much... From a logical point of view recnick was right... bat the truth is that any king of string manipulation using QT QByteArray, std or mem is going to stop when findind the first \0 char, Qt QString can do it without problems but when converting it to c string (char*) the data will be again trimmed with the first \0
I found that using QDataStream::readRawData reads the file into a char* given the size to read. So thats how I accomplished the deal...
QFile file("test.pdf");
file.open(QIODevice::ReadOnly);
int size = file.size();
char* buffer = new char[size];
QDataStream stream(&file);
stream.readRawData(buffer, size);
soap_set_mime(this, "MIME_boundary", NULL);
if(soap_set_mime_attachment(this, buffer, size, SOAP_MIME_BINARY, "application/pdf", NULL, NULL, NULL))
{
soap_clr_mime(this);
soapMessage = this->error;
}
Note that in the line
if(soap_set_mime_attachment(this, buffer, size, SOAP_MIME_BINARY, "application/pdf", NULL, NULL, NULL))
I'm still using the size var instead of doing sizeof(buffer) or any other aproach since this one is going to trimm again the data qhen finding the first \0...
Hope this helps...
David G Ortega
I'm trying to find a way to replace all instances of a string token in a file with another string.
How can I do this in C++ with the win32 API?
In other languages this is an easy thing to do, but in C++ I am just lost.
EDIT: For some context, this is for a WiX custom action. So portability is not a main priority, just the most simplest solution.
If the file fits in memory – it's simpler. Call OpenFile() to open file, GetFileSize() to determine file size, allocate enough memory, call ReadFile() to read file, then CloseFile. Do replacement in memory (use strstr() or similar function), then again OpenFile(), WriteFile(), CloseFile().
If the file is large - create a temporary file and read the source file in chunks and write filtered text to the temporary file, then call DeleteFile() to delete the original file and MoveFile() to move the filtered file.
You could use the Boost.Regex Library which should resemble most of the functionality you find on other platforms.
It would work like this:
In this example you’ll find how you can replace a string matching a pattern.
#include <boost/regex.hpp>
#include <string>
int main()
{
boost::regex pattern ("b.lug",boost::regex_constants::icase|boost::regex_constants::perl);
std::string stringa ("Searching for bolug");
std::string replace ("BgLug");
std::string newString;
newString = boost::regex_replace (stringa, pattern, replace);
printf("The new string is: |%s|\n",newString.c_str());
return 0;
}
but you would have of course to add the file reading/writing.
As per sharptooth's solution, I knocked up some C code to do a find and replace on a file. I used stdio calls (strlen, strstr, strcpy and strcat) to do the string manipulation (rather than win32 calls), so your only dependancy is the C run time.
This is certainly not code I would use in a production system. I would use stuff from toolkit string manipulation libraries to make this much cleaner (and not so much with the fixed length buffers). I probably wouldn't use boost, I don't like the overhead. But I figured you might like an example with just the basics (N.B. This writes the altered buffers out to .temp).
#include <stdio.h>
#define BUF_LEN 2048
int findAndReplace (const char * file, const char * find, const char * replace)
{
int replaceCount = 0;
FILE * f = fopen (file, "rt");
if (strstr(replace, find))
return 0; // replacing blah with stuff_blah_stuff
unsigned int findLen = strlen (find);
char tempFile [BUF_LEN];
strcpy (tempFile, file);
strcat (tempFile, ".temp");
FILE * writeF = fopen (tempFile, "wt");
if (!f || !writeF)
return 0;
printf ("Processing %s - %s to %s\n", file, find, replace);
char lineBuf [BUF_LEN];
memset (lineBuf, 0, BUF_LEN);
char tempLineBuf [BUF_LEN];
memset (tempLineBuf, 0, BUF_LEN);
// read each line of the file
while (fgets (lineBuf, BUF_LEN, f))
{
// get the position of find in the line buffer
char * pos = strstr (lineBuf, find);
while (pos)
{
strncpy (tempLineBuf, lineBuf, pos - lineBuf);
strcat (tempLineBuf, replace);
strcat (tempLineBuf, pos + findLen);
replaceCount++;
// replace the current buf with the replaced buffer
strncpy (lineBuf, tempLineBuf, BUF_LEN);
memset (tempLineBuf, 0, BUF_LEN);
pos = strstr (lineBuf, find);
}
printf ("writing new line %s\n", lineBuf);
fputs (lineBuf, writeF);
}
fclose (f);
fclose (writeF);
return replaceCount;
}
int main ()
{
printf ("Made %d replacements\n", findAndReplace ("blah.txt", "marker", "testing_blah"));
}
Why do you have to use the Win32 API? It's easy enough using straight C++, I wouldn't confuse the issue by adding artificial constraints. Just open your input file, open an output file, and read a line from your input. While you haven't hit EOF in your input file, use a regex to look for your token. If you find it, then replace it with your text. Write the line to the output file. Read another line from the input. When you get EOF on the input, close it. Be sure any pending output gets flushed from the output buffer. Close the output file. Done.