Here's the situation.
Let's say I have a gltf file containing :
"uri" : "data:application/gltf-buffer;base64,AAAIAAcAAAABAAgAAQAJAAgAAQACAAkAAgAKAAkAAgADAAoAAwALAAoAAwAEAAsABAAMAAsABAAFAAwABQANAAwABQAGAA0AAAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAAAAAQAAAAAAAAAAAAABAQAAAAAAAAAAAAACAQAAAAAAAAAAAAACgQAAAAAAAAAAAAADAQAAAAAAAAAAAAAAAAAAAgD8AAAAAAACAPwAAgD8AAAAAAAAAQAAAgD8AAAAAAABAQAAAgD8AAAAAAACAQAAAgD8AAAAAAACgQAAAgD8AAAAAAADAQAAAgD8AAAAACAAKAAwAAAAAAIA/AAAAQAAAAAAAAEBAAABAQAAAAAAAAKBAAACAQAAAAAA=",
I understand how to extract the type and the base using QJsonValue::toString() and QString parsing functions (like split(";")) but my problems remain : I have now a string that correspond to my gltf buffer and I don't know how to read it. (I know what it is, an d how the data should be interpreted thanks to accessors and bufferviews)
tldr: how to convert that string of letters to a QByteArray ?
you tried it like that?))
QByteArray by = QByteArray::fromBase64("AAAIAAcAAAABAAgAAQAJAAgAAQACAAkAAgAKAAkAAgADAAoAAwALAAoAAwAEAAsABAAMAAsABAAFAAwABQANAAwABQAGAA0AAAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAAAAAQAAAAAAAAAAAAABAQAAAAAAAAAAAAACAQAAAAAAAAAAAAACgQAAAAAAAAAAAAADAQAAAAAAAAAAAAAAAAAAAgD8AAAAAAACAPwAAgD8AAAAAAAAAQAAAgD8AAAAAAABAQAAAgD8AAAAAAACAQAAAgD8AAAAAAACgQAAAgD8AAAAAAADAQAAAgD8AAAAACAAKAAwAAAAAAIA/AAAAQAAAAAAAAEBAAABAQAAAAAAAAKBAAACAQAAAAAA=");
Related
I am serializing multiple objects and want to save the given Strings to a file. The structure is the following:
A few string and long attributes, then a variable amount of maps<long, map<string, variant> >. My first idea was creating one valid JSONFile but this is very hard to do (all of the maps are very big and my temporary memory is not big enough). Since I cant serialize everything together I have to do it piece by piece. I am planning on doing that and I then want to save the recieved strings to a file. Here is how it will look like:
{ "Name": "StackOverflow"}
{"map1": //map here}
{"map2": //map here}
As you can see this is not one valid JSON object but 3 valid JSONObjects in one file. Now I want to deserialize and I need to give a valid JSONObject to the deserializer. I already save tellp() everytime when I write a new JSONObject to file, so in this example I would have the following adresses saved: 26, endofmap1, endofmap2.
Here is what I want to do: I want to use these addresses, to extract the strings from the file I wrote to. I need one string which is from 0 to (26-1), one string from 26 to(endofmap1-1) and one string from endofmap1 to (endofmap2-1). Since these strings would be valid JSONObjects i could deserialize them without problem.
How can I do this?
I would create a serialize and deserialize class that you can use as part of a hierarchy.
So for instance, in rough C++ psuedo-code:
class Object : public serialize, deserialize {
public:
int a;
float b;
Compound c;
bool serialize(fstream& fs) {
fs << a;
fs << b;
c->serialize(fs);
fs->flush();
}
// same for deserialize
};
class Compound : serialize, deserialize {
public:
map<> things;
bool serialize(fstream& fs) {
for(thing : things) {
fs << thing;
}
fs->flush();
}
};
With this you can use JSON as the file will be written as your walk the heirarchy.
Update:
To extract a specific string from a file you can use something like this:
// pass in an open stream (streams are good for unit testing!)
std::string extractString(fstream& fs) {
int location = /* the location of the start from file */;
int length = /* length of the string you want to extract */;
std::string str;
str.resize(length);
char* begin = *str.begin();
fs->seekp(location);
fs->read(begin, length);
return str;
}
Based on you saying "my temporary memory is not big enough", I'm going to assume two possibilities (though some kind of code example may help us help you!).
possibility one, the file is too big
The issue you would be facing here isn't a new one - a file too large for memory, assuming your algorithm isn't buffering all the data, and your stack can handle the recursion of course.
On windows you can use the MapViewOfFile function, the MSDN has plenty of detail on that. This function will effectively grab a "view" of a section of a file - allowing you to load enough of the file to modify only what you need, before closing and opening a view at a later offset.
If you are on a different platform, there will be similar functions.
possibility two, you are doing too much at once
The other option is more of a "software engineering" issue. You have so much data then when holding them in your std::maps, you run out of heap-memory.
If this is the case, you are going to need to use some clever thinking - here are some ideas!
Don't load all your data into the maps. wherever the data is coming from, take a CRC, Index, or Filename of the data-source. Store that information in the map, and leave the actual "big strings" on the hard disk. - This way you can load each item of data when you need it.
This works really well for data that needs to be sorted, or correlated.
Process or load your data when you need to write it. If you don't need to sort or correlate the data, why load it into a map beforehand at all? Just load each "big string" of data in sequence, then write them to the file with an ofstream.
Unity's IL2CPP/LLVM back-end generates cpp files from C#. You can certainly read these files, and drop breakpoints into them to view variables. However C# strings are transcompiled into a custom class called String_t. Xcode doesn't seem to know how to print the inner strings of these classes and I'm not sure how to read the raw bytes.
Here's the String_t definition:
struct String_t : public Object_t
{
// System.Int32 System.String::length
int32_t ___length;
// System.Char System.String::start_char
uint16_t ___start_char;
};
...can anyone figure out how to read the contained string from an Xcode breakpoint?
You can use p il2cpp::utils::StringUtils::Utf16ToUtf8. So if the name of the variable is L_3 for example, then you can do this:
p il2cpp::utils::StringUtils::Utf16ToUtf8(&L_3->___start_char_1)
You can see the string in memory if you right-click the string in the locals window and select View Memory of "*foo". Then the string starts 12 bytes in. Because of IL2CPP's 16bit characters, the string is printed with dots inbetween. This probably doesn't work for unicode characters!
If anyone has a more robust solution I'd love to accept their answer.
I'd like to use libusb to retrieve information about my devices. I can read every descriptor and print every number associated inside theese descriptors.
But I have troubles with the strings. How can I manage the string descriptors in a good way with c++?
I'd like to implement a simple function like this:
std::string get_string(std::uint8_t index);
which internally retrieves the string associated a index. The device handle is got from the attributes of the class(the function is a class member) and the buffer where the libusb_get_string_descriptor is allocated statically because seeing that the dimension is contained in a 8bit field the length must be at most 256 charachters, mustn't it?
How can I manage unicode with theese things? Any ideas? Is right the use of the std::string?
Im using qt 5.0 and its support such classes as QJsonObject QJsonDocument and QJsonArray. In my programm i need to serialize json array and convert it to qstring/qbytearray but i didn't found any serialize or encode methods in those classes. Is there any way i can serialize data using included qt 5.0. libs? I found this example:
QVariant id(1), name("John Doe");
QJsonObject json;
json["Name"] = name.toString();
json.insert("id", id.toInt());
But i can't find how i can make an array from it.
Question closed. Use QJsonDocument::toJson to get data from QJsonObject.
I think the secret is that when serializing to JSON, Qt writes out a UTF-8 binary encoded version of the text into a QByteArray. You can recover the string using QString::fromUtf8. Also, even though toJson writes out a QByteArray it is not a true binary representation; as I just mentioned, it is the bytes of the UTF-8 encoded string representation of the JSON. For binary serialization you must use either Qt's own QJsonDocument::toBinaryData or find a BSON library.
Text serialization
Use QJsonDocument::toJson to serialize as text via a QString, and QJsonDocument::fromJson to deserialize from text.
Tip: Consider passing the QJsonDocument::Compact format option to get compact JSON output during serialization.
To QString
QString json = QString::fromUtf8(doc.toJson(QJsonDocument::Compact));
From QString
QJsonDocument doc = QJsonDocument::fromJson(json);
Binary serialization
Use QJsonDocument::toBinaryData to serialize as binary via a QByteArray, and QJsonDocument::fromBinaryData to deserialize from binary.
Caveat: The binary format used by the above functions is Qt's own binary format (qbjs) so may not be suitable for interoperation with other binary formats such as BSON. For that, you might want to look into choosing a BSON implementation.
To QByteArray
QByteArray bytes = doc.toBinaryData();
From QByteArray
QJsonDocument doc = QJsonDocument::fromBinaryData(bytes);
I created an xml file using libxml and c++. What I want to do now, is reading from a .txt and put this text between some specific tags.
I have tried the following code, just reading from a file and write it between tags:
char * s ;
double d;
fichier>>i>>s>>d;
// fichier.close();
cout << s << endl ;
xmlNewChild(root_node, NULL, BAD_CAST "metadata",
BAD_CAST s );
While running this code, I get this error:
output error : string is not in UTF-8
So I guess that there is a format incompatibility between the input and output. Can you help me please? I don't know how to fix this.
You need to convert your input string into UTF-8 input using one of the functions defined in the encoding module. (Or using any other encoding library you like like icu ) you can find details about the encoding module here http://www.xmlsoft.org/html/libxml-encoding.html
My guess is that you want to preserve the bytes so that what you need is something like (VERY untested and derived purely from the docs.)
//Get the encoding
xmlCharEncodingHandlerPtr encoder = xmlGetCharEncodingHandler(XML_CHAR_ENCODING_ASCII);
// Each ascii byte should take up at most 2 utf-8 bytes IIRC so allocate enough space.
char* buffer_utf8 = new char[length_of_s*2];
//Do the encoding
int consumed = length_of_s;
int encoded_length=length_of_s*2;
int len = (*encoder.input)(buffer_utf8, &encoded,s,&consumed);
if( len<0 ) { .. error .. }
buffer_utf8[len]=0; // I'm not sure if this is automatically appended or not.
//Now you can use buffer_utf8 rather than s.
If your input is in a different encoding supported by libxml then it should just be a matter of changing XML_CHAR_ENCODING_ASCII to the right constant, though you may need to change thenumber of bytes allocated in the in buffer_utf8 too.