I am reading in data on JavaScript an pass the Jsonstring like that:{"data_size":500, "array":[0,0,0,0,..,0,0]} to the webserver. The numbers in the array could be anything between 0 to 4294967295.
On the Mongoose webserver I am using the lib rapidjson to work with the Jsonstring. Therefore, I create a Document d and reads values from the "jsonstring" into an uint32_t Array using this:
#include "rapidjson/document.h"
int i_data_size=0;
Document d;
conn->content[conn->content_len]=0; //to zero terminate
if (d.Parse(conn->content).HasParseError())
{
//Error
}
else
{
Value& s = d["data_size"];
i_data_size=s.GetInt();
uint32_t *Data=NULL;
Data=new uint32_t[i_data_size];
Value& a = d["array"];
for(SizeType i=0;i<a.Size();i++)
{
Data[i]=a[i].GetUint();
}
}
conn->content is containing the json char*.
When I am sending: {"data_size":500, "array":[0,0,0,0,..,0,0]} everything works find. But sometimes, not everytime, when the a number becomes greater, like this:
{"data_size":500, "array":[123,222,0,0,..,0,0]}
I get the Error:
free(): invalid next size (normal)
This error is not related to rapidJson it is from C, showing because something is messed up with memory management. C++, the heir to the language C, won't check anything regarding to memory management because it supposed to be fast and it will trust you with this. Because of that, the error you are getting won't just tell you what is wrong exactly. In your code, you are using an array but you are not managing its boundaries well, double check what you are doing regarding to reading and writing that unsigned array (check boundaries more carefully) or use a container like std::vector which will do that for you. Take a look at here:
http://www.codeguru.com/cpp/cpp/cpp_mfc/stl/article.php/c4027/C-Tutorial-A-Beginners-Guide-to-stdvector-Part-1.htm
Solved the problem!
Writing a zero out of the boundry was causing the error:
conn->content[conn->content_len]=0; //to zero terminate
Solved the problem by using a string instead:
string json="";
json = string(conn->content);
json=json.substr(0,conn->content_len);
if (d.Parse(json.c_str()).HasParseError())
{ ...
Related
I am using Rocksdb as database for the program(C++). For one of the use case, I am making a key of following format to store: key=< fix-prefix >< string-type-element >< foo-type >
I am getting the following error while accessing the "string-type-element" piece of data from the key:
Received signal: Segmentation fault (11)
The piece of code to iterate the stored data is something like:
auto prefix = // defined here
auto from = // defined here
auto to = // defined here
std::unique_ptr<rocksdb::Transaction> trans(db_.BeginTransaction(rocksdb::WriteOptions()));
rocksdb::ReadOptions opts;
opts.snapshot = trans->GetSnapshot();
std::unique_ptr<rocksdb::Iterator> iter(trans->GetIterator(opts));
iter->Seek(from);
for (; iter->Valid() && iter->key().compare(to) < 0; iter->Next())
{
if (iter->key().starts_with(prefix))
{
// This line of code is producing the error
const auto string-type-element = *reinterpret_cast<const string-type*>(iter->key().data() + prefix.size());
// some stuffs here
}
}
My Attempt:
As you can see in the piece of code above that I pointed out the buggy line. As the error is Segmentation fault (11) which usually means attempt to undefined/outside memory location, so my guess is reinterpret_cast is not able to deduce the size of "string-type-element" element as std::string is not a fixed size unlike "int" etc and it eventually end up accessing the memory portion it should not access...
I want to ask:
If it is actually the case that std::string cannot be extracted from in between of the key because of unknown size, what can we do about it?
Could there be any other reason for the issue I mentioned and how to approach it?
reinterpret_cast is a wildly dangerous tool to use and should only be used in extraordinary circumstances, which you don't have here. Look at the std::string constructors and you'll find one that accepts a pointer to a null-terminated character array. It's not clear what the type of iter->key().data() is, but if it is a null-terminated string, you could just change the line to:
const auto string-type {iter->key().data() + prefix.size()};
to initialize string-type from the part of the data after the prefix.
In your case, reinterpret_cast is trying to pretend that the data pointed to is a string object, which is more than just raw bytes of character type, but has other elements to the structure as well.
#include <iostream>
using namespace std;
int main() {
string s,s_new;
cin>>s;
int len=s.length();
cout<<len<<"\n";
for(int i=0;i<len;i++){
s_new[i]=s[i];
}
cout<<s[len-1]<<"\n";
cout<<s_new[len-1];
return 0;
}
I am trying to copy string 's' to another string 's_new'. String 's' is taken as an input from user.The code snippet outputs the string length for reference and the other two lines prints the
last character from 's' and 's_new'
But for a particular length this program is creating segmentation fault in various IDE.
For example, I got segmentation fault for length 25 in Ideone. Also in onlineGDB I got segmentation fault for length 1961.These lengths were not always constant in these IDE.
I was only using lower_case alphabetical characters for these strings and i used randomly generated strings of various lengths for these tests.
I didnot receive any error when I used character arrays for the same job.
I wanted to know if this issue was because of the code or If there is some problem in using the string STL like I did ?
s_new[i]=s[i]; has undefined behavior since
s_new.size() == 0.
You need to resize it to do what you are doing:
s_new.resize(len);
for(int i=0;i<len;i++){
s_new[i]=s[i];
}
for a particular length this program is creating segmentation fault in various IDE
Writing out-of-bounds always has undefined behavior. In this case, what happens is most likely this:
A std::string often uses small string optimization where the complete string data is stored internally in the std::string. For longer strings a pointer to the string data is used instead (but the user of the class won't notice this). When you do s_new[i] = ... and the string length is 0 and you pass the small string optimization threshold, you start overwriting other internal data in std::string or something else in the memory stored after the std::string.
I am kinda new to programming, so pardon me for any stupid questions :P, i noticed that when i write this
int main()
{
string s;
s[0]='a';
cout<<s.size();
cout<<s<<" ";
cout<<s[0];
return 0;
}
The output is 0 a , firstly why is the size 0? and why didn't anything get printed when i write cout<<s; but it gives a for cout<<s[0]. if i use push_back it gives normal out put.
int main()
{
string s;
s.push_back('a');
cout<<s.size();
cout<<s<<" ";
cout<<s[0];
return 0;
}
output :- 1a a
I might have overlooked something but i would be really appreciate if some could point out the cause.
Thank you.
EDIT: Such fast replies! thanks for your answers, i couldn't figues out how to reply to comments so edited the question body(first time using stackoverflow),
(any help on this would be appreciated as well), one more thing so when i use cout<<s[0] does it give a because a was stored on the next address of string
s?
and once again thanks for clearing that up!
What you've overlooked is that in C++ strings don't automatically grow when you assign characters to them. So
string s;
s[0]='a';
is an error because the s has size zero so there is no 'room' for the character 'a'. The correct way to add a character to a string is to use push_back which is why your second example works.
Because of the error your first example has what's called undefined behaviour (UB for short). This means the output of your program is not predictable at all, and it's more or less a waste of time asking why it outputs what it does. It could just as easily crash.
This:
string s;
creates a string containing no characters. Then this:
s[0]='a';
attempts to make a change to one of those non-existent characters. The result of this is undefined behaviour - your program goes into an unknown state.
If you would like to make your compiler warn you about this problem, you can use the at() member function of string:
s.at(0) = 'a';
Now your program will throw an std::out_of_range exception when you try to change that non-existant character.
Containers don't automatically allocate storage, so you are writing outside the allocated storage. In other words, that's a bug in your program. One advise: Many C++ implementations have a way to activate diagnostics for debugging programs, those would have caught this error.
when I write this
string s;
s[0]='a';
the output is 0, firstly why is the size 0?
The output is zero because operator[i] assumes that the string has sufficient space to store i+1 characters. In your case, string's size is zero, so accessing element at index 0 is undefined behavior.
and why didn't anything get printed when I write to cout
The same thing reason applies: after undefined behavior the program could output anything, but it happens to output nothing.
int main()
{
string s;
s='a';
cout<<s.size();
cout<<s<<" ";
cout<<s[0];
return 0;
}
Just take off [0] after s in initialisation because s is of type string not type char.
Just write s and it will work.
Im currently working on a project which uses different language settings. To solve this a table is used to store all the texts in different languages that are used in the program. So whenever a text is about to be written on the screen this table is called and depending on what the current language setting is a text string is returned. I recently joined this project and I noticed that the way of storing this wasn't very optimized and for every new language that was added the time it would take to look up the correct string would increase. I therefore came up with a (in my mind) better solution. However, when I tried to implement it I ran into the problem of getting an error that too much memory is used and I don't understand why. I am using IAR enbedded workbench.
The original solution in pseudo/c++ code:
typedef struct
{
enum textId;
enum language;
string textString;
} Text;
static const Text s_TextMap[] =
{
{ TextId::RESTORE_DATA_Q ,Language::ENGLISH ,"Restore Data?" },
{ TextId::RESTORE_DATA_Q ,Language::SWEDISH ,"Återställa data?" },
{ TextId::RESTORE_DATA_Q ,Language::GERMAN ,"Wiederherstellen von Daten?" },
{ TextId::CHANGE_LANGUAGE ,Language::ENGLISH ,"Change Language" },
{ TextId::CHANGE_LANGUAGE ,Language::SWEDISH ,"Välj språk" },
{ TextId::CHANGE_LANGUAGE ,Language::GERMAN ,"Sprache wählen" },
};
My solution in pseudo/c++ code:
typedef struct
{
const char* pEngText;
const char* pSweText;
const char* pGerText;
} Texts;
static Texts addTexts(const char* pEngText, const char* pSweText, const char* pGerText)
{
Texts t;
t.pEngText = pEngText;
t.pSweText = pSweText;
t.pGerText = pGerText;
return t;
}
typedef struct
{
enum textId;
Texts texts;
} Text;
static const TextTest s_TextMapTest[] =
{
{TextId::RESTORE_DATA_Q, addTexts("Restore Data?","Återställa data?","Wiederherstellen von Daten?")},
{TextId::CHANGE_LANGUAGE, addTexts("Change Language","Välj språk","Sprache wählen")},
};
My solution is obviously faster to lookup in the average case and based on my calculations it should also use less memory. When the full tables are used I've calculated that the original solution requires 7668 bytes and that my solution requires 4248 bytes. The way I did this was to implement the full tables in a small testprogram and use sizeof(s_TextMap). However, when I try to compile the code I get linking errors saying:
Error[Lp011]: section placement failed
unable to allocate space for sections/blocks with a total estimated minimum size of 0x130301 bytes (max align 0x1000) in <[0x0000a000-0x0007ffff]> (total uncommitted space 0x757eb).
Error[Lp011]: section placement failed
unable to allocate space for sections/blocks with a total estimated minimum size of 0x47de4 bytes (max align 0x20) in <[0x1fff0000-0x2000fff0]> (total uncommitted space 0x1fff1).
Error[Lp021]: the destination for compressed initializer batch "USER_DEFAULT_MEMORY-1" is placed at an address that is dependent on the size of the batch, which is not allowed when using lz77 compression. Consider using "initialize by copy with packing = zeros" (or none) instead.
Error[Lp021]: the destination for compressed initializer batch "USER_DEFAULT_MEMORY-1" is placed at an address that is dependent on the size of the batch, which is not allowed when using lz77 compression. Consider using "initialize by copy with packing = zeros" (or none) instead.
The error I'm most confused about is the first one which states that my code is estimated to take 0x130301 bytes of memory and I see no way this is possible. Could this be some bug in IAR or am I missing something?
In your solution s_TextMapTest[] will necessarily be located in RAM rather than ROM because the pointers are set at run-time - though it is not clear how you managed to use a function call as an array element initialiser. On most microcontrollers RAM is a far more limited resource. You have provided no information about the target or its memory map.
Either way you should check the memory map output from your linker to verify that data is located appropriately and where you expect it to be.
The original code and the solution you propose in your own answer is ROMable.
A solution that maintains the form of your original proposal while remaining ROMable is to write addTexts() as a macro rather than a function:
#define addTexts( eng, swe, ger, fin ) {eng, swe, ger, fin}
though I cannot see what the advantage is, and you are not really "adding" text - the text was always there by initialisation.
Ok, so I managed to get it to work. I remowed the extra struct and the functions and just simplified it to:
typedef struct
{
TextId::e textId;
const char* pEngText;
const char* pSweText;
const char* pGerText;
const char* pFinText;
} Text;
It doesn't look as good to me but at least it works
I'm currently unpacking one of blizzard's .mpq file for reading.
For accessing the unpacked char buffer, I'm using a boost::interprocess::stream::memorybuffer.
Because .mpq files have a chunked structure always beginning with a version header (usually 12 bytes, see http://wiki.devklog.net/index.php?title=The_MoPaQ_Archive_Format#2.2_Archive_Header), the char* array representation seems to truncate at the first \0, even if the filesize (something about 1.6mb) remains constant and (probably) always allocated.
The result is a streambuffer with an effective length of 4 ('REVM' and byte nr.5 is \0). When attempting to read further, an exception is thrown. Here an example:
// (somewhere in the code)
{
MPQFile curAdt(FilePath);
size_t size = curAdt.getSize(); // roughly 1.6 mb
bufferstream memorybuf((char*)curAdt.getBuffer(), curAdt.getSize());
// bufferstream.m_buf.m_buffer is now 'REVM\0' (Debugger says so),
// but internal length field still at 1.6 mb
}
//////////////////////////////////////////////////////////////////////////////
// wrapper around a file oof the mpq_archive of libmpq
MPQFile::MPQFile(const char* filename) // I apologize my naming inconsistent convention :P
{
for(ArchiveSet::iterator i=gOpenArchives.begin(); i!=gOpenArchives.end();++i)
{
// gOpenArchives points to MPQArchive, wrapper around the mpq_archive, has mpq_archive * mpq_a as member
mpq_archive &mpq_a = (*i)->mpq_a;
// if file exists in that archive, tested via hash table in file, not important here, scroll down if you want
mpq_hash hash = (*i)->GetHashEntry(filename);
uint32 blockindex = hash.blockindex;
if ((blockindex == 0xFFFFFFFF) || (blockindex == 0)) {
continue; //file not found
}
uint32 fileno = blockindex;
// Found!
size = libmpq_file_info(&mpq_a, LIBMPQ_FILE_UNCOMPRESSED_SIZE, fileno);
// HACK: in patch.mpq some files don't want to open and give 1 for filesize
if (size<=1) {
eof = true;
buffer = 0;
return;
}
buffer = new char[size]; // note: size is 1.6 mb at this time
// Now here comes the tricky part... if I step over the libmpq_file_getdata
// function, I'll get my truncated char array, which I absolutely don't want^^
libmpq_file_getdata(&mpq_a, hash, fileno, (unsigned char*)buffer);
return;
}
}
Maybe someone could help me. I'm really new to STL and boost programming and also inexperienced in C++ programming anyways :P Hope to get a convenient answer (plz not suggest to rewrite libmpq and the underlying zlib architecture^^).
The MPQFile class and the underlying uncompress methods are acutally taken from a working project, so the mistake is either somewhere in the use of the buffer with the streambuffer class or something internal with char array arithmetic I haven't a clue of.
By the way, what is the difference between using signed/unsigned chars as data buffers? Has it anything to do with my problem (you might see, that in the code randomly char* unsigned char* is taken as function arguments)
If you need more infos, feel free to ask :)
How are you determining that your char* array is being 'truncated' as you call it? If you're printing it or viewing it in a debugger it will look truncated because it will be treated like a string, which is terminated by \0. The data in 'buffer' however (assuming libmpq_file_getdata() does what it's supposed to do) will contain the whole file or data chunk or whatever.
Sorry, messed up a bit with these terms (not memorybuffer actually, streambuffer is meant as in the code)
Yeah you where right... I had a mistake in my exception handling. Right after that first bit of code comes this:
// check if the file has been open
//if (!mpf.is_open())
pair<char*, size_t> temp = memorybuf.buffer();
if(temp.first)
throw AdtException(ADT_PARSEERR_EFILE);//Can't open the File
notice the missing ! before temp.first . I was surprized by the exception thrown, looked at the streambuffer .. internal buffer at was confused of its length (C# background :P).
Sorry for that, it's working as expected now.