I want a function that will take a time_t parameter and an arbitrary format string and format it. I want something like this:
std::string GetTimeAsString(std::string formatString, time_t theTime)
{
struct tm *timeinfo;
timeinfo = localtime( &theTime);
char buffer[100];
strftime(buffer, 100, formatString.c_str(), timeinfo);
std::string result(buffer);
return result;
}
However one problem I'm running into is the buffer length. I was thinking of doing something like formatString * 4 as the buffer length. But I guess you can't dynamically set the buffer length? Maybe I could pick an arbitrarily large buffer? I'm a little stuck as to how to make it generic.
How can I write a function to achieve this?
If you have C++11:
std::string GetTimeAsString(std::string formatString, time_t theTime)
{
struct tm *timeinfo;
timeinfo = localtime( &theTime);
formatString += '\a'; //force at least one character in the result
std::string buffer;
buffer.resize(formatstring.size());
int len = strftime(&buffer[0], buffer.size(), formatString.c_str(), timeinfo);
while (len == 0) {
buffer.resize(buffer.size()*2);
len = strftime(&buffer[0], buffer.size(), formatString.c_str(), timeinfo);
}
buffer.resize(len-1); //remove that trailing '\a'
return buffer;
}
Note I take formatString as a const reference, (for speed and safety), and use the result string as the buffer, which is faster than doing an extra copy later. I also start at the same size as the formatstring, and double the size with each attempt, but that's easily changable to something more appropriate for the results of strftime.
C++11 solution with std::put_time():
std::string GetTimeAsString(std::string formatString, time_t theTime)
{
const struct tm* timeinfo = localtime(&theTime);
std::ostringstream os;
os << std::put_time(timeinfo, formatString.c_str());
return os.str();
}
Use a vector<char> for the buffer instead of an array. Repeatedly increase the size until strftime returns non-zero.
I would think your best bet would be to provide a fixed buffer that is likely to handle the vast majority of cases, and then do special handling for the rest. Something like (untested, except in the wetware inside my skull):
std::string GetTimeAsString (std::string formatString, time_t theTime) {
struct tm *timeinfo;
char buffer[100], *pBuff = buffer;
int rc, buffSize = 100;
timeinfo = localtime (&theTime);
rc = strftime(pBuff, 100, formatString.c_str(), timeinfo);
// Most times, we shouldn't enter this loop.
while (rc == 0) {
// Free previous in it was allocated.
if (pBuff != buffer)
delete[] pBuff;
// Try with larger buffer.
buffSize += 100;
pBuff = new char [buffSize];
rc = strftime(pBuff, buffSize, formatString.c_str(), timeinfo);
}
// Make string then free buffer if it was allocated.
std::string result(pBuff);
if (pBuff != buffer)
delete[] pBuff;
return result;
}
strftime will return zero if the provided buffer wasn't big enough. In that case, you start allocating bigger buffers until it fits.
Your non-allocated buffer size and the increment you use for allocation size can be tuned to your needs. This method has the advantage that you won't notice an efficiency hit (however small it may be) except for the rare cases - no allocation is done for that vast majority.
In addition, you could choose some other method (e.g., +10%, doubling, etc) for increasing the buffer size.
The strftime() function returns 0 if the buffer's size is too small to hold the expected result. Using this property, you could allocate the buffer on the heap and try the consecutive powers of 2 as its size: 1, 2, 4, 8, 16 etc. until the buffer is big enough. The advantage of using the powers of 2 is that the solution's complexity is logarithmically proportional to the result's length.
There's also a special case that needs to be thought of: the format might be such that the result's size will always be 0 (e.g. an empty format). Not sure how to handle that.
Related
I have a char array defined like this
char buffer[100];
When I run Flawfinder scan for hits I get the one says:
(buffer) char:
Statically-sized arrays can be improperly restricted, leading to potential
overflows or other issues (CWE-119!/CWE-120). Perform bounds checking, use
functions that limit length, or ensure that the size is larger than the
maximum possible length.
I know I have to do the checks when needed to make sure my code will be exceptions free but do we have any way to solve this (define a char array in other ways) and make the Flawfindr output without any hit?
UPDATE
Here's the full code of the function in case it would help
std::string MyClass::randomGenerator(odb::nullable<int> maxLength) {
struct timeval tmnow;
struct tm *tm;
char buf[100];
gettimeofday(&tmnow, NULL);
tm = localtime(&tmnow.tv_sec);
strftime(buf, 100, "%m%d%H%M%S", tm);
string micro = std::to_string(((int)tmnow.tv_usec / 10000));
strlcat(buf, micro.c_str(), sizeof(buf));
std::stringstream stream;
stream << std::hex << stoll(buf);
std::string result(stream.str());
Utilities::find_and_replace(result, "0", "h");
Utilities::find_and_replace(result, "1", "k");
std::transform(result.begin(), result.end(),result.begin(), ::toupper);
if (maxLength) {
return result.substr(result.size() - maxLength.get(), result.size() - 1);
} else {
return result ;
}
}
Flawfinder is really a slightly glorified grep - it's not a true static-analysis tool that does data flow analysis, so I have always taken its output with a healthy dose of salt!
The way you should really write this code is to write true C++ code rather than glorified-C using C runtime functions, which are absolutely subject to memory corruption issues.
This routine is called a zillion times to create large csv files full of numbers. Is there a more efficient way to to this?
static std::string dbl2str(double d)
{
std::stringstream ss;
ss << std::fixed << std::setprecision(10) << d; //convert double to string w fixed notation, hi precision
std::string s = ss.str(); //output to std::string
s.erase(s.find_last_not_of('0') + 1, std::string::npos); //remove trailing 000s (123.1200 => 123.12, 123.000 => 123.)
return (s[s.size()-1] == '.') ? s.substr(0, s.size()-1) : s; //remove dangling decimal (123. => 123)
}
Before you start, check whether significant time is spent in this function. Do this by measuring, either with a profiler or otherwise. Knowing that you call it a zillion times is all very well, but if it turns out your program still only spends 1% of its time in this function, then nothing you do here can possibly improve your program's performance by more than 1%. If that were the case the answer to your question would be "for your purposes no, this function cannot be made significantly more efficient and you are wasting your time if you try".
First thing, avoid s.substr(0, s.size()-1). This copies most of the string and it makes your function ineligible for NRVO, so I think generally you'll get a copy on return. So the first change I'd make is to replace the last line with:
if(s[s.size()-1] == '.') {
s.erase(s.end()-1);
}
return s;
But if performance is a serious concern, then here's how I'd do it. I'm not promising that this is the fastest possible, but it avoids some issues with unnecessary allocations and copying. Any approach involving stringstream is going to require a copy from the stringstream to the result, so we want a more low-level operation, snprintf.
static std::string dbl2str(double d)
{
size_t len = std::snprintf(0, 0, "%.10f", d);
std::string s(len+1, 0);
// technically non-portable, see below
std::snprintf(&s[0], len+1, "%.10f", d);
// remove nul terminator
s.pop_back();
// remove trailing zeros
s.erase(s.find_last_not_of('0') + 1, std::string::npos);
// remove trailing point
if(s.back() == '.') {
s.pop_back();
}
return s;
}
The second call to snprintf assumes that std::string uses contiguous storage. This is guaranteed in C++11. It is not guaranteed in C++03, but is true for all actively-maintained implementations of std::string known to the C++ committee. If performance really is important then I think it's reasonable to make that non-portable assumption, since writing directly into a string saves copying into a string later.
s.pop_back() is the C++11 way of saying s.erase(s.end()-1), and s.back() is s[s.size()-1]
For another possible improvement, you could get rid of the first call to snprintf and instead size your s to some value like std::numeric_limits<double>::max_exponent10 + 14 (basically, the length that -DBL_MAX needs). The trouble is that this allocates and zeros far more memory than is typically needed (322 bytes for an IEEE double). My intuition is that this will be slower than the first call to snprintf, not to mention wasteful of memory in the case where the string return value is kept hanging around for a while by the caller. But you can always test it.
Alternatively, std::max((int)std::log10(d), 0) + 14 computes a reasonably tight upper bound on the size needed, and might be quicker than snprintf can compute it exactly.
Finally, it may be that you can improve performance by changing the function interface. For example, instead of returning a new string you could perhaps append to a string passed in by the caller:
void append_dbl2str(std::string &s, double d) {
size_t len = std::snprintf(0, 0, "%.10f", d);
size_t oldsize = s.size();
s.resize(oldsize + len + 1);
// technically non-portable
std::snprintf(&s[oldsize], len+1, "%.10f", d);
// remove nul terminator
s.pop_back();
// remove trailing zeros
s.erase(s.find_last_not_of('0') + 1, std::string::npos);
// remove trailing point
if(s.back() == '.') {
s.pop_back();
}
}
Then the caller can reserve() plenty of space, call your function several times (presumably with other string appends in between), and write the resulting block of data to the file all at once, without any memory allocation other than the reserve. "Plenty" doesn't have to be the whole file, it could be one line or "paragraph" at a time, but anything that avoids a zillion memory allocations is a potential performance boost.
Efficient in terms of speed or brevity?
char buf[64];
sprintf(buf, "%-.*G", 16, 1.0);
cout << buf << endl;
Displays "1". Formats up to significant 16 digits, with no trailing zeros, before reverting to scientific notation.
use snprintf and an array of char instead of stringstream and string
pass a pointer to char buffer to dbl2str into which it prints (in order to avoid the copy constructor of string called when returning). Assemble the string to be printed in a character buffer (or convert the char buffer when called to a string or add it to an existing string)
declare the function inline in a header file
#include <cstdio>
inline void dbl2str(char *buffer, int bufsize, double d)
{
/** the caller must make sure that there is enough memory allocated for buffer */
int len = snprintf(buffer, bufsize, "%lf", d);
/* len is the number of characters put into the buffer excluding the trailing \0
so buffer[len] is the \0 and buffer[len-1] is the last 'visible' character */
while (len >= 1 && buffer[len-1] == '0')
--len;
/* terminate the string where the last '0' character was or overwrite the existing
0 if there was no '0' */
buffer[len] = 0;
/* check for a trailing decimal point */
if (len >= 1 && buffer[len-1] == '.')
buffer[len-1] = 0;
}
I'm having a string is not null terminated error, though I'm not entirely sure why. The usage of std::string in the second part of the code is one of my attempt to fix this problem, although it still doesn't work.
My initial codes was just using the buffer and copy everything into client_id[]. The error than occurred. If the error is correct, that means I've got either client_ id OR theBuffer does not have a null terminator. I'm pretty sure client_id is fine, since I can see it in debug mode. Strange thing is buffer also has a null terminator. No idea what is wrong.
char * next_token1 = NULL;
char * theWholeMessage = &(inStream[3]);
theTarget = strtok_s(theWholeMessage, " ",&next_token1);
sendTalkPackets(next_token1, sizeof(next_token1) + 1, id_clientUse, (unsigned int)std::stoi(theTarget));
Inside sendTalkPackets is. I'm getting a string is not null terminated at the last line.
void ServerGame::sendTalkPackets(char * buffer, unsigned int buffersize, unsigned int theSender, unsigned int theReceiver)
{
std::string theMessage(buffer);
theMessage += "0";
const unsigned int packet_size = sizeof(Packet);
char packet_data[packet_size];
Packet packet;
packet.packet_type = TALK;
char client_id[MAX_MESSAGE_SIZE];
char theBuffer[MAX_MESSAGE_SIZE];
strcpy_s(theBuffer, theMessage.c_str());
//Quick hot fix for error "string not null terminated"
const char * test = theMessage.c_str();
sprintf_s(client_id, "User %s whispered: ", Usernames.find(theSender)->second.c_str());
printf("This is it %s ", buffer);
strcat_s(client_id, buffersize , theBuffer);
Methinks that problem lies in this line:
sendTalkPackets(next_token1, sizeof(next_token1) + 1, id_clientUse, (unsigned int)std::stoi(theTarget));
sizeof(next_token1)+1 will always gives 5 (on 32 bit platform) because it return size of pointer not size of char array.
One thing which could be causing this (or other problems): As
buffersize, you pass sizeof(next_token1) + 1. next_token1 is
a pointer, which will have a constant size of (typically) 4 or 8. You
almost certainly want strlen(next_token1) + 1. (Or maybe without the
+ 1; conventions for passing sizes like this generally only include
the '\0' if it is an output buffer. There are a couple of other
places where you're using sizeof, which may have similar problems.
But it would probably be better to redo the whole logic to use
std::string everywhere, rather than all of these C routines. No
worries about buffer sizes and '\0' terminators. (For protocol
buffers, I've also found std::vector<char> or std::vector<unsigned char>
quite useful. This was before the memory in std::string was
guaranteed to be contiguous, but even today, it seems to correspond more
closely to the abstraction I'm dealing with.)
You can't just do
std::string theMessage(buffer);
theMessage += "0";
This fails on two fronts:
The std::string constructor doesn't know where buffer ends, if buffer is not 0-terminated. So theMessage will potentially be garbage and include random stuff until some zero byte was found in the memory beyond the buffer.
Appending string "0" to theMessage doesn't help. What you want is to put a zero byte somewhere, not value 0x30 (which is the ascii code for displaying a zero).
The right way to approach this, is to poke a literal zero byte buffersize slots beyond the start of the buffer. You can't do that in buffer itself, because buffer may not be large enough to accomodate that extra zero byte. A possibility is:
char *newbuffer = malloc(buffersize + 1);
strncpy(newbuffer, buffer, buffersize);
newbuffer[buffersize] = 0; // literal zero value
Or you can construct a std::string, whichever you prefer.
This routine is called a zillion times to create large csv files full of numbers. Is there a more efficient way to to this?
static std::string dbl2str(double d)
{
std::stringstream ss;
ss << std::fixed << std::setprecision(10) << d; //convert double to string w fixed notation, hi precision
std::string s = ss.str(); //output to std::string
s.erase(s.find_last_not_of('0') + 1, std::string::npos); //remove trailing 000s (123.1200 => 123.12, 123.000 => 123.)
return (s[s.size()-1] == '.') ? s.substr(0, s.size()-1) : s; //remove dangling decimal (123. => 123)
}
Before you start, check whether significant time is spent in this function. Do this by measuring, either with a profiler or otherwise. Knowing that you call it a zillion times is all very well, but if it turns out your program still only spends 1% of its time in this function, then nothing you do here can possibly improve your program's performance by more than 1%. If that were the case the answer to your question would be "for your purposes no, this function cannot be made significantly more efficient and you are wasting your time if you try".
First thing, avoid s.substr(0, s.size()-1). This copies most of the string and it makes your function ineligible for NRVO, so I think generally you'll get a copy on return. So the first change I'd make is to replace the last line with:
if(s[s.size()-1] == '.') {
s.erase(s.end()-1);
}
return s;
But if performance is a serious concern, then here's how I'd do it. I'm not promising that this is the fastest possible, but it avoids some issues with unnecessary allocations and copying. Any approach involving stringstream is going to require a copy from the stringstream to the result, so we want a more low-level operation, snprintf.
static std::string dbl2str(double d)
{
size_t len = std::snprintf(0, 0, "%.10f", d);
std::string s(len+1, 0);
// technically non-portable, see below
std::snprintf(&s[0], len+1, "%.10f", d);
// remove nul terminator
s.pop_back();
// remove trailing zeros
s.erase(s.find_last_not_of('0') + 1, std::string::npos);
// remove trailing point
if(s.back() == '.') {
s.pop_back();
}
return s;
}
The second call to snprintf assumes that std::string uses contiguous storage. This is guaranteed in C++11. It is not guaranteed in C++03, but is true for all actively-maintained implementations of std::string known to the C++ committee. If performance really is important then I think it's reasonable to make that non-portable assumption, since writing directly into a string saves copying into a string later.
s.pop_back() is the C++11 way of saying s.erase(s.end()-1), and s.back() is s[s.size()-1]
For another possible improvement, you could get rid of the first call to snprintf and instead size your s to some value like std::numeric_limits<double>::max_exponent10 + 14 (basically, the length that -DBL_MAX needs). The trouble is that this allocates and zeros far more memory than is typically needed (322 bytes for an IEEE double). My intuition is that this will be slower than the first call to snprintf, not to mention wasteful of memory in the case where the string return value is kept hanging around for a while by the caller. But you can always test it.
Alternatively, std::max((int)std::log10(d), 0) + 14 computes a reasonably tight upper bound on the size needed, and might be quicker than snprintf can compute it exactly.
Finally, it may be that you can improve performance by changing the function interface. For example, instead of returning a new string you could perhaps append to a string passed in by the caller:
void append_dbl2str(std::string &s, double d) {
size_t len = std::snprintf(0, 0, "%.10f", d);
size_t oldsize = s.size();
s.resize(oldsize + len + 1);
// technically non-portable
std::snprintf(&s[oldsize], len+1, "%.10f", d);
// remove nul terminator
s.pop_back();
// remove trailing zeros
s.erase(s.find_last_not_of('0') + 1, std::string::npos);
// remove trailing point
if(s.back() == '.') {
s.pop_back();
}
}
Then the caller can reserve() plenty of space, call your function several times (presumably with other string appends in between), and write the resulting block of data to the file all at once, without any memory allocation other than the reserve. "Plenty" doesn't have to be the whole file, it could be one line or "paragraph" at a time, but anything that avoids a zillion memory allocations is a potential performance boost.
Efficient in terms of speed or brevity?
char buf[64];
sprintf(buf, "%-.*G", 16, 1.0);
cout << buf << endl;
Displays "1". Formats up to significant 16 digits, with no trailing zeros, before reverting to scientific notation.
use snprintf and an array of char instead of stringstream and string
pass a pointer to char buffer to dbl2str into which it prints (in order to avoid the copy constructor of string called when returning). Assemble the string to be printed in a character buffer (or convert the char buffer when called to a string or add it to an existing string)
declare the function inline in a header file
#include <cstdio>
inline void dbl2str(char *buffer, int bufsize, double d)
{
/** the caller must make sure that there is enough memory allocated for buffer */
int len = snprintf(buffer, bufsize, "%lf", d);
/* len is the number of characters put into the buffer excluding the trailing \0
so buffer[len] is the \0 and buffer[len-1] is the last 'visible' character */
while (len >= 1 && buffer[len-1] == '0')
--len;
/* terminate the string where the last '0' character was or overwrite the existing
0 if there was no '0' */
buffer[len] = 0;
/* check for a trailing decimal point */
if (len >= 1 && buffer[len-1] == '.')
buffer[len-1] = 0;
}
I have a need to serialize int, double, long, and float
into a character buffer and this is the way I currently do it
int value = 42;
char* data = new char[64];
std::sprintf(data, "%d", value);
// check
printf( "%s\n", data );
First I am not sure if this is the best way to do it but my immediate problem is determining the size of the buffer. The number 64 in this case is purely arbitrary.
How can I know the exact size of the passed numeric so I can allocate exact memory; not more not less than is required?
Either a C or C++ solution is fine.
EDIT
Based on Johns answer ( allocate large enough buffer ..) below, I am thinking of doing this
char *data = 0;
int value = 42;
char buffer[999];
std::sprintf(buffer, "%d", value);
data = new char[strlen(buffer)+1];
memcpy(data,buffer,strlen(buffer)+1);
printf( "%s\n", data );
Avoids waste at a cost of speed perhaps. And does not entirely solve the potential overflow Or could I just use the max value sufficient to represent the type.
In C++ you can use a string stream and stop worrying about the size of the buffer:
#include <sstream>
...
std::ostringstream os;
int value=42;
os<<42; // you use string streams as regular streams (cout, etc.)
std::string data = os.str(); // now data contains "42"
(If you want you can get a const char * from an std::string via the c_str() method)
In C, instead, you can use the snprintf to "fake" the write and get the size of the buffer to allocate; in facts, if you pass 0 as second argument of snprintf you can pass NULL as the target string and you get the characters that would have been written as the return value. So in C you can do:
int value = 42;
char * data;
size_t bufSize=snprintf(NULL, 0 "%d", value)+1; /* +1 for the NUL terminator */
data = malloc(bufSize);
if(data==NULL)
{
// ... handle allocation failure ...
}
snprintf(data, bufSize, "%d", value);
// ...
free(data);
I would serialize to a 'large enough' buffer then copy to an allocated buffer. In C
char big_buffer[999], *small_buffer;
sprintf(big_buffer, "%d", some_value);
small_buffer = malloc(strlen(big_buffer) + 1);
strcpy(small_buffer, big_buffer);