Small code bits:
int main()
{
char buf[18];
char buf2[18];
int newlength = 16;
memset(buf, '0', 16);
for (int i = newlength; i < 18; i++)
buf[i] = 0x00;
memcpy(buf2, buf, 18);
return 0;
}
First I want to set a portion of an array to a specific value and then I would like to fill the rest with 0x00. Then I'd like to copy it to another array.
On MS VS2013, I receive a warning as readable range for buf to be between 0 and 15. (Code Analysis for C/C++ Warnings. C6385 Read Overrun) Why? Does memcpy ignore the bits set to 0x00?
This message from the code analyzer seems to be based on the principle that the buffer content would be defined alone as output from memset(). It misses the point that the loop after memset() completes this input.
If you double click on the warning, you can get a highlighting of the lines considered for triggering this warning.
But the code you write is correct, so you don't have to worry about the result here. The online documentation says "might" no "will" :
This warning indicates that the readable extent of the specified
buffer might be smaller than the index used to read from it.
Additional remarks:
When making what is going on more obvious for the analyzer, it still brings the same abusive warnings:
memset(buf, '0', 16);
memset(buf + 16, 0x00, 2); // for replacing the loop
In this case, the analyzer notices the second memset(). But as it doesn't affect buf from its beginning, it as an input/output to a buffer operation without taking into consideration the additional length.
Even this kind over-precautiononous code raises the warning:
memset(buf, 0x00, sizeof(buf)); // completeky initalize the buffer
memset(buf, '0', 16); // overwrite just the beginning
Here, it seems that as soon as a memxxx() operation targets the begin of the buffer, the length of that operation is considered to be the sole initialized part.
So, yes the warning is annoying, but trust your code. I could only get rid of the warning by making really weird an unefficient coding:
memset(buf, 0x00, sizeof(buf)); // 18 bytes get initalized
memset(buf + 1, '0', 15); // not begin of buffer
buf[0] = '0'; // not a memxxx() operation
Unfortunately the configuration of the analyzer doesn't allow to disable just this single rule, but the whole set of security verification rules.
Seems to be a bug in compiler / lint tool (depends on who shows you the warning)
You're initializing 16 bytes. You're accessing 18 bytes. It seems to think that the access is reading, even though it shouldn't.
Related
You can see the parameter sent to the function does not exceed the buffer size from the code below.
This problem happened randomly, and only happened in debug build.
#include <thread>
#include <sstream>
#define BUF_SZ 32
int main()
{
wchar_t src[BUF_SZ]{};
bool running = true;
std::thread th([&] {
for (double g = 0; g < 100000; g += .1)
{
std::wstringstream ws;
ws << g;
wcscpy_s(src, BUF_SZ, ws.str().c_str());
}
running = false;
});
wchar_t dst[BUF_SZ]{};
while (running)
wcscpy_s(dst, src); // assert on "Buffer is too small" randomly
th.join();
return 0;
}
Thanks to Mr. Steve Wishnousky from MSFT VC++ team, here is the complete explanation of the problem.
Wcscpy_s does not operate atomically on the buffers and will only work
correctly if the buffers do not change contents during the runtime of
wcscpy_s.
Another thing to note is that in Debug mode, the wcscpy_s function
will fill the rest of the buffer in with a debug mark (0xFE) to
indicate that the data there is now invalid to assume it's contents,
in order to detect potential runtime errors.
The error happens differently every time of course, but lets assume
this error happens when src=1269.9 and wcscpy_s(dst, src) is called.
The actual contents of src is: "1 2 6 9 . 9 null 0xfe 0xfe ...".
wcscpy_s copies over the 1269.9 but as it's about to read the null,
the other wcscpy_s just wrote a new value to src so it's now: "1 2 7 0
null 0xfe 0xfe ...". Instead of reading the null corresponding from
the previous src, it reads the 0xfe, so it thinks this is a real
character. Since there is no null terminator until we reach the end of
the buffer, the Debug runtime asserts that the buffer was too small
for the input.
In the Release build, the 0xFE debug marks aren't placed in the
buffer, so it will eventually find a null character. You can also
disable the debug marks by calling _CrtSetDebugFillThreshold:
https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/crtsetdebugfillthreshold?view=vs-2019.
Note that the Debug marks are actually catching a real correctness
problem here though. This "buffer changed during wcscpy_s" issue could
happen for any value. For example, if src=1269.9, wcscpy_s could copy
over the 126, but then as it's about to read the 9, src is updated to
1270 and the value that would end up in dest would be "1260".
Since the string copy will need to copy "src" characters and a terminating null character you'll need to provide a buffer which is at least one character bigger than sizeof "src".
I suggest you could try to use:
wcscpy_s(dst, sizeof src+1, src);
I'm having a string is not null terminated error, though I'm not entirely sure why. The usage of std::string in the second part of the code is one of my attempt to fix this problem, although it still doesn't work.
My initial codes was just using the buffer and copy everything into client_id[]. The error than occurred. If the error is correct, that means I've got either client_ id OR theBuffer does not have a null terminator. I'm pretty sure client_id is fine, since I can see it in debug mode. Strange thing is buffer also has a null terminator. No idea what is wrong.
char * next_token1 = NULL;
char * theWholeMessage = &(inStream[3]);
theTarget = strtok_s(theWholeMessage, " ",&next_token1);
sendTalkPackets(next_token1, sizeof(next_token1) + 1, id_clientUse, (unsigned int)std::stoi(theTarget));
Inside sendTalkPackets is. I'm getting a string is not null terminated at the last line.
void ServerGame::sendTalkPackets(char * buffer, unsigned int buffersize, unsigned int theSender, unsigned int theReceiver)
{
std::string theMessage(buffer);
theMessage += "0";
const unsigned int packet_size = sizeof(Packet);
char packet_data[packet_size];
Packet packet;
packet.packet_type = TALK;
char client_id[MAX_MESSAGE_SIZE];
char theBuffer[MAX_MESSAGE_SIZE];
strcpy_s(theBuffer, theMessage.c_str());
//Quick hot fix for error "string not null terminated"
const char * test = theMessage.c_str();
sprintf_s(client_id, "User %s whispered: ", Usernames.find(theSender)->second.c_str());
printf("This is it %s ", buffer);
strcat_s(client_id, buffersize , theBuffer);
Methinks that problem lies in this line:
sendTalkPackets(next_token1, sizeof(next_token1) + 1, id_clientUse, (unsigned int)std::stoi(theTarget));
sizeof(next_token1)+1 will always gives 5 (on 32 bit platform) because it return size of pointer not size of char array.
One thing which could be causing this (or other problems): As
buffersize, you pass sizeof(next_token1) + 1. next_token1 is
a pointer, which will have a constant size of (typically) 4 or 8. You
almost certainly want strlen(next_token1) + 1. (Or maybe without the
+ 1; conventions for passing sizes like this generally only include
the '\0' if it is an output buffer. There are a couple of other
places where you're using sizeof, which may have similar problems.
But it would probably be better to redo the whole logic to use
std::string everywhere, rather than all of these C routines. No
worries about buffer sizes and '\0' terminators. (For protocol
buffers, I've also found std::vector<char> or std::vector<unsigned char>
quite useful. This was before the memory in std::string was
guaranteed to be contiguous, but even today, it seems to correspond more
closely to the abstraction I'm dealing with.)
You can't just do
std::string theMessage(buffer);
theMessage += "0";
This fails on two fronts:
The std::string constructor doesn't know where buffer ends, if buffer is not 0-terminated. So theMessage will potentially be garbage and include random stuff until some zero byte was found in the memory beyond the buffer.
Appending string "0" to theMessage doesn't help. What you want is to put a zero byte somewhere, not value 0x30 (which is the ascii code for displaying a zero).
The right way to approach this, is to poke a literal zero byte buffersize slots beyond the start of the buffer. You can't do that in buffer itself, because buffer may not be large enough to accomodate that extra zero byte. A possibility is:
char *newbuffer = malloc(buffersize + 1);
strncpy(newbuffer, buffer, buffersize);
newbuffer[buffersize] = 0; // literal zero value
Or you can construct a std::string, whichever you prefer.
I am opening a file and placing it's contents into a string buffer to do some lexical analysis on a per-character basis. Doing it this way enables parsing to finish faster than using a subsequent number of fread() calls, and since the source file will always be no larger than a couple MBs, I can rest assured that the entire contents of the file will always be read.
However, there seems to be some trouble in detecting when there is no more data to be parsed, because ftell() often gives me an integer value higher than the actual number of characters within the file. This wouldn't be a problem with the use of the EOF (-1) macro, if the trailing characters were always -1... But this is not always the case...
Here's how I am opening the file, and reading it into the string buffer:
FILE *fp = NULL;
errno_t err = _wfopen_s(&fp, m_sourceFile, L"rb, ccs=UNICODE");
if(fp == NULL || err != 0) return FALSE;
if(fseek(fp, 0, SEEK_END) != 0) {
fclose(fp);
fp = NULL;
return FALSE;
}
LONG fileSize = ftell(fp);
if(fileSize == -1L) {
fclose(fp);
fp = NULL;
return FALSE;
}
rewind(fp);
LPSTR s = new char[fileSize];
RtlZeroMemory(s, sizeof(char) * fileSize);
DWORD dwBytesRead = 0;
if(fread(s, sizeof(char), fileSize, fp) != fileSize) {
fclose(fp);
fp = NULL;
return FALSE;
}
This always appears to work perfectly fine. Following this is a simple loop, which checks the contents of the string buffer one character at a time, like so:
char c = 0;
LONG nPos = 0;
while(c != EOF && nPos <= fileSize)
{
c = s[nPos];
// do something with 'c' here...
nPos++;
}
The trailing bytes of the file are usually a series of ý (-3) and « (-85) characters, and therefore EOF is never detected. Instead, the loop simply continues onward until nPos ends up being of higher value than fileSize -- Which is not desirable for proper lexical analysis, because you often end up skipping the final token in a stream which omits a newline character at the end.
In a Basic Latin character set, would it be safe to assume that an EOF char is any character with a negative value? Or perhaps there is just a better way to go about this?
#EDIT: I have just tried to implement the feof() function into my loop, and all the same, it doesn't seem to detect EOF either.
Assembling comments into an answer...
You leak memory (potentially a lot of memory) when you fail to read.
You haven't allowed for a null terminator at the end of the string read.
There's no point in zeroing the memory when it is all about to be overwritten by the data from the file.
Your test loop is accessing memory out of bounds; nPos == fileSize is one beyond the end of the memory you allocated.
char c = 0;
LONG nPos = 0;
while(c != EOF && nPos <= fileSize)
{
c = s[nPos];
// do something with 'c' here...
nPos++;
}
There are other problems, not previously mentioned, with this. You did ask if it is 'safe to assume that an EOF char is any character with a negative value', to which I responded No. There are several issues here, that affect both C and C++ code. The first is that plain char may be a signed type or an unsigned type. If the type is unsigned, then you can never store a negative value in it (or, more accurately, if you attempt to store a negative integer into an unsigned char, it will be truncated to the least significant 8* bits and will be treated as positive.
In the loop above, one of two problems can occur. If char is a signed type, then there is a character (ÿ, y-umlaut, U+00FF, LATIN SMALL LETTER Y WITH DIAERESIS, 0xFF in the Latin-1 code set) that has the same value as EOF (which is always negative and usually -1). Thus, you might detect EOF prematurely. If char is an unsigned type, then there will never be any character equal to EOF. But the test for EOF on a character string is fundamentally flawed; EOF is a status indicator from I/O operations and not a character.
During I/O operations, you will only detect EOF when you've attempted to read data that isn't there. The fread() won't report EOF; you asked to read what was in the file. If you tried getc(fp) after the fread(), you'd get EOF unless the file had grown since you measured how long it is. Since _wfopen_s() is a non-standard function, it might be affecting how ftell() behaves and the value it reports. (But you later established that wasn't the case.)
Note that functions such as fgetc() or getchar() are defined to return characters as positive integers and EOF as a distinct negative value.
If the end-of-file indicator for the input stream pointed to by stream is not set and a
next character is present, the fgetc function obtains that character as an unsigned
char converted to an int.
If the end-of-file indicator for the stream is set, or if the stream is at end-of-file, the end-of-
file indicator for the stream is set and the fgetc function returns EOF. Otherwise, the
fgetc function returns the next character from the input stream pointed to by stream.
If a read error occurs, the error indicator for the stream is set and the fgetc function
returns EOF.289)
289) An end-of-file and a read error can be distinguished by use of the feof and ferror functions.
This indicates how EOF is separate from any valid character in the context of I/O operations.
You comment:
As for any potential memory leakage... At this stage in my project, memory leaks are one of many problems with my code which, as of yet, are of no concern to me. Even if it didn't leak memory, it doesn't even work to begin with, so what's the point? Functionality comes first.
It is easier to head off memory leaks in error paths at the initial coding stage than to go back later and fix them — because you may not spot them because you may not trigger the error condition. However, the extent to which that matters depends on the intended audience for the program. If it is a one-off for a coding course, you may be fine. If you're the only person who'll use it, you may be fine. But if it will be installed by millions, you'll have problems retrofitting the checks everywhere.
I have swapped _wfopen_s() with fopen() and the result from ftell() is the same. However, after changing the corresponding lines to LPSTR s = new char[fileSize + 1], RtlZeroMemory(s, sizeof(char) * fileSize + 1); (which should also null-terminate it, btw), and adding if(nPos == fileSize) to the top of the loop, it now comes out cleanly.
OK. You could use just s[fileSize] = '\0'; to null terminate the data too, but using RtlZeroMemory() achieves the same effect (but would be slower if the file is many megabytes in size). But I'm glad the various comments and suggestions helped get you back on track.
* In theory, CHAR_BITS might be larger than 8; in practice it is almost always 8 and for simplicity, I'm assuming it is 8 bits here. The discussion has to be more nuanced if CHAR_BITS is 9 or more, but the net effect is much the same.
typedef struct {
unsigned char b1, b2;
} cont;
cont buf[1024];
int main(int argc, char *argv[]) {
FILE* fp;
fp = fopen(argv[1], "rb")
if(fp!=NULL)
fread(buf, sizeof (cont), sizeof (buf), fp);
//do something with buf
return 0;
}
Hello there, I am facing a segmentation fault error when I try to run this program. It used to work fine all of the sudden the segm. fault error appeared. The fread function call is generating the error. Please help me!
You're using fread() wrong - arg#1 is the size of elements to read and arg#2 is the number of elements to read (which should be 1024 in your case).
As a result, what you do reads sizeof (cont) * sizeof (buf) bytes, and that overflows your buffer.
See:
http://www.opengroup.org/onlinepubs/009695399/functions/fread.html
for the function documentation.
To clarify, you want to read 1024 elements but sizeof(buf) is 2048 (at least, maybe more if the struct is padded by the ABI of your platform).
Examples (coded so that they don't rely on a specific number of elements):
fread(buf, 1, sizeof(buf), fp); // fills the buffer (assuming it's buf[...])
fread(buf, sizeof(*buf), sizeof(buf)/sizeof(*buf), fp); // ditto
I.e. if you want to pass the total size of the destination buffer, via sizeof(), then the other argument must be one, while if you want to pass the size of the data structure, then the other argument is the number of these that fits into the buffer.
Always check return values. How else do you know if you actually managed to read anything?
I think this may be because of padding. The "cont" type is defined as 2 bytes big, but will probably be padded to 4. However this should not cause a problem because even if sizeof(cont) returns 2 or 4, "buf" must be using the padded size and so still be big enough.
sizeof(buf) gives you the grand total of buf, not just the number of elements in it. Nevertheless, you should never every read directly into structs. Bad things await you, if you do it that way.
Also structs may be padded anywhere between their members, so you don't even know the exact memory layout of that struct you define.
To keep your program portable and safe always read files element by element and construct the data from that.
int i;
for(i = 0; i < MAX_ELEMENTS && !feof(fil); ++i) {
int c1, c2;
c1 = fgetc(fil);
c2 = fgetc(fil);
if(c1 == EOF || c2 == EOF)
break;
buf[i].c1 = c1;
buf[i].c2 = c2;
}
Does this look tedious and verbose? Yes, but that is for good reason. Always assume the contents of a file to be possibly corrupted. Just reading a file into memory assuming is dangerous!
I'm having a really hard time understanding why is this piece of code making my computer beep. I've isolated this section of code to be the one producing the occasional beep, but I don't see what's the problem with it.
const int BUFFER_LENGTH = 8192;
char buffer [BUFFER_LENGTH + 1];
int recvResult;
do
{
recvResult = recv(webSocket, buffer, BUFFER_LENGTH, 0);
buffer[recvResult] = '\0';
printf("%s", buffer);
if (recvResult > 0)
{
sendResult = send(clientSocket, buffer, recvResult, 0);
}
}while (recvResult > 0);
shutdown(webSocket, SD_SEND);
To give a little bit of context, this is for a computer networks class in which we have to code a proxy. So what I'm doing is listening to the answer and simply forward it to the client.
I can't tell you how high I jumped out of my chair when I first heard the beeping noise...
The buffer probably contains a '\a' char which makes the computer beep. From 5.2.2 (Character display semantics) :
Alphabetic escape sequences representing nongraphic characters in the execution character set are intended to produce actions on display devices as follows:
\a (alert) Produces an audible or visible alert without changing the active position.
Nevermind, found it, it was actually the printf statement that was doing an occasionnal beep!
Agree with the '\a' beep explanation.
One more point about your code:
recvResult = recv(webSocket, buffer, BUFFER_LENGTH, 0);
buffer[recvResult] = '\0';
Note that recvResult will be -1 if there's an I/O error (or if you're working in the non-blocking mode and no data to read so far).
In such a case you'll write into forbidden memory, which is (damn, how I hate this phrase) undefined behavior. Simply speaking - memory overwrite, which is bad.
You should check for socket error before writing into buffer