Failed to read the same file using multithread - c++

I created 50 threads to read the same file at the same time and then, in each thread, tried to write its content to new file that create with different name.
The code was supposed to generate 50 different files.
But I got unexpected results that it just generate 3~5 files.
When all the read the same file, there is no race-condition, and each thread is aimed to write its content to different file.
Can somebody help me? Thank you!
My code is listed below and it is a modification from Reference
#include <unistd.h>
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <string.h>
#include <iostream>
#include <vector>
#include <thread>
void copy(const char *src_path, const char *dst_path);
int main(int argc, char **argv)
{
std::vector<std::thread> vth;
char *tmp = "Copy.deb";
for (int i = 0; i < 50; ++i)
{
char src[40];
memset(src, '\0', sizeof(src));
sprintf(src, "%d", i);
strcat(src, tmp);
vth.emplace_back(std::bind(copy, "Original.deb", strdup(src)));
}
for (int i = 0; i < 50; ++i)
{
vth[i].join();
}
return 0;
}
void copy(const char *src_path, const char *dst_path)
{
FILE *src, *dst;
int buffer_size = 8 * 1024;
char buffer[buffer_size];
size_t length;
src = fopen(src_path, "rb");
dst = fopen(dst_path, "wb");
while (!feof(src))
{
length = fread(buffer, 1, buffer_size, src);
fwrite(buffer, 1, length, dst);
}
fclose(src);
fclose(dst);
}

I believe your problem is that you are passing src (which is a pointer to a local variable on your main thread's stack) to your thread's entry function, but since your copy() function runs asynchronously in a separate thread, the char src[40] array that you are passing a pointer to has already been been popped off of the main thread's stack (and likely overwritten by other data) before the copy() function gets a chance to read its contents.
The easy fix would be to make a copy of the string on the heap, so that you can guarantee the string will remain valid until the the copy() function executes and reads it:
vth.emplace_back(std::bind(copy, "Original.deb", strdup(src)));
... and be sure to have your copy() function free the heap-allocation when it's done using it:
void copy(const char *src_path, const char *dst_path)
{
FILE *src, *dst;
int buffer_size = 8 * 1024;
char buffer[buffer_size];
size_t length;
src = fopen(src_path, "rb");
dst = fopen(dst_path, "wb");
free(dst_path); // free the string previously allocated by strdup()
[...]
Note that you don't currently have the same problem with the "Original.deb" argument since "Original.deb" is a string-literal and therefore stored statically in the executable, which means it remains valid for as long as the program is running -- but if/when you change your code to not use a string-literal for that argument, you'd likely need to do something similar for it as well.

Related

How can I get the latest changes in a file using ifstream?

It's a real-time capture system, I need to get the latest changes from a file which is occasionally edited(mostly add content) by other applications.
In other words, how can I get content that added in the period when I open it without reopening the file?
#include <iostream>
#include <fstream>
#include <string>
using namespace std;
int main(){
ifstream tfile("temp.txt",ios::in);
if(!tfile){
cout<<"open failed"<<endl;
return 0;
}
string str;
while(1){
if(tfile.eof())
continue;
getline(tfile,str);
cout<<str<<endl;
}
tfile.close();
}
C++ / C Solution
If you are looking for a c++ solution you can use the following functions that I had created a while back:
#include <iostream>
#include <string>
// For sleep function
#ifdef _WIN32
#include <Windows.h>
#else
#include <unistd.h>
#endif
using namespace std;
void watchLogs(const char *FILENAME) {
FILE * f;
unsigned size = 0;
f = fopen(FILENAME , "r");
char c;
while (true) {
if (!size) { // will print content of your log file. If you just want the updates you can remove the current content except the first two lines;
fseek(f, 0, SEEK_END);
size =(unsigned long)ftell(f) ;
fseek (f, 0, SEEK_SET);
char buffer[size + 1];
fread ( buffer, 1, size, f );
buffer[size] = '\0';
cout << buffer << "\n";
}
else if ((c = (char)fgetc(f)) >= 0) {
fseek(f, 0, SEEK_END); // reach end of file
int BUFFER_SIZE =(unsigned long)ftell(f) - size; // save the length of the update to your logs
char buffer[BUFFER_SIZE + 1]; // prepare a buffer to print the characters
fseek(f,-BUFFER_SIZE,SEEK_END); // rewind BUFFER_SIZE characters before the EOF
int i = 0;
do {buffer[i++] = (char)fgetc(f);} while(i < BUFFER_SIZE); // copy to buffer
buffer[i] = '\0'; // don't forget to NULL terminate your buffer
cout << buffer << "\n";
size += i; // increment the size of the current file
}
}
sleep(3); // updates are checked every 3 seconds to avoid running the cpu at fullspeed, you could set the new logs to show up every minutes or every seconds, up to you.
fclose(f);
}
And you can test it with:
int main(int argc, char **argv) {
if (argc < 2)
return 1;
const char *FILENAME = argv[1];
watchLogs(FILENAME);
return 0;
}
./a.out mysql_binary.log
I could have used stringstreamer but I like that this version would also work with c files with some minor tweaks (can't use string).
I hope you will find it helpful!
NB: This assume that your file will only grow and that the changes will be appended to the end of your file.
NB2: This program is not segfault proof, you may want to check the return of fopen etc
Inotify
If you use Linux you could also potentially go for inotify:
Download inotify: sudo apt-get install -y inotify-tools
Then create the following script mywatch.sh
while inotifywait -e close_write $1; do ./$1; done
Give permission to execute:
add chmox +x mywatch.sh
and call it with ./watchit.sh mysql_binary.log

How to define an interface/API which is used in multiple cpp files?

I get the following error:
"Symbol SBPTest multiply defined (by ../../build/typeFind.LPC1768.o and ../../build/main.LPC1768.o)."
I have declared SBPTest in common.h like this:
#ifndef COMMON_H_
#define COMMON_H_
extern RawSerial SBPTest(USBTX, USBRX);
#endif
Other files look like this...
typeFind.h:
#include "mbed.h"
#include <string>
extern unsigned int j;
extern unsigned int len;
extern unsigned char myBuf[16];
extern std::string inputStr;
void MyFunc(char* inputBuffer);
typeFind.cpp:
#include "typeFind.h"
#include "common.h"
void MyFunc(char* inputBuffer) {
inputStr = inputBuffer;
if (inputStr == "01") {
len = 16;
for ( j=0; j<len; j++ )
{
myBuf[j] = j;
}
for ( j=0; j<len; j++ )
{
SBPTest.putc( myBuf[j] );
}
}
}
main.cpp:
#include "typeFind.h"
#include "common.h"
#include "stdlib.h"
#include <string>
LocalFileSystem local("local"); // define local file system
unsigned char i = 0;
unsigned char inputBuff[32];
char inputBuffStr[32];
char binaryBuffer[17];
char* binString;
void newSBPCommand();
char* int2bin(int value, char* buffer, int bufferSize);
int main() {
SBPTest.attach(&newSBPCommand); //interrupt to catch input
while(1) { }
}
void newSBPCommand() {
FILE* WriteTo = fopen("/local/log1.txt", "a");
while (SBPTest.readable()) {
//signal readable
inputBuff[i] = SBPTest.getc();
//fputc(inputBuff[i], WriteTo);
binString = int2bin(inputBuff[i], binaryBuffer, 17);
fprintf (WriteTo, "%s\n", binString);
inputBuffStr[i] = *binString;
i++;
}
fprintf(WriteTo," Read input once. ");
inputBuffStr[i+1] = '\0';
//fwrite(inputBuff, sizeof inputBuffStr[0], 32, WriteTo);
fclose(WriteTo);
MyFunc(inputBuffStr);
}
char* int2bin(int value, char* buffer, int bufferSize)
{
//..................
}
I am programming on mbed, a LPC1768. The serial is used both in main.cpp and typeFind.cpp. I have looked on stack overflow and a common.h file is recommended, yet I get a compiler error.
You musn't define the variable in the header, or else you end up defining it in all translation units that include the header, which violates the one definition rule. Only declare it:
// common.h
extern RawSerial SBPTest;
And define in exactly one source file:
// common.cpp (or any other, but exactly one source file)
RawSerial SBPTest(USBTX, USBRX);
I recommend using either list initialisation or copy initilisation, since direct initilisation grammar is ambiguous with a function declaration and may confuse anyone who doesn't know whether USBTX and USBRX are types or values:
// common.cpp (or any other, but exactly one source file)
RawSerial SBPTest{USBTX, USBRX}; // this
auto SBPTest = RawSerial(USBTX, USBRX); // or this

I am trying to pass some strings trough espeak and it reads them but I get "segmentation fault"

This is my code. I want to get 5 strings from the user and espeak reads each of them when user interred it. But I get segmentation fault(core dumped) message.
#include <string.h>
#include <malloc.h>
#include <espeak/speak_lib.h>
int test()
{
espeak_POSITION_TYPE position_type;
espeak_AUDIO_OUTPUT output;
char *path=NULL;
int Buflength = 500, Options=0;
void* user_data;
t_espeak_callback *SynthCallback;
espeak_PARAMETER Parm;
char Voice[] = {"English"};
int i=0;
char text[1000];
unsigned int Size,position=0, end_position=0, flags=espeakCHARS_AUTO, *unique_identifier;
output = AUDIO_OUTPUT_PLAYBACK;
espeak_Initialize(output, Buflength, path, Options );
espeak_SetVoiceByName(Voice);
const char *langNativeString = "en_US";
espeak_VOICE voice={0};
voice.languages = langNativeString;
voice.name = "US";
voice.variant = 2;
voice.gender = 1;
Size = strlen(text)+1;
for (i=0; i<5; i++)
{
scanf("%s ", &text);
printf("%s", text);
espeak_Synth( text, Size, position, position_type, end_position, flags,
unique_identifier, user_data );
espeak_Synchronize( );
fflush(stdout);
}
return 0;
}
int main(int argc, char* argv[] )
{
test();
return 0;
}
I tried some modification but none of them worked. I want the program works like this:
User input: hi
espeak says: hi
user input: one
espeak says: one
(for 5
inputs)
But when I try to interring more than 4 characters as input,it gives segmentation fault error!
The two main issues are:
you use strlen on an uninitialized array of chars;
the unique_identifier argument of espeak_Synth must be NULL or point to an unsigned int (see the source code) while now it is an unsigned pointer to random memory.
Move strlen after scanf, use NULL instead of unique_identifier and your code will suddenly work (kind of).
There are many other issues though: useless variables, uninitialized variables, no input sanitization and more. IMO a better approach would be to throw away the test function and rewrite it from scratch properly.
Addendum
This is how I'd rewrite the above code. It is still suboptimal (no input sanitization, no error checking) but IMO it is much cleaner.
#include <stdio.h>
#include <string.h>
#include <espeak/speak_lib.h>
static void say(const char *text)
{
static int initialized = 0;
if (! initialized) {
espeak_Initialize(AUDIO_OUTPUT_PLAYBACK, 0, NULL, 0);
espeak_SetVoiceByName("en");
initialized = 1;
}
espeak_Synth(text, strlen(text)+1,
0, POS_CHARACTER, 0,
espeakCHARS_UTF8, NULL, NULL);
espeak_Synchronize();
}
int main()
{
char text[1000];
int i;
for (i = 0; i < 5; ++i) {
scanf("%s", text);
say(text);
}
return 0;
}

Visual Studio 2008 C++ misreporting memory address location

I'm attempting to debug a stack overwrite (no pun intended) corruption problem with some c/c++ code in Visual Studio 2008.
When I compile the solution in win32 debug mode, I'm able to run the debugger and see a class get instantiated.
In the constructor call, we initialize some fixed length char[] member variables using memset.
If I use printf to print the memory location of the member variable, I get a value that is exactly 14 bytes past what the VS2008 watch/local var window reports as the memory location.
How does VS calculate what it thinks the memory address is of a variable?
Edit: I have compiled with Run Time Check options, and what I see is "Run-time Check Failure #2 - stack around variable 'varName' was corrupted.
Edit 3: Here is the abbreviated source
Header file:
//////////////////////////////////////////////////////////////////////
#include "commsock.h"
#include "buffer.h"
#include "BufferedReader.h"
#define AUTHTYPE_PASSKEY 1
#define AUTHTYPE_PAGER 2
#define AUTHTYPE_PASSWORD 3
#define AUTHTYPE_RADIUS 4
#define AUTHTYPE_INFOCARD_RO 5
#define AUTHTYPE_INFOCARD_CR 6
#define AUTHSTATE_NOT_LOGGED_IN 0
#define AUTHSTATE_IDENTIFIED 1
#define AUTHSTATE_AUTHENTICATED 2
#define AUTHSTATE_MAX_FAILS 32
#define AUTHSTATE_NO_LOG 65536
class PRClientSession {
public:
PRClientSession();
virtual ~PRClientSession();
void Reset();
BOOL BeginAuth(LPCSTR szUserID, LPCSTR szClientIP, LPCSTR szOtherUserID="");
BOOL CompleteAuth(LPCSTR szResponse);
BOOL Logoff();
BOOL DetachAsync();
BOOL Detach();
BOOL Attach(const char *authstr=NULL);
BOOL Connected(){return m_Socket.Connected();};
BOOL EncryptText(char *pPassword, char *pOut, char *pKey);
BOOL EncryptText(char *pPassword, char *pOut);
BOOL DecryptText(char *pPassword, char *pOut, char *pKey);
BOOL DecryptText(char *pPassword, char *pOut);
LPCSTR Error(LPCSTR szErr=NULL){if (szErr)strncpy_s(m_szError,szErr,sizeof(m_szError));return m_szError;};
LPCSTR Challenge(LPCSTR szChal=NULL){if (szChal)strncpy_s(m_szChallenge,szChal,sizeof(m_szChallenge));return m_szChallenge;};
LPCSTR UserID(LPCSTR szUID=NULL){if (szUID)strncpy_s(m_szUserID,szUID,sizeof(m_szUserID));return m_szUserID;};
LPCSTR SessionID(LPCSTR szSID=NULL){if (szSID)strncpy_s(m_szSessionID,szSID,sizeof(m_szSessionID));return m_szSessionID;};
LPCSTR SessionLogID(LPCSTR szSLID=NULL){if (szSLID)strncpy_s(m_szSessionLogID,szSLID,sizeof(m_szSessionLogID));return m_szSessionLogID;};
LPCSTR SessionStateID(LPCSTR szSSID=NULL){if (szSSID)strncpy_s(m_szSessionStateID,szSSID,sizeof(m_szSessionStateID));return m_szSessionStateID;};
int AuthType(int iType=-1){if (iType != -1)m_iAuthType = iType;return m_iAuthType;};
int SendRequest(LPCSTR szType, ...);
int Write(char *szBuf, int iLen=-1);
int Read(char *szBuf, int iLen, int iTimeout=-1);
BOOL ReadResponse(int iBlock=FALSE);
int ReadResponseTimeout(int iBlock) ;
BOOL GetField(LPCSTR szField, char *szBuf, int iLen, int total=-1);
BOOL GetField(LPCSTR szField, int &iBuf );
char *GetData(){return m_pData;};
int GetDataSize(){return m_iDataSize;}
char *GetMessage(){return m_pMessage;};
SOCKET getSocket(){return m_Socket.getSocket();}
private:
BOOL LoadConfig();
BOOL GetMsgField(LPCSTR szIn, LPCSTR szSrch, char *szBuf, int iLen, int total=-1);
int ReadMessage( char *buf, int len, const char *terminator = NULL );
public:
bool sendCommand(char* szCommand, char *szArglist);
LPCSTR ReplyMessage(){ return m_szReplyMessage; }
int IsRadiusChallenge(){ return m_iIsRadiusChallenge; }
static char PRIISMS_USER_TAG[];
static char DEST_USER_TAG[];
static char RADIUS_USER_TAG[];
static char RADIUS_PASSWORD_TAG[];
static char GENERATE_LOGIN[];
static char VALIDATE_LOGIN[];
static char PR_RADIUS_GENERATED[];
private:
BOOL doConnect();
// Response reader vars...
char *m_pMessage;
char *m_pData;
Buffer m_Buf;
BOOL m_bLoaded;
BufferedReader m_reader;
Buffer m_ReqBuf;
CCommSocket m_Socket;
char m_szServer[128];
int m_iServerPort;
char m_szError[128];
int m_iAuthState;
int m_iDataSize;
int m_iAuthType;
char m_szChallenge[1024];
char m_szUserID[64];
char m_szSessionID[64];
char m_szSessionLogID[64];
char m_szSessionStateID[64];
char m_szSessionSecret[16];
long m_RequestID;
int m_iIsRadiusChallenge;
char m_szReplyMessage[1024];
};
and the source code with constructor...
#include "stdafx.h"
#include "PRclntsn.h"
#include "iondes.h"
#include "prsystemparameters.h"
#include "prsessionlog.h"
#include "util.h"
#include "PRClntSn.h"
#include <string>
using namespace std;
#define LoadConfigFailedMsg "Unable to retrieve database configuration entries."
#ifndef AUTH_TYPE_RADIUS
#define AUTH_TYPE_RADIUS 4
#endif
//------------------------------------------------------------------------------------
// initialize static members
char PRClientSession::DEST_USER_TAG[] = "DEST_USER=";
char PRClientSession::PRIISMS_USER_TAG[] = "PRIISMS_USER=";
char PRClientSession::RADIUS_USER_TAG[] = "RADIUS_USER=";
char PRClientSession::RADIUS_PASSWORD_TAG[] = "RADIUS_PASSWORD=";
char PRClientSession::GENERATE_LOGIN[] = "GENERATE_LOGIN";
char PRClientSession::VALIDATE_LOGIN[] = "VALIDATE_LOGIN";
char PRClientSession::PR_RADIUS_GENERATED[] = "PR_RADIUS_GENERATED";
PRClientSession::PRClientSession()
{
Reset();
}
void PRClientSession::Reset()
{
//LOG4CXX_TRACE(mainlogger, CLASS_METHOD_TAG);
printf("m_szServer mem location: %p", (void *)&m_szServer);
memset(m_szServer, 0, sizeof(m_szServer));
m_iServerPort = 0;
memset(m_szError, 0, sizeof(m_szError));
m_iAuthState = 0;
m_iDataSize = 0;
m_iAuthType = 0;
memset(m_szChallenge, 0, sizeof(m_szChallenge));
memset(m_szUserID, 0, sizeof(m_szUserID));
memset(m_szSessionID, 0, sizeof(m_szSessionID));
memset(m_szSessionLogID, 0, sizeof(m_szSessionLogID));
memset(m_szSessionStateID, 0, sizeof(m_szSessionStateID));
memset(m_szSessionSecret, 0, sizeof(m_szSessionSecret) );
// memset(m_szReadBuf, 0, sizeof(m_szReadBuf));
m_RequestID = 0;
m_iIsRadiusChallenge = 0;
memset(m_szReplyMessage, 0, sizeof(m_szReplyMessage));
m_reader.setComm(&m_Socket);
}
Output of printf:
m_szServer mem location: 01427308
Visual Studio 2008 Locals window
m`_szServer 0x014272fa "" char [128]`
It's off by 14... and when the code actually runs the very first memset, it does so starting with address 7308, not 72fa. It basically tramples contiguous memory regions (and thus, variables)
The debugger inserts extra space. This space is set to a pre-defined value, and if it's changed, the debugger knows that something has gone wrong. It's normal for compilers and debuggers to insert space that serves no apparent purpose.
If you have used a fixed-size char[], that immediately signals to me as a problem, since there's no explicit encapsulation. You could use a custom array type (such as boost::array) to write your own buffer overrun detection code. If you throw at this time, you will get a stack trace.
This is (probably) not going to fix your bug, but I would try to avoid C-style arrays if you can.
Unless you have some compelling reason to use the fixed-length arrays, replace them with std::vector which will be filled with 0x00 by default.
so instead of
const size_t MYCLASS::BUFLEN(16);
class myClass {
public:
myClass()
{
memset(buffer, 0, BUFLEN);
}
private:
static const size_t BUFLEN;
char buffer[BUFLEN];
};
you have
const size_t MYCLASS::BUFLEN(16);
class myClass {
public:
myClass() : buffer(BUFLEN)
{
memset(buffer, 0, BUFLEN);
}
private:
static const size_t BUFLEN;
std::vector<char> buffer;
};

How to write to a memory buffer with a FILE*?

Is there any way to create a memory buffer as a FILE*. In TiXml it can print the xml to a FILE* but i cant seem to make it print to a memory buffer.
There is a POSIX way to use memory as a FILE descriptor: fmemopen or open_memstream, depending on the semantics you want: Difference between fmemopen and open_memstream
I guess the proper answer is that by Kevin. But here is a hack to do it with FILE *. Note that if the buffer size (here 100000) is too small then you lose data, as it is written out when the buffer is flushed. Also, if the program calls fflush() you lose the data.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv)
{
FILE *f = fopen("/dev/null", "w");
int i;
int written = 0;
char *buf = malloc(100000);
setbuffer(f, buf, 100000);
for (i = 0; i < 1000; i++)
{
written += fprintf(f, "Number %d\n", i);
}
for (i = 0; i < written; i++) {
printf("%c", buf[i]);
}
}
fmemopen can create FILE from buffer, does it make any sense to you?
I wrote a simple example how i would create an in-memory FILE:
#include <unistd.h>
#include <stdio.h>
int main(){
int p[2]; pipe(p); FILE *f = fdopen( p[1], "w" );
if( !fork() ){
fprintf( f, "working" );
return 0;
}
fclose(f); close(p[1]);
char buff[100]; int len;
while( (len=read(p[0], buff, 100))>0 )
printf(" from child: '%*s'", len, buff );
puts("");
}
C++ basic_streambuf inheritance
In C++, you should avoid FILE* if you can.
Using only the C++ stdlib, it is possible to make a single interface that transparently uses file or memory IO.
This uses techniques mentioned at: Setting the internal buffer used by a standard stream (pubsetbuf)
#include <cassert>
#include <cstring>
#include <fstream>
#include <iostream>
#include <ostream>
#include <sstream>
/* This can write either to files or memory. */
void write(std::ostream& os) {
os << "abc";
}
template <typename char_type>
struct ostreambuf : public std::basic_streambuf<char_type, std::char_traits<char_type> > {
ostreambuf(char_type* buffer, std::streamsize bufferLength) {
this->setp(buffer, buffer + bufferLength);
}
};
int main() {
/* To memory, in our own externally supplied buffer. */
{
char c[3];
ostreambuf<char> buf(c, sizeof(c));
std::ostream s(&buf);
write(s);
assert(memcmp(c, "abc", sizeof(c)) == 0);
}
/* To memory, but in a hidden buffer. */
{
std::stringstream s;
write(s);
assert(s.str() == "abc");
}
/* To file. */
{
std::ofstream s("a.tmp");
write(s);
s.close();
}
/* I think this is implementation defined.
* pusetbuf calls basic_filebuf::setbuf(). */
{
char c[3];
std::ofstream s;
s.rdbuf()->pubsetbuf(c, sizeof c);
write(s);
s.close();
//assert(memcmp(c, "abc", sizeof(c)) == 0);
}
}
Unfortunately, it does not seem possible to interchange FILE* and fstream: Getting a FILE* from a std::fstream
You could use the CStr method of TiXMLPrinter which the documentation states:
The TiXmlPrinter is useful when you
need to:
Print to memory (especially in non-STL mode)
Control formatting (line endings, etc.)
https://github.com/Snaipe/fmem is a wrapper for different platform/version specific implementations of memory streams
It tries in sequence the following implementations:
open_memstream.
fopencookie, with growing dynamic buffer.
funopen, with growing dynamic buffer.
WinAPI temporary memory-backed file.
When no other mean is available, fmem falls back to tmpfile()