creating/opening file on disk returns EAGAIN - c++

As title says I'm getting this error while trying to open file for binary writing(mode doesnt seem matter).
My app uses libev to handle sockets(non blocking/epoll backend) and while parsing client packets i want at some point where i receive fileupload message to start writing down to disk data i get from server.
I couldn't google anything about EAGAIN(Resource temporarily unavailable) message and file opening..
These are methods I've tried:
fopen( ... ) returns EAGAIN
using ofstream/fstream's open(...) by creating them on heap(new) returns EAGAIN
using ofstream/fstream's open(...) staticly as class member (ofstream m_ofFile;) works, but strangly compiler generates code which calls ofstream destructor and closes file before exiting class method im calling .open from. Now that contradicts with my C++ knowledge where for class members which are class types, destructors are called right before class owner's..
edit:
#Joachim
You're right, I'm not acually getting this error..(method #1. gonna test method #2 again soon). File opens regulary and i get regular FILE*. That happens in Init(...) function of my class, but then when I call OnFileChunk later on m_hFile is 0 and therefor i cant write to it. Here is complete class code:
class CFileTransferCS
{
wstring m_wszfile;
wstring m_wszLocalUserFolderPath;
int m_nChunkIndex;
int m_nWrittenBytes;
int m_nFileSize;
FILE* m_hFile;
CFileTransferCS( const CFileTransferCS& c ){}
CFileTransferCS& operator=( const CFileTransferCS& c ){}
public:
CFileTransferCS( );
CFileTransferCS( wstring file, uint32_t size );
void OnFileChunk( char* FileChunk, int size );
void Init( wstring file, uint32_t size );
void SetLocalUserLocalPath( wstring path );
};
CFileTransferCS::CFileTransferCS( )
{
m_hFile = NULL;
m_wszLocalUserFolderPath = L"";
m_nChunkIndex = 0;
m_nWrittenBytes = 0;
}
CFileTransferCS::CFileTransferCS( wstring file, uint32_t size )
{
m_nChunkIndex = 0;
m_nWrittenBytes = 0;
m_wszfile = file;
m_nFileSize = size;
wstring wszFullFilePath = m_wszLocalUserFolderPath + m_wszfile.substr( m_wszfile.find_last_of(L"\\") + 1 );
// string fp = string( file.begin(),file.end() );
string fp ="test.bin"; //for testing purposes
this->m_hFile = fopen(fp.c_str(),"wb");
printf("fp: %s hFile %d\n",fp.c_str(),this->m_hFile); //everything's fine here...
if(!this->m_hFile)
{
perror ("cant open file ");
}
}
void CFileTransferCS::SetLocalUserLocalPath( wstring path )
{
m_wszLocalUserFolderPath = path;
}
void CFileTransferCS::Init( wstring file, uint32_t size )
{
// If previous transfer session got interrupted for whatever reason
// close and delete old file and open new one
if( this->m_hFile )
{
printf("init CS transfer: deleting old file///\n");
fclose( this->m_hFile );
string fp = string( file.begin(),file.end() );
if( remove( fp.c_str() ))
{
//cant delete file...
}
}
CFileTransferCS( file, size );
}
void CFileTransferCS::OnFileChunk( char* FileChunk, int size )
{
for (;;)
{
printf("ofc: hFile %d\n",this->m_hFile); //m_hFile is 0 here...
if( !this->m_hFile )
{
// m_pofFile->open("kurac.txt",fstream::out);
printf("file not opened!\n");
break;
}
int nBytesWritten = fwrite( FileChunk, 1, size, this->m_hFile );
if( !nBytesWritten )
{
perror("file write!!\n");
break;
}
m_nWrittenBytes+=size;
if( m_nWrittenBytes == m_nFileSize )
{
fclose( m_hFile );
printf("file uplaod transfer finished!!!\n");
}
break;
}
printf("CFileTransferCS::OnFileChunk size: %d m_nWrittenBytes: %d m_nFileSize: %d\n",size,m_nWrittenBytes,m_nFileSize);
}
final edit:
I got it.. Calling explicitly CFileTransferCS( wstring file, uint32_t size ) constructor made problems.. Calling constructor like this explicitly caused that this pointer in it wasnt original one(that Init function was using) so when i was opening file from it and saving handle to m_hFile, i was doing it in some other object(now im not sure if CFileTransferCS(..) call allocated memory for CFileTransferCS object or it corrupted some other part of memory randomly.. will check it out with IDA later on )
Thanks to everyone and my apologies.
Regards, Mike –

#MikeJacksons answer:
Calling explicitly CFileTransferCS( wstring file, uint32_t size ) constructor made problems. Calling constructor like this explicitly caused that this pointer in it wasnt original one(that Init function was using) so when i was opening file from it and saving handle to m_hFile, i was doing it in some other object(now im not sure if CFileTransferCS(..) call allocated memory for CFileTransferCS object or it corrupted some other part of memory randomly.. will check it out with IDA later on ) Thanks everyone and my apologies.
Removed: CFileTransferCS( file, size );
(No need to appologize Mike, looks like you did a great job hunting down the bug).

Related

Mounting memory buffer as a file without writing to disk

I have a server and needs to feed data from clients to a library; however, that library only supports reading files (it uses open to access the file).
Since the data can get pretty big, I rather not write it out to a temporary file, read it in with the library then delete it afterwards. Instead I would like to do something similar to a ramdisk where there's a file in which the content is actually in memory.
However, there can be multiple clients sending over large data, I don't think constantly calling mount and umount to create a ramdisk for each client is efficient. Is there a way for me to mount an existing memory buffer as a file without writing to disk?
The library does not support taking in a file descriptor nor FILE*. It will only accept a path which it feeds directly to open
I do have the library's source code and attempted to add in a function that uses fmemopen; however, fmemopen returns a FILE* with no file descriptor. The internals of the library works only with file descriptors and it is too complex to change/add support to use FILE*
I looked at mmap, but it appears to be no different than writing out the data to a file
Using mount requires sudo access and I prefer not to run the application as sudo
bool IS_EXITING = false;
ssize_t getDataSize( int clientFD ) { /* ... */}
void handleClient( int clientFD ) {
// Read in messages to get actual data size
ssize_t dataSize = getDataSize( clientFD );
auto* buffer = new char[ dataSize ];
// Read in all the data from the client
ssize_t bytesRead = 0;
while( bytesRead < dataSize ) {
int numRead = read( clientFD, buffer + bytesRead, dataSize - bytesRead );
bytesRead += numRead;
// Error handle if numRead is <= 0
if ( numRead <= 0 ) { /* ... */ }
}
// Mount the buffer and get a file path... How to do this
std::string filePath = mountBuffer( buffer );
// Library call to read the data
readData( filePath );
delete[ ] buffer;
}
void runServer( int socket )
while( !IS_EXITING ) {
auto clientFD = accept( socket, nullptr, nullptr );
// Error handle if clientFD <= 0
if ( clientFD <= 0 ) { /* ... */ }
std::thread clientThread( handleClient, clientFD );
clientThread.detach( );
}
}
Use /dev/fd. Get the file descriptor of the socket, and append that to /dev/fd/ to get the filename.
If the data is in a memory buffer, you could create a thread that writes to a pipe. Use the file descriptor of the read end of the pipe with /dev/fd.

How to receive a JPEG image over serial port

So I am trying to send a jpeg image (4Kb) from a raspberry pi to my Mac wirelessly using Xbee Series 1. I have an image on the raspberry pi and can read it into binary format. I've used this binary format to save it into another image file and it creates a copy of the image correctly. That tells me that I am reading it correctly. So I am trying to send that data over a serial port (to be transferred by the xbee's) to my Mac. Side note, Xbee's can only transmit I think 80 bytes of data per packet or something. I don't know how that affects what I'm doing though.
My problem is, I do not know how to read the data and properly store it into a jpeg file itself. Most of the Read() functions I have found require you to enter a length to read and I don't know how to tell how long it is since its just a serial stream coming in.
Here is my code to send the jpeg.
#include "xSerial.hpp"
#include <iostream>
#include <cstdlib>
using namespace std;
int copy_file( const char* srcfilename, const char* dstfilename );
int main(){
copy_file("tylerUseThisImage.jpeg", "copyImage.jpeg");
return 0;
}
int copy_file( const char* srcfilename, const char* dstfilename )
{
long len;
char* buf = NULL;
FILE* fp = NULL;
// Open the source file
fp = fopen( srcfilename, "rb" );
if (!fp) return 0;
// Get its length (in bytes)
if (fseek( fp, 0, SEEK_END ) != 0) // This should typically succeed
{ // (beware the 2Gb limitation, though)
fclose( fp );
return 0;
}
len = ftell( fp );
std::cout << len;
rewind( fp );
// Get a buffer big enough to hold it entirely
buf = (char*)malloc( len );
if (!buf)
{
fclose( fp );
return 0;
}
// Read the entire file into the buffer
if (!fread( buf, len, 1, fp ))
{
free( buf );
fclose( fp );
return 0;
}
fclose( fp );
// Open the destination file
fp = fopen( dstfilename, "wb" );
if (!fp)
{
free( buf );
return 0;
}
// this is where I send data in but over serial port.
//serialWrite() is just the standard write() being used
int fd;
fd = xserialOpen("/dev/ttyUSB0", 9600);
serialWrite(fd, buf, len);
//This is where the file gets copied to another file as a test
// Write the entire buffer to file
if (!fwrite( buf, len, 1, fp ))
{
free( buf );
fclose( fp );
return 0;
}
// All done -- return success
fclose( fp );
free( buf );
return 1;
}
On the receive side I know I need to open up the serial port to read and use some sort of read() but I don't know how that is done. Using a serial library it has some functions to check if serial data is available and return the number of characters available to read.
One question about the number of characters available to read, will that number grow as the serial stream comes over or will it immediately tell the entire length of the data to be read?
But finally, I know after I open the serial port, I need read the data into a buffer and then write that buffer to a file but I have not had any luck. This is what I have tried thus far.
// Loop, getting and printing characters
char temp;
bool readComplete = false;
int bytesRead = 0;
fp = fopen("copyImage11.jpeg", "rwb");
for (;;)
{
if(xserialDataAvail(fd) > 0)
{
bytesRead = serialRead(fd, buf, len);
readComplete = true;
}
if (readComplete)
{
if (!fwrite(buf, bytesRead, 1, fp))
{
free(buf);
fclose(fp);
return 0;
}
fclose(fp);
free(buf);
return 1;
}
}
I don't get errors with my code, it just doesnt create the jpeg file correctly. Maybe I'm not transmitting it right, or maybe I'm not reading/writing to file correctly. Any help would be appreciated. Thanks everyone you rock!
If you are defining your own protocol, then you need to have a method for sending the length first.
I would recommend testing your code by sending short blocks of ascii text to confirm your i/o. Once that is working you can use the ascii to set up the transfer; ie send the length, and have your receiver ready for an expected block.

Suppress system() output

First off, I do mostly C#, .Net development so go easy on me if this is a stupid question.
I am implementing an Ericcson open source project to convert an image to another format. The problem is that on conversion an output to a console happens as follows...
1 file(s) copied.
I need to suppress this dialog that pops up. I just want to execute the system command with no output. I think I have isolated the area of the code causing this.
void writeOutputFile(char *dstfile, uint8* img, uint8* alphaimg, int width, int height)
{
char str[300];
if(format!=ETC2PACKAGE_R_NO_MIPMAPS&&format!=ETC2PACKAGE_RG_NO_MIPMAPS)
{
fWritePPM("tmp.ppm",width,height,img,8,false);
//PRINTF("Saved file tmp.ppm \n\n");
}
else if(format==ETC2PACKAGE_RG_NO_MIPMAPS)
{
fWritePPM("tmp.ppm",width,height,img,16,false);
}
if(format==ETC2PACKAGE_RGBA_NO_MIPMAPS||format==ETC2PACKAGE_RGBA1_NO_MIPMAPS||format==ETC2PACKAGE_sRGBA_NO_MIPMAPS||format==ETC2PACKAGE_sRGBA1_NO_MIPMAPS)
fWritePGM("alphaout.pgm",width,height,alphaimg,false,8);
if(format==ETC2PACKAGE_R_NO_MIPMAPS)
fWritePGM("alphaout.pgm",width,height,alphaimg,false,16);
// Delete destination file if it exists
if(fileExist(dstfile))
{
sprintf(str, "del %s\n",dstfile);
system(str);
}
int q = find_pos_of_extension(dstfile);
if(!strcmp(&dstfile[q],".ppm")&&format!=ETC2PACKAGE_R_NO_MIPMAPS)
{
// Already a .ppm file. Just rename.
sprintf(str,"move tmp.ppm %s\n",dstfile);
//PRINTF("Renaming destination file to %s\n",dstfile);
}
else
{
// Converting from .ppm to other file format
//
// Use your favorite command line image converter program,
// for instance Image Magick. Just make sure the syntax can
// be written as below:
//
// C:\imconv source.ppm dest.jpg
//
if(format==ETC2PACKAGE_RGBA_NO_MIPMAPS||format==ETC2PACKAGE_RGBA1_NO_MIPMAPS||format==ETC2PACKAGE_sRGBA_NO_MIPMAPS||format==ETC2PACKAGE_sRGBA1_NO_MIPMAPS)
{
// Somewhere after version 6.7.1-2 of ImageMagick the following command gives the wrong result due to a bug.
// sprintf(str,"composite -compose CopyOpacity alphaout.pgm tmp.ppm %s\n",dstfile);
// Instead we read the file and write a tga.
//PRINTF("Converting destination file from .ppm/.pgm to %s with alpha\n",dstfile);
int rw, rh;
unsigned char *pixelsRGB;
unsigned char *pixelsA;
fReadPPM("tmp.ppm", rw, rh, pixelsRGB, 8);
fReadPGM("alphaout.pgm", rw, rh, pixelsA, 8);
fWriteTGAfromRGBandA(dstfile, rw, rh, pixelsRGB, pixelsA, true);
free(pixelsRGB);
free(pixelsA);
sprintf(str,""); // Nothing to execute.
}
else if(format==ETC2PACKAGE_R_NO_MIPMAPS)
{
sprintf(str,"imconv alphaout.pgm %s\n",dstfile);
//PRINTF("Converting destination file from .pgm to %s\n",dstfile);
}
else
{
sprintf(str,"imconv tmp.ppm %s\n",dstfile);
//PRINTF("Converting destination file from .ppm to %s\n",dstfile);
}
}
// Execute system call
system(str);
free(img);
if(alphaimg!=NULL)
free(alphaimg);
}
I am lost at this point about how to suppress the console that pops up. As we iterate through images via a reference to the dll, many many console windows flash on the screen. Need to stop this from happening.
Help is greatly appreciated.
Try doing:
strcat( str, " > nul" ) // for Windows or
//strcat( str, " > /dev/null" ) // for Unix
system( str )
If it doesn't help then this may help:
#include <string>
#include <ShellAPI.h>
int system_no_output( std::string command )
{
command.insert( 0, "/C " );
SHELLEXECUTEINFOA ShExecInfo = {0};
ShExecInfo.cbSize = sizeof(SHELLEXECUTEINFO);
ShExecInfo.fMask = SEE_MASK_NOCLOSEPROCESS;
ShExecInfo.hwnd = NULL;
ShExecInfo.lpVerb = NULL;
ShExecInfo.lpFile = "cmd.exe";
ShExecInfo.lpParameters = command.c_str();
ShExecInfo.lpDirectory = NULL;
ShExecInfo.nShow = SW_HIDE;
ShExecInfo.hInstApp = NULL;
if( ShellExecuteExA( &ShExecInfo ) == FALSE )
return -1;
WaitForSingleObject( ShExecInfo.hProcess, INFINITE );
DWORD rv;
GetExitCodeProcess( ShExecInfo.hProcess, &rv );
CloseHandle( ShExecInfo.hProcess );
return rv;
}
and replace all your system() calls to system_no_output() ones.
To fully suppress the output, redirect both stdout and stderr:
system("command >nul 2>nul");
Take a look here if you're using C#. The class used in C# is ProcessStartInfo.
In the example in the link, look at the OpenWithStartInfo member function which will minimize the console.
As far as doing this in C++, take a look at the spawn family of functions here.

C++ write to front of file

I need to open a file as ofstream and write to the front of the file, while preserving
the remaining contents of the file, which will be "moved". Similar to "prepend"
a file.
Is this possible using the STL or boost ?
No -- the language (or library) doesn't really make much difference here. Most file systems just don't allow it, full stop.
The usual way to get the same effect is to write your new data to a new file, then copy the data in the old file to the new file following the data you wrote.
No it isn't. And this has been asked here many times before. If you want to do this you have to create new file, write the "prepend" data to it, then open the existing file and copy its contents to the new file.
A new iostream class can wrap that functionality. This assumes your prepend data isn't too large to comfortably fit in memory. Use it like a regular ofstream.
#include <fstream>
#include <sstream>
#include <vector>
class prepend_ofstream
: public std::ostringstream {
std::filebuf file;
public:
prepend_ofstream() {}
prepend_ofstream( char const *name, openmode mode = out ) {
open( name, mode );
}
~prepend_ofstream() {
if ( is_open() ) close();
}
void open( char const *name, openmode mode ) {
if ( ! file.open( name, mode & binary | in | out ) ) {
setstate( failbit );
}
}
bool is_open() { return file.is_open(); }
void close() {
if ( ! is_open() ) {
setstate( failbit );
return;
}
char *strbuf = &str()[0];
std::vector<char> buf( str().size() );
int rdsz;
do {
rdsz = file.sgetn( &buf[0], buf.size() );
file.pubseekoff( -rdsz, cur );
file.sputn( strbuf, buf.size() );
file.pubseekoff( 0, cur ); // "update the output sequence"
std::copy( &buf[0], &buf[0]+rdsz, strbuf );
} while ( rdsz == buf.size() );
file.sputn( &buf[0], rdsz );
if ( ! file.close() ) {
setstate( failbit );
}
}
};
Typically features are added through new stream buffer classes, not actual streams, but in this case the new functionality is in close, which is unfortunately not virtual.

Read() from file descriptor hangs

Hey, hopefully this should be my last PTY-related question and I can move onto more exciting issues. (c;
Here's a set of small functions I have written for creating and reading/writing to a pty: http://pastebin.com/m4fcee34d The only problem is that they don't work! After I run the initializer and writeToPty( "ls -l" ) , 'output' from readFromPty is still empty.
Ubuntu, QT C++
EDITED: Ok, I can confirm all this stuff works except for the read loop. In the debuggers' locals/watchers tab it shows that the QString 'output' actually does get the right data put in it, but after it ( the read() ) runs out of characters from the output it runs and then hangs. What is going on and how can I fix it?
Thanks! (c:
#include <iostream>
#include <unistd.h>
#include <utmp.h>
#include <pty.h>
#include <QString>
#include <QThread>
// You also need libutil in your .pro file for this to compile.
class CMkPty
{
public:
CMkPty( int *writeChannel, int *readChannel );
~CMkPty();
int runInPty( char *command );
int writeToPty( char *input );
int readFromPty( QString output );
int m_nPid;
private:
int m_nMaster, m_nSlave, m_nPosition, m_nBytes;
char *m_chName;
void safe_print( char *s );
char m_output;
};
CMkPty::CMkPty( int *masterFD, int *slaveFD )
{
openpty( &m_nMaster, &m_nSlave, (char*)0, __null, __null );
m_nPid = fork();
*masterFD = m_nMaster;
*slaveFD = m_nSlave;
if( m_nPid == 0 )
{
login_tty( m_nSlave );
execl( "/bin/bash", "-l", (char*)0 );
return;
}
else if( m_nPid > 0 )
{
return;
}
else if( m_nPid < 0 )
{
std::cout << "Failed to fork." ;
return;
}
}
CMkPty::~CMkPty()
{
close( m_nMaster );
close( m_nSlave );
}
int CMkPty::writeToPty( char *szInput )
{
int nWriteTest;
write( m_nMaster, szInput, sizeof( szInput ) );
nWriteTest = write( m_nMaster, "\n", 1 );
if( nWriteTest < 0 )
{
std::cout << "Write to PTY failed" ;
return -1;
}
return 0;
}
int CMkPty::readFromPty( QString output )
{
char buffer[ 160 ];
m_nBytes = sizeof( buffer );
while ( ( m_nPosition = read( m_nMaster, buffer, m_nBytes ) ) > 0 )
{
buffer[ m_nPosition ] = 0;
output += buffer;
}
return 0;
}
EDIT: Here's a link to the question with the code that finally worked for me.
I'm note entirely familiar with posix, but after reading this page http://pwet.fr/man/linux/fonctions_bibliotheques/posix/read I had some insight. What's more, I don't see you adjusting your M_nBytes value if you haven't read as much as you were expecting on the first pass of the loop.
edit: from that link, perhaps this will be of some help:
If some process has the pipe open for writing and O_NONBLOCK is clear, read() shall block the calling thread until some data is written or the pipe is closed by all processes that had the pipe open for writing.
When attempting to read a file (other than a pipe or FIFO) that supports non-blocking reads and has no data currently available:
*
If O_NONBLOCK is clear, read() shall block the calling thread until some data becomes available.
so essentially, if you're not in an error state, and you tell it to keep reading, it will block until it finds something to read.