I wrote the following code to capture packets; but, it actually save the last packet.
process_Packet(const struct pcap_pkthdr *header,
const u_char * packet)
{
FILE* pFile = NULL;
pFile = fopen ("myfile.pcap" , "wb"); // open for writing in binary mode
pcap_dumper_t * dumpfile = pcap_dump_fopen(pcap_handle,pFile);
if (dumpfile == NULL)
{
printf("***NOOOO Dump!!!!!!!***");
}
else
{
pcap_dump((unsigned char *) dumpfile, header, packet);
printf("***Dumped!!!!!!!***");
}
pcap_dump_close(dumpfile);
}
I want to write a code that collect packets and append the new packet to previous ones.
I should say that fopen("...", "ab") corrupts the file and doesn't work.
pcap_dump_fopen writes some initialization headers, so it should be called only once on empty file. After file with headers created you actually can pass FILE* instance opened in append mode to pcap_dump directly casted to unsigned char *. But it is not safe approach - better at least write all required fields yourself (it's like 10 lines anyway) since function implementation may change in the future and file format will not. And I don't really understand why you would like to reopen file on every packet dumped. If you want to ensure all data is written you can just call fflush.
Related
I am trying to read pcap file to some data structure like vector or array and later use gathered data (only selected one like packet length, timestamp) in application. I've found some sample application for reading pcap:
#include <stdio.h>
#include <pcap.h>
#define LINE_LEN 16
void dispatcher_handler(u_char *, const struct pcap_pkthdr *, const u_char *);
int main(int argc, char **argv)
{
pcap_t *fp;
char errbuf[PCAP_ERRBUF_SIZE];
if(argc != 2)
{
printf("usage: %s filename", argv[0]);
return -1;
}
/* Open the capture file */
if ((fp = pcap_open_offline(argv[1], // name of the device
errbuf // error buffer
)) == NULL)
{
fprintf(stderr,"\nUnable to open the file %s.\n", argv[1]);
return -1;
}
/* read and dispatch packets until EOF is reached */
pcap_loop(fp, 0, dispatcher_handler, NULL);
pcap_close(fp);
return 0;
}
void dispatcher_handler(u_char *temp1,
const struct pcap_pkthdr *header,
const u_char *pkt_data)
{
u_int i=0;
/*
* unused variable
*/
(VOID*)temp1;
/* print pkt timestamp and pkt len */
printf("%ld:%ld (%ld)\n", header->ts.tv_sec, header->ts.tv_usec, header->len);
printf("\n\n");
}
The problem is with pcap_loop(). I've found documentation for this, but the only information is that this is reading whole file until end of file is reached. I've been trying to treat file as a typical FILE, and in while loop read until EOF, but it doesn't work, because I cannot simply treat fp as FILE.
Also I don't see any possibility to pass pointer to pcap_handler to retrieve it later.
Can someone suggest how I can do it other way?
According to documentation your code looks right. pcap_loop should be doing the FILE reading, you shouldn't try to do it.
One thing the doc mentions is that in older pcap versions 0 for count in pcap_loop is undefined, so it should be safer to use -1 in case you are linking to an older version.
I after further documentation and Internet investigation I've found function: pcap_next_ex()
Thanks to that I can use now while loop and read line by line (or more precisely packet by packet). The general idea is as follows:
struct pcap_pkthdr *header;
const u_char *pkt_data;
int res;
while((res = pcap_next_ex(fp, &header, &pkt_data)) >= 0)
{
//Process packet
}
I've been trying to treat file as a typical FILE, and in while loop read until EOF, but it doesn't work, because I cannot simply treat fp as FILE.
No, because it's not a FILE. (You also get a pcap_t * from pcap_open_live() or from a pcap_create()/pcap_activate() combination, but that gives you a handle for a live capture, not for a file.)
Also I don't see any possibility to pass pointer to pcap_handler to retrieve it later.
The fourth argument to pcap_loop() is passed as the first argument to pcap_handler, so you could do
pcap_loop(fp, 0, dispatcher_handler, pointer);
and then, in dispatcher_handler(), cast temp1 to the appropriate type and use it - it'll point to the same thing that pointer does.
I am baffled with the way fopen() is responding to my code. Note that I am using it to write binary files read from FPGA. The scenario is this
When I WRITE data to FPGA based system and then read back (after processing), only 307 files in binary can be saved, after which the fopen() returns NULL value
When I only read from FPGA (no writing at all) then it saves perfectly in all the 500 files.
When I do both cases for writing in .txt files, every thing goes perfect
CASE 1 is my problem and the real-world implementation I need. Hence I cannot understand this strange behaviour. One thing is assured with several testing that the whole problem is with the PC/C++ side and FPGA has nothing to do with it.
The first code is the calling of function, the second code is the function for writing binary files itself
...
sprintf(fwname, "data/uwpi%d.bin", fnum++);
opfile = fopen(fwname , "w");
rc = sipif_readdata(pInDMA, (40000)*2);
save_binary(pInDMA, 40000, fwname, opfile , fnum);
...
The following is the function details
static ULONG save_binary(void *buf, int bufsize, const char *filename, FILE *fOutFile, int fnum){
// cast our stamp less pointer to a short
short *buf16 = (short *)buf;
fOutFile = fopen(filename, "ab");
if(fOutFile==NULL) {
//comment added to verify my claim
printf("Null value found at file %d\n", fnum);
return -3; }
printf("fnum is %d\n", fnum);
fwrite(buf, 2, bufsize, fOutFile);
fclose(fOutFile);
return 0;
}
I have a program that records data from a serial port. Every so often, I want to split up files such that the data logs don't become very large. The problem is, after I recreate the FILE* and try to write into it, the program crashes. No compiler errors/warnings before hand also...
The program does create one log for the first time interval, but once it's time to create a new data log, it crashes at the fwrite.
First off, initializations/declarations.
char * DATA_DIR = "C:\DATA";
sprintf(path,"%s%s%s",DATA_DIR,curtime,".log"); //curtime is just the current time in a string
FILE * DATA_LOG = fopen(path, "wb+");
And later on in a while loop
if(((CURRENT_TIME-PREVIOUS_TIME) > (SEC_IN_MINUTE * MINUTE_CHUNKS) ) && (MINUTE_CHUNKS != 0) && FIRST_TIME == 0) //all this does is just checks if its time to make a new file
{
fclose(DATA_LOG); //end the current fileread
char * path;
char curtime[16];
//gets the current time and saves it to a file name
sprintf(curtime , "%s" , currentDateTime());
sprintf(path,"%s%s%s",DATA_DIR,curtime,".log");
DATA_LOG = fopen(path, "wb+"); //open the new file
//just some logic (not relevant to problem)
PREVIOUS_TIME = CURRENT_TIME;
newDirFlag = 1;
}
fwrite(cdata , sizeof(char) , numChars , DATA_LOG); //crashes here. cdata, sizeof, and numChars don't change values
Any ideas why is this happening? I'm stumped.
Couple of problems, path has no memory allocated (you're writing stuff to some random memory address which is bad). You also should check the return values of fwrite fopen for errors. If there is one use perror so you know what the problem is. It's likely the fopen is failing or you're corrupting your stack by writing to path.
Also use snprintf it's much safter than just sprintf which is vulnerable to buffer overflow.
EDIT: just saw your comment that it's c++. Why not use std::string and fstream instead? They are much safer than what you're currently doing (and probably easier).
Your MAIN problem is that char * path; has no memory assigned to it. This means that you are writing to some RANDOM [1] location in memory.
I would suggest that you use char path[PATH_MAX]; - that way you don't have to worry about allocating and later deallocating the storage for your path.
Alternatively, you could use:
stringstream ss;
ss << DATA_DIR << currentDateTime() << ".log";
string path = ss.str();
fopen(path.c_str(), "wb+")
which is a more C++ style solution.
[1] By random, I don't mean truly a random number, but some unknown value that happens to be in that location on the stack. It is almost always NOT a good place to store a string.
I am programming a face detection algorithm. In my code I'm parsing an XML file (in a recursion way, very inefficient takes my about 4 minutes to parse the whole XML file). I'd like to save the XML content using Iosteam binary to a file. I'm using a struct in C++ in order to use the raw data.
My goal is to parse the XML only if the raw data file is not exist.
The method work like this:
If the raw data file is not exist, parse the XML file and save the data to a file.
If the raw data file exist, read the raw data from the file
My problem is: whenever I open the raw data file and read from it. I get to read only small amount of byte from the file, I don't know how much, but in a certain point I receive only 0x00 data on my buffer.
My guess: I believe this has to do with the OS buffer, Which has a certain amount of buffer for read and write operations. I might be wrong about this. Though I'm not sure which one from the operations doesn't work well, it's either the write or read.
I was thinking to write / read the raw data char by char or line by line. In the other hand the file doesn't contain a text, which means that I can't read line by line or char by char.
The raw data size is
size_t datasize = DataSize(); == 196876 (Byte)
Which is retrieve in this function
/* Get the upper bound for predefined cascade size */
size_t CCacadeInterpreter::DataSize()
{
// this is an upper boundary for the whole hidden cascade size
size_t datasize = sizeof(HaarClassifierCascade) * TOTAL_CASCADE+
sizeof(HaarStageClassifier)*TOTAL_STAGES +
sizeof(HaarClassifier) * TOTAL_CLASSIFIERS +
sizeof(void*)*(TOTAL_CASCADE+TOTAL_STAGES+TOTAL_CLASSIFIERS);
return datasize;
}
The method work like this
BYTE * CCacadeInterpreter::Interpreter()
{
printf("|Phase - Load cascade from memory | CCacadeInterpreter::Interpreter | \n");
size_t datasize = DataSize();
// Create a memory structure
nextFreeSpace = pStartMemoryLocation = new BYTE [datasize];
memset(nextFreeSpace,0x00,datasize);
// Try to open a predefined cascade file on the current folder (instead of parsing the file again)
fstream stream;
stream.open(cascadeSavePath); // ...try existing file
if (stream.is_open())
{
stream.seekg(0,ios::beg);
stream.read((char*)pStartMemoryLocation , datasize); // **ream from file**
stream.close();
printf("|Load cascade from saved memory location | CCacadeInterpreter::Interpreter | \n");
printf("Completed\n\n");
stream.close();
return pStartMemoryLocation;
}
// Open the cascade file and parse the cascade xml file
std::fstream cascadeFile;
cascadeFile.open(cascadeDestanationPath, std::fstream::in); // open the file with read only attributes
if (!cascadeFile.is_open())
{
printf("Error: couldn't open cascade XML file\n");
delete pStartMemoryLocation;
return NULL;
}
// Read the file XML file , line by line
string buffer, str;
getline(cascadeFile,str);
while(cascadeFile)
{
buffer+=str;
getline(cascadeFile,str);
}
cascadeFile.close();
split(buffer, '<',m_tokens);
// Parsing begins
pHaarClassifierCascade = (HaarClassifierCascade*)nextFreeSpace;
nextFreeSpace += sizeof(HaarClassifierCascade);
pHaarClassifierCascade->count=0;
pHaarClassifierCascade->orig_window_size_height=20;
pHaarClassifierCascade->orig_window_size_width=20;
m_deptInTree=0;
m_numOfStage = 0;
m_numOfTotalClassifiers=0;
while (m_tokens.size())
{
Parsing();
}
// Save the current cascade into a file
SaveBlockToMemory(pStartMemoryLocation,datasize);
printf("\nCompleted\n\n");
return pStartMemoryLocation;
}
bool CCacadeInterpreter::SaveBlockToMemory(BYTE * pStartMemoryLocation,size_t dataSize)
{
fstream stream;
if (stream.is_open() )
stream.close();
stream.open(cascadeSavePath); // ...try existing file
if (!stream.is_open()) // ...else, create new file...
stream.open(cascadeSavePath, ios_base::in | ios_base::out | ios_base::trunc);
stream.seekg(0,ios::beg);
stream.write((char*)pStartMemoryLocation,dataSize);
stream.close();
return true;
}
Try using the Boost IOstreams library.
It has an easy to use wrrapers for file handling
I'm pretty close to losing my head here ;)
I'm developing a service that uses gsoap. I would like to return a mime response.
I have everything working, but when reading binary files, all kind of files like jpeg, pdf, etc... contains the \0 char several times over the data (if opened with notepad can see a lot of NUL).
So any code for reading a raw file fails miserably once it finds the end-of-file char. I have tried to replace the \0 but the file becomes incorrect to display.
I have also tried several methods including the example that comes with gsoap.
So resuming,
fstream generic code doesn't work.
for (i = 0; i < MAX_FILE_SIZE; i++)
{ if ((c = fgetc(fd)) == EOF)
break;
image.__ptr[i] = c;
}
doesn't work also
QFile::ReadAll works but when converting QString to char* the array is trimmed in the first NUL.
So, which is the best aproach to read an entire binary file? Its crazy how sometimes C++ at the basic.
Thanks in advance.
I have tried this as retnick suggested below
UrlToPdf urlToPdf;
urlToPdf.getUrl(&input, &result);
QByteArray raw = urlToPdf.getPdf(QString(result.data.c_str()));
int size = raw.toBase64().size();
char* arraydata = new char[size];
strcpy(arraydata, raw.toBase64().data());
soap_set_mime(this, "MIME_boundary", NULL);
if(soap_set_mime_attachment(this, arraydata, size, SOAP_MIME_BASE64, "application/pdf", NULL, NULL, NULL))
{
soap_clr_mime(this);
soapMessage = this->error;
}
but no luck... the mime response is bigger than the actual file...
David G Ortega
to read binary files use fread()
Once you read it treat it as an array of bytes not as a string. No string functions allowed.
EDIT: The gSOAP documentation section 14.1 explains how to send MIME attachments. I only refer to the relevant function (please read it all).
int soap_set_mime_attachment(struct soap *soap, char *buf_ptr, size_t buf_size,
enum soap_mime_encoding encoding,
const char *type, const char *id,
const char *location, const char *description);
char *buf_ptr is your buffer.
size_t buf_size is the length of your buffer.
So just do your QFile::ReadAll.
this gives you back a QByteArray. The QByteArray has the method
QByteArray QByteArray::toBase64 () const
this will return a
QByteArray base64image = QByteArray::toBase64(rawImage);
so now just do
soap_set_mime(soap, "MIME_boundary", "<boundary.xml#just-testing.com>");
/* add a base64 encoded image (base64image points to base64 data) */
soap_set_mime_attachment(soap,
base64image.data(), base64image.size(),
SOAP_MIME_BASE64, "image/jpeg",
"<boundary.jpeg#just-testing.com>", NULL, NULL);
I have not tested this but should be close to finished.
QFile::ReadAll works but when converting QString to char* the array is trimmed in the first NUL.
Are you sure it's actually trimmed or you just can't print/view the array in the debugger [since C-style strings are 0 terminated]?
If the QString itself is not enough for your needs you may want to convert it to a std::vector or similar using the range constructor or range assign, you'll have lots less grief towards the how much data the container holds.
EDIT:
Here's some sample code for fstream reading from a binary file:
std::ifstream image( <image_file_name>, std::ios_base::in | std::ios_base::binary );
std::istream_iterator< char > image_begin( image ), image_end;
std::vector< char > vctImage( image_begin, image_end );
The std::ios_base::binary is the most important part of the thing (similar to fopen/fread ["rb"] & probably QFile has something similar)
Also posting some sample code usually helps in getting the right answer.
HIH
I have the solution for this... As renick suggested I tried his idea but it failed without undestanding it so much... From a logical point of view recnick was right... bat the truth is that any king of string manipulation using QT QByteArray, std or mem is going to stop when findind the first \0 char, Qt QString can do it without problems but when converting it to c string (char*) the data will be again trimmed with the first \0
I found that using QDataStream::readRawData reads the file into a char* given the size to read. So thats how I accomplished the deal...
QFile file("test.pdf");
file.open(QIODevice::ReadOnly);
int size = file.size();
char* buffer = new char[size];
QDataStream stream(&file);
stream.readRawData(buffer, size);
soap_set_mime(this, "MIME_boundary", NULL);
if(soap_set_mime_attachment(this, buffer, size, SOAP_MIME_BINARY, "application/pdf", NULL, NULL, NULL))
{
soap_clr_mime(this);
soapMessage = this->error;
}
Note that in the line
if(soap_set_mime_attachment(this, buffer, size, SOAP_MIME_BINARY, "application/pdf", NULL, NULL, NULL))
I'm still using the size var instead of doing sizeof(buffer) or any other aproach since this one is going to trimm again the data qhen finding the first \0...
Hope this helps...
David G Ortega