I have a device that sends serial data to my esp8266.
I need to parse that data to my application.
The format of each line I receive from the serial port is like this:
"\x02""Q,328,013.83,N,00,\x03""1D"
where the 1st character is char 2 (start of transmission) and the 3rd from the end is char 3 (end of transmission). The last number ("1C") is the checksum.
The numbers between, are the values I want to parse.
I 've written the following code which works, but I wonder if this the correct way of doing it.
#include <iostream>
#include <stdio.h>
#include <string.h>
uint16_t calcCRC(char* str)
{
uint16_t crc=0; // starting value as you like, must be the same before each calculation
for (uint16_t i=0; i<strlen(str); i++) // for each character in the string
{
crc ^= str[i]; // update the crc value
}
printf("CRC: %X\n", crc);
return crc;
}
int main()
{
char * data= "\x02""Q,328,013.83,N,00,\x03""1D";
const char start = 2; //stx start transmission
const char end = 3; //etx end transmission
char* pos1 = strchr(data, start);
char* pos2 = strchr(data, end);
int p1,p2;
if (pos1 && pos2) {
p1 = pos1-data+1;
p2 = pos2-data+1;
printf ("found at %d, %d\n", p1, p2);
} else
return 0;
char* checksumStr;
checksumStr = strrchr(data, end);
int checksum = 0;
if (checksumStr) {
checksum = strtol(checksumStr+1, NULL, 16);
}
printf("checksum char: |%s| check number %X\n", checksumStr+1, checksum);
char cleanData[25];
strncpy(cleanData, data+p1, p2-p1-1);
cleanData[p2-p1-1] = '\0';
uint16_t crc = calcCRC(cleanData);
printf("Clean data to checksum: |%s|\n", cleanData);
char* addr = strtok(cleanData, ","); // Q
int WindDir = atoi(strtok(NULL, ",")); // 328
float WindSpeed = atof(strtok(NULL, ",")); // 13.83
char* unit = strtok(NULL, ","); // N
char* deviceStatus = strtok(NULL,","); // 00
printf("CRC: %X, Speed %3.2f, dir %d, ", crc, WindSpeed, WindDir);
return(0);
}
Run it here
Thank you !!
Break your code into two parts:
Message extraction – extracts the actual message from buffer and checks the checksum.
Message processing – converts the message to tokens which then can be easily take care of and extracts the information.
Your calling code should look like this:
// extract the message
char msg[ 1024 ];
if ( !extract_message( msg, data ) )
return false;
// process the message
int dir;
double speed;
process_message( msg, dir, speed );
A message extraction function idea:
#define STX '\x02'
#define ETX '\x03'
bool extract_message( char* d, const char* s )
{
// start of text
if ( *s != STX )
return false;
++s;
// actual message
char cs = 0; // checksum
while ( *s && ETX != *s )
{
cs ^= *s;
*d++ = *s++;
}
*(d-1) = '\0';
// end of text
if ( *s != ETX )
return false;
++s;
// sent checksum
char scs = (char)strtol( s, 0, 16 );
//
return scs == cs;
}
A processing function idea:
#define MAX_TOKENS 5
bool process_message(char* buf, int& dir, double& speed) // assumes the message is correct
{
// breaks the input into tokens (see strtok)
const char* delim = ",";
const char* tokens[ MAX_TOKENS ];
int token_count = 0;
for ( char* cursor = strtok( buf, delim ); cursor; cursor = strtok( 0, delim ) )
{
tokens[ token_count ] = cursor;
++token_count;
}
// ...
dir = atoi( tokens[ 1 ] ); // assumes the token is valid
speed = atof( tokens[ 2 ] ); // assumes the token is valid
//
return true;
}
I do not have a handy Arduino compiler, so the code might need some tweaking.
If your code works well, it's fine, but I recommend to you to write C++ style codes. I wrote part of your code in C++. I believe you can understand how to read the rest of your data using istream.
I also hope you perfer using std::string than const char*, use std::iostream to get or put data from and to. There's more flexible way to split strings using std::regex but I think it's too hard to understand currently.
#include <iostream>
#include <sstream>
#include <string>
using namespace std;
int main(void)
{
string data = "Q,328,013.83,N,00,1D";
istringstream iss(data);
string tmp;
int wind_dir;
float wind_speed;
getline(iss, tmp, ','); // read until ','
iss >> wind_dir;
getline(iss, tmp, ','); // read until ','
iss >> wind_speed;
return 0;
}
It is a little unclear what you are asking. When you say:
I've written the following code which works, ...
There is no way the code as written above can work. You do not have the control characters 2 (ASCII stx - start transmission) or 3 (ASCII etx - end transmission) embedded in data, so your conditional if (pos1 && pos2) tests false and you exit with return 0; (normally return 1; is used to indicate error, that is the value of EXIT_FAILURE)
To make your code work and include stx at the beginning of data and etx at the end, you can do:
char data[32] = "";
char stx[2] = { 2 };
char etx[2] = { 3 };
char *serial= "Q,328,013.83,N,00,1D";
strcat (data, stx);
strcat (data, serial);
strcat (data, etx);
That now provides data in a form where strchr(data, start); and strchr(data, end); will properly locate the start and end of the serial data in the example.
The remainder of your code DOES work, though note all your variables addr, WinDir, WindSpeed, unit and deviceStatus are pointers to address within cleanData and are only valid while cleanData remains in scope. That is fine here as your code ends thereafter, but if you intend to do this within a function and want to return the values, then you will need to provide storage for each and copy each to storage that will remain valid after cleanData has gone out of scope.
Currently addr, unit and deviceStatus are unused in your code. You can write a short printf to output their values to have your code compile cleanly, without warning (you can also cast to (void*), but I'm not positive that is supported on Arduino). Adding a short printf solves the warning issue and gives you a look at all the data parse from cleanData, e.g.
printf ("\naddr : %s\n"
"WinDir : %d\n"
"WindSpeed : %.2f\n"
"unit : %s\n"
"deviceStatus : %s\n\n",
addr, WindDir, WindSpeed, unit, deviceStatus);
With that, your code compiles cleanly and does work (though you need a final '\n' at the end of printf("CRC: %X, Speed %3.2f, dir %d, ", crc, WindSpeed, WindDir); (or a simple putchar ('\n'); for your code to be POSIX compliant.
I'm not sure if the embedded stx and etx were your stumbling blocks or not, that's not 100% clear from the question, but without them the code does not work. Let me know if you have further questions.
After Question Update
If you are looking for an alternative to parsing the data, you could declare addr, unit and devicestatus a short character arrays, e.g. char [16] (or whatever the longest is), and then simply use sscanf to parse data directly, e.g.
if (sscanf (data, "\x02 %15[^,],%d,%f, %15[^,], %15[^,], \x03",
addr, &winddir, &windspeed, unit, devicestatus) != 5) {
fputs ("error parsing data.\n", stderr);
return 1;
}
sscanf can adequately handle the embedded stx and etx. A minimal example using sscanf on data directly could be:
#include <stdio.h>
#define MINSTR 16
int main (void) {
char *data = "\x02""Q,328,013.83,N,00,\x03""1D",
addr[MINSTR],
unit[MINSTR],
devicestatus[MINSTR];
int winddir;
float windspeed;
if (sscanf (data, "\x02 %15[^,],%d,%f, %15[^,], %15[^,], \x03",
addr, &winddir, &windspeed, unit, devicestatus) != 5) {
fputs ("error parsing data.\n", stderr);
return 1;
}
printf ("addr : %s\n"
"windir : %d\n"
"windspeed : %.2f\n"
"unit : %s\n"
"devicestatus : %s\n", addr, winddir, windspeed, unit, devicestatus);
}
Example Use/Output
$ ./bin/windspeed+dir
addr : Q
windir : 328
windspeed : 13.83
unit : N
devicestatus : 00
Related
I'm using CNG to generate a hash.
Result of BCryptFinishHash call is MD5 of a input in hex form.
Example:
char *outHash = "\x02\x34\x75\01..."
I want to convert it to printable string: 02347501...
How can I do that?
To encode a byte array in hex and write the encoded data to a std::string, do this:
static inline char
hex_digit(unsigned int n)
{
if (n < 10) return '0' + n;
if (n < 16) return 'a' + (n - 10);
abort();
}
std::string
encode_bytes(const unsigned char *bytes, size_t len)
{
std::string rv;
rv.reserve(len * 2);
for (size_t i = 0; i < len; i++) {
rv.push_back(hex_digit((bytes[i] & 0xF0) >> 4));
rv.push_back(hex_digit((bytes[i] & 0x0F) >> 0));
}
return rv;
}
Note that you must know the length of the byte array. It is not safe to treat it as a NUL-terminated "C string", because binary data can contain internal zero bytes. To know the length of a hash generated by CNG, call BCryptGetProperty to get the BCRYPT_HASH_LENGTH property.
we can use CryptBinaryToString here with CRYPT_STRING_HEXASCII or CRYPT_STRING_HEX or CRYPT_STRING_HEXRAW or CRYPT_STRING_HEX | CRYPT_STRING_NOCRLF or CRYPT_STRING_HEXRAW | CRYPT_STRING_NOCRLF depen how you want format string. for example
void print(PUCHAR pbHash, ULONG cbHash, DWORD dwFlags = CRYPT_STRING_HEXRAW | CRYPT_STRING_NOCRLF)
{
ULONG cch = 0;
if (CryptBinaryToStringW(pbHash, cbHash, dwFlags, 0, &cch))
{
if (PWSTR sz = (PWSTR)_malloca(cch * sizeof(WCHAR)))
{
if (CryptBinaryToStringW(pbHash, cbHash, dwFlags, sz, &cch))
{
DbgPrint("%S\n", sz);
}
_freea(sz);
}
}
}
If you need an easy, one time solution, this is a useful tool:
https://codebeautify.org/hex-string-converter
However, if you're looking to do this within your code itself, I found this from an earlier thread (AKA, this is not my work but that of #KEINE LUST from here )
int main(void)
{
unsigned char readingreg[4];
readingreg[0] = 0x4a;
readingreg[1] = 0xaa;
readingreg[2] = 0xaa;
readingreg[3] = 0xa0;
char temp[4];
sprintf(temp, "%x", readingreg[0]);
printf("This is element 0: %s\n", temp);
return 0;
}
You can print it like this:
for(const char *wsk=outHash; *wsk; ++wsk){
printf("%02hhx", *wsk);
}
Edit based that cstring can have 0x00 numbers.
C
const char outHash[] = "\x02\x34\x75";
const int size = sizeof(outHash)/sizeof(char) - 1;
for(int i = 0; i < size; ++i){
printf("%02hhx", outHash [i]);
}
C++
std::string outHash = "\x02\x34\x75";
for(int i = 0; i < outHash.size(); ++i) {
printf("%02hhx", outHash [i]);
}
Loop over the characters and print the numerical value (in hex).
#include <iostream>
#include <iomanip>
int main()
{
char* outHash = "\x02\x34\x75\x01\x23\xff"; // Get from your Hash function.
int sizeOfHash = 6; // Use appropriate size for BCryptFinishHash()
// Set up the characteristics of the stream.
// setw(2): Each printed object will use a min width of 2
// setfill('0'): If the object is less than 2 char then fill the space with '0'
// hex: Print numbers in hex.
std::cout << std::setw(2) << std::setfill('0') << std::hex;
// Create a view of the object.
// Makes it simpler to loop over.
std::string_view view(outHash, sizeOfHash);
// Loop over the string.
for(unsigned char val: view) {
// Convert to `unsigned char` to make sure you don't print
// negative numbers. Then convert from there to `int` so that
// the `std::hex will kick in and convert to hex value.
std::cout << static_cast<int>(val);
}
std::cout << "\n";
}
I am working on C++ wrapper around Windows Crypto API & CNG which I am using in my projects. I plan to move all of it to github but for now it is just a work in progress, but you can find it useful for Crypto basics like HEX / Base64 encode / decode etc.
https://github.com/m4x1m1l14n/Crypto
You can use Crypto::Hex::Encode() method to achieve what you want.
#include <Crypto\Hex.hpp>
#include <Crypto\Random.hpp>
using namespace m4x1m1l14n;
char arr[] = { 0xaa, 0xbb, 0xcc, 0xdd, 0x99, 0x00 };
encoded = Crypto::Hex::Encode(arr, sizeof(arr));
/* encoded = "aabbccdd9900" */
Also you can use wrapper for MD5 which is located in Hash namespace, like this. (If you are not using large amount of data)
#include <Crypto\Hex.hpp>
#include <Crypto\Hash.hpp>
using namespace m4x1m1l14n;
encoded = Crypto::Hex::Encode(Crypto::Hash::MD5("Whatever you want to hash"));
I have following piece of code that is supposed to calculate the SHA256 of a file. I am reading the file chunk by chunk and using EVP_DigestUpdate for the chunk. When I test the code with the file that has content
Test Message
Hello World
in Windows, it gives me SHA256 value of 97b2bc0cd1c3849436c6532d9c8de85456e1ce926d1e872a1e9b76a33183655f but the value is supposed to be 318b20b83a6730b928c46163a2a1cefee4466132731c95c39613acb547ccb715, which can be verified here too.
Here is the code:
#include <openssl\evp.h>
#include <iostream>
#include <string>
#include <fstream>
#include <cstdio>
const int MAX_BUFFER_SIZE = 1024;
std::string FileChecksum(std::string, std::string);
int main()
{
std::string checksum = FileChecksum("C:\\Users\\Dell\\Downloads\\somefile.txt","sha256");
std::cout << checksum << std::endl;
return 0;
}
std::string FileChecksum(std::string file_path, std::string algorithm)
{
EVP_MD_CTX *mdctx;
const EVP_MD *md;
unsigned char md_value[EVP_MAX_MD_SIZE];
int i;
unsigned int md_len;
OpenSSL_add_all_digests();
md = EVP_get_digestbyname(algorithm.c_str());
if(!md) {
printf("Unknown message digest %s\n",algorithm);
exit(1);
}
mdctx = EVP_MD_CTX_create();
std::ifstream readfile(file_path,std::ifstream::in|std::ifstream::binary);
if(!readfile.is_open())
{
std::cout << "COuldnot open file\n";
return 0;
}
readfile.seekg(0, std::ios::end);
long filelen = readfile.tellg();
std::cout << "LEN IS " << filelen << std::endl;
readfile.seekg(0, std::ios::beg);
if(filelen == -1)
{
std::cout << "Return Null \n";
return 0;
}
EVP_DigestInit_ex(mdctx, md, NULL);
long temp_fil = filelen;
while(!readfile.eof() && readfile.is_open() && temp_fil>0)
{
int bufferS = (temp_fil < MAX_BUFFER_SIZE) ? temp_fil : MAX_BUFFER_SIZE;
char *buffer = new char[bufferS+1];
buffer[bufferS] = 0;
readfile.read(buffer, bufferS);
std::cout << strlen(buffer) << std::endl;
EVP_DigestUpdate(mdctx, buffer, strlen(buffer));
temp_fil -= bufferS;
delete[] buffer;
}
EVP_DigestFinal_ex(mdctx, md_value, &md_len);
EVP_MD_CTX_destroy(mdctx);
printf("Digest is: ");
//char *checksum_msg = new char[md_len];
//int cx(0);
for(i = 0; i < md_len; i++)
{
//_snprintf(checksum_msg+cx,md_len-cx,"%02x",md_value[i]);
printf("%02x", md_value[i]);
}
//std::string res(checksum_msg);
//delete[] checksum_msg;
printf("\n");
/* Call this once before exit. */
EVP_cleanup();
return "";
}
I tried to write the hash generated by program as string using _snprintf but it didn't worked. How can I generate the correct hash and return the value as string from FileChecksum Function? Platform is Windows.
EDIT: It seems the problem was because of CRLF issue. As Windows in saving file using \r\n, the Checksum calculated was different. How to handle this?
MS-DOS used the CR-LF convention,So basically while saving the file in windows, \r\n comes in effect for carriage return and newline. And while testing on online (given by you), only \n character comes in effect.
Thus either you have to check the checksum of Test Message\r\nHello World\r\n in string which is equivalent to creating and reading file in windows(as given above), which is the case here.
However, the checksum of files,wherever created, will be same.
Note: your code works fine :)
It seems the problem was associated with the value of length I passed in EVP_DigestUpdate. I had passed value from strlen, but replacing it with bufferS did fixed the issue.
The code was modified as:
while(!readfile.eof() && readfile.is_open() && temp_fil>0)
{
int bufferS = (temp_fil < MAX_BUFFER_SIZE) ? temp_fil : MAX_BUFFER_SIZE;
char *buffer = new char[bufferS+1];
buffer[bufferS] = 0;
readfile.read(buffer, bufferS);
EVP_DigestUpdate(mdctx, buffer, bufferS);
temp_fil -= bufferS;
delete[] buffer;
}
and to send the checksum string, I modified the code as:
EVP_DigestFinal_ex(mdctx, md_value, &md_len);
EVP_MD_CTX_destroy(mdctx);
char str[128] = { 0 };
char *ptr = str;
std::string ret;
for(i = 0; i < md_len; i++)
{
//_snprintf(checksum_msg+cx,md_len-cx,"%02x",md_value[i]);
sprintf(ptr,"%02x", md_value[i]);
ptr += 2;
}
ret = str;
/* Call this once before exit. */
EVP_cleanup();
return ret;
As for the wrong checksum earlier, the problem was associated in how windows keeps the line feed. As suggested by Zangetsu, Windows was making text file as CRLF, but linux and the site I mentioned earlier was using LF. Thus there was difference in the checksum value. For files other than text, eg dll the code now computes correct checksum as string
I want to exchange data over TCP/IP with C++ by writing my own small binary protocol. My idea was to implement the protocol by defining values in form of the following bytes:
BEGIN
LENGTH
FUNCTIONCODE
[data bytes] (optional)
CRC
END
How can I define a binary value for "BEGIN" that is unique allowing the receiver to recognize the beginning of a new telegram? E.g. if I do that:
static const int BEGIN = 0x41;
and the optional data bytes randomly include also the 0x41 value, this might be a problem for the receiver. Or am I wrong? If not, how may I define a unique BEGIN and END value?
I usually write something like this for simple C++ networking. (But I wrote this code from scratch this time and haven't tested it at all!)
class Packet : public std::string {
public:
struct __attibute__((__packed__)) Header {
static const int BEGIN = 0x41;
uint32_t begin;
uint16_t length;
uint16_t funcode;
void init(uint16_t len) { begin = BEGIN; length = len; }
bool verify() { return (BEGIN == begin); }
char *buffer() { return reinterpret_cast<char *>(this); }
};
class __attibute__((__packed__)) Footer {
uint32_t crc;
char term;
char newline;
void init() { term = ';'; newline = '\n'; }
bool verify() { return (';' == term && '\n' == newline); }
char *buffer() { return reinterpret_cast<char *>(this); }
};
public:
void init(size_t n = 0) {
resize(sizeof(Header) + n + sizeof(Footer));
Header * const h = header();
h->init(n);
Footer * const f = footer();
f->init();
}
// these two are valid only after init()
Header *header() { return reinterpret_cast<Header*>(&(*this)[0]); }
Footer *footer() { return reinterpret_cast<Footer*>(&(*this)[size() - sizeof(Footer)]); }
template <typename T>
T *buffer() { return reinterpret_cast<T *>(&(*this)[sizeof(Header)]); }
void extend(size_t n) { resize(size() + n); }
};
int sendMessage(int funcode) {
Packet p;
switch (funcode) {
case TIME: {
timeval tv;
gettimeofday(&tv, NULL);
p.init(sizeof tv);
Packet::Header * const h = p.header();
h->funcode = funcode;
timeval * const dst = p.buffer<timeval>();
*dst = tv;
} break;
case PID: {
const pid_t pid = getpid();
p.init(sizeof pid);
Packet::Header * const h = p.header();
h->funcode = funcode;
pid_t * const dst = p.buffer<pid_t>();
*dst = pid;
} break;
...
}
Packet::Footer * const f = p.footer();
f->crc = ...;
const ssize_t nSent = send(sock, p.data(), p.size(), 0);
...
}
Packet receiveMessage() {
Packet ret;
ret.init();
Packet::Header * const h = ret.header();
ssize_t nRecv = recv(sock, h->buffer(), sizeof *h, 0);
...
if ( ! h->verify()) ...
p.extend(h->length);
nRecv = recv(sock, p.buffer<char>(), h->length, 0);
switch (h->funcode) {
case TIME: {
timeval tv = *p->buffer<timeval>();
tm theTm;
localtime_r(&tv.tv_sec, &theTm);
char buf[128];
strftime(buf, sizeof buf, "%x %X", &tv);
cout << "Server time == " << buf << endl;
} break;
case PID: {
const pid_t pid = *p.buffer<pid_t>();
cout << "Server PID == " << pid << endl;
} break;
...
}
Packet::Footer * const f = ret.footer();
nRecv = recv(sock, f->buffer(), sizeof *f, 0);
if ( ! f->verify() || f->crc != ...) ...
return ret; // inefficient data copy, only for a sample code
}
I recommend you to define your BEGIN code as something like 0x48454c4f which can be read as ASCII "HELO" rather than a random int like 0x41. And I added two extra bytes ';' and '\n'to the footer; you will find them useful when you dump packet capture of your protocol to the console. Remember, you can always use sed, perl, python or whatever (but not grep; it won't accept arbitrary hex search strings) to analyse arbitrary binary data including your packet dump. If you design your protocol well, your debugging will be much easier. (First of all, you should consider using ASCII protocol rather than binary one.)
Frequently a protocol definition goes like this:
FUNCTIONCODE
LENGTH
DATA
CRC
and there's no need to fish for "BEGIN". Reading proceeds by reading FUNCTIONCODE and LENGTH using a fixed length, and then the data + CRC, according to LENGTH, and then, again...
However, if you think that the additional "BEGIN" marker helps you, select "BEGIN" as a value that's not a FUNCTIONCODE, and then you have
BEGIN
FUNCTIONCODE
LENGTH
DATA
CRC
And for every byte after BEGIN equal to BEGIN send BEGIN twice. But, in this case, reading is much more complicated...
I am currently doing some testing with a new addition to the ICU dictionary-based break iterator.
I have code that allows me to test the word-breaking on a text document but when the text document is too large it gives the error: bash: ./a.out: Argument list too long
I am not sure how to edit the code to break-up the argument list when it gets too long so that a file of any size can be run through the code. The original code author is quite busy, would someone be willing to help out?
I tried removing the printing of what is being examined to see if that would help, but I still get the error on large files (printing what is being examined isn't necessary - I just need the result).
If the code could be modified to read the source text file line by line and export the results line by line to another text file (ending up with all the lines when it is done), that would be perfect.
The code is as follows:
/*
Written by George Rhoten to test how word segmentation works.
Code inspired by the break ICU sample.
Here is an example to run this code under Cygwin.
PATH=$PATH:icu-test/source/lib ./a.exe "`cat input.txt`" > output.txt
Encode input.txt as UTF-8.
The output text is UTF-8.
*/
#include <stdio.h>
#include <unicode/brkiter.h>
#include <unicode/ucnv.h>
#define ZW_SPACE "\xE2\x80\x8B"
void printUnicodeString(const UnicodeString &s) {
int32_t len = s.length() * U8_MAX_LENGTH + 1;
char *charBuf = new char[len];
len = s.extract(0, s.length(), charBuf, len, NULL);
charBuf[len] = 0;
printf("%s", charBuf);
delete charBuf;
}
/* Creating and using text boundaries */
int main(int argc, char **argv)
{
ucnv_setDefaultName("UTF-8");
UnicodeString stringToExamine("Aaa bbb ccc. Ddd eee fff.");
printf("Examining: ");
if (argc > 1) {
// Override the default charset.
stringToExamine = UnicodeString(argv[1]);
if (stringToExamine.charAt(0) == 0xFEFF) {
// Remove the BOM
stringToExamine = UnicodeString(stringToExamine, 1);
}
}
printUnicodeString(stringToExamine);
puts("");
//print each sentence in forward and reverse order
UErrorCode status = U_ZERO_ERROR;
BreakIterator* boundary = BreakIterator::createWordInstance(NULL, status);
if (U_FAILURE(status)) {
printf("Failed to create sentence break iterator. status = %s",
u_errorName(status));
exit(1);
}
printf("Result: ");
//print each word in order
boundary->setText(stringToExamine);
int32_t start = boundary->first();
int32_t end = boundary->next();
while (end != BreakIterator::DONE) {
if (start != 0) {
printf(ZW_SPACE);
}
printUnicodeString(UnicodeString(stringToExamine, start, end-start));
start = end;
end = boundary->next();
}
delete boundary;
return 0;
}
Thanks so much!
-Nathan
The Argument list too long error message is coming from the bash shell and is happening before your code even gets started executing.
The only code you can fix to eliminate this problem is the bash source code (or maybe it is in the kernel) and then, you're always going to run into a limit. If you increase from 2048 files on command line to 10,000, then some day you'll need to process 10,001 files ;-)
There are numerous solutions to managing 'too big' argument lists.
The standardized solution is the xargs utility.
find / -print | xargs echo
is a un-helpful, but working example.
See How to use "xargs" properly when argument list is too long for more info.
Even xargs has problems, because file names can contain spaces, new-line chars, and other unfriendly stuff.
I hope this helps.
The code below reads the content of a file whos name is given as the first parameter on the command-line and places it in a str::buffer. Then, instead of calling the function UnicodeString with argv[1], use that buffer instead.
#include<iostream>
#include<fstream>
using namespace std;
int main(int argc, char **argv)
{
std::string buffer;
if(argc > 1) {
std::ifstream t;
t.open(argv[1]);
std::string line;
while(t){
std::getline(t, line);
buffer += line + '\n';
}
}
cout << buffer;
return 0;
}
Update:
Input to UnicodeString should be char*. The function GetFileIntoCharPointer does that.
Note that only the most rudimentary error checking is implemented below!
#include<iostream>
#include<fstream>
using namespace std;
char * GetFileIntoCharPointer(char *pFile, long &lRet)
{
FILE * fp = fopen(pFile,"rb");
if (fp == NULL) return 0;
fseek(fp, 0, SEEK_END);
long size = ftell(fp);
fseek(fp, 0, SEEK_SET);
char *pData = new char[size + 1];
lRet = fread(pData, sizeof(char), size, fp);
fclose(fp);
return pData;
}
int main(int argc, char **argv)
{
long Len;
char * Data = GetFileIntoCharPointer(argv[1], Len);
std::cout << Data << std::endl;
if (Data != NULL)
delete [] Data;
return 0;
}
I have written a read function which reads values from serial port(LINUX) . It returns values as pointer to char . I am calling this function in another function and storing it again in a variable as pointer to char . I occasionally got stack over flow problem and not sure if this function is creating problem.
The sample is provided below. Please give some suggestions or criticism .
char *ReadToSerialPort( )
{
const int buffer_size = 1024;
char *buffer = (char *)malloc(buffer_size);
char *bufptr = buffer;
size_t iIn;
int iMax = buffer+buffer_size-bufptr;
if ( fd < 1 )
{
printf( "port is not open\n" );
// return -1;
}
iIn = read( fd, bufptr, iMax-1 );
if ( iIn < 0 )
{
if ( errno == EAGAIN )
{
printf( "The errror in READ" );
return 0; // assume that command generated no response
}
else
printf( "read error %d %s\n", errno, strerror(errno) );
}
else
{
// *bufptr = '\0';
bufptr[(int)iIn<iMax?iIn:iMax] = '\0';
if(bufptr != buffer)
return bufptr;
}
free(buffer);
return 0;
} // end ReadAdrPort
int ParseFunction(void)
{
// some other code
char *sResult;
if( ( sResult = ReadToSerialPort()) >= 0)
{
printf("Response is %s\n", sResult);
// code to store char in string and put into db .
}
}
Thanks and regards,
SamPrat
You do not deallocate the buffer. You need to make free after you finished working with it.
char * getData()
{
char *buf = (char *)malloc(255);
// Fill buffer
return buf;
}
void anotherFunc()
{
char *data = getData();
// Process data
free(data);
}
In your case I think you should free the buffer after printf:
if( ( sResult = ReadToSerialPort()) >= 0)
{
printf("Response is %s\n", sResult);
// code to store char in string and put into db .
free(sResult);
}
UPDATE Static buffer
Another option to use static buffers. It could increase performance a little bit, but getData method will be not a thread-safe.
char buff[1024];
char *getData()
{
// Write data to buff
return buff;
}
int main()
{
char *data = getData();
printf("%s", data);
}
UPDATE Some notes about your code
int iMax = buffer+buffer_size-bufptr; - iMax will always be 1024;
I do not see any idea of using bufptr since its value is the same as buffer and you do not change it anywhere in your function;
iIn = read( fd, bufptr, buffer_size-1 );
You can replace bufptr[(int)iIn<iMax?iIn:iMax] = '\0'; with bufptr[iIn] = '\0';
if(bufptr != buffer) is always false and this is why your pointer is incorrect and you always return 0;
Do not forget to free the buffer if errno == EAGAIN is true. Currently you just return 0 without free(buffer).
Good luck ;)
Elalfer is partially correct. You do free() your buffer, but not in every case.
For example, when you reach if ( errno == EAGAIN ) and it evaluates to true, you return without doing free on your buffer.
The best would be to pass the buffer as a parameter and make it obvious that the user must free the buffer, outside the function. (this is what basically Elalfer sais in his edited answer).
Just realized this is a C question, I blame SO filtering for this :D sorry! Disregard the following, I'm leaving it so that comments still make sense.
The correct solution should use std::vector<char>, that way the destructor handles memory deallocation for you at the end of scope.
what is the purpose of the second pointer?
char *buffer = (char *)malloc(buffer_size);
char *bufptr = buffer;
what is the purpose of this?
int iMax = buffer+buffer_size-bufptr; // eh?
What is the purpose of this?
bufptr[(int)iIn<iMax?iIn:iMax] = '\0'; // so you pass in 1023 (iMax - 1), it reads 1023, you've effectively corrupted the last byte.
I would start over, consider using std::vector<char>, something like:
std::vector<char> buffer(1500); // default constructs 1500 chars
int iRead = read(fd, &buffer[0], 1500);
// resize the buffer if valid
if (iRead > 0)
buffer.resize(iRead); // this logically trims the buffer so that the iterators begin/end are correct.
return buffer;
Then in your calling function, use the vector<char> and if you need a string, construct one from this: std::string foo(vect.begin(), vect.end()); etc.
When you are setting the null terminator "bufptr[(int)iIn
bufptr[iMax]=>bufptr[1024]=>one byte beyond your allocation since arrays start at 0.
Also int this case "int iMax = buffer+buffer_size-bufptr;" can be re-written as iMax = buffer_size. It makes the code less readable.