Error while trying to send a message with FIFO - c++

I'm new to this kind of programming so i'm sorry in advance if it's a dump question.I'm trying to do a very simple task but i don't seem to get what goes wrong.
I have a parent process that creates a number of children processes, and with the usage of FIFOs i want to send a message to all of the children (e.g "hi"), the message get's received by the processes but there is an error coming up no matter what i do and can't seem to find what's wrong.
Here is the parent's main function:
int main(int argc, char *argv[])
{
int num_monitors, buf_size; //command line arguments
num_monitors = stoi(argv[1]);
buf_size = stoi(argv[2]);
// Structures to store monitor info
Monitors *m_info = new Monitors[num_monitors]; // Stores monitor pid & pipe ends
create_n_monitors(m_info,num_monitors,buf_size,input_dir_path);
sleep(1); // making sure all pipes get created
for(int i=0; i<num_monitors; i++) // opening the write end now that the monitors have been created
{
m_info[i].write_fd = open(m_info[i].write_p, O_WRONLY | O_NONBLOCK);
if (m_info[i].write_fd == -1){perror("open # 27 main parent");exit(1);}
}
for(int i=0; i<num_monitors; i++)
send_message(m_info[i].write_fd, (char*)"hi", buf_size);
delete [] m_info;
return 0;
}
Here's the class where i keep every process information stored:
class Monitors
{
public:
pid_t m_pid; // monitors's PID
char read_p[32]; // write monitor - read parent pipe name
char write_p[32]; // read monitor - write parent pipe name
int read_fd; // file descriptor for read fifo of monitor
int write_fd; // file descriptor for write fifo of monitor
Monitors();
~Monitors();
};
Here's how i create the processes and the pipes(FIFOs):
void create_n_monitors(Monitors *m_info, int num_monitors, int buf_size, char *input_dir)
{
create_unique_fifo(true, NULL, 0); // creates a fifo file
for (int i = 0; i < num_monitors; ++i) // create num monitors
create_monitor(m_info, i, buf_size, input_dir);
}
/* ========================================================================= */
// Create a monitor and it's named fifos. Store it's info in <m_info[index]>.
void create_monitor(Monitors *m_info, int index, int buf_size, char *input_dir)
{
create_unique_fifo(false, m_info, index); // Create fifos
pid_t pid = fork();
if(pid == -1) {
perror("fork");
exit(1);
}
else if ( pid == 0) { // we are in the child monitor, read_p(read parent) : monitor's write end of the fifo
// write_p(write parent): monitor's read end
char buf_size_str[15];
sprintf(buf_size_str, "%d", buf_size); // buf_size must be a char*
execl("./Monitor","Monitor", buf_size_str, m_info[index].read_p, m_info[index].write_p, (char * )NULL);
perror("execl");
exit(1);
}
//else
m_info[index].m_pid = pid; // Store it's pid
}
/* ========================================================================= */
// If <setup> is true, create a directory to store fifos.
void create_unique_fifo(bool setup, Monitors *m_info, int index)
{
static char fifo_name[32];
static int counter = 0;
if (setup == true)
{
char dir_path[] = "named_fifos";
if (access(dir_path, F_OK) == 0) // If dir already exists (due to abnormal previous termination, eg: SIGKILL)
delete_flat_dir(dir_path); // completely remove it
if (mkdir(dir_path, 0777) == -1){perror("mkdir # unique_fifo");exit(1);}
sprintf(fifo_name, "named_fifos/f");
return;
}
struct stat stat_temp;
// Create a unique name (e.g named_fifos1R , named_fifos6W )
sprintf(m_info[index].read_p, "%s%d%c", fifo_name, counter, 'R');
// Create fifos
if(stat(m_info[index].read_p, &stat_temp) == -1){
if (mkfifo(m_info[index].read_p,0666) < 0 ){perror("mkfifo # unique_fifo");exit(1);}
}
m_info[index].read_fd = open(m_info[index].read_p, O_RDONLY | O_NONBLOCK);
if (m_info[index].read_fd == -1) {perror("open # 73 setup_monitors");exit(1);}
sprintf(m_info[index].write_p, "%s%d%c", fifo_name, counter, 'W');
++counter; // counter used for pipe names
}
/* ========================================================================= */
// Remove a flat directory and its contents.
void delete_flat_dir(char *init_flat_path)
{
char flat_path[32];
strcpy(flat_path, init_flat_path);
DIR *dir = opendir(flat_path);
if (dir == NULL){perror("opendir # delete_flat_dir"); exit(1);}
struct dirent *entry;
while ((entry = readdir(dir)) != NULL) // Delete contents/files
{
char *f_name = entry->d_name;
if (!strcmp(f_name, ".") || !strcmp(f_name, ".."))
continue;
char f_path[32];
snprintf(f_path, 32, "%s/%s", flat_path, f_name); // Remove file
if (remove(f_path) == -1){perror("remove # delete_flat_dir"); exit(1);}
}
// Remove dir
if (closedir(dir) == -1){perror("closedir 2 # delete_flat_dir"); exit(1);}
if (rmdir(flat_path) == -1){perror("rmdir # delete_flat_dir"); exit(1);}
}
Here are the functions used for the communication of the processes and parent:
// Sends <message> to file descriptor <fd>
void send_message(int fd, char *message, int buf_size)
{
int length = strlen(message);
char buffer[10];
sprintf(buffer, "%d#", length);
write(fd, buffer, 9); // sending the number of bytes reader is about to read
write(fd, message, length); // sending the message itself
}
char *read_message(int read_end_fd, int buf_size)
{
char buffer[10];
int fifo_buffer_size = buf_size;
read(read_end_fd, buffer, 9);
char * tok = strtok(buffer, "#");
int length = atoi(tok); // how many characters will be received
char * input_read = new char[length + 1];
char * str = input_read;
int bytes_read = 0, total_bytes = 0; // We might need to read less or more bytes
fifo_buffer_size = length < fifo_buffer_size ? length : fifo_buffer_size; // // than <buf_size>
while(total_bytes < length)
{
str += bytes_read; // move str pointer
bytes_read = read(read_end_fd, str, fifo_buffer_size); //and read the next <buf_size> characters
total_bytes += bytes_read; // adding them to the total amount of bytes read altogether
if((total_bytes + fifo_buffer_size) > length)
fifo_buffer_size = length - total_bytes; // reading exactly the amount that's left
}
input_read[length] = '\0';
return input_read;
}
And lastly the processes main function:
int create_unique_fifo(char* read_fifo)
{
int read_fd;
struct stat stat_temp;
// Create fifos
if(stat(read_fifo, &stat_temp) == -1){
if (mkfifo(read_fifo,0666) < 0 ){perror("mkfifo # unique_fifo 37");exit(1);}
}
read_fd = open(read_fifo, O_RDONLY); // Open named pipe for reading
if (read_fd == -1){perror("open # monitor.cpp 1"); exit(1);}
return read_fd;
}
int main(int argc, char* argv[])
{
int buf_size = stoi(argv[1]); // Process command line args
char read_fifo[100], write_fifo[100];
strcpy(write_fifo, argv[2]); // parent read - monitor write
strcpy(read_fifo, argv[3]); // parent write - monitor read
int read_fd, write_fd;
read_fd = create_unique_fifo(read_fifo);
write_fd = open(write_fifo, O_WRONLY | O_NONBLOCK); // Open named pipe for writing
if (write_fd == -1){perror("open # monitor.cpp 2"); exit(1);}
char* message;
message = read_message(read_fd, buf_size);
cout << "2:" << message << endl;
return 0;
}
I'm using the valgrind debugger and get the following result when num_monitors=5 and buf_size=2:
==29783== Syscall param write(buf) points to uninitialised byte(s)
==29783== at 0x4B691E7: write (write.c:26)
==29783== by 0x10B85B: send_message(int, char*, int) (in /home/sofia/Desktop/syspro-2/travelMonitor)
==29783== by 0x10A80F: main (in /home/sofia/Desktop/syspro-2/travelMonitor)
==29783== Address 0x1ffefffc81 is on thread 1's stack
==29783== in frame #1, created by send_message(int, char*, int) (???:)
==29783==
2:hi
2:hi
2:hi
2:hi
2:hi
==29783==
==29783== HEAP SUMMARY:
==29783== in use at exit: 0 bytes in 0 blocks
==29783== total heap usage: 4 allocs, 4 frees, 138,864 bytes allocated
==29783==
==29783== All heap blocks were freed -- no leaks are possible
==29783==
==29783== Use --track-origins=yes to see where uninitialised values come from
==29783== For lists of detected and suppressed errors, rerun with: -s
==29783== ERROR SUMMARY: 5 errors from 1 contexts (suppressed: 0 from 0)
Any idea why this could be happening? I added sleep(1) after the creation of the processes to make sure all the fifos get created in time but still get this error... Any help would be much appreciated

In send_message, you always write 9 bytes from buffer, but not all of those bytes have had a value written to them. This is because the sprintf that populates buffer only writes to the first few locations, possibly as few as 3.
The solution is to either initialize them all to 0
char buffer[10] = 0;
or only write the number of bytes that are in the string. This is easily known from the value returned by sprintf.
auto buf_len = sprintf(buffer, "%d#", length);
write(fd, buffer, buf_len);

Related

Why is msgrcv() feeding garbage characters into the buffer?

right now, I am currently trying to output the contents of buf.mtext so I can make sure take the correct input before moving on with my program. Everything seems to work fine, except one thing; msgrcv() puts garbage characters into the buffer, and the reciever process outputs garbage characters.
Here is my sender process:
int main (void)
{
int i; // loop counter
int status_01; // result status
int msqid_01; // message queue ID (#1)
key_t msgkey_01; // message-queue key (#1)
unsigned int rand_num;
float temp_rand;
unsigned char eight_bit_num;
unsigned char counter = 0;
unsigned char even_counter = 0;
unsigned char odd_counter = 0;
srand(time(0));
struct message {
long mtype;
char mtext[BUFFER_SIZE];
} buf_01;
msgkey_01 = MSG_key_01; // defined at top of file
msqid_01 = msgget(msgkey_01, 0666 | IPC_CREAT)
if ((msqid_01 <= -1) { exit(1); }
/* wait for a key stroke at the keyboard ---- */
eight_bit_num = getchar();
buf_01.mtype = 1;
/* send one eight-bit number, one at a time ------------ */
for (i = 0; i < NUM_REPEATS; i++)
{
temp_rand = ((float)rand()/(float)RAND_MAX)*255.0;
rand_num = (int)temp_rand;
eight_bit_num = (unsigned char)rand_num;
if ((eight_bit_num % 2) == 0)
{
printf("Even number: %d\n", eight_bit_num);
even_counter = even_counter + eight_bit_num;
}
else
{
printf("Odd number: %d\n", eight_bit_num);
odd_counter = odd_counter + eight_bit_num;
}
/* update the counters ------------------------------ */
counter = counter + eight_bit_num;
if((eight_bit_num % 2) == 0) { even_counter = even_counter + eight_bit_num; }
else { odd_counter = odd_counter + eight_bit_num; }
buf_01.mtext[0] = eight_bit_num; // copy the 8-bit number
buf_01.mtext[1] = '\0'; // null-terminate it
status_01 = msgsnd(msqid_01, (struct msgbuf *)&buf_01, sizeof(buf_01.mtext), 0);
status_01 = msgctl(msqid_01, IPC_RMID, NULL);
}
Here is my receiver process:
int main() {
struct message {
long mtype;
char mtext[BUFFER_SIZE];
} buf;
int msqid;
key_t msgkey;
msgkey = MSG_key_01;
msqid = msgget(msgkey, 0666); // connect to message queue
if (msqid < 0) {
printf("Failed\n");
exit(1);
}
else {
printf("Connected\n");
}
if (msgrcv(msqid, &buf, BUFFER_SIZE, 0, 0) < 0) { // read message into buf
perror("msgrcv");
exit(1);
}
printf("Data received is: %s \n", buf.mtext);
printf("Done receiving messages.\n");
return 0;
}
The output is usually something like as follows:
Data received is: ▒
Done receiving messages.
I have made sure to clear my message queues each time after running the sender and receiver processes, as well, since I have come to find out this can cause issues. Thanks in advance for your help.
Turns out neither of the suggested solutions were the issue, as I suspected; the sender process actually works just fine. The problem was that I was trying to print buf.mtext instead of buf.mtext[0] which isn't an actual integer value. I fixed the issue by just doing this:
int temp_num = buf.mtext[0];
printf("Data recieved is %d \n", temp_num);

C++: segmentation fault in memset () when returning from procedure

Could anyone help me with my C++ program? I'm experiencing a segmentation fault and I can't find the problem. I'm writing a program for my raspberry pi that communicates with an nRF24L01 sensor network and sends data to a dashboard (Dashing) hosted on my RPi using CURL.
I ran my program with a debugger (gdb), and this is the backtrace I got: (see below for the full source code)
Program received signal SIGSEGV, Segmentation fault.
0xb6fbe0dc in memset () from /usr/lib/arm-linux-gnueabihf/libarmmem.so
(gdb) bt
#0 0xb6fbe0dc in memset () from /usr/lib/arm-linux-gnueabihf/libarmmem.so
#1 0x000119d4 in handleSensorMsg () at myNetworkMaster.cpp:174
#2 0x00000000 in ?? ()
I also checked my program with valgrind, using --leak-check=full. The output:
==20790== Invalid read of size 4
==20790== at 0x484F808: bcm2835_peri_read (in /usr/local/lib/librf24.so.1.1.7)
==20790== Address 0xffffffff is not stack'd, malloc'd or (recently) free'd
==20790==
==20790==
==20790== Process terminating with default action of signal 11 (SIGSEGV)
==20790== Access not within mapped region at address 0xFFFFFFFF
==20790== at 0x484F808: bcm2835_peri_read (in /usr/local/lib/librf24.so.1.1.7)
==20790== If you believe this happened as a result of a stack
==20790== overflow in your program's main thread (unlikely but
==20790== possible), you can try to increase the size of the
==20790== main thread stack using the --main-stacksize= flag.
==20790== The main thread stack size used in this run was 8388608.
==20790==
==20790== HEAP SUMMARY:
==20790== in use at exit: 75,908 bytes in 643 blocks
==20790== total heap usage: 1,232 allocs, 589 frees, 84,668 bytes allocated
==20790==
==20790== LEAK SUMMARY:
==20790== definitely lost: 0 bytes in 0 blocks
==20790== indirectly lost: 0 bytes in 0 blocks
==20790== possibly lost: 0 bytes in 0 blocks
==20790== still reachable: 75,908 bytes in 643 blocks
==20790== suppressed: 0 bytes in 0 blocks
==20790== Reachable blocks (those to which a pointer was found) are not shown.
==20790== To see them, rerun with: --leak-check=full --show-reachable=yes
==20790==
==20790== For counts of detected and suppressed errors, rerun with: -v
==20790== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
The source code: the crash occurs in handleSensorMsg(), after the first call to sendDataToDashBoard(). What happens is that the program starts and listens for messages on the nRF24L01. The first one it will receive is a sensor message. It reads the contents and sends the temperature data to the dashboard. After that it will crash. As you can see I'm not used to programming in C(++), my experience is limited to a few simple arduino programs.
/** use Raspberry pi as master node for RF24Network, based on example code from TMRh20
*
* This example sketch shows how to manually configure a node via RF24Network as a master node, which
* will receive all data from sensor nodes.
*
* send received data to Dashing dashboard using curl
*
* listen for messages on the program queue to send data to a node
*
*/
#include <RF24/RF24.h>
#include <RF24Network/RF24Network.h>
#include <curl/curl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <errno.h>
#include <mqueue.h>
#include <time.h>
#include "common.h"
/**
* g++ -L/usr/lib main.cc -I/usr/include -o main -lrrd
**/
//using namespace std;
// CE Pin, CSN Pin, SPI Speed
// Setup for GPIO 22 CE and GPIO 25 CSN with SPI Speed # 1Mhz
//RF24 radio(RPI_V2_GPIO_P1_22, RPI_V2_GPIO_P1_18, BCM2835_SPI_SPEED_1MHZ);
// Setup for GPIO 22 CE and CE0 CSN with SPI Speed # 4Mhz
//RF24 radio(RPI_V2_GPIO_P1_15, BCM2835_SPI_CS0, BCM2835_SPI_SPEED_4MHZ);
// Setup for GPIO 22 CE and CE1 CSN with SPI Speed # 8Mhz
RF24 radio(RPI_V2_GPIO_P1_15, BCM2835_SPI_CS0, BCM2835_SPI_SPEED_8MHZ);
RF24Network network(radio);
CURL *curl;
//CURLcode res;
mqd_t mq;
const char* AUTH_TOKEN = "MODEL_M";
const char* DASHBOARD_IP = "http://192.168.0.21:80/";
const int CURL_BUFFER_SIZE = 100;
class timer {
private:
time_t begTime;
public:
bool isRunning = false;
void start() {
begTime = time(NULL);
isRunning = true;
}
int elapsedTime() {
return (time(NULL) - begTime);
}
bool isTimeout(int seconds) {
int elapsed = elapsedTime();
//printf("%f\n", elapsed);
return elapsed >= seconds;
}
};
const int AVG_TEMP_SAMPLESIZE = 60;
const float TEMP_HIVAL = 999;
const float TEMP_LOVAL = -273.15;
const int DAILY_TEMP_RST_HOUR = 0;
float avgTemperature = TEMP_HIVAL;
float minTemperature = TEMP_HIVAL;
float maxTemperature = TEMP_LOVAL;
float minDailyTemp = TEMP_HIVAL;
float maxDailyTemp = TEMP_LOVAL;
// TODO: create 'message queue'?
const int SEND_RELAY_INTERVAL = 1;
timer relayMsgTimer;
const int SEND_DASH_WAIT = 1;
timer relayDashTimer;
const int SEND_QUERY_INTERVAL = 10;
timer queryMsgTimer;
bool timedRelayMode = true;
int timedRelayState = 1;
const int TIMED_RELAY_ON_HOUR = 6;
const int TIMED_RELAY_OFF_HOUR = 0;
const int TIMED_RELAY_RST_HOUR = 12;
void setCurrentTimestamp(char* timestamp) {
time_t rawTime = time(NULL);
struct tm *tm = localtime(&rawTime);
strftime(timestamp, 20, "%Y-%m-%d %H:%M:%S", tm);
}
void sendDataToDashBoard(const char* const widget, const char* const data) {
char url[CURL_BUFFER_SIZE];
char postFields[CURL_BUFFER_SIZE];
std::fill(url, url + CURL_BUFFER_SIZE, 0);
std::fill(postFields, postFields + CURL_BUFFER_SIZE, 0);
sprintf(url, "%swidgets/%s", DASHBOARD_IP, widget);
curl_easy_setopt(curl, CURLOPT_URL, url);
sprintf(postFields, "{\"auth_token\":\"%s\",%s", AUTH_TOKEN, data);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, postFields);
printf("%s%s\n", url, postFields);
CURLcode res = curl_easy_perform(curl);
if (res != CURLE_OK)
printf("curl_easy_perform failed: %s\n", curl_easy_strerror(res));
curl_easy_reset(curl);
} // <-- this is where the debugger stops !!!
void handleSensorMsg() {
RF24NetworkHeader header = 0;
network.read(header,&sensorData,sizeof(sensorData));
char timestamp[20];
std::fill(timestamp, timestamp + 20, 0);
setCurrentTimestamp(timestamp);
printf("%s - %c: Rcv %d %u from 0%o\n", timestamp, SENSOR_MSG, sensorData.temperature, sensorData.humidity, header.from_node);
// ignore probable invalids -- TODO: what if first reading is invalid? => avgTemperature
if (header.from_node != SENSOR_NODE ||
(avgTemperature != TEMP_HIVAL &&
(sensorData.temperature > avgTemperature * 20.0 ||
sensorData.temperature < avgTemperature * 5.0))) {
printf("ignored invalid data\n");
return;
}
float temperature = sensorData.temperature / 10.0;
int humidity = sensorData.humidity / 10.0 + 0.5;
if (avgTemperature == TEMP_HIVAL)
avgTemperature = temperature;
else {
avgTemperature -= avgTemperature / AVG_TEMP_SAMPLESIZE;
avgTemperature += temperature / AVG_TEMP_SAMPLESIZE;
}
if (temperature > maxDailyTemp) {
maxDailyTemp = temperature;
if (temperature > maxTemperature)
maxTemperature = temperature;
} else if (temperature < minDailyTemp) {
minDailyTemp = temperature;
if (temperature < minTemperature)
minTemperature = temperature;
}
char data[CURL_BUFFER_SIZE];
std::fill(data, data + CURL_BUFFER_SIZE, 0);
sprintf(data, "\"current\":%.1f,\"average\":%f,\"moreinfo\":\"Min : %.1f (%.1f) - Max : %.1f (%.1f)\"}",
temperature, avgTemperature, minDailyTemp, minTemperature, maxDailyTemp, maxTemperature);
sendDataToDashBoard("temperature", data); // This is the call that crashes!!!
std::fill(data, data + CURL_BUFFER_SIZE, 0);
sprintf(data, "\"value\":%d}", humidity);
sendDataToDashBoard("humidity", data);
}
void handleRelayMsg() {
RF24NetworkHeader header = 0;
network.read(header, &relayData, sizeof(relayData));
char timestamp[20];
std::fill(timestamp, timestamp + 20, ' ');
setCurrentTimestamp(timestamp);
printf("%s - %c: Rcv %d from 0%o\n", timestamp, RELAY_MSG, relayData.state, header.from_node);
// handle invalid reading
if (header.from_node != RELAY_NODE ||
(relayData.state != true && relayData.state != false)) {
printf("ignored invalid data\n");
queryMsgTimer.start();
return;
}
// delay send of relayData to dashboard to avoid race condition
relayDashTimer.start();
relayMsgTimer.isRunning = false;
queryMsgTimer.isRunning = false;
}
bool sendRelayData() {
RF24NetworkHeader header(/*to node*/ RELAY_NODE, RELAY_MSG);
char timestamp[20];
std::fill(timestamp, timestamp + 20, ' ');
setCurrentTimestamp(timestamp);
printf("%s - %c: Snd %d to %o\n", timestamp, RELAY_MSG, relayData.state, RELAY_NODE);
if (!network.write(header, &relayData, sizeof(relayData))) {
relayMsgTimer.start();
printf("Send failed...\n");
return false;
}
printf("Send OK\n");
return true;
}
void sendQueryMsgToRelay() {
RF24NetworkHeader header(/*to node*/ RELAY_NODE, QUERY_MSG);
char timestamp[20];
std::fill(timestamp, timestamp + 20, ' ');
setCurrentTimestamp(timestamp);
printf("%s - %c: Snd to %o\n", timestamp, QUERY_MSG, RELAY_NODE);
if (!network.write(header, NULL, 0)) {
queryMsgTimer.start();
printf("Send failed...\n");
return;
}
printf("Send OK\n");
queryMsgTimer.isRunning = false;
}
bool readMessageQueue() {
ssize_t bytes_read;
char buffer[MAX_SIZE + 1];
std::fill(buffer, buffer + MAX_SIZE + 1, ' ');
/* receive the message */
bytes_read = mq_receive(mq, buffer, MAX_SIZE, NULL);
if (bytes_read >= 0) {
buffer[bytes_read] = '\0';
char timestamp[20];
std::fill(timestamp, timestamp + 20, ' ');
setCurrentTimestamp(timestamp);
printf("%s - Received: %s\n", timestamp, buffer);
if (!strncmp(buffer, MSG_ON, strlen(MSG_ON))) {
if (relayMsgTimer.isRunning)
return false;
relayData.state = true;
queryMsgTimer.isRunning = false;
sendRelayData();
} else if (!strncmp(buffer, MSG_OFF, strlen(MSG_OFF))) {
if (relayMsgTimer.isRunning)
return false;
relayData.state = false;
queryMsgTimer.isRunning = false;
sendRelayData();
} else if (!strncmp(buffer, MSG_SET, strlen(MSG_SET))) {
if (relayMsgTimer.isRunning)
return false;
relayData.state = !relayData.state;
queryMsgTimer.isRunning = false;
sendRelayData();
} else if (!strncmp(buffer, MSG_GET, strlen(MSG_GET))) {
if (queryMsgTimer.isRunning)
return false;
sendQueryMsgToRelay();
} else if (!strncmp(buffer, MSG_TIMER, strlen(MSG_TIMER))) {
timedRelayMode = !timedRelayMode;
timedRelayState = 1;
printf("timedRelayMode=%d\n", timedRelayMode);
} else if (!strncmp(buffer, MSG_STOP, strlen(MSG_STOP)))
return true;
}
return false;
}
void sendRelayDataToDashboard() {
char data[CURL_BUFFER_SIZE];
std::fill(data, data + CURL_BUFFER_SIZE, 0);
sprintf(data, "\"status\":\"%s\"}", relayData.state ? "ON" : "OFF");
sendDataToDashBoard("relay", data);
relayDashTimer.isRunning = false;
}
int getCurrentHour() {
time_t rawTime = time(NULL);
struct tm *tm = localtime(&rawTime);
return tm->tm_hour;
}
void setTimedRelayState() {
char timestamp[20];
std::fill(timestamp, timestamp + 20, ' ');
setCurrentTimestamp(timestamp);
printf("%s - Timed relay event:\n", timestamp);
relayData.state = !relayData.state;
sendRelayData();
}
void updateTimedRelay(int hour) {
if (timedRelayState == 1 && relayData.state && hour == TIMED_RELAY_OFF_HOUR) {
setTimedRelayState();
timedRelayState = 2;
} else if (timedRelayState == 2 && !relayData.state && hour == TIMED_RELAY_ON_HOUR) {
setTimedRelayState();
timedRelayState = 1;
} else if (timedRelayState == 2 && hour == TIMED_RELAY_RST_HOUR)
timedRelayState = 1;
}
void updateDailyTemperature(int hour) {
if (hour == DAILY_TEMP_RST_HOUR) {
char timestamp[20];
std::fill(timestamp, timestamp + 20, ' ');
setCurrentTimestamp(timestamp);
minDailyTemp = avgTemperature;
maxDailyTemp = avgTemperature;
printf("%s - daily temperatures reset\n", timestamp);
}
}
int main(int argc, char** argv)
{
// Refer to RF24.h or nRF24L01 DS for settings
radio.begin();
delay(5);
network.begin(CHANNEL, MASTER_NODE);
radio.setDataRate(RF24_250KBPS);
radio.setCRCLength(RF24_CRC_8);
// increase delay for network ACK
network.routeTimeout = 300;
radio.printDetails();
// CURL setup for sending data to dashboard
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl == NULL) {
printf("Error getting curl handle.\n");
exit(EXIT_FAILURE);
}
// setup the message queue
struct mq_attr attr;
/* initialize the queue attributes */
attr.mq_flags = 0;
attr.mq_maxmsg = 10;
attr.mq_msgsize = MAX_SIZE;
attr.mq_curmsgs = 0;
/* create the message queue */
mq = mq_open(QUEUE_NAME, O_CREAT | O_RDONLY | O_NONBLOCK, 0644, &attr);
if (mq == -1) {
printf("Failed to create queue.\n");
exit(EXIT_FAILURE);
}
queryMsgTimer.start();
int currentHour = getCurrentHour();
bool exitReceived = false;
while(!exitReceived)
{
network.update();
while ( network.available() ) { // Is there anything ready for us?
RF24NetworkHeader header; // If so, grab it and print it out
network.peek(header);
switch(header.type) {
case SENSOR_MSG: handleSensorMsg();
break;
case RELAY_MSG: handleRelayMsg();
break;
default: network.read(header,0,0);
char timestamp[20];
setCurrentTimestamp(timestamp);
printf("%s - Rcv bad type %d from 0%o\n", timestamp, header.type, header.from_node);
printf("%s\n", header.toString());
break;
}
}
// check for incoming messages on the program queue
exitReceived = readMessageQueue();
// are there any messages that need to be sent?
if (relayMsgTimer.isRunning && relayMsgTimer.isTimeout(SEND_RELAY_INTERVAL))
relayMsgTimer.isRunning = !sendRelayData();
if (relayDashTimer.isRunning && relayDashTimer.isTimeout(SEND_DASH_WAIT))
sendRelayDataToDashboard();
if (queryMsgTimer.isRunning && queryMsgTimer.isTimeout(SEND_QUERY_INTERVAL))
sendQueryMsgToRelay();
// timed events
if (currentHour != getCurrentHour()) {
currentHour = getCurrentHour();
// check if relay needs to be switched on or off based on timer
if (timedRelayMode)
updateTimedRelay(currentHour);
// reset daily temperature min&max
updateDailyTemperature(currentHour);
}
delay(100);
fflush(stdout);
}
/* cleanup */
curl_easy_cleanup(curl);
curl_global_cleanup();
mq_close(mq);
mq_unlink(QUEUE_NAME);
return 0;
}
The common.h include:
#ifndef COMMON_H_
#define COMMON_H_
#define CHANNEL 1
#define QUEUE_NAME "/rf24_queue"
#define MAX_SIZE 1024
#define MSG_STOP "exit"
#define MSG_ON "ON"
#define MSG_OFF "OFF"
#define MSG_SET "SET"
#define MSG_GET "GET"
#define MSG_TIMER "TIMER"
#define MASTER_NODE 0
#define SENSOR_NODE 01
#define RELAY_NODE 01
#define SENSOR_MSG '1'
#define RELAY_MSG 'R'
#define QUERY_MSG 'Q'
#ifndef AM2320_H
struct sensorData_t {
int16_t temperature;
uint16_t humidity;
} sensorData;
#endif
struct relayData_t {
bool state;
} relayData;
typedef enum { wdt_16ms = 0, wdt_32ms, wdt_64ms, wdt_128ms, wdt_250ms, wdt_500ms, wdt_1s, wdt_2s, wdt_4s, wdt_8s } wdt_prescalar_e;
#endif /* #ifndef COMMON_H_ */
The RF24 libraries are available from here: https://github.com/TMRh20/RF24. Thanks for your time!
The offending code was this block here (in sendDataToDashBoard):
CURLcode res = curl_easy_perform(curl);
if (res != CURLE_OK)
printf("curl_easy_perform failed: %s\n", curl_easy_strerror(res));
curl_easy_reset(curl);
I commented out the curl_easy_reset() call and now it doesn't crash anymore. I don't know why, but there you have it. I think I don't need to do a reset since I always set the same options on every perform (URL & POSTFIELDS), so this works for me.

Sub-functions to send and receive string over socket

I assume that for messages that are of only 1 byte (a char), I will use read() and write() directly.
For those messages having size > 1 bytes, I use two subfunctions to read and write them over sockets.
For example, I have the server construct a string called strcities (list of city) and print it out --> nothing strange. Then send the number of bytes of this string to the client, and then the actual string.
The client will first read the number of bytes, then the actual city list.
For some reason my code sometimes work and sometimes doesn't. If it works, it also prints out some extra characters that I have no idea where they come from. If it doesn't, it hangs and forever waits in the client, while the server goes back to the top of the loop and wait for next command from the client. Could you please take a look at my codes below and let me know where I did wrong?
Attempt_read
string attempt_read(int rbytes) { // rbytes = number of bytes of message to be read
int count1, bytes_read;
char buffer[rbytes+1];
bool notdone = true;
count1 = read(sd, buffer, rbytes);
while (notdone) {
if (count1 == -1){
perror("Error on write call");
exit(1);
}
else if (count1 < rbytes) {
rbytes = rbytes - count1; // update remaining bytes to be read
count1 = read(sd, buffer, rbytes);
}
else {notdone = false;}
} // end while
string returnme;
returnme = string(buffer);
return returnme;
}
Attempt_write
void attempt_write(string input1, int wbytes) { // wbytes = number of bytes of message
int count1;
bool notdone = true;
count1 = write(sd, input1.c_str(), wbytes);
while (notdone) {
if (count1 == -1){
perror("Error on write call");
exit(1);
}
else if (count1 < wbytes) {
wbytes = wbytes - count1;
count1 = write(sd, input1.c_str(), wbytes);
}
else {notdone = false;}
} // end while
return;
}
1) string class has a method size() that will return the length of the string, so you do not actually need a second attempt_write parameter.
2) You can transfer length of message before message or you can transfer a terminating 0 after, if you only will sent an ASCII strings. Because your connection could terminate at any time, it is better to send exact length before sending the string, so your client could know, what to expect.
3) What compilator do you use, that would allow char buffer[rbytes+1]; ? A standard c++ would require char buffer = new char[rbytes+1]; and corresponding delete to avoid a memory leaks.
4) In your code, the second read function call use same buffer with no adjustment to length, so you, practically, overwrite the already received data and the function will only work, if all data will be received in first function call. Same goes for write function
I would suggest something like this:
void data_read(unsigned char * buffer, int size) {
int readed, total = 0;
do {
readed = read(sd, buffer + total, size - total);
if (-1 == writted) {
perror("Error on read call");
exit(1);
}
total += readed;
} while (total < size);
}
string attempt_read() {
int size = 0;
data_read((unsigned char *) &size, sizeof(int));
string output(size, (char) 0x0);
data_read((unsigned char *) output.c_str(), size);
return output;
}
void data_write(unsigned char * buffer, int size) {
int writted, total = 0;
do {
writted = write(sd, buffer + total, size - total);
if (-1 == writted) {
perror("Error on write call");
exit(1);
}
total += writted;
} while (total < size);
}
void attempt_write(string input) {
int size = input.size();
data_write((unsigned char *) &size, sizeof(int));
data_write((unsigned char *) input.c_str(), size);
}

Why Windows C++ muti-threading IOPS is much faster than IOMeter?

I have a SSD and I am trying to use it to simulate my program I/O performance, however, IOPS calculated from my program is much much faster than IOMeter.
My SSD is PLEXTOR PX-128M3S, by IOMeter, its max 512B random read IOPS is around 94k (queue depth is 32).
However my program (32 windows threads) can reach around 500k 512B IOPS, around 5 times of IOMeter! I did data validation but didn't find any error in data fetching. It's because my data fetching in order?
I paste my code belwo (it mainly fetch 512B from file and release it; I did use 4bytes (an int) to validate program logic and didn't find problem), can anybody help me figure out where I am wrong?
Thanks so much in advance!!
#include <stdio.h>
#include <Windows.h>
//Global variables
long completeIOs = 0;
long completeBytes = 0;
int threadCount = 32;
unsigned long long length = 1073741824; //4G test file
int interval = 1024;
int resultArrayLen = 320000;
int *result = new int[resultArrayLen];
//Method declarison
double GetSecs(void); //Calculate out duration
int InitPool(long long,char*,int); //Initialize test data for testing, if successful, return 1; otherwise, return a non 1 value.
int * FileRead(char * path);
unsigned int DataVerification(int*, int sampleItem); //Verify data fetched from pool
int main()
{
int sampleItem = 0x1;
char * fPath = "G:\\workspace\\4G.bin";
unsigned int invalidIO = 0;
if (InitPool(length,fPath,sampleItem)!= 1)
printf("File write err... \n");
//start do random I/Os from initialized file
double start = GetSecs();
int * fetchResult = FileRead(fPath);
double end = GetSecs();
printf("File read IOPS is %.4f per second.. \n",completeIOs/(end - start));
//start data validation, for 4 bytes fetch only
// invalidIO = DataVerification(fetchResult,sampleItem);
// if (invalidIO !=0)
// {
// printf("Total invalid data fetch IOs are %d", invalidIO);
// }
return 0;
}
int InitPool(long long length, char* path, int sample)
{
printf("Start initializing test data ... \n");
FILE * fp = fopen(path,"wb");
if (fp == NULL)
{
printf("file open err... \n");
exit (-1);
}
else //initialize file for testing
{
fseek(fp,0L,SEEK_SET);
for (int i=0; i<length; i++)
{
fwrite(&sample,sizeof(int),1,fp);
}
fclose(fp);
fp = NULL;
printf("Data initialization is complete...\n");
return 1;
}
}
double GetSecs(void)
{
LARGE_INTEGER frequency;
LARGE_INTEGER start;
if(! QueryPerformanceFrequency(&frequency))
printf("QueryPerformanceFrequency Failed\n");
if(! QueryPerformanceCounter(&start))
printf("QueryPerformanceCounter Failed\n");
return ((double)start.QuadPart/(double)frequency.QuadPart);
}
class input
{
public:
char *path;
int starting;
input (int st, char * filePath):starting(st),path(filePath){}
};
//Workers
DWORD WINAPI FileReadThreadEntry(LPVOID lpThreadParameter)
{
input * in = (input*) lpThreadParameter;
char* path = in->path;
FILE * fp = fopen(path,"rb");
int sPos = in->starting;
// int * result = in->r;
if(fp != NULL)
{
fpos_t pos;
for (int i=0; i<resultArrayLen/threadCount;i++)
{
pos = i * interval;
fsetpos(fp,&pos);
//For 512 bytes fetch each time
unsigned char *c =new unsigned char [512];
if (fread(c,512,1,fp) ==1)
{
InterlockedIncrement(&completeIOs);
delete c;
}
//For 4 bytes fetch each time
/*if (fread(&result[sPos + i],sizeof(int),1,fp) ==1)
{
InterlockedIncrement(&completeIOs);
}*/
else
{
printf("file read err...\n");
exit(-1);
}
}
fclose(fp);
fp = NULL;
}
else
{
printf("File open err... \n");
exit(-1);
}
}
int * FileRead(char * p)
{
printf("Starting reading file ... \n");
HANDLE mWorkThread[256]; //max 256 threads
completeIOs = 0;
int slice = int (resultArrayLen/threadCount);
for(int i = 0; i < threadCount; i++)
{
mWorkThread[i] = CreateThread(
NULL,
0,
FileReadThreadEntry,
(LPVOID)(new input(i*slice,p)),
0,
NULL);
}
WaitForMultipleObjects(threadCount, mWorkThread, TRUE, INFINITE);
printf("File read complete... \n");
return result;
}
unsigned int DataVerification(int* result, int sampleItem)
{
unsigned int invalid = 0;
for (int i=0; i< resultArrayLen/interval;i++)
{
if (result[i]!=sampleItem)
{
invalid ++;
continue;
}
}
return invalid;
}
I didn't look in enough detail to be certain, but I didn't see any code there to flush the data to the disk and/or ensure your reads actually came from the disk. That being the case, it appears that what you're measuring is primarily the performance of the operating system's disk caching. While the disk might contribute a little to the performance you're measuring, it's probably only a small contributor, with other factors dominating.
Since the code is apparently written for Windows, you might consider (for one example) opening the file with CreateFile, and passing the FILE_FLAG_NO_BUFFERING flag when you do so. This will (at least mostly) remove the operating system cache from the equation, and force each read or write to deal directly with the disk itself.

How to guarantee read() actually sends 100% of data sent by write() through named pipes

I've got the following two programs, one acting as a reader and the other as a writer. The writer seems to only send about 3/4 of the data correctly to be read by the reader. Is there any way to guarantee that all the data is being sent? I think I've got it set up so that it reads and writes reliably, but it still seems to miss 1/4 of the data.
Heres the source of the writer
#define pipe "/tmp/testPipe"
using namespace std;
queue<string> sproutFeed;
ssize_t r_write(int fd, char *buf, size_t size) {
char *bufp;
size_t bytestowrite;
ssize_t byteswritten;
size_t totalbytes;
for (bufp = buf, bytestowrite = size, totalbytes = 0;
bytestowrite > 0;
bufp += byteswritten, bytestowrite -= byteswritten) {
byteswritten = write(fd, bufp, bytestowrite);
if(errno == EPIPE)
{
signal(SIGPIPE,SIG_IGN);
}
if ((byteswritten) == -1 && (errno != EINTR))
return -1;
if (byteswritten == -1)
byteswritten = 0;
totalbytes += byteswritten;
}
return totalbytes;
}
void* sendData(void *thread_arg)
{
int fd, ret_val, count, numread;
string word;
char bufpipe[5];
ret_val = mkfifo(pipe, 0777); //make the sprout pipe
if (( ret_val == -1) && (errno != EEXIST))
{
perror("Error creating named pipe");
exit(1);
}
while(1)
{
if(!sproutFeed.empty())
{
string s;
s.clear();
s = sproutFeed.front();
int sizeOfData = s.length();
snprintf(bufpipe, 5, "%04d\0", sizeOfData);
char stringToSend[strlen(bufpipe) + sizeOfData +1];
bzero(stringToSend, sizeof(stringToSend));
strncpy(stringToSend,bufpipe, strlen(bufpipe));
strncat(stringToSend,s.c_str(),strlen(s.c_str()));
strncat(stringToSend, "\0", strlen("\0"));
int fullSize = strlen(stringToSend);
signal(SIGPIPE,SIG_IGN);
fd = open(pipe,O_WRONLY);
int numWrite = r_write(fd, stringToSend, strlen(stringToSend) );
cout << errno << endl;
if(errno == EPIPE)
{
signal(SIGPIPE,SIG_IGN);
}
if(numWrite != fullSize )
{
signal(SIGPIPE,SIG_IGN);
bzero(bufpipe, strlen(bufpipe));
bzero(stringToSend, strlen(stringToSend));
close(fd);
}
else
{
signal(SIGPIPE,SIG_IGN);
sproutFeed.pop();
close(fd);
bzero(bufpipe, strlen(bufpipe));
bzero(stringToSend, strlen(stringToSend));
}
}
else
{
if(usleep(.0002) == -1)
{
perror("sleeping error\n");
}
}
}
}
int main(int argc, char *argv[])
{
signal(SIGPIPE,SIG_IGN);
int x;
for(x = 0; x < 100; x++)
{
sproutFeed.push("All ships in the sea sink except for that blue one over there, that one never sinks. Most likley because it\'s blue and thats the mightiest colour of ship. Interesting huh?");
}
int rc, i , status;
pthread_t threads[1];
printf("Starting Threads...\n");
pthread_create(&threads[0], NULL, sendData, NULL);
rc = pthread_join(threads[0], (void **) &status);
}
Heres the source of the reader
#define pipe "/tmp/testPipe"
char dataString[50000];
using namespace std;
char *getSproutItem();
void* readItem(void *thread_arg)
{
while(1)
{
x++;
char *s = getSproutItem();
if(s != NULL)
{
cout << "READ IN: " << s << endl;
}
}
}
ssize_t r_read(int fd, char *buf, size_t size) {
ssize_t retval;
while (retval = read(fd, buf, size), retval == -1 && errno == EINTR) ;
return retval;
}
char * getSproutItem()
{
cout << "Getting item" << endl;
char stringSize[4];
bzero(stringSize, sizeof(stringSize));
int fd = open(pipe,O_RDONLY);
cout << "Reading" << endl;
int numread = r_read(fd,stringSize, sizeof(stringSize));
if(errno == EPIPE)
{
signal(SIGPIPE,SIG_IGN);
}
cout << "Read Complete" << endl;
if(numread > 1)
{
stringSize[numread] = '\0';
int length = atoi(stringSize);
char recievedString[length];
bzero(recievedString, sizeof(recievedString));
int numread1 = r_read(fd, recievedString, sizeof(recievedString));
if(errno == EPIPE)
{
signal(SIGPIPE,SIG_IGN);
}
if(numread1 > 1)
{
recievedString[numread1] = '\0';
cout << "DATA RECIEVED: " << recievedString << endl;
bzero(dataString, sizeof(dataString));
strncpy(dataString, recievedString, strlen(recievedString));
strncat(dataString, "\0", strlen("\0"));
close(fd);
return dataString;
}
else
{
return NULL;
}
}
else
{
return NULL;
}
close(fd);
}
int main(int argc, char *argv[])
{
int rc, i , status;
pthread_t threads[1];
printf("Starting Threads...\n");
pthread_create(&threads[0], NULL, readItem, NULL);
rc = pthread_join(threads[0], (void **) &status);
}
You are definitely using signals the wrong way. Threads are completely unnecessary here - at least in the code provided. String calculations are just weird. Get this book and do not touch the keyboard until you finished reading :)
The general method used to send data through named pipes is to tack on a header with the length of the payload. Then you read(fd, header_len); read(rd, data_len); Note the latter read() will need to be done in a loop until data_len is read or eof. Note also if you've multiple writers to a named pipe then the writes are atomic (as long as a reasonable size) I.E. multiple writers will not case partial messages in the kernel buffers.
It's difficult to say what is going on here. Maybe you are getting an error returned from one of your system calls? Are you sure that you are successfully sending all of the data?
You also appear to have some invalid code here:
int length = atoi(stringSize);
char recievedString[length];
This is a syntax error, since you cannot create an array on the stack using a non-constanct expression for the size. Maybe you are using different code in your real version?
Do you need to read the data in a loop? Sometimes a function will return a portion of the available data and require you to call it repeatedly until all of the data is gone.
Some system calls in Unix can also return EAGAIN if the system call is interrupted - you are not handling this case by the looks of things.
You are possibly getting bitten by POSIX thread signal handling semantics in your reader main thread.
The POSIX standard allows for a POSIX thread to receive the signal, not necessarily the thread you expect. Block signals where not wanted.
signal(SIG_PIPE,SIG_IGN) is your friend. Add one to reader main.
POSIX thread handling semantics, putting the POS into POSIX. ( but it does make it easier to implement POSIX threads.)
Examine the pipe in /tmp with ls ? is it not empty ?