I am testing application's latency during UDP communication on windows 10.
I tried to send a message every 1 second and receive a response sent immediately from the remote.
Send thread
It works every 1 second.
auto start = std::chrono::system_clock::now();
unsigned int count = 1;
while (destroyFlag.load(std::memory_order_acquire) == false)
{
if (isReady() == false)
{
break;
}
/*to do*/
worker_();
std::this_thread::sleep_until(start + std::chrono::milliseconds(interval_)* count++);
}
worker_()
Send thread call this. just send message and make log string.
socket_.send(address_);
logger_.log("," + std::string("Send") + "\n");
Receiver
When message arrives, it creates a receive log string and flushes it to a file.
auto& queueData = socket_.getQueue();
while (queueData.size() > 0)
{
auto str = queueData.dequeue();
logger_.log(",Receive" + str + "\n");
logger_.flush();
}
I've been testing it overnight and I can't figure out why I got this result.
chart for microseconds
x-axis : Hour_Minute_second
y-axis : microseconds
For a few hours it seemed to work as expected. But after that, the time gradually changed and went to a different time zone.
Does anyone know why this is happening?
std::chrono::steady_clock is working.
It made my charts straight.
And another way, turn off the windows automatically time synchronize.
Related
I developed a little application that read data from a sensor, store them in SPIFFS memory of my wemos D1 mini (esp8266) and then create a JSON Document and send it via MQTT to my topic. The problem is that as long as I send a JSON Doc with 10 object everything works great, but when I increase the size of the doc over 10 object nothing works. Eventually I need to send a JSON doc with 100 object inside.
What have I already done?
I'm using PubSubClient and I already set the MAX_PACKET_SIZE to the correct value
Using arduinojson assistant I found out the size of my JSON Document (8192 bytes)
I tried to use mqtt.fx to test if the problem was the esp8266 or the mqtt broker. Using mqtt.fx I'm able to send a JSON doc with 100 objects
As soon as I increase the size of the JSON doc I get a wdt error from the serial monitor of my arduino IDE.
I search the internet for wdt error but I don't get what they are and how to solve my problem
Last things I already tried to show on the serial monitor the file.txt in the SPIFFS where I store the data and I can store and then read the 100 object
So in the end I think it's an esp8266 problem and not PubSubClient or MQTT. Am I right?
Does anyone of you here ever encountered this problem before or have some other test I can run?
I search the internet for wdt error but I don't get what they are and how to solve my problem
WDT stands for a Watch Dog Timer. https://os.mbed.com/cookbook/WatchDog-Timer#:~:text=A%20watchdog%20timer%20(WDT)%20is,a%20software%20or%20hardware%20fault.
A watchdog timer (WDT) is a hardware timer that automatically generates a system reset if the main program neglects to periodically service it. It is often used to automatically reset an embedded device that hangs because of a software or hardware fault. Some systems may also refer to it as a computer operating properly (COP) timer. Many microcontrollers including the mbed processor have watchdog timer hardware.
Let's paint a better picture with an example. Let's say that you setup a WDT with a time of 10 seconds. Then the WDT starts counting down from 10 seconds. If it reaches 0 the processor will reset. "Feeding" the WDT will reset the countdown to the original value in this case 10 seconds. So if the WDT has counted down to 4 seconds remaining and you feed it, it resets the countdown back to 10 and starts counting down again.
Does anyone of you here ever encountered this problem before or have some other test I can run?
It looks to me like sending a larger JSON object takes a longer period of time than what the WDT is set for. One possibility would be to break up the JSON object into multiple pieces and send it in smaller chunks instead of one large one. This way the time between WDT "feedings" is reduced. I have no idea if this would be possible for you to change. But this should at least give you a better idea of what's happening.
OK in the end the problem was that sending a large JsonDocument triggered the WDT and the only way I found to overcome this problem was, as suggested by adamvz, to create a main file with all the 100 object, then call a function to split that file in 10 smaller one and send each of them over the internet through an HTTP request or Mosquitto.
Supposing you already created the main file in the spiffs memory, then:
This to split the main file:
void WritePacks() {
sourceFile = LittleFS.open("/file.txt", "r");
if (!sourceFile) {
Serial.println(F("Error: file.txt open failed"));
} else {
Serial.println("File open w/ success");
for (byte idx = 0; idx < outputCount; idx++) {
String aLine;
aLine.reserve(capacity);
if (sourceFile.available() == 0) break;
destinationFile = LittleFS.open(outputFileNames[idx], "w");
if (!destinationFile) {
Serial.print(F("can't open destination "));
Serial.println(outputFileNames[idx]);
break;
} else {
int lineCount = 0;
while (sourceFile.available() && (lineCount <= 10)) {
aLine = sourceFile.readStringUntil('\n');
destinationFile.println(aLine); // double check if the '\n' is in the String or not (--> print or println accordingly)
lineCount++;
}
outputIndex = idx;
Serial.println(outputIndex);
destinationFile.close();
}
} // end for
sourceFile.close();
}
}//end WritePacks
This to publish:
//------ HTTP Publish ------
void httpPublish(){
const char * outputFileNames[] = {"/out1.txt", "/out2.txt", "/out3.txt", "/out4.txt", "/out5.txt", "/out6.txt", "/out7.txt", "/out8.txt", "/out9.txt", "/out10.txt"};
const byte outputCount = sizeof outputFileNames / sizeof outputFileNames[0];
byte outputIndex = 0;
File sourceFile;
File destinationFile;
//Serial.println(capacity);
for (byte idx = 0; idx < outputCount; idx++) {
DynamicJsonDocument doc(capacity);
DynamicJsonDocument globalDoc(capacity);
StaticJsonDocument <1024> localDoc;
String aLine;
aLine.reserve(capacity);
destinationFile = LittleFS.open(outputFileNames[idx], "r");
if (!destinationFile) {
Serial.print(F("can't open destination "));
Serial.println(outputFileNames[idx]);
break;
} else {
Serial.print("Reading: ");
Serial.println(outputFileNames[idx]);
//int lineCount = 0;
while (destinationFile.available()) {
aLine = destinationFile.readStringUntil('\n');
DeserializationError error = deserializeJson(localDoc, aLine);
if (!error) globalDoc.add(localDoc);
else{ Serial.println("Error Writing All files");}
}//while
JsonObject Info = doc.createNestedObject("Info");
Info["Battery"] = battery;
Info["ID"] = id;
Info["Latitudine"] = latitudine;
Info["Longitudine"] = longitudine;
JsonArray Data = doc.createNestedArray("Data");
Data.add(globalDoc);
HTTPClient http;
//Send request
http.begin("yourURL");
char buffer[capacity];
size_t n = serializeJson(doc, buffer);
http.POST(buffer);
Serial.println(buffer);
http.end();
destinationFile.close();
}
}// end for
}//end httpPublish
I have this simple code to pull the messages from Google PubSub Subscription:
#include "google/pubsub/v1/pubsub.grpc.pb.h"
#include "google/pubsub/v1/pubsub.pb.h"
#include "grpc++/grpc++.h"
#include "base/logging.h"
int main() {
auto creds = grpc::GoogleDefaultCredentials();
auto stub = std::make_unique<google::pubsub::v1::Subscriber::Stub>(
grpc::CreateChannel("pubsub.googleapis.com", creds));
grpc::ClientContext context;
std::unique_ptr<
grpc::ClientReaderWriter<google::pubsub::v1::StreamingPullRequest,
google::pubsub::v1::StreamingPullResponse>>
stream(stub->StreamingPull(&context));
google::pubsub::v1::StreamingPullRequest request;
request.set_subscription("my_subscription");
request.set_stream_ack_deadline_seconds(10);
stream->Write(request);
google::pubsub::v1::StreamingPullResponse response;
size_t count = 0;
while (stream->Read(&response)) {
google::pubsub::v1::StreamingPullRequest ack_request;
for (const auto& message : response.received_messages()) {
ack_request.add_ack_ids(message.ack_id());
if (++count % 1000 == 0) {
LOG(Info, "count: " << count << " message_size: " << message.message().data().size());
}
}
stream->Write(ack_request);
}
return 0;
}
It turned out that while (stream->Read(&response)) doesn't work forever and stops after ~30 minutes (I don't know why that happens). I tried to wrap the code in while (true) so messages will be pulled in an infinite loop but it turned out that the second iteration can't pull any messages (I see in Google Cloud monitoring that messages are coming).
What is wrong with this code?
I know that GCP didn't implement C++ client yet and StreamingPull is a low level API but I don't wanna wait until they make it (it's unclear when it'll happen) and also don't wanna switch to other language (my application is in C++).
Streaming connections to GCP can be closed for a variety of reasons, e.g. transient network issues or max TTLs on connection lifetimes. A stream should not be expected to be open indefinitely.
When stream->Read(&response) returns false, that's an indication that the stream has been closed [source]. Your code should then recreate the stream to continue pulling messages.
I am using CodeBlocks with MinGW compiler and wxWidgets library.
I am writing a program that read some data from the microcontroller, by sending messages (using index and subindex) and getting response messages with said data.
My plan was to send messages one-by-one and waiting for response message using __atomic int variables__ to check when I get response message.
This is my function for sending a message:
typedef std::chrono::high_resolution_clock Clock;
void sendSDO(int index, int subindex)
{
int nSent = 0;
atomic_index.store(index);
atomic_subindex.store(subindex);
canOpenClient->SDORead(index, subindex);
auto start = Clock::now();
nSentMessages++;
nSent++;
Sleep(10);
while ((atomic_index.load() != 0) && (atomic_subindex.load() != 0))
{
auto t = chrono::duration_cast<chrono::milliseconds>(Clock::now() - start);
if(t.count() > 20)
{
if (nSent > 5)
{
MainFrame->printTxt("[LOG] response not received\n");
return;
}
atomic_index.store(index);
atomic_subindex.store(subindex);
canOpenClient->SDORead(index, subindex);
nSentMessages++;
nSent++;
start = Clock::now();
}
}
}
Pseudocode is set atomic int to index and subindex of what I want value I want to read from microcontroller, then send message to it SDORead(), and if no response was received in 20 ms, send the message again, up to 5 times.
For receiving messages, I have a __separate thread__ with a callback function which is called when I get response message from the controller:
void notifyEvent(unsigned char ev_type)
{
SDO_msg_t msg;
msg = canOpenClient->Cmd_CustomMessageGet(); //get response message
if(ev_type == CO_EVENT_SDO_READ)
{
if ((msg.index == atomic_index.load()) && (msg.subindex == atomic_subindex.load()))
{
//does stuff, like saves message data to set container
atomic_index.store(0);
atomic_subindex.store(0);
}
}
if (message data not in container)
printf("not in container!")
}
Here I set the same atomic int values to 0, when the correct response message is received, and save response message data
I also have variables nSentMessages and nReceivedMessages, which hold the number of messages sent and messages received. I check at the end if these values are the same. Normally, I wouldn't need this (since I wait for every response), I put it there as an extra safety measure.
Now onto the problem:
1) My problem is in callback function notifyEvent(), where I presumably save response message data to a container, but I still sometimes get "not in container!" from that if statement and I don't know why. (My container is just normal set set<EDSobject, cmp> container, it's not atomic or anything, since I know there won't be reads/writes to it at the same time from different threads.)
2) If you check my function sendSDO(), there is a line Sleep(10). The program works ok with it, but if I remove it, the program returns a different value for nSentMessages and nReceivedMessages - 576 and 575. This happens every time I run the program and I don't understand why.
I'm implementing RS485 on arm developement board using serial port and gpio for data enable.
I'm setting data enable to high before sending and I want it to be set low after transmission is complete.
It can be simply done by writing:
//fd = open("/dev/ttyO2", ...);
DataEnable.Set(true);
write(fd, data, datalen);
tcdrain(fd); //Wait until all data is sent
DataEnable.Set(false);
I wanted to change from blocking-mode to non-blocking and use poll with fd. But I dont see any poll event corresponding to 'transmission complete'.
How can I get notified when all data has been sent?
System: linux
Language: c++
Board: BeagleBone Black
I don't think it's possible. You'll either have to run tcdrain in another thread and have it notify the the main thread, or use timeout on poll and poll to see if the output has been drained.
You can use the TIOCOUTQ ioctl to get the number of bytes in the output buffer and tune the timeout according to baud rate. That should reduce the amount of polling you need to do to just once or twice. Something like:
enum { writing, draining, idle } write_state;
while(1) {
int write_event, timeout = -1;
...
if (write_state == writing) {
poll_fds[poll_len].fd = write_fd;
poll_fds[poll_len].event = POLLOUT;
write_event = poll_len++
} else if (write == draining) {
int outq;
ioctl(write_fd, TIOCOUTQ, &outq);
if (outq == 0) {
DataEnable.Set(false);
write_state = idle;
} else {
// 10 bits per byte, 1000 millisecond in a second
timeout = outq * 10 * 1000 / baud_rate;
if (timeout < 1) {
timeout = 1;
}
}
}
int r = poll(poll_fds, poll_len, timeout);
...
if (write_state == writing && r > 0 && (poll_fds[write_event].revent & POLLOUT)) {
DataEnable.Set(true); // Gets set even if already set.
int n = write(write_fd, write_data, write_datalen);
write_data += n;
write_datalen -= n;
if (write_datalen == 0) {
state = draining;
}
}
}
Stale thread, but I have been working on RS-485 with a 16550-compatible UART under Linux and find
tcdrain works - but it adds a delay of 10 to 20 msec. Seems to be polled
The value returned by TIOCOUTQ seems to count bytes in the OS buffer, but NOT bytes in the UART FIFO, so it may underestimate the delay required if transmission has already started.
I am currently using CLOCK_MONOTONIC to timestamp each send, calculating when the send should be complete, when checking that time against the next send, delaying if necessary. Sucks, but seems to work
I am using ActiveMQ-CPP 3.4.5 to connect from C++ program to message broker.
The problem I encountered is connected with redelivery policy.
Messages which are returned to queue are redelivered immediately. This is not the
behaviour I expect. I would expect the messages to be returned after certain period
of time which is set through redelivery policy.
This is a code snipped showing the way I set up redelivery policy:
policy = new activemq::core::policies::DefaultRedeliveryPolicy();
policy->setInitialRedeliveryDelay(0);
policy->setRedeliveryDelay(10000);
policy->setMaximumRedeliveries((int)activemq::core::RedeliveryPolicy::NO_MAXIMUM_REDELIVERIES);
connectionFactory.setRedeliveryPolicy(policy);
As I said before I would except the messages to be redelivered after 10000 ms, but the are not.
They come back to consumer immediately.
Does anyone know what could be the reason of such behaviour?
You set the initial delay to zero so they are going to be redelivered immediately the first time a transaction is rolled back. If you want them to be delayed on the first redelivery cycle then you need to set the initial delay to 10000 as well.
When I looked into ActiveMQ-CPP sources I found the following code snippet in ActiveMQConsumer.cpp file:
if( internal->redeliveryDelay > 0 && !this->internal->unconsumedMessages->isClosed() ) {
// TODO - Can't do this until we can control object lifetime.
// Start up the delivery again a little later.
// this->internal->scheduler->executeAfterDelay(
// new StartConsumerTask(this), internal->redeliveryDelay);
start();
} else {
start();
}
So it seems that redeliveryDelay is not taken into account after rollback at all.
That is why, I suppose, my messages arrive immediately after rollback.
onMessage method:
void BaseProdListener::onMessage( const cms::Message* message ){
log4cxx::Logger::getLogger("BaseProdListener")->info("onMessage");
_message = message;
try {
const cms::TextMessage* textMessage = dynamic_cast< const cms::TextMessage* >( message );
std::string text = "";
if( textMessage != NULL ) {
text = textMessage->getText();
log4cxx::Logger::getLogger("BaseProdListener")->debug("Received message:" + text);
handleMessage(text);
}
} catch (cms::CMSException& e){
log4cxx::Logger::getLogger("BaseProdListener")->error(e.getStackTraceString());
}
}