I'm writing a local network scanner on Windows to find online hosts with IP Helper Functions, which is equivalent to nmap -PR but without WinPcap. I know SendARP will block and send arp request 3 times if the remote host doesn't respond, so I use std::aync to create one threads for each host, but the problem is I want to send an ARP request every 20ms so it would not be too much arp packets in a very short time.
#include <iostream>
#include <future>
#include <vector>
#include <winsock2.h>
#include <iphlpapi.h>
#pragma comment(lib, "iphlpapi.lib")
#pragma comment(lib, "ws2_32.lib")
using namespace std;
int main(int argc, char **argv)
{
ULONG MacAddr[2]; /* for 6-byte hardware addresses */
ULONG PhysAddrLen = 6; /* default to length of six bytes */
memset(&MacAddr, 0xff, sizeof (MacAddr));
PhysAddrLen = 6;
IPAddr SrcIp = 0;
IPAddr DestIp = 0;
char buf[64] = {0};
size_t start = time(NULL);
std::vector<std::future<DWORD> > vResults;
for (auto i = 1; i< 255; i++)
{
sprintf(buf, "192.168.1.%d", i);
DestIp = inet_addr(buf);
vResults.push_back(std::async(std::launch::async, std::ref(SendARP), DestIp, SrcIp, MacAddr, &PhysAddrLen));
Sleep(20);
}
for (auto it= vResults.begin(); it != vResults.end(); ++it)
{
if (it->get() == NO_ERROR)
{
std::cout<<"host up\n";
}
}
std::cout<<"time elapsed "<<(time(NULL) - start)<<std::endl;
return 0;
}
At first I can do this by calling Sleep(20) after launching a thread, but once SendARP in these threads re-send ARP requests if no replies from remote host, it's out of my control, and I see many requests in a very short time(<10ms) in Wireshark, so my question is:
Any way to make SendARP asynchronous?
if not, can I control the sent timing of SendARP in threads?
There doesn't seem to be any way to force SendARP to act in a non-blocking manner, it would appear that when a host is unreachable, it will try to re-query several times before giving up.
As for the solution, nothing you want to hear. the MSDN Docs state that there's a newer API that deprecates SendARP called ResolveIpNetEntry2 that can also do the same thing, but it also appears to behave in the same manner.
The struct it receives contains a field called ReachabilityTime.LastUnreachable which is: The time, in milliseconds, that a node assumes a neighbor is unreachable after not having received a reachability confirmation.
However, it does not appear to have any real effect.
The best way to do it, is to use WinPCap or some other driver, there doesn't seem to be a way of solving your problem in userland.
Related
I'm adapting a tcp PubSub example to using inproc with multithread. It ends up hanging forever.
My setup
macOS Mojave, Xcode 10.3
zmq 4.3.2
The source code reeproducing the issue:
#include <string.h>
#include <stdio.h>
#include <unistd.h>
#include <thread>
#include "zmq.h"
void hello_pubsub_inproc() {
void* context = zmq_ctx_new();
void* publisher = zmq_socket(context, ZMQ_PUB);
printf("Starting server...\n");
int pub_conn = zmq_bind(publisher, "inproc://*:4040");
void* subscriber = zmq_socket(context, ZMQ_SUB);
printf("Collecting stock information from the server.\n");
int sub_conn = zmq_connect(subscriber, "inproc://localhost:4040");
sub_conn = zmq_setsockopt(subscriber, ZMQ_SUBSCRIBE, 0, 0);
std::thread t_pub = std::thread([&]{
const char* companies[2] = {"Company1", "Company2"};
int count = 0;
for(;;) {
int which_company = count % 2;
int index = (int)strlen(companies[0]);
char update[12];
snprintf(update, sizeof update, "%s",
companies[which_company]);
zmq_msg_t message;
zmq_msg_init_size(&message, index);
memcpy(zmq_msg_data(&message), update, index);
zmq_msg_send(&message, publisher, 0);
zmq_msg_close(&message);
count++;
}
});
std::thread t_sub = std::thread([&]{
int i;
for(i = 0; i < 10; i++) {
zmq_msg_t reply;
zmq_msg_init(&reply);
zmq_msg_recv(&reply, subscriber, 0);
int length = (int)zmq_msg_size(&reply);
char* value = (char*)malloc(length);
memcpy(value, zmq_msg_data(&reply), length);
zmq_msg_close(&reply);
printf("%s\n", value);
free(value);
}
});
t_pub.join();
// Give publisher time to set up.
sleep(1);
t_sub.join();
zmq_close(subscriber);
zmq_close(publisher);
zmq_ctx_destroy(context);
}
int main (int argc, char const *argv[]) {
hello_pubsub_inproc();
return 0;
}
The result
Starting server...
Collecting stock information from the server.
I've also tried adding this before joining threads to no avail:
zmq_proxy(publisher, subscriber, NULL);
The workaround: Replacing inproc with tcp fixes it instantly. But shouldn't inproc target in-process usecases?
Quick research tells me that it couldn't have been the order of bind vs. connect, since that problem is fixed in my zmq version.
The example below somehow tells me I don't have a missing shared-context issue, because it uses none:
ZeroMQ Subscribers not receiving message from Publisher over an inproc: transport class
I read from the Guide in the section Signaling Between Threads (PAIR Sockets) that
You can use PUB for the sender and SUB for the receiver. This will correctly deliver your messages exactly as you sent them and PUB does not distribute as PUSH or DEALER do. However, you need to configure the subscriber with an empty subscription, which is annoying.
What does it mean by an empty subscription?
Where am I doing wrong?
You can use PUB for the sender and SUB for the receiver. This will correctly deliver your messages exactly as you sent them and PUB does not distribute as PUSH or DEALER do. However, you need to configure the subscriber with an empty subscription, which is annoying.
Q : What does it mean by an empty subscription?
This means to set ( configure ) a subscription, driving a Topic-list message-delivery filtering, using an empty subscription string.
Q : Where am I doing wrong?
Here :
// sub_conn = zmq_setsockopt(subscriber, ZMQ_SUBSCRIBE, 0, 0); // Wrong
sub_conn = zmq_setsockopt(subscriber, ZMQ_SUBSCRIBE, "",0); // Empty string
Doubts also here, about using a proper syntax and naming rules :
// int pub_conn = zmq_bind(publisher, "inproc://*:4040");
int pub_conn = zmq_bind(publisher, "inproc://<aStringWithNameMax256Chars>");
as inproc:// transport-class does not use any kind of external stack, but maps the AccessPoint's I/O(s) onto 1+ memory-locations ( a stack-less, I/O-thread not requiring transport-class ).
Given this, there is nothing like "<address>:<port#>" being interpreted by such (here missing) protocol, so the string-alike text gets used as-is for identifying which Memory-location are the message-data going to go into.
So, the "inproc://*:4040" does not get expanded, but used "literally" as a named inproc:// transport-class I/O-Memory-location identified as [*:4040] ( Next, asking a .connect()-method of .connect( "inproc://localhost:4040" ) will, and must do so, lexically miss the prepared Memory-location: ["*:4040"] as the strings do not match
So this ought fail to .connect() - error-handling might be silent, as since the versions +4.x there is not necessary to obey the historical requirement to first .bind() ( creating a "known" named-Memory-Location for inproc:// ) before one may call a .connect() to get it cross-connected with an "already existing" named-Memory-location, so the v4.0+ will most probably not raise any error on calling and creating a different .bind( "inproc://*:4040" ) landing-zone and next asking a non-matching .connect( "inproc://localhost:4040" ) ( which does not have a "previously prepared" landing-zone in an already existing named-Memory-location.
I need to make a statistical printout of a socket program.
I am using method Listen(uint32_t port) in c++ thread to listen to clients on the specified port (more than one) and send/receive client's transactions to/from a server.
Now i need to write a log file of how many packet received/sent by this method.
my implementation is shown in the skeleton below:
hub.cpp
//set up necessary header
#include <iostream>
....
#include <vector>
//global variables
std::map<uint32_t,long> * received_pk;
std::map<uint32_t,long> * sent_pk;
void Listen(uint32_t port ); // method
int main (int argc, char **argv){
//set up client ports
vector<uint32_t> client_ports;
client_ports.push_back(50002);
client_ports.push_back(50003);
//initialize variables
received_pk = new std::map<uint32_t,uint32_t>();
sent_pk = new std::map<uint32_t,uint32_t>();
for(uint32_t i=0;i<client_ports.size();i++){
received_pk->insert(std::pair<uint32_t,uint32_t>(client_ports.at(i),0) );
sent_pk->insert(std::pair<uint32_t,uint32_t>(client_ports.at(i),0) );
}
//set up thread
vector<thread*> threads;
for(uint32_t i=0;i<client_ports.size();i++){
cout << "Create Listener in port " << client_ports.at(i) << endl;
threads.push_back(new thread(Listen,client_ports.at(i)));
}
//Wait for the threads to finish
for(uint32_t i=0;i<client_ports.size();i++){
threads.at(i)->join();
}
}
void Listen(uint32_t port){
...
set up struct sockaddr_in client, host;
listen on port: port
...
while(1){
receive packet from client;
received_pk->at(port)++;
check packet type
if(packet==status packet){
update the packet id number
}
if (packet==transaction){
send packet to Server
receive reply
send reply back to client
sent_pk->at(port)++;
}
}
}
Now i need to access received_pk and sent_pk while hub.cpp is still running (probably in the while loop)
I thought of two options:
Access received_pk and sent_pk from an external program: like define a method that can get the packet information while the thread is till running
problem: I don't know if i can access a variable/method while program is executing .
or print received_pk and sent_pk to a log file every 5 seconds.
problem: I don't know if it makes sense to have a timer in the multiple thread.
Please any advice will be appreciated.
Kehinde
Quite possibly, the easiest solution is to put the data in shared memory. The map x is a bit suspect - did you mean std::map<Key, Value>? That doesn't fit well in shared memory. Instead, use simple arrays. There are just 64K ports, and sizeof(long long[65536]) isn't excessive.
I need to port a multiprocess application that uses the Windows API functions SetEvent, CreateEvent and WaitForMultipleObjects to Linux. I have found many threads concerning this issue, but none of them provided a reasonable solution for my problem.
I have an application that forks into three processes and manages thread workerpool of one process via these Events.
I had multiple solutions to this issue. One was to create FIFO special files on Linux using mkfifo on linux and use a select statement to awaken the threads. The Problem is that this solution will operate differently than WaitForMultipleObjects. For Example if 10 threads of the workerpool will wait for the event and I call SetEvent five times, exactly five workerthreads will wake up and do the work, when using the FIFO variant in Linux, it would wake every thread, that i in the select statement and waiting for data to be put in the fifo. The best way to describe this is that the Windows API kind of works like a global Semaphore with a count of one.
I also thought about using pthreads and condition variables to recreate this and share the variables via shared memory (shm_open and mmap), but I run into the same issue here!
What would be a reasonable way to recreate this behaviour on Linux? I found some solutions doing this inside of a single process, but what about doing this with between multiple processes?
Any ideas are appreciated (Note: I do not expect a full implementation, I just need some more ideas to get myself started with this problem).
You could use a semaphore (sem_init), they work on shared memory. There's also named semaphores (sem_open) if you want to initialize them from different processes. If you need to exchange messages with the workers, e.g. to pass the actual tasks to them, then one way to resolve this is to use POSIX message queues. They are named and work inter-process. Here's a short example. Note that only the first worker thread actually initializes the message queue, the others use the attributes of the existing one. Also, it (might) remain(s) persistent until explicitly removed using mq_unlink, which I skipped here for simplicity.
Receiver with worker threads:
// Link with -lrt -pthread
#include <fcntl.h>
#include <mqueue.h>
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
void *receiver_thread(void *param) {
struct mq_attr mq_attrs = { 0, 10, 254, 0 };
mqd_t mq = mq_open("/myqueue", O_RDONLY | O_CREAT, 00644, &mq_attrs);
if(mq < 0) {
perror("mq_open");
return NULL;
}
char msg_buf[255];
unsigned prio;
while(1) {
ssize_t msg_len = mq_receive(mq, msg_buf, sizeof(msg_buf), &prio);
if(msg_len < 0) {
perror("mq_receive");
break;
}
msg_buf[msg_len] = 0;
printf("[%lu] Received: %s\n", pthread_self(), msg_buf);
sleep(2);
}
}
int main() {
pthread_t workers[5];
for(int i=0; i<5; i++) {
pthread_create(&workers[i], NULL, &receiver_thread, NULL);
}
getchar();
}
Sender:
#include <fcntl.h>
#include <stdio.h>
#include <mqueue.h>
#include <unistd.h>
int main() {
mqd_t mq = mq_open("/myqueue", O_WRONLY);
if(mq < 0) {
perror("mq_open");
}
char msg_buf[255];
unsigned prio;
for(int i=0; i<255; i++) {
int msg_len = sprintf(msg_buf, "Message #%d", i);
mq_send(mq, msg_buf, msg_len, 0);
sleep(1);
}
}
I'd like to convert all my log messages in my multi-threaded project, to use Apple System Log facility (or asl).
according to the following asl manual - https://developer.apple.com/library/ios/documentation/System/Conceptual/ManPages_iPhoneOS/man3/asl_get.3.html
When logging from multiple threads, each thread must open a separate client handle using asl_open.
For that reason, I've defined asl client per thread to be used in all my log commands. However, in facing some major difficulties in binding asl client to each asl_log command.
1. what if some of my asl log commands reside in a code that is common for
more than one thread - which asl client should i decide use on such message.
2. Even on thread unique code, one should be consistent in choosing the same
asl_client on all log functions on a single thread code scope (this is
not always easy to find in complex projects.).
Is there any easier way to adopt my project logging messages to use asl ?
I'd think about something like binding asl client to thread,
thanks
Ok, so the best solution I've found out so far is by creating a global variable asl client that is thread-specific.
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <asl.h>
#define NUMTHREADS 4
pthread_key_t glob_var_key;
void print_func() //take global var and use it as the aslclient per thread
{
asl_log(*((aslclient*) pthread_getspecific(glob_var_key)),NULL,ASL_LEVEL_NOTICE, "blablabla");
}
void* thread_func(void *arg)
{
aslclient *p = malloc(sizeof(aslclient));
// added tid to message format to distinguish between messages
uint64_t tid;
pthread_threadid_np(NULL, &tid);
char tid_str[20];
sprintf(tid_str, "%llu", tid);
*p = asl_open(tid_str,"Facility",ASL_OPT_STDERR);
pthread_setspecific(glob_var_key, p);
print_func();
sleep(1); // enable ctx switch
print_func();
pthread_setspecific(glob_var_key, NULL);
free(p);
pthread_exit(NULL);
}
int main(void)
{
pthread_t threads[NUMTHREADS];
int i;
pthread_key_create(&glob_var_key,NULL);
for (i=0; i < NUMTHREADS; i++)
pthread_create(&threads[i],NULL,thread_func,NULL);
for (i=0; i < NUMTHREADS; i++)
pthread_join(threads[i], NULL);
}
I've searched the internet for a while now and found different solutions but then all don't really work or are to complicated for my use.
I used C++ until 2 years ago so it might be a bit rusty :D
I'm currently writing a program that posts data to an URL. It only posts the data nothing else.
For posting the data I use curl, but it blocks the main thread and while the first post is still running there will be a second post that should start.
In the end there are about 5-6 post operations running at the same time.
Now I want to push the posting with curl into another thread. One thread per post.
The thread should get a string parameter with the content what to push.
I'm currently stuck on this. Tried the WINAPI for windows but that crashes on reading the parameter. (the second thread is still running in my example while the main thread ended (waiting on system("pause")).
It would be nice to have a multi plattform solution, because it will run under windows and linux!
Heres my current code:
#define CURL_STATICLIB
#include <curl/curl.h>
#include <curl/easy.h>
#include <cstdlib>
#include <iostream>
#include <stdio.h>
#include <stdlib.h>
#include <string>
#if defined(WIN32)
#include <windows.h>
#else
//#include <pthread.h>
#endif
using namespace std;
void post(string post) { // Function to post it to url
CURL *curl; // curl object
CURLcode res; // CURLcode object
curl = curl_easy_init(); // init curl
if(curl) { // is curl init
curl_easy_setopt(curl, CURLOPT_URL, "http://10.8.27.101/api.aspx"); // set url
string data = "api=" + post; // concat post data strings
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, data.c_str()); // post data
res = curl_easy_perform(curl); // execute
curl_easy_cleanup(curl); // cleanup
} else {
cerr << "Failed to create curl handle!\n";
}
}
#if defined(WIN32)
DWORD WINAPI thread(LPVOID data) { // WINAPI Thread
string pData = *((string*)data); // convert LPVOID to string [THIS FAILES]
post(pData); // post it with curl
}
#else
// Linux version
#endif
void startThread(string data) { // FUnction to start the thread
string pData = data; // some Test
#if defined(WIN32)
CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)thread, &pData, 0, NULL); // Start a Windows thread with winapi
#else
// Linux version
#endif
}
int main(int argc, char *argv[]) {
// The post data to send
string postData = "test1234567890";
startThread(postData); // Start the thread
system("PAUSE"); // Dont close the console window
return EXIT_SUCCESS;
}
Has anyone a suggestion?
Thanks for the help!
Consider using Boost.Thread or the new C++11 threading facilities (like std::thread etc.).
Some remarks to the code of the initial question:
If staying away from std::thread or boost::thread, use _beginthreadex(..) instead of CreateThread(..) because the latter one can cause resource leaks if used with certain functions of the C runtime.
When using CreateThread(..), a cast to LPTHREAD_START_ROUTINE is not required if the signature of the passed function is correct. So casting it is simply wrong.
The were already some remarks about the lifetime of stack allocated variables and what happens if the address of these are passed to a thread function.
Don't use system("PAUSE") in order to keep the code protable. Instead use the following snippet:
void wait_for_key_press()
{
std::cin.clear();
std::cin.ignore(std::cin.rdbuf()->in_avail());
std::cin.get();
}
Use std::thread for threads. It's a relatively new thing, part of the newest C++11 standard, but it will probably be the most portable way to do threads in the near future.
See how easy it is to make 5 threads that will busy-wait (unless optimized by the compiler):
#include<thread>
#include<vector>
int main()
{
std::vector<std::thread> threads;
for (int i=0; i< 5; i++)
{
threads.push_back(std::thread([] () {
for (long long j=0; j < 1000000000000LL; j++) ;
}));
}
for (auto & thread : threads)
{
thread.join();
}
}
You might want to avoid multi-threading by using libcurl 's multi-operations interface, which enables you to run several concurrent HTTP requests in the same (single) thread.