It's something that seems deceptively simple, but comes with a lot of nasty details and compatibility problems. I have some code that kinda works on Linux and... sorta works on Windows but it's having various problems, for what seems like a common and simple problem. I know async is all the rage these days, but I have good reasons to want a process per connection.
I'm writing a server that hosts simulation processes. So each connection is long-running and CPU intensive. But more importantly, these simulators (Ngspice, Xyce) have global state and sometimes segfault or reach unrecoverable errors. So it is essential that each connection has its own process so they can run/crash in parallel and not mess with each other's state.
Another semi-important detail is that the protocol is based on Capnp RPC, which has a nice cross-platform async API, but not a blocking one. So what I do is have my own blocking accept loop that forks a new process and then starts the Capnp event loop in the new process.
So I started with a simple accept loop, added a ton of ifdefs to support windows, and then added fork to make it multiprocess and then added a SIGCHLD handler to try to avoid zombie processes. But Windows doesn't have fork, and if many clients disconnect simultaneously I still get zombies.
My current code lives here: https://github.com/NyanCAD/SimServer/blob/1ba47205904fe57196498653ece828c572579717/main.cpp
I'm fine with either some more ifdefs and hacks to make Windows work and avoid zombies, or some sort of library that either offers a ready made multiprocess socket server or functionality for writing such a thing. The important part is that it can accept a socket in a new process and pass the raw FD to the Capnp event loop.
In my C++ application I'm using a third party library for Bluetooth discovering process. I'm looking at the examples provided to learn how to use it.
The example that best match my needs is a simple GUI application that call a Discovery(long timeout) function from the library to start the Bluetooth discovery.
That function returns immediatly (so that the GUI is not freezed) and fires an __event called OnDeviceFound once a new BT device has been discovered and OnDiscoveryComplete once the timeout has elapsed.
So in the GUI constructor (of the example) there're __hook defined like this:
__hook(&BluetoothDiscovery::OnDiscoveryComplete, &m_Discovery, &BluetoothClientDlg::OnDiscoveryComplete);
Now, I need to implement the same in my application, that is not a Window application but a console application that runs as a Windows Service, doing a continuos discovering on a separate thread looking for new devices.
So, actually, since my implementation makes use of a thread for discovery, I don't need an event based discovery procedure, but I need a blocking discovering. The library does not provide a blocking API for discovering.
So here comes the question: is it possible to use an event based function in a blocking function? In other words, is it possible to write a function that could be called in the thread main loop every n seconds that does a discovery procedure and return the founded Bluetooth devices (using that event-based library API)?
What you want is a Semaphore which your main thread sits on until the discovery thread completes and then notifies your main thread to wake.
Active waits like what you suggest are nasty, and should be avoided where you can.
I have used Qt quite a lot, but recently needed to debug the threads I have been creating and found many more threads then I was expecting.
So my program is a simple console only (no GUI) Qt application (linux).
Threads that I have created:
It has a main() (which executes the QtCoreApplication) - so that is the main thread.
A thread to process received data from the com port (using FTDI D2XX thirdparty code drivers)
And that is all. When I do ps -T... and find my application there are 7 threads. I have two classes that are QObjects using signals and slots, so maybe they need a thread each for message handling, that takes me to 4 threads... so I am at a loss as to why I might have 7 threads for my application.
Can anyone explain more about what is going on? can post code if needed. Note I only use new QThread once in my code (for the moment).
Qt doesn't create any per-QObject threads. It creates helper threads for some plaform-specific reasons, e.g. QProcess sometimes needs helper threads.
The FTDI D2XX unix driver uses libusb and that implementation is completely backwards and uses additional threads on top of the thread you've provided for it. Frankly said, you shouldn't be using the D2XX driver on Linux or OS X. Just use the kernel driver.
You should simply run the D2XX driver in a trivial non-Qt test application that opens the device and reads from it continuously and see how many threads it spawns. You'll be dismayed...
Please let me explain what is my problem:
I have a Gui application, that has to connect to remote server and keep connected to it for the time untill a user decides to quit the connection, or the server will. I wish to create the client connection mechanism in a separate thread. If the client should be able to asynchronusly receive data and in event driven style inform the main gui thread about it. The thread should also be able to receive data from gui thread to be sent to the server.
I come from a low level microcontroller place, where I would handle this task simply using interrupts and while(1) loop and flags. The problem is on a pc, this would take to much processor time. I have watched and read a lot of tutorials about sockets and threads in qt, but i still dont know what is the best aproach and how to do it properly.
For now, I have a test server on a remote target that is able to receive connections from my Qt client that I am trying to write. I have a class now for my client in Qt, that inherits from Qthread, but then I read that it is not the best aproach anymore.
I wish to create a client instance in new thread (triggered from the gui thread) that will hang forever with exec(). Now I dont know how to handle, using signals the incoming data from the server and incoming commands from the main GUI thread. In general, I would maybe know how to implement this on a low level, but i read about a lot of high level functions for this that qt delivers, i wish to use that.
I would really aprichiate help in this matter. I tried searching, but havent found any solid, working up to date code examples. Could someone please explain me how to create a client instannce in a new thread that wont disconnect after sending/ receiving some data, but instead stay connected and stay responsive to to server calls and gui thread calls in event driven style?
May be use general Qt socket mechanism instead separate thread will be better for you. Sockets is very similar to MCU interrupts and simple to use. For your application requests it must be enough.
In this thread (posted about a year ago) there is a discussion of problems that can come with running Word in a non-interactive session. The (quite strong) advice given there is not to do so. In one post it is stated "The Office APIs all assume you are running Office in an interactive session on a desktop, with a monitor, keyboard and mouse and, most importantly, a message pump." I'm not sure what that is. (I've been programming in C# for only about a year; my other programming experience has primarily been with ColdFusion.)
Update:
My program runs through a large number of RTF files to extract two pieces of information used to construct a medical report number. Rather than try and figure out how the formatting instructions in RTF work, I decided to just open them in Word and pull the text out from there (without actually starting the GUI). Occasionally, the program hiccuped in the middle of processing one file, and left a Word thread open attached to that document (I still have to figure out how to shut that one down). When I re-ran the program, of course I got a notification that there was a thread using that file, and did I want to open a read-only copy? When I said Yes, the Word GUI suddenly popped up from nowhere and started processing the files. I was wondering why that happened; but it looks like maybe once the dialog box popped up the message pump started pushing the main GUI to Windows as well?
A message loop is a small piece of code that exists in any native Windows program. It roughly looks like this:
MSG msg;
while (GetMessage(&msg, NULL, 0, 0))
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}
The GetMessage() Win32 API retrieves a message from Windows. Your program typically spends 99.9% of its time there, waiting for Windows to tell it something interesting happened. TranslateMessage() is a helper function that translates keyboard messages. DispatchMessage() ensures that the window procedure is called with the message.
Every GUI enabled .NET program has a message loop, it is started by Application.Run().
The relevance of a message loop to Office is related to COM. Office programs are COM-enabled programs, that's how the Microsoft.Office.Interop classes work. COM takes care of threading on behalf of a COM coclass, it ensures that calls made on a COM interface are always made from the correct thread. Most COM classes have a registry key in the registry that declares their ThreadingModel, by far the most common ones (including Office) use "Apartment". Which means that the only safe way to call an interface method is by making the call from the same thread that created the class object. Or to put it another way: by far most COM classes are not thread-safe.
Every COM enabled thread belongs to a COM apartment. There are two kinds, Single Threaded Apartments (STA) and a Multi Thread Apartment (MTA). An apartment threaded COM class must be created on an STA thread. You can see this back in .NET programs, the entry point of the UI thread of a Windows Forms or WPF program has the [STAThread] attribute. The apartment model for other threads is set by the Thread.SetApartmentState() method.
Large parts of Windows plumbing won't work correctly if the UI thread is not STA. Notably Drag+Drop, the clipboard, Windows dialogs like OpenFileDialog, controls like WebBrowser, UI Automation apps like screen readers. And many COM servers, like Office.
A hard requirement for an STA thread is that it should never block and must pump a message loop. The message loop is important because that's what COM uses to marshal an interface method call from one thread to another. Although .NET makes marshaling calls easy (Control.BeginInvoke or Dispatcher.BeginInvoke for example), it is actually a very tricky thing to do. The thread that executes the call must be in a well-known state. You can't just arbitrarily interrupt a thread and force it to make a method call, that would cause horrible re-entrancy problems. A thread should be "idle", not busy executing any code that is mutating the state of the program.
Perhaps you can see where that leads: yes, when a program is executing the message loop, it is idle. The actual marshaling takes place through a hidden window that COM creates, it uses PostMessage to have the window procedure of that window execute code. On the STA thread. The message loop ensures that this code runs.
The "message pump" is a core part of any Windows program that is responsible for dispatching windowing messages to the various parts of the application. This is the core of Win32 UI programming. Because of its ubiquity, many applications use the message pump to pass messages between different modules, which is why Office applications will break if they are run without any UI.
Wikipedia has a basic description.
John is talking about how the Windows system (and other window based systems - X Window, original Mac OS....) implement asynchronous user interfaces using events via a message system.
Behind the scenes for each application there is a messaging system where each window can send events to other windows or event listeners -- this is implemented by adding a message to the message queue. There is a main loop which always runs looking at this message queue and then dispatching the messages (or events) to the listeners.
The Wikipedia article Message loop in Microsoft Windows shows example code of a basic Windows program -- and as you can see at the most basic level a Windows program is just the "message pump".
So, to pull it all together. The reason a windows program designed to support a UI can't act as a service is because it needs the message loop running all the time to enable UI support. If you implement it as a service as described, it won't be able to process the internal asynchronous event handling.
In COM, a message pump serialises and de-serialises messages sent between apartments. An apartment is a mini process in which COM components can be run. Apartments come in single threaded and free threaded modes. Single threaded apartments are mainly a legacy system for applications of COM components that don't support multi-threading. They were typically used with Visual BASIC (as this did not support multi-threaded code) and legacy applications.
I guess that the message pump requirement for Word stems from either the COM API or parts of the application not being thread safe. Bear in mind that the .NET threading and garbage collection models don't play nicely with COM out of the box. COM has a very simplistic garbage collection mechanism and threading model that requires you to do things the COM way. Using the standard Office PIAs still requires you to explicitly shut down COM object references, so you need to keep track of every COM handle created. The PIAs will also create stuff behind the scenes if you're not careful.
.NET-COM integration is a whole topic all by itself, and there are even books written on the subject. Even using COM APIs for Office from an interactive desktop application requires you to jump through hoops and make sure that references are explicitly released.
Office can be assumed to be thread-unsafe, so you will need a separate instance of Word, Excel or other Office applications for each thread. You would have to incur the starting overhead or maintain a thread pool. A thread pool would have to be meticulously tested to make sure all COM references were correctly released. Even starting and shutting down instances requires you to make sure all references are released correctly. Failure to dot your i's and cross your t's here will result in large numbers of dead COM objects and even whole running instances of Word being leaked.
Wikipedia suggests it means the program's main Event Loop.
I think that this Channel 9 discussion has a nice succinct explanation:
This process of window communication is made possible by the so-called Windows Message Pump. Think of the Message Pump as an entity that enables cooperation between application windows and the desktop.