Good day folks,
I am currently trying to figure a way to pass data between a 64bit process and a 32bit process. Since it's a real-time application and both are running on the same computer I tough using shared memory (shm).
While I was looking for some synchronization mechanism using shm, I felt on boost::message_queue. However it's not working.
My code is basically the following:
Sender part
message_queue::remove("message_queue");
message_queue mq(create_only, "message_queue", 100, sizeof(uint8_t));
for (uint8_t i = 0; i < 100; ++i)
{
mq.send(&i, sizeof(uint8_t), 0);
}
Receiver part
message_queue mq(open_only, "message_queue");
for (uint8_t i = 0; i < 100; ++i)
{
uint8_t v;
size_t rsize;
unsigned int rpriority;
mq.receive(&v, sizeof(v), rsize, rpriority);
std::cout << "v=" << (int) v << ", esize=" << sizeof(uint8_t) << ", rsize=" << rsize << ", rpriority=" << rpriority << std::endl;
}
This code works perfectly if the two process are 64bit or 32bit. But doesn't work if the two process aren't the same.
Looking deeper in boost (1.50.0) code you'll see the following line in message_queue_t::do_receive (boost/interprocess/ipc/message_queue.hpp):
scoped_lock lock(p_hdr->m_mutex);
For some reason, in the mutex seems to be locked when working with heterogeneous processes. My wild guess would be that the mutex is offsetted and therefore it's value is corrupted but I'm not quite sure.
Am I trying to accomplish something that is simply not supported?
Any help or advice will be appreciated.
I think this is about the portability of offset_ptr used in message_queue to point to each message, including the header mutex. 32-/64-bit interoperability should be supported since Boost 1.48.0, as addressed in https://svn.boost.org/trac/boost/ticket/5230.
Following the ticket suggestion, the following definition has (so far) worked fine for me in leiu of message_queue:
typedef message_queue_t< offset_ptr<void, int32_t, uint64_t> > interop_message_queue;
On Boost 1.50.0 under MSVC this also seems to require a small patch in message_queue.hpp to resolve template ambiguity: casting arguments in calls to ipcdetail::get_rounded_size(...).
I've spent my whole workday figure out the solution and finally I made it work. The solution is partially what James provided, so I used the interop_message_queue on both the 32-bit and the 64-bit processes.
typedef boost::interprocess::message_queue_t< offset_ptr<void, boost::int32_t, boost::uint64_t>> interop_message_queue;
The problem is that with this modification the code wouldn't compile, so I also had to add the following, which I found on the boost bug report list (#6147: message_queue sample fails to compile in 32-bit), this code has to be place before the boost includes of the message_queue:
namespace boost {
namespace interprocess {
namespace ipcdetail {
//Rounds "orig_size" by excess to round_to bytes
template<class SizeType, class ST2>
inline SizeType get_rounded_size(SizeType orig_size, ST2 round_to) {
return ((orig_size-1)/round_to+1)*round_to;
}
}
}
}
Related
Can anyone give me an example of how I can use segmented stacks with boost coroutines? Do I have to annotate every function that is called from the coroutine with a special split-stack attribute?
When I try and write a program that should use segmented stacks, it just segfaults.
Here is what I have done so far
https://wandbox.org/permlink/TltQwGpy4hRoHgDY The code seems to segfault very quickly, if segmented stacks were used I would expect it to be able to handle more iterations. The program errors out after 35 iterations.
#include <boost/coroutine2/all.hpp>
#include <iostream>
#include <array>
using std::cout;
using std::endl;
class Int {
int a{2};
};
void foo(int num) {
cout << "In iteration " << num << endl;
std::array<Int, 1000> arr;
static_cast<void>(arr);
foo(num + 1);
}
int main() {
using Coroutine_t = boost::coroutines2::coroutine<int>::push_type;
auto coro = Coroutine_t{[&](auto& yield) {
foo(yield.get());
}};
coro(0);
}
Compiling that code with -fsplit-stack solves the problem. Annotations are not required. All functions are by default treated as split stacks. Example - https://wandbox.org/permlink/Pzzj5gMoUAyU0h7Q
Easy as that.
compile boost (boost.context and boost.coroutine) with b2 property segmented-stacks=on (enables special code inside boost.coroutine and boost.context).
your app has to be compiled with -DBOOST_USE_SEGMENTED_STACKS and -fsplit-stack (required by boost.coroutines headers).
see documentation: http://www.boost.org/doc/libs/1_65_1/libs/coroutine/doc/html/coroutine/stack/segmented_stack_allocator.html
boost.coroutine contains an example that demonstrates segmented stacks
(in directory coroutine/example/asymmetric/ call b2 toolset=gcc segmented-stacks=on).
please note: while llvm supports segmented stacks, clang seams not to provide the __splitstack_<xyz> functions.
I am trying to create custom tcl channel and use it to get the output of tcl Interpreter. I added the implementation of few function of Tcl_ChannelType but I am getting segfault.
#include <tcl.h>
#include <iostream>
int driverBlockModeProc(ClientData instanceData, int mode) {
std::cout << "driverBlockModeProc\n";
return 0;
}
int driverCloseProc(ClientData instanceData, Tcl_Interp *interp) {
std::cout << "driverCloseProc\n";
return 0;
}
int driverInputProc(ClientData instanceData, char* buf, int bufSize, int *errorCodePtr) {
std::cout << "driverInputProc\n";
return 0;
}
int driverOutputProc(ClientData instanceData, const char* buf, int toWrite, int *errorCodePtr) {
std::cout << "driverOutputProc\n";
return 0;
}
int main() {
Tcl_ChannelType *typePtr = new Tcl_ChannelType;
typePtr->blockModeProc = driverBlockModeProc;
typePtr->outputProc = driverOutputProc;
typePtr->closeProc = driverCloseProc;
typePtr->inputProc = driverInputProc;
typePtr->seekProc = NULL;
typePtr->setOptionProc = NULL;
typePtr->getOptionProc = NULL;
typePtr->watchProc = NULL;
typePtr->getHandleProc = NULL;
typePtr->close2Proc = NULL;
typePtr->blockModeProc = NULL;
typePtr->flushProc = NULL;
typePtr->handlerProc = NULL;
typePtr->wideSeekProc = NULL;
typePtr->threadActionProc = NULL;
ClientData data = new char[200];
Tcl_CreateChannel(typePtr, "im_chanel", data, TCL_WRITABLE | TCL_READABLE);
}
I cant debug the segfault because its source are not available. I think the segfault is because a function is called which is NULL. I only need to use the channel to get the output of interpreter. Which functions I needn't implement here and is it right direction to solve the problem.
You're advised to download the source to Tcl when working at this level. I'm not sure what version you're using, but all official distributions of the source going back a very long way are on the SourceForge file distribution system; pick the exact match for the version you've got.
Creating a custom channel driver is not easy. There's a significant amount of complexity involved, and it isn't especially well-documented what “methods” within the channel driver type are mandatory and what are optional. (They're not C++ methods in a class — Tcl is pure C code for reasons too long to go into here — but they function in a conceptually similar way.)
If we look at the documentation for Tcl_CreateChannel, we see (quite a long way down that page) a definition of the channel type structure. The channel type structure should be statically allocated (Tcl's implementation assumes very strongly that it never changes location) and the following fields must be set to something meaningful:
typeName — This is the name of the channel type, useful for debugging!
version — This is the version of the channel type; you should set it to the latest version supported by your target source level. (You're recommended to use at least TCL_CHANNEL_VERSION_2 or things get rather more complex.)
closeProc or close2Proc — Channels must be closeable, but you have two choices for ways to do it. Bidirectional channels ought to use the close2Proc, but aren't strictly required to.
inputProc — Only needed if you're reading; take care to handle this correctly.
outputProc — Only needed if you're writing; take care to handle this correctly.
watchProc — Called to tell the channel driver to install itself into the event system so as to receive suitable events (as instructed by the supplied bitmask). Channels that don't have backing OS handles use timer events, or simply never actually generate events (in which case they'll never become readable or writable from the perspective of fileevent).
Looking at your code, I see that you're missing a watchProc. I know it's hard to see (not many people write channel drivers, to be honest, so the documentation isn't very hard “tested”) but it's really necessary.
I need to check if a dynamic library is present, so that later I can safely call functions that use this library.
Is there a multiplatform way to check this? I am targeting MS Windows 7 (VC++11) and Linux (g++).
To dynamically "use" a function from a shared library requires that the library isn't part of the executable file, so you will need to write code to load the library and then use the function. There may well be ways to to do that in a portable fashion, but I'm not aware of any code available to do that.
It isn't very hard code to write. As "steps", it involves the following:
Load the library given a name of a file (e.g. "xx", which is then translated to "xx.so" or "xx.dll" in the architecture specific code).
Find a function based on either index ("function number 1") or name ("function blah"), and return the address.
Repeat step 2 for all relevant functions.
When no longer needing the library, close it with the handle provided.
If step 1 fails, then your library isn't present (or otherwise "not going to work"), so you can't call functions in it...
Clearly, there are many ways to design an interface to provide this type of functionality, and exactly how you go about that would depend on what your actual problem setting is.
Edit:
To clarify the difference between using a DLL directly, and using one using dynamic loading from the code:
Imagine that this is our "shared.h", which defines the functions for the shared library
(There is probably some declspec(...) or exportsymbol or other such stuff in a real header, but I'll completely ignore that for now).
int func1();
char *func2(int x);
In a piece of code that directly uses the DLL, you'd just do:
#include <shared.h>
int main()
{
int x = func1();
char *str = func2(42);
cout << "x=" << x << " str=" << str << endl;
return 0;
}
Pretty straight forward, right?
When we use a shared library that is dynamically loaded by the code, it gets a fair bit more complex:
#include <shared.h>
typedef int (*ptrfunc1)();
typedef char * (*ptrfunc2)(int x);
int main()
{
SOMETYPE handle = loadlibrary("shared");
if (handle == ERROR_INDICATOR)
{
cerr << "Error: Couldn't load shared library 'shared'";
return 1;
}
ptrfunc1 pf1 = reinterpret_cast<ptrfunc1>(findfunc("func1"));
ptrfunc2 pf2 = reinterpret_cast<ptrfunc2>(findfunc("func2"));
int x = pf1();
char *str = pf2(42);
cout << "x=" << x << " str=" << str << endl;
return 0;
}
As you can see, the code suddenly got a lot more "messy". Never mind what hoops you have to jump through to find the constructor for a QObject, or worse, inherit from a QObject. In other words, if you are using Qt in your code, you are probably stuck with linking directly to "qt.lib" and your application WILL crash if a Qt environment isn't installed on the machine.
LoadLibrary calls should fail, then you can know if the dynamic library is present or not. Also with dynamic loading you get the function pointer from the dynamic library and if the pointer is null then the platform doesn't support that function on that platform.
On windows you have LoadLibrary API to load a dynamic lib. And GetProcAddress API to look up the desired function in that lib. If GetProcAddress returns NULL for that particular function that you are looking for that functionality is not present for that platform. You can log then and decide fallback.
I'm working in a project where I'm writing a plugin for a particular package.
This package implements a "new" method in one of its headers, and as such, I am unable to include <strstream> as it also implements "new".
The package sdk also includes a thinned out and very old version of boost, which means that I can't use the boost serialization classes. It is built on Qt for VS2008, and we are (required for this project) to be in VS2005, so I can't include Qt either.
I need to be able to get data from an externally running application, sending the data over TCPIP. What is the best way for me to serialize out the data from the source and read it back in with these limitations?
I'm currently tempted to make a struct which could contain all possible data that might be sent over, and then just copying the memory of that struct into a block of bytes which gets sent over, but this sounds like a bad approach to me.
Thanks,
Liron
Google Protobuf
boost.serialization is the way to go here. It is the most comprehensible serialization library for C++ that I know of, and it comes with support for standard containers.
You can extract the bytes from your data and pass it around, see a basic example.
QByteArray vectorToBin(const QVector<qint32> & vec)
{
size_t size = sizeof(qint16);
QByteArray result;
foreach(qint16 e, vec) {
for(int n = 0; n<=(size-1)*8; n+=8) {
char c = static_cast<char>((e >> n));
result.append(c);
}
}
return result;
}
QVector<qint32> binToVector(const QByteArray & bytes)
{
QVector<qint32> result;
size_t size = sizeof(qint16);
for(int i=0; i<bytes.size(); i+=size) {
qint16 e = ((bytes[i+1] & 0xff)<<8) | (bytes[i] & 0xff);
result << e;
}
return result;
}
I'm just getting started with DirectShow programming, and the 'C' nature of the DirectShow API is making my skin crawl. Endless return codes instead of exceptions, addref/release all over the place, functions taking pointer to pointers...
Are there any 'C++ friendly' wrappers for DirectShow programming that hide all the COM ugliness underneath.?
For 98% of DirectShow code, you should never see a call to AddRef or Release. Always use CComPtr<>. There are a few minor exceptions to this rule.
Also, understanding locking is important, as is CCritSec and CAutoLock; once again, I'd avoid manually locking CCritSec instances because it's a good way to deadlock an application.
Also, this code can be pretty handy:
#include <dxerr9.h>
...
HRESULT hr = S_OK;
... something goes wrong ...
CString err(::DXGetErrorString9(hr));
err += ::DXGetErrorDescription9(hr);
Lastly, make sure you're using the DShow eventing. All sorts of useful info comes in over events from a DShow graph, and it's surprising how many applications don't have this implemented (or have it implemented incorrectly).
A while ago, I wrote this, which I've sort of tossed together a few of the pitfalls of working with DShow. Sadly, I know these pitfalls because I've screwed up in almost all these ways.
CComPtr is a pretty handy one,
Beyond that you just have to live with things like the HRESULTS. I rather prefer them to exceptions ... find them much easier to work with ... each to their own.
It's worth knowing that DirectShow is implemented in quartz.dll, and DirectShow Editing Services in qedit.dll.
You can use those two with the Compiler COM Support from Visual C++ to write DirectShow client applications.
Here is a small play media_file example (compile with cl /D_UNICODE playfile.cpp):
#include <iostream>
#import <quartz.dll> rename_namespace("dshow")
#import <qedit.dll> rename_namespace("dshow")
// Required for CLSID_FilterGraph
#include <uuids.h>
#pragma comment(lib, "strmiids.lib")
int wmain(int argc, wchar_t* argv[])
{
using namespace dshow;
using namespace std;
if (argc != 2)
{
wcout << "Usage: play media_file" << endl;
return 1;
}
struct ComInitializer
{
ComInitializer()
{
::CoInitialize(0);
}
~ComInitializer()
{
::CoUninitialize();
}
} comInit;
try
{
IGraphBuilderPtr graphBuilder;
graphBuilder.CreateInstance(CLSID_FilterGraph);
graphBuilder->RenderFile(argv[1], 0);
IMediaControlPtr mediaControl = graphBuilder;
mediaControl->Run();
wcout << "Press Return to stop playback." << endl;
wcin.get();
mediaControl->Stop();
}
catch (const _com_error& err)
{
wcout << L"Error code: 0x" << hex << err.Error() << endl;
}
}
I've used this method in building a console DirectShow oggenc application.
If you don't find a wrapper then you can possibly benefit from the coding pattern discussed in my blog entry "B true, or B thrown! (Using the >> throwing pattern)".
Example from that blog entry (this can easily be adapted for COM, and in fact I first "invented" it for COM usage):
std::ostream& operator<<( std::ostream& stream, wchar_t const s[] )
{
Size const nBytes = wcstombs( 0, s, 0 );
(nBytes >= 0)
|| throwX( "wcstombs failed to deduce buffer size" );
Size const bufSize = nBytes + 1;
std::vector< char > buf( bufSize );
// The count of bytes written does not include terminating nullbyte.
wcstombs( &buf[0], s, bufSize )
>> Accept< IsNonNegative >()
|| throwX( "wcstombs failed to convert string" );
return (stream << &buf[0]);
}
There is some discussion at the blog, evidently everybody has his or her own preference for suitable operators etc., so I'm just throwing in the general principle here.
Oh, should add, instead of AddRef and Release calls, just use some COM smart pointer. There are so many of them around (e.g. in ATL/MFC, and the "intrinsic" one for Visual C++) that it's just pick and choose the one you like. Or create your own e.g. based on boost::intrusive_ptr.
Cheers & hth.,