Use 64 bit lib in 32 bit code - c++

I am wondering whether there exists a way to use a 64 bit lib in a 32bit executable.
I can design the 64 bit lib accordingly hence expose specific APIs for that purpose
that will have explicit 32 bit aligned variable in them, if that will help.
Basically we have a large piece of software which is compiled 32 bit, and for the time being
we do not want to recompile it as 64 bit, since it will cause a lot of headache.
Now within that existing code we'd like to access a new API that we are building, that new API is hiding a 64bit implementation, but the API itself 32bit on the "outside"
We've been toying around with ideas such as remote execution or passing the data via sockets but that will be simply to slow for our purposes, using shared memory as IPC between 32/64 bits would be quite challenging to develop hence not worth the development effort.
Any ideas?

Related

How can a 32 bit client communicate with a 64 bit server if long type is passed?

We have a 32-bit C++ GUI application running on 32 bit windows 7. We are planning to migrate our server with C++ apps to 64-bit linux. We have noticed that long types are larger in 64 bit. This will be incompatible with the client-server message passing from 64-bit to 32-bit. What is a good way to solve this incompatibility? Do we need to change the code? How? or Do we use a third party software to do the conversion? What software is it?
That's why there is the standardized int32_t and uint32_t types etc., so you could specifically select type depending on your needs.
It might be quite a lot of work to replace all long types to int32_t in all structures you send, especially if it's a big project, but you (hopefully) only have to do it once. Another way of solving this problem is to serialize the data into a text-format and then deserialize it on the receiving side, this has the big advantage that it will make the communication almost completely platform independent.

c++ FIFO implementation for VERY LARGE 10G buffer

I have a need to implement a 10G ring buffer (FIFO). I need to write this program in C++ for integration (have to integrate with C++ API) reasons even though I think it will be easier in Java with it's FileChannel class. I know I'll need to use a memory-mapped file, just not sure how to do it in C++. I figure I'll have to use a buffer and re-map it for different parts of the underlying file? Something like that. Anyway, if anyone has some example code (non MFC), that would be great.
I'll be implementing this on 32 bit hardware running Windows 7 32 bit.
Thanks.
One way to create a memory mapped file, that works both on Windows and UNIX, is to use the Boost C++ libraries, see here. This code handles large files (> 4 GB) on most 32-bit platforms; see here.

Using 32-bit library in 64-bit C++ program

Is there any way how use an old 32-bit static library *.a in a 64-bit system.
The is no chance to obtain a source code of this old library to compile it again.
I also do not want to use -m32 in gcc, because the program use many 64bit libraries.
Thanks.
That depends entirely on the platform on which you're running. OS X on PowerPC, for example, that would "Just Work".
On x86 platforms, you can't link a 32-bit library into a 64-bit executable. If you really need to use that library, you'll need to start a separate 32-bit process to handle your calls to the library, and use some form of IPC to pass those calls between your 64-bit application and that helper process. Be forewarned: this is a lot of hassle. Make sure that you really need that library before starting down this road.
On the x86/x86_64 platform, you can't do this. I mean, maybe you could if you wrote custom assembly language wrappers for each and every 32 bit function you wanted to call. But that's the only way it's even possible. And even if you were willing to do that work I'm not sure it would work.
The reason for this is that the calling conventions are completely different. The x864_64 platform has many more registers to play with, and the 64-bit ABI (Application Binary Interface, basically how parameters are passed, how a stack frame is set up and things like that) standards for all of the OSes make use of these extra registers for parameter passing and the like.
This makes the ABI of 32-bit and 64-bit x86/x86_64 systems completely incompatible. You'd have to write a translation layer. And it's possible the 32-bit ABI allows 32-bit code to fiddle around with CPU stuff that 64-bit code is not allowed to fiddle with, and that would make your job even harder since you'd be required to restore the possibly modified state before returning to the 64-bit code.
And that's not even talking about this issue of pointers. How do you pass a pointer to a data structure that's sitting at a 64-bit address to 32-bit code?
Simple answer: You can't .
You need to use -m32 in order to load a 32-bit library.
Probably your best approach is to create a server wrapping the library. Then a 64-bit application can use IPC (various methods, e.g. sockets, fifos) in order to communicate to and from the process hosting the library.
On Windows this would be called out-of-process COM. I don't know that there's a similar framework on unix, but the same approach will work.

64 bit features in a 32 bit application?

I have a 32 bit application which I plan to run on 64bit Windows 7.
At this stage I cannot convert the entire application to 64 bit due to dependencies to thirdparty functionality.
However, I would like to have access to the xmm9-xmm15 registers in my SSE optimizations and also use the additional registers which 64 bit cpus provides in general while executing my application.
Is this possible to achieve with some compiler flag?
It seems to me that the best way would be to divide your program in more than one executable. The EXE compiled as 64bit can communicate with another 32-bit EXE used the 32-bit thirdparty DLL which you need. You will have some overhead in the communication and will have to implement Starting/Stopping of depended process, but you will have clear program architecture.
If you develop native C++ application you can implement the second EXE for example as the COM++ out-of-process object (LocalServer or even LocalService). You can consider the way to implement the COM server in C# (see here). Sometimes that way could simplify the implementation and you can use the advantages of both .NET and native programming.
Yes, you can. See this example for how.
As Christopher pointed out, this technique will not always work, but for this express purpose, to jump into a few lines of handcrafted assembly and make use of more registers, then put the result somewhere and then jump back to 32 bit mode, it should work out just fine.

What is the big deal with BUILDING 64-bit versions of binaries?

There are a ton of drivers & famous applications that are not available in 64-bit. Adobe for instance does not provider a 64-bit Flash player plugin for Internet Explorer. And because of that, even though I am running 64-bit Vista, I have to run 32-bit IE. Microsoft Office, Visual Studio also don't ship in 64-bit AFAIK.
Now personally, I haven't had much problems building my applications in 64-bit. I just have to remember a few rules of thumb, e.g. always use SIZE_T instead of UINT32 for string lengths etc.
So my question is, what is preventing people from building for 64-bit?
If you are starting from scratch, 64-bit programming is not that hard. However, all the programs you mention are not new.
It's a whole lot easier to build a 64-bit application from scratch, rather than port it from an existing code base. There are many gotchas when porting, especially when you get into applications where some level of optimization has been done. Programmers use lots of little assumptions to gain speed, and these are not always easy to quickly port to 64-bit. A few examples I've had to deal with:
Proper alignment of elements within a struct. As data sizes change, assumptions that certain fields in a struct will be aligned on an optimal memory boundary may fail
Length of long integers change, so if you pass values over a socket to another program that may not be 64-bit, you need to refactor your code
Pointer lengths change, as so hard to decipher code written be a guru that has left the company become a little trickier to debug
Underlying libraries will also need to have 64-bit support to properly link. This is a large part of the problem of porting code if you rely on any libraries that are not open source
In addition to the things in #jvasak's post, the major thing that can causes bugs:
pointers are larger than ints - a huge amount of code makes the assumption that the sizes are the same.
Remember that Windows will not even allow an application (whether 32-bit or 64-bit) to handle pointers that have an address above 0x7FFFFFFF (2GB or above) unless they have been specially marked as "LARGE_ADDRESS_AWARE" because so many applications will treat the pointer as a negative value at some point and fall over.
The biggest issues that I've run into porting our C/C++ code to 64 bit is support from 3rd party libraries. E.g. there is currently only 32 bit versions of the Lotus Notes API and also MAPI so you can't even link against them.
Also since you can't load a 32 bit DLL into your 64 bit process you get burnt again trying to load things dynamically. We ran into this problem again trying to support Microsoft Access under 64 bit. From wikipedia:
The Jet Database Engine will remain
32-bit for the foreseeable future.
Microsoft has no plans to natively
support Jet under 64-bit versions of
Windows
Just a guess, but I would think a large part of it would be support - If Adobe compiles the 64 bit version, they have to support it. Even though it may be a simple compile switch, they'd still have to run through a lot of testing, etc, followed by training their support staff to respond correctly, when they do run into issues fixing them either results in a new version of the 32 bit binary or a branch in the code, etc. So while it seems simple, for a large application it can still end up costing a lot.
Another reason that a lot of companies have not gone through the effort of creating 64 bit versions is simply they don't need to.
Windows has WoW64 (Windows on Windows 64 bit) and Linux can have the 32 bit libraries available alongside the 64 bit. Both of these allow us to run 32 bit applications in 64 bit environments.
As long as the software is able to run in this way, there is not a major incentive to convert to 64 bit.
Exceptions to this are things such as device drivers as they are tied in deeper with the operating systems and cannot run in the 32 bit layer that the x86-64/AMD64 based 64-bit operating systems offer (IA64 is unable to do this from what I understand).
I agree with you on flash player though, I am very disappointed in Adobe that they have not updated this product. As you have pointed out, it does not work properly in 64 bit requiring you to run the 32 bit version of Internet Explorer.
I think it is a strategic mistake on Adobe's part. Having to run the 32 bit browser for flash player is an inconvenience for users, and many will not understand this solution. This could lead to developers being apprehensive about using flash. The most important thing for a web site is to make sure everyone can view it, solutions that alienate users are typically not popular ones. Flash's popularity was fed by its own popularity, the more sites that used it, the more users had it on their systems, the more users that had it on their systems, the more sites were willing to use it.
The retail market pushes these things forward, when a general consumer goes to buy a new computer, they aren't going to know they don't need a 64 bit OS they are going to get it either because they hear it is the latest and greatest thing, the future of computing, or just because they don't know the difference.
Vista has been out for about 2 years now, and Windows XP 64-bit was out before that. In my mind that is too long for a major technology such as Flash to not be upgraded if they want to hold on to their market. It may have to do with Adobe taking over Macromedia and this is a sign that Adobe does not feel Flash is part of their future, I find it hard to believe as I think Flash and Dreamweaver were the top parts of what they got out of Macromedia, but then why have they not updated it yet?
It is not as simple as just flipping a switch on your compiler. At least, not if you want to do it right. The most obvious example is that you need to declare all your pointers using 64-bit datatypes. If you have any code which makes assumptions about the size of these pointers (e.g. a datatype which allocates 4 bytes of memory per pointer), you'll need to change it. All this needs to have been done in any libraries you use, too. Further, if you miss just a few then you'll end up with pointers being down-casted and at the wrong location. Pointers are not the only sticky point but are certainly the most obvious.
Primarily a support and QA issue. The engineering work to build for 64-bit is fairly trivial for most code, but the testing effort, and the support cost, don't scale down the same way.
On the testing side, you'll still have to run all the same tests, even though you "know" they should pass.
For a lot of applications, converting to a 64-bit memory model doesn't actually give any benefit (since they never need more than a few GB of RAM), and can actually make things slower, due to the larger pointer size (makes every object field twice as large).
Add to that the lack of demand (due to the chicken/egg problem), and you can see why it wouldn't be worth it for most developers.
Their Linux/Flash blog goes some way to explain why there isn't a 64bit Flash Player as yet. Some is Linux-specific, some is not.