Python ints passed into C++ functions as longs using ctypes get truncated - c++

I'm developing on a Mac using MacOSX 10.8.2 with the latest xcode and the stock python interpreter.
Here's some code I put into fails.cpp
#include <iostream>
using namespace std;
extern "C" {
void mysort(long *data, long data2) {
cout << hex << *data << ' ' << data2 << endl;
}
}
Here's some code to call it from python that I put in fails.py:
import ctypes
sort_dll = ctypes.CDLL("fails.dylib")
mysort = ctypes.CFUNCTYPE(None, ctypes.POINTER(ctypes.c_long), ctypes.c_long)(sort_dll.mysort)
a = ctypes.c_long(0x987654321)
mysort(ctypes.byref(a), 0x123456789)
I compile and run with
c++ -arch x86_64 -o fails.o -c fails.cpp && g++ -o fails.dylib -dynamiclib fails.o && python fails.py
The result is:
987654321 23456789
Why is the 64-bit integer passed by value being truncated to 32 bits? Surprisingly, the pointer to a 64-bit long isn't truncated.

I suspect it is because the Python ctypes library has decided that c_long is 32-bits in size. There are some hints in the docs:
Represents the C signed int datatype. The constructor accepts an optional integer initializer; no overflow checking is done. On platforms where sizeof(int) == sizeof(long) it is an alias to c_long.
This code will tell you how big c_long actually is:
import ctypes
print ctypes.sizeof(ctypes.c_long())
The value the pointer references wouldn't be truncated since ctypes only has to marshall the pointer itself. It's possible the pointer is being truncated, but it doesn't matter as the high bits are all zero anyway. It's also possible that ctypes decided that ctypes.POINTER is 64-bit. You can find out by modifying the above example just slightly.

Related

How to convert a 32 bit signed integer value to 64 bit signed integer equivalent in C

I have a library that is compiled to use 32 bit signed integer. When other applications compile theirs with a flag e.g: -DODBC64 it promotes the same type I have used in my library to a 64 bit signed integer.
e.g:
#ifdef ODBC64
typedef sint64 SLEN;
#else
#define SLEN int
#endif
When the application passes reference to my library as :
SLEN count;
mylibraryfunction(&count);
the values returned to application looks like these:
sizeof(SLEN) = 8
sizeof(SLEN) in my library = 4
m_AffectedRows BEFORE = 0x3030303030303030
m_AffectedRows AFTER = 0x3030303000000000 0
You can see that the assignment from my lib is copying 4 bytes (value 0).
I need to know a way to reset the upper 4 bytes to 0.
e.g:
0x0000000000000000
I have tried both static_cast and reinterpret_cast, but none are helpful.
I made a MCVE where I resembled what happens in OPs case.
I even didn't need an extra library for this, just two translation units (resulting in two object files).
First lib.cc:
#include <cstdint>
extern "C" void func(int32_t *pValue);
void func(std::int32_t *pValue)
{
*pValue = 0;
}
Second prog.cc:
#include <iostream>
#include <iomanip>
// how prog.cc "knows" func():
extern "C" void func(int64_t *pValue);
int main()
{
int64_t value = 0x0123456789ABCDEFull;
std::cout << "value before: " << std::hex << value << '\n';
func(&value);
std::cout << "value after : " << std::hex << value << '\n';
return 0;
}
Compiler errors? No. Each translation unit uses prototype of func() conformant.
Linker errors? No. The symbols match, anything else is beyond view of linker.
I must admit I had to use extern "C" to achieve this. Otherwise, at least, the C++ name mangling had prevented the proper linking. (When I became aware of this, I made code in C.)
Output:
value before: 123456789abcdef
value after : 123456700000000
Live Demo on wandbox
This is very dangerous! Any use of any extern symbol should use a 100 % compatible declaration. (And yes, C and C++ provide various ways to shoot into your own foot.)
Imagine what would happen if the lib.cc function func() would write int64_t where the prog.cc would pass a pointer to int32_t: Out of bound access with possible more disastrous consequences.
Your library can't access the upper 4 bytes, but you can before you call it. So try to initialize it with 0's first:
SLEN count = 0; // initialize with 0's
mylibraryfunction(&count);

Is there a way for a Eclipse compiler to show address fields from variables as an integer value? [duplicate]

This question already has answers here:
Printing the addresses of variables in decimal
(2 answers)
Closed 4 years ago.
I'm currently doing projects on pointer variables in general and I'm using a virtual machine to mimic linux from my 64 bit OS. Keep in mind, this is C++. One thing I'm currently struggling with are using hexadecimal values. Rather than using that as a reference to where certain values are located, I want the output to simply show up as an integer. I tried type casting with int() but this isn't working. Is there a way to work around this for Linux Ubuntu on virtual machines using Eclipse compilers to show those address fields at integers?
Thanks in advance.
EDIT:!!!!
int x = 0;
int * p = &x;
std::cout << p << std::endl;
std::cout << &x << std::endl;
return 0;
libraries that were used include iostream and nothing else.
Since you are on a 64 bit computer, the addresses will be 64 bits long, while an int is typically only 32 bits. Therefore a cast to an int will lose precision, which is not permitted. long long on the other hand is an integer type that holds 64 bits, so casting to that would call an overload of operator<< that by default prints in decimal.

Why can't I assign a 300 or less digits integer with the GMP library?

I have this little code with a big integer:
int main()
{
mpz_clears;
mpf_clears;
mpz_class a;
a=130637788386308069046861449260260571291678458515671364436805375996643405376682659882150140370119739570729696093810308688223886144781635348688713392214619435345787110033188140509357535583193264801721383236152235906221860161085667905721519797609516199295279707992563172152784123713076584911245631751842633105652153513186684155079079372385923352208421842040532051768902602579344300869529063620569896872621227499787666438515766191438772844982077590564825560915004123788524793626088046688154064374425340131073611440941376503643793012676721171310302652283866154666880487476095144107907540698417260347310774677574064007810935083421437442654204085311165490420993090855747058348793757769523336364858305492927387281493416741250273266926840468154062676311322374882380011804120628601384191;
return 0;
and got this error, so that I cannot do anything with the integer...
error: integer literal is too large to be represented in any
integer type
I am on mac, compiled with the tool of xcode and use the last 6.0.0a gmp library.
What did I miss?
Using GMP doesn't magically make C/C++ understand literals larger than the base language spec. Luckily, with C++11 with user defined literals, GMP has a literal syntax defined so the literal type is mpz_class and not int. Just add _mpz to the end of the integer literal, e.g.
a=130637788386308069046861449260260571291678458515671364436805375996643405376682659882150140370119739570729696093810308688223886144781635348688713392214619435345787110033188140509357535583193264801721383236152235906221860161085667905721519797609516199295279707992563172152784123713076584911245631751842633105652153513186684155079079372385923352208421842040532051768902602579344300869529063620569896872621227499787666438515766191438772844982077590564825560915004123788524793626088046688154064374425340131073611440941376503643793012676721171310302652283866154666880487476095144107907540698417260347310774677574064007810935083421437442654204085311165490420993090855747058348793757769523336364858305492927387281493416741250273266926840468154062676311322374882380011804120628601384191_mpz;
C++ types can't handle the big number which you are passing and its throwing that error.
see this answer for the details.
What range of values can integer types store in C++
and in the document its clearly mentioned that.
Construct an mpz_class. All the standard C++ types may be used, except long long and long double, and all the GMP C++ classes can be used, although conversions from mpq_class and mpf_class are explicit. Any necessary conversion follows the corresponding C function, for example double follows mpz_set_d (see Assigning Integers).
Just because you use a big-integer class does not mean that the compiler will suddenly become capable of working with ultra-large integer literals. Your suffix-less literal is treated as an int by the compiler, as usual. And it is too large for int.
You should either use C++11 literal support and supply your literal value with _mpz suffix, or represent your values as strings.
Well, even with the _mpz option, there might be a limit to the number of digits you can assign. I reduced to this size the number of digits and I could store the integer:
130637788386308069046861449260260571291678458515671364436805375996643405376682659882150140370119739570729696093810308688223886144781635348688713392214619435345787110033188140509357535583193264801721383236152235906221860161085667905721519797609516199295279707992563172152784123713076584911245631751842633105652153513186684155079079372385923352208421842040532051768902602579344300869529063620569896872621227499787666438515766191438772844982077590564825560915004123788524793626088046688154064374425340131073611440941376503643793012676721171310302652283866154666880487476095144107907540698417260347310774677574064007810935083421437442654204085311165490420993090855747058348793757769523336364858305492927387281493416741250273266926840468154062676311322374882380011804120628601384191443885715160918938894478991212554338474935909274442208280226020332302710637502228813106477844481700372333640604211874260838332822176968781235304962300880267221110401606508880971834777831402249082184410637749400023282419270071233303228854128584088916313729295257781669730936517951304701393525757057288415991731506781288275420005462290126288405806701552761743270631625705587885293887371663631869096785158480771725887503591755610
To operate with big numbers you better look at crypto-software, like openssl/bn.h
$ sudo apt-get install libssl-dev
MUL.cpp
#include <openssl/bn.h>
#include <ctime>
#include <string>
#include <stdio.h>
#include <iostream>
#include <time.h>
using namespace std;
// man bn
int main(int argc, char *argv[]) {
BN_CTX *ctx = BN_CTX_new( );
BIGNUM *a = NULL;
BIGNUM *b = NULL;
BIGNUM *mul = BN_new( );
BN_dec2bn(&a, argv[1]);
BN_dec2bn(&b, argv[2]);
BN_mul( mul, a, b, ctx);
string mul_str = BN_bn2dec( mul );
cout << mul_str << endl;
BN_free( a );
BN_free( b );
BN_free( mul );
BN_CTX_free( ctx );
return 0;
}
Thereafter,
$ g++ -v -o MUL MUL.cpp -lcrypto -lrt
$ ./MUL 1111111111111111111111 11111111111111111111111111111111111111
12345679012345679012344444444444444444320987654320987654321
Or your numbers:
$ ./MUL 130637788386308069046861449260260571291678458515671364436805375996643405376682659882150140370119739570729696093810308688223886144781635348688713392214619435345787110033188140509357535583193264801721383236152235906221860161085667905721519797609516199295279707992563172152784123713076584911245631751842633105652153513186684155079079372385923352208421842040532051768902602579344300869529063620569896872621227499787666438515766191438772844982077590564825560915004123788524793626088046688154064374425340131073611440941376503643793012676721171310302652283866154666880487476095144107907540698417260347310774677574064007810935083421437442654204085311165490420993090855747058348793757769523336364858305492927387281493416741250273266926840468154062676311322374882380011804120628601384191443885715160918938894478991212554338474935909274442208280226020332302710637502228813106477844481700372333640604211874260838332822176968781235304962300880267221110401606508880971834777831402249082184410637749400023282419270071233303228854128584088916313729295257781669730936517951304701393525757057288415991731506781288275420005462290126288405806701552761743270631625705587885293887371663631869096785158480771725887503591755610130637788386308069046861449260260571291678458515671364436805375996643405376682659882150140370119739570729696093810308688223886144781635348688713392214619435345787110033188140509357535583193264801721383236152235906221860161085667905721519797609516199295279707992563172152784123713076584911245631751842633105652153513186684155079079372385923352208421842040532051768902602579344300869529063620569896872621227499787666438515766191438772844982077590564825560915004123788524793626088046688154064374425340131073611440941376503643793012676721171310302652283866154666880487476095144107907540698417260347310774677574064007810935083421437442654204085311165490420993090855747058348793757769523336364858305492927387281493416741250273266926840468154062676311322374882380011804120628601384191443885715160918938894478991212554338474935909274442208280226020332302710637502228813106477844481700372333640604211874260838332822176968781235304962300880267221110401606508880971834777831402249082184410637749400023282419270071233303228854128584088916313729295257781669730936517951304701393525757057288415991731506781288275420005462290126288405806701552761743270631625705587885293887371663631869096785158480771725887503591755610 130637788386308069046861449260260571291678458515671364436805375996643405376682659882150140370119739570729696093810308688223886144781635348688713392214619435345787110033188140509357535583193264801721383236152235906221860161085667905721519797609516199295279707992563172152784123713076584911245631751842633105652153513186684155079079372385923352208421842040532051768902602579344300869529063620569896872621227499787666438515766191438772844982077590564825560915004123788524793626088046688154064374425340131073611440941376503643793012676721171310302652283866154666880487476095144107907540698417260347310774677574064007810935083421437442654204085311165490420993090855747058348793757769523336364858305492927387281493416741250273266926840468154062676311322374882380011804120628601384191443885715160918938894478991212554338474935909274442208280226020332302710637502228813106477844481700372333640604211874260838332822176968781235304962300880267221110401606508880971834777831402249082184410637749400023282419270071233303228854128584088916313729295257781669730936517951304701393525757057288415991731506781288275420005462290126288405806701552761743270631625705587885293887371663631869096785158480771725887503591755610
17066231754465807402900420358384027625063828147296398970430735667107038565643532131029262436685775911914313380489264979150809498798501933158462145225528180551769789575574797447397445413420626589337764710733729094394489883580640065014326441503312557193537990957436917477463646907709648408216434453480535184067598892956346170907569131377903528411111853111366291979626086698383298307345834392212231046634063413666384508324419179368057389725477538706904589815650381926976015993762890040156790405503832003268981872633663593094738685190702324170124859881504272985658187233430078072074850856278428071856485129710354193089376603399949840797557269752404324626163227377490362755801388382649764643331010423816641928437135475590362260226952505888042595117331291587671568162632704073970431619934337538342580157600657762301792637641467188915728074681445126309949503670453577866959321057221058159530832970764856624653513410124322745428166790635701352502052256269558858525352939994015481804509533547301113993717358659764948833130929069642122698202129401970681574382516490304799923426506769877124709868229514010141819989355368364493152148025681360589017051459966742937386231341355101280349206933137558553397562834428892415453233148641176771563712134361104022807550303488698075859517457925640610491054979460976360319484563783286684527208609132950696416209270231736324410954950684352328008061594161733388271145413633779170942608933174781023093090166766097051317310022316050630811133583173184115645711370920550992076973922926411319105318826733544096711620052335678566140497081147253812896893420327059981797167126306338464682511753397155370448528529202208913712194212784573851259886750498508574755936271159501806286495680771291521438702898980393172655721876464226059755372992679372981146919582948873581755274102131988783345665739059331911729901433053990191883389538118335419875393994271684373681810615128035637305671352255611195822986723919966346614504211249964663541945984204985492046339446890342152602121983526036026284992402505355803385348589918367728082706040916985152635360901742191941842971992737551826097028139759998758915104121256359451340107345440703495802418055755094797175689110387208266417084593991014867282339917152309333902180775075816625489223370772738533933334770778564779976389719378964794125379024410157550786106085604852610915830834787247811239349553909171864850989169291577662971436559192161855924800900992398387001394175369368663291775977076397743722156192299105428781790818602044847522848431713923633708651868973304037943629982141197617707337073274179185426770132582538432486796714335942857724787044441406231875204080386533209509526701082724875806709758857092820176146255706651998803661272142775642520442391227251506425870387373189142488674432150155028643969780961833270806721943761674451332734094107418048448339730770862124109349834151523986716674077669261444236368571532558762173381119345015880991848768022539566069235387298433970531174140055934895491488406387322959686152870801506904417823703916926860536028705138822353298033104149394326119785714010793712166616781321617880422232478270993974660928438969267385851248329604086387551705361922068424614692617313923883706742816371720982187645987693446127391744743054010747707122729452000008024595790675203131690448164324982521914771679392295264057374903374105401693996933614023173316709739350993750161786196236271822749116371726461907551046689900873564981257387360500771251705433693927287093968802056959550818280473764856549882949501840084257149510400015730796750717240359458767805153426658794187889587166234478619647814190297313764503421633608458361966472100

Passing pointer of unsigned int to pointer of long int

I have a sample code which is working properly in 32 bit system, but when I cross compile it for 64-bit system and try to run on 64 bit Machine, it behaves differently.
Can anyone tell me why this is happening?
#include <usr/include/time.h>
#include <sys/time.h>
void func(time_t * inputArg)
{
printf("%ld\n",*inputArg);
}
int main()
{
unsigned int input = 123456;
func((time_t *)&input);
}
Here "time_t" is a type defined in linux system library header file which is of type "long int".
This code is working fine with a 32-bit system but it isn't with 64-bit.
For 64-bit I have tried this:
#include <usr/include/time.h>
#include <sys/time.h>
void func(time_t * inputArg)
{
printf("%ld\n",*inputArg);
}
int main()
{
unsigned int input = 123456;
time_t tempVar = (time_t)input
func(&tempVar);
}
Which is working fine, but I have used the first method in my whole application a number of times. Any alternate solutions would be appreciated.
can anyone tell me why this is happening?
Dereferencing an integer pointer of different size than the type of the pointed object has undefined behaviour.
If the pointed to integer is smaller than the pointers pointed type, you will read unrelated bytes as part of the dereferenced number.
Your fix works because you pass a pointer to an object of proper type, but consider that your input cannot represent all of the values that time_t can.
Best fix is to use the proper type initially. Use time_t as the input.
Your "fixed" code has a cast that lets the compiler convert your unsigned int value to a time_t. Your original code assumes that they're identical.
On your 32-bit system they are identical, so you get lucky. On your 64-bit system you find out what happens when you invoke Undefined Behavior.
In other words, both C and C++ allow you to cast pointers to whatever you want, but it's up to you to make sure such casts are safe.
thank you for response.
Actually I found my mistake when I printed sizeof 'long int' in 32bit machine and 64bit machine.
32bit machine :
sizeof(int) = 32bit & sizeof(long int) = 32 bit & sizeof(long long int) = 64 bit
64bit machine:
sizeof(int) = 32bit & sizeof(long int) = 64 bit & sizeof(long long int) = 64 bit

Why do I get wrong conversion from hex to decimal with strtoul function in Visual Studio compiler?

I am converting a string from hex to decimal. The problem is that in Visual Studio compiler the conversion returns a wrong value. However when I compile the same code in a Mac at the terminal using the g++ compiler, the value is returned correctly.
Why this is happening?
#include <string>
#include <iostream>
using namespace std;
int main()
{
string hex = "412ce69800";
unsigned long n = strtoul( hex.c_str(), nullptr, 16 );
cout<<"The value to convert is: "<<hex<<" hex\n\n";
cout<<"The converted value is: "<<n<<" dec\n\n";
cout<<"The converted value should be: "<<"279926183936 dec\n\n";
return 0;
}
output:
Because in Windows long is a 32-bit type, unlike most Unix/Linux implementations which use LP64 memory model in which long is 64 bits. The number 412ce69800 has 39 bits and inherently it can't be stored in a 32-bit type. Read compiler warnings and you'll know the issue immediately
C standard only requires long to have at least 32 bits. C99 added a new long long type with at least 64 bits, and that's guaranteed in all platforms. So if your value is in 64-bit type's range, use unsigned long long or uint64_t/uint_least64_t and strtoull instead to get the correct value.