I am trying to store large amount of data into multiple malloc array
I have three malloc array, two 2d char array and one int array. In a test case the array name are defined as:
cres=12163;
catm=41241;
matm = (char**) malloc(catm*sizeof(char*));
for(i=0;i<catm;i++)
matm[i]=(char*) malloc(5*sizeof(char));
mres = (char**) malloc(cres*sizeof(char*));
for(i=0;i<cres;i++)
mres[i]=(char*) malloc(5*sizeof(char));
mrin = (int*) malloc(cres*sizeof(int));
I read the data from a file. The data stored in these array if printed as it is stored in the these array is in right format. But when I try to retrieve data from the character arrays, after assigning value to the int array the character; array change the column length to 14 and the value is set to 8.50000000E-01.
I am using Linux Opensuse and g++ comiler.
Any Solution or alternate method to store large amount of data.
Sorry for all the confusion the blunder was on my part i was assigning the file-handling line pointer to all the values.
So matm is an array of char* with length catm. You then assign to its elements arrays of char of length 5. Then you do the same for res instead of atm.
Finally, you allocate and store in mrin an array of cres integers.
Almost certainly you are overflowing one of these arrays. You can use valgrind to figure out which, most likely automatically, by simply running valgrind ./a.out or whatever your program is called. It will print stack traces where memory errors occur.
You may simply have strings longer than 4 characters (plus the terminating null). You don't show the code where you populate the arrays.
Since you're using a C++ compiler, you should consider using C++ containers like std::vector<char> and std::string instead of raw C arrays which are error-prone as you have discovered.
OK, so I am going to take a crack at this... in C!
What you are making are arrays of pointers to char.
So two arrays of pointer to char, each holding 41241 pointers to char
One array holding pointers to int ( although why I have no idea since just declaring an array of int of size 12163 would do the trick.
Further you are declaring each entry on the char pointer array to be 5 chars which will hold a C style string of 4 bytes plus the null terminator.
char* strArray1 [41241] ;
char* strArray2 [41241] ;
int* intArray [12163] ;
for( int x=0 ; int < 41241;x++){
strArray1[x] = malloc(5*sizeof(char)) ;
strcopy("fred",strArray1[x]);
}
for( int x=0 ; int < 41241;x++){
strArray2[x] = malloc(5*sizeof(char)) ;
strcopy("Tom",strArray2[x]);
}
for(x=0;x<12163;x++){
inArray[x*] = rand() % 50 ;
}
for( int x=0 ; int < 41241;x++){
printf(" This entry = %s \n",strArray1[x]) ;
}
for( int x=0 ; int < 41241;x++){
printf(" This entry = %s \n",strArray2[x]) ;
}
for( int x=0 ; int < 12163;x++){
printf(" This entry = %i \n",intArray[x*]) ;
}
DO NOT try and get cute with C as it will bite you in the ass every time.
Need to have an array full of random numbers within range. Code:
void fillArray(int *arr){
for(int i=0;i<(sizeof arr);i++){
arr[i] = rand() % 2 - 2;
}
}
int *arrPoint(int *arr, int max){
for (int i=0;i<max;i++){
printf("%d is %d\n",i,arr[i]);
}
return arr;
}
int main(int argc, char *argv[]){
srand ( time(NULL) );
int arr_f[15];
fillArray(arr_s);
arrPoint(arr_s, 15);
system("PAUSE");
return EXIT_SUCCESS;
}
Output:
0 is -2
1 is -1
2 is -2
3 is -1
4 is 0
5 is 0
6 is 4633240
7 is 2686652
8 is 1973724226
9 is 1974338216
10 is 2686716
11 is 1973744850
12 is 8
13 is 1973752206
14 is 1973752162
Press any key to continue . . .
What the hell? Putting rand() % 2 into brackets doesn't help either. What these numbers are and how do I get rid of them?
P.S. Tried this in crappy Dev-C++ and Code::Blocks with the same result. Need the program to be small sized (putting it to dropbox), so no, I can't use 100mb boost lib.
void fillArray(int *arr){
for(int i=0;i<(sizeof arr);i++){
Since arr is a pointer to an integer, sizeof arr is equivalent to sizeof (int *), which is apparently 4 (32-bits) on your platform. That's clearly not what you want. You only pass fillArray a pointer to the first element of the array.
If you need the number of elements in the array a pointer points to, you need to pass that information. The C and C++ languages provides no way to tell how many bytes a pointer points to given just the pointer.
You do it correctly in arrPoint. Do it that way in fillArray.
Arrays don't work like you think. You cannot use sizeof to get the array size magically. Arrays decay to pointers to their first element when passed as a function argument, and you must provide the size information separately:
int main()
{
int * arr = new int[20];
the_function(arr, 20);
//...
delete[] arr;
}
void (int * arr, std::size_t size)
{
for (std::size_t i = 0; i != size; ++i)
{
// something with arr[i]
}
}
Much better yet, use std::vector<int> to spare yourself all this headache, and tons of further headache that you may not even have thought about.
(There is an exception to my statement: For an automatic array in the local scope, int b[10], sizeof(b) will indeed return 10 * sizeof(int). But that is only because the compiler knows the array size. The size information is not passed around, and it is not available at runtime.)
In fillArray (sizeof arr) is the size of a pointer, not the size of your array.
Your problem is the sizeof operator. arr is a pointer to an int, which is usually 4 or 8 bytes long. You must pass the length of the array (not the size in bytes) as an additional parameter to the function.
Try using _countof(arr) instead of sizeof(arr)._countof returns size of array, not pointers-be aware of this.
Is there a way to cross over all elements in integer array using pointer ( similiar to using pointer to cross over string elements).I know that integer array is not NULL terminated so when I try to cross over array using pointer it overflows.So I added NULL as a last element of an array and it worked just fine.
int array[7]={1,12,41,45,58,68,NULL};
int *i;
for(i=array;*i;i++)
printf("%d ",*i);
But what if one of the elements in array is 0 ,that will behave just as NULL.Is there any other way that will implement pointer in crossing over all elements in integer array?
In general, no unless you pick a sentinel value that's not part of the valid range of the data. For example, the valid range might be positive numbers, so you can use a negative number like -1 as a sentinel value that indicates the end of the array. This how C-style strings work; the NULL terminator is used because it's outside of the valid range of integers that could represent a character.
However, it's usually better to somehow pair up the array pointer with another variable that indicates the size of the array, or another pointer that points one-past-the-end of the array.
In your specific case, you can do something like this:
// Note that you don't have to specify the length of the array.
int array[] = {1,12,41,45,58,68};
// Let the compiler count the number of elements for us.
int arraySize = sizeof(array)/sizeof(int);
// or int arraySize = sizeof(array)/sizeof(array[0]);
int main()
{
int* i;
for(i = array; i != array + arraySize; i++)
printf("%d ",*i);
}
You can also do this:
int arrayBegin[] = {1,12,41,45,58,68};
int* arrayEnd = arrayBegin + sizeof(arrayBegin)/sizeof(arrayBegin[0]);
int main()
{
int* i;
for(i = arrayBegin; i != arrayEnd; i++)
printf("%d ",*i);
}
But given only a pointer, no you can't know how long the array it points to is. In fact, you can't even tell if the pointer points to an array or a single object! (At least not portably.)
If you have functions that must accept an array, either have your function require:
the pointer and the size of the array pointed by the pointer,
or two pointers with one pointing to the first element of the array and one pointing one-past-the-end of the array.
I'd like to give some additional advice: Never use some kind of sentinel/termination value in arrays for determining their bounds. This makes your programs prone to error and is often the cause for security issues. You should always store the length of arrays to limit all operations to their bounds and test against that value.
In C++ you have the STL and its containers.
In C you'll effectively end up using structures like
typedef struct t_int_array
{
size_t length;
int data[1]; /* note the 1 (one) */
} int_array;
and a set of manipulation functions like this
int_array * new_int_array(size_t length)
{
int_array * array;
/* we're allocating the size of basic t_int_array
(which already contains space for one int)
and additional space for length-1 ints */
array = malloc( sizeof(t_int_array) + sizeof(int) * (length - 1) );
if(!array)
return 0;
array->length = length;
return array;
}
int_array * concat_int_arrays(int_array const * const A, int_array const * const B);
int_array * int_array_push_back(int_array const * const A, int const value);
/* and so on */
This method will make the compiler align the t_int_array struct in a way, that it's optimal for the targeted architecture (also with malloc allocation), and just allocating more space in quantities of element sizes of the data array element will keep it that way.
The reason that you can iterate across a C-style string using pointers is that of the 256 different character values, one has been specifically reserved to be interpreted as "this is the end of the string." Because of this, C-style strings can't store null characters anywhere in them.
When you're trying to use a similar trick for integer arrays, you're noticing the same problem. If you want to be able to stop at some point, you'll have to pick some integer and reserve it to mean "this is not an integer; it's really the end of the sequence of integers." So no, there is no general way to take an array of integers and demarcate the end by a special value unless you're willing to pick some value that can't normally appear in the string.
C++ opted for a different approach than C to delineate sequences. Instead of storing the elements with some sort of null terminator, C++-style ranges (like you'd find in a vector, string, or list) store two iterators, begin() and end(), that indicate the first element and first element past the end. You can iterate over these ranges by writing
for (iterator itr = begin; itr != end; ++itr)
/* ... visit *itr here ... */
This approach is much more flexible than the C-string approach to defining ranges as it doesn't rely on specific properties of any values in the range. I would suggest opting to use something like this if you want to iterate over a range of integer values. It's more explicit about the bounds of the range and doesn't run into weird issues where certain values can't be stored in the range.
Apart from the usual suggestion that you should go and use the STL, you can find the length of a fixed array like this:
int array[6]={1,12,41,45,58,68};
for (int i = 0; i < sizeof(array) / sizeof(array[0]); ++i)
{ }
If you use a templated function, you can implicitly derive the length like this:
template<size_t len> void func(int (&array)[len])
{
for (int i = 0; i < len; ++i) { }
}
int array[6]={1,12,41,45,58,68};
func(array);
If 0 is a value that may occur in a normal array of integers, you can specify a different value:
const int END_OF_ARRAY = 0x80000000;
int array[8]={0,1,12,41,45,58,68,END_OF_ARRAY};
for (int i = 0; array[i] != END_OF_ARRAY; ++i)
{ }
If every value is a possibility, or if none of the other approaches will work (for example, a dynamic array) then you have to manage the length separately. This is how strings that allow embedded null characters work (such as BSTR).
In your example you are using (or rather abusing) the NULL macro as a sentinel value; this is the function of the NUL('\0') character in a C string, but in the case of a C string NUL is not a valid character anywhere other than as the terminal (or sentinel) value .
The NULL macro is intended to represent an invalid pointer not an integer value (although in C++ when implicitly or explicitly cast to an int, its value is guaranteed to be zero, and in C this is also almost invariably the case). In this case if you want to use zero as the sentinel value you should use a literal zero not NULL. The problem is of course that if in this application zero is a valid data value it is not suitable for use as a sentinel.
So for example the following might suit:
static const int SENTINEL_VALUE = -1 ;
int array[7] = { 1, 12, 41, 45, 58, 68, SENTINEL_VALUE } ;
int* i ;
for( i = array; *i != SENTINEL_VALUE; i++ )
{
printf( "%d ", *i ) ;
}
If all integer values are are valid data values then you will not be able to use a sentinel value at all, and will have to use either a container class (which knows its length) or iterate for the known length of the array (from sizeof()).
Just to pedanticize and expand a little on a previous answer: in dealing with integer arrays in C, it's vanishingly rare to rely on a sentinel value in the array itself. No(1) sane programmer does that. Why not? Because by definition an integer can hold any value within predefined negative/positive limits, or (for the nowadays-not-unusual 32-bit integer) 0 to 0xffffff. It's not a good thing to redefine the notion of "integer" by stealing one of its possible values for a sentinel.
Instead, one always(1) must(1) rely on a controlling up-to-date count of integers that are in the array. Suppose we are to write a C function
that returns an int pointer to the first array member whose value is greater than the function's argument or, if there's no such member, returns NULL (all code is untested):`
int my_int_array[10]; // maximum of 10 integers in my_int_array[], which must be static
int member_count = 0; // varies from 0 to 10, always holds number of ints in my_int_array[]
int *
first_greater_than ( int val ) {
int i;
int *p;
for ( i = 0, p = my_int_array; i < member_count; ++i, ++p ) {
if ( *p > val ) {
return p;
}
}
return NULL;
}
Even better is also to limit the value of i to never count past the last possible member of my_int_array[], i.e., it never gets bigger than 9, and p never points at my_int_array[10] and beyond:
int my_int_array[10]; // maximum of 10 integers in my_int_array[], which must be static
int member_count = 0; // varies from 0 to 10, always holds number of ints in my_int_array[]
int *
first_greater_than ( int val ) {
#define MAX_COUNT sizeof(my_int_array)/sizeof(int)
int i;
int* p;
for ( i = 0, p = my_int_array; i < member_count && i < MAX_COUNT; ++i, ++p ) {
if ( *p > val ) {
return p;
}
}
return NULL;
}
HTH and I apologize if this is just too, too elementary.
--pete
Not strictly true but believe it for now
In ANSI C it's very easy and shorter than solution before:
int array[]={1,12,41,45,58,68}, *i=array;
size_t numelems = sizeof array/sizeof*array;
while( numelems-- )
printf("%d ",*i++);
Another way is to manage array of pointers to int:
#include <stdlib.h>
#include <stdio.h>
#define MAX_ELEMENTS 10
int main() {
int * array[MAX_ELEMENTS];
int ** i;
int k;
// initialize MAX_ELEMENTS,1 matrix
for (k=0;k<MAX_ELEMENTS;k++) {
array[k] = malloc(sizeof(int*));
// last element of array will be NULL pointer
if (k==MAX_ELEMENTS-1)
array[k] = NULL;
else
array[k][0] = k;
}
// now loop until you get NULL pointer
for (i=array;*i;i++) {
printf("value %i\n",**i);
}
// free memory
for (k=0;k<MAX_ELEMENTS;k++) {
free(array[k]);
}
return 0;
}
In this way loop condition is totally independent from the values of integers. But... for this to work you must use 2D array (matrix) instead of ordinary 1D array. Hope that helps.
I am finding it difficult to determine the length of the columns in a 2D unsigned short pointer array. I have done memory allocation correctly as far as I know. and can print them correctly.
plz see the following code segment:
int number_of_array_index_required_for_pointer_abc=3;
char A[3][16];
strcpy(A[0],"Hello");
strcpy(A[1],"World");
strcpy(A[2],"Tumanicko");
cout<<number_of_array_index_required_for_pointer_abc*sizeof(unsigned short)<<endl;
unsigned short ** pqr=(unsigned short **)malloc(number_of_array_index_required_for_pointer_abc*sizeof(unsigned short));
for(int i=0;i<number_of_array_index_required_for_pointer_abc;i++)
{
int ajira = strlen(A[i])*sizeof(unsigned short);
cout<<i<<" = "<<ajira<<endl;
pqr[i]=(unsigned short *)malloc(ajira);
cout<<"alocated pqr[i]= "<<sizeof pqr<<endl;
int j=0;
for(j=0;j<strlen(A[i]);j++)
{
pqr[i][j]=(unsigned short)A[i][j];
}
pqr[i][j]='\0';
}
for(int i=0;i<number_of_array_index_required_for_pointer_abc;i++)
{
//ln= (sizeof pqr[i])/(sizeof pqr[0]);
//cout<<"Size of pqr["<<i<<"]= "<<ln<<endl;
// I want to know the size of the columns i.e. pqr[i]'s length instead of finding '\0'
for(int k=0;(char)pqr[i][k]!='\0';k++)
cout<<(char)pqr[i][k];
cout<<endl;
}
You're almost there. You have this loop:
for(int k=0;(char)pqr[i][k]!='\0';k++) ...
Once this loop is done, k will have the length of the row. So this will give you the length of pqr[i] (not including the null terminator):
int k;
for (k=0; pqr[i][k] != 0; k++)
;
cout<<"The length is "<< k <<endl;
Edit:
You now added that you want to know the size even if the null terminator is not there. There is no way to do that. You will need to either have some kind of terminator, or store the size somewhere. If you use vector<unsigned short>, it will store the size for you. Since it also handles allocation and deallocation, it's the recommended choice.
</Edit>
Note that you have two errors in your allocation:
pqr is an array of pointers, but you're allocating a size of C*sizeof(unsigned short). that should be C*sizeof(unsigned short *) instead.
You're not allocating memory for the null terminator at the end of each string: You should be allocating (strlen(A[i])+1) * sizeof(unsigned short) for each string.
You have a bug at this line:
pqr[i][j]='\0';
At this point j is equal to strlen(A[i]) - which is outside the bounds you setup for pqr:
int ajira = strlen(A[i])*sizeof(unsigned short);
pqr[i]=(unsigned short *)malloc(ajira);
pqr[i] goes from [0] to [strlen(A[i])-1] so writing to pqr[i][strlen(A[i])] overflows the array. The compiler won't pick up on this as you allocated the memory yourself.
The solution to that bug is to do malloc(ajira+sizeof(unsigned short))
Edited after comments
I'm trying to build a char array for storing the return value of a function. In the following function the data is stored in *****valv**. How to build a extern variable to access the data?
int credis_lrange(REDIS rhnd, const char *key,
int start, int end, char ***valv)
{
int rc;
if ((rc = cr_sendfandreceive(rhnd, CR_MULTIBULK, "LRANGE %s %d %d\r\n",
key, start, end)) == 0)
{
*valv = rhnd->reply.multibulk.bulks;
rc = rhnd->reply.multibulk.len;
}
return rc;
}
Solution:
char **elements;
int size = credis_lrange(this->redis,"object_2",600,603,&elements);
for (int i=0; i<size;i++) {
cout << "element: " << elements[i] << endl;
}
Thanks to everyone!
char ***element[size];
Is not exactly a 3D array, but an array of size elements of pointers-to-pointers-to-pointers to char.
Use any one of the following:
char e[ D1 ][ D2 ][ D3 ]; /* D1, D2, D3 are integral constants */
char *e[ D2 ][ D3 ];
char e[][ D2 ][ D3 ];
Also, you can pass it on by simply speficying e as the argument to your function.
On further reading, it appears that the parameter is not really a 3D array but a pointer to an array of C-style strings. Note, the syntax may be the same, the intent is different.
In that case, you'll need to do two things:
Specify the number of strings you want to store in the array
For each string
Allocate memory
Copy string data to the char array
And finally, you'll be passing in the address of this array of strings on to the credis_lrange function.
I only found one hit on Google for this, but it looks like the cr_sendfandreceive function allocates its rhnd->reply.multibulk.bulks member, so you don't actually have to pass it back (since you were passed rhnd in the first place).
If you want to copy it, then you would declare elements as a char** and pass its address (or use references), and then inside the method you would clone the bulks member and also each string in the array (in a loop).
In Arduino Studio environment, for MCUs programming like ST or ESP32 using PSRAM, one can allocate dynamic memory for a 3D multiarray of char array, sized 255, like so:
char*** 3d_array = (char***) heap_caps_malloc( sizeof(char)*255*(size_x*size_y*size_z), MALLOC_CAP_SPIRAM);
for X86/ X64 architectures a 3D multiaaray of char array, sized 255 can be declared as follows:
char*** 3d_array = (char***) malloc( sizeof(char)*255*(size_x*size_y*size_z));