I am trying to store large amount of data into multiple malloc array
I have three malloc array, two 2d char array and one int array. In a test case the array name are defined as:
cres=12163;
catm=41241;
matm = (char**) malloc(catm*sizeof(char*));
for(i=0;i<catm;i++)
matm[i]=(char*) malloc(5*sizeof(char));
mres = (char**) malloc(cres*sizeof(char*));
for(i=0;i<cres;i++)
mres[i]=(char*) malloc(5*sizeof(char));
mrin = (int*) malloc(cres*sizeof(int));
I read the data from a file. The data stored in these array if printed as it is stored in the these array is in right format. But when I try to retrieve data from the character arrays, after assigning value to the int array the character; array change the column length to 14 and the value is set to 8.50000000E-01.
I am using Linux Opensuse and g++ comiler.
Any Solution or alternate method to store large amount of data.
Sorry for all the confusion the blunder was on my part i was assigning the file-handling line pointer to all the values.
So matm is an array of char* with length catm. You then assign to its elements arrays of char of length 5. Then you do the same for res instead of atm.
Finally, you allocate and store in mrin an array of cres integers.
Almost certainly you are overflowing one of these arrays. You can use valgrind to figure out which, most likely automatically, by simply running valgrind ./a.out or whatever your program is called. It will print stack traces where memory errors occur.
You may simply have strings longer than 4 characters (plus the terminating null). You don't show the code where you populate the arrays.
Since you're using a C++ compiler, you should consider using C++ containers like std::vector<char> and std::string instead of raw C arrays which are error-prone as you have discovered.
OK, so I am going to take a crack at this... in C!
What you are making are arrays of pointers to char.
So two arrays of pointer to char, each holding 41241 pointers to char
One array holding pointers to int ( although why I have no idea since just declaring an array of int of size 12163 would do the trick.
Further you are declaring each entry on the char pointer array to be 5 chars which will hold a C style string of 4 bytes plus the null terminator.
char* strArray1 [41241] ;
char* strArray2 [41241] ;
int* intArray [12163] ;
for( int x=0 ; int < 41241;x++){
strArray1[x] = malloc(5*sizeof(char)) ;
strcopy("fred",strArray1[x]);
}
for( int x=0 ; int < 41241;x++){
strArray2[x] = malloc(5*sizeof(char)) ;
strcopy("Tom",strArray2[x]);
}
for(x=0;x<12163;x++){
inArray[x*] = rand() % 50 ;
}
for( int x=0 ; int < 41241;x++){
printf(" This entry = %s \n",strArray1[x]) ;
}
for( int x=0 ; int < 41241;x++){
printf(" This entry = %s \n",strArray2[x]) ;
}
for( int x=0 ; int < 12163;x++){
printf(" This entry = %i \n",intArray[x*]) ;
}
DO NOT try and get cute with C as it will bite you in the ass every time.
Related
Normally it's a question about a buffer with a null-terminated string, but we can extrapolate it to a general case.
I have a big array of a fixed length, let's say 10:
char outputArray[10] = {'-','-','-','-','-','-','-','-','-','-'};
And I have some other (Edited: smaller) array (in my case it's a char buffer with null terminator) with a variable length. Let's say it's a buffer of 6 elements, but the actual length is indicated by another variable.
char inputArray[10] = {'h','i','/0',...some other values, i'm not interested in};
int arrLength = 2; // For my task it means a strlen(inputArray);
How to put a small array at the end of a big array, to get this:
outputArray = {'-','-','-','-','-','-','-','-','h','i'} // the null terminator isn't important, it's not about the strings, it's about arrays.
Constraints:
I can't use std, so only "native" solutions (it's for Arduino)
C++11
Code should be memory and time efficient (some elegant algorithm without too much loops and too much temporary variables or calculations please)
Thank you in advance
Edited:
Thank to #ThomasWeller for an answer. I have a small precision though. What if I need to clean all the elements before the inserted array?
For example I had some garbage
{'a','k','$','-','n','"','4','i','*','%'};
And I need to get
{'-','-','-','-','-','-','-','-','h','i'};
Do I need 2 loops? First to reset an array and the second one to set the actual result?
It can be done with a single for loop and a single variable:
for (char i=0; i<arrLength; i++)
{
outputArray[10-arrLength+i] = inputArray[i];
}
If you make arrLength a char instead of an int, this will even save you 2 bytes of memory ;-)
Use memset() to set all memory to an initial value and then memcpy() the contents at the end:
char output[10];
char input[10] = "hi\0------";
char arrLength = 2;
void setup() {
memset(output, '-', 10); // "----------"
memcpy(output+10-arrLength, input, arrLength);
Serial.begin(9600);
Serial.write(output, 10);
}
void loop() { }
I am an Arduino noob attempting to select a random name from this array:
char ns[ ][3] = {"Carlos Alberto Castronovo","Tom Erbaugh","Caterina De Giacco","Di Puglia Pugliese Filomena","Manishwar Dhillon","Mel Richards","Connie Hvidtfeldt","Amy Namehere","Tim Beck","Sanil Sethi","Christophe Lavault","Steven Grimes","Jessica Serra","Mariateresa Petrucci","Patricia Anderson","Felma Roberto Cinco","Mai Ahmed","Tobe Levy","Indah Suspriati Wibawa","Dain Turgeon Orbe","Li Wang","Ed Clark","Elodie da Silva","Jason Garcia","Allan Litswa","Pietro Zubani","Cyril Jeanpierre","Kate Denali Princess","Maria Pilar Gl","Jefferson Ricarte","Adam Reed","László Lipták","Thalia Dbl","Maria Jose Calle Salas","William Alexander","Nicole Richardson","Andrea Hescher","Ismail Sholeh","Simone Spacci","Jason Jankow"};
But I receive this error, and I am not sure about different data types and how to approach fixing this array:
error: initializer-string for array of chars is too long
Is there something basic that I am missing?
It is exactly what it is informing: your character strings are way too long to fit in your char array, so your compiler is telling you that it will not proceed any further.
You can make it work by enhancing the size of your arrays like this:
char ns[ ][30] = //... ;
The 30 here is just to represent your biggest char string; it needs to have the size of your largest predefined char string + 1 (so that the null terminating character \0 can be added). For example, if your biggest string were "apple", your array would need to be of, at least, length 6.
You can iterate through these strings by doing this, for example:
int array_items = sizeof(ns) / sizeof(*ns); // this will gives you the amount of items stored in your array
int i;
int j;
for (i = 0; i < array_items; ++i) {
size_t strSize = strlen(ns[i]); // strSize now contains, if ns[i] contained the example of apple, 5
for (j = 0; j < strSize; ++j) {
printf("%c", ns[i][j]);
}
printf("\n");
}
That [3] means each string is limited to a maximum of 3 characters. And since one has to be the null terminator, it really means two. Your strings are a lot longer than that. Choose a number that's big enough to accommodate all of them.
ETA: #JLF: are you my long lost brother? :)
I am trying to call a function that takes char** as a parameter. Its job is to fill an array of strings (i.e. an array of char*). I know the max length of the strings, and I can pass the max number to fill as another parameter, so I was hoping to stack allocate it like this:
fill_my_strings(char** arr_str, int max_str); // function prototype
char fill_these[max_strings][max_chars_per_string]; // allocating chars
fill_my_strings(fill_these, max_strings); // please fill them!
Of course, I get the "cannot convert char[max_strings][max_chars_per_string] to char**" error.
I know this is some subtle (or not-so-subtle) problem with my understanding of the difference between arrays and pointers. I'm just not sure why it's not possible to pass this block of memory to something wanting a char** and have it fill in my stack-allocated chars. Could somebody please explain if this is possible, or if not, why not?
Is it possible to call a function like this without calling malloc / new?
The simple answer to your question is no; a two dimensional array is different than a pointer-to pointer type. Arrays decay to pointers to their first element, but pointers actually are that value.
The difference between these types is clear, if you cast both to char*
int x;
char *arr_pp[] = {"foo", "bar", "baz"};
char arr_2d[][4] = {"foo", "bar", "baz"};
char *cp = (char*)arr_pp;
for(x=0; x<3; x++)
printf("%d ", cp[x]);
printf("\n");
cp = (char*)arr_2d;
for(x=0; x<3; x++)
printf("%d ", cp[x]);
printf("\n");
The output (on my computer) is:
-80 -123 4
102 111 111
Where the first row is gibberish formed by the fact that I'm printing an address cast into bytes, and the second row is the ascii values of "foo".
In a function taking a char ** the compiler can't know to decay array types, which don't actually contain pointers.
Suppose you have n pointers to strings of m-1 maximum characters (m characters including the NULL).
So, in pure C:
sizeof(char[n][m]) will return n*m.
sizeof(char**) will return the size of a pointer in your architecture, probably 32 (if x86) or 64 (if x86_64).
char[n][m] actually allocates the n*m byte contiguously. char** allocates a single pointer. This pointer references a memory stripe of *n bytes. Each of these n pointers points to a memory stripe of m characters.
So, considering that sizeof(char) == u, if you declare char a[n][m], when you use a[i][j], the compiler understands *(a + i*m*u + j*u).
So, considering that sizeof(char *) == w, if you declare char **a, when you use a[i][j], the compiler understands ((a + i*w) + j*w).
Completely different data management.
The closes thing you could do to handle your special case is to create a char** variable, and populate it with the addresses of your stack allocated matrix.
char **tmp = malloc(max_strings * sizeof(char *));
int i;
for(i = 0; i < max_strings; i++){
tmp[i] = &(fill_these[i][0]); //you probably can't reference a char[][] with a single index - not confirmed
}
I am not sure why fill_my_strings() need a char** parameter. From your example, caller have already allocated the memory from stack. So using a char* should be OK.
But if you want to use char** or you can't modify the fill_my_strings() function, try following example code:
void fill_my_strings(char** arr_str, int max_chars_per_string, int max_strings)
{
for(int i = 0; i < max_strings; ++i)
{
//Make sure you have enough space
memcpy(*arr_str, "ABCD", sizeof("ABCD"));
*arr_str += max_chars_per_string;
}
}
char fill_these[max_strings][max_chars_per_string];
char* pointer = (char*)fill_these;
fill_my_strings(&pointer, max_strings, max_chars_per_string);
The obvious thing to do is build an index
In c use something like:
char string_list[num_strings][str_length];
// ...
char**index = calloc( (num_strings+1), sizeof(*index) ); // calloc insures NULL termination
for (int i=0; i<num_strings; ++i) {
index[i] = string_list[i]
}
In c++ prefer new[] to calloc;
Is there a way to cross over all elements in integer array using pointer ( similiar to using pointer to cross over string elements).I know that integer array is not NULL terminated so when I try to cross over array using pointer it overflows.So I added NULL as a last element of an array and it worked just fine.
int array[7]={1,12,41,45,58,68,NULL};
int *i;
for(i=array;*i;i++)
printf("%d ",*i);
But what if one of the elements in array is 0 ,that will behave just as NULL.Is there any other way that will implement pointer in crossing over all elements in integer array?
In general, no unless you pick a sentinel value that's not part of the valid range of the data. For example, the valid range might be positive numbers, so you can use a negative number like -1 as a sentinel value that indicates the end of the array. This how C-style strings work; the NULL terminator is used because it's outside of the valid range of integers that could represent a character.
However, it's usually better to somehow pair up the array pointer with another variable that indicates the size of the array, or another pointer that points one-past-the-end of the array.
In your specific case, you can do something like this:
// Note that you don't have to specify the length of the array.
int array[] = {1,12,41,45,58,68};
// Let the compiler count the number of elements for us.
int arraySize = sizeof(array)/sizeof(int);
// or int arraySize = sizeof(array)/sizeof(array[0]);
int main()
{
int* i;
for(i = array; i != array + arraySize; i++)
printf("%d ",*i);
}
You can also do this:
int arrayBegin[] = {1,12,41,45,58,68};
int* arrayEnd = arrayBegin + sizeof(arrayBegin)/sizeof(arrayBegin[0]);
int main()
{
int* i;
for(i = arrayBegin; i != arrayEnd; i++)
printf("%d ",*i);
}
But given only a pointer, no you can't know how long the array it points to is. In fact, you can't even tell if the pointer points to an array or a single object! (At least not portably.)
If you have functions that must accept an array, either have your function require:
the pointer and the size of the array pointed by the pointer,
or two pointers with one pointing to the first element of the array and one pointing one-past-the-end of the array.
I'd like to give some additional advice: Never use some kind of sentinel/termination value in arrays for determining their bounds. This makes your programs prone to error and is often the cause for security issues. You should always store the length of arrays to limit all operations to their bounds and test against that value.
In C++ you have the STL and its containers.
In C you'll effectively end up using structures like
typedef struct t_int_array
{
size_t length;
int data[1]; /* note the 1 (one) */
} int_array;
and a set of manipulation functions like this
int_array * new_int_array(size_t length)
{
int_array * array;
/* we're allocating the size of basic t_int_array
(which already contains space for one int)
and additional space for length-1 ints */
array = malloc( sizeof(t_int_array) + sizeof(int) * (length - 1) );
if(!array)
return 0;
array->length = length;
return array;
}
int_array * concat_int_arrays(int_array const * const A, int_array const * const B);
int_array * int_array_push_back(int_array const * const A, int const value);
/* and so on */
This method will make the compiler align the t_int_array struct in a way, that it's optimal for the targeted architecture (also with malloc allocation), and just allocating more space in quantities of element sizes of the data array element will keep it that way.
The reason that you can iterate across a C-style string using pointers is that of the 256 different character values, one has been specifically reserved to be interpreted as "this is the end of the string." Because of this, C-style strings can't store null characters anywhere in them.
When you're trying to use a similar trick for integer arrays, you're noticing the same problem. If you want to be able to stop at some point, you'll have to pick some integer and reserve it to mean "this is not an integer; it's really the end of the sequence of integers." So no, there is no general way to take an array of integers and demarcate the end by a special value unless you're willing to pick some value that can't normally appear in the string.
C++ opted for a different approach than C to delineate sequences. Instead of storing the elements with some sort of null terminator, C++-style ranges (like you'd find in a vector, string, or list) store two iterators, begin() and end(), that indicate the first element and first element past the end. You can iterate over these ranges by writing
for (iterator itr = begin; itr != end; ++itr)
/* ... visit *itr here ... */
This approach is much more flexible than the C-string approach to defining ranges as it doesn't rely on specific properties of any values in the range. I would suggest opting to use something like this if you want to iterate over a range of integer values. It's more explicit about the bounds of the range and doesn't run into weird issues where certain values can't be stored in the range.
Apart from the usual suggestion that you should go and use the STL, you can find the length of a fixed array like this:
int array[6]={1,12,41,45,58,68};
for (int i = 0; i < sizeof(array) / sizeof(array[0]); ++i)
{ }
If you use a templated function, you can implicitly derive the length like this:
template<size_t len> void func(int (&array)[len])
{
for (int i = 0; i < len; ++i) { }
}
int array[6]={1,12,41,45,58,68};
func(array);
If 0 is a value that may occur in a normal array of integers, you can specify a different value:
const int END_OF_ARRAY = 0x80000000;
int array[8]={0,1,12,41,45,58,68,END_OF_ARRAY};
for (int i = 0; array[i] != END_OF_ARRAY; ++i)
{ }
If every value is a possibility, or if none of the other approaches will work (for example, a dynamic array) then you have to manage the length separately. This is how strings that allow embedded null characters work (such as BSTR).
In your example you are using (or rather abusing) the NULL macro as a sentinel value; this is the function of the NUL('\0') character in a C string, but in the case of a C string NUL is not a valid character anywhere other than as the terminal (or sentinel) value .
The NULL macro is intended to represent an invalid pointer not an integer value (although in C++ when implicitly or explicitly cast to an int, its value is guaranteed to be zero, and in C this is also almost invariably the case). In this case if you want to use zero as the sentinel value you should use a literal zero not NULL. The problem is of course that if in this application zero is a valid data value it is not suitable for use as a sentinel.
So for example the following might suit:
static const int SENTINEL_VALUE = -1 ;
int array[7] = { 1, 12, 41, 45, 58, 68, SENTINEL_VALUE } ;
int* i ;
for( i = array; *i != SENTINEL_VALUE; i++ )
{
printf( "%d ", *i ) ;
}
If all integer values are are valid data values then you will not be able to use a sentinel value at all, and will have to use either a container class (which knows its length) or iterate for the known length of the array (from sizeof()).
Just to pedanticize and expand a little on a previous answer: in dealing with integer arrays in C, it's vanishingly rare to rely on a sentinel value in the array itself. No(1) sane programmer does that. Why not? Because by definition an integer can hold any value within predefined negative/positive limits, or (for the nowadays-not-unusual 32-bit integer) 0 to 0xffffff. It's not a good thing to redefine the notion of "integer" by stealing one of its possible values for a sentinel.
Instead, one always(1) must(1) rely on a controlling up-to-date count of integers that are in the array. Suppose we are to write a C function
that returns an int pointer to the first array member whose value is greater than the function's argument or, if there's no such member, returns NULL (all code is untested):`
int my_int_array[10]; // maximum of 10 integers in my_int_array[], which must be static
int member_count = 0; // varies from 0 to 10, always holds number of ints in my_int_array[]
int *
first_greater_than ( int val ) {
int i;
int *p;
for ( i = 0, p = my_int_array; i < member_count; ++i, ++p ) {
if ( *p > val ) {
return p;
}
}
return NULL;
}
Even better is also to limit the value of i to never count past the last possible member of my_int_array[], i.e., it never gets bigger than 9, and p never points at my_int_array[10] and beyond:
int my_int_array[10]; // maximum of 10 integers in my_int_array[], which must be static
int member_count = 0; // varies from 0 to 10, always holds number of ints in my_int_array[]
int *
first_greater_than ( int val ) {
#define MAX_COUNT sizeof(my_int_array)/sizeof(int)
int i;
int* p;
for ( i = 0, p = my_int_array; i < member_count && i < MAX_COUNT; ++i, ++p ) {
if ( *p > val ) {
return p;
}
}
return NULL;
}
HTH and I apologize if this is just too, too elementary.
--pete
Not strictly true but believe it for now
In ANSI C it's very easy and shorter than solution before:
int array[]={1,12,41,45,58,68}, *i=array;
size_t numelems = sizeof array/sizeof*array;
while( numelems-- )
printf("%d ",*i++);
Another way is to manage array of pointers to int:
#include <stdlib.h>
#include <stdio.h>
#define MAX_ELEMENTS 10
int main() {
int * array[MAX_ELEMENTS];
int ** i;
int k;
// initialize MAX_ELEMENTS,1 matrix
for (k=0;k<MAX_ELEMENTS;k++) {
array[k] = malloc(sizeof(int*));
// last element of array will be NULL pointer
if (k==MAX_ELEMENTS-1)
array[k] = NULL;
else
array[k][0] = k;
}
// now loop until you get NULL pointer
for (i=array;*i;i++) {
printf("value %i\n",**i);
}
// free memory
for (k=0;k<MAX_ELEMENTS;k++) {
free(array[k]);
}
return 0;
}
In this way loop condition is totally independent from the values of integers. But... for this to work you must use 2D array (matrix) instead of ordinary 1D array. Hope that helps.
I am finding it difficult to determine the length of the columns in a 2D unsigned short pointer array. I have done memory allocation correctly as far as I know. and can print them correctly.
plz see the following code segment:
int number_of_array_index_required_for_pointer_abc=3;
char A[3][16];
strcpy(A[0],"Hello");
strcpy(A[1],"World");
strcpy(A[2],"Tumanicko");
cout<<number_of_array_index_required_for_pointer_abc*sizeof(unsigned short)<<endl;
unsigned short ** pqr=(unsigned short **)malloc(number_of_array_index_required_for_pointer_abc*sizeof(unsigned short));
for(int i=0;i<number_of_array_index_required_for_pointer_abc;i++)
{
int ajira = strlen(A[i])*sizeof(unsigned short);
cout<<i<<" = "<<ajira<<endl;
pqr[i]=(unsigned short *)malloc(ajira);
cout<<"alocated pqr[i]= "<<sizeof pqr<<endl;
int j=0;
for(j=0;j<strlen(A[i]);j++)
{
pqr[i][j]=(unsigned short)A[i][j];
}
pqr[i][j]='\0';
}
for(int i=0;i<number_of_array_index_required_for_pointer_abc;i++)
{
//ln= (sizeof pqr[i])/(sizeof pqr[0]);
//cout<<"Size of pqr["<<i<<"]= "<<ln<<endl;
// I want to know the size of the columns i.e. pqr[i]'s length instead of finding '\0'
for(int k=0;(char)pqr[i][k]!='\0';k++)
cout<<(char)pqr[i][k];
cout<<endl;
}
You're almost there. You have this loop:
for(int k=0;(char)pqr[i][k]!='\0';k++) ...
Once this loop is done, k will have the length of the row. So this will give you the length of pqr[i] (not including the null terminator):
int k;
for (k=0; pqr[i][k] != 0; k++)
;
cout<<"The length is "<< k <<endl;
Edit:
You now added that you want to know the size even if the null terminator is not there. There is no way to do that. You will need to either have some kind of terminator, or store the size somewhere. If you use vector<unsigned short>, it will store the size for you. Since it also handles allocation and deallocation, it's the recommended choice.
</Edit>
Note that you have two errors in your allocation:
pqr is an array of pointers, but you're allocating a size of C*sizeof(unsigned short). that should be C*sizeof(unsigned short *) instead.
You're not allocating memory for the null terminator at the end of each string: You should be allocating (strlen(A[i])+1) * sizeof(unsigned short) for each string.
You have a bug at this line:
pqr[i][j]='\0';
At this point j is equal to strlen(A[i]) - which is outside the bounds you setup for pqr:
int ajira = strlen(A[i])*sizeof(unsigned short);
pqr[i]=(unsigned short *)malloc(ajira);
pqr[i] goes from [0] to [strlen(A[i])-1] so writing to pqr[i][strlen(A[i])] overflows the array. The compiler won't pick up on this as you allocated the memory yourself.
The solution to that bug is to do malloc(ajira+sizeof(unsigned short))
Edited after comments