I am having trouble with a certain piece of documentation from MSDN. I am using C++ (or C, rather) to connection to an SQL Server instance via ODBC. Take at the code sample at the bottom of this piece of documentation.
Notice there is a function in the sample called AllocParamBuffer(). The documentation describes what it should do, but doesn't provide any further help. Could someone please give me a few pointers (no pun intended) as to how I could replicate the definition of this function for this particular case, or, better yet, show it could be done? I'm at a real roadblock, and I can't find any assistance elsewhere.
Any help would be greatly appreciated.
Thank you for your time.
You are referring to:
// Call a helper function to allocate a buffer in which to store the parameter
// value in character form. The function determines the size of the buffer from
// the SQL data type and parameter size returned by SQLDescribeParam and returns
// a pointer to the buffer and the length of the buffer.
AllocParamBuffer(DataType, ParamSize, &PtrArray[i], &BufferLenArray[i]);
All this does is allocate some memory one presumes with malloc (since the later free calls) to store the input parameter (PtrArray[i]) then set the buffer length BufferLenArray[i] (i.e. the amount of memory allocated for PtrArrayp[i]).
We'd only be guessing how it calculates how much memory to allocate since the amount required in this case will differ depending on the DataType and ParamSize returned by SQLDescribeParameter. The guess work is down to the fact all the parameters are bound as SQL_C_CHAR and some of them might not be string columns e.g., they could be dates.
All you need to do is malloc some memory, assign the pointer to PtrArray[i] and set the amount allocated in BufferLenArray[i].
Related
Would it be at all possible (I don't care about practicality or usefulness) to write a C or C++ program that monitored memory usage in the following, very basic way?
Given that declaring a variable without assigning it a value results in it having the value of whatever is already at its memory location, one could create a large array (thousands or millions of elements) and leave all the values unassigned. Then to see if any of these elements have been overwritten, we would simply need to repeatedly compare their current values to a previous value.
I highly doubt this would be as simple as I posited above. Assuming my doubt is well-founded, wherein would the problem lie and, more importantly, would it be something we could circumvent with some creative or esoteric code? I imagine that the problem would be attributable to something along the lines of the declared, uninitialized elements being not allowing other system processes to write to their memory address. Please give me some pointers! (heehee) Thanks.
Lets say your program is in C
Creating a large array is limited to the extent free memory is allowed and how the OS limits you.
So let's say you created a pretty large array (uninitialized).
Now that memory is given to your process(program you ran) and no other process can access it ! (It's OS role to avoid such things , basic requirements of Virtualization).
So as no other process can access its value won't be changed once its allocated to you.
I'm writing a fairly straightforward function that sends an array over to a file descriptor. However, in order to send the data, I need to append a one byte header.
Here is a simplified version of what I'm doing and it seems to work:
void SendData(uint8_t* buffer, size_t length) {
uint8_t buffer_to_send[length + 1];
buffer_to_send[0] = MY_SPECIAL_BYTE;
memcpy(buffer_to_send + 1, buffer, length);
// more code to send the buffer_to_send goes here...
}
Like I said, the code seems to work fine, however, I've recently gotten into the habit of using the Google C++ style guide since my current project has no set style guide for it (I'm actually the only software engineer on my project and I wanted to use something that's used in industry). I ran Google's cpplint.py and it caught the line where I am creating buffer_to_send and threw some comment about not using variable length arrays. Specifically, here's what Google's C++ style guide has to say about variable length arrays...
http://google-styleguide.googlecode.com/svn/trunk/cppguide.xml#Variable-Length_Arrays_and_alloca__
Based on their comments, it appears I may have found the root cause of seemingly random crashes in my code (which occur very infrequently, but are nonetheless annoying). However, I'm a bit torn as to how to fix it.
Here are my proposed solutions:
Make buffer_to_send essentially a fixed length array of a constant length. The problem that I can think of here is that I have to make the buffer as big as the theoretically largest buffer I'd want to send. In the average case, the buffers are much smaller, and I'd be wasting about 0.5KB doing so each time the function is called. Note that the program must run on an embedded system, and while I'm not necessarily counting each byte, I'd like to use as little memory as possible.
Use new and delete or malloc/free to dynamically allocate the buffer. The issue here is that the function is called frequently and there would be some overhead in terms of constantly asking the OS for memory and then releasing it.
Use two successive calls to write() in order to pass the data to the file descriptor. That is, the first write would pass only the one byte, and the next would send the rest of the buffer. While seemingly straightforward, I would need to research the code a bit more (note that I got this code handed down from a previous engineer who has since left the company I work for) in order to guarantee that the two successive writes occur atomically. Also, if this requires locking, then it essentially becomes more complex and has more performance impact than case #2.
Note that I cannot make the buffer_to_send a member variable or scope it outside the function since there are (potentially) multiple calls to the function at any given time from various threads.
Please let me know your opinion and what my preferred approach should be. Thanks for your time.
You can fold the two successive calls to write() in your option 3 into a single call using writev().
http://pubs.opengroup.org/onlinepubs/009696799/functions/writev.html
I would choose option 1. If you know the maximum length of your data, then allocate that much space (plus one byte) on the stack using a fixed size array. This is no worse than the variable length array you have shown because you must always have enough space left on the stack otherwise you simply won't be able to handle your maximum length (at worst, your code would randomly crash on larger buffer sizes). At the time this function is called, nothing else will be using the further space on your stack so it will be safe to allocate a fixed size array.
I was asked this question in an interview- "how much memory does a function use?". So I tried to answer by saying you could add up all the memory taken by all the data variables , data structures it instantiates- for example add 4 bytes for long, 1 for char , 4 for int, 32 bits for a pointer on 32 bits system, and adding any inputs that were dynamically allotted. The interviewer was not happy with my answer.
I am learning C++, and will appreciate any insight.
Question is quite undefined. A function itself will occupy just the space for its activation record from the caller, for parameters and for its local variables on the stack. According to architecture the activation record will contain things like saved registers, address to return when the function is called and whatever.
But a function can allocate how much memory it requires on the heap so there is no a precise answer.
Oh in addition, if the function is recursive then it could use a lot of memory, always because of activation records which are needed between each call.
From point of view of static behavior,
1. Data used by it - Sum of all variables memory sizes
2. Size of instructions - Each instruction written inside a function will occupy some memory in binary. That is how size of your function will be identified. This is nothing but your compiled code size.
From point of view of dynamic behavior (run time),
1. Heap memory resulted because of a function call is function memory.
i think this guide on function footprints is what you were talking about. they were probably looking for "32/64 bits (integer) because its a pointer"...
I bet the right answer could be "Undefined". An empty function consumes nothing.
function func(){}
A chaining one takes more than we can actually estimate.
function funcA()
{
funcB();
funcC();
//...
}
A local object without being used in its scope will be optimized away by most compilers so it too takes zero memory in its container.
function func()
{
var IamIgnored=0;
//don't do anything with IamIgnored
}
And please don't miss the memory alignment so I think calculating memory used by an object or a function can't be simply done by accumulating all objects' memory sizes within their scopes.
Suppose there is a variable a and a pointer p which points to address of a.
int a;
int *p=&a;
Now since I have a pointer pointing to the location of the variable, I know the exact memory location (or the chunk of memory).
My questions are:
Given an address, can we find which variable is using them? (I don't think this is possible).
Given an address, can we atleast find how big is the chunk of memory to which that memory address belongs. (I know this is stupid but still).
You can enumerate all your (suspect) variables and check if they point to the same location as your pointer (e.g. you can compare pointers for equality)
If your pointer is defined as int *p, you can assume it points to an integer. Your assumption can be proven wrong, of course, if for example the pointer value is null or you meddled with the value of the pointer.
You can think of memory as a big array of bytes:
now if you have a pointer to somewhere in middle of array, can you tell me how many other pointers point to same location as your pointer?? Or can you tell me how much information I stored in memory location that you point to it?? Or can you at least tell me what kind of object stored at location of your pointer?? Answer to all of this question is really impossible and the question look strange. Some languages add extra information to their memory management routines that they can track such information at a later time but in C++ we have the minimum overhead, so your answer is no it is not possible.
For your first question you may handle it using smart pointers, for example shared_ptr use a reference counter to know how many shared_ptr are pointing to a memory location and be able to control life time of the object(but current design of shared_ptr do not allow you to read that counter).
There is non-standard platform dependent solution to query size of dynamically allocated memory(for example _msize on Windows and memory_size on Unix) but that only work with dynamic memories that allocated using malloc and is not portable, in C++ the idea is you should care for this, if you need this feature implement a solution for it and if you don't need it, then you never pay extra cost of it
Given an address ,can we find which variable is using them ?
No, this isn't possible. variables point to memory, not the other way around. There isn't some way to get to variable-names from compiled code, except maybe via the symbol table, reading which in-turn would probably need messing around with assembly.
Given an address ,can we atleast find how big is the chunk of memory
to which that memory address belongs..
No. There isn't a way to do that given just the address. You could find the sizeof() after dereferencing the address but not from the address itself.
Question 1.
A: It cannot be done natively, but could be done by Valgrind memcheck tool. The VM tracks down all variables and allocated memory space/stack. However, it is not designed to answer such question, but with some modification, memcheck tool could answer this question. For example, it can correlate invalid memory access or memory leakage address to variables in the source code. So, given a valid and known memory address, it must be able to find the corresponding variable.
Question 2.
A: It can be done like above, but it can also be done natively with some PRELOADED libraries for malloc, calloc, strdup, free, etc. By manual instructed memory allocation functions, you can save allocated address and size. And also save the return address by __builtin_return_address() or backtrace() to know where the memory chunk is being allocated. You have to save all allocated address and size to a tree. Then you should be able to query the address belongs to which chunk and the chunk size, and what function allocated the chunk.
In my game engine I expose my Vector and Color objects to Lua, using userdata.
Now, for every even locally created Vector and Color from within Lua scripts, Luas memory usage goes up a bit, it doesn't fall until the garbage collector runs.
The garbage collector causes a small lagspike in my game.
Shouldn't the Vector and Color objects be immediately deleted if they are only used as arguments? For example like: myObject:SetPosition( Vector( 123,456 ) )
They aren't right now - the memory usage of Lua rises to 1,5 MB each second, then the lag spike occurs and it goes back to about 50KB.
How can I solve this problem, is it even solvable?
You can run a lua_setgcthreshold(L,0) to force an immediate garbage collection after you exit the function.
Edit: for 5.1 I'm seeing the following:
int lua_gc (lua_State *L, int what, int data);
Controls the garbage collector.
This function performs several tasks, according to the value of the parameter what:
* LUA_GCSTOP: stops the garbage collector.
* LUA_GCRESTART: restarts the garbage collector.
* LUA_GCCOLLECT: performs a full garbage-collection cycle.
* LUA_GCCOUNT: returns the current amount of memory (in Kbytes) in use by Lua.
* LUA_GCCOUNTB: returns the remainder of dividing the current amount of bytes of memory in use by Lua by 1024.
* LUA_GCSTEP: performs an incremental step of garbage collection. The step "size" is controlled by data (larger values mean more steps) in a non-specified way. If you want to control the step size you must experimentally tune the value of data. The function returns 1 if the step finished a garbage-collection cycle.
* LUA_GCSETPAUSE: sets data as the new value for the pause of the collector (see §2.10). The function returns the previous value of the pause.
* LUA_GCSETSTEPMUL: sets data as the new value for the step multiplier of the collector (see §2.10). The function returns the previous value of the step multiplier.
In Lua, the only way an object like userdata can be deleted is by the garbage collector. You can call the garbage collector directly, like B Mitch wrote (use lua_gc(L, LUA_CGSTEP, ...)), but there is no warranty that exactly your temporary object will be freed.
The best way to solve this is to avoid the creation of temporary objects. If you need to pass fixed parameters to methods like SetPosition, try to modify the API so that it also accepts numeric arguments, avoiding the creation of a temporary object, like so:
myObject:SetPosition(123, 456)
Lua Gems has a nice piece about optimization for Lua programs.
Remember, Lua doesn't know until runtime whether or not you saved those objects- you could have put them in a table in the registry, for example. You shouldn't even notice the impacts of collecting 1.5MB, there's another problem here.
Also, you're really being a waste making a new object for that. Remember that in Lua every object has to be dynamically allocated, so you're calling malloc to .. make a Vector object to hold two numbers? Write your function to take a pair of numeric arguments as an overload.