Compared to
Simple memory access
Disk access
Memory access on another computer(on the same network)
Disk access on another computer(on the same network)
in C++ on windows.
relative timings (shouldn't be off by more than a factor of 100 ;-)
memory-access in cache = 1
function call/return in cache = 2
memory-access out of cache = 10 .. 300
disk access = 1000 .. 1e8 (amortized depends upon the number of bytes transferred)
depending mostly upon seek times
the transfer itself can be pretty fast
involves at least a few thousand ops, since the user/system threshold must be crossed at least twice; an I/O request must be scheduled, the result must be written back; possibly buffers are allocated...
network calls = 1000 .. 1e9 (amortized depends upon the number of bytes transferred)
same argument as with disk i/o
the raw transfer speed can be quite high, but some process on the other computer must do the actual work
A function call is simply a shift of the frame pointer in memory onto the stack and addition of a new frame on top of that. The function parameters are shifted into local registers for use and the stack pointer is advanced to the new top of the stack for execution of the function.
In comparison with time
Function call ~ simple memory access
Function call < Disk Access
Function call < memory access on another computer
Function call < disk access on another computer
Compared to a simple memory access - slightly more, negligible really.
Compared to every thing else listed - orders of magnitude less.
This should hold true for just about any language on any OS.
In general, a function call is going to be slightly slower than memory access since it in fact has to do multiple memory accesses to perform the call. For example, multiple pushes and pops of the stack are required for most function calls using __stdcall on x86. But if your memory access is to a page that isn't even in the L2 cache, the function call can be much faster if the destination and the stack are all in the CPU's memory caches.
For everything else, a function call is many (many) magnitudes faster.
Hard to answer because there are a lot of factors involved.
First of all, "Simple Memory Access" isn't simple. Since at modern clock speeds, a CPU can add two numbers faster than it get a number from one side of the chip to the other (The speed of light -- It's not just a good idea, it's the LAW)
So, is the function being called inside the CPU memory cache? Is the memory access you're comparing it too?
Then we have the function call will clear the CPU instruction pipeline, which will affect speed in a non-deterministic way.
Assuming you mean the overhead of the call itself, rather than what the callee might do, it's definitely far, far quicker than all but the "simple" memory access.
It's probably slower than the memory access, but note that since the compiler can do inlining, function call overhead is sometimes zero. Even if not, it's at least possible on some architectures that some calls to code already in the instruction cache could be quicker than accessing main (uncached) memory. It depends how many registers need to be spilled to stack before making the call, and that sort of thing. Consult your compiler and calling convention documentation, although you're unlikely to be able to figure it out faster than disassembling the code emitted.
Also note that "simple" memory access sometimes isn't - if the OS has to bring the page in from disk then you've got a long wait on your hands. The same would be true if you jump into code currently paged out on disk.
If the underlying question is "when should I optimise my code to minimise the total number of function calls made?", then the answer is "very close to never".
This link comes up a lot in Google. For future reference, I ran a short program in C# on the cost of a function call, and the answer is: "about six times the cost of inline". Below are details, see //Output at the bottom. UPDATE: To better compare apples with apples, I changed Class1.Method to return 'void', as so: public void Method1 () { // return 0; }
Still, inline is faster by 2x: inline (avg): 610 ms; function call (avg): 1380 ms. So the answer, updated, is "about two times".
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
namespace FunctionCallCost
{
class Program
{
static void Main(string[] args)
{
Debug.WriteLine("stop1");
int iMax = 100000000; //100M
DateTime funcCall1 = DateTime.Now;
Stopwatch sw = Stopwatch.StartNew();
for (int i = 0; i < iMax; i++)
{
//gives about 5.94 seconds to do a billion loops,
// or 0.594 for 100M, about 6 times faster than
//the method call.
}
sw.Stop();
long iE = sw.ElapsedMilliseconds;
Debug.WriteLine("elapsed time of main function (ms) is: " + iE.ToString());
Debug.WriteLine("stop2");
Class1 myClass1 = new Class1();
Stopwatch sw2 = Stopwatch.StartNew();
int dummyI;
for (int ie = 0; ie < iMax; ie++)
{
dummyI = myClass1.Method1();
}
sw2.Stop();
long iE2 = sw2.ElapsedMilliseconds;
Debug.WriteLine("elapsed time of helper class function (ms) is: " + iE2.ToString());
Debug.WriteLine("Hi3");
}
}
// Class 1 here
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace FunctionCallCost
{
class Class1
{
public Class1()
{
}
public int Method1 ()
{
return 0;
}
}
}
// Output:
stop1
elapsed time of main function (ms) is: 595
stop2
elapsed time of helper class function (ms) is: 3780
stop1
elapsed time of main function (ms) is: 592
stop2
elapsed time of helper class function (ms) is: 4042
stop1
elapsed time of main function (ms) is: 626
stop2
elapsed time of helper class function (ms) is: 3755
The cost of actually calling the function, but not executing it in full? or the cost of actually executing the function? simply setting up a function call is not a costly operation (update the PC?). but obviously the cost of a function executing in full depends on what the function is doing.
Let's not forget that C++ has virtual calls (significantly more expensive, about x10) and on WIndows you can expect VS to inline calls (0 cost by definition, as there is no call left in the binary)
Depends on what that function does, it would fall 2nd on your list if it were doing logic with objects in memory. Further down the list if it included disk/network access.
A function call usually involves merely a couple of memory copies (often into registers, so they should not take up much time) and then a jump operation. This will be slower than a memory access, but faster than any of the other operations mentioned above, because they require communication with other hardware. The same should usually hold true on any OS/language combination.
If the function is inlined at compile time, the cost of the function becomes equivelant to 0.
0 of course being, what you would have gotten by not having a function call, ie: inlined it yourself.
This of course sounds excessively obvious when I write it like that.
The cost of a function call depends on the architecture. x86 is considerably slower (a few clocks plus a clock or so per function argument) while 64-bit is much less because most function arguments are passed in registers instead of on the stack.
Function call is actually a copy of parameters onto the stack (multiple memory access), register save, the actual code execution, and finally result copy and and registers restore (the registers save/restore depend on the system).
So.. speaking relatively:
Function call > Simple memory access.
Function call << Disk access - compared with memory it can be hundreds of times more expensive.
Function call << Memory access on another computer - the network bandwidth and protocol are the grand time killers here.
Function call <<< Disk access on another computer - all of the above and more :)
Only memory access is faster than a function call.
But the call can be avoided if compiler with inline optimization (for GCC compiler(s) and not only it is activated when using level 3 of optimization (-O3) ).
Related
I am performing some execution time benchmarks for my implementation of quicksort. Out of 100 successive measurements on exactly the same input data it seems like the first call to quicksort takes roughly 10 times longer than all consecutive calls. Is this a consequence of the operating system getting ready to execute the program, or is there some other explanation? Moreover, is it reasonable to discard the first measurement when computing an average runtime?
The below bar chart illustrates execution time (miliseconds) versus method call number. Each time the method is called it processes the exact same data.
To produce this particular graph the main method makes a call to quicksort_timer::time_fpi_quicksort(5, 100) whose implementation can be seen below.
static void time_fpi_quicksort(int size, int runs)
{
std::vector<int> vector(size);
for (int i = 0; i < runs; i++)
{
vector = utilities::getRandomIntVectorWithConstantSeed(size);
Timer timer;
quicksort(vector, ver::FixedPivotInsertion);
}
}
The getRandomIntVectorWithConstantSeed is implemented as follows
std::vector<int> getRandomIntVectorWithConstantSeed(int size)
{
std::vector<int> vector(size);
srand(6475307);
for (int i = 0; i < size; i++)
vector[i] = rand();
return vector;
}
CPU and Compilation
CPU: Broadwell 2.7 GHz Intel Core i5 (5257U)
Compiler Version: Apple LLVM version 10.0.0 (clang-1000.11.45.5)
Compiler Options: -std=c++17 -O2 -march=native
Yes, it could be a page fault on the page holding the code for the sort function (and the timing code itself). The 10x could also include ramp-up to max turbo clock speed.
Caching is not plausible, though: you're writing the (tiny) array outside the timed region, unless the compiler somehow reordered the init with the constructor of your Timer. Memory allocation being much slower the first time would easily explain it, maybe having to make a system call to get a new page the first time, but later calls to new (to construct std::vector) just grabbing already-hot-in-cache memory from the free list.
Training the branch predictors could also be a big factor, but you'd expect it to take more than 1 run before the TAGE branch predictors in a modern Intel CPU, or the perceptron predictors in a modern AMD, "learned" the full pattern of all the branching. But maybe they get close after the first run.
Note that you produce the same random array every time, by using srand() on every call. To test if branch prediction is the explanation, remove the srand so you get different arrays every time, and see if the time stays much higher.
What CPU, compiler version / options, etc. are you using?
Probably is because of caching, as the memory needs to be fetched from DRAM and allocated in CPU's data cache the first time. That takes (much) more latency more than loads that hit in the CPU's cache.
Then as your instructions are in the pipeline they follow the same branch as it is the instructions from the same memory source as it doesn't need to be invalidated because is the same pointer.
Would be interesting if you implement 4 methods with more or less the same functionality and then swap between them to see what happen.
I am trying to time a void function
for (size_t round = 0; round < 5; round++) {
cpu_time_start = get_cpu_time();
wall_time_start = get_wall_time();
scan.assign_clusters(epsilon, mu);
cpu_time_end = get_cpu_time();
wall_time_end = get_wall_time();
...
}
The first timing yields 300 seconds, while the next four timings yields 0.000002 seconds. This indicates that the void function call to assign_clusters is optimized out. How can I force my program to execute this time consuming function call every time, and yet still use optimization for the rest of the code?
What I usually do is to save the result of the function in question and then print it, but since this is a void function, do I have the same option?
I use the following optimization flags: -std=c++0x -march=native -O2
It depends on what is taking the time, to make the fix.
This could be caused by :-
Loading services. Your clustering may be database based, and requires the database services to start (the first time)
Disk caching. The OS will remember data it has read, and be able to provide the data as if it was in memory.
Memory caching. The CPU has different speeds of memory available to it, using the same memory twice, would be faster the second time.
State caching. The data may be in a more amenable state for subsequent runs. This can be thought of as sorting an array twice. The second time is already sorted, which can produce a speed up.
Service starting can be a number of seconds.
Disk cache approx 20x speed up.
Memory cache approx 6x speed up
State caching, can be unbounded.
I think your code needs to reset the scan object, to ensure it does the work again
I've just started working with a chunk of code that the authors claim is "highly optimized". At some point they do this:
namespace somename
{
static float array[N];
}
float Someclass::some_function(std::vector<float>& input)
{
// use somename::array in some way
return result;
}
The authors haven't included somename::array in the class because of issues with persistence code (which we have little control over). The class does O(N^2) operations on the array when some_function is called. So If I move array inside the function call,
float Someclass::some_function(std::vector<float>& input)
{
float array[N];
// use somename::array in some way
return result;
}
is it reasonable to expect a decrease in performance? In other words, is it obvious that, across many different systems and compilers, the author's optimization (using a global array rather than one inside the function) will help performance?
Since numbers matter:
./trial2.out 59.08s user 0.01s system 88% cpu 1:07.01 total
./trial.out 59.40s user 0.00s system 99% cpu 59.556 total
The source code: http://pastebin.com/YA2WpTSU (With alternate code commented and tested)
So, no difference. Compiled with:
gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)
Time results while using a non-static array within the function:
./trial.out 57.32s user 0.04s system 97% cpu 58.810 total
./trial.out 57.77s user 0.04s system 97% cpu 59.259 total
So again, no difference. Since when you use a array it is part of your function stack and not heap, there is no overhead with reserving memory every time the function is called. It would be an entirely different scenario in case you used a dynamic allocation (in which case, I do suspect that there would have been a huge difference in performance).
Maybe you don't notice the difference, but there is one! With the static keyword, the array exists in the DATA segment of the program and it remains the whole runtime. Without the static keyword, the array resides in the stack and is initialized every time you call the function. The stack version nevertheless can be the better choice because of better locality and therefore less cache misses. You have to measure which version is better in your case. In my case (one array with 69 64-Bit numbers and a second two dimensional array of 48 * 12 characters) the static version was significantly faster.
The only difference is that "array" is allocated globally (if declared static) and it would be "allocated" on the stack if declared in the function body. What really matters here is the size of your array (N).
If it's a big array, you can leave it static because you may not be able to declare it on the stack.
A third option would be to allocate it dynamically (heap, with new keyword). However, all these suppositions won't really affect the performance of the function itself, as once, allocated, there's no overhead for any of these methods.
I'm looking for a simple operation / routine which can "waste" time if repeated continuously.
I'm researching how gprof profiles applications, so this "time waster" needs to waste time in the user space and should not require external libraries. IE, calling sleep(20) will "waste" 20 seconds of time, but gprof will not record this time because it occurred within another library.
Any recommendations for simple tasks which can be repeated to waste time?
Another variant on Tomalak's solution is to set up an alarm, and so in your busy-wait loop, you don't need to keep issuing a system call, but instead just check if the signal has been sent.
The simplest way to "waste" time without yielding CPU is a tight loop.
If you don't need to restrict the duration of your waste (say, you control it by simply terminating the process when done), then go C style*:
for (;;) {}
(Be aware, though, that the standard allows the implementation to assume that programs will eventually terminate, so technically speaking this loop — at least in C++0x — has Undefined Behaviour and could be optimised out!**
Otherwise, you could time it manually:
time_t s = time(0);
while (time(0) - s < 20) {}
Or, instead of repeatedly issuing the time syscall (which will lead to some time spent in the kernel), if on a GNU-compatible system you could make use of signal.h "alarms" to end the loop:
alarm(20);
while (true) {}
There's even a very similar example on the documentation page for "Handler Returns".
(Of course, these approaches will all send you to 100% CPU for the intervening time and make fluffy unicorns fall out of your ears.)
* {} rather than trailing ; used deliberately, for clarity. Ultimately, there's no excuse for writing a semicolon in a context like this; it's a terrible habit to get into, and becomes a maintenance pitfall when you use it in "real" code.
** See [n3290: 1.10/2] and [n3290: 1.10/24].
A simple loop would do.
If you're researching how gprof works, I assume you've read the paper, slowly and carefully.
I also assume you're familiar with these issues.
Here's a busy loop which runs at one cycle per iteration on modern hardware, at least as compiled by clang or gcc or probably any reasonable compiler with at least some optimization flag:
void busy_loop(uint64_t iters) {
volatile int sink;
do {
sink = 0;
} while (--iters > 0);
(void)sink;
}
The idea is just to store to the volatile sink every iteration. This prevents the loop from being optimized away, and makes each iteration have a predictable amount of work (at least one store). Modern hardware can do one store per cycle, and the loop overhead generally can complete in parallel in that same cycle, so usually achieves one cycle per iteration. So you can ballpark the wall-clock time in nanoseconds a given number of iters will take by dividing by your CPU speed in GHz. For example, a 3 GHz CPU will take about 2 seconds (2 billion nanos) to busy_loop when iters == 6,000,000,000.
Suppose I have a function in a single threaded program that looks like this
void f(some arguments){
char buffer[32];
some operations on buffer;
}
and f appears inside some loop that gets called often, so I'd like to make it as fast as possible. It looks to me like the buffer needs to get allocated every time f is called, but if I declare it to be static, this wouldn't happen. Is that correct reasoning? Is that a free speed up? And just because of that fact (that it's an easy speed up), does an optimizing compiler already do something like this for me?
No, it's not a free speedup.
First, the allocation is almost free to begin with (since it consists merely of adding 32 to the stack pointer), and secondly, there are at least two reasons why a static variable might be slower
you lose cache locality. Data allocated on the stack are going to be in the CPU cache already, so accessing it is extremely cheap. Static data is allocated in a different area of memory, and so it may not be cached, and so it will cause a cache miss, and you'll have to wait hundreds of clock cycles for the data to be fetched from main memory.
you lose thread safety. If two threads execute the function simultaneously, it'll crash and burn, unless a lock is placed so only one thread at a time is allowed to execute that section of the code. And that would mean you'd lose the benefit of having multiple CPU cores.
So it's not a free speedup. But it is possible that it is faster in your case (although I doubt it).
So try it out, benchmark it, and see what works best in your particular scenario.
Incrementing 32 bytes on the stack will cost virtually nothing on nearly all systems. But you should test it out. Benchmark a static version and a local version and post back.
For implementations that use a stack for local variables, often times allocation involves advancing a register (adding a value to it), such as the Stack Pointer (SP) register. This timing is very negligible, usually one instruction or less.
However, initialization of stack variables takes a little longer, but again, not much. Check out your assembly language listing (generated by compiler or debugger) for exact details. There is nothing in the standard about the duration or number of instructions required to initialize variables.
Allocation of static local variables is usually treated differently. A common approach is to place these variables in the same area as global variables. Usually all the variables in this area are initialized before calling main(). Allocation in this case is a matter of assigning addresses to registers or storing the area information in memory. Not much execution time wasted here.
Dynamic allocation is the case where execution cycles are burned. But that is not in the scope of your question.
The way it is written now, there is no cost for allocation: the 32 bytes are on the stack. The only real work is you need to zero-initialize.
Local statics is not a good idea here. It wont be faster, and your function can't be used from multiple threads anymore, as all calls share the same buffer. Not to mention that local statics initialization is not guaranteed to be thread safe.
I would suggest that a more general approach to this problem is that if you have a function called many times that needs some local variables then consider wrapping it in a class and making these variables member functions. Consider if you needed to make the size dynamic, so instead of char buffer[32] you had std::vector<char> buffer(requiredSize). This is more expensive than an array to initialise every time through the loop
class BufferMunger {
public:
BufferMunger() {};
void DoFunction(args);
private:
char buffer[32];
};
BufferMunger m;
for (int i=0; i<1000; i++) {
m.DoFunction(arg[i]); // only one allocation of buffer
}
There's another implication of making the buffer static, which is that the function is now unsafe in a multithreaded application, as two threads may call it and overwrite the data in the buffer at the same time. On the other hand it's safe to use a separate BufferMunger in each thread that requires it.
Note that block-level static variables in C++ (as opposed to C) are initialized on first use. This implies that you'll be introducing the cost of an extra runtime check. The branch potentially could end up making performance worse, not better. (But really, you should profile, as others have mentioned.)
Regardless, I don't think it's worth it, especially since you'd be intentionally sacrificing re-entrancy.
If you are writing code for a PC, there is unlikely to be any meaningful speed advantage either way. On some embedded systems, it may be advantageous to avoid all local variables. On some other systems, local variables may be faster.
An example of the former: on the Z80, the code to set up the stack frame for a function with any local variables was pretty long. Further, the code to access local variables was limited to using the (IX+d) addressing mode, which was only available for 8-bit instructions. If X and Y were both global/static or both local variables, the statement "X=Y" could assemble as either:
; If both are static or global: 6 bytes; 32 cycles
ld HL,(_Y) ; 16 cycles
ld (_X),HL ; 16 cycles
; If both are local: 12 bytes; 56 cycles
ld E,(IX+_Y) ; 14 cycles
ld D,(IX+_Y+1) ; 14 cycles
ld (IX+_X),D ; 14 cycles
ld (IX+_X+1),E ; 14 cycles
A 100% code space penalty and 75% time penalty in addition to the code and time to set up the stack frame!
On the ARM processor, a single instruction can load a variable which is located within +/-2K of an address pointer. If a function's local variables total 2K or less, they may be accessed with a single instruction. Global variables will generally require two or more instructions to load, depending upon where they are stored.
With gcc, I do see some speedup:
void f() {
char buffer[4096];
}
int main() {
int i;
for (i = 0; i < 100000000; ++i) {
f();
}
}
And the time:
$ time ./a.out
real 0m0.453s
user 0m0.450s
sys 0m0.010s
changing buffer to static:
$ time ./a.out
real 0m0.352s
user 0m0.360s
sys 0m0.000s
Depending on what exactly the variable is doing and how its used, the speed up is almost nothing to nothing. Because (on x86 systems) stack memory is allocated for all local vars at the same time with a simple single func(sub esp,amount), thus having just one other stack var eliminates any gain. the only exception to this is really huge buffers in which case a compiler might stick in _chkstk to alloc memory(but if your buffer is that big you should re-evaluate your code). The compiler cannot turn stack memory into static memory via optimization, as it cannot assume that the function is going to be used in a single threaded enviroment, plus it would mess with object constructors & destructors etc
If there are any local automatic variables in the function at all, the stack pointer needs to be adjusted. The time taken for the adjustment is constant, and will not vary based on the number of variables declared. You might save some time if your function is left with no local automatic variables whatsoever.
If a static variable is initialized, there will be a flag somewhere to determine if the variable has already been initialized. Checking the flag will take some time. In your example the variable is not initialized, so this part can be ignored.
Static variables should be avoided if your function has any chance of being called recursively or from two different threads.
It will make the function substantially slower on most real cases. This is because the static data segment is not near the stack and you will lose cache coherency, so you will get a cache miss when you try to access it. However when you allocate a regular char[32] on the stack, it is right next to all your other needed data and costs very little to access. The initialization costs of a stack-based array of char are meaningless.
This is ignoring that statics have many other problems.
You really need to actually profile your code and see where the slowdowns are, because no profiler will tell you that allocating a statically-sized buffer of characters is a performance problem.