I noticed earlier that when my VS is building my big c++ solution, my CPU usage was less than 25%. Wondering if I can set VS to always use 100% CPU, I did some research:
Found two options that can be configured for this purpose:
Maximum Number Of Parallel Project Builds
Maximum Concurrent C++ Compilations
What is the difference?
And to achieve my goal, bonus question is how can I set VS to use more CPU when it builds?
It can often be insightful to see all the files considered in a build. It's not uncommon to access 10.000 files, especially when using bigger libraries.
The fastest way to access these files would be if they're in RAM already, i.e. the OS file cache. Otherwise, an SSD is a reasonable alternative. But if they have to come from a mechanical HDD, the CPU will spend a large amount of time just sleeping while it waits for files to be read.
Hence, the way to improve CPU utilization is to make sure it's not waiting for I/O. Hardware is much cheaper than C++ programmers; get a fast SSD and sufficient RAM.
Related
I have an application that loads files and processes data. Let's assume I have like 10...20 files to process.
some requirements, to make the question clearer:
files are small, maybe a few MB max
there might be a dozen files, maybe a hundred
one example might be parsing CSV data, or JSON, loading game 3d models
One idea is to use some thread pool and process files in parallel. Is this efficient? Can my operating system handle file access from multiple threads?
I found this question:
Accessing a single file with multiple threads
But in my application one thread would access its "own" file, so there wouldn't be any collisions.
In my application, I'm using C++/STL, but I'd like to know the general opinion about filesystems on Linux and Windows.
You need to benchmark. (probably in your case it could be worth to use several threads; however in your case, the loading should be so quick, even done sequentially, that your average user won't notice)
In many cases, when you deal with medium sized files (e.g. less than a dozen of megabytes each, or perhaps even half a gigabyte each) which have been accessed recently, these files practically sit in the page cache. So you won't access the disk itself, and your program practically works in RAM (and then multithreading should be effective).
BTW, Linux has readahead(2), posix_fadvise(2), madvise(2) to hint the kernel virtual memory subsystem (that is, to give hints to the page cache).
If your common use case is accessing the disk itself (e.g. because the files are quite big, or because you have not accessed them recently before, so they are not in the page cache), then multi-threading won't help, because the bottleneck becomes the hardware disk.
Remember that a disk (even an SSD one) is many thousands times slower than RAM and it does IO operations sequentially.
Also, you may spend some amount of CPU time in parsing the files. If that takes a significant amount of CPU, it is worth to be run in several independent threads.
In my experience you get more performance if the processing of the data is heavy. In this case you really make parallel the execution of your program. You also need to know how many core your cpu have. It is not worth have more threads than cpu cores.
If your processing is "light", probably your threads are always waiting of disk to complete reading, with little, if ever, gain in performance.
I'm working on a project that has thousands of .cpp files plus thousands more .h and .hpp and the build takes 28min running from an SSD.
We inherited this project from a different company just weeks ago but perusing the makefiles, they explicitly disabled parallel builds via the .NOPARALLEL phony target; we're trying to find out if they have a good reason.
Worst case, the only way to speed this up is to use a RAM drive.
So I followed the instructions from Tekrevue and installed Imdisk and then ran benchmarks using CrystalDiskMark:
SSD
RAM Drive
I also ran dd using Cygwin and there's a significant speedup (at least 3x) on the RAM drive compared to my SSD.
However, my build time changes not one minute!
So then I thought: maybe my proprietary compiler calls some Windows API and causes a huge slowdown so I built fftw from source on Cygwin.
What I expected is that my processor usage would increase to some max and stay there for the duration of the build. Instead, my usage was very spiky: one for each file compiled. I understand even Cygwin still has to interact with windows so the fact that I still got spiky proc usage makes me assume that it's not my compiler that's the issue.
Ok. New theory: invoking compiler for each source-file has some huge overhead in Windows so, I copy-pasted from my build-log and passed 45 files to my compiler and compared it to invoking the compiler 45 times separately. Invoking ONCE was faster but only by 4 secs total for the 45 files.
And I saw the same "spiky" processor usage as when invoking compiler once for each file.
Why can't I get the compiler to run faster even when running from RAM drive? What's the overhead?
UPDATE #1
Commenters have been saying, I think, that the RAM drive thing is kind of unnecessary bc windows will cache the input and output files in RAM anyway.
Plus, maybe the RAM drive implementation (ie drivers) is sub-optimal.
So, I'm not using the RAM drive anymore.
Also, people have said that I should run the 45-file build multiple times so as to remove the overhead for caching: I ran it 4 times and each time it was 52secs.
CPU usage (taken 5 secs before compilation ended)
Virtual memory usage
When the compiler spits out stuff to disk, it's actually cached in RAM first, right?
Well then this screenshot indicates that IO is not an issue or rather, it's as fast as my RAM.
Question:
So since everything is in RAM, why isn't the CPU % higher more of the time?
Is there anything I can do to make single- threaded/job build go faster?
(Remember this is single-threaded build for now)
UPDATE 2
It was suggested below that I should set the affinity, of my compile-45-files invocation, to 1 so that windows won't bounce around the invocation to multiple cores.
The result:
100% single-core usage! for the same 52secs
So it wasn't the hard drive, RAM or the cache but CPU that's the bottleneck.
**THANK YOU ALL! ** for your help
========================================================================
My machine: Intel i7-4710MQ # 2.5GHz, 16GB RAM
I don't see why you are blaming so much the operating system, besides sequential, dumb IO (to load sources/save intermediate output - which should be ruled out by seeing that an SSD and a ramdisk perform the same) and process starting (ruled out by compiling a single giant file) there's very little interaction between the compiler and the operating system.
Now, once you ruled out "disk" and processor, I expect the bottleneck to be the memory speed - not for the RAM-disk IO part (which probably was already mostly saturated by the SSD), but for the compilation process itself.
That's actually quite a common problem, at this moment of time processors are usually faster than memory, which is often the bottleneck (that's the reason why currently it's critical to write cache-friendly code). The processor is probably wasting some significant time waiting for out of cache data to be fetched from main memory.
Anyway, this is all speculation. If you want a reliable answer, as usual you have to profile. Grab some sampling profiler from a list like this and go see where the compiler is wasting time. Personally, I expect to see a healthy dose of cache misses (or even page faults if you burned too much RAM for the ramdisk), but anything can be.
Reading your source code from the drive is a very, very small part of the overhead of compiling software. Your CPU speed is far more relevant, as parsing and generating binaries are the slowest part of the process.
**Update
Your graphs show a very busy CPU, I am not sure what you expect to see. Unless the build is multithreaded AND your kernel stops scheduling other, less intensive threads, this is certainly the graph of a busy processor.
Your trace is showing 23% CPU usage. Your CPU has 4 actual cores (with hyperthreading to make it look like 8). So, you're using exactly one core to its absolute maximum (plus or minus 2%, which is probably better accuracy than you can really expect).
The obvious conclusion from this would be that your build process is CPU bound, so improving your disk speed is unlikely to make much difference.
If you want substantially faster builds, you need to either figure out what's wrong with your current makefiles or else write entirely new ones without the problems, so you can support both partial and parallel builds.
That can gain you a lot. Essentially anything else you do (speeding up disks, overclocking the CPU, etc.) is going to give minor gains at best (maybe 20% if you're really lucky, where a proper build environment will probably give at least a 20:1 improvement for most typical builds).
I want to try to speed up my compile-time of our C++ projects. They have about 3M lines of code.
Of course, I don't need to always compile every project, but sometimes there are lot of source files modified by others, and I need to recompile all of them (for example, when someone updates an ASN.1 source file).
I've measured that compiling a mid-project (that does not involves all the source files) takes about three minutes. I know that's not too much, but sometimes it's really boring waiting for a compile..
I've tried to move the source code to an SSD (an old OCZ Vertex 3 60 GB) that, benchmarked, it's from 5 to 60 times faster than the HDD (especially in random reading/writing). Anyway, the compile-time is almost the same (maybe 2-3 seconds faster, but it should be a chance).
Maybe moving the Visual Studio bin to SSD would grant additional increment in performance?
Just to complete the question: I've a W3520 Xeon #2.67 GHz and 12 GB of DDR3 ECC.
This all greatly depends on your build environment and other setup. For example, on my main compile server, I have 96 GiB of RAM and 16 cores. The HDD is rather slow, but that doesn't really matter as about everything is cached in RAM.
On my desktop (where I also compile sometimes) I only have 8 Gib of RAM, and six cores. Doing the same parallel build there could be greatly sped up, because six compilers running in parallel eat up enough memory for the SSD speed difference being very noticeable.
There are many things that influence the build times, including the ratio of CPU to I/O "boundness". In my experience (GCC on Linux) they include:
Complexity of code. Lots of metatemplates make it use more CPU time, more C-like code might make the I/O of generated objects (more) dominant
Compiler settings for temporary files, like -pipe for GCC.
Optimization being used. Usually, the more optmization, the more the CPU work dominates.
Parallel builds. Compiling a single file at a time will likely never produce enough I/O to get today's slowest harddisk to any limit. Compiling with eight cores (or more) at once however might.
OS/filesystem being used. It seems that some filesystems in the past have choked on the access pattern for many files built in parallel, essentially putting the I/O bottleneck into the filesystem code, rather than the underlying hardware.
Available RAM for buffering. The more aggressively an OS can buffer your I/O, the less important the HDD speed gets. This is why sometimes a make -j6 can be a slower than a make -j4 despite having enough idle cores.
To make it short: It depends on enough things to make any "yes, it will help you" or "no, it will help you not" pure speculation, so if you have the possibility to try it out, do it. But don't spend too much time on it, for every hour you try to cut your compile times into half, try to estimate how often you (or your coworkers if you have any) could have rebuilt the project, and how that relates to the possible time saved.
C++ compilation/linking is limited by processing speed, not HDD I/O. That's why you're not seeing any increase in compilation speed.
(Moving the compiler/linker binaries to the SSD will do nothing. When you compile a big project, the compiler/linker and the necessary library are read into memory once and stay there.)
I have seen some minor speedups from moving the working directory to an SSD or ramdisk when compiling C projects (which is a lot less time consuming than C++ projects that make heavy use of templates etc), but not enough to make it worth it.
I found that compiling a project of around 1 million lines of C++ sped up by about a factor of two when the code was on an SSD (system with an eight-core Core i7, 12 GB RAM). Actually, the best possible performance we got was with one SSD for the system and a second one for the source -- it wasn't that the build was much faster, but the OS was much more responsive while a big build was underway.
The other thing that made a huge difference was enabling parallel building. Note that there are two separate options that both need to be enabled:
Menu Tools → Options → Projects and Solutions → maximum number of parallel project builds
Project properties → C++/General → Multi-processor compilation
The multiprocessor compilation is incompatible with a couple of other flags (including minimal rebuild, I think) so check the output window for warnings. I found that with the MP compilation flag set all cores were hitting close to 100% load, so you can at least see the CPU is being used aggressively.
One point not mentioned is that when using ccache and a highly parallel build, you'll see benefits to using an SSD.
I did replace my hard disk drive with an SSD hoping that it will reduce the compilation time of my C++ project. Simply replacing the hard disk drive with an SSD did not solve the problem and compilation time with both were almost the same.
However, after initial failures, I got success in speeding up the compilation by approximately six times.
The following steps were done to increase the compilation speed.
Turned off hibernation: "powercfg -h off" in command prompt
Turned off drive indexing on C drive
Shrunk page file to 800 min/1024 max (it was initially set to system managed size of 8092).
Apparently the speed of the C++ linker in Visual Studio 2010 hasn't improved that much (about 25% in our case). This means that we're still stuck with linking times between 30 seconds and two minutes. Surely there are linkers out there that perform better? Does anyone have experience with switching to another linker or even a complete tool set and seeing linking times go down drastically?
Cheers,
Sebastiaan
You may well find a faster linker but, unless it's ten times as fast and I'm linking thirty times an hour, I think I'd prefer to use the tools that Microsoft has tested with.
I would rather have relatively slow link times than potentially unstable software.
And you kids are spoilt nowadays. In my day, we had to submit our 80-column cards to the computer centre and, if we were lucky, the operator would get it typed in by next Thursday and we could start debugging from the hardcopy output :-)
When we checked the linker speed, we have identified the disk speed to be the most limiting factor. The amount of file traffic is huge, especially because of debugging info (just check the size of the pdb).
For us the solution was:
install insane amounts of RAM, so that a lot of file traffic can be cached (go for 4 GB, or even more, if you are on 64b OS). Note: you may need to change some system settings so that system is able to dedicate more memory for the cache
use very fast hard drive (connecting multiple of them as RAID may help even more)
We have also experimented with SSD, but the SSD we have tried had very slow write performance, therefore the net effect was negative. This might have changed meanwhile, especially with the best of SSDs.
As a first step I would suggest to launch Process Explorer (or even Task Manager will do) and check your CPU load and I/O traffic during the link phase, so that you can verify of you are CPU limited, or I/O limited.
There may be, but I imagine you'd be talking improvements in the range of a few percentage points. You're unlikely to find anything that's magnitudes faster (which is, I assume, what you'd like).
There are ways to improve your link times, though. What options do you have turned on? Things like "Enable Incremental Linking" and "Enable Function-level Linking" can have dramatic effects on linking performance (well, obviously the first time you link it'll be a "full" link, but subsequent links can be made much quicker with these settings).
Wow and i get nervous when my link time is above 10 sec.
Use modern style SSD Disks. I have 2x 60 GB OCZ Vertex2 E Disks as a RAID 0 and IO is not a problem anymore. SSD are now good enough for daily use even for heavy writes.
And get a few gigabyte of memory. Can't see any reason anymore to work with less then 8 GB RAM.
Turn on incremental linking, and linking should not take more than 1 sec.
I'm processing data from a hard disk from one large file (processing is fast and not a lot of overhead) and then have to write the results back (hundreds of thousands of files).
I started writing the results straight away in files, one at a time, which was the slowest option. I figured it gets a lot faster if I build a vector of a certain amount of the files and then write them all at once, then go back to processing while the hard disk is occupied in writing all that stuff that i poured into it (that at least seems to be what happens).
My question is, can I somehow estimate a convergence value for the amount of data that I should write from the hardware constraints ? To me it seems to be a hard disk buffer thing, I have 16MB buffer on that hard disk and get these values (all for ~100000 files):
Buffer size time (minutes)
------------------------------
no Buffer ~ 8:30
1 MB ~ 6:15
10 MB ~ 5:45
50 MB ~ 7:00
Or is this just a coincidence ?
I would also be interested in experience / rules of thumb about how writing performance is to be optimized in general, for example are larger hard disk blocks helpful, etc.
Edit:
Hardware is a pretty standard consumer drive (I'm a student, not a data center) WD 3,5 1TB/7200/16MB/USB2, HFS+ journalled, OS is MacOS 10.5. I'll soon give it a try on Ext3/Linux and internal disk rather than external).
Can I somehow estimate a convergence value for the amount of data that I should write from the hardware constraints?
Not in the long term. The problem is that your write performance is going to depend heavily on at least four things:
Which filesystem you're using
What disk-scheduling algorithm the kernel is using
The hardware characteristics of your disk
The hardware interconnect you're using
For example, USB is slower than IDE, which is slower than SATA. It wouldn't surprise me if XFS were much faster than ext2 for writing many small files. And kernels change all the time. So there are just too many factors here to make simple predictions easy.
If I were you I'd take these two steps:
Split my program into multiple threads (or even processes) and use one thread to deliver system calls open, write, and close to the OS as quickly as possible. Bonus points if you can make the number of threads a run-time parameter.
Instead of trying to estimate performance from hardware characteristics, write a program that tries a bunch of alternatives and finds the fastest one for your particular combination of hardware and software on that day. Save the fastest alternative in a file or even compile it into your code. This strategy was pioneered by Matteo Frigo for FFTW and it is remarkably effective.
Then when you change your disk, your interconnect, your kernel, or your CPU, you can just re-run the configuration program and presto! Your code will be optimized for best performance.
The important thing here is to get as many outstanding writes as possible, so the OS can optimize hard disk access. This means using async I/O, or using a task pool to actually write the new files to disk.
That being said, you should look at optimizing your read access. OS's (at least windows) is already really good at helping write access via buffering "under the hood", but if your reading in serial there isn't too much it can do to help. If use async I/O or (again) a task pool to process/read multiple parts of the file at once, you'll probably see increased perf.
Parsing XML should be doable at practically disk read speed, tens of MB/s. Your SAX implementation might not be doing that.
You might want to use some dirty tricks. 100.000s of files to write is not going to be efficient with the normal API.
Test this by writing sequentially to a single file first, not 100.000. Compare the performance. If the difference is interesting, read on.
If you really understand the file system you're writing to, you can make sure you're writing a contiguous block you just later split into multiple files in the directory structure.
You want smaller blocks in this case, not larger ones, as your files are going to be small. All free space in a block is going to be zeroed.
[edit] Do you really have an external need for those 100K files? A single file with an index could be sufficient.
Expanding on Norman's answer: if your files are all going into one filesystem, use only one helper thread.
Communication between the read thread and write helper(s) consists of a two-std::vector double-buffer per helper. (One buffer owned by the write process and one by the read process.) The read thread fills the buffer until a specified limit then blocks. The write thread times the write speed with gettimeofday or whatever, and adjusts the limit. If writing went faster than last time, increase the buffer by X%. If it went slower, adjust by –X%. X can be small.