How do you detect the number of CPU cores in crystal?
In Go you can do runtime.NumCPU() and in Node.js you can do os.cpus(). Is there something similar in Crystal?
The last time I checked (admittedly that was long ago) there was no direct way to do that, but you can get access to this information through the command line. I ended up combining multiple of these answers for redundancy.
THREADS = `getconf _NPROCESSORS_ONLN || nproc --all || grep -c '^processor' /proc/cpuinfo`.to_i
Use System.cpu_count. This will give you the number of logical cores available. This method was introduced in Crystal 0.23.0.
Here's the source for it in the tree for Crystal 0.29.0: https://github.com/crystal-lang/crystal/blob/fbfe8b62f44eca4596090b7580801bd184f73c7a/src/system.cr#L22
Related
Is there a way to limit CPU usage in C++?
I mean can I write a program which takes input %CPU to be used and it uses maximum that much amount of CPU.
I basically want to add the functionality of cpulimit command internally in the program.
If there is, how to do it ?
Edits:
Environment: Linux (debian) with gcc 6.1. It should support as many arbitrary numbers as possible. i.e a range of 1% - 100% . If the OS cannot do, an error can be logged and the nearest value to it can be used or any other solution that is recommended when the OS restricts that number.
Linux does not provide the means to set a specific percentage. However, the nice(2) system call lowers the priority of the process in relation to other processes on the system, thus achieving some sort of a relative percentage of CPU, in relation to other processes on the system.
You can also use the setrlimit(2) system call to set your process's RLIMIT_CPU, as a fixed amount.
Use getrusage(), see Linux commands to detect the computer resource usage of a program
And when you check, and you've used however many milliseconds of runtime you want, use nanosleep() to sleep a few milliseconds. Adjust percentages to match your requirements.
I have written and cross compiled a small c++ program, and I could run it in an ARM or a PC. Since ARM and a PC have different instruction set architectures, I wanna to compare them. Is that possible for me to get the number of executed instructions in this c++ program for both ISAs?
What you need is a profiler. perf would be one easy to use. It will give you the number of instructions that executed, which is the best metric if you want to compare ISA efficiency.
Check the tutorial here.
You need to use: perf stat ./your binary
Look for instructions metric. This approach uses a register in your CPU's performance monitoring unit - PMU - that counts the number of instructions.
Are you trying to get the number of static instructions or dynamic instructions? So, for instance, if you have the following loop (pseudocode):
for (i 0 to N):
a[i] = b[i] + c[i]
Static instruction count will be just under 10 instructions, give or take based on your ISA, but the dynamic count would depend on N, on the branch prediction implementation and so on.
So for static count I would recommend using objdump, as per recommendations in the comments. You can find the entry and exit labels of your subroutine and count the number of instructions in between.
For dynamic instruction count, I would recommend one of two things:
You can simulate running that code using an instruction set simulator (there are open source ISA simulators for both ARM and x86 out there - Gem5 for instance implements both of them, there are others out there that support one or the other.
Your second option is to run this natively on the target system and setup performance counters in the CPU to report dynamic instruction count. You would reset before executing your code, and read it afterwards (there might be some noise here associated with calling your subroutine and exiting, but you should be able to isolate that out)
Hope this helps :)
objdump -dw mybinary | wc -l
On Linux and friends, this gives a good approximation of the number of instructions in an executable, library or object file. This is a static count, which is of course completely different than runtime behavior.
Linux:
valgrind --tool=callgrind ./program 1 > /dev/null
There are many online judge sites which can verify your program by comparing its output to the correct answers. What's more, they also check the running time and memory usage to make sure that your program doesn't exceed the maximum limit.
So here is my question, since some online judge sites run several test programs at the same time, how do they achieve performance isolation? and how do they achieve same running time on same program that run at another time?
I think there are isolated environment processes like 'VMware' or 'Sandbox' that always return same result. is this correct? and any idea about how to implement these things?
Current Solution
I'm using docker for sandboxing. it's a dead simple and the safest way.
Unfortunately it is VERY hard to actually guarantee consistent running times even on a dedicated machine versus a VM. If you do want to implement something like this as was mentioned you probably want a VM to keep all the code that will run sandboxed. Usually you don't want to service more than a couple of requests per core so I would say for algorithms that are memory and cpu bound use at most 2 VMs per physical core of the machine.
Although I can only speculate why not try different numbers of VMs per core and see how it performs. Try to aim for about a 90% or higher rate of SLO compliance (or 98-99 if you really need to) and you should be just fine. Again its hard to tell you exactly what to do as a lot of these things require just testing it out and seeing how it does.
May be overly simplistic depending on your other requirements which aren't in the question, but;
If the algorithms are CPU bound, simply running it in an isolated VM (or FreeBSD jail, or...) and using the built-in operating system instrumentation would be the simplest.
(Could be as simple as using the 'time' command in unix and setting memory limits with "limit")
I want to check the performance of an application (whose exe i have, no source code) by running it multiple times and possibly compare the results, dint find much on the internet regarding this topic,
Since i have to do it with multiple input times, i thought doing it through code(no bar on the language used) can make things easier, as i may have to repeat them many times,
can anyone help me start off???
Note: by Performance i mean the memory usage, cpu and possibly the time taken to do it!
(I'm currently using perfmon on windows by using necessary counters to check these parameters and manually noting it down)
Thanks
It strongly depends upon your operating system. On Linux, you could use the time utility. And strace might help you understanding the system calls that are used.
I have no idea of the equivalent on Windows systems.
I think that you could create a bash/batch script to call your program as many times as you need and with different inputs.
You could then have your script create a CSV file that contains the time it took to execute your program (start date and end date for example). CSV files are usually compatible with most spreadsheet programs like Excel, so I think that can make it easier for you to process your data, like creating means and standard deviations.
I don't have much to say regarding the memory and CPU usage, but if you are in Windows it wouldn't hurt to take a look at the Process Explorer and the Process Monitor (you can find them in this page). I think that they might help you in your task.
Finally if you are in Linux I think that you might be able to use grep with the top command to gather some statistics.
Regards,
Felipe
If you want exact results, Rational Purify (on Windows), or valgrind (on Linux) are the best tools; these run your application in a virtual machine that can be instructed to do exact cycle counting.
In another post an utility named timethis.exe was mentioned for measuring time under Windows. Maybe it is useful for your purposes.
I used the perform im using to manually note down in an automated way,
that is, i used the performance counter class available in dot net and obtained samples of the particular application at regular intervals and generated a graph with those values..
Thanks :)
What I want to do
I have a computationally intensive OCaml application and I'd like it to run in the background without disturbing normal computer usage. I'd like to present the users with two options:
(1) the application only runs when CPU usage is virtually 0%;
(2) the application only uses "free" processing power (e.g. if other processes add up to 100%, the OCaml application pauses; if other processes are virtually 0%, then there are no restrictions for the OCaml application; if other processes add up to, say, 50% then OCaml will use up to 50%).
Some thoughts
My idea is to check CPU usage at various check points in the code and pause execution if necessary.
In (1), we just check if CPU is below say 2% and, if not, pause until it becomes lower than 2% again.
In (2), things are trickier. Since when no restrictions are present the application always consumes 100% of CPU and checkpoints will be quite frequent, to reduce CPU usage to, say, half, I just have to delay it at every checkpoint by exactly the time it took between check points. If check points are frequent, this would be similar to using 50% CPU, I'd say. For other percentages we can do something similar by suspending for appropriate periods of time. However, this looks very contrived, full of overhead, and above all, I'm not sure it really does what I want. A better alternative could be to invoke Unix.nice n with some appropriate integer at the start of the application. I suppose that setting n=15 would probably be right.
My questions
(Q1) How can I know from within my OCaml application what the CPU usage for the application process is? (I'd like to do this with an OCaml function and not by invoking "ps" or something similar on the command line...)
(Q2) Do you see problems with my idea to achieve (2). Which are the practical differences to changing niceness of process?
(Q3) Do you have any other suggestions for (2)?
Use Caml's Unix library to periodically capture your CPU times and your elapsed times. Your CPU usage is the ratio. Try Unix.gettimeofday and Unix.times. N.B. You'll need to link with the -lunix option.
I too would just run the process under nice and be done with it.
Get your PID then parse the contents of /proc/<PID>/stat to get info about your process and /proc/stat to get global CPU info. They both have a bunch of statistics that you can use to decide when to do work and when to sleep. Do man proc to see the documentation for all the fields (long). Related question with good info: stackoverflow.com/questions/1420426
Setting niceness is easy and reliable. Doing things yourself is much more work but potentially gives you more control. If your actual goal is to just run as a background task, I would go with nice and be done with it.