I wrote the code below to get the current cpu usage on a window machine. However I notice some unusual spikes. Nine out of ten times the percentage I'm getting is between 2-5 and suddenly it spikes to over 95% which ofc doesn't agree with what the task-manger is showing... Could someone please point out if there is anything I'm doing wrong?
#include <iostream>
#include <sstream>
#include <chrono>
#include <thread>
#include <vector>
#include <Windows.h>
float roundPercentage(const double& d) {
std::ostringstream tmp;
tmp << d;
std::string str = tmp.str();
return (float)(std::round(std::stof(str.substr(0, 5)) * 100) / 100);
}
int main() {
while (true) {
FILETIME idle_time, kernel_time, user_time;
std::vector<FILETIME> last_values;
GetSystemTimes(&idle_time, &kernel_time, &user_time);
last_values.push_back(kernel_time);
last_values.push_back(user_time);
std::this_thread::sleep_for(std::chrono::milliseconds(500));
GetSystemTimes(&idle_time, &kernel_time, &user_time);
unsigned long long current_value = (kernel_time.dwLowDateTime + kernel_time.dwHighDateTime) + (user_time.dwLowDateTime + user_time.dwHighDateTime);
unsigned long long last_value = (last_values[0].dwLowDateTime + last_values[0].dwHighDateTime) + (last_values[1].dwLowDateTime + last_values[1].dwHighDateTime);
double d = (double)(last_value - current_value) / (double)(last_value + current_value) * 100.0;
std::cout << "cpu usage: " << roundPercentage(d) << "%" << std::endl;
}
return 0;
}
Based on comments I had a better look into FILETIME and ULARGE_INTEGER and the below code seems to work...
#include <iostream>
#include <chrono>
#include <thread>
#include <Windows.h>
uint64_t FromFileTime(const FILETIME& ft) {
ULARGE_INTEGER uli = { 0 };
uli.LowPart = ft.dwLowDateTime;
uli.HighPart = ft.dwHighDateTime;
return uli.QuadPart;
}
int main() {
while (true) {
FILETIME idle_time, kernel_time, user_time;
GetSystemTimes(&idle_time, &kernel_time, &user_time);
uint64_t k1 = FromFileTime(kernel_time);
uint64_t u1 = FromFileTime(user_time);
uint64_t i1 = FromFileTime(idle_time);
std::this_thread::sleep_for(std::chrono::milliseconds(500));
GetSystemTimes(&idle_time, &kernel_time, &user_time);
uint64_t k2 = FromFileTime(kernel_time);
uint64_t u2 = FromFileTime(user_time);
uint64_t i2 = FromFileTime(idle_time);
uint64_t ker = k2 - k1;
uint64_t usr = u2 - u1;
uint64_t idl = i2 - i1;
uint64_t cpu = (ker + usr - idl) * 100 / (ker + usr);
std::cout << "cpu usage: " << static_cast<int>(cpu) << "%" << std::endl;
}
return 0;
}
Related
i got some timestamp where seconds in int32 and nanosecond in uint32, i wish to get a accuracy delta-time, like:
int32 last_seconds; // = some value
int32 this_seconds; // = some value
uint32 last_nanosec; // = some value, e.g. 178922366
uint32 this_nanosec; // = some value, e.g. 58887157
float delta_seconds = float(this_seconds - last_seconds);
float delta_nanosec = float(this_nanosec - last_nanosec);
float delta_time = delta_seconds + delta_nanosec/1e9;
but i found this_nanosec - last_nanosec easily overflow as they are uint32 and frequently this_nanosec < last_nanosec, as example delta_nanosec = 4.17493, but delta_nanosec = -0.120035209 is more reasonable . (however, results of seconds look fine)
how can i get a accuracy delta-time as expected in this case? thanks
You could use your input timestamps to initialize timespec struct/s, then plug them in diff_timespec
#include <time.h>
#include <cinttypes>
#include <iostream>
double diff_timespec(const struct timespec *time1, const struct timespec * time0)
{
return (time1->tv_sec - time0->tv_sec) + (time1->tv_nsec - time0->tv_nsec) / 1000000000.0;
}
int main(int, char**)
{
int32_t last_seconds = 3; // = some value
uint32_t last_nanosec = 178922366; // = some value, e.g. 178922366
int32_t this_seconds = 6; // = some value
uint32_t this_nanosec = 58887157; // = some value, e.g. 58887157
struct timespec last{last_seconds, last_nanosec}, now{this_seconds, this_nanosec};
double delta_time = diff_timespec(&now, &last);
std::cout << delta_time << std::endl;
}
output: 2.87996
I need to execute multiple methods with specified interval(1 millisecond). Is there any way to call the methods in a loop without sleep because I am not getting 1000 entries in a log file for a second
Currently I have implemented using sleep. Data collection also might take little time.
#include <iostream>
#include <string>
#include <unordered_map>
#include <functional>
#include <csignal>
#include <chrono>
#include <thread>
#include <fstream>
#include <ctime>
#include <iomanip>
#include <sstream>
#include <windows.h>
#include <atomic>
volatile std::sig_atomic_t gStatus{};
void signalHandler(int sig) {
gStatus = sig;
}
void feature1(std::string &output) { output = "feature1"; }
void feature2(std::string &output) { output = "feature2"; }
void feature3(std::string &output) { output = "feature3"; }
void feature4(std::string &output) { output = "feature4"; }
void feature5(std::string &output) { output = "feature5"; }
void feature6(std::string &output) { output = "feature6"; }
void processInfoRequest(std::string logFile, std::unordered_map<std::string, std::function<void(std::string &value)>> methodMap)
{
std::ofstream ofs(logFile);
std::uint64_t prevTime = 0, curTime = 0, diffTime = 0, tmpTime = 0, scheduleTime = 0, pollPeriod = 1;
prevTime = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
while (gStatus != SIGINT)
{
prevTime = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
prevTime = prevTime - scheduleTime;
**//get value**
auto timenow = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
ofs << ctime(&timenow);
for(auto it : methodMap)
{
std::string output;
it.second(output);
ofs << output << ",";
}
ofs << "\n";
curTime = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
diffTime = curTime - prevTime;
if (diffTime < pollPeriod)
{
diffTime = pollPeriod - diffTime;
curTime = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
std::uint64_t i = 0;
std::uint64_t n = diffTime / 10;
std::uint64_t remainingMs = diffTime % 10;
for (i = 0; i < n; i++)
{
std::this_thread::sleep_for(std::chrono::milliseconds(10));
}
std::this_thread::sleep_for(std::chrono::milliseconds(remainingMs));
tmpTime = std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
scheduleTime = tmpTime - (curTime + diffTime);
}
else
{
scheduleTime = 0;
}
}
ofs.close();
}
int main()
{
std::signal(SIGINT, signalHandler);
std::string output;
std::unordered_map<std::string, std::function<void(std::string &value)>> methodMap = {
{"f1", feature1},
{"f2", feature2},
{"f3", feature3},
{"f4", feature4},
{"f5", feature5},
{"f6", feature6}
};
std::thread th1 = std::thread(processInfoRequest, "threadlog.csv", methodMap);
if (th1.joinable())
th1.join();
return 0;
}
I have simplified my code, and it compiles, but it doesn't do anything. It doesn't error out though either. I am trying to get 7 threads (on my 8-core processor) in this example to write to a variable to benchmark my system. I would like to do this with multiple threads to see if it's faster. It's based off other code that worked before I added multithreading. When I run, it just terminates. It should show progress each second of how many total iterations all the threads have done together. Some of the includes are there from other code I am working on.
I would like to also gracefully terminate all 7 threads when Ctrl-C is pressed. Help would be appreciated. Thanks!
//Compiled using: g++ ./test.cpp -lpthread -o ./test
#include <stdio.h>
#include <string>
#include <iostream>
#include <time.h>
#include <ctime>
#include <ratio>
#include <chrono>
#include <iomanip>
#include <locale.h>
#include <cstdlib>
#include <pthread.h>
using namespace std;
using namespace std::chrono;
const int NUM_THREADS = 7;
const std::string VALUE_TO_WRITE = "TEST";
unsigned long long int total_iterations = 0;
void * RunBenchmark(void * threadid);
class comma_numpunct: public std::numpunct < char > {
protected: virtual char do_thousands_sep() const {
return ',';
}
virtual std::string do_grouping() const {
return "\03";
}
};
void * RunBenchmark(void * threadid) {
unsigned long long int iterations = 0;
std::string benchmark;
int seconds = 0;
std::locale comma_locale(std::locale(), new comma_numpunct());
std::cout.imbue(comma_locale);
auto start = std::chrono::system_clock::now();
auto end = std::chrono::system_clock::now();
do {
start = std::chrono::system_clock::now();
while ((std::chrono::duration_cast < std::chrono::seconds > (end - start).count() != 1)) {
benchmark = VALUE_TO_WRITE;
iterations += 1;
}
total_iterations += iterations;
iterations = 0;
cout << "Total Iterations: " << std::setprecision(0) << std::fixed << total_iterations << "\r";
} while (1);
}
int main(int argc, char ** argv) {
unsigned long long int iterations = 0;
int tc, tn;
pthread_t threads[NUM_THREADS];
for (tn = 0; tn < NUM_THREADS; tn++) {
tc = pthread_create( & threads[tn], NULL, & RunBenchmark, NULL);
}
return 0;
}
I wrote a small test to compare file writing operations between boost file_mapping and std::ofstream. I was under the impression that file_mapping performance would be superior but it is apparently not the case.
Can someone explain why I would get better numbers with std::ofstream?
[EDIT]: So I did a profiling of my benchmark test and noticed that boost::iostreams::detail::direct_streambuf was spending lots of time copying bytes. I've added a new test which is using std::copy_n instead of ostream.write. The performance seems much better now. I have also updated the test code to compare with different file size.
The boost iostream direct_streambuf is really struggling on high volumes compared to std::copy_n. I'd like to find a better alternative instead as my app is based on ostream and I can't afford the refactoring.
#include <boost/interprocess/file_mapping.hpp>
#include <boost/interprocess/mapped_region.hpp>
#include <boost/iostreams/device/array.hpp>
#include <boost/iostreams/stream.hpp>
#include <vector>
#include <chrono>
#include <iostream>
#include <fstream>
int test_mapped_file_ostream(size_t TOTAL_SIZE, size_t BLOCK_SIZE, size_t N)
{
const std::string filename = "test_filemapping.dat";
boost::interprocess::file_mapping::remove(filename.data());
{
std::ofstream file(filename, std::ios::binary | std::ios::trunc);
file.seekp(static_cast<std::streamoff>(TOTAL_SIZE-1));
file.write("", 1);
}
std::chrono::system_clock::time_point start;
std::chrono::system_clock::time_point end;
{
boost::interprocess::file_mapping fmap(filename.data(), boost::interprocess::read_write);
boost::interprocess::mapped_region mreg(fmap, boost::interprocess::read_write);
mreg.advise( boost::interprocess::mapped_region::advice_sequential );
std::shared_ptr<std::streambuf> buf( new boost::iostreams::stream_buffer<boost::iostreams::array_sink>((char*)(mreg.get_address()), mreg.get_size()));
std::ostream ostream( buf.get() );
const std::vector<char> data(BLOCK_SIZE,1);
start=std::chrono::system_clock::now();
for ( size_t i=0; i<N; i++ ) {
ostream.write( data.data(), data.size() );
}
end=std::chrono::system_clock::now();
}
auto total = end-start;
std::cout << "test_mapped_file_ostream (ms): " << std::chrono::duration_cast<std::chrono::milliseconds>(total).count() << std::endl;
return 0;
}
int test_mapped_file_stdcopy_n(size_t TOTAL_SIZE, size_t BLOCK_SIZE, size_t N)
{
const std::string filename = "test_filemapping_stdcopy.dat";
boost::interprocess::file_mapping::remove(filename.data());
{
std::ofstream file(filename, std::ios::binary | std::ios::trunc);
file.seekp(static_cast<std::streamoff>(TOTAL_SIZE-1));
file.write("", 1);
}
std::chrono::system_clock::time_point start;
std::chrono::system_clock::time_point end;
{
boost::interprocess::file_mapping fmap(filename.data(), boost::interprocess::read_write);
boost::interprocess::mapped_region mreg(fmap, boost::interprocess::read_write);
mreg.advise( boost::interprocess::mapped_region::advice_sequential );
char* regptr = (char*)mreg.get_address();
const std::vector<char> data(BLOCK_SIZE,1);
start=std::chrono::system_clock::now();
for ( size_t i=0; i<N; i++ ) {
std::copy_n( data.data(), data.size(), regptr );
regptr += data.size();
}
end=std::chrono::system_clock::now();
}
auto total = end-start;
std::cout << "test_mapped_file_stdcopy_n (ms): " << std::chrono::duration_cast<std::chrono::milliseconds>(total).count() << std::endl;
return 0;
}
int test_fstream_file(size_t TOTAL_SIZE, size_t BLOCK_SIZE, size_t N)
{
const std::string filename = "test_fstream.dat";
std::chrono::system_clock::time_point start;
std::chrono::system_clock::time_point end;
{
const std::vector<char> data(BLOCK_SIZE,1);
std::ofstream file(filename, std::ios::binary | std::ios::trunc);
start=std::chrono::system_clock::now();
for ( size_t i=0; i<N; i++ ) {
file.write( data.data(), data.size() );
}
end=std::chrono::system_clock::now();
}
auto total = end-start;
std::cout << "test_fstream_file (ms): " << std::chrono::duration_cast<std::chrono::milliseconds>(total).count() << std::endl;
return 0;
}
int main(int argc, char **argv)
{
if ( argc != 2 ) {
std::cout << "Usage: " << argv[0] << " <size of output file in gigabytes>" << std::endl;
exit(1);
}
uint64_t totalsize = std::stoull(argv[1]);
if (totalsize==0) {
totalsize = 1;
}
const std::size_t GB = (uint64_t)1 << 30;
const std::size_t TOTAL_SIZE = totalsize << 30;
const std::size_t BLOCK_SIZE = (uint64_t)1 << 20;
const std::size_t N = TOTAL_SIZE/BLOCK_SIZE;
std::cout << "TOTAL_SIZE (GB)=" << TOTAL_SIZE/GB << std::endl;
test_mapped_file_ostream(TOTAL_SIZE,BLOCK_SIZE,N);
test_mapped_file_stdcopy_n(TOTAL_SIZE,BLOCK_SIZE,N);
test_fstream_file(TOTAL_SIZE,BLOCK_SIZE,N);
return 0;
}
results: Windows 7, HHD, 64gb RAM
Performance ratios compared to fstream.write in (ms):
TOTAL_SIZE (GB)=5
test_mapped_file_ostream (ms): 24610 (-1.88x)
test_mapped_file_stdcopy_n (ms): 3307 (3.9x)
test_fstream_file (ms): 13052
TOTAL_SIZE (GB)=10
test_mapped_file_ostream (ms): 49524 (-1.3x)
test_mapped_file_stdcopy_n (ms): 6610 (5.8x)
test_fstream_file (ms): 38219
TOTAL_SIZE (GB)=15
test_mapped_file_ostream (ms): 85041 (1.52x)
test_mapped_file_stdcopy_n (ms): 12387 (10.5x)
test_fstream_file (ms): 129964
TOTAL_SIZE (GB)=20
test_mapped_file_ostream (ms): 122897 (1.7x)
test_mapped_file_stdcopy_n (ms): 17542 (12.2x)
test_fstream_file (ms): 213697
Profiling
You're using a text oriented ostream anyways. This is going to account for a large portion of the time taken formatting to the stream.
Other than that consider madvising for sequential access.
Finally profile to find your bottle necks
I've hit this problem with all the tricks I know and came up with the following really bare-bones POSIX mmap vs.write comparison.
I used madvise and fadvise with SEQUENTIAL|WILL_NEED where applicable, and made sure that sparseness wasn't a cause for slowness.
The short summary of it all is:
your code could really be a lot simpler (see rev. 176f546ea8f65050c)
the maps are fast for smaller volumes
the buffering is probably what makes the streambased implementations shine instead of the mmap based ones
Live On Coliru
#include <boost/chrono.hpp>
#include <boost/chrono/chrono_io.hpp>
#include <iostream>
#include <vector>
#include <algorithm>
// mmap the manual way
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#ifndef COLIRU
const std::size_t TOTAL_SIZE = 5ul << 30;
const std::size_t BLOCK_SIZE = 1ul << 20;
#else
const std::size_t TOTAL_SIZE = 1ul << 20;
const std::size_t BLOCK_SIZE = 1ul << 9;
#endif
static_assert(0 == TOTAL_SIZE%BLOCK_SIZE, "not divisable by block size");
const int N = TOTAL_SIZE/BLOCK_SIZE;
template <typename Caption, typename F>
auto timed(Caption const& task, F&& f) {
using namespace boost::chrono;
struct _ {
high_resolution_clock::time_point s;
Caption const& task;
~_() { std::cout << " -- (" << task << " completed in " << duration_cast<milliseconds>(high_resolution_clock::now() - s) << ")\n"; }
} timing { high_resolution_clock::now(), task };
return f();
}
void test_mapped_file() {
std::vector<char> const data(BLOCK_SIZE, 1);
const std::string filename = "test_filemapping.dat";
std::remove(filename.c_str());
int fd = open(filename.c_str(), O_RDWR|O_CREAT, 0644);
if (fd==-1) {
perror("open");
exit(255);
}
if(posix_fallocate64(fd, 0, TOTAL_SIZE)) {
perror("fallocate64");
exit(255);
}
posix_fadvise64(fd, 0, TOTAL_SIZE, POSIX_FADV_WILLNEED | POSIX_FADV_SEQUENTIAL);
char* fmap = static_cast<char*>(mmap64(nullptr, TOTAL_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0));
if (!fmap || fmap == MAP_FAILED) {
perror("mmap");
exit(255);
}
madvise(fmap, TOTAL_SIZE, MADV_SEQUENTIAL | MADV_WILLNEED);
timed(filename, [output=fmap, &data] () mutable {
for (size_t i = 0; i < N; i++) {
std::copy_n(data.data(), data.size(), output);
output += data.size();
}
});
munmap(fmap, TOTAL_SIZE);
close(fd);
}
void test_posix_write() {
std::vector<char> const data(BLOCK_SIZE, 1);
const std::string filename = "test_posix.dat";
std::remove(filename.c_str());
int fd = open(filename.c_str(), O_RDWR|O_CREAT, 0644);
if (fd==-1) {
perror("open");
exit(255);
}
posix_fadvise64(fd, 0, TOTAL_SIZE, POSIX_FADV_WILLNEED | POSIX_FADV_SEQUENTIAL);
timed(filename, [&] () mutable {
for (size_t i = 0; i < N; i++) {
ptrdiff_t count = ::write(fd, data.data(), data.size());
if (-1 == count) {
perror("write");
exit(255);
}
assert(count == BLOCK_SIZE);
}
});
close(fd);
}
int main() {
test_mapped_file();
test_posix_write();
}
When tested on Coliru prints:
./a.out; md5sum *.dat
-- (test_filemapping.dat completed in 0 milliseconds)
-- (test_posix.dat completed in 8 milliseconds)
d35bb2e58b602d94ccd9628f249ae7e5 test_filemapping.dat
d35bb2e58b602d94ccd9628f249ae7e5 test_posix.dat
Run locally (5GiB volumes):
$ ./test
-- (test_filemapping.dat completed in 1950 milliseconds)
-- (test_posix.dat completed in 1307 milliseconds)
You might be experiencing thrashing, which will significantly slow down the time needed to write the file via memory mapping. Your benchmark writes out nearly 5 gigabytes of data. If you do not have 5 gigabytes of RAM available, then the OS will be busy exchanging dirtied pages in memory for data on disk.
As sehe suggested, you might consider madvising because in this scenario, you are sequentially accessing the memory-mapped file:
mreg.advise( boost::interprocess::mapped_region::advice_sequential );
However, note that this will not fix thrashing problems.
I am using below method to validate Date.
How to format month in string ?
bool CDateTime :: IsValidDate(char* pcDate) //pcDate = 25-Jul-2012 15:08:23
{
bool bVal = true;
int iRet = 0;
struct tm tmNewTime;
iRet = sscanf_s(pcDate, "%d-%d-%d %d:%d:%d", &tmNewTime.tm_mon, &tmNewTime.tm_mday, &tmNewTime.tm_year, &tmNewTime.tm_hour, &tmNewTime.tm_min, &tmNewTime.tm_sec);
if (iRet == -1)
bVal = false;
if (bVal == true)
{
tmNewTime.tm_year -= 1900;
tmNewTime.tm_mon -= 1;
bVal = IsValidTm(&tmNewTime);
}
return bVal;
}
Using strptime:
#include <time.h>
char *str = "25-Jul-2012 15:08:23";
struct tm tm;
if (strptime (str, "%d-%b-%Y %H:%M:%S", &tm) == NULL) {
/* Bad format !! */
}
The C++11 way of doing this is:
#include <iostream>
#include <iomanip>
#include <ctime>
#include <chrono>
int main()
{
auto now = std::chrono::system_clock::now();
auto now_c = std::chrono::system_clock::to_time_t(now);
std::cout << "Now is " << std::put_time(std::localtime(&now_c), "%d-%b-%Y %H:%M:%S") << '\n';
}
Note: The stream I/O manipulator std::put_time is not implemented fully in all compilers yet. GCC 4.7.1 does not have it for example.