Boost log rotating file - size exceeded - c++

I try to log with Boost library 1.61, rotating file once size exceeded 5Ko. But, every time program runs, it begins from the first log file, exceeding max size !
That is, it appends text to flog000.txt, then to flog001.txt.
But the desired behaviour was: append to the last file (flog002.txt) then continue with flog003.txt...
Here is my code:
#include <boost/log/common.hpp>
#include <boost/log/sinks/text_ostream_backend.hpp>
#include <boost/log/core.hpp>
#include <boost/log/utility/setup/file.hpp>
#include <boost/log/sources/logger.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
void init()
{
boost::log::add_file_log
(
boost::log::keywords::file_name = "flog%3N.txt",
boost::log::keywords::open_mode = std::ios_base::app,
boost::log::keywords::rotation_size = 5 * 1024,
boost::log::keywords::auto_flush = true
);
}
int main()
{
init();
boost::log::sources::logger lg;
boost::log::add_common_attributes();
for (int i(0); i < 1000; ++i)
{
BOOST_LOG(lg) << "Hello : " << i << std::endl;
}
return 0;
}

"There's no file collecting" - this is at least a part of the problem.
It does seem impossible to have scanning and append to an existing as well.
However, if you do scanning, you can at least prevent extra log messages from being wrongly appended to the older log files:
Live On Coliru
#include <boost/log/common.hpp>
#include <boost/log/core.hpp>
#include <boost/log/sinks/text_ostream_backend.hpp>
#include <boost/log/sources/logger.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>
#include <boost/log/utility/setup/file.hpp>
void init() {
auto s = boost::log::add_file_log(
boost::log::keywords::file_name = "flog%3N.txt",
boost::log::keywords::open_mode = std::ios_base::app,
boost::log::keywords::rotation_size = 5 * 1024,
boost::log::keywords::auto_flush = true
);
auto be = s->locked_backend();
be->set_file_collector(boost::log::sinks::file::make_collector(
boost::log::keywords::target = "./", // log file & target have same dir
boost::log::keywords::max_size = 50 * 1024,
boost::log::keywords::min_free_space = 100000
));
be->scan_for_files(boost::log::sinks::file::scan_method::scan_matching, true);
}
int main() {
init();
boost::log::sources::logger lg;
boost::log::add_common_attributes();
for (int i(0); i < 1000; ++i) {
BOOST_LOG(lg) << "Hello : " << i << std::endl;
}
}
After one run:
-rw-rw-r-- 1 sehe sehe 5116 jun 6 00:27 flog000.txt
-rw-rw-r-- 1 sehe sehe 5109 jun 6 00:27 flog001.txt
-rw-rw-r-- 1 sehe sehe 2665 jun 6 00:27 flog002.txt
After two runs:
-rw-rw-r-- 1 sehe sehe 5116 jun 6 00:27 flog003.txt
-rw-rw-r-- 1 sehe sehe 5109 jun 6 00:27 flog004.txt
-rw-rw-r-- 1 sehe sehe 2665 jun 6 00:27 flog005.txt
After three runs:
-rw-rw-r-- 1 sehe sehe 5116 jun 6 00:28 flog006.txt
-rw-rw-r-- 1 sehe sehe 5109 jun 6 00:28 flog007.txt
-rw-rw-r-- 1 sehe sehe 2665 jun 6 00:28 flog008.txt

Related

boost asio thread_group only run one thread

I am studing boost library. I want to use fiber with blocking api so I try to use a thread pool to run blocking api. When the blocking api is finished, the fiber get the result. I have created a thread pool of 4 threads, but I found that only one thread is running. Why?
Here is my code.
#include <boost/asio.hpp>
#include <boost/fiber/all.hpp>
#include <boost/fiber/fiber.hpp>
#include <boost/fiber/future/future.hpp>
#include <boost/fiber/future/promise.hpp>
#include <boost/fiber/operations.hpp>
#include <chrono>
#include <ctime>
#include <iostream>
#include <string>
#include <thread>
#include <unistd.h>
using namespace std;
using namespace boost;
time_t block_api() {
std::this_thread::sleep_for(std::chrono::seconds(1));
return clock();
}
class AsyncWrapper {
public:
AsyncWrapper(asio::thread_pool &pool) : pool_(&pool) {}
~AsyncWrapper() {}
time_t async_api() {
fibers::promise<time_t> promise;
boost::fibers::future<time_t> future(promise.get_future());
auto func = [&]() mutable {
auto ret = block_api();
std::cout << "run on:"<<getpid()<<std::endl;
promise.set_value(ret);
};
asio::post(pool_->get_executor(), func);
return future.get();
}
private:
asio::thread_pool *pool_ = nullptr;
};
static asio::thread_pool g_pool(4);
int main(int argc, char **argv) {
if (argc != 2) {
return -1;
}
std::string arg = argv[1];
std::cout << "press enter to start" << std::endl;
getchar();
for (int32_t i = 0; i < stoi(arg); i++) {
fibers::fiber f([]() {
AsyncWrapper async_wrapper(g_pool);
auto ret = async_wrapper.async_api();
});
f.detach();
}
this_fiber::yield();
return 0;
}
When I use command to analysis my program ,I saw that
top - 17:32:03 up 5:29, 0 users, load average: 0.19, 0.17, 0.20
Threads: 5 total, 1 running, 4 sleeping, 0 stopped, 0 zombie
%Cpu(s): 8.3 us, 0.0 sy, 0.0 ni, 91.6 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 7931.7 total, 5037.4 free, 2405.4 used, 488.9 buff/cache
MiB Swap: 2048.0 total, 1259.0 free, 789.0 used. 5229.7 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10338 wuhaotao 20 0 431880 15732 3424 R 99.7 0.2 0:03.63 with_blocking_a
10339 wuhaotao 20 0 431880 15732 3424 S 0.0 0.2 0:00.00 with_blocking_a
10340 wuhaotao 20 0 431880 15732 3424 S 0.0 0.2 0:00.00 with_blocking_a
10341 wuhaotao 20 0 431880 15732 3424 S 0.0 0.2 0:00.00 with_blocking_a
10342 wuhaotao 20 0 431880 15732 3424 S 0.0 0.2 0:00.00 with_blocking_a
Only the 10338 thread is running......

Check all files sizes in a path (C++)

I'm trying to loop so that my program can get the weight of all files in a folder, and if the weight of any of these is equal to X, it will do an action, I need to know how I can loop like this, and i have a func to know whats the file size
std::ifstream::pos_type filesize(const char* filename)
{
std::ifstream in(filename, std::ifstream::ate | std::ifstream::binary);
return in.tellg();
}
Here's a short example program that demonstrates how to use C++17's <filesystem> library to iterate over a directory. If your compiler is fairly up-to-date, it should support C++17 without issue.
#include <filesystem>
#include <iostream>
int main() {
namespace fs = std::filesystem;
fs::path pwd(""); // Current directory program was executed from
pwd = fs::absolute(pwd);
for (auto& i : fs::directory_iterator(pwd)) {
try {
if (fs::file_size(i.path()) / 1024 > 2048) {
std::cout << i.path() << " is larger than 2MB\n";
}
} catch (fs::filesystem_error& e) {
std::cerr << e.what() << '\n';
}
}
}
This was the contents of the directory:
.
├── a.out
├── fiveKB
├── fourMB
├── main.cpp
└── oneMB
0 directories, 5 files
And information about the files:
drwxr-xr-x 7 user staff 224B Jul 29 22:11 ./
drwxr-xr-x 13 user staff 416B Jul 29 21:59 ../
-rwxr-xr-x 1 user staff 47K Jul 29 22:10 a.out*
-rw-r--r-- 1 user staff 5.0K Jul 29 21:58 fiveKB
-rw-r--r-- 1 user staff 4.0M Jul 29 21:59 fourMB
-rw-r--r-- 1 user staff 450B Jul 29 22:11 main.cpp
-rw-r--r-- 1 user staff 1.0M Jul 29 21:59 oneMB
And finally, the output:
"/Users/user/Documents/tmp/test/fourMB" is larger than 2MB

Writing last n lines from one file to another

This is the program I have written which copies all the lines from diskData.dat file to 24HrDiskData.dat file. As of now I am copying all the lines from one file to another I wish to copy last n lines from diskData.dat to 24HrDiskData.dat
#include <iostream>
#include <stdio.h>
#include <fstream>
#include <stdlib.h>
using namespace std;
int main(int argc, char *argv[]) {
FILE *HrData;
char tempData[1024];
int flag = 0;
ofstream fout;
fout.open("24HrDiskData.dat", ios::app); // open file for appending
assert (!fout.fail());
if ((HrData = fopen("/home/xvishuk/diskData.dat", "r")) == NULL) {
printf("\nFile cannot be opened");
}
while (fgets(tempData, 1024, HrData) != NULL) {
fout << tempData; // write the data to the file
}
return 0;
}
Sample pattern of Delimited data
|Sat Mar 26 18:47:57 2016|1|1|182|60.66|3|30|4782|31|68|4|3467750|110|43.1562|64|
|Sat Mar 26 19:01:49 2016|1|1|140|46.26|3.03|30|4782|30|68|4|3467764|96|43.1562|64|
|Sat Mar 26 19:15:40 2016|1|1|184|61.07|3.01|30|4782|30|68|4|3467777|112|43.1562|64|
|Sat Mar 26 19:29:30 2016|1|1|180|59.91|3|30|4782|32|68|4|3467791|98|43.1562|64|
|Sat Mar 26 19:43:20 2016|1|1|194|64.61|3|30|4782|32|68|4|3467805|114|43.1562|64|
|Sat Mar 26 19:57:17 2016|1|1|170|56.62|3|30|4782|30|68|4|3467818|102|43.1562|64|
|Sat Mar 26 20:11:14 2016|1|1|140|46.32|3.02|30|4782|30|68|4|3467832|118|43.1562|64|
|Sat Mar 26 20:25:12 2016|1|1|176|58.35|3.02|30|4782|30|68|4|3467846|104|43.1562|64|
|Sat Mar 26 20:39:10 2016|1|1|202|66.9|3.02|30|4782|30|68|4|3467859|120|43.1562|64|
|Sat Mar 26 20:53:11 2016|1|1|198|65.85|3.01|30|4782|31|68|4|3467873|106|43.1562|64|
|Sat Mar 26 21:07:12 2016|1|1|184|60.97|3.02|30|4782|32|68|4|3467887|92|43.1562|64|
|Sat Mar 26 21:21:11 2016|1|1|152|50.28|3.02|30|4782|31|68|4|3467901|108|43.1562|64|
|Sat Mar 26 21:35:16 2016|1|1|168|55.77|3.01|30|4782|30|68|4|3467915|94|43.1562|64|
|Sat Mar 26 21:49:20 2016|1|1|172|57.03|3.02|30|4782|31|68|4|3467928|112|43.1562|64|
|Sat Mar 26 22:03:26 2016|1|1|152|50.56|3.01|30|4782|33|68|4|3467942|98|43.1562|64|
|Sat Mar 26 22:17:32 2016|1|1|174|57.86|3.01|30|4782|31|68|4|3467956|114|43.1562|64|
|Sat Mar 26 22:31:38 2016|1|1|156|51.86|3.01|30|4782|30|68|4|3467970|100|43.1562|64|
|Sat Mar 26 22:45:44 2016|1|1|202|66.57|3.03|30|4782|30|68|4|3467984|116|43.1562|64|
|Sat Mar 26 22:59:55 2016|1|1|188|62.4|3.01|30|4782|31|68|4|3467998|102|43.1562|64|
|Sat Mar 26 23:14:06 2016|1|1|164|53.95|3.04|30|4782|32|68|4|3468012|118|43.1562|64|
|Sat Mar 26 23:28:17 2016|1|1|168|55.78|3.01|30|4782|31|68|4|3468026|104|43.1562|64|
|Sat Mar 26 23:42:28 2016|1|1|176|58.33|3.02|30|4782|30|68|4|3468040|120|43.1562|64|
|Sat Mar 26 23:56:39 2016|1|1|170|55.76|3.05|30|4782|30|68|4|3468053|106|43.1562|64|
|Sun Mar 27 00:10:54 2016|1|1|172|56.97|3.02|30|4782|31|68|4|3468067|92|43.1562|64|
|Sun Mar 27 00:25:13 2016|1|1|184|61.23|3.01|30|4782|33|68|4|3468081|108|43.1562|64|
|Sun Mar 27 00:39:26 2016|1|1|188|62.12|3.03|30|4782|31|68|4|3468096|94|43.1562|64|
|Sun Mar 27 00:53:42 2016|1|1|170|56.42|3.01|30|4782|30|68|4|3468110|110|43.1562|64|
|Sun Mar 27 01:07:58 2016|1|1|174|57.8|3.01|30|4782|31|68|4|3468124|96|43.1562|64|
Expected Output
The last 10 records.
|Sat Mar 26 22:45:44 2016|1|1|202|66.57|3.03|30|4782|30|68|4|3467984|116|43.1562|64|
|Sat Mar 26 22:59:55 2016|1|1|188|62.4|3.01|30|4782|31|68|4|3467998|102|43.1562|64|
|Sat Mar 26 23:14:06 2016|1|1|164|53.95|3.04|30|4782|32|68|4|3468012|118|43.1562|64|
|Sat Mar 26 23:28:17 2016|1|1|168|55.78|3.01|30|4782|31|68|4|3468026|104|43.1562|64|
|Sat Mar 26 23:42:28 2016|1|1|176|58.33|3.02|30|4782|30|68|4|3468040|120|43.1562|64|
|Sat Mar 26 23:56:39 2016|1|1|170|55.76|3.05|30|4782|30|68|4|3468053|106|43.1562|64|
|Sun Mar 27 00:10:54 2016|1|1|172|56.97|3.02|30|4782|31|68|4|3468067|92|43.1562|64|
|Sun Mar 27 00:25:13 2016|1|1|184|61.23|3.01|30|4782|33|68|4|3468081|108|43.1562|64|
|Sun Mar 27 00:39:26 2016|1|1|188|62.12|3.03|30|4782|31|68|4|3468096|94|43.1562|64|
|Sun Mar 27 00:53:42 2016|1|1|170|56.42|3.01|30|4782|30|68|4|3468110|110|43.1562|64|
To only write the last n lines, you could allocate an array of n strings and store the lines into it as you read them, only keeping the last n when the array fills up, using modular index to avoid unnecessary copying.
When you reach the end of file, output the lines from the array.
Also avoid mixing C and C++. Either use <stdio.h> or <iostream>, but using both at the same time is error prone and inelegant.
Here is a simple implementation in C:
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define N 100
int main(void) {
char line[1024];
char *stash[N] = { NULL };
int i, j;
FILE *HrData, *fout;
if ((fout = fopen("24HrDiskData.dat", "a")) == NULL) {
fprintf(stderr, "cannot open 24HrDiskData.dat for appending: %s\n",
strerror(errno));
return 1;
}
if ((HrData = fopen("/home/xvishuk/diskData.dat", "r")) == NULL) {
fprintf(stderr, "cannot open input file: %s\n",
strerror(errno));
return 1;
}
for (i = 0; fgets(line, 1024, HrData) != NULL;) {
free(stash[i]);
stash[i] = strdup(line);
i = (i + 1) % N;
}
fclose(HrData);
for (j = i;;) {
if (stash[i])
fputs(stash[i], fout);
i = (i + 1) % N;
if (i == j)
break;
}
fclose(fout);
return 0;
}
Two passes.
First pass, count the lines and store their file positions.
Second pass, calculate the line number you want to start copying from. Look up its file position in the container. Seek to the file position. Start copying.
Edit 1: Example code
#include <iostream>
#include <fstream>
#include <string>
#include <vector>
int main(void)
{
std::string text_line;
std::vector<std::streampos> file_positions;
std::ifstream input_file("my_file.txt");
file_positions.push_back(input_file.tellg());
while (std::getline(input_file, text_line))
{
file_positions.push_back(input_file.tellg());
}
// The total lines is file_positions.size().
// Return the last 13 lines
const unsigned int total_lines = file_positions.size();
std::streampos seek_position = 0;
unsigned int index = 0U;
if (total_lines > 13U)
{
index = total_lines - 13U;
}
input_file.clear();
input_file.seekg(file_positions[index]);
// Copy text lines to output file...
return 0;
}
Edit 2 - More efficient method
You could maintain a container of N text lines as you read through the file.
#include <deque>
//...
int main(void)
{
std::deque<std::string> text_lines(N);
while (std::getline(input_file, text_line))
{
if (text_lines.size() == N)
{
text_lines.pop_front();
}
text_lines.push_back(text_line);
}
// Now copy the text lines from the `std::deque` to the output file.
//...
return 0;
}

trying to connect to postgresql via c++ program

Background
I'm running linux... and I'm trying to write a basic little c++ program that connects to a postgresql database.
I'm trying to follow this article
http://www.tutorialspoint.com/postgresql/postgresql_c_cpp.htm
Problem
I've been able to compile the library... and I can see now that I have the following folder on my computer
/usr/local/include/pqxx
But when i try to write some basic code and compile it, I get the following error:
devbox2:/var/abus# g++ testdb.cpp -lpqxx -lpq
testdb.cpp:2:22: fatal error: pqxx/pqxx: No such file or directory
#include <pqxx/pqxx>
^
compilation terminated.
Source Code
Here's what the code looks like:
1 #include <iostream>
2 #include <pqxx/pqxx>
3
4 using namespace std;
5 using namespace pqxx;
6
7 int main(int argc, char* argv[])
8 {
9 try{
10 connection C("dbname=testdestination user=testuser password=testpassword \
11 hostaddr=127.0.0.1 port=5432");
12 if (C.is_open()) {
13 cout << "Opened database successfully: " << C.dbname() << endl;
14 } else {
15 cout << "Can't open database" << endl;
16 return 1;
17 }
18 C.disconnect ();
19 }catch (const std::exception &e){
20 cerr << e.what() << std::endl;
21 return 1;
22 }
23 }
What I've tried so far:
I've been poking around the /usr/local/include/pqxx folder and I can see that there is a file called pqxx... but it doesn't have any extension on it.
Here's a snippet from the ls -lah command for that folder:
-rw-r--r-- 1 root root 637 Dec 8 21:42 pipeline
-rw-r--r-- 1 root root 7.5K Dec 8 21:42 pipeline.hxx
-rw-r--r-- 1 root root 1.1K Dec 8 21:42 pqxx
-rw-r--r-- 1 root root 728 Dec 8 21:42 prepared_statement
-rw-r--r-- 1 root root 8.2K Dec 8 21:42 prepared_statement.hxx
I've also made sure that my PATH includes the /usr/local/include/pqxx folder. This is what my PATH looks like:
PATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/gcc:/usr/local/include/pqxx:/usr/local/include'
I'm not sure what else I should check. Any suggestions would be appreciated.
Thanks.
To find the include files, you must add an -I option, e.g.
g++ -I/usr/local/include testdb.cpp -lpqxx -lpq
Adding directories to PATH doesn't help here, PATH is for locating executables from the shell.

What do /proc/fd file descriptors show?

Learning about the /proc/ directory today, in particular I'm interested in the security implications of having all the information about a process semi-publicly available, so I wrote a simple program that does some simple whatnot that allows me to explore some properties of the /proc/ directory:
#include <iostream>
#include <unistd.h>
#include <fcntl.h>
using namespace std;
extern char** environ;
void is_linux() {
#ifdef __linux
cout << "this is running on linux" << endl;
#endif
}
int main(int argc, char* argv[]) {
is_linux();
cout << "hello world" << endl;
int fd = open("afile.txt", O_RDONLY | O_CREAT, 0600);
cout << "afile.txt open on: " << fd << endl;
cout << "current pid: " << getpid() << endl;;
cout << "launch arguments: " << endl;
for (int index = 0; index != argc; ++index) {
cout << argv[index] << endl;
}
cout << "program environment: " << endl;
for (char** entry = environ; *entry; ++entry) {
cout << *entry << endl;
}
pause();
}
Interestingly though (to me anyway), when I check the file-descriptors folder (/pid/<PID#>/fd), I see this:
root#excalibur-VirtualBox:/proc/1546/fd# ls -l
total 0
lrwx------ 1 root root 64 Nov 7 09:12 0 -> /dev/null
lrwx------ 1 root root 64 Nov 7 09:12 1 -> /dev/null
lrwx------ 1 root root 64 Nov 7 09:12 2 -> /dev/null
lrwx------ 1 root root 64 Nov 7 09:12 3 -> socket:[11050]
why do the file descriptors point to /dev/null? Is that to prevent user's from being able to inject content into a file without actually being the process itself, or am I off base on that? And even more curious, why does the file descriptor to an open file point to a socket? That seems really odd. If anyone can shed some light on this for me, I would really appreciate it. Thanks!
You are definitely looking at the wrong /proc directory (for other PID or on another computer). The contents of /proc/<pid>/fd for your program should look like here:
lrwx------ 1 user group 64 Nov 7 22:15 0 -> /dev/pts/4
lrwx------ 1 user group 64 Nov 7 22:15 1 -> /dev/pts/4
lrwx------ 1 user group 64 Nov 7 22:15 2 -> /dev/pts/4
lr-x------ 1 user group 64 Nov 7 22:15 3 -> /tmp/afile.txt
Here we can see that file descriptors 0, 1, and 2 are shown as symbolic links to the pseudo terminal in which the program is running. It could be /dev/null if you started your program with input, output, and error redirection. The file descriptor #3 points to the file afile.txt which is currently opened.