I am trying to write my first program in C++, and I need to use Boost library. I am trying to write a program which recursively goes through a directory tree and returns the date of the newest and the oldest file.
This is where I'm now:
#define BOOST_FILESYSTEM_VERSION 3
#include "boost/filesystem.hpp"
#include <iostream>
#include <ctime>
using namespace std;
namespace fs = boost::filesystem;
int main() {
fs::recursive_directory_iterator it_end;
fs::recursive_directory_iterator it_dir("e:\\");
fs::path p;
time_t oldest( time(NULL) );
time_t newest(0);
try {
for ( ; it_dir != it_end; ++it_dir ) {
p = *it_dir;
try {
time_t t( last_write_time(p) );
if (t<oldest) oldest=t;
if (t>newest) newest=t;
if (fs::is_directory(p)) cout << (p) << " " << t << endl;
}
catch (const fs::filesystem_error& ex) {
cout << "\n" << ex.what() << "\n";
}
}
}
catch (const fs::filesystem_error& ex) {
cout << "\n" << ex.what() << "\n";
}
cout << "\nOldest: " << ctime(&oldest);
cout << "Newest: " << ctime(&newest) << endl;
return 0;
}
The problems I've met are that:
1.When I encounter a too long path (more than 256 or 260 characters, I think), there is an error:
boost::filesystem::last_write_time: The system cannot find the path specified:
2.When I meet with a non accessable directory, like "System Volume Information", I have two more:
boost::filesystem::last_write_time: Access is denied: "e:\System Volume Information"
boost::filesystem::directory_iterator::construct: Access is denied: "e:\System Volume Information"
How can I modify the code above to handle long path names under Windows? Is it really hard to do it? Some programs, like Total Commander for example has no problems with long paths, but many programs still have.
The more important question is that how can I actually make the above code work (not caring about long paths). The problem is that when for ( ; it_dir != it_end; ++it_dir ) meets with a not accessible directory, it throws an exception, and to catch this exception, I need to define the outside catch. But when I'm outside it means that the for cycle is not continuing. So it means that the above code works as far as the first not accessible folder. There it throws an exception and ends.
Is there any way to go back into the for cycle after an exception has been thrown?
My idea is to do a ++it_dir inside the catch and start the for cycle again. But how can I start it again? Shell I move it out to a separate function?
Sorry if my understanding is not clear, it's my first project. I never used C++ before but I'm trying my best!
EDIT:
Any other answer? The problem is that the catch is not working inside the cycle for "not accessible" kind of errors. How can I make it work inside? Here is the smallest code producing the error. Is there any way to catch this error inside the for cycle? Or catch it in a way that it could continue after skipping the non-accessible element with a it_dir++?
int main() {
fs::recursive_directory_iterator it_end;
fs::recursive_directory_iterator it_dir("e:\\");
for ( ; it_dir != it_end; ++it_dir ) {
//something here
}
}
It turned out it is a bug in boost. I've found a bug support ticket for it and contributed to it.
Just put the try/catch inside the for loop...
Related
I created a C++ wrapper to access my Python modules. everything is working until I try to use threads in my application.
On my Python module there is a method which reads from a webcam (so its uses an infinite loop) and I send callbacks from C++ to get the image and other needed information from it.
Since we have a blocking method here, I decided to use threads.
The threading on Python part seems not to be working on the C++ side that is if I call the async counter part of the webcam_feed loop, none of my callbacks are actually executed (on python part the routines are all executed however, it seems it doesn't reach to C++ section somehow. I don't get any feedback in C++ side, however, on Python part, those routines responsible for executing the callbacks save the info to the disk so I know for sure they are executed).
I asked a separate question for it here.
Therefore I decided to use the threading inside C++ client. However, whenever I execute the code (given below), I get an access violation whenever I want to use any methods after the thread is started.
Here are the sample callbacks I have for now:
void default_callback(bool status, std::string id, py::array_t<uint8_t>& img)
{
auto rows = img.shape(0);
auto cols = img.shape(1);
auto type = CV_8UC3;
cv::Mat img1(rows, cols, type, img.mutable_data());
cv::imshow("from callback", img1);
cv::waitKey(1);
auto timenow = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
std::cout << "\narg1: " << status << " arg2: " << id << " arg3: " << typeid(img).name() << " " << ctime(&timenow) << std::endl;
}
void default_c_callback_temporary(bool status, char* message)
{
std::cout << "status is: " << status << " id/name: " << message << " ptr:" << "" << std::endl;
std::ofstream myfile;
myfile.open("example.txt");
myfile << "Writing this to a file: " << status << message << std::endl;
myfile.close();
}
And this is the actual test
void thread_test_start(Core* core)
{
try
{
core->SetCpuAffinity(2);
core->AddCallback(default_callback);
core->AddCallback_C_tmp(default_c_callback_temporary);
//set true to run the async version (implemented in python)
core->Start(false);
}
catch (const std::exception& ex)
{
std::cout << ex.what() << std::endl;
}
}
int main()
{
Core* core = new Core(false);
std::thread t(thread_test_start, core);
py::print(core->GetCallbacks());
std::cout << "\nGet C Callbacks:\n";
py::print(core->GetCallbacks_C_tmp());
std::cout << "\nEverything done. press Enter to Exit";
t.join();
std::getchar();
return 0;
}
The call to core->GetCallbacks() causes the memory access violation:
Exception thrown at 0x000000006FCC6D80 (python36.dll) in TestDLL.exe: 0xC0000005: Access violation reading location 0x0000000000000010.
And here is a snapshot showing the access violation error inside VS2019:
Doing something like this is also the same :
void thread_test_start2()
{
try
{
Core* core = new Core(false);
core->SetCpuAffinity(2);
core->AddCallback(default_callback);
core->AddCallback_C_tmp(default_c_callback_temporary);
std::thread t(&Core::Start, core, false);
py::print(core->GetCallbacks());
std::cout << "\nGet C Callbacks:\n";
py::print(core->GetCallbacks_C_tmp());
t.join();
}
catch (const std::exception& ex)
{
std::cout << ex.what() << std::endl;
}
}
results in :
Exception thrown at 0x000000006FCC0CDF (python36.dll) in TestDLL.exe: 0xC0000005: Access violation writing location 0x0000000000000020.
like the former one.
Why am I getting this error ? Can we not use threading with Pybind11? What am I missing here?
Here is a sample project to re-create this issue : https://workupload.com/file/6LmfRtbztHK
The reason for memory access violations were due to trying to run methods using different threads. That is, all Pybind11 related methods (methods that use Pybind11) need to be executed under the very same thread it seems.
Therefore executing some portion of the code under one thread and trying to execute some other methods in the main thread will result in memory access violation.
In order to get around this, I ended up implementing a simple dispatcher in one callback where any method that needs to be run, first sets a flag, then each time the callback is run, the flag is checked and the corresponding method is run.
int flag=0;
void callback(...)
{
switch(flag)
{
case 1: //e.g. stop
core->stop();
break;
case 2: // e.g. get_callbacks()
core->get_callbacks();
break;
case 3:
//some other op
break;
....
}
//reset flag
flag = 0;
}
i have a problem running my C++ project on our faculty server. The runtime error i get is this:
terminate called after throwing an instance of 'std::runtime_error'
what(): locale::facet::_S_create_c_locale name not valid
Aborted (core dumped)
I determined that the problem lies somewhere within this filesystem iterator (by utilizing a test program):
bf::path dir("ImageData/" + m_object_type);
vector<bf::path> tmp;
copy(bf::directory_iterator(dir), bf::directory_iterator(), back_inserter(tmp));
sort(tmp.begin(), tmp.end());
for (vector<bf::path>::const_iterator it(tmp.begin()); it != tmp.end(); ++it)
{
auto name = *it;
image_names.push_back(name.string());
}
The program ran perfectly on two other Linux-based system (kubuntu & linux mint, but as my project is quite runtime heavy and running it with different parameters would take about 28 days on my machine i really want to use the server). I already tried various paths but none of it worked. I read about a boost bug which led to this error before 1.47, but i'm using 1.54 on the server. I also checked the system locales but that doesn't really gave me a clue since they are almost similar to my system. The other specification of the server are:
Ubuntu 12.04.1 LTS (GNU/Linux 3.2.0-29-generic x86_64)
g/c++ (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
If anyone has any idea to share, i would appreciate it.
This is an issue with Boost < 1.56. Boost internally tries to construct a std::locale("") (see http://www.boost.org/doc/libs/1_55_0/libs/filesystem/src/path.cpp, and compare the updated version in v1.56). This call will fail if the locale (or LC_ALL) is invalid.
In my case, it was a call to boost::filesystem::create_directories() that triggered the locale("") call.
The following workaround worked for me: Override the LC_ALL environment variable in your program. std::locale("") seems to use that variable to determine what the “sensible default” locale should be.
#include <locale>
#include <cstdlib>
#include <iostream>
int main(int argc, char **)
{
try {
std::locale loc("");
std::cout << "Setting locale succeeded." << std::endl;
} catch (const std::exception& e) {
std::cout << "Setting locale failed: " << e.what() << std::endl;
}
// Set LC_ALL=C, the "classic" locale
setenv("LC_ALL", "C", 1);
// Second attempt now works for me:
try {
std::locale loc("");
std::cout << "Setting locale succeeded." << std::endl;
} catch (const std::exception& e) {
std::cout << "Setting locale failed: " << e.what() << std::endl;
}
}
After the setenv call, I can create a default locale, and the boost::filesystem calls work as well.
I don't know for sure, but I suspect that this program will behave the same:
#include <locale>
#include <iostream>
#include <stdexcept>
int main () {
try { std::locale foo (""); }
catch ( std::runtime_error & ex ) { std::cout << ex.what() << std::endl; }
}
Also, this (old) ticket https://svn.boost.org/trac/boost/ticket/5289 may shed some light on the subject.
EDIT: Technically, this is not an answer.
for anyone who is interested, here is a version of the above directory iterator utilizing the QT-lib:
string str1 = "ImageData/";
QString dir_string1 = QString::fromStdString(str1);
QString dir_string2 = QString::fromStdString(m_object_type);
dir_string1.append(dir_string2);
QDir dir(dir_string1);
dir.setFilter(QDir::Files);
dir.setSorting(QDir::Name);
QStringList entries = dir.entryList();
string tmp;
for (QStringList::ConstIterator it=entries.begin(); it != entries.end(); ++it)
{
auto name = *it;
tmp = name.toUtf8().constData();
image_names.push_back(str1 + m_object_type + "/" + tmp);
}
I was writing a code for exception handling on Visual C++ 2010 .Here is the code
#include <iostream>
using namespace std;
// Localize a try/catch to a function.
void Xhandler(int test)
{
try{
if(test) throw test;
}
catch(int i) {
cout << "Caught Exception #: " << i << '\n';
}
}
int main()
{
cout << "Start\n";
Xhandler(1);
Xhandler(2);
Xhandler(0);
Xhandler(3);
cout << "End";
return 0;
}
The Program executed properly and the output was the as expected.But when I pressed the close button for closing the console then an error came that cmd has stopped working
.Then I ran my previous code that executed properly ,they also gave the same error
.
Can anybody tell why it is happening?Is it a problem with the Visual c++ 2010 or the code
I think your problem is not with your code. The problem is within your compiler tool chain. You probably are using Qt, and the tool chain has a problem causing this. Google the message you get when you crash with your IDE.
Here's a simple experiment to prove what I'm saying: just run this code:
int main()
{
cout << "Start\n";
cout << "End";
return 0;
}
And your program will crash, which means you have no problems with exceptions or anything else in your code, but with your tool chain.
I'm currently writing a web-crawler/spider in C++ on Linux and I'm having some problems with updating a database. I'm fairly new to C/C++, just FYI.
The database updates are executed by a seperate thread (using pthreads) but the same problem exists if executed in main() so I, perhaps naively, discarded the threading stuff as the cause of anything.
I'm using libmysqlcppconn for the database API.
I am compiling with gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5.1) with -O2 -Wall -pedantic and it compiles cleanly.
Nevertheless, when the function commitChangesToDatabase() below is called, it basically picks out items from a std::map (url_queue), throws them in a std::vector (updates) and erases said item from the original std::map, then proceeds to iterate over the std::vector, executing a MySQL prepared statement for each item in the vector. Here is where it fails hard.
It randomly either:
Crashes without any error output (no segfault, no stacktrace, no nothing)
Crashes with a glibc memory corruption detected (see output here: http://pastie.org/private/wlkuorivq5tptlcr7ojg)
Reports that the MySQL server has gone away (caught exception), but keeps trying (doesn't crash)
I have tried switching the prepared statement to a simple executeUpdate(), but to no avail.
I have tried eliminating the step with picking out items and rather just execute the updates whenever I find an item to update, in the first loop over url_queue.
Other functions in this application uses prepared statements as well (another UPDATE) and that works fine. Those functions are also run by seperate threads.
I would run the application through valgrind, but quite frankly, I don't understand most of the output so it wouldn't help me much - but if anyone wants the output from it, let me know which options to run it with and I'll provide it.
I have no clue how to proceed from here. Anyone have a clue what's wrong?
struct queue_item_t {
int id;
int sites_id;
int priority;
int depth;
int handler;
int state; // 0 = Pending, 1 = Working, 2 = Completed, 3 = Checked
double time_allowed_crawl;
bool status;
bool was_redirected;
double time;
double time_end;
double time_curl;
double size;
std::string hash;
std::string url;
std::string file;
std::string host;
};
void commitChangesToDatabase()
{
map< string, queue_item_t >::iterator it, end;
sql::PreparedStatement *pstmt;
int i = 0;
if (!url_queue.size()) {
return;
}
pthread_mutex_lock(&dbCommitMutex);
pthread_mutex_lock(&itemMutex);
cout << "commitChangesToDatabase()" << endl;
pstmt = dbPrepareStatement("UPDATE crawler_queue SET process_hash = NULL, date_crawled = NOW(), url = ?, hash = ? WHERE id = ?");
for (it = url_queue.begin(); it != url_queue.end();)
{
if (it->second.state == 2)
{
pstmt->setString(1, it->second.url);
pstmt->setString(2, it->second.hash);
pstmt->setInt(3, it->second.id);
try {
pstmt->executeUpdate();
++i;
} catch (sql::SQLException &e) {
cerr << "# ERR: SQLException in " << __FILE__;
cerr << "(" << __FUNCTION__ << ") on line " << __LINE__ << endl;
cerr << "# ERR: " << e.what();
cerr << " (MySQL error code: " << e.getErrorCode();
cerr << ", SQLState: " << e.getSQLState() << " )" << endl;
}
url_queue.erase(it++);
}
else {
++it;
}
}
delete pstmt;
cout << "~commitChangesToDatabase()" << endl;
pthread_mutex_unlock(&itemMutex);
pthread_mutex_unlock(&dbCommitMutex);
}
// this function is defined in another file but is written here just to show the contents of it
sql::PreparedStatement *dbPrepareStatement(const std::string &query)
{
return con->prepareStatement(query);
}
Edit:
Some seem to believe the problem is with the iteration over the url_queue collection, however I have ruled that out but commenting out everything that operates on the database, but not the iteration. Furthermore, the iteration here is a simplified (but working) version of the original which picks out items from the map, throws in a vector and erases from the map, as demonstrated below, and that part of the program works fine - it only crashes whenever the database is used.
for (it = url_queue.begin(); it != url_queue.end();)
{
if (it->second.state == 2)
{
update_item.type = (!it->second.was_redirected ? 1 : 2);
update_item.item = it->second;
updates.push_back(update_item);
url_queue.erase(it++);
}
else {
++it;
}
}
Edit 2:
Output from valgrind --leak-check=yes: http://pastie.org/private/2ypk0bmawwsqva3ikfazw
It seems, the iterator is incremented unnecessarily; first in loop body, and also in for statement. In this code, it is possible to increment the end iterator, this is a problematic operation and might be source of the problem.
The following loop structure is more suitable for this case:
it = url_queue.begin();
while( it != url_queue.end() ){
//loop body
}
I don't think it is a good idea to mess with iterators. replace:
else {
++it;
}
by:
else continue;
or just remove it.
I'm trying to use the ofstream class to write some stuff to a file, but all that happens is that the file gets created, and then nothing. I have some simply code here:
#include <iostream>
#include <fstream>
#include <cstring>
#include <cerrno>
#include <time.h>
using namespace std;
int main(int argc, char* argv[])
{
ofstream file;
file.open("test.txt");
if (!file) {
cout << strerror(errno) << endl;
} else {
cout << "All is well!" << endl;
}
for (int i = 0; i < 10; i++) {
file << i << "\t" << time(NULL) << endl;
}
file.flush();
file.close();
return 0;
}
When I create a console application, everything works fine, so I'm afraid this code is not completely representative. However, I am using code like this in a much larger project that - to be honest - I don't fully understand (Neurostim). I'm supposed to write some class that is compiled to a dll which can be loaded by Neurostim.
When the code is run, "test.txt" is created and then "No error!" is printed, as this is apparently the output from strerror. Obviously this is wrong however. The application runs perfectly otherwise, and is not phased by the fact that I'm trying to write to a corrupted stream. It just doesn't do it. It seems to me like there is no problem with permissions, because the file is in fact created.
Does anyone have any ideas what kind of things might cause this odd behavior? (I'm on WinXP Pro SP3 and use Visual C++ 2008 Express Edition)
Thanks!
Just a thought :- in your real code are you re-using your stream object?
If so, you need to ensure that you call clear() on the stream before re-using the object otherwise, if there was a previous error state, it won't work. As I recall, not calling clear() on such a stream would result in an empty file that couldn't be written to, as you describe in your question.
ofstream file;
file.open("test.txt");
Just a nit: you can combine that into a single line. ofstream file("test.txt");
if (file) {
cout << strerror(errno) << endl;
} else {
cout << "All is well!" << endl;
}
Your test is backwards. If file is true, it's open and ready for writing.
Also, I wouldn't count on strerror() working correctly on Windows. Most Windows APIs don't use errno to signal errors. If your failure is happening outside the C/C++ run-time library, this may not tell you anything interesting.
UPDATE Thinking more about this, failing to open a file via fstreams is not guaranteed to set errno. It's possible that errno ends up set on some platforms (espeically if those platforms implement fstream operations with FILE* or file descriptors, or some other library that sets errno) but that is not guaranteed. The official way to check for failure is via exceptions, std::io_state or helper methods on std::fstream (like fail or bad). Unfortunately you can't get as much information out of std::streams as you can from errno.
You've got the if statement wrong. operator void* returns NULL (a.k.a. false) if the file is not writable. It returns non-zero (a.k.a. true) if the file is writeable. So you want:
if (!file) {
cout << strerror(errno) << endl;
} else {
cout << "All is well!" << endl;
}
Or:
if (!file.good()) {
cout << strerror(errno) << endl;
} else {
cout << "All is well!" << endl;
}