I am getting this error when running my c++ program under ubuntu 20.04 with clang 9 :
GLib-ERROR : Creating pipes for GWakeup: Too many open files
I know I have a lot of open files, this is a CGI renderer with a mip-map tiled texture cache, it loads only parts of images in the cache when needed and not the full images, thus keeping files open. There are about 3000 images but it opened only about 1000 before crashing.
What I don't understand is that these texture files are read using stdlib fopen(), not GLib/GTK. So why is the error coming from GLib?
The limit on open files is process- or system-wide. So it doesn't matter whether they are opened by GLib or not (which also uses the same operating system calls to open files, anyway). If you have 1000 files opened at the same time with fopen(), consider closing them if you can.
Related
I have an external USB, NTFS-formatted hard drive which contains many files which I need to eventually copy to a drive on a Windows Server 2008 R2 machine.
The files on the drive were placed there by scripts run with the drive mounted on Solaris. The user who did this copy was careless and edited their copy script on a Windows machine, resulting in shell script lines such as:
cp /sourceDir/sourceFileName /externalDrivePath/targetFileName\r\n
and as such, the files on the external drive have a trailing carriage return in their filenames. Standard Windows copy utilities (copy, xcopy, robocopy) fail to copy these files with error 0x7B / 123 : "The filename, directory name, or volume label syntax is incorrect."
I have tested, and am fairly sure that if I had the drive mounted again on a Linux box, I should be able to repair the files with commands such as:
mv /externalDrive/targetFileName\r /externalDrivePath/targetFileName\n
However, I do not have immediate access to a Linux machine.
What I have tried so far to repair/move these files:
"Application" solutions on Windows Server 2008 R2:
Renaming files in Windows Explorer -- would be unfeasible solution due to sheer volume of files, but it doesn't work anyways.
Wildcard pattern matching the filenames from cmd prompt, e.g. copy E:\externalDrivePath\targetFileName* anotherPath. Fails with 0x7B error.
Copying files from cmd prompt using 8.3 (short) filenames. Files in question do not have short names, per output of dir /x
"Programming" solutions on Windows Server 2008 R2:
Copying/Renaming files using Python/Java: any attempt to open/copy the carriage-return file throws exception tracing back to the same 0x7B Windows error.
Copying files using Windows C 'CopyFile' API: fails with 0x7B error. Here I found the files using FindNextFile API, and passed that source path into CopyFile, but the OS still fails to copy the file.
Writing my own file copy function in C using fopen, ofstream, etc. The fopen call again fails with 0x7B.
Copying files using C++ boost::filesystem APIs: fails with 0x7B error. Again, found the files using a boost::filesystem::directory_iterator and passed the found file's path to boost::filesystem::copy_file()
Providing file path to Win32 APIs CopyFile / MoveFile as "\?\E:\externalDrivePath\targetFileName\r". Calls fail again with 0x7B error.
I also dabbled with mounting this drive on an OS X machine to run the copy, expecting it would provide support for the NTFS drive more like Solaris did. However, it fails to copy with similar error messages to Windows -- I guess OS X's NTFS implementation is more "Windows-like"?
If this is solvable on Windows, I feel like it's going to either require a very low-level C function that manipulates the FILE itself, without 'opening' it based on its string filename. Not sure how to go about that. That, or some file repair utility that I'm unaware of which incorporates this functionality already.
Any alternative approaches or suggestions how to implement what I'm describing would be most appreciated.
TLDR: Try CreateFileW with a unicode path prefixed with \\?\ and containing the trailing carriage return.
The \\?\ path syntax bypasses a lot of the usual validation rules, unicode exansion, etc and allows long file paths and even (dangerously) allows characters like slashes inside a filename.
Given that, I'd imagine a carriage returns should be fairly trivial to handle...
This page relating to long filenames has more details. Relevant parts quoted below
There is no need to perform any Unicode normalization on path and file name strings for use by the Windows file I/O API functions because the file system treats path and file names as an opaque sequence of WCHARs. Any normalization that your application requires should be performed with this in mind, external of any calls to related Windows file I/O API functions.
When using an API to create a directory, the specified path cannot be so long that you cannot append an 8.3 file name (that is, the directory name cannot exceed MAX_PATH minus 12).
The shell and the file system have different requirements. It is possible to create a path with the Windows API that the shell user interface is not able to interpret properly.
And from here
On newer file systems, such as NTFS, ex-FAT, UDFS, and FAT32, Windows stores the long file names on disk in Unicode, which means that the original long file name is always preserved. This is true even if a long file name contains extended characters and regardless of the code page that is active during a disk read or write operation. The case of the file name is preserved, even when the file system is not case-sensitive ...
I had a similar problem in the mid 90ies, on Windows 3.11.
I ended up using rename from a C program (declared in <stdio.h>).
If that fails you could try low-level C system calls: open, read and write to copy the file to a new name.
Low-level calls usually bypass limitations imposed by user-friendly high-level functions.
I have recently started working on a small program with SDL2 and c++. In this program I used IMG_LoadTexture() to load a texture however it just would not want to work, the program could not find the image. So I chanced my program to use SDL_loadBMP(), did not work. After hours of endlessly looking for bugs I still could not find anything.
When I started my project I saved it to my D: disk and not on my C: disk. So when I was about to give up on the whole project I copied my exe, image and libraries to my C: disk and Boom it worked.
So I have no idea why this happens. I refer to my image with a relative path but it just doesn't want to work on my D: disk. Is this a bug? or might it have something to do with settings related to my disk? And how can I make it work on my D: disk?
P.S. I am using windows 7
P.S.S. My apologies in advance for any horrible grammar and spelling mistakes.
Is the working directory of you executable the same you assume it is? On windows happened more than one time that working directory was not the path I believed. With certain compiler configurations the working directory is not the same on wich the executable is, you have to change working directory looking at documentation of your IDE.
What frequently happened to me
CURRENT WORKING DIR
D:/projectdir/
EXPECTED WORKING DIR
D:/projectdir/bin/debug/
where files really are:
D:/projectdir/bin/debug/app.exe
D:/projectdir/bin/debug/texture.png
which files app is trying to loading when launched from IDE:
D:/projectdir/app.exe
D:/projectdir/texture.png
You can quickly test wich is current working directory by calling equivalent on windows of
getcwd (answer from here )
#include <WinBase.h>
int main() {
TCHAR pwd[MAX_PATH];
GetCurrentDirectory(MAX_PATH,pwd);
MessageBox(NULL,pwd,pwd,0);
}
I'm trying to open a file where my program runs, I could open a file in directories like this:
myfile.open("D:\\users.txt");
But I want to open this file:
myfile.open("users.txt");
users.txt is placed where my program is.
users.txt is placed where my program is.
The current working directory of your process may not be where your program executable is. The two are not bound together.
This:
myfile.open("users.txt");
should work just fine. However, I have encountered situations where the program could not read the file. That was due to the white spaces being included within the full path:
eg: "C:\Folder1\Folder 2\file.txt"
Make sure you don't have any white spaces there...
I recommend reading up on Naming Files, Paths, and Namespaces to give you a better understanding how Win32 API handles File paths, and also Namespaces. It will help you in the long run when you need to open USB and Serial connections to external devices.
I am trying to differentiate between multiple wav files using acoustic fingerprinting. I am using Chromaprint from AcoustID. I am using 32 bit windows. I have downloaded the file fpcalc.exe and am trying to run it on multiple wav files. The problem is, it is generating blank FINGERPRINT for those wav files.
I am running the command
fpcalc.exe -raw <FILENAME>.wav
The files are in wav format and the size of the files is 1 SEC. I am running the same command on bigger files, then the FINGERPRINT is working fine.
Any pointers?
The Chromaprint library only works for larger files, I think the files should have at least 10 seconds to fingerprint.
Maybe you should look for alternatives like Python and Ruby who have libraries to fingerprint small audio files (if you search a bit).
You can use the gem that I made for wav files:
https://rubygems.org/gems/audio-fingerprint
I'm using Wischik's example code for creating AVI files, however after I run the code and it exits successfully no AVI file is created. I looked through the code and cannot find where it writes/saves/creates an AVI file on your computer. Does anyone know where the code saves AVI file or if it doesn't, a way to make it save the AVI file created to your computer? Thanks
link to source code here
Wischik source code
You should find test.avi in the project directory. Since you know it's name, you could also use Windows explorer to search for it.
To examine further, (assuming you are using VC++), goto
Tools > Options > Projects and Solutions > VC++ Directories,
and Show Directories for Include files.
That will tell you where the headers are.
Alternatively, set a break point at the line where AVIFileOpen is called, and follow the execution path to see what it does.