How to check whether a folder exists or not in gcp cloud storage out of 1 million folders - google-cloud-platform

For example I have structure like this.
bucketname/checked/folder1/some files
bucketname/checked/folder2/some files
bucketname/checked/folder3/some files
bucketname/checked/folder4/some files
bucketname/checked/folder5/some files
bucketname/checked/folder6/some files
bucketname/checked/folder7/some files
bucketname/checked/folder8/some files
bucketname/checked/folder9/some files
bucketname/checked/folder10/some files
bucketname/checked/folder11/some files
......
......
bucketname/checked/folder-1million/some files
Now,
1. If I have to check whether folder99999 exists or not. So,what would be the best way to check it (we have information of folder name - folder99999) ?
2. If we simply check path that exists or not, and if not then it means, folder don't exists. would it work fine If we have millions of folders?
3. Which data structure gcp uses to retrieve the folder data ?

The true answer is this one provided by John: folder doesn't exist. All the files are stored at the root directory (bucket level) and the file name is the full path. By human convention, the / is the folder separator and the console display fake folders.
If you haven't files in a "folder", the "folder" doesn't exist, it's not interpreted/deduced from the name fully qualified path. The folder is not a Cloud Storage resource
It's also for that reason that you search only by path prefix
However, it depends what you want to check. If you exactly know which folder you want to check and validate, and if there is at least one file in it, you can directly list the files with the folder path as prefix.

Related

How do I prevent libtorrent from creating a folder with a name?

When downloading a torrent via libtorrent (a library for C ++), a folder with the name of the torrent is created, into which all the files are downloaded. In uTorrent, for example, you can remove the "Create subfolder" check mark when adding a torrent and everything will be fine. How to do it in libtorrent?
Is it possible to change the names of the torrent (folder)?
To remove the directory from the file hierarchy, you can rename every file to the same name but with the torrent name root directory stripped off.
You should be able to do this both on the torrent_info before adding the torrent, or on the torrent_handle after adding it. But if you add it first the root directory will still be created.
See rename_file()

Filezilla lets to duplicate files with utf8 filenames in same folder

When I'm uploading all folder tree with all files, filezilla corrupts files with utf8 filenames. It writes them there, the files are OK, but they isn't served by nginx.
When I open some folder and select files from my computer same folder, draging them by hand, they duplicates, but these new files, are shown by nginx!
So if I originally uploaded 6 files to current folder when uploading all folders tree, then opening just this one current folder Filezilla lets me again to upload same 6 files, but it becomes as 12 files. When I select to drag files again, it won't become 18, Filezilla overwriting these 6 which I had draged by hand, leaves another 6 untouched.
I have about 2000 different folders in my folders tree, with files inside. When I upload all folder tree, no files which has utf8 in filename are shown by nginx. When I open every dir and redrag files by hand, they start work on nginx, also they are draged as new ones in that folder as I said before.
Resume:
Filezilla uploading files somehow wrong when uploading all folder tree... But Correctly when dragging only files.
I googled for you, and found this advice:
I am not sure if it solves your problem, but try under "File
server manager" "character set" and there force UTF-8.

Informatica B2B Data Exchange

I am new to B2B DX, using since 2 months. I have a requirement where files are getting generated in dynamic folder. For eg. File name is 20170503test.txt then it will get generated in /2017_05/20170503/20170503test.txt.
The next day means tomorrow it will get generated in /2017_05/20170504/20170504test.txt. So how my endpoint can pick these files as they are getting generated in different folders? So what I can set file pattern is *test.txt. But how endpoint can go in different directories?
If you go high enough in the target file system, there will be a single directory which is common among all the target files... even if it is the root directory... set your target folder to that. Then in the mapping itself, create a special FileName port on your target... this will dynamically set the name of the output file to whatever value this port is set to so you can fully qualify the string you set this to to include the full dynamic file path up until that shared directory described above.

How can I rename files of which I don't know the extension?

I have a directory with these files:
one.txt
two.mp3
three.bmp
Is there any way to rename files using MoveFile() but without specifying the extension of the file (all the files have different filenames)?
If not, how can I rename these files to anything I want, when I only know the
one
two
three
?
You'll have to use the underlying OS API to scan through the directory for files and compare each filename with your desired prefix. Here is another question that shows you how to list the contents of a directory in Windows.
Once you know prefix read the filelist of the directory and find a valid filename with that prefix, and use that filename with the function.

having file names and delete them

I want to have access to all files of a folder and have a list of them and work with them.
For example: there is a folder named "new folder" and consist of files : 1.txt and 2.txt
I don't know what are in the folder new folder. So I want a list of files in it .
Hence the questions are :
1- How can I have such this list?
2- How can I delete a file (e.g 2.txt) whether i know there is file with this name or not.
3- Is it possible to figure out has a txt file been used or not (whether it is empty or not)
thanks;
I'd use Boost filesystem to analyze folder content, and remove to delete the file. You will find in filesystem tutorial some sample that will ease your work.
edit: remove(path) it's available in boost filesystem.