I am writing a c++ program that interfaces with an Apache FtpServer using libcurl. I was originally using the LIST command to get the contents of a directory but it was giving me a lot of information I didn't but had to parse any ways which lead to a lot unneeded overhead (especially when I was working with hundreds of thousands of files). In addition I needed a valid time stamp and it was giving me a shorthand that didn't include the year (so on January 1 all of the files on my computer looked outdated compared to the FTP server's). My solution was to use the NLST command to get only the names, then download the timestamps of each using MDTM. This worked awesome but then I ran into the major problem of not being able to tell if a file was a directory or not.
I am thinking the easiest way to do this is using the permissions to see if the first flag is set to d. FTP doesn't appear to have this functionality. Is there an easy way to tell if a filename is a directory or file?
You can try to use the CWD command on the file to test if it is a directory or not. But failure may mean lack of permission, so you need to check the error code. For example:
ftp> cd atom.xml
550 Can't change directory to atom.xml: Not a directory
Alternatively, you can use the NLST command again on the file you want to test. If it is a plain file, you will just get the filename back. Otherwise, you will get a list of contents of the directory.
ftp> nlist atom.xml
200 PORT command successful
150 Connecting to port 53912
atom.xml
226 1 matches total
ftp> mkdir foo
257 "foo" : The directory was successfully created
ftp> nlist foo
200 PORT command successful
150 Connecting to port 53928
226 0 matches total
Related
This is driving me crazy, I must admit. After finally being able to successfully compile two functions I need to process voice files, from C/C++ code that I downloaded from a trustworthy online repository (code that had been thoroughly tested in Linux), I am now struggling to launch those files from Matlab...
When I type the following command in cmd (dos)
Analysis b2.wav config_default
it works, no problem (see here Works).
Then, I build the exact same command into a string and feed it to the "system" Matlab function. Then the code crashes... (see here Fails) I've tried with full paths (c:\b2.wav, etc) but still does not work...
Any ideas as to why this might be happening?
Your image shows that the program Analysis stopped unexpectedly.
It might be a lot of reasons why, so let's go step by step:
1) Try executing Analysis from Terminal and passing wrong parameters (a file that doesn't exist, only one param (missing the config_defalut), no parameters at all, three parameters, etc...)
Can you make the program crash from terminal by passing wrong params?
2) Try creating the command first, checking that it's correct (\b is actually \b instead of a string modifier)
command_to_be_run = 'C:\Analysis C:\b2.wav C:\config_default'
disp(command_to_be_run) % is it showing exacly what you want?
system(command_to_be_run); % if so, run it.
3) Try creating a dummy executable dummy.exe in C that accepts two parameters and prints the received parameters (keep it super simple, just printing). Call it from Terminal. Does it work? Call it from Matlba. Does it Work?
With this 3 tests you can considerably narrow down where your error comes from.
By the way, is "config_default" a file or just a string that tells analysis how to behave? In some examples you treat it as a file, in others as a parameter without path.
Based on what's been tried so far and the outputs, here's my theory:
Premise: Analysis.exe came from code that's well tested in Linux. It works in Windows command line when run from the same directory where both it and the target file reside. But it stops working from Matlab console.
Assertion 1: Matlab console does not operate within the context of the directory where the binary is but rather within the Matlab directory. As such, Analysis.exe will try to find the target from the Matlab directory.
Validation for Assertion 1: Try putting the binary and the target wav in the Matlab directory. Then run system with the binary and target specified just by name (no path).
Assertion 2: If the file's full path is specified to address this issue, it still doesn't work. This may be because the code assumed a Linux file system where the delimiter is "/" rather than "\".
Validation for Assertion 2: Run with paths specified from the command line while in a diferent directory to see if it fails or not.
Possible Solution 1: Add the directory where both Analysis.exe and the target are into the Matlab path: (1) On the Home tab, in the Environment section, click Set Path. Add the path there. (2) addpath (folderName1,...,folderNameN) adds the specified folders to the top of the search path for the current MATLAB session. -> Then run the system command without the full paths.
Possible Solution 2: Add the directory where both Analysis.exe and the target are into the Windows environment path. Then run the system command without the full paths.
EDIT: Possible hackish solution - Create a batch file where: (1) you would cd to the directory where Analysis.exe and the target wav are; and (2) do a Matlab system call to the batch file.
EDIT 2: Possible experiment to validate assertion 2.
I have an access to some server where there is a lot of data. I can't copy the whole of data on my computer.
I can't compile on the server the program I want because the server doesn't have all libs I need.
I don't think that the server admin would be very happy to see me coming and asking to him to install some libs just for me...
So, I try to figure if there is a way to open a file like with,
FILE *fopen(const char *filename, const char *mode);
or
void std::ifstream::open(const char* filename, ios_base::openmode mode = ios_base::in);
but over a SSH connection. Then reading the file like I do for usual program.
both computer and server are running linux
I assume you are working on your Linux laptop and the remote machine is some supercomputer.
First non-technical advice: ask permission first to access the data remotely. In some workplaces you are not allowed to do that, even if it technically possible.
You could sort-of use libssh for that purpose, but you'll need some coding and read its documentation.
You could consider using some FUSE file system (on your laptop), e.g. some sshfs; you would then be able to access some supercomputer files as /sshfilesystem/foo.bar). It is probably the slowest solution, and probably not a very reliable one. I don't really recommend it.
You could ask permission to use NFS mounts.
Maybe you might consider some HTTPS access (if the remote computer has it for your files) using some HTTP/HTTPS client library like libcurl (or the other way round, some HTTP/HTTPS server library like libonion)
And you might (but ask permission first!) use some TLS connection (e.g. start manually a server like program on the remote supercomputer) perhaps thru OpenSSL or libgnutls
At last, you should consider installing (i.e. asking politely the installation on the remote supercomputer) or using some database software (e.g. a PostgreSQL or MariaDB or Redis or MongoDB server) on the remote computer and make your program become a database client application ...
BTW, things might be different if you access a few dozen of terabyte sized files in a random access (each run reading a few kilobytes inside them), or a million files, of which a given run access only a dozen of them with sequential reads, each file of a reasonable size (a few megabytes). In other words, DNA data, video films, HTML documents, source code, ... are all different cases!
Well, the answer to your question is no, as already stated several times (unless you think about implementing ssh yourself which is out of scope of sanity).
But as you also describe your real problem, it's probably just asking the wrong question, so -- looking for alternatives:
Alternative 1
Link the library you want to use statically to your binary. Say you want to link libfoo statically:
Make sure you have libfoo.a (the object archive of your library) in your library search path. Often, development packages for a library provided by your distribution already contain it, if not, compile the library yourself with options to enable the creation of the static library
Assuming the GNU toolchain, build your program with the following flags: -Wl,-Bstatic -lfoo -Wl,-Bdynamic (instead of just -lfoo)
Alternative 2
Create your binary as usual (linked against the dynamic library) and put that library (libfoo.so) e.g. in ~/lib on the server. Then run your binary there with LD_LIBRARY_PATH=~/lib ./a.out.
You can copy parts of file to your computer over SSH connection:
copy part of source file using dd command to temporary file
copy temporary file to your local box using scp or rsync
You can create a shell script to automate this if you need to do that multiple times.
Instead of fopen on a path, you can use popen on an ssh command. (Don't forget that FILE * streams obtained from popen are closed with pclose and not fclose).
You can simplify the interface by writing a function which wraps popen. The function accepts just the remote file name, and then generates the ssh command to fetch that file, properly escaping everything, like spaces in the file name, shell meta-characters and whatnot.
FILE *stream = popen("ssh user#host cat /path/to/remote/file", "r");
if (stream != 0) {
/* ... */
pclose(stream);
}
popen has some drawbacks because it processes a shell command. Because the argument to ssh is also a shell command that is processed on the remote end, it raises issues of double escaping: passing a command through as a shell command.
To do something more robust, you can create a pipe using pipe, then fork and exec* the ssh process, installing the write end of the pipe as its stdout, and use fdopen to create a FILE * stream on the reading end of the pipe in the parent process. This way, there is accurate control over the arguments which are handed to the process: at least locally, you're not running a shell command.
You can't directly(1) open a file over ssh with fopen() or ifstream::open. But you can leverage the existing ssh binary. Simply have your program read from stdin, and pipe the file to it via ssh:
ssh that_server cat /path/to/largefile | ./yourprogram
(1) Well, if you mount the remote system using sshfs you can access the files over ssh as if they were local.
I have an application that transfers files via socket, desire that this also make transfers directories.
How can I download a full DIRECTORY via socket?
The program works like this, it asks the user to enter the remote directory where the file is to transfer Example: C:\users\server\file.dat
After it makes a validation to confirm the existence of the file,
and finally it transfers byte by byte.
The problem is that when the user type a directory validation fail, an example is if I type C:\users\SERVER\DIRECTORY
the program then returns an error FAILED IN READ BYTES
The solution would be to zip the directory, but the server only works with command line "shell", and Windows has nothing to zip by native command line.
Any suggestions?
You cannot read a directory like an ordinary file. You should search for all files yourself and send them one by one. This means you should also send paths to all files and reproduce their hierarchy yourself.
I wish to view the samples in the "Samples Environments for Microsoft Chart Controls" code available on the Microsoft website. However when I try to build the project in order to create an exe that I can open, I get the following error:
Cannot write to the output file "obj\Release\System.Windows.Forms.DataVisualization.Charting.Utilities.SampleMain.VerticalTabPageButton.resources". The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
Am I missing something here? How am I supposed to navigate this set of samples?
Oh, and I manually edited the .sln file so it opens in VS2008. But I don't think that has anything to do with my problem.
I encountered the same problem when trying to unzip the "Samples Environments for Microsoft Chart Controls.zip" file.
I was attempting to unzip to my "Documents\Visual Studio 2010\Project" folder. Obviously this folder is quite far down a directory structure on the PC, so I received the same error.
So here's the solution: unzip the file to somewhere quite high up your PC's directory structure - for example, the root of your c drive, i.e. "c:\".
You'll then find you don't receive the "Path Too Long" error message.
Best wishes,
Mike.
Is it possible to use regex for matching file names in FTP to get files from server ?
I need to do FTP to server and need to download the files whose file names are ending with the same value. In my case, it is 14_04_25_144238.
I am not sure if it is doable. But, just out of curiosity, asking this.
Can we use regex like .*14_04_25_144238 in the ftp get command ?
Thanks in advance.
Dinesh S
You want the mget command.
From the Unix man page
mget remote-files
Expand the remote-files on the remote machine and
do a get for each file name thus produced. See glob for details on
the filename expansion. Resulting file names will then be
processed according to case, ntrans, and nmap settings. Files are
transferred into the local working directory, which can be changed
with 'lcd directory'; new local directories can be created with '!
mkdir directory'.
If you want to turn of the prompting of each file, then you also need this:
prompt
Toggle interactive prompting. Interactive prompting occurs
during multiple file transfers to allow the user to selectively retrieve or store files. If prompting is turned off (default is on), any mget or mput will transfer all files, and any
mdelete will delete all files
.