I have a mapping like
SA-->SQ--->EXPR--->TGT
The source will be of the same structure and the tartget also.
There are a bunch of files(with the same structure) which will go through this mapping .
So i want to use a parameter file through which i will give the file names for every run manually.
How to use the param file in session for Source filename attribute
Please suggest..
you could use indirect source type, wherein your source file is basically a list of files, and in turn the session reads each of the files one by one.
the parameter file could reference a source file name (the list) as
$InputFile_myName=/a/b/c.list
In line with what Raghav says, indicate the name of a file that will hold a list of input files in the 'Source filename' property box for the SQ in question in the Mapping tab, making the file 'Source filetype' be 'Indirect', specified in the Session Properties. If you already know ahead of time the names of the input files, you can specify them in that file and deploy that file with the workflow to the location you indicate in the 'Source file directory' property box. However if you won't know the names of the input files until run-time but know the files' naming standard (e.g: "Input_files_name_ABC_" where "" represents variable text, such as a numeric value incremented per input file generated by some other process), then one way to deal with that is to use a Pre-Session Command specifiable in the 'Components' tab of the Session. Create one that will build a new file at the location and with the name specified for the Indirect input file referenced above by using the Unix shell (or if running on Windows, the cmd shell) to list the files conforming to the naming standard for them and redirect the listing output to that file.
Tricky thing is that there must be one or more files listed in that Indirect type of input file. If that file is empty, the workflow will fail (abend). An Indirect file type must have in it listed at least one file (even if that file is empty) and that file must exist. The workflow fails if the indirect file reader gets no files to read from or if a file listed in it is not present on the server to be read from. One way to get around this is to make sure an empty file is present at all times that conforms to the naming standard. This can be assured by creating a "touchfile" before executing the listing command to build the Indirect file type listing file. In Unix, you'd use the 'touch {path}/{filename}' command ({filename} could be, for example, "Input_files_name_ABC_TOUCHFILE"), or on Windows you'd redirect an empty string to a file likewise named via cmd shell process. Either way, that will help you avoid an abend. Cleaning up that file is easy to do: a Post-Session command can be used to delete the empty touchfile. Likewise, you can do the same for the Indirect type of file if desired.
Related
I need to be able to distinguish two cases in a command line utility:
The user has specified an output directory (-o output/): create a file called output/file.ext
The user has specified an output file (-o output): create a file called output
What's the correct way in C++17 to determine that output/ or output\ is a path to a directory even when the output directory doesn't exist?
The free function std::filesystem::is_directory() returns false unless the directory exists.
Is checking that std::filesystem::path::has_filename() returns false reliable?
Edit: clarified use case based on comments.
That is not possible even theoretically, because if the user specified a path whose last element is non-existent, it could be created either as a file or as a directory.
However, you can check the special case of a path ending in a directory separator (std::path::preferred_separator), which can happen only for directories, not regular files. That's pretty easy to do.
How could I possible open a symbolic link and get the content of the file instead of the file it is pointing to?
By doing:
with open('/home/symlink.txt', 'rb') as f:
data=f.read()
If the symbolic link points to /foo/faa.txt, the variable data will contain the content of faa.txt. This is a big security and file problem from my server because I'm generating zip archives.
If for example, a folder contains multiple symbolic links with different names to avoid duplicating files, the zip archive will contain multiple files instead of multiple symbolic links!
I hope to be clear enough!
An extra explanation:
The point of this is to allow downloading symlinks in a django server. The way of returning files is the following one:
response = HttpResponse()
response.write(data))
return response
This means that data must contain the content that the user will download. I can not just give it a path. So what I need to do is to give it a symbolic link. The problem is that reading a symbolic link makes python read the content where it is pointing to instead of its real content. In a few words, the user downloads the real file instead of the symbolic link!
A possible solution to this would be to get the path where the symlink points to, and then generate the link in the buffer. Is this possible?
It looks like there are 2 questions here: How can you read a symlink from the filesystem, and how can you store this in a .zip file such that it will be recreated when you unzip it.
Reading a symlink
The contents of a symlink are defined here:
http://man7.org/linux/man-pages/man7/symlink.7.html
A symbolic link is a special type of file whose contents are a string that is the pathname of another file, the file to which the link refers
You can read that path by using os.readlink (https://docs.python.org/2/library/os.html#os.readlink) - this is analogous to C's readlink function.
It's also important to note that these symlinks aren't distinguished by their content or file attributes, but by the fact that the file entry on disk points to a string rather than a file object:
In other words, a symbolic link is a pointer to another name, and not to an underlying object.
This means that there isn't really a "file" you could store in the ZIP. So how do the existing zip & unzip utilities do it?
Storing a symlink in a zip file
The spec for the ZIP format is here: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
Note that section 4.5.7 (defining UNIX Extra Field) says:
The variable length data field will contain file type specific data. Currently the only values allowed are the original "linked to" file names for hard or symbolic links, and the major and minor device node numbers for character and block device nodes. [...] Link files will have the name of the original file stored.
This means that to store a symlink, all you need to do is add the UNIX extra field block to the data you are writing (these appear to live immediately after the filename is written, and you need to set the extra field length accordingly), and populate its "Variable length data field" with the path you get from readlink. The content you store for the node will be empty.
If you're using a library to generate the zip data (recommended!), it will probably have an abstraction available for that. If not, I'd suggest you put in a feature request!
Of course, most existing zip and unzip utilities follow the same definition, which is why you are able to zip and unzip symbolic links as if they were regular files.
I am creating a json string in C++ and save it to a file using fstream.
Here is the code for creating the file:
string json="{ \"a\"= 1 }";
fstream datei1("jsonfile.json",ios::out);
file1 << json << endl;
file1.close();
How could one set the mime-type to 'application/json'??
file -i jsonfile.json in linux shell gives me: jsonfile.json: text/plain; charset=utf-8
file command try to guess the type of your file by reading it.
And read your file again: it is a plain text file. There is only a simple object stored, nothing that can lead to the application answer.
So without changing your file data, there is nothing you can do from your code to change file command answer.
From the documentation of file command:
Causes the file command to output mime type strings rather than the
more traditional human readable ones. Thus it may say 'text/plain;
charset=us-ascii' rather than 'ASCII text'. In order for this option
to work, file changes the way it handles files recognized by the
command itself (such as many of the text file types, directories etc),
and makes use of an alternative 'magic' file. (See the FILES section,
below).
/usr/share/file/magic.mgc Default compiled list of magic.
/usr/share/file/magic Directory containing default magic files.
You can read about magic files on the wiki.
Also you can add your own signatures in /etc/magic.
But *.json is a plain text file, without any signatures, thus, probably, it's impossible to make OS think, that some file has application/json mime type without any hacks.
I have source and target in an informatica powercenter developer. I heed some other header name to be imported in the target file automatically without any manual entry. How can I import customized headers to informatica target.
What have you tried?
You can use a header command in the session configuration for the target, I haven't used it, and couldn't find any documentation on it (i.e. what is possible and how, whether parameters can be used or not, etc.). I did test using (on Windows) an ECHO command to output its text to the header row, but it didn't seem to recognize parameters.
Or you can try to include the header as the first data output row. That means your output will have to be all string types and length restrictions may compound the issue.
Or you can try using two mappings, one that truncates the files and writes the header and one which outputs the data specifying append in the session. You may need two target definitions pointing to the same files. I don't know if the second mapping would attempt to load the existing data (i.e. typecheck), in which case it might throw an error if it didn't match.
Other options may be possible, we don't do much with flat files.
The logic is,
In session command, there is an option called user defined headers. Type echo followed by column name separated by comma delimited
echo A, B, C
at the moment I'm writing a kind of lib, which gets from outside the file name 'sFilename'. With it data were written to a file it will be created, data were append to an existing file with data, data were updated in an existing file with data or the data were read from an existing data.
The user of the application with my lib should get as much as possible on information about errors of file handling.
For this purpose I code a method FileExists(...) which uses _stat() to determine if a file exists or not and a method "bool checkPermission(std::string sFilename, CFile::EOpenmode iOpenmode)" which gives back a bool if the specified file (sFilename) with the iOpenmode (Read, Write, Readwrite) have the permission to be read, written or read and written.
This method works with _stat(sFilename.c_str(), &buf) too and gives the desired information of the file back in buf.
Before checking any file I want to check if the directory containing the specified file has the desired permissions and for that I want to use the checkPermission method [works with _stat()] for the directory!
Now the problem: how can I determine easyly the containing directory? If the user just give a filename "test.txt" the file will be created or read in working directory. So its easy to get the up-directory. Its the same like the working directory. (And there its simple to use checkPermission to get detailed information about the directory).
But what about when the user not only give the file name? For exaample "....\test.txt" or "dir1\dir2\test.txt". How to combine the working directory with a specific path to gain the up-directory of the file and then to check the permissions?
Phew, I hope all is clear and it was'nt too long ;-)
Rumo
I'd suggest using the Boost FileSystem library at www.boost.org. In particular, check out the path class, which has methods such as make_absolute and parent_path.
This is Windows example code GetFileNameFromHandle to show you how to get the path from a HANDLE. I think it is what you are looking for.
http://msdn.microsoft.com/en-us/library/aa366789%28v=vs.85%29.aspx
I found out that _stat() and _access() doesn't really works for the permissions of the directories. See this stackoverflow page.
With _stat() you can't use ".\" to get information about the current directory. But _access() at least can check if a directory exists as well ".\" or "..\".
In conclusion I use _access() to check the existence of a directory and _stat() to check the permissions of an existing file. If a file should be created I'll check it by doing.
And by the way ;-) I don't need to combine working directory with the user specified file because I can use the specified file alone in _access() to determine if directory exists.
Rumo