SharpLibZip: Add file without path - compression

I'm using the following code, using the SharpZipLib library, to add files to a .zip file, but each file is being stored with its full path. I need to only store the file, in the 'root' of the .zip file.
string[] files = Directory.GetFiles(folderPath);
using (ZipFile zipFile = ZipFile.Create(zipFilePath))
{
zipFile.BeginUpdate();
foreach (string file in files)
{
zipFile.Add(file);
}
zipFile.CommitUpdate();
}
I can't find anything about an option for this in the supplied documentation. As this is a very popular library, I hope someone reading this may know something.

My solution was to set the NameTransform object property of the ZipFile to a ZipNameTransform with its TrimPrefix set to the directory of the file. This causes the directory part of the entry names, which are full file paths, to be removed.
public static void ZipFolderContents(string folderPath, string zipFilePath)
{
string[] files = Directory.GetFiles(folderPath);
using (ZipFile zipFile = ZipFile.Create(zipFilePath))
{
zipFile.NameTransform = new ZipNameTransform(folderPath);
foreach (string file in files)
{
zipFile.BeginUpdate();
zipFile.Add(file);
zipFile.CommitUpdate();
}
}
}
What's cool is the the NameTransform property is of type INameTransform, allowing customisation of the name transforms.

How about using System.IO.Path.GetFileName() combined with the entryName parameter of ZipFile.Add()?
string[] files = Directory.GetFiles(folderPath);
using (ZipFile zipFile = ZipFile.Create(zipFilePath))
{
zipFile.BeginUpdate();
foreach (string file in files)
{
zipFile.Add(file, System.IO.Path.GetFileName(file));
}
zipFile.CommitUpdate();
}

The MSDN entry for Directory.GetFiles() states that The returned file names are appended to the supplied path parameter. (http://msdn.microsoft.com/en-us/library/07wt70x2.aspx), so the strings you are passing to zipFile.Add() contain the path.
According to the SharpZipLib documentation, there is an overload of the Add method,
public void Add(string fileName, string entryName)
Parameters:
fileName(String) The name of the file to add.
entryName (String) The name to use for the ZipEntry on the Zip file created.
Try this approach:
string[] files = Directory.GetFiles(folderPath);
using (ZipFile zipFile = ZipFile.Create(zipFilePath))
{
zipFile.BeginUpdate();
foreach (string file in files)
{
zipFile.Add(file, Path.GetFileName(file));
}
zipFile.CommitUpdate();
}

Related

Reading multiple files from FileChooserDialog GTK 3

I am trying to open a Gtkmm::FileChooserDialog to choose multiple files and print their paths along with filenames to a label. I can open the dialog and choose the files but I am having a hard time reading the filenames to my variables.
FileChooserDialog openFileDialog("", FILE_CHOOSER_ACTION_OPEN);
openFileDialog.add_button("Cancel", RESPONSE_CANCEL);
openFileDialog.add_button("Open", RESPONSE_OK);
openFileDialog.set_current_folder(ustring::compose("%1/Desktop", ustring(getenv("HOME"))));
openFileDialog.set_transient_for(*this);
openFileDialog.set_select_multiple(true);
Glib::RefPtr<Gtk::FileFilter> fileFilter = Gtk::FileFilter::create();
fileFilter->set_name("Text Files (*.txt)");
fileFilter->add_pattern("*.txt");
openFileDialog.add_filter(fileFilter);
fileFilter = Gtk::FileFilter::create();
fileFilter->set_name("All Files (*.*)");
fileFilter->add_pattern("*.*");
openFileDialog.add_filter(fileFilter);
if (openFileDialog.run() == RESPONSE_OK)
label.set_text(ustring::compose("File = %1", ustring(openFileDialog.get_filename())));
return true;
You can use Gtk::FileChooser::get_filenames (Gtkmm 3.24):
if (openFileDialog.run() == Gtk::RESPONSE_OK)
{
for(const auto& fileName : openFileDialog.get_filenames())
{
label.set_text(Glib::ustring::compose("File = %1", Glib::ustring(fileName)));
}
}
which returns a std::vector of file names.
Note: In my answer, I keep overwriting the label variable because it was all the context there was in your code snippet. My be you have multiple labels or you want to pack all the file names somehow in a single label. I let this part to you.

How to modify the filename of the S3 object uploaded using the Kafka Connect S3 Connector?

I've been using the S3 connector for a couple of weeks now, and I want to change the way the connector names each file. I am using the HourlyBasedPartition, so the path to each file is already enough for me to find each file, and I want the filenames to be something generic for all the files, like just 'Data.json.gzip' (with the respective path from the partitioner).
For example, I want to go from this:
<prefix>/<topic>/<HourlyBasedPartition>/<topic>+<kafkaPartition>+<startOffset>.<format>
To this:
<prefix>/<topic>/<HourlyBasedPartition>/Data.<format>
The objective of this is to only make one call to S3 to download the files later, instead of having to look for the filename first and then download it.
Searching through the files from the folder called 'kafka-connect-s3', I found this file:
https://github.com/confluentinc/kafka-connect-storage-cloud/blob/master/kafka-connect-s3/src/main/java/io/confluent/connect/s3/TopicPartitionWriter.java which at the end has some of the following functions:
private RecordWriter getWriter(SinkRecord record, String encodedPartition)
throws ConnectException {
if (writers.containsKey(encodedPartition)) {
return writers.get(encodedPartition);
}
String commitFilename = getCommitFilename(encodedPartition);
log.debug(
"Creating new writer encodedPartition='{}' filename='{}'",
encodedPartition,
commitFilename
);
RecordWriter writer = writerProvider.getRecordWriter(connectorConfig, commitFilename);
writers.put(encodedPartition, writer);
return writer;
}
private String getCommitFilename(String encodedPartition) {
String commitFile;
if (commitFiles.containsKey(encodedPartition)) {
commitFile = commitFiles.get(encodedPartition);
} else {
long startOffset = startOffsets.get(encodedPartition);
String prefix = getDirectoryPrefix(encodedPartition);
commitFile = fileKeyToCommit(prefix, startOffset);
commitFiles.put(encodedPartition, commitFile);
}
return commitFile;
}
private String fileKey(String topicsPrefix, String keyPrefix, String name) {
String suffix = keyPrefix + dirDelim + name;
return StringUtils.isNotBlank(topicsPrefix)
? topicsPrefix + dirDelim + suffix
: suffix;
}
private String fileKeyToCommit(String dirPrefix, long startOffset) {
String name = tp.topic()
+ fileDelim
+ tp.partition()
+ fileDelim
+ String.format(zeroPadOffsetFormat, startOffset)
+ extension;
return fileKey(topicsDir, dirPrefix, name);
}
I don't know if this can be customised to what I want to do but seems to be somehow near/related to my intentions. Hope it helps.
(Submitted an issue to Github as well: https://github.com/confluentinc/kafka-connect-storage-cloud/issues/369)

Append files to an existing zip file with Poco::Zip

After successfully compress the folder, here is my situation :
If append = true and overWrite = false I have to check whether if the target zip file exists or not if existed I will check the existed zip file which files it doesn't contain and append new file from the source folder to it.
My question is:
How can I open the zip file and put it to the compress object? or which others library in Poco should I use to open zip stream? I'm trying to use std::ifstream but Poco::zip::Compress doesn't seem to receive an std::ifstream
I surely have to modify the Poco source code itself to match with my requirement. Thanks in advance.
void ZipFile(string source, string target, List extensions, bool append, bool overWrite)
{
Poco::File tempFile(source);
if (tempFile.exists())
{
if (Poco::File(target).exists() && append && !overWrite) {
fs::path targetPath = fs::path(target);
std::ifstream targetFileStream(targetPath.string(), std::ios::binary);
std::ofstream outStream(target, ios::binary);
CompressEx compress(outStream, false, false);
if (tempFile.isDirectory())
{
Poco::Path sourceDir(source);
sourceDir.makeDirectory();
compress.addRecursive(sourceDir, Poco::Zip::ZipCommon::CompressionMethod::CM_AUTO,
Poco::Zip::ZipCommon::CL_NORMAL, false);
}
else if (tempFile.isFile())
{
Poco::Path path(tempFile.path());
compress.addFile(path, path.getFileName(), Poco::Zip::ZipCommon::CompressionMethod::CM_AUTO,
Poco::Zip::ZipCommon::CL_NORMAL);
}
compress.close(); // MUST be done to finalize the Zip file
outStream.close();
}
}
No need to modify the Poco source code. Poco allows you to get the contents of an archive and add files to it.
First, open the target archive to check which files are already in there:
Poco::ZipArchive archive(targetFileStream);
Then collect all files you want to add, that are not in the archive, yet:
std::vector<fs::path> files;
if (fs::is_directory(source)) {
for(auto &entry : fs::recursive_directory_iterator())
// if entry is file and not in zip
if (fs::is_regular_file(entry)
&& archive.findHeader(fs::relative(entry.path, source)) == archive.headerEnd()) {
files.push_back(entry.path);
}
} else if (fs::is_regular_file(entry)
&& archive.findHeader(source) == archive.headerEnd()) {
files.push_back(source);
}
Finally, add the files to your zip
Poco::Zip::ZipManipulator manipulator(target, false);
for(auto &file : files)
manipulator.addFile(fs::relative(file, source), file,
Poco::Zip::ZipCommon::CompressionMethod::CM_AUTO,
Poco::Zip::ZipCommon::CL_NORMAL);
I had no opportunity to test this. So try it out and see what needs to be done to make it work.

Java: return a LinkedHashSet

Basically, I'm trying to return a collection of strings in Java.
But...
each string must be unique because they're all the names of ".db" files in current folder, so I thought this collection should be LinkedHashSet.
The elements (filenames) must maintain the exact same order, so I can choose one of them by it's order number in the collection.
My main routine will show this collection in a GUI component (maybe a JList) for the user to choose one of them (without the .db extension).
I'm totally newbie (as you can see), so if you think there are better options than LinkedHashSet please tell me.
Also, how can I grab this collection in the main class?
What I've got so far:
public Set GetDBFilesList() {
//returns ORDERED collection of UNIQUE strings with db filenames
LinkedHashSet a = new LinkedHashSet();
FilenameFilter dbFilter = (File file, String name) -> {
return name.toLowerCase().endsWith(".db");
};
String dirPath = "";
File dir = new File(dirPath);
File[] files = dir.listFiles(dbFilter);
if (files.length > 0) {
for (File aFile : files) {
a.add(aFile.getName());
}
}
return a;
}
You want an ordered and unique collection - LinkedHashSet is a good choice.
Some comments on your methode:
Your should use Generics f.e.: LinkedHashSet<File> or LinkedHashSet<String>
The check for files.length is unnecessary, but you could check for null if the path is not a directory or an I/O error occured
You should name your variables properly: a is not a good name
Your methode can be static - maybe in a static helper class?
The Set.add methode returns true or false if the item was added or not, you should check that just in case
Putting all together:
//Your Main class
public class Main
{
public static void main(String[] args)
{
File dir = new File("");
Collection<File> dbFiles = DbFileManager.getDatabaseFiles(dir);
}
}
//Your DB File Reader Logic
public class DbFileManager
{
public static Collection<File> getDatabaseFiles(File directory)
{
Collection<File> dbFiles = new LinkedHashSet<>();
//filter code etc.
boolean success = dbFiles.addAll(directory.listFiles(filter));
//Check if everthing was added
return dbFiles;
}
}

Sitecore FileUtil.ZipFiles creating empty zip file

I am trying to use the ZipFiles() utility method and its producing an empty zip file. I am using Sitecore 6.5. There are no error, permissions or otherwise.
Any thoughts? Here is the code.
public void CreateZipFile(string zipfileName, List<string> files)
{
var zipfile = string.Format("{0}/{1}/{2}", TempFolder.Folder, "myfolder", zipfileName) ;
var fileArray = files.ToArray();
var x = FileUtil.ZipFiles(zipfile, fileArray);
}
EDIT:
I am passing the files like this
var files = new List<string> { FileUtil.MapPath("/temp/sample.xlf") };
The proper usage of FileUtil.ZipFiles method is:
FileUtil.ZipFiles("/test.zip", new []{"/web.config", "/otherfile.txt"})
Sitecore automatically maps paths. The zip file will be created in your web app root.
EDIT AFTER COMMENT
If you want to create a zip file outside the web root and with a flat structure inside, you can use Sitecore ZipWriter class like this:
public static string ZipFiles(string absolutePathToZipfile, string[] files)
{
using (ZipWriter zipWriter = new ZipWriter(absolutePathToZipfile))
{
foreach (string path in files)
{
using (FileStream fileStream = System.IO.File.OpenRead(path.StartsWith("/") ? FileUtil.MapPath(path) : path))
zipWriter.AddEntry(FileUtil.GetFileName(path), fileStream);
}
}
return absolutePathToZipfile;
}