I'm trying to compress a folder into cpio.gz archive with following code. But its not compressing empty folders and symlinks.
void write_archive(string archivename, vector<string> files) {
struct archive *a;
struct archive_entry *entry;
struct stat st;
char buff[8192];
int len;
int fd;
a = archive_write_new();
archive_write_add_filter_gzip(a);
archive_write_set_format_cpio(a);
archive_write_open_filename(a, archivename.c_str());
for (string file : files) {
string filename = file;
stat(file.c_str(), &st);
entry = archive_entry_new();
archive_entry_set_pathname(entry, trim(filename));
archive_entry_set_size(entry, st.st_size);
archive_entry_set_filetype(entry, AE_IFREG);
archive_entry_set_perm(entry, 0644);
archive_write_header(a, entry);
fd = open(file.c_str(), O_RDONLY);
len = read(fd, buff, sizeof(buff));
while ( len > 0 ) {
archive_write_data(a, buff, len);
len = read(fd, buff, sizeof(buff));
}
close(fd);
archive_entry_free(entry);
}
archive_write_close(a);
archive_write_free(a);
}
I'm using this code to repack extracted ramdisk of Android. Extracting files using libarchive is working fine. It extracted all files, folders and symlinks....
full code for compressing here
Anyway fixed it https://github.com/Devil7DK/SimpleShits/commit/2859df5855ae26c63240a0ea99bf5f015d810b66
Key things are
archive_entry_set_filetype(entry, AE_IFLNK); - to set the type of entity which we can determine with help of lstat
and
archive_entry_set_symlink(entry, link.c_str()); // - to set the path of link
References:
https://github.com/libarchive/libarchive/issues/1034
https://www.unix.com/man-page/freebsd/3/archive_entry_set_symlink/
https://groups.google.com/forum/#!topic/libarchive-discuss/iz5wCAW2GZ4
Related
I've never used LibArchive before and am trying to use it to create a .tar archive of a directory, since it is installed on my remote compute instance.
LibArchive has an example of how to create an archive from a flat list of files (shown below). But I can't find any examples of how to actually create an archive from a directory path. During the "Extract" example from the same website, it appears everything is contained within the "archive_entry" struct but there doesn't appear to be a counterpart for Create.
void
write_archive(const char *outname, const char **filename)
{
struct archive *a;
struct archive_entry *entry;
struct stat st;
char buff[8192];
int len;
int fd;
a = archive_write_new();
archive_write_add_filter_gzip(a);
archive_write_set_format_pax_restricted(a); // Note 1
archive_write_open_filename(a, outname);
while (*filename) {
stat(*filename, &st);
entry = archive_entry_new(); // Note 2
archive_entry_set_pathname(entry, *filename);
archive_entry_set_size(entry, st.st_size); // Note 3
archive_entry_set_filetype(entry, AE_IFREG);
archive_entry_set_perm(entry, 0644);
archive_write_header(a, entry);
fd = open(*filename, O_RDONLY);
len = read(fd, buff, sizeof(buff));
while ( len > 0 ) {
archive_write_data(a, buff, len);
len = read(fd, buff, sizeof(buff));
}
close(fd);
archive_entry_free(entry);
filename++;
}
archive_write_close(a); // Note 4
archive_write_free(a); // Note 5
}
The archive_read_disk_descend function from the libtar api allows you to traverse a directory. It should be called before archive_write_header(a, entry); To use this function prior to starting the for loop you need to call struct archive *disk = archive_read_disk_new() in order to create an archive representing your disk.
Using zlib version 1.2.7 uncompress gzip data, but I couldn't know how to get the files' name in the compression package, or some one you are extracting.The method I find,it looks like read all data to buffer, and then return it.
like this:
int gzdecompress(Byte *zdata, uLong nzdata, Byte *data, uLong *ndata)
{
int err = 0;
z_stream d_stream = {0}; /* decompression stream */
static char dummy_head[2] = {
0x8 + 0x7 * 0x10,
(((0x8 + 0x7 * 0x10) * 0x100 + 30) / 31 * 31) & 0xFF,
};
d_stream.zalloc = NULL;
d_stream.zfree = NULL;
d_stream.opaque = NULL;
d_stream.next_in = zdata;
d_stream.avail_in = 0;
d_stream.next_out = data;
//only set value "MAX_WBITS + 16" could be Uncompress file that have header or trailer text
if(inflateInit2(&d_stream, MAX_WBITS + 16) != Z_OK) return -1;
while(d_stream.total_out < *ndata && d_stream.total_in < nzdata) {
d_stream.avail_in = d_stream.avail_out = 1; /* force small buffers */
if((err = inflate(&d_stream, Z_NO_FLUSH)) == Z_STREAM_END) break;
if(err != Z_OK) {
if(err == Z_DATA_ERROR) {
d_stream.next_in = (Bytef*) dummy_head;
d_stream.avail_in = sizeof(dummy_head);
if((err = inflate(&d_stream, Z_NO_FLUSH)) != Z_OK) {
return -1;
}
} else return -1;
}
}
if(inflateEnd(&d_stream) != Z_OK) return -1;
*ndata = d_stream.total_out;
return 0;
}
Using Example:
// file you want to extract
filename = "D:\\gzfile";
// read file to buffer
ifstream infile(filename, ios::binary);
if(!infile)
{
cerr<<"open error!"<<endl;
}
int begin = infile.tellg();
int end = begin;
int FileSize = 0;
infile.seekg(0,ios_base::end);
end = infile.tellg();
FileSize = end - begin;
char* buffer_bin = new char[FileSize];
char buffer_bin2 = new char[FileSize * 2];
infile.seekg(0,ios_base::beg);
for(int i=0;i<FileSize;i++)
infile.read(&buffer_bin[i],sizeof(buffer_bin[i]));
infile.close( );
// uncompress
uLong ts = (FileSize * 2);
gzdecompress((Byte*)buffer_bin, FileSize, (Byte*)buffer_bin2, &ts);
Array "buffer_bin2" get the extracted data.Attribute "ts" is the data length.
The question is, I don't know what is it name, is there only one file.How can I get the infomation?
Your question is not at all clear, but if you are trying to get the file name that is stored in the gzip header, then it would behoove you to read the zlib documentation in zlib.h. In fact that would be good idea if you plan to use zlib in any capacity.
In the documentation, you will find that the inflate...() functions will decompress gzip data, and that there is an inflateGetHeader() function that will return the gzip header contents.
Note that when gzip decompresses a .gz file, it doesn't even look at the name in the header, unless explicitly asked to. gzip will decompress to the name of the .gz file, e.g. foo.gz becomes foo when decompressed, even if the gzip header says the name is bar. If you use gzip -dN foo.gz, then it will call it bar. It is not clear why you even care what the name in the gzip header is.
I'm using libzip in a c++ application for Linux that will need to be able to zip/unzip directories containing symbolic links. I want to add the link itself without following it. Reading out the link with readlink() and adding it to the zip archive results in a nonsense standard file when unzipped with unzip.
Solution does not need to be portable, it will only be used under Linux. The linux zip command has a --symlinks flags so the zip standard should support it. System calls are not really an option, the number of files is quite large and this makes the application extremely slow.
Is it possible to add symlinks with libzip, and how?
Thanks,
Sander
Based on documentation: no
According to its webpage, libzip is based on zlib. The zip program used in Linux, etc, is info-zip, which does not use zlib, but is self-contained (and contains features not in zlib).
Yes it's possible.
Below a function i use for zipping a list of files in c-code.
The files to zip are stored in a cJSON struct,no uid/gid set and files/directories relative to a directory "base" (as that is my appliction).
The Function returns 0 on success.
int list_zip_it(char * upload_zip_name,char * base, cJSON * filelist)
{
int result=0;
int error_n = 0;
struct zip *archive = zip_open(upload_zip_name, ZIP_TRUNCATE | ZIP_CREATE, &error_n);
if(!archive)
{
printf(stderr,"could not open or create archive\n");
return -1;
}
mode_t mode=0;
cJSON * item;
cJSON_ArrayForEach(item,filelist)
{
char * path=NULL;
path=item->valuestring;
// stat the item
struct stat sb;
if (stat(path, &sb) == 0 ) mode=sb.st_mode;
zip_uint32_t attr=0;
attr=((mode ) << 16L);
char rel_file[1024];
if (strncmp(path,CI_PROJECT_DIR,strlen(base))==0 )
{
snprintf(rel_file,1024,"%s",path+strlen(base)+1);
printf("archive filename: %s\n",rel_file);
}
else
{
fprintf(stderr,"filename outside base-derectory\n");
continue;
}
if (S_ISDIR(mode))
{
int index = (int)zip_add_dir(archive, rel_file);
if (index>0) zip_file_set_external_attributes(archive, index, 0, ZIP_OPSYS_UNIX, attr);
}
else if (S_ISLNK(mode)) // symlink
{
char link[1024];//=calloc(1, 1024);
memset(link, 0, 1024);
ssize_t size_link=readlink(path , link, 1023);
if (size_link > 0)
{
struct zip_source *source = zip_source_buffer(archive , link, ( zip_uint64_t)size_link,0);
if (source)
{
int index = (int)zip_add(archive, rel_file, source);
if (index>0) zip_file_set_external_attributes(archive, index, 0, ZIP_OPSYS_UNIX, attr);
}
else
{
printf(stderr,"failed to create source buffer: %s \n", zip_strerror(archive) );
zip_source_free(source);
}
}
else error("failed to read link: %s \n",path );
}
else if (S_ISREG(mode))
{
struct zip_source *source = zip_source_file(archive, path, 0, 0);
if(source == NULL)
{
error("failed to create source buffer: %s \n", zip_strerror(archive) );
result=1;
break;
}
// todo calculate filename relative to project_dir
int index = (int)zip_add(archive, rel_file, source);
if(index < 0 )
{
int zep,sep;
zip_error_get(archive, &zep, &sep);
if (zep== ZIP_ER_EXISTS )
{
fprintf(stderr,"failed to add file to archive: %s \n", zip_strerror(archive) );
zip_source_free(source);
}
else
{
fprintf(stderr,"failed to add file to archive: %s \n", zip_strerror(archive) );
zip_source_free(source);
result=1;
break;
}
}
else
{
zip_file_set_external_attributes(archive, index, 0, ZIP_OPSYS_UNIX, attr);
}
}
}
zip_close(archive);
return result;
}
I am trying to encrypt all the files in a particular folder and that folder has sub folders when i try to print all the files it works good but when i try to encrypt all the files it keeps on encrypting the same file again and again
void dirListFiles(wchar_t *startDir) {
HANDLE hFind;
WIN32_FIND_DATA wfd;
wchar_t path[99999];
char *enName;
const char *extension = ".enc";
int wcsChars;
wsprintf(path, L"%s\\*", startDir);
if ((hFind = FindFirstFile(path, &wfd)) == INVALID_HANDLE_VALUE) {
return;
}
do {
if ((wcsncmp(L".", wfd.cFileName, 1) !=0) && (wcsncmp(L"..", wfd.cFileName, 2) != 0) ) {
wsprintf(path, L"%s\\%s", startDir, wfd.cFileName);
if (wfd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) {
dirListFiles(path);
} else {
wcsChars = wcslen(path);
char *szTo = new char[wcsChars + 1];
szTo[wcsChars] = '\0';
WideCharToMultiByte(CP_ACP, 0, path, -1, szTo, wcsChars, NULL, NULL);
enName = (char *)malloc(strlen(szTo) + 1 + 4);
strcpy(enName, szTo);
strcat(enName, extension);
// If i add this line it keeps on encrypting the same file
//fencrypt(szTo, enName, (unsigned const char*)"1234567812345678");
printf("%s\n", enName);
delete[] szTo;
free(enName);
}
}
} while(FindNextFile(hFind, &wfd));
if (GetLastError() != ERROR_NO_MORE_FILES) {
FindClose(hFind);
return;
}
}
If i add fencrypt(szTo, enName, (unsigned const char*)"1234567812345678"); then it encrypts the files in the main folder that is G:\WinApp but when it enters G:\WinApp\ipch\winapp-1918e0a3 it keeps on encrypting the same file again and again there is no file in G:\WinApp\ipch\ only a folder winapp-1918e0a3 this is my encryption function please tell where am i wrong
void fencrypt(char* read, char* write, const unsigned char* enc_key) {
RAND_bytes(iv, AES_BLOCK_SIZE);
readFile = fopen(read,"rb");
writeFile = fopen(write,"wb");
fwrite(iv, 1, 8, writeFile);
fwrite("\0\0\0\0\0\0\0\0", 1, 8, writeFile);
AES_set_encrypt_key(enc_key, 256, &key);
init_ctr(&state, iv);
while(1) {
bytes_read = fread(indata, 1, AES_BLOCK_SIZE, readFile);
AES_ctr128_encrypt(indata, outdata, bytes_read, &key, state.ivec, state.ecount, &state.num);
bytes_written = fwrite(outdata, 1, bytes_read, writeFile);
if (bytes_read < AES_BLOCK_SIZE) {
break;
}
}
fclose(writeFile);
fclose(readFile);
}
I'm far from 100% sure, but I have a feeling that when you create a new file in your directory, the FindNextFile gets confused and finds the same filename again because it's not actually the same file. You may need to either collect a list of all files in a directory and process them one at a time, or keep a list of what you have already done and skip over the ones you have done.
The FindNextFile call will probably see the newly appeared encrypted file and therefore you will keep processing the new files also.
Since you are appending an ".enc" extension to your encrypted files, it would be relatively simple to skip files that already have this extension.
In our application, we are generating some large ASCII log files to an Windows NTFS compressed directory. My users want to know both the compressed and uncompressed size of the files on a status screen for the application. We are using Rad Studio 2010 C++ for this application.
I found this nice recursive routine online to read the size of the files on the disk -
__int64 TransverseDirectory(string path)
{
WIN32_FIND_DATA data;
__int64 size = 0;
string fname = path + "\\*.*";
HANDLE h = FindFirstFile(fname.c_str(), &data);
if (h != INVALID_HANDLE_VALUE)
{
do
{
if ((data.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY))
{
if (strcmp(data.cFileName, ".") != 0 && strcmp(data.cFileName, "..") != 0)
{
// We found a sub-directory, so get the files in it too
fname = path + "\\" + data.cFileName;
// recurrsion here!
size += TransverseDirectory(fname);
}
}
else
{
LARGE_INTEGER sz;
sz.LowPart = data.nFileSizeLow;
sz.HighPart = data.nFileSizeHigh;
size += sz.QuadPart;
// ---------- EDIT ------------
if (data.dwFileAttributes & FILE_ATTRIBUTE_COMPRESSED)
{
unsigned long doNotCare;
fname = path + "\\" + data.cFileName;
DWORD lowWordCompressed = GetCompressedFileSize(fname.c_str(),
&doNotCare);
compressedSize += lowWordCompressed;
}
// ---------- End EDIT ------------
}
}
while (FindNextFile(h, &data) != 0);
FindClose(h);
}
return size;
}
But what I cannot find is any information on how to read compressed/uncompressed file size information. Suggestions on where to look?
The Win32 API GetFileSize will return the uncompressed file size. The API GetCompressedFileSize will return the compressed file size.