I have small lambda function which donwload files from s3 buckets, create zip of it and upload on S3 bucket again.
My lambda function use .net core3.1.
I can write text file but not create zip file.
Zip File Code in C# (as it deployed in linux system, hence I tried to create zip file in Tmp direcotry)
ZipFile.CreateFromDirectory(#"/tmp/", "test.zip");
Lambda function return below error.
Error :
at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
at System.IO.FileStream.OpenHandle(FileMode mode, FileShare share, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, Boolean useAsync)
at System.IO.Compression.ZipFile.Open(String archiveFileName, ZipArchiveMode mode, Encoding entryNameEncoding)
at System.IO.Compression.ZipFile.DoCreateFromDirectory(String sourceDirectoryName, String destinationArchiveFileName, Nullable`1 compressionLevel, Boolean includeBaseDirectory, Encoding entryNameEncoding)
at System.IO.Compression.ZipFile.CreateFromDirectory(String sourceDirectoryName, String destinationArchiveFileName)
at XXX.DownloadService.CreateZip(DownloadRequest downloadRequest) in D:\Projects\DownloadService.cs:line 133
I don't know what did I wrong. or what is alternative to create zip file if this approach is wrong.
Help me.
Thanks in advance.
I suspect that it is trying to create the Zip file in a read-only directory. The second parameter can include the path of the destination Zip file:
public static void CreateFromDirectory (string sourceDirectoryName, string destinationArchiveFileName);
destinationArchiveFileName: The path of the archive to be created, specified as a relative or absolute path. A relative path is interpreted as relative to the current working directory.
However, in an AWS Lambda function, only the /tmp/ directory can be written to. This might conflict with the fact that you are zipping the entire /tmp/ directory. You might need to put your files in a sub-directory of the /tmp/ directory:
ZipFile.CreateFromDirectory(#"/tmp/files_to_zip/", "/tmp/test.zip");
Related
I want to load the AWS config file and edit the contents of the file.
I found #aws-sdk/shared-ini-file-loader, that works well to load the config file data as the JSON object.
import { loadSharedConfigFiles } from '#aws-sdk/shared-ini-file-loader'
let awsFileContents = await loadSharedConfigFiles({ configFilepath: '~/.aws/config' })
console.log(awsFileContents.configFile)
Now I want to perform some changes in the awsFileContents.configFile object, parse it back to the correct format, and write it back to the ~/.aws/config file.
Is there an AWS module available that can do that?
I have tried ini, multi-ini, and conf-cfg-ini. But they have issues while parsing the JSON back to the correct format.
You do not need the SDK to read/write the config file. It is a normal INI file you can modify with standard tools for INI files.
You can use e.g. the ini package for this: https://www.npmjs.com/package/ini
In other languages like Python and also for my "AWS Manager" (Windows application written in Delphi) I use simple ini function to read and also write the config file without any issues.
Getting following error when using custom json configuration file.
Severity Code Description Project File Line Suppression State
Error The "SlowCheetah.TransformTask" task failed unexpectedly.
System.UnauthorizedAccessException: Access to the path 'D:\Data\…\Settings.Release.json' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Microsoft.VisualStudio.Jdt.JsonTransformation..ctor(String transformFile, IJsonTransformationLogger logger)
at Microsoft.VisualStudio.SlowCheetah.JsonTransformer.Transform(String sourcePath, String transformPath, String destinationPath)
at Microsoft.VisualStudio.SlowCheetah.TransformTask.Execute()
at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
at Microsoft.Build.BackEnd.TaskBuilder.d__26.MoveNext() OzCruisingHangfireAgent.Program
Best Regards,
Damodar
This is a known issue.
The JSON transformer in SlowCheetah v3.1.66 doesn't support the source or transform files being read-only yet.
See the following issues:
https://github.com/Microsoft/slow-cheetah/issues/86
https://github.com/Microsoft/json-document-transforms/issues/16
The merged PR#17 has a fix for Microsoft.VisualStudio.Jdt, but it hasn't been released yet.
Here are a few work arounds:
Make all files writeable when checking out of source control.
Call the Attrib MSBuild task before the transformation to remove the read-only file attribute.
For example:
<Attrib Files="D:\Data\…\Settings.Release.json" Normal="true"/>
Call the Exec MSBuild task before the transformation to remove the read-only file attribute.
For example:
<Exec Command="attrib -R "D:\Data\…\Settings.Release.json""/>
I need to receive the absolute file path like
C:/Users/Dima/YandexDisk/_Application/WeldAnalysis/ExperimentDefaults.xlsx
from path of QRC file like
:/Data/ExperimentDefaults.xlsx.
How can I do this?
Resources are packed all together into the QRC file, they are not exposed to the filesystem, thus they cannot be converted to a standard path; it is like accessing a file inside a zip file.
What are you willing to do with such file path? QFile can deal with resource paths (":/...") and open such files, but other file handling APIs don't (such as C's fopen or C++'s std::ifstream). Also, and for the same reason, non-Qt application won't be able to open such files if passed as a parameter.
Options you have include:
opening the file using QFile and read it using related Qt classes (given you are dealing with an Excel file I think it doesn't worth the effort, unless you have some library to read it from a memory buffer).
copying the file out of the resources container to a temporary directory (you can use QTemporaryDir to create one), and use the file from there:
QTemporaryDir tempDir;
if (tempDir.isValid()) {
const QString tempFile = tempDir.path() + "/yourfile.xlsx";
if (QFile::copy(":/yourfile.xlsx", tempFile)) {
// your file is now extracted to the filesystem
}
}
Only be aware that when tempDir is destroyed, the temporary directory is also deleted, therefore your file. You can avoid this by disabling the auto-remove feature: tempDir.setAutoRemove(false) and manually deleting the temporary directory when you finish with the extracted file.
First of all, if there is an easier way than using inotify, please tell me!
Basically what I would like to do is watching a root directory with inotify with these flags: IN_CREATE | IN_MODIFY | IN_DELETE.
When it's IN_CREATE and IN_ISDIR I would like to watch that folder too. But the main thing I need is whether a file was created, deleted or modified even in subdirectories. Now I know I can just add multiple directories with inotify_add_watch(), but when I read the event->name how can I know which directory it belongs to? The inotify_event struct doesn't seem to hold that value. So if I have a structure like this:
/root
Then I create a directory "a":
/root/a
Then create a file:
/root/a/tmp.txt
When I read event->name it'll only say tmp.txt, but how can I know it is in the "a" subdirectory? How can I know what the watch descriptor was?
In inotify_event structure the name field contains the name of the object to which the event occurred, relative to wd. You need to get the absolute path of the parent directory and concatenate the name of the file/directory to get the full path.
Also in Inotify_event structure the mask field, you can use the IN_ISDIR mask bit to know if the event that has occurred for that wd is a file or a directory.
This is from the inotify
here
The name field is only present when an event is returned for a file inside a watched directory; it identifies the file pathname relative to the watched directory. This pathname is null-terminated, and may include further null bytes to align subsequent reads to a suitable address boundary.
Here's how I did it:
I created a hashmap (QHash<int, QString> fd_to_path) during inotify_add_watch() time to couple the received wd with its corresponding directory string:
int wd = inotify_add_watch(...next_dir_path..);
if (wd != -1)
fd_to_path.insert(wd, next_dir_path);
Then when reading the received inotify event after struct inotify_event *ev = (...); you just query the corresponding dir path with:
QString dir_path = fd_to_path.value(ev->wd);
So I was able to get the AppData folder via SHGetKnownFolderPath and converted the memory address it printed to a readable string via
SHGetKnownFolderPath(FOLDERID_RoamingAppData, NULL, NULL, &wszPath);
_bstr_t bstrPath(wszPath);
std::string strPath((char*)bstrPath);
newstring.append(strPath);
newstring.append(secondvar);
So you probably noticed the newstring.append. What I do is append the folder name I want and file to the end of the AppData location which is C:\Users\*Username*\AppData\Roaming(and append here my folder and file).
Then I use cURL to send the file from newstring to my cURL function called sendfile, however because of the hacks I tried to convert the memory address to a readable string and then append the other information, I then got a heap corrupted message.
I then tried manually putting the path to the directory and file to cURL function curl_formadd and it still wouldn't work. However, if there is a file where the application is, and use it for sending through HTTP, it works. If I add a full path, i.e C:\Users\*Username*\AppData\Roaming\myfolder\myfile.txt to the curl_formadd function, nothing happens.
So how do I go about fetching that file from the directory I want and sending it with cURL?