Amazon web service batch file upload using specific key - amazon-web-services

I would like to ask if there is any way to set a key for each uploaded file using the TransferManager (or any other class)? I am currently using the method uploadFileList for this and I noticed that I can define a callback for each file sent using the ObjectMetadataProvider interface, but I only have the ObjectMetadata at my disposal. I thought it would be possible to get the parent ObjectRequest and set the key value in there, but that does not seem to be possible.
What I am trying to achieve:
MultipleFileUpload fileUpload = tm.uploadFileList(bucketName, "", new File(directory), files, new ObjectMetadataProvider() {
#Override
public void provideObjectMetadata(File file, ObjectMetadata objectMetadata) {
objectMetadata.getObjectRequest().setKey(myOwnKey);
}
});
I am most likely missing something obvious, but I spent some time looking for the answer and cannot find it anywhere. My problem is that if I supply some files for this method, it takes their absolute path (or something like that) as a key name and that is not acceptable for me. Any help is appreciated.

I almost forgot about this post.
There was no elegant solution, so I had to resort to making my own transfer manager (MultiUpload) and check the list of each upload manually.
I can then set the key for each object upon creating the Upload object.
List<Upload> uploads = new ArrayList();
MultiUpload mu = new MultiUpload(uploads);
for (File f : files) {
// Check, if file, since only files can be uploaded.
if (f.isFile()) {
String key = ((!directory.isEmpty() && !directory.equals("/"))?directory+"/":"")+f.getName();
ObjectMetadata metadata = new ObjectMetadata();
uploads.add(tm.upload(
new PutObjectRequest(bucketName,
key, f)
.withMetadata(metadata)));
}
}

Related

Can't upload file to aws s3 asp.net

fileTransferUtility = new TransferUtility(s3Client);
try
{
if (file.ContentLength > 0)
{
var filePath = Path.Combine(Server.MapPath("~/Files"), Path.GetFileName(file.FileName));
var fileTransferUtilityRequest = new TransferUtilityUploadRequest
{
BucketName = bucketName,
FilePath = filePath,
StorageClass = S3StorageClass.StandardInfrequentAccess,
PartSize = 6291456, // 6 MB.
Key = keyName,
CannedACL = S3CannedACL.PublicRead
};
fileTransferUtilityRequest.Metadata.Add("param1", "Value1");
fileTransferUtilityRequest.Metadata.Add("param2", "Value2");
fileTransferUtility.Upload(fileTransferUtilityRequest);
fileTransferUtility.Dispose();
}
Im getting this error
The file indicated by the FilePath property does not exist!
I tried changing the path to the actual path of the file to C:\Users\jojo\Downloads but im still getting the same error.
(Based on a comment above indicating that file is an instance of HttpPostedFileBase in a web application...)
I don't know where you got Server.MapPath("~/Files") from, but if file is an HttpPostedFileBase that's been uploaded to this web application code then it's likely in-memory and not on your file system. Or at best it's on the file system in a temp system folder somewhere.
Since your source (the file variable contents) is a stream, before you try to interact with the file system you should see if the AWS API you're using can accept a stream. And it looks like it can.
if (file.ContentLength > 0)
{
var transferUtility = new TransferUtility(/* constructor params here */);
transferUtility.Upload(file.InputStream, bucketName, keyName);
}
Note that this is entirely free-hand, I'm not really familiar with AWS interactions. And you'll definitely want to take a look at the constructors on TransferUtility to see which one meets your design. But the point is that you're currently looking to upload a stream from the file you've already uploaded to your web application, not looking to upload an actual file from the file system.
As a fallback, if you can't get the stream upload to work (and you really should, that's the ideal approach here), then your next option is likely to save the file first and then upload it using the method you have now. So if you're expecting it to be in Server.MapPath("~/Files") then you'd need to save it to that folder first, for example:
file.SaveAs(Path.Combine(Server.MapPath("~/Files"), Path.GetFileName(file.FileName)));
Of course, over time this folder can become quite full and you'd likely want to clean it out.

SmtpClient.send - Could not find a part of the path

I'm trying to write an email to my local folder. I successfully wrote an email to my documents folder using this code:
using (var client = new SmtpClient())
{
client.UseDefaultCredentials = true;
client.DeliveryMethod = SmtpDeliveryMethod.SpecifiedPickupDirectory;
client.PickupDirectoryLocation = tempDocsPath;
client.Send(message);//Writes to the PickupDirectoryLocation
}
However, when I ported this same code to another project, it gives me this error:
System.Net.Mail.SmtpException : Failure sending mail. ---> System.IO.DirectoryNotFoundException : Could not find a part of the path 'C:\Users\josh.bowdish\source\repos\GenerateEmail\GenerateEmail\bin\Debug\net461\tempFiles\AAMkAGUyODNhN2JkLThlZWQtNDE4MS1hODM1LWU0ZDY4Y2NhYmMxOQBGAAAAAABKB1jlHZSIQZSWN7AYZH2SBwDZdOTdKcayQ5NMwcwkNT7UAAAAAAEMAADZdOTdKcayQ5NMwcwkNT7UAACn\0a5b24a5-d625-4ecd-9990-af5654679820.eml'.
I've verified that the directory it's trying to write to exists, even rewrote it to look like this:
private static string WriteEmail(MailMessage message, string messageDirectory)
{
if (Directory.Exists(messageDirectory))
{
using (var client = new SmtpClient())
{
client.UseDefaultCredentials = true;
client.DeliveryMethod = SmtpDeliveryMethod.SpecifiedPickupDirectory;
client.PickupDirectoryLocation = messageDirectory;
client.Send(message);//Writes to the PickupDirectoryLocation
}
...
}
//stuff that returns the full email path
}
It breaks on the client.Send() line with the above error. As far as I can tell the code paths are identical. I've tried writing to the same folder that the other project is working with to no avail. The only thing I can think of is it's trying to write the email file before it exists, but the other project is writing it just fine.
Can someone tell me what is generating this error?
Thanks,
~Josh
This could be a permissions problem. Ensure that the account that your application is running under has permissions to "write" to this directory. Your Directory.Exists could be passing since it is only checking if the directory is there, but failing when trying to actually write to it.

Sharepoint document library storing files on filesystem

I'm in a bit of trouble here. Here is the context:
One of our customers asked us to develop an alternative solution to storing documents of a document library in the content database as their content database is growing too fast. They provided us with a network storage so that the documents could be stored in the filesystem instead. After googling a bit, I've found a feature called Remote Blob Storage RBS RBS, but as the references say, this is a per content database feature which is not acceptable for the context. The other option I've come up with is the use of SPItemEventReceiver so that in the ItemAdded event I could save the SPFile associated with the ListItem of the SPItemEventProperties property to the filesystem and possibly delete or truncate the SPFile object
public static void DeleteAssociatedFile(SPWeb web, SPListItem item)
{
try
{
if (item == null) { throw new ArgumentNullException("item"); }
if (item.FileSystemObjectType == SPFileSystemObjectType.File)
{
web.AllowUnsafeUpdates = true;
using (var fileStream = item.File.OpenBinaryStream())
{
if (fileStream.CanWrite)
{
fileStream.SetLength(0);
}
}
item.File.Update();
}
}
catch (Exception ex)
{
// log error message
Logger.Unexpected("ListItemHelper.DeleteAssociatedFile", ex.Message);
throw;
}
finally
{
web.AllowUnsafeUpdates = false;
}
}
forcing it to not store its content into the content database. But it didn't work out. Everytime that I somehow manage to delete or truncate the SPFile associated with the ListItem, the ListItem itself either gets deleted from the document library or the file doesn't get affected by the change. So my is question is: is there a solution for this problem? Any other thoughts that could help me in this quest?
Thanks in advance!
As you have asked other thoughts
One thing coming into my mind is one drive for business instead of network storage
Another is develop custom file upload, upload the file directly to network storage and once uploaded, add an entry in SharePoint list.

Move a file after an email is sent

I am writing a program that looks for files in a folder, attaches the files to the MailMessage and sends an email using SmtpClient.
After the email is sent out successfully, I want to move the emailed files to a different folder.
I get this message "The process cannot access the file because it is being used by another process.". I tried Thread.Sleep() but did not work.
smtpClient.Send(mail);
foreach (var report in reports)
{
string source = Path.Combine(reportsFolder, report);
string destination = Path.Combine(sentReportsFolder, report);
File.Move(source, destination);
}
First, try to dispose your smtpclient class:
smtpClient.Send(mail);
smtpClient.Dispose();
http://msdn.microsoft.com/pt-br/library/system.net.mail.smtpclient.dispose.aspx
But, when creating the class, you could use an using statemant.
Like:
using (SmtpClient smtpClient = new SmtpClent()) {
//attach file
smtpClient.Send();
}
This will ensure that, after send an email, the class will releases any resources that might be locked by the class. So, you not need to call .Dispose() explicitly.
http://msdn.microsoft.com/pt-br/library/system.net.mail.smtpclient.aspx
http://msdn.microsoft.com/en-us/library/yh598w02.aspx

Flash Builder (Mobile) - Dynamic Web Service URL

For my Flash Builder 4.6 Project I have a http service defined which looks at a url from our website.
What I'd like to be able to do though is to change the web service url on the fly within the app. i.e. using the existing url as default but having an admin/settings screen to change where the web service points (either stored in our sqlite database or in local memory).
This would be so that we could allow our customers to host their own version of the website/database but still be able to use/download the app through the app stores.
Has anyone had any experience with doing this?
EDIT: Adding some more details after the comments below.
When I created the HTTP Service through the FlashBuilder wizard it creates two web service classes a super class and a sub class which inherits from the super class. All of the code that the wizard populates goes into the super class.
I can assume that the code I need to put in would be in the sub class. But I do not know which function I'd put it in or how.
Below is a sample of the Super's constructor:
// initialize service control
_serviceControl = new mx.rpc.http.HTTPMultiService("websitehere");
var operations:Array = new Array();
var operation:mx.rpc.http.Operation;
var argsArray:Array;
operation = new mx.rpc.http.Operation(null, "loginRequest");
operation.url = "login.php";
operation.method = "GET";
argsArray = new Array("un","pw");
operation.argumentNames = argsArray;
operation.serializationFilter = serializer0;
operation.properties = new Object();
operation.properties["xPath"] = "/";
operation.contentType = "application/x-www-form-urlencoded";
operation.resultType = valueObjects.Data;
operations.push(operation);
_serviceControl.operationList = operations;
I'm not sure what property of the _serviceControl variable I would need to alter.
Also when I search for my website in my code it brings back a .fml file inside a .model directory which seems to get auto refreshed if I change the service url through the wizard. Would this not cause an issue?
I then have the challenge of accessing the user defined url. Within the app we use an sqlite database to store data but I think it would probably be better to use a 'SharedObject' which we also use to know what account they are logged into. How reliable is this? I assume I would be able to access this via the Service?
Though the awkward thing is that we were planning to have this configurable on a settings screen that would have been accessed after logging in. But to log in it would already need to know which server to point to.
if im reading your question correctly then your main ambition is to dynamically change the url for the services based on a user defined variable.
This is very easy to accomplish and even easier to accomplish if you are using parsley / spicelib.
a few points
dont change the code in the super file, this will get overwritten whenever the service gets refreshed. change everything in its generated sub-Class.
Shared Objects are very good for small quantities of data but should never be used for massive datasets i.e storing a big arraycollection.
Anyway here is how i achieve this.
In the SubClass you can change the constructor function.
Here is how i change my urls based on a config variable but you can just as easily use a SharedObject instead.
public function SubClassConstructor(){
if(CONFIG::DOMAIN_IDENT == "development" || CONFIG::DOMAIN_IDENT == "dev" || CONFIG::DOMAIN_IDENT == "d"){
_serviceControl.endpoint = "http://yoururl1";
}
else if(CONFIG::DOMAIN_IDENT == "production" || CONFIG::DOMAIN_IDENT == "prod" || CONFIG::DOMAIN_IDENT == "p"){
_serviceControl.endpoint = "http://yoururl2";
}
}
Of course this isn't exactly what your looking for but its a working solution, of course you can use Bindings to a Global ApplicationModel or direct reference to the SharedObject i guess you already know how to use the SharedObject.
Ask if you need any further help or guidance.
As cghrmauritius' solution didn't quite work for me, I am posting up the final solution that did work in my situation.
public function subConstructor()
{
super();
_serviceControl.baseURL = "http://url1";
}
Obviously for my final solution I need to implement the shareobject as well but overriding the url was my main priority.