I'm trying to upload a file with presigned AWS url.
I generate a presigned url like that :
use AsyncAws\S3\S3Client;
$s3 = new S3Client();
$bucket = 'my-bucket';
$key = 'myfile.pdf';
$date = new \DateTimeImmutable('+15minutes');
$contentType = 'application/pdf';
$input = new GetObjectRequest([
'Bucket' => $bucket,
'Key' => $key,
'ContentType' => $contentType
]);
$presignUrl = $s3->presign($input, $date);
This code above works fine, I get a presigned url like this : "https://my-bucket.s3.eu-west-3.amazonaws.com/myfile.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20220809T141153Z&X-Amz-Expires=900&X-Amz-Credential=XXXXXXXXXXXXXXX%2F20220809%2Feu-west-3%2Fs3%2Faws4_request&x-amz-content-sha256=UNSIGNED-PAYLOAD&X-Amz-SignedHeaders=host&X-Amz-Signature=XXXXXXXXXXXXXXXXXXXXXXXXXX"
Next, I try to use this url to upload a file with Postman.
I select my pdf file via Body / binary / Select file
This is my postman config and response :
As you can see, I have a 403 code response.
These are my headers (of course I've tried with differents Content-Type but nothing better than the exactly same 403 response.
Important information : With the same $s3 instance and authentification, if I try to get object url ($content = $s3->getObject($input); with AsyncAws), it works well. So I suppose, it is not an authentification issue.
Thanks in advance if you have any idea for me !
Related
I am working with AWS S3 Bucket, and trying to upload image from react native project managed by expo. I have express on the backend. I have created a s3 file on backend that handles getting the presigned url, and this works, and returns the url to the front end inside this thunk function from reduxjs toolkit. I used axios to send request to my server, this works. I have used axios and fetch to try the final put to the presigned url but when it reached the s3 bucket there is nothing in the file just an empty file with 200 bytes everytime. When I use the same presigned url from postman and upload and image in binary section then send the post request the image uploads to the bucket no problems. When I send binary or base64 to bucket from RN app it just uploads those values in text form. I attempted react-native-image-picker but was having problems with that too. Any ideas would be helpful thanks. I have included a snippet from redux slice. If you need more info let me know.
redux slice projects.js
// create a project
// fancy funtion here ......
export const createProject = createAsyncThunk(
"projects/createProject",
async (postData) => {
// sending image to s3 bucket and getting a url to store in d
const response = await axios.get("/s3")
// post image directly to s3 bucket
const s3Url = await fetch(response.data.data, {
method: "PUT",
body: postData.image
});
console.log(s3Url)
console.log(response.data.data)
// make another request to my server to store extra data
try {
const response = await axios.post('/works', postData)
return response.data.data;
} catch (err) {
console.log("Create projects failed: ", err)
}
}
)
I am currently creating a basic app with React-Native (frontend) and Flask/MongoDB (backend). I am planning on using AWS S3 as cheap cloud storage for all the images and videos that are going to be uploaded and viewed. My current idea (and this could be totally off), is when a user uploads content, it will go through my Flask API and then to the S3 storage. When a user wants to view content, I am not sure what the plan of attack is here. Should I use my Flask API as a proxy, or is there a way to simply send a link to the content directly on S3 (which would avoid the extra traffic through my API)?
I am quite new to using AWS and if there is already a post discussing this topic, please let me know, and I'd be more than happy to take down this duplicate. I just can't seem to find anything.
Should I use my Flask API as a proxy, or is there a way to simply send a link to the content directly on S3 (which would avoid the extra traffic through my API)?
If the content is public, you just provide an URL which points directly to the file on the S3 bucket.
If the content is private, you generate presigned url on your backend for the file for which you want to give access. This URL should be valid for a short amount of time (for example: 15/30 minutes). You can regenerate it, if it becomes unavailable.
Moreover, you can generate a presigned URL which can be used for uploads directly from the front-end to the S3 bucket. This might be an option if you don't want the upload traffic to go through the backend or you want faster uploads.
There is an API boto3, try to use it.
It is not so difficult, I have done something similar, will post code here.
I have done like #Ervin said.
frontend asks backend to generate credentials
backend sends to frontend the credentials
Frontend upload file to S3
Frontend warns backend it has done.
Backend validate if everything is ok.
Backend will create a link to download, you have a lot of security options.
example of item 6) To generate a presigned url to download content.
bucket = app.config.get('BOTO3_BUCKET', None)
client = boto_flask.clients.get('s3')
params = {}
params['Bucket'] = bucket
params['Key'] = attachment_model.s3_filename
params['ResponseContentDisposition'] = 'attachment; filename={0}'.format(attachment_model.filename)
if attachment_model.mimetype is not None:
params['ResponseContentType'] = attachment_model.mimetype
url = client.generate_presigned_url('get_object', ExpiresIn=3600, Params=params)
example of item 2) Backend will create presigned credentials to post your file on S3, send s3_credentials to frontend
acl_permission = 'private' if private_attachment else 'public-read'
condition = [{'acl': acl_permission},
["starts-with", "$key", '{0}/'.format(folder_name)],
{'Content-Type': mimetype }]
bucket = app.config.get('BOTO3_BUCKET', None)
fields = {"acl": acl_permission, 'Bucket': bucket, 'Content-Type': mimetype}
client = boto_flask.clients.get('s3')
s3_credentials = client.generate_presigned_post(bucket, s3_filename, Fields=fields, Conditions=condition, ExpiresIn=3600)
example of item 5) Here are an example how backend can check if file on S3 are ok.
bucket = app.config.get('BOTO3_BUCKET', None)
client = boto_flask.clients.get('s3')
response = client.head_object(Bucket=bucket, Key=s3_filename)
if response is None:
return None, None
md5 = response.get('ETag').replace('"', '')
size = response.get('ContentLength')
Here are an example how frontend will ask for credentials, upload file to S3 and inform backend it is done.
I tried to remove a lot of particular code.
//frontend asking backend to create credentials, frontend will send some file metadata
AttachmentService.createPostUrl(payload).then((responseCredentials) => {
let form = new FormData();
Object.keys(responseCredentials.s3.fields).forEach(key => {
form.append(key, responseCredentials.s3.fields[key]);
});
form.append("file", file);
let payload = {
data: form,
url: responseCredentials.s3.url
}
//Frontend will send file to S3
axios.post(payload.url, payload.data).then((res) => {
return Promise.resolve(true);
}).then((result) => {
//when it is done, frontend informs backend
AttachmentService.uploadSuccess(...).then((refreshCase) => {
//Success
});
});
});
trying to generate a pre-signed query string to POST to an S3 bucket using the AWS SDK for PHP. Getting the following error:
The request signature we calculated does not match the signature you provided.
Here's the php file to generate the URL, adapted from here.
<?php
require('./aws/aws-autoloader.php');
$client = new \Aws\S3\S3Client([
'version' => 'latest',
'region' => 'us-east-1',
'credentials' => [
'key' => '[KEY]',
'secret' => '[SECRET]',
],
]);
$bucket = '[mybucket]';
// Set some defaults for form input fields
$formInputs = ['acl' => 'public-read'];
// Construct an array of conditions for policy
$options = [
['acl' => 'bucket-owner-full-control'],
['bucket' => $bucket],
];
// Optional: configure expiration time string
$expires = '+2 hours';
$postObject = new \Aws\S3\PostObjectV4(
$client,
$bucket,
$formInputs,
$options,
$expires
);
// Get attributes to set on an HTML form, e.g., action, method, enctype
$formAttributes = $postObject->getFormAttributes();
// Get form input fields. This will include anything set as a form input in
// the constructor, the provided JSON policy, your AWS Access Key ID, and an
// auth signature.
$formInputs = $postObject->getFormInputs();
echo "https://[mybucket].s3.amazonaws.com/?".http_build_query($formInputs)."&X-Amz-Expires=7200&X-Amz-SignedHeaders=host";
?>
And the curl command:
curl --request POST --upload-file "file.gif" -k [query string here]
I am uploading PDF's to AmazonS3 manually, using Panic Transmis and via a PHP script/API.
For some reason, some display in your browser, and some force download.
I have checked permission and can not seem to see any issues, Can anyone help explain how to make PDF's always display in browser ( unless the user specifies otherwise ).
I don't think it is as browser issue.
You need to change the Content-Type and Content-Disposition.
Content-Type: application/pdf;
Content-Disposition: inline;
Using the AWS S3 console, find the file and using the context menu (right click) select Properties then it's under Metadata.
Or change this programmatically:
http://docs.aws.amazon.com/AWSSDKforPHP/latest/index.html#m=AmazonS3/create_object
In companion with well's answer, here an example:
public function save($bucket, $name, $content, $options = [])
{
$this->s3->putObject([
'Bucket' => $bucket,
'Key' => $name,
'Body' => $content,
] + $options);
}
$this->bucket->save('my-bucket', 'SofiaLoren.pdf', $content, [
'ContentType' => 'application/pdf',
'ContentDisposition' => 'inline',
]);
At the moment I have to download images on the server and post them like this:
$photo = array(
'message' => 'Status',
'source' => '#/full/path/of/the/image.png'
);
$response = $fb->api('/'.$album.'/photos', 'POST', $photo);
I'm using curl on the backend to post this request and it's working like a charm.
I'm wondering if it's possible to post the remote image directly instead to download a local copy?
I tried to do something like this:
$photo = array(
'message' => 'Status',
'source' => file_get_contents('http://www.domain.com/image.png')
);
$response = $fb->api('/'.$album.'/photos', 'POST', $photo);
But I got an exception from the graph API: "(#324) Requires upload file"
It looks like this is happening when you are not sending the multipart/data header which is set automatically when sending an array of data ($data is an array).
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
So I'm doubtful that it's possible to post a remote image.
What do you think?
It is possible to upload a photo by just giving that photo’s URL under the parameter name url.
This info is a little hidden in the description of the photo endpoint, where it just says,
“You can also publish a photo by providing a url param with the photo's URL.”
So instead of providing the sourceparameter, just provide url with the value of the photo’s publicly reachable URL. (All other parameters except source stay the same and are still usable in the same way.)
I tried this recently, and it worked fine. Although, I only tried it for photos with URLs from my app domain – I can’t say for sure if it works for URLs from “anywhere” on the web as well (although i can’t see a good reason for why it shouldn’t).