Is it possible to download Zip file present in S3 using API Gateway alone.
Configurations:
Integration type : AWS Service (S3)
endpoint : GET
Content-Type: application/zip (or) application/octet-stream
A corrupted zip file is getting downloaded.
I could able to do a workaround using S3 presigned url and don't want to make the bucket public.
The file can get corrupted when some headers are missing from the upload PUT request. Please make sure the file is not-corrupted by downloading it manually and checking. If it's corrupted from the GET request only please make sure you include all the needed headers in the GET request too.
Related
We have a client incorrectly setting the Content-Type to application/json when uploading a pdf file to s3. This results in the client library we use to download the file to give it the wrong extension, also happens when you download the file from the s3 console, even though it has .pdf as the suffix, it downloads to .json.
Is there anyway to prevent clients from doing this? The Content type should always be application/x-www-form-urlencoded from my understanding.
I have an GET API created which uses nodejs lambda to return a zip file with base64 encoding and content-type - application/zip. And it is working fine by downloading zip file while hitting from browser or postman when it doesn't configured with any authorizer.
But when I add an authorizer to this API, I am not able to see this zip file downloaded instead getting as json response where the response header sets content-type to "application/json".
{
"statusCode":200,
"isBase64Encoded":true,
"body":"UEsFBgAAAAAAAAAAAAAAAAAAAAAAAA==",
"headers":{
"Content-Type":"application/zip, application/octet-stream",
"Content-Disposition":"attachment; filename=Testfile.zip",
"Content-Encoding":"base64"
}
}
How to get the result in the same way i.e. getting zip file downloaded?
I changed the api integration type to lambda-proxy instead of lambda.
This solved my issue and able to download the binary file.
My question came out when I experienced two different behaviors in object URL from json files stored in a s3 bucket.
Consider a json file: mydata.json
If I upload this file using s3 dashboard from AWS website, I am able to see data in browser: //s3-us-west-2.amazonaws.com/bucket/folder/mydata.json. I am also able to read this data from a different application if I create a specific configuration in s3 bucket.
For the other hand, if I use boto3 library for python and upload the same file in the same bucket (making file public in the process), when I click object URL it downloads the file, but it doesn't open data in browser.
This is the code I used:
# upload json file
bucket.upload_file(path, jsonkey)
object_acl = s3.ObjectAcl('bucket_name', jsonkey)
bucket_response = object_acl.put(ACL='public-read')
I explored file properties such as metadata. When I upload file via dashboard, the metadata assigned is Content-Type: application/json, and via boto3 is Content-Type: binary/octet-stream. I don't really know if metadata affects the object URL behavior.
In this context, how can I properly configure files in json format to be downloaded or to be read? I mean, what is the main configuration that affects object URL behavior?
I couldn't find a significant difference between both methods (dashboard and boto3) in properties or permissions, besides Content-Type in metadata. However, when I tried to change Content-Type, behavior was the same.
Any other information I can provide to clarify this question, be free to ask. Thanks in advance.
The documentation for the S3 bucket resource's upload_file() method is not ideal as it simply refers you to the equivalent S3Transfer docs for how extra arguments can be used.
Try the following:
bucket.upload_file(path, jsonkey, ExtraArgs={'ContentType': "application/json"})
As in the title, I can't seem to get it to work, i'm following the high level guide detailed here but any images uploaded seem to be blank.
What i've set up:
/images/{object} - PUT
> Integration Request
AWS Region: ap-southeast-2
AWS Service: S3
AWS Subdomain [bucket name here]
HTTP method: PUT
Path override: /{object}
Execution Role [I have one set up]
> URL Path Paramaters
object -> method.request.path.object
I'm trying to use Postman to send a PUT request with Content-Type: image/png and the body is a binary upload of a png file.
I've also tried using curl:
curl -X PUT -H "Authorization: Bearer [token]" -H "Content-Type: image/gif" --upload-file ~/Pictures/bart.gif https://[api-url]/dev/images/cool.gif
It creates the file on the server and the size seems to be double what ever was uploaded, when viewed I just get "image has an error".
When I try with .txt files (content-type: text/plain) it seems to work though.
Any ideas?
After reading alot and chatting to AWS technical support, the problem seems to be that you can't do binary uploads through API Gateway as anything that passes through automatically goes through a UTF-8 encode.
There are a few workarounds for this I can think of, my solution will be to base64 the files before upload and trigger a lambda when they hit the bucket to decode them
This is a old post, but I got a solution.
AWS now support binary upload through APIGateway READ.
In general, go to your API settings, and add a Binary Media type.
After that, you can handle the file in base64
I have Amazon S3 where all of my files are stored. Currently my users can go to a link where they can stream, but not download, audio and video files. How can I set up a link through either Amazon S3 or perhaps Amazon CloudFront that will allow someone to download an MP3 file or something of that nature?
Thanks for any advice!
You must set the file's content header to something other than the media type the browser understands. For example:
Content-Disposition: attachment; filename=FILENAME.EXT
Content-Type: application/octet-stream
This used to be a big issue if you wanted to have both features (ability to display/view and ability to download) and you used to have to proxy the file download through your EC2 or other annoying ways. Now S3 has it built in:
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html
You can override values for a set of response headers using the query
parameters listed in the following table. These response header values
are only sent on a successful request, that is, when status code 200
OK is returned. The set of headers you can override using these
parameters is a subset of the headers that Amazon S3 accepts when you
create an object. The response headers that you can override for the
GET response are Content-Type, Content-Language, Expires,
Cache-Control, Content-Disposition, and Content-Encoding. To override
these header values in the GET response, you use the request
parameters described in the following table. (linke above)