I am attempting to generate a presigned url that only allows one visit/use of the url.
I have been trying just using an expire time but from what I have tested anything under 70 seconds always gives an expired url error.
aws s3 presign s3://bucket/object --expires-in 70
The other alternative would be a short url expire time (e.g. 5 seconds) but I cannot get anything under 70 seconds to work without an expired url error.
If anything under 70 seconds gives you an error, your clock is almost certainly wrong on the machine where you are generating the signed URL.
Expiration is calculated as --expires-in seconds in the future relative to the clock on the machine where you are running aws-cli. There's an assumption that this is a trusted environment (your credentials are there) and the clock is also trusted to have been set accurately.
(The clock on the machine where the browser is being used to access the URL doesn't matter.)
Note that the fixed expiration time associated with a given URL is shown in the error message.
Related
I'm having issues uploading files, that take a bit of time, using presigned put object urls together with transfer acceleration. Sometimes it works and sometimes it doesn't.
I've performed the following tests using Java SDK 2.15.9 to generate urls and curl to upload. Also I'm uploading from Sweden to an s3 bucket located in the us-east-2 region using transfer acceleration.
file size
url expire time
transfer speed
time to upload
status
20mb
1min
100k/s
x
failed after 1min 23sec on 403 forbidden
42mb
10min
50k/s
13min
success
42mb
10min
10k/s
x
failed after 12min on 403 forbidden
42mb
10min
25k/s
28min
success
What is going on here? First theory of mine was that the expire time needed to be longer than the upload time. However reading through the docs it seems that expire time is validated on start of request. Is that true when transfer acceleration is enabled as well? Also using an expire time of 10min worked even tho one upload took 28min.
Do I need to set a longer expire time?
Used the following curl command:
curl -v -H "Content-Type: $contentType" --limit-rate $rateLimit --upload-file $file "$url"
Code to generate URL:
private val presigner by lazy {
S3Presigner.builder()
.credentialsProvider(DefaultCredentialsProvider.create())
.region(s3Region)
.serviceConfiguration(S3Configuration.builder()
.accelerateModeEnabled(true)
.checksumValidationEnabled(false)
.build())
.build()
}
override fun run() {
val url = presigner.presignPutObject { builder ->
builder.putObjectRequest {
it.bucket(s3Bucket)
it.key(UUID.randomUUID().toString())
it.contentType(contentType)
}.signatureDuration(Duration.ofSeconds(expire))
}.url()
println(url)
}
Got a response from AWS support that Cloudfront does not send the request to S3 until it has received the entire body. So the resolution is to increase the expire time of the upload URL.
From the SDK documentation (link: https://docs.aws.amazon.com/aws-sdk-php/v3/api/class-Aws.S3.S3Client.html#_createPresignedRequest) the $expires parameter should denote the time at which the URL should expire.
So if I specified 2 minutes as an expiration time, after 2 minutes the URL should be invalid. My code looks like this
<?php
$s3 = $this->cloudProvider->getClient(); // S3 client
$cmd = $s3->getCommand(
'GetObject',
[
'Bucket' => $this->getSdkBucket(), // Bucket name
'Key' => "$s3Name",
]
);
$urlReq = $s3->createPresignedRequest($cmd, $expirationTime); // $expirationTime is a Unix timestamp
And I get the url that has the correct expiry time (in my case the client wanted it to be the session expiry time, and session time is 4 hours)
X-Amz-Content-Sha256=UNSIGNED-PAYLOAD
&X-Amz-Security-Token=long_string_goes_here
&X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=another_string_goes_here
&X-Amz-Date=20200907T110127Z
&X-Amz-SignedHeaders=host
&X-Amz-Expires=14400 // This decreases depending on how long the user is logged in - max 4hrs
&X-Amz-Signature=another_string_here
The problem is that this url will be valid after 4 hours.
From what I've read in this answer about expiry time (https://stackoverflow.com/a/57792699/629127), the URL will be valid from 6 hours up to 7 days, depending on the credentials used to generate it.
And I'm using the IAM credentials (ECS provider).
So does this means that no matter what value I put in the expiry variable, I won't be able to make the link valid for that time period?
The expiration time is checked by S3 and if the browser or a proxy (if you are using CloudFront, for example) has the file cached then the request won't reach S3. If it is cached in the browser then don't worry, that means the same user can access the URL after expiration only.
You can use the browser devtools to check for this. On the network tab, find the request to the signed URL and you can see if it was returned from the cache (from memory/disk cache).
Is X-Amz-Expires a required header/parameter? Official documentation is inconsistent and uses it in some examples, while not in others.
If it is not required, what is the default expiration value of a signed request? Does it equal to the maximum possible value for X-Amz-Expires parameter, which is 604800 (seven days)?
The documentation (see above links) talks about X-Amz-Expires parameter only in context of passing signing parameters in a query string. If X-Amz-Expires parameter is required, is it only required for passing signing parametes in query string (as opposed to passing them with Authorization header)?
Update:
Introduction to AWS Security Processes paper, on page 17 says
A request must reach AWS within 15 minutes of the
time stamp in the request. Otherwise, AWS denies the request.
Now what time stamp are we talking about here? My guess is that it is X-Amz-Date. If I am correct, then another question crops up:
How do X-Amz-Date and X-Amz-Expires parameters relate to each other? To me it sounds like request expiration algorithm falls back to 15 mins from X-Amz-Date timestamp, if X-Amz-Expire is not present.
Is X-Amz-Expires a required header/parameter?
X-Amz-Expires is only used with query string authentication, not with the Authorization: header.
There is no default value with query string authentication. It is a required parameter, and the service will reject a request if X-Amz-Algorithm=AWS4-HMAC-SHA256 is present in the query string but X-Amz-Expires=... is not.
<Error>
<Code>AuthorizationQueryParametersError</Code>
...
Now what time stamp are we talking about here?
This refers to X-Amz-Date: when used with the Authorization: header. Because X-Amz-Date: is part of the input to the signing algorithm, a change in the date or time also changes the signature. An otherwise-identical request signed 1 second earlier or later has an entirely different signature. AWS essentially allows your server clock to be wrong by up to 15 minutes without breaking your ability to authenticate requests. It is not a fallback or a default. It is a fixed window.
The X-Amz-Date: of Authorization: header-based requests is compared by AWS to their system time, which is of course synched to UTC, and the request is rejected out if hand if this value is more than 15 minutes skewed from UTC when the request arrives. No other validation related to authentication occurs prior to the time check.
Validation of Query String authentication expiration involves different logic:
X-Amz-Expires must not be a value larger than 604800 or smaller than 0; otherwise the request is immediately denied without further processing, including a message similar to the one above.
X-Amz-Date must not be more than 15 minutes in the future, according to the AWS system clock. The error is Request is not yet valid.
X-Amz-Date must not be more than X-Amz-Expires number of seconds in the past, relative to the AWS system clock, and no 15 minute tolerance applies. The error is Request has expired.
If any of these conditions occur, no further validation is done on the signature, so these messages will not change depending on the validity of the signature. This is checked first.
Also, the leftmost 8 characters of your X-Amz-Date: must match the date portion of your Credential component of the Authorization: header. The date itself has zero tolerance for discrepancy against the credential (so, when signing, don't read your system time twice, else you risk generating an occasional invalid signature around midnight UTC).
Finally, requests do not expire while in the middle of processing. If you send a request using either signing method that is deemed valid when it arrives but would have expired very soon thereafter, it is always allowed to run to completion -- for example, a large S3 download or an EBS snapshot creation request will not start, then fail to continue, because the expiration timer struck while the request had already started on the AWS side. If the action was authorized when requested, then it continues and succeeds as normal.
I am using Wso2 API Manager 1.8. I have created some apis in it, and subscribed it to an application. I Want to make the token expire time to infinte , means never expire token. I have gone through the tutorial given in wso2 site and done following changes.
refered link : Changing the default token expiration time
I have changed ApplicationAccessTokenDefaultValidityPeriod to -1 . Rest remains same. And I created new application to get the changes, Now get long integer value 9223372036854452224 ie 106751991 days) as expire time. And working fine .
Then I changed the system date to next day check whether the expire not set to zero. But my token expire time chages to 3600000 milliseconds, How this happens, even if the time reduces this much change never expecting.
I have chages the default H2 database to mysql and done same thing with this.
Then I noticed that when the day changes,the token expire time reduces from 9223372036854452224 to 3600000 milliseconds means 1hr of expire time, which is default time. Actually it have 106751991 days expire time. How this happening ?
Please help. How can I solve this problem
Set token validity time to -1 (minus one) - that means never expires (also, see if that is the value that you have in API Store on Subscriptions tab)
We are getting "Forbidden error(403)" while trying to upload data on Google cloud when there is a time skew on my machine i.e. my machine clock is not synchronized/updated with the NTP server.
Why does Google not return the proper error information?
It is very likely that you are setting the "Date" field incorrectly. All (signed) API v1.0 requests must include a "Date" header, and that header must be part of the signature for the request. The Date field must be within 15 minutes of the real clock time that Google's servers receive your request. If your clock is more than 15 minutes skewed, your signed requests will be rejected.
For more, please see the v1.0 API documentation here: https://developers.google.com/storage/docs/reference/v1/developer-guidev1#authentication under the CanonicalHeaders section.
This is also the case with S3. See here: http://aws.amazon.com/articles/1109?_encoding=UTF8&jiveRedirect=1#04