Is there a way we can run select object content (s3 select) on specific version of s3 object using version Id?
I cannot find any references in select object content documentation to specify the version Id like we have version Id field in get Object request.
It does not look like this is possible.
The select_object_content() function takes a Bucket and Key, but not a VersionId.
Finally found a solution based on the request posted to aws sdk git channel.
request.putCustomQueryParameter("versionId", "<Your-version-Id>");
The aws docs should be updated soon since it was missing from there.
Related
I've create an S3 bucket for hosting my website. For that I've used the below code from the AWS CDK for python docs
self.bucket = s3.Bucket(
self,
"my-bucket-name",
bucket_name="my-bucket-name",
removal_policy=core.RemovalPolicy.DESTROY,
website_index_document="index.html",
public_read_access=True
)
For a reason, I want to send this bucket object as an argument to another object and get the bucket name from the argument. So, I've tried
self.bucket.bucket_name
self.bucket.bucket_arn
nothing seems working, instead the object returns ${Token[TOKEN.189]}. Could anyone guide me through this?
If the bucket name is hard coded like the example you pasted above, you can always externalize it to the cdk context file. As you've seen, when you access the bucket name from the Bucket construct, it creates a reference to it and that is so if you need it in another resource, cloud formation will depend on the value from the Bucket resource by using the Ref/GetAtt capabilities in CloudFormation. Then it will be guaranteed that the bucket actually exists before it is used downstream.
If you don't care about that and just want the actual bucket name in the cdk app code then put the value in the cdk context json file and use node.try_get_context to retrieve it wherever.
There is a handy method called fromBucketName you can use if it wasn't defined in your current app:
const bucket = aws_s3.Bucket.fromBucketName(this, 'bucketLabel", "nameYouGaveBucket")
Otherwise, I believe you are looking for bucket.bucketName (typescript) or bucket.bucket_name (python).
See typescript docs python docs. This is also available in the CDK wrappers in other languages.
Note that there are similar methods for all sorts of CDK constructs, so you should refer often to the API docs, as there is lots like this you can find easily there.
So I am using quick sight java SDK for integrating s3 with quicksight, and for that, I am using CreateDataSource.so while using this method, I have to pass one parameter in this method DataSourceId. The description of this parameter in the AWS documentation is like "An ID for the data source. This ID is unique per AWS Region for each AWS account," and I do not know how to get this.i to have to get this parameter programmatically. Type of this parameter is String
getClient().createDataSource(new CreateDataSourceRequest().withDataSourceId("DataSourceID").withAwsAccountId("AWS ACCOUNT").withName("display name of data soure").withType(DataSourceType.S3));
getClient is a client for using quicksight API
and yeah for the integration of s3 and quicksight I can not do this through AWS console I have to do this programmatically
I do not know how to get this.i to have to get this parameter programmatically
This is something that you set yourself, for example:
DataSourceId = "my-first-test-data-source-id"
For my app I need users to be able to upload their profile pictures.
The way it works is they send their info (name, email...) and their pictures to a lambda function. The Lambda function stores the pictures in S3 and stores the info and the link to the picture in S3 in DynamoDB.
Users should be able to upload a new picture and use it as their profile picture. Would it be possible to upload a picture that would use the same link in S3 (meaning I would replace the old picture by the new one while keeping the link the same)?
This way I don't have to update any table in dynamoDB. The thing is that I need to use the link in other tables and this would avoid having to update every tables it is in.
To replace the file upload it again with the same key. e.g.
aws s3 cp ./hello1.text s3://document/hello.text
May receive old data until replication is completed. refer - https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#BasicsKeys
I upload folders/files by:
aws s3 cp files s3://my_bucket/
aws s3 cp folder s3://my_bucket/ --recursive
Is there a way to return/rollback to previous version?
Like git revert or something similar?
Here the is test file that I uploaded 4 times.
How to get to previous version (make it the "Latest version")
For example make this "Jan 17, 2018 12:48:13" or "Jan 17, 2018 12:24:30"
to become the "Latest version" not in gui but by using command line?
Here is how to get that done:
If you are using cli,
https://docs.aws.amazon.com/cli/latest/reference/s3api/get-object.html
Get the object with the version you want.
Then perform a put object for the downloaded object.
https://docs.aws.amazon.com/cli/latest/reference/s3api/put-object.html
Your old S3 object will be the latest object now.
AWS S3 object is immutable and you can only put and delete. Rename is GET and PUT of the same object with a different name.
Hope it helps.
No. However, to protect against this in the future, you can enable versioning on your bucket and even configure the bucket to prevent automatic overwrites and deletes.
To enable versioning on your bucket, visit the Properties tab in the bucket and turn it on. After you have done so, copies or versions of each item within the bucket will contain version meta data and you will be able to retrieve older versions of the objects you have uploaded.
Once you have enabled versioning, you will not be able to turn it off.
EDIT (Updating my answer for your updated question):
You can't version your objects in this fashion. You are providing each object a unique Key, so S3 is treating it as a new object. You are going to need to use the same Key for each object PUTS to use versioning correctly. The only way to get this to work would be to GETS all of the objects from the bucket and find the most current date in the Key programmatically.
EDIT 2:
https://docs.aws.amazon.com/AmazonS3/latest/dev/RestoringPreviousVersions.html
To restore previous versions you can:
One of the value propositions of versioning is the ability to retrieve
previous versions of an object. There are two approaches to doing so:
Copy a previous version of the object into the same bucket The copied
object becomes the current version of that object and all object
versions are preserved.
Permanently delete the current version of the object When you delete
the current object version, you, in effect, turn the previous version
into the current version of that object.
I wasn't able to get answer I was looking to get for this question. I figured out myself by going to aws s3 console and would like to share here.
So, the quickest way is to simply navigate to:
--AWS Console -> to s3 console -> the bucket -> the s3 object
You will see the following:
At this point you can simpyl navigate to all your object
versions by clicking at the "Versions" and pick (download or move)
whichever version of the object you are interested in
S3 allows you to enable versioning for your bucket. If you have versioning on, you should be able to find previous versions back. If not, you are out of luck.
See the following page for more information: https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html
I need to move some files (thousands) to Amazon S3 bucket, from where they will be displayed to the end-user by another application (instead of the current one).
Problem is, that these files have creation/upload date now (dates very between 2012 and 2017, when they were uploaded to current application), and when I move them they all start to be of the same date. That is a problem because when you look at the files in the new application, you don't understand the time hierarchy which is sometimes very important.
Is there any way I can modify upload date of a file(s) in S3?
The Last Modification Date is generated by Amazon S3 and cannot be set via the API.
If dates and other information (eg user) are important to your application, you can store it as metadata on the object. Then, retrieve the metadata when displaying dates, user, etc.
What I did was renaming the file to something else and then renaming it again to its original name.
As you cannot rename directly, you have to copy the file to a new name, and then copy it back to its original name. (and delete the auxiliary file, of course)
It is not optimal, but that's the solution when using AWS client. I hope one day AWS will have all function the FTP used to have.
You can just copy over the same object and the timestamp will update.
This technique is also used to prolong the expire of an object in a bucket with a lifecycle rule.