With S3 replication, if a previously replicated file is deleted in the destination bucket, is the default behaviour that the file will be re-copied? I assume this is the case and if so, is there any way to change this behaviour so files are only ever replicated once?
if a previously replicated file is deleted in the destination bucket, is the default behaviour that the file will be re-copied
NO, it wont be recopied because according to docs "By default, replication only supports copying new Amazon S3 objects after it is enabled."
So literally s3 sees it as an existing object which is already replicated to destination no matter whether it is deleted in destination or not, it won't replicate AGAIN!
Related
I want to use the AWS S3 sync command to sync a large bucket with another bucket.
I found this answer that say that the files from the bucket synced over the AWS backbone and are not copied to the local machine but I can't find a reference anywhere in the documentation. Does anyone has a proof for this behavior? any formal documentation that explains how it works?
I tried to find something in the documentation but nothing there.
To learn more about the sync command, check CLI docs. You can directly refer to the section named -
Sync from S3 bucket to another S3 bucket
The following sync command syncs objects to a specified bucket and
prefix from objects in another specified bucket and prefix by copying
s3 objects. An s3 object will require copying if one of the following
conditions is true:
The s3 object does not exist in the specified bucket and prefix
destination.
The sizes of the two s3 objects differ.
The last modified time of the source is newer than the last modified time of the destination.
Use the S3 replication capability if you only want to replicate the data that moves from bucket1 to bucket2.
Long time listener, first time caller...
If I have an S3 bucket which is versioned (as per the documentation for replication) and bucket replication is enabled; what happens if the source bucket object is deleted before replication has taken place?
I'm assuming (as the source bucket is versioned) the replication will still take place. The replication logic surely sees a new object creation event and replicates this in the destination bucket. Because I've not specified any other configuration, the delete marker subsequently placed on the source bucket object will not be replicated onto the destination object. Therefore it should just appear as normal in the destination bucket and not be visible in the source bucket.
I can't find anything concrete in the documentation that clarifies the position on this situation.
It depends on which option you choose in your replication configuration:
V1: the delete marker is replicated - a subsequent GET request to the deleted object does not return the object in either the source or the destination bucket.
V2: the delete marker is not replicated - a subsequent GET request to the deleted object returns the object only in the destination bucket.
For more, see Managing delete marker replication in Amazon S3.
Unfortunately, this morning I accidentally deleted a number of images from my S3 account, and I need to restore them. I have read about versioning, however this was not enabled on the bucket at the time of deletion (I have now enabled).
Is there any way of restoring these files either manually, or via Amazon directly?
Thanks,
Pete
Unfortunately, I don't think you can. Here is what AWS says in their docs -
To be able to undelete a deleted object, you must have had versioning
enabled on the bucket that contains the object before the object was
deleted.
I am trying to copy S3 bucket's objects across region. My source bucket is having version enabled and I need my destination bucket should retain all the versions present in my source bucket(though I think we can't preserve the actual timestamp while copying). I would prefer to have some officially supported tool.
Amazon S3 Cross-Region Replication (CRR) will do this for you. In fact, it requires versioning to be activated.
However, it will only take effect after it is activated. Any existing objects and versions will not be replicated.
A while ago when the price difference between standard storage and glacier was closer to 10:1 than 3:1, I moved a couple of bucket completely to Glacier using a life-cycle policy. Admittedly, I hadn't investigated how to reverse that process permanently.
I know the documentation states that I would have to "use the copy operation to overwrite the object as a Standard or RRS object", but I guess I'm unclear what that looks like. Do I just copy and paste within that bucket?
Restoration can only be done at the Object level, not Bucket or Folder. When an object is restored, it is a available in Amazon S3 for the period requested (eg 2 days), then reverts to only being in Glacier. You'll need to copy the objects out of the bucket during that time to keep a copy of them in S3.
The easiest method would be to use a utility to restore multiple Objects for you, eg:
How to restore whole directories from Glacier?
Also, see Stackoverflow: How to restore folders (or entire buckets) to Amazon S3 from Glacier?