Get S3 bucket size from aws GO SDK - amazon-web-services

Question

I am using Amazon S3 bucket to upload files (using GO SDK). I have a requirement to charge client when their directory size exceeds 2GB.
The directory hierarchy inside bucket is like: /BUCKET/uploads/CLIENTID/yyyy/mm/dd
For this, I have searched a lot about it. But could not find anything.
How can I get the directory size inside a bucket using SDK ?

Answer 1

First of all, /uploads/CLIENTID/yyyy/mm/dd is not a directory in S3 bucket, but a prefix. The S3 management UI in AWS Console may trick you to think a bucket has subdirectories, just like your computer file system, but they are prefixes.
Your question really is: How can I get the total size of all objects inside a bucket, with a given prefix?
Hope this code snippet can clear your doubts.
package main
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
// iterate all objects in a given S3 bucket and prefix, sum up objects' total size in bytes
// use: size, err := S3ObjectsSize("example-bucket-name", "/a/b/c")
func S3ObjectsSize(bucket string, prefix string, s3client S3Client) (int64, error) {
output, err := s3client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
Bucket: aws.String(bucket),
Prefix: aws.String(prefix),
})
if err != nil {
return -1, fmt.Errorf("cannot ListObjectsV2 in %s/%s: %s", bucket, prefix, err.Error())
}
var size int64
for _, object := range output.Contents {
size += object.Size
}
return size, nil
}
// stub of s3.Client for dependency injection
type S3Client interface {
ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error)
}