I'm reading objects from Amazon S3 using the GetObject method from AWS SDK go v2
input := &s3.GetObjectInput{
Bucket: aws.String(w.Bucket),
Key: key,
}
object, _ := w.Client.GetObject(ctx, input)
return object
I have access to the object's content size, and to the file type, and there is a parameter
Object.Body
that should have the file content.. But I can't seem to find a way to access it.
the Body is of type io.ReadCloser
Add
import "io/ioutil"
Then
bodyInBytes, err := ioutil.ReadAll(object.Body)
If you are using go after 1.16 then io.ReadAll is preferred, import "io"
You mention reading JSON in the comments. To read JSON, make a struct that matches the structure of your JSON document (use an online converter like https://mholt.github.io/json-to-go/ with a sample) then add import "encoding/json" and:
data := mystruct{}
err := json.Unmarshal(bodyInBytes, &data)
Related
I am trying to write an example of how to use a web identity token with a container to perform EC2 operations.
The container spec contains the service account and has the necessary permission to access the token path and its namespace is a trusted entity in the role.
package main
import (
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/sts"
)
func main() {
sess, _ := session.NewSession()
config := aws.NewConfig().WithRegion("us-east-1")
stsSTS := sts.New(sess)
roleARN := "arn:aws:iam::1234567:role/s2-p0o5-csi-drivers-ebs-cloud-credentials"
roleProvider := stscreds.NewWebIdentityRoleProviderWithOptions(stsSTS, roleARN, "gosession", stscreds.FetchTokenPath("/build/token"))
creds := credentials.NewCredentials(roleProvider)
credValue, _ := roleProvider.Retrieve()
fmt.Printf("credValue.AccessKeyID: %v\n", credValue.AccessKeyID)
fmt.Printf("credValue.SecretAccessKey: %v\n", credValue.SecretAccessKey)
fmt.Printf("credValue.SessionToken: %v\n", credValue.SessionToken)
config = config.WithCredentials(creds)
nodeID := "i-00843f27cfeb0beff"
svc := ec2.New(sess, config)
request := &ec2.DescribeInstancesInput{
InstanceIds: []*string{&nodeID},
}
result, _ := svc.DescribeInstances(request)
fmt.Printf("result: %v\n", result)
}
The value of result get empty. Whereas, i have exported (credValue.AccessKeyID, credValue.SecretAccessKey,credValue.SessionToken) as environment variables and aws cli is giving me output related to describing the instance.
I tried various methods like credentials.NewStaticCredentials() with the credential information, but no luck. Can some help share hint on what is going wrong and correct way of doing it.
The instance-id used should be avaiable in the region, else it returns an empty map with err == nil.
I am trying to upload base64 encoded string to aws s3 bucket in the form of PNG image. I had tried following code:
import (
awsconfig "github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/feature/s3/manager"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
signature:= "iVBORw0KGgoAAAANSUhEUgAAAfQAAAC0CAYAAABi+d5SAAAAAXNSR0IArs4c6QAAHANJREFUeF7tnWusdkdZhm+kVUSxYg+2klirFWMrrdD+wPKjEg+QCtQayw8PqYApQUPaRhJDhBQUEyJqDyHGeihtQI1oLI02WklBYrWgrWBaRKxQEKgoLWjVWA5ac/utRd9v9333embeWWvNrHVNsrP39+1nZs1cM+++1zwz88zjRIIABCAAAQhAoHkCj2u+BTQAAhCAAAQgAAEh6AwCCEAAAhCAwAIIIOgL6ESaAAEIQAACEEDQGQMQgAAEIACBBRBA0BfQiTQBAhCAAAQggKAzBiAAAQhAAAILIICgL6ATaQIEIAABCEAAQWcMQAACEIAABBZAAEFfQCfSBAhAAAIQgACCzhiAAAQgAAEILIAAgr6ATqQJEIAABCAAAQSdMQABCEAAAhBYAAEEvb1O/HZJx0n6zq7q/vdXdz/7u7++YaNZ/ybJX04Hvx9svfMeL+mzkh6W9MABgz+T9GFJ90u6rT101BgCEIDAcgkg6PX2rcX17E64LdoWaX+vIT0i6c8lvUbSO2uoEHWAAAQgsHYCCHo9I+D8TrA98+4FvJ7a7a7J6yS9uoWKUkcIQAACSyaAoM/Tu559P0fSj0k6uaKZdy6NmyT9QG5m8kEAAhCAwP4EEPT9GUZK2HSff/8CBHxbm6+RdHkEBjYQgAAEIFCeAIJenmlfol3oFu/ehT7ek+op+XRJH6qnOtQEAhCAwHoIIOhl+9pr35d0rvR+53nZJ9Rd2lsk/WjdVaR2EIAABJZJAEHfv18t3BZxu5s3j4vtX3J7JTzUHalrr+bUGAIQgEDjBBD0/A60kF/WCfkaZ+O7yPko22vzsZITAhCAAARyCCDoOdSOzMgtXGufkW+jd4ek8/KwkgsCEIAABHIJIOhp5Czgb9qI0paWu4z1u7qIb+/b+O6SHQXO/xdNm9HlNvO4jR+V9BFJZ0g6qfvlyyVdFCz8BEkPBm0xgwAEIACBAgQQ9DhEu9c9K5/SvW7xtkg75KoFNkWw4y2LWZ4j6c6YqZ7d1TlojhkEIAABCOxLAEEfJmgBd+CUPnb6cI48C8dOv7UTQgv4nOK9qwXXSvJMfSg5YM6NQ0b8HgIQgAAEyhFA0A9n6XPkdrGPMSu3W9vCfY+kd0u6vVy3jlaSPRRXBkr3pjjbkiAAAQhAYCICCPp20BZwC1fJyGf/LultGzNwu9BbSwh6az1GfSEAgdUQQNAf29UODuNZeYmbzTwLt4jfUKkLPXWgI+ipxLCHAAQgMBEBBP1o0F4jfpmkY/bkf7Okqxe4MeyqoNfiiq79e2IkOwQgAAEIRAkg6EdI2cXuTVwviILbYecyPItt0Z0eabrvPo9sDmSXe4QmNhCAAAQKEkDQjwSH8S72fVzsPl7m9fYad6YXHC56b5DTWZLuLvlgyoIABCAAgcMJrF3QPdu0mOfuYvdGN8/I7V5fQ7pf0imBhq59XAUQYQIBCECgLIE1/+H1WWlvfstNf9tdj7pU9/o2Lo8EYH1qI7pcwBwTCEAAAhAoQWCtgr6vmK/1nHVE0D0u1zquSnwmKQMCEIBAFoE1/uH1Wrd3a+cku9gdbMYBYdaYIoLOFaprHBm0GQIQmJ3A2gR9n5n52yW9sLsEZfaOm6kCX5D0+IFn/6ekJ81UPx4LAQhAYLUE1iTo3gDnY1c56Xck/VBOxoXliczQ/1vSExfWbpoDAQhAoHoCaxF072K/L2M3+yckPW8Fx9GiAzUi6C5rLeMqyg07CEAAAqMTWMsfXodfvTCRpnexe1bve8ZJRwgg6IwECEAAApUSWIOg56ybO+Kb85GOJhANLHPagqPlMSYgAAEIVElg6YLuKHAWoZTAMYj57qEaDf16rqS7qhzxVAoCEIDAQgksXdBTXe2I+eED3YF4Ip6L50v6o4V+ZmgWBCAAgSoJLFnQfV7cYV2jCTEfJnWdpEuHzf4/HK6D75AgAAEIQGAiAksVdLvY7Wq3yz2S2AAXoXQkbv2VAdO1RtILoMEEAhCAwDgElirovizlsiAyR3/zbval35QWxHGoGYJegiJlQAACEBiBwBIF3degenYeTcwmo6SOuNvtdh9KMB0ixO8hAAEIFCawREH3TPvsICe72ve5Bz34mMWYMUNfTFfSEAhAYGkEliboUcHp+/HpuNqThvQ7JD07kMM2a73AJoAHEwhAAALlCSxJ0C+QdEsComsk+eY1UpyA7zo/IWCOoAcgYQIBCECgJIElCbpDtB4XhPPRztVOWNcgsM7sc5KODWSx6D8YsMMEAhCAAAQKEViKoKfsajc6ZpDpA8hHAH3BzVD6rKQnDBnxewhAAAIQKEtgCYKeei3qzZIcdIaURiAaqOceSU9LKxprCEAAAhDYl0Drgp56LarPnHumias9feRENxxyZC2dLTkgAAEI7E2gdUGPXhbSg7pIkuO7k9IJROPis5yRzpYcEIAABPYm0LKgR2eMPaQPSDpjb2LrLeAjkk4NNJ+rUwOQMIEABCBQmkCrgp66bv4ZSd+Iq32v4fNIILeXNFKuqg0UiQkEIAABCEQItCjoqevm5oAbODIadttEX6De1cXF3+9p5IYABCAAgWQCLQq647SnhGtlk1bysHhMBt+B7rvQhxLBeoYI8XsIQAACIxFoTdAtKhaXaCJWe5TU4XbRc/5XSLItCQIQgAAEJibQkqBHZ4k9Qq/neibvzVyk/Qg4Lvv5gSJY2ghAwgQCEIDAGARaEXQLs4+opWy4epGkG8aAtsIyozvcn8zGwxWODpoMAQhUQaAFQbeIe93cAWGi6cZE13y03DXamb9PCQwldrgPEeL3EIAABEYk0IKgpwaPYd287IBZ0g53vxTuOkvvFxd/9VEEt0UT9KU+LOGUHV+UBgEIFCJQu6CnBo9h3bzQwNgoJrp3oeQO915c/f3gDXr+Pwvz5vKLf+7j81uI/dXbpCzTpNLzcx6W9HlJj5f0QUm+YvZEST7C90/d122pBWMPAQhAIJVAzYLudXO72lMSm7JSaMVsoy9Vu3a495vp3J+9uG7+vE2gYzVrw8oBee6V9BOSEPY2+oxaQqBJArUKuv/I29Wect6cI1PjDMHoDvc3SvrCRp/ZVU96lIDZ/IKknwEKBCAAgTEI1Cro0XPPPRM2wY0xOo6UabfyQbf3eE9bfskvkXT98ptJCyEAgakJ1Cjoqa52b4LzbJArUcuMHrvIz5X0PEmnSPqWMsVSygYBb8zz+joJAhCAQDECNQp6yq52b4KzmL+vGJF1FNSva5udlzf8ErW5rr0OCvO10ssTL5/v8TwZAhBYIoHaBD26o7rvC4LHbB+VFme7yXuRPvh9iWO5tTZ9TfB8f2vtor4QgMBMBGoT9PsSAsisfd3cs2wfzdr8YpY90wcp47Fe1rgrIx9ZIAABCGwlUJOgp8zO7Wq3kK1h3dwu8bO7pYXeNZ4SNY+hXyeBX5P00jqrRq0gAIEWCdQk6CnXol4k6W0tAg/U2WJ9Yecu9xo34h2AdoiJX/784rctwpv/f9f+i557H+Rm8xH9ccp9dv/7bPpT92sauSEAAQg8SqAWQXeUr5uCHeMIXEs742wBd5vMYe0CbgG2+B70vvj/Dorywf/7tKSHurxTe29Ok3S8pKclHEur5fMX/OhhBgEI1Eyglj8onm1b1CJpKdHgPMu7rBPxMcOTRpiOZdPHPt+cCW+KcD9znlp8x2qvyz1D0vsDD7hf0lMCdphAAAIQCBGoQdCjt3m5Qa3Pzpco4vdIelCSI8r1Yr1tNh0akAswil5m8x5Jz1xAe2kCBCBQCYEaBP1ySVcFebQ4O/cLyyXdda4poWyDSGY1a/0Fawx4b5V0caDgX5L0ioAdJhCAAARCBGoQ9OhmuNauRfVMrRfyUGdUYOT1a8+4nxWsC/HzHwvK6/hPDvB7oaTfC9hhAgEIQCBEYG5BTwnz2koQGYu4vQ61zsYt2t7Z3bvF/bPXsO0yd0rxmHgjGPeDH/1R81WqxwQ+fSd0SxUBU0wgAAEIDBOYW9BTxMOznlo3T3ln+hskfU8lF5nYFe5kke43pB12RGtzpFjgfe59KLXmMRlqT4nfnyPpzkBBvnnt2IAdJhCAAATCBOYW9GhkuFqjwtmt7rXQZ4SJlzO0aPez7H523X/PfcoSPSa5LHLyOT77tYGMd0g6L2CHCQQgAIEwgTkFPWV3e22BZCzkV054Ht6zYYu1v3p3ebiTEwxv6Nb9I1lq9phE6j+GzZu6zY9DZb9S0uuHjPg9BCAAgRQCcwp61N3us8y1BFtxPbwj3wFgxkxus8Xb5/N7t/mYz3PZfsGyxyRyJr5Wj8nYjIbKj3qcniPpT4cK4/cQgAAEUgjMKejRa1JrEQ+/gHhWHhG8lD7obS3iFnDPkue4DvY1Xfsida/NYxKp89g20YAyrsfJkv5l7ApRPgQgsC4Ccwr6/0j6kgDuucXDAu6wtGOEm+1F/OoKdos/0IUuHeqSmjwmQ3Wd8vc/GDyG9o+SvnnKivEsCEBgHQTmEvSUzVdz1dEjwPW0J6H0rNxeB8/E993EVmqU/rikXw8Wxtnz7aB+V5LPlg8lblkbIsTvIQCBLAJziWX0qlQHOfFlF3Mkz8g9My8l5t7Y5pm43eq1Hb+7W9K3BSD7DLtfcjh7/lhY/ysp8nn6EUm/FWCNCQQgAIEkApE/QEkFBo0tbL6YZCi9WtLrhoxG+H2pmbkF0ALu9s6xLh5Bk+ItqWU/Q6RdU9o8V9IfBx/49ZI+FrTFDAIQgECYwFyCblfz+QO1/Jykr51hNpuy23tXExw4xOfTf7WB2WzKUbWnV/xiEh70Ixj+Q3Bd3OPe9xGQIAABCBQnMJegPxJoyc0THA/bVo1opLRteT/RzcZ/MdC+GkxSYgF8UtIpNVS6sjpEg8m42hckzOQraybVgQAEaicwh6D7LLfP6w6lOXa3pxzd2qy/18ed1+71llJ06cNtYu13e8+6788Kdvocn7dg1TCDAARaJzDHH5jofdFTRyJLWUvu+91r5BZyC2NrKWV2zlG17b3rl5w3Bzsed3sQFGYQgEAegTkEPRIhbo6LP1Jd7a6jI8a1uuM7ZXbOUbXtn6+/k/StwY+e185rOaYYrDJmEIBASwTmEPSIW3tqAYnU6aCL3Z6G2o6fRcdeyuzcXggvk7Ta1iiTVLuop8nlMjtPpYs9BCCQTGAOQfc684UDNZ1yN7XF6r0J5809M29ZzI0+ZXb+2m5ZIXlwLTzDrZK+N9hGZudBUJhBAAL5BOYQ9MiRtSnrFXnB6AkvYbaa+gIz9V6G/NE8Xc5Lukh/kSf+a3f8MmKbYvOVks6V9BWSHHb2uyR5fH55F1K5v9DIS0K+e92hln26ZNeGVJfzdQG7zTra03OCJB/T9Gd2s+yTJDm+/Wb6+BZbvxz7b4JD4vqUyB9IcqAjEgQgkEhgSuHsqza0Vu17vseIm74NjdfAHQ0umubYeR+tW9Qu8kLVl3WNJO95IB1NIHqrmnO9KEH8t3H+UklPkXRq5xF4pqQTg5H9Wuy3/+iuoLWwkyAAgQQCcwi612KPO6SOU7l4Pbuwqz16Netc5+ITunPQNPUFhtn5Y5FeJ+nSQdJHDLxp7sygbW/m0xbfJ+n53ezb+ef4nCZWu6i5PQmOi//7RUulMAgsnMAcfyiGgspMtSEuZR15CTHMUyPgTfVi1dJH7BWS3pBQ4V1r5+6Ls7uXSb9QWsT77wnFL97UkSK9ZEGCAAQCBGoU9Ck2EDli1y0BPr3JVC8ZCVVKNh1a6tgscAl7BZIBDWQ4R9KdCYV6b4aXaJy8hORQxxbuXrwTilqtqe9x8H0OJAhAIEBgrYI+5PbfRDfHmfhA1yWZpMRrd8H7rvsmVa4R47/uNqFFq+tY/n4JmGo/SLRerdnZm+EXTBIEIDBAoEZBH3vdNvXM+ZRH6MYYsNGravtnL+EFpjRHR4NzVDjS9AQcVpdd79Nz54kNEqhR0MesU+qRrdZ3eadugvMQbv0FpvTH8FWSfq50oZQXJvDDkn47bI0hBFZMYEzx3IV1yN09Zp1SXM//3J2jbTVCWs6d7myEO3rUet2bcK3z/oFk+Wde/jy9IQJjiucuDEPnoMeqU/SWt77eLZ85zxFzXO1Hj9jjJb0j4Sa1hj72TVXVY9ljkwQBCAwQGEs8D3vsXIKeMjv/kKTTGx09qefr3cwlHMsr2V0O3nJHyQJHKstBmOxBcl19I957uudsXhhkL4OTj399oPM6OTCNA7g8aUu9bPdAF7zGdruS8z/c2d3f1cP2fR7//m8OZHZdbHuapD8JMnF5rg8JAhCoUNCHQq2O8ZKRchmJ/0j5Bq0Wb1FzO9/ZHY1KGfwteyNS2hmx9YvcvRHDCW08Ji3Wfhn+cBci9bYJn1/6UdGNmn8p6VmlH055EFgqgTHEc4jV0C5zv72XFtPoHxDXveV15KGXpW1903J7h8Za6u8de/w3JH1HasaC9nYve/w7bsAHJX1M0u0Fy6+hKEfac8S9oeSXGHtLSBCAQIBAjYI+RmCZqNC1HFAlZUmhHxo3dnGzA0Nl8SYWDs8Ip/hMeJxZsP3Vi7e/l36RrbXTvHP9LYHK/ZckX0JDggAEAgSm+ON1sBpDR6nGEPShnfV9HVudraaEse3b6vVX90Wru/gDwzvJZCgkcVJhB4wt4HaX918W8jWnp3behwgDe/T8uSRBAAIDBOYQdEfO8jrvrlRa0L1L1pewRNLYQW0idUi1eb2kn07MtIQ73RObfKj5X0g6r2SBkj4v6Vc6EbeHiHQ0gc9I8p6PSPKmOF+O5AA/Hru8hEaoYbM6AnMI+pDAlp4lR9fPW7xNbejlaNuA9m5o9wF/FB+lU3J27stbHNnM4kPaTcCcfNlNTvLYPbhk0S9j5JRHHggsgsAcgm5wh/0BLS3oQ5vw+o4s/dyxB0jOWXP/0fNLwNpdvpt984Ju9rdPf/klyXsYvKHu4/sUtKK8Hr9egjjsKuVcHCX3IzxB0smS/N2eAn/5hWLzhfjgv3PrnZJv07vR/7zt/6Jl9u08pmvb2o4K+m+ilyGb9qatQdCHzr33A760qz/6Qcqx8x/DmxLucvczEPPtpKMbtHb1k18E/VVylp8zJlrMk7P3o8V2Uud2CGzekthOrbuaziXofoM+dQet0vHTD3vWZhXGOC43xoDIOWtuMffSQ9Nvn2PAlJSyQWuzCl6isSARGja/Y07qThZ8U34R5IRAcQKteWu/CGAuQT9s1lx6LTs6c5qLRcpozBFzl0887MMp35fo7bCb/g9TOg7bnQR+UtIb4QOBygi0MsE7CttcInaYoHsdo9Qd0tFNYyWfOda4zBXz6yW9ZKxKLaTcSJz/uyS9XdIrF9Lmmprxy5KuqKlC1GX1BJqMnrl0QY/ucK89wEqumN8q6bmr/2jGAHxZF1/cUeL8c598TMr7FX6WdfIYyEyrt0q6ODMv2SBQmkCTXs2lC/oSdrjn7Gb34G7B61D6Q1iqPLvbvMeD9fFSRGPlvFjST3UXyMRyYAWBcQi0tEn6iwSWLuh2kz4j0N81dp5n5ZdJ8ktJauIq1FRi2NdC4ExJ/rJHxBsW5/obVQsP6jE9gWb/fs71YZlqDd3RunyucijVFiHOSwVXJUTS2mxfy/Hoh/qJ36+PgPc32EvlL++J8b93nZBZHx1aXJpA08d75xL0wy5LKeUqjm6Ie1DSCaVHRWZ5rvOVe2wKbHowZjIj2/oI2Htlgbe4+6vfRIvYr28slGyxtceTqWYvSVqyoFsYI+7qv+/uPy85MFLLOqu7TnKfqyIR81Tq2C+ZgAU/Git+8w+4o771+fpocJuR2CJl2qb/6hlH8u3qj74efUS6fcM2b8vv//MLUc/CP/fP69tyMJ///zBec46vvl6b3w/Wp2+P27qI/TJLFnTvTPZtYkPp2m6teshujN/7A+EXD5/FPXaPByDme8AjKwQgAIElEJhL0KdYQ4/e5vTdkm6boTPtPfCmt33e3F1txHyGzuOREIAABGojsGRBrzFCnMX7km4pYF8h91jybkyv+XDZSm2fLOoDAQhAYGICNQp6qSAvNQm61/M8G/cSQAkh9zBxzHvP8vddT5t4yPE4CEAAAhAYgwCCPt45V4v4+d0M2j+XSr6q83IuWimFk3IgAAEILIPAXIJ+2LG1Ujfd3Cvp9EA3vVuSw33umzzztoB7Ft6fl923zIP5mZWXJkp5EIAABBZCYC5B33VG3Bu8PJstcQ7wOkmXBvsph0Mv4G6Lv0rOwg9W2+cjPStnrTzYoZhBAAIQWBuBHCErxcgi6DVgC6HXgS1W/ncp0YoGlnF7/Ezf9jR0FtEz+Zd1F56cWArEIeV8urspjXvMJ4DNIyAAAQi0TGBOQZ+CW+o913dL+pSkn5f0ye7nM7oLIyzmU0WUe1jSb0p6FZvephgmPAMCEIBA+wSWLui3SLqgoW7yksPV3Re71xvqOKoKAQhAYG4CSxd0h1T1jWuRC1rm7AsL+Q2dkJfYPzBnW3g2BCAAAQjMQGDpgm6kvqj++hnYRh6JkEcoYQMBCEAAAoME1iDohvB+SV4LryX1rnXPypmR19Ir1AMCEIBAwwTWIuhnSrqngn66uQsI413rrJFX0CFUAQIQgMBSCKxF0N1fc4i6Y637KFz/hYgv5ZNDOyAAAQhURmBNgm70Pnrm8+YXj9QPDstq8fYM3N8R8JFAUywEIAABCBxNYG2C3rfeN5T51jMHn9knPdTdeOZNdxZw1sP3oUleCEAAAhDIJrBWQe+BOTSs472/VNJXBSh6xu0wrN7M5p+HIssFisQEAhCAAAQgsD+BtQv6JsEXS3I411MknS3pJElP7GbdFvG/knS7JM/KSRCAAAQgAIGqCCDoVXUHlYEABCAAAQjkEUDQ87iRCwIQgAAEIFAVAQS9qu6gMhCAAAQgAIE8Agh6HjdyQQACEIAABKoigKBX1R1UBgIQgAAEIJBHAEHP40YuCEAAAhCAQFUEEPSquoPKQAACEIAABPIIIOh53MgFAQhAAAIQqIoAgl5Vd1AZCEAAAhCAQB4BBD2PG7kgAAEIQAACVRFA0KvqDioDAQhAAAIQyCOAoOdxIxcEIAABCECgKgIIelXdQWUgAAEIQAACeQQQ9Dxu5IIABCAAAQhURQBBr6o7qAwEIAABCEAgjwCCnseNXBCAAAQgAIGqCCDoVXUHlYEABCAAAQjkEUDQ87iRCwIQgAAEIFAVAQS9qu6gMhCAAAQgAIE8Agh6HjdyQQACEIAABKoigKBX1R1UBgIQgAAEIJBHAEHP40YuCEAAAhCAQFUEEPSquoPKQAACEIAABPIIIOh53MgFAQhAAAIQqIoAgl5Vd1AZCEAAAhCAQB4BBD2PG7kgAAEIQAACVRFA0KvqDioDAQhAAAIQyCOAoOdxIxcEIAABCECgKgIIelXdQWUgAAEIQAACeQQQ9Dxu5IIABCAAAQhURQBBr6o7qAwEIAABCEAgjwCCnseNXBCAAAQgAIGqCCDoVXUHlYEABCAAAQjkEUDQ87iRCwIQgAAEIFAVAQS9qu6gMhCAAAQgAIE8Agh6HjdyQQACEIAABKoigKBX1R1UBgIQgAAEIJBH4P8AbePq4jPqwCcAAAAASUVORK5CYII="
decodedSignature, decodeErr := base64.StdEncoding.DecodeString(signature)
if decodeErr != nil {
// handle error here
}
r := bytes.NewReader(decodedSignature)
bucketName := "test"
filepath := "uploads"
contentType := "image/png"
cfg, err := awsconfig.LoadDefaultConfig(context.TODO(),
awsconfig.WithRegion(config.AwsRegion),
)
client := s3.NewFromConfig(cfg)
// Create an uploader with the client and options
uploader := manager.NewUploader(client)
cacheExpireTime := time.Now().AddDate(1, 0, 0)
cacheMaxAgeSecs := "31556926" // seconds in 1 year
// Upload the file to S3.
uploadResp, err := uploader.Upload(context.TODO(), &s3.PutObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(filePath),
Body: r,
ContentType: aws.String(contentType),
Expires: &cacheExpireTime,
CacheControl: aws.String(cacheMaxAgeSecs),
})
But when I try to load the s3 url on which the image is uploaded, then there is black screen as shown in the screenshot:
The Body parameter in the uploader.Upload function accepts io.Reader objects as body, meaning that you either need to use os.Open to open a png you have saved, or in this case, use bytes.NewReader(data) to make your data conform to the io.Reader interface (have the .Read() method).
I am using Amazon S3 bucket to upload files (using GO SDK). I have a requirement to charge client when their directory size exceeds 2GB.
The directory hierarchy inside bucket is like: /BUCKET/uploads/CLIENTID/yyyy/mm/dd
For this, I have searched a lot about it. But could not find anything.
How can I get the directory size inside a bucket using SDK ?
First of all, /uploads/CLIENTID/yyyy/mm/dd is not a directory in S3 bucket, but a prefix. The S3 management UI in AWS Console may trick you to think a bucket has subdirectories, just like your computer file system, but they are prefixes.
Your question really is: How can I get the total size of all objects inside a bucket, with a given prefix?
Hope this code snippet can clear your doubts.
package main
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
// iterate all objects in a given S3 bucket and prefix, sum up objects' total size in bytes
// use: size, err := S3ObjectsSize("example-bucket-name", "/a/b/c")
func S3ObjectsSize(bucket string, prefix string, s3client S3Client) (int64, error) {
output, err := s3client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
Bucket: aws.String(bucket),
Prefix: aws.String(prefix),
})
if err != nil {
return -1, fmt.Errorf("cannot ListObjectsV2 in %s/%s: %s", bucket, prefix, err.Error())
}
var size int64
for _, object := range output.Contents {
size += object.Size
}
return size, nil
}
// stub of s3.Client for dependency injection
type S3Client interface {
ListObjectsV2(ctx context.Context, params *s3.ListObjectsV2Input, optFns ...func(*s3.Options)) (*s3.ListObjectsV2Output, error)
}
I'm looking to integrade an S3 bucket with an API im developing, I'm running into this error wherever I go -
SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
status code: 403
I have done the following
Installed SDK & AWS CLI, and AWS configured
Double(triple) checked spelling of key & secret key & bucket permissions
Attempted with credentials document, .env, and even hard coding the values directly
Tested with AWS CLI (THIS WORKS), so I believe I can rule out permissions, keys, as a whole.
I'm testing by trying to list buckets, here is the code taken directly from the AWS documentation-
sess := session.Must(session.NewSessionWithOptions(session.Options{ <--- DEBUGGER SET HERE
SharedConfigState: session.SharedConfigEnable,
}))
svc := s3.New(sess)
result, err := svc.ListBuckets(nil)
if err != nil { exitErrorf("Unable to list buckets, %v", err) }
for _, b := range result.Buckets {
fmt.Printf("* %s created on %s\n", aws.StringValue(b.Name), aws.TimeValue(b.CreationDate))
}
Using debugger, I can see the sessions config files as the program runs, the issue is potentially here
config -
-> credentials
-> creds
-> v
-> Access Key = ""
-> Secret Access Key = ""
-> Token = ""
-> provider
->value
-> Access Key With Value
-> Secret Access Key With Value
-> Token With Value
I personally cannot find any documentation regarding "creds" / "v", and I don't know if this is causing the issue. As I mentioned, I can use the AWS CLI to upload into the bucket, and even when I hard code my access key etc in to the Go SDK I receive this error.
Thank you for any thoughts, greatly appreciated.
I just compiled your code and its executing OK ... one of the many ways to supply credentials to your binary is to populate these env vars
export AWS_ACCESS_KEY_ID=AKfoobarT2IJEAU4
export AWS_SECRET_ACCESS_KEY=oa6oT0Xxt4foobarbambazdWFCb
export AWS_REGION=us-west-2
that is all you need when using the env var approach ( your values are available using the aws console browser )
the big picture is to create a wrapper shell script ( bash ) which contains above three lines to populate the env vars to supply credentials then in same shell script execute the golang binary ( typically you compile the golang in some preliminary process ) ... in my case I store the values of my three env vars in encrypted files which the shell script decrypts just before it calls the above export commands
sometimes its helpful to drop kick and just use the aws command line equivalent commands to get yourself into the ballpark ... from a terminal run
aws s3 ls s3://cairo-mombasa-zaire --region us-west-2
which can also use those same env vars shown above
for completeness here is your code with boilerplate added ... this runs OK and lists out the buckets
package main
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
// "github.com/aws/aws-sdk-go/service/s3/s3manager"
"fmt"
"os"
)
func exitErrorf(msg string, args ...interface{}) {
fmt.Fprintf(os.Stderr, msg+"\n", args...)
os.Exit(1)
}
func main() {
region_env_var := "AWS_REGION"
curr_region := os.Getenv(region_env_var)
if curr_region == "" {
exitErrorf("ERROR - failed to get region from env var %v", region_env_var)
}
fmt.Println("here is region ", curr_region)
// Load session from shared config
sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))
svc := s3.New(sess)
result, err := svc.ListBuckets(nil)
if err != nil { exitErrorf("Unable to list buckets, %v", err) }
for _, b := range result.Buckets {
fmt.Printf("* %s created on %s\n", aws.StringValue(b.Name), aws.TimeValue(b.CreationDate))
}
}
numBytes, err := downloader.Download(tempFile,
&s3.GetObjectInput{
Bucket: aws.String(bucket),
Key: aws.String(fileName),
},
)
In my case the bucket value was wrong, it is missing literal "/" at the end. Adding that fixes my problem.
Error i got - err: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
status code: 403,
If anyone else happens to have this problem,
The issue was regarding environment variables much like Scott suggest above, however it was due to lacking
export AWS_SDK_LOAD_CONFIG="true"
If this environment variable is not present, then the Golang SDK will not look for a credentials file, along with this, I instantiated environment variables for both my keys which allowed the connection to succeed.
To recap
if you're attempting to use the shared credentials folder, you must use the above noted environment variable to enable it.
If you're using environment variables, you shouldn't be affected by this problem.
I'd like to upload files to my s3 bucket via the aws golang sdk. I have a web server listening to POST requests and I'm expecting to receive multiple files of any type.
Using the sdk, the s3 struct PutObjectInput expects Body to be of type io.ReadSeeker and I'm not sure how to extract the content from the files uploaded and in turn satisfy the io.ReadSeeker interface.
images := r.MultipartForm.File
for _, files := range images {
for _, f := range files {
# In my handler, I can loop over the files
# and see the content
fmt.Println(f.Header)
_, err = svc.PutObjectWithContext(ctx, &s3.PutObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("key"),
Body: FILE_CONTENT_HERE,
})
}
}
Use the FileHeader.Open method to get an io.ReadSeeker.
f, err := f.Open()
if err != nil {
// handle error
}
_, err = svc.PutObjectWithContext(ctx, &s3.PutObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("key"),
Body: f,
})
Open returns a File. This type satisfies the io.ReadSeeker interface.
Use the S3 Manager's Uploader.Upload method, http://docs.aws.amazon.com/sdk-for-go/api/service/s3/s3manager/#Uploader.Upload. We have an example at http://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/s3-example-basic-bucket-operations.html#s3-examples-bucket-ops-upload-file-to-bucket.