Whenever I try to generate a Presign URL using AWS API I get something like this:
https://{OBJECT_PATH}?X-Amz-Algorithm=AWS4-HMAC-SHA256\\u0026X-Amz-Credential=AKIAJMALZY6GKVGFVOCQ%2F20191003%2Fus-east-1%2Fs3%2Faws4_request\\u0026X-Amz-Date=20191003T111419Z\\u0026X-Amz-Expires=300\\u0026X-Amz-SignedHeaders=host\\u0026X-Amz-Signature=c72a66c249fe2dada4c3925388efc62cbc734c84e01ffbf4748a5f6be9a99026\
When I try to enter it, I get the following message:
<Error>
<Code>AuthorizationQueryParametersError</Code>
<Message>X-Amz-Algorithm only supports "AWS4-HMAC-SHA256"</Message
<RequestId>7236615A79C34A4B</RequestId>
<HostId>PHoX1otuX6apJrXRTOMM/8GvgiajrNPdElrnfSzAZCzpLzMG8NFu9RIJEiyEYHWAls91n882QNE=</HostId>
</Error>
I tried to replace \\u0026 with & because I saw on documentation (https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html) that this is how presign URL should be structured, I get:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
<AWSAccessKeyId>AKIAJMALZY6GKVGFVOCQ</AWSAccessKeyId>
<StringToSign>AWS4-HMAC-SHA256
20191003T112220Z
20191003/us-east-1/s3/aws4_request
a1d8f89ff0472249afb8f1320314eef950bcef423f05fe98420f53cc7f7d8e8f</StringToSign>
<SignatureProvided>a1b2b7080c67e6456cbd2ab678565e5f3857483378e69ca03e35cc83232b2844\</SignatureProvided>
<StringToSignBytes>{SEQUENCE_OF_BYTES}</StringToSignBytes>
<CanonicalRequest>GET/{OBJECT_KEY}?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJMALZY6GKVGFVOCQ%2F20191003%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191003T112220Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host host:{BUCKET_NAME} host UNSIGNED-PAYLOAD</CanonicalRequest>
<CanonicalRequestBytes>{SEQUENCE_OF_BYTES}</CanonicalRequestBytes>
<RequestId>F3A0303ADD62D8FD</RequestId
<HostId>yU7FdXLoEO+j+KpZtQH1YjF6l3RnVugBi6rzCRJOgVYk7FEQqreobMuFdSiYtzIKUgl0Qr2GcAQ=</HostId>
</Error>
Finally, here is the code I use in order to generate these URLs:
var accessableURLs []string
for _, fileName := range fileNames {
objectKey := filepath.Join(fileDir, fileName)
req, _ := s3Obj.Client.GetObjectRequest(&s3.GetObjectInput{
Bucket: aws.String(s3Obj.Bucket),
Key: aws.String(objectKey),
})
urlStr, err := req.Presign(expireAfter)
if err != nil {
glog.Errorf("Failed during generating pre-signed S3 url, reason -%v ", err)
return nil, err
}
accessableURLs = append(accessableURLs, urlStr)
}
The following works with the v1 aws sdk. I'm not sure why your solution fails:
package main
import (
"fmt"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/s3"
)
func main() {
MyBucketName := "myAwesomeBucket"
MyKeyName := "/path/filename.txt"
days := 5
session, err := session.NewSession(&aws.Config{
Region: aws.String("us-east-1")},
)
s := s3.New(session)
r, _ := s.GetObjectRequest(&s3.GetObjectInput{
Bucket: aws.String(MyBucketName),
Key: aws.String(MyKeyName),
})
url, err := r.Presign(time.Duration(days) * 24 * time.Hour)
if err != nil {
fmt.Println("failure")
}
fmt.Println(url)
}
One thing to note is that the IAM creds used to sign the url must be valid for as long as the signed url is intended to be valid for. This ends up being an issue with ec2 machines that use temporary creds. A workaround is for the application to have a permanent IAM cred that it uses to sign the url with.
Related
I am trying to write an example of how to use a web identity token with a container to perform EC2 operations.
The container spec contains the service account and has the necessary permission to access the token path and its namespace is a trusted entity in the role.
package main
import (
"fmt"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/sts"
)
func main() {
sess, _ := session.NewSession()
config := aws.NewConfig().WithRegion("us-east-1")
stsSTS := sts.New(sess)
roleARN := "arn:aws:iam::1234567:role/s2-p0o5-csi-drivers-ebs-cloud-credentials"
roleProvider := stscreds.NewWebIdentityRoleProviderWithOptions(stsSTS, roleARN, "gosession", stscreds.FetchTokenPath("/build/token"))
creds := credentials.NewCredentials(roleProvider)
credValue, _ := roleProvider.Retrieve()
fmt.Printf("credValue.AccessKeyID: %v\n", credValue.AccessKeyID)
fmt.Printf("credValue.SecretAccessKey: %v\n", credValue.SecretAccessKey)
fmt.Printf("credValue.SessionToken: %v\n", credValue.SessionToken)
config = config.WithCredentials(creds)
nodeID := "i-00843f27cfeb0beff"
svc := ec2.New(sess, config)
request := &ec2.DescribeInstancesInput{
InstanceIds: []*string{&nodeID},
}
result, _ := svc.DescribeInstances(request)
fmt.Printf("result: %v\n", result)
}
The value of result get empty. Whereas, i have exported (credValue.AccessKeyID, credValue.SecretAccessKey,credValue.SessionToken) as environment variables and aws cli is giving me output related to describing the instance.
I tried various methods like credentials.NewStaticCredentials() with the credential information, but no luck. Can some help share hint on what is going wrong and correct way of doing it.
The instance-id used should be avaiable in the region, else it returns an empty map with err == nil.
I am trying to upload base64 encoded string to aws s3 bucket in the form of PNG image. I had tried following code:
import (
awsconfig "github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/feature/s3/manager"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
signature:= "iVBORw0KGgoAAAANSUhEUgAAAfQAAAC0CAYAAABi+d5SAAAAAXNSR0IArs4c6QAAHANJREFUeF7tnWusdkdZhm+kVUSxYg+2klirFWMrrdD+wPKjEg+QCtQayw8PqYApQUPaRhJDhBQUEyJqDyHGeihtQI1oLI02WklBYrWgrWBaRKxQEKgoLWjVWA5ac/utRd9v9333embeWWvNrHVNsrP39+1nZs1cM+++1zwz88zjRIIABCAAAQhAoHkCj2u+BTQAAhCAAAQgAAEh6AwCCEAAAhCAwAIIIOgL6ESaAAEIQAACEEDQGQMQgAAEIACBBRBA0BfQiTQBAhCAAAQggKAzBiAAAQhAAAILIICgL6ATaQIEIAABCEAAQWcMQAACEIAABBZAAEFfQCfSBAhAAAIQgACCzhiAAAQgAAEILIAAgr6ATqQJEIAABCAAAQSdMQABCEAAAhBYAAEEvb1O/HZJx0n6zq7q/vdXdz/7u7++YaNZ/ybJX04Hvx9svfMeL+mzkh6W9MABgz+T9GFJ90u6rT101BgCEIDAcgkg6PX2rcX17E64LdoWaX+vIT0i6c8lvUbSO2uoEHWAAAQgsHYCCHo9I+D8TrA98+4FvJ7a7a7J6yS9uoWKUkcIQAACSyaAoM/Tu559P0fSj0k6uaKZdy6NmyT9QG5m8kEAAhCAwP4EEPT9GUZK2HSff/8CBHxbm6+RdHkEBjYQgAAEIFCeAIJenmlfol3oFu/ehT7ek+op+XRJH6qnOtQEAhCAwHoIIOhl+9pr35d0rvR+53nZJ9Rd2lsk/WjdVaR2EIAABJZJAEHfv18t3BZxu5s3j4vtX3J7JTzUHalrr+bUGAIQgEDjBBD0/A60kF/WCfkaZ+O7yPko22vzsZITAhCAAARyCCDoOdSOzMgtXGufkW+jd4ek8/KwkgsCEIAABHIJIOhp5Czgb9qI0paWu4z1u7qIb+/b+O6SHQXO/xdNm9HlNvO4jR+V9BFJZ0g6qfvlyyVdFCz8BEkPBm0xgwAEIACBAgQQ9DhEu9c9K5/SvW7xtkg75KoFNkWw4y2LWZ4j6c6YqZ7d1TlojhkEIAABCOxLAEEfJmgBd+CUPnb6cI48C8dOv7UTQgv4nOK9qwXXSvJMfSg5YM6NQ0b8HgIQgAAEyhFA0A9n6XPkdrGPMSu3W9vCfY+kd0u6vVy3jlaSPRRXBkr3pjjbkiAAAQhAYCICCPp20BZwC1fJyGf/LultGzNwu9BbSwh6az1GfSEAgdUQQNAf29UODuNZeYmbzTwLt4jfUKkLPXWgI+ipxLCHAAQgMBEBBP1o0F4jfpmkY/bkf7Okqxe4MeyqoNfiiq79e2IkOwQgAAEIRAkg6EdI2cXuTVwviILbYecyPItt0Z0eabrvPo9sDmSXe4QmNhCAAAQKEkDQjwSH8S72fVzsPl7m9fYad6YXHC56b5DTWZLuLvlgyoIABCAAgcMJrF3QPdu0mOfuYvdGN8/I7V5fQ7pf0imBhq59XAUQYQIBCECgLIE1/+H1WWlvfstNf9tdj7pU9/o2Lo8EYH1qI7pcwBwTCEAAAhAoQWCtgr6vmK/1nHVE0D0u1zquSnwmKQMCEIBAFoE1/uH1Wrd3a+cku9gdbMYBYdaYIoLOFaprHBm0GQIQmJ3A2gR9n5n52yW9sLsEZfaOm6kCX5D0+IFn/6ekJ81UPx4LAQhAYLUE1iTo3gDnY1c56Xck/VBOxoXliczQ/1vSExfWbpoDAQhAoHoCaxF072K/L2M3+yckPW8Fx9GiAzUi6C5rLeMqyg07CEAAAqMTWMsfXodfvTCRpnexe1bve8ZJRwgg6IwECEAAApUSWIOg56ybO+Kb85GOJhANLHPagqPlMSYgAAEIVElg6YLuKHAWoZTAMYj57qEaDf16rqS7qhzxVAoCEIDAQgksXdBTXe2I+eED3YF4Ip6L50v6o4V+ZmgWBCAAgSoJLFnQfV7cYV2jCTEfJnWdpEuHzf4/HK6D75AgAAEIQGAiAksVdLvY7Wq3yz2S2AAXoXQkbv2VAdO1RtILoMEEAhCAwDgElirovizlsiAyR3/zbval35QWxHGoGYJegiJlQAACEBiBwBIF3degenYeTcwmo6SOuNvtdh9KMB0ixO8hAAEIFCawREH3TPvsICe72ve5Bz34mMWYMUNfTFfSEAhAYGkEliboUcHp+/HpuNqThvQ7JD07kMM2a73AJoAHEwhAAALlCSxJ0C+QdEsComsk+eY1UpyA7zo/IWCOoAcgYQIBCECgJIElCbpDtB4XhPPRztVOWNcgsM7sc5KODWSx6D8YsMMEAhCAAAQKEViKoKfsajc6ZpDpA8hHAH3BzVD6rKQnDBnxewhAAAIQKEtgCYKeei3qzZIcdIaURiAaqOceSU9LKxprCEAAAhDYl0Drgp56LarPnHumias9feRENxxyZC2dLTkgAAEI7E2gdUGPXhbSg7pIkuO7k9IJROPis5yRzpYcEIAABPYm0LKgR2eMPaQPSDpjb2LrLeAjkk4NNJ+rUwOQMIEABCBQmkCrgp66bv4ZSd+Iq32v4fNIILeXNFKuqg0UiQkEIAABCEQItCjoqevm5oAbODIadttEX6De1cXF3+9p5IYABCAAgWQCLQq647SnhGtlk1bysHhMBt+B7rvQhxLBeoYI8XsIQAACIxFoTdAtKhaXaCJWe5TU4XbRc/5XSLItCQIQgAAEJibQkqBHZ4k9Qq/neibvzVyk/Qg4Lvv5gSJY2ghAwgQCEIDAGARaEXQLs4+opWy4epGkG8aAtsIyozvcn8zGwxWODpoMAQhUQaAFQbeIe93cAWGi6cZE13y03DXamb9PCQwldrgPEeL3EIAABEYk0IKgpwaPYd287IBZ0g53vxTuOkvvFxd/9VEEt0UT9KU+LOGUHV+UBgEIFCJQu6CnBo9h3bzQwNgoJrp3oeQO915c/f3gDXr+Pwvz5vKLf+7j81uI/dXbpCzTpNLzcx6W9HlJj5f0QUm+YvZEST7C90/d122pBWMPAQhAIJVAzYLudXO72lMSm7JSaMVsoy9Vu3a495vp3J+9uG7+vE2gYzVrw8oBee6V9BOSEPY2+oxaQqBJArUKuv/I29Wect6cI1PjDMHoDvc3SvrCRp/ZVU96lIDZ/IKknwEKBCAAgTEI1Cro0XPPPRM2wY0xOo6UabfyQbf3eE9bfskvkXT98ptJCyEAgakJ1Cjoqa52b4LzbJArUcuMHrvIz5X0PEmnSPqWMsVSygYBb8zz+joJAhCAQDECNQp6yq52b4KzmL+vGJF1FNSva5udlzf8ErW5rr0OCvO10ssTL5/v8TwZAhBYIoHaBD26o7rvC4LHbB+VFme7yXuRPvh9iWO5tTZ9TfB8f2vtor4QgMBMBGoT9PsSAsisfd3cs2wfzdr8YpY90wcp47Fe1rgrIx9ZIAABCGwlUJOgp8zO7Wq3kK1h3dwu8bO7pYXeNZ4SNY+hXyeBX5P00jqrRq0gAIEWCdQk6CnXol4k6W0tAg/U2WJ9Yecu9xo34h2AdoiJX/784rctwpv/f9f+i557H+Rm8xH9ccp9dv/7bPpT92sauSEAAQg8SqAWQXeUr5uCHeMIXEs742wBd5vMYe0CbgG2+B70vvj/Dorywf/7tKSHurxTe29Ok3S8pKclHEur5fMX/OhhBgEI1Eyglj8onm1b1CJpKdHgPMu7rBPxMcOTRpiOZdPHPt+cCW+KcD9znlp8x2qvyz1D0vsDD7hf0lMCdphAAAIQCBGoQdCjt3m5Qa3Pzpco4vdIelCSI8r1Yr1tNh0akAswil5m8x5Jz1xAe2kCBCBQCYEaBP1ySVcFebQ4O/cLyyXdda4poWyDSGY1a/0Fawx4b5V0caDgX5L0ioAdJhCAAARCBGoQ9OhmuNauRfVMrRfyUGdUYOT1a8+4nxWsC/HzHwvK6/hPDvB7oaTfC9hhAgEIQCBEYG5BTwnz2koQGYu4vQ61zsYt2t7Z3bvF/bPXsO0yd0rxmHgjGPeDH/1R81WqxwQ+fSd0SxUBU0wgAAEIDBOYW9BTxMOznlo3T3ln+hskfU8lF5nYFe5kke43pB12RGtzpFjgfe59KLXmMRlqT4nfnyPpzkBBvnnt2IAdJhCAAATCBOYW9GhkuFqjwtmt7rXQZ4SJlzO0aPez7H523X/PfcoSPSa5LHLyOT77tYGMd0g6L2CHCQQgAIEwgTkFPWV3e22BZCzkV054Ht6zYYu1v3p3ebiTEwxv6Nb9I1lq9phE6j+GzZu6zY9DZb9S0uuHjPg9BCAAgRQCcwp61N3us8y1BFtxPbwj3wFgxkxus8Xb5/N7t/mYz3PZfsGyxyRyJr5Wj8nYjIbKj3qcniPpT4cK4/cQgAAEUgjMKejRa1JrEQ+/gHhWHhG8lD7obS3iFnDPkue4DvY1Xfsida/NYxKp89g20YAyrsfJkv5l7ApRPgQgsC4Ccwr6/0j6kgDuucXDAu6wtGOEm+1F/OoKdos/0IUuHeqSmjwmQ3Wd8vc/GDyG9o+SvnnKivEsCEBgHQTmEvSUzVdz1dEjwPW0J6H0rNxeB8/E993EVmqU/rikXw8Wxtnz7aB+V5LPlg8lblkbIsTvIQCBLAJziWX0qlQHOfFlF3Mkz8g9My8l5t7Y5pm43eq1Hb+7W9K3BSD7DLtfcjh7/lhY/ysp8nn6EUm/FWCNCQQgAIEkApE/QEkFBo0tbL6YZCi9WtLrhoxG+H2pmbkF0ALu9s6xLh5Bk+ItqWU/Q6RdU9o8V9IfBx/49ZI+FrTFDAIQgECYwFyCblfz+QO1/Jykr51hNpuy23tXExw4xOfTf7WB2WzKUbWnV/xiEh70Ixj+Q3Bd3OPe9xGQIAABCBQnMJegPxJoyc0THA/bVo1opLRteT/RzcZ/MdC+GkxSYgF8UtIpNVS6sjpEg8m42hckzOQraybVgQAEaicwh6D7LLfP6w6lOXa3pxzd2qy/18ed1+71llJ06cNtYu13e8+6788Kdvocn7dg1TCDAARaJzDHH5jofdFTRyJLWUvu+91r5BZyC2NrKWV2zlG17b3rl5w3Bzsed3sQFGYQgEAegTkEPRIhbo6LP1Jd7a6jI8a1uuM7ZXbOUbXtn6+/k/StwY+e185rOaYYrDJmEIBASwTmEPSIW3tqAYnU6aCL3Z6G2o6fRcdeyuzcXggvk7Ta1iiTVLuop8nlMjtPpYs9BCCQTGAOQfc684UDNZ1yN7XF6r0J5809M29ZzI0+ZXb+2m5ZIXlwLTzDrZK+N9hGZudBUJhBAAL5BOYQ9MiRtSnrFXnB6AkvYbaa+gIz9V6G/NE8Xc5Lukh/kSf+a3f8MmKbYvOVks6V9BWSHHb2uyR5fH55F1K5v9DIS0K+e92hln26ZNeGVJfzdQG7zTra03OCJB/T9Gd2s+yTJDm+/Wb6+BZbvxz7b4JD4vqUyB9IcqAjEgQgkEhgSuHsqza0Vu17vseIm74NjdfAHQ0umubYeR+tW9Qu8kLVl3WNJO95IB1NIHqrmnO9KEH8t3H+UklPkXRq5xF4pqQTg5H9Wuy3/+iuoLWwkyAAgQQCcwi612KPO6SOU7l4Pbuwqz16Netc5+ITunPQNPUFhtn5Y5FeJ+nSQdJHDLxp7sygbW/m0xbfJ+n53ezb+ef4nCZWu6i5PQmOi//7RUulMAgsnMAcfyiGgspMtSEuZR15CTHMUyPgTfVi1dJH7BWS3pBQ4V1r5+6Ls7uXSb9QWsT77wnFL97UkSK9ZEGCAAQCBGoU9Ck2EDli1y0BPr3JVC8ZCVVKNh1a6tgscAl7BZIBDWQ4R9KdCYV6b4aXaJy8hORQxxbuXrwTilqtqe9x8H0OJAhAIEBgrYI+5PbfRDfHmfhA1yWZpMRrd8H7rvsmVa4R47/uNqFFq+tY/n4JmGo/SLRerdnZm+EXTBIEIDBAoEZBH3vdNvXM+ZRH6MYYsNGravtnL+EFpjRHR4NzVDjS9AQcVpdd79Nz54kNEqhR0MesU+qRrdZ3eadugvMQbv0FpvTH8FWSfq50oZQXJvDDkn47bI0hBFZMYEzx3IV1yN09Zp1SXM//3J2jbTVCWs6d7myEO3rUet2bcK3z/oFk+Wde/jy9IQJjiucuDEPnoMeqU/SWt77eLZ85zxFzXO1Hj9jjJb0j4Sa1hj72TVXVY9ljkwQBCAwQGEs8D3vsXIKeMjv/kKTTGx09qefr3cwlHMsr2V0O3nJHyQJHKstBmOxBcl19I957uudsXhhkL4OTj399oPM6OTCNA7g8aUu9bPdAF7zGdruS8z/c2d3f1cP2fR7//m8OZHZdbHuapD8JMnF5rg8JAhCoUNCHQq2O8ZKRchmJ/0j5Bq0Wb1FzO9/ZHY1KGfwteyNS2hmx9YvcvRHDCW08Ji3Wfhn+cBci9bYJn1/6UdGNmn8p6VmlH055EFgqgTHEc4jV0C5zv72XFtPoHxDXveV15KGXpW1903J7h8Za6u8de/w3JH1HasaC9nYve/w7bsAHJX1M0u0Fy6+hKEfac8S9oeSXGHtLSBCAQIBAjYI+RmCZqNC1HFAlZUmhHxo3dnGzA0Nl8SYWDs8Ip/hMeJxZsP3Vi7e/l36RrbXTvHP9LYHK/ZckX0JDggAEAgSm+ON1sBpDR6nGEPShnfV9HVudraaEse3b6vVX90Wru/gDwzvJZCgkcVJhB4wt4HaX918W8jWnp3behwgDe/T8uSRBAAIDBOYQdEfO8jrvrlRa0L1L1pewRNLYQW0idUi1eb2kn07MtIQ73RObfKj5X0g6r2SBkj4v6Vc6EbeHiHQ0gc9I8p6PSPKmOF+O5AA/Hru8hEaoYbM6AnMI+pDAlp4lR9fPW7xNbejlaNuA9m5o9wF/FB+lU3J27stbHNnM4kPaTcCcfNlNTvLYPbhk0S9j5JRHHggsgsAcgm5wh/0BLS3oQ5vw+o4s/dyxB0jOWXP/0fNLwNpdvpt984Ju9rdPf/klyXsYvKHu4/sUtKK8Hr9egjjsKuVcHCX3IzxB0smS/N2eAn/5hWLzhfjgv3PrnZJv07vR/7zt/6Jl9u08pmvb2o4K+m+ilyGb9qatQdCHzr33A760qz/6Qcqx8x/DmxLucvczEPPtpKMbtHb1k18E/VVylp8zJlrMk7P3o8V2Uud2CGzekthOrbuaziXofoM+dQet0vHTD3vWZhXGOC43xoDIOWtuMffSQ9Nvn2PAlJSyQWuzCl6isSARGja/Y07qThZ8U34R5IRAcQKteWu/CGAuQT9s1lx6LTs6c5qLRcpozBFzl0887MMp35fo7bCb/g9TOg7bnQR+UtIb4QOBygi0MsE7CttcInaYoHsdo9Qd0tFNYyWfOda4zBXz6yW9ZKxKLaTcSJz/uyS9XdIrF9Lmmprxy5KuqKlC1GX1BJqMnrl0QY/ucK89wEqumN8q6bmr/2jGAHxZF1/cUeL8c598TMr7FX6WdfIYyEyrt0q6ODMv2SBQmkCTXs2lC/oSdrjn7Gb34G7B61D6Q1iqPLvbvMeD9fFSRGPlvFjST3UXyMRyYAWBcQi0tEn6iwSWLuh2kz4j0N81dp5n5ZdJ8ktJauIq1FRi2NdC4ExJ/rJHxBsW5/obVQsP6jE9gWb/fs71YZlqDd3RunyucijVFiHOSwVXJUTS2mxfy/Hoh/qJ36+PgPc32EvlL++J8b93nZBZHx1aXJpA08d75xL0wy5LKeUqjm6Ie1DSCaVHRWZ5rvOVe2wKbHowZjIj2/oI2Htlgbe4+6vfRIvYr28slGyxtceTqWYvSVqyoFsYI+7qv+/uPy85MFLLOqu7TnKfqyIR81Tq2C+ZgAU/Git+8w+4o771+fpocJuR2CJl2qb/6hlH8u3qj74efUS6fcM2b8vv//MLUc/CP/fP69tyMJ///zBec46vvl6b3w/Wp2+P27qI/TJLFnTvTPZtYkPp2m6teshujN/7A+EXD5/FPXaPByDme8AjKwQgAIElEJhL0KdYQ4/e5vTdkm6boTPtPfCmt33e3F1txHyGzuOREIAABGojsGRBrzFCnMX7km4pYF8h91jybkyv+XDZSm2fLOoDAQhAYGICNQp6qSAvNQm61/M8G/cSQAkh9zBxzHvP8vddT5t4yPE4CEAAAhAYgwCCPt45V4v4+d0M2j+XSr6q83IuWimFk3IgAAEILIPAXIJ+2LG1Ujfd3Cvp9EA3vVuSw33umzzztoB7Ft6fl923zIP5mZWXJkp5EIAABBZCYC5B33VG3Bu8PJstcQ7wOkmXBvsph0Mv4G6Lv0rOwg9W2+cjPStnrTzYoZhBAAIQWBuBHCErxcgi6DVgC6HXgS1W/ncp0YoGlnF7/Ezf9jR0FtEz+Zd1F56cWArEIeV8urspjXvMJ4DNIyAAAQi0TGBOQZ+CW+o913dL+pSkn5f0ye7nM7oLIyzmU0WUe1jSb0p6FZvephgmPAMCEIBA+wSWLui3SLqgoW7yksPV3Re71xvqOKoKAQhAYG4CSxd0h1T1jWuRC1rm7AsL+Q2dkJfYPzBnW3g2BCAAAQjMQGDpgm6kvqj++hnYRh6JkEcoYQMBCEAAAoME1iDohvB+SV4LryX1rnXPypmR19Ir1AMCEIBAwwTWIuhnSrqngn66uQsI413rrJFX0CFUAQIQgMBSCKxF0N1fc4i6Y637KFz/hYgv5ZNDOyAAAQhURmBNgm70Pnrm8+YXj9QPDstq8fYM3N8R8JFAUywEIAABCBxNYG2C3rfeN5T51jMHn9knPdTdeOZNdxZw1sP3oUleCEAAAhDIJrBWQe+BOTSs472/VNJXBSh6xu0wrN7M5p+HIssFisQEAhCAAAQgsD+BtQv6JsEXS3I411MknS3pJElP7GbdFvG/knS7JM/KSRCAAAQgAIGqCCDoVXUHlYEABCAAAQjkEUDQ87iRCwIQgAAEIFAVAQS9qu6gMhCAAAQgAIE8Agh6HjdyQQACEIAABKoigKBX1R1UBgIQgAAEIJBHAEHP40YuCEAAAhCAQFUEEPSquoPKQAACEIAABPIIIOh53MgFAQhAAAIQqIoAgl5Vd1AZCEAAAhCAQB4BBD2PG7kgAAEIQAACVRFA0KvqDioDAQhAAAIQyCOAoOdxIxcEIAABCECgKgIIelXdQWUgAAEIQAACeQQQ9Dxu5IIABCAAAQhURQBBr6o7qAwEIAABCEAgjwCCnseNXBCAAAQgAIGqCCDoVXUHlYEABCAAAQjkEUDQ87iRCwIQgAAEIFAVAQS9qu6gMhCAAAQgAIE8Agh6HjdyQQACEIAABKoigKBX1R1UBgIQgAAEIJBHAEHP40YuCEAAAhCAQFUEEPSquoPKQAACEIAABPIIIOh53MgFAQhAAAIQqIoAgl5Vd1AZCEAAAhCAQB4BBD2PG7kgAAEIQAACVRFA0KvqDioDAQhAAAIQyCOAoOdxIxcEIAABCECgKgIIelXdQWUgAAEIQAACeQQQ9Dxu5IIABCAAAQhURQBBr6o7qAwEIAABCEAgjwCCnseNXBCAAAQgAIGqCCDoVXUHlYEABCAAAQjkEUDQ87iRCwIQgAAEIFAVAQS9qu6gMhCAAAQgAIE8Agh6HjdyQQACEIAABKoigKBX1R1UBgIQgAAEIJBH4P8AbePq4jPqwCcAAAAASUVORK5CYII="
decodedSignature, decodeErr := base64.StdEncoding.DecodeString(signature)
if decodeErr != nil {
// handle error here
}
r := bytes.NewReader(decodedSignature)
bucketName := "test"
filepath := "uploads"
contentType := "image/png"
cfg, err := awsconfig.LoadDefaultConfig(context.TODO(),
awsconfig.WithRegion(config.AwsRegion),
)
client := s3.NewFromConfig(cfg)
// Create an uploader with the client and options
uploader := manager.NewUploader(client)
cacheExpireTime := time.Now().AddDate(1, 0, 0)
cacheMaxAgeSecs := "31556926" // seconds in 1 year
// Upload the file to S3.
uploadResp, err := uploader.Upload(context.TODO(), &s3.PutObjectInput{
Bucket: aws.String(bucketName),
Key: aws.String(filePath),
Body: r,
ContentType: aws.String(contentType),
Expires: &cacheExpireTime,
CacheControl: aws.String(cacheMaxAgeSecs),
})
But when I try to load the s3 url on which the image is uploaded, then there is black screen as shown in the screenshot:
The Body parameter in the uploader.Upload function accepts io.Reader objects as body, meaning that you either need to use os.Open to open a png you have saved, or in this case, use bytes.NewReader(data) to make your data conform to the io.Reader interface (have the .Read() method).
AWS Config has a set of Managed Rules and I am trying to use the Golang AWS SDK to use the DescribeConfigRules API to retrieve the list of AWS Config Managed Rule Names and other details.
It seems like every request receives a response of 25 rules and a NextToken for the next set of results. What I am having trouble understanding is how do I use this NextToken to retrieve the next set of results?
Here is what I have so far.
package main
import (
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/configservice"
)
func main() {
//Create an aws session
sess, err := session.NewSession(&aws.Config{Region: aws.String("us-west-2"), Credentials: credentials.NewSharedCredentials("", "my-aws-profile")})
// Create a ConfigService client from just a session.
configsvc := configservice.New(sess)
rules := (*configservice.DescribeConfigRulesInput)(nil)
configrulesoutput, err := configsvc.DescribeConfigRules(rules)
if err != nil {
log.Fatal(err)
}
for _, rule := range configrulesoutput.ConfigRules {
fmt.Println("Rule: ", *rule.ConfigRuleName)
}
}
The above code successfully prints the first 25 rules received in the response. However I am not sure how to use the NextToken received in the response to get the next set of results.
Sample Response.
ConfigRules: [
{
ConfigRuleArn: "ConfigRuleARN",
ConfigRuleId: "config-rule-ppwclr",
ConfigRuleName: "cloudtrail-enabled",
ConfigRuleState: "ACTIVE",
Description: "Checks whether AWS CloudTrail is enabled in your AWS account. Optionally, you can specify which S3 bucket, SNS topic, and Amazon CloudWatch Logs ARN to use.",
InputParameters: "{}",
MaximumExecutionFrequency: "TwentyFour_Hours",
Source: {
Owner: "AWS",
SourceIdentifier: "CLOUD_TRAIL_ENABLED"
}
},
{ Rule 2 }, ....{ Rule 25}
],
NextToken: "nexttoken"
}
Code extracts the rulenames from the response and output is as below.
Rule: cloudtrail-enabled
Rule: restricted-ssh
Rule: securityhub-access-keys-rotated
Rule: securityhub-autoscaling-group-elb-healthcheck-required
Rule: securityhub-cloud-trail-cloud-watch-logs-enabled
Rule: securityhub-cloud-trail-encryption-enabled
Rule: securityhub-cloud-trail-log-file-validation-enabled
Rule: securityhub-cloudtrail-enabled
Rule: securityhub-cmk-backing-key-rotation-enabled
Rule: securityhub-codebuild-project-envvar-awscred-check
Rule: securityhub-codebuild-project-source-repo-url-check
Rule: securityhub-ebs-snapshot-public-restorable-check
Rule: securityhub-ec2-managedinstance-patch-compliance
Rule: securityhub-ec2-security-group-attached-to-eni
Rule: securityhub-eip-attached
Rule: securityhub-elasticsearch-encrypted-at-rest
Rule: securityhub-elasticsearch-in-vpc-only
Rule: securityhub-iam-password-policy-ensure-expires
Rule: securityhub-iam-password-policy-lowercase-letter-check
Rule: securityhub-iam-password-policy-minimum-length-check
Rule: securityhub-iam-password-policy-number-check
Rule: securityhub-iam-password-policy-prevent-reuse-check
Rule: securityhub-iam-password-policy-symbol-check
Rule: securityhub-iam-password-policy-uppercase-letter-check
Rule: securityhub-iam-policy-no-statements-with-admin-access
End Goal: Using golang AWS SDK, extract the AWS Config Managed Rule details and put it in an excel format using Excelize to review which AWS Config rules we want enabled.
Thanks for your help in advance.
---New based on #Adrian's comment and doc reference---
As per doc
type DescribeConfigRulesInput struct {
// The names of the AWS Config rules for which you want details. If you do not
// specify any names, AWS Config returns details for all your rules.
ConfigRuleNames []*string `type:"list"`
// The nextToken string returned on a previous page that you use to get the
// next page of results in a paginated response.
NextToken *string `type:"string"`
// contains filtered or unexported fields }
So here is what I am trying. Specifying nil should give me back all rules. nextToken is blank string for the first call.
configsvc := configservice.New(sess)
rules := (*configservice.DescribeConfigRulesInput)(nil)
nextToken := ""
rules.SetNextToken(nextToken)
getConfigRulesFunc(configsvc, rules)
//getConfigRulesFunc function
func getConfigRulesFunc(cfgsvc *configservice.ConfigService, ruleset *configservice.DescribeConfigRulesInput) {
configrulesoutput, err := cfgsvc.DescribeConfigRules(ruleset)
if err != nil {
log.Fatal(err)
}
for i, r := range configrulesoutput.ConfigRules {
fmt.Println("Rule: ", i, ""+*r.ConfigRuleName)
}
if *configrulesoutput.NextToken != "" {
ruleset := (*configservice.DescribeConfigRulesInput)(nil)
ruleset.SetNextToken(*configrulesoutput.NextToken)
getConfigRulesFunc(cfgsvc, ruleset)
}
}
Above code compiles fine but here the runtime error I believe because of nil.
configsvc type: *configservice.ConfigService
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x13c7ed2]
goroutine 1 [running]:
github.com/aws/aws-sdk-go/service/configservice.(*DescribeConfigRulesInput).SetNextToken(...)
/Users/user/go/src/github.com/aws/aws-sdk-go/service/configservice/api.go:12230
main.main()
/Users/user/golang/awsgotest/awsgotest.go:26 +0x232
Ok, finally figured it out with the help of a very kind Alex Diehl via this ticket https://github.com/aws/aws-sdk-go/issues/3293 on the official aws-sdk-go repo.
I would still say the aws sdk for go definitely lacks simple examples for configservice at the least on recommended usage.
Here the code that works. This will also show how to use simple recursive function in go to use NextToken for pagination of api results that span multiple pages especially apis that do not have built in paginators.
Also note that DescribeConfigRules API does not list all AWS Managed Config Rules, only the Config rules enabled for your account.
package main
import (
"fmt"
"log"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/configservice"
)
var i int = 0
func main() {
sess, err := session.NewSession(&aws.Config{Region: aws.String("us-west-2"), Credentials: credentials.NewSharedCredentials("", "my-profile")})
if err != nil {
log.Fatal(err)
}
//Create a ConfigService client from just a session.
configsvc := configservice.New(sess)
fmt.Printf("configsvc type: %T\n", configsvc)
rules := &configservice.DescribeConfigRulesInput{}
getConfigRulesFunc(configsvc, rules)
}
func getConfigRulesFunc(cfgsvc *configservice.ConfigService, ruleset *configservice.DescribeConfigRulesInput) {
configrulesoutput, err := cfgsvc.DescribeConfigRules(ruleset)
if err != nil {
log.Fatal(err)
}
for _, r := range configrulesoutput.ConfigRules {
fmt.Println("Rule: ", i, ""+*r.ConfigRuleName)
i = i + 1
}
if configrulesoutput.NextToken != nil {
fmt.Println("In if nexttoken is not empty")
fmt.Println("Print NextToken: ", *configrulesoutput.NextToken)
ruleset := &configservice.DescribeConfigRulesInput{}
ruleset.SetNextToken(*configrulesoutput.NextToken)
getConfigRulesFunc(cfgsvc, ruleset)
}
}
Code in Bold were the ones giving me grief on how to use the NextToken based on best practices atleast for the go sdk for aws.
FYI, you could have looked in the AWS Go guide as there is a section on pagination: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/making-requests.html#using-pagination-methods.
I am working in Golang,now I am attempting to upload an image to AWS S3, but I get:
NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
My code is like this:
func firstFunction(){
//Connect to S3
AWSsession, err := ConnectAWS()
if err != nil {
fmt.Println("Error Connecting to AWS S3")
}
GetSingleMedia(AWSsession)
}
func ConnectAWS()(*session.Session, error){
//Create S3 Session
AWSsession, err := session.NewSession(&aws.Config{
Region: aws.String("us-west-2")},
)
if err != nil {
fmt.Println("Error AWS:", err.Error())
}
return AWSsession,err
}
func GetSingleMedia(...someparams,AWSsession *session.Session){
//o.Blob is correct, this is valid
data, err := ioutil.ReadAll(bytes.NewReader(o.Blob))
//Store: bytes.NewReader(o.Blob)
UploadImage(AWSsession,bytes.NewReader(o.Blob),bucket,"SomeID")
}
func UploadImage(AWSsession *session.Session,reader *bytes.Reader,bucket string, key string) (*s3manager.UploadOutput,error){
uploader := s3manager.NewUploader(AWSsession)
result, err := uploader.Upload(&s3manager.UploadInput{
Body : reader,
Bucket: aws.String(bucket),
Key : aws.String(key),
})
if err != nil {
fmt.Println("Error uploagin img: ",err.Error())
}
return result,err
}
Also, I have placed the creentials under /home/myuser/.aws/ there's a credential file, I don't get any error on creating the session, then, what could be the problem?
The error is triggered in UploadImage
EDIT:
Currently in the credentials file I have:
[default]
awsBucket = "someBucket"
awsAccessKey = "SOME_ACCESS_KEY"
awsSecretKey = "SOME_AWS_SECRET_KEY"
Sould I change any permission or something?
I would suggest you follow the guide here: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
This command will ask you for access/secret key and write them in a correct format:
aws configure
It would appear you have a wrong format of credentials file. The correct format would be something like this:
[default]
aws_access_key_id = SOME_ACCESS_KEY
aws_secret_access_key = SOME_AWS_SECRET_KEY
I need to get public permanent (not signed) URL of a resource using golang and official aws go sdk. In Java AWS S3 SDK there's a method called getResourceUrl() what's the equivalent in go?
This is how you get presigned URLs using the go sdk:
func GetFileLink(key string) (string, error) {
svc := s3.New(some params)
params := &s3.GetObjectInput{
Bucket: aws.String(a bucket name),
Key: aws.String(key),
}
req, _ := svc.GetObjectRequest(params)
url, err := req.Presign(15 * time.Minute) // Set link expiration time
if err != nil {
global.Log("[AWS GET LINK]:", params, err)
}
return url, err
}
If what you want is just the URL of a public access object you can build the URL yourself:
https://<region>.amazonaws.com/<bucket-name>/<key>
Where <region> is something like us-east-2. So using go it will be something like:
url := "https://%s.amazonaws.com/%s/%s"
url = fmt.Sprintf(url, "us-east-2", "my-bucket-name", "some-file.txt")
Here is a list of all the available regions for S3.
Looks almost clean:
import "github.com/aws/aws-sdk-go/private/protocol/rest"
...
params := &s3.GetObjectInput{
Bucket: aws.String(a bucket name),
Key: aws.String(key),
}
req, _ := svc.GetObjectRequest(params)
rest.Build(req) // aws method to build URL in request object
url = req.HTTPRequest.URL.String() // complete URL to resource