I am working in Golang,now I am attempting to upload an image to AWS S3, but I get:
NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
My code is like this:
func firstFunction(){
//Connect to S3
AWSsession, err := ConnectAWS()
if err != nil {
fmt.Println("Error Connecting to AWS S3")
}
GetSingleMedia(AWSsession)
}
func ConnectAWS()(*session.Session, error){
//Create S3 Session
AWSsession, err := session.NewSession(&aws.Config{
Region: aws.String("us-west-2")},
)
if err != nil {
fmt.Println("Error AWS:", err.Error())
}
return AWSsession,err
}
func GetSingleMedia(...someparams,AWSsession *session.Session){
//o.Blob is correct, this is valid
data, err := ioutil.ReadAll(bytes.NewReader(o.Blob))
//Store: bytes.NewReader(o.Blob)
UploadImage(AWSsession,bytes.NewReader(o.Blob),bucket,"SomeID")
}
func UploadImage(AWSsession *session.Session,reader *bytes.Reader,bucket string, key string) (*s3manager.UploadOutput,error){
uploader := s3manager.NewUploader(AWSsession)
result, err := uploader.Upload(&s3manager.UploadInput{
Body : reader,
Bucket: aws.String(bucket),
Key : aws.String(key),
})
if err != nil {
fmt.Println("Error uploagin img: ",err.Error())
}
return result,err
}
Also, I have placed the creentials under /home/myuser/.aws/ there's a credential file, I don't get any error on creating the session, then, what could be the problem?
The error is triggered in UploadImage
EDIT:
Currently in the credentials file I have:
[default]
awsBucket = "someBucket"
awsAccessKey = "SOME_ACCESS_KEY"
awsSecretKey = "SOME_AWS_SECRET_KEY"
Sould I change any permission or something?
I would suggest you follow the guide here: http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
This command will ask you for access/secret key and write them in a correct format:
aws configure
It would appear you have a wrong format of credentials file. The correct format would be something like this:
[default]
aws_access_key_id = SOME_ACCESS_KEY
aws_secret_access_key = SOME_AWS_SECRET_KEY
Related
I have setup cross account reading kinesis stream, but i get security token expired error when kinesis client is reading records. I used sts assume role to assume roleA in accountA, then use roleA credentials to assume roleB, lastly return the kinesis client, so there is no refresh feature applied to it and the client will expire in 1 hr by default. I looked up the stscreds AssumeRoleProvider and the doc says it will refresh the credentials. But i have no idea on how to refresh the first credential for assumed roleA then refresh the second credential for assumed roleB. Or is it better to call the method to reinitialize the kinesis client?
Here is the code block.
cfg, err := config.LoadDefaultConfig(
context.TODO(),
config.WithRegion("us-west-2"),
)
if err != nil {
return nil, err
}
stsclient := sts.NewFromConfig(cfg)
assumingcnf, err := config.LoadDefaultConfig(
context.TODO(),
config.WithRegion("us-west-2"),
config.WithCredentialsProvider(aws.NewCredentialsCache(
stscreds.NewAssumeRoleProvider(
stsclient,
roleToAssumeArn1,
)),
),
)
if err != nil {
return nil, err
}
stsclient = sts.NewFromConfig(assumingcnf)
cnf, err := config.LoadDefaultConfig(
context.TODO(),
config.WithRegion("us-west-2"),
config.WithCredentialsProvider(aws.NewCredentialsCache(
stscreds.NewAssumeRoleProvider(
stsclient,
roleToAssumeArn2,
)),
),
)
if err != nil {
return nil, err
}
kClient := kinesis.NewFromConfig(cnf)
return kClient
You should be able to do this with the providers provided by AWS. I'm assuming you're using aws-sdk-go-v2.
This would make the resulting CredentialsProvider return the cached credentials until they expire; then it will call provider2, which uses sts2 to get new credentials for roleB, and sts2 will always first call provider1 first to get new credentials for roleA.
func createProvider(cfg aws.Config) aws.CredentialsProvider {
sts1 := sts.NewFromConfig(cfg)
provider1 := stscreds.NewAssumeRoleProvider(sts1, "roleA")
sts2 := sts.NewFromConfig(cfg, func (o *sts.Options) { o.Credentials = provider1 })
provider2 := stscreds.NewAssumeRoleProvider(sts2, "roleB")
return aws.NewCredentialsCache(provider2)
}
I am attempting to build an Amazon S3 client in GoLang but I am having trouble making API calls. I'm receiving an error that says "no such host" but I am positive the credentials I'm providing are correct.
Defining a struct to hold the client
// the Client struct holding the client itself as well as the bucket.
type S3Client struct {
S3clientObject s3.S3
bucket string
}
// Initialize the client
func CreateS3Client() S3Client{
S3clientCreate := S3Client{S3clientObject: Connect(), bucket: GetS3Bucket()}
if (!CheckBuckets(S3clientCreate)) {
exitErrorf("Bucket does not exist, try again.")
}
return S3clientCreate
}
Connecting to the bucket
func Connect() s3.S3{
// Initialize a session
sess, err := session.NewSession(&aws.Config{
Credentials: credentials.NewStaticCredentials("myCredentials", "myCreds", ""),
Endpoint: aws.String("myDomain"),
Region: aws.String("myRegion"),
},
)
if err != nil {
exitErrorf("Unable to use credentials, %v", err)
}
// Create S3 service client
svc := s3.New(sess)
return *svc
}
At this point, I am able to establish a connection and use the ListBuckets functionality to receive a list of all the buckets (like this: https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#S3.ListBuckets)
When I try to call the GetObject API, it tells me it cannot find the host
// Gets an object from the bucket
func Get(client S3Client, key string) interface{} {
// golang does not support "default values" so I used a nil (same as null)
if (key == "") {
return nil
}
svc := client.S3clientObject
input := &s3.GetObjectInput{
Bucket: aws.String("myBucket"),
Key: aws.String("myPathKey"),
}
result, err := svc.GetObject(input)
if err != nil {
if aerr, ok := err.(awserr.Error); ok {
switch aerr.Code() {
case s3.ErrCodeNoSuchKey:
fmt.Println(s3.ErrCodeNoSuchKey, aerr.Error())
case s3.ErrCodeInvalidObjectState:
fmt.Println(s3.ErrCodeInvalidObjectState, aerr.Error())
default:
fmt.Println(aerr.Error())
}
} else {
fmt.Println(err.Error())
}
}
return result
}
This returns:
dial tcp: lookup "hostname": no such host
I cannot figure out why this is happening, because I am able to successfully make a connection to the bucket, and list them out using ListBuckets, but when using another API call, it fails to find the host. Is there something wrong with my code? Is there another configuration that I forgot about?
Any help or guidance is greatly appreciated as I'm somewhat new to using GoLang and S3.
Apparently the issue was with the bucket name. All I did to resolve this was put a "/" in front of the bucket name when creating it and it worked.
I'm attempting to use the docker go-sdk to push an image to AWS ECR.
This is the code I'm using to push the image.
where tag = ".dkr.ecr.us-east-1.amazonaws.com/api:mytag"
func Push(c context.Context, tag string, credentials string) error {
cli, err := client.NewClient(apiSocket, apiVersion, nil, apiHeaders)
if err != nil {
return err
}
fmt.Println(credentials)
resp, err := cli.ImagePush(c, tag, types.ImagePushOptions{
RegistryAuth: credentials,
})
if err != nil {
panic(err)
}
io.Copy(os.Stdout, resp)
resp.Close()
return nil
}
But I keep getting this response:
{"status":"The push refers to repository [<id>.dkr.ecr.us-east-1.amazonaws.com/api]"}
{"status":"Preparing","progressDetail":{},"id":"23432919a50a"}
{"status":"Preparing","progressDetail":{},"id":"9387ad10e44c"}
{"status":"Preparing","progressDetail":{},"id":"e2a4679276bf"}
{"status":"Preparing","progressDetail":{},"id":"31c5c8035e63"}
{"status":"Preparing","progressDetail":{},"id":"a73789d39a06"}
{"status":"Preparing","progressDetail":{},"id":"f36942254806"}
{"status":"Preparing","progressDetail":{},"id":"4a2596f9aa79"}
{"status":"Preparing","progressDetail":{},"id":"5cf3066ccdbc"}
{"status":"Preparing","progressDetail":{},"id":"76a1661c28fc"}
{"status":"Preparing","progressDetail":{},"id":"beefb6beb20f"}
{"status":"Preparing","progressDetail":{},"id":"df64d3292fd6"}
{"status":"Waiting","progressDetail":{},"id":"beefb6beb20f"}
{"status":"Waiting","progressDetail":{},"id":"df64d3292fd6"}
{"errorDetail":{"message":"no basic auth credentials"},"error":"no basic auth credentials"}
Any ideas?
Notes:
I've verified that the credentials string I'm passing in is a base64 encoded user:pass for the ECR registry.
I've verified that the ECR credentials I'm getting are from the same AWS Region as where im attempting to push the image.
I found out in a GitHub comment that RegistryAuth actually needs to be a base64 JSON string with username and password fields. Ugh. This is undocumented in the Docker repository.
RegistryAuth = "{ \"username\": \"myusername\", \"password\": \"mypassword\", \"email\": \"myemail\" }
Relevant GitHub comment.
It is working for me now.
I wanted to upload an image in aws s3.
const (
S3_REGION = ""
S3_BUCKET = ""
)
func main() {
// Create a single AWS session (we can re use this if we're uploading many files)
s, err := session.NewSession(&aws.Config{Region: aws.String(S3_REGION)})
if err != nil {
log.Fatal(err)
}
// Upload
err = AddFileToS3(s, "result.csv")
if err != nil {
log.Fatal(err)
}
}
I am stuck here.
Where can I get S3_REGION as per this code standard?
Source : https://golangcode.com/uploading-a-file-to-s3/
When you log in to the AWS console you see on the top right which region you are logged in to, for example "Oregon" which refers to the "us-west-2" region.
Please refer to this table and this link.
I need to get public permanent (not signed) URL of a resource using golang and official aws go sdk. In Java AWS S3 SDK there's a method called getResourceUrl() what's the equivalent in go?
This is how you get presigned URLs using the go sdk:
func GetFileLink(key string) (string, error) {
svc := s3.New(some params)
params := &s3.GetObjectInput{
Bucket: aws.String(a bucket name),
Key: aws.String(key),
}
req, _ := svc.GetObjectRequest(params)
url, err := req.Presign(15 * time.Minute) // Set link expiration time
if err != nil {
global.Log("[AWS GET LINK]:", params, err)
}
return url, err
}
If what you want is just the URL of a public access object you can build the URL yourself:
https://<region>.amazonaws.com/<bucket-name>/<key>
Where <region> is something like us-east-2. So using go it will be something like:
url := "https://%s.amazonaws.com/%s/%s"
url = fmt.Sprintf(url, "us-east-2", "my-bucket-name", "some-file.txt")
Here is a list of all the available regions for S3.
Looks almost clean:
import "github.com/aws/aws-sdk-go/private/protocol/rest"
...
params := &s3.GetObjectInput{
Bucket: aws.String(a bucket name),
Key: aws.String(key),
}
req, _ := svc.GetObjectRequest(params)
rest.Build(req) // aws method to build URL in request object
url = req.HTTPRequest.URL.String() // complete URL to resource