How to Store Datetime Values in Dynamodb - amazon-web-services

Given the following functions to create and then retrieve an item...
type Auth struct {
UserID string `dynamodbav:"UserId"`
ExpiresOn time.Time `dynamodbav:"ExpiresOn,unixtime"`
}
func (r *AuthRepository) Create(auth *Auth) (*Auth, error) {
deviceCode, err := c.NewDeviceCode()
if err != nil {
return nil, err
}
item, err := attributevalue.MarshalMap(auth)
if err != nil {
return nil, err
}
input := &dynamodb.PutItemInput{
Item: item,
TableName: aws.String("MyTable"),
}
_, err = r.svc.PutItem(r.ctx, input)
return auth, nil
}
func (r *AuthRepository) Get(userID string) (*Auth, error) {
input := &dynamodb.GetItemInput{
TableName: aws.String("MyTable"),
Key: map[string]types.AttributeValue{
"UserId": &types.AttributeValueMemberS{*aws.String(userID)},
},
}
result, err := r.svc.GetItem(r.ctx, input)
if err != nil {
return nil, err
}
var auth *Auth = nil
if len(result.Item) > 0 {
auth = &Auth{}
if err := attributevalue.UnmarshalMap(result.Item, &auth); err != nil {
return nil, err
}
}
return auth, nil
}
... I create a new item with that should expire in 30 minutes like this:
repo.Create(&auth.Auth{
UserID: "XYZ",
ExpiresOn: time.Now().Add(time.Duration(1800) * time.Second), // expires in 30 mins
})
Then, I retrieve the item created above and I check whether it has expired:
authEntry, err := repo.GetByUserCode("XYZ")
if authEntry.ExpiresOn.Before(time.Now()) {
// I always get here as Before evaluates to true
}
As I noted in the code snippet above, if I create an entry with an expiry time of now + 30 minutes, and then I immediately retrieve that record to check if ExpiresOn is before now, I always get true...
What's the correct way to store datetime in DynamoDB?

Related

How to check if a QLDB result is empty using Go?

In Go, how can I check if a AWS QLDB result is empty?
In node, the following would do the trick:
txn.execute(statement).then((result: Result) => {
const resultList: dom.Value[] = result.getResultList();
if (resultList.length === 0) {
// DO something
}
)}
I've been reading the official quick start, but examples "assumes the results are not empty":
p, err := driver.Execute(context.Background(), func(txn qldbdriver.Transaction) (interface{}, error) {
result, err := txn.Execute("SELECT firstName, lastName, age FROM People WHERE age = 54")
if err != nil {
return nil, err
}
// Assume the result is not empty
hasNext := result.Next(txn)
if !hasNext && result.Err() != nil {
return nil, result.Err()
}
ionBinary := result.GetCurrentData()
temp := new(Person)
err = ion.Unmarshal(ionBinary, temp)
if err != nil {
return nil, err
}
return *temp, nil
})

How do I add all resource arn's? Lambda Golang ListTags

I am trying to list out all tags of my lambda functions, struggling a lot, please help me if anyone knows.
func main() {
svc := lambda.New(session.New())
input := &lambda.ListTagsInput{
Resource: aws.String("arn:aws:lambda:us-east-1:657907747545:function-function-name"),
I'm expecting to list all tag arn's for my lambda functions
You can use the following code:
package main
import (
"context"
awsutils "github.com/alessiosavi/GoGPUtils/aws"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/service/lambda"
"os"
"sync"
)
var lambdaClient *lambda.Client = nil
var once sync.Once
func init() {
once.Do(func() {
cfg, err := awsutils.New()
if err != nil {
panic(err)
}
lambdaClient = lambda.New(lambda.Options{Credentials: cfg.Credentials, Region: cfg.Region})
})
}
func ListLambdas() ([]string, error) {
f, err := lambdaClient.ListFunctions(context.Background(), &lambda.ListFunctionsInput{})
if err != nil {
return nil, err
}
var functions = make([]string, len(f.Functions))
for i, functionName := range f.Functions {
functions[i] = *functionName.FunctionName
}
continuationToken := f.NextMarker
for continuationToken != nil {
f, err = lambdaClient.ListFunctions(context.Background(), &lambda.ListFunctionsInput{Marker: continuationToken})
if err != nil {
return nil, err
}
continuationToken = f.NextMarker
for _, functionName := range f.Functions {
functions = append(functions, *functionName.FunctionName)
}
}
return functions, nil
}
func DescribeLambda(lambdaName string) (*lambda.GetFunctionOutput, error) {
function, err := lambdaClient.GetFunction(context.Background(), &lambda.GetFunctionInput{FunctionName: aws.String(lambdaName)})
if err != nil {
return nil, err
}
return function, nil
}
func ListTags(lambdaARN string) (*lambda.ListTagsOutput, error) {
return lambdaClient.ListTags(context.Background(), &lambda.ListTagsInput{
Resource: aws.String(lambdaARN),
})
}
Then you can use the ListLambdas method in order to list all your lambda. After, you can iterate the slice returned and call the DescribeLambda method in order to get the lambdaARN, then you can call the ListTags.
You can refer to the following repository in order to understand how to work with AWS (lambda, S3, glue etc etc) in Golang: https://github.com/alessiosavi/GoGPUtils/tree/master/aws

How do I get pagination working with exclusiveStartKey for dynamodb aws-sdk-go-v2?

I'm trying to create a pagination endpoint for a dynamodb table I have. But I've tried everything to get the exclusiveStartKey to be the correct type for it to work. However, everything I've tried doesn't seem to work.
example code:
func GetPaginator(tableName string, limit int32, lastEvaluatedKey string) (*dynamodb.ScanPaginator, error) {
svc, err := GetClient()
if err != nil {
logrus.Error(err)
return nil, err
}
input := &dynamodb.ScanInput{
TableName: aws.String(tableName),
Limit: aws.Int32(limit),
}
if lastEvaluatedKey != "" {
input.ExclusiveStartKey = map[string]types.AttributeValue{
"id": &types.AttributeValueMemberS{
Value: lastEvaluatedKey,
},
}
}
paginator := dynamodb.NewScanPaginator(svc, input)
return paginator, nil
}
Edit:
Okay so I'm creating a API that requires pagination. The API needs to have a query parameter where the lastEvaluatedId can be defined. I can then use the lastEvaluatedId to pass as the ExclusiveStartKey on the ScanInput. However when I do this I still received the same item from the database. I've created a test.go file and will post the code below:
package main
import (
"context"
"fmt"
"os"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
"github.com/aws/aws-sdk-go-v2/service/dynamodb"
)
type PaginateID struct {
ID string `dynamodbav:"id" json:"id"`
}
func main() {
lastKey := PaginateID{ID: "ae82a99d-486e-11ec-a7a7-0242ac110002"}
key, err := attributevalue.MarshalMap(lastKey)
if err != nil {
fmt.Println(err)
return
}
cfg, err := config.LoadDefaultConfig(context.TODO(), func(o *config.LoadOptions) error {
o.Region = os.Getenv("REGION")
return nil
})
if err != nil {
fmt.Println(err)
return
}
svc := dynamodb.NewFromConfig(cfg, func(o *dynamodb.Options) {
o.EndpointResolver = dynamodb.EndpointResolverFromURL("http://localhost:8000")
})
input := &dynamodb.ScanInput{
TableName: aws.String("TABLE_NAME"),
Limit: aws.Int32(1),
ExclusiveStartKey: key,
}
paginator := dynamodb.NewScanPaginator(svc, input)
if paginator.HasMorePages() {
data, err := paginator.NextPage(context.TODO())
if err != nil {
fmt.Println(err)
return
}
fmt.Println(data.Items[0]["id"])
fmt.Println(data.LastEvaluatedKey["id"])
}
}
When I run this test code. I get this output:
&{ae82a99d-486e-11ec-a7a7-0242ac110002 {}}
&{ae82a99d-486e-11ec-a7a7-0242ac110002 {}}
So the item that is returned is the same Id that I am passing to the ScanInput.ExclusiveStartKey. Which means it's not starting from the ExclusiveStartKey. The scan is starting from the beginning everytime.
The aws-sdk-go-v2 DynamoDB query and scan paginator constructors have a bug (see my github issue, includes the fix). They do not respect the ExclusiveStartKey param.
As an interim fix, I copied the paginator type locally and added one line in to the constructor: nextToken: params.ExclusiveStartKey.
so basically what you need to do is to get the LastEvaluatedKey and to pass it to ExclusiveStartKey
you can not use the scan paginator attributes because it's not exported attributes, therefore instead I suggest that you use the returned page by calling NextPage
in the following snippet I have an example :
func GetPaginator(ctx context.Context,tableName string, limit int32, lastEvaluatedKey map[string]types.AttributeValue) (*dynamodb.ScanOutput, error) {
svc, err := GetClient()
if err != nil {
logrus.Error(err)
return nil, err
}
input := &dynamodb.ScanInput{
TableName: aws.String(tableName),
Limit: aws.Int32(limit),
}
if len(lastEvaluatedKey) > 0 {
input.ExclusiveStartKey = lastEvaluatedKey
}
paginator := dynamodb.NewScanPaginator(svc, input)
return paginator.NextPage(), nil
}
keep in mind that paginator.NextPage(ctx) could be nil incase there is no more pages or you can use HasMorePages()

Go GORM Mocking Begin Expected

I am trying to test a function that gets some details then update details in db. I am using gorm for my ORM and mocking db exec with DATA-DOG/sqlmock. But I keep getting expected begin error but not sure where I am messing up. I tried many variations of the code where I remove the expectBegin expectCommit etc.
Here is the code I am trying to test:
// UpsertUserProfile saves the user if doesn't exists and adds the OAuth profile
// and updates existing user info if record exist in db
func (o *ORM) UpsertUserProfile(gu *goth.User) (*models.User, error) {
db := o.DB
up := &models.UserProfile{}
u, err := models.GothUserToDBUser(gu, false)
if err != nil {
return nil, err
}
tx := db.Where("email = ?", gu.Email).First(u)
if tx.Error != nil && tx.Error != gorm.ErrRecordNotFound {
return nil, tx.Error
}
if tx := db.Model(u).Save(u); tx.Error != nil {
return nil, tx.Error
}
tx = db.Where("email = ? AND provider = ? AND external_user_id = ?", gu.Email, gu.Provider, gu.UserID).First(up)
if tx.Error != nil && tx.Error != gorm.ErrRecordNotFound {
return nil, tx.Error
}
up, err = models.GothUserToDBUserProfile(gu, false)
if err != nil {
return nil, err
}
up.User = *u
if tx := db.Model(up).Save(up); tx.Error != nil {
return nil, tx.Error
}
return u, nil
}
And here is the test code
func TestORM_UpsertUserProfile(t *testing.T) {
mockdb, mock, err := sqlmock.New()
if err != nil {
t.Fatalf("an error '%s' was not expected when opening a stub database connection", err)
}
gormDB, err := gorm.Open(postgres.New(postgres.Config{
Conn: mockdb,
}), &gorm.Config{})
if err != nil {
t.Fatal(err.Error())
}
defer mockdb.Close()
id, _ := uuid.NewV4()
rows := sqlmock.NewRows([]string{"id", "first_name", "last_name"}).
AddRow(id.String(), "fname", "lname")
profileRows := sqlmock.NewRows([]string{"email"}).AddRow("email")
// Test cases
t.Run("success", func(t *testing.T) {
o := &ORM{
DB: gormDB,
}
now := time.Now()
fName := "fname"
lName := "lname"
wantUsr := models.User{
Email: "email",
FirstName: &fName,
LastName: &lName,
BaseModelSoftDelete: models.BaseModelSoftDelete{
BaseModel: models.BaseModel{
ID: id,
CreatedAt: &now,
UpdatedAt: &now,
},
},
}
mock.ExpectQuery(regexp.QuoteMeta(`SELECT * FROM "users"`)).WithArgs("email").WillReturnRows(rows)
mock.ExpectBegin()
mock.ExpectExec(regexp.QuoteMeta(`UPDATE`)).WillReturnResult(sqlmock.NewResult(1, 1))
mock.ExpectCommit()
mock.ExpectBegin()
mock.ExpectQuery(regexp.QuoteMeta(`SELECT * FROM "user_profiles"`)).WithArgs("email", "provider", "testID").WillReturnRows(profileRows)
mock.ExpectCommit()
mock.ExpectBegin()
got, err := o.UpsertUserProfile(&goth.User{Email: "email", UserID: "testID", Provider: "provider"})
if (err != nil) != false {
t.Errorf("ORM.UpsertUserProfile() error = %v, wantErr %v", err, false)
return
}
if !reflect.DeepEqual(got, wantUsr) {
t.Errorf("ORM.UpsertUserProfile() = %v, want %v", got, wantUsr)
}
})
}
but the test keeps returning this error
SELECT * FROM "user_profiles" WHERE email = 'email' AND provider = 'provider' AND external_user_id = 'testID' ORDER BY "user_profiles"."id" LIMIT 1
/Users/public/Projects/go-rest-service/internal/orm/orm_test.go:260: ORM.UpsertUserProfile() error = call to Query 'SELECT * FROM "user_profiles" WHERE email = $1 AND provider = $2 AND external_user_id = $3 ORDER BY "user_profiles"."id" LIMIT 1' with args [{Name: Ordinal:1 Value:email} {Name: Ordinal:2 Value:provider} {Name: Ordinal:3 Value:testID}], was not expected, next expectation is: ExpectedBegin => expecting database transaction Begin, wantErr false

Insert a slice of items in DynamoDB

Is it possible to insert a slice of struct (item) into dynamoDB? Currently, I can only insert one item. Here is how my current function does this.
func Insert(doc interface{}) error {
av, err := dynamodbattribute.MarshalMap(doc)
if err != nil {
return err
}
input := &dynamodb.PutItemInput{
Item: av,
TableName: "schools",
}
_, err = svc.PutItem(input)
if err != nil {
return err
}
return nil
}
How do I adjust the Insert function to insert a slice of items?
func InsertMulti(doc []interface{}) error { - what should be here?
Here is what I have been able to do
func InsertMulti (doc []interface{}) error {
for _, v := range doc {
av, err := dynamodbattribute.MarshalMap(doc)
if err != nil {
return err
}
}
input := &dynamodb.BatchWriteItemInput{
RequestItems: map[string][]*dynamodb.WriteRequest{
"schools" : {
}
}
}
}