AWS DynamoDB batchWriteItem in Lamda is not working properly - amazon-web-services

Question

I have an app created via AWS Amplify and I created an AWS Lambda function that aim to insert multiple data to one of the table in DynamoDB.
Firstly, I tried map attributes to documentClient.put({...params}).promise() and run Promise.all() on it. But some items got lost. For example, I add 40 items at once, but only 5-10 got added. I thought this could be some lambda limit issue. So I switch to batchWriteItem() and seems it remain the same or even worse (only 1 out of 40 got added). Here is the code I wrote:
export const addAvailabilities = async (
docClient: AWS.DynamoDB.DocumentClient,
newAvailabilities: Availability[],
expertId: string,
cognitoUserId: string
) => {
console.info('#addAvailabilities: Start')
try {
let restArray = [...newAvailabilities]
const promiseArray = []
while (restArray.length > 0) {
const executingArray = restArray.slice(0, 25)
const temp = batchAddAvailability(docClient, executingArray, expertId, cognitoUserId)
promiseArray.push(temp)
restArray = restArray.slice(25)
}
const result = await Promise.all(promiseArray)
console.info(result)
console.info('#addAvailabilities: End')
return result
} catch (err) {
console.error('#addAvailabilities: Error')
throw err
}
}
const mapBatchAddAvailabilityParams = (newAvailabilities: Availability[], expertId: string, cognitoUserId: string) => {
return newAvailabilities.map((availability, index) => {
const currentTime = `${moment().utc().format('YYYY-MM-DD[T]HH:mm:ss.SS')}${index}Z`
return {
PutRequest: {
Item: {
id: uuid(),
__typename: 'ExpertAvailability',
expertId: expertId,
owner: cognitoUserId,
startTime: availability.start,
status: 'available',
createdAt: currentTime,
updatedAt: currentTime
}
}
}
})
}
const batchAddAvailability = async (
docClient: AWS.DynamoDB.DocumentClient,
newAvailabilities: Availability[],
expertId: string,
cognitoUserId: string
) => {
console.info('#batchAddAvailability: Start')
try {
const batchParams = mapBatchAddAvailabilityParams(newAvailabilities, expertId, cognitoUserId)
const param = {
RequestItems: {
[process.env.API_TAP_EXPERTAVAILABILITYTABLE_NAME]: batchParams
}
}
console.info('params', JSON.stringify(param))
return docClient.batchWrite(param).promise()
} catch (err) {
console.error('#batchAddAvailability: Error')
throw err
}
}
I add this const currentTime = ${moment().utc().format('YYYY-MM-DD[T]HH:mm:ss.SS')}${index}Z` because I saw this post has similar issue as mine and that's the solution solve his problem. But it doesn't solve mine.
From the logs, params get format correctly like this:
{
"RequestItems": {
"Availability": [
{
"PutRequest": {
"Item": {
"id": "66a7b63e-a14b-4ba2-94a9-0dd7bf457efe",
"__typename": "ExpertAvailability",
"expertId": "8d30c1a4-685e-40bf-b884-6e50bb422e99",
"owner": "test-keycloak_eeb71cb1-6c11-4fb9-a721-ce5dc7d06269",
"startTime": "2021-10-03T23:00:00.000Z",
"status": "available",
"createdAt": "2021-09-08T10:24:28.880Z",
"updatedAt": "2021-09-08T10:24:28.880Z"
}
}
},
{
"PutRequest": {
"Item": {
"id": "162d839d-7fde-417e-994b-2dc12336c4cf",
"__typename": "ExpertAvailability",
"expertId": "8d30c1a4-685e-40bf-b884-6e50bb422e99",
"owner": "test-keycloak_eeb71cb1-6c11-4fb9-a721-ce5dc7d06269",
"startTime": "2021-10-04T00:00:00.000Z",
"status": "available",
"createdAt": "2021-09-08T10:24:28.881Z",
"updatedAt": "2021-09-08T10:24:28.881Z"
}
}
},
{
"PutRequest": {
"Item": {
"id": "dc257c75-9a27-482a-88c5-1747ffe97361",
"__typename": "ExpertAvailability",
"expertId": "8d30c1a4-685e-40bf-b884-6e50bb422e99",
"owner": "test-keycloak_eeb71cb1-6c11-4fb9-a721-ce5dc7d06269",
"startTime": "2021-10-04T01:00:00.000Z",
"status": "available",
"createdAt": "2021-09-08T10:24:28.932Z",
"updatedAt": "2021-09-08T10:24:28.932Z"
}
}
},
{
"PutRequest": {
"Item": {
"id": "75b2e911-e842-4f11-99ed-702c6cf1c485",
"__typename": "ExpertAvailability",
"expertId": "8d30c1a4-685e-40bf-b884-6e50bb422e99",
"owner": "test-keycloak_eeb71cb1-6c11-4fb9-a721-ce5dc7d06269",
"startTime": "2021-10-04T02:00:00.000Z",
"status": "available",
"createdAt": "2021-09-08T10:24:28.943Z",
"updatedAt": "2021-09-08T10:24:28.943Z"
}
}
},
{
"PutRequest": {
"Item": {
"id": "df65e151-1699-446d-ab3b-06aca707a2fb",
"__typename": "ExpertAvailability",
"expertId": "8d30c1a4-685e-40bf-b884-6e50bb422e99",
"owner": "test-keycloak_eeb71cb1-6c11-4fb9-a721-ce5dc7d06269",
"startTime": "2021-10-04T03:00:00.000Z",
"status": "available",
"createdAt": "2021-09-08T10:24:28.944Z",
"updatedAt": "2021-09-08T10:24:28.944Z"
}
}
},
{
"PutRequest": {
"Item": {
"id": "98a5bdd2-60d3-4d5d-9913-40b5c84c6d62",
"__typename": "ExpertAvailability",
"expertId": "8d30c1a4-685e-40bf-b884-6e50bb422e99",
"owner": "test-keycloak_eeb71cb1-6c11-4fb9-a721-ce5dc7d06269",
"startTime": "2021-10-04T04:00:00.000Z",
"status": "available",
"createdAt": "2021-09-08T10:24:28.945Z",
"updatedAt": "2021-09-08T10:24:28.945Z"
}
}
}
]
}
}
There is no UnprocessedItems or any errors return from the execution. But the items are just missing. The capacity of the table is on-demand so I think capacity shouldn't be a problem. Any ideas what's wrong? Many thanks