I have the following iteration state defined in a Map State:
"WriteRteToDB": {
"Comment": "Write Rte to DB. Also records the risk calculations in the same table.",
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"End": true,
"Parameters": {
"FunctionName": "logger-lambda",
"RtInfo.$": "States.Array($)",
"ExecutionId.$": "$$.Execution.Id",
"InitTime.$": "$$.Execution.StartTime"
}
The parameters defined produce the following input:
{
"FunctionName": "logger-lambda",
"RtInfo": {
"status": 200,
"rte": {
"date": "2022-06-05 00:00:00",
"rt_value": 778129128.6631782,
"lower_80": 0,
"upper_80": 0.5,
"location_id": "WeWork Office Space & Coworking, Town Square, Alpharetta, GA, USA",
"syndrome": "Gastrointestinal"
}
},
"InitTime": "2022-06-05T15:04:57.297Z",
"ExecutionId": "arn:aws:states:us-east-1:1xxxxxxxxxx1:execution:RadaRx-rteForecast:0dbf2743-abb5-e0b6-56d0-2cc82a24e3b4"
}
But the following Error is produced:
{
"error": "States.Runtime",
"cause": "An error occurred while executing the state 'WriteRteToDB' (entered at the event id #28). The Parameters '{\"FunctionName\":\"logger-lambda\",\"RtInfo\":[{\"status\":200,\"rte\":{\"date\":\"2022-12-10 00:00:00\",\"rt_value\":1.3579795204795204,\"lower_80\":0,\"upper_80\":0.5,\"location_id\":\"Atlanta Tech Park, Technology Parkway, Peachtree Corners, GA, USA\",\"syndrome\":\"Influenza Like Illnesses\"}}],\"InitTime\":\"2022-06-05T16:06:10.132Z\",\"ExecutionId\":\"arn:aws:states:us-east-1:1xxxxxxxxxx1:execution:RadaRx-rteForecast:016a37f2-d01c-9bfd-dc3f-1288fb7c1af6\"}' could not be used to start the Task: [The field \"RtInfo\" is not supported by Step Functions]"
}
I have already tried wrapping the RtInfo inside an array of length 1 as you can observe from above, considering that it is a state within the Map State. I have also checked Input size to make sure that it does not cross the Max Input/Output quota of 256KB.
Your task's Parameters has incorrect syntax. Pass RtInfo and the other user-defined inputs under the Payload key:
"Parameters": {
"FunctionName": "logger-lambda",
"Payload": {
"RtInfo.$": "States.Array($)",
"ExecutionId.$": "$$.Execution.Id",
"InitTime.$": "$$.Execution.StartTime"
}
}
Related
i have the following chain: lambda->sns->sqs-> ec2 script->dynamodb table.
the test event for lambda
{
"body": {
"MessageAttributes": {
"vote": {
"Type": "String",
"StringValue": "b"
},
"voter": {
"Type": "String",
"StringValue": "count"
}
}
}
}
originally the table should have 1 partition and 2 attributes - a and b, created with this json code:
{
"voter": {
"S": "count"
},
"a": {
"N": "11"
},
"b": {
"N": "20"
}
}
the issue is in ec2 script that runs continuously. It has a function update_count(vote) that should just increase the existing a or b by 1. The code:
def update_count(vote):
logging.info('update count....')
table.update_item(
Key={'voter': 'count'},
UpdateExpression="ADD #vote = #vote + :incr",
ExpressionAttributeNames={'#vote': vote},
ExpressionAttributeValues={':incr': 1}
)
when i send my test event it overwrites my dynamodb table and it has the following structure:
An UpdateItem will never overwrite attributes which are not set in the UpdateExpression.
I would ensure that you have no other code which is inadvertantly overwriting your item, perhaps from a Lambda function listening to a stream, or another function being called unknowingly in your application.
You can enable DynamoDB Dataplane logs on Cloudtrail to understand what is happening:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/logging-using-cloudtrail.html
In AWS Step Functions, we can use parameters from previous steps as inputs using the following syntax.
"Parameters": {
"Details": {
"weight.$": "$.product.weight",
"unit": "grams"
}
}
As an example, say product.weight. is 500. Can I configure a parameter that will receive a value of 500 grams?
For example, something like this:
"Parameters": {
"Details": {
"formatted_weight.$": "{$.product.weight} grams",
}
}
Use the States.Format intrinsic function to interpolate a string:
"Parameters": {
"Details": {
"formatted_weight.$": "States.Format('{} grams', $.product.weight)",
}
}
I found the pricing documentation ambiguous with regard to the Map states that fan out based on an array in the input. Does anyone know if each fan out ends up being counted as a "state transition" incurring the $0.025 cost? Here is an example input and state machine definition for reference.
Input:
{ "data": [
// Is each of these going to be a "state transition"?
{ "name": "a" },
{ "name": "b" },
] }
Definition:
{
"StartAt": "Start",
"States": {
"Start": {
"Type": "Map",
"ItemsPath": "$.data",
"End": true,
"Iterator": {
"StartAt": "Monitor",
"States": {
"Monitor": {
"Type": "Task",
"Resource": "some-lambda",
"End": true
}
}
}
}
}
}
The cost of Step Functions State Transitions is:
$0.000025 PER STATE TRANSITION THEREAFTER
$0.025 per 1,000 state transitions
With AWS Step Functions, you pay for the number state transitions.
So let's see how many state transitions will Map State creates.
AWS Step Functions only charge customers for events that ends with Entered
For each Map State we have at least these 4 state transitions:
MapStateEntered (Counted as state transition)
MapStateStarted (Not Counted)
MapStateSucceeded (Not Counted)
MapStateExited (Not Counted)
And for each iterations of map state we have these 2 state transitions:
MapIterationStarted (Not Counted)
MapIterationSucceeded (Not Counted)
So for a Map State we can assume the cost is defined by:
cost = (1 + iterations * (steps inside iteration) ) * $0.000025
So for your example (An execution with a Map state with 2 iterations), the overhead of Map State is:
transitions: 1+2*1 = 3
cost: 3 * 0.000025 = $0.000075
I am currently using Hyperledger Fabric. I am using the REST API to make GET request as so:
curl 172.18.0.3:7050/chain/blocks/31
And the output I am getting back is :
{
"transactions": [
{
"type": 2,
"chaincodeID": "BMBQHHg2y0RnadYEaZZT8icjMvZbDPjkn5mFb+clFORxJqz8qsMs/QlalCT+A3msuc59KYM5sbZyhM3OeSplTWo91WAHTUgqIKVrm1gUzsouBIqLNvpqgimN36+s0ywF0Rx4gn27RmQYBbB+877Nh+w7A8Ezz92T1MgHcmzfRgVaDmiN0ga+jAfufNYglmeM4ZSysmSsz6xJtrcD5mTmHXZtvtw6uGCI1TCOMBaWTpLhNHfM2/5EB5jatdMjDi1GAlaXkDWcLgGjScL1yZpWcntz/N0cT90r6i9ycXZ0kk9wodBq2cFutDTdkl8S90kzd0gXig==",
"payload":"BBYZD6S/hRILcf21zVbhMAhA+qLQvAq+KvOBuXOknPCAMjas2LI9f42AKG6r+uWP71LYEkbo1XXANuDmukZjDsFGltzoIfq+Mry5n/CNXzXgiVLX0J7z08kGfEfw2vnywgmVFX4UtKPpl8pMTmRxJWn5Q0HY1pFnA6ZaXluoLRf7f17Ko4SPahi19k2NszcJ0SHE7xRllfLXZJxaOlT2J56nqjTBKTJ86bdqn6AdQXHA6Px7yz5XpgJhccyecaLS4sYcsrqHoOlO+kk+bw5Q6qnkHfIIhLXCEgHxKoT00L8I8B2luO1RlmQd4mNfXb7GrLOJXvCNPrcpSEmQDByEGwn1j3Zy0lilwKVaNYTPNThMwQ==",
"txid": "72bd2ab7-f769-49c9-a754-c7be0c481cf0",
"timestamp": {
"seconds": 1496062124,
"nanos": 474977395
},
"confidentialityLevel": 1,
"confidentialityProtocolVersion": "1.2",
"nonce": "2YgU+0WYPuTKGsKkT1hx7McOURPTIRgG",
"toValidators": "BJWJi5aSycSaJBaLIciUxlhZNyRsW6es2pO7ljUmqxP2SLzgUJtDtAeG8S5SMq+RQ9iX9m8+HIUocrD2J1MBTJaxPWcs/dYFNp1zi8k1ogbEuIQJDe/Gb0mbYVoBqGgFjofiE2lrZTO+RBVmUBQkAoybloOMUSfMawpOPTt/cIeNBq3M+t6gbTSl0ZVs5ofITWtonwhG8PNnlZwEmTLkC7evX1ImivMqo47ONxHXJlbbtjf+pL5kaqU5DrXWiv2L6Wt0xc11od4rbotnAQP2w2dqKTy2fj4ON6qCBp8i+t2FRi/iO0INJpI0aDjdkVCR",
"cert": "MIICUjCCAfigAwIBAgIRAJOBK8HG3E/Pmw8fZwL4iuswCgYIKoZIzj0EAwMwMTELMAkGA1UEBhMCVVMxFDASBgNVBAoTC0h5cGVybGVkZ2VyMQwwCgYDVQQDEwN0Y2EwHhcNMTcwNTI5MTEyNzQwWhcNMTcwODI3MTEyNzQwWjBFMQswCQYDVQQGEwJVUzEUMBIGA1UEChMLSHlwZXJsZWRnZXIxIDAeBgNVBAMTF1RyYW5zYWN0aW9uIENlcnRpZmljYXRlMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE6QVLJ48eCVlS1S8/BiSTU1XiWR0tZ6NGF3OZr306sTcgG/nYtcjx6/yJNwDgdYz5Boi7sA2QWUcqUkWfIPNWPKOB3DCB2TAOBgNVHQ8BAf8EBAMCB4AwDAYDVR0TAQH/BAIwADANBgNVHQ4EBgQEAQIDBDAPBgNVHSMECDAGgAQBAgMEME0GBioDBAUGBwEB/wRArYyx9l4zJL4TbxDHuGZBsJ545Jsph/D/Q/FgMTTtxPh93B+LV6AI1tyFVHWiKNS4GgvDVlmgfwFuMAca+/PaujBKBgYqAwQFBggEQPEdAS1h/9LJJmqriV+42k0bL+ghGFbHa5GiEAitiMjlduiwgfelPK/rbAq0a6NrnPXCEYe1aWCSqyqsEfHGBoIwCgYIKoZIzj0EAwMDSAAwRQIhAOidYaESZ3xyZBTgcBOm3zyXvGb4YCCt7I7+M0gZF4xzAiAgYuCf7FPGx3fnJdABlZjszA1pR6jaPtIOQN2ndfAFZA==",
"signature": "MEUCIQCVBtfjk3yzwfOFyOojH5tynq3HrG7dFN9URXB5C6kYDAIgLPcwJBAIVlD1I4dxzczfxmywlZn1ZMSvL2djioWgqFQ="
}
],
"stateHash": "9KEsiBp4t/VZyETXMASSYtuPuf8JowktCSbX7daPt69uqDzrJvifrPIXpI5N1kOayoq6H0afM8zN/WZpWsesHQ==",
"previousBlockHash": "v6Fo6SARD0xdE0B/jvIq22kgV5uLAKhTwLjrA4YRBskWcZOjECFbNgzlwFQhEmbar1zcAbcZVo9eo/3tx2y68g==",
"consensusMetadata": "CCA=",
"nonHashData": {
"localLedgerCommitTimestamp": {
"seconds": 1496062125,
"nanos": 496018341
},
"chaincodeEvents": [
{
},
{
}
]
}
}
So I had performed a invoke to
transfer 10 from a to b.
And i got this payload.
The payload is encrypted as the
CORE_SECURITY_ENABLED=true and
CORE_SECURITY_PRIVACY=true
I know we have to use the certificate to decrypt the payload and then might be use base64 decoding to get the exact payload back.
But my question is what are the exact function calls or exact steps involved in doing so ?
I am working on AWS Elasticsearch. It doesn't allow open/close index, so setting change can not be applied on the index.
In order to change the setting of a index, I have to create a new index with new setting and then move the data from the old index into new one.
So first I created a new index with
PUT new_index
{
"settings": {
"max_result_window":3000000,
"analysis": {
"filter": {
"german_stop": {
"type": "stop",
"stopwords": "_german_"
},
"german_keywords": {
"type": "keyword_marker",
"keywords": ["whatever"]
},
"german_stemmer": {
"type": "stemmer",
"language": "light_german"
}
},
"analyzer": {
"my_german_analyzer": {
"tokenizer": "standard",
"filter": [
"lowercase",
"german_stop",
"german_keywords",
"german_normalization",
"german_stemmer"
]
}
}
}
}
}
it succeeded. Then I try to move data from old index into new one with query:
POST _reindex
{
"source": {
"index": "old_index"
},
"dest": {
"index": "new_index"
}
}
It failed with
Request failed to get to the server (status code: 504)
I checked the indices with _cat api, it gives
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open old_index AGj8WN_RRvOwrajKhDrbPw 5 1 2256482 767034 7.8gb 7.8gb
yellow open new_index WnGZ3GsUSR-WLKggp7Brjg 5 1 52000 0 110.2mb 110.2mb
Seemingly some data are loaded into there, just wondering why the _reindex doesn't work.
You can check the status of reindex with api:
GET _tasks?detailed=true&actions=*reindex
There is a "status" object in response which has field "total":
total is the total number of operations that the reindex expects to perform. You can estimate the progress by adding the updated, created, and deleted fields. The request will finish when their sum is equal to the total field.
Link to ES Documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html