i'm using Akka on one of my projects and i need to get the state of an actor, the way i'm doing it is as follows.
a REST request comes in
#GET
#Produces(Array(MediaType.APPLICATION_JSON))
def get() = {
try {
Await.result((getScanningActor ? WorkInfo), 5.second).asInstanceOf[ScanRequest]
}
catch{
case ex: TimeoutException => {
RequestTimedOut()
}
}
}
on the actor i respond with the current work state
case WorkInfo => sender ! currentWork
for some reason the first time i call this function i get the correct value, on the following requests i get the same value i received on the first call
I'm also using DCEVM if that makes any difference.
Related
I have a lambda function which does a series of actions. I have a react application which triggers the lambda function.
Is there a way I can send a partial response from the lambda function after each action is complete.
const testFunction = (event, context, callback) => {
let partialResponse1 = await action1(event);
// send partial response to client
let partialResponse2 = await action2(partialResponse1);
// send partial response to client
let partialResponse3 = await action3(partialResponse2);
// send partial response to client
let response = await action4(partialResponse3);
// send final response
}
Is this possible in lambda functions? If so, how we can do this. Any ref docs or sample code would be do a great help.
Thanks.
Note: This is fairly a simple case of showing a loader with % on the client-side. I don't want to overcomplicate things SQS or step functions.
I am still looking for an answer for this.
From what I understand you're using API Gateway + Lambda and are looking to show the progress of the Lambda via UI.
Since each step must finish before the next step begin I see no reason not to call the lambda 4 times, or split the lambda to 4 separate lambdas.
E.g.:
// Not real syntax!
try {
res1 = await ajax.post(/process, {stage: 1, data: ... });
out(stage 1 complete);
res2 = await ajax.post(/process, {stage: 2, data: res1});
out(stage 2 complete);
res3 = await ajax.post(/process, {stage: 3, data: res2});
out(stage 3 complete);
res4 = await ajax.post(/process, {stage: 4, data: res3});
out(stage 4 complete);
out(process finished);
catch(err) {
out(stage {$err.stage-number} failed to complete);
}
If you still want all 4 calls to be executed during the same lambda execution you may do the following (this especially true if the process is expected to be very long) (and because it's usually not good practice to execute "long hanging" http transaction).
You may implement it by saving the "progress" in a database, and when the process is complete save the results to the database as well.
All you need to do is query the status every X seconds.
// Not real syntax
Gateway-API --> lambda1 - startProcess(): returns ID {
uuid = randomUUID();
write to dynamoDB { status: starting }.
send sqs-message-to-start-process(data, uuid);
return response { uuid: uuid };
}
SQS --> lambda2 - execute(): returns void {
try {
let partialResponse1 = await action1(event);
write to dynamoDB { status: action 1 complete }.
// send partial response to client
let partialResponse2 = await action2(partialResponse1);
write to dynamoDB { status: action 2 complete }.
// send partial response to client
let partialResponse3 = await action3(partialResponse2);
write to dynamoDB { status: action 3 complete }.
// send partial response to client
let response = await action4(partialResponse3);
write to dynamoDB { status: action 4 complete, response: response }.
} catch(err) {
write to dynamoDB { status: failed, error: err }.
}
}
Gateway-API --> lambda3 -> getStatus(uuid): returns status {
return status from dynamoDB (uuid);
}
Your UI Code:
res = ajax.get(/startProcess);
uuid = res.uuid;
in interval every X (e.g. 3) seconds:
status = ajax.get(/getStatus?uuid=uuid);
show(status);
if (status.error) {
handle(status.error) and break;
}
if (status.response) {
handle(status.response) and break;
}
}
Just remember that lambda's cannot exceed 15 minutes execution. Therefore, you need to be 100% certain that whatever the process does, it never exceeds this hard limit.
What you are looking for is to have response expose as a stream where you can write to the stream and flush it
Unfortunately its not there in Node.js
How to stream AWS Lambda response in node?
https://docs.aws.amazon.com/lambda/latest/dg/programming-model.html
But you can still do the streaming if you use Java
https://docs.aws.amazon.com/lambda/latest/dg/java-handler-io-type-stream.html
package example;
import java.io.InputStream;
import java.io.OutputStream;
import com.amazonaws.services.lambda.runtime.RequestStreamHandler;
import com.amazonaws.services.lambda.runtime.Context;
public class Hello implements RequestStreamHandler{
public void handler(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
int letter;
while((letter = inputStream.read()) != -1)
{
outputStream.write(Character.toUpperCase(letter));
}
}
}
Aman,
You can push the partial outputs into SQS and read the SQS messages to process those message. This is a simple and scalable architecture. AWS provides SQS SDKs in different languages, for example, JavaScript, Java, Python, etc.
Reading and writing into SQS is very easy using SDK and that too can be implemented in serverside or in your UI layer (with proper IAM).
I found AWS step function may be what you need:
AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly.
Check this link for more detail:
In our example, you are a developer who has been asked to create a serverless application to automate handling of support tickets in a call center. While you could have one Lambda function call the other, you worry that managing all of those connections will become challenging as the call center application becomes more sophisticated. Plus, any change in the flow of the application will require changes in multiple places, and you could end up writing the same code over and over again.
I have Lambda function tranportKickoff which receives an input and then sends/proxies that input forward into a Step Function. The code below does run and I am getting no errors but at the same time the step function is NOT executing.
Also critical to the design, I do not want the transportKickoff function to wait around for the step function to complete as it can be quite long running. I was, however, expecting that any errors in the calling of the Step Function would be reported back synchronously. Maybe this thought is at fault and I'm somehow missing out on an error that is thrown somewhere. If that's the case, however, I'd like to find a way which is able to achieve the goal of having the kickoff lambda function exit as soon as the Step Function has started execution.
note: I can execute the step function independently and I know that it works correctly
const stepFn = new StepFunctions({ apiVersion: "2016-11-23" });
const stage = process.env.AWS_STAGE;
const name = `transport-steps ${message.command} for "${stage}" environment at ${Date.now()}`;
const params: StepFunctions.StartExecutionInput = {
stateMachineArn: `arn:aws:states:us-east-1:999999999:stateMachine:transportion-${stage}-steps`,
input: JSON.stringify(message),
name
};
const request = stepFn.startExecution(params);
request.send();
console.info(
`startExecution request for step function was sent, context sent was:\n`,
JSON.stringify(params, null, 2)
);
callback(null, {
statusCode: 200
});
I have also checked from the console that I have what I believe to be the right permissions to start the execution of a step function:
I've now added more permissions (see below) but still experiencing the same problem:
'states:ListStateMachines'
'states:CreateActivity'
'states:StartExecution'
'states:ListExecutions'
'states:DescribeExecution'
'states:DescribeStateMachineForExecution'
'states:GetExecutionHistory'
Ok I have figured this one out myself, hopefully this answer will be helpful for others:
First of all, the send() method is not a synchronous call but it does not return a promise either. Instead you must setup listeners on the Request object before sending so that you can appropriate respond to success/failure states.
I've done this with the following code:
const stepFn = new StepFunctions({ apiVersion: "2016-11-23" });
const stage = process.env.AWS_STAGE;
const name = `${message.command}-${message.upc}-${message.accountName}-${stage}-${Date.now()}`;
const params: StepFunctions.StartExecutionInput = {
stateMachineArn: `arn:aws:states:us-east-1:837955377040:stateMachine:transportation-${stage}-steps`,
input: JSON.stringify(message),
name
};
const request = stepFn.startExecution(params);
// listen for success
request.on("extractData", req => {
console.info(
`startExecution request for step function was sent and validated, context sent was:\n`,
JSON.stringify(params, null, 2)
);
callback(null, {
statusCode: 200
});
});
// listen for error
request.on("error", (err, response) => {
console.warn(
`There was an error -- ${err.message} [${err.code}, ${
err.statusCode
}] -- that blocked the kickoff of the ${message.command} ITMS command for ${
message.upc
} UPC, ${message.accountName} account.`
);
callback(err.statusCode, {
message: err.message,
errors: [err]
});
});
// send request
request.send();
Now please bear in mind there is a "success" event but I used "extractData" to capture success as I wanted to get a response as quickly as possible. It's possible that success would have worked equally as well but looking at the language in the Typescript typings it wasn't entirely clear and in my testing I'm certain that the "extractData" method does work as expected.
As for why I was not getting any execution on my step functions ... it had to the way I was naming the function ... you're limited to a subset of characters in the name and I'd stepped over that restriction but didn't realize until I was able to capture the error with the code above.
For anyone encountering issues executing state machines from Lambda's make sure the permission 'states:StartExecution' is added to the Lambda permissions and the regions match up.
Promise based version:
import { StepFunctions } from 'aws-sdk';
const clients = {
stepFunctions: new StepFunctions();
}
const createExecutor = ({ clients }) => async (event) => {
console.log('Executing media pipeline job');
const params = {
stateMachineArn: '<state-machine-arn>',
input: JSON.stringify({}),
name: 'new-job',
};
const result = await stepFunctions.startExecution(params).promise();
// { executionArn: "string", startDate: number }
return result;
};
const startExecution = createExecutor({ clients });
// Pass in the event from the Lambda e.g S3 Put, SQS Message
await startExecution(event);
Result should contain the execution ARN and start date (read more)
I want to send notifications to clients via websockets. This notifications are generated by actors, hence I'm trying to create a stream of actor's messages at server startup and subscribe websockects connections to this stream (sending only those notifications emitted since subscription)
With Source.actorRef we can create a Source of actor messages.
val ref = Source.actorRef[Weather](Int.MaxValue, fail)
.filter(!_.raining)
.to(Sink foreach println )
.run()
ref ! Weather("02139", 32.0, true)
But how can I subscribe (akka http*) websockets connections to this source if has been materialized already?
*WebSockets connections in Akka HTTP requires a Flow[Message, Message, Any]
What I'm trying to do is something like
// at server startup
val notifications: Source[Notification,ActorRef] = Source.actorRef[Notificacion](5,OverflowStrategy.fail)
val ref = notifications.to(Sink.foreach(println(_))).run()
val notificationActor = system.actorOf(NotificationActor.props(ref))
// on ws connection
val notificationsWS = path("notificationsWS") {
parameter('name) { name ⇒
get {
onComplete(flow(name)){
case Success(f) => handleWebSocketMessages(f)
case Failure(e) => throw e
}
}
}
}
def flow(name: String) = {
val messages = notifications filter { n => n.name equals name } map { n => TextMessage.Strict(n.data) }
Flow.fromSinkAndSource(Sink.ignore, notifications)
}
This doensn't work because the notifications source is not the one that is materialized, hence it doens't emit any element.
Note: I was using Source.actorPublisher and it was working but ktoso discourages his usage and also I was getting this error:
java.lang.IllegalStateException: onNext is not allowed when the stream has not requested elements, totalDemand was 0.
You could expose the materialised actorRef to some external router actor using mapMaterializedValue.
Flow.fromSinkAndSourceMat(Sink.ignore, notifications)(Keep.right)
.mapMaterializedValue(srcRef => router ! srcRef)
The router can keep track of your sources actorrefs (deathwatch can help tidying things up) and forward messages to them.
NB: you're probably already aware, but note that by using Source.actorRef to feed your flow, your flow will not be backpressure aware (with the strategy you chose it will just crash under load).
I'm an akka noob so apologies!
I'm playing around with a system that uses Spray and Akka.
I'm using the following code snippet to send a message to another actor.
It uses ask which, from what I understand will return a future which is resolved in "mapTo" and "map". I then return the result to the users using Sprays "complete" directive.
val response = (worker ? Create(json))
.mapTo[Ok]
.map(result => s"I got a response: ${result}")
.recover { case _ => "error" }
complete(response)
My question is, since this is a future, do I need to be worried about sending the correct response to the client? In some code samples I see examples where the actorRef to reply to is sent as part of the request...
// set reply to actor
val replyTo = sender() // important to not close over sender()
// create actor that collects replies from workers
val aggregator = context.actorOf(Props(
classOf[StatsAggregator], words.size, replyTo))
Then in the receiving actor...
replyTo ! SendResult
Should I be passing the "replyTo" actor as part of the request or is this all taken care of in the mapTo?
Thanks in advance!
The complete directive will send back a response to http/https client of your service. You don't need to do more than that. Please note that your code swallows errors by making recover on a future. Spray will treat it as a success and will return status code 200.
The last and most importantly, your worker has to reply with Ok message back like this.
class Worker extends Actor {
def receive: Receive = {
case Create(json) =>
//do some staff with json
sender() ! Ok // This Ok message will be passed in the future in spray route
}
}
The replyTo idiom is needed only when worker uses Future internally to process the work load. As it in the following example
class Worker extends Actor {
def recieve: Recieve = {
case Create(json) =>
val future = Future{
//do some staff with json
}
val replyTo = sender()
future.onComplete {
case scala.util.Success(result) =>
replyTo ! Ok
case scala.util.Failure(ex) =>
replyTo ! akka.actor.Status.Failure(ex)
}
}
}
The replyTo is needed to fix actual sender of the message since onComplete may be executed within a different actor context that can point to a different sender resulting in message being sent to a wrong actor.
I am trying to use ember-data to get a simple registration form to save on my server. The call technically works, but the success callback is never trigger on the promise, and I have no idea why.
The server receives the data from the front end and successfully saves it to the database. It then returns status code 201 for CREATED. I can see the successful response happening in the Chrome debugger. But even when the server responds with a successful status, the error callback is triggered on the save's promise. I've confirmed this happens every time by putting a debugger; statement in the error callback.
My router's model is hooked up like this:
model: function() {
return this.store.createRecord('registerUser');
}
And I have a simple register function in my controller:
register: function() {
var self = this;
this.get('model').save().then(function() {
self.transitionToRoute('index');
}, function(resp) {
if (resp.responseJSON) {
self.get('model').set('errors', resp.responseJSON.errors);
}
});
}
Every time my server comes back with a response, success or failure, the failure callback is hit. If I have errors in the response (for invalid data or something), the errors are successfully displayed in the form. I can see the request coming in properly, and the data is stored in the database. So, the save is technically successful, but ember doesn't seem to know that it is even though a successful 201 status is returned from the server (which can be verified in the Chrome debugger).
The only thing I can think of is that ember-data's adapter is doing something that I'm not aware of, but I am just using the default RESTAdapter and haven't touched it. Is there anything else
If it makes a difference, the server is running Play 1.2.5. I don't know if that makes a difference in the response's header or something like that.
Any help would be greatly appreciated. Thank you for your time!
Mike
SOLUTION
So, the issue was to do with the JSON response. The two problems:
I did not include an ID in the response
I did not "wrap" the response in a "registerUser". This is necessary to match the model name.
Below is a valid response:
{
"registerUser": {
"id": 11,
"email": "mike999#test.com",
"password": "12345",
"password2": "12345",
"name": "Mike"
}
}
Ember Data is expecting the model in the response, so sending back a success http status doesn't mean it will hit the success endpoint. When it tries to serialize your response (or lack of response) it's probably failing which would be why it's hitting the failure function. A big reason for the response is the id of the record.
The model returned should be in the following format
{
registerUser:{
id: "123",
attr: "asdf"
}
}
https://github.com/emberjs/data/blob/master/TRANSITION.md