I'm running down a rabbit hole so I'm asking my question here.
I have a useMutation mutation that uploads up to 10MB JSON data to my DB. This obviously takes a long time and I received a timeout error which I believed caused the query to run twice which then uploaded some of the data twice.
How can I state for one mutation to have a longer timeout? Where do I specify the timeout? Can I specify it not to be retried?
I am currently using a couple of links:
import { onError } from '#apollo/client/link/error'
import { RetryLink } from '#apollo/client/link/retry'
import { setContext } from '#apollo/client/link/context'
I believe the RetryLink (https://www.apollographql.com/docs/link/links/retry/) may help. Currently I have default settings:
// default settings
const retryLink = new RetryLink({
delay: {
initial: 300,
max: Infinity,
jitter: true,
},
attempts: {
max: 5,
retryIf: (error, _operation) => !!error,
},
})
500 level errors are server side issues. In this case, your server is the one indicating that there is a timeout, so there is nothing to do on the client side. You'll have to check your server configuration and figure it out on that end.
Related
I'm currently working on trying to SSR a page with NextJS and change the links on a component based on the presence of a http-only cookie. I've got it working to find the cookie and pass a prop based on that cookie, but the data is not rehydrating after either addition or removal of that cookie. More specifically:
If you initially land on the page with cookie present it creates the page correctly. If you then remove the cookie and reload the data still acts like the cookie is present.
Vice versa
here is what i'm using to pass the prop:
return {
props: {
seo: { seo: seo },
navData: navItems,
footerData: footerItems.data.navItem || [],
footerCallout: footerCallout.data.footerCallout,
hero: content.heroSection,
customer: ctx.req.cookies.token ? true : false,
customer_id: ctx.req.cookies.token || '',
},
}
I'm also using a useEffect and useState to try and trigger rehydration:
const [isCustomer, setIsCustomer] = useState(customer)
useEffect(() => {
console.log(`customer is ${customer}`)
setIsCustomer(customer)
}, [customer])
One other bit of info, i'm deploying with Netlify, and when i locally use netlify dev and spin up a server everything works. The second it gets deployed on netlify is when i get the above behavior.
I'm not sure what exactly is going on, and it may just be a lack of understanding of Next and SSR. Any help would be greatly appreciated. i've been beating my head against the wall on this for a while.
I have a lambda function which does a series of actions. I have a react application which triggers the lambda function.
Is there a way I can send a partial response from the lambda function after each action is complete.
const testFunction = (event, context, callback) => {
let partialResponse1 = await action1(event);
// send partial response to client
let partialResponse2 = await action2(partialResponse1);
// send partial response to client
let partialResponse3 = await action3(partialResponse2);
// send partial response to client
let response = await action4(partialResponse3);
// send final response
}
Is this possible in lambda functions? If so, how we can do this. Any ref docs or sample code would be do a great help.
Thanks.
Note: This is fairly a simple case of showing a loader with % on the client-side. I don't want to overcomplicate things SQS or step functions.
I am still looking for an answer for this.
From what I understand you're using API Gateway + Lambda and are looking to show the progress of the Lambda via UI.
Since each step must finish before the next step begin I see no reason not to call the lambda 4 times, or split the lambda to 4 separate lambdas.
E.g.:
// Not real syntax!
try {
res1 = await ajax.post(/process, {stage: 1, data: ... });
out(stage 1 complete);
res2 = await ajax.post(/process, {stage: 2, data: res1});
out(stage 2 complete);
res3 = await ajax.post(/process, {stage: 3, data: res2});
out(stage 3 complete);
res4 = await ajax.post(/process, {stage: 4, data: res3});
out(stage 4 complete);
out(process finished);
catch(err) {
out(stage {$err.stage-number} failed to complete);
}
If you still want all 4 calls to be executed during the same lambda execution you may do the following (this especially true if the process is expected to be very long) (and because it's usually not good practice to execute "long hanging" http transaction).
You may implement it by saving the "progress" in a database, and when the process is complete save the results to the database as well.
All you need to do is query the status every X seconds.
// Not real syntax
Gateway-API --> lambda1 - startProcess(): returns ID {
uuid = randomUUID();
write to dynamoDB { status: starting }.
send sqs-message-to-start-process(data, uuid);
return response { uuid: uuid };
}
SQS --> lambda2 - execute(): returns void {
try {
let partialResponse1 = await action1(event);
write to dynamoDB { status: action 1 complete }.
// send partial response to client
let partialResponse2 = await action2(partialResponse1);
write to dynamoDB { status: action 2 complete }.
// send partial response to client
let partialResponse3 = await action3(partialResponse2);
write to dynamoDB { status: action 3 complete }.
// send partial response to client
let response = await action4(partialResponse3);
write to dynamoDB { status: action 4 complete, response: response }.
} catch(err) {
write to dynamoDB { status: failed, error: err }.
}
}
Gateway-API --> lambda3 -> getStatus(uuid): returns status {
return status from dynamoDB (uuid);
}
Your UI Code:
res = ajax.get(/startProcess);
uuid = res.uuid;
in interval every X (e.g. 3) seconds:
status = ajax.get(/getStatus?uuid=uuid);
show(status);
if (status.error) {
handle(status.error) and break;
}
if (status.response) {
handle(status.response) and break;
}
}
Just remember that lambda's cannot exceed 15 minutes execution. Therefore, you need to be 100% certain that whatever the process does, it never exceeds this hard limit.
What you are looking for is to have response expose as a stream where you can write to the stream and flush it
Unfortunately its not there in Node.js
How to stream AWS Lambda response in node?
https://docs.aws.amazon.com/lambda/latest/dg/programming-model.html
But you can still do the streaming if you use Java
https://docs.aws.amazon.com/lambda/latest/dg/java-handler-io-type-stream.html
package example;
import java.io.InputStream;
import java.io.OutputStream;
import com.amazonaws.services.lambda.runtime.RequestStreamHandler;
import com.amazonaws.services.lambda.runtime.Context;
public class Hello implements RequestStreamHandler{
public void handler(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {
int letter;
while((letter = inputStream.read()) != -1)
{
outputStream.write(Character.toUpperCase(letter));
}
}
}
Aman,
You can push the partial outputs into SQS and read the SQS messages to process those message. This is a simple and scalable architecture. AWS provides SQS SDKs in different languages, for example, JavaScript, Java, Python, etc.
Reading and writing into SQS is very easy using SDK and that too can be implemented in serverside or in your UI layer (with proper IAM).
I found AWS step function may be what you need:
AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly.
Check this link for more detail:
In our example, you are a developer who has been asked to create a serverless application to automate handling of support tickets in a call center. While you could have one Lambda function call the other, you worry that managing all of those connections will become challenging as the call center application becomes more sophisticated. Plus, any change in the flow of the application will require changes in multiple places, and you could end up writing the same code over and over again.
I have Lambda function tranportKickoff which receives an input and then sends/proxies that input forward into a Step Function. The code below does run and I am getting no errors but at the same time the step function is NOT executing.
Also critical to the design, I do not want the transportKickoff function to wait around for the step function to complete as it can be quite long running. I was, however, expecting that any errors in the calling of the Step Function would be reported back synchronously. Maybe this thought is at fault and I'm somehow missing out on an error that is thrown somewhere. If that's the case, however, I'd like to find a way which is able to achieve the goal of having the kickoff lambda function exit as soon as the Step Function has started execution.
note: I can execute the step function independently and I know that it works correctly
const stepFn = new StepFunctions({ apiVersion: "2016-11-23" });
const stage = process.env.AWS_STAGE;
const name = `transport-steps ${message.command} for "${stage}" environment at ${Date.now()}`;
const params: StepFunctions.StartExecutionInput = {
stateMachineArn: `arn:aws:states:us-east-1:999999999:stateMachine:transportion-${stage}-steps`,
input: JSON.stringify(message),
name
};
const request = stepFn.startExecution(params);
request.send();
console.info(
`startExecution request for step function was sent, context sent was:\n`,
JSON.stringify(params, null, 2)
);
callback(null, {
statusCode: 200
});
I have also checked from the console that I have what I believe to be the right permissions to start the execution of a step function:
I've now added more permissions (see below) but still experiencing the same problem:
'states:ListStateMachines'
'states:CreateActivity'
'states:StartExecution'
'states:ListExecutions'
'states:DescribeExecution'
'states:DescribeStateMachineForExecution'
'states:GetExecutionHistory'
Ok I have figured this one out myself, hopefully this answer will be helpful for others:
First of all, the send() method is not a synchronous call but it does not return a promise either. Instead you must setup listeners on the Request object before sending so that you can appropriate respond to success/failure states.
I've done this with the following code:
const stepFn = new StepFunctions({ apiVersion: "2016-11-23" });
const stage = process.env.AWS_STAGE;
const name = `${message.command}-${message.upc}-${message.accountName}-${stage}-${Date.now()}`;
const params: StepFunctions.StartExecutionInput = {
stateMachineArn: `arn:aws:states:us-east-1:837955377040:stateMachine:transportation-${stage}-steps`,
input: JSON.stringify(message),
name
};
const request = stepFn.startExecution(params);
// listen for success
request.on("extractData", req => {
console.info(
`startExecution request for step function was sent and validated, context sent was:\n`,
JSON.stringify(params, null, 2)
);
callback(null, {
statusCode: 200
});
});
// listen for error
request.on("error", (err, response) => {
console.warn(
`There was an error -- ${err.message} [${err.code}, ${
err.statusCode
}] -- that blocked the kickoff of the ${message.command} ITMS command for ${
message.upc
} UPC, ${message.accountName} account.`
);
callback(err.statusCode, {
message: err.message,
errors: [err]
});
});
// send request
request.send();
Now please bear in mind there is a "success" event but I used "extractData" to capture success as I wanted to get a response as quickly as possible. It's possible that success would have worked equally as well but looking at the language in the Typescript typings it wasn't entirely clear and in my testing I'm certain that the "extractData" method does work as expected.
As for why I was not getting any execution on my step functions ... it had to the way I was naming the function ... you're limited to a subset of characters in the name and I'd stepped over that restriction but didn't realize until I was able to capture the error with the code above.
For anyone encountering issues executing state machines from Lambda's make sure the permission 'states:StartExecution' is added to the Lambda permissions and the regions match up.
Promise based version:
import { StepFunctions } from 'aws-sdk';
const clients = {
stepFunctions: new StepFunctions();
}
const createExecutor = ({ clients }) => async (event) => {
console.log('Executing media pipeline job');
const params = {
stateMachineArn: '<state-machine-arn>',
input: JSON.stringify({}),
name: 'new-job',
};
const result = await stepFunctions.startExecution(params).promise();
// { executionArn: "string", startDate: number }
return result;
};
const startExecution = createExecutor({ clients });
// Pass in the event from the Lambda e.g S3 Put, SQS Message
await startExecution(event);
Result should contain the execution ARN and start date (read more)
I am trying to use ember-data to get a simple registration form to save on my server. The call technically works, but the success callback is never trigger on the promise, and I have no idea why.
The server receives the data from the front end and successfully saves it to the database. It then returns status code 201 for CREATED. I can see the successful response happening in the Chrome debugger. But even when the server responds with a successful status, the error callback is triggered on the save's promise. I've confirmed this happens every time by putting a debugger; statement in the error callback.
My router's model is hooked up like this:
model: function() {
return this.store.createRecord('registerUser');
}
And I have a simple register function in my controller:
register: function() {
var self = this;
this.get('model').save().then(function() {
self.transitionToRoute('index');
}, function(resp) {
if (resp.responseJSON) {
self.get('model').set('errors', resp.responseJSON.errors);
}
});
}
Every time my server comes back with a response, success or failure, the failure callback is hit. If I have errors in the response (for invalid data or something), the errors are successfully displayed in the form. I can see the request coming in properly, and the data is stored in the database. So, the save is technically successful, but ember doesn't seem to know that it is even though a successful 201 status is returned from the server (which can be verified in the Chrome debugger).
The only thing I can think of is that ember-data's adapter is doing something that I'm not aware of, but I am just using the default RESTAdapter and haven't touched it. Is there anything else
If it makes a difference, the server is running Play 1.2.5. I don't know if that makes a difference in the response's header or something like that.
Any help would be greatly appreciated. Thank you for your time!
Mike
SOLUTION
So, the issue was to do with the JSON response. The two problems:
I did not include an ID in the response
I did not "wrap" the response in a "registerUser". This is necessary to match the model name.
Below is a valid response:
{
"registerUser": {
"id": 11,
"email": "mike999#test.com",
"password": "12345",
"password2": "12345",
"name": "Mike"
}
}
Ember Data is expecting the model in the response, so sending back a success http status doesn't mean it will hit the success endpoint. When it tries to serialize your response (or lack of response) it's probably failing which would be why it's hitting the failure function. A big reason for the response is the id of the record.
The model returned should be in the following format
{
registerUser:{
id: "123",
attr: "asdf"
}
}
https://github.com/emberjs/data/blob/master/TRANSITION.md
I have a unit test where I am checking that a service polls a URL using $http. The first time round I want to return a value that will cause the service wait 10 seconds (using $timeout) and then to poll again. The second time round I want to return a value that will stop the service from polling.
When I do this
httpBackend.expectGET(url).respond(200, { status: 'busy'});
httpBackend.expectGET(ulr).respond(200, { status: 'complete}');
service.poll();
httpBackend.flush();
timeout.flush();
httpBackend.verifyNoAllRequests();
However it never gets to the verification part. As soon as I do the timeout.flush() I get
Unsatisfied requests: GET /url/status/check
Have you tried :
httpBackend.expectGET(url).respond(200, { status: 'busy'});
service.poll();
httpBackend.flush();
timeout.flush();
httpBackend.expectGET(url).respond(200, { status: 'complete}');
httpBackend.flush();