I’m trying to build an application on aws that is 100% serverless (minus the database for now) and what I’m running into is that the database is the bottleneck. My application can scale very well but my database has a finite number of connections it can accommodate and at some point, my lambdas will run into that limit. I can do connection pooling outside of the handler in my lambdas so that there is a database connection per lambda container instead of per invocation and while that does increase the number of concurrent invocations before I hit my connection limit, the limit still exists.
I have two questions.
1. Does serverless aurora solve this by autoscaling to increase the number of instances to meet the need for more connections.
2. Are there any other solutions to this problem?
Also, from other developers interested in serverless, am I trying to do something that’s not worth doing? I love how easy deployment is with serverless framework but is it better just to work with Microservices in something like Kubernetes instead?
I believe there are two potential solutions to that problem:
The first and the simplest option is to take advantage of "lambda hot state", it's the concept when Lambda reuses the execution context for subsequent invocations. As per AWS suggestion
Any declarations in your Lambda function code (outside the handler code, see Programming Model) remains initialized, providing additional optimization when the function is invoked again. For example, if your Lambda function establishes a database connection, instead of reestablishing the connection, the original connection is used in subsequent invocations. We suggest adding logic in your code to check if a connection exists before creating one.
Basically, while the lambda function is the hot stage it "might/should" reuse opened connection(s).
The limitations of the following:
you only reuse connection for single lambda type, so if you have 5 lambda functions invoked all the time you still will be using 5 connections
when you have a spike in lambda invocations, including parallel executions this approach becomes less effective since, lambda will be executed in a new execution context for majority of requests
The second option would be to use a connection pool, connection pool is an array of established database connections, so that the connections can be reused when future requests to the database are required.
While the second option provides a more consistent solution it requires much more infrastructure.
you would be required to run a separate instance for the pool, and if you want to do things properly probably at least two instances and a load balancer (unless use containers).
While it might be overwhelming to provision that much additional infrastructure for connection pooler, it still might be a valid option depending on the scale of the project, your existing infrastructure (may be you already using containers) and cost benefits
Best practices by AWS recommends to take advantage of hot start. You can read more about it here: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.Lambda.BestPracticesWithDynamoDB.html
Related
I have one lambda function to test the URLs using puppeteer and chrome.
When I invoke 50 lambdas at the same time chrome is not able to load all the passed URLs.
What could be the reason for it?
I suspect it shares the CPU with time slicing.
One of the best features of AWS Lambda functions is scalability. It means it will increase the needed resources to perform the task. It is impossible to share the CPU because it will destroy the whole concept of Serverless in Lambda Functions. BUT, these scenarios could be your problem:
Multiple invocations at the same will share /tmp directory. Your code might store more than allowed ephemeral storage in your invocation which might be the reason of your problem. I suggest checking to invocation logs to see if you can find any errors for regarding the ephemeral storage.
As you said, you are sending 50 requests at same time. If the target server is just a single server, it might be flooded and the memory might get full. In that case, the server can't respond to you anymore.
The Postgres query builder my Lambda functions use, Knex, uses prepared statements so I'm unable to fully take advantage of RDS Proxy since the sessions are pinned. I try to ensure that the lambdas run for as little time as possible so that the pinned session completes as quickly as possible and its connection is returned to the pool.
I was wondering how I might be able to make the sessions shorter and more granular and thinking about creating and closing a connection to AWS RDS Proxy with each query.
What performance considerations should I be considering to determine the viability of this approach?
Things I'm thinking of:
RDS Proxy connection overhead (latency and memory)
The time that RDS Proxy takes to return a closed connection back to the pool and make it reusable by others (haven't been able to find documentation on this)
Overhead of Knex's local connection pool
Using RDS proxy when building applications with Lambda functions is a recommended infrastructure pattern by AWS. Relational Databases are not built to handle tons of connections, while Lambdas can scale to thousands of instances.
RDS Proxy connection overhead (latency and memory)
This would definitely increase your latency, but you will see a great improvement in the CPU and memory usage of your database, which would ultimately prevent unnecessary failures. It's a good trade-off when you can do a lot of other optimizations on the lambda side.
The time that RDS Proxy takes to return a closed connection back to the pool and make it reusable by others (haven't been able to find documentation on this)
While working with Lambdas, you should drop the connection to your RDS proxy, as soon as you finish processing your logic without worrying about the time the RDS proxy would take to return the closed connection back. Once the connection is dropped, the RDS proxy keeps it warm in the pool of connections it maintains for a certain duration of time. If another lambda tries to make a connection meanwhile, it can share the same connection which is still warm in the pool. Dropping the database connection at the right time from your lambda would save you lambda processing time -> money.
Overhead of Knex's local connection pool
I would suggest not using Knex local connection pool with lambda as it won't do any good (Keep the pool max to 1). Every lambda execution is independent of another, the pool will never be shared and the connection doesn't persist after the execution completes unless you plan to use it with a serverless-offline kind of local framework for development purposes.
Read More about AWS Lambda + RDS Proxy usage: https://aws.amazon.com/blogs/compute/using-amazon-rds-proxy-with-aws-lambda/
AWS Documentation on where to use RDS Proxy: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy-planning.html
I'm trying to find a good architecture for connecting to the database. It is required that the connection to the database is not repeated in each lambda function. In addition, this way will create many connections for individual lambdas instead of one common. Can I implement the structure as in the figure below, so that one lambda connects to the database, and everyone else uses its connection in its code
Your proposed architecture will not work because unless your Innovation of DB Lambda is too frequent to always keep it warm and that you are storing your connection in /tmp for reusing on subsequent innovations your DB Lambda will create new connections for each invocation. Moreover if your invocations of DB Lambda create multiple containers to serve simultaneous requests then you will anyways have those many connections instead of just one
Ideal solution will be to replace the DB Lambda with a tiny EC2 instance
The DB connection can be cached in your "DB Lambda" when the Lambda stays warm. If the Lambda does not stay warm, then invoking that lambda will suffer the price of a cold lambda, which could be dominated by having to recreate a DB connection, or could be dominated by what other work you do in "DB Lambda".
How frequently you expect your Lambdas to go cold is something to take into consideration. It depends on the statistics of your incoming traffic. Are you willing to suffer the latency of recreating a DB connection once in a while or not, another consideration?
Managing a tiny EC2 instance like someone else said, could be a lot of extra work depending on whether your cloud service is a complex set of backend services, whether that service shuts down during periods of inactivity. Managing EC2 instances is more work that Lambdas.
I do see one potential problem with your architecture. If for whatever reason your "DB Lambda" fails, the calling Lambda won't know. That could be a problem if you need to handle that situation and do cleanup.
If i understood the whole concept correctly, the "serverless" architecture assumes that instead of using own servers or containers, one should use bunch of aws services. Usually such architecture includes Amazon API Gateway, bunch of Lambda functions and DynamoDB (or alternative) for storing data and state, as Lambda can't keep state. And such services as EC2 is not participating in all this, well, because this is a virtual server and it diminish all the benefits of serverless architecture.
All this looks really cool, but i feel like i'm missing something important, because right now this seems to be not applicable for such cases as real time applications.
Say, i have 2 users online. One of them performs an action in an app, which triggers changes in database, which in turn, should trigger changes in the second user app.
The conventional way to send some data or command from server to client is websocket connection. But with serverless architecture there seem to be no way to establish and maintain websocket connection. So... where did i misunderstood the concept? Or, if i understood everything correctly, then how do i implement the interactions between 2 users as described above?
where did i misunderstood the concept?
Your observation is correct. It doesn't work out of the box using API Gateway and Lambda.
Applicable solution as described here is to use AWS IoT - yes, another AWS Service.
Serverless isn't just a matter of Lambda, API Gateway and DynamoDB, it's much bigger than that. One of the big advantages to Serverless is the operational burden that it takes off your plate. No more patching, no more capacity planning, no more config management. Those may seem trivial but doing those things well and across a significant fleet of instances is complex, expensive and time consuming. Another benefit is the economics. Public cloud leverages utility billing, meaning you pay for what you run whether or not you actually use it. With AWS most of the billing per service is by hour but with Lambda it's per 100ms. The cheapest EC2 instance running for a full month is about $10/m (double that for redundancy). $20 in Lambda pricing gets you millions of invocations so for most cases serverless is significantly cheaper.
Serverless isn't for everything though, it has it's limitations, for example it's not meant for running binaries. You can't run nginx in Lambda (for example), it's only meant to be a runtime environment for the programming languages that it supports. It's also specifically meant for event based workloads, which is perfect for microservice based architectures. Small independent discrete pieces of compute doing work that when done they send an event to another(s) to do something else and if needed return a response.
To address your concerns about realtime processing, depending on what your code is doing your Lambda function could complete in less than 100ms all the way up to 5 minutes. There are strategies to optimize it's duration time but in general it's for short lived work which is conducive of realtime scenarios.
In your example about the 2 users interacting with the web app and the db, that could very easily be built using serverless technologies with one or 2 functions and a DynamoDB table. The total roundtrip time could be as low as milliseconds if not seconds, it really all depends on your code and what it's doing. These would all be HTTP calls so no websockets needed. Think of a number of APIs calling each other and your Lambda code is the orchestrator.
You might want to look at SNS (simple notification service). In your example, if app user 2 is a a subscriber to an SNS topic, then when app user 1 makes a change that triggers an SNS message, it will be pushed to the subscriber (app user 2). The message can be pushed over several supported protocols (Amazon, Apple, Google, MS, Baidu) in addition to SMTP or SMS. The SNS message can be triggered by a lambda function or directly from a DynamoDB stream after an update (a database trigger). It's up to the app developer to select a message protocol and format. The app only has to receive messages through its native channels. This may not exactly be millisecond-latency 'real-time', but it's fast enough for all but the most latency-sensitive applications.
I've been working on an AWS serverless application for several months now, and am amazed at the variety of services available. The rate of improvement and new features being added is enough to leave you out-of-breath.
I am using several AWS lambda functions (and multiple instances of the same ones) to access objects within AWS. Any information on thread safety of lambdas? While order of writing is not important (in my case), atomicity of read-modify-write is.
Anyone came across proper/intended solution?
I came across this because I'm trying to achieve the same thing.
Problem: I have a cluster of state machines running in EC2 instances. Each invokes a Lambda asynchronously (InvocationType.Event). I want to run the lambda code each time but perform a downstream network call only once from within the lambda.
Solution: Using a DynamoDB table with a relatively low TTL, I perform a conditional put of an item with unique id generated from event data that will be the same across all lambda invocations for that unique event. Then, I only perform the downstream network call if the conditional put succeeds.
It's a distributed synchronization technique often used in distributed systems for "fail-fast" systems, just via a lambda.
A strict read-modify-write atomicity requires serialization of processing (maybe using locks etc.). This is not possible with Lambda.
What you can do is this: Write your requests into a SQS Queue and then use a single-threaded (or properly synchronized) piece of software on a single EC2 instance to process the queue entries.
The CAP theorem (https://en.wikipedia.org/wiki/CAP_theorem) says that this impacts your availability, because this process on a single EC2 instance by design becomes your single point of failure.
You could also use a RDS DB with transactions (or even DynamoDB with conditional writes/consistent reads).