So I was finding caching solutions for my AWS Lambda functions and I find out something called 'Simple Caching'. It's fits perfectly for what I want since my data is not changed frequently. However one thing that I was unable to find that what is the timeout for this cache. When is the data refreshed by the function and is there any way I can control it ?
An example of the code I am using for the function:
let cachedValue;
module.exports.handler = function(event, context, callback) {
console.log('Starting Lambda.');
if (!cachedValue) {
console.log('Setting cachedValue now...');
cachedValue = 'Foobar';
} else {
console.log('Cached value is already set: ', cachedValue);
}
};
What you're doing here is taking advantage of a side effect of container reuse. There is no lower or upper bound for how long such values will persist, and no guarantee that they will persist at all. It's a valid optimization to use, but it's entirely outside your control.
Importantly, you need to be aware that this stores the value in one single container. It lives for as long as the Node process in the container are alive, and is accessible whenever a future invocation of the function reuses that process in that container.
If you have two or more invocations of the same function running concurrently, they will not be in the same container, and they will not see each other's global variables. This doesn't make it an invalid technique, but you need to be aware of that fact. The /tmp/ directory will exhibit very similar behavior, which is why you need to clean that up when you use it.
If you throw any exception, the process and possibly the container will be destroyed, either way the cached values will be gone on the next invocation, since there's only one Node process per container.
If you don't invoke the function at all for an undefined/undocumented number of minutes, the container will be released by the service, so this goes away.
Re-deploying the function will also clear this "cache," since a new function version won't reuse containers from older function versions.
It's a perfectly valid strategy as long as you recognize that it is a feature of a black box with no user-serviceable parts.
See also https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/ -- a post that is several years old but still accurate.
Related
I currently have a Firebase Function and to do its task it needs a key. This key changes every 4-20 days and I want to be able to have the functions update the key themselves. What would the best way to do this be? To get the key it is a slow network call to a 3rd party API so I'd rather store it. Currently I have an environment variable that I change myself when I find the functions failing, but I would rather have this process done automatically.
I don't think I can change the environment variables at run time so is the only option to store the value in my database and query for that every time I need it? This seems a bit slow, but I'm not sure.
is the only option to store the value in my database and query for that every time I need it?
Cloud Functions is stateless and will not retain any information outside of the code and data that was deployed with the function. So, you will need some sort of persistent storage to hold the key. It doesn't have to be a database. It can be any persistent storage you want.
You can certainly just read the key once (from wherever you choose to store it) and store it in memory if it was not previously read, for as long as you are allowed to keep using it without refreshing the value. Memory does persist for some time per server instance, but it is not shared among all of your function invocations, as each one might run on a different instance.
As informed to me by Paul Rudin on the Google Cloud Slack, you could cache the key as a global variable which is, in practice, often reused: https://cloud.google.com/functions/docs/bestpractices/tips#use_global_variables_to_reuse_objects_in_future_invocations
Use global variables to reuse objects in future invocations There is
no guarantee that the state of a Cloud Function will be preserved for
future invocations. However, Cloud Functions often recycles the
execution environment of a previous invocation. If you declare a
variable in global scope, its value can be reused in subsequent
invocations without having to be recomputed.
This way you can cache objects that may be expensive to recreate on
each function invocation. Moving such objects from the function body
to global scope may result in significant performance improvements.
The following example creates a heavy object only once per function
instance, and shares it across all function invocations reaching the
given instance:
// Global (instance-wide) scope
// This computation runs at instance cold-start
const instanceVar = heavyComputation();
/**
* HTTP function that declares a variable.
*
* #param {Object} req request context.
* #param {Object} res response context.
*/
exports.scopeDemo = (req, res) => {
// Per-function scope
// This computation runs every time this function is called
const functionVar = lightComputation();
res.send(`Per instance: ${instanceVar}, per function: ${functionVar}`);
};
I would like to backup a running rocksdb-instance to a location on the same disk in a way that is safe, and without interrupting processing during the backup.
I have read:
Rocksdb Backup Instructions
Checkpoints Documentation
Documentation in rocksdb/utilities/{checkpoint.h,backupable_db.{h,cc}}
My question is whether the call to CreateNewBackupWithMetadata is marked as NOT threadsafe to express, that two concurrent calls to this function will have unsafe behavior, or to indicate that ANY concurrent call on the database will be unsafe. I have checked the implementation, which appears to be creating a checkpoint - which the second article claims are used for online backups of MyRocks -, but I am still unsure, what part of the call is not threadsafe.
I currently interpret this as, it is unsafe, because CreateBackup... calls DisableFileDeletions and later EnableFileDeletions, which, of course, if two overlapping calls are made, may cause trouble. Since the SST-files are immutable, I am not worried about them, but am unsure whether modifying the WAL through insertions can corrupt the backup. I would assume that triggering a flush on backup should prevent this, but I would like to be sure.
Any pointers or help are appreciated.
I ended up looking into the implementation way deeper, and here is what I found:
Recall a rocksdb database consists of Memtables, SSTs and a single WAL, which protects data in the Memtables against crashes.
When you call rocksdb::BackupEngine::CreateBackupWithMetadata, there is no lock taken internally, so this call can race, if two calls are active at the same time. Most notably this call does Disable/EnableFileDeletions, which, if called by one call, while another is still active spells doom for the other call.
The process of copying the files from the database to the backup is protected from modifications while the call is active by creating a rocksdb::Checkpoint, which, if flush_before_backup was set to true, will first flush the Memtables, thus clearing the active WAL.
Internally the call to CreateCustomCheckpoint calls DB::GetLiveFiles in db_filecheckpoint.cc. GetLiveFiles takes the global database lock (_mutex), optionally flushes the Memtables, and retrieves the list of SSTs. If a flush in GetLiveFiles happens while holding the global database-lock, the WAL must be empty at this time, which means the list should always contain the SST-files representing a complete and consistent database state from the time of the checkpoint. Since the SSTs are immutable, and since file deletion through compaction is turned off by the backup-call, you should always get a complete backup without holding writes on the database. However this, of course, means it is not possible to determine the exact last write/sequence number in the backup when concurrent updates happen - at least not without inspecting the backup after it has been created.
For the non-flushing version, there maybe WAL-files, which are retrieved in a different call than GetLiveFiles, with no lock held in between, i.e. these are not necessarily consistent, but I did not investigate further, since the non-flushing case was not applicable to my use.
I have two pieces of code
Lambda A : (written in Python)
Lambda B : (written in NodeJs)
Scenario 1:
Lambda A calls Lambda B with some payload and waits for output from lambda B.
Lambda B as part of its logic makes api calls and returns data. I call the other lambda using boto3 (InvocationType: RequestResponse).
Scenario 2:
I create a zip file containing both the pieces of the code and create a lambda using the zip file. In the python code, I invoke the NodeJs code using subprocess.call().
Can anyone tell me which approach is faster. what are the pros and cons of above approaches (w.r.t : billingTime, duration time, scalability etc..)
As per my understanding the cons for the above approaches will be :
Scenario 1:
Because of the network call, I will be billed twice (for the network call duration)
has some network overhead.
Scenario 2:
Sub process creation overhead.
The answer here boils down to "benchmark it."
The process creation overhead, itself, should be minimal, but the overhead of starting up the Node child could be a performance killer.
The reason centers around container reuse.
When a Node Lambda function is invoked for the first time, then finishes, the container and the process inside it remain on a warm standby for the next invocation. When that happens, your process is already running, and the handler function is invoked in a matter of microseconds. There is no time required to set up the container and start the process and run through any initialization code on that second invocation.
This means that, in scenario 1, the time for the function to get started is minimized. The overhead is how long it takes for the caller to make the request to Lambda and for Lambda to return the response, once available. In between those two things, there is very little time.
By contrast, if you spin up a child process with each request in scenario 2, you have all of that initialization overhead with each request.
I recently had occasion to run some code in Lambda that was in a language Lambda doesn't support, called by a Lambda function written in Node.js I do this with a child process, but with a twist: the child process was written to read from STDIN and write to STDOUT, for IPC from and to the JS code. I can then send a "request" to the child process and an event is triggered when the child writes the response.
So, the child is started from Node, with its controlling Node object in a global variable, only if not already present... but it is likely to be already present, again, due to container reuse.
In Node/Lambda, setting context.callbackWaitsForEmptyEventLoop allows the Lambda callback to consider the invocation finished, even if the event loop is still running, and this means I can leave that child process running across invocations.
With this mechanism in place, I achieve best-case runtimes for each Lambda invocation of under 3 milliseconds when the container is reused. For each new container, then first initiation of that child process is in excess of 1000 ms. The 3ms time is doubtless better than I could achieve if calling a second Lambda function from inside the first one, but the savings come fron keeping the inner process alive while the container remains alive.
Since your outer function is Python, it's not clear to me just exacrly what implications there are for you, or how useful this might be, but I thought it might serve to illustrate the value of the concept of keeping your child process alive between invocations.
But start with what you have, and benchmark both of your scenarios, multiple tines, to ensure that any longer than expected runtines aren't an artifact of new container creation.
I'm looking for guidelines to using a consistent value of the current date and time throughout a transaction.
By transaction I loosely mean an application service method, such methods usually execute a single SQL transaction, at least in my applications.
Ambient Context
One approach described in answers to this question is to put the current date in an ambient context, e.g. DateTimeProvider, and use that instead of DateTime.UtcNow everywhere.
However the purpose of this approach is only to make the design unit-testable, whereas I also want to prevent errors caused by unnecessary multiple querying into DateTime.UtcNow, an example of which is this:
// In an entity constructor:
this.CreatedAt = DateTime.UtcNow;
this.ModifiedAt = DateTime.UtcNow;
This code creates an entity with slightly differing created and modified dates, whereas one expects these properties to be equal right after the entity was created.
Also, an ambient context is difficult to implement correctly in a web application, so I've come up with an alternative approach:
Method Injection + DeterministicTimeProvider
The DeterministicTimeProvider class is registered as an "instance per lifetime scope" AKA "instance per HTTP request in a web app" dependency.
It is constructor-injected to an application service and passed into constructors and methods of entities.
The IDateTimeProvider.UtcNow method is used instead of the usual DateTime.UtcNow / DateTimeOffset.UtcNow everywhere to get the current date and time.
Here is the implementation:
/// <summary>
/// Provides the current date and time.
/// The provided value is fixed when it is requested for the first time.
/// </summary>
public class DeterministicTimeProvider: IDateTimeProvider
{
private readonly Lazy<DateTimeOffset> _lazyUtcNow =
new Lazy<DateTimeOffset>(() => DateTimeOffset.UtcNow);
/// <summary>
/// Gets the current date and time in the UTC time zone.
/// </summary>
public DateTimeOffset UtcNow => _lazyUtcNow.Value;
}
Is this a good approach? What are the disadvantages? Are there better alternatives?
Sorry for the logical fallacy of appeal to authority here, but this is rather interesting:
John Carmack once said:
There are four principle inputs to a game: keystrokes, mouse moves, network packets, and time. (If you don't consider time an input value, think about it until you do -- it is an important concept)"
Source: John Carmack's .plan posts from 1998 (scribd)
(I have always found this quote highly amusing, because the suggestion that if something does not seem right to you, you should think of it really hard until it seems right, is something that only a major geek would say.)
So, here is an idea: consider time as an input. It is probably not included in the xml that makes up the web service request, (you wouldn't want it to anyway,) but in the handler where you convert the xml to an actual request object, obtain the current time and make it part of your request object.
So, as the request object is being passed around your system during the course of processing the transaction, the time to be considered as "the current time" can always be found within the request. So, it is not "the current time" anymore, it is the request time. (The fact that it will be one and the same, or very close to one and the same, is completely irrelevant.)
This way, testing also becomes even easier: you don't have to mock the time provider interface, the time is always in the input parameters.
Also, this way, other fun things become possible, for example servicing requests to be applied retroactively, at a moment in time which is completely unrelated to the actual current moment in time. Think of the possibilities. (Picture of bob squarepants-with-a-rainbow goes here.)
Hmmm.. this feels like a better question for CodeReview.SE than for StackOverflow, but sure - I'll bite.
Is this a good approach?
If used correctly, in the scenario you described, this approach is reasonable. It achieves the two stated goals:
Making your code more testable. This is a common pattern I call "Mock the Clock", and is found in many well-designed apps.
Locking the time to a single value. This is less common, but your code does achieve that goal.
What are the disadvantages?
Since you are creating another new object for each request, it will create a mild amount of additional memory usage and additional work for the garbage collector. This is somewhat of a moot point since this is usually how it goes for all objects with per-request lifetime, including the controllers.
There is a tiny fraction of time being added before you take the reading from the clock, caused by the additional work being done in loading the object and from doing lazy loading. It's negligible though - probably on the order of a few milliseconds.
Since the value is locked down, there's always the risk that you (or another developer who uses your code) might introduce a subtle bug by forgetting that the value won't change until the next request. You might consider a different naming convention. For example, instead of "now", call it "requestRecievedTime" or something like that.
Similar to the previous item, there's also the risk that your provider might be loaded with the wrong lifecycle. You might use it in a new project and forget to set the instancing, loading it up as a singleton. Then the values are locked down for all requests. There's not much you can do to enforce this, so be sure to comment it well. The <summary> tag is a good place.
You may find you need the current time in a scenario where constructor injection isn't possible - such as a static method. You'll either have to refactor to use instance methods, or will have to pass either the time or the time-provider as a parameter into the static method.
Are there better alternatives?
Yes, see Mike's answer.
You might also consider Noda Time, which has a similar concept built in, via the IClock interface, and the SystemClock and FakeClock implementations. However, both of those implementations are designed to be singletons. They help with testing, but they don't achieve your second goal of locking the time down to a single value per request. You could always write an implementation that does that though.
Code looks reasonable.
Drawback - most likely lifetime of the object will be controlled by DI container and hence user of the provider can't be sure that it always be configured correctly (per-invocation and not any longer lifetime like app/singleton).
If you have type representing "transaction" it may be better to put "Started" time there instead.
This isn't something that can be answered with a realtime clock and a query, or by testing. The developer may have figured out some obscure way of reaching the underlying library call...
So don't do that. Dependency injection also won't save you here; the issue is that you want a standard pattern for time at the start of the 'session.'
In my view, the fundamental problem is that you are expressing an idea, and looking for a mechanism for that. The right mechanism is to name it, and say what you mean in the name, and then set it only once. readonly is a good way to handle setting this only once in the constructor, and lets the compiler and runtime enforce what you mean which is that it is set only once.
// In an entity constructor:
this.CreatedAt = DateTime.UtcNow;
I'm trying to explore all the options of the new C++11 standard in depth, while using std::async and reading its definition, I noticed 2 things, at least under linux with gcc 4.8.1 :
it's called async, but it got a really "sequential behaviour", basically in the row where you call the future associated with your async function foo, the program blocks until the execution of foo it's completed.
it depends on the exact same external library as others, and better, non-blocking solutions, which means pthread, if you want to use std::async you need pthread.
at this point it's natural for me asking why choosing std::async over even a simple set of functors ? It's a solution that doesn't even scale at all, the more future you call, the less responsive your program will be.
Am I missing something ? Can you show an example that is granted to be executed in an async, non blocking, way ?
it's called async, but it got a really "sequential behaviour",
No, if you use the std::launch::async policy then it runs asynchronously in a new thread. If you don't specify a policy it might run in a new thread.
basically in the row where you call the future associated with your async function foo, the program blocks until the execution of foo it's completed.
It only blocks if foo hasn't completed, but if it was run asynchronously (e.g. because you use the std::launch::async policy) it might have completed before you need it.
it depends on the exact same external library as others, and better, non-blocking solutions, which means pthread, if you want to use std::async you need pthread.
Wrong, it doesn't have to be implemented using Pthreads (and on Windows it isn't, it uses the ConcRT features.)
at this point it's natural for me asking why choosing std::async over even a simple set of functors ?
Because it guarantees thread-safety and propagates exceptions across threads. Can you do that with a simple set of functors?
It's a solution that doesn't even scale at all, the more future you call, the less responsive your program will be.
Not necessarily. If you don't specify the launch policy then a smart implementation can decide whether to start a new thread, or return a deferred function, or return something that decides later, when more resources may be available.
Now, it's true that with GCC's implementation, if you don't provide a launch policy then with current releases it will never run in a new thread (there's a bugzilla report for that) but that's a property of that implementation, not of std::async in general. You should not confuse the specification in the standard with a particular implementation. Reading the implementation of one standard library is a poor way to learn about C++11.
Can you show an example that is granted to be executed in an async, non blocking, way ?
This shouldn't block:
auto fut = std::async(std::launch::async, doSomethingThatTakesTenSeconds);
auto result1 = doSomethingThatTakesTwentySeconds();
auto result2 = fut.get();
By specifying the launch policy you force asynchronous execution, and if you do other work while it's executing then the result will be ready when you need it.
If you need the result of an asynchronous operation, then you have to block, no matter what library you use. The idea is that you get to choose when to block, and, hopefully when you do that, you block for a negligible time because all the work has already been done.
Note also that std::async can be launched with policies std::launch::async or std::launch::deferred. If you don't specify it, the implementation is allowed to choose, and it could well choose to use deferred evaluation, which would result in all the work being done when you attempt to get the result from the future, resulting in a longer block. So if you want to make sure that the work is done asynchronously, use std::launch::async.
I think your problem is with std::future saying that it blocks on get. It only blocks if the result isn't already ready.
If you can arrange for the result to be already ready, this isn't a problem.
There are many ways to know that the result is already ready. You can poll the future and ask it (relatively simple), you could use locks or atomic data to relay the fact that it is ready, you could build up a framework to deliver "finished" future items into a queue that consumers can interact with, you could use signals of some kind (which is just blocking on multiple things at once, or polling).
Or, you could finish all the work you can do locally, and then block on the remote work.
As an example, imagine a parallel recursive merge sort. It splits the array into two chunks, then does an async sort on one chunk while sorting the other chunk. Once it is done sorting its half, the originating thread cannot progress until the second task is finished. So it does a .get() and blocks. Once both halves have been sorted, it can then do a merge (in theory, the merge can be done at least partially in parallel as well).
This task behaves like a linear task to those interacting with it on the outside -- when it is done, the array is sorted.
We can then wrap this in a std::async task, and have a future sorted array. If we want, we could add in a signally procedure to let us know that the future is finished, but that only makes sense if we have a thread waiting on the signals.
In the reference: http://en.cppreference.com/w/cpp/thread/async
If the async flag is set (i.e. policy & std::launch::async != 0), then
async executes the function f on a separate thread of execution as if
spawned by std::thread(f, args...), except that if the function f
returns a value or throws an exception, it is stored in the shared
state accessible through the std::future that async returns to the
caller.
It is a nice property to keep a record of exceptions thrown.
http://www.cplusplus.com/reference/future/async/
there are three type of policy,
launch::async
launch::deferred
launch::async|launch::deferred
by default launch::async|launch::deferred is passed to std::async.