I am slowly understanding the asio way of doing things as I migrate legacy callback code to use completion tokens. So far I have been following the pattern described in the callback_wrapper.cpp example which uses asio::asio::async_initiate. Then I was reading the article "The great mystery on how to make durable, asynchronous functions for asio." and the author uses asio::async_compose instead. Is there an advantage of one over the other? The API looks similar save for one takes generic arguments and the other an io context and executor with the arguments supplied via the wrapped function. Is it just a difference between being able to have bi-directional data flow via the lambda capture vs uni-directional via the arguments?
Related
I've recently been dealing with a project which has a peculiar design - the server invokes an aws lambda which itself invokes another lambda - and the second lambda is failing.
However, it seems the first lambda (the server-invoked one) is not receiving any error from the second one, so it just waits, and so does the server, until there is a timeout.
I was wondering if this is a common problem and if there is a "typical" solution, best-practices to consider, etc.
I don't think language is particularly important - I want to know if there's a simple, quick, direct way of dealing with the problem in the title without doing things like overhauling everything or setting global flags.
Have you taken a look at Step Functions? They're a great way of managing Lambda functions that need to happen sequentially or conditionally.
There's some pretty good debug features as well; you'll be able to see input and output data, and any errors.
https://aws.amazon.com/step-functions/
I'm trying to create a modular application in clojure.
Lets suppose that we have a blog engine, which consists of two modules, for example - database module, and article module (something that stores articles for blog), all with some configuration parameters.
So - article module depends on storage, And having two instances of article module and database module (with different parameters) allows us to host two different blogs in two different databases.
I tried to implement this creating new namespaces for each initialized module on-the-fly, and defining functions in this namespaces with partially applied parameters. But this approach is some sort of hacking, i think.
What is right way to do this?
A 'module' is a noun, as in the 'Kingdom of Nouns' by Steve Yegge.
Stick to non side-effecting or pure functions of their parameters (verbs) as much as possible except at the topmost levels of your abstractions. You can organize those functions however you like. At the topmost levels you will have some application state, there are many approaches to manage that, but the one I use the most is to hide these top-level services under a clojure protocol, then implement it in a clojure record (which may hold references to database connections or some-such).
This approach maximizes flexibility and prevents you from writing yourself into a corner. It's analagous to java's dependency injection. Stuart Sierra did a good talk recently on these topics at Clojure/West 2013, but the video is not yet available.
Note the difference from your approach. You need to separate the management and resolution of objects from their lifecycles. Tying them to namespaces is quick for access, but it means any functions you write as clients that use that code are now accessing global state. With protocols, you can separate the implementation detail of global state from the interface of access.
If you need a motivating example of why this is useful, consider, how would you intercept all access to a service that's globally accessible? Well, you would push the full implementation down and make the entry point a wrapper function, instead of pushing the relevant details closer to the client code. What if you wanted some behavior for some clients of the code and not others? Now you're stuck. This is just anticipating making those inevitable trade-offs preemptively and making your life easier.
I work on a multithreaded message processing application written in C++. The application receives xml messages, performs some action, and may publish out an xml message to another service if required.
Currently, the app works by extracting data while parsing the message and performing some action on that message in the middle of parsing. This seems like poor practice to me. I have the opportunity to create an alternative, and I'm considering approaches I can use.
One method I've thought of is to serialize the xml data into a data object, and once that is finished, extract and process data as needed. The disadvantage would be that I have to build a new class for each different xml message I process (probably around 30), but that approach seems cleaner than what I have now.
Is there a better way than this? Should also mention the caveat that any code libraries developed outside the U.S. are unlikely to be approved.
Currently, the app works
Then what exactly are you fixing?
Don't fix what isn't broken.
There are typically two approaches to XML parsing: DOM and SAX. DOM builds up a document object model (like what you are proposing), whereas SAX invokes callbacks as parts of the document are visited during parsing. The free, well-known libxml2 library supports both parsing methods.
Typically, the SAX approach (i.e., using callbacks that get executed as the document is visited), uses less memory and can result in lower end-user latency, because you can start processing immediately, instead of having to wait for the entire document to have been parsed and built up.
The fact that your program is multithreaded is a red-herring. As long as you always pass an object to each of your callbacks, and that object is not shared between threads, you can safely do this with multiple different such objects in multiple different threads. Using a standard library such as libxml2 to do your parsing is also sensible from a reuse perspective.
There were probably some design decisions that were made which led to this approach (say for example, it's faster to process using a SAX like model than a DOM like model), with the latter you need to parse the entire message, with the former you can make decisions as you are called back with data.
I'd try to understand these first before making any changes, secondly aside from keeping you busy, is there a real business need for it? If not, move on and do something else...
I'm fairly new to advanced C++ program techniques such as templates,
but I am developing a simple API for a project I'm working on.
The function or method that you call can take a long time to complete.
Essentially it's transferring a file over the network.
It looks a bit like this.
Client
{
int WriteFile();
int ReadFile();
}
But I want to have a couple of options here.
call WriteFile and have it block.
Call WriteFileAsync and not have it block.
In the async version be flexible about how I know the task is done.
Be able to poll the client to find out where it's up to with my current Read or Write
operation.
I'm at a bit of a loss as to how to design this nicely the C++ way.
It's a requirement to avoid using boost, but I could use a boost-like approach.
Although, I looked through some of the headers and got very much confused. Anything beyond
basic template programming for me I find confusing.
What I'm after is a nice way of being notified of event completion and be able to wait for
an event to complete.
My advice would be looking at the docs and tutorial for boost::asio (which you can use as part of boost or as part of the independent asio project, but I guess that the requirement is no external libs, not just no boost).
Usually blocking calls are simple to define, while non-blocking operations require some callback mechanism as to notify the user of the result of the operation whenever that completes. Again, take a look at the tutorials and docs to get an idea of a clean interface, that will be much easier to browse over than the headers.
EDIT: ASIO has support for different protocols, so it might be more complex than what you need, read one of the examples and get the ideas of how to use callback mechanisms.
Regarding the use of asynchronous calls, I would suggest reading about the design of the future for C++0x.
Basically, the idea is to hand a proxy to the user, instead of a plain type. This proxy is aware of the threading and can be used to:
poll about the completion
get the result
You can also add clever mechanisms like trying to get the result for a fixed duration or up to a fixed point in time and abandon (for the moment) if the task hasn't completed in time (for example to do something else and try again later, or to simple go forward and forget about this).
The new threading API of C++0x has been very cleverly designed (taking mainly after Boost.Threads) and would give you much insight as to how to design for multi-threading.
I am tasked to maintain and update a library which allows a computer to send commands at a hardware device and then receive its response. Currently the code is setup in such a way that every single possible command the device can receive is sent via its own function. Code repetition is everywhere; a DRY advocate's worst nightmare.
Obviously there is much opportunity for improvement. The problem is each command has a different payload. Currently the data that is to be the payload is passed to each command function in the form of arguments. It's difficult to consolidate functionality without pushing the complexity to a level that calls the library.
When a response is received from the device its data is put into an object of a class solely responsible for holding this data, they do nothing else. There are hundreds of classes which do this. These objects are then used to access the returned data by the app layer.
My objectives:
Throughly reduce code repetition
Maintain similiar level of complexity at application layer
Make it easier to add new commands
My idea:
Have one function to send a command and one to receive (the receiving function is automatically called when a response from the device is detected). Have a struct holding all command/response data which will be passed to sending function and returned by receiving function. Since each command has a corresponding enum value, have a switch statement which sets up any command specific data for sending.
Is my idea the best way to do it? Is there a design pattern I could use here? I've looked and looked but nothing seems to fit my needs.
Thanks in advance! (Please let me know if clarification is necessary)
This reminds me of the REST vs. SOA debate, albeit on a smaller physical scale.
If I understand you correctly, right now you have calls like
device->DoThing();
device->DoOtherThing();
and then sometimes I get a callback like
callback->DoneThing(ThingResult&);
callback->DoneOtherTHing(OtherThingResult&)
I suggest that the user is the key component here. Do the current library users like the interface at the level it is designed? Is the interface consistent, even if it is large?
You seem to want to propose
device->Do(ThingAndOtherThingParameters&)
callback->Done(ThingAndOtherThingResult&)
so to have a single entry point with more complex data.
The downside from a library user perspective may that now I have to use a manual switch() or other type statement to tell what really happened. While the dispatching to the appropriate result callback used to be done for me, now you have made it a burden upon the library user.
Unless this bought me as a user some level of flexibility, that I as as user wanted I would consider this a step backwards.
For your part as an implementor, one suggestion would be to go to the generic form internally, and then offer both interfaces externally. Perhaps the old specific interface could even be auto-generated somehow.
Good Luck.
Well, your question implies that there is a balance between the library's complexity and the client's. When those are the only two choices, one almost always goes with making the client's life easier. However, those are rarely really the only two choices.
Now in the text you talk about a command processing architecture where each command has a different set of data associated with it. In the olden days, this would typically be implemented with a big honking case statement in a loop, where each case called a different routine with different parameters and perhaps some setup code. Grisly. McCabe complexity analysers hate this.
These days what you can do with an OO language is use dynamic dispatch. Create a base abstract "command" class with a standard "handle()" method, and have each different command inherit from it to add their own members (to represent the different "arguments" to the different commands). Then you create a big honking array of these at startup, usually indexed by the command ID. For languages like C++ or Ada it has to be an array of pointers to "command" objects, for the dynamic dispatch to work. Then you can just call the appropriate command object for the command ID you read from the client. The big honking case statement is now handled implicitly by the dynamic dispatch.
Where you can get the big savings in this scenario is in subclassing. Do you have several commands that use the exact same parameters? Make a subclass for them, and then derive all of those commands from that subclass. Do you have several commands that have to perform the same operation on one of the parameters? Make a subclass for them with that one method implemented for that operation, and then derive all those commands from that subclass.
Your first objective should be to produce a library that decouples higher software layers from the hardware. Users of your library shouldn't care that you have a hardware device that can execute a number of functions with a different payload. They should only care what the device does in a higher level. In this sense, it is in my opinion a good thing that every command is mapped to each one function.
My plan will be:
Identify the objects the higher data layers need to get the job done. Model the objects in C++ classes from their perspective, not from the perspective of the hardware
Define the interface of the library using the above objects
Start the implementation of the library. Perhaps an intermediate layer that maps software objects to hardware objects is necessary
There are many things you can do to reduce code repetition. You can use polymorphism. Define a class with the base functionality and extend it. You can also use utility classes, that implement functions needed for many commands.