I am working on an application which consists of 2 layers: The GUI built in Electron and the "backend" built in C++ running in the background. The GUI needs to be able to (amongst other things such as streaming data) send and request data to and from the backend for configuration purposes. For communication Redis is being used, mainly for its pub/sub capability.
What would be the preferred way to request and send data from/to the backend? I came up with the following ideas but I'm not sure if any of these are the way to go.
Publish value on a configuration channel and handle the request via switch case. E.g. configuration.set_sensor_frequency is handled by a set_sensor_frequency(value) function in the backend.
Write the configuration to configuration.sensor_frequency on the redis server and listen to the set event on the backend and react accordingly. But this kinda seems like method 1 but more complicated.
Like method 2, write the config to the redis server and periodically check (every few cycles or so) in the backend whether the value has been updated
Something else. Please elaborate.
Related
I have a nodeJS server which receives user POST/Streaming requests from a web-UI.
I have a C++ back-end engine process which does some calculations and sends API calls to other 3rd party services. The API call requires certain info provided by the web users.
My question is what is the best solution to pass the request data received on NodeJS and send over to the C++ process?
WebUI -> NodeJS ->???->> C++ engine
Make your C++ application listen on a TCP or Unix socket.
Make your NodeJs application connect to that socket and exchange messages. For messages you can use Google Protocol Buffers, JSON, etc..
If the information what you have is still at JavaScript layer, then you have to implement C/C++ Addons implementation. If you already have some type of native module, then you may follow the same design based on that (very likely existing module could be based on NAN). If you are plan to introduce a brand new native module then it is a good time to consider N-API. You can get more information about it from.
https://nodejs.org/dist/latest-v11.x/docs/api/n-api.html
https://github.com/nodejs/node-addon-api
I have a question on how can we integrate kafka producer with a front end web app. get the data for every minute or second . Can the web app pass the JSON object to a running producer each time the it is created ? or do we need to initiate the kafka client each time we get a JSON object ?
You would want to probably open a new Producer for every session, probably not open and close for each and every request. And this would be done on the backend, not the frontend.
But a web server consisting of a Kafka client is no different underneath the HTTP layer vs a regular console app; you accept an incoming request, deserialize it, then optionally parse, then serialize again for Kafka output, then optionally render something back to the user.
If you're really asking, "is Kafka with HTTP requests possible", regardless of the language and platforms, then sure, the Confluent REST Proxy operates similarly, only written in Java
As far as webapps tracking goes, I would suggest looking into Divolte Collector
I'm trying to implement a rather complex architecture for a desktop application (not to be distributed, so it's ok to use technology usually adopted for servers - please don't tell me to use electron or .NET).
It basically must store data coming from a UDP stream (with new data frequency at ~90Hz). The application should also open a websocket server and accept new clients, specifically from a tablet. The tablet user should be able to set a flag, enabling or disabling data storage.
This is a very simple block scheme of the system
I already used Django before, but for more standard usage (CMS, REST APIs, etc). After some research, I found some tools I could use to build the system:
1 - Celery, which to my understanding enables running asynchronous tasks (I guess I could use it to store the data coming from the UDP stream, maybe after accumulating a hundred values or so)
2 - Django channels, which should help me in the websocket communication
3 - Twisted, to receive UDP messages.
What confuses me is how to integrate these components, and exchange data between them. Looks like twisted is a completely separated server, so how can i run a celery task that takes input as data and writes it to a django model?
How should I implement the flag coming from webosckets? global variable?
any help appreciated!
I have 2 applications (a GUI in javascript and another in C++) which need to communicate to each other.
The C++ application (server) contains multiple realtime sensor data which it has to stream to the GUI (client). The data is buffered and sent as a big chunk. The GUI simply renders the data and doesn't buffer it locally (current library renders relatively slow).
We want to use Redis where each channel is a sensor. On the client side the user can select which sensor has to be streamed. This requires to let the server somehow know which channels to publish to.
Now the question is more about performance and extensibility. Which scenario is best?
Publish all sensor data. +-30 sensors with data at max 64 bit. Each up to 10,000 samples streamed at up to 50hz. (This is maxing out absolutely everything, but does give a ballpark).
Store the channel names in Redis as a JSON object or namespaced keys. Listen for a set event server-side, get the channels and cache them and dynamically publish to the channels.
Same as above but get the channels during every cycle from Redis without listening to any set event.
Use a configuration channel where the client publishes the configuration (via JSON string) when it's changed. Server side we subscribe to the configuration channel and handle the new channels appropriately.
Something else. Please elaborate.
Try to use redis streams feature from recently released redis 5.0. If you are looking for performant C++ library, which supports redis streams try to use bredis, for example.
I've got a Grails app (version 2.2.4) with a controller method that "logs" all requests to an external web service (JSON over HTTP - one way message, response is not needed). I want to decouple the controller method from calling the web service directly/synchronously and provide a simple "queue" which can store the calls if the web service is unavailable and then send them through once the service is back up again.
This sounds like a good fit for some sort of JMS solution but I've not got any experience with using JMS (so learning curve could be an issue). Should I be using one of the available messaging plugins or is that overkill for my simple requirements? I don't want a separate messaging app, it has to be embedded in my webapp and I'd prefer something small and simple vs more complicated and robust (so advice on which plugin would be welcome).
The alternative is to implement an async service myself and queue the "messages" in the database (reading them via a Quartz job) or with something like java.util.concurrent.ConcurrentLinkedQueue?
EDIT: Another approach could be to use log4j with a custom appender set up as a AsyncAppender.
The alternative is to implement an async service myself and queue the "messages" in the database (reading them via a Quartz job)
I went ahead and tried this approach. It was very straight forward and was only a "screen" length of code in the end. I tested it with a failing web service end point as well as an app restart (crash) and it handled both. I used a single service class to both persist the messages (Grails domain class) and to flush the queue (triggered by Quartz scheduler) which reads the DB and fires off the web service calls, removing the DB entity when web service returns 200 status code.