Web-app with clojure using hot-swapping of code - clojure

I'm thinking of writing a web-app in clojure that can update itself without restarting or loosing state.
I've seen some articles where Clojure apps can perform so-called hot-swapping of code. Meaning that they can update their own functions at runtime. Would this be safe to perform on a web-server?

To get hot-swap for code is tricky to get right, if possible at all.
It depends on the changeset and the running application too.
Issues:
old vars may litter namespaces and cause subtle conflicts, bugs
redefinition of multiple vars is not atomic
There may be old vars in a namespace that will not be there if you restart the application, however will interfere if you just redefine some of the functions and keep the app running without restart.
The other issue is atomicity: redefining multiple functions i.e. changing multiple vars is not atomic. If you change functions in one or more namespace that code in some other namespace depends on, reloading the namespaces with the new code is not atomic.
Generally, you are better off either
having a proxy hold the requests until your app restarts
spinning up a new app instance parallel to the "old version" and use a proxy to switch from the new version after the new version is ready to process requests

OTP applications in Erlang support this. Basically, it will spin the new version of your application up and start sending requests to the new version of your application. It will keep the old version alive until it has completed processing requests and then shut it down.

Related

Persistence of data for MSI installation

The MSI installation would call my (native/C++) custom action functions. Since the DLL is freshly loaded, and the MSIEXEC.EXE process is launched separately for each function (the callable actions, as specified in MSI/WiX script), I cannot use any global data in C/C++ program.
How (or Where) can I store some information about the installation going on?
I cannot use named objects (like shared-memory) as the "process" that launches the DLL to call the "action" function would exit, and OS will not keep the named-object.
I may use an external file to store, but then how would I know (in the DLL's function):
When to delete the external file.
When to find that this function call is the first call (Action/function call Before="LaunchConditions" may help, not very sure).
If I cannot delete the file, I cannot know if "information" is current or stale (i.e. belonging to earlier failed/succeeded MSI run).
"Temporary MSI tables" I have heard of, but not sure how to utilize it.
Preserve Settings: I am a little confused what your custom actions do, to be honest. However, it sounds like they preserve settings from an older application and setup version and put them back in place if the MSI fails to install properly?
Migration Suggestion (please seriously consider this option): Could you install your new MSI package and delete all shortcuts and access to the old application whilst leaving it
installed instead? Your new application version installs to a new path
and a new registry hive, and then you migrate all settings on first
launch of the new application and then kick off the uninstall of the
old application - somehow - or just leave it installed if that is
acceptable? Are there COM servers in your old install? Other things that have global registration?
Custom Action Abstinence: The above is just a suggestion to avoid custom actions. There are many reasons to avoid custom actions (propaganda piece against custom actions). If you migrate settings on application launch you avoid all sequencing, conditioning, impersonation issues along with the technical issues you have already faced (there are many more) associated with custom action use. And crucially you are in a familiar debugging context (application launch code) as opposed to the unfamiliar world of setups and their poor debugability.
Preserving Settings & Data: With regards to saving data and settings in a running MSI instance, the built in mechanism is basically to set properties using Session.Property (COM / VBScript) or MsiSetProperty (Win32) calls. This allows you to preserve strings inside the MSI's Session object. Sort of global data.
Note that properties can only be set in immediate mode (custom actions that don't change the system), and sending the data to deferred mode custom actions (that can make system changes) is quite involved centering around the CustomActionData concept (more on deferred mode & CustomActionData).
Essentially you send a string to the deferred mode custom action by means of a SetProperty custom action in immediate mode. Typically a "home grown" delimited string that you construct in immediate mode and chew up into information pieces when receiving it in deferred mode. You could try to use JSON-strings and similar to make transfer easier and more reliable by serializing and de-serializing objects via JSON strings.
Alternatives?: This set property approach is involved. Some people write to and from the registry during installation, or to a temp file (in the temp folder) and then they clean up during the commit phase of MSI, but I don't like this approach for several reasons. For one thing commit custom actions might not run based on policies on target systems (when rollback is disabled, no commit script is created - see "Commit Execution" section), and it isn't best practice. Adding temporary rows is an interesting option that I have never spent much time on. I doubt you would be able to easily use this to achieve what you need, although I don't really know what you need in detail. I haven't used it properly. Quick sample. This RemoveFile example from WiX might be better.

AlarmManager or JobScheduler compatible with all Android versions?

I need some help deciding which classes to adopt in my app that has to fulfill these requirements:
Firing of the alarms at exact times (down to the second)
Can have multiple alarms in the system (triggering at different times).
Should awaken the device if asleep and keep it awake until the user completes a defined task.
Should run on as many Android versions as possible, at least from 4.0 and up (the fact that you can use an old phone to run this app is a plus).
I've coded a small demo with AlarmManager & BroadcastReceiver to get an understanding of how all of this works. I've studied the docs for AlarmManager, BroadcastReceiver, WakefulBroadcastReceiver, and JobScheduler and even though I understand the differences, I don't know which one(s) will satisfy my requirements.
From the documentation, I get the impression that I might need to write multiple versions of my app to accommodate the different Android versions it will run on. This would be a nightmare!
Any suggestions on which classes to use? I would hate to go in one given direction only to later find out that I need to rewrite everything.
Thanks!
You don't need multiple versions of your app, you just need multiple "scheduler" implementations, each with just a couple files, and all call the same app logic.
Firing of alarms at exact times (down to the second)
Well, there's only one option that handles this. You need an AlarmManager.setExact. This is strongly discouraged, because it tends to waste battery.
Can have multiple alarms in the system (triggering at different times).
Every option there handles that, as long as you give them different ids.
Should awaken the device if asleep and keep it awake until the user completes a defined task.
Sounds like you need to have your alarm call Context.StartForegroundService, and leave that foreground service running until the user completes the task. Again, this is discouraged, because it wastes battery.
Should run on as many Android versions as possible, at least from 4.0 and up (the fact that you can use an old phone to run this app is a plus).
AlarmManager and foreground services run on all versions of Android, though the call to start a foreground Service has changed slightly with Android O, the concept is identical.
While your app is doing something important that needs to execute RIGHT NOW, even if the screen is off, then you should grab a wakelock. A wakelock prevents the CPU from pausing itself, so that you can process what needs to immediately occur. If the code can wait until the screen is turned on, then please do not use a wake lock. JobService grabs a wakelock always, so JobService code does not need to grab a separate wakelock. If you have no service running, including a JobService, then Android will randomly stop your app, even if you have a wakelock. So you always need a service of some sort, while doing any important work.

Possible to make QML application "offline capable" using caches?

I'm trying to make one of my QML apps "offline capable" - that means I want users to be able to use the application when not connected to the internet.
The main problem I'm seeing is the fact that I'm pretty much pulling a QML file with the UI from one of my HTTP servers, allowing me to keep the bulk of the code within reach and easily updatable.
My "main QML file" obviously has external dependencies, such as fonts (using FontLoader), images (using Image) and other QML components (using Loader).
AFAIK all those resources are loaded through the Qt networking stack, so I'm wondering what I'll have to do to make all resources available when offline without having to download them all manually to the device.
Is it possible to do this by tweaking existing/implementing my own cache at Qt/C++ level or am I totally on the wrong track?
Thanks!
A simple solution is to invert the approach: include baseline files within your application's executable/bundle. Upon first startup, copy them to the application's data directory. Then, whenever you have access to your server, you can update the data directory.
All modifications of the data directory should be atomic - they must either completely succeed, or completely fail, without leaving the data directory in an unusable state.
Typically, you'd create a new, temporary data folder, and copy/hardlink the files there, and download what's needed, and only once everything checks out you'd swap the old data directory with the new one.
Letting your application access QML and similar resources directly online is pretty much impossible to get right, unless you insist on explicitly versioning all the resources and having the version numbers in the url.
Suppose your application was started, and has loaded some resources. There are no guarantees that the user has went to all the QML screens - thus only some resources will be loaded. QML also makes no guarantees as to how often and when will the resources be reloaded: it maintains its own caches, after all. Sometime then you update the contents on the server. The user proceeds to explore more of the application after you've done the changes, but now the application he experiences is a frankenstein of older and newer pieces, with no guarantees that these pieces are still meant to work together. It's a bad idea.

Does any web site test framework facilitate fine-grained concurrent testing

I have a legacy .NET application that is implemented using Application variables and makes heavy use of Session data as well. There are some anecdotal reports of bugs that seem to point toward concurrency errors, i.e. multiple sessions clobbering shared application-level data.
I want to develop some automated tests that let me control concurrent access in a fine-grained fashion, i.e.
Create two HTTP clients with fresh sessions
Request /my/page/1 with client 1
Request /my/page/2 with client 2
POST data with client 2
POST data with client 1
Issue parallel request for /my/page/results from both clients
etc.
Are there any libraries that make this sort of testing easier or will I have to roll my own to some extent?
I'm aware of Selenium and WatiN, but have not personally used either project. From reading the docs, neither appears to be a good match.
Perhaps the best option is just plain NUnit and making good use of the .NET WebClient class?
Clarification: What you want is a well-defined series of steps to be executed synchronously. This means your test code does not need to be multithreaded, more precisely, it must not be multithreaded, or you lose control over the order the steps are executed.
The only thing you need is two browser instances and the ability to dispatch the test steps to the two browsers.
To do so, I presume you can use either of Selenium or WatiN as follows (not verified, it's only in my mind)
Selenium (with WebDriver):
using (var firefox1 = new FirefoxDriver(profile))
using (var firefox2 = new FirefoxDriver(profile))
{
requestPageOne(firefox1);
requestPageTwo(firefox2);
postPageTwo(firefox2);
postPageOne(firefox1);
//// ...
}
WatiN
using (var firefox1 = new FireFox(url1))
using (var firefox2 = new FireFox(url2))
//// ...

Logging for multi=threaded server component

I have a multi-threaded sever application that I'm writing in C++ and I need to implement a good and fairly efficient logging system. By efficient I mean that whatever amount of logging is configured, the application shouldn't ever come to a grinding halt. So preferably there is some thread that is dedicated to writing it's log files.
I want to log each request that the server component handles in it's own file, having a rotation system that removes files older then some threshold. A request is handled by 2 threads, one that does some conversion work and the a worker-thread that is part of thread pool (BOOST threadpool) that does all the other actions (database gets, calculations, etc). So the logging need be threadsafe and I have to be able to configure it for levels and let each Logger class instance (my own logger that implements some library) accept a new file name. So that each new Logger instance is created for a specific request.
My ultimate question is: Which logging library allows me to have a new Log file for each request and allows me to configure log levels? (IE: error, warning, critical, etc)
Or should I implement everything myself? (no logging is not an option)
I have looked at Boost::Logging v2 and since the main logger object, that holds all state (levels, files) is global, I cannot change the files for each request.
I have looked at templog.org and this I can't even get to compile. No matter what I include or which references I set, it can never find the templog namespace or any of its classes.
Have a look at Apache log4cxx. It a great logging library !