How to add a replication hook to Riak v2.0+ - database-replication

Background
Riak v2.0 changed the configuration system from:
vm.args (set of erlang parameters for the Riak node)
app.config (configuration of various attributes for the Riak node)
to:
riak.conf (replacing the majority of what was in vm.args and app.conf with a single configuration file [using named settings])
advanced.config (a small specific set of settings [mostly related to riak_repl])
Question
In Multi-Data-Center-Replication-Hooks the Riak v2.0+ documentation still refers to:
Add a -pa argument to your vm.args file to specify the path where your
compiled .beam file lives:
Shell
-pa /path/to/replication/hook Finally, add a -run argument to your vm.args file to register the hook:
Shell
-run riak_repl_hook_sample register
What are the riak.conf or advanced.config versions of the -pa & -run parameters?
Note: I know that in the doc:Upgrading Your Configuration System we can:
Keep your configuration files from the older system, which are still
recognized in Riak 2.0.
but we would like to move to the new configuration system as it gives us more visibility.

in your advanced.config file, try someting like that:
[
{riak_kv, [
{add_paths, ["/path/to/replication/hook"]}
]},
{vm_args, [
{'-run riak_repl_hook_sample register', ""}
]}
].
I know that starting an app works, I'm using this:
[
{riak_kv, [
{add_paths, ["/path/to/my/beams/"]}
]},
{vm_args, [
{'-s my_app', ""}
]}
].
So with a little fiddling you should be able to make it work :)

Possible solution
Note: If you want to understand how riak.conf settings are mapped to vm.args the below file is very useful, however be warned that if you do not use the below settings you will end up with blank entries in the generated vm.args file:
/var/lib/riak/generated.configs/vm..args
-pa
-run
Riak v2.0+ schema file for config
Turns out that there is a nice schema file in the riak lib directory that can be expanded to add on any missing erlang flags, for me this was:
/usr/lib64/riak/lib/10-riak.schema
To this I added:
%% Add the replication hook directory to the code path
{mapping, "repl.hook.path", "vm_args.-pa", [
{default, ""}
]}.
%% Call Module:start/0
{mapping, "repl.hook.run", "vm_args.-run", [
{default, ""}
]}.
Then I added to the riak.conf file:
# add the repl hook path and register it
repl.hook.path = /path/to/replication/hook
repl.hook.run = riak_repl_hook_sample register
after this I needed to restart the node.

Related

Generation of jacoco.exec using bazel not permitted in other paths except /tmp

In my BUILD.bazel my java_test looks like this:
java_test(
name = "SomeServiceTest",
srcs = [
"src/test/java/com/service/SomeServiceTest.java",
],
test_class = "com.service.SomeServiceTest",
deps = [
"SomeService",
"#junit_junit//jar",
"#commons_logging_commons_logging//jar",
"#org_hamcrest_hamcrest_core//jar",
"#com_fasterxml_jackson_core_jackson_annotations//jar",
"#javax_servlet_javax_servlet_api//jar",
"#org_springframework_spring_aop//jar",
"#org_springframework_spring_beans//jar",
"#org_springframework_spring_context//jar",
"#org_springframework_spring_test//jar",
"#org_springframework_spring_web//jar",
"#org_mockito_mockito_core//jar",
"#net_bytebuddy_byte_buddy//jar",
],
size = "medium",
jvm_flags = ["-javaagent:$$workspacepath/jacocoagent-runtime.jar=destfile=$$workspacepath/jacoco.exec"]
)
I want to make the path of jacocoagent-runtime.jar and the path where the jacoco.exec will be generated to be dynamic, thus, the jvm_flags setup. I defined $$workspacepath in my execution of bazel test below:
bazel test --test_output=all --action_env=workspacepath=/Users/Someone/Desktop some-service:all_tests
Now, I am getting the error below:
java.io.FileNotFoundException: /Users/Someone/Desktop/jacoco.exec (Operation not permitted)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at org.jacoco.agent.rt.internal_290345e.output.FileOutput.openFile(FileOutput.java:67)
at org.jacoco.agent.rt.internal_290345e.output.FileOutput.writeExecutionData(FileOutput.java:53)
at org.jacoco.agent.rt.internal_290345e.Agent.shutdown(Agent.java:137)
at org.jacoco.agent.rt.internal_290345e.Agent$1.run(Agent.java:54)
If I change the workspacepath to /tmp, it works fine. What is wrong with other paths other than /tmp?
I agree with #Godin -- sounds like the input path is not in the sandbox. Does --spawn_strategy=standalone[1] help?
If that's indeed the problem then to fix the build with sandboxing you need to make the .jar file an input of the java_test's action and reference its path correctly from the jvm_flags.
To do that:
either create a new package in your workspace and copy the jacoco jar there, or add a new_local_repository rule to your WORKSPACE file and reference the jar's directory and specify the build_file_contents attribute as exports_files(["jacoco-runtime.jar"])
now that you can reference Jacoco by a label (e.g. #jacoco//:jacoco-runtime.jar) you need to add it to the java_test rule's data attribute
finally you need to change the java_test rule's jvm_flags attribute to reference the jar using $(location <label>), e.g. $(location #jacoco//:jacoco-runtime.jar)
[1] https://docs.bazel.build/versions/master/user-manual.html#flag--spawn_strategy

How to turn on .sass extensions in Ionic 2?

I have set up a basic project and only .scss files are picked up. I would like to write my CSS with the .sass format. How can I turn that on?
You can override the app script config files:
https://github.com/driftyco/ionic-app-scripts#overriding-config-files
In my package.json, I added the following:
"config": {
"ionic_sass": "./config/sass.config.js"
}
I created the sass.config.js (copy from the github project) and added the sass extension:
includeFiles: [
/\.(scss|sass)$/i
],
It's unfortunately impossible to update the watch config file, so I directly edited it in the node module:
Under #ionic/app-scripts/config/watch.config.js
Add the following .sass line below .scss:
'{{SRC}}/**/*.scss',
'{{SRC}}/**/*.sass'

How publish a directory with ring middleware?

I am trying to use Clojure + Compojure + Ring in combination with the qooxdoo JS library. This is actually going well, but qooxdoo runs in two modes "build" (that works for me) and "source" (not so good). In the latter case, the JS generated by qooxdoo actually hardcodes references (well, using relative addresses ../../..) back to the qooxdoo installation and at run time it asks for sth like:
http://localhost:3000/opt/qooxdoo-5.0.1-sdk/framework/source/class/qx/bom/client/OperatingSystem.js
...since I have the library installed under /opt/qooxdoo-5.0.1-sdk.
Serious sanity check: if I directly open the index.html in the browser, it works fine. So it seems I just have to figure out how to serve the static files under the /opt library install.
I have tried wrap-file from the ring.middleware.file because that sounds like what I want but no matter what path I give it I get hundreds of 404s as it tries to pick up each framework file individually from the installed library.
I can work OK under "build" (qooxdoo cobbles together a single minified .js I serve with wrap-resource) but on occasion I need to run in source mode to find JS bugs.
Am I missing something simple?
The correct way to handle this is to configure Qooxdoo to tell it what URIs you would like to use - by default the source build does just use relative paths, but you can easily override this by editing the config.json.
In your config.json you will have a "jobs" section containing a "libraries" section, containing a "library" array - your application is a library, as is Qooxdoo, as is any contribs so it will look something like this:
"jobs" : {
"libraries" : {
"=library" : [ {
"manifest" : "${QOOXDOO_PATH}/framework/Manifest.json"
}, {
"manifest" : "Manifest.json"
}
},
Each "library" object can have a "uri" property, so for your example you probably want something like this:
"jobs" : {
"libraries" : {
"=library" : [ {
"manifest" : "${QOOXDOO_PATH}/framework/Manifest.json",
"uri" : "/opt/qooxdoo-5.0.1-sdk"
}, {
"manifest" : "Manifest.json"
}
},
Simple indeed: (wrap-file "/")
In development/source mode qooxdoo works off the installation directory instead of code pulled into my local tree, and does so by hardcoding relative paths that resolve to an absolute /opt/qooxdoo-etc path.
This looks to the server like a relative "opt/qooxdoo..." file request, so I had to offer root ("/") as a search directory.
(wrap-file "/") is a serious security issue since you're offering the root directory for all.
What you can potentially do is to make a dedicate directory to server your static file, and serve your content from there.
su
mkdir -p /var/clojure/static/opt
cd /var/clojure/static/opt
ln -s /opt/qooxdoo-5.0.1-sdk qooxdoo-5.0.1-sdk
chown -r YOUR-USER-ID /var/clojure/static/opt
And use the middleware: (wrap-file "/var/clojure/static") to serve your file.
There is a section in the manual that deals with the issue of serving a source version through a web server.
I guess you found the basic insight yourself, that the web server must export a root where the relative paths used in the generated loader match URL paths under that web server. (The rational behind this is that the source version uses all the JS files from all involved libraries directly from disk.)
In the worst case that might mean you need to export the file system root ("/") from your web server. As you are doing this on a local development machine this shouldn't be much of a problem. If security is a concern you might want to collect your qooxdoo environment under some innocent path like /home/kenny/devel/qooxdoo, containing the qooxdoo SDK, your app, and all libs/contribs you might be using.
If you follow the above link you will also find some help from the qooxdoo tool chain. E.g. if you run generate.py source-httpd-config[*] it will tell you which path on your local disk is the closest parent directory that will encompass all necessary libraries, i.e. needs to be exported in your web server for the source version to work.
Alternatively, as John wrote you can export each qooxdoo lib through an individual path under your web server, and then tell your main application where to find it. You might actually need to give a proper URL like http://localhost/libs/qooxdoo-5.0.1-sdk/framework, not just a file system path as John suggests. Also remember that you have to go as far as the directory where the Manifest file resides (hence the above path ending in .../framework). See this manual section for a deep dive.

Building multiple bundles with r.js

Hopefully a low-ball conceptual question. I'm having trouble understanding the modules option of r.js build configs. I want to build multiple modules with nested dependencies with one r.js build config.
Say I have the following project structure:
+---build
+---src
+---moduleOne
| +---moduleOne.js
| \---dependecyForModuleOne.js
+---moduleTwo
| +---moduleOne.js
| \---dependecyForModuleTwo.js
|---buildConfig.js
|---devModuleConfig.js
\---prodModuleConfig.js
devModuleConfig and prodModuleConfig are the dev and prod runtime configs, and buildConfig.js is the r.js build config.
Now, I can build moduleOne no problem using this config:
({
"baseUrl": "./",
"name": "moduleOne/moduleOne",
"out": "../build/moduleOneBundle.js",
mainConfigFile: 'devModuleConfig.js',
optimize: 'none'
})
I end up with a bundle in build that I can run after specifying different paths in the build config:
+---build
\---moduleOneBundle.js
I want to build two modules, so specify moduleOne using modules config option:
({
"baseUrl": "./",
modules: [
{
"name": "moduleOne/moduleOne",
"out": "../build/moduleOne.js"
}
],
dir:"../build", // <-- r.js says I need to add this. why?
mainConfigFile: 'devModuleConfig.js',
optimize: 'none'
})
As well as the required dir option, I get all my configs in the build dir, but I did not specify them, and I do not get my bundled module, and I get a text file containing r.js build output. In fact, build ends up looking exactly the same as src:
+---build
+---moduleOne
| +---moduleOne.js
| \---dependecyForModuleOne.js
+---moduleTwo
| +---moduleOne.js
| \---dependecyForModuleTwo.js
|---build.txt
|---buildConfig.js
|---devModuleConfig.js
\---prodModuleConfig.js
How do I configure multiple modules to build using one r.js config? I've read the docs a few times and can't get my head around it.
You can see my project that contains all this here: https://github.com/sennett/r.js-multiple-modules
When specifying modules, r.js includes all files then optimizes ones based on the rules specified in modules. removeCombined: true deletes modules from build that were combined into another module, and then manual deletion of the other files can be used to manually clean up any unwanted build artefacts.

Nodemon - exclusion of files

I would like to exclude some specific files from monitoring of NodeMon. How can I do this?
My existing configuration:
nodemon: {
all: {
script: 'app.js',
options: {
watchedExtensions: ['js']
}
}
In order to make NodeMon ignore a bunch of files from monitoring, you can start it as
nodemon --ignore PATTERN [--ignore PATTERN2]
where PATTERN is the name of a specific file, directory, or wildcard pattern. Make sure that if you use a wildcard, it is escaped.
For example
nodemon --ignore 'lib/*.js' --ignore README
Alternatively, if you want to configure that behaviour instead, try creating a nodemon.json file in your current working directory, or your home directory. You can configure ignoring some files by adding something like the following to this config file:
{
"ignore": ["lib/*.js", "README"]
}
Refer the README file at https://github.com/remy/nodemon for more details.
You can add nodemon configuration within package.json file
For example:
{
"name": "nlabel",
"version": "0.0.1",
// other regular stuff
"nodemonConfig": {
"ignore": ["public/data/*.json", "public/javascripts/*.js"]
},
"author": "#sziraqui",
"license": "GPL-3.0"
}
The key must be "nodemonConfig". Ignore rules can be specified as array of globs or complete filenames
More info: https://github.com/remy/nodemon#packagejson
For me (Mac and nodemon 1.18.3), the only way to ignore entire directories is to run e.g.
nodemon --ignore "**/old/**"
with the double quote and **. The config file won't work.
I have set up an alias like this:
alias nm='nodemon server.js -i "**/old/**" -i "**/img/**"'
Check what files are monitored by running
DEBUG=nodemon:watch nodemon server.js -i "**/old/**" -i "**/img/**"
-i is an alternative to --ignore. Check out the available parameters with nodemon --help
If like me nothing is working for you, follow my instructions.
Do not use the '' around your path('login.json' is wrong for example)
And CTRL + S won't suffice, you need to close the terminal and use the command npm run devStart AGAIN if you want your changes to take place in your packages.json
"scripts": {
"devStart": "nodemon server.js --ignore login.json"
},