I am wondering what approach/library is considered a good practise in clojure app development for configuration files?
I would like to keep some settings e.g. database connection, credentials to access external web services etc, in the external configuration file.
Thanks
EDN is a good solution. However, you might also like the clojure environ library. This giv3es quite a nice abstraction for using environment variables, java property files, end etc.
Clojure Cookbook has a section exactly on that: https://github.com/clojure-cookbook/clojure-cookbook/blob/master/04_local-io/4-15_edn-config.asciidoc.
Related
I have a pythonic serverless project on AWS, in which several services are contained in a single repository (a monorepo) which looks like:
/
serverless.yml
/service1
lambda_handler.py
/service2
lambda_handler.py
/general
__init__.py
utils.py
'general' is a package that is shared between different services, and therefore we must use a single 'serverless.yml' file at root directory (otherwise it won't be deployed).
We have two difficulties:
A single 'serverless.yml' may be too messy and hard to maintain, and it prevents us from using global configuration (which may be quite useful).
Deplyoing a single service is complicated. I guess that 'package' feature may help but I'm not quite sure how to use it right.
Any advise or best practices to use for this case?
Its better to use individual serverless.yml files per each service. To use the shared code,
You can convert the code into a library and use it as a dependency and installed via a package manager for each individual service similar to a library. (This is useful since updating a version of common code won't affect the other services)
Keep the shared code in a different repository and use git submodule for individual service.
For more information, refer the article Can we share code between microservices which I have originally written considering serverless.
In a scenario where the project's settings.py file is split into base, development, and production, and only the base file is tracked in VCS. Is it a problem if the SECRET_KEY is hard-coded in the production settings file. Or will having it in an environment variable a better choice? If so, why?
Is having it pulled from the system somehow more secure than written in plain text inside the file?
I would say the security for both methods are the same. Written down in a file (which is not committed to the source code repository) or as a environment variable would have the same effect.
If your system is compromised in a way someone got access the server, both methods would expose your security key. So, it wouldn't make much difference.
Now, I would say using environment variable is a better strategy. Not related to security though. But usually it is not a good idea to rely on uncommitted files to run a project. It's one of the causes of the famous in my machine it works problems. And it also make initial setup of a project difficult for newcomers.
For this kind of settings and configuration management, there is a great python library called Python Decouple. It's worth checking it out. I use it in every Django project I work with.
Is there any recommended way to load configuration inside a .env file in clojure?
I've found https://github.com/rentpath/clj-dotenv and https://github.com/jackmorrill/dotenv which seemed to do what I want, but both of them are not available on clojars.org anymore with github activity being very low.
There also is https://github.com/weavejester/environ/ but I have not quite gotten my head around how to use it, since the project.clj is tracked inside my git repository and my configuration (in dev also) contains potentially sensible information such as API tokens.
Any help would be greatly appreciated.
The most basic approach is to edn/read an .edn file that contains a map of configuration. You don't need a library to do this. You just need to manage the file (don't check it in if it contains passwords, but do deploy it to where it needs to go).
Environ is great for getting values from the environment, but how you get them into your environment is up to you. One way would be source an env file before launching your application.
This library https://github.com/outpace/config can help for more complicated needs. It allows you to pull configuration from many different sources (files, environment, or specify something else) in different formats (edn/string).
Ultimately you have to decide where you want configuration to be and how it will get there, both of which are not directly something you do from your Clojure project, but are instead deployment concerns. Feel free to add more specifics if this is missing your needs.
I'm learning about Strongloop, it's pretty good so far.
Question: What is the appropriate place to put AWS keys? config.json? ..and how would I access them from my application?
Thanks
Ideally you would not put those credentials in any file that is committed. I usually find environment variables to be the best balance of convenience and security.
If you are using strong-pm, then you would do this with slc ctl env-set. If you are using some other supervisor, then you'll need to consult its docs.
A lot of times it is enough to use Upstart or systemd directly, which both make it fairly easy to set environment variables in the service process.
Other than above answer, what you can do is put these in your release procedure.
What we have done in our product is all these entries are kept in a config file which is deployed from the shared folder.
Let me elaborate it.
we have local config files in the git. and separate config files on production servers in a folder names as shared, now, when ever a tag release is deployed from git, the shared folder overwrite these config files.
My open source project it is C++ dynamic linking library. Most of bugs - crash.
I want create public symbol server to simplify debugging with memory dump.
See also: Setting up a Symbol Server
I assume you're using Microsoft tools? If so, all you should need to do is expose your 'symstore' directory with a web server then configure debuggers to access that store:
srv*symbol-cache-location*http://your.web.server.com/symboldir
The "Debugging Tools for Windows" docs (debugger.chm) has details for configuring IIS - I'm sure any other HTTP server will work just as well if you don't need authentication, which I imagine would be the case for an open source project. As far as I know, symsrv.dll just makes normal HTTP GET requests for symbol files when it's trying to get them from an HTTP server.
You'll also need to build the symbol store using the 'symstore' utility. Hopefully that can be integrated into your build or packaging process so it happens automatically. Again, debugger.chm has good docs on the tool.
This will not be a real answer, but you might want to take a moment to vote for C++ support in NuGet in work item Support Managed C++ Project Types or have a look at the discussion about C++ Project support. When that gets in, SymbolSource support will follow shortly (currently it only supports hosting symbols for .NET assemblies).