I am using a config file to store passwords / keys for DB and connection to other services like AWS.
I am using Travis CI for build, and running my tests, and then deploying it to live server.
I would like to encrypt the variables in my config file and only the application should be able to read it. This is the criteria -
1) The application should be able to decrypt it in multiple environments, from the build server to multiple deployment servers.
2) The password used to decrypt the config file is not avalaible to the developers.
The project is in clojure, are there any leiningen plugins / features that will aid in this ?
Thanks,
Murtaza
I think the best solution to what you are trying to do is simply to remove the config files from the application source tree. Save the config files to the build and deployment severs and don't give the developers direct access to those.
The chief problem with what you are trying to do is that you are trying to make it impossible for your developers to have access to something but at the same make it possible for something that the developers have complete control over, the application code, to have access to the same thing. That is, for the application to be able to decrypt the configs, the developers have to have access to the decyption keys.
You could try making it only possible to decrypt your configs through tools available only on the build and deployment servers, (ie a Leiningen plugin,) but it would be simpler to just store the config files separately.
Related
I am deploying a Django application using the following steps:
Push updates to GIT
Log into AWS
Pull updates from GIT
The issue I am having is my settings production.py file. I have it in my .gitignore so it does not get uploaded to GITHUB due to security. This, of course, means it is not available when I PULL updates onto my server.
What is a good approach for making this file available to my app when it is on the server without having to upload it to GITHUB where it is exposed?
It is definitely a good idea not to check secrets into your repository. However, there's nothing wrong with checking in configuration that is not secret if it's an intrinsic part of your application.
In large scale deployments, typically one sets configuration using a tool for that purpose like Puppet, so that all the pieces that need to be aware of a particular application's configuration can be generated from one source. Similarly, secrets are usually handled using a secret store like Vault and injected into the environment when the process starts.
If you're just running a single server, it's probably just fine to adjust your configuration or application to read secrets from the environment (or possibly a separate file) and set those values on the server. You can then include other configuration settings (secrets excluded) as a file in the repository. If, in the future, you need more flexibility, you can pick up other tools in the future.
I have an application I am currently working on for which I am integrating Travis CI. I am running into the problem of API keys being accessed by Travis. Given below is my current setup (without Travis):
I have a config.py (and is git ignored) that has API keys for all my interfacing applications. I use ConfigParser to read this file and get the required keys.
Travis asks me to look at environment variables as an option to encrypt the keys and add them to .travis.yml. How would Travis know or what needs to be done in order to make travis know that a particular key belongs to a specific interfacing application. Does there need to be changes to the code?
You would read these important variables into your application as system variables.
However, this will only work for builds that are run against master. These environment variables aren't available for builds that are run as part of pull requests.
I'm developing an tool using Django for internal use at my organization. It's used to search and tag documents (using Haystack and Solr), and will be employed on different projects. My team currently has a working prototype and we want to deploy it 'in the wild.'
Our security environment is strict. Project documents are located on subfolders on a network drive, and access to these folders is restricted based on users' Windows credentials (we also have an MS SQL server that uses the same credentials). A user can only access the projects they are involved in. Since we're an exclusively Microsoft shop, if we want to deploy our app on the company intranet, we'll need to use an IIS server to deal with these permissions. No one on the team has the requisite knowledge to work with IIS, Active Directory, and our IT department is already over-extended. In short, we're not web developers and we don't have immediate access to anybody experienced.
My hacky solution is to forgo IIS entirely and have each end user run a lightweight server locally (namely, CherryPy) while each retaining access to a common project-specific database (e.g. a SQLite DB living on the network drive or a DB on the MS SQL server). In order to use the tool, they would just launch an all-in-one batch script and point their browser to 127.0.0.1:8000. I recognize how ugly this is, but I feel like it leverages the security measures already in place (note that never expect more than 10 simultaneous users on a given project). Is this a terrible idea, and if so, what's a better solution?
I've dealt with a similar situation (primary development was geared toward a normal deployment situation, but some users have a requirement to use the application on a standalone workstation). Rather than deploy web and db servers on a standalone workstation, I just run the app with the Django internal development server and a SQLite DB. I didn't use CherryPy, but hopefully this is somewhat useful to you.
My current solution makes a nice executable for users not familiar with the command line (who also have trouble remembering the URL to put in their browser) but is also relatively easy development:
Use PyInstaller to package up the Django app into single executable. Once you figure this out, don't continue to do it by hand, add it to your continuous integration system (or at least write a script).
Modify the manage.py to:
Detect if the app is frozen by PyInstaller and there are no arguments (i.e.: user executed it by double clicking it) and if so, then run execute_from_command_line(..) with arguments to start the Django development server.
Right before running the execute_from_command_line(..), pop off a thread that does a time.sleep(2) (to let the development server come up fully) and then webbrowser.open_new("http://127.0.0.1:8000").
Modify the app's settings.py to detect if frozen and change things around such as the path to the DB server, enabling the development server, etc.
A couple additional notes.
If you go with SQLite, Windows file locking on network shares may not be adequate if you have concurrent writing to the DB; concurrent readers should be fine. Additionally, since you'll have different DB files for different projects you'll have to figure out a way for the user to indicate which file to use. Maybe prompt in app, or build the same app multiple times with different settings.py files. Variety of a ways to hit this nail...
If you go with MSSQL (or any client/server DB), the app will have to know the DB credentials (which means they could be extracted by a knowledgable user). This presents a security risk that may not be acceptable. Basically, don't try to have the only layer of security within the app that the user is executing. The DB credentials used by the app that a user is executing should only have the access that the user is allowed.
I'm learning about Strongloop, it's pretty good so far.
Question: What is the appropriate place to put AWS keys? config.json? ..and how would I access them from my application?
Thanks
Ideally you would not put those credentials in any file that is committed. I usually find environment variables to be the best balance of convenience and security.
If you are using strong-pm, then you would do this with slc ctl env-set. If you are using some other supervisor, then you'll need to consult its docs.
A lot of times it is enough to use Upstart or systemd directly, which both make it fairly easy to set environment variables in the service process.
Other than above answer, what you can do is put these in your release procedure.
What we have done in our product is all these entries are kept in a config file which is deployed from the shared folder.
Let me elaborate it.
we have local config files in the git. and separate config files on production servers in a folder names as shared, now, when ever a tag release is deployed from git, the shared folder overwrite these config files.
Where should the configuration.yml file of Dropwizard be saved?
I'm using Dropwizard which is a Java web framework.
Dropwizard uses configuration.yml files to load in environment specific configuration files.
In the example I found online the configuration.yml files contains username and password of databases.
Now the question is where to save this configuration files which contain password in plain text.
OPTION 1 GIT REPOSITORY
In the example the configuration.yml are part of the project. So one could keep them in the git repository with the rest of the code. This though is a well-known bad security practice.
If someone crack the git repository has access to the code and to the database. Also this way every single developer has access to all the passwords of all the environments.
OPTION 2 FILE ON THE COMPUTER
Safe the configuration.yml on the machine but do not store on the git repository
OPTION 3 ENVIRONMENT VARIABLES
Use configuration.yml file which point to environment variables on the specific machine.
This is not so practical since all this environment variables needs to be set manually on all the machines. Also what is the syntax to use ENVIRONMENT VARIABLES in Dropwizard's configuration.yml files?
I'd go with environment variables if you cannot control read access to the config file or are concerned that your machine is owned by an untrusted third party.
Environment variables are trivial to script.
You should use a file on the computer: this is how many frameworks out there work.
If you use a unix/linux server you can chmod 0600 [filename] and be sure that nobody (almost nobody as root can do anything) can read that file.
On the dropwizard ML it was also cited to use software like puppet/chef to deploy your application and using these frameworks to handle all variables (eg: different configurations for test/staging/production).
Bye
Piero