In a Django project, I want to keep user uploaded files secure on the server. Should this be done at the OS level (we are using ubuntu) or at the application level?
Encrypting at the application level will be easier to maintain. But, aside from some drawbacks like possible negative effect on performance, I am not even sure if this will have any point. If a hacker compromises the server, he will also have access to the encryption keys and how it is encrypted / decrypted.
Any suggestions are greatly appreciated. Thanks.
How you protect your data depends on what kinds of attacks you want to protect against. Of course, you probably don't know how an attacker is most likely to compromise your system, unless there are certain threat models you're particularly trying to protect against, like say a rogue sysadmin.
The attacker might gain access to the OS that the web server is running on. In this case, filesystem level encryption probably does you no good. In fact file-system level encryption is probably only useful protection against somebody walking off with the physical server (which is a totally valid threat model). However, if the files are encrypted with keys stored in the database, then an attacker who has access to the webserver OS but not the database is thwarted.
In contrast, an attacker might gain access to the database but not the OS, through a hole in your application. I would expect this to be less likely since modern operating systems present huge and well-studied attack surfaces.
To protect your user's data against an attacker with full access to your servers is very difficult. You need to encrypt the data with a key that your servers don't have. This could be something like a password or a key stored in a user cookie. The problem with all these schemes is that users can't be trusted to hold on to critical data like this -- they always want a way to reset their password if they forget. In most cases, it's not realistic to protect data against an attacker with full access to your OS and your database.
So I'd choose what you're trying to protect against. Personally, I'd expect an OS penetration to be most likely, and thus encrypt the files with keys that are stashed in a part of the database that is extra protected somehow. The challenge here is that the OS has to store database login credentials (in settings.py) in order for the web app to function. So try to keep those files as restricted as possible within the OS i.e. chmod 600 on a user account that does as little else as possible.
You're right that if the key used to encrypt the files is stored on the server you don't get a whole lot of added security by encrypting the files.
However, if you use a key provided by the user, then you do get some security. For example, if you store the encryption key in a cookie, then it will only be available for the duration of each request. I don't believe this will create any new security issues (if an attacker can steal the cookie, they can also steal the user's session), and it will make it much harder for an attacker to access files belonging to users who aren't currently online.
If you're really paranoid, you could do what 1Password does, and send encrypted data back to the browser, which can decrypt it with JavaScript encryption routines…
Related
Google CloudSQL documentation states that the data is encrypted in transit and at rest.
I'm using pgcrypto in a Django app to encrypt sensitive information. However I'm wondering if there's any point in doing this since it's already encrypted at rest. The only thing I can imagine is an event where the Google App Engine server with the deployed code gets compromised and the password to the database is somehow leaked - the hackers would eventually have access to unencrypted data as they 'read' it in. But then even with pgcrypto, in the event the GAE server is compromised, they'd still be able to run code to fetch unencrypted data.
Am I overthinking this? The goal is to provide total piece of mind to the end-user with as many 'hurdles' introduced as possible to ensure their data stays completely secure. I have a feeling I don't really need pgcrypto, but looking for an educated reply.
The rather educated answer is: Yes.
Underlying encryption offered by CloudSQL is like FileVault offered by OS X - your stuff truly is encrypted, but if you're logged in, everything is world-readable to you.
The main worry is that you, or someone who is able to compromise your server, is able to read data in plain-text. Data needs to to be encrypted, and I've personally gone beyond the default AES 128-bit offered by most databases and switched to AES 256-bit with initialisation vector (a different one for each encrypted content). This will ensure that the data is encrypted, inaccessible and unreadable by even yourself. Yes, your code is eventually able to decrypt but storing and protecting the decryption keys is a different topic altogether.
I'm launching a Python Django web app on Heroku, using the default PostgreSQL database. I'll also be using a AWS S3 to store some files. The client I'm creating the site for is rightly concerned with security and asks is we can encrypt the database and the files stored in S3.
Am I correct in saying the only benefit that encryption will have, is it will protect our data in the unlikely case somebody breaks into one of Amazon's datacenters and happens to steal a hard drive on which our data is located?
I've come to the following conclusions:
Unless somebody gets hold of my AWS credentials or Heroko login details, the data is as safe as it can be.
Also, even if the data is encrypted and they get hold of my credentials/login details, they will still be able to read the data.
The key in keeping the site secure is just making sure nobody gets hold of my credentials/login details.
It is therefore not necessary to encrypt the database and files unless we believe there is a strong possibility of somebody breaking into an AWS datacenter.
Are my statements above correct?
I am working on a web project and I want to (as far as possible) handle user data in a way that reduces damage to the users privacy in case of someone compromising our servers/databases.
Of course we only have user dat'a that is needed for the website to do it's job but because of the nature of the project we have quite a bit of information on our users (part of the functionality is to apply yourself to jobs and sending your cv with it)
We thought about encrypting/decrypting sensitive data with a private/public keypair of which the private key is encrypted with the users password but found some security and implementation problems with that :P
the question is how do you implement user privacy and a protection against data theft on centralised web sever with browser compatible protocols while for functionality it is required that users can exchange sensible data?
To give some additional insight: this project is not yet in production stage so there is still time to make things right.
we are already doing some basic stuff like
serving https
enforcing https for sites that may handle sensitive data
hashing salted passwords
some hardening of our server and services on it
encrypted harddrives to prevent someone from reading all client information after stealing our servers / harddrives
but that's about it, there is besides the password hashes no mechanism that would stop/at least make it harder for someone who managed to get into (part of) the server to gain all data on all our users. Nor do we see a way to encrypt user data to disable our self from reading them as we need the data (we wouldn't have collected it otherwise) for some part of the website / the functionality we want it to provide. Even if we for example managed somehow (maybe with some javascript) that all data would get to us encrypted (by the client's browser) and we serve the client his privatekey encrypted with some passphrase (like for example his login password) we could not for examle scan user uploaded files for viruses and the like. On the other hand would a client side encryption at least with the browser/webserver concept leave some issues with security at least as we imagine it (you are welcome to prove me wrong) and seems quite like reinventing the wheel, and maybe as this project is not primarily about privacy, but rather privacy is a prefarable property we might not want to reinvent the wheel for it. I strongly believe I am not the first webdeveloper thinking about this, am I? So what have other projects done? What have you done to try to protect your users data?
if relevant we are using django and postrgreSQL for most things and javascript for some UI
The common way to deal with this issue is to split (partition) your data.
Keep minimal data on the Internet-facing web server and pass any sensitive data as quickly as possible to another server that is kept inside a second firewall. Often, data is pulled from the web server by the internal secure server to further increase security. This is how banks and finance houses handle sensitive data from the internet (or at least they should). There is even a set of standards (PCI) that cover the secure handling of credit card transactions that explain all of this in mind-numbing detail.
To further secure the internal server, you can put it on a separate network and secure physical access to it. You can also focus other security tools on it such as Data Loss Protection and Intrusion Protection.
In addition, if you have any data that you don't need to see in the clear, use a client-side encryption library to encrypt it locally. There are still risks of course since the users workstation might be compromised by malware but it still removes risks during data transmission and from server storage risks. It also puts responsibility onto the user rather than just on to your central servers.
You already seem to be a long way ahead of most web developers in ensuring that your customers are kept safe and secure. One other small change it would be worth considering would be to turn on enforced HTTPS for all transactions with your site. That way, there is very little chance of unexpected data leakage such as data being unexpectedly cached.
UPDATE:
Client side encryption can help a lot since it puts the encryption responsibility on the user. Check out LastPass for example. Without doing the encryption client-side, you could never trust the service. Similarly with backup services where you set your key locally so that the backups can never be unlocked by someone on the server - they never have the key.
Partitioning is one of the primary methods for enterprises to secure services that have Internet facing components. As I said, typically, the secure server PULLs data from the less secure one so the less secure server can never have any access to anything more secure even if fully compromised. Indeed there will be a firewall that prevents any traffic from the DMZ (where the less secure service is located) getting to the secure network. Only connections from the secure side are allowed through and they will be tightly controlled by security processes. In a typical bank or other high security setting, you may well find several layers like this, each of which having separate security controls, all partitioned from each other enforcing separation of data and security.
Hope that adds some clarity. Continue to ask if not!
UPDATE 2:
Even for simple, low cost setups, I would still recommend partitioning. For a low cost version, consider having two virtual servers with the dedicated firewall replaced by careful control of the software firewall on the more secure server. Follow the same principals outlined above for everything else.
I am developing an application that works with PostgreSQL and other database features that require a username and password to login to the remote server to store the data. The user has to be able to set the username and password which would then be stored in a configuration file on disk. The problem is that anybody can open the configuration file and read the credentials creating a serious security problem.
I have done some research on encrypting the whole configuration file and then de-crypting it when needed, but the problem is that a hacker could put the program though a debugger and easily find out the decryption key. What is the best method to keep configuration data secret on Windows using C/C++?
The moment an Attacker is able to attach a debugger to your running program is the moment the game is over. Being able to debug your program means that your user account or the underlying OS is compromised, which means every security measure on your app's behalf is futile. The attacker will (with knowledge, persistence and motivation) know everything you enter into your computer, or have entered and stored before.
The user has to be able to set the username and password which would
then be stored in a configuration file on disk
This is the weak spot and this is what you need to change.
(On a side note, is the password you store never going to change? That's another security weak spot.)
As stated in Eugen Rieck's answer, if the attacker has physical access to your system he will, in time, break all your defenses.
The simple solution is clear: don't let him have access to the system that handles security/authorization. Have the SQL server on a dedicated, remote machine and let it handle the username/password validation.
Or, make your app multi tiered with part on a remote machine that handles the user authentication and routs your DB queries.
This will mean that your user will have to login every time they start your application.
(Preferably also after a pre-set period of inactivity.)
It all depends on how safe you need to be. It's important to understand that security is not easy to create and you should always try to use existing frameworks if possible.
I'm fleshing out an idea for a web service that will only allow requests from desktop applications (and desktop applications only) that have been registered with it. I can't really use a "secret key" for authentication because it would be really easy to discover and the applications that use the API would be deployed to many different machines that aren't controlled by the account holder.
How can I uniquely identify an application in a cross-platform way that doesn't make it incredibly easy for anyone to impersonate it?
You can't. As long as you put information in an uncontrolled place, you have to assume that information will be disseminated. Encryption doesn't really apply, because the only encryption-based approaches involve keeping a key on the client side.
The only real solution is to put the value of the service in the service itself, and make the desktop client be a low-value way to access that service. MMORPGs do this: you can download the games for free, but you need to sign up to play. The value is in the service, and the ability to connect to the service is controlled by the service (it authenticates players when they first connect).
Or, you just make it too much of a pain to break the security. For example, by putting a credential check at the start and end of every single method. And, because eventually someone will create a binary that patches out all of those checks, loading pieces of the application from the server. With credentials and timestamp checks in place, and using a different memory layout for each download.
You comment proposes a much simpler scenario. Companies have a much stronger incentive to protect access to the service, and there will be legal agreements in effect regarding your liability if they fail to protect access.
The simplest approach is what Amazon does: provide a secret key, and require all clients to encrypt with that secret key. Yes, rogue employees within those companies can walk away with the secret. So you give the company the option (or maybe require them) to change the key on a regular basis. Perhaps daily.
You can enhance that with an IP check on all accesses: each customer will provide you with a set of valid IP addresses. If someone walks out with the desktop software, they still can't use it.
Or, you can require that your service be proxied by the company. This is particularly useful if the service is only accessed from inside the corporate firewall.
Encrypt it (the secret key), hard-code it, and then obfuscate the program. Use HTTPS for the web-service, so that it is not caught by network sniffers.
Generate the key using hardware speciffic IDs - processor ID, MAC Address, etc. Think of a deterministic GUID.
You can then encrypt it and send it over the wire.