Google CloudSQL documentation states that the data is encrypted in transit and at rest.
I'm using pgcrypto in a Django app to encrypt sensitive information. However I'm wondering if there's any point in doing this since it's already encrypted at rest. The only thing I can imagine is an event where the Google App Engine server with the deployed code gets compromised and the password to the database is somehow leaked - the hackers would eventually have access to unencrypted data as they 'read' it in. But then even with pgcrypto, in the event the GAE server is compromised, they'd still be able to run code to fetch unencrypted data.
Am I overthinking this? The goal is to provide total piece of mind to the end-user with as many 'hurdles' introduced as possible to ensure their data stays completely secure. I have a feeling I don't really need pgcrypto, but looking for an educated reply.
The rather educated answer is: Yes.
Underlying encryption offered by CloudSQL is like FileVault offered by OS X - your stuff truly is encrypted, but if you're logged in, everything is world-readable to you.
The main worry is that you, or someone who is able to compromise your server, is able to read data in plain-text. Data needs to to be encrypted, and I've personally gone beyond the default AES 128-bit offered by most databases and switched to AES 256-bit with initialisation vector (a different one for each encrypted content). This will ensure that the data is encrypted, inaccessible and unreadable by even yourself. Yes, your code is eventually able to decrypt but storing and protecting the decryption keys is a different topic altogether.
Related
So I'm new to the whole cloud computing infrastructure and I'm trying to grasp the best practices and today it came to my mind. How do I store sensitive data in AWS what services do I need to utilize and what architecture shall I build for it, I wrote a scenario down to further explain my question.
Let's say I have a user registration and I need from every user to input a secret key that I need to access some kind of 3rd party service on their behalf (let's assume that it's the only to access that service and no other way to access it) how do I store it in my database let's say RDS for example without compromising other IAM users from accessing the database but all they say is an encrypted secret key not the plain text.
I searched online and found some saying KMS and some saying Secrets Manager some saying Backend Encrypting and some says Frontend Encrypting which way shall I go with?
Whoever decides to answer this question thanks in advance but please elaborate as much as you can because I'm still trying the get the concepts and trying to leverage the "Cloud" capabilities as much as possible.
Two common approaches would be to encrypt the secret key at either a) the application level, or b) the database level. To encrypt the key inside your application, you would use some reliable encryption method, such as SHA-256 or SHA-512. The key would be encrypted and non accessible even before you write it out to your database as binary content. To encrypt at the database level, there are a number of options, depending on your particular database. If your RDBMS support encrypted columns, then, from your application, you may simply write out the secret key to its column. The database would then automatically handle encrypting on the way in, and also decrypting the secret key on the way out, when you go to read it.
I'm launching a Python Django web app on Heroku, using the default PostgreSQL database. I'll also be using a AWS S3 to store some files. The client I'm creating the site for is rightly concerned with security and asks is we can encrypt the database and the files stored in S3.
Am I correct in saying the only benefit that encryption will have, is it will protect our data in the unlikely case somebody breaks into one of Amazon's datacenters and happens to steal a hard drive on which our data is located?
I've come to the following conclusions:
Unless somebody gets hold of my AWS credentials or Heroko login details, the data is as safe as it can be.
Also, even if the data is encrypted and they get hold of my credentials/login details, they will still be able to read the data.
The key in keeping the site secure is just making sure nobody gets hold of my credentials/login details.
It is therefore not necessary to encrypt the database and files unless we believe there is a strong possibility of somebody breaking into an AWS datacenter.
Are my statements above correct?
I am working on a web project and I want to (as far as possible) handle user data in a way that reduces damage to the users privacy in case of someone compromising our servers/databases.
Of course we only have user dat'a that is needed for the website to do it's job but because of the nature of the project we have quite a bit of information on our users (part of the functionality is to apply yourself to jobs and sending your cv with it)
We thought about encrypting/decrypting sensitive data with a private/public keypair of which the private key is encrypted with the users password but found some security and implementation problems with that :P
the question is how do you implement user privacy and a protection against data theft on centralised web sever with browser compatible protocols while for functionality it is required that users can exchange sensible data?
To give some additional insight: this project is not yet in production stage so there is still time to make things right.
we are already doing some basic stuff like
serving https
enforcing https for sites that may handle sensitive data
hashing salted passwords
some hardening of our server and services on it
encrypted harddrives to prevent someone from reading all client information after stealing our servers / harddrives
but that's about it, there is besides the password hashes no mechanism that would stop/at least make it harder for someone who managed to get into (part of) the server to gain all data on all our users. Nor do we see a way to encrypt user data to disable our self from reading them as we need the data (we wouldn't have collected it otherwise) for some part of the website / the functionality we want it to provide. Even if we for example managed somehow (maybe with some javascript) that all data would get to us encrypted (by the client's browser) and we serve the client his privatekey encrypted with some passphrase (like for example his login password) we could not for examle scan user uploaded files for viruses and the like. On the other hand would a client side encryption at least with the browser/webserver concept leave some issues with security at least as we imagine it (you are welcome to prove me wrong) and seems quite like reinventing the wheel, and maybe as this project is not primarily about privacy, but rather privacy is a prefarable property we might not want to reinvent the wheel for it. I strongly believe I am not the first webdeveloper thinking about this, am I? So what have other projects done? What have you done to try to protect your users data?
if relevant we are using django and postrgreSQL for most things and javascript for some UI
The common way to deal with this issue is to split (partition) your data.
Keep minimal data on the Internet-facing web server and pass any sensitive data as quickly as possible to another server that is kept inside a second firewall. Often, data is pulled from the web server by the internal secure server to further increase security. This is how banks and finance houses handle sensitive data from the internet (or at least they should). There is even a set of standards (PCI) that cover the secure handling of credit card transactions that explain all of this in mind-numbing detail.
To further secure the internal server, you can put it on a separate network and secure physical access to it. You can also focus other security tools on it such as Data Loss Protection and Intrusion Protection.
In addition, if you have any data that you don't need to see in the clear, use a client-side encryption library to encrypt it locally. There are still risks of course since the users workstation might be compromised by malware but it still removes risks during data transmission and from server storage risks. It also puts responsibility onto the user rather than just on to your central servers.
You already seem to be a long way ahead of most web developers in ensuring that your customers are kept safe and secure. One other small change it would be worth considering would be to turn on enforced HTTPS for all transactions with your site. That way, there is very little chance of unexpected data leakage such as data being unexpectedly cached.
UPDATE:
Client side encryption can help a lot since it puts the encryption responsibility on the user. Check out LastPass for example. Without doing the encryption client-side, you could never trust the service. Similarly with backup services where you set your key locally so that the backups can never be unlocked by someone on the server - they never have the key.
Partitioning is one of the primary methods for enterprises to secure services that have Internet facing components. As I said, typically, the secure server PULLs data from the less secure one so the less secure server can never have any access to anything more secure even if fully compromised. Indeed there will be a firewall that prevents any traffic from the DMZ (where the less secure service is located) getting to the secure network. Only connections from the secure side are allowed through and they will be tightly controlled by security processes. In a typical bank or other high security setting, you may well find several layers like this, each of which having separate security controls, all partitioned from each other enforcing separation of data and security.
Hope that adds some clarity. Continue to ask if not!
UPDATE 2:
Even for simple, low cost setups, I would still recommend partitioning. For a low cost version, consider having two virtual servers with the dedicated firewall replaced by careful control of the software firewall on the more secure server. Follow the same principals outlined above for everything else.
I am serializing/pickling an object, encoding it as a compressed string, and passing it as a parameter in the URL for the next page to deserialize. My web app does not have a database; I am doing this because the app gets data from external web services, which are slow.
Is this acceptable practice? Is this a security risk? Is there a way to make this secure?
If you need to share data between views, do it with the session. That's what sessions are made for. Session info is stored in the database by default, but it doesn't have to be, you can also use the filesystem, some caching system (memcache, Redis, etc), or signed-cookies (Django 1.4+ only).
See:
Configuring the Session Engine
How to Use Sessions
Is this a security risk?
If the serialisation you are using is pickle then yes that is definitely a problem, as alluded to on the doc:
Never unpickle data received from an untrusted or unauthenticated source
Use a form of serialisation designed only to hold safe static values (eg JSON).
You can protect a value that you send to the client side from tampering by signing it with a MAC, eg using hmac. You may need to consider adding other properties to the MAC-signed data such as username or timestamp, to prevent signed data blocks being freely interchangeable, if that's a threat to whatever integrity you are trying to achieve.
If you also need to protect the value from being viewed and interpreted by the client side user you would need to use an encryption algorithm (eg AES - not part of stdlib) in addition to the signing.
(I still wouldn't personally trust a MAC-signed and encrypted pickle. Even though it would need the server-side secret to be leaked to make it exploitable, you don't really want an information-leakage vulnerability to escalate to an arbitrary-code-execution vulnerability, which is what pickle represents.)
It is not the best option, since URL parameter fields will show in server logs. You're
probably better of sending data with POST method or better yet, creating a rudimentary database (if you don't have access to anything else, use Sqlite) and just pass the ID to the next screen.
In a Django project, I want to keep user uploaded files secure on the server. Should this be done at the OS level (we are using ubuntu) or at the application level?
Encrypting at the application level will be easier to maintain. But, aside from some drawbacks like possible negative effect on performance, I am not even sure if this will have any point. If a hacker compromises the server, he will also have access to the encryption keys and how it is encrypted / decrypted.
Any suggestions are greatly appreciated. Thanks.
How you protect your data depends on what kinds of attacks you want to protect against. Of course, you probably don't know how an attacker is most likely to compromise your system, unless there are certain threat models you're particularly trying to protect against, like say a rogue sysadmin.
The attacker might gain access to the OS that the web server is running on. In this case, filesystem level encryption probably does you no good. In fact file-system level encryption is probably only useful protection against somebody walking off with the physical server (which is a totally valid threat model). However, if the files are encrypted with keys stored in the database, then an attacker who has access to the webserver OS but not the database is thwarted.
In contrast, an attacker might gain access to the database but not the OS, through a hole in your application. I would expect this to be less likely since modern operating systems present huge and well-studied attack surfaces.
To protect your user's data against an attacker with full access to your servers is very difficult. You need to encrypt the data with a key that your servers don't have. This could be something like a password or a key stored in a user cookie. The problem with all these schemes is that users can't be trusted to hold on to critical data like this -- they always want a way to reset their password if they forget. In most cases, it's not realistic to protect data against an attacker with full access to your OS and your database.
So I'd choose what you're trying to protect against. Personally, I'd expect an OS penetration to be most likely, and thus encrypt the files with keys that are stashed in a part of the database that is extra protected somehow. The challenge here is that the OS has to store database login credentials (in settings.py) in order for the web app to function. So try to keep those files as restricted as possible within the OS i.e. chmod 600 on a user account that does as little else as possible.
You're right that if the key used to encrypt the files is stored on the server you don't get a whole lot of added security by encrypting the files.
However, if you use a key provided by the user, then you do get some security. For example, if you store the encryption key in a cookie, then it will only be available for the duration of each request. I don't believe this will create any new security issues (if an attacker can steal the cookie, they can also steal the user's session), and it will make it much harder for an attacker to access files belonging to users who aren't currently online.
If you're really paranoid, you could do what 1Password does, and send encrypted data back to the browser, which can decrypt it with JavaScript encryption routines…