I run a REST service on AppEngine (which may not be relevant). Each REST request is accompanied by a user id and password, and at the beginning of each request I hash the password to see if it matches my records before proceeding.
This is working well, theoretically, but in practice I get bursts of requests from users - 4 or 5 a second. It's taking BCrypt 500ms to hash the password for each request! What a waste!
Clearly, I don't want to optimize the BCrypt time down. Are there standard practices for caching hashes? Would memcache be a safe place to store a table of recently hashed passwords and their hashes? I guess at that point I might as well store the users' plain-text passwords in Memcache. I'd love to do a 3ms memcache lookup instead of a 500ms hash, but security is paramount. Would it make more sense to implement some sort of session abstraction?
Thanks for any advice!
Edit for extra context: this is a gradebook application that stores sensitive student data (grades). Teachers and students log in from everywhere, including over wifi, etc. Every successful request is sent over https.
The usual rule of thumb with REST APIs is to have them remain fully stateless, however, as with goto there is a time and a place depending on your requirements. If you're not averse to the idea of having the client store a temporary session key which you only need to regenerate occasionally then you might try that out.
When a client makes a request, check whether they're sending a session key variable along with the user ID and password. If they are not, generate one based on the current server time and their password, then pass it back to the client. Store it in your own database along with its creation time. Then, when a client makes a request that includes a session key, you can verify it by directly comparing it to the session key stored in your database without requiring a hash. As long as you invalidate the key every few hours, it shouldn't be much of a security concern. Then again, if you're currently sending the password and user ID in the clear then you already have security issues.
Your best bet given your current approach is to keep a mapping of attempted passwords to their bcrypted form in memcache. If you're concerned for some reason about storing a plaintext password in memcache, then use an md5 or sha1 hash of the attempted password as a key instead.
The extra step isn't really necessary. Item stored in memcache don't leak to other apps.
Related
I'm currently running a Django application with SESSION_ENGINE configured as django.contrib.sessions.backends.db. I'd like to change this to django.contrib.sessions.backends.cached_db for a performance boost.
Can I make this change without destroying the existing sessions?
Yes, you can make this change without logged in users suddenly finding themselves being logged out. That's because cached_db checks memcache first for the key and if it cannot be found in it, goes to the database. Thus making this change will not cause a loss of session data. Fragment of code from cached_db
def load(self):
try:
data = self._cache.get(self.cache_key)
except Exception:
# Some backends (e.g. memcache) raise an exception on invalid
# cache keys. If this happens, reset the session. See #17810.
data = None
if data is None:
# Duplicate DBStore.load, because we need to keep track
# of the expiry date to set it properly in the cache.
However please note that cached sessions backends are a bit over rated. Depending on the middleware that you have, the session object may be updated very often, as often as every request if only to change the expire date. In that case you will find that the database is being written to all the time. Which means the cached value has to be discarded too.
You should be able to. cached_db backend is just a write-through cache to a database backed, persistent, db backend which speeds up your read queries. It will not speed up your write queries, so you should try and find out how much you are reading and writing the session data.
Your Django SECRET_KEY setting determines your session key hashing parametrs along with Session settings that determine the cache you will use for sessions and session your TTLs, so if you are not changing those variables, you should be good.
Good day, I'v implemented a REST service. In the URL of resource end-point I use ID's which are primary keys of tables of the database. For example http://host/myapp/items/item/4. I'v learned using the database ID in the URL is a bad practice and I should use UUID instead. On the other hand I'v learned that using UUIDs in indexes is a performance issue if there's many records in the database because they are not sequential (1,2,3,...). So I'v got an idea to encrypt the database ID. This is how it could work:
1) Client POSTs an item to `http://host/myapp/items`.
2) The back-end creates a new item in the database.
3) Autoincremented ID '4' is generated by the database.
4) The back-end encrypts the ID '4' to 'fa4ce3178a045b2a' using a cipher key and returns encrypted ID of a created resource.
And then:
5) Client sends a request to GET `http://myapp/items/item/fa4ce3178a045b2a`.
6) The back-end decrypts 'fa4ce3178a045b2a' to '4' using an cipher key.
7) The back-end fetches item with primary key '4' and sends it to the client.
What are the cons of such solution? Will the encryption/decryption will be fast enough so that it's not worse then using UUID? And what encryption algorithm should I use so that it is fast and doesn't consume much resources? Could someone more experienced advise or recommend a better solution? Thank you in advance. Vojtech
yes, ids are sometimes undesired. client can predict and generate the links. can check how big your database is etc. but sometimes it doesn't matter and then ids are perfectly ok.
the fastest ciphers are symmetric ones. for details you have to find/do some benchmarks. example is here: http://stateless.geek.nz/2004/10/13/scp-performance/
but i don't think anyone can tell you if the process of encryption will be faster than using uuid. it depends on your database size, indexes you use, cache, hardware etc. do the performance tests. if speed is critical for you, you can think of storing translation map/table (uuid -> id) in memory
I don't think we can predict which is faster: using UUID in your database or encrypting and decrypting the ids. It can depend on the type of the database, the computer the database is on and the actual request as well.
For example when you want to list many resources and you want to add links to the detailed views, you have to encrypt the id of each resource in order to compose the response. Now by a long list this can take a much longer time than a slightly slower select, so I would not use it.
I don't think this is a real bottleneck. I think the HTTP communication is the bottleneck, so in order to make things faster you should consider setting the HTTP cache properly instead. Btw. if you really want to crypt your ids, you should measure the speeds, instead of asking us to guess them.
There are two approaches I've been thinking about for storing data in cookies. One way is to use one cookie for all the data and store it as a JSON string.
The other approach is to use a different cookies for each piece of data.
The negatives I see with the first approach is that it'll take up more space in the headers because of the extra JSON characters in the cookie. Also I'll have to parse and stringify the JSON which will take a little processing time. The positive is that only one cookie is being used. Are there other positives I am missing?
The negatives I see with the second approach is that there will be more cookie key value pairs used.
There are about 15-20 cookies that I will be storing. The expires date will be the same for each cookie.
From what I understand the max number of cookies per domain is around 4000. We are not close to that number yet.
Are there any issue I am overlooking? Which approach would be best?
Edit - These cookies are managed by the JavaScript.
If you hand out any data for storage to your users (which is what cookies do), you should encrypt the data, or at the very very least sign it.
This is needed to protect the data from tampering.
At this point, size considerations are way off (due to padding), and so is the performance overhead of parsing the JSON (encryption will cause significantly greater overhead).
Conclusion: store your data as JSON, (encrypt it), sign it, encode it as base64, and store it in a single cookie. Keep in mind that there is a maximum size for cookies (and it's 4K).
Reference: among numerous other frameworks and applications, this is what Rails does.
A best-practice for cookies is to minimize their use. For instance, limit your cookie usage to just remembering the session id, and then store your data on the server side.
In the EU, cookies are subject to legal regulations, and using cookies for almost anything but session ids require explicit client consent.
Good morning.
I think i understand you. At sometime ago, i use cookies stored as json data encrypted, but for intranet, or administration accounts. For users of shop, i used this same practice. Whetever, to store products on shop site, i don't use encryption.
Important: sometimes i have problems with json decode before decrypt data. Depending your use, you can adopt a system storing data separated by ; and : encrypted like:
encrypt_function($key, "product:K10072;qtd:1|product:1042;qtd:1|product:3790;qtd:1") to store products; and
encrypt_function($key, "cad_products:1;mdf_products:2;cad_collabs:0") to store security grants.
Any system can be hacked. You need to create an applycation with constant user data verification and log analyzing. This system, yes, needs to be fast.
Suppose that a staff member using a web site can exchange tickets for a customer. It is convenient to store data about the multi-view exchange in the session. But more than one exchange might be going on at the same time.
One way to keep track of the separate data in the session is to create a sub-session key and use that to access the session data. This key would need to be part of the view as a hidden input or it would need to be in the URL. This all gets pretty messy and the hidden variable method isn't great since redirects might occur during the exchange.
Is there a clean way to do this?
Use a database table that tracks information for a particular exchange and read/write from it when opening/submitting your wizard pages. Sessions are much more volatile by nature.
I need to push changes to my app's session scopes in real time. Each user in session in my app has a similar struct to this:
session.user =
{
name = "Foo",
mojo = "100"
};
Users can modify each others' "mojo." For example, if user Foo received 10 mojo points, and he now has 110, I need to update his session.user.mojo to reflect the additional "mojo" received. I need to modify his session struct, in other words.
Example 2: User in session 1 does something where user in session 2 receives "mojo." The session.user.mojo in session 2 needs to be updated to reflect this change.
Some info:
The inital mojo value is pulled from the database and stored in the session when a user logs in.
"Mojo" updates always take place in the database. "Mojo" stored in the session is used to govern user privileges.
What are my options? Is this even possible? I have absolutely no idea on how to do something like that.
UPDATE I don't want pass the updated values back to the user (the data will refresh when the user navigates between pages). I only want to change them in the appropriate user's session scope.
This answer is ColdFusion 9 specific.
Cache user data (e.g. cachePut()) by user ID, and keep track of their user ID in session. Every update to mojo should retrieve the user data in cache - if present - and update it there as well. Finally, if this is a multi-server environment, setup messaging between the machines that broadcasts the user ID of any change to mojo, servers receiving the message then update their own cached user data.
What this buys you is limiting the amount of database activity that goes on, pretty good liveness, and makes the mojo value available globally, which has the added benefit of being available for purposes other than the user session (e.g. another user can review their profile to see the mojo score).
If you really need to change vars in a particular Session, there's no built-in way to do that. Maybe you can abstract out the logic, instead of accessing the mojo from Session, always access mojo from DB?
update: Why session? How about a big struct in Application scope, and use userID or sessionID as key, and mojo as value? You can also store a timestape like lastUpdated and delete the ones that has not been updated to reclaim your memory. Then from time to time, update your DB? Or... update your DB async if u're worry about performance.