Django Cache Implementation - django

Well, I'm designing a web application using Django. The application allow users to select a photo from the computer system and keep populating onto the users timeline. The timeline view have a list/grid of all the photos which the user has uploaded sorted chronologically, showing 50 photos and then a pull to refresh to fetch the next 50 photos on the timeline.The implementation works for multiple users.
Now for fast user experience of the app I'm considering caching. Like most sites store the timeline of the user onto cache so that whenever the user logs in it the first place to check for information the request is served out of the cache and if it is not available there then you go to the DB to query for the information.
Primarily in one line I'm trying to cache all the timelines for different users in cache for now.
I'm done with building of the webapp minus the cache part. So , my question is how do I cache all the timelines of different users??

There is a big difference between public caching and the caching of private data. I feel your data is private and thus needs a different strategy. There is a nice overview of the different ways to implement testing and, more importantly, the different things you need to take into account: The Server Side (Tom Eastman). This has a part on speed and caching (16:20 onward). It explains how to use etag and last_modified headers with django.

Related

Tracking user web search history in Django

I am implementing a Django powered website in which I need to track the websites visited by a logged in user.I had googled it but didn't find anything specific to Django.
I know there is a documentation available at link
Can anyone please recommend some django specific document or some algorithm which is independent of the web browsers.
Thanks .
Edit: The django application would be acting like a portal which will authenticate users to use internet.The search will not be on the django powered website as it would just act as a portal for login,logout.
That depends on various things and you will probably get very different answers.
One way would be to store a list of urls in request.session as a session cookie value.
If you need more persistence, you could create your own model for it and save it on each request. Something like:
class Tracker(models.Model):
url = models.URLField()
user = models.ForeignKey('auth.User')
time = models.DateTimeField(auto_now_add=True)
However this can be relatively slow depending how intense you plan to use it.
If you need to store much of this kind of data, it might be worth it to investigate adding a database backend like influxdb which is designed to store a lot of time series data for later statistical evaluation.

Display content on web page depending on Google Analytics data

Is it possible to use data from Google Analytics when someone visits my web page?
I would like to get information about the visitor, eg:
location
how many times he visited the page
from where he comes
time spent on website
etc
And depending on those informations display proper content. Or even add them (silently) to the contact form.
Is it possible to use the data collected by Google Analytics or should I create my own mechanism based on cookies? Maybe any other solution?
This isn't possible. Universal Analytics uses a cookie with an ID for the user and handles the rest on the server, so there's no data available to you. However, all of the info that Google is tracking is accessible to you.
location you can get via the users IP
visits you can get by setting a cookie on each user and tracking sessions
referrer should be in the request headers
time spent can be tracked the same way that ga does, but keeping track of the time everytime the user creates another hit

Saving Sitecore DMS Visitor info to a user

With Sitecore DMS you can create Profiles and show specific pages based on the user's visiting habits. This information is stored using a cookie so whenever the user comes back they have the same visitor profile. But if they delete their cookies or switch browsers that information is gone. Now what I want to do is save this visitor information to a sitecore user, in that they can log in and their visitor profiles will be the same as when they left, no matter where they are or what browser they are using. I've been trying to figure this out for a while now but without success. Whenever I login with a user and create a specific profile, that profile is deleted as soon as i log out. Is this even possible for Sitecore, because it seems rather silly to only have profiles based on cookies when cookies are deleted constantly.
With Sitecore Analytics, you have two tracking cookies, one is for Session--the other for GlobalSession (which doesn't expire across normal asp.net sessions). Unfortunately, Sitecore doesn't track profile key scores based on GlobalSession, but by a single asp.net session. The entire system is based around this, and it's very disappointing. '
To get around this, I was using OMS and use Reflector to disasssemble and rewrite some of the analytics code to record by global session. It also required a couple of schema changes as well as new rules... pretty extensive work. At the end of the day, because of the size of the database and our need for a very limited number of features, I ended up creating a single table and recording profile key values in that by globalsession.
From what I understand, the schema has changed from OMS to DMS, but the single most important factor, that data is recorded by session, has not changed.
With regard to storing the User's id, you can associate this with a 'tag' which is stored, I believe, with the globalsessionid, at least in OMS. However, Sitecore doesn't update the tag records, so you'll end up with multiple records per global session. If you're storing this value on every request, that table will bloat quickly.
There are a couple of reasons for this... not the least of which is that in many cases, you don't have a logged in user to correlate this information with. The profile data isn't stored in cookies... it's stored in the Analytics table. But it's associated with a cookie that has a unique ID and once that has been deleted, the ID (hopefully!) won't be used again.
A suggestion for how to get around this here in this StackOverflow answer.

Are there any concerns in using django session to store information?

Are they any pitfalls to using django session to store user information? in what situations should I avoid using this mechanism?
"The session framework lets you store and retrieve arbitrary data on a per-site-visitor basis", so if you're fine with that (the stored dictionary based information is available only for the specific user of that session, as long as you don't access the selected session backend otherwise) it's perfectly ok.
The only pitfalls I see could be introduced when using a cache-based session backend (cache invalidation, persistence of data, distribution of to-be-cached-data to multiple servers, things like that), specifically when the storage of data is different from your main storage (database) - say with memcached or file based caching.
One thing that surprised me with storing data in the sessions is what happens (or doesn't happen) when the user has the site open in two browsers (say once on their mobile, once on their desktop).
For example, I was having a performance problem and decided to fix it by making fewer hits to the database. The site's premise was that the mobile app was for viewing data but you do changes through the desktop site.
There was some logic like this:
if not session_data then:
fetch_data_and_put_in_session
else:
get_session_data_ftw()
If the user logged in on their mobile the session data was created from the database. If they then used their browser to make changes to the data they couldn't view it on their mobile until their session expired.

Web Dev - Where to store state of a shopping-cart-like object?

You're building a web application. You need to store the state for a shopping cart like object during a user's session.
Some notes:
This is not exactly a shopping cart, but more like an itinerary that the user is building... but we'll use the word cart for now b/c ppl relate to it.
You do not care about "abandoned" carts
Once a cart is completed we will persist it to some server-side data store for later retrieval.
Where do you store that stateful object? And how?
server (session, db, etc?)
client (cookie key-vals, cookie JSON object, hidden form-field, etc?)
other...
Update: It was suggested that I list the platform we're targeting - tho I'm not sure its totally necessary... but lets say the front-end is built w/ASP.NET MVC.
It's been my experience with the Commerce Starter Kit and MVC Storefront (and other sites I've built) that no matter what you think now, information about user interactions with your "products" is paramount to the business guys. There's so many metrics to capture - it's nuts.
I'll save you all the stuff I've been through - what's by far been the most successful for me is just creating an Order object with "NotCheckedOut" status and then adding items to it and the user adds items. This lets users have more than one cart and allows you to mine the tar out of the Orders table. It also is quite easy to transact the order - just change the status.
Persisting "as they go" also allows the user to come back and finish the cart off if they can't, for some reason. Forgiveness is massive with eCommerce.
Cookies suck, session sucks, Profile is attached to the notion of a user and it hits the DB so you might as well use the DB.
You might think you don't want to do this - but you need to trust me and know that you WILL indeed need to feed the stats wonks some data later. I promise you.
I have considered what you are suggesting but have not had a client project yet to try it. The closest actually is a shopping list that you can find here...
http://www.scottcommonsense.com/toolbox.aspx
Click on Grocery Checklist to open the window. It does use ASPX, but only to manage the JS references placed on the page. The rest is done via AJAX using web services.
Previously I built an ASP.NET 2.0 site for a commerce site which used anon/auth cookies automatically. Each provides you with a GUID value which you can use to identify a user which is then associated with data in your database. I wanted the auth cookies so a user could move to different computers; work, home, etc. I avoided using the Profile fields to hold onto a complex ShoppingBasket object which was popular during the time in all the ASP.NET 2.0 books. I did not want to deal with "magic" serialization issues as the data structure changed over time. I prefer to manage db schema changes with update/alter scripts synced with software changes.
With the anon/auth cookies identifying the user on the client you can use the ASP.NET AJAX client-side to call the authentication web services using the JS proxies that are provided for you as a part of ASP.NET. You need to implement the Membership API to at least authenticate the user. The rest of the provider implementation can throw a NotImplementedException safely. You can then use your own custom ASMX web services via AJAX (see ScriptReference attribute) and update the pages with server-side data. You can completely do away with ASPX pages and just use static HTML/CSS/JS if you like.
The one big caveat is memory leaks in JS. Staying on the same page a long time increases your potential issue with memory leaks. It is a risk you can minimize by testing for long sessions and using tools like Firebug and others to look for memory leaks. Use the JS Lint tool as well as it will help identify major problems as you go.
I'd be inclined to store it as a session object. This is because you're not concerned with abandoned carts, and can therefore remove the overhead of storing it in the database as it's not necessary (not to mention that you'd also need some kind of cleanup routine to remove abandoned carts from the database).
However, if you'd like users to be able to persist their carts, then the database option is better. This way, a user who is logged in will have their cart saved across sessions (so when they come back to the site and login, their cart will be restored).
You could also use a combination of the two. Users who come to the site use the session-based cart by default. When they log in, all items are moved from the session-based cart to a database-based cart, and any subsequent cart activity is applied directly to the database.
In the DB tied to whatever you're using for sessions (db/memcache sessions, signed cookies) or to an authenticated user.
Store it in the database.
Do you envision folks needing to be able to start on one machine (e.g. their work PC) but continue/finsih from a different machine (e.g. home PC)? If so, the answer is obvious.
If you don't care about abandoned carts and have things in place for someone messing with the data on the client side... I think a cookie would be good -- especially if it's just a cookie of JSON data.
I'd use an (encrypted) cookie on the client which holds the ID of the users basket. Unless it's a really busy site then abandoned baskets won't fill up the database by too much, and you can run a regular admin task to clear the abandoned orders down if you care that much. Also doing it this way the user will keep their order if they close their browser and go away, a basket in the session would be cleared at this point..
Finally this means that you don't have to worry about writing code to deal with de/serialising the data from a client-side cookie, while later worrying about actually putting that data into the database when it gets converted into an order (too many points of failure for my liking)..
Without knowing the platform I can't give a direct answer. However, since you don't care about abandoned carts, then I would differ from my colleagues here and suggest storing it on the client. Why store it in the database if you don't care if it's abandoned?
Then again, it does depend on the size of the object you're storing -- cookies have their limits after all.
Edit: Ahh, asp.net MVC? Why not use the profile system? You can enable an anonymous profile if you don't want to bother making them log in
I'd say store the state somewhere on the server and correlate it to the user's session. While a cookie could ostensibly be an equal place to store things, if you consider security and data size, keeping as much data on the server as possible becomes a good thing.
For example, in a public terminal setting, would it be OK for someone to look at the contents of the cookie and see the list? If so, cookie's fine; if not, you'll just want an ID that links the user to the data. Doing that would also allow you to ensure the user is authenticated to the site in order to get to that data rather than storing everything on the machine - they'd need some form of credentials as well as the session identifier.
From a size perspective, sure, you're not going to be too concerned about a 4K cookie or something for a browser/broadband user, but if one of your targets is to allow a mobile phone or BlackBerry (not on 3G) to connect and have a snappy experience (and not get billed for the data), minimizing the amount of data getting passed to the client will be key.
The server storage also gives you some flexibility mentioned in some of the other answers - the user can save their cart on one machine and resume working with it on another; you can tie the cart to some form of credentials (rather than a transient session) and persist the cart long after the user has cleared their cookies; you get a little more in the way of fault tolerance - if the user's browser crashes, the site still has the data safe and sound.
If fault tolerance is important, you'll need some sort of persistent store like a database. If not, in application memory is probably fine, but you'll lose data if the app restarts. If you're in a farm environment, the store has to be centrally accessible, so you're again looking at a database.
Whether you choose to key by transient session or by credentials is going to depend on whether the users can save their data and come back later to get it. Transient session will eventually get cleaned up as "abandoned," and maybe that's OK. Tying to a user profile will let the user keep their data and explicitly abandon it. Either way, I'd make use of some sort of backing store like a database for fault tolerance and central accessibility. (Or maybe I'm overengineering the solution?)
If you care about supporting users without Javascript enabled, then the server side sessions will let you use URL rewriting.
If a relatively short time-out (around 2 hours, depending on your server config) is OK for the cart, then I'd say the server-side session. It's faster and more efficient than accessing the DB.
If you need a longer persistence (say some users like to leave and come back the next day), then store it in a cookie that is tamper-evident (use encryption or hashes).