So I am trying to implement moreover learn how to cache Django views per URL. I am able to do so and here is what is happening...
I visit a URL for 1st time and Django sets the caches.
I get my result from cache and not from querying the database during the second visit if the browser is same.
Now the doubt is - if I change my browser from the first visit and second visit, for example, do the first visit from Chrome (it sets the cache) and during the second visit from Mozilla, it again sets the cache. I was expecting it to return the result from the cache.
During my research on StackOverflow and checking what it sets as cache, I found there are two important things first being a header and the second being the content. And I think every time a browser is changed the header is new so it sets the cache instead of returning the result from cache. Do let me know if I am wrong.
I have a public URL and I was thinking to show data from the cache if a subsequent request is made, irrespective of browser or mobile/laptop/desktop, only based on Url, is that anyhow possible?
**(I was thinking if someone from the north part of the country visit a URL, subsequent visit to the same URL from the south part of the country should get data from the cache, based on my cache expiry time though)
Also if my understanding is wrong please correct me.
I am learning to cache using Redis on Django.
So i manually set key for some of my public url(views), adjust cache on create and delete and during get-list I check for the key values in cache, get the result from cache if cache timeout or unavailable then get the result from database. Somehow response time for this is little bit slower than cache_page(), the default django function, I dont know why. Any explanation ?? or Am i correct ?
Related
I have read Ember docs related to Substates etc and I understand how it works. In my current application my loading.hbs and other child loading.hbs templates work fine.
I just want to discuss a use case.
In my route A, in my model function I do fetchAll for my model.
I go to Route A, First time api request is sent and I see my loading screen.
now I navigate to some other route B.
now I come back to first route A, api request is sent again but this
time loading screen is not shown.
I want to develop my understanding here. Now the second time loading screen is not shown which tells us that store had data so there is no sense to put it on loading and after fetch store sent request to sync with backend.
QUESTION
Now I want to know if this is a default behaviour of Ember with Ember-data?
To show this loading screen, will I have to do something manually?
Ideally what I want is, if on second request data was fetched then show it and at the same time show loader to tell user that it is syncing with backend as well.
I know everything can be done manually, I don't want to reinvent the wheel or do things in non-conventional way. So I want to know best optimized solution for this as provided by Ember which an experienced Ember developer can help me understand.
Thanks in advance.
Now I want to know if this is a default behaviour of Ember with
Ember-data?
Yes, that's the default behavior of Ember data when you do a findRecord or findAll where shouldBackgroundReloadRecord or shouldBackgroundReloadAll event of the adapter respectively, is defaulted to true. You can turn this off by returning false and ensuring shouldReloadAll or shouldReloadRecord respectively are set at true to ensure the request always hits the API and not fetches from cache.
Ideally, showing data immediately on the screen is always advisable as it has a better UX in terms of giving the user a feel that data is already there and that some parts of the data is being fetched. Check here
To show this loading screen, will I have to do something manually?
You may also want to read this
To add further details after my own research, I found helpful and relevant details in Ember docs.This is all about caching.
If records were already there then promise will be resolved immediately that's why I don't see loading screen for already loaded record, at the same time Ember-Data syncs with backend as well and re-render the template.
Ember Model Docs
Caching
The store will automatically cache records for you. If a record had already been loaded, asking for it a second time will always return the same object instance. This minimizes the number of round-trips to the server, and allows your application to render its UI to the user as fast as possible.
For example, the first time your application asks the store for a person record with an ID of 1, it will fetch that information from your server.
However, the next time your app asks for a person with ID 1, the store will notice that it had already retrieved and cached that information from the server. Instead of sending another request for the same information, it will give your application the same record it had provided it the first time. This feature—always returning the same record object, no matter how many times you look it up—is sometimes called an identity map.
Using an identity map is important because it ensures that changes you make in one part of your UI are propagated to other parts of the UI. It also means that you don't have to manually keep records in sync—you can ask for a record by ID and not have to worry about whether other parts of your application have already asked for and loaded it.
One downside to returning a cached record is you may find the state of the data has changed since it was first loaded into the store's identity map. In order to prevent this stale data from being a problem for long, Ember Data will automatically make a request in the background each time a cached record is returned from the store. When the new data comes in, the record is updated, and if there have been changes to the record since the initial render, the template is re-rendered with the new information.
I will have a sidebar that appears on almost every page of my web app. The sidebar will have some drop-downs, which will consist of a total of say, 1000 different options, which are pulled from the db. Rather than doing the query to get these choices on every page load, I think it makes more sense to just do the query once, say in my config.py and store them in a variable that my views have access to. Is this OK? Is there a better way to accomplish this?
You could do that, but then you'd need to restart your Flask server every time you wanted to update the sidebar.
I'd use some other form of caching here. You could use Flask-Cache and memoize your query results. You can pick a nice long cache timeout, and then clear the cached result whenever you update the sidebar.
I'm building a Django web application which allow users to select a photo from the computer system and keep populating onto the users timeline. The timeline will be showing 10 photos initially and then have a pull to refresh to fetch the next 10 photos on the timeline.
So my first question is I'm able to upload images which gets store on the file system,but how do I show only first 10 and then pull a refresh to fetch the next 10 and so on.
Next, I want the user experience of the app to be fast. So, I'm considering caching. So, i was thinking, what do I cache. Since there are 3 types of cache in Django- Database cache, MemCache, or FileSystem Caching.
So my secon question is should I cache the first 10 photos of each user or something else?
Kindly answer with your suggestions.
So my first question is I'm able to upload images which gets store on the file system,but how do I show only first 10 and then pull a refresh to fetch the next 10 and so on.
Fetch first 10 with your initial logic, fetch next photos in chronological order. You must have some timestamp relating to your photo posting. Fetch images according to that. You can use Django Paginator for this.
what do I cache
Whatever static data you want to show to the user frequently and wont change right away. You can cache per user or for all users. According to that you choose what to cache.
should I cache the first 10 photos of each user or something else
Depends on you. Are those first pictures common to all the users? Then you can cache. If not and the pictures are user dependent, there is no point caching them. The user will anyway have to fetch the first images. And I highly doubt the user will keep asking for the same first 10 photos frequently. Again, it's your logic. If you think caching will help, you can go ahead and cache.
The DiskCache project was first created for a similar problem (caching images). It includes a couple of features that will help you to cache and serve images efficiently. DiskCache is an Apache2 licensed disk and file backed cache library, written in pure-Python, and compatible with Django.
diskcache.DjangoCache provides a Django-compatible cache interface with a few extra features. In particular, the get and set methods permit reading and writing files. An example:
from django.core.cache import cache
with open('filename.jpg', 'rb') as reader:
cache.set('filename.jpg', reader, read=True)
Later you can get a reference to the file:
reader = cache.get('filename.jpg', read=True)
If you simply wanted the name of the file on disk (in the cache):
try:
with cache.get('filename.jpg', read=True) as reader:
filename = reader.name
except AttributeError:
filename = None
The code above requests a file from the cache. If there is no such value, it will return None. None will cause an exception to be raised by the with statement because it lacks an __exit__ method. In that case, the exception is caught and filename is set to None.
With the filename, you can use something like X-Accel-Redirect to tell Nginx to serve the file directly from disk.
I am using Django + memcached and have a (hopefully) simple question.
My database is updated once a day. My pages are set to time out after 24 hours.
Is there a way to generate all the pages of the site into the cache once each day, just after the database is updated, in advance of any users coming to them?
I'd like the first user of the day to see the fast-loading cached version, not the slow-loading non-cached version.
I guess I could do this by scraping the site, but is there a neater way?
I think this is going to depend on how you have your urls.py set up.
If your urls are all either
A.) Straight out of the urls.py
or
B.) Predictable based on your database
If so, you might be able to use django_extensions show_urls
by doing python manage.py show_urls it outputs a list of all the urls.
From there, you can just capture them in a list and loop over them while hitting each one with a requests.get(some_url)
The output will tell if a variable is needed. If so, just replace it with the correct variable(s) and your done.
What is a good approach to keeping accurate counts of how many times a page has been viewed?
I'm using Django. Specifically, I don't want refreshing the page to up the count.
As far as I'm aware, no browsers out there at the moment send any kind of message/header to the server saying whether the request was from a refresh or not.
The only way I can see to not count a user refreshing the page is to track the IPs and times that a user views a page, and then if the user last viewed the page less than 30 minutes ago, say, you would dismiss it as a refresh and not increment the page view count.
IMO most page refreshes should be counted as a page view anyway, as the only reason I have for refreshing is to see new data that might have been added, or the occasional accidental refresh/reloading after a browser crash (which the above method would dismiss).
You could give each user cookie, that expires at the end of the day, containing a unique number. If he reloads a page you can check wether she has been counted already that day.
You could create a table with unique visitors of the pages, e.g. VisitorIP + X-Forwarded-For content, User-Agent string along with a PageID of some sorts. If the data itself is irrelevant, you can create a md5/sha1 hash from these values (besides the PageID of course). Be warned however that this table will grow really fast.
I'd advise against setting cookies for that purpose. They have a limited size and with many visited pages by the user, you could reach that limit and make the solution unreliable. Also it makes it harder to cache such page on client-side (see Cacheability), since it becomes interactive content.
You can write a django middleware and catch request.url, then setup a table with url / accesses columns. Beware of transactions for concurrent update.
If you have load problems, you can use memcached with incr or add function and periodicaly update the database table to avoid transaction locks.