One part of my application requires fetching around 200 objects of a Model (Customer) to update certain fields on each and every request.
It first fetches a Customer object on every request and attains a list of Grouped Customers each with certain values from an outside API. This list of grouped customers has around 200 entries (each being of type Customer). I want to update all the Customer's fields with the values by fetching all 200 of them and updating and saving. Getting 200 objects for every single request and then waiting for it to finish updating then returning a response seems rather silly and slow. Is there anyway to do this updating asynchronously? Like I can return the response right after the I fetch the first Customer and then have a an async function handle the updating process.
Is there some possibility this can be done or do I have to wait for all 200 objects to be done updating?
As the first Google result for "django asynchronous tasks" would have shown you, the canonical way of doing this is to use Celery.
Related
I've been self studying web design and want to implement something, but I'm really not sure how to accomplish it, even if I can.
The only frontend I have dealt with is angular 4, and the only backend I have dealt with is django rest framework. I have managed to get user models done in drf, and the frontend to get the user authenticated with json web tokens, and done different kinds of get and post requests.
What I want to do is on the front end have a button, when the button is hit, it will send some get request, that basically runs a text mining algorithm that will produce a list, it may take some time to fully complete, maybe in the range of 20-30 seconds, but I don't want the user to wait that long to get back the single response containing the fully compiled list.
Is it possible to say create a table in angular, and then every couple of seconds the backend sends another response containing more data, where the backend then appends the new results to that table. Something like:
00.00s | button -> GET request
01.00s drf starts analysis
05.00s drf returns the first estimated 10% of overall list
09.00s drf finds 10% more, returns estimated 20% of overall list
then repeat this process until the algorithm has stopped. The list will be very small in size, probably a list of around 20 strings, of about 15 words in each,..
I already tried in django to send multiple responses in a for loop, but the angular front end just receives the first one and then doesnt listen anymore.
No, that's not possible. For each request will be one response, not multiple.
You have two options:
- Just start your algorithm with an endpoint like /start, and check the state in an interval on an endpoint like /state
- Read about websockets or try firebase (or angularfire). This provides a two way communication
I currently have an ember app that's trying to retrieve data from a 3rd party API using Ember Data. The urls are in the format, /user_id/date/food, which would retrieve the user's consumed food on the given day. I want to retrieve the list of foods the user consumed given a date range (2015-06-07, 2015-08-10).
I tried to use Ember.query and filter out the unnecessary data, but the API doesn't have an endpoint which would return all of the consumed foods.
Currently I'm supporting the single day query using queryRecord and passing the day in.
If /user_id/date/food is the only endpoint the 3rd party API is providing, the only option you would have is to make a request for every day in the range you want to get. You might want to use Ember.RSVP.all to wait till all requests are retrieved and finally concatenate the result for example.
But this is not really recommended, because a large range would be very inefficient.
We are considering moving from Backbone to Ember. There are a few issues through I can't get answers to from the docs.
1) Ember-Data caches it's data. Our application is multi-user so other users need to be able to see new records created by everyone. Is there a way around this? I read on another post that when a query string is passed, ember data does not cache data, is this true? If it is, can I then just always send query string and nothing will be cached?
2) Ember data has a single model in the router that appears to be instantiated at route load time. I can see that your can request data from multiple sources by returning an object with many this.store.find calls. Say I have a select element and when you select an option, another select gets populated with items based on the first select (which requires a call back to the server). How would that work, how can I get model data on demand (not at route load time)?
I'm not sure if it answers your question but you can always call
model.reload()
to refetch data from server so you can work with up to date data.
You may want to consider Faye (http://faye.jcoglan.com/), which would let you have a pub/sub setup that could update your store by listening to topics of interest. This uses WebSocket for the streaming interface. You could then put new objects into the store, remove or update existing objects which the server could publish to the client.
I have a User model which hasMany phones. The UI for the user allows to add/delete/update phones on the single form.
When user submits the form all changes to the phone list are sent to the server with a single request.
I have extended the App.UserSerializer with custom serializeHasMany to include all the phone details in the single request.
The real problem is to sync the store state after the request is complete.
Basically I need to solve these two problems:
Remove deleted records from the store. I could not find any methods which just removes a record from a store.
Update new records with the ids generated by server. (Or just remove the new records from the store and hasMany array since response creates the dups for the added records)
Is there any best practices or work arounds for this kind of scenarios?
Thank you.
I think the best practice for now is just sticking to regular REST. In your case this will mean a few extra requests (really though, how many phones can a user have?), but it will spare you a lot of effort in handling things manually.
Ember may support bulk updates in the future (https://github.com/emberjs/data/blob/master/TRANSITION.md, "We plan to support batch saving with a single HTTP request through a dedicated API in the future.")
I was told that in order to not make a request all the time, one can use .all() method to load data that is kept in the store. But how does Ember deal with cache? I have a couple of questions.
How do you control cache? When do you use .find() and when .all(). Do you use .find() and then .all()? For how long?
Does .all() have some expiration date so that after some time it can make a new request? Or it uses Local Storage so that I have to clear it manually?
Suppose that I have some data I'd like to refresh only once a week? How should I go about this? Now every time I enter or re-visit the same route a new request is made. How can I avoid this?
So will start by answering question from your comment:
I'd rather to know how can I load data when an app starts (not via routes as I don't have to update it so often). Is it possible
So OK technically this is still via routes, but the best way to load data when an app "starts" is via the Application Route's model hook.
App.ApplicationRoute = Ember.Route.extend({
model: function({
return App.Post.find();
})
})
The router will wait for promise returned by find() to resolve, so you can be sure that response from server has come back before any other routes are entered.
How do you control cache?
Mostly you don't worry about it. You can refresh() an individual record after some timeout if needed.
When do you use .find() and when .all(). Do you use .find() and then .all()? For how long?
Depends what you want to achieve. In our app we use find() in the application route, then either all() or a filter() in other routes.
Does .all() have some expiration date so that after some time it can make a new request?
Nope. It will never make a new request
Or it uses Local Storage so that I have to clear it manually?
It does not use local storage, records are in memory. So for sure an F5 will clear the cache.
Suppose that I have some data I'd like to refresh only once a week? How should I go about this? Now every time I enter or re-visit the same route a new request is made. How can I avoid this?
So OK let's assume you use find() only in the application route, and that user keeps browser open for 1 week and the records have expired. There are many ways to refresh, what's easy/best depends on if they all expire at once or if they time-out one at a time.
Have some timer checks for expired records and calls refresh() as needed.
Have a Ping model that you update on some schedule. When server responds to update it can sideload any changed records.
Or can just refresh the browser once per week (via window.location...)
What you call cache is the content of the store. There are usually 2 ways to update the store to reflect changes made on the backend side:
the change happens with a user interaction/call to the server. If you update mulitple records on the backend side, you can sideload them with the response of that request.
the change happens asynchronously on the backend side (background job). You can use a websocket to push those changes to the client.