I'm working on a template inside one of my apps and I need to have a lot of records in a table to see how it looks like (and several other behaviors) inside the template when queried. I don't want to waste my time inserting over 30 records one by one. I'm trying to do a bulk insert but I have no previously dumped data or such to populate using it.
The correctness of data is not important to me. the quantity is important.
Does that have anything to do with mocking?
I'm not trying to unit test anything.
Thanks,
Which type of data do you want...??
i mean.. blog posts.. or any other things...try use Faker for django which provieds fake data.
Faker - Documentation
Faker - github
Related
I'm developing an app in flask. I have to tables, one renders on the server side using jinja2 and the other one is a live table that renders dynamically using socketio. They are in different routes but the tables look the same in design.
My problem relies on the rendering, I iterate through the same database in both cases but in the dynamic part i get the json and render it with mustache and the static table do the same but with jinja2. I need to store data related with states and categories that i obtain from the database for every row and use it for rendering in both routes.
basically I want to know where to store this relationship:
{category_id:{icon:x, color:y, name:z}}
I'm almost sure that whatever solution I get I would eventually need this as a jquery object (my current solution but a single change in that data means change multiple places on different templates) so i can then access on rendering to get the dynamic data, but... that doesn't mean i now how to get there nor how to share the same data structure between flask jinja and js. Thanks in advance.
If I understand your question correctly, you can use Flask sessions to store data between views.
from flask import session
Then you can set it:
session['config'] = {category_id:{icon:x, color:y, name:z}}
And get it, and pass it to your view:
default_config = {category_id:{icon:x, color:y, name:z}}
config = session.get('config', default_config)
I'm building a site using mongodb and django-nonrel. I've read in various places that for mongo, it's better to use straight pymongo than the django ORM. This jives with my experience as well -- django's ORM is awesome for relational databases, but for doesn't give you much that pymongo doesn't do already.
My problem is that I don't know how to set up the database tables (err... "collections") initially without using django's ORM. What do I need to do to cast off the shackles of models.py and syncdb, and just write the code myself?
Seems like somebody should have created a guide for this already, but I can't find one.
A little more detail:
Right now, I'm building models and running syncdb to configure the DB. So far, django's ORM magic has made it work. But I need to do some slightly fancier stuff, like indexing on sub-elements, so I don't think the ORM is going to work for me anymore.
On top of that, I don't use models (other than auth_users and sessions) anywhere else in the project. The real schemas are defined elsewhere in json. I don't want to maintain the model classes when and the json schemas at the same time -- it's just bad practice.
Finally, I have a "loadfixtures" management command that I use to flush, syncdb, and load fixtures. It seems like this would be a very good place for the new ORM-replacing code to live, I just don't know what that code should look like....
With MongoDB you don't need an extra step to predeclare the schema to "set up" collections. The document-oriented nature of MongoDB actually does not enforce a strict schema; documents within a collection may have different fields as needed. It's a different concept to get used to, but the collection will be created as soon as you start saving data to it.
Indexes can be added using pymongo's ensureIndex on a collection.
Similar to the collection creation on data insertion, a collection will also be created if it does not exist when an index is added.
An article that should help you get started: Using MongoDB with Django.
If you're new to MongoDB, you also might want to try the short online tutorial.
I have a simple Django website (just a form, really) which asks a few questions and saves the data in a SQL database using Model.save(). Pretty simple. I want to add a model to do page counting, though -- it'll just be a single object with a field that gets incremented each time the page's view function is called.
Now, I know little to nothing about SQL. I imagine this is not terribly difficult to do, but I would like to avoid losing or breaking all my data because of a slight misunderstanding of how the database works. So how can I go about doing this? I've heard of some third-party apps that will implement such functionality, but I'd like to do it myself just for learning purposes.
I don't understand why your existing data would be affected at all. You're talking about adding a completely new table to the database, which is supported within Django by simply running manage.py syncdb. The case where that doesn't work is when you're modifying existing tables, but you're not doing that here.
I must say though that learning and using South would be of benefit in any case. It's good practice to have a tool that can maintain your model tables.
(Plus, of course, you would never lose any data, because your database is backed up, right? Right?)
Since you're adding new model, you can just run syncdb and it will create new table for your model. If you were to change existing model, then you'd need to manually update database schema using "ALTER TABLE" statements or use South instead.
I need to create some fixtures for django test. Does anyone know of a shortcut that lets me get x number of rows from every table in database and serialize it?
Background:
I have Multiple tables with 10's of millions of entries. I have tried to use the ./manage.py dumpdata, but in addition to taking too long there is no way fixture should be that large.
Each table has multiple foreign keys.
The Problem:
The code I am trying to test frequently calls select_related() Meaning I need all the foreign key relationships filled in.
Does anyone know of any tools that can help me follow foreign relationships for serializing DB data??? Any suggestions would be greatly appreciated.
Thank you for your time.
I have used the django-autofixture pluggable apps in a couple projects. You could give that a shot. Instead of using data from your database for tests, create a development database filled with autofixtures.
This link has a few other examples of similar pluggable apps.
http://djangopackages.com/grids/g/fixtures/
Another option is the tool Dynamic Dynamic Fixture, that follow Foreign Keys and Many to Many fields. Also, you can use the option "number_of_laps" that may help you.
I have the following model
class Plugin(models.Model):
name = models.CharField(max_length=50)
# more fields
which represents a plugin that can be downloaded from my site. To track downloads, I have
class Download(models.Model):
plugin = models.ForiegnKey(Plugin)
timestamp = models.DateTimeField(auto_now=True)
So to build a view showing plugins sorted by downloads, I have the following query:
# pbd is plugins by download - commented here to prevent scrolling
pbd = Plugin.objects.annotate(dl_total=Count('download')).order_by('-dl_total')
Which works, but is very slow. With only 1,000 plugins, the avg. response is 3.6 - 3.9 seconds (devserver with local PostgreSQL db), where a similar view with a much simpler query (sorting by plugin release date) takes 160 ms or so.
I'm looking for suggestions on how to optimize this query. I'd really prefer that the query return Plugin objects (as opposed to using values) since I'm sharing the same template for the other views (Plugins by rating, Plugins by release date, etc.), so the template is expecting Plugin objects - plus I'm not sure how I would get things like the absolute_url without a reference to the plugin object.
Or, is my whole approach doomed to failure? Is there a better way to track downloads? I ultimately want to provide users some nice download statistics for the plugins they've uploaded - like downloads per day/week/month. Will I have to calculate and cache Downloads at some point?
EDIT: In my test dataset, there are somewhere between 10-20 Download instances per Plugin - in production I expect this number would be much higher for many of the plugins.
That does seem unusually slow. There's nothing obvious in your query that would cause that slowness, though. I've done very similar queries in the past, with larger datasets, and they have executed in milliseconds.
The only suggestion I have for now is to install the Django debug toolbar, and in its SQL tab find the offending query and go to EXPLAIN to get the database to tell you exactly what it is doing when it executes. If it's doing subqueries, for example, check that they are using an index - if not, you may need to define one manually in the db. If you like, post the result of EXPLAIN here and I'll help further if possible.
Annotations are obviously slow, as they need to update every record in the db.
One direct way would be to denormalize the db field. Use a download_count field on the plugin models that is incremented on the new save of Download. Use the sort by the aggregate query on Plugins.
If you think there are going to be too many downloads to update another record of the Plugin all the time, you can update the download_count field on the Plugin via a cron.