Can or should I ever do this in a view?
a = SomeTable.objects.all()
for r in a:
if r.some_column == 'foo':
r.some_column = 'bar'
It worked like a champ, but I tried a similar thing somewhere else and I was getting strange results, implying that QuerySet objects don't like to be trifled with. And, I didn't see anything in the docs good or bad for this sort of trick.
I know there are other ways to do this, but I'm specifically wanting to know if this is a bad idea, why it's bad, and if it is indeed bad, what the 'best' most django/pythonic way to change values on the fly would be.
This is fine as long as you don't do anything later that will cause the queryset to be re-evaluated - for example, slicing it. That will make another query to the database, and all your modified objects will be replaced with fresh ones.
A way to protect yourself against that would be to convert to a list first:
a = list(SomeTable.objects.all())
This way, further slicing etc won't cause a fresh db call, and any modifications will be preserved.
Yup. See docs here
SomeTable.objects.filter(some_column='foo').update(some_column='bar')
I would go with Django's idiom. It executes the SQL with a single statement with 'where' and 'update' rather sending multiple SQL statements like your code would. This saves time. Check with Django's 'connection' to test SQL time.
Related
I use signals for things that should always be done when an object is deleted, saved, updated, etc. However, there are times when I don't want to call my save signals, so I use
Model.objects.filter(id=instance.id).update(field=value)
instead of the instance's save method:
instance.save()
In the case of deleting objects, there are also times when I don't want to call the delete signals, but I haven't found a way to avoid calling them. Is there a way??
UPDATE:
I'm using django 1.6.2 and calling the delete method like this:
Model.objects.filter(id=instance.id).delete()
on the queryset still still calls the delete signal.
There is unsafe and undocumented way which probably can change between versions and also can lead to unexpected consequences which will be hard to debug.
This is a hack that probably you don't want to use but.... it's possible.
qs = Model.objects.filter(id=instance.id)
qs.order_by().select_related(None)._raw_delete(qs.db)
If you have any related objects to this one it will probably fail with database error because records will not be deleted automatically by Django
In the place where you don't want signals to be called, you have to disconnect them, run you code and then reconnect them again. You can take a look on how mute_signal decorator is implemented in FactoryBoy or just use (but it's basically intended for django tests)
You can do this.
Model.objects.filter(id=instance.id).delete()
Try using raw SQL and don't resist it. It is always a powerful ultimate weapon you could use.
from django.db import connection
with connection.cursor() as cursor:
cursor.execute(f'DELETE from tbl_model where id={instance.id}')
If I perform a prefetch_related('toppings') for a queryset, and I want to later filter(spicy=True) by fields in the related table, Django ignores the cached info and does a database query. I found that this is documented (under the Note box) and seems to happen for all forms of caching (select_related(), already evaluated querysets, etc.) when another filter() is performed.
However, is there some sort of super secret hidden time-saving shortcut to filter locally (using the cache and not hitting the database) without having to write the python code to loop the queryset (using list/dict comprehension, etc.)? Maybe something like a filter_locally(spicy=True)?
EDIT:
One of the reasons why a list/comprehension doesn't work well for me is because a list/dict does not have the queryset methods. In my case, the first level M2M field, toppings, isn't the end goal for me and I need to check a 2nd related M2M field (which I have already pre-fetched as well). While this is also possible using list comprehension, it's just much simpler to have something such as filter_locally(spicy=True, origin__country='Spain') because:
it allows accessing many levels of related fields with minimal effort
it allows chaining other queryset methods
it's easier to read because it's consistent with the familiar filter()
it's easier to modify existing code using filter() without prefetch to add this optimization in without much changes.
But from the responses, Django has no such support :(
You have to write the python code to loop through the queryset (a list/dict comprehension is ideal). All the filter() code knows how to do is add filtering language to the SQL sent to the database. Filtering locally is a totally different problem than filtering remotely, so the solutions to those two separate problems won't be able to share any logic.
A list comprehension one-liner would be pretty straightforward, though; the syntax might not be much more complex than with filter().
If you're filtering on a boolean doing the list comprehension is pretty easy. You can also swap out the topping.spicy==True for a string comparison or whatever.
I would do something like:
qs = Pizza.objects.all().prefetch_related('toppings')
res = list(qs)
def get_spicy(qs):
res = list(qs)
return [pizza for pizza in res if any(topping.spicy==True for
topping in pizza.toppings.all())]
That is if you want to return the pizza object if any of its toppings is spicy. You can also replace the any() with all() to check for all, and do a lot of pretty powerful queries with this syntax. I'm somewhat surprised that there is no easy way to do this in django. It seems like a lot of these simple queries should be easy to implement in a generic manner.
The above code assumes a many2many. It should be easy to modify to work with a simple FK relationship such as a one2one or one2many.
Hope this was helpful.
I'm running a Django shop where we serve each our clients an object graph which is completely separate from the graphs of all the other clients. The data is moderately sensitive, so I don't want any of it to leak from one client to another, nor for one client to delete or alter another client's data.
I would like to structure my code such that I by default write code which adheres to the security requirements (No hard guarantees necessary), but lets me override them when I know I need to.
My main fear is that in a Twig.objects.get(...), I forget to add client=request.client, and likewise for Leaf.objects.get where I have to check that twig__client=request.client. This quickly becomes error-prone and complicated.
What are some good ways to get around my own forgetfulness? How do I make this a thing I don't have to think about?
One candidate solution I have in mind is this:
Set the default object manager as DANGER = models.Manager() on my abstract base class(es).
Have a method ok(request) on said base classes which applies .filter(leaf__twig__branch__trunk__root__client=request.client) as applicable.
use MyModel.ok(request) instead of MyModel.objects wherever feasible.
Can this be improved upon? One not so nice issue is when a view calls a model method, e.g. branch.get_twigs_with_fruit, I now have to either pass a request for it to run through ok or I have to invoke DANGER. I like neither :-\
Is there some way of getting access to the current request? I think that might mitigate the situation...
Ill explain a different problem I had however I think the solution might be something to look into.
Once I was working on a project to visualize data where I needed to have a really big table which will store all the data for all visualizations. That turned out to be a big problem because I would have to do things like Model.objects.filter(visualization=5) which was just not very elegant and not efficient.
To make things simpler and more efficient I ended up creating dynamic models on the fly. Essentially I would create a separate table in the db on the fly and then store a data only for that one visualization in that. My code is something like:
def get_model_class(table_name):
class ModelBase(ModelBase):
def __new__(cls, name, bases, attrs):
name = '{}_{}'.format(name, table_name)
return super(ModelBase, cls).__new__(cls, name, bases, attrs)
class Data(models.Model):
# fields here
__metaclass__ = ModelBase
class Meta(object):
db_table = table_name
return Data
dynamic_model = get_model_class('foo')
This was useful for my purposes because it allowed queries to be much faster but getting back to your issue I think something like this can be useful because this will make sure that each client's data is separate not only via a foreign key, but is actually separated in the db.
Using this method is pretty straight forward except before using the model, you have to call the function to get it for each client. To make things more efficient you can cache/memoize the results of the function call so that it does not have to recompute the same thing more than once.
Using Django 1.3 with PostgreSQL 9.0, I have a multi-step object creation function/view, where:
The main object is created (have tried both MyModel.objects.create() and manually using object.save() methods) and,
Then m2m relationships are setup (they must follow the main object creation so that said object has an id to relate to).
Some of those relationships may fail, or some other problem may arise, thus I need the entire function to behave atomically.
I've tried wrapping the function with the transaction.commit_on_success decorator, as well as tried using commit_manually (and setting the commit point at the end of the function); but neither works. That is, the main object is created and saved in the database, even when an exception is raised later on in the function. This leaves the database in an inconsistent state, to put it politely. So, how to debug this? I've seen similar questions, but they had to do with using MySQL, whereas this kind of broken transaction is not supposed to happen with Postgres. There were tickets on the Django Trac about this issue from years back, but they were supposedly fixed/resolved. Could any Djangonauts out there provide enlightenment please?
See this ticket: https://code.djangoproject.com/ticket/6669
I think for now you'll just need to call transaction.rollback() explicitly when you get an IntegrityError
I don't know if this applies to you, but the problem that brought me here was a failure to read the manual with regard to Django testing.
If you are testing code with transactions in it you need to use TransactionTestCase instead of TestCase, failure to do so will result in the tests seeing the behavior you describe.
I'm using QSqlQuery::prepare() and ::addBindValue() for my queries in a Qt project I'm working on. There's a lot of repeated code and though I think that's the "right" way, I wanted to make sure. Perhaps someone has alternative ideas? Example:
QSqlQuery newQuery;
newQuery.prepare("INSERT INTO table "
"(foo,bar,baz,"
"herp,derp,biggerp,"
"alpha,beta,gamma,"
"etc) VALUES "
"(?,?,?,"
"?,?,?,"
"?,?,?,"
"?)");
newQuery.addBindValue(this->ui->txtFoo->text());
newQuery.addBindValue(this->ui->txtBar->text());
newQuery.addBindValue(this->ui->txtBaz->text());
newQuery.addBindValue(this->ui->txtHerp->text());
newQuery.addBindValue(this->ui->txtDerp->text());
newQuery.addBindValue(this->ui->txtBiggerp->text());
newQuery.addBindValue(this->ui->txtAlpha->text());
newQuery.addBindValue(this->ui->txtBeta->text());
newQuery.addBindValue(this->ui->txtGamma->itemText(0));
newQuery.addBindValue(this->ui->txtEtc->text());
newQuery.exec();
You can see there's a bunch of the same "newQuery.addBindValue(this->ui->__________" over and over. Is this the 'best' way to go about it?
Also, I asked in #qt on freenode the other night but didn't get a definitive answer; will the above (::prepare with ::addBindValue) protect agains SQL injection? The reference didn't really say.
It might look a bit tidier if you first create a QMap or QStringList with the bindings, then iterate through that data structure and call addBindValue() for each item in the list/map.
In relation to your sub-question on SQL injection, that combination of ::prepare and ::addBindValue does indeed fully protect against it. This is because the bound values are never parsed by the SQL engine; they're just values that slot in after compilation (the preparation step) and before execution.
Of course, you have to be careful when taking values out of the DB too, but that's not protecting the database but rather ensuring that the values aren't used to cause other mischief (e.g., injecting unexpected malicious <script> tags into HTML or, worse still, a <blink> or <marquee> monstrosity). But that's another problem, and doesn't apply to all uses anyway; putting the values in a strictly plain text GUI field is usually no problem.