Here's one for my Django comrades.
I am trying to build an application in Django 1.6.5 and MySQL where some database fields are encrypted in order to respect the privacy of users. I have successfully installed django-extensions and successfully used the EncryptedCharField to do a simple save and data retrieval.
However, I am running into a problem when using EncryptedCharField for floats. Now, simple typecasting has been useful--this problem doesn't seem to be that problem. I'm trying to start with an encrypted field that is a value, change the value by adding/subtracting some number, then save the back to the database. I've managed to reduce and reproduce the error like this:
>>> user_id = 1
>>> account = AccountCash.objects.get( id = 1 )
>>> account.id
1L
>>> account.portfolio_id
1L
>>> account.account_name
u'My Cash'
>>> account.current_value
u'200'
>>> account.current_value = account.current_value - 10
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: unsupported operand type(s) for -: 'unicode' and 'int'
Here, account_name and current_value are both EncryptedCharFields. We see that current_value is a unicode, so typecasting to a float should (I thought) resolve the issue, as it has for me in other places.
However, doing so leads to another problem in
>>> account.current_value = float(account.current_value) - 10
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/jasonnett/anaconda/envs/bb_env/lib/python2.7/site-packages/django/db/models/fields/subclassing.py", line 35, in __set__
obj.__dict__[self.field.name] = self.field.to_python(value)
File "/Users/jasonnett/anaconda/envs/bb_env/lib/python2.7/site-packages/django_extensions/db/fields/encrypted.py", line 69, in to_python
elif value and (value.startswith(self.prefix)):
AttributeError: 'float' object has no attribute 'startswith'
I haven't been able to figure out what is different about assigning a float value to the encrypted field here vs. where I originally set the value like this:
# Make the new account, passing in the portfolio it belongs to
new_account = AccountCash(
portfolio = portfolio,
account_name = newCashAccountForm.cleaned_data['account_name'],
starting_balance = newCashAccountForm.cleaned_data['starting_balance'],
current_value = newCashAccountForm.cleaned_data['starting_balance'],
)
new_account.save()
Related
I have a date of birth (dob) DateField in my Django Model (Author). I tried to annotate age parameter. I searched for many possible ways to do it and each procedure generated some kind of error.
Here I tried in python console first to make sure that the expression would be a valid one:
>>> from datetime import datetime
>>> (datetime.now() - datetime(2000,1,1)).days #output: 7506
First Try:
>>> from django.db.models import F
>>> authors = Author.objects.annotate(age = (datetime.now()-F('dob')).days) #no-error here
>>> print(authors) # Exception thrown here
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 324, in __getitem__
qs._fetch_all()
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 1305, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 70, in __iter__
for row in compiler.results_iter(results):
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/sql/compiler.py", line 1100, in apply_converters
value = converter(value, expression, connection)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/backends/sqlite3/operations.py", line 291, in convert_datefield_value
value = parse_date(value)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/utils/dateparse.py", line 75, in parse_date
match = date_re.match(value)
TypeError: expected string or bytes-like object
Second time I used ExpressionWrapper to define that output type will be DateTimeField
>>> from django.db.models import DateTimeField, ExpressionWrapper
>>> authors = Author.objects.annotate(age = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField()).days)
AttributeError: 'ExpressionWrapper' object has no attribute 'days'
I have also tried the RawSQL. I am using Sqlite database here. so date('now') would provide me current date.
>>> from django.db.models.expressions import RawSQL
>>> authors = Author.objects.annotate(age=RawSQL("date('now')-dob"))
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'params'
So is there any way, I could solve the issue?
You were almost there, the issue was the output_field value. It should be DurationField instead of DateTimeField
age_expr = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField())
queryset = Author.objects.annotate(age=age_expr)
NOTE: You can't use .days along with ExpressionWrapper since the annotate operation performs in the DB level.
To calculate the age, you may need to use the .total_seconds() method
print(queryset[0].age.total_seconds() / 60 / 60 / 24 / 365.25)
Empty string or list params worked for the RawSQL command.
authors = Author.objects.annotate(age=RawSQL("date('now')-dob", params=""))
Trying to get the city from the zip code, so I installed the pyzipcode package. However, whenever I try to get the city I get the following error. The state, zipcode, lat/long work normally. I've tried numerous zipcodes and none of them seem to pull any city records.
from pyzipcode import ZipCodeDatabase
zcdb = ZipCodeDatabase()
zipc=zcdb['90210']
zipc.city
Traceback (most recent call last):
File "<console>", line 1, in <module>
AttributeError: 'ZipCode' object has no attribute 'city'
The documentation for the package uses 'city' as an example: https://pypi.org/project/pyzipcode/.
I don't understand what I am doing wrong here. Here is my Django model:
class VMMigrationEvent(models.Model):
created = models.DateTimeField(auto_now_add=True) # DateTime because there may be multiple events in a day.
dc = models.TextField(max_length=150), # data center
destination_host = models.TextField(max_length=150), # host migrated to.
message = models.TextField(), # The message from virtual center.
updated = models.DateTimeField(auto_now=True) # DateTime because there may be multifple events in a day.
user = models.TextField(max_length=150), # The user logged into the virtual center that execute the migration.
vc = models.TextField(max_length=150), # virtual center
vm = models.ForeignKey(VirtualMachine) # The VirtualMachine record associated with this event.
And from the python console I do this:
>>> from cm.models import *
>>> dc='DCM01N-01'
>>> destination_host='auroravm2-1.example.com'
>>> message='my hovercraft is full of eels.'
>>> user='mister_gumby'
>>> vc='vca-001-s.example.com'
>>> vm='ads-108'
>>> vm_db_obj = VirtualMachine.objects.filter(name=vm).latest('create_date')
>>> vmme = VMMigrationEvent.objects.create(dc=dc,
... destination_host=destination_host,
... message=message, user=user, vc=vc, vm=vm_db_obj)
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/query.py", line 392, in create
obj = self.model(**kwargs)
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/base.py", line 573, in __init__
raise TypeError("'%s' is an invalid keyword argument for this function" % list(kwargs)[0])
TypeError: 'vc' is an invalid keyword argument for this function
Why is vc invalid? It is a field in my model?
You have commas after several of your field definitions: vc, but also dc, destination_host and user. This turns them into tuples, which can't be used as fields on the model.
Delete those commas
(Also, you probably meant CharField rather than TextField.)
INSTALLED_APPS in django is obviously a tuple and therefore immutable.
Why does django-userena try to append a module to it at runtime?
Reference userena/tests/profiles/test.py
from django import test
class ProfileTestCase(test.TestCase):
""" A custom TestCase that loads the profile application for testing purposes """
def _pre_setup(self):
# Add the models to the db.
self._original_installed_apps = list(settings.INSTALLED_APPS)
settings.INSTALLED_APPS.append('userena.tests.profiles')
loading.cache.loaded = False
call_command('syncdb', interactive=False, verbosity=0)
# Call the original method that does the fixtures etc.
super(ProfileTestCase, self)._pre_setup()
def _post_teardown(self):
# Call the original method.
super(ProfileTestCase, self)._post_teardown()
# Restore the settings.
settings.INSTALLED_APPS = self._original_installed_apps
loading.cache.loaded = False
And obviously, when I run the unit tests with userena, I get errors such as:-
======================================================================
ERROR: test_can_view_profile (userena.tests.models.BaseProfileModelTest)
Test if the user can see the profile with three type of users.
----------------------------------------------------------------------
Traceback (most recent call last):
File "./django-trunk/django/test/testcases.py", line 499, in __call__
self._pre_setup()
File "./_thirdparty/django-userena/userena/tests/profiles/test.py", line 11, in _pre_setup
settings.INSTALLED_APPS.append('userena.tests.profiles')
AttributeError: 'tuple' object has no attribute 'append'
======================================================================
ERROR: test_get_full_name_or_username (userena.tests.models.BaseProfileModelTest)
Test if the full name or username are returned correcly
----------------------------------------------------------------------
Traceback (most recent call last):
File "./django-trunk/django/test/testcases.py", line 499, in __call__
self._pre_setup()
File "./_thirdparty/django-userena/userena/tests/profiles/test.py", line 11, in _pre_setup
settings.INSTALLED_APPS.append('userena.tests.profiles')
AttributeError: 'tuple' object has no attribute 'append'
How do I solve this problem?
I think this:
self._original_installed_apps = list(settings.INSTALLED_APPS)
settings.INSTALLED_APPS.append('userena.tests.profiles')
Should be
self._original_installed_apps = list(settings.INSTALLED_APPS)
settings.INSTALLED_APPS += ('userena.tests.profiles',)
Looks like a bug to me.
INSTALLED_APPS is a tuple by default but can be changed to a list. The author of that app probably did change that for themselves, wrote the tests and didn't realize that it won't work for people who have INSTALLED_APPS as a tuple. You can most likely fix the problem by changing your settings.INSTALLED_APPS to a list.
Btw, there are better ways how to override settings.
I have some date filtering in django that I just can't see what's wrong.
I have a model with this field:
date_of_birth = models.DateField(blank = True, null=True, auto_now=False, auto_now_add=False)
I then select all data
users = UserDetails.objects.all()
I have 2 rows in the database and two object returned
>>> users
[<UserDetails: UserDetails object>, <UserDetails: UserDetails object>]
>>> users[0]
<UserDetails: UserDetails object>
I can see each objects date_of_birth value
>>> users[0].date_of_birth
datetime.date(1971, 9, 28)
However any filter I try always fails?
>>> users = users.filter(date_of_birth__year >= 1970)
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'date_of_birth__year' is not defined
>>> users = users.filter(date_of_birth == datetime.date(1971, 9, 28))
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'date_of_birth' is not defined
>>> users = users.filter(date_of_birth == '1971-09-28')
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'date_of_birth' is not defined
Can anyone help explain what's wrong here?
I'm at a loss.
Thanks.
You don't filter using the comparison operators. What you're doing is actually passing a keyword argument to the filter method. To filter based on equality, simply do:
users = users.filter(date_of_birth=datetime.date(1971, 9, 28))
To filter using greater than or equals, use:
users = users.filter(date_of_birth__year__gte=1970)
That is, you append __gte to the keyword argument.
You can also filter on less than or equals (date_of_birth__year__lte), less than (date_of_birth__year__lt), or greater than (date_of_birth__year__gt).