Django EncryptedCharField Error When Editing Value - django

Here's one for my Django comrades.
I am trying to build an application in Django 1.6.5 and MySQL where some database fields are encrypted in order to respect the privacy of users. I have successfully installed django-extensions and successfully used the EncryptedCharField to do a simple save and data retrieval.
However, I am running into a problem when using EncryptedCharField for floats. Now, simple typecasting has been useful--this problem doesn't seem to be that problem. I'm trying to start with an encrypted field that is a value, change the value by adding/subtracting some number, then save the back to the database. I've managed to reduce and reproduce the error like this:
>>> user_id = 1
>>> account = AccountCash.objects.get( id = 1 )
>>> account.id
1L
>>> account.portfolio_id
1L
>>> account.account_name
u'My Cash'
>>> account.current_value
u'200'
>>> account.current_value = account.current_value - 10
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: unsupported operand type(s) for -: 'unicode' and 'int'
Here, account_name and current_value are both EncryptedCharFields. We see that current_value is a unicode, so typecasting to a float should (I thought) resolve the issue, as it has for me in other places.
However, doing so leads to another problem in
>>> account.current_value = float(account.current_value) - 10
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/jasonnett/anaconda/envs/bb_env/lib/python2.7/site-packages/django/db/models/fields/subclassing.py", line 35, in __set__
obj.__dict__[self.field.name] = self.field.to_python(value)
File "/Users/jasonnett/anaconda/envs/bb_env/lib/python2.7/site-packages/django_extensions/db/fields/encrypted.py", line 69, in to_python
elif value and (value.startswith(self.prefix)):
AttributeError: 'float' object has no attribute 'startswith'
I haven't been able to figure out what is different about assigning a float value to the encrypted field here vs. where I originally set the value like this:
# Make the new account, passing in the portfolio it belongs to
new_account = AccountCash(
portfolio = portfolio,
account_name = newCashAccountForm.cleaned_data['account_name'],
starting_balance = newCashAccountForm.cleaned_data['starting_balance'],
current_value = newCashAccountForm.cleaned_data['starting_balance'],
)
new_account.save()

Related

Django: Annotating age from date_of_birth

I have a date of birth (dob) DateField in my Django Model (Author). I tried to annotate age parameter. I searched for many possible ways to do it and each procedure generated some kind of error.
Here I tried in python console first to make sure that the expression would be a valid one:
>>> from datetime import datetime
>>> (datetime.now() - datetime(2000,1,1)).days #output: 7506
First Try:
>>> from django.db.models import F
>>> authors = Author.objects.annotate(age = (datetime.now()-F('dob')).days) #no-error here
>>> print(authors) # Exception thrown here
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 324, in __getitem__
qs._fetch_all()
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 1305, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 70, in __iter__
for row in compiler.results_iter(results):
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/sql/compiler.py", line 1100, in apply_converters
value = converter(value, expression, connection)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/backends/sqlite3/operations.py", line 291, in convert_datefield_value
value = parse_date(value)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/utils/dateparse.py", line 75, in parse_date
match = date_re.match(value)
TypeError: expected string or bytes-like object
Second time I used ExpressionWrapper to define that output type will be DateTimeField
>>> from django.db.models import DateTimeField, ExpressionWrapper
>>> authors = Author.objects.annotate(age = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField()).days)
AttributeError: 'ExpressionWrapper' object has no attribute 'days'
I have also tried the RawSQL. I am using Sqlite database here. so date('now') would provide me current date.
>>> from django.db.models.expressions import RawSQL
>>> authors = Author.objects.annotate(age=RawSQL("date('now')-dob"))
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'params'
So is there any way, I could solve the issue?
You were almost there, the issue was the output_field value. It should be DurationField instead of DateTimeField
age_expr = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField())
queryset = Author.objects.annotate(age=age_expr)
NOTE: You can't use .days along with ExpressionWrapper since the annotate operation performs in the DB level.
To calculate the age, you may need to use the .total_seconds() method
print(queryset[0].age.total_seconds() / 60 / 60 / 24 / 365.25)
Empty string or list params worked for the RawSQL command.
authors = Author.objects.annotate(age=RawSQL("date('now')-dob", params=""))

Pyzipcode city lookup not working properly

Trying to get the city from the zip code, so I installed the pyzipcode package. However, whenever I try to get the city I get the following error. The state, zipcode, lat/long work normally. I've tried numerous zipcodes and none of them seem to pull any city records.
from pyzipcode import ZipCodeDatabase
zcdb = ZipCodeDatabase()
zipc=zcdb['90210']
zipc.city
Traceback (most recent call last):
File "<console>", line 1, in <module>
AttributeError: 'ZipCode' object has no attribute 'city'
The documentation for the package uses 'city' as an example: https://pypi.org/project/pyzipcode/.

Why am I getting TypeError: 'vc' is an invalid keyword argument for this function error?

I don't understand what I am doing wrong here. Here is my Django model:
class VMMigrationEvent(models.Model):
created = models.DateTimeField(auto_now_add=True) # DateTime because there may be multiple events in a day.
dc = models.TextField(max_length=150), # data center
destination_host = models.TextField(max_length=150), # host migrated to.
message = models.TextField(), # The message from virtual center.
updated = models.DateTimeField(auto_now=True) # DateTime because there may be multifple events in a day.
user = models.TextField(max_length=150), # The user logged into the virtual center that execute the migration.
vc = models.TextField(max_length=150), # virtual center
vm = models.ForeignKey(VirtualMachine) # The VirtualMachine record associated with this event.
And from the python console I do this:
>>> from cm.models import *
>>> dc='DCM01N-01'
>>> destination_host='auroravm2-1.example.com'
>>> message='my hovercraft is full of eels.'
>>> user='mister_gumby'
>>> vc='vca-001-s.example.com'
>>> vm='ads-108'
>>> vm_db_obj = VirtualMachine.objects.filter(name=vm).latest('create_date')
>>> vmme = VMMigrationEvent.objects.create(dc=dc,
... destination_host=destination_host,
... message=message, user=user, vc=vc, vm=vm_db_obj)
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/query.py", line 392, in create
obj = self.model(**kwargs)
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/base.py", line 573, in __init__
raise TypeError("'%s' is an invalid keyword argument for this function" % list(kwargs)[0])
TypeError: 'vc' is an invalid keyword argument for this function
Why is vc invalid? It is a field in my model?
You have commas after several of your field definitions: vc, but also dc, destination_host and user. This turns them into tuples, which can't be used as fields on the model.
Delete those commas
(Also, you probably meant CharField rather than TextField.)

(Obviously) Not possible to append to tuple. Why does django userena test do it?

INSTALLED_APPS in django is obviously a tuple and therefore immutable.
Why does django-userena try to append a module to it at runtime?
Reference userena/tests/profiles/test.py
from django import test
class ProfileTestCase(test.TestCase):
""" A custom TestCase that loads the profile application for testing purposes """
def _pre_setup(self):
# Add the models to the db.
self._original_installed_apps = list(settings.INSTALLED_APPS)
settings.INSTALLED_APPS.append('userena.tests.profiles')
loading.cache.loaded = False
call_command('syncdb', interactive=False, verbosity=0)
# Call the original method that does the fixtures etc.
super(ProfileTestCase, self)._pre_setup()
def _post_teardown(self):
# Call the original method.
super(ProfileTestCase, self)._post_teardown()
# Restore the settings.
settings.INSTALLED_APPS = self._original_installed_apps
loading.cache.loaded = False
And obviously, when I run the unit tests with userena, I get errors such as:-
======================================================================
ERROR: test_can_view_profile (userena.tests.models.BaseProfileModelTest)
Test if the user can see the profile with three type of users.
----------------------------------------------------------------------
Traceback (most recent call last):
File "./django-trunk/django/test/testcases.py", line 499, in __call__
self._pre_setup()
File "./_thirdparty/django-userena/userena/tests/profiles/test.py", line 11, in _pre_setup
settings.INSTALLED_APPS.append('userena.tests.profiles')
AttributeError: 'tuple' object has no attribute 'append'
======================================================================
ERROR: test_get_full_name_or_username (userena.tests.models.BaseProfileModelTest)
Test if the full name or username are returned correcly
----------------------------------------------------------------------
Traceback (most recent call last):
File "./django-trunk/django/test/testcases.py", line 499, in __call__
self._pre_setup()
File "./_thirdparty/django-userena/userena/tests/profiles/test.py", line 11, in _pre_setup
settings.INSTALLED_APPS.append('userena.tests.profiles')
AttributeError: 'tuple' object has no attribute 'append'
How do I solve this problem?
I think this:
self._original_installed_apps = list(settings.INSTALLED_APPS)
settings.INSTALLED_APPS.append('userena.tests.profiles')
Should be
self._original_installed_apps = list(settings.INSTALLED_APPS)
settings.INSTALLED_APPS += ('userena.tests.profiles',)
Looks like a bug to me.
INSTALLED_APPS is a tuple by default but can be changed to a list. The author of that app probably did change that for themselves, wrote the tests and didn't realize that it won't work for people who have INSTALLED_APPS as a tuple. You can most likely fix the problem by changing your settings.INSTALLED_APPS to a list.
Btw, there are better ways how to override settings.

Django Date Filter Weirdness

I have some date filtering in django that I just can't see what's wrong.
I have a model with this field:
date_of_birth = models.DateField(blank = True, null=True, auto_now=False, auto_now_add=False)
I then select all data
users = UserDetails.objects.all()
I have 2 rows in the database and two object returned
>>> users
[<UserDetails: UserDetails object>, <UserDetails: UserDetails object>]
>>> users[0]
<UserDetails: UserDetails object>
I can see each objects date_of_birth value
>>> users[0].date_of_birth
datetime.date(1971, 9, 28)
However any filter I try always fails?
>>> users = users.filter(date_of_birth__year >= 1970)
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'date_of_birth__year' is not defined
>>> users = users.filter(date_of_birth == datetime.date(1971, 9, 28))
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'date_of_birth' is not defined
>>> users = users.filter(date_of_birth == '1971-09-28')
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'date_of_birth' is not defined
Can anyone help explain what's wrong here?
I'm at a loss.
Thanks.
You don't filter using the comparison operators. What you're doing is actually passing a keyword argument to the filter method. To filter based on equality, simply do:
users = users.filter(date_of_birth=datetime.date(1971, 9, 28))
To filter using greater than or equals, use:
users = users.filter(date_of_birth__year__gte=1970)
That is, you append __gte to the keyword argument.
You can also filter on less than or equals (date_of_birth__year__lte), less than (date_of_birth__year__lt), or greater than (date_of_birth__year__gt).