I'm trying to delete a table hosted in heroku. Because it is in the shared database I can't connect to postgres directly to perform sentence. I execute delete through django shell. Issue is that table name is in uppercase and, for unknow reason, django try to delete it in lower cases:
>>> sql = '''drop table campings_horesEntrega cascade; <---see uppercases
... drop table campings_horesRecollida; '''
>>>
>>> from django.db import connections, transaction
>>> cursor = connections['default'].cursor()
>>> cursor.execute(sql)
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/app/.heroku/venv/lib/python2.7/site-packages/django/db/backends/util.py", line 40, in execute
return self.cursor.execute(sql, params)
File "/app/.heroku/venv/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 52, in execute
return self.cursor.execute(query, args)
DatabaseError: table "campings_horesentrega" does not exist <--- see lowercases
someone has a workaround to my issue?
thanks a lot.
You need to wrap your table names in quotes if you want postgres to respect case.
DROP TABLE "campings_horesEntrega" CASCADE;
Related
I have a date of birth (dob) DateField in my Django Model (Author). I tried to annotate age parameter. I searched for many possible ways to do it and each procedure generated some kind of error.
Here I tried in python console first to make sure that the expression would be a valid one:
>>> from datetime import datetime
>>> (datetime.now() - datetime(2000,1,1)).days #output: 7506
First Try:
>>> from django.db.models import F
>>> authors = Author.objects.annotate(age = (datetime.now()-F('dob')).days) #no-error here
>>> print(authors) # Exception thrown here
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 324, in __getitem__
qs._fetch_all()
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 1305, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 70, in __iter__
for row in compiler.results_iter(results):
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/sql/compiler.py", line 1100, in apply_converters
value = converter(value, expression, connection)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/backends/sqlite3/operations.py", line 291, in convert_datefield_value
value = parse_date(value)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/utils/dateparse.py", line 75, in parse_date
match = date_re.match(value)
TypeError: expected string or bytes-like object
Second time I used ExpressionWrapper to define that output type will be DateTimeField
>>> from django.db.models import DateTimeField, ExpressionWrapper
>>> authors = Author.objects.annotate(age = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField()).days)
AttributeError: 'ExpressionWrapper' object has no attribute 'days'
I have also tried the RawSQL. I am using Sqlite database here. so date('now') would provide me current date.
>>> from django.db.models.expressions import RawSQL
>>> authors = Author.objects.annotate(age=RawSQL("date('now')-dob"))
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'params'
So is there any way, I could solve the issue?
You were almost there, the issue was the output_field value. It should be DurationField instead of DateTimeField
age_expr = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField())
queryset = Author.objects.annotate(age=age_expr)
NOTE: You can't use .days along with ExpressionWrapper since the annotate operation performs in the DB level.
To calculate the age, you may need to use the .total_seconds() method
print(queryset[0].age.total_seconds() / 60 / 60 / 24 / 365.25)
Empty string or list params worked for the RawSQL command.
authors = Author.objects.annotate(age=RawSQL("date('now')-dob", params=""))
my dev environment is in python 3.6 and virtualenv(virtualenv 15.1.0) and with dependencies below:
django 2.2
ibm-db 3.0.1
ibm-db-django 1.2.0.0
as i used "(env_django2) λ python manage.py check", it return "System check identified no issues (0 silenced)." that is no problom.
however, when i used "python manage.py inspectdb --database db2 DEMO.DEMOUSER", it return like these below:
(env_django2) λ python manage.py inspectdb --database db2 DEMO.DEMOUSER
# This is an auto-generated Django model module.
# You'll have to do the following manually to clean this up:
# * Rearrange models' order
# * Make sure each model has one field with primary_key=True
# * Make sure each ForeignKey has `on_delete` set to the desired behavior.
# * Remove `managed = False` lines if you wish to allow Django to create, modify, and delete the table
# Feel free to rename the models, but don't rename db_table values or field names.
from django.db import models
# Unable to inspect table 'DEMO.DEMOUSER'
# The error was: 'NoneType' object is not subscriptable
this is my db2 setting:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
},
'db2':{
'ENGINE': 'ibm_db_django',
'NAME': 'test',
'USER': 'db2inst1',
'PASSWORD': 'XXXXXX',
'HOST': 'XX.XXX.XXX.XXX',
'PORT': 'XXXXX',
'PCONNECT':True,
}
}
and i had test it by coding like these below, and that is no problom.
with connections['db2'].cursor() as cursor:
cursor.execute("""
select * from DEMO.DEMOUSER
""")
columns = [col[0] for col in cursor.description]
listResult = [dict(zip(columns, row)) for row in cursor.fetchall()]
print(listResult)
so my question is how can i make the inspectdb work?
i have check inspectdb.py, and i found that
table_description = connection.introspection.get_table_description(cursor, table_name)
except Exception as e:
yield "# Unable to inspect table '%s'" % table_name
func get_table_description will invole the code below:
sql = "SELECT TYPE FROM SYSIBM.SYSTABLES WHERE CREATOR='%(schema)s' AND NAME='%(table)s'" % {'schema': schema.upper(), 'table': table_name.upper()}
it seem i didn't set the table and schema name right.
ok, i have to go to sleep, very tired.
2020.3.13 update........
so when i readed the code in ibm_db_django\introspection.py, i found that below:
schema = cursor.connection.get_current_schema()
that mean, ibm_db_django will get the schema by default, which is your user name setted in django's setting file. so it will always set the table name like 'db2inst1.DEMO.DEMOUSER' in my case. u can't change the schema by yourself.
as i know, setting the schema immutable is unreasonable,because the schema and user was different things in db2 unlike oracle. everyone can use the granted schema.
hence, i had to change the code like these,so that i can set my schema correctly:
# change the code below
#schema = cursor.connection.get_current_schema()
print( '-----------change by HK 20200313-----------------')
if len(table_name.split('.')) == 2:
cursor.connection.set_current_schema(table_name.split('.')[0].upper())
table_name = table_name.split('.')[1]
schema = cursor.connection.get_current_schema()
print('----------------- HK TEST schema:%s'%schema)
# change the code below
#cursor.execute( "SELECT * FROM %s FETCH FIRST 1 ROWS ONLY" % qn( table_name ) )
sql = "SELECT * FROM %s FETCH FIRST 1 ROWS ONLY" % ( schema+'.'+table_name )
print('------------ HK TEST LOG:%s'%sql)
cursor.execute( sql )
so the problem had been solved,however,another emerged !!
when i typed the command "python manage.py inspectdb --database db2 DEMO.DEMOUSER", it show below:
class DemoDemouser(models.Model):
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "D:\MyProject\python\django-sample\env_django2\lib\site-packages\django\core\management\__init__.py", line 381, in execute_from_command_line
utility.execute()
File "D:\MyProject\python\django-sample\env_django2\lib\site-packages\django\core\management\__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "D:\MyProject\python\django-sample\env_django2\lib\site-packages\django\core\management\base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "D:\MyProject\python\django-sample\env_django2\lib\site-packages\django\core\management\base.py", line 364, in execute
output = self.handle(*args, **options)
File "D:\MyProject\python\django-sample\env_django2\lib\site-packages\django\core\management\commands\inspectdb.py", line 34, in handle
for line in self.handle_inspection(options):
File "D:\MyProject\python\django-sample\env_django2\lib\site-packages\django\core\management\commands\inspectdb.py", line 102, in handle_inspection
column_name = row.name
AttributeError: 'list' object has no attribute 'name'
checking inspectdb.py，i find out the answer. the description of specified table ,returned from the func in introspection.py implemented by ibm_db_django, is different from inspectdb.py in django 2.2.
so check it with django release history and ibm_db_ango.
ibm_db_ango 1.2 released in 2018.4.3, django 2.2 released in 2019.3.
it seem django 2.2 dont support the ibm_db_ango 1.2(the latest version)
therefore i had to install django 2.0 instead.
finally it work!!!.
I am using raw_sql queries for my convenience for keeping my database minimal I am deleting extra records. By this query
#d is from a loop and has values
res=MyModel.objects.raw("DELETE FROM mydb_mymodel WHERE mydb_mymodel.s_type = '%s' and mydb_mymodel.barcode = '%s' and mydb_mymodel.shopcode = '%s' and mydb_mymodel.date = '%s'" ,[d.s_type,d.barcode,d.shopcode,d.date])
It is not deleting records in database but
when I do res.query and run it from postgres console it works!
Yes I can use
MyModel.objects.filter(s_type=d.s_type,barcode=d.barcode,
shopcode=d.shopcode,date=d.date).delete()
but what I am missing in raw_sql?
A .raw(..) is not executed eagerly, it is, just like most Django ORM queries performed lazily. It thus returns a RawQuerySet object with the query in the object. For example:
>>> User.objects.raw('BLA BLA BLA', [])
<RawQuerySet: BLA BLA BLA>
A query like BLA BLA BLA does not make any sense: a database will error on it, but still we retrieve a RawQuerySet.
You can force evaluation by for example iterating over it, and then we get:
>>> list(User.objects.raw('BLA BLA BLA', []))
Traceback (most recent call last):
File "/djangotest/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/djangotest/env/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 71, in execute
return self.cursor.execute(query, args)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 250, in execute
self.errorhandler(self, exc, value)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
raise errorvalue
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute
res = self._query(query)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 412, in _query
rowcount = self._do_query(q)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 375, in _do_query
db.query(q)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/connections.py", line 276, in query
_mysql.connection.query(self, query)
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'BLA BLA BLA' at line 1")
So the list(..) forces evaluation, and now the database of course produces an error. However even if it was a valid DELETE query, it would still raise an error, since such query does not return any record.
In order to make DELETE calls, the Django manual specifies that you should use a cursor [Django-doc]:
from django.db import connection
with connection.cursor() as cursor:
cursor.execute(
"DELETE FROM mydb_mymodel WHERE s_type = '%s' AND barcode = '%s' AND shopcode = '%s' AND date = '%s'" ,
[d.s_type,d.barcode,d.shopcode,d.date]
)
But I think it is probably a lot simpler to specify it like:
MyModel.objects.filter(
s_type=d.s_type,
barcode=d.barcode,
shopcode=d.shopcode,
date=d.date
).delete()
This will construct a DELETE query, and serialize the parameters properly. A .delete() query is done eagerly, so the odds of making above discussed mistakes is a lot lower: if the ORM is implemented correctly, then we do not need to worry about that.
try this:
from django.db import connection
def executeQuery(self, sql, params=[]):
with connection.cursor() as cursor:
cursor.execute(
sql,
params
)
use it like:
executeQuery("DELETE FROM mydb_mymodel WHERE mydb_mymodel.s_type = '%s' and mydb_mymodel.barcode = '%s' and mydb_mymodel.shopcode = '%s' and mydb_mymodel.date = '%s'" ,[d.s_type,d.barcode,d.shopcode,d.date])
I changed my models and made migrations. Then i changed my models another one time and when try python manage.py migrate i get error:
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions, shop
Running migrations:
Applying shop.0004_auto_20180128_1331...Traceback (most recent call last):
File "/home/morilon/dj/intshop/venv/lib/python3.6/site-packages/django/db/backends/utils.py", line 83, in _execute
return self.cursor.execute(sql)
File "/home/morilon/dj/intshop/venv/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 301, in execute
return Database.Cursor.execute(self, query)
sqlite3.OperationalError: table "shop_brand" already exists
So my question is - how can i delete table "shop_brand"???
I already tried flush and sqlflush but that only delete data from table but not actually table "shop_brand".
I use django 2.0.1 and python 3.6
Use the dbshell command
python manage.py dbshell
then while in the shell, depending on what database you are using, you type the command to show tables to identify the table you want to drop.
For instance, for sqlite, you would use
.tables
still in the shell, you can use SQL command to drop the table
DROP TABLE shop_brand;
So I was developing an app in Django and needed a function from the 1.4 version so I decided to update.
But then a weird error appeared when I wanted to do syncdb
I am using the new manage.py and as You can see it makes some of the tables but then fails :
./manage.py syncdb
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_user_permissions
Creating table auth_user_groups
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site
Traceback (most recent call last):
File "./manage.py", line 9, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/__init__.py", line 443, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/__init__.py", line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/base.py", line 196, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/base.py", line 232, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/base.py", line 371, in handle
return self.handle_noargs(**options)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/core/management/commands/syncdb.py", line 91, in handle_noargs
sql, references = connection.creation.sql_create_model(model, self.style, seen_models)
File "/usr/local/lib/python2.7/dist-packages/Django-1.4-py2.7.egg/django/db/backends/creation.py", line 44, in sql_create_model
col_type = f.db_type(connection=self.connection)
TypeError: db_type() got an unexpected keyword argument 'connection'
I had the same issue, the definition for my custom field was missing the connection parameter.
from django.db import models
class BigIntegerField(models.IntegerField):
def db_type(self, connection):
return "bigint"
Although already old, answered and accepted question but I am adding my understanding I have added it because I am not using customized type and it is a Django Evolution error (but not syncdb)evolve --hint --execute. I think it may be helpful for someone in future. .
I am average in Python and new to Django. I also encounter same issue when I added some new features to my existing project. To add new feature I had to add some new fields of models.CharField() type,as follows.
included_domains = models.CharField(
"set of comma(,) seprated list of domains in target emails",
default="",
max_length=it_len.EMAIL_LEN*5)
excluded_domains = models.CharField(
"set of comma(,) seprated list of domains NOT in target emails",
default="",
max_length=it_len.EMAIL_LEN*5)
The Django version I am using is 1.3.1:
$ python -c "import django; print django.get_version()"
1.3.1 <--------# version
$python manage.py syncdb
Project signature has changed - an evolution is required
Django Evolution: Django Evolution is an extension to Django that allows you to track changes in your models over time, and to update the database to reflect those changes.
$ python manage.py evolve --hint
#----- Evolution for messagingframework
from django_evolution.mutations import AddField
from django.db import models
MUTATIONS = [
AddField('MessageConfiguration', 'excluded_domains', models.CharField, initial=u'', max_length=300),
AddField('MessageConfiguration', 'included_domains', models.CharField, initial=u'', max_length=300)
]
#----------------------
Trial evolution successful.
Run './manage.py evolve --hint --execute' to apply evolution.
The trial was susses and when I tried to apply changes in DB
$ python manage.py evolve --hint --execute
Traceback (most recent call last):
File "manage.py", line 25, in <module>
execute_manager(settings)
File "/var/www/sites/www.taxspanner.com/django/core/management/__init__.py", line 362, in execute_manager
utility.execute()
File "/var/www/sites/www.taxspanner.com/django/core/management/__init__.py", line 303, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/var/www/sites/www.taxspanner.com/django/core/management/base.py", line 195, in run_from_argv
self.execute(*args, **options.__dict__)
File "/var/www/sites/www.taxspanner.com/django/core/management/base.py", line 222, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/management/commands/evolve.py", line 60, in handle
self.evolve(*app_labels, **options)
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/management/commands/evolve.py", line 140, in evolve
database))
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/mutations.py", line 426, in mutate
return self.add_column(app_label, proj_sig, database)
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/mutations.py", line 438, in add_column
sql_statements = evolver.add_column(model, field, self.initial)
File "/usr/local/lib/python2.7/dist-packages/django_evolution-0.6.9.dev_r225-py2.7.egg/django_evolution/db/common.py", line 142, in add_column
f.db_type(connection=self.connection), # <=== here f is field class object
TypeError: db_type() got an unexpected keyword argument 'connection'
To understand this exception I check that this exception is something similar to:
>>> def f(a):
... print a
...
>>> f('b', b='a')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() got an unexpected keyword argument 'b'
>>>
So the function signature has been changed.
Because I have not added any new customized or enum fields but only two similar fields that was already in model and char type field is supported by most of database (I am ussing PostgreSQL) even I was getting this error!
Then I read from #: Russell Keith-Magee-4 Reply.
What you've hit here is the end of the deprecation cycle for code that
doesn't support multiple databases.
In Django 1.2, we introduced multiple database support; in order to
support this, the prototype for get_db_preb_lookup() and
get_db_prep_value() was changed.
For backwards compatibility, we added a shim that would transparently
'fix' these methods if they hadn't already been fixed by the
developer.
In Django 1.2, the usage of these shims raised a
PendingDeprecationWarning. In Django 1.3, they raised a
DeprecationWarning.
Under Django 1.4, the shim code was been removed -- so any code that
wasn't updated will now raise errors like the one you describe.
But I am not getting any DeprecationWarning warning assuming because of newer version of Django Evolution.
But from above quote I could understand that to support multiple databases function signature is added and an extra argument connection is needed. I also check the db_type() signature in my installation of Django as follows:
/django$ grep --exclude-dir=".svn" -n 'def db_type(' * -R
contrib/localflavor/us/models.py:8: def db_type(self):
contrib/localflavor/us/models.py:24: def db_type(self):
:
:
Ialso refer of Django documentation
Field.db_type(self, connection):
Returns the database column data type for the Field, taking into account the connection
object, and the settings associated with it.
And Then I could understand that to resolve this issue I have to inherited models.filed class and overwrite def db_type() function. And because I am using PostgreSQL in which to create 300 chars type field I need to return 'char(300)'. In my models.py I added:
class CharMaxlengthN(models.Field):
def db_type(self, connection):
return 'char(%d)' % self.max_length # because I am using postgresql
If you encounter similar problem please check your underline DB's manual that which type of column you need to create and return a string.
And changed the definition of new fields (that I need to add) read comments:
included_domains = CharMaxlengthN( # <--Notice change
"set of comma(,) seprated list of domains in target emails",
default="",
max_length=it_len.EMAIL_LEN*5)
excluded_domains = CharMaxlengthN( # <-- Notice change
"set of comma(,) seprated list of domains NOT in target emails",
default="",
max_length=it_len.EMAIL_LEN*5)
Then I executed same command that was failing previously:
t$ python manage.py evolve --hint --execute
You have requested a database evolution. This will alter tables
and data currently in the None database, and may result in
IRREVERSABLE DATA LOSS. Evolutions should be *thoroughly* reviewed
prior to execution.
Are you sure you want to execute the evolutions?
Type 'yes' to continue, or 'no' to cancel: yes
Evolution successful.
I also check my DB and tested my new added features It is now working perfectly, and no DB problem.
If you wants to create ENUM field read Specifying a mySQL ENUM in a Django model.
Edit: I realized instead of sub classing models.Field I should have inherit more specific subclass that is models.CharField.
Similarly I need to create Decimal DB fields so I added following class in model:
class DecimalField(models.DecimalField):
def db_type(self, connection):
d = {
'max_digits': self.max_digits,
'decimal_places': self.decimal_places,
}
return 'numeric(%(max_digits)s, %(decimal_places)s)' % d