django/postgres: Transaction managed block ended with pending COMMIT/ROLLBACK - django

I want to run manage.py sqldiff myapp (command from django extension), but I get the following error:
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django_extensions/management/commands/sqldiff.py", line 596, in handle
sqldiff_instance.find_differences()
File "/usr/local/lib/python2.7/dist-packages/django/db/transaction.py", line 222, in inner
self.__exit__(None, None, None)
File "/usr/local/lib/python2.7/dist-packages/django/db/transaction.py", line 207, in __exit__
self.exiting(exc_value, self.using)
File "/usr/local/lib/python2.7/dist-packages/django/db/transaction.py", line 302, in exiting
leave_transaction_management(using=using)
File "/usr/local/lib/python2.7/dist-packages/django/db/transaction.py", line 56, in leave_transaction_management
connection.leave_transaction_management()
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/__init__.py", line 115, in leave_transaction_management
raise TransactionManagementError("Transaction managed block ended with "
django.db.transaction.TransactionManagementError: Transaction managed block ended with pending COMMIT/ROLLBACK
manage.py runserver, shell, shell_plus all work fine, but the sqldiff command chokes. I have tried:
restart postgres server
manually connect to postgres from shell via psycopg2, ran
connection.rollback(), and connection.commit()
but the error persists.
Any ideas on what can be done are welcome!
Cheers,
Hoff

I updated an issue with sqldiff causing similar problems. Your best bet is probably to obtain django-extensions from the github repository and do a bit of investigation. Edit sqldiff.py and comment out the transaction code decorating the find_differences method:
# #transaction.commit_manually
def find_differences(self):
...
# transaction.rollback() # reset transaction
...
# transaction.commit()
Now your real problem should be revealed, no longer masked by the transaction exception.

Related

DeserializationError when adding fixture on server (loaddata)

I am trying to add my data to database in my server. I use exactly the same JSON file in my local machine and it works. But when I do the same in server it gives me Deserialization Error.
The JSON file is so big that I can't show it here but I am sure there is no typo.
I did all migrations and database works - I can add object from admin dashboard. How can I solve this error?
The error I get:
Traceback (most recent call last):
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/serializers/json.py", line 69, in Deserializer
objects = json.loads(stream_or_string)
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 22, in <module>
main()
File "manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/base.py", line 330, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/base.py", line 371, in execute
output = self.handle(*args, **options)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 72, in handle
self.loaddata(fixture_labels)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 114, in loaddata
self.load_label(fixture_label)
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/management/commands/loaddata.py", line 172, in load_label
for obj in objects:
File "/home/testuser/myprojectt/myvenv/lib/python3.8/site-packages/django/core/serializers/json.py", line 74, in Deserializer
raise DeserializationError() from exc
django.core.serializers.base.DeserializationError: Problem installing fixture '/home/testuser/myprojectt/myprojectt/articles/fixtures/article.json':
Checked the JSON again and again but it still working on local but the same error on server.
I am using Ubuntu on the server. And didn't have git-lfs installed. To solve this issue:
Install it with sudo apt-get install git-lfs.
Then run git lfs pull on project directory.
And then I managed to work with the JSON data as I wanted.

SQLite objects created in a thread can only be used in that same thread with Django 2.2.2 and ipdb

I updated my Django project to version 2.2.2 and now when I call to ipdb for debugging, the server tell me that error. If I go back to Django 2.2.1, the error disappears.
System check identified no issues (0 silenced).
June 06, 2019 - 08:19:36
Django version 2.2.2, using settings 'laserapp.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
> /home/lynde/public_html/laserapp/gestion/views.py(192)get_context_data()
191 import ipdb; ipdb.set_trace()
--> 192 context = super(ResumeOrderCodeView, self).get_context_data(**kwargs)
193 try:
Traceback (most recent call last):
File "manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 60, in execute
super().execute(*args, **options)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
output = self.handle(*args, **options)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 95, in handle
self.run(**options)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 102, in run
autoreload.run_with_reloader(self.inner_run, **options)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 585, in run_with_reloader
start_django(reloader, main_func, *args, **kwargs)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 570, in start_django
reloader.run(django_main_thread)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 288, in run
self.run_loop()
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 294, in run_loop
next(ticker)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 334, in tick
for filepath, mtime in self.snapshot_files():
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 350, in snapshot_files
for file in self.watched_files():
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 249, in watched_files
yield from iter_all_python_module_files()
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 103, in iter_all_python_module_files
return iter_modules_and_files(modules, frozenset(_error_files))
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/django/utils/autoreload.py", line 120, in iter_modules_and_files
sys_file_paths.append(module.__file__)
AttributeError: module '__main__' has no attribute '__file__'
If you suspect this is an IPython bug, please report it at:
https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev#python.org
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.
Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
%config Application.verbose_crash=True
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/IPython/core/history.py", line 780, in writeout_cache
self._writeout_input_cache(conn)
File "/home/lynde/public_html/env/laser/lib/python3.6/site-packages/IPython/core/history.py", line 764, in _writeout_input_cache
(self.session_number,)+line)
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140030717908736 and this is thread id 140031011022656.
Can somebody know what's the problem there? I'm using django runserver in local machine, with celery and postgresql database.
Thanks!! :)
Try using the --reload flag to run runserver.
./manage.py runserver --noreload
It stops the server reloading on code changes but I think avoids the issue you're having.

Why am I getting errors when trying to invoke Django Python shell?

My current version of python is 2.7.10 and version of Django is 1.9.1, path.py is 8.1.2 but every time when I'm trying to invoke python shell by command:
$ python manage.py shell I'm getting a lot of errors and the ending of them looks like:
Error in sys.excepthook:
Traceback (most recent call last):
File "W:\SVN\vendors\python\win32\lib\site-packages\IPython\CrashHandler.py", line 157, in __call__
report.write(self.make_report(traceback))
File "W:\SVN\vendors\python\win32\lib\site-packages\IPython\CrashHandler.py", line 215, in make_report
rpt_add('BZR revision : %s \n\n' % Release.revision)
AttributeError: 'module' object has no attribute 'revision'
Original exception was:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "W:\SVN\vendors\python\win32\lib\site-packages\django\core\management\__init__.py", line 353, in execute_from_command_line
utility.execute()
File "W:\SVN\vendors\python\win32\lib\site-packages\django\core\management\__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "W:\SVN\vendors\python\win32\lib\site-packages\django\core\management\base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "W:\SVN\vendors\python\win32\lib\site-packages\django\core\management\base.py", line 399, in execute
output = self.handle(*args, **options)
File "W:\SVN\vendors\python\win32\lib\site-packages\django\core\management\commands\shell.py", line 69, in handle
self.run_shell(shell=options['interface'])
File "W:\SVN\vendors\python\win32\lib\site-packages\django\core\management\commands\shell.py", line 58, in run_shell
return getattr(self, shell)()
File "W:\SVN\vendors\python\win32\lib\site-packages\django\core\management\commands\shell.py", line 41, in ipython
ip()
File "W:\SVN\vendors\python\win32\lib\site-packages\django\core\management\commands\shell.py", line 22, in _ipython_pre_011
shell = IPShell(argv=[])
File "W:\SVN\vendors\python\win32\lib\site-packages\IPython\Shell.py", line 73, in __init__
debug=debug,shell_class=shell_class)
File "W:\SVN\vendors\python\win32\lib\site-packages\IPython\ipmaker.py", line 521, in make_IPython
IP.pre_config_initialization()
File "W:\SVN\vendors\python\win32\lib\site-packages\IPython\iplib.py", line 835, in pre_config_initialization
self.db = pickleshare.PickleShareDB(rc.ipythondir + "/db")
File "W:\SVN\vendors\python\win32\lib\site-packages\IPython\Extensions\pickleshare.py", line 53, in __init__
if not self.root.isdir():
TypeError: _isdir() takes exactly 1 argument (0 given)
How can I overcome the problem?
From doing a quick search, it looks like this error happens when both Django and IPython are installed, since apparently Django will attempt to use the IPython shell.
The first resource with someone who had a similar issue:
run python manage.py shell occurs errors
And in the link above, one of the comments on the OP's question suggested looking at this separate thread for answers:
https://superuser.com/questions/318655/error-running-ipython3-on-xp-typeerror-isdir-takes-exactly-1-argument-0-gi
The winning answer says the following:
Finally decided to give this another stab, and managed to get it to
work. The solution is a two-line change in the
ipython-0.11-py3.2.egg\IPython\utils\pickleshare.py file, line 52:
Before:
if not self.root.isdir():
self.root.makedirs()
After:
if not os.path.isdir(self.root):
os.makedirs(self.root)

"'django_mongodb_engine' isn't an available database backend" when pushing to Heroku

I have a Django app that successfully ran on Heroku and uses mongodb engine. All of a sudden, I can not push it anymore. It does not seem to recognize the django_mongodb_engine module anymore. I even reverted all changes to the version that last ran, but it's still the same. I suspect a version conflict or so, but can not figure out how to fix it.
This is the error I'm getting:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 429, in execute_from_command_line
utility.execute()
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 351, in handle
return self.handle_noargs(**options)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/commands/shell.py", line 46, in handle_noargs
from django.db.models.loading import get_models
File "/app/.heroku/python/lib/python2.7/site-packages/django/db/__init__.py", line 78, in <module>
connection = connections[DEFAULT_DB_ALIAS]
File "/app/.heroku/python/lib/python2.7/site-packages/django/db/utils.py", line 93, in __getitem__
backend = load_backend(db['ENGINE'])
File "/app/.heroku/python/lib/python2.7/site-packages/django/db/utils.py", line 51, in load_backend
raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: 'django_mongodb_engine' isn't an available database backend.
Try using django.db.backends.XXX, where XXX is one of:
'dummy', 'mysql', 'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3'
Error was: No module named utils
And these are the important parts from the requirements.txt:
hg+https://bitbucket.org/wkornewald/django-nonrel
git+https://github.com/django-nonrel/mongodb-engine
hg+https://bitbucket.org/wkornewald/djangotoolbox
pymongo==2.2.1
pytz==2012d
I'm especially confused because I reverted all changes. Exactly this code with the same configuration ran perfectly (and still does on my production server), just pushing does not work.
Thanks
Simon
Of course, just when I asked on SO I figure out the answer :-)
I don't know why exactly, but defining the exact revision that i want to check out solved the problem.
So the requirements.txt looks now like this:
-e hg+http://bitbucket.org/wkornewald/django-nonrel#be48c152abc6b15e45155e2bbcfd69c665ccb536#egg=Django-dev
-e git+https://github.com/django-nonrel/mongodb-engine#52257b5d90dcfaa564e708264a9cfe591d301fe6#egg=django_mongodb_engine-dev
-e hg+https://bitbucket.org/wkornewald/djangotoolbox#a8cdf61ba9c0cdc7cbbbd37d693e9a222d9a8e5f#egg=djangotoolbox-dev
pymongo==2.2.1
pytz==2012d

Heroku Django Postgres syncdb error

Heroko Config
DATABASE_URL: postgres://xxxxxxxx:xxxxxxxxxx#ec2-54-243-215-140.compute-1.amazonaws.com:5432/xxxxxxxxx
HEROKU_POSTGRESQL_GOLD_URL: postgres://yyyyyyyyyyy:yyyyyyyyyyyyy#ec2-54-243-243-20.compute-1.amazonaws.com:5432/yyyyyyyyyy
HEROKU_POSTGRESQL_YELLOW_URL: postgres://zzzzzzzzzz:zzzzzzzzzzzz#ec2-54-243-215-140.compute-1.amazonaws.com:5432/zzzzzzzzz
settings.py
DATABASES = {'default': dj_database_url.config(default='postgres://xxxxxxxx:xxxxxxxxxx#ec2-54-243-215-140.compute-1.amazonaws.com:5432/xxxxxxxxx'),}
heroku run python manage.py syncdb
Running `python manage.py syncdb` attached to terminal... up, run.4673
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
utility.execute()
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
self.execute(*args, **options.__dict__)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
output = self.handle(*args, **options)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle
return self.handle_noargs(**options)
File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 60, in handle_noargs
tables = connection.introspection.table_names()
File "/app/.heroku/python/lib/python2.7/site-packages/django/db/backends/__init__.py", line 910, in table_names
return self.get_table_list(cursor)
File "/app/.heroku/python/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/introspection.py", line 33, in get_table_list
AND pg_catalog.pg_table_is_visible(c.oid)""")
File "/app/.heroku/python/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 52, in execute
return self.cursor.execute(query, args)
django.db.utils.DatabaseError: current transaction is aborted, commands ignored until end of transaction block
Many do not speak English and therefore can not explanation. What's the problem?
The error you have posted merely means that there was a previous error in the transaction and the transaction has already aborted. You cannot fix this problem by troubleshooting this error. You must find out what your real problem is and try to fix it.
To do this, check your PostgreSQL logs, and search backwards from this error until you come to a different error. That will be the error that terminated the transaction and you can work on fixing that one instead. This error however is meaningless from a troubleshooting perspective. Find your real problem and fix it.