Thanks for taking the time to look at my question!
I am trying to run the following raw query in django on googleappgine:
models.MyModel.objects.raw(
'select count(1) from myapp_mymodel where %s >= my_column',
[4])
I keep getting the error:
NotImplementedError: Cursors are not supported.
However, cursors do seem to be implemented in djangoappengine.
Has anyone else run into this issue??
Traceback:
File "..../django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "..../promotions/views.py", line 562, in task_alert_stats
auto_join_sql, auto_join_params)[0]
File "..../django/db/models/query.py", line 1381, in __getitem__
return list(self)[k]
File "..../django/db/models/query.py", line 1326, in __iter__
query = iter(self.query)
File "..../django/db/models/sql/query.py", line 67, in __iter__
self._execute_query()
File "..../django/db/models/sql/query.py", line 81, in _execute_query
self.cursor.execute(self.sql, self.params)
File "..../django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "..../djangotoolbox/db/base.py", line 620, in __getattribute__
raise NotImplementedError("Cursors are not supported.")
To anyone who comes across this:
I ended up doing a simple filter and then going through the results to further filter them.
Related
Stack:
Ubuntu (20.04 LTS)
Nginx
Postgresql (v13.3)
An AWS load balancer sends traffic to the Ubuntu instance(k8s cluster), which is handled by Nginx, which forwards on to Django (4.0.3) running in gunicorn (19.9.0). Django connects to the database using psycopg2 (2.8.6).
The issue I have is that the database connection seems to shut down randomly.
Django reports errors like this:
`InterfaceError: connection already closed
File "django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
File "django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "contextlib.py", line 79, in inner
return func(*args, **kwds)
File "django/views/generic/base.py", line 84, in view
return self.dispatch(request, *args, **kwargs)
File "django/utils/decorators.py", line 46, in _wrapper
return bound_method(*args, **kwargs)
File "django/views/decorators/cache.py", line 62, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "django/views/generic/base.py", line 119, in dispatch
return handler(request, *args, **kwargs)
File "django/views/generic/detail.py", line 108, in get
self.object = self.get_object()
File "django/views/generic/detail.py", line 53, in get_object
obj = queryset.get()
File "django/db/models/query.py", line 492, in get
num = len(clone)
File "django/db/models/query.py", line 302, in __len__
self._fetch_all()
File "django/db/models/query.py", line 1507, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "django/db/models/query.py", line 57, in __iter__
results = compiler.execute_sql(
File "django/db/models/sql/compiler.py", line 1359, in execute_sql
cursor = self.connection.cursor()
File "django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "django/db/backends/base/base.py", line 284, in cursor
return self._cursor()
File "django/db/backends/base/base.py", line 262, in _cursor
return self._prepare_cursor(self.create_cursor(name))
File "django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "django/db/backends/base/base.py", line 262, in _cursor
return self._prepare_cursor(self.create_cursor(name))
File "django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "django/db/backends/postgresql/base.py", line 256, in create_cursor
cursor = self.connection.cursor()`
Postgresql does not report any major errors: I can only assume that the connection was closed correctly and Django was not expecting it.
This happens quite rarely, but enough to be a concern: once every 3 days.
Any suggestions of how to investigate this further would be gratefully accepted. Thanks in advance!
I'm using django-service-objects and Django-Q for a distributed data mining code. I ran into a error "django.db.utils.InterfaceError: connection already closed" when I was running Django-Q cluster async_task(..., sync=True) to debug and attempted to query the Django PostgreSQL 13 DB using Model.object.get(pk=...).
I resolved the issue by setting db_transaction = False within the Service class.
class dataMining(Service):
db_transaction = False
https://django-service-objects.readthedocs.io/en/stable/pages/usage.html#database-transactions
The effect of db_transaction=True (enabled by default) is to "...process method on services ... inside a transaction". Meaning changes to the Django PostgreSQL DB are not actually saved until the def process(self) method of the Service class completes successfully. https://django-service-objects.readthedocs.io/en/stable/pages/usage.html#service
I suspect the self.object = self.get_object() call in your code sample is experiencing the same problem of being 'locked out' during a process method within a DB transaction.
I have a problem with my django server time, it is 2 hours late.
I tried to find a solution but whatever i found (as How to change my django server time, or Django documentation) is not what I think I need.
Right now my computer's time is 23:14:37
When i write in bash date +%H:%M:%S I will get:
(python_env) bash-3.2$ date +%H:%M:%S
23:17:03
So I don't think this is my console problem.
But when I am running my server here is what I get:
(python_env) bash-3.2$ python3 manage.py runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
April 04, 2020 - 21:18:47
Django version 2.2.4, using settings 'myVote365.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
The time is now 2 hours late.
I have red that it could be because of wrong timezone, but I have my code similar to How to change my django server time
LANGUAGE_CODE = 'pl'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
What's more it worked perfectly fine just before I change my os to Windows, and to OS X again
Edit:
I changed USE_TZ to False, ran and closed the server, and again to True.
And now although I still have a server 2 hour late. But from error:
Internal Server Error: /panel/
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 189, in _get_session
return self._session_cache
AttributeError: 'SessionStore' object has no attribute '_session_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 383, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: django_session
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/icookiez/Documents/praca/dla taty/myVote365/myVote365/audytor/views.py", line 26, in panel
if 'auditor' in request.session and request.session['auditor']['logged'] is True:
File "/usr/local/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 51, in __contains__
return key in self._session
File "/usr/local/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 194, in _get_session
self._session_cache = self.load()
File "/usr/local/lib/python3.7/site-packages/django/contrib/sessions/backends/db.py", line 43, in load
s = self._get_session_from_db()
File "/usr/local/lib/python3.7/site-packages/django/contrib/sessions/backends/db.py", line 34, in _get_session_from_db
expire_date__gt=timezone.now()
File "/usr/local/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 402, in get
num = len(clone)
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 256, in __len__
self._fetch_all()
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 55, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/sqlite3/base.py", line 383, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: django_session
I got a working site.
First step
I changed USE_TZ to False, ran and closed the server, and again to True. This allowed my side to load, but still had the wrong time.
Second step
I changed TIME_ZONE not to ‘PL’ as #jatrp5 suggested, but to 'Europe/Warsaw' as I read on this question. Using the zone name form Wikipedia. And now I have time I was wishing for.
To match your bash time you should change the time zone to TIME_ZONE = 'PL'. The universal time zone (UTC) is two hours behind the Polish.
I have a django rest Backend app, and i use swagger to look and document my apis to the FE.
This worked fine, but I made some changes and now I get this error:
Internal Server Error: /
Traceback (most recent call last):
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/rest_framework/views.py", line 497, in dispatch
response = self.handle_exception(exc)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/rest_framework/views.py", line 457, in handle_exception
self.raise_uncaught_exception(exc)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/rest_framework/views.py", line 468, in raise_uncaught_exception
raise exc
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/rest_framework/views.py", line 494, in dispatch
response = handler(request, *args, **kwargs)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/rest_framework_swagger/views.py", line 32, in get
schema = generator.get_schema(request=request)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/rest_framework/schemas/coreapi.py", line 153, in get_schema
links = self.get_links(None if public else request)
File "/home/notsoshabby/.local/share/virtualenvs/panda_pitch-UBt5SNMA/lib/python3.7/site-packages/rest_framework/schemas/coreapi.py", line 140, in get_links
link = view.schema.get_link(path, method, base_url=self.url)
AttributeError: 'AutoSchema' object has no attribute 'get_link'
HTTP GET / 500 [0.15, 127.0.0.1:44214]
/home/notsoshabby/Desktop/panda_pitch/django_project/settings.py
This error is not very clear as the AutoSchema is not a part of my code and the traceback is not showing me where in My code the problem is.
I made too many changes to go one by one and check which one caused that.
Anyone experienced this issue before? Any ideas on how to debug to find which change causes this issue?
I ran into the same issue, the fix is described here: https://www.django-rest-framework.org/community/3.10-announcement/
To summarize, Django Rest Framework 3.10 (released a few days ago) deprecated the CoreAPI based schema generation, and introduced the OpenAPI schema generation in its place. Currently to continue to use django-rest-swagger as is you need to re-enable the CoreAPI schema generation by adding the following config to the settings file:
REST_FRAMEWORK = { ... 'DEFAULT_SCHEMA_CLASS': 'rest_framework.schemas.coreapi.AutoSchema' }
I've started getting this error on the production environment this morning, after getting Django-storages, Boto, and Django-compressor working to put static files on S3 yesterday, though I don't know if that is related...
OperationalError: could not fork new process for connection: Cannot allocate memory
could not fork new process for connection: Cannot allocate memory
could not fork new process for connection: Cannot allocate memory
Stacktrace (most recent call last):
File "django/core/handlers/base.py", line 89, in get_response
response = middleware_method(request)
File "reversion/middleware.py", line 17, in process_request
if hasattr(request, "user") and request.user.is_authenticated():
File "django/utils/functional.py", line 184, in inner
self._setup()
File "django/utils/functional.py", line 248, in _setup
self._wrapped = self._setupfunc()
File "django/contrib/auth/middleware.py", line 16, in <lambda>
request.user = SimpleLazyObject(lambda: get_user(request))
File "django/contrib/auth/middleware.py", line 8, in get_user
request._cached_user = auth.get_user(request)
File "django/contrib/auth/__init__.py", line 98, in get_user
user_id = request.session[SESSION_KEY]
File "django/contrib/sessions/backends/base.py", line 39, in __getitem__
return self._session[key]
File "django/contrib/sessions/backends/base.py", line 165, in _get_session
self._session_cache = self.load()
File "django/contrib/sessions/backends/db.py", line 19, in load
expire_date__gt=timezone.now()
File "django/db/models/manager.py", line 131, in get
return self.get_query_set().get(*args, **kwargs)
File "django/db/models/query.py", line 361, in get
num = len(clone)
File "django/db/models/query.py", line 85, in __len__
self._result_cache = list(self.iterator())
File "django/db/models/query.py", line 291, in iterator
for row in compiler.results_iter():
File "django/db/models/sql/compiler.py", line 763, in results_iter
for rows in self.execute_sql(MULTI):
File "django/db/models/sql/compiler.py", line 817, in execute_sql
cursor = self.connection.cursor()
File "django/db/backends/__init__.py", line 308, in cursor
cursor = util.CursorWrapper(self._cursor(), self)
File "django/db/backends/postgresql_psycopg2/base.py", line 177, in _cursor
self.connection = Database.connect(**conn_params)
File "psycopg2/__init__.py", line 178, in connect
return _connect(dsn, connection_factory=connection_factory, async=async)
I am deploying the site on Heroku. It works for a bit after I restart the application, but stops working again after a few minutes.
Any ideas as to what might be causing this?
I encountered the same problem trying to set up a simple django web application with a postgresql database on heroku and managed to solve it.
I don't fully understand the error but the fix is fairly simple: when you are passing python lists created by queries to your database, you need to limit the size of the list.
So for example if you are passing as context the following list:
set_list = userSetTable.objects.all()
return render(request, 'fc/user.html', {'set_list': set_list,})
That will cause an error because set_list might be really big. You need to specify a maximum size:
set_list = userSetTable.objects.all()[0:20]
So in a real-world application, you might want to display the list as page results or whatever... you get the point.
add_to_builtins('gravatar.gravatar')
is being used in my settings.py file, and it causes a crash on page load with this error:
Traceback (most recent call last):
File "C:\development\python\Lib\site-packages\django\core\servers\basehttp.py", line 283, in run
self.result = application(self.environ, self.start_response)
File "C:\development\python\Lib\site-packages\django\contrib\staticfiles\handlers.py", line 68, in call
return self.application(environ, start_response)
File "C:\development\python\Lib\site-packages\django\core\handlers\wsgi.py", line 272, in call
response = self.get_response(request)
File "C:\development\python\Lib\site-packages\django\core\handlers\base.py", line 169, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "C:\development\python\Lib\site-packages\django\core\handlers\base.py", line 203, in handle_uncaught_exception
return debug.technical_500_response(request, *exc_info)
File "C:\development\python\Lib\site-packages\django\views\debug.py", line 59, in technical_500_response
html = reporter.get_traceback_html()
File "C:\development\python\Lib\site-packages\django\views\debug.py", line 128, in get_traceback_html
t = Template(TECHNICAL_500_TEMPLATE, name='Technical 500 template')
File "C:\development\python\Lib\site-packages\django\template\base.py", line 108, in init
self.nodelist = compile_string(template_string, origin)
File "C:\development\python\Lib\site-packages\django\template\base.py", line 135, in compile_string
parser = parser_class(lexer.tokenize())
File "C:\development\python\Lib\site-packages\django\template\debug.py", line 34, in init
super(DebugParser, self).init(lexer)
File "C:\development\python\Lib\site-packages\django\template\base.py", line 208, in init
print lib.tags
AttributeError: 'NoneType' object has no attribute 'tags'
If i don't have any issue with using gravatar if i don't use the add_to_builtins functionality - that is, with plain old load tags in my views. I'm just using it a fair bit, so thought i might make use of the add_to_builtins functionality.
From the looks of things, the imported version crashes because the import_library(module) for gravatar.gravatar returns None (i know this because i printed out that line, and it returns None) - what is wrong with what i've done?
oh gravatar.
Ok, so i was all wrong in how to import this using the add_to_builtins - what i needed to do was:
add_to_builtins('gravatar.templatetags.gravatar')
i'm unsure as to why, but that works a treat. Now i don't need to have any messy "load" statements, and i can happily call
{% gravatar story.user 40 %}
from any of my templates. yay!