I am developing my first django website.
I have written code in my view layer (the handlers that return an HttpResponse object to the view template (hope I am using the correct terminology).
In any case, I want to put print statements in my views.py file, so that I can debug it. However, it looks like stdout has been redirect to another stream, so I am not seeing anything printed out on my console (or even the browser).
What is the recommended way (best practice) for debugging django view layer scripts?
there are more advanced ways of doing it, but i find dropping
import pdb
pdb.set_trace()
does the job.
Use the Python logging module. Then use the Django debug toolbar, which will catch and display all the things you send to the log.
I'd upvote dysmsyd, but I don't have the reputation.
pdb is good because it lets you step thru your procedure and follow the control flow.
If you are using the django runserver, you can print to stdout or stderr.
If you are using the mod_wsgi, you can print to stderr.
The pprint module is also useful:
import sys
from pprint import pprint
def myview(request):
pprint (request, sys.stderr)
Try django-sentry. Especially if your project is in production stage.
Configure django debug toolbar: pip install django-debug-toolbar and follow the instructions to configure it in: https://github.com/django-debug-toolbar/django-debug-toolbar
import logging
Use the logging to debug: logging.debug('My DEBUG message')
Here is how it works on my class view:
from django.views.generic import TemplateView
import logging
class ProfileView(TemplateView):
template_name = 'profile.html'
def get(self, request, *args, **kwargs):
logging.debug(kwargs)
return render(request, self.template_name)
Related
I want to use scrapy spider in Django views and I tried using CrawlRunner and CrawlProcess but there are problems, views are synced and further crawler does not return a response directly
I tried a few ways:
# Core imports.
from scrapy.crawler import CrawlerProcess
from scrapy.utils.project import get_project_settings
# Third-party imports.
from rest_framework.views import APIView
from rest_framework.response import Response
# Local imports.
from scrapy_project.spiders.google import GoogleSpider
class ForFunAPIView(APIView):
def get(self, *args, **kwargs):
process = CrawlerProcess(get_project_settings())
process.crawl(GoogleSpider)
process.start()
return Response('ok')
is there any solution to handle that and run spider directly in other scripts or projects without using DjangoItem pipeline?
you didn't really specify what the problems are, however, I guess the problem is that you need to return the Response immediately, and leave the heavy call aka function to run in the background, you can alter your code as following, to use the Threading module
from threading import Thread
class ForFunAPIView(APIView):
def get(self, *args, **kwargs):
process = CrawlerProcess(get_project_settings())
process.crawl(GoogleSpider)
thread = Thread(target=process.start)
thread.start()
return Response('ok')
after a while of searching for this topic, I found a good explanation here: Building a RESTful Flask API for Scrapy
I use the python-telegram-bot 20.0a2 library and Django 4.1
The bot runs by main.py script:
if __name__ == "__main__":
asyncio.run(main())
Inside of the main script I also run uvicorn in the same ascynhronous context as Application instance
# Run application and webserver together
async with application_tg:
await application_tg.start()
await server.serve() # uvicorn
await application_tg.stop()
What is the problem?
I use webhook for my bot
Django's url.py calls async view but the view can't get initalized Application instance of the bot.
so the question is:
How can to rearrange a scheme of interaction between python-telegram-bot 20 and Django 4.1 in a way that I can access Application instance from a Django hook?
Addition:
It's easy to achieve by using other frameworks such as starlette as it mentioned on the official wiki page of PTB library: https://docs.python-telegram-bot.org/en/v20.0a2/examples.customwebhookbot.html
My main script:
https://gist.github.com/SergSm/6843fadf505b826f83a10bf7eebc3fa0
my view:
import json
from django.views import View
from django.http import JsonResponse, HttpResponse
from django.views.decorators.csrf import csrf_exempt
from telegram import Update
from bot.tgbot.main import application_tg
async def telegram_handle(request):
if request.method == 'POST':
await application_tg.update_queue.put(
Update.de_json(data=json.loads(request.body), bot=application_tg.bot)
)
return JsonResponse({"ok": "POST processed"})
else:
return JsonResponse({"ok": "GET processed"})
UPDATE 1
I was desperate to make it run this way.
I tried to use contextvars module and read a lot of asyncio related stuff/
In the end I made an awful assumption that if I put my python-telegram-bot code into the Django view async function it's gonna work. But it does work!
And now I will try to wrap it using middleware to make my code cleaner
UPDATE 2
If you want to use Django orm with sync functions you need to use #sync_to_async(thread_sensitive=False)
the thread_sensitive=False parameter is important in this case otherwise you will never get the result of awaitables
I'm aware of pdb built-in Python library for debugging Python programs and scripts. However, you can't really use it for debugging Django apps (I get errors when I try to use it in views.py). Are there any tools that I can use when Django's traceback isn't helpful ?
EDIT:
from .forms import TestCaseForm, TestCaseSuiteForm
from .models import TestCase, TestSuite
from django.contrib.auth.forms import UserCreationForm
from django.views.generic import FormView, ListView
from django.contrib.auth.models import User
from django.utils.decorators import method_decorator
from django.contrib.auth.decorators import login_required
from django.contrib.auth import logout
import pdb
class ListAllTestSuites(ListView):
template_name = 'list.html'
context_object_name = 'username'
def get_queryset(self):
pdb.set_trace() # <-- setting a trace here to diagnose the code below
queryset = {'test_suites': TestSuite.objects.filter(user=self.request.user),
'username': self.request.user}
return queryset
you forgot the exact error message and full traceback, and, more importantly, you forgot to explain how you executed this code to get this result...
But anyway: from the error message, you're obviously trying to execute your view file as a plain python script (cf the reference to __main__). This cannot work. A view is a module, not a script, and, moreover, any module dependending on Django needs some setup done to be imported (which is why we use the django shell - ./manage.py shell - instead of the regular Python one).
For most case, you can just launch the django shell, import your module and use pdb.runcall() to execute some function / method under the debugger (no need to put a breakpoint then, but that's still possible).
Now views require a HTTPRequest as first argument which make them a bit more cumbersome to debug that way (well, there is django.tests.RequestFactory but still...), so your best bet here is to set your breakpoint, launch your devserver (or restart it - if it didn't already did, as it should), point your browser to the view's url, and then you should see the debugger's prompt in your devserver's terminal.
In a Django project I'm using selenium to run some UI tests, using a LiveServerTestCase.
One of my test cases is failing, and when using the Firefox driver I can see a page throwing "Server Error (500)", which means DEBUG is set to False which is not the case when I run the local development server.
How is the test server being launched? Why is not using my settings which define DEBUG = True?
Other URLs (such as the homepage URL) return fine, so the server is working. But I just don't get why it's not showing debug information, and which settings it's using.
My test case for reference:
class LoginTest(LiveServerTestCase):
#classmethod
def setUpClass(cls):
try:
from selenium.webdriver import PhantomJS
cls.selenium = PhantomJS()
except:
from selenium.webdriver.firefox.webdriver import WebDriver
cls.selenium = WebDriver()
super(LoginTest, cls).setUpClass()
#classmethod
def tearDownClass(cls):
cls.selenium.quit()
super(LoginTest, cls).tearDownClass()
def test_fb_login(self):
self.selenium.get('%s%s' % (self.live_server_url, reverse('account_login')))
# TEST SERVER RETURNS 500 ON THIS URL WITH NO DEBUG INFO
According to Testing Django Application - Django Documentation:
Regardless of the value of the DEBUG setting in your configuration
file, all Django tests run with DEBUG=False. This is to ensure that
the observed output of your code matches what will be seen in a
production setting.
It should still be possible to override this using:
with self.settings(DEBUG=True):
...
Although I wouldn't recommend it, it can still be useful from time to time. (Thomas Orozco's comment)
You can also change your settings in TestCase setUp() method.
from django.conf import settings
class MyTest(LiveServerTestCase):
def setUp(self):
# Change settings here
settings.DEBUG = True
# ...
I ran into the same issue and it is possible to override settings with a decorator.
based on your example you would import override_settings and place the decorator above the class:
from django.conf import settings
from django.test import override_settings
#override_settings(DEBUG=True)
class LoginTest(LiveServerTestCase):
...
details in django docs
I'm currently running some Django tests and it looks that DEBUG=False by default. Is there a way to run a specific test where I can set DEBUG=True at the command line or in code?
For a specific test inside a test case, you can use the override_settings decorator:
from django.test.utils import override_settings
from django.conf import settings
class TestSomething(TestCase):
#override_settings(DEBUG=True)
def test_debug(self):
assert settings.DEBUG
Starting with Django 1.11 you can use --debug-mode to set the DEBUG setting to True prior to running tests.
The accepted answer didn't work for me. I use Selenium for testing, and setting #override_settings(DEBUG=True) makes the test browser always display 404 error on every page. And DEBUG=False does not show exception tracebacks. So I found a workaround.
The idea is to emulate DEBUG=True behaviour, using custom 500 handler and built-in django 500 error handler.
Add this to myapp.views:
import sys
from django import http
from django.views.debug import ExceptionReporter
def show_server_error(request):
"""
500 error handler to show Django default 500 template
with nice error information and traceback.
Useful in testing, if you can't set DEBUG=True.
Templates: `500.html`
Context: sys.exc_info() results
"""
exc_type, exc_value, exc_traceback = sys.exc_info()
error = ExceptionReporter(request, exc_type, exc_value, exc_traceback)
return http.HttpResponseServerError(error.get_traceback_html())
urls.py:
from django.conf import settings
if settings.TESTING_MODE:
# enable this handler only for testing,
# so that if DEBUG=False and we're not testing,
# the default handler is used
handler500 = 'myapp.views.show_server_error'
settings.py:
# detect testing mode
import sys
TESTING_MODE = 'test' in sys.argv
Now if any of your Selenium tests encounters 500 error, you'll see a nice error page with traceback and everything. If you run a normal non-testing environment, default 500 handler is used.
Inspired by:
Where in django is the default 500 traceback rendered so that I can use it to create my own logs?
django - how to detect test environment
Okay let's say you want to write tests for error testcase for which the urls are :-
urls.py
if settings.DEBUG:
urlpatterns += [
url(r'^404/$', page_not_found_view),
url(r'^500/$', my_custom_error_view),
url(r'^400/$', bad_request_view),
url(r'^403/$', permission_denied_view),
]
test_urls.py:-
from django.conf import settings
class ErroCodeUrl(TestCase):
def setUp(self):
settings.DEBUG = True
def test_400_error(self):
response = self.client.get('/400/')
self.assertEqual(response.status_code, 500)
Hope you got some idea!
Nothing worked for me except https://stackoverflow.com/a/1118271/5750078
Use Python 3.7
breakpoint()
method.
Works fine on pycharm
You can't see the results of DEBUG=True when running a unit test. The pages don't display anywhere. No browser.
Changing DEBUG has no effect, since the web pages (with the debugging output) are not visible anywhere.
If you want to see a debugging web page related to a failing unit test, then do this.
Drop your development database.
Rerun syncdb to build an empty development database.
Run the various loaddata scripts to rebuild the fixtures for that test in your development database.
Run the server and browse the page.
Now you can see the debug output.