Is it necessary to catch errors stemming from HTTP inputs? Is it ever a good idea to let it fail naturally (allow the exception to bubble up)?
I have a Django view for a server side interface for an AJAX call that looks something like this:
def some_view(request):
try:
some_int = int(request.POST.get('some_int')) # May raise ValueError or TypeError
except (ValueError, TypeError):
return HttpResponseBadRequest('some_int must be an int')
# ... Code that assumes some_int is an int
return HttpResponse('The normal response')
Is it ever acceptable in production code to have something like this?
def some_view(request):
some_int = int(request.POST.get('some_int')) # Ignore ValueError or TypeError raised
# ... Code that assumes some_int is an int
return HttpResponse('normal_response')
As I accept more parameters, I find that it is frustrating to maintain so many try/except blocks which are mostly the same and I end up with a ton of boiler plate code.
Of course I tried to refactor this into a separate function but since Django requires an HttpResponse to be returned, not raised as an exception, I can't plug it into a view without a try/except block. Also, conversions to int aren't the only thing I check... there are a lot of business logic sanity checks performed depending on the input as well. For example, I would validate that the JSON passed is of a specific format (i.e. array of objects of int array, etc.).
My views end up being 70+ lines of code just for sanity checks and a few lines of code that actually generate the response. Somehow I feel like there should be a more elegant way but I haven't found one so I'm considering forgoing all checks and just letting Django take care of it. Is this a bad idea?
I'm aware of the following potential problems if I don't catch the exceptions:
The same HTTP 500 is returned for all errors
If logging is enabled in production, it would probably log an error every time an invalid input occurs
Are there other problems I should be aware of? It just feels wrong not to catch exceptions from user inputs even though there's not much I can do about it in terms of recovery logic.
I think the best way to handle this is by writing your own middleware that catches the exceptions and turns them into your desired response.
That might look something like this:
# views.py
def some_view(request):
some_int = int(request.POST.get('some_int'))
# ... Code that assumes some_int is an int
return HttpResponse('normal_response')
# middleware.py
class MyValidationMiddleware(object):
def process_exception(self, request, e):
if isinstance(e, ValueError):
return HttpResponseBadRequest('Input did not validate')
else:
return None # let it bubble up
(Since middleware is site-wide, you might want to explicitly define your own input-validation exceptions to distinguish them from other sorts of exceptions that might occur.)
Alternatively, you could do the same sort of thing with per-view decorators.
I have many similar functions in my Django views.py that start off like this:
#login_required
def processMyObject(request, myObjectID, myObjectSlug=None):
logger.info("In processMyObject(myObjectID=%s, myObjectSlug=%s)" % (myObjectID, myObjectSlug))
try:
myObject = MyObject.objects.get(id=myObjectID)
except:
logger.error("Failed to get MyObject Object by ID")
raise "Failed to get MyObject Object by ID"
if myObjectSlug != None and myObject.slug != myObjectSlug:
logger.error("myObjectSlug '%s' doesn't match myObject #%s's slug" % (myObjectSlug, myObject.id))
raise "myObjectSlug '%s' doesn't match myObject #%s's slug" % (myObjectSlug, myObject.id)
Each of these functions has the same argument signature and contains the same chunk of code at the top, but then goes on to do implement some unique functionality. However this is common code in each one of them. It seems like a horrible violation of DRY for me to have typed the same code so many multiple times.
How can I use inheritance or some other technique to elegantly factor out this code so that it only appears once but is used in each of these view functions?
You can write decorator that recieves the MyObjectId and slug from view as parameter. logs the info line and raises an error if object is missing.
Just check information about function decorators and read django code for examples. For example, look up the code for the decorator you are already using (login_required) and look up the user_passes_test decorator in django.contrib.auth.decorators. That is probably the best real example for your case.
And then use the decorator in front of each view that needs it - just like you are using #login_required
I'm a little thrown by this. I have the following code, which works perfectly well:
urlresolvers.reverse('admin:cards_card_change', args=([92]))
To further my understanding, I wanted to try rewriting the line as:
urlresolvers.reverse('admin:cards_card_change', kwargs={'object_id':92})
as seemingly suggested by the documentation on reversing admin views (I'm using Django 1.4).
However, this doesn't seem to match anything at all. Why not? I tried looking in the django source code for answers, but couldn't find the view used for change, so links to the relevant module there would be really helpful as well!
The urlpattern of the change view is in admin/options.py:
url(r'^(.+)/$',
wrap(self.change_view),
name='%s_%s_change' % info),
You could find that it dispatches a request to the change_view method of the ModelAdmin instance. The change_view method also resides in admin/options.py:
def change_view(self, request, object_id, form_url='', extra_context=None):
...
It does accept a parameter object_id.
The reason of the missing match of reverse is that the urlpattern above does not accept named parameter, if you change it to something like
url(r'^(?P<object_id>.+)/$',
wrap(self.change_view),
name='%s_%s_change' % info),
The urlresolvers.reverse('admin:cards_card_change', kwargs={'object_id':92}) should work.
I've no idea whether it was intended to avoid some edge cases or it's just a bug and there's already a ticket fixing this. I'll check it later.
I think this is a python syntax issue.
The thing is that **kwargs cannot be taken as *args.
The function you're trying to call has a first argument (object_id) which is positional and cannot be used as a named argument. (Someone tell me if I'm wrong)
For you, object_id is a positional argument, and thus will be ignored if used in kwargs
If you want your view function to accept object_id also in kwargs, you have to change it with for example
my_id = kwargs.get('object_id', args[0])
I have a custom model manager that looks like this:
class MyManager(models.Manager)
def get_query_set(self):
'''Only get items that are 'approved' and have a `pub_date` that is in
the past. Ignore the rest.'''
queryset = super(MyManager, self).get_query_set()
queryset = queryset.filter(status__in=('a',))
return queryset.filter(pub_date__lte=datetime.utcnow())
And this works well enough; however, I have a problem using Django's generic.list_detail views object_detail and object_list: the queryset seems to be only loading once and, because of this, it isn't fetching the items it should be because, I assume, the utcnow() time has been called only once (when it first loaded).
I assume this is intentional and meant as a performance boost - however, it means that video's show up elsewhere on the site (in places I am not in a object_detail view) before they are available in an object_detail view (see urls.py below). This is leading to 404s ...
Any ideas ? Or do I have to write my own custom views to avoid this ?
Thanks!
urls.py
url(r'^video/(?P<object_id>\d+)$',
list_detail.object_detail,
{ 'queryset': Video.objects.all(), },
name='video_detail',
),
It is not a problem of cache: as you do it now, the queryset definition is evaluated once, while parsing urls, and then, it is never evaluated again.
Solution is actually pretty simple and described in the online documentation: Complex filtering with wrapper functions: just create a small custom view, that will simply call the generic view.
I am actually using a similar solution quite a lot and I feel it quite comfortable.
By the way, a small side note, for this case I would suggest not using a custom manager, and go back instead on a normal filtering.
Try correcting urls.py to:
url(r'^video/(?P<object_id>\d+)$',
list_detail.object_detail,
{ 'queryset': Video.objects.all, }, # here's the difference
name='video_detail',
)
Edit:
If this fail, try apply similar technique(passing callable instead of calling it) to filter():
return queryset.filter(pub_date__lte=datetime.utcnow)
I have an almost identical model Manager to thornomad, and the same problem with generic views.
I have to point out that neither of the suggestions above work:
doing Video.objects.all without parentheses gives an error
doing queryset.filter(pub_date__lte=datetime.utcnow), again without the parentheses, does not give an error but does not fix the problem
I have also tried another way, which is to use a lambda to return the queryset, eg:
qs = lambda *x: Video.objects.all()
url(r'^video/(?P<object_id>\d+)$',
list_detail.object_detail,
{ 'queryset': qs(), },
name='video_detail',
),
...it didn't work either and I can see now I must have been desperate to think it would :)
lazy_qs = lambda *x: lazy(Post.live_objects.all, QuerySet)
blog_posts = {
'queryset': lazy_qs(),
...doesn't work either (gives an error) as utils.functional.lazy doesn't know how to convert the result to a QuerySet properly, as best I can tell.
I think Roberto's answer of wrapping the generic view is the only one that will help.
The django docs should be amended to point out the limitations of the queryset used by generic views (currently the docs have a special note to tell you everything will be okay!)
I am currently defining regular expressions in order to capture parameters in a URL, as described in the tutorial. How do I access parameters from the URL as part the HttpRequest object?
My HttpRequest.GET currently returns an empty QueryDict object.
I'd like to learn how to do this without a library, so I can get to know Django better.
When a URL is like domain/search/?q=haha, you would use request.GET.get('q', '').
q is the parameter you want, and '' is the default value if q isn't found.
However, if you are instead just configuring your URLconf**, then your captures from the regex are passed to the function as arguments (or named arguments).
Such as:
(r'^user/(?P<username>\w{0,50})/$', views.profile_page,),
Then in your views.py you would have
def profile_page(request, username):
# Rest of the method
To clarify camflan's explanation, let's suppose you have
the rule url(regex=r'^user/(?P<username>\w{1,50})/$', view='views.profile_page')
an incoming request for http://domain/user/thaiyoshi/?message=Hi
The URL dispatcher rule will catch parts of the URL path (here "user/thaiyoshi/") and pass them to the view function along with the request object.
The query string (here message=Hi) is parsed and parameters are stored as a QueryDict in request.GET. No further matching or processing for HTTP GET parameters is done.
This view function would use both parts extracted from the URL path and a query parameter:
def profile_page(request, username=None):
user = User.objects.get(username=username)
message = request.GET.get('message')
As a side note, you'll find the request method (in this case "GET", and for submitted forms usually "POST") in request.method. In some cases, it's useful to check that it matches what you're expecting.
Update: When deciding whether to use the URL path or the query parameters for passing information, the following may help:
use the URL path for uniquely identifying resources, e.g. /blog/post/15/ (not /blog/posts/?id=15)
use query parameters for changing the way the resource is displayed, e.g. /blog/post/15/?show_comments=1 or /blog/posts/2008/?sort_by=date&direction=desc
to make human-friendly URLs, avoid using ID numbers and use e.g. dates, categories, and/or slugs: /blog/post/2008/09/30/django-urls/
Using GET
request.GET["id"]
Using POST
request.POST["id"]
Someone would wonder how to set path in file urls.py, such as
domain/search/?q=CA
so that we could invoke query.
The fact is that it is not necessary to set such a route in file urls.py. You need to set just the route in urls.py:
urlpatterns = [
path('domain/search/', views.CityListView.as_view()),
]
And when you input http://servername:port/domain/search/?q=CA. The query part '?q=CA' will be automatically reserved in the hash table which you can reference though
request.GET.get('q', None).
Here is an example (file views.py)
class CityListView(generics.ListAPIView):
serializer_class = CityNameSerializer
def get_queryset(self):
if self.request.method == 'GET':
queryset = City.objects.all()
state_name = self.request.GET.get('q', None)
if state_name is not None:
queryset = queryset.filter(state__name=state_name)
return queryset
In addition, when you write query string in the URL:
http://servername:port/domain/search/?q=CA
Do not wrap query string in quotes. For example,
http://servername:port/domain/search/?q="CA"
def some_view(request, *args, **kwargs):
if kwargs.get('q', None):
# Do something here ..
For situations where you only have the request object you can use request.parser_context['kwargs']['your_param']
You have two common ways to do that in case your URL looks like that:
https://domain/method/?a=x&b=y
Version 1:
If a specific key is mandatory you can use:
key_a = request.GET['a']
This will return a value of a if the key exists and an exception if not.
Version 2:
If your keys are optional:
request.GET.get('a')
You can try that without any argument and this will not crash.
So you can wrap it with try: except: and return HttpResponseBadRequest() in example.
This is a simple way to make your code less complex, without using special exceptions handling.
I would like to share a tip that may save you some time.
If you plan to use something like this in your urls.py file:
url(r'^(?P<username>\w+)/$', views.profile_page,),
Which basically means www.example.com/<username>. Be sure to place it at the end of your URL entries, because otherwise, it is prone to cause conflicts with the URL entries that follow below, i.e. accessing one of them will give you the nice error: User matching query does not exist.
I've just experienced it myself; hope it helps!
These queries are currently done in two ways. If you want to access the query parameters (GET) you can query the following:
http://myserver:port/resource/?status=1
request.query_params.get('status', None) => 1
If you want to access the parameters passed by POST, you need to access this way:
request.data.get('role', None)
Accessing the dictionary (QueryDict) with 'get()', you can set a default value. In the cases above, if 'status' or 'role' are not informed, the values are None.
If you don't know the name of params and want to work with them all, you can use request.GET.keys() or dict(request.GET) functions
This is not exactly what you asked for, but this snippet is helpful for managing query_strings in templates.
If you only have access to the view object, then you can get the parameters defined in the URL path this way:
view.kwargs.get('url_param')
If you only have access to the request object, use the following:
request.resolver_match.kwargs.get('url_param')
Tested on Django 3.
views.py
from rest_framework.response import Response
def update_product(request, pk):
return Response({"pk":pk})
pk means primary_key.
urls.py
from products.views import update_product
from django.urls import path
urlpatterns = [
...,
path('update/products/<int:pk>', update_product)
]
You might as well check request.META dictionary to access many useful things like
PATH_INFO, QUERY_STRING
# for example
request.META['QUERY_STRING']
# or to avoid any exceptions provide a fallback
request.META.get('QUERY_STRING', False)
you said that it returns empty query dict
I think you need to tune your url to accept required or optional args or kwargs
Django got you all the power you need with regrex like:
url(r'^project_config/(?P<product>\w+)/$', views.foo),
more about this at django-optional-url-parameters
This is another alternate solution that can be implemented:
In the URL configuration:
urlpatterns = [path('runreport/<str:queryparams>', views.get)]
In the views:
list2 = queryparams.split("&")
url parameters may be captured by request.query_params
It seems more recommended to use request.query_params. For example,
When a URL is like domain/search/?q=haha, you would use request.query_params.get('q', None)
https://www.django-rest-framework.org/api-guide/requests/
"request.query_params is a more correctly named synonym for request.GET.
For clarity inside your code, we recommend using request.query_params instead of the Django's standard request.GET. Doing so will help keep your codebase more correct and obvious - any HTTP method type may include query parameters, not just GET requests."