Django URLField likes to add a trailing slash (/) at the end of the user input, forcing all URLs to be stored with the extra character, this is wrong. How can I stop this behavior and save URLs as submitted by users?
Check to_python of URLField at https://github.com/django/django/blob/master/django/forms/fields.py.
You can see it has a line url_fields[2] = '/' almost at the end of method to_python. It appends a trailing slash / at the end of url. You can see the logic for doing this as a comment before this line.
This slash is necessary in case some query params are given.
If you want to avoid this behaviour, write you own field which extends from URLField and override to_python in your custom class.
I've been struggling with this as well, because it's causing a problem for certain urls. For example, http://www.nasa.gov/mission_pages/kepler/news/kepler-62-kepler-69.html/ fails, but it works without the slash.
To expand on akshar's answer, the method to do this is explained here. For example, defining this in my models.py file and setting url = NoSlashURLField() rather than models.URLField() in my model removes the slash:
try:
from urllib.parse import urlsplit, urlunsplit
except ImportError: # Python 2
from urlparse import urlsplit, urlunsplit
class NoSlashURLField(models.URLField):
description = "Remove the goddamn slash"
__metaclass__ = models.SubfieldBase
def __init__(self, *args, **kwargs):
super(NoSlashURLField, self).__init__(*args, **kwargs)
def to_python(self, value):
def split_url(url):
"""
Returns a list of url parts via ``urlparse.urlsplit`` (or raises a
``ValidationError`` exception for certain).
"""
try:
return list(urlsplit(url))
except ValueError:
# urlparse.urlsplit can raise a ValueError with some
# misformatted URLs.
raise ValidationError(self.error_messages['invalid'])
value = super(NoSlashURLField, self).to_python(value)
if value:
url_fields = split_url(value)
if not url_fields[0]:
# If no URL scheme given, assume http://
url_fields[0] = 'http'
if not url_fields[1]:
# Assume that if no domain is provided, that the path segment
# contains the domain.
url_fields[1] = url_fields[2]
url_fields[2] = ''
# Rebuild the url_fields list, since the domain segment may now
# contain the path too.
url_fields = split_url(urlunsplit(url_fields))
# if not url_fields[2]:
# # the path portion may need to be added before query params
# url_fields[2] = '/'
value = urlunsplit(url_fields)
return value
For those using the usual Django admin forms for their site, and also using South for DB migrations, you may want to use the following method instead of stonefury's. His method changes the model field, which confuses South unless you add some special code. The below method changes only the admin code, allowing South to remain blissfully unaware.
Define this class somewhere in your app:
class NoSlashURLFormField(forms.URLField):
def to_python(self, value):
def split_url(url):
"""
Returns a list of url parts via ``urlparse.urlsplit`` (or raises a
``ValidationError`` exception for certain).
"""
try:
return list(urlsplit(url))
except ValueError:
# urlparse.urlsplit can raise a ValueError with some
# misformatted URLs.
raise ValidationError(self.error_messages['invalid'])
if value:
url_fields = split_url(value)
if not url_fields[0]:
# If no URL scheme given, assume http://
url_fields[0] = 'http'
if not url_fields[1]:
# Assume that if no domain is provided, that the path segment
# contains the domain.
url_fields[1] = url_fields[2]
url_fields[2] = ''
# Rebuild the url_fields list, since the domain segment may now
# contain the path too.
url_fields = split_url(urlunsplit(url_fields))
value = urlunsplit(url_fields)
return value
Then edit your admin.py file as follows:
from your_app.path.to.noslash import NoSlashURLFormField
from django.contrib.admin.widgets import AdminURLFieldWidget
class MyModelAdmin(admin.ModelAdmin):
...
formfield_overrides = {
models.URLField: {
'form_class': NoSlashURLFormField,
# Need to specify the AdminURLFieldWidget here because it would
# otherwise get defaulted back to URLInput.
'widget': AdminURLFieldWidget,
}
}
Related
Get and post work pretty well for me .
but put and delete i use "U_repo_name" to look in table
i got the error message :
Page not found (404)
Request Method: PUT
Request URL: http://localhost:8000/gitapi/repo/authoo/
Using the URLconf defined in gitit.urls, Django tried these URL patterns, in this order:
admin/
gitapi/ [name='gitapi']
gitapi/ repo/
gitapi/ repo/str:U_repo_name
this is my model :
class UserRepos(models.Model):
U_repo_name =models.CharField( max_length=50)
this is my url :
urlpatterns = [
path('repo/',view=userreposapi),
path('repo/<str:U_repo_name>',view=userreposapi),
]
project urls :
urlpatterns = [
path('admin/', admin.site.urls),
path('gitapi/',include('gitapi.urls')),
]
this is my serializer :
class UserReposSerializer(serializers.ModelSerializer):
class Meta:
model = UserRepos
fields ='__all__'
and this is my views :
#csrf_exempt
def userreposapi(request,id=0):
if request.method=='GET':
userrepos = UserRepos.objects.all()
userreposserializer = UserReposSerializer(userrepos , many=True)
return JsonResponse(userreposserializer.data , safe=False)
elif request.method=='POST':
userrepos_data=JSONParser().parse(request)
userreposerializer = UserReposSerializer(data=userrepos_data)
if userreposerializer.is_valid():
userreposerializer.save()
return JsonResponse("added successfully!!",safe=False)
return JsonResponse("failed to add",safe=False)
elif request.method=='Put':
userrepos_data=JSONParser().parse(request)
userrepos = UserRepos.objects.get(U_repo_name=userrepos_data['U_repo_name'])
userreposserializer=UserReposSerializer(userrepos,data=userrepos_data)
if userreposserializer.is_valid():
userreposserializer.save()
return JsonResponse("updated successfully", safe=False)
return JsonResponse("failed to update",safe=False)
elif request.method=='DELETE':
userrepos = UserRepos.objects.all()
userrepos.delete()
return JsonResponse("deleted",safe=False)
Hi! I see a few issues with your code that might be causing this problem.
request.method is always full capitalized, so you should use the string 'PUT' instead of 'Put' on this line:
elif request.method=='Put':
You are adding a trailing slash to your route when trying to access it (/gitapi/repo/authoo/), but the pattern set in urlpatterns doesn't have this trailing slash:
path('repo/<str:U_repo_name>',view=userreposapi),
You can use re_path() instead of path() to set a regex URL pattern that works whether you add a trailing slash or not:
re_path(r'repo/(?P<U_repo_name>\w+)/?', view=userreposapi),
(?P<U_repo_name>\w+) creates a named group called U_repo_name that matches any word character ([a-zA-Z0-9_]) from one to more times. /? matches a single optional /.
Your route 'repo/<str:U_repo_name>' captures a string in the variable U_repo_name. This variable is provided to the view userreposapi() as a keyword argument, but the view currently only accepts the id kwarg. You should either add the U_repo_name kwarg as an optional argument or make the view accept other keyword arguments with **kwargs:
# Both are valid:
def userreposapi(request, id=0, U_repo_name=None):
...
def userreposapi(request, id=0, **kwargs):
...
Fixing these 3 issues should make the route work.
I think the trailing slash in the url is the problem. I should be not there.
http://localhost:8000/gitapi/repo/authoo # here I removed the trailing slash.
I'm trying to write a custom rest_framework Permission to prevent users from querying information that's not of the same company as them. Unfortunately, I can't seem to access any of the URL's parameters from within has_permission() or has_object_permissions().
Here's the beginning of my router:
# Create a basic router
router = routers.SimpleRouter()
# Establish some variables to assist with nested routes
root_elem = 'companies'
root_elem_id = '/(?P<company_id>[0-9]+)'
loca_elem = '/locations'
loca_elem_id = '/(?P<location_id>[0-9]+)'
# Companies will be the root from which all other relations branch
router.register(r'' + root_elem, views.CompanyViewSet)
router.register(r'' + root_elem + root_elem_id + loca_elem,
views.LocationViewSet)
Here's my custom permission:
# Only permit actions originating from location managers or company admins
class IsLocationManagerOrHigher(BasePermission):
# Checked when displaying lists of records
def has_permission(self, request, *args, **kwargs):
is_correct_level = False
# Admins can see every location if their location_id
# matches a location that's a child of the company
# specified in the URL
if request.employee.is_admin:
is_correct_level = True
return request.user and is_correct_level
# Checked when viewing specific records
def has_object_permission(self, request, view, obj):
is_correct_level = False
# Admins can see location details if their location's company_id
# matches a Location's company_id
if request.employee.is_admin:
is_correct_level = True
# Managers can see location details if it's their location
elif obj.id == request.employee.location_id and request.employee.is_manager:
is_correct_level = True
return request.user and is_correct_level
Right now checking request.employee.is_admin is only half of what I need - I also need to access the company_id from the URL and make sure it matches the admin's location's company_id:
# Pseudocode
try:
user_location = Location.objects.get(id=request.employee.location_id)
return user_location.company_id == kwargs['company_id']
except ObjectDoesNotExist:
pass
I've yet to figure out how to pass these parameters into the Permission so that it can perform this extra step. Or perhaps there's a better way of accomplishing what I'm trying to do?
If you can't pass them in directly (which would be preferable), they are available on the request object:
company_id = request.resolver_match.kwargs.get('company_id')
request.resolver_match.args and request.resolver_match.kwargs contain the positional/keyword arguments captured in your url.
As an alternative to the correct response posted by knbk, you can also get the URL parameters using the view object passed to has_permission method. Like this:
company_id = view.kwargs.get('company_id')
How can I remove whitespace, prior to validation of a URLField?
Using "clean_[fieldname]()" would seem to be the documented way from https://docs.djangoproject.com/en/dev/ref/forms/validation/ , but it does not work for the URLField. I've reduced it to a basic test case which can be run in the django shell:
class XXXTestModel(models.Model):
url = models.URLField('URL',null=True,blank=True)
name = models.CharField(max_length=200)
class XXXTestForm(ModelForm):
def clean_url(self):
return self.cleaned_data['url'].strip()
def clean_name(self):
return self.cleaned_data['name'].strip()
class Meta:
model = XXXTestModel
fields = (
'url',
)
Tested from the Django shell with:
>>> django.VERSION
(1, 5, 1, 'final', 0)
>>> from xxx import XXXTestForm,XXXTestModel
>>> data = dict(url=' http://www.example.com/ ',name=' example ')
>>> f=XXXTestForm(data)
>>> f.is_valid();f.errors
False
{'url': [u'Enter a valid URL.']}
>>> f.cleaned_data
{'name': example'}
There are a number of close dupes of this question on stack overflow, but none of the answers guide toward a solution.
The issue here is how the django.forms.URLField works.
django.forms.Field.clean is defined as:
def clean(self, value):
"""
Validates the given value and returns its "cleaned" value as an
appropriate Python object.
Raises ValidationError for any errors.
"""
value = self.to_python(value)
self.validate(value)
self.run_validators(value)
return value
Note that to_python is performed before any validation. This is the issue here - django.forms.URLField can't understand the value you're giving it, so the value it produces fails the set of validators already defined as part of django.forms.URLField (namely, django.core.validators.URLValidator).
The reason it fails is django tries to "normalize" the URL. This includes things such as adding "http://" where needed. When given your example url, " http://www.example.com ", django uses urlparse.urlsplit to get it "parts" of the url. The leading space, however, messes it up and the entire value becomes part of the path. As such, django finds no scheme, and reconstitutes the URL as "http:// http://www.example.com ". This is then given to django.core.validators.URLValidator, which obviously fails.
To avoid this, we'll need to define our own URLField for our form
from django import forms
class StrippedURLField(forms.URLField):
def to_python(self, value):
return super(StrippedURLField, self).to_python(value and value.strip())
Using this ensures the process will all go as expected, and we wont need a clean_url method. (note: you should use clean_* where possible, but here it is not)
class XXXTestForm(forms.ModelForm):
url = StrippedURLField(blank=True, null=True)
I'm using the Django Feeds Framework and it's really nice, very intuitive and easy to use. But, I think there is a problem when creating links to feeds in HTML.
For example:
<link rel="alternate" type="application/rss+xml" title="{{ feed_title }}" href="{{ url_of_feed }}" />
Link's HREF attribute can be easily found out, just use reverse()
But, what about the TITLE attribute? Where the template engine should look for this? Even more, what if the feed is build up dinamically and the title depends on parameters (like this)?
I can't come up with a solution that "seems" DRY to me... All that I can come up with is using context processors o template tags, but it gets messy when the context procesor/template tag has to find parameters to construct the Feed class, and writing this I realize I don't even know how to create a Feed instance myself within the view.
If I put all this logic in the view, it would not be just one view. Also, the value for TITLE would be in the view AND in the feed.
Just a guess (as I have not used feeds yet in my django app), but you could add a special template_context for your feed with your feed object and use it in your base.html.
I'm not fully satisfied with this solution, it may break feeds using Request and depends on a magic method. There it goes:
#coding:utf-8
# Author: Armando Pérez Marqués <mandx#rbol.org>
# Purpose: Django TemplateTag to output feed links in templates in a DRY way
# Created: 05/07/2010
import re
from django import template
from django.conf import settings
from django.contrib.syndication.views import Feed
from django.core.urlresolvers import reverse, resolve, NoReverseMatch
from django.template import Node
from django.template import TemplateSyntaxError
from django.utils.encoding import smart_str
from django.utils.html import escape as html_escape
from django.utils.safestring import mark_safe
register = template.Library()
kwarg_re = re.compile(r"(?:(\w+)=)?(.+)")
class FeedInfoNode(Node):
def __init__(self, view_name, args, kwargs, asvar):
self.view_name = view_name
self.args = args
self.kwargs = kwargs
self.asvar = asvar
def render(self, context):
args = [arg.resolve(context) for arg in self.args]
kwargs = dict([(smart_str(k,'ascii'), v.resolve(context))
for k, v in self.kwargs.items()])
# Try to look up the URL twice: once given the view name, and again
# relative to what we guess is the "main" app. If they both fail,
# re-raise the NoReverseMatch unless we're using the
# {% feed_info ... as var %} construct in which cause return nothing.
url = ''
try:
url = reverse(self.view_name, args=args, kwargs=kwargs, current_app=context.current_app)
except NoReverseMatch, e:
if settings.SETTINGS_MODULE:
project_name = settings.SETTINGS_MODULE.split('.')[0]
try:
url = reverse(project_name + '.' + self.view_name,
args=args, kwargs=kwargs, current_app=context.current_app)
except NoReverseMatch:
if self.asvar is None:
# Re-raise the original exception, not the one with
# the path relative to the project. This makes a
# better error message.
raise e
else:
if self.asvar is None:
raise e
if 'request' in context:
request = context['request']
else:
request = None
feed_instance, feed_args, feed_kwargs = resolve(url)
if not isinstance(feed_instance, Feed):
raise NoReverseMatch, \
'feed_info can only reverse class-based feeds'
feed_obj = feed_instance.get_object(request, *feed_args, **feed_kwargs)
feed_data = {
'url': url,
'obj': feed_instance,
'args': feed_args,
'kwargs': feed_kwargs,
#'title': html_escape(feed_instance.__get_dynamic_attr('title', obj)),
'title': html_escape(
feed_instance._Feed__get_dynamic_attr('title', feed_obj)
),
'type': feed_instance.feed_type.mime_type,
}
if self.asvar:
context[self.asvar] = feed_data
return ''
else:
return mark_safe(
'<link rel="alternate" type="%(type)s" title="%(title)s" href="%(url)s" />' \
% feed_data
)
def feed_info(parser, token):
"""
Returns an mapping containing populated info about the reversed feed
Works exactly as the url tag, but the mapping is not returned, instead
a variable is always set in the context.
This is a way to define links that aren't tied to a particular URL
configuration::
{% feed_info path.to.some_feed_view_class arg1 arg2 as feed_info_var %}
or
{% feed_info path.to.some_feed_view_class name1=value1 name2=value2 as feed_info_var %}
"""
bits = token.split_contents()
if len(bits) < 2:
raise TemplateSyntaxError("'%s' takes at least one argument"
" (path to a feed view)" % bits[0])
viewname = bits[1]
args = []
kwargs = {}
asvar = None
bits = bits[2:]
if len(bits) >= 2 and bits[-2] == 'as':
asvar = bits[-1]
bits = bits[:-2]
# Backwards compatibility: check for the old comma separated format
# {% url urlname arg1,arg2 %}
# Initial check - that the first space separated bit has a comma in it
if bits and ',' in bits[0]:
check_old_format = True
# In order to *really* be old format, there must be a comma
# in *every* space separated bit, except the last.
for bit in bits[1:-1]:
if ',' not in bit:
# No comma in this bit. Either the comma we found
# in bit 1 was a false positive (e.g., comma in a string),
# or there is a syntax problem with missing commas
check_old_format = False
break
else:
# No comma found - must be new format.
check_old_format = False
if check_old_format:
# Confirm that this is old format by trying to parse the first
# argument. An exception will be raised if the comma is
# unexpected (i.e. outside of a static string).
match = kwarg_re.match(bits[0])
if match:
value = match.groups()[1]
try:
parser.compile_filter(value)
except TemplateSyntaxError:
bits = ''.join(bits).split(',')
# Now all the bits are parsed into new format,
# process them as template vars
if len(bits):
for bit in bits:
match = kwarg_re.match(bit)
if not match:
raise TemplateSyntaxError("Malformed arguments to url tag")
name, value = match.groups()
if name:
kwargs[name] = parser.compile_filter(value)
else:
args.append(parser.compile_filter(value))
return FeedInfoNode(viewname, args, kwargs, asvar)
feed_info = register.tag(feed_info)
I'm starting with the code of the {% url %} template tag, and then, after obtaining the feed's URL, use resolve() to get the Feed subclass instance, and then get the needed attributes.
Caveats
Requires Django 1.2 Class Feeds, don't know exactly how to do this with the old way of feeds.
If the feed class uses the request object, the request context processor must be configured, since None is passed if it isn't present in the context.
There's an oddity with Feed.__get_dynamic_attr(). The Feed subclass instance doesn't have this method; instead, it appears with another name. Don't know how to figure the name out at runtime...
I've got a view in Django that uses memcached to cache data for the more highly trafficked views that rely on a relatively static set of data. The key word is relatively: I need invalidate the memcached key for that particular URL's data when it's changed in the database. To be as clear as possible, here's the meat an' potatoes of the view (Person is a model, cache is django.core.cache.cache):
def person_detail(request, slug):
if request.is_ajax():
cache_key = "%s_ABOUT_%s" % settings.SITE_PREFIX, slug
# Check the cache to see if we've already got this result made.
json_dict = cache.get(cache_key)
# Was it a cache hit?
if json_dict is None:
# That's a negative Ghost Rider
person = get_object_or_404(Person, display = True, slug = slug)
json_dict = {
'name' : person.name,
'bio' : person.bio_html,
'image' : person.image.extra_thumbnails['large'].absolute_url,
}
cache.set(cache_key)
# json_dict will now exist, whether it's from the cache or not
response = HttpResponse()
response['Content-Type'] = 'text/javascript'
response.write(simpljson.dumps(json_dict)) # Make sure it's all properly formatted for JS by using simplejson
return response
else:
# This is where the fully templated response is generated
What I want to do is get at that cache_key variable in it's "unformatted" form, but I'm not sure how to do this--if it can be done at all.
Just in case there's already something to do this, here's what I want to do with it (this is from the Person model's hypothetical save method)
def save(self):
# If this is an update, the key will be cached, otherwise it won't, let's see if we can't find me
try:
old_self = Person.objects.get(pk=self.id)
cache_key = # Voodoo magic to get that variable
old_key = cache_key.format(settings.SITE_PREFIX, old_self.slug) # Generate the key currently cached
cache.delete(old_key) # Hit it with both barrels of rock salt
# Turns out this doesn't already exist, let's make that first request even faster by making this cache right now
except DoesNotExist:
# I haven't gotten to this yet.
super(Person, self).save()
I'm thinking about making a view class for this sorta stuff, and having functions in it like remove_cache or generate_cache since I do this sorta stuff a lot. Would that be a better idea? If so, how would I call the views in the URLconf if they're in a class?
URLConf should point to any callable. There's no strict requirement to make it point to function exactly. You could implement base class with your cache methods then extend it:
class RealView(BaseViewWithCacheMethods):
def __call__(self, request):
if request.is_ajax():
return self.ajax_view()
return self.html_view()
URLConf definition would be something like that:
from django.conf.urls.defaults import *
from views import RealView
urlpattrens = patterns('',
(r'^$', RealView()),
)