How do you put a file in a fixture in Django? - django

I can easily fill the field of a FileField or ImageField in a Django fixture with a file name, but that file doesn't exist and when I try to test my application it fails because that file doesn't exist.
How do I correctly populate a FileField or Imagefield in a Django fixture so that the file itself is available too?

I'm afraid the short answer is that you can't do this using the FileField or ImageField classes; they just store a file path and have no real concept of the file's actual data. The long answer, however, is that anything is possible if you leverage the Django API for writing your own custom model fields.
At a minimum, you'll want to implement the value_to_string method to convert the data for serialization (there's an example in the django docs at the link above). Note that the examples at the URL link above also include mention of subclassing FileField and ImageField, which is helpful for your situation!
You'll also have to decide if the data should therefore be stored in the database, or on the file system. If the former, you will have to implement your custom class as a Blob field, including customization for every DB you wish to support; you'll also have to provide some support for how the data should be returned to the user out of the database when the HTML requests a .gif/.jpg/.png/.whatever url. If the latter, which is the smarter way to go IMHO, you'll have to implement methods for serializing, de-serializing binary data to the filesystem. Either way, if you implement these as subclasses of FileField and ImageField, you should still be able to use the Admin tools and other modules that expect such django features.
If and only if you elect to use the more involved blob approach, here's a snippet of code from an old project of mind (back when I was learning Django) that handles blob for MySQL and PostgreSQL; you'll probably be able to find a number of improvements as I haven't touched it since :-) It does not handle serialization, though, so you'll have to add that using the method above.
from django.db import models
from django.conf import settings
class BlobValueWrapper(object):
"""Wrap the blob value so that we can override the unicode method.
After the query succeeds, Django attempts to record the last query
executed, and at that point it attempts to force the query string
to unicode. This does not work for binary data and generates an
uncaught exception.
"""
def __init__(self, val):
self.val = val
def __str__(self):
return 'blobdata'
def __unicode__(self):
return u'blobdata'
class BlobField(models.Field):
"""A field for persisting binary data in databases that we support."""
__metaclass__ = models.SubfieldBase
def db_type(self):
if settings.DATABASE_ENGINE == 'mysql':
return 'LONGBLOB'
elif settings.DATABASE_ENGINE == 'postgresql_psycopg2':
return 'bytea'
else:
raise NotImplementedError
def to_python(self, value):
if settings.DATABASE_ENGINE == 'postgresql_psycopg2':
if value is None:
return value
return str(value)
else:
return value
def get_db_prep_save(self, value):
if value is None:
return None
if settings.DATABASE_ENGINE =='postgresql_psycopg2':
return psycopg2.Binary(value)
else:
return BlobValueWrapper(value)

There's no way to "include" the files in the serialized fixture. If creating a test fixture, you just need to do it yourself; make sure that some test files actually exist in locations referenced by the FileField/ImageField values. The values of those fields are paths relative to MEDIA_ROOT: if you need to, you can set MEDIA_ROOT in your test setUp() method in a custom test_settings.py to ensure that your test files are found wherever you put them.
EDIT: If you want to do it in your setUp() method, you can also monkeypatch default_storage directly:
from django.core.files.storage import default_storage
class MyTest(TestCase):
def setUp(self):
self._old_default_storage_location = default_storage.location
default_storage.location = '/some/other/place'
def tearDown(self):
default_storage.location = self._old_default_storage_location
That seems to work. default_storage is a documented public API, so this should be reliable.

Related

DRF serializer remove imagefield absolute url

In DRF, now if i save the image the serializer is giving me the image with absolute path. But in my scenario, i don't want absolute path. I just need a image path which is there in DB. In my scenario, i'm hard-coding the image path in json. So i can't use the absolute path. Can anyone suggest some ideas to implement it.
You can use SerializerMethodField. You can return the value as it is stored in the database by doing the following:
class YourSerializer(serializers.ModelSerializer):
image = serializers.SerializerMethodField()
def get_image(self, obj):
return obj.image.file.name
docs: https://www.django-rest-framework.org/api-guide/fields/#serializermethodfield
Create custom FileField will solve the issue
class CustomFileField(serializers.FileField):
def to_representation(self, value):
if not value:
return None
if not getattr(value, 'path', None):
# If the file has not been saved it may not have a path.
return None
return value.path
class MyFooSerializer(serializers.ModelSerializer):
my_file_field = CustomFileField()
...
# other code
Above answers are working for list and retrieve method, but i need to add for post and retrieve instead of querying and serializing it again. Below method will work.
Normally for ImageField and FileField will return name but in DRF it's been converted to url for usefulness. So u just need to change there that's it.
In your serailizer, just specify below like that:
Field_name = serializers.ImageField(use_url=False)
So this line will convert absolute url to name as response after Save and Update.
For further reference refer this link

How to safely access request object in Django models

What I am trying to do:
I am trying to access request object in my django models so that I can get the currently logged in user with request.user.
What I have tried:
I found a hack on this site. But someone in the comments pointed out not to do it when in production.
I also tried to override model's __init__ method just like mentioned in this post. But I got an AttributeError: 'RelatedManager' object has no attribute 'request'
Models.py:
class TestManager(models.Manager):
def user_test(self):
return self.filter(user=self.request.user, viewed=False)
class Test(models.Model):
def __init__(self, *args, **kwargs):
self.request = kwargs.pop('request', None)
super(Test, self).__init__(*args, **kwargs)
user = models.ForeignKey(User, related_name='test')
viewed = models.BooleanField(default=False)
objects = TestManager()
I trying to access request object in my Django models so that I can get the currently logged in user with request.user.
Well a problem is that models are not per se used in the context of a request. One for example frequently defines custom commands to do bookkeeping, or one can define an API where for example the user is not present. The idea of the Django approach is that models should not be request-aware. Models define the "business logic" layer: the models define entities and how they interact. By not respecting these layers, one makes the application vulnerable for a lot of problems.
The blog you refer to aims to create what they call a global state (which is a severe anti-patten): you save the request in the middleware when the view makes a call, such that you can then fetch that object in the model layer. There are some problems with this approach: first of all, like already said, not all use cases are views, and thus not all use cases pass through the middleware. It is thus possible that the attribute does not exist when fetching it.
Furthermore it is not guaranteed that the request object is indeed the request object of the view. It is for example possible that we use the model layer with a command that thus does not pass through the middleware, in which case we should use the previous view request (so potentially with a different user). If the server processes multiple requests concurrently, it is also possible that a view will see a request that arrived a few nanoseconds later, and thus again take the wrong user. It is also possible that the authentication middleware is conditional, and thus that not all requests have a user attribute. In short there are more than enough scenario's where this can fail, and the results can be severe: people seeing, editing, or deleting data that they do not "own" (have no permission to view, edit, or delete).
You thus will need to pass the request, or user object to the user_test method. For example with:
from django.http import HttpRequest
class TestManager(models.Manager):
def user_test(self, request_or_user):
if isinstance(request_or_user, HttpRequest):
return self.filter(user=request_or_user.user, viewed=False)
else:
return self.filter(user=request_or_user, viewed=False)
one thus has to pass the request object from the view to the function. Even this is not really pure. A real pure approach would only accept a user object:
class TestManager(models.Manager):
def user_test(self, user):
return self.filter(user=user, viewed=False)
So in a view one can use this as:
def some_view(request):
some_tests = Test.objects.user_test(request.user)
# ...
# return Http response
For example if we want to render a template with this queryset, we can pass it like:
def some_view(request):
some_tests = Test.objects.user_test(request.user)
# ...
return render(request, 'my_template.html', {'some_tests': some_tests})

How do I test modelform validation when the model includes a FileField?

I took an age looking for the answer to this so now I've figured it out I'm putting it here for future users. I'm using Python 2.7 and Django 1.5 but this answer should also work for Python 3+ and Django 1.6+
The problem
I have a model that includes a FileField. I'm using a ModelForm to let users add instances of this model. I want all my app's code to be under testing, including any forms. Passing file paths, file objects or byte strings in a data dictionary does not result in a valid form.
The question
How do I test validation on a ModelForm including a FileField?
You can infer the answer from Django's documentation. The trick is that you don't pass the FileField data in the same dictionary as the rest of your form data. Instead, your test_forms.py should look something like:
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase
class CorrespondenceFormTest(TestCase):
def setUp(self):
...
def test_correspondence_form_with_good_data(self):
data = {
'direction': 'OU',
'incoming_mail_type': '',
'outgoing_mail_type': 'CL',
}
file_data = {
'correspondence_file': SimpleUploadedFile('test.txt', 'Hi!')
}
form = CorrespondenceForm(data, file_data)
self.assertTrue(form.is_valid())
This will pass. You can now write additional tests for the things you want to fail with confidence that they fail for the right reason.

Getting Google App Engine blob info in Django view

This is a follow up question for Django on Google App Engine: cannot upload images
I got part of the upload of images to GAE Blobstore working. Here's what I did:
In models.py I created a model PhotoFeature:
class PhotoFeature(models.Model):
property = models.ForeignKey(
Property,
related_name = "photo_features"
)
caption = models.CharField(
max_length = 100
)
blob_key = models.CharField(
max_length = 100
)
In admin.py I created an admin entry with an override for the rendering of the change_form to allow for insert of the correct action to the Blobstore upload url:
class PhotoFeatureAdmin(admin.ModelAdmin):
list_display = ("property", "caption")
form = PhotoFeatureForm
def render_change_form(self, request, context, *args, **kwargs):
from google.appengine.ext import blobstore
if kwargs.has_key("add"):
context['blobstore_url'] = blobstore.create_upload_url('/admin/add-photo-feature')
else:
context['blobstore_url'] = blobstore.create_upload_url('/admin/update-photo-feature')
return super(PhotoFeatureAdmin, self).render_change_form(request, context, args, kwargs)
As I use standard Django, I want to use the Django views to process the result once GAE has updated the BlobStore in stead of BlobstoreUploadHandler. I created the following views (as per the render_change_form method) and updated urls.py:
def add_photo_feature(request):
def update_photo_feature(request):
This all works nicely but once I get into the view method I'm a bit lost. How do I get the Blob key from the request object so I can store it with PhotoFeature? I use standard Django, not Django non-rel. I found this related question but it appears not to contain a solution. I also inspected the request object which gets passed into the view but could not find anything relating to the blob key.
EDIT:
The Django request object contains a FILES dictionary which will give me an instance of InMemoryUploadedFile. I presume that somehow I should be able to retrieve the blob key from that...
EDIT 2:
Just to be clear: the uploaded photo appears in the Blobstore; that part works. It's just getting the key back from the Blobstore that's missing here.
EDIT 3:
As per Daniel's suggestion I added storage.py from the djangoappengine project which contains the suggested upload handler and added it to my SETTINGS.PY. This results in the following exception when trying to upload:
'BlobstoreFileUploadHandler' object has no attribute 'content_type_extra'
This is really tricky to fix. The best solution I have found is to use the file upload handler from the djangoappengine project (which is associated with django-nonrel, but does not depend on it). That should handle the required logic to put the blob key into request.FILES, as you'd expect in Django.
Edit
I'd forgotten that django-nonrel uses a patched version of Django, and one of the patches is here to add the content-type-extra field. You can replicate the functionality by subclassing the upload handler as follows:
from djangoappengine import storage
class BlobstoreFileUploadHandler(storage.BlobstoreFileUploadHandler):
"""Handler that adds blob key info to the file object."""
def new_file(self, field_name, *args, **kwargs):
# We need to re-process the POST data to get the blobkey info.
meta = self.request.META
meta['wsgi.input'].seek(0)
fields = cgi.FieldStorage(meta['wsgi.input'], environ=meta)
if field_name in fields:
current_field = fields[field_name]
self.content_type_extra = current_field.type_options
super(BlobstoreFileUploadHandler, self).new_file(field_name,
*args, **kwargs)
and reference this subclass in your settings.py rather than the original.

Use proxy class instead of User for request.user

I have a meta class for the Django User model that I use to add extra methods (overly simplified version):
# project.models.pie_lover.py
from django.contrib.auth.models import User
class PieLover(User):
class Meta:
app_label = "core"
proxy = True
def likes_pie(self):
return True
In my view, I wish to get the logged in PieLover and see if he likes pie (silly thing to do because a PieLover always loves pie, but in a real world situation this may not be the case). My problem lies in the way Django logs in the users, I use the built-in login(request) function for this and as a result the object stored in request.user is a User object, not a PieLover.
# project.views.like_pie.py
from ..models.pie_lover import PieLover
def index(request):
pie_lover = request.user
if pie_lover.likes_pie():
print "Try pie, try!"
else:
print "BLASPHEMER!"
If I try to do this Django tells me that the User object has no method likes_pie which is to be expected as request.user is not a PieLover instance.
As a quick workaround I just get the PieLover which has the same ID as the User but that means an extra DB hit.
How can I make Django use PieLover by default in the request?
I was thinking that instead of making another database query to get the proper PieLover object to create a new PieLover object and pass request.user to it at initialization but I don't know what the implications of this are.
After poking around I found, what seems to me, the easiest and non-hackish way to access the PieLover methods for a given User instance. I have added this to a custom middleware:
from models.pie_lover import PieLover
class PieLoverMiddleware(object):
def process_request(self, request):
request.pie_lover = PieLover(request.user)
How can I make Django use PieLover by default in the request?
You don't.
Read this before you do anything else: https://docs.djangoproject.com/en/1.3/topics/auth/#storing-additional-information-about-users
Your "extension" to user should be a separate model with all of your extension methods in that separate model.
You can then navigate from User (in the request) to your extension trivially using the get_profile() method already provided.
from django.contrib.auth import models as auth_models
def _my_func(self):
return True
auth_models.User.my_func = _my_func