I am new to Django, and I am looking for the best approach to the following problem.
I have an application that is producing two reports. One is a JSON blob so I store it in psql with data model that uses JSONField.
The second report is a .html file.
The .html file will be generated multiple times a day so the first thing that came to mind was storing it in the db.
I need to be able to pull the report as well so it can be displayed to the user in the UI.
I created a test data model using TextField:
class TestResultsHTML(models.Model):
name = models.CharField(max_length=200)
report = models.TextField()
It makes it into the Db no problem, however when I attempt to retrieve it I can't seem to get the actual report:
In [3]: html_results = TestResultsHTML.objects.get(id=4)
In [4]: html_results.name
Out[4]: 'b0f5c336-867a-44a3-a5ef-6297bf6042cf'
In [5]: html_results.report
Out[5]: "<_io.TextIOWrapper name='report.html' mode='r' encoding='UTF-8'>"
I was expected that .report would return the actual contents of the file. The file itself is 1800+ lines.
Is this a good approach or is this not the intended use of TextField?
A TextField doesn't store the file, Django has a FileField for this (see here). This saves the file to a certain location/folder and the object saved in the DB essentially stores that location which you can then access later. Something like this:
class TestResultsHTML(models.Model):
name = models.CharField(max_length=200)
file_loc = models.FileField(upload_to=upload_location)
Then open the file at a later date with something like this:
with open(html_results.file_loc, 'w'):
Related
I'm a big fan of Django-parler, but I've run into a problem when storing a translated model in two different databases.
My model is:
class InstrumentFamily(TranslatableModel):
primary_key = True
translations = TranslatedFields(
label=CharNullField(_('Label'), max_length=100, unique=False, null=True,)
I have 2 database aliases 'default' and 'test' and my database router directs my model to 'test'.
I insert models in both databases by doing this:
fam = InstrumentFamily(code=TEST_CODE)
with switch_language(fam, 'en'):
fam.label = "test_family_test EN"
with switch_language(fam, 'fr'):
fam.label = "test_family_test FR"
fam.save()
which stores the object and its translations in database 'test', or by doing this:
fam = InstrumentFamily(code="TEST_FAM")
with switch_language(fam, 'en'):
fam.label = "test_family_default_EN"
with switch_language(fam, 'fr'):
fam.label = "test_family_default_FR"
fam.save(using='default')
which saves the object and its translations to database 'default'. So far, so good.
But when accessing the object previously saved in 'default' by doing this (after properly clearing all caches to force a database read):
fam = InstrumentFamily.objects.using('default').get(code=TEST_CODE)
print(f" label: {fam.label}")
django-parler properly retrieves the object from database 'default', but looks for the translation from database 'test' ! (SQL trace below, see the very end of each line):
SELECT "orchestra_instrumentfamily"."id", "orchestra_instrumentfamily"."code" FROM "orchestra_instrumentfamily" WHERE "orchestra_instrumentfamily"."code" = 'TEST_FAM' LIMIT 21; args=('TEST_FAM',); alias=default
SELECT "orchestra_instrumentfamily_translation"."id", "orchestra_instrumentfamily_translation"."language_code", "orchestra_instrumentfamily_translation"."label", "orchestra_instrumentfamily_translation"."master_id" FROM "orchestra_instrumentfamily_translation" WHERE ("orchestra_instrumentfamily_translation"."master_id" = 34 AND "orchestra_instrumentfamily_translation"."language_code" = 'en') LIMIT 21; args=(34, 'en'); alias=test
I'm obviously missing something big... What am I supposed to do to have the 'using("default")' information propagated to the second query? I couldn't find anything in the documentation about storing TranslatableModels in more than one database. Am I trying to achieve something parler does not support?
Thanks in advance for enlightening me!
This looks like a bug in django-parler. It doesn't pass the using information to its internal queries that retrieve translation model data. You can file a bit in the GitHub repository so this can be addressed.
A workaround would be to implement a database-router that enforces using a particular database for this model.
How can I direct the destination of the output file to my db?
My models.py is structured like so:
class Model(models.Model):
char = models.CharField(max_length=50, null=False, blank=False)
file = models.FileField(upload_to=upload_location, null=True, blank=True)
I have the user enter a value for 'char', and then the value of 'char' is printed on to a file. The process of successfully printing onto the file is working, however, the file is outputting to my source directory.
My goal is to have the output file 'pdf01.pdf' output to my db and be represented as 'file' so that the admin can read it.
Much of the information in the Dango docs has been focussed on directing the path of objects imported by the user directly, not on files that have been created internally. I have been reading mostly from these docs:
Models-Fields
Models
File response objects
Outputting PDFs
I have seen it recommend to write to a buffer, not a file, then save the buffer contents to my db however I haven't been able to find many examples of how to do that relevant to my situation online.
Perhaps there is a relevant gap in my knowledge regarding buffers and BytesIO? Here is the function I have been using to alter the pdf, I have been using BytesIO to temporarily store files throughout the process but have not been able to figure out how to use it to direct the output anywhere specific.
can = canvas.Canvas(BytesIO(), pagesize=letter)
can.drawString(10, 10, char)
can.save()
BytesIO().seek(0)
text_pdf = PdfFileReader(BytesIO())
base_file = PdfFileReader(open("media/01.pdf", "rb"))
page = base_file.getPage(0)
page.mergePage(text_pdf.getPage(0))
PdfFileWriter().addPage(page)
PdfFileWriter().write(open("pdf01.pdf", "wb")
FileField does not store files directly in the database. Files get uploaded in a location on the filesystem determined by the upload_to argument. Only some metadata are stored in the DB, including the path of the file in your filesystem.
If you want to have the contents of the files in the database, you could create a new File model that includes a BinaryField to store the data and a CharField to store the URL from which the file can be fetched. To feed the data of PdfFileWriter to the binary field of Django, perhaps the most appropriate would be to use BytesIO.
I found this workaround to direct the file to a desired location (in this case both my media_cdn folder and also output it to an admin.)
I set up an admin action to perform the function that outputs the file so the admin will have access to both the output version in the form of both an HTTP response and through the media_cdn storage.
Hope this helps anyone who struggles with the same problem.
#admin.py
class edit_and_output():
def output:
author = Account.email
#alter file . . .
with open('media_cdn/account/{0}.pdf'.format(author), 'wb') as out_file:
output.write(out_file)
response = HttpResponse(content_type='application/pdf')
response['Content-Disposition'] = 'attachment;filename="{0}.pdf"'.format(author)
output.write(response)
I am pretty new to Django. I need some help.
I am working on an admin page that allows me to edit objects. My model has a text field that I want to fill with a file contents which will be uploaded with a FileInput widget in a form.
So I want to read the file contents, put it in a string and store it in the model text field. Therefore it should be saved in the database.
Any help with this? I dont know how to get the file , read it and store it in my model as string. I am using a ModelAdmin btw.
First off, you might want to just use a model.FileField (https://docs.djangoproject.com/en/2.0/ref/models/fields/#filefield). The database only has to store a path to the file, instead of the entire contents of the file. You might not want to store the contents of a file directly in the database, especially if its a large file.
But if you do want to read a file in to your model. Try something along the lines of:
models.py
class SomeModel(Model):
textfield = TextField()
views.py or whatever script you are calling
with open('data.txt', 'r') as myfile:
data=myfile.read()
newmodel = SomeModel()
newmodel.textfield = data
newmodel.save()
I have an API endpoint with Django Rest Framework to upload an image.
class MyImageSerializer(serializers.ModelSerializer):
image = serializers.ImageField(source='image')
I can upload images but they are saved with the filename that is sent from the client which can result to collisions. I would like instead to upload the file to my CDN with a timestamp filename.
Generating the filename is not the problem, just saving the image with it.
Any one knows how to do that?
Thanks.
If your image is of type ImageField from django, then you don't really have to do anything, not even declare it in your serializer like you did. It's enough to add it in the fields attribute and django will handle collisions. This means django will add _index on each new file which might generate a collision, so if you upload a file named 'my_pic.jpg' 5 times, you will actually have files 'my_pic.jpg', 'my_pic_1.jpg', 'my_pic_2.jpg', 'my_pic_3.jpg', 'my_pic_4.jpg' on your server.
Now, this is done using django's implementation for FileSystemStorage (see here), but if you want it to append a timestamp to your filename, all you have to do is write a storage class where you overwrite the get_available_name(name) method. Example:
class MyFileSystemStorage(FileSystemStorage):
def get_available_name(self, name):
''' name is the current file name '''
now = time.time()
stamp = datetime.datetime.fromtimestamp(now).strftime('%Y-%m-%d-%H-%M-%S')
return '{0}_{1}'.format(name, str(stamp))
And the image field in your model:
image = models.ImageField(upload_to='your upload dir', storage= MyFileSystemStorage)
Important update
As of August 20, 2014 this is no longer an issue, since Django found a vulnerability related to this behaviour (thanks #mlissner for pointing it out) . Important excerpt :
We’ve remedied the issue by changing the algorithm for generating file
names if a file with the uploaded name already exists.
Storage.get_available_name() now appends an underscore plus a random 7
character alphanumeric string (e.g. "_x3a1gho"), rather than iterating
through an underscore followed by a number (e.g. "_1", "_2", etc.).
I am using a file upload in my Django model like this :
def upload_path(self, filename):
return 'upload/actualities/%s/%s' % (self.id, filename)
photo = models.ImageField(upload_to=upload_path)
and my adminModel is :
from actualities.models import *
from django.contrib import admin
class ActualityAdmin(admin.ModelAdmin):
class Media:
js = ('/static/js/tiny_mce/tiny_mce.js', '/static/js/textareas.js')
admin.site.register(Actuality, ActualityAdmin)
Everything works fine except when i edit mu model because it has an id. But when I create it, the file upload happens before the model saving... So i put my file in /media/actualities/None/filename.jpg, and I want /media/2/filename.jpg
How can I force to make the file upload after the model saving?
Thank you!!!
You will probably want to override the Model's save() method, and maybe come up with a custom "don't do anything" UploadHandler, then switch back to the original one and call save again.
https://docs.djangoproject.com/en/dev/topics/http/file-uploads/
https://docs.djangoproject.com/en/dev/topics/db/models/
What I would do in this situation however, is make a custom upload handler that saves the file off into some temp space. Then I'd override the save method (in a mixin or something) that moves the file from temp to wherever you wanted it.
#Tomek's answer is also another way. If you have your model generate it's own id, then you can use that.
A second to last suggestion which is what I do with my photo blog is instead of saving all the images in a directory like media/2/filename.jpg I save the image by date uploaded. 2011/10/2/image.jpg This kind of helps any directory from getting too unwieldy.
Finally, you could hash the file names and store them in directories of hash name to kind of equally spread out the images in a directory.
I've picked the date style because that's meaningful for me with that project. Perhaps there is another way you can name an image for saving that would mean something more than "model with id 2's pics" that you could use for this problem.
Good Luck!
As workaround, try generating UUID for file name (instead of using self.id).