Error: A bytes-like object is required, not 'list' - django

I am developing a small project that aims to enable to the user to submit a zip file. The system should read the file, extract it and save its contents to the database.
I am having this error : a bytes-like object is required, not 'list' because I tried to use BytesIo. without it, the error said :" fpin.seek(0, 2)
AttributeError: 'list' object has no attribute 'seek'"
My code looks like this
files = request.FILES.getlist('document[]')
with zipfile.ZipFile(io.BytesIO(files), "r")as archive:
for zippedFileName in archive.namelist():
with archive.open(zippedFileName) as myfile:
with io.BytesIO() as buf:
buf.write(myfile.read())
buf.seek(0)
file = File(buf, zippedFileName).decode('utf-8')
rbe = UploadedFile.objects.create(document=file)
rbe.user= request.user
rbe.save()
return render(request, 'uploader/index.html', {'files': files})
The error or traceback looks like this

In Django, request.FILES.getlist returns a list of UploadedFile objects, each of which has a file attribute, a file-like object that you can pass directly to the ZipFile constructor.
Since your code apparently assumes that there is only one UploadedFile object in the list returned by request.FILES.getlist, you can unpack it like this:
uploaded_file, = request.FILES.getlist('document[]')
with zipfile.ZipFile(uploaded_file.file, "r") as archive:
for zippedFileName in archive.namelist():
...
# you should return after the for loop, not within it
return render(request, 'uploader/index.html', {'files': [uploaded_file]})

Related

'NoneType' object is not iterable: Loop for a list

I am trying to convert a loop for a list, but i'm getting the 'NoneType' object is not iterable' error.
assets = input("What assets you desire to analyze?v: ").upper().split(",")
for ticker in assets:
assets1 = print(ticker + ".SA")
When i print "assets1", i get a output like this:
PGMN3.SA
COGN3.SA
TASA4.SA
PCAR3.SA
PETZ3.SA
But i wish to make this loop as a list, to do a WebScraping after, but when put "assets1" for webscraping, i get the message error: 'NoneType' object is not iterable'
How can i fix it?
Thanks!
print() function return None - so you're only printing the tickers, not storing them in anything. This example uses list-comprehension to add .SA to each ticker and store it in a list names assets:
assets = input("What assets you desire to analyze?").upper().split(",")
assets = [ticker + ".SA" for ticker in assets]
print(assets)
Prints (for example):
What assets you desire to analyze?PGMN3,COGN3,TASA4,PCAR3,PETZ3
['PGMN3.SA', 'COGN3.SA', 'TASA4.SA', 'PCAR3.SA', 'PETZ3.SA']

Saving a ZipFile to FileField in Django

I have the following model in Django:
class AksOrder(models.Model):
zip_file = models.FileField(upload_to='aks_zips/%M/%S/', blank=True)
and in my views I have in essential these functions:
def gen_zip(pk, name, vars):
zipObj = ZipFile(os.path.join('/tmp/', str(name) + '_' + str(pk) + '.zip'), 'w')
zipObj.write(pdf_files[0].path, '/filea.pdf')
zipObj.write(pdf_files[1].path, '/fileb.pdf')
def aksorder_complete(request, pk):
ao = get_object_or_404(AksOrder, id=pk)
zipObj = generate_shop_zip(ao.c.pk, ao.dl, ao.vars)
ao.zip_file.save('file.zip', zipObj)
I did not only try this version, but this one seems the most reasonable and logic one to me. I get a There is no item named 65536 in the archive. When I modify it slightly and close the file at the end of zip-writing in the first function, I get a ValueError: Attempt to use ZIP archive that was already closed message. Both times, the zip-File is generated properly in /tmp/ I could not work arount it. And that's only locally, I need to do it for S3 later...
I finally achieved it: I added a zipObj.close() to the first function at the end and I modified the 2nd function like so:
file = open('path/to/file.zip', 'rb')
ao.zip_file.save('name.zip', file)
apparently, the rb mode in file-open was decisive.

Django ImageField with tempfile

Folks, I need help understanding some details about how Django saves model files. I've written a test that involves creation of files (in a temporary directory via tempfile) and has the following lines:
TEMP_DIR = tempfile.TemporaryDirectory()
TEMP_DIR_PATH = TEMP_DIR.name
...
#override_settings(MEDIA_ROOT=TEMP_DIR_PATH)
def create_photo(self, album_number, photo_number):
...
p = Photo.objects.create(
number=photo_number,
album=album,
added_by=self.user,
image=SimpleUploadedFile(
name=...,
content=open(..., 'rb').read(),
content_type='image/jpeg'
),
remarks='-'
)
p.full_clean()
p.save()
return p
This code works, except for one thing that confuses me. The line p = Photo.objects.create causes a file to appear in the temporary directory. Then p.full_clean() does nothing to the file. However when I execute p.save(), the file disappears from the temporary directory. If I remove p.save(), the file stays there when the function returns.
So my test function
def test_image_file_present(self):
"""When a photo is added to DB, the file actually appears in MEDIA."""
p = self.create_photo(3, 2)
image_filename = p.image.file.name
if not os.path.exists(image_filename):
self.fail('Image file not found')
fails if p.save() is there but passes if I remove p.save().
Why would object.save() cause the file to disappear?
As a bonus question, what's the purpose of .save() if the file and the Django model object appear already during Photo.objects.create? I've checked that the pre-save signal is sent by Photo.object.create() as well as by p.save().

Keeping Original Filename for FileField in Django

I would like to keep the original file name of an UploadedFile in Django that has its location stored in a FileField. Right now I am observing that if two files have the same name, the first file uploaded keeps its original name but the second time a file with that name is uploaded, it has a random string appended to make the file name unique. One solution is to add an additional field to the model: Django: How to save original filename in FileField? or Saving Original File Name in Django with FileField but these solutions seem suboptimal as they require changing the Model fields.
An alternative would be to prepend a random directory path to the front of the file make sure that in a given directory the file name is unique and allowing the basename to remain unchanged. One way to do this would be to pass in a callable upload_to that does just that. Another option would be to subclass FileField and override get_filename to not strip the input filename to the basename allowing the caller to pass in a filename with a prepended path. The latter option is not ideal if I want to use an ImageField as I would have to subclass that as well.
In looking at the code that actually generates the unique filename by appending the random string, it looks like the best solution to this problem might be to subclass the Storage class in-use and override get_available_name method to create unique filenames by prepending a directory rather than post-pending the string to the base name.
Sorry for the quick answere, here is another approach to your question :
The idea here is to create an unique folder for each uploaded file.
# in your settings.py file
MY_FILE_PATH = 'stored_files/'
The path were your files will be stored : /public/media/stored_files
# somewhere in your project create an utils.py file
import random
try:
from hashlib import sha1 as sha_constructor
except ImportError:
from django.utils.hashcompat import sha_constructor
def generate_sha1(string, salt=None):
"""
Generates a sha1 hash for supplied string.
:param string:
The string that needs to be encrypted.
:param salt:
Optionally define your own salt. If none is supplied, will use a random
string of 5 characters.
:return: Tuple containing the salt and hash.
"""
if not isinstance(string, (str, unicode)):
string = str(string)
if isinstance(string, unicode):
string = string.encode("utf-8")
if not salt:
salt = sha_constructor(str(random.random())).hexdigest()[:5]
hash = sha_constructor(salt+string).hexdigest()
return (salt, hash)
In your models.py
from django.conf import settings
from utils.py import generate_sha1
def upload_to_unqiue_folder(instance, filename):
"""
Uploads a file to an unique generated Path to keep the original filename
"""
salt, hash = generate_sha1('{}{}'.format(filename, get_datetime_now().now))
return '%(path)s%(hash_path)s%(filename)s' % {'path': settings.MY_FILE_PATH,
'hash_path': hash[:10],
'filename': filename}
#And then add in your model fileField the uplaod_to function
class MyModel(models.Model):
file = models.FileField(upload_to=upload_to_unique_folder)
The file will be uploaded to this location :
public/media/stored_file_path/unique_hash_folder/my_file.extention
Note : I got the code from Django userena sources, and adapted it to my needs
Note2 : For more informations take a look at this greate post on Django File upload : File upload example
Have a good day.
Edit : Trying to provide a working solution :)
To my understanding, during the form submission/file upload process, you can add form validation functions.
During the validation and cleaning process, you could check that the database does not already have a duplicate name (ie. query to see if that file name exists).
If it is duplicate, you could just rename it xyz_1, xyz_2, etc

'NoneType' object has no attribute 'get_server_msg', Socketio (chat with Django)

I'm new with django and I try to implement a chat, using socketio. I used ://github.com/stephenmcd/django-socketio and I followed this tutorial: http://codysoyland.com/2011/feb/6/evented-django-part-one-socketio-and-gevent/
The example works well. Now I would like to integrate this chat inside the templates of my django app. I didn't find any tutorial about that.
So I tried to take the example, I put the html inside my template and I used the .py inside my view.
So here is the part of my views.py that is creating the problem:
buffer = []
socketio = request.environ['socketio']
if socketio.on_connect():
socketio.send({'buffer': buffer})
socketio.broadcast({'announcement': socketio.session.session_id + ' connected'})
while True:
message = socketio.recv()
if len(message) == 1:
message = message[0]
message = {'message': [socketio.session.session_id, message]}
buffer.append(message)
if len(buffer) > 15:
del buffer[0]
socketio.broadcast(message)
else:
if not socketio.connected():
socketio.broadcast({'announcement': socketio.session.session_id + ' disconnected'})
break
But when I go to my page I get this error message:
'NoneType' object has no attribute 'get_server_msg'
Exception Type: AttributeError
Exception Value:'NoneType' object has no attribute 'get_server_msg'
Exception Location: /Users/marc-antoinelacroix/virtualenv/lib/python2.7/site- packages/socketio/protocol.py in recv, line 41
Do you have any idea on how to fix that?
Thank you,