When I run heroku run python manage.py collectstatic it does upload staticfiles to AWS S3, however when I run django in production, my static files do not load to the page.
To be honest i dont know why do i have bad request, since i already did collectstatic to S3.
My AWS settings # settings.py
AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')
STATIC_URL = 'https://' + AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/'
AWS_URL = os.environ.get('AWS_URL')
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
STATIC_URL = "https://" + AWS_STORAGE_BUCKET_NAME + ".s3.amazonaws.com/"
MEDIA_URL = STATIC_URL + "media/"
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
STATIC_ROOT = 'staticfiles'
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)
Related
This is currently my configuration for static and media files. This works fine locally, however now, I'm trying to host them using S3.
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATIC_URL = '/static/'
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
This is also my initial S3 configuration:
AWS_ACCESS_KEY_ID = '*******'
AWS_SECRET_ACCESS_KEY = '*************'
AWS_STORAGE_BUCKET_NAME = '*******'
AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = None
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
Note: I'm also currently hosting my website on heroku.
I'm using S3 to store my static files, but I'm having an issue where only the admin static files are uploading to the bucket.
I expected to see the css and other folders from inside static upload to the bucket, but instead, this is all I got:
static/
- admin/
- flags/
What can I do to get my app's static folder to upload also?
Looking at other answers to this question, it seems like the main difference is that my project uses django-sass-processor, but I'm not sure why that would mean my entire folder is not picked up.
Project folder structure:
backend/
- accounts/
- core/
-- settings.py
-- ...
- static/
- templates/
settings.py
USE_S3 = os.getenv('USE_S3') == 'TRUE'
if USE_S3:
# aws settings
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = 'public-read'
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
# s3 static settings
AWS_LOCATION = 'static'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
else:
STATIC_URL = '/staticfiles/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'sass_processor.finders.CssFinder',
]
SASS_PROCESSOR_ROOT = os.path.join(BASE_DIR, 'static')
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
I use AWS S3 to store my Django static files (using django-storages). And alternate cloudfront domain name to serve these files.
I am unable to finish Django compressor via python manage.py compress.
This is the error I see:
CommandError: An error occurred during rendering /home/ubuntu/foldername/templates/home/xxx.html: 'https://xxx.domainname.com/static/plugins/xxx-xxx/xxx.js' isn't accessible via COMPRESS_URL ('https://s3bucket-static.s3.amazonaws.com/') and can't be compressed
So I tried using the default cloudfront address, alternate domain name and S3 bucket for COMPRESS_URL:
https://s3bucket-static.s3.amazonaws.com/
https://s3bucket-static.s3.amazonaws.com/static/
https://d1231232131241123.cloudfront.net/
https://d1231232131241123.cloudfront.net/static/
https://xxx.domainname.com/
https://xxx.domainname.com/static/
Only https://domainname.com/static/ don't have error. What happen is that after compress is done, the CACHE folder appear inside the local instance staticfiles folder. Which is wrong since I wanted the CACHE folder to be inside the S3 bucket.
Here are my settings for django compressor:
# django-compressor
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder',
)
COMPRESS_ENABLED = True
COMPRESS_OFFLINE = True
if not DEBUG:
COMPRESS_URL = env('COMPRESS_URL')
Here are my django-storages settings:
if not DEBUG:
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = env('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = env('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = env('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = None
AWS_CLOUDFRONT_DOMAIN_NAME = env('AWS_CLOUDFRONT_DOMAIN_NAME')
AWS_S3_CUSTOM_DOMAIN = f'{AWS_CLOUDFRONT_DOMAIN_NAME}.com'
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'
settings for static files:
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles/')
STATICFILES_DIRS = os.path.join(BASE_DIR, 'static/')
It worked after I changed my django-compressor settings to this:
# django-compressor
COMPRESS_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder',
)
COMPRESS_ENABLED = True
COMPRESS_OFFLINE = True
if not DEBUG:
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/static/'
COMPRESS_ROOT = STATIC_ROOT
COMPRESS_URL = STATIC_URL
I want to upload my staticfiles to amazon s3 storage, but I can't stop django from just putting them in a directory staticfiles in the project root. I have boto3 in my requirements.txt and have set
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
I have successfully set up s3 with media files, to the same bucket (under different directory).
Collectstatic also seems disregards the STATIC_ROOT variable in settings.py when I comment out the s3-settings. When running python3 manage.py collectstatic I expect it to gather static files into STATIC_ROOT as it says in the docs here https://docs.djangoproject.com/en/2.1/ref/settings/ (under static_root). But even if I give a different value to STATIC_ROOT, collectstatic always collects into a directory on the root called staticfiles.
STATIC_ROOT = os.path.join(BASE_DIR, 'this_is_not_used_by_collectstatic')
The rest of my s3 settings:
# Amazon S3
AWS_ACCESS_KEY_ID = os.environ['AWS_ACCESS_KEY_ID']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_SECRET_ACCESS_KEY']
AWS_STORAGE_BUCKET_NAME = 'my_bucket_name'
AWS_S3_REGION_NAME = 'eu-north-1'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_DEFAULT_ACL = None
AWS_LOCATION = 'static'
STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
# media
DEFAULT_FILE_STORAGE = 'korvir.storage_backends.MediaStorage'
If you're using django-heroku and and you have the following in your code, as per the instructions:
django_heroku.settings(locals())
... then it will overwrite your STATIC_ROOT setting. If you want to preserve your own STATIC_ROOT setting, then make sure you set STATIC_ROOT after calling django_heroku.settings.
Intro: I have made my application in Django I am trying to get my static and media files hosted in aws s3. My Django project is on AWS Lambda and AWS Api gateway using Zappa. below is my settings.py
AWS_DEFAULT_ACL = None
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static'),
]
DEFAULT_FILE_STORAGE = 'aws_storage_classes.MediaStorage'
AWS_ACCESS_KEY_ID = os.getenv("ACCESS_KEY")
AWS_SECRET_ACCESS_KEY = os.getenv("ACCESS_SECRET_KEY")
AWS_STORAGE_BUCKET_NAME = os.getenv("AWS_STORAGE_BUCKET_NAME")
STATICFILES_STORAGE = 'aws_storage_classes.StaticStorage'
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_S3_DOMAIN = "%s.s3.amazonaws.com" % AWS_STORAGE_BUCKET_NAME
STATIC_URL = 'https://%s.static/' % AWS_S3_DOMAIN
MEDIA_URL = 'https://%s.media/' % AWS_S3_DOMAIN
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
I then created a file in my project folder the same as my manage.py called aws_storage_classes.py
Below are the contents of my file aws_storage_classes.py
from storages.backends.s3boto3 import S3Boto3Storage
class StaticStorage(S3Boto3Storage):
location = 'static'
class MediaStorage(S3Boto3Storage):
location = 'media'
Below are my urls
if settings.DEBUG:
urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
Also in my settings.py
DEBUG = False
When I do python manage.py collectstatic all the staticfiles are downloaded and I don't get any errors. But when go on admin page the static files are not uploaded. See images below
Below is the image of my S3 bucket
Below is what is inside the static folder
Static files not loading
<link rel="stylesheet" type="text/css" href="https://<bucketname>.s3.amazonaws.com/static/style.css?AWSAccessKeyId=AKIAJHJGHJGJGJGJJGJGJMHIFQ&Signature=yfS3%2BvA0q15aUxw7OBySuQWZfjg%3D&x-amz-security-token=FQoGZXIvYXdgljdaksfgdjsgfkjertert;hre;thtires=151232173">
Updated settings.py
AWS_DEFAULT_ACL = None
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static'),
]
DEFAULT_FILE_STORAGE = 'aws_storage_classes.MediaStorage'
AWS_ACCESS_KEY_ID = os.getenv("ACCESS_KEY")
AWS_SECRET_ACCESS_KEY = os.getenv("ACCESS_SECRET_KEY")
AWS_STORAGE_BUCKET_NAME = os.getenv("AWS_STORAGE_BUCKET_NAME")
STATICFILES_STORAGE = 'aws_storage_classes.StaticStorage'
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_S3_DOMAIN = "%s.s3.amazonaws.com" % AWS_STORAGE_BUCKET_NAME
STATIC_URL = 'https://%s.static/' % AWS_S3_DOMAIN
MEDIA_URL = 'https://%s.media/' % AWS_S3_DOMAIN
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
Below is the link after adding ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
<link rel="stylesheet" type="text/css" href="https://django-static-media.s3.amazonaws.com/static/style.css?AWSAccessKeyId=AKIAGGGGGGGGGMHIFQ&Signature=6gFQTsOSDFSDFA%3D&x-amz-security-token=FQoGZvwbJhd9amp;Expires=1549168642">
Still staticfiles not loading
I have given my user full s3 access so I don't think I should need the below. Also I was able to successfully add the static files in the S3 and it is not giving me any permission related errors. still trying it out as per a suggestions below
Below are my permissions
I tried to give permission to lambda function but it gives me the bellow error
After changing the static and media URLS
I can see STATIC_URL and MEDIA_URL not setup correctly.
change
STATIC_URL = 'https://%s.static/' % AWS_S3_DOMAIN
MEDIA_URL = 'https://%s.media/' % AWS_S3_DOMAIN
To
STATIC_URL = 'https://%s/static/' % AWS_S3_DOMAIN
MEDIA_URL = 'https://%s/media/' % AWS_S3_DOMAIN
It should work.
The following link has a very good explanation.
https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html
I had the same issue but with me it was a ACL issue. I could only get collectstatic to run if I set AWS_DEFAULT_ACL=None but then all the files would be set to private by default and couldn't access them from the app.
To fix I had to change the S3 bucket Object Ownership to Bucket owner preferred so that it accepted the public-read ACL.