This is currently my configuration for static and media files. This works fine locally, however now, I'm trying to host them using S3.
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATIC_URL = '/static/'
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
This is also my initial S3 configuration:
AWS_ACCESS_KEY_ID = '*******'
AWS_SECRET_ACCESS_KEY = '*************'
AWS_STORAGE_BUCKET_NAME = '*******'
AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = None
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
Note: I'm also currently hosting my website on heroku.
Related
I'm using S3 to store my static files, but I'm having an issue where only the admin static files are uploading to the bucket.
I expected to see the css and other folders from inside static upload to the bucket, but instead, this is all I got:
static/
- admin/
- flags/
What can I do to get my app's static folder to upload also?
Looking at other answers to this question, it seems like the main difference is that my project uses django-sass-processor, but I'm not sure why that would mean my entire folder is not picked up.
Project folder structure:
backend/
- accounts/
- core/
-- settings.py
-- ...
- static/
- templates/
settings.py
USE_S3 = os.getenv('USE_S3') == 'TRUE'
if USE_S3:
# aws settings
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = 'public-read'
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
# s3 static settings
AWS_LOCATION = 'static'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
else:
STATIC_URL = '/staticfiles/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'sass_processor.finders.CssFinder',
]
SASS_PROCESSOR_ROOT = os.path.join(BASE_DIR, 'static')
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
I am using Django storages to host my static files (css, js, images) on s3. When I load my webpage django keeps pointing to the incorrect url of my s3 public bucket. For example, it keeps returning https://mysite.amazonaws.com/assets/images/cat.png despite the correct public url for the file being https://mysite.s3-ap-southeast-2.amazonaws.com/assets/images/cat.png
settings.py
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = os.environ.get('aws_access_key_id')
AWS_SECRET_ACCESS_KEY = os.environ.get('aws_secret_key')
AWS_STORAGE_BUCKET_NAME = 'mysite'
AWS_DEFAULT_ACL = "private"
AWS_S3_SIGNATURE_VERSION = "s3v4"
AWS_S3_REGION_NAME = 'ap-southeast-2'
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static'),
]
STATIC_URL = 'https://mysite.s3-ap-southeast-2.amazonaws.com/static/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
Try setting a s3 custom domain.
AWS_S3_CUSTOM_DOMAIN = 'mysite.s3-ap-southeast-2.amazonaws.com'
I use AWS S3 to store my Django static files (using django-storages). And alternate cloudfront domain name to serve these files.
I am unable to finish Django compressor via python manage.py compress.
This is the error I see:
CommandError: An error occurred during rendering /home/ubuntu/foldername/templates/home/xxx.html: 'https://xxx.domainname.com/static/plugins/xxx-xxx/xxx.js' isn't accessible via COMPRESS_URL ('https://s3bucket-static.s3.amazonaws.com/') and can't be compressed
So I tried using the default cloudfront address, alternate domain name and S3 bucket for COMPRESS_URL:
https://s3bucket-static.s3.amazonaws.com/
https://s3bucket-static.s3.amazonaws.com/static/
https://d1231232131241123.cloudfront.net/
https://d1231232131241123.cloudfront.net/static/
https://xxx.domainname.com/
https://xxx.domainname.com/static/
Only https://domainname.com/static/ don't have error. What happen is that after compress is done, the CACHE folder appear inside the local instance staticfiles folder. Which is wrong since I wanted the CACHE folder to be inside the S3 bucket.
Here are my settings for django compressor:
# django-compressor
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder',
)
COMPRESS_ENABLED = True
COMPRESS_OFFLINE = True
if not DEBUG:
COMPRESS_URL = env('COMPRESS_URL')
Here are my django-storages settings:
if not DEBUG:
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = env('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = env('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = env('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = None
AWS_CLOUDFRONT_DOMAIN_NAME = env('AWS_CLOUDFRONT_DOMAIN_NAME')
AWS_S3_CUSTOM_DOMAIN = f'{AWS_CLOUDFRONT_DOMAIN_NAME}.com'
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'
settings for static files:
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles/')
STATICFILES_DIRS = os.path.join(BASE_DIR, 'static/')
It worked after I changed my django-compressor settings to this:
# django-compressor
COMPRESS_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder',
)
COMPRESS_ENABLED = True
COMPRESS_OFFLINE = True
if not DEBUG:
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/static/'
COMPRESS_ROOT = STATIC_ROOT
COMPRESS_URL = STATIC_URL
Intro: I have made my application in Django I am trying to get my static and media files hosted in aws s3. My Django project is on AWS Lambda and AWS Api gateway using Zappa. below is my settings.py
AWS_DEFAULT_ACL = None
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static'),
]
DEFAULT_FILE_STORAGE = 'aws_storage_classes.MediaStorage'
AWS_ACCESS_KEY_ID = os.getenv("ACCESS_KEY")
AWS_SECRET_ACCESS_KEY = os.getenv("ACCESS_SECRET_KEY")
AWS_STORAGE_BUCKET_NAME = os.getenv("AWS_STORAGE_BUCKET_NAME")
STATICFILES_STORAGE = 'aws_storage_classes.StaticStorage'
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_S3_DOMAIN = "%s.s3.amazonaws.com" % AWS_STORAGE_BUCKET_NAME
STATIC_URL = 'https://%s.static/' % AWS_S3_DOMAIN
MEDIA_URL = 'https://%s.media/' % AWS_S3_DOMAIN
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
I then created a file in my project folder the same as my manage.py called aws_storage_classes.py
Below are the contents of my file aws_storage_classes.py
from storages.backends.s3boto3 import S3Boto3Storage
class StaticStorage(S3Boto3Storage):
location = 'static'
class MediaStorage(S3Boto3Storage):
location = 'media'
Below are my urls
if settings.DEBUG:
urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
Also in my settings.py
DEBUG = False
When I do python manage.py collectstatic all the staticfiles are downloaded and I don't get any errors. But when go on admin page the static files are not uploaded. See images below
Below is the image of my S3 bucket
Below is what is inside the static folder
Static files not loading
<link rel="stylesheet" type="text/css" href="https://<bucketname>.s3.amazonaws.com/static/style.css?AWSAccessKeyId=AKIAJHJGHJGJGJGJJGJGJMHIFQ&Signature=yfS3%2BvA0q15aUxw7OBySuQWZfjg%3D&x-amz-security-token=FQoGZXIvYXdgljdaksfgdjsgfkjertert;hre;thtires=151232173">
Updated settings.py
AWS_DEFAULT_ACL = None
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static'),
]
DEFAULT_FILE_STORAGE = 'aws_storage_classes.MediaStorage'
AWS_ACCESS_KEY_ID = os.getenv("ACCESS_KEY")
AWS_SECRET_ACCESS_KEY = os.getenv("ACCESS_SECRET_KEY")
AWS_STORAGE_BUCKET_NAME = os.getenv("AWS_STORAGE_BUCKET_NAME")
STATICFILES_STORAGE = 'aws_storage_classes.StaticStorage'
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_S3_DOMAIN = "%s.s3.amazonaws.com" % AWS_STORAGE_BUCKET_NAME
STATIC_URL = 'https://%s.static/' % AWS_S3_DOMAIN
MEDIA_URL = 'https://%s.media/' % AWS_S3_DOMAIN
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
Below is the link after adding ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
<link rel="stylesheet" type="text/css" href="https://django-static-media.s3.amazonaws.com/static/style.css?AWSAccessKeyId=AKIAGGGGGGGGGMHIFQ&Signature=6gFQTsOSDFSDFA%3D&x-amz-security-token=FQoGZvwbJhd9amp;Expires=1549168642">
Still staticfiles not loading
I have given my user full s3 access so I don't think I should need the below. Also I was able to successfully add the static files in the S3 and it is not giving me any permission related errors. still trying it out as per a suggestions below
Below are my permissions
I tried to give permission to lambda function but it gives me the bellow error
After changing the static and media URLS
I can see STATIC_URL and MEDIA_URL not setup correctly.
change
STATIC_URL = 'https://%s.static/' % AWS_S3_DOMAIN
MEDIA_URL = 'https://%s.media/' % AWS_S3_DOMAIN
To
STATIC_URL = 'https://%s/static/' % AWS_S3_DOMAIN
MEDIA_URL = 'https://%s/media/' % AWS_S3_DOMAIN
It should work.
The following link has a very good explanation.
https://simpleisbetterthancomplex.com/tutorial/2017/08/01/how-to-setup-amazon-s3-in-a-django-project.html
I had the same issue but with me it was a ACL issue. I could only get collectstatic to run if I set AWS_DEFAULT_ACL=None but then all the files would be set to private by default and couldn't access them from the app.
To fix I had to change the S3 bucket Object Ownership to Bucket owner preferred so that it accepted the public-read ACL.
I'm trying to configure my django setup to serve static files in debug and production mode. I want in production mode to be from S3 and in debug mode to be from the local installation. I have the settings as below:
COMPRESS_ENABLED = not DEBUG
COMPRESS_PARSER = 'compressor.parser.LxmlParser'
COMPRESS_CSS_FILTERS = ['compressor.filters.cssmin.CSSMinFilter']
COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter']
COMPRESS_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
DEFAULT_FILE_STORAGE = 'libs.storages.S3Storage.S3Storage'
AWS_ACCESS_KEY_ID = # Key
AWS_SECRET_ACCESS_KEY = # Secret
AWS_STORAGE_BUCKET_NAME = # Bucket
AWS_QUERYSTRING_AUTH = False
MEDIA_URL = '/media/'
if COMPRESS_ENABLED:
from boto.s3.connection import SubdomainCallingFormat
AWS_S3_CALLING_FORMAT = SubdomainCallingFormat()
COMPRESS_OFFLINE = True
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
# Path for all static files
STATIC_URL = 'https://%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
STATIC_ROOT = STATIC_URL
else:
STATIC_ROOT = path.join(PROJECT_ROOT, '..', 'assets')
STATIC_URL = '/static/'
I'm able to get everything to work except one tiny thing which I'm not able to figure out. Often times my CSS files will have a background url as /images/logo.png. The folder /images/ exists in S3 but for my local it needs to be /static/images/. I tried to set the S3 URL as /static/ at the end but it wasn't working. Is there anything else i'm missing? How do I get the CSS image urls to redirect to /static/?
I read through the source code of s3boto.py. You can set the following property:
AWS_LOCATION = "static"
That variable creates a static folder in your s3. The other properties should be:
STATIC_URL = 'https://%s.s3.amazonaws.com/static/' % AWS_STORAGE_BUCKET_NAME
STATIC_ROOT = STATIC_URL