I use AWS S3 to store my Django static files (using django-storages). And alternate cloudfront domain name to serve these files.
I am unable to finish Django compressor via python manage.py compress.
This is the error I see:
CommandError: An error occurred during rendering /home/ubuntu/foldername/templates/home/xxx.html: 'https://xxx.domainname.com/static/plugins/xxx-xxx/xxx.js' isn't accessible via COMPRESS_URL ('https://s3bucket-static.s3.amazonaws.com/') and can't be compressed
So I tried using the default cloudfront address, alternate domain name and S3 bucket for COMPRESS_URL:
https://s3bucket-static.s3.amazonaws.com/
https://s3bucket-static.s3.amazonaws.com/static/
https://d1231232131241123.cloudfront.net/
https://d1231232131241123.cloudfront.net/static/
https://xxx.domainname.com/
https://xxx.domainname.com/static/
Only https://domainname.com/static/ don't have error. What happen is that after compress is done, the CACHE folder appear inside the local instance staticfiles folder. Which is wrong since I wanted the CACHE folder to be inside the S3 bucket.
Here are my settings for django compressor:
# django-compressor
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder',
)
COMPRESS_ENABLED = True
COMPRESS_OFFLINE = True
if not DEBUG:
COMPRESS_URL = env('COMPRESS_URL')
Here are my django-storages settings:
if not DEBUG:
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = env('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = env('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = env('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = None
AWS_CLOUDFRONT_DOMAIN_NAME = env('AWS_CLOUDFRONT_DOMAIN_NAME')
AWS_S3_CUSTOM_DOMAIN = f'{AWS_CLOUDFRONT_DOMAIN_NAME}.com'
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'
settings for static files:
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles/')
STATICFILES_DIRS = os.path.join(BASE_DIR, 'static/')
It worked after I changed my django-compressor settings to this:
# django-compressor
COMPRESS_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder',
)
COMPRESS_ENABLED = True
COMPRESS_OFFLINE = True
if not DEBUG:
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/static/'
COMPRESS_ROOT = STATIC_ROOT
COMPRESS_URL = STATIC_URL
Related
I'm using S3 to store my static files, but I'm having an issue where only the admin static files are uploading to the bucket.
I expected to see the css and other folders from inside static upload to the bucket, but instead, this is all I got:
static/
- admin/
- flags/
What can I do to get my app's static folder to upload also?
Looking at other answers to this question, it seems like the main difference is that my project uses django-sass-processor, but I'm not sure why that would mean my entire folder is not picked up.
Project folder structure:
backend/
- accounts/
- core/
-- settings.py
-- ...
- static/
- templates/
settings.py
USE_S3 = os.getenv('USE_S3') == 'TRUE'
if USE_S3:
# aws settings
AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_STORAGE_BUCKET_NAME')
AWS_DEFAULT_ACL = 'public-read'
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
# s3 static settings
AWS_LOCATION = 'static'
STATIC_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{AWS_LOCATION}/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
else:
STATIC_URL = '/staticfiles/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'sass_processor.finders.CssFinder',
]
SASS_PROCESSOR_ROOT = os.path.join(BASE_DIR, 'static')
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
I am using Django storages to host my static files (css, js, images) on s3. When I load my webpage django keeps pointing to the incorrect url of my s3 public bucket. For example, it keeps returning https://mysite.amazonaws.com/assets/images/cat.png despite the correct public url for the file being https://mysite.s3-ap-southeast-2.amazonaws.com/assets/images/cat.png
settings.py
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
AWS_ACCESS_KEY_ID = os.environ.get('aws_access_key_id')
AWS_SECRET_ACCESS_KEY = os.environ.get('aws_secret_key')
AWS_STORAGE_BUCKET_NAME = 'mysite'
AWS_DEFAULT_ACL = "private"
AWS_S3_SIGNATURE_VERSION = "s3v4"
AWS_S3_REGION_NAME = 'ap-southeast-2'
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static'),
]
STATIC_URL = 'https://mysite.s3-ap-southeast-2.amazonaws.com/static/'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
Try setting a s3 custom domain.
AWS_S3_CUSTOM_DOMAIN = 'mysite.s3-ap-southeast-2.amazonaws.com'
When I run heroku run python manage.py collectstatic it does upload staticfiles to AWS S3, however when I run django in production, my static files do not load to the page.
To be honest i dont know why do i have bad request, since i already did collectstatic to S3.
My AWS settings # settings.py
AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')
STATIC_URL = 'https://' + AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/'
AWS_URL = os.environ.get('AWS_URL')
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
STATIC_URL = "https://" + AWS_STORAGE_BUCKET_NAME + ".s3.amazonaws.com/"
MEDIA_URL = STATIC_URL + "media/"
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
STATIC_ROOT = 'staticfiles'
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)
I have a working Django site where I've been using whitenoise to serve static files.
I've then switched to S3 storage for both site static and media. And it works really well.
I have one problem with this setup though: collecting the admin static files (select2, tinyMCE, etc) takes lots of times and regularly fails in Heroku.
So here's my ideal scenario:
keep everything as is BUT
DO NOT upload the admin static files to S3 (during collectsatic) and serve them from the app server using whitenoise
So how do I achieve that?
Here's the relevant excerpt from my settings file
ENVIRONMENT = os.environ.get('DJANGO_ENVIRONMENT', 'dev')
IS_PROD = ENVIRONMENT == 'production'
IS_DEV = not IS_PROD
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DEBUG = bool(os.environ.get('DJANGO_DEBUG', IS_DEV))
# Static files (CSS, JavaScript, Images) and media
STATIC_URL = '/static/'
STATIC_ROOT = '.static'
if IS_PROD:
AWS_STORAGE_BUCKET_NAME = os.environ.get('S3_BUCKET', '')
AWS_S3_REGION_NAME = os.environ.get('S3_REGION', '')
AWS_ACCESS_KEY_ID = os.environ.get('S3_KEY_ID', '')
AWS_SECRET_ACCESS_KEY = os.environ.get('S3_SECRET', '')
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'
AWS_S3_OBJECT_PARAMETERS = {
'Expires': 'Thu, 31 Dec 2100 20:00:00 GMT',
'CacheControl': 'max-age=94608000',
}
STATICFILES_LOCATION = 'static'
STATICFILES_STORAGE = 'custom_storages.StaticStorage'
MEDIAFILES_LOCATION = 'media'
DEFAULT_FILE_STORAGE = 'custom_storages.MediaStorage'
MEDIA_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/{MEDIAFILES_LOCATION}/'
TINYMCE_JS_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/static/tiny_mce/tiny_mce.js'
else:
MEDIA_URL = '/media/'
MEDIA_ROOT = '.media'
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
]
I'm trying to configure my django setup to serve static files in debug and production mode. I want in production mode to be from S3 and in debug mode to be from the local installation. I have the settings as below:
COMPRESS_ENABLED = not DEBUG
COMPRESS_PARSER = 'compressor.parser.LxmlParser'
COMPRESS_CSS_FILTERS = ['compressor.filters.cssmin.CSSMinFilter']
COMPRESS_JS_FILTERS = ['compressor.filters.jsmin.JSMinFilter']
COMPRESS_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
DEFAULT_FILE_STORAGE = 'libs.storages.S3Storage.S3Storage'
AWS_ACCESS_KEY_ID = # Key
AWS_SECRET_ACCESS_KEY = # Secret
AWS_STORAGE_BUCKET_NAME = # Bucket
AWS_QUERYSTRING_AUTH = False
MEDIA_URL = '/media/'
if COMPRESS_ENABLED:
from boto.s3.connection import SubdomainCallingFormat
AWS_S3_CALLING_FORMAT = SubdomainCallingFormat()
COMPRESS_OFFLINE = True
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
# Path for all static files
STATIC_URL = 'https://%s.s3.amazonaws.com/' % AWS_STORAGE_BUCKET_NAME
STATIC_ROOT = STATIC_URL
else:
STATIC_ROOT = path.join(PROJECT_ROOT, '..', 'assets')
STATIC_URL = '/static/'
I'm able to get everything to work except one tiny thing which I'm not able to figure out. Often times my CSS files will have a background url as /images/logo.png. The folder /images/ exists in S3 but for my local it needs to be /static/images/. I tried to set the S3 URL as /static/ at the end but it wasn't working. Is there anything else i'm missing? How do I get the CSS image urls to redirect to /static/?
I read through the source code of s3boto.py. You can set the following property:
AWS_LOCATION = "static"
That variable creates a static folder in your s3. The other properties should be:
STATIC_URL = 'https://%s.s3.amazonaws.com/static/' % AWS_STORAGE_BUCKET_NAME
STATIC_ROOT = STATIC_URL