I have an Vuejs application running with Django framework, currently application running in production mode with static files are in local server, instead of serving files from location server want to keep in Azure storage
Followed below URL and made the changes to keep media and static files to azure
https://medium.com/#DawlysD/django-using-azure-blob-storage-to-handle-static-media-assets-from-scratch-90cbbc7d56be
Here is my setting.py in djando
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'dist', 'static'),
]
# Added for Azure storage
STATICFILES_STORAGE ='storages.backends.azure_storage.AzureStorage'
DEFAULT_FILE_STORAGE = 'backend.custom_azure.AzureMediaStorage'
AZURE_ACCOUNT_NAME = os.environ['AZURE_ACCOUNT_NAME']
AZURE_ACCOUNT_KEY = os.environ['AZURE_ACCOUNT_KEY']
AZURE_CUSTOM_DOMAIN = f'{AZURE_ACCOUNT_NAME}.blob.core.windows.net'
STATIC_LOCATION = 'static'
STATIC_URL = f'https://{AZURE_CUSTOM_DOMAIN}/{STATIC_LOCATION}/'
AZURE_LOCATION = 'containertest04'
AZURE_CONTAINER = 'containertest04'
MEDIA_LOCATION = "media"
MEDIA_URL = f'https://{AZURE_CUSTOM_DOMAIN}/{MEDIA_LOCATION}/'
Facing 2 issues
1) Getting below error in django log while running "python3.7 manage.py collectstatic" also uploading files from my application
2020-01-04 14:50:59,888 azure.storage.common.storageclient INFO Client-Request-ID=9e944449-2f01-11ea-9a8b-000d3a3be0ea Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Sat, 04 Jan 2020 14:50:59 GMT, Server-Request-ID=a1e7153a-401e-00df-690e-c38686000000, HTTP status code=404, Exception=The specified blob does not exist. ErrorCode: BlobNotFound.
2020-01-04 14:50:59,888 azure.storage.common.storageclient ERROR Client-Request-ID=9e944449-2f01-11ea-9a8b-000d3a3be0ea Retry policy did not allow for a retry: Server-Timestamp=Sat, 04 Jan 2020 14:50:59 GMT, Server-Request-ID=a1e7153a-401e-00df-690e-c38686000000, HTTP status code=404, Exception=The specified blob does not exist. ErrorCode: BlobNotFound.
2) Both static and media files are going to the container containertest04, even though I have created separate container "static" and "media"
Related
This Django app is deployed on Azure as an App Service its static and media files are stored in an Azure storage account - blob.
The project used to work well in the past, but something has changed and now the problem is as following.
Relevant part of the app settings file:
STATIC_URL = 'https://myappstorage.blob.core.windows.net/static/'
MEDIA_URL = 'https://myappstorage.blob.core.windows.net/media/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
# any static paths you want to publish
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static')
]
so one would expect that for example a favicon, which is in the root directory of the storage could be found on https://myappstorage.blob.core.windows.net/static/favicon and it is indeed!
But all static files that the app on azure tries to load it tries to load from
https://myappstorage.myappstorage.blob.core.windows.net/static/
(note the duplication of myappstorage), same for media files.
This results in no static files being applied to the page as they are being loaded from the wrong url. When I run the app locally, it works fine. I destroyed it and recreated it, no success. Now I have two copies running, one deployed through FTP and startup command and on using Github action. Still the same problem.
I have also tried little hack-ish workarounds in the setting file, with no success.
Try with adding below in app setting
DEFAULT_FILE_STORAGE = 'backend.custom_azure.AzureMediaStorage'
STATICFILES_STORAGE = 'backend.custom_azure.AzureStaticStorage'
STATIC_LOCATION = "static"
MEDIA_LOCATION = "media"
AZURE_ACCOUNT_NAME = "djangoaccountstorage"
AZURE_CUSTOM_DOMAIN = f'{AZURE_ACCOUNT_NAME}.blob.core.windows.net'
STATIC_URL = f'https://{AZURE_CUSTOM_DOMAIN}/{STATIC_LOCATION }/'
MEDIA_URL = f'https://{AZURE_CUSTOM_DOMAIN}/{MEDIA_LOCATION}
refer this document for more details
using Django 3
I followed the Django Doc
https://docs.djangoproject.com/en/3.0/ref/contrib/staticfiles/#manifeststaticfilesstorage
to export my static files with a hash appending.
settings.py production
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
static_root folder (output)
static_root/
staticfiles.json
static_root/css/
project_styles.87c2920e7bc3.css
project_styles.css
everything is collected correctly.
Afterwards i uploaded everything to my apache static server.
And i set off / comment the STATICFILES_STORAGE . That is how i understand the Doc´s? If i leave this setting on in production i get an 500 Error.
settings.py production
# STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
After restarting my Django app in production, my site is still loading project_styles.css but not the hash Version project_styles.87c2920e7bc3.css in my browser. Even if i delete project_styles.css Django will not serve the hash version.
Question
Did i miss some settings in the settings.py in production mode?
In the Doc´s they mention to set STATICFILES_STORAGE = django.contrib.staticfiles.storage.StaticFilesStorage but it shows no difference. And as it is mentioned it´s only for testing.
What i have to do to load the correct static hash version in production? do i have to set something in my templates, so that django will look into the json file for the correct hash version? Or do i have to name the hash file?
Alright, the Problem was that i wanted two different STATIC_ROOT Paths. One for Development and one for Production, because i want all my Development stuff in one Project folder. Because if you collectstatic with the STATIC_ROOT of your apache server, django will export it for instance into c:/var/www/your/server/static while i wanted it to c:/webprojects/myproject_1/static_root_exports and later upload these files on my server separately.
So i set two different Path depending on DEV_STATIC off / on in my django-environ file. Django will set the correct Path.
.env
DEBUG=off
# --- applies media server & sets MEDIA_ROOT & STATIC_ROOT
DEV_STATIC=on
<...>
STATIC_ROOT_DEV=static_root_exports
STATIC_ROOT_PROD=/var/www/myUserName/html/myproject_assets/static
<...>
setting.py
# -- Set for Hash
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
# --- STATIC_ROOT
if DEV_STATIC == True:
STATIC_ROOT = SERVER_DIR.joinpath(env('STATIC_ROOT_DEV'))
else:
STATIC_ROOT = env('STATIC_ROOT_PROD')
After deployment today, for some reason, the project no longer shows style from items that would have been collected during the collectstatic process. I have full access to the files from any browser. I have checked all permissions. I have browsed to the site from the development server, and other machines to eliminate software\font possibilities. No idea what's going on here.
I serve the files from a different server. Other django projects are unaffected. No other django projects use anything like wagtail though. Pulling my hair out at this point and probably just missing something simple. What am I missing?
base.py config
STATIC_ROOT = '/var/www/html/static.xxxxxx.net'
STATIC_URL = 'http://static.xxxxxx.net/'
MEDIA_ROOT = '/var/www/html/media.xxxxxx.net'
MEDIA_URL = 'http://media.xxxxxx.net/'
Checking for file existence on server:
-rw-rw-r-- 1 xxxxxx xxxxxx 13648 Aug 24 09:18 /var/www/html/static.xxxxxx.net/wagtailadmin/css/userbar.3d5222a7eba2.css
Checking CSS relative references
-rw-rw-r-- 1 xxxxxx xxxxxx 68812 Aug 24 09:18 /var/www/html/static.xxxxxx.net/wagtailadmin/css/../../wagtailadmin/fonts/opensans-regular.45f80416d702.woff2
Django Debug Toolbar shows 4 static files used for both the production and development environments. Everything is identical.
In the chrome inspect view, if I replace a relative CSS stylesheet link in the development environment with a link from the file server, it breaks the same way.
From:
/static/wagtailadmin/css/userbar.css
To:
http://static.xxxxxx.net/wagtailadmin/css/userbar.css
Again, I can stick that address in my browser, any browser, and I see the stylesheet. I really have no idea how my file server could be stopping browsers from processing CSS, but that's what it's starting to look like.
Update: in the chrome inspect view, if I remove a css reference that uses a stylesheet from my fileserver, the page loses all the style\colors\etc. If I reapply the link to my server, it applys the styles again. It seems to apply all the style except icons\glyphs.
Update 2: If I change to STATIC_URL = '/static/' I get the styles... Until I turn debug back off :-/
Wagtail's recommended setup for STATIC_ROOT, STATIC_URL, MEDIA_ROOT, and MEDIA_URL is typically much simpler and the ..._URL declarations are typically not absolute references. I set things up in base.py like this:
PROJECT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
BASE_DIR = os.path.dirname(PROJECT_DIR)
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
]
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
STATICFILES_DIRS = [
os.path.join(os.path.join(BASE_DIR, directory_structure_down_to..., 'static'),
--- include as many lines like above as necessary here ---
]
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
If you use wagtail-react-streamfield, update it. The font wagtail.woff is duplicated there and it gets picked up first (before wagtail's original one) by Django's static subsystem when collecting statics.
Here is the application hosted on heroku + aws s3 used for storing only media files.
When user upload's a images the error occures .
CertificateError at /create/
hostname 'shuboy.media.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'
Request Method: POST
Request URL: http://shuboy2015.herokuapp.com/create/
Django Version: 1.9.6
Exception Type: CertificateError
Exception Value:
hostname 'shuboy.media.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'
Exception Location: /app/.heroku/python/lib/python2.7/ssl.py in match_hostname, line 271
Python Executable: /app/.heroku/python/bin/python
Python Version: 2.7.10
Python Path:
['/app',
'/app/.heroku/python/bin',
'/app/.heroku/python/lib/python2.7/site-packages/setuptools-20.4-py2.7.egg',
'/app/.heroku/python/lib/python2.7/site-packages/pip-8.1.1-py2.7.egg',
'/app',
'/app/.heroku/python/lib/python27.zip',
'/app/.heroku/python/lib/python2.7',
'/app/.heroku/python/lib/python2.7/plat-linux2',
'/app/.heroku/python/lib/python2.7/lib-tk',
'/app/.heroku/python/lib/python2.7/lib-old',
'/app/.heroku/python/lib/python2.7/lib-dynload',
'/app/.heroku/python/lib/python2.7/site-packages']
Server time: Tue, 7 Jun 2016 20:54:31 +0000
In settings.py file , I only want to store media files there no static files.
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_S3_SECURE_URLS = False
AWS_QUERYSTRING_AUTH = False
AWS_S3_ACCESS_KEY_ID = '*******'
AWS_S3_SECRET_ACCESS_KEY = '******'
AWS_STORAGE_BUCKET_NAME ='shuboy.media'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = '/static/'
MEDIA_URL = 'https://%s/media/' % AWS_S3_CUSTOM_DOMAIN
STATICFILES_DIRS = [
os.path.join(BASE_DIR, "appname/static"),
]
STATIC_ROOT = os.path.join(BASE_DIR, "STATIC_CDN")
MEDIA_ROOT = '%s.s3.amazonaws.com/media/' % AWS_STORAGE_BUCKET_NAME
STATICFILES_STORAGE = 'whitenoise.django.GzipManifestStaticFilesStorage'
If anybody tell me why this error is occurring and how can I solve this issue ? Any further suggestions will be appreciable .
The above error is happening because your bucket name has a "." in it.
The work around for this issue is go to ~/.boto configuration file search for https_validate_certificates and set it to False it will work.
Or remove the "." from your bucket name for it to work correctly.
The boto package doesn't fully support S3 buckets with dots in the name, because of SSL certificate scope.
Either create and use a new bucket without a dot, like "shuboymedia", or add this monkey patch in your settings:
import ssl
if hasattr(ssl, '_create_unverified_context'):
ssl._create_default_https_context = ssl._create_unverified_context
I have just installed Grunt in my Django app. In my blogengine app I have the folder: assets/css/global.scss. Grunt minifies this .scss file to static/css/global.css.
I am still developing the app locally. I have been running grunt sass and watch to minify the scss file to css as I'm working on it.
However, I've set the static url, etc to be my Amazon S3 bucket. This means when I run collectstatic, I have to wait ages for it to upload to S3 so I can see my changes.
I am wanting to eventually deploy this to Heroku, but in the meantime, how do I set my static content to work locally and setup production settings to use S3?
This is in settings.py:
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.6/howto/static-files/
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXXX'
AWS_SECRET_ACCESS_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
AWS_STORAGE_BUCKET_NAME = 'ingledow'
STATIC_URL = 'http://ingledow.s3.amazonaws.com/'
You could mess with the DEBUG settings. In local development, set DEBUG to True, and Django will handle serving all the static files. Once you push to production, set DEBUG to False and the S3 settings will kick in. You could have different settings files or you could set an environment variable both locally and on Heroku and call it in your settings (i.e.: `DEBUG = os.environ['DEBUG'].
In your bashrc, set a environment flag:
alias DJANGO_ENV=local
(Alternatively, just do this in your local shell: export DJANGO_ENV=local)
Then in settings.py:
import os
if os.environ.get( 'DJANGO_ENV', '' ) == 'local':
# SETUP LOCAL SETTINGS
else:
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXXX'
AWS_SECRET_ACCESS_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
AWS_STORAGE_BUCKET_NAME = 'ingledow'
STATIC_URL = 'http://ingledow.s3.amazonaws.com/'
Turn off the local settings when doing pushstatic (eg. "unset DJANGO_ENV"). On production (ie. Heroku), you won't have the DJANGO_ENV system variable, so it will default to AWS files.