How can i solve CertificateError by AWS -S3 in Django? - django

Here is the application hosted on heroku + aws s3 used for storing only media files.
When user upload's a images the error occures .
CertificateError at /create/
hostname 'shuboy.media.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'
Request Method: POST
Request URL: http://shuboy2015.herokuapp.com/create/
Django Version: 1.9.6
Exception Type: CertificateError
Exception Value:
hostname 'shuboy.media.s3.amazonaws.com' doesn't match either of '*.s3.amazonaws.com', 's3.amazonaws.com'
Exception Location: /app/.heroku/python/lib/python2.7/ssl.py in match_hostname, line 271
Python Executable: /app/.heroku/python/bin/python
Python Version: 2.7.10
Python Path:
['/app',
'/app/.heroku/python/bin',
'/app/.heroku/python/lib/python2.7/site-packages/setuptools-20.4-py2.7.egg',
'/app/.heroku/python/lib/python2.7/site-packages/pip-8.1.1-py2.7.egg',
'/app',
'/app/.heroku/python/lib/python27.zip',
'/app/.heroku/python/lib/python2.7',
'/app/.heroku/python/lib/python2.7/plat-linux2',
'/app/.heroku/python/lib/python2.7/lib-tk',
'/app/.heroku/python/lib/python2.7/lib-old',
'/app/.heroku/python/lib/python2.7/lib-dynload',
'/app/.heroku/python/lib/python2.7/site-packages']
Server time: Tue, 7 Jun 2016 20:54:31 +0000
In settings.py file , I only want to store media files there no static files.
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_S3_SECURE_URLS = False
AWS_QUERYSTRING_AUTH = False
AWS_S3_ACCESS_KEY_ID = '*******'
AWS_S3_SECRET_ACCESS_KEY = '******'
AWS_STORAGE_BUCKET_NAME ='shuboy.media'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = '/static/'
MEDIA_URL = 'https://%s/media/' % AWS_S3_CUSTOM_DOMAIN
STATICFILES_DIRS = [
os.path.join(BASE_DIR, "appname/static"),
]
STATIC_ROOT = os.path.join(BASE_DIR, "STATIC_CDN")
MEDIA_ROOT = '%s.s3.amazonaws.com/media/' % AWS_STORAGE_BUCKET_NAME
STATICFILES_STORAGE = 'whitenoise.django.GzipManifestStaticFilesStorage'
If anybody tell me why this error is occurring and how can I solve this issue ? Any further suggestions will be appreciable .

The above error is happening because your bucket name has a "." in it.
The work around for this issue is go to ~/.boto configuration file search for https_validate_certificates and set it to False it will work.
Or remove the "." from your bucket name for it to work correctly.

The boto package doesn't fully support S3 buckets with dots in the name, because of SSL certificate scope.
Either create and use a new bucket without a dot, like "shuboymedia", or add this monkey patch in your settings:
import ssl
if hasattr(ssl, '_create_unverified_context'):
ssl._create_default_https_context = ssl._create_unverified_context

Related

Getting BlobNotFound while configuring django static and media files

I have an Vuejs application running with Django framework, currently application running in production mode with static files are in local server, instead of serving files from location server want to keep in Azure storage
Followed below URL and made the changes to keep media and static files to azure
https://medium.com/#DawlysD/django-using-azure-blob-storage-to-handle-static-media-assets-from-scratch-90cbbc7d56be
Here is my setting.py in djando
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'dist', 'static'),
]
# Added for Azure storage
STATICFILES_STORAGE ='storages.backends.azure_storage.AzureStorage'
DEFAULT_FILE_STORAGE = 'backend.custom_azure.AzureMediaStorage'
AZURE_ACCOUNT_NAME = os.environ['AZURE_ACCOUNT_NAME']
AZURE_ACCOUNT_KEY = os.environ['AZURE_ACCOUNT_KEY']
AZURE_CUSTOM_DOMAIN = f'{AZURE_ACCOUNT_NAME}.blob.core.windows.net'
STATIC_LOCATION = 'static'
STATIC_URL = f'https://{AZURE_CUSTOM_DOMAIN}/{STATIC_LOCATION}/'
AZURE_LOCATION = 'containertest04'
AZURE_CONTAINER = 'containertest04'
MEDIA_LOCATION = "media"
MEDIA_URL = f'https://{AZURE_CUSTOM_DOMAIN}/{MEDIA_LOCATION}/'
Facing 2 issues
1) Getting below error in django log while running "python3.7 manage.py collectstatic" also uploading files from my application
2020-01-04 14:50:59,888 azure.storage.common.storageclient INFO Client-Request-ID=9e944449-2f01-11ea-9a8b-000d3a3be0ea Operation failed: checking if the operation should be retried. Current retry count=0, Server-Timestamp=Sat, 04 Jan 2020 14:50:59 GMT, Server-Request-ID=a1e7153a-401e-00df-690e-c38686000000, HTTP status code=404, Exception=The specified blob does not exist. ErrorCode: BlobNotFound.
2020-01-04 14:50:59,888 azure.storage.common.storageclient ERROR Client-Request-ID=9e944449-2f01-11ea-9a8b-000d3a3be0ea Retry policy did not allow for a retry: Server-Timestamp=Sat, 04 Jan 2020 14:50:59 GMT, Server-Request-ID=a1e7153a-401e-00df-690e-c38686000000, HTTP status code=404, Exception=The specified blob does not exist. ErrorCode: BlobNotFound.
2) Both static and media files are going to the container containertest04, even though I have created separate container "static" and "media"

django s3 storages does not recognise the bucket name

I am using django-s3-storage==0.11.2 and boto3==1.4.4. These are in the settings.py:
STATIC_URL = '/static/'
STATICFILES_DIRS = [
os.path.join(BASE_DIR, "static"),
]
STATIC_ROOT = os.path.join(BASE_DIR, 'static_cdn')
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media_cdn')
AWS_S3_BUCKET_NAME = "my-bucket-name"
AWS_ACCESS_KEY_ID = 'test_id_x'
AWS_SECRET_ACCESS_KEY = 'test_id_x+test_id_x'
DEFAULT_FILE_STORAGE = "django_s3_storage.storage.S3Storage"
STATICFILES_STORAGE = "django_s3_storage.storage.StaticS3Storage"
AWS_S3_ADDRESSING_STYLE = "auto"
AWS_S3_BUCKET_AUTH_STATIC = False
AWS_S3_MAX_AGE_SECONDS_STATIC = 60 * 60 * 24 * 365 # 1 year.
AWS_S3_BUCKET_AUTH = False
AWS_S3_MAX_AGE_SECONDS = 60 * 60 * 24 * 365 # 1 year.
I have also ran these command:
manage.py s3_sync_meta django.core.files.storage.default_storage
But when I run collectstatic or this command
manage.py s3_sync_meta django.contrib.staticfiles.storage.staticfiles_storage
I get this error:
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"
I have already created the bucket, and the bucket name is correct. Because this works and does not gives any error:
s3.meta.client.head_bucket(Bucket='my-bucket-name')
I don't know what am I missing here? Could you help me out please.
Alright, it looks confusing for me too.
Below are my observations -
1 . Bucket name pattern
Bucket name should not have '/' in them .
It would be good if you can update the AWS_S3_BUCKET_NAME from
"my-bucket-name" to the pattern which actually resembles with your
bucket name.
Source: https://github.com/boto/botocore/issues/680
2 . In the Django S3 Storage Documentation , it says
If your are updating a project that used django-storages
just for S3 file storage, migration is trivial.
Follow the installation instructions, replacing 'storages' in INSTALLED_APPS.
Be sure to scrutinize the rest of your settings file for changes,
most notably AWS_S3_BUCKET_NAME for AWS_STORAGE_BUCKET_NAME.
Can you please try to change AWS_S3_BUCKET_NAME_STATIC = bass-line-shop in your settings.py ?
Let me know, if it helps!

S3ResponseError: 301 Moved Permanently - Django

I have a very big issue with Amazon S3. I am working on a Django app and I want to store file on S3:
My settings are:
AWS_STORAGE_BUCKET_NAME = 'tfjm2-inscriptions'
AWS_ACCESS_KEY_ID = 'id'
AWS_SECRET_ACCESS_KEY = 'key'
AWS_S3_CUSTOM_DOMAIN = '%s.s3-eu-west-1.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
And I get this error: S3ResponseError: 301 Moved Permanently
Some same issues on the Internet say that it is because it is a non-US bucket and I did tried with a US-standard bucket but it get a 401 Forbidden error.....
I do not know what to do.
Please help me.
Thank you
You can do this:
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
in terminal run 'nano ~/.boto'
if there is some configs try to comment or rename file and connect again. (it helps me)
http://boto.cloudhackers.com/en/latest/boto_config_tut.html
there is boto config file directories. take a look one by one and clean them all, it will work by default configs. also configs may be in .bash_profile, .bash_source...
I guess you must allow only KEY-SECRET
Solve by changing your code to:
AWS_STORAGE_BUCKET_NAME = 'tfjm2-inscriptions'
AWS_ACCESS_KEY_ID = 'id'
AWS_SECRET_ACCESS_KEY = 'key'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_REGION_NAME = 'us-east-2' ##### Use the region name where your bucket is created
AWS allow you to access a create and access buckets in the same region as an optimization measure and as such if you create a bucket in say 'us-west-2' you'll get a 301 if try accessing it from a different region (Africa, Europe and even East US).
You should specify the region if requesting the bucket from outside its region.

Can't get the TinyMCE to work in django admin

I follow the instruction in here to install the TinyMCE into the django Admin backend.
But it is not working. When checking the console log, I saw this:
http://127.0.0.1:8000/media/js/tiny_mce/tiny_mce.js Failed to load
DO I need to manually adding the js file? The instruction in the github does not mention this.
UPDATE
Indeed to make it work, will have to move the tiny_mce to your static folder.
Here is my solution for anyone who also have similar problem.
settins.py
STATIC_URL = '/static/'
STATIC_ROOT = ''
STATICFILES_DIRS = (
os.path.join(BASE_DIR, "static"),
)
#this is for localhost development, if you are in production enviroment, you will need to remove the STATICFILES_DIRS and define your STATIC_ROOT
TINYMCE_DEFAULT_CONFIG = {
'plugins' : "pagebreak,style,layer,table,save,advhr,advimage,advlink,emotions,iespell,inlinepopups,insertdatetime,preview,media,searchreplace,print,contextmenu,paste,directionality,fullscreen,noneditable,visualchars,nonbreaking,xhtmlxtras,template,wordcount,advlist,autosave,pagebreak",
'theme': "advanced",
'theme_advanced_buttons1' : "bold,italic,underline,strikethrough,|,justifyleft,justifycenter,justifyright,justifyfull,fontselect,fontsizeselect,fullscreen,code,|,preview,image,media",
'theme_advanced_buttons2' : "table,|,bullist,numlist,|,outdent,indent,blockquote,|,undo,redo,|,link,unlink,|,forecolor,backcolor, emotions,|,pagebreak,paste",
'theme_advanced_buttons3 ': "",
'theme_advanced_toolbar_location' : "top",
'theme_advanced_toolbar_align' : "left",
'width': '700',
'height': '400'
}
admin.py
class AdminPost(admin.ModelAdmin):
class Media:
js = ('/static/js/tiny_mce/tiny_mce.js',)
Django TinyMCE has the media url as default value, as you can see in the docs:
TINYMCE_JS_URL (default: settings.MEDIA_URL + 'js/tiny_mce/tiny_mce.js')
TINYMCE_JS_ROOT (default: settings.MEDIA_ROOT + 'js/tiny_mce')
If you prefer to use the static files in the static folder, you have to set these values to the correct path. I'd sugest:
TINYMCE_JS_URL = settings.STATIC_URL + 'js/tiny_mce/tiny_mce.js'
TINYMCE_JS_ROOT = settings.STATIC_ROOT + 'js/tiny_mce'
Now you have to ensure you're using the "django.contrib.staticfiles.finders.AppDirectoriesFinder" in your STATICFILES_FINDERS settings, in order to not have to copy the files in development environment, and to be collected with collectstatic.
There are 2 possible reasons for this issue.
1.) You have the required files but on a different location than specified in your script tag. Change the url in your script tag to the valid location and it will work
2.) You do not have the required files. Download the source files and place them in the specified location and it will work.
You should try:-
First uninstalling tinymce
pip uninstall django-tinymce4
and Then re-installing tinymce
It worked for me

How to use S3 for Production on Heroku and local static css for development?

I have just installed Grunt in my Django app. In my blogengine app I have the folder: assets/css/global.scss. Grunt minifies this .scss file to static/css/global.css.
I am still developing the app locally. I have been running grunt sass and watch to minify the scss file to css as I'm working on it.
However, I've set the static url, etc to be my Amazon S3 bucket. This means when I run collectstatic, I have to wait ages for it to upload to S3 so I can see my changes.
I am wanting to eventually deploy this to Heroku, but in the meantime, how do I set my static content to work locally and setup production settings to use S3?
This is in settings.py:
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.6/howto/static-files/
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXXX'
AWS_SECRET_ACCESS_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
AWS_STORAGE_BUCKET_NAME = 'ingledow'
STATIC_URL = 'http://ingledow.s3.amazonaws.com/'
You could mess with the DEBUG settings. In local development, set DEBUG to True, and Django will handle serving all the static files. Once you push to production, set DEBUG to False and the S3 settings will kick in. You could have different settings files or you could set an environment variable both locally and on Heroku and call it in your settings (i.e.: `DEBUG = os.environ['DEBUG'].
In your bashrc, set a environment flag:
alias DJANGO_ENV=local
(Alternatively, just do this in your local shell: export DJANGO_ENV=local)
Then in settings.py:
import os
if os.environ.get( 'DJANGO_ENV', '' ) == 'local':
# SETUP LOCAL SETTINGS
else:
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXXX'
AWS_SECRET_ACCESS_KEY = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxx'
AWS_STORAGE_BUCKET_NAME = 'ingledow'
STATIC_URL = 'http://ingledow.s3.amazonaws.com/'
Turn off the local settings when doing pushstatic (eg. "unset DJANGO_ENV"). On production (ie. Heroku), you won't have the DJANGO_ENV system variable, so it will default to AWS files.