Django and AWS S3: Static File Requests to Wrong URL - django
I'm trying to serve a Django 2.1 with Python 3.7 application with React 16.6 frontend to AWS Elastic Beanstalk. I build the React app using Create React App.
So far I succesfully got the server running (I serve React's index.html via a TemplateView) and connected with the database. I'm now kinda stuck connecting the S3 bucket to my template view.
Here is the issue:
python manage.py collectstatic
Worked correctly (when I access the default Django admin page the login is correctly styled), but when I try to access one of my React routes, the page is just blank. Using the chrome dev tools I was able to deduce that index.html was correctly loaded, but that the network request fails with:
GET http://<my-eb-url>.elasticbeanstalk.com/static/js/main.a1cf6ce7.chunk.js net::ERR_ABORTED 404 (Not Found)
So apparently the error is that Django doesn't look for React in the bucket, but in a relative file path. How can I change this? I mean the Python configuration should be good, since collectstatic worked.
Here is React's index.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<link rel="shortcut icon" href="/favicon.ico" />
<meta
name="viewport"
content="minimum-scale=1,initial-scale=1,width=device-width,shrink-to-fit=no"
/>
<meta name="theme-color" content="#000000" />
<link
rel="stylesheet"
href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700|Roboto+Slab:400,700"
/>
<!-- <link rel="stylesheet" href="/fonts/fonts.css" /> -->
<link rel="manifest" href="/manifest.json" />
<title>Pontem</title>
<link href="/static/css/main.0ebf21be.chunk.css" rel="stylesheet" />
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<script>
!(function(l) {
function e(e) {
for (var r, t, n = e[0], o = e[1], u = e[2], f = 0, i = []; f < n.length; f++)
(t = n[f]), p[t] && i.push(p[t][0]), (p[t] = 0);
for (r in o) Object.prototype.hasOwnProperty.call(o, r) && (l[r] = o[r]);
for (s && s(e); i.length; ) i.shift()();
return c.push.apply(c, u || []), a();
}
function a() {
for (var e, r = 0; r < c.length; r++) {
for (var t = c[r], n = !0, o = 1; o < t.length; o++) {
var u = t[o];
0 !== p[u] && (n = !1);
}
n && (c.splice(r--, 1), (e = f((f.s = t[0]))));
}
return e;
}
var t = {},
p = { 2: 0 },
c = [];
function f(e) {
if (t[e]) return t[e].exports;
var r = (t[e] = { i: e, l: !1, exports: {} });
return l[e].call(r.exports, r, r.exports, f), (r.l = !0), r.exports;
}
(f.m = l),
(f.c = t),
(f.d = function(e, r, t) {
f.o(e, r) || Object.defineProperty(e, r, { enumerable: !0, get: t });
}),
(f.r = function(e) {
"undefined" != typeof Symbol &&
Symbol.toStringTag &&
Object.defineProperty(e, Symbol.toStringTag, { value: "Module" }),
Object.defineProperty(e, "__esModule", { value: !0 });
}),
(f.t = function(r, e) {
if ((1 & e && (r = f(r)), 8 & e)) return r;
if (4 & e && "object" == typeof r && r && r.__esModule) return r;
var t = Object.create(null);
if (
(f.r(t),
Object.defineProperty(t, "default", { enumerable: !0, value: r }),
2 & e && "string" != typeof r)
)
for (var n in r)
f.d(
t,
n,
function(e) {
return r[e];
}.bind(null, n)
);
return t;
}),
(f.n = function(e) {
var r =
e && e.__esModule
? function() {
return e.default;
}
: function() {
return e;
};
return f.d(r, "a", r), r;
}),
(f.o = function(e, r) {
return Object.prototype.hasOwnProperty.call(e, r);
}),
(f.p = "/");
var r = (window.webpackJsonp = window.webpackJsonp || []),
n = r.push.bind(r);
(r.push = e), (r = r.slice());
for (var o = 0; o < r.length; o++) e(r[o]);
var s = n;
a();
})([]);</script
><script src="/static/js/1.f9c0bd2f.chunk.js"></script
><script src="/static/js/main.a1cf6ce7.chunk.js"></script>
</body>
</html>
Appendix
I feel like the error is not caused by Python. But just because it might help, here are my python settings. Also my pip packages.
base.py:
"""
Django settings for hrdinner project.
"""
import os
import sys
from pathlib import Path
from django.core.exceptions import ImproperlyConfigured
def get_env_variable(var_name):
"""Get the environment variable or return exception."""
try:
return os.environ[var_name]
except KeyError:
error_msg = 'Set the {} environment variable'.format(var_name)
raise ImproperlyConfigured(error_msg)
# Build paths inside the project like this: BASE_DIR / 'media'
BASE_DIR = Path(__file__).resolve().parent.parent
# Tell Django where to look for the apps.
sys.path.append(str(BASE_DIR.parent / 'src'))
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = get_env_variable('DJANGO_SECRET_KEY')
ALLOWED_HOSTS = []
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'model_utils',
'authtools',
'rest_framework',
'rest_framework.authtoken',
# my apps ...
]
ROOT_URLCONF = 'config.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [BASE_DIR.parent / 'build'],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'config.wsgi.application'
ATOMIC_REQUESTS = True
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"
# Password validation
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
AUTHENTICATION_BACKENDS = (
# Needed to login by username in Django admin, regardless of `allauth`
'django.contrib.auth.backends.ModelBackend',
)
LOGIN_URL = "/"
DEBUG_PROPAGATE_EXCEPTIONS = False
# Internationalization
# https://docs.djangoproject.com/en/1.11/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
STATICFILES_DIRS = [BASE_DIR.parent / 'build' / 'static']
# Security
X_FRAME_OPTIONS = 'DENY'
# django-authtools configuration
AUTH_USER_MODEL = 'accounts.User'
# django-rest-framework configuration
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.TokenAuthentication',
'rest_framework.authentication.BasicAuthentication',
'rest_framework.authentication.SessionAuthentication',
),
'DEFAULT_PERMISSION_CLASSES': (
'rest_framework.permissions.IsAuthenticated',
)
}
production.py:
from .aws.conf import *
from .base import *
DEBUG = False
INSTALLED_APPS += [
'storages',
]
ALLOWED_HOSTS += [
# my urls
]
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': get_env_variable('RDS_DB_NAME'),
'USER': get_env_variable('RDS_USERNAME'),
'PASSWORD': get_env_variable('RDS_PASSWORD'),
'HOST': get_env_variable('RDS_HOSTNAME'),
'PORT': get_env_variable('RDS_PORT'),
}
}
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
SESSION_COOKIE_SECURE = True
SESSION_COOKIE_HTTPONLY = True
CSRF_COOKIE_SECURE = True
CSRF_COOKIE_HTTPONLY = True
SECURE_SSL_HOST = True
# TODO: CHANGE THIS TO A YEAR ONCE YOUR ARE READY! 5 minutes for testing.
SECURE_HSTS_SECONDS = 300
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
SECURE_CONTENT_TYPE_NOSNIFF = True
conf.py:
import os
from django.core.exceptions import ImproperlyConfigured
def get_env_variable(var_name):
"""Get the environment variable or return exception."""
try:
return os.environ[var_name]
except KeyError:
error_msg = 'Set the {} environment variable'.format(var_name)
raise ImproperlyConfigured(error_msg)
AWS_ACCESS_KEY_ID = get_env_variable("ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = get_env_variable("SECRET_ACCESS_KEY")
AWS_S3_SIGNATURE_VERSION = 's3v4'
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_STORAGE_BUCKET_NAME = get_env_variable("AWS_BUCKET_NAME")
AWS_S3_CUSTOM_DOMAIN = '%s.s3.eu-central-1.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_LOCATION = 'static'
STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)
STATICFILES_STORAGE = 'config.settings.aws.utils.StaticRootS3BotoStorage'
DEFAULT_FILE_STORAGE = 'config.settings.aws.utils.MediaRootS3BotoStorage'
MEDIA_URL = 'https://%s.s3.amazonaws.com/media/' % AWS_STORAGE_BUCKET_NAME
MEDIA_ROOT = MEDIA_URL
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
utils.py:
from storages.backends.s3boto3 import S3Boto3Storage
def StaticRootS3BotoStorage(): return S3Boto3Storage(location='static')
def MediaRootS3BotoStorage(): return S3Boto3Storage(location='media')
requirements.txt:
awsebcli==3.14.6
blessed==1.15.0
boto==2.49.0
boto3==1.9.42
botocore==1.12.42
cached-property==1.5.1
cement==2.8.2
certifi==2018.10.15
chardet==3.0.4
colorama==0.3.9
Django==2.1.2
django-annoying==0.10.4
django-authtools==1.6.0
django-model-utils==3.1.2
django-storages==1.7.1
djangorestframework==3.9.0
docker==3.5.1
docker-compose==1.21.2
docker-pycreds==0.3.0
dockerpty==0.4.1
docopt==0.6.2
docutils==0.14
idna==2.6
isort==4.3.4
jmespath==0.9.3
jsonschema==2.6.0
pathspec==0.5.5
psycopg2==2.7.5
python-dateutil==2.7.5
pytz==2018.6
PyYAML==3.13
requests==2.18.4
s3transfer==0.1.13
semantic-version==2.5.0
termcolor==1.1.0
texttable==0.9.1
urllib3==1.22
wcwidth==0.1.7
websocket-client==0.54.0
Your browser is trying fetch your stylesheets and javascript files from a relative path because you specified one in your index.html:
<script src="/static/js/1.f9c0bd2f.chunk.js"></script>
Since you are using Django to collect your static files, you can also use its template tag to reference static files in your templates:
<script src="{% static 'js/1.f9c0bd2f.chunk.js' %}"></script>
Related
DJANGO Templates dont share static files
I have a currently have a problem. I got two django templates:cart.html and index.html. In the cart.html there is a javascript file linked. Both of these files should access the static files in the static folder.
The problem now is that, when I try to acces the static files from the linked cart.html javascript file it gives me this error:
[15/Aug/2022 20:14:40] "GET /cart/images/fruits/Kornknacker.jpg HTTP/1.1" 404 2419
settings.py
BASE_DIR = Path(__file__).resolve().parent.parent
DEBUG = True
ALLOWED_HOSTS = []
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
]
STATIC_URL = 'static/'
STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'),)
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'SemmelBrothers.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [BASE_DIR, 'template'],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'SemmelBrothers.wsgi.application'
Here is the project structure:
Here is the javascript file:
{% load static %}
function loadCart() {
let productsSection = document.getElementById("products_section");
productsSection.innerHTML = '';
let productHTML = '';
let totalPrice = 0;
let cartItems = JSON.parse(localStorage.getItem('cart'));
if (cartItems && cartItems.length > 0) {
for (let item of cartItems) {
totalPrice = totalPrice + (item.quantity * item.price);
productHTML = productHTML + `
<div class="product-card" data-name="${item.itemName}" data-price="${item.price}" data-id="${item.itemId}">
<div>
<img src="{%static '/images/fruits/${item.itemName}.jpg'%}" alt="FRUIT" width="180">
</div>
<h3>
${item.itemName}
</h3>
<div>
Anzahl: ${item.quantity}
</div>
<div>
Preis: ${item.quantity * item.price}€
</div>
</div>
`;
}
document.getElementById("total_price_container").style.display = 'block';
document.getElementById("total_price").innerHTML = totalPrice;
document.getElementById("no-products_section").style.display = 'none';
document.getElementById("checkout-section").style.display = 'flex';
document.getElementById("order-process_section").style.display = 'none';
productsSection.innerHTML = productHTML;
}
else {
document.getElementById("no-products_section").style.display = 'block';
document.getElementById("checkout-section").style.display = 'none';
document.getElementById("total_price_container").style.display = 'none';
}
};
loadCart();
document.getElementById('checkout_cart').addEventListener('click', function () {
console.log(localStorage);
localStorage.removeItem('cart');
document.getElementById("products_section").innerHTML = '';
document.getElementById("order-process_section").style.display = 'block';
document.getElementById("checkout-section").style.display = 'none';
})
document.getElementById('clear_cart').addEventListener('click', function () {
localStorage.removeItem('cart');
loadCart();
})
the problem is that django can't didn't find (404)
"Kornknacker.jpg"
in
"static/cart/images/fruits/Kornknacker.jpg"
But you should probably point it to
"static/images/fruits/Kornknacker.jpg"
if that's the case try Im not sure if it will actually work😂😂:
<img src="{%static '/../images/fruits/${item.itemName}.jpg'%}" alt="FRUIT" width="180">
cannot post a new DB to apache-superset, 400 Error with CSRF
I am unsure how I can set the CSRF token so apache superset is happy.. here is my code
Create a DB Connection for Apache Superset
import requests
import json
base_url = "http://SOMEIP:8088/"
login_url = base_url + "api/v1/security/login"
csrf_url = base_url + "api/v1/security/csrf_token"
def create_db_connection(session, db_ip, db_port, db_user, db_pass, db_name):
url = base_url + "api/v1/database/"
sqlalchemy_url = "postgresql://" + db_user + ":" + db_pass + "#" + db_ip + ":" + str(db_port)+"/" + db_name
data_out = {
"allow_csv_upload": True,
"allow_ctas": True,
"allow_cvas": True,
"allow_dml": True,
"allow_multi_schema_metadata_fetch": True,
"allow_run_async": True,
"cache_timeout": 0,
"database_name": db_name,
"expose_in_sqllab": True,
"impersonate_user": True,
"sqlalchemy_uri": sqlalchemy_url
}
response = session.post(url=url, headers=dict(Referrer=login_url), json=data_out)
response.raise_for_status()
return response
if __name__ == "__main__":
username = "mysupersetuser"
password = "mysupersetpass"
db_user = "MYUSER"
db_password = "MYDBPASS"
db_host = "MYPOSTGRESIP"
db_port = 5432
db_name = "MYDBNAME"
session = requests.session()
login_url = base_url + "api/v1/security/login"
login_data = {
"password": password,
"provider": "db",
"refresh": False,
"username": username
}
response = session.post(url=login_url, json=login_data)
response.raise_for_status()
head = {
"Authorization": "Bearer " + json.loads(response.text)['access_token']
}
response = session.get(csrf_url, headers=head)
response.raise_for_status()
response = create_db_connection(session, db_ip=db_host, db_port=db_port, db_user=db_user, db_pass=db_password, db_name=db_name)
print(str(response.text))
I see multiple ways people are stating to set the CSRF token.. but none have worked so far.
Always it's the same response 400
'{"errors": [{"message": "400 Bad Request: The CSRF token is missing.", "error_type": "GENERIC_BACKEND_ERROR", "level": "error", "extra": {"issue_codes": [{"code": 1011, "message": "Issue 1011 - Superset encountered an unexpected error."}]}}]}'
It looks like Apache-Superset uses Flask-AppBuilder which uses Flask_WTF. You should be able to set it in the request headers.
def create_db_connection(session, db_ip, db_port, db_user, db_pass, db_name, token_from_your_crsf_url):
headers = {
'Referrer': login_url,
'X-CSRFToken': token_from_your_crsf_url
}
...
response = session.post(url=url, headers=headers, json=data_out)
...
How to use PythonAnywhere SFTP to store uploaded files to get past file size limitations in Django Filer?
I am currently working on a Django website project in PythonAnywhere where users can upload files using Django Filer in the admin. I have successfully implemented it with the public Django filer settings. However, it seems I cannot upload files larger than 256 Mb. Is using the SFTP server provided to PythonAnywhere users an alternative way to store files? Or are there other ways to get past this size limitation? I am unsure how to write the server into my code. The SFTP address format and settings file for filer is provided below.
SFTP Address and Hostname Format
username#ssh.pythonanywhere.com
filer.settings.py
# -*- coding: utf-8 -*-
from __future__ import absolute_import
import logging
import os
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
from django.core.files.storage import get_storage_class
from .utils.loader import load_object
from .utils.recursive_dictionary import RecursiveDictionaryWithExcludes
logger = logging.getLogger(__name__)
# FILER_IMAGE_MODEL setting is used to swap Image model.
# If such global setting does not exist, it will be created at this point (with default model name).
# This is needed especially when using this setting in migrations.
if not hasattr(settings, 'FILER_IMAGE_MODEL'):
setattr(settings, 'FILER_IMAGE_MODEL', 'filer.Image')
FILER_IMAGE_MODEL = settings.FILER_IMAGE_MODEL
FILER_DEBUG = getattr(settings, 'FILER_DEBUG', False) # When True makes
FILER_SUBJECT_LOCATION_IMAGE_DEBUG = getattr(settings, 'FILER_SUBJECT_LOCATION_IMAGE_DEBUG', False)
FILER_WHITESPACE_COLOR = getattr(settings, 'FILER_WHITESPACE_COLOR', '#FFFFFF')
FILER_0_8_COMPATIBILITY_MODE = getattr(settings, 'FILER_0_8_COMPATIBILITY_MODE', False)
FILER_ENABLE_LOGGING = getattr(settings, 'FILER_ENABLE_LOGGING', False)
if FILER_ENABLE_LOGGING:
FILER_ENABLE_LOGGING = (
FILER_ENABLE_LOGGING and (getattr(settings, 'LOGGING')
and ('' in settings.LOGGING['loggers']
or 'filer' in settings.LOGGING['loggers'])))
FILER_ENABLE_PERMISSIONS = getattr(settings, 'FILER_ENABLE_PERMISSIONS', False)
FILER_ALLOW_REGULAR_USERS_TO_ADD_ROOT_FOLDERS = getattr(settings, 'FILER_ALLOW_REGULAR_USERS_TO_ADD_ROOT_FOLDERS', False)
FILER_IS_PUBLIC_DEFAULT = getattr(settings, 'FILER_IS_PUBLIC_DEFAULT', True)
FILER_PAGINATE_BY = getattr(settings, 'FILER_PAGINATE_BY', 20)
_ICON_SIZES = getattr(settings, 'FILER_ADMIN_ICON_SIZES', ('16', '32', '48', '64'))
if not _ICON_SIZES:
raise ImproperlyConfigured('Please, configure FILER_ADMIN_ICON_SIZES')
# Reliably sort by integer value, but keep icon size as string.
# (There is some code in the wild that depends on this being strings.)
FILER_ADMIN_ICON_SIZES = [str(i) for i in sorted([int(s) for s in _ICON_SIZES])]
# Filer admin templates have specific icon sizes hardcoded: 32 and 48.
_ESSENTIAL_ICON_SIZES = ('32', '48')
if not all(x in FILER_ADMIN_ICON_SIZES for x in _ESSENTIAL_ICON_SIZES):
logger.warn(
"FILER_ADMIN_ICON_SIZES has not all of the essential icon sizes "
"listed: {}. Some icons might be missing in admin templates.".format(
_ESSENTIAL_ICON_SIZES))
# This is an ordered iterable that describes a list of
# classes that I should check for when adding files
FILER_FILE_MODELS = getattr(
settings, 'FILER_FILE_MODELS',
(FILER_IMAGE_MODEL, 'filer.File'))
DEFAULT_FILE_STORAGE = getattr(settings, 'DEFAULT_FILE_STORAGE', 'django.core.files.storage.FileSystemStorage')
MINIMAL_FILER_STORAGES = {
'public': {
'main': {
'ENGINE': None,
'OPTIONS': {},
},
'thumbnails': {
'ENGINE': None,
'OPTIONS': {},
}
},
'private': {
'main': {
'ENGINE': None,
'OPTIONS': {},
},
'thumbnails': {
'ENGINE': None,
'OPTIONS': {},
},
},
}
DEFAULT_FILER_STORAGES = {
'public': {
'main': {
'ENGINE': DEFAULT_FILE_STORAGE,
'OPTIONS': {},
'UPLOAD_TO': 'filer.utils.generate_filename.randomized',
'UPLOAD_TO_PREFIX': 'filer_public',
},
'thumbnails': {
'ENGINE': DEFAULT_FILE_STORAGE,
'OPTIONS': {},
'THUMBNAIL_OPTIONS': {
'base_dir': 'filer_public_thumbnails',
},
},
},
'private': {
'main': {
'ENGINE': 'filer.storage.PrivateFileSystemStorage',
'OPTIONS': {
'location': os.path.abspath(os.path.join(settings.MEDIA_ROOT, '../smedia/filer_private')),
'base_url': '/smedia/filer_private/',
},
'UPLOAD_TO': 'filer.utils.generate_filename.randomized',
'UPLOAD_TO_PREFIX': '',
},
'thumbnails': {
'ENGINE': 'filer.storage.PrivateFileSystemStorage',
'OPTIONS': {
'location': os.path.abspath(os.path.join(settings.MEDIA_ROOT, '../smedia/filer_private_thumbnails')),
'base_url': '/smedia/filer_private_thumbnails/',
},
'THUMBNAIL_OPTIONS': {},
},
},
}
MINIMAL_FILER_SERVERS = {
'private': {
'main': {
'ENGINE': None,
'OPTIONS': {},
},
'thumbnails': {
'ENGINE': None,
'OPTIONS': {},
},
},
}
DEFAULT_FILER_SERVERS = {
'private': {
'main': {
'ENGINE': 'filer.server.backends.default.DefaultServer',
'OPTIONS': {},
},
'thumbnails': {
'ENGINE': 'filer.server.backends.default.DefaultServer',
'OPTIONS': {},
},
},
}
FILER_STORAGES = RecursiveDictionaryWithExcludes(MINIMAL_FILER_STORAGES, rec_excluded_keys=('OPTIONS', 'THUMBNAIL_OPTIONS'))
if FILER_0_8_COMPATIBILITY_MODE:
user_filer_storages = {
'public': {
'main': {
'ENGINE': DEFAULT_FILE_STORAGE,
'UPLOAD_TO': 'filer.utils.generate_filename.randomized',
'UPLOAD_TO_PREFIX': getattr(settings, 'FILER_PUBLICMEDIA_PREFIX', 'filer_public'),
},
'thumbnails': {
'ENGINE': DEFAULT_FILE_STORAGE,
'OPTIONS': {},
'THUMBNAIL_OPTIONS': {
'base_dir': 'filer_public_thumbnails',
},
},
},
'private': {
'main': {
'ENGINE': DEFAULT_FILE_STORAGE,
'UPLOAD_TO': 'filer.utils.generate_filename.randomized',
'UPLOAD_TO_PREFIX': getattr(settings, 'FILER_PRIVATEMEDIA_PREFIX', 'filer_private'),
},
'thumbnails': {
'ENGINE': DEFAULT_FILE_STORAGE,
'OPTIONS': {},
'THUMBNAIL_OPTIONS': {
'base_dir': 'filer_private_thumbnails',
},
},
},
}
else:
user_filer_storages = getattr(settings, 'FILER_STORAGES', {})
FILER_STORAGES.rec_update(user_filer_storages)
def update_storage_settings(user_settings, defaults, s, t):
if not user_settings[s][t]['ENGINE']:
user_settings[s][t]['ENGINE'] = defaults[s][t]['ENGINE']
user_settings[s][t]['OPTIONS'] = defaults[s][t]['OPTIONS']
if t == 'main':
if 'UPLOAD_TO' not in user_settings[s][t]:
user_settings[s][t]['UPLOAD_TO'] = defaults[s][t]['UPLOAD_TO']
if 'UPLOAD_TO_PREFIX' not in user_settings[s][t]:
user_settings[s][t]['UPLOAD_TO_PREFIX'] = defaults[s][t]['UPLOAD_TO_PREFIX']
if t == 'thumbnails':
if 'THUMBNAIL_OPTIONS' not in user_settings[s][t]:
user_settings[s][t]['THUMBNAIL_OPTIONS'] = defaults[s][t]['THUMBNAIL_OPTIONS']
return user_settings
update_storage_settings(FILER_STORAGES, DEFAULT_FILER_STORAGES, 'public', 'main')
update_storage_settings(FILER_STORAGES, DEFAULT_FILER_STORAGES, 'public', 'thumbnails')
update_storage_settings(FILER_STORAGES, DEFAULT_FILER_STORAGES, 'private', 'main')
update_storage_settings(FILER_STORAGES, DEFAULT_FILER_STORAGES, 'private', 'thumbnails')
FILER_SERVERS = RecursiveDictionaryWithExcludes(MINIMAL_FILER_SERVERS, rec_excluded_keys=('OPTIONS',))
FILER_SERVERS.rec_update(getattr(settings, 'FILER_SERVERS', {}))
def update_server_settings(settings, defaults, s, t):
if not settings[s][t]['ENGINE']:
settings[s][t]['ENGINE'] = defaults[s][t]['ENGINE']
settings[s][t]['OPTIONS'] = defaults[s][t]['OPTIONS']
return settings
update_server_settings(FILER_SERVERS, DEFAULT_FILER_SERVERS, 'private', 'main')
update_server_settings(FILER_SERVERS, DEFAULT_FILER_SERVERS, 'private', 'thumbnails')
# Public media (media accessible without any permission checks)
FILER_PUBLICMEDIA_STORAGE = get_storage_class(FILER_STORAGES['public']['main']['ENGINE'])(**FILER_STORAGES['public']['main']['OPTIONS'])
FILER_PUBLICMEDIA_UPLOAD_TO = load_object(FILER_STORAGES['public']['main']['UPLOAD_TO'])
if 'UPLOAD_TO_PREFIX' in FILER_STORAGES['public']['main']:
FILER_PUBLICMEDIA_UPLOAD_TO = load_object('filer.utils.generate_filename.prefixed_factory')(FILER_PUBLICMEDIA_UPLOAD_TO, FILER_STORAGES['public']['main']['UPLOAD_TO_PREFIX'])
FILER_PUBLICMEDIA_THUMBNAIL_STORAGE = get_storage_class(FILER_STORAGES['public']['thumbnails']['ENGINE'])(**FILER_STORAGES['public']['thumbnails']['OPTIONS'])
FILER_PUBLICMEDIA_THUMBNAIL_OPTIONS = FILER_STORAGES['public']['thumbnails']['THUMBNAIL_OPTIONS']
# Private media (media accessible through permissions checks)
FILER_PRIVATEMEDIA_STORAGE = get_storage_class(FILER_STORAGES['private']['main']['ENGINE'])(**FILER_STORAGES['private']['main']['OPTIONS'])
FILER_PRIVATEMEDIA_UPLOAD_TO = load_object(FILER_STORAGES['private']['main']['UPLOAD_TO'])
if 'UPLOAD_TO_PREFIX' in FILER_STORAGES['private']['main']:
FILER_PRIVATEMEDIA_UPLOAD_TO = load_object('filer.utils.generate_filename.prefixed_factory')(FILER_PRIVATEMEDIA_UPLOAD_TO, FILER_STORAGES['private']['main']['UPLOAD_TO_PREFIX'])
FILER_PRIVATEMEDIA_THUMBNAIL_STORAGE = get_storage_class(FILER_STORAGES['private']['thumbnails']['ENGINE'])(**FILER_STORAGES['private']['thumbnails']['OPTIONS'])
FILER_PRIVATEMEDIA_THUMBNAIL_OPTIONS = FILER_STORAGES['private']['thumbnails']['THUMBNAIL_OPTIONS']
FILER_PRIVATEMEDIA_SERVER = load_object(FILER_SERVERS['private']['main']['ENGINE'])(**FILER_SERVERS['private']['main']['OPTIONS'])
FILER_PRIVATEMEDIA_THUMBNAIL_SERVER = load_object(FILER_SERVERS['private']['thumbnails']['ENGINE'])(**FILER_SERVERS['private']['thumbnails']['OPTIONS'])
# By default limit number of simultaneous uploads if we are using SQLite
if settings.DATABASES['default']['ENGINE'].endswith('sqlite3'):
_uploader_connections = 1
else:
_uploader_connections = 3
FILER_UPLOADER_CONNECTIONS = getattr(
settings, 'FILER_UPLOADER_CONNECTIONS', _uploader_connections)
FILER_DUMP_PAYLOAD = getattr(settings, 'FILER_DUMP_PAYLOAD', False) # Whether the filer shall dump the files payload
FILER_CANONICAL_URL = getattr(settings, 'FILER_CANONICAL_URL', 'canonical/')
THUMBNAIL_HIGH_RESOLUTION = True
THUMBNAIL_PROCESSORS = (
'easy_thumbnails.processors.colorspace',
'easy_thumbnails.processors.autocrop',
#'easy_thumbnails.processors.scale_and_crop',
'filer.thumbnail_processors.scale_and_crop_with_subject_location',
'easy_thumbnails.processors.filters',
)
You probably should not allow users to upload via sftp (because then you would be giving them your pythonanywhere username and password). PythonAnywhere does not allow uploading >100mb files- so you could either chunk it up into smaller pieces and upload it piece by piece, or perhaps upload directly to say an aws s3 bucket.
django-pipeline FileNotFoundError when i perform manage.py collectstatic
It's last some lines of traceback :
File "C:\PycharmDev\TestVirtualEnv\testEnv1\lib\site-packages\pipeline\compressors\__init__.py", line 247, in execute_command
stdin=subprocess.PIPE, stderr=subprocess.PIPE)
File "C:\Python34\Lib\subprocess.py", line 859, in __init__
restore_signals, start_new_session)
File "C:\Python34\Lib\subprocess.py", line 1112, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] ...
And in settings.py :
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATIC_URL = '/static/'
STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'
STATIC_URL = '/static_prepared/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
STATICFILES_DIRS = [
os.path.join(BASE_DIR, 'static_prepared'),
]
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'pipeline.finders.PipelineFinder',
)
PIPELINE = {
'STYLESHEETS': {
'static_prepared': {
'source_filenames': (
'*.css',
),
'output_filename': 'colors.css'
},
},
'JAVASCRIPT': {
'static_prepared': {
'source_filenames': (
'*.js',
),
'output_filename': 'stats.js',
}
}
}
What's wrong with me?
When i use manage.py collectstatic, it doesn't work at all with pipeline.
It seems like only works like there's no pipeline.
The same as without pipeline. Normal manage.py collectstatic.
Why that error has been occurred?
I already made static and static_prepared directory on os.path.join(BASE_DIR, ...) manually.
How can i set pipeline correctly?
Replace other name like
PIPELINE = {
'STYLE': {
'static_prepared': {
'source_filenames': (
'*.css',
),
'output_filename': 'colors.css'
},
},
'JSCRIPT': {
'static_prepared': {
'source_filenames': (
'*.js',
),
'output_filename': 'stats.js',
}
}
}
django - am I hitting a memory limit with my list?
I'm trying to inline my stylesheets with the following code:
views.py
def index(request):
latest_course_list = Course.objects.order_by('-start_date')
template = loader.get_template('index.html')
style_sheets = ""
style_sheets = addStyle(style_sheets)
ctx = {
'latest_course_list': latest_course_list,
'style_sheets': style_sheets,
}
return render_to_response('index.html', ctx, context_instance=RequestContext(request))
def addStyle(style):
style_sheets=""
style_sheet_list = [
"bootstrap",
"custom-style",
"responsive",
"animate",
"flexslider",
"theme-style"
]
for sheet in style_sheet_list:
with open (DJANGO_ROOT + "/assets/css/" + sheet + ".css", "r") as myfile:
style_sheets+=myfile.read()
style_sheets+="\n"
return style_sheets
template:
<style type="text/css">{{ style_sheets }}</style>
But it only gets down to half way through the animate.css before cutting off and just adding "..."
What limit am I hitting here? How can I solve it?