I am working with SQLAlchemy and Flask-Admin right now. Let us say that I am trying to get a String of the file name or a String of the absolute path of the file name of an image file. I am trying to insert either of these into a database automatically when I create a user. The code that I am using is structured similar to this.
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
url_pic = Column(String(50), nullable=False)
pic = Column(LargeBinary, nullable=False)
...
from flask.ext.admin.contrib.sqla import ModelView
from flask.ext.admin.form.upload import FileUploadField
from wtforms.validators import ValidationError
from flask.ext.admin import Admin
from flask.ext.sqlalchemy import SQLAlchemy
from flask import Flask
import imghdr
app = Flask(__name__)
db = SQLAlchemy(app)
class UserAdminView(ModelView):
def picture_validation(form, field):
if field.data:
filename = field.data.filename
if filename[-4:] != '.jpg':
raise ValidationError('file must be .jpg')
if imghdr.what(field.data) != 'jpeg':
raise ValidationError('file must be a valid jpeg image.')
field.data = field.data.stream.read()
return True
form_columns = ['id','url_pic', 'pic']
column_labels = dict(id='ID', url_pic="Picture's URL", pic='Picture')
def pic_formatter(view, context, model, name):
return 'NULL' if len(getattr(model, name)) == 0 else 'a picture'
column_formatters = dict(pic=pic_formatter)
form_overrides = dict(pic= FileUploadField)
form_args = dict(pic=dict(validators=[picture_validation]))
admin = Admin(app)
admin.add_view(UserAdminView(User, db.session, category='Database Administration'))
...
How could we get a string version of the file name or absolute path when using the picture_validation function? Right now, the function only provides the binary data of the file, which isn’t as useful.
I am using django-rest-framework in conjuntion with django-ckeditor. I'm serving some images with absolute url-s without any problem. But images and files uploaded by ckeditor are served as relative paths, and they can't be displayed client side since it is in a different domain.
Here is an example of what I'm getting:
{
image: "http://example.com/media/myimage.png",
body: "<p>download my file</p>"
}
And this is what I woul like to get:
{
image: "http://example.com/media/myimage.png",
body: "<p>download my file</p>"
}
Edit:
This would be the model of my example:
from django.db import models
from ckeditor_uploader.fields import RichTextUploadingField
image: models.ImageField()
body: RichTextUploadingField(blank=True,null=True)
I would use a custom serializer to fix that:
from rest_framework import serializers
def relative_to_absolute(url):
return 'http://127.0.0.1:8000' + url
class FileFieldSerializer(serializers.Field):
def to_representation(self, value):
url = value.url
if url and url.startswith('/'):
url = relative_to_absolute(url)
return url
When filefield.url contains a relative url, relative_to_absolute() is called to prepend the domain.
Here I just used a constant string; you can either save it in your settings, or, if Django Site framework is installed, retrieve it as follows:
from django.contrib.sites.models import Site
domain=Site.objects.get_current().domain
Sample usage of the custom serializer:
class Picture(BaseModel):
...
image = models.ImageField(_('Image'), null=True, blank=True)
...
class PictureSerializer(serializers.ModelSerializer):
image = FileFieldSerializer()
class Meta:
model = Picture
fields = '__all__'
Variation for RichTextUploadingField
If, on the other hand, you're using RichTextUploadingField by CKEditor, your field is, basically, a TextField where an HTML fragment is saved upon images
upload.
In this HTML fragment, CKEditor will reference the uploaded images with a relative path, for very good reasons:
your site will still work if the domain is changed
the development instance will work in localhost
after all, we're using Django, not WordPress ;)
So, I wouldn't touch it, and fix the path at runtime in a custom serializer instead:
SEARCH_PATTERN = 'href=\\"/media/ckeditor/'
SITE_DOMAIN = "http://127.0.0.1:8000"
REPLACE_WITH = 'href=\\"%s/media/ckeditor/' % SITE_DOMAIN
class FixAbsolutePathSerializer(serializers.Field):
def to_representation(self, value):
text = value.replace(SEARCH_PATTERN, REPLACE_WITH)
return text
Alternatively, domain can be saved in settings:
from django.conf import settings
REPLACE_WITH = 'href=\\"%s/media/ckeditor/' % settings.SITE_DOMAIN
or retrieved from Django Site framework as follows:
from django.contrib.sites.models import Site
REPLACE_WITH = 'href=\\"{scheme}{domain}/media/ckeditor/'.format(
scheme="http://",
domain=Site.objects.get_current().domain
)
You might need to adjust SEARCH_PATTERN according to your CKEditor configuration; the more specific, the better.
Sample usage:
class Picture(BaseModel):
...
body = RichTextUploadingField(blank=True,null=True)
...
class PictureSerializer(serializers.ModelSerializer):
body = FixAbsolutePathSerializer()
class Meta:
model = Picture
fields = '__all__'
In my Django site there are some permissions entries linked to applications that I've removed. For example I have permissions entries linked to "Dashboard" and "Jet" applications. How can you remove them?
Permissions have foreign keys to content types under the hood, so removing the content types for the models that no longer exist will also remove the permissions for those models.
Fortunately, Django also provides a manage.py command to remove old content types: remove_stale_contenttypes. Running that command will list the content types that no longer exist and the related objects (including permissions) that will be deleted, allowing you to review the changes and approve them.
$ manage.py remove_stale_contenttypes
Some content types in your database are stale and can be deleted.
Any objects that depend on these content types will also be deleted.
The content types and dependent objects that would be deleted are:
- Content type for stale_app.removed_model
- 4 auth.Permission object(s)
This list doesn't include any cascade deletions to data outside of Django's
models (uncommon).
Are you sure you want to delete these content types?
If you're unsure, answer 'no'.
Type 'yes' to continue, or 'no' to cancel:
To start, make an empty migration file:
python manage.py makemigrations --empty yourappname
Change the migration (this is an example, adjust to your needs):
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations
def add_permissions(apps, schema_editor):
pass
def remove_permissions(apps, schema_editor):
"""Reverse the above additions of permissions."""
ContentType = apps.get_model('contenttypes.ContentType')
Permission = apps.get_model('auth.Permission')
content_type = ContentType.objects.get(
model='somemodel',
app_label='yourappname',
)
# This cascades to Group
Permission.objects.filter(
content_type=content_type,
codename__in=('add_somemodel', 'change_somemodel', 'delete_somemodel'),
).delete()
class Migration(migrations.Migration):
dependencies = [
('yourappname', '0001_initial'),
]
operations = [
migrations.RunPython(remove_permissions, add_permissions),
]
I did it this way:
import re
for perm in Permission.objects.all():
if re.match( r".+modelname.+permissionname.+",str(perm)):
print(perm)
perm.delete()
If you have custom or model based (default) permissions you wish to remove you could write a command like this to accomplish this task:
from django.conf import settings
from django.contrib.auth.models import Permission
from django.core.management.base import BaseCommand
import django.apps
class Command(BaseCommand):
help = 'Remove custom permissions that are no longer in models'
def handle(self, *args, **options):
# get the db name needed for removal...
database_name = input('Database Name: ')
default_perm_names = list()
# are real perms in db, may not be accurate
db_custom_perm_names = list()
# will be used to ensure they are correct.
meta_custom_perm_names = list()
default_and_custom_perms = list()
for model in django.apps.apps.get_models():
# add to models found to fix perms from removed models
app_label = model._meta.app_label
lower_model_name = model._meta.model_name
all_model_permissions = Permission.objects.using(database_name).filter(content_type__app_label=app_label, content_type__model=lower_model_name)
default_and_custom_perms.extend([x for x in all_model_permissions])
# get the custom meta permissions, these should be in the meta of the class
# will be a list or tuple or list, [0=codename, 1=name]
meta_permissions = model._meta.permissions
if meta_permissions:
for perm in all_model_permissions:
# will be the model name from the content type, this is how django makes default perms
# we are trying to remove them so now we can figure out which ones are default by provided name
model_name_lower = perm.content_type.name
# default_perms = ['add', 'change', 'view', 'delete', 'undelete']
# append them to the list of default names
default_perm_names.append(f'Can add {model_name_lower}')
default_perm_names.append(f'Can change {model_name_lower}')
default_perm_names.append(f'Can view {model_name_lower}')
default_perm_names.append(f'Can delete {model_name_lower}')
default_perm_names.append(f'Can undelete {model_name_lower}')
# will mean this is a custom perm...so add it
if not perm.name in default_perm_names:
db_custom_perm_names.append(perm.name)
# the perms to ensure are correct...
for model_perm in meta_permissions:
# get the meta perm, will be a list or tuple or list, [0=codename, 1=name]
custom_perm = Permission.objects.using(database_name).get(codename=model_perm[0], name=model_perm[1])
meta_custom_perm_names.append(custom_perm.name)
perms_to_remove = [perm for perm in db_custom_perm_names if perm not in meta_custom_perm_names]
if not perms_to_remove:
print('There are no stale custom permissions to remove.')
# print(perms_to_remove)
# now remove the custom permissions that were removed from the model
for actual_permission_to_remove in Permission.objects.using(database_name).filter(name__in=perms_to_remove):
# print(actual_permission_to_remove)
actual_permission_to_remove.delete(using=database_name)
print(actual_permission_to_remove, '...deleted')
for perm in [x for x in Permission.objects.using(database_name)]:
# loop all perms...if it is not in the model perms it does not exist...
if perm.content_type.model not in [x.content_type.model for x in default_and_custom_perms]:
perm.delete(using=database_name)
print(perm, 'regular permission...deleted')
If you also wish to ensure that the default permissions are added from Django you can add this snippet in the command:
from django.apps import apps
from django.contrib.auth.management import create_permissions
from apps.client.models import ClientInformation
# add all permissions the django way
# get the db name needed from settings.py
database_name = 'default' # or whatever DB you are looking for
print(f'adding all permissions if not there to {database_name}')
for app_config in apps.get_app_configs():
# print(app_config)
app_config.models_module = True
create_permissions(app_config, using=database_name)
app_config.models_module = None
Then call via python manage.py fix_permissions if you name your command file fix_permissions.py
I've reworked #ViaTech's code to use Django's contrib.auth.management._get_all_permissions() functions which makes it more straight forward:
from typing import List, Set, Tuple
import django.apps
# noinspection PyProtectedMember
from django.contrib.auth.management import _get_all_permissions
from django.contrib.auth.models import Permission
from django.contrib.contenttypes.models import ContentType
from django.core.management.base import BaseCommand
from django.db import DEFAULT_DB_ALIAS
class Command(BaseCommand):
help = "Remove custom permissions that are no longer defined in models"
def add_arguments(self, parser):
parser.add_argument(
"--database",
default=DEFAULT_DB_ALIAS,
help=f'Specifies the database to use. Default is "{DEFAULT_DB_ALIAS}".',
)
parser.add_argument(
"--dry",
action="store_true",
help="Do a dry run not actually deleting any permissions",
)
def handle(self, *args, **options) -> str:
using = options["database"]
# This will hold the permissions that models have defined,
# i.e. default permissions plus additional custom permissions:
# (content_type.pk, codename)
defined_perms: List[Tuple[int, str]] = []
for model in django.apps.apps.get_models():
ctype = ContentType.objects.db_manager(using).get_for_model(
model, for_concrete_model=False
)
# noinspection PyProtectedMember
for (codename, _) in _get_all_permissions(model._meta):
defined_perms.append((ctype.id, codename))
# All permissions in current database (including stale ones)
all_perms = Permission.objects.using(using).all()
stale_perm_pks: Set[int] = set()
for perm in all_perms:
if (perm.content_type.pk, perm.codename) not in defined_perms:
stale_perm_pks.add(perm.pk)
self.stdout.write(f"Delete permission: {perm}")
# Delete all stale permissions
if options["dry"]:
result = f"DRY RUN: {len(stale_perm_pks)} stale permissions NOT deleted"
else:
if stale_perm_pks:
Permission.objects.filter(pk__in=stale_perm_pks).delete()
result = f"{len(stale_perm_pks)} stale permissions deleted"
return result
In the Admin console, I can add a group and add a bunch of permissions that relate to my models, e.g.
api | project | Can add project
api | project | Can change project
api | project | Can delete project
How can I do this programmatically. I can't find any information out there on how to do this.
I have:
from django.contrib.auth.models import Group, Permission
from django.contrib.contenttypes.models import ContentType
from api.models import Project
new_group, created = Group.objects.get_or_create(name='new_group')
# Code to add permission to group ???
ct = ContentType.objects.get_for_model(Project)
# Now what - Say I want to add 'Can add project' permission to new_group?
UPDATE: Thanks for the answer you provided. I was able to use that to work out what I needed. In my case, I can do the following:
new_group, created = Group.objects.get_or_create(name='new_group')
proj_add_perm = Permission.objects.get(name='Can add project')
new_group.permissions.add(proj_add_perm)
Use below code
from django.contrib.auth.models import Group, Permission
from django.contrib.contenttypes.models import ContentType
from api.models import Project
new_group, created = Group.objects.get_or_create(name='new_group')
# Code to add permission to group ???
ct = ContentType.objects.get_for_model(Project)
# Now what - Say I want to add 'Can add project' permission to new_group?
permission = Permission.objects.create(codename='can_add_project',
name='Can add project',
content_type=ct)
new_group.permissions.add(permission)
I needed to create a default set of groups and permission (view only) for those groups. I came up with a manage.py command that may be useful to others (create_groups.py). You can add it to your <app>/management/commands dir, and then run via manage.py create_groups:
"""
Create permission groups
Create permissions (read only) to models for a set of groups
"""
import logging
from django.core.management.base import BaseCommand
from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
GROUPS = ['developers', 'devops', 'qa', 'operators', 'product']
MODELS = ['video', 'article', 'license', 'list', 'page', 'client']
PERMISSIONS = ['view', ] # For now only view permission by default for all, others include add, delete, change
class Command(BaseCommand):
help = 'Creates read only default permission groups for users'
def handle(self, *args, **options):
for group in GROUPS:
new_group, created = Group.objects.get_or_create(name=group)
for model in MODELS:
for permission in PERMISSIONS:
name = 'Can {} {}'.format(permission, model)
print("Creating {}".format(name))
try:
model_add_perm = Permission.objects.get(name=name)
except Permission.DoesNotExist:
logging.warning("Permission not found with name '{}'.".format(name))
continue
new_group.permissions.add(model_add_perm)
print("Created default group and permissions.")
UPDATE: A bit more sophisticated now:
from django.contrib.auth.models import Group
from django.contrib.auth.models import User
from django.contrib.auth.models import Permission
from django.contrib.contenttypes.models import ContentType
READ_PERMISSIONS = ['view', ] # For now only view permission by default for all, others include add, delete, change
WRITE_PERMISSIONS = ['add', 'change', 'delete']
EMAIL_USER_DOMAIN = 'your-domain.com'
# Add your groups here, app and model code
GROUP_MODEL = ('auth', 'group')
USER_MODEL = ('auth', 'user')
PERMISSION_MODEL = ('auth', 'permission')
LOG_ENTRY_MODEL = ('admin', 'logentry')
def add_group_permissions(group_names, model_natural_keys, permissions):
"""
Add permissions to the provided groups for the listed models.
Error raised if permission or `ContentType` can't be found.
:param group_names: iterable of group names
:param model_natural_keys: iterable of 2-tuples containing natural keys for ContentType
:param permissions: iterable of str (permission names i.e. add, view)
"""
for group_name in group_names:
group, created = Group.objects.get_or_create(name=group_name)
for model_natural_key in model_natural_keys:
perm_to_add = []
for permission in permissions:
# using the 2nd element of `model_natural_key` which is the
# model name to derive the permission `codename`
permission_codename = f"{permission}_{model_natural_key[1]}"
try:
perm_to_add.append(
Permission.objects.get_by_natural_key(
permission_codename, *model_natural_key
)
)
except Permission.DoesNotExist:
# trying to add a permission that doesn't exist; log and continue
logging.error(
f"permissions.add_group_permissions Permission not found with name {permission_codename!r}."
)
raise
except ContentType.DoesNotExist:
# trying to add a permission that doesn't exist; log and continue
logging.error(
"permissions.add_group_permissions ContentType not found with "
f"natural name {model_natural_key!r}."
)
raise
group.permissions.add(*perm_to_add)
def set_users_group(users, group):
"""
Adds users to specific permission group.
If user or group does not exist, they are created.
Intended for use with special users for api key auth.
:param users: list of str, usernames
:param group: str, group for which users should be added to
:return: list, user objects added to group
"""
users = users or []
user_objs = []
for user_name in users:
try:
user = User.objects.get(username=user_name)
except User.DoesNotExist:
user = User.objects.create_user(username=user_name,
email=f'{user_name}#{EMAIL_USER_DOMAIN}',
password='')
user_objs.append(user)
group, created = Group.objects.get_or_create(name=group)
group.user_set.add(user)
return user_objs
API_READ_GROUP = 'api-read-users'
API_WRITE_GROUP = 'api-write-users'
READ_GROUPS = [API_READ_GROUP, ]
WRITE_GROUPS = [API_WRITE_GROUP, ] # Can be used in same way as read users below
# Adding users to a group
set_users_group(READ_USERS, API_READ_GROUP)
# Setting up the group permissions i.e. read for a group of models
add_group_permissions(READ_GROUPS, [GROUP_MODEL, USER_MODEL, LOG_ENTRY_MODEL], READ_PERMISSIONS)
I also found that using manage.py update_permissions is useful to sort out/clean up stale permissions if models have changed etc.. Its part of django-extensions commands.
Inspired by radtek's answer I created a bit better version (in my opinion).
It allows specifying model as object (instead of string) and specifying all configuration in one dictionary (instead of several lists)
# backend/management/commands/initgroups.py
from django.core.management import BaseCommand
from django.contrib.auth.models import Group, Permission
from backend import models
GROUPS_PERMISSIONS = {
'ConnectionAdmins': {
models.StaticCredentials: ['add', 'change', 'delete', 'view'],
models.NamedCredentials: ['add', 'change', 'delete', 'view'],
models.Folder: ['add', 'change', 'delete', 'view'],
models.AppSettings: ['view'],
},
}
class Command(BaseCommand):
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
help = "Create default groups"
def handle(self, *args, **options):
# Loop groups
for group_name in GROUPS_PERMISSIONS:
# Get or create group
group, created = Group.objects.get_or_create(name=group_name)
# Loop models in group
for model_cls in GROUPS_PERMISSIONS[group_name]:
# Loop permissions in group/model
for perm_index, perm_name in \
enumerate(GROUPS_PERMISSIONS[group_name][model_cls]):
# Generate permission name as Django would generate it
codename = perm_name + "_" + model_cls._meta.model_name
try:
# Find permission object and add to group
perm = Permission.objects.get(codename=codename)
group.permissions.add(perm)
self.stdout.write("Adding "
+ codename
+ " to group "
+ group.__str__())
except Permission.DoesNotExist:
self.stdout.write(codename + " not found")
Taking ideas from the answers of #radtek and #Pavel I created my own version of create_groups.py which I call using python manage.py create_groups. This file is stored on app_name/management/commands/create_groups.py. I created a __init__.py inside each the management and commands folders.
I have created the possibility to control each model permissions separately because I had a group of users called Member that must have different permissions on different models.
I also added the possibility to create users with emails and a default password that would have to be changed afterwards and associate them with a certain group.
from django.core.management import BaseCommand
from django.contrib.auth.models import User, Group , Permission
import logging
GROUPS = {
"Administration": {
#general permissions
"log entry" : ["add","delete","change","view"],
"group" : ["add","delete","change","view"],
"permission" : ["add","delete","change","view"],
"user" : ["add","delete","change","view"],
"content type" : ["add","delete","change","view"],
"session" : ["add","delete","change","view"],
#django app model specific permissions
"project" : ["add","delete","change","view"],
"order" : ["add","delete","change","view"],
"staff time sheet" : ["add","delete","change","view"],
"staff" : ["add","delete","change","view"],
"client" : ["add","delete","change","view"],
},
"Member": {
#django app model specific permissions
"project" : ["view"],
"order" : ["view"],
"staff time sheet" : ["add","delete","change","view"],
},
}
USERS = {
"my_member_user" : ["Member","member#domain.cu","1234*"],
"my_admin_user" : ["Administration","admin#domain.ca","1234"],
"Admin" : ["Administration","superuser#domain.cu","1234"],
}
class Command(BaseCommand):
help = "Creates read only default permission groups for users"
def handle(self, *args, **options):
for group_name in GROUPS:
new_group, created = Group.objects.get_or_create(name=group_name)
# Loop models in group
for app_model in GROUPS[group_name]:
# Loop permissions in group/model
for permission_name in GROUPS[group_name][app_model]:
# Generate permission name as Django would generate it
name = "Can {} {}".format(permission_name, app_model)
print("Creating {}".format(name))
try:
model_add_perm = Permission.objects.get(name=name)
except Permission.DoesNotExist:
logging.warning("Permission not found with name '{}'.".format(name))
continue
new_group.permissions.add(model_add_perm)
for user_name in USERS:
new_user = None
if user_name == "Admin":
new_user, created = User.objects.get_or_create(username=user_name,is_staff = True,is_superuser = True, email = USERS[user_name][1])
else:
new_user, created = User.objects.get_or_create(username=user_name,is_staff = True, email = USERS[user_name][1])
new_user.set_password(USERS[user_name][2])
new_user.save()
if USERS[user_name][0] == str(new_group):
new_group.user_set.add(new_user)
print("Adding {} to {}".format(user_name,new_group))
I've extended the Django cms Page model into ExtendedPage model and added page_image to it.
How can I now acces the page_image property in a template.
I'd like to access the page_image property for every child object in a navigation menu... creating a menu with images...
I've extended the admin and I have the field available for editing (adding the picture)
from django.db import models
from django.utils.translation import ugettext_lazy as _
from cms.models.pagemodel import Page
from django.conf import settings
class ExtendedPage(models.Model):
page = models.OneToOneField(Page, unique=True, verbose_name=_("Page"), editable=False, related_name='extended_fields')
page_image = models.ImageField(upload_to=settings.MEDIA_ROOT, verbose_name=_("Page image"), blank=True)
Thank you!
BR
request.current_page.extended_fields.page_image
should work if you are using < 2.4. In 2.4 they introduced a new two page system (published/draft) so you might need
request.current_page.publisher_draft.extended_fields.page_image
I usually write some middleware or a template processor to handle this instead of doing it repetitively in the template. Something like:
class PageOptions(object):
def process_request(self, request):
request.options = dict()
if not request.options and request.current_page:
extended_fields = None
try:
extended_fields = request.current_page.extended_fields
except:
try:
custom_settings = request.current_page.publisher_draft.extended_fields
except:
pass
if extended_fields:
for field in extended_fields._meta.fields:
request.options[field.name] = getattr(extended_fields, field.name)
return None
will allow you to simply do {{ request.options.page_image }}