Wagtail and Django are very efficient and sophisticated frameworks, but they do not offer any simple way of importing (i.e bulk creating) pages. I have tried dozens of suggestions to do so (eg 1, eg 2, eg 3, eg 4, etc). I've also tried CodeRedCMS (which is based on Wagtail), but it offers a very rudimentary function and I do not want to add additional dependencies. Gazmans solution in this post seemed very efficient but I could not figure out exactly how to go about solving the issue.
So my question is simple: If I have a csv/json file with thousands of rows, where each row represents a post for a specific Page Model I've created in Wagtail, how do I import it to Wagtail? Any example with psuedo code would be immensely appreciated.
The solutions that I almost succeed with were by (1) either using DataGrip and directly import data into the table, or (2) using the loaddata management command in Django. However, both these methods resulted in Wagtail not registering the entries, such that I could verify that they existed in the database table but not in the parent table and not visible in the admin backend, which I believe is due to multi-table inheritance.
Here's a recipe which should point you in the right direction:
https://gist.github.com/tomdyson/ef8c2f684620b84feaddfd7454e09647
This shows the creation of an example Film page model for Wagtail, along with a management command which creates ~35k pages from a downloaded CSV file.
You could also check out my wagtail-fakenews app, which generates Wagtail pages with dummy content. It doesn't import from CSV, but it does include images and StreamField content (see make_fake_items.py), which the first recipe doesn't.
Related
I'm trying some Django + React stuff and I'm a bit confused with the particular role of Django model in this scheme.
As I can understand Django model provide smooth way to create some typical forms by users of the site (please correct me). And I can't feel the edge of what should be hardcoded and what I need to store in model.
In case of React (I'm trying to connect react with django through api via djangorestframework) should I create a model for Header? And store bg image with it and my slogans. I feel it's should be purely in frontend. But next section with 3 typical screens (they will be listed horizontally and swap each other). They are just copy of each other with different data (title, bg, fg, text, link), for me it seems closer to model usage. But model initially empty and if I want to get this data from model I firstly need to somehow store this data to model.
So in general my question is what the right cases for using Django models and when it's no needed. And if it possible with applying to my example to better understanding for me )
ofc I searched this info widely but so far can't create clear understanding by myself.
Thanks )
You might actually be in search of a headless CMS.
To combine React and Django and still use a CMS to allow administrating text blocks and images, Django Wagtail's StreamField is a good choice.
https://docs.wagtail.io/en/v2.0/topics/streamfield.html
See for example our company's homepage: https://www.blu-beyond.com. It runs on Django Wagtail, having JS animated text blocks that are administered in the CMS (just jQuery and other JS libs, no React, in this case).
Django Wagtail offers a JSON API, as well, that can be used in React:
https://docs.wagtail.io/en/v2.8/advanced_topics/api/v2/usage.html#fetching-content
It also offers a GraphQL API.
I have a django project with 3 models. The user, the project (in which he works on) and the Change (which logs start and finish working time as well as the project and user).
I want to export a custom CSV report. Ex: total working hours of users per project, total hours devoted to a project etc. This means that there are some calculations to be done across models before exporting. So far I have found out how to export in CSV just filtered model entries, which isn't very helpful. I also found some tools online but most of them are outdated.
Can anyone point me to a direction or give me advice or links where I can learn more? Thank you.
Couple of ways to do this. You can use django-report-builder which is a pretty neat tool.
Other ways are using custom views to admin site (check Django AdminPlus), overwrite queryset and use custom admin action etc.
we are trying to migrate a project written with Drupal to Django CMS and we faced a problem with article module. Our site is divided in sections and we have a news module installed in every section with a category, url structure is looking like this:
/section1
/news-category1
/section2
/news-category2
/etc..
This is the same news module, just split in categories (some news articles can pop up in multiple sections, in this case one section is chosen as base to form unique article URL). The only one method I found makes this structure:
/news
/caregory1
/category2
/etc...
Which is not good for us as we would prefer to keep the current URL structure for SEO purposes. Is there a correct way to implement this in Django CMS beside creating each section as a module and plugging in in to a page? Or can I some-how install the same module to multiple pages and pass the section information to it?
One way to do this I found myself would be to plug-in the same module to every page it will be on and then have it to parse the path of the page to figure out it's category. Not super-officiant but might work. Not sure if there is any other way.
I've been playing around with python/django for the last couple of weeks and whilst the overall structure and makeup of the framework is making sense I'm rather confused on how to create advanced interfaces (in relation to tasks administrators would perform). One trivial example I'm playing around with at the moment is a bulk csv product import for different suppliers which will update various fields of a particular product (keeping track of any changes), creating items where they don't already exist and applying other business logic etc.
With the data successfully in the database and the models reflecting this I envisage a view whereby one could select a supplier from a drop down, which would load all the products silently in the background and display a datagrid on success. The user could then interact with each product individually, for example selecting would display a stacked line chart of pricing history above the datagrid and an optional fly-in panel to the right with options to update prices, add notes etc.
Are there any best practice approaches for achieving something along these lines, does one create custom views/templates or put some heavy lifting into overriding the default Django admin functionality?
Any help is appreciated, thanks in advance.
You can either:
Create a custom django admin action that will appear as an option in the changelist page dropdown menu (of the Supplier model for example.) You can then apply this action to the selected rows. You can also have intermediate pages when using admin actions
You can hook your own views into the django admin for particular models and and then overwrite the appropriate django admin templates to link the two together
What would be the best way to port an existing Drupal site to a Django application?
I have around 500 pages (mostly books module) and around 50 blog posts. I'm not using any 3rd party modules.
I would like to keep the current URLS (for SEO purposes) and migrate database to Django. I will create a simple blog application, so migrating blog posts should be ok. What would be the best way to serve 500+ pages with Django? I would like to use Admin to edit/add new pages.
All Django development is similar, and yours will fit the pattern.
Define the Django model for your books and blog posts.
Unit test that model using Django's built-in testing capabilities.
Write some small utilities to load your legacy data into Django. At this point, you'll realize that your Django model isn't perfect. Good. Fix it. Fix the tests. Redo the loads.
Configure the default admin interface to your model. At this point, you'll spend time tweaking the admin interface. You'll realize your data model is wrong. Which is a good thing. Fix your model. Fix your tests. Fix your loads.
Now that your data is correct, you can create templates from your legacy pages.
Create URL mappings and view functions to populate the templates from the data model.
Take the time to get the data model right. It really matters, because everything else is very simple if your data model is solid.
It may be possible to write Django models which work with the legacy database (I've done this in the past; see docs on manage.py inspectdb).
However, I'd follow advice above and design a clean database using Django conventions, and then migrate the data over. I usually write migration scripts which write to the new database through Django and read the old one using the raw Python DB APIs (while it is possible to tie Django to multiple databases simultaneously, too).
I also suggest taking a look at the available blogging apps for Django. If the one included in Pinax suits your need, go ahead and use Pinax as a starting point.
S.Lott answer is still valid after years, I try to complete the analysis with the tools and format to do the job.
There are many Drupal export tools out of there by now but with the very same request I go for Views Datasource choosing JSON as format. This module is very solid and available for the last version of Drupal. The JSON format is very fast in both parsing and encoding and it's easy to read and very Python-friendly (import json).
Using Views Datasource you can create a node view sorted by node id (nid), show a limited number of elements per page, configure a view path, add to it a filter identifier and pass to it the nid to read all elements until you get an empty JSON response.
When importing in Django you have a wide set of tools as well, starting from loaddata to load fixtures. Views Datasource exported JSON but it's not formatted as Django expects fixtures: you can write a custom admin command to do the import, where you can have the full control of the import flow.
You can start your command passing a nid=0 as argument and then let the procedure read, import and then fetch data from the next page passing simply the last nid read in the previous HTTP request. You can even restrict access to the path on view but you need additional configuration on the import side.
Regarding performance, just for example I parsed and imported 15.000+ nodes in less than 10 minutes via a Django 1.8 custom admin command on an 8 core / 8 GB Linux virtual machine and PostgreSQL as DBMS, logging success and error information into a custom model for each node.
These are the basics for import/export between these two platform, for detailed information I described all the major steps for export from Drupal and then import to Django in this guide.