I am looking for some advice before I start on a new project.
I am creating a Web Application using Django 1.10. I have experience with Django and creating general "content-based" websites with it. However, since this project is going to be a web-based application, I plan on doing more "complex" things than just rendering HTML templates and doing some basic CRUD operations. When I say "complex" things, the most specific example I can give at this point is to leverage more asynchronous requests so my web-application can remain responsive to the user and provide that "real-time" experience that comes with an application that might be installed on their local machine or whatever. Plus, since this is a web-application and not just a website, the project is definitely going to be more data-driven which could potentially mean requesting large amounts of data that would be best served in say a paginated manner, etc.
So, my though was this.....Since I am familiar with Django and have read such good things about the Django REST Framework, I could create a RESTful API to perform all of my CRUD operations and basically interface with my web application's core database.
At that point, I could essentially have two "layers": (1) A presentation layer that will render my web-application's pages and (2) an application layer that will do all my backend CRUD operations. Since the two are separated, I will also get the added benefit of being able to leverage the API from other avenues (other than my Django web-app) if that is ever required down the road.
I guess my first question is whether or not this makes sense and if so, thoughts on the best way to implement it. I believe I have two options.
Create a single Django project and include the API as a separate application. This seems like it would work fine but it would couple my API with my presentation since they would both be hosted by the same server. Which...if the only "consumer" of the API is my one Django site, then this may not be an issue in the near term but could cause problems later on.
Separate the two into two distinct Django projects. This offers the most flexibility and is probably the best answer, I think.
There may be other options as well. At the end of the day, I am looking for some advice/guidance from those who have done this before and what other things I should consider before starting on this effort.
Basically I would not recommend the way number 2. This is because you really fast will start to struggle with issues like that: where to store models? Project A, Project B, Both?? Also I am almost sure that you will have other code that can be shared between both of the django projects.
Worth to note here that in two separate django projects - you will have troubles with synchronization of the migrations (just consider simultaneously change of the models in both projects). Personally - I've never was able to solve this in some nice, acceptable way.
Maybe I will share with you my personal experience - hope this will be helpful.
First is that if your are making a big application - consider resignation from django templates - use some modern js framework for frontend: react, angular; and make a separate project for fontend. This is tricky because you need to have right competences in the team. If this is not possible - well do separate django app for front.
Use DRF - you can create rest app inside your project and use all of the models defined elsewhere - it will provide you simple CRUD out of the box and with some work it can be really powerful. If you will be able to create front in modern js framework - this REST can be also used to feed data to the front. And also any other client you can imagine. This basically shift the amount of work needed to be done in the backend (only REST instead of REST + templates and rendering) to the frontend.
Asynchronous tasks. Well nothing new here - use celery. Define your tasks inside application; make sure that you have correct number of workers. And let the magic begin.
As for real-time experience, you may consider using django-channels: https://github.com/django/channels/ It is nice way to handle websocket connections.
Things to consideration:
In big application usually you will end up with something like this:
myapp.com -> point to frontend; api.myapp.com -> point to the api
You should make a clear separation in the code between this two. In the case when your front is js-based - this is not a problem, but rest + rendering it is important to be able to run only-api-node and only-rendering-node. It's hard to say which one of them will be used more heavily.
So basically you can end up with something like this:
core
models
users.py
my_app_frontend
users
views.py
forms.py
tasks.py
urls.py
my_app_rest
users
serializers.py
views.py
signals.py
urls.py
my_app
settings.py
rest_settings.py
manage.py
wsgi.py
rest_wsgi.py
urls.py
rest_urls.py
This have some advantages that I like:
Models are in one place;
There is a possibility to run two separates nodes - based on needs.
Your development settings should be able to run dev server as a whole. Combing the settings from front and rest.
There is a clear code separation - and we know what is what in above structure.
If you have any more question - please ask.
Also I am curious what the others have to say - and how they handle issues pointed by me :)
Happy coding!
Related
I've read articles and posts about what a project and an app is for Django, and basically end up using the typical example of Pool and Users, however a real program generally use a complex relational database, therefore its design gravitates around this RDB; and the eternal conflict raises once again about: which ones to consider an application and which one to consider components of that application?
Let's take as an example this RDB (courtesy of Visual Paradigm):
I could consider the whole set as an application or to consider every entity as an application, the outlook looks gray. The only thing I'm sure is about this:
$ django-admin startproject movie_rental
So I wish to learn from the expertise of all of you: What approach (not necessarily those mentioned before) would you use to create applications based on this RDB for a Django project?
Thanks in advance.
PS1: MORE DETAILS RELATED ABOUT MY REQUEST
When programming something I follow this steps:
Understand the context what you are going to program about,
Identify the main actors and objects in this context,
If needed, make an UML diagram,
Design a solid-relational-database diagram, (solid=constraints, triggers, procedures, etc.)
Create the relational database,
Start coding... suffer and enjoy
When I learn something new I hope they follow these same steps to understand where they want to go with their actions.
When reading articles and posts (and viewing videos), almost all of them omit the steps 1 to 5 (because they choose simple demo apps), and when programming they take the easy route, and don't show other situations or the many supposed features that Django offers (reusability, pluggability, etc).
When doing this request, I wish to know what criteria is used for experienced programmers in Django to determine what applications to create based on this sample RDB diagram.
With the (2) answers obtained so far, "application" for...
brandonris1 is about features/services
Jeff Hui is about implementing entities of a DB
James Bennett is about every action on a object, he likes doing a lot of apps
Conclusion so far: Django application is a personal creed.
My initial request was about creating applications, but as models are mentioned, I have this another question: is with a legacy relational database (as showed in the picture) possible to create a Django project with multiple apps? this is because in every Django demo project showed, every app created has a model with their own tables, giving the impression that tables do not interact with those of other applications.
I hope my request is more clear. Thanks again for your help.
It seems you are trying to decide between building a single monolithic application vs microservices. Both approaches have their pros and cons.
For example, a single monolithic application is a good solution if you have a small amount of support resources and do not need to be able to develop new features in fast sprints across the different areas of the application (i.e. Film Management Features vs Staff Management Features)
One major downside to large monolithic applications is that eventually their feature sets grow too large and with each new feature, you have a significant amount of regression testing which will need to be done to ensure there aren't any negative repercussions in other areas of the application.
Your other option is to go with a microservice strategy. In this case, you would divide these entities amongst a series of smaller services and provide them each methods to integrate/communicate with each other (APIs).
Example:
- Film Service
- Customer Service
- Staff Service
The benefits of this approach is it allows you to separate capabilities and features by specific service areas thus reducing risk and regression testing across the application when new features are deployed or there is a catastrophic issue (i.e. DB goes down).
The downside to this approach is that under true microservice architecture, all resources are separated therefore you need to have unique resources (ie Databases, servers) for each service thus increasing your operating cost.
Either of these options is a good option but is totally dependent on your support model and expected volumes. Hope this helps.
ADDITIONAL DETAIL:
After reading through your additional details, since this DB already exists and my assumption is that you cannot migrate it, you still have the same choice as to whether or not you follow a monolithic application or a microservices architecture.
For both approaches, you would need to connect your django webapp the the specific DB you are already using. I can't speak for every connector out there but I know that the MySQL connector allows django to read from the pre-existing db to systematically generate the models.py file for the application. As a part of that connector, there is a model variable which allows you to define whether or not Django is responsible for actually managing the DB tables themselves.
The only thing this changes from an architecture perspective is how many times do you want to code this connection?
If you only want to do it once and completely comply with the DRY method, you can build a monolithic application knowing that as new features become required, application wide regression testing will be an absolute requirement.
If you want ultimate flexibility for future changes with this collection of features and don't mind recoding the migration across multiple apps while reducing the need for application wide regression testing as new features become required, a microservice architecture strategy is more appropriate.
I want to know how to divide a project having a hierarchical structure into applications. Let's say that I'm trying to build something like github.com.
In github.com, an account has some repositories, which have some features like code, issues, or pull requests. And those features have references to other features. In this case, which is an application and which is not? At that time, should I put applications in the root directory or in an application directory as sub-applications?
In an ideal world, each app would be independent of the others, or only loosely coupled to the others. But in many real world situations, there are often so interdependencies that it's hardly worth trying to abstract them.
So, then, in that case.. the best way to separate them is to divide them into functional groups where the majority of the views, models etc in each app are used solely within the app. So, given your github example, the "issues" could be their own app. The issues app would have specific views that are related solely to displaying, editing and serving (ajax requests, etc) issues, models for storing issues and their ongoing status, templates which are solely responsible for rendering issue views, issue entry for example, issues per user, issues per project, details of a particular issues. There's actually a lot of issue-specific code.
And yes, by the time you're done, you'll have for example foreign keys from those issue models to user models and to perhaps a commit model, a project model.. many interdependencies that would prevent the issues app from working without the presence of other apps. But logically, when it's time to work on the issue system, you'll know where to go.. because all the issue code is in one place. All the default issue settings are in issues/settings.py for example, all the tables primarily related to issues will be prefixed with the app_label eg. issues_issue, issues_comment.. etc..
So basically, try to break it up on the basis of core functionality, and minimize the number of dependencies.. or at least, try to avoid circular dependencies.. eg, some apps will have many other apps depending upon them, some will have none. Try to avoid a deadly embrace. But, in the end, dependencies will happen.
In some cases, you may be able to implement optional dependencies, eg.. when something happens in App A, Model_A, it should trigger something happening in App B, Model_B.. but only if App B is installed. There are ways to do this less-closely-coupled behavior, such as Django's signal system
https://docs.djangoproject.com/en/2.0/ref/signals/
But this is not as reliable as a foreign key, so do not go out of your way to loosely couple things which will never be uncoupled.
Try to divide things into apps on the basis of closely coupled functionality, eg. views that are related to other views. Put things which all your apps rely upon into your master app or into a library.. and you'll find that your code is much easier to maintain as it grows.
I would put the applications at the level of your manage.py file in your main project, then you can easily run this command: python manage.py startapp login_app. Then you can have a structure like so:
main_project
login_app
codeissues_app
pullrequests_app
It's not possible to create independent apps for every app in your project. I suggest you to follow the domain driven design. (google it)
So imagine you are building a ecommerce shop. You would have something like:
your_project_folder
docs
readme
static
your_project
domain # here you put the models logic
cart
products
payment
shipping
tax
infrastructure # your packages to interface with other services
paypal
stripe
interface
rest
another_rest
presentation
public_site
...
This is just an example of how you can divide the project. Than you must have boundaries. In the domain folder you must group the packages (and so design your model) to not permit cross references.
Interface, Infrastructure and Presentation can access to the Domain.
The Domain should be more stricter. Have a look here: https://martinfowler.com/bliki/BoundedContext.html
Anyway this is just the surface of the subject. Depends a lot of what kind of project you are building and what are the requirements. Have a look at the Domain Driven Design.
A newbie question:
I've been able to deploy a few simple Django apps on Heroku very easily.
However, now I am tackling a much larger project that consists of several apps (all Django and all with restricted access) under one Django project that will all use one central Heroku PostgreSQL db. The catch, of course, is that Heroku would deploy this as just one big Heroku app thus preventing one from taking advantage of scaling dynos up and down per Django app (some apps will have much greater usage than others).
I've spent a few days researching this and it looks like I will have to build a REST api to allow each app to interact (they will need to) and then deploy them as separate Heroku apps i.e. each as its own Django project. There is obviously quite a bit of overhead and admin (managing multiple settings etc.) to develop and maintain this api but I can't see another way if I need to scale up and down individual apps.
Before I embark on this, I'd be most interested in hearing any alternative suggestions (bearing in mind that we really like Heroku and we really need to scale the individual dynos!). There does not seem to be much documentation on the web on this topic but one would think it a fairly common scenario.
What's the disadvantage of scaling the entire project? It doesn't matter if the dyno serves one app or many: it's the number of requests overall that counts. An app isn't really a discrete thing in Django terms, it's just a collection of models and views, and it doesn't make much sense to try and separate them. Separate services connected by REST are a really great thing - my own projects often work like that - but they're not required, and there's no point rewriting your code that way unless you need to.
I have to create a django project which has two dictinct parts. The producer provides some UI to the user and eventually writes to some models. The consumer uses this data and does some processing and shows some pretty graphs etc.
Technically, these are completely isolated apps used by completely distinct user set. I would love to make them separate django projects altogether but unfortunately they share the db structure. The database is effectively a pipe that connects the two.
How should I structure the project? From what I read, I should ideally have isolated models per app. Unfortunately that is not possible.
Any recommendations?
If you define a "app" and use it inside of your "Django Project" I assume you use INSTALLED_APP's in settings.py to make it known within your environment.
If you look it from this point - its the same as if you use "django-social-auth" in two different projects/services and you share the same DB. I can't judge if its common or uncommon to use the same DB, its a design decision you have to make and you need to be happy with.
If you just like to have users and webusers seperated in your admin, please have a look at
separating-staff-and-user-accounts
I am relatively new to Django and this is a more general 'concept' question.
For a client I need to construct an expansive database holding data returned from a series of questionnaires as well as some basic biological data. The idea is to move away from the traditional tools (i.e. Microsoft Access) and manage the data in a mysql database using a basic CRUD interface. Initially the project doesn't need to live on the web, but the next phase will to be to have a centralized db with login and admin page.
I have started building the db with Django models which is great, and I want to use the Django admin for the management of the data.
My question is: Is this a good use of Django? Is there anything I should consider before relying on django for the whole process? And is it advisable to us the Django runserver for db admin on a client's local machine (before we get to the web phase).
Any advice would be much appreciated.
Actually, your description sounds exactly like the sort of thing for which Django is an ideal solution. It sounds more complex and customized than a CMS, and if it's as straightforward as your description then the ORM is definitely a good tool for this. Then again, this sounds exactly like an appserver-ready problem, so Rails, Express for Node.js, or even ChicagoBoss (if you're brave) would be good platforms for this kind of application.
And sure, Django is solid enough you can run it with the test server for local clients before you go whole-hog and run the thing on the web. For that, though, I recommend Apache/mod_wsgi, and if you're going to be fault tolerant there are diamond architectures (one front end proxy with monitoring failover, two or more appserver machines, one database with hot spare) and more complex (see: sharding) architectural layouts you can approach later.
If you're going to run it in a client's local setting, and you're not running Windows, I recommend looking into the screen program. It will allow you to detach the running job into the background while making diagnostics accessible in an ongoing fashion.