I am learning Django, so please correct any issues with my way of thinking here.
To start, I am using the Django Rest framework and I'm not interested in just creating a list of items, which is all that I can find examples of on the entire Internet...
I will have some pretty large functional classes that will POST and GET data to and from a pipeline server (TFS to be specific) and obviously, I'll need to insert this logic into a file within my app, I'm not just displaying things from my database here.
So, my question is, where does this go within my app? my understanding is that the "Django way" is to keep models thick.. does that mean that all code logic to make requests and interact with the pipeline server should end up in my models.py?
It's kinda frustrating that everyone else seems to just be listing furniture or some other sales item from their database. Which does not really help me very much... I need to know what a very large web app would look like (one that has a lot of logic and not just a stupid list of red and blue chairs).
Related
So I've made and been running a beta test of my project for a couple of months. It is a CRUD application, where user posts some stuff.
The thing is, I realized that I might have to pivot the item a little, so I'm planning to build a new project from the beginning. But the posts of my beta testers have to be kept as-is, so I want to migrate(or duplicate) the entire data to this new project.
Given that I have the exact same model as the original project in the new one, is there a way to make this possible? Or (I strongly hope not, but) do I have to manually copy+paste all the data? The posts are quite private so the users wouldn't want me to copy+paste them.
Thank you very much in advance. :)
FYI, I'm using AWS EC2 and Postgres.
I've read articles and posts about what a project and an app is for Django, and basically end up using the typical example of Pool and Users, however a real program generally use a complex relational database, therefore its design gravitates around this RDB; and the eternal conflict raises once again about: which ones to consider an application and which one to consider components of that application?
Let's take as an example this RDB (courtesy of Visual Paradigm):
I could consider the whole set as an application or to consider every entity as an application, the outlook looks gray. The only thing I'm sure is about this:
$ django-admin startproject movie_rental
So I wish to learn from the expertise of all of you: What approach (not necessarily those mentioned before) would you use to create applications based on this RDB for a Django project?
Thanks in advance.
PS1: MORE DETAILS RELATED ABOUT MY REQUEST
When programming something I follow this steps:
Understand the context what you are going to program about,
Identify the main actors and objects in this context,
If needed, make an UML diagram,
Design a solid-relational-database diagram, (solid=constraints, triggers, procedures, etc.)
Create the relational database,
Start coding... suffer and enjoy
When I learn something new I hope they follow these same steps to understand where they want to go with their actions.
When reading articles and posts (and viewing videos), almost all of them omit the steps 1 to 5 (because they choose simple demo apps), and when programming they take the easy route, and don't show other situations or the many supposed features that Django offers (reusability, pluggability, etc).
When doing this request, I wish to know what criteria is used for experienced programmers in Django to determine what applications to create based on this sample RDB diagram.
With the (2) answers obtained so far, "application" for...
brandonris1 is about features/services
Jeff Hui is about implementing entities of a DB
James Bennett is about every action on a object, he likes doing a lot of apps
Conclusion so far: Django application is a personal creed.
My initial request was about creating applications, but as models are mentioned, I have this another question: is with a legacy relational database (as showed in the picture) possible to create a Django project with multiple apps? this is because in every Django demo project showed, every app created has a model with their own tables, giving the impression that tables do not interact with those of other applications.
I hope my request is more clear. Thanks again for your help.
It seems you are trying to decide between building a single monolithic application vs microservices. Both approaches have their pros and cons.
For example, a single monolithic application is a good solution if you have a small amount of support resources and do not need to be able to develop new features in fast sprints across the different areas of the application (i.e. Film Management Features vs Staff Management Features)
One major downside to large monolithic applications is that eventually their feature sets grow too large and with each new feature, you have a significant amount of regression testing which will need to be done to ensure there aren't any negative repercussions in other areas of the application.
Your other option is to go with a microservice strategy. In this case, you would divide these entities amongst a series of smaller services and provide them each methods to integrate/communicate with each other (APIs).
Example:
- Film Service
- Customer Service
- Staff Service
The benefits of this approach is it allows you to separate capabilities and features by specific service areas thus reducing risk and regression testing across the application when new features are deployed or there is a catastrophic issue (i.e. DB goes down).
The downside to this approach is that under true microservice architecture, all resources are separated therefore you need to have unique resources (ie Databases, servers) for each service thus increasing your operating cost.
Either of these options is a good option but is totally dependent on your support model and expected volumes. Hope this helps.
ADDITIONAL DETAIL:
After reading through your additional details, since this DB already exists and my assumption is that you cannot migrate it, you still have the same choice as to whether or not you follow a monolithic application or a microservices architecture.
For both approaches, you would need to connect your django webapp the the specific DB you are already using. I can't speak for every connector out there but I know that the MySQL connector allows django to read from the pre-existing db to systematically generate the models.py file for the application. As a part of that connector, there is a model variable which allows you to define whether or not Django is responsible for actually managing the DB tables themselves.
The only thing this changes from an architecture perspective is how many times do you want to code this connection?
If you only want to do it once and completely comply with the DRY method, you can build a monolithic application knowing that as new features become required, application wide regression testing will be an absolute requirement.
If you want ultimate flexibility for future changes with this collection of features and don't mind recoding the migration across multiple apps while reducing the need for application wide regression testing as new features become required, a microservice architecture strategy is more appropriate.
I am looking for some advice before I start on a new project.
I am creating a Web Application using Django 1.10. I have experience with Django and creating general "content-based" websites with it. However, since this project is going to be a web-based application, I plan on doing more "complex" things than just rendering HTML templates and doing some basic CRUD operations. When I say "complex" things, the most specific example I can give at this point is to leverage more asynchronous requests so my web-application can remain responsive to the user and provide that "real-time" experience that comes with an application that might be installed on their local machine or whatever. Plus, since this is a web-application and not just a website, the project is definitely going to be more data-driven which could potentially mean requesting large amounts of data that would be best served in say a paginated manner, etc.
So, my though was this.....Since I am familiar with Django and have read such good things about the Django REST Framework, I could create a RESTful API to perform all of my CRUD operations and basically interface with my web application's core database.
At that point, I could essentially have two "layers": (1) A presentation layer that will render my web-application's pages and (2) an application layer that will do all my backend CRUD operations. Since the two are separated, I will also get the added benefit of being able to leverage the API from other avenues (other than my Django web-app) if that is ever required down the road.
I guess my first question is whether or not this makes sense and if so, thoughts on the best way to implement it. I believe I have two options.
Create a single Django project and include the API as a separate application. This seems like it would work fine but it would couple my API with my presentation since they would both be hosted by the same server. Which...if the only "consumer" of the API is my one Django site, then this may not be an issue in the near term but could cause problems later on.
Separate the two into two distinct Django projects. This offers the most flexibility and is probably the best answer, I think.
There may be other options as well. At the end of the day, I am looking for some advice/guidance from those who have done this before and what other things I should consider before starting on this effort.
Basically I would not recommend the way number 2. This is because you really fast will start to struggle with issues like that: where to store models? Project A, Project B, Both?? Also I am almost sure that you will have other code that can be shared between both of the django projects.
Worth to note here that in two separate django projects - you will have troubles with synchronization of the migrations (just consider simultaneously change of the models in both projects). Personally - I've never was able to solve this in some nice, acceptable way.
Maybe I will share with you my personal experience - hope this will be helpful.
First is that if your are making a big application - consider resignation from django templates - use some modern js framework for frontend: react, angular; and make a separate project for fontend. This is tricky because you need to have right competences in the team. If this is not possible - well do separate django app for front.
Use DRF - you can create rest app inside your project and use all of the models defined elsewhere - it will provide you simple CRUD out of the box and with some work it can be really powerful. If you will be able to create front in modern js framework - this REST can be also used to feed data to the front. And also any other client you can imagine. This basically shift the amount of work needed to be done in the backend (only REST instead of REST + templates and rendering) to the frontend.
Asynchronous tasks. Well nothing new here - use celery. Define your tasks inside application; make sure that you have correct number of workers. And let the magic begin.
As for real-time experience, you may consider using django-channels: https://github.com/django/channels/ It is nice way to handle websocket connections.
Things to consideration:
In big application usually you will end up with something like this:
myapp.com -> point to frontend; api.myapp.com -> point to the api
You should make a clear separation in the code between this two. In the case when your front is js-based - this is not a problem, but rest + rendering it is important to be able to run only-api-node and only-rendering-node. It's hard to say which one of them will be used more heavily.
So basically you can end up with something like this:
core
models
users.py
my_app_frontend
users
views.py
forms.py
tasks.py
urls.py
my_app_rest
users
serializers.py
views.py
signals.py
urls.py
my_app
settings.py
rest_settings.py
manage.py
wsgi.py
rest_wsgi.py
urls.py
rest_urls.py
This have some advantages that I like:
Models are in one place;
There is a possibility to run two separates nodes - based on needs.
Your development settings should be able to run dev server as a whole. Combing the settings from front and rest.
There is a clear code separation - and we know what is what in above structure.
If you have any more question - please ask.
Also I am curious what the others have to say - and how they handle issues pointed by me :)
Happy coding!
I'm involve in a new proyect, let explain it briefly.
We have an application server, this server can be interfaces with other systems using udp (this is a design policy, I can change it).
Now we need a web app to collect the information sended by the application servers to show some reports and send back some configuration when we need.
We are very interested in Django with Twisted (or Hendrix), we are very new in python world, I surf in Internet for days and so far I undersand some concepts but I mess about how to start with a proof of concept, so I'm here.
I need an example or guide about how to start, our idea is simple for this proof of concept.
An Application server send some data over udp.
A twisted/django app receive that info and save it in a database table.
A django web page read the database table and show a report. I don't know if the web page need to run on twisted or not.
Please, I read a lot of post for days and I'm lost about how to start.
Thank you a lot in advance.
Best Regards.
Learn each part of your stack one-by-one. Start by following a Django tutorial, then once you're comfortable with Django run the app using Hendrix instead of Django's manage.py, finally add your UDP tasks via Twisted from Hendrix. Lucky for you all the projects you've mentioned have good communities and documentation. You don't have to get everything working perfectly, just get some working pieces.
Write some code, then when you have some running code and you get stuck somewhere, hop back onto Stackoverflow and ask your specific questions.
Update
As it so happens, there's a video which shows how to integrate Django + Hendrix. Give it a shot and tell us how it goes.
I need to write an API which would provide access to data being served as HTML documents from a web server. I need for my users to be able to perform queries over the data.
Say on a web site there is a page which lists items and their owners. Then there is additional set of profile pages for owners which for each owner provide information about their reputation. An example query I may need to answer is "Give me ID's and owners of all items submitted in 2013 whose owners have reputation of at least 10".
Given a query to answer, I need to be able to screen scrape only the parts of the web site I need for answering the query at hand. And ideally cache the obtained information for future use with new queries.
I have no problem writing the screen scraping part, but I am struggling with designing the storage/query/cache part. Is there something about Clojure/Datomic that makes it an especially suitable technology choice for this kind of processing of data? I have been pointed in this direction before.
It seems a nice challenge but not sure about a few things: a) would you like to expose to your users a Datalog query box and so make them learn datalog-like syntax? b) what exact kind of results do you wish to cache, raw DB responses, html fomatted text, json ?
Anyway I suggest you to install and play a little bit with the Datomic console to get a grasp if you didn't before as it seems to me the more close idea to what you want to achieve atm https://www.youtube.com/watch?v=jyuBnl0XQ6s http://blog.datomic.com/2013/10/datomic-console.html
For the API I suggest you to use http://clojure-liberator.github.io/liberator/ as it provides sane defaults to implement REST services and let you focus on your app behaviour