I've an app which is growing pretty big, doing too much thing as a single app, so I'd like to split it in 2 or 3 "sub-apps"
The problem is that there are a dozen of models which are linked to each other (foreing key, manytomanyfields, etc.)
I've read LOTS of times that apps should be self-consistent, so, are there any best practices to split a big app in several ones linked to each other?
--> how bad is importing models from other apps?
I didn't hear about a best practice solution, but here's what I would usually do, and I split apps a lot:
Step 0 - When is an app "too big"?:
An app should be an (independent) logical unit. Independent is actually misleading, of course you can have dependencies like django.conrib.auth, what you should have tho are cross dependencies. They will eventually lead to looping imports. That being said, you app can grow quite large, with is totally fine.
If you having problems organizing your code, I may remind you of the fact that every module can be build as a package. You simply split your models.py into models/__init__.py and models/LOGICAL_UNITS.py.
The only reason why you should split an app is because you can, not because you want to ;)
Step 1 - Overview
Use django_extensions' graph printing capabilities.
This should give you a good overview an might help you to find so called "communities". Groups of models that have strong cross dependencies.
Those communities usually make a pretty good app.
Step 2 - Naming:
If you cant find a name for you're new application, it probably isn't one.
Related
I'm building/managing a django project, with multiple apps inside of it. One stores survey data, and another stores classifiers, that are used to add features to the survey data. For example, Is this survey answer sad? 0/1. This feature will get stored along with the survey data.
We're trying to decide how and where in the app to actually perform this featurization, and I'm being recommended a number of approaches that don't make ANY sense to me, but I'm also not very familiar with django, or more-than-hobby-scale web development, so I wanted to get another opinion.
The data app obviously needs access to the classifiers app, to be able to run the classifiers on the data, and then reinsert the featurized data, but how to get access to the classifiers has become contentious. The obvious approach, to me, is to just import them directly, a la
# from inside the Survey App
from ClassifierModels import Classifier
cls = Classifier.where(name='Sad').first() # or whatever, I'm used to flask
data = Survey.where(question='How do you feel?').first()
labels = cls(data.responses)
# etc.
However, one of my engineers is saying that this is bad practice, because apps should not import one another's models. And that instead, these two should only communicate via internal APIs, i.e. posting all the data to
http://our_website.com/classifiers/sad
and getting it back that way.
So, what feels to me like the most pressing question: Why in god's name would anybody do it this way? It seems to me like strictly more code (building and handling requests), strictly less intuitive code, that's more to build, harder to work with, and bafflingly indirect, like mailing a letter to your own house rather than talking to the person who lives there, with you.
But perhaps in easier to answer chunks,
1) Is there REALLY anything the matter with the first, direct, import-other-apps-models approach? (The only answers I've found say 'No!,' but again, this is being pushed by my dev, who does have more industrial experience, so I want to be certain.)
2) What is the actual benefit of doing it via internal API's? (I've asked of course, but only get what feel like theoretical answers, that don't address the concrete concerns, of more and more complicated code for no obvious benefit.)
3) How much do the size of our app, and team, factor into which decision is best? We have about 1.75 developers, and only, even if we're VERY ambitious, FOUR users. (This app is being used internally, to support a consulting business.) So to me, any questions of Best Practices etc. have to factor in that we have tiny teams on both sides, and need something stable, functional, and lean, not something that handles big loads, or is externally secure, or fast, or easily worked on by big teams, etc.
4) What IS the best approach, if NEITHER of these is right?
It's simply not true that apps should not import other apps' models. For a trivial refutation, think about the apps in django.contrib which contain models such as User and ContentType, which are meant to be imported and used by other apps.
That's not to say there aren't good use cases for an internal API. I'm in the planning process of building one myself. But they're really only appropriate if you intend to split the apps up some day into separate services. An internal API on its own doesn't make much sense if you're not in a service-based architecture.
I cant see any reason why you should not import an app model from another one. Django itself uses several applications and theirs models internally (like auth and admin). Reading the applications section of documentation we can see that the framework has all the tools to manage multiple applications and their models inside a project.
However it seems quite obvious to me that it would make your code really messy and low-performance to send requests to your applications API.
Without context it's hard to understand why your engineer considers this a bad practice. He was maybe referring to database isolation (thus, see "Working multiple databases" in documentation) or proper code isolation for testing.
It is right to think about decoupling your apps. But I do not think that internal REST API is a good way.
Neither direct import of models, calling queries and updates in another app is a good approach. Every time you use model from another app, you should be careful. I suggest you to try to separate communication between apps to the simple service layer. Than you Survey app do not have to know models structure of Classifier app::
# from inside the Survey App
from ClassifierModels.services import get_classifier_cls
cls = get_classifier_cls('Sad')
data = Survey.where(question='How do you feel?').first()
labels = cls(data.responses)
# etc.
For more information, you should read this thread Separation of business logic and data access in django
In more general, you should create smaller testable components. Nowadays I am interested in "functional core and imperative shell" paradigm. Try Gary Bernhardt lectures https://gist.github.com/kbilsted/abdc017858cad68c3e7926b03646554e
I'm trying to figure out the best way to build a large web application, but keeping the various logical sections modularized. For example, there will be a number of different logical sections:
Estimates
Work Orders
Accounting
Contacts
etc...
As much as possible, I want to have the code for each of these modules disconnected from all the other modules. If code from one section starts to throw errors or cause incorrect data, I don't want that to affect other modules.
My first thought is just to separate by using Django 'apps'. Each module is its own app. Then, if I build each app to be "pluggable", then the code is self sufficient. However, the problem with this approach is that the modules will likely need to access the models of other modules. For example, the 'Accounting' module will need access to the 'Contacts' model, since we want to send invoices to people in our contacts list.
I looked into having a completely separate Django project for each module, but that poses problems (I think) for user authentication. Plus, to my knowledge, multiple Django projects can't (easily) share one database.
Just wondering if there are any good ways or best practices to make logically separate 'modules' while, as much as possible, keeping the code from one module completely isolated from the rest, but still having a cohesive web application.
I have intended to have an app. where I want to have different things having relations with each other and want to know that whether I should have them as just different models or as differnt apps. Obviously if this is student, teacher in LMS then they are necessary component of LMS while if this is Job, Professional and Company then there can be different things associated with a job , a professional can have his full profile with different features, company can have different directory listing e.t.c. like features.
So Company and professionals who are users also should be as diff. apps. and job as different app.? Will this way be fine? as Jobs app. don't always everywhere need to have professional data or employer all data other than just name. So it seems like it is more convenient to have them as diff. apps, so that it can be used somewhere else.So is that right way?
Or
As I also want this project to be flexible so will the above make it more complex? And should I just treat them as diff. models instead of diff. apps. as Company and Professional are users , for which django gives Profile features also. So is this right way?
Which way is better one?
thanks in advance.
There is no exact answer here, so it's my opinion.
It is always good to have several apps rather than one big app. Reasons:
apps becomes smaller and it's easier to maintain small pieces of code;
project structure becomes more clear, I just need to look at the file manager to see main parts of the project;
interaction between apps become explicit: easy to test and prevent unnecessary coupling.
Not every Django app should be pluggable. It's ok to have two apps that depend on each other (if you aren't going to distribute them seperately). It's like having two dependent functions: nothing is wrong with it.
I am going to pick up a task that no one has ever attempted to try at my workplace. It is a CF app first written using CF 2.0 (Yes, 2.0!) 10 yrs ago with > 10 cfscheduler tasks.. We explored the idea of rewriting the app, but 10 yrs of work simply can't be rewrote in 2-3 months.
What steps shall one take to modernize the app into a maintainable, extendable state? The one that I keep hearing is "write tests", but how can I write tests when it wasn't even in MVC?
Any advice would be appreciated, thanks!
p.s. I should thank Allaire, Macromedia and Adobe for keeping CF so freaking backward compatible all the way back to 2.0!
btw, what's the most modern, maintainable state for a CF app without MVC framework? or should my end goal be ultimately refactoring it into a MVC app?? I can't image how many links I will break if I do... seems impossible... thought?
update: found 2 related Q's...
https://softwareengineering.stackexchange.com/questions/6395/how-do-you-dive-into-large-code-bases
https://softwareengineering.stackexchange.com/questions/29788/how-do-you-dive-into-a-big-ball-of-mud
I am not sure if you need to move the whole site to a MVC application. Recently I did helped with an site that was not MVC, that still had a library with the Models, Services and Assemblers in a clean and organized manor. It worked great, and we didn't need to do anything more than what was necessary.
That being said, my first step would be to organize the spaghetti code into their different purposes. It may be hard to properly create the models, but at the very least you could break out the services like functions from the pages. With that done, it should be a lot cleaner already.
Then, I would try to take the repeated code and put them into custom tags. It will make the code more reusable, and easier to read.
Good Luck!
Consider, whether a full fledged framework is really necessary. In its most basic form a framework is merely highly organized code. So if procedural, that is well organized, works leave it.
Keep in mind something like FW/1 as migration path can be better than say Coldbox if you don't need all the other stuff.
Lastly, consider this I was able to migrate a 4.5 almost 70% of the way to Coldbox (very simple and really more about directory and file organization versus IOC, plugins, modules, etc...) just using a few extra lines per file plus onMissingMethod functions.
Good Luck.
I had to deal with a similar situation for about two years at my last job, however, it wasn't quite as old as yours. I think I was dealing with code from 4.0 on. There's no silver bullet here, and you'll need to be careful that you don't get too caught up in re-factoring the code and costing your company tons of money in the process. If the app works as it is rewriting it would be a pretty big wast of money.
What I did was update small chunks at a time, I wouldn't even refactor whole templates at a time, just small portions of one at a time. If I saw a particular ugly loop, or nested if statements I'd try to clean it up the best I could. If the app can be broken down into smaller modules or areas of functionality and you have the extra time you can try to clean up the code a module at a time.
A good practice I heard from the Hearding Code podcast is create a testing harness template that would use a particular cfm page that has a known output that you can re-run to make sure that it still has the same output once you've done refactoring. Its not nearly as granular as a unit test, but its something and something is almost always better than nothing, right?
I suspect that the reason this app hasn't been touched for years is because for the most part it works. So the old adage "if it ain't broken don't fix it" probably applies; However, code can always be improved :)
The first thing I'd do is switch to Application.cfc and add some good error logging. That way you may find out about things that need to be fixed, and also if you do make changes you're know if they break anything else.
The next thing I'd do is before you change any code is use selenium to create some tests - it can be used as a FireFox plugin and will record what you do. It's really good for testing legacy apps without much work on your part.
Chances are that you won't have much if any protection from SQL injection attacks so you will want to add cfqueryparam to everything!!
After that I'd be looking for duplicated code - eliminating duplicate code is going to make maintenance easier.
Good luck!
Funnily enough, I'm currently involved in converting an old CF app into an MVC3 application.
Now this isn't CF2, it was updated as recently as a year ago so all of this may not apply at all to your scenario, apologies if this is the case.
The main thing I had to do consolidate the mixed up CFQuerys and their calls into logical units of code that I could then start porting in functionality either to C# or JavaScript.
Thankfully this was a very simple application, the majority of the logic was called on a database using the DWR Ajax library; that which wasn't was mostly consolidated in a functions.cfm file.
Obviously a lot of that behavior doesn't need to be replicated as packaging up the separate components of logic (such as they were) in the CF app did map quite neatly to the various Partial Views and Editor Templates that I envisaged in the MVC application.
After that, it was simply a case of, page by page, finding out which logic was called when, what it relied upon that then finally creating a series of UML class and sequence diagrams.
Honestly though, I think I gained the most ground when I simply hit File-New Project and started trying to replicate the behavior of the app from the top of index.cfm.
I would break logical parts of the app into CFC's
Pick a single view, look at the logic within. Move that out to a CFC and invoke it.
Keep doing that you will have something much easier to work with that can be plugged into an MVC later. Its almost no work to do this, just copy and paste sections of code and call them.
You can consider using object factory to layer your application. We have similar situation at work and we started refactoring by putting Lightwire DI framework.
First we migrated all the sql statement into gateways, then we started using services and take a lot of code out of the templates to the services.
The work is not finished yet but the application is looking better already.
For large, really complex applications I'd prefer ColdBox for a re-factor project. However, I just saw a presentation at the D2W Conference on F/W 1 (Framework One), a VERY simple ColdFusion MVC framework. Check out code from the presentation here.
It's 1 (one) CFC file and a set of conventions for organizing your code. I highly recommend evaluating it for your project.
I have a project that I wrote in PHP/symfony that uses 45 tables. I'm in the process of porting it to Python/Django.
There's a belief I've held for years that you should split up your projects into a bunch of small files rather than a few huge files. From what I understand, that's not an odd thing to believe. In Rails and symfony, there's a one-model-per-file convention. In Django, however, it seems that most developers put all of each app's models in one file.
This makes sense to me if your apps are each small enough. It doesn't make sense to me for large apps, though, and what I have is at least one large app.
Out of the 45 tables my project uses, 35 are closely related. I have a script that imports data from CSV files. For each line in each CSV file, I save 50-80 pieces of data into 30-35 different tables in one fell swoop.
Maybe I'm just thinking about this the wrong way but it would seem incredibly odd to me to divide my project into 6 or 7 different apps when almost all my tables are inextricably linked. When I touch one table, I touch all 35 tables. The delineations would have to be arbitrary. What would be the point of that?
Please forgive me if I come off as biased because I certainly am biased. I'm not having this problem in symfony and I wouldn't be having it in Rails. (I chose Django because of GeoDjango and Python's GIS capabilities.)
In a perfect world, I would have one model per file.
If I try to have one model per file, I get circular reference problems.
I could avoid the circular reference problems by putting all my models in one file but that feels wrong to me.
I could avoid putting all my models in the same file by splitting them into separate apps, but in order to end up with sufficiently small apps, I'd have to break up my project in arbitrary (and therefore pointless) ways.
What should I do?
If having one model per file would be the perfect answer for you, there's an app for that.
I've never done it on a scale of 80 model files but I can certainly point you towards another stack question:
About 20 models in 1 django app
http://djangosnippets.org/snippets/1838/
What kind of circular reference problems are you having by the way? If it's with ForeignKey definitions, here's a way around that...
http://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ForeignKey
You can also look into django.db.loading.get_model, but some may frown on this.