Has anyone of you done evaluation of a django project and how to improve/refactor it's code base? A pet project in company I work at is becoming more widely used and it would be good to improve its quality before further development. Are there any techniques or methodologies of analyzing django projects before we start putting more and more features into it? We wouldn't like to suddenly realise, that due to earlier poor choice we have to live with something really bad.
What I noticed after working more than a year on a quite large site was mostly this concerning our design/coding. It's not purely refactoring related and probably you already know a lot, but maybe it can help :).
Most importantly, we did not always put our code in the right place. There was too many functionality in the view methods, too few in forms and models. I see this problem many times. To handle input, use forms, to format/tweak/... model data, use model methods or properties. Seriously, at one moment I reduced a view of about 150 lines to 20 by just putting code in better places. Improves maintainability and readability very well
Many code we wrote actually did not really use the full force of python and/or django. For the first one, reading something like Dive Into Python was a great help, for the second one I just tracked down all complex constructs we built (mostly legacy code from 0.96) and looked if there were alternatives in the django docs. Of course, don't waste your time trying to reduce everything to a one-liner, but certainly with legacy code this can help improve readability and maintainability.
Always look on sites like django-snippets, google code, ... if there are decent existing django projects that can take away a lot of your functionality. Often these projects are looked upon by more people and are therefore more stable and performant. If a project doesn't meet all your demands it can even be a good idea of just adding your requirements yourself instead of doing it all custom for your site.
Try to keep the application cross-over dependencies minimal. When you would draw a dependency graph (e.g. by linking each app that have foreign keys to each other) it should still be clear and not something where every app is linked to any other app. Typically you'll have some 'helper' apps (e.g. user system, tagging) that are used by many and all other apps are actually only dependent on those apps.
Write tests, django has an excellent test suite, so use it. Certainly for parts of code that are common to many apps and are likely to change. Really, nothing so annoying as suddenly noticing a bug you actually solved 4 months ago and you have no clue which update since then broke it again.
Take a second look at database normalization, the django orm model is still tightly coupled to relational databases and so it's an important concept, certainly when you work with models that are likely to be expanded later on.
Concerning real refactoring, the only important tool I can think of is South, can help you a lot if your database scheme changes. Otherwise, like a hint I already gave: write tests to make sure your functionality before and after refactoring remains the same.
You can learn about code smells. Some folks say that if your code smells it means you need to consider refactoring.
This is very wide subject in fact. Remeber that:
Premature optimization is the root of all
evil -- DonaldKnuth
I personally think that this is also valid for (premature) refactoring.
EDIT: this is also a good resource about code smells: http://www.codinghorror.com/blog/2006/05/code-smells.html
Related
I am working on a Flask site I inherited from the last person to hold my position. It is a fairly large project with a lot of code, so any fundamental changes I want to make involve a lot of refactoring and rewriting code. With this being said, are there many tangible benefits in trying to integrate Flask-Security this late in the game? The site seems to have a pretty solid authentication system to me. It already uses Flask-Login, Flask-Admin, Flask-Mail, and a bunch of the other packages included in Flask-Security, so it seems like I'm deciding on whether a small change in functionality is worth all the work of implementing it.
This might not be the right place for it, but I am hoping to hear from people who have used Flask-Security and have a good idea of why it would be beneficial to add it rather than using the packages separately. It probably won't be a high-traffic site, but I don't want to create security risks by making the wrong choice.
Thanks!
I am working on a fat client legacy C++ application which has a lot of business logic mixed in with the presentation side of things. I want to clean things out and refactor the code out completely, so there is a clear seperation of concerns. I am looking at MVC or some other suitable design pattern in order to achieve this.
I would like to get recommendations from people who have walked this road before -
Do I use MVP or MVC (or another pattern)?
What is/are the best practices for undertaking something like this (i.e. useful steps/checks) ?
The most useful step are to create very robust and full set of regression tests, no matter which pattern you choose.
To choose between MVP and MVC, please review their differences in this SO question:
What are MVP and MVC and what is the difference?
If MVP vs. MVC is your main problem, you can probably choose freely.
There are three factors affecting that decision: your previous experience, support of your framework/libraries, and which fits the existing code base better.
In my epxerience both patterns fit on C++ legacy applications like cat puke on a wedding cake. Your main challenges will be:
Not breaking anything. Heck, it's probably not breaking everything
Noticing that you are actually moving forward
Doing that with small changes that don't require a three month rewrite of some components
The remainder is very generic to dealing with legacy applications, not related to the specifics of your question. I'll leave it here since you also have a generic part.
Rewrite vs. Refactor You already stated your decision, so I just put forward the pro rewrite arguments to consider: if you have a clear spec and understand how current users use this app, a rewrite might be faster and cheaper when 30% or more of the code need changes. (This doesn't mean "rewrite everything", this might also mean cutting back the application to logic-only, then planting a new presentation layer on top of it)
Assuming you go for refactor:
Define your goals Why do you need to refactor the app at all? Good reasons could be a lot of new features to be implemented, presentation layer needs to be replaced, or to buggy to remain sane. From that, decide what needs to change, and what can stay the way it is.
You have already picked your goal: MV*. I just want you to think about the benefits for the client and the code owner in the long run.
Read the code. No, really, take your time to get used to the code base. Step through it with the debugger. Try to understand the things involved. Take notes on improvements you think you should make.
Create tests - mostly regression tests for the currently desired behavior. With legacy code bases, they are more often than not manual, so create test protocols that a moron can follow, and try to get hold of a not-so-moron that can run these tests for your from time to time. Try to get some use cases from existing users.
Stick to small, reversible changes If a refactoring step goes wrong, it should be small enough to be thrown away without hesitation. Sometimes, this is not easy, my typical worst case steps are:
- decide which functionality to replace. Make plans how the new code should look like.
- move the existing code behind an interface that also works for the new code
- test
- replace the functionality with your new code
- test
- sometimes, the interface is "final", somethimes you can remove/reduce it
Always improve Don't accept functionality setbacks tha "can be fixed later".
You should take a look at "Working Effectively With Legacy Code" by Michael Feathers. The book outlines how to get your existing code into a test harness, as well as how to safely break the dependencies in the code.
We have an existing "legacy" app written in C++/powerbuilder running on Unix with it's own Sybase databases. For complex organizational(existing apps have to go through lot of red-tape to be modified) and code reasons(no re-factoring has been done in yrs so the code is spaghetti), so it's difficult to get modifications done to this application. Hence I am considering writing a new modern, maybe grails, based web app to do some "admin" type things directly into the database. For example to add users, or to add "constraint rows".
What does the software community think of this approach of doing a run-around of the existing app like this? Good idea? Tips and hints?
There is always a debate between a single monolithic app and several more focused apps. I don't think it's necessarily a bad idea to separate things - you make things more modular, reduce dependecies, etc.. The thing to avoid is duplication of functionality. If you split off an adminstration app separately, make sure to remove that functionality from the old app, or else you will have an unmaintained set of adminstration tools that will likely come back to haunt you.
Good idea? No.
Sometimes necessary? Yes.
Living in a world where you sometimes have to do things you know aren't a good idea? Priceless.
In general, you should always follow best practices. For everything else, there's kludges.
See this, from Joel, first!
Update: I had somewhat misconstrued the question and thought that more was being rewritten.
My perspective on your suggested "utility" system is not nearly so reserved as would be suggested by my link to Joel's article. Indeed, I would heartily recommend that you take this approach for a number of reasons.
First, this may well be the fastest route to your desired outcome since the old code is so difficult to work with.
Second, this gives you experience with a new development technology and does so in the context of your existing work - this is a real advantage.
Third, I took this approach years ago when transitioning an application from C++ to Delphi. In time, the Delphi app grew to be so capable that a complete leap onto that platform became possible. At no point were users without the functionality that they already knew because the old app wasn't phased out until the replacement functionality had been proven. However, it is at this stage that you'll want to heed Joel's warnings: remember that some of the "messiness" you see is actually knowledge embodied in the old code.
Good idea? That depends on how well the database is documented and/or understood. Make a mistake about some implicit application-level implemented rule, relation, or constraint, and your legacy app may end up doing cartwheels down the aisle.
Take a hypothetical example. Let's say adding a user with the legacy system adds records to the following tables:
app_users
app_constraints
app_permissions
user_address
Let's assume you catch the first three, miss the fourth. It can't be important, right? But what if in the app, in the 50 places that app_users is used, one place does an inner join to user_address. (And why not? The app writer knew that he always wrote a record to user_address!) The newly added user suddenly disappears from the application's view, a condition that "could never happen" according to the original coder, and the application coughs up a hair ball. Orders can't be taken. Production stops. A VP puts his new cardiac bypass surgery to the test.
Who gets blamed? Not the long-gone developer who should have coded for more exceptions. Not the guys who set up the red tape to control change. The guy that did an end run around those controls.
Good luck,
Terry.
I feel the need to refactor my old CF5 based code into CFC's. We already have some code in ColdSpring and Transfer but feel a large rewrite to ColdSpring and Transfer is pointless.
What tips, approaches and gotchas will I hit.
How can I make this easy?
I don't mind keeping ColdSpring in the mix but Transfer is the bit I'm scared of with the size of the project.
edit: my code base has been going for 7-8 years and is vast. To describe it would be difficult, however I'm looking for generic suggestions on approaches
Changing the whole code base just for the sake of it if it basically works would be introducing a lot of potential bugs into your system. I don’t think there is an easy way to do it.
If you look at the areas of your site which are 1: most likely to change and 2: executed the most you may be able to target some areas which could benefit from change and see how easily they would fit into a CFC based framework, and what benefits. But for most of the code if it is working OK, there may be no pressing need to change.
However whenever you need to do a major alteration to part of the system it may be worth looking at that from an OO perspective and moving the existing code over, where applicable.
In one of my ongoing projects (almost same situation, even more -- most of code is really bad) I am using technique I'd called "wave-style". General ideas I use are following:
Splitting processing from output. I can not implement true MVC here, but at least I can move view into separate templates (sometimes re-use them) and prepare all data in basic (model) templates.
Move all repeating code into components -- this is one of most important tips.
Group related functions into components. Say, all customer-related info grouped into CustomerManager.cfc, invoices into InvoiceManager.cfc etc.
Why "wave"? In a big project I can't just sit and rewrite all customer-related code. So I have make it step by step. For example, I have to work on customer signup, extend it with few attributes. I've created basic component, moved there methods to validate form (check login, email etc.) and add customer - so this page works in new style. Lated I will need to improve invoice page, where I need to get invoice owner details: I just add method into customer manager and get rid of direct queries. Later edit customer page... Also it can be called "on demand refactoring" or smth.
There can be additional stuff relying on your current project state. But it helped me a lot. Hope you'll find these tips useful.
Before you change anything: create a full set of regression tests!
When refactoring, the goal has to be preserve functionality first, so that you don't directly affect your clients.
I agree with Sergii's wave-style refactoring also - this allows you to break things into manageable chunks rather than doing everything in one go.
But whatever method you have, the more regression tests you can create, the better - it's really the only way you can confirm you haven't unintentionally changed something.
This is extremely hard (bordering on impossible) to answer without knowing any of your code.
The question is a bit like "I want to disassemble my old Volkswagen and build a new one from the parts, what should I consider?" :-)
My advice would be to start off by encapsulating your business logic into CFCs instead of worrying about the whole presentation layer of your site.
By just concentrating on the business logic, you'll be able to get the most important functionality into CFCs and ease the maintenance nightmare. It also won't be too hard to just "drop-in" these CFCs into your existing site.
After getting as much business logic into the CFCs as you can, you'll notice that the enormous monster has been cut down to size. At that point you can now decide on what you want to do with the presentation layer of your site. You're now free to pick from a multitude of frameworks available to use (CFWheels, FuseBox, ColdBox, Mode-Glue) to port over the presentation layer.
Or you could just say "the heck with it" and rewrite the whole thing in CFWheels from the start :)
If you are not using version control get that set up before you do anything else. Being able to back out of broken refactoring is a serious life saver. After that I agree with what has been posted. You will want to take on small chunks at a time - divide and conquer.
Last term (August - December 2008) me and some class mates wrote an application in C++. Nothing spectacular, it is an ORM for Sqlite3. We implemented some stuff like reflection to make it work and release the end user from the ugly stuff. Personally, i think we made a nice job, and that our ORM could actually be useful for someone (even though its writen specifically for Sqlite3, its easily adaptable for oter databases).
Consequently, i`ve come to the conclusion that it should be published somewhere (sourceforge most likely) as an open source project. But, as it was a term project, there are some things that need to be addresesed before doing that. Namely, it has some memory leaks that should be fixed, and some parts of the code could be refactored to make everyone´s life easier in the future.
I would like to know more experienced C++ programmers opinion on some issues:
Is it worth rewriting some parts to
apply new techonologies (for example,
boost).
Should our ORM be adapted to latest
C++ standard? Is there any benefit in
doing this?
How will we know when our code is
ready for release?
What are the chances that this ORM
will be forgotten into the mists of
the internet? (i.e is it worth
publishing it beyond personal pride
as a programmer?)
Right now i can`t think of many more questions, but i would like to read on similar experiences.
EDIT: I should probably translate my code + comments to english right? (self question)
Thanks in advance.
I guess I am "more experienced" with regard to your particular question. I co-developed an open source web application language & template system a lot like ColdFusion back in the early days of web design before Java or ASP were around. You can still see it at http://www.steelblue.com/ if you are interested. It's still used at the company I was at when it was developed, but I don't think anywhere else.
What I found is that unless you are already well connected and people are watching what you are doing, getting people to use your open source code is just about as hard as selling somone your closed source program. You really need to advocate for your project and it should have some kind of unique selling proposition that distinguishes it from the compitition.
So, that's the unsolicited advice. Here are some specific answers to the questions you had...all purely my opinion, of course.
I wouldn't rewrite any code unless you have a featuer you want to put in. That feature might be compatibility with a specific platforms or compilers. It might be to support a new db datatype or smarter indicies or whatever. If you are going to put some more serious work into the applicaiton, think about a roadmap of what you can realistically accomplish in the next iteration and what choices will make the app the "most better" at the end of your cycle.
Release the code as soon as it is usable for a specific purpose, any purpose. Two reasons. First off, there might be someone who wants it for that purpose right now. If it's not available, they will use something else. Also, if it's open source, they might contribute back to the project. Second, the sooner you find out how much people want to use the code, the better. Either it will be more popular than you expect and you can get excited about continuing the development....or....you will find that no one is even visiting your web page to see what you've got. In either case, better to know sooner than later what people really want from your project so you can take that into account when planning new releases.
About the "forgotten into the mists." I think most projects are. I don't want to be a downer, but looking at Wikipedia, there were 5 C++ ORM tools popular enough to get mention and they were all open source. As I said above, unless you can sell your idea to people, they are going to go with another proven open source solution. For someone to choose you over them, three things have to happen: 1. They need a feature you have that the others don't. 2. They find your project web site and it demonstrates the superiority of your code. 3. They trust your code enough to give it a shot.
On the other hand, if you are in this for the long haul and want to continue development thigns get easier over time. Eventually the project will get all the basics covered and you can start developing those new featuers that aren't in the other solutions. Also, the longer you are in active development the more trustworthy the project will seem. Finally, you will get more experience in the nitch. 2 years from now you will be better positioned to say where your effort will have the most impact on bettering the project.
A final thought: If you are enjoying it, learning from it, and it's not getting in the way of you keeping food on the table, it's a good use of your time.
Good luck!
-Al
Regarding the open source part:
If you really want to make it an open source project, you really should publish it regardless of it's current state - fully working and debugged - or half working and full of memory leaks.
Just, if it's state is bad, make sure to document it, and give it a suitable version number (less than one?). then others may view your code, suggest improving, join your team, etc...
My--rather random--thoughts on the matter (in the order I think is most important):
How will we know when our code is ready for release?
Like Liran Orevi said: if you're going open source release early. Document it reasonable well, and take the time to provide a road map of planned or hoped for future improvements (these are a invitation for people to help you, so note which ones have no one working on them).
Is it worth rewriting some parts to apply new technologies (for example, boost).
Should our ORM be adapted to latest C++ standard? Is there any benefit in doing this?
SQLite relies on a fairly limited base. Maybe you don't want your tool to demand a much heavier environment. If the code in not currently a tangled and unmaintainable mess, you might want to avoid boost and newest frills. Once you have a stable release (1.0 at least) you can starting thinking about the improvements that can be made for version 2.
What are the chances that this ORM will be forgotten into the mists of the internet? (i.e is it worth publishing it beyond personal pride as a programmer?)
Most things end up in the big /dev/null in the sky, and there is only one way to find out... If it goes anywhere at all, you win. If it doesn't it was a modest investment, and maybe you learned something while you were at it.