I feel the need to refactor my old CF5 based code into CFC's. We already have some code in ColdSpring and Transfer but feel a large rewrite to ColdSpring and Transfer is pointless.
What tips, approaches and gotchas will I hit.
How can I make this easy?
I don't mind keeping ColdSpring in the mix but Transfer is the bit I'm scared of with the size of the project.
edit: my code base has been going for 7-8 years and is vast. To describe it would be difficult, however I'm looking for generic suggestions on approaches
Changing the whole code base just for the sake of it if it basically works would be introducing a lot of potential bugs into your system. I don’t think there is an easy way to do it.
If you look at the areas of your site which are 1: most likely to change and 2: executed the most you may be able to target some areas which could benefit from change and see how easily they would fit into a CFC based framework, and what benefits. But for most of the code if it is working OK, there may be no pressing need to change.
However whenever you need to do a major alteration to part of the system it may be worth looking at that from an OO perspective and moving the existing code over, where applicable.
In one of my ongoing projects (almost same situation, even more -- most of code is really bad) I am using technique I'd called "wave-style". General ideas I use are following:
Splitting processing from output. I can not implement true MVC here, but at least I can move view into separate templates (sometimes re-use them) and prepare all data in basic (model) templates.
Move all repeating code into components -- this is one of most important tips.
Group related functions into components. Say, all customer-related info grouped into CustomerManager.cfc, invoices into InvoiceManager.cfc etc.
Why "wave"? In a big project I can't just sit and rewrite all customer-related code. So I have make it step by step. For example, I have to work on customer signup, extend it with few attributes. I've created basic component, moved there methods to validate form (check login, email etc.) and add customer - so this page works in new style. Lated I will need to improve invoice page, where I need to get invoice owner details: I just add method into customer manager and get rid of direct queries. Later edit customer page... Also it can be called "on demand refactoring" or smth.
There can be additional stuff relying on your current project state. But it helped me a lot. Hope you'll find these tips useful.
Before you change anything: create a full set of regression tests!
When refactoring, the goal has to be preserve functionality first, so that you don't directly affect your clients.
I agree with Sergii's wave-style refactoring also - this allows you to break things into manageable chunks rather than doing everything in one go.
But whatever method you have, the more regression tests you can create, the better - it's really the only way you can confirm you haven't unintentionally changed something.
This is extremely hard (bordering on impossible) to answer without knowing any of your code.
The question is a bit like "I want to disassemble my old Volkswagen and build a new one from the parts, what should I consider?" :-)
My advice would be to start off by encapsulating your business logic into CFCs instead of worrying about the whole presentation layer of your site.
By just concentrating on the business logic, you'll be able to get the most important functionality into CFCs and ease the maintenance nightmare. It also won't be too hard to just "drop-in" these CFCs into your existing site.
After getting as much business logic into the CFCs as you can, you'll notice that the enormous monster has been cut down to size. At that point you can now decide on what you want to do with the presentation layer of your site. You're now free to pick from a multitude of frameworks available to use (CFWheels, FuseBox, ColdBox, Mode-Glue) to port over the presentation layer.
Or you could just say "the heck with it" and rewrite the whole thing in CFWheels from the start :)
If you are not using version control get that set up before you do anything else. Being able to back out of broken refactoring is a serious life saver. After that I agree with what has been posted. You will want to take on small chunks at a time - divide and conquer.
Related
I am working on a fat client legacy C++ application which has a lot of business logic mixed in with the presentation side of things. I want to clean things out and refactor the code out completely, so there is a clear seperation of concerns. I am looking at MVC or some other suitable design pattern in order to achieve this.
I would like to get recommendations from people who have walked this road before -
Do I use MVP or MVC (or another pattern)?
What is/are the best practices for undertaking something like this (i.e. useful steps/checks) ?
The most useful step are to create very robust and full set of regression tests, no matter which pattern you choose.
To choose between MVP and MVC, please review their differences in this SO question:
What are MVP and MVC and what is the difference?
If MVP vs. MVC is your main problem, you can probably choose freely.
There are three factors affecting that decision: your previous experience, support of your framework/libraries, and which fits the existing code base better.
In my epxerience both patterns fit on C++ legacy applications like cat puke on a wedding cake. Your main challenges will be:
Not breaking anything. Heck, it's probably not breaking everything
Noticing that you are actually moving forward
Doing that with small changes that don't require a three month rewrite of some components
The remainder is very generic to dealing with legacy applications, not related to the specifics of your question. I'll leave it here since you also have a generic part.
Rewrite vs. Refactor You already stated your decision, so I just put forward the pro rewrite arguments to consider: if you have a clear spec and understand how current users use this app, a rewrite might be faster and cheaper when 30% or more of the code need changes. (This doesn't mean "rewrite everything", this might also mean cutting back the application to logic-only, then planting a new presentation layer on top of it)
Assuming you go for refactor:
Define your goals Why do you need to refactor the app at all? Good reasons could be a lot of new features to be implemented, presentation layer needs to be replaced, or to buggy to remain sane. From that, decide what needs to change, and what can stay the way it is.
You have already picked your goal: MV*. I just want you to think about the benefits for the client and the code owner in the long run.
Read the code. No, really, take your time to get used to the code base. Step through it with the debugger. Try to understand the things involved. Take notes on improvements you think you should make.
Create tests - mostly regression tests for the currently desired behavior. With legacy code bases, they are more often than not manual, so create test protocols that a moron can follow, and try to get hold of a not-so-moron that can run these tests for your from time to time. Try to get some use cases from existing users.
Stick to small, reversible changes If a refactoring step goes wrong, it should be small enough to be thrown away without hesitation. Sometimes, this is not easy, my typical worst case steps are:
- decide which functionality to replace. Make plans how the new code should look like.
- move the existing code behind an interface that also works for the new code
- test
- replace the functionality with your new code
- test
- sometimes, the interface is "final", somethimes you can remove/reduce it
Always improve Don't accept functionality setbacks tha "can be fixed later".
You should take a look at "Working Effectively With Legacy Code" by Michael Feathers. The book outlines how to get your existing code into a test harness, as well as how to safely break the dependencies in the code.
I wish to present the idea of MVC to a bunch of old C++ spaghetti coders (at my local computer club).
One of them that has alot of influence on the rest of the group seems to finally be getting the idea of encapsulation (largely due in part to this website).
I was hoping that I could also point him in the right direction by showing him Model View Controller, but I need to do it in a way that makes sense to him, as well as it probably needs to be written in C/C++!
I realize that MVC is a very old architectural pattern so it would seem to me that there should be something out there that would do the job.
I'm more of a web developer, so I was wondering if anybody out there who is a good C/C++ coder could tell me what it is that made the MVC light switch turn on in your head.
Don't start off with MVC. Start off with Publish / Subscribe (AKA the "listener" pattern).
Without the listener pattern fully understood, the advantages of MVC will never be understood. Everyone can understand the need to update something else when something changes, but few think about how to do it in a maintainable manner.
Present one option after another, showing each option's weaknesses and strengths: making the variable a global, merging the other portion of code into the variable holder, modifying the holder to directly inform the others, and eventually creating a standard means of registering the intent to listen.
Then show how the full blown listener can really shine. Write a small "model" class and add half a dozen "listeners" and show how you never had to compromise the structure of the original class to add "remote" updates.
Once you get this down, move the idea into to the "model view" paradigm. Throw two or three different views on the same model, and have everyone amazed on how comparatively easy it is to add different views of the same information.
Finally discuss the need to manage views and update data. Note that the input is partially dependent on items which are not in the view or the model (like the keyboard and mouse). Introduce the idea of centralizing the processing where a "controller" needs to coordinate which models to create and maintain in memory, and which views to present to the user.
Once you do that, you'll have a pretty good introduction to MVC.
You might find it easier to sell them on the Document/View or Document/Presenter patterns. MVC was invented on Smalltalk where everything about the different UI elements had to be coded by the developer (as I understand, never used the thing). Thus the controller element was necessary because didn't have things like TextElement::OnChange. Now days, more modern GUI API's use Document/View but Document/Presenter is something I've seen proposed.
You might also consider reading Robert Martin's article on the TaskMaster framework.
You might also consider that any C++ developer who is not familiar with these patterns and already understands their purpose and necessity is either a complete newb or a basket-case best avoided. People like that cause more harm than good and are generally too arrogant to learn anything new or they already would have.
Get some spaghetti C++ code (theirs?), refactor it to use MVC, and show them what advantages it has, like easier unit testing, re-use of models, making localized changes to the view with less worry, etc.
Recently I inherited a business critical project at work to "enhance". The code has been worked on and passed through many hands over the past five years. Consultants and full-time employees who are no longer with the company have butchered this very delicate and overly sensitive application. Most of us have to deal with legacy code or this type of project... its part of being a developer... but...
There are zero units and zero system tests. Logic is inter-mingled (and sometimes duplicated for no reason) between stored procedures, views (yes, I said views) and code. Documentation? Yeah, right.
I am scared. Yes, very sacred to make even the most minimal of "tweak" or refactor. One little mishap, and there would be major income loss and potential legal issues for my employer.
So, any advice? My first thought would be to begin writing assertions/unit tests against the existing code. However, that can only go so far because there is a lot of logic embedded in stored procedures. (I know its possible to test stored procedures, but historically its much more difficult compared to unit testing source code logic).
Another or additional approach would be to compare the database state before and after the application has performed a function, make some code changes, then do database state compare.
I just rewrote thousands of lines of the most complex subsystem of an enterprise filesystem to make it multi-threaded, so all of this comes from experience. If the rewrite is justified (it is if the rewrite is being done to significantly enhance capabilities, or if existing code is coming in the way of putting in more enhancements), then here are the pointers:
You need to be confident in your own abilities first of all to do this. That comes only if you have enough prior experience with the technologies involved.
Communicate, communicate, communicate. Let all involved stake-holders know, this is a mess, this is risky, this cannot be done in a hurry, this will need to be done piece-meal - attack one area at a time.
Understand the system inside out. Document every nuance, trick and hack. Document the overall design. Ask any old-timers about historical reasons for the existence of any code you cannot justify. These are the mines you don't want to step on - you might think those are useless pieces of code and then regret later after getting rid of them.
Unit test. Work the system through any test-suite which already exists, otherwise first write the tests for existing code, if they don't exist.
Spew debugging code all over the place during the rewrite - asserts, logging, console prints (you should have the ability to turn them on and off, as well specify different levels of output i.e. control verbosity). This is a must in my experience, and helps tremendously during a rewrite.
When going through the code, make a list of all things that need to be done - things you need to find out, things you need to write tests for, things you need to ask questions about, notes to remind you how to refactor some piece of code, anything that can affect your rewrite... you cannot afford to forget anything! I do this using Outlook Tasks (just make sure whatever you use is always in front of you - this is the first app I open as soon as I sit down on the desk). If I get interrupted, I write down anything that I have been thinking about and hints about where to continue after coming back to the task.
Try avoiding hacks in your rewrite (that's one of the reasons you are rewriting it). Think about tough problems you encounter. Discuss them with other people and bounce off your ideas against them (nothing beats this), and put in clean solutions. Look at all the tasks you put into the todo list - make a 10,000 feet picture of existing design, then decide how the new rewrite would look like (in terms of modules, sub-modules, how they fit together etc.).
Tackle the toughest problems before any other. That'll save you from running into problems you cannot solve near the end of tunnel, and save you from taking any steps backward. Of course, you need to know what the toughest problems will be - so again, better document everything first during your forays into existing code.
Get a very firm list of requirements.
Make sure you have implicit requirements as well as explicit ones - i.e. what programs it has to work with, and how.
Write all scenarios and use cases for how it is currently being used.
Write a lot of unit tests.
Write a lot of integration tests to test the integration of the program with existing programs it has to work with.
Talk to everyone who uses the program to find out more implicit requirements.
Test, test, test changes before moving into production.
CYA :)
Two things, beyond #Sudhanshu's great list (and, to some extent, disagreeing with his #8):
First, be aware that untested code is buggy code - what you are starting with almost certainly does not work correctly, for any definition of "correct" other than "works just like the unmodified code". That is, be prepared to find unexpected behavior in the system, to ask experts in the system about that behavior, and for them to conclude that it's not working the way it should. Prepare them for it to - warn them that without tests or other documentation, there's no reason to think it works they way they think it's working.
Next: Refactor The Low-Hanging Fruit Take it easy, take it slow, take it very careful. Notice something easy in the code - duplication, say - and test the hell out of whatever methods contain the duplication, then eliminate it. Lather, rinse, repeat. Don't write tests for everything before making changes, but write tests for whatever you're changing. This way, it stays releasable at every stage and you are continuously adding value, continuously improving the code base.
I said "two things", but I guess I'll add a third: Manage expectations. Let your customer know how scared you are of this task; let them know how bad what they've got is. Let them know how slow progress will be, and let them know you'll keep them informed of that progress (and, of course, do it). Your customer may think s/he's asking for "just a little fix" - and the functionality may indeed change only a little - but that doesn't mean it's not going to be a lot of work and a lot of time. You understand that; your customer needs to, too.
I've had this problem before and I've asked around (before the days of stack overflow) and this book has always been recommended to me. http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052
Ask yourself this: what are you trying to achieve? What is your mission? How much time do you have? What is the measurement for success? What risks are there? How do you mitigate and deal with them?
Don't touch anything unless you know what it is you're trying to achieve.
The code might be "bad" but what does that mean? The code works right? So if you rewrite the code so it does the same thing you'll have spent a lot of time rewriting something introducing bugs along the way so the code does the same thing? To what end?
The simplest thing you can do is document what the system does. And I don't mean write mind-numbing Word documents no one will ever read. I mean writing tests on key functionality, refactoring the code if necessary to allow such tests to be written.
You said you are scared to touch the code because of legal, income loss and that there is zero documentation. So do you understand the code? The first thing you should do is document it and make sure you understand it before you even think about refactoring. Once you have done that and identified the problem areas make a list of your refactoring proposals in the order of maximum benefit with minimum changes and attack it incrementally. Refactoring makes additional sense if: the expected lifespan of the code will be long, new features will be added, bug fixes are numerous. As for testing the database state - I worked on a project recently where that is exactly what we did with success.
Is it possible to get a separation of the DB and non-DB parts, so that a DBA can take on the challenge of the stored procedures and databases themselves freeing you up to work on the other parts of the system? This also presumes that there is a DBA who can step up and take that part of the application.
If that isn't possible, then I'd make the suggestion of seeing how big is the codebase and if it is possible to get some assistance so it isn't all on you. While this could be seen as side-stepping responsibility, the point would be that things shouldn't be in just one person's hands usually as they can disappear at times.
Good luck!
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions concerning problems with code you've written must describe the specific problem — and include valid code to reproduce it — in the question itself. See SSCCE.org for guidance.
Closed 9 years ago.
Improve this question
When developing an application which mostly interacts with a database, what is a good way to start? The application requires a lot of filtering based on user input, sorting and structuring.
The best way to start is by figuring out "user stories" (or "use cases" -- but the "story" approach tends to really work great and start dragging shareholder into the shared storytelling...!-); on top of that, designing the database schema as the best-normalized idea you can find to satisfy all data layer needs of the user stories.
Thirdly, you may sketch layers such as views on top of the schema; fourthly, and optionally, triggers and stored procedures that might live in the DB to ensure consistency and ease of use for higher layers (but, no matter how strongly DBAs will push you towards those, don't accept their assurances that they're a MUST: they aren't -- if your storage layer is well designed in terms of normalization and maybe useful views on top, non-storage-layer functionality CAN always reside elsewhere, it's an issue of convenience and performance, NOT logical consistency, completeness, correctness).
I think the business layer and user-experience layers should come after. I realize that's a controversial position, but my point is that the user stories (and implied business-rules that come with them) have ALREADY told you a LOT about the business and user layers -- so, "nailing down" (relatively speaking -- agility and "embrace change!" should always rule;-) the data storage layer is the next order of business, and refining ("drilling down") the higher layers can and should come after.
When you get to the database layer you'll want to handle the database access via stored procedures. This will help give you additional protection against SQL Injection attacks, and make it much easier to push logic changes to the database layer.
If it's mostly users interacting with data, you can design using a form perspective.
What forms are needed for user input?
What forms are needed for output reports?
Once you've determined that, the use of the forms will dictate the business logic needed to be coded behind the scenes. You'll take the inputs, create the set of procedures or methods to deal with them, and output what is necessary. Once you know the inputs and outputs, you will be able to easily design the necessary functions.
The scope of the question is very broad. You are expecting me to tell what to do. I can only do a good job of telling how to do things. Do investigate upon using Hibernate/Spring. Since most of your operations looks like querying db, hibernate should help. Make sure the tables are sufficiently indexed so your queries can run faster if filtered based on index fields. The challenging task is design your DB layer which will be the glue between your application and db. Design your db layer generic enough so that it can build queries based on the params that you pass to it. Then move on to develop the above presentation layer. Developing your application layer by layer helps since it will force you to decouple the db logic from the presentation logic. When you develop the db layer, assume that not just your presentation layer but any client can call it. This will help you to design applications that can be scalable and adaptable to new requirements.
So bottom line : Start with DB, DB integeration layer, Controller and last Presentation Layer.
For the purpose of discussion, I'm going to assume that you are working with a starting application that doesn't have a pre-existing database. If this is false, I'd probably move the order of steps around quite a bit.
1 - Understand the Universe
First, you've got to get a sense of what's around you so you can really understand the problem that you are trying to solve.
User stories or use cases are often a good starting point. Starting with what tasks the user will try to do, and evaluating how frequently they are likely to be is a great starting point. I like to start with screen mockups as well, with or without lots of hands on time with users, I find that having a screen gives our team something really finite to argue about.
What other tools exist in this sphere? These days, it seems to me that users never use just one tool, they swap around alot. You need to know two main things about the other tools you users use:
(1) - what will they be using as part of the process, along side your tool? Consider direct input/output needs - what might they want to cut/copy/paste from or to? What tools might you want to offer file upload/download for with specific formats, what tools are they using alongside your tool that you might want to share terminology, layout, color coding, icons or other GUI elements with. Focus especially on the edges of the tools - a real gotcha I hit in a recent project was emulating the databases of previous tools. It turned out that we had massive database shift, and we would likely have been better starting fresh.
(2) What (if anything) are you replacing or competing with? Steal the good stuff, dump and improve the bad stuff. Asking users is always best. If you can't at least understanding the management initiative is important - is this tool replacing a horrible legacy tool? It may be legacy, but there may be the One True Feature that has kept the tool in business all these years...
At this stage, I find that things are really mushy - there's some screen shots, some writing, some schemas or ICDs - but not a really gelled clue.
2 - Logical Entities
Or at least that's what the OO books call it.
I don't care much for all the writing I see on this task - but I find that any any given system, I have one true diagram that I draw over and over. It's usually about 3-10 boxes, and hopefully less than an exponentially large number of lines connecting them. W
The earlier you can get that diagram the better.
It doesn't matter to me if it's in UML, a database logical model, something older, or on the back of a napkin (as long as the napkin is shrouded in plastic and hung where everyone can see it).
The earlier you can make this diagram correctly, the better.
After the diagram is made, you can start working on the follow on work that may be more official.
I think it's a chicken and egg question on whether you start with your data or you start with your screens and business logic. I know that you certianly want to optimize for database sizing and searchability... but how do you know exactly what your database needs are without screens and interfaces giving you a sense for the data?
In practice, I think this is an ever-churning cycle. You do a little bit everywhere, and then you change it all.
Even if you don't get to do a formal agile lifecycle, I think you're best bet is to view design as agile -- it will take many repetitions and arguments before you really feel it's "right".
The most important thing to keep in mind is that your first, and most likely 2nd 3rd attempt at designing the database will be wrong in some way. That might sound negative, maybe even a little rash, (it's certainly more towards the 'agile' software design philosophy) but it's important thing to keep in mind.
You still need to do your analysis thoroughly of course, try to implement one feature at a time, but try to get all layers working first. That way you won't have to do to much rework when the specs change and you understand the issues better. One you have a lot of data loaded into a system, changing things becomes increasingly difficult.
The main benefit of this approach is you find out quickly where you design is broken, where you haven't separated you design layers correctly. One trick I find extremely useful is to do both a sqllite and a mysql version, so seamless switching between the two is possible. Because the two use a different accent of SQL it highlights where you have too tight a coupling between the layers.
A good start would be to get familiar with Multitier architecture
Then you design your presentation layer.
In your business logic layer implement all logic
And finally you implement your data access layer.
Try to setup a prototype with something that is more productive then C++ for example Ruby, Python and well maybe even PHP.
When the prototype works and you see your data model is okay and your queries are too slow then you can start using C++.
But as your questions suggests you have more options then data and in this case the speed of a scripting langauge should be enough.
We have an existing "legacy" app written in C++/powerbuilder running on Unix with it's own Sybase databases. For complex organizational(existing apps have to go through lot of red-tape to be modified) and code reasons(no re-factoring has been done in yrs so the code is spaghetti), so it's difficult to get modifications done to this application. Hence I am considering writing a new modern, maybe grails, based web app to do some "admin" type things directly into the database. For example to add users, or to add "constraint rows".
What does the software community think of this approach of doing a run-around of the existing app like this? Good idea? Tips and hints?
There is always a debate between a single monolithic app and several more focused apps. I don't think it's necessarily a bad idea to separate things - you make things more modular, reduce dependecies, etc.. The thing to avoid is duplication of functionality. If you split off an adminstration app separately, make sure to remove that functionality from the old app, or else you will have an unmaintained set of adminstration tools that will likely come back to haunt you.
Good idea? No.
Sometimes necessary? Yes.
Living in a world where you sometimes have to do things you know aren't a good idea? Priceless.
In general, you should always follow best practices. For everything else, there's kludges.
See this, from Joel, first!
Update: I had somewhat misconstrued the question and thought that more was being rewritten.
My perspective on your suggested "utility" system is not nearly so reserved as would be suggested by my link to Joel's article. Indeed, I would heartily recommend that you take this approach for a number of reasons.
First, this may well be the fastest route to your desired outcome since the old code is so difficult to work with.
Second, this gives you experience with a new development technology and does so in the context of your existing work - this is a real advantage.
Third, I took this approach years ago when transitioning an application from C++ to Delphi. In time, the Delphi app grew to be so capable that a complete leap onto that platform became possible. At no point were users without the functionality that they already knew because the old app wasn't phased out until the replacement functionality had been proven. However, it is at this stage that you'll want to heed Joel's warnings: remember that some of the "messiness" you see is actually knowledge embodied in the old code.
Good idea? That depends on how well the database is documented and/or understood. Make a mistake about some implicit application-level implemented rule, relation, or constraint, and your legacy app may end up doing cartwheels down the aisle.
Take a hypothetical example. Let's say adding a user with the legacy system adds records to the following tables:
app_users
app_constraints
app_permissions
user_address
Let's assume you catch the first three, miss the fourth. It can't be important, right? But what if in the app, in the 50 places that app_users is used, one place does an inner join to user_address. (And why not? The app writer knew that he always wrote a record to user_address!) The newly added user suddenly disappears from the application's view, a condition that "could never happen" according to the original coder, and the application coughs up a hair ball. Orders can't be taken. Production stops. A VP puts his new cardiac bypass surgery to the test.
Who gets blamed? Not the long-gone developer who should have coded for more exceptions. Not the guys who set up the red tape to control change. The guy that did an end run around those controls.
Good luck,
Terry.