Clean Development Series

Posted on Sunday December 02, 2012 / by Adam Culp

Clean Development Series: Part 1, Dirty Code (cause/effect)

Lately I have been talking about clean application development, and how developers can do a better job of it. Due to the strong focus on this I decided to write some content from my latest talks into a series of blog posts. Enabling everyone to reap the benefits of the subject matter, and allow attendees of my talks to refresh the content covered at one of the events I spoke at.

Whether we’re experienced developers or newcomers, we’ve all seen code that could/should have been done better. Many times it’s even code we ourselves wrote and revisited later for one reason or another. I, for one, have seen my share of code written by a past me and wondered what on earth I was thinking when I wrote that. Of course there has also been times I was hired to fix another developers code, and it can be scary too.

The code is pretty dirty if this is your reaction!

Side effects of dirty code

Causes of dirty code (the perfect storm)

As you can see from this first post in the series, dirty code is dangerously easy to create and can carry disastrous outcomes. I hope this post helps some to spot these pitfalls a bit easier, and make it easier to avoid them. Please return for future installments as I attempt to highlight more areas to help us all perform clean development, and write better applications.

Clean Development Series: Part 2, Dirty Code (why we do it)

Lately I have been talking about clean application development, and how developers can do a better job of it. Due to the strong focus on this I decided to write some content from my latest talks into a series of blog posts. Enabling everyone to reap the benefits of the subject matter, and allow attendees of my talks to refresh the content covered at one of the events I spoke at.

In Part 1 of this series I discussed the causes and effects of dirty code, and how disastrous it can be for us and/or our employer. I think it’s easy to see how the terrible effects caused by writing dirty code is not desirable. Most of us try to avoid wasting time, introducing bugs, increasing technical debt, causing our companies to lose money, and looking for a new job because our employer went out of business. So if we don’t like these things, why do we write dirty code in the first place? It is our own fault after all, but it’s not something we consciously do…most of the time. Let’s take a look at a sample scenario, and see how we sometimes get ourselves in this type of situation:

New Job and a Simple Request

We start a new job, of perhaps we are approached by our supervisor with a request. We know how it works. The boss mentions how wonderful it would be if the application was able to do some sort of functionality, and asks if it’s possible. Of course in our haste to please and impress we say, Of course, and head off to our lair to perform our magic. We sit down behind the keyboard and already have an excellent idea of how to build the new functionality our boss was asking for. What do we do first?

The Preparation

Do we write unit tests? No. Do we document the requirements? No. Perhaps we at least write down some sort of decision making flow chart? No. So we at least have an idea of standards we will use and follow? No. Well, maybe we download a framework, or figure out what functions we already have rather than recreating the wheel? No say.

Execution

What actually happens is pretty mystical. Like a maestro ready to conduct an elegant symphony we go back to our desk, fire up our favorite IDE or editor capable of helping us create the best code imaginable by man. These tools have built in auto complete, connectors to version control, able to link to ticket and bug tracking systems, debuggers, project file management, structure tree display, code inspectors, refactor modules, and even direct connections to hosting and command line tools. We can now create true masterpieces of intellectual property.

But we don’t have any time for all of those bells and whistles today. We are excited and in a hurry to prove that our boss made the right decision by hiring us. So instead of using these awesome tools at our disposal we code the entire thing in very short order, and then spend a few minutes to do some functional testing and ensure it doesn’t blow up on us while we demo it.

Presentation

Barely able to contain ourselves we rush into our bosses office to announce we have completed the functionality he requested. Our excitement is quickly contagious and our boss looks on wide-eyed as we demo our accomplishment. He is a little surprised we were able to finish it so quickly, and thanks us for our hard work.

Then, faster than we can think of what we will have for lunch, our boss immediately thinks of some sort of enhancement to the item we just completed. He quickly outlines how it could work and what it could bring the company. In the excitement of the moment, and our newly found feelings of accomplishment, we tell our boss it’s a small task and we should be able to get it done pretty fast.

We head back to our desk feeling the afterglow of the moment, feeling like we saved the world as well as earned our keep at this new employer. Once again we have a task to accomplish.

Second Iteration

True to form we perform exactly as well on this second task assigned to us. In our haste to please we ignore best practices, testing, and standards as we quickly complete the new request. Upon completion, the meeting and presentation go pretty much like the first, and ends with yet another task.

More of the Same

This cycle continues over and over again, and we continue to perform admirably. Our boss seems to be a wealth of knowledge and ideas, and we continue to turn these ideas into application at an astounding rate. However, with each iteration there are more bugs, more overlap between one functionality and the next, and more gotcha moments as we try to demo the newly created functionality. We are starting to struggle with the code base a bit as our technical debt has been steadily increasing little by little.

Longer Time frames

Of course we are smart, and realize very fast that those little gotcha moments as we demo to our boss are taking a toll on his trust of our ability. We quickly realize we need to test a little better before presenting the functionality. This requires some time to complete so we start to pad our time estimates a bit to compensate. But with each task this time frame grows longer and longer, and soon these time estimates become a week, a month, two months, a quarter, then six months.

Unstable

By now our code has become so unstable we fear it, and are tripping over quick decisions made months ago. There are no unit tests, no standard functional tests, no documented regression tests. Our code is just layer upon layer of classes and functions that are so tightly coupled we constantly trip over them. A change in one place causes things to break in 4 other places. We have not even sat down and set a coding standard, let alone done any documentation of the application.

It’s at this point that every time someone makes a request our first answer is 6 months. However, we know in our mind that there is no way we can get it done in that time frame. But marketing needed some sort of answer and there is no way they would settle for a number larger than 6 months.

Danger

We’ve dug a hole of technical debt over time by not using best practices, and we are in a danger area. Our once pleased boss is questioning every choice and estimate we make. Perhaps he is forgiving a little because he realizes we have worked so hard up to now, and we have earned some professional credit to this point. We realize that action needs to happen…and happen fast.

Time to Pay

As with all debt, whether it is financial or technical, we must eventually pay it off somehow. With financial debt we either pay it off by making payment arrangements, or we go bankrupt. Technical debt is similar. With technical debt we can either refactor bit by painful bit (making payments), or we can do a full rewrite of the application (go bankrupt). Of course we could also leave the company and therefore leave the mess for someone else to clean up, but lets assume we are responsible developers and feel obligated to do the right thing.

Summary

I am sure most of us have played a part in the story above, and can nod our heads in understanding. Hopefully most of us have learned from the experience and no longer do it. (Yeah, right.) Those who have not had the luxury of making these mistakes should take heed of the story and learn from these common mistakes. The story above IS NOT how to do things.

IMPORTANT: Our employers hire us to be professionals. By following the sequence above we set the standard ourselves, in a bad way. Please do things the right way, and be the professional you were hired to be.

Throughout this series I will continue to outline ways to avoid making these mistakes, and how we can accomplish Clean Development. I hope you found this breakdown helpful, and learned from it.

Clean Development Series: Part 3, Dirty Code (how to spot/smell it)

Lately I have been talking about clean application development, and how developers can do a better job of it. Due to the strong focus on this I decided to write some content from my latest talks into a series of blog posts. Enabling everyone to reap the benefits of the subject matter, and allow attendees of my talks to refresh the content covered at one of the events I spoke at.

In part 1 of this series we discussed some causes of dirty code, and then talked about the things that dirty code can cause. In part 2 we followed that by giving a possible scenario of how developers get caught in a situation where they end up with dirty code. Now in this part we will cover how to spot dirty code, and some possible situations that can be a bad smell and highlight potential problems.

Most experienced developers, and development how to books, have come to associate smells to help find dirty code. Meaning that most dirty code can be found by looking for certain clues/smells, which like a bad odor can tell us something rotten is nearby. When something smells we usually are forced to deal with the problem. (If milk smells bad we throw it out.) The same is true of smells in our code. Below are some common stenches we may encounter to help point out problems.

Dirty things smell. So does dirty code.

How to spot common dirty code

NOTE: All of the items below are not necessarily dirty code, but are merely indications that something may be wrong and require more investigation.

While these are not the only smells that may be encountered, they are the most common indicators that ugly and spoiled things lurk inside. Like a spoiled package of meat, you don’t need to open the package to know something is very wrong. We should always analyze our code to look for these and other smells, then fix them quickly. Remember, one bad apple will ruin a bunch if left unattended. Don’t let these bad apples infect the rest of the code. Fix them, take them out before they multiply, leave the code better than when you arrived.

As I have said many times, we read code more than we write code. By eliminating smells we ensure that our reading, or other developers reading, of the code goes faster and enables creation of new code to be faster. Not to mention how this affects our reputation. If you write dirty code please believe that other developers WILL talk about you, and it can determine what jobs you get hired for in the future. So please, write clean code and clean up old code.

Clean Development Series: Part 4, Rewrite dilemma

Lately I have been talking about clean application development, and how developers can do a better job of it. Due to the strong focus on this I decided to write some content from my latest talks into a series of blog posts. Enabling everyone to reap the benefits of the subject matter, and allow attendees of my talks to refresh the content covered at one of the events I spoke at.

In part 1 of this series we discussed reasons for creating dirty code, and the things that dirty code can cause. We followed with part 2 by giving a possible scenario of how developers get caught in a situation and start writing dirty code. Then, in part 3 we covered smells that hint to dirty code, and some possible situations that can be a bad smell highlighting potential problems. So, what happens when our code carries many of the bad situation covered in the first 3 parts?

Code that made us proud at one time has become an embarrassment we try to hide, and hide from. In part 2 the developer was very happy with his work in the beginning, and so was his boss. They were able to complete requests quickly, and everyone was happy to have such fast turn-around on new features. Then, little by little the problems of the application started to grow. The bugs began popping out, a change in one place leads to multiple problems in other areas, and time estimates start growing to compensate as the developers are forced to pad them for the unknown. As things become worse the developers tend to care less and less about the code, and this lack of caring shows in newly created code as problems continue to mount.

Many times the first signs of surrender come when the boss asks if hiring more developers will help get things done faster. It’s an honest question, and makes perfect sense on paper. If it takes two or three developers six months to create a new feature surely twice as many could do it in half the time. So the boss instructs us to start a search for more developers to hire, and we spend two or three months (or more) of our spare time searching, evaluating, and interviewing candidates until we finally manage to find a couple. After all, more resources equal more output…right?


Needless to say we go through a few developers, because all of them are not able to grasp the mess we’ve created. Others just leave for greener pastures rather than try to figure things out. Then a couple, either due to desperation or being hard workers, manage to stick around and get to work. But for some strange reason the time-lines never seem to improve, and it still takes a long time to introduce new features.

The boss is very confused, and not happy, because he thought more resources would allow development to get done faster. He even went to his boss and asked for permission to augment the team, and fought a good fight to get his way. Meanwhile the developers, seeing the writing on the wall, start scurrying to provide answers for failing to meet expectation. Technical debt has caught up to them, and it’s time to pay it off.

We talked briefly in part 1 about technical debt, and explained it was similar to buying things on credit. Using credit to buy things means the debt continues to grow and grow, until we’re forced to deal with it. Either we pay it off, or we go bankrupt and start fresh. In relation to coding, if we continue to develop poorly taking the short way, our technical debt continues to climb with each bad thing we do. Eventually we find ourselves in a situation where we either repair the code through refactoring, or we rewrite the application entirely and start fresh.

Unfortunately the first reaction to a poorly written application is usually a resounding rewrite. Developers feel like a weight is lifted from their shoulders, as the manager hears a distant cha-ching sounds of money slipping away. The old application took months, maybe even years, to build and cost a HUGE amount of money. Developers, hosting, tools, sales, customer support, and the hidden costs of insurance/rent/utilities/furniture to keep it all running add up. In the managers mind a rewrite means it must all be duplicated, and that’s not far from the truth when we consider the salaries of an entire development team for the time it will take to rewrite the whole application.

Challenging sale

In the past there has been times when I inherited an application full of dirty code and recommended a rewrite to the client. On the surface it seemed like a good idea, and a win/win situation for both me and the client. It allowed me to make a little more money, because a rewrite take longer than simply doing updates or creating some new functionality. It also meant I could create the application my way. Because if it was built using my favorite framework, with my coding standards, and unit tests, it would be so much easier for me to work with.

The customer would benefit from a better application, pay for less time on future enhancements because it was better, and might even get some performance gains with less bugs from the application. The downside of course is that it costs the customer a much larger sum of money, and much longer time period, to have a rewrite done.

The rewrite

After much debate the developers may eventually win, and the manager will allow the rewrite. But only if the team is divided so part will be dedicated to building the new vision while the rest will support the old application. It only makes sense the developers who have been there the longest should develop the new application, since they understand the business better. (Even though they were the ones who created most of the bad code to start with.) The newly split up developer teams start working.

Rewrite

As the experienced team starts the new application the other team continues to fix bugs, and develops new features in the old application. It happens this way because business must continue to make money and service customers who continually need new functionality, or they will leave. Which causes endless scope creep for the team developing the new application. Not only are they attempting to build a new application, but they must also keep pace with new features added to the old application as well as incorporate any bug fixes and/or changes to the business logic. We start to see how difficult the task of rewriting becomes.

Rewriting started with the best intentions as everything is planned ahead of time, code written in an organized way, and perhaps unit test are even written. But as time moves on, and the scope continues to creep, the business side of the company starts to become impatient. The developers start to hear, Is it done yet?, and under mounting pressure to speed things up they fall back into the bad practices used in the old application. Before long the new application starts to suffer the same technical debt issues, and the downward spiral drags on. Ultimately the developers end up in the same position of apologizing for padded time-lines, unpredictable bugs, and cascading errors.

Usually by the time this entire process plays out there has been a change of developers, and maybe none of the originals are still around. The current team is now faced with two similar applications that both contain dirty code full of smells, and may not even have tests. They are now faced with the same choices of the previous team…to rewrite or refactor. And if they refactor, which application do they refactor?

Meanwhile the company, waiting for the new application, has been dragging its feet on some new features. Or perhaps have been promising the new application to their customers and building expectation. This costs the company money in lost revenue from lack of features, lost customers who went elsewhere to get the new features, and higher costs for the extra developers required for a dual-application path. Eventually, if left unmonitored, companies cannot recover and go out of business from scenarios like this.

Refactor instead

On the other hand, what if we fought the urge to push a rewrite? If the current application is working and already contains all the business logic needed, but has a very dirty code base, why should the customer endure the loss of time and money for a rewrite? Plus, if we are new to the application and don’t really know all of the business logic, why should we endure the headaches and heated meetings while we discover every little nuance of a new application.

Now don’t get me wrong. I realize there are some circumstances where a rewrite is absolutely necessary. Perhaps the framework the application uses has had a major upgrade, or perhaps the application business logic is undergoing a large overhaul, or maybe the code is just…that…bad! Yes, there are times a rewrite is inevitable, and simply MUST happen. What I am introducing here is the idea that many times when we do a rewrite it may have actually been better for the customer, and maybe us as well, if we had simply performed refactoring as we went.

Refactor

What I have come to realize as a much better solution than the rewrite story above is to follow some of the steps, but take a different turn. What I mean is that I still split the team up, but rather than instructing one team to build a new application I dub them as the refactor team. Their job is to start refactoring the code one module at a time. First they write unit tests, if there weren’t any. Then they start performing scheduled refactors. Meanwhile the other team starts following the new coding standards and styles, and writing unit tests as they continue to do bug fixes and adding enhancements.

Of course these two teams need to be closely monitored, and must communicate very often to ensure they are not stepping on each other. IT IS VITAL that everything be managed carefully, and that the teams are communicating open and often. Did I mention that communication is important, and it must happen often?

Sure there will be times when commits clash, and code will need to be altered using pair programming. However, the cost of time and money will be greatly reduced from a full rewrite. Not to mention it will help bring the development team closer together in the process.

After the initial modules are refactored in this manner it may be beneficial to rotate some members around on each team. This will ensure that team members on both sides of the fence do not become bored, or fall into bad habits.

In closing

Remember, I am not saying one way is better than another in every circumstance. It’s really a judgement call on which method is used to deal with the technical debt of the application we are working on. However, I hope this post has helped highlight that a rewrite is not always the best solution. Sometimes the best approach is to plan a series of refactors to recover the application from the junk pile. We must carefully analyze the entire picture and determine if a rewrite is a must, or is it a nice to have. Then we can honestly create a full list of pros and cons to more fully gain the big picture, and use that list to sell what we believe to be the best solution.

It may very well save the company from being a used to be, and instead make it a will be.