Growing Object Oriented Software, Guided By Tests

Few weeks ago I finished to read the famous book
Growing Object-Oriented Software, Guided by Tests

In this post, I try to summarize what I think is the most interesting content.

The book present the interesting approach of starting a new project with a Walking Skeleton that is a tiny implementation of the system that performs a small end-to-end function

The point of the walking skeleton is to help us understand the requirements well enough to propose and validate a broad-brush system structure.

In most Agile projects, there’s a first stage where the team is doing initial analysis, setting up its physical and technical environments, and otherwise getting started. This is usually called Iteration Zero.

Book: Working Effectively with Legacy Code

Every professional developer have to deal with legacy code in the course of his career.

The book Working Effectively with Legacy Code written by Michael Feather is considered a must read and I really recommend it. This is a summary of the book.
Michael Feather definition of Legacy Code:

Legacy code is simply code without tests.

The goal of every competent software developer is to create designs that tolerate change! This is why it is critical to learn how to confidently make changes in any code base.

The main problem is to make changes while preserving existing behaviour.

Preserving existing behaviour is one of the largest challenges in software development.

The three main questions to ask are:

1. What changes do we have to make?
2. How will we know that we've done them correctly?
3. How will we know that we haven't broken anything?

The main approach is to use tests as a way to detect changes.

But, when there are no tests, we end up in the:

Legacy Code Dilemma

When we change code, we should have tests in place. To put tests in place, we often have to change code.

Sometimes, it is possible to make changes without getting existing classes under tests:

• Sprout Method: develop a new method and call it from the existing code.
• Sprout Class: develop a new class and use it from the existing code.
• Wrap Method: develop a new method that wrap an existing method.
• Wrap Class: create a new class that wrap an existing class (the Decorator Pattern).

A technique like TDD is perfect in this context and only the new method or class will be tested.

However, when you need to touch existing code, it is necessary to do very conservative refactoring to get tests in place first.

The goal is to change as little code as possible to get tests in place.

The steps to follow are:

1. Identify change points
2. Break dependencies
3. Write tests
4. Make changes
5. Refactor

Identify Change Points

Reading through code and playing with the system helps you to identify the change points. It often pays to draw pictures and make notes to improve your understanding.

Scratch Refactoring is a useful technique for learning. Grab the code and start applying refactoring without tests and at the end throw away all your changes. As a result, your knowledge about the code base will be increased.

Object-Oriented Reengineering Patterns is a recommended book for learning how to read and understand large code bases and build a big picture of a project. Once you identified the areas of the code that you need to change, it is important to answer the following question:

If I make some changes here, where I can see the effects?

Answering this questions is very important because it helps to identify which code you should be cover with tests in order to create a good safety net that gives you the confidence you need for subsequent changes.

It is important to know what can be affected by the changes we are making!

Effect Sketching is a useful technique that can be used to answer this question. The idea is to create a little graph that shows the effect of your changes. Navigation tools like ReSharper can help you identify all usages of a particular piece of code. However, if your class has a superclass or subclasses pay attention because there might be other clients that you haven't considered.

A Pinch Point is a narrowing in an effect sketch, a place where it is possible to write tests to cover a wide set of changes. If you can find a pinch point in a design, it can make your work a lot easier. It is a place where tests against a couple of methods can detect changes in many methods. Writing tests at pinch points is an ideal way to start some invasive work in part of a program.

The tests that we are going to write are called Characterization Tests. What the system does is more important than what it is supposed to do. You are not fixing bugs at this stage! The goal is to capture what the system does to increase your confidence of making changes.

How to break dependencies?

Adding tests is not always easy. Often, it is necessary to break dependencies.

The book contains a catalogue of dependency breaking techniques:

• Extract Interface
• Adapt Parameter: wrap a parameter behind an interface
• Parametrized Constructor: create a new constructor that takes the dependency and update the old constructor to use it
• Break Out Method Object: move a long method to a new class
• Extract and Override Call: extract a call to a new method and override it in a testing subclass (ideal to break dependencies on global variables and static methods)
• Extract and Override Factory Method: extract hard-coded initialization work from a constructor to a factory method and override it in a testing subclass.
• Parametrized Method: if a method creates an object internally, pass the object from the outside
• Pull Up Feature: you can pull up a cluster of methods into an abstract superclass and you can subclass it to create instances in your tests.
• Push Down Dependency: make a class abstract and create a subclass that will be the new production class and push down all problematic dependencies into that class.
• ...

For languages like C and C++ it is possible to use some specific techniques like:

• Preprocessing seams: use ad-hoc macros to replace behaviour in tests
• Link seams: override behaviour linking to a different implementation in a test module
• Replace Function with Function Pointers: use pointers in tests to swap real implementation with fakes
• Template Redefinition (C++ only): make a class a template and instantiate the template with a different type in the test file.

In using these techniques it is very important to try to preserve signatures whenever possible! It is always better to do this work using pair programming.

Working in legacy code is surgery, and doctors never operate alone.

Consider reading the book Pair Programming Illuminated. Refactoring

One of the most important principle to keep in mind is the Single Responsibility Principle. It is important to find responsibilities and extract classes when required. You can start applying it at the implementation level. It makes it easier to introduce it at interface level later.

Try to describe the responsibility of the class in a single sentence.

The best way to get better at finding responsibilities is to read more books about design patterns and in particular to read more code.

You can identify responsibilities using the following techniques:

• Method Grouping
• Hidden Methods: many private or protected methods indicates that there is another class
• Coupling between variables and methods
• Sketch (dependency graph between methods)

When you do refactoring, remember to do just one thing at a time.

Programming is the art of doing one thing at a time!

Ask your partner to challenge you constantly asking: What are you doing?

Remove duplication as much as possible! 

You end up with very small focused methods. The goal is to have orthogonality in the system. Your system is orthogonal when there is only one place you have to go to make a change. Removing duplication often reveals design.

Remember, code is your house, and you have to live in it.

Rename Class is the most powerful refactoring. It changes the way people see code and lets them notice possibilities that they might not have considered before.

Consider the Command/Query Separation Design Principle. 

A method should be a command or a query, but not both. A command is a method that can modify the state of the object but that doesn't return a value. A query is a method that returns a value but that does not modify the object.

Extract Method is a core technique for working with legacy code. You can use it to extract duplication, separate responsibilities, and break down long methods. It is also recommended to extract methods in a bottom up approach. Extract small pieces first!

Conclusion

It is true that working with legacy code can be frustrating at times. However, it is only a matter of perspective and approach. Working with legacy code can be fun and doing it using pair programming is a way to get a better result.

Surely, working with legacy code is a challenge and offers the opportunity to significantly improve your software developer skills. In any case, I totally agree with what Michael Feather say at the end of the book.

There isn't much that can replace working in a good environment with people you respect who know how to have fun at work

The Pragmatic Programmer

In this post I would like to collect the best advices from the book The Pragmatic Programmer.

The book start with some interesting sentences about the role of programmers that is often wrongly perceived.

If you don’t think carefully you might think that programming is just typing statements in a programming language.

Are you a pragmatic programmer?

If you are a pragmatic programmer, you'll share many of the following characteristics:

• Early adopter / fast adapter
• You have an instinct for technologies and techniques, and you love trying things out. When given something new, you can grasp it quickly and integrate it with the rest of your knowledge. Your confidence is born of experience.
• Inquisitive
• You tend to ask questions.
• Critical thinker
• You rarely take things as given without first getting the facts.
• Realistic
• You try to understand the underlying nature of each problem you face. This realism gives you a good feel for how difficult things are, and how long things will take.
• Jack of all trades
• You try hard to be familiar with a broad range of technologies and environments, and you work to keep abreast of new developments. Although your current job may require you to be a specialist, you will always be able to move on to new areas and new challenges

Self-Development and Career

• Take responsibility for everything you do.
• Every day, work to refine the skills you have and to add new tools to you repertoire.
• A pragmatic programmer takes charge of his or her own career, and isn't afraid to admit ignorance or error.
• Don’t blame someone or something else, or make up an excuse. Don’t blame all the problems on a vendor, a programming language, management, or your co-workers. Any and all of these may play a role, but it is up to you to provide solutions, not excuses. Instead of excuses, provide options. Don’t say it can’t be done; explain what can be done to salvage the situation.
• Don’t be afraid to ask, or to admit that you need help.
• You must invest in your knowledge portfolio regularly
• You need to know the ins and outs of the particular technology you are working with currently. But don’t stop there. The face of computing change rapidly. The more technologies you are comfortable with, the better you will be able to adjust to change. Stay current!
• Learning an emerging technology before it becomes popular can be just as hard as finding an undervalued stock, but the payoff can be just as rewarding.
• Learn at least one new language every year.
• Read a technical book each quarter (and then each month).
• Read non-technical books, too.
• Think critically about what you read and hear.
• Make a point of reading other people’s source code and documentation, either informally or during code reviews. You’re not snooping- you’re learning from them. Even if your project doesn't use that technology, perhaps you can borrow some ideas.
• There are two world-class professional societies for programmers: the Association for Computing Machinery (ACM) and the IEEE Computer Society. We recommend that all programmers belong to one (or both) of these societies. In addition, developers outside the USA may want to join their national societies, such as the BCS in the United Kingdom.

Perfectionism

• Don’t spoil a perfectly good program by over embellishment and over-refinement. Move on, and let your code stand in its own right for a while. It may not be perfect. Don’t worry: it could never be perfect.
• In addition to doing your own personal best, you must analyze the situation for risks that are beyond your control.
• There is no such thing as a best solution, be it a tool, a language, or an operating system. There can only be systems that are more appropriate in a particular set of circumstances.You shouldn't be wedded to any particular technology, but have a broad enough background and experience base to allow you to choose good solutions in particular situations.
• Because we can’t write perfect software, it follows that we can’t write perfect test software either. We need to test the tests. After you have written a test to detect a particular bug, cause the bug deliberately and make sure the test complains.

People

• Participate in local user groups.
• Care and cultivation of gurus is important.
• Turn the meeting into a dialog, and you’ll make your point more effectively. Who knows, you might even learn something.
• There’s one technique that you must use if you want people to listen to you: listen to them.
• A good idea is an orphan without effective communication.
• Dealing with computer systems is hard. Dealing with people is even harder.
• Work out what you can reasonably ask for. Develop it well. Once you've got it, show people, and let them marvel. Then say “of course, it would be better if we added…”. Pretend it’s not important. Sit back and wait for them to start asking you to add the functionality you originally wanted. People find it easier to join an ongoing success.
• Organize your resources (people) using the same techniques you use to organize code.

General Advices

• Always respond to e-mails and voice mails, even if the response is simply “I’ll get back to you later”.
• DRY – Don’t Repeat Yourself
• It’s a great idea to record your estimates so you can see how close you were. When an estimate turns out wrong, don’t just shrug and walk away. Find out why it differed from you guess.
• What to say when asked for an estimate: “I’ll get back to you”.
• It’s important to discover the underlying reason why users do a particular thing, rather than just the way they currently do it.
• We want to see pride of ownership. People should see your name on a piece of code and expect it to be solid, well written, tested, and documented.

Technical Advices

• Never run on auto-pilot. Constantly be thinking, critiquing your work in real time.
• We want to design components that are self-contained: independent, and with a single, well-defined purpose. An orthogonal approach reduces the risks inherent in any development.
• Be careful to preserve the orthogonality of your system as you introduce third-party toolkits and libraries. Choose your technologies wisely.
• Normally, you can simply hide a third-party product behind a well-defined, abstract interface. In fact, we've always been able to do so on any project we've worked on.
• Write code using the vocabulary of the application domain.
• Tools amplify your talent. The better you tools, and the better you know how to use them, the more productive you can be.
• The best format for storing knowledge persistently is plain text.
• GUI environments are normally limited to the capabilities that their designers intended. The command line is better suited when you want to quickly combine a couple of commands to perform a query or some other task. Invest some energy in becoming familiar with your shell and things will soon start falling into place. Investigate alternatives to you current shell.
• Make sure that the editor you choose is available on all platforms you use.
• Embrace the fact that debugging is just problem solving and attack it as such. It doesn't really matter whether the bug is your fault or someone else’s. It is still your problem.
• Set compiler warning levels as high as possible. Concentrate on the harder problems at hand. You must brutally test both boundary conditions and realistic end-user usage patterns. You need to do this systematically.
• The best way to start fixing a bug is to make it reproducible.
• A very simple but particularly useful technique for finding the cause of a problem is simply to explain it to someone else. They do not need to say a word; the simple act of explaining, step by step, what the code is supposed to do often causes the problem to leap off the screen and announce itself.
• Once a human tester finds a bug, it should be the last time a human tester finds that bug.
• When faced with a “surprising” failure, you must realize that one or more of your assumptions is wrong. Don’t gloss over a routing or piece of code involved in the bug because you “know” it works. Prove it. Prove it in this context, with this data, with these boundary conditions.
• If it can’t happen, use assertions to ensure that it won’t. Never put code that must be executed into an assert. Don’t use assertions in place of real error handling. Assertions check for things that should never happen.
• Exceptions should rarely be used as part of a program’s normal flow.
• We need to allow for concurrency and to think about decoupling any time or order dependencies.
• Coding is not mechanical. There are decisions to be made every minute- decisions that require careful thought and judgment if the resulting program is to enjoy a long, accurate, and productive life. Pragmatic programmers thing critically about all code, including our own. We constantly see room for improvement in our programs and our designs. You should be careful of tools that write reams of code on your behalf unless you understand what they’re doing.
• For routines you call, rely only on documented behavior. If you can’t, for whatever reason, then document your assumption well.
• Don’t be a slave to history. Don’t let existing code dictate future code.
• Software development is still not a science. Let your instincts contribute to your performance.
• The only timing that counts is the speed of your code, running in the production environment, with real data. Be wary of premature optimization. It’s always a good idea to make sure an algorithm really is a bottleneck before investing your precious time trying to improve it.
• Time pressure is often used as an excuse for not refactoring. But this excuse just doesn't hold up: fail to refactor now, and there’ll be a far greater time investment to fix the problem down the road- when there are more dependencies to reckon with.
• Keep track of the things that need to be refactored.
• Refactoring is an activity that needs to be undertaken slowly, deliberately and carefully. Don’t try to refactor and add functionality at the same time. Make sure you have good tests before you begin refactoring.
• We need to build testability into the software from the very beginning, and test each piece thoroughly before trying to wire them together.
• It’s convenient, but not always practical, for each class or module to contain its own unit test.
• Don’t be a slave to any notation; use whatever method best communicates the requirements with your audience. A big danger in producing a requirements document is being too specific. Good requirements documents remain abstract.
• Many projects failures are blamed on an increase in scope. It’s easy to get sucked into the “just one more feature” maelstrom.
• Create and maintain a project glossary.
• Publishing of project documents to internal web sites for easy access by all participants is particularly useful for requirements documents.
• When faced with an intractable problem, enumerate all the possible avenues you have before you. Don’t dismiss anything, no matter how unusable or stupid it sounds. Now go through the list and explain why a certain path cannot be taken. Are you sure? Can you prove it?
• Great performers share a trait: they know when to start and when to wait.
• Software development is still not a science. Let your instincts contribute to your performance.
• Blindly adopting any technique without putting it into the context of your development practices and capabilities is a recipe for disappointment.
• We prefer to understand the whole of the system we’re working on. It may not be possible to have an in-depth grasp of every aspect of a system, but you should know how the components interact, where the data lives, and what the requirements are.
• Never underestimate the cost of adopting new tools and methods.
• Pragmatic programmers look at methodologies critically, then extract the best from each and meld them into a set of working practices that gets better each month.
• There are advantages to being a pragmatic individual, but these advantages are multiplied many fold if the individual is working on a pragmatic team. The team speaks with one voice-externally. Internally, we strongly encourage lively, robust debate. Good developers tend to be passionate about their work.
• Compile projects with make files, even when using an IDE environment.
• Many development teams use an internal web site for project communication, and we think this is a great idea.
• As with validation and verification, you need to perform usability testing as early as you can, while there is still time to make corrections.
• Use metrics: cyclomatic complexity, inheritance fan-in and fan-out, response set, class coupling ratio, …
• Pragmatic programmers embrace documentation as an integral part of the overall development process.
• In reality, the success of a project is measured by how well it meets the expectations of its users. Never lose sight of the business problems your application is intended to solve.

Programmers are constantly in maintenance mode. Our understanding changes day by day.

Pragmatic programmers should always be learning.