Category Archives: Design Patterns

Evolving Toward a Persistence Layer

One of the most confusing design pattern is persistence. The need for an application to persist its internal state and data is so tremendous that there are likely tens – if not hundreds – of different technologies to address this single problem. Unfortunately, no technology is a magic bullet. Each application, and sometimes each component of the application, is unique in its own way – thus, requiring a unique solution.

In this tutorial, I will teach you some best practices to help you determine which approach to take, when working on future applications. I will briefly discuss some high level design concerns and principles, followed by a more detailed view on the Active Record design pattern, combined with a few words about the Table Data Gateway design pattern.

Of course, I will not merely teach you the theory behind the design, but I will also guide you through an example that begins as random code and transforms into a structured persistence solution.

Today, no programmer can understand this archaic system.

The oldest project I have to work on began in the year 2000. Back then, a team of programmers started a new project by evaluating different requirements, thought about the workloads the application will have to handle, tested different technologies and reached a conclusion: all the PHP code of the application, except the index.php file, should reside in a MySQL database. Their decision may sound outrageous today, but it was acceptable twelve years ago (OK… maybe not).

They started by creating their base tables, and then other tables for each web page. The solution worked… for a time. The original authors knew how to maintain it, but then each author left one by one–leaving the code base in the hands of other newcomers.

Today, no programmer can understand this archaic system. Everything starts with a MySQL query fromindex.php. The result of that query returns some PHP code that executes even more queries. The simplest scenario involves at least five database tables. Naturally, there are no tests or specifications. Modifying anything is a no-go, and we simply have to rewrite the entire module if something goes wrong.

The original developers ignored the fact that a database should only contain data, not business logic or presentation. They mixed PHP and HTML code with MySQL and ignored high level design concepts.

All applications should concentrate on respecting a clean, high level design.

As time passed, the new programmers needed to add additional features to the system while, at the same time, fixing old bugs. There was no way to continue using MySQL tables for everything, and everyone involved in maintaining the code agreed that its design was horribly flawed. So the new programmers evaluated different requirements, thought about the workloads the application will have to handle, tested different technologies and reached a conclusion: they decided to move as much code as possible to the final presentation. Again, this decision may sound outrageous today, but it was light years from the previous outrageous design.

The developers adopted a templating framework and based the application around it, starting every new feature and module with a new template. It was easy; the template was descriptive and they knew where to find the code that performs a specific task. But that’s how they ended up with template files containing the engine’s Domain Specific Language (DSL), HTML, PHP and of course MySQL queries.

Today, my team just watches and wonders. It is a miracle that many of the views actually work. It can take a hefty amount of time just to determine how information gets from the database to the view. Like its predecessor, it’s all a big mess!

Those developers ignored the fact that a view should not contain business or persistence logic. They mixed PHP and HTML code with MySQL and ignored high level design concepts.

High level schema

A mock is an object that acts like its real counterpart, but doesn’t execute the real code.

All applications should concentrate on respecting a clean, high level design. This is not always achievable, but it should be a high priority. A good high level design has well-isolated business logic. Object creation, persistence, and delivery are outside of the core and dependencies point only toward the business logic.

Isolating the business logic opens the door to great possibilities, and everything becomes somewhat of a plugin, if the external dependencies always point towards the business logic. For example, you could swap the heavy MySQL database with a lightweight SQLite3 database.

  • Imagine being able to drop your current MVC framework and replacing it with another, without touching the business logic.
  • Imagine delivering the results of your application through a third party API and not over HTTP, or changing any third party technology you use today (except the programming language of course) without touching the business logic (or without much hassle).
  • Imagine making all these changes and your tests would still pass.

To better identify the problems with a bad, albeit working, design, I will start with a simple example of, you guessed it, a blog. Throughout this tutorial, I will follow some test-driven development (TDD) principles and make the tests easily understandable – even if you don’t have TDD experience. Let’s imagine that you use a MVC framework. When saving a blog post, a controller named BlogPost executes a save() method. This method connects to an SQLite database to store a blog post in the database.

Let’s create a folder, called Data in our code’s folder and browse to that directory in the console. Create a database and a table, like this:

Our save() method gets the values from the form as an array, called $data:

This code works, and you can verify it by calling it from another class, passing a predefined $data array, like this:

The content of the $data variable was indeed saved in the database:

Inheritance is the strongest type of dependency.

A characterization test describes and verifies the current behavior of preexisting code. It is most frequently used to characterize legacy code, and it makes refactoring that code much easier.

A characterization test can test a module, a unit, or go all the way from the UI to the database; it all depends on what we want to test. In our case, such a test should exercise the controller and verify the contents of the database. This is not a typical unit, functional, or integration test, and it usually cannot be associated with either of those testing levels.

Characterization tests are a temporary safety net, and we typically delete them after the code is properly refactored and unit tested. Here is an implementation of a test, placed in the Test folder:

This test creates a new controller object and executes its save() method. The test then reads the information from the database and compares it with the predefined $data[] array. We preform this comparison by using the $this->assertEquals() method, an assertion that presumes that its parameters are equal. If they are different, the test fails. Also, we clean the BlogPosts database table each time we run the test.

Legacy code is untested code. – Michael Feathers

With our test up and running, let’s clean a little of the code. Open the database with the whole directory name and use sprintf() to compose the query string. This results in much simpler code:

Gateway Pattern

We recognize that our code needs to be moved from the controller to the business logic and persistence layer, and the Gateway Pattern can help us get started down that path. Here is the revised testSave()method:

This represents how we want to use the save() method on the controller. We expect the controller to call a method named persist($blogPostObject) on the gateway object. Let’s change our BlogPostController to do that:

A good high level design has a well isolated business logic.

Nice! Our BlogPostController became much simpler. It uses the gateway (either supplied or instantiated) to persist the data by calling its persist() method. There is absolutely no knowledge about how the data is persisted; the persistence logic became modular.

In the previous test, we created the controller with a mock persistence object, ensuring that data never gets written to the database when running the test. In production code, the controller creates its own persisting object to persist the data using a SqlitePost object. A mock is an object that acts like its real counterpart, but it doesn’t execute the real code.

Now let’s retrieve a blog post from the data store. It’s just as easy as saving data, but please note that I refactored the test a bit.

And the implementation in the BlogPostController is just a one statement method:

Isn’t this cool? The BlogPost class is now part of the business logic (remember the high level design schema from above). The UI/MVC creates BlogPost objects and uses concrete Gateway implementations to persist the data. All dependencies point to the business logic.

There’s only one step left: create a concrete implementation of Gateway. Following is the SqlitePost class:

Note: The test for this implementation is also available in the source code, but, due to its complexity and length, I did not include it here.

Active Record is one of the most controversial patterns. Some embrace it (like Rails and CakePHP), and others avoid it. Many Object Relational Mapping (ORM) applications use this pattern to save objects in tables. Here is its schema:

Active record pattern

As you can see, Active Record-based objects can persist and retrieve themselves. This is usually achieved by extending an ActiveRecordBase class, a class that knows how to work with the database.

The biggest problem with Active Record is the extends dependency. As we all know, inheritance is the strongest type of dependency, and it’s best to avoid it most of the time.

Before we go further, here is where we are right now:

Gateway in high level schema

The gateway interface belongs to the business logic, and its concrete implementations belong to the persistence layer. Our BlogPostController has two dependencies, both pointing toward the business logic: the SqlitePost gateway and BlogPost class.

 

There are many other patterns, like the Proxy Pattern, that are closely related to persistence.

If we were to follow the Active Record pattern exactly as it is presented by Martin Fowler in his 2003 book,Patterns of Enterprise Application Architecture, then we would need to move the SQL queries into theBlogPost class. This, however, has the problem of violating both the Dependency Inversion Principle and the Open Closed Principle. The Dependency Inversion Principle states that:

  • High-level modules should not depend on low-level modules. Both should depend on abstractions.
  • Abstractions should not depend upon details. Details should depend upon abstractions.

And the Open Closed Principle states: software entities (classes, modules, functions, etc.) should be open for extension but closed for modification. We will take a more interesting approach and integrate the gateway into our Active Record solution.

If you try to do this on your own, you probably already realized that adding the Active Record pattern to the code will mess things up. For this reason, I took the option of disabling the controller and SqlitePost tests to concentrate only on the BlogPost class. The first steps are: make BlogPost load itself by setting its constructor as private and connect it to the gateway interface. Here is the first version of the BlogPostTestfile:

It tests that a blog post is correctly initialized and that it can have a gateway if set. It is a good practice to use multiple asserts when they all test the same concept and logic.

Our second test has several assertions, but all of them refer to the same common concept of empty blog post. Of course, the BlogPost class has also been modified:

It now has a load() method that returns a new object with a valid gateway. From this point on, we will continue with the implementation of a load($title) method to create a new BlogPost with information from the database. For easy testing, I implemented an InMemoryPost class for persistence. It just keeps a list of objects in memory and returns information as desired:

Next, I realized that the initial idea of connecting the BlogPost to a gateway via a separate method was useless. So, I modified the tests, accordingly:

As you can see, I radically changed the way BlogPost is used.

The load() method checks the $content parameter for a value and creates a new BlogPost if a value was supplied. If not, the method tries to find a blog post with the given title. If a post is found, it is returned; if there is none, the method creates an empty BlogPost object.

In order for this code to work, we will also need to change how the gateway works. Our implementation needs to return an associative array with title, content, and timestamp elements instead of the object itself. This is a convention I’ve chosen. You may find other variants, like a plain array, more attractive. Here are the modifications in SqlitePostTest:

And the implementation changes are:

We are almost done. Add a persist() method to the BlogPost and call all the newly implemented methods from the controller. Here is the persist() method that will just use the gateway’s persist() method:

And the controller:

The BlogPostController became so simple that I removed all of its tests. It simply calls the BlogPostobject’s persist() method. Naturally, you’ll want to add tests if, and when, you have more code in the controller. The code download still contains a test file for the BlogPostController, but its content is commented.

This is just the tip of the iceberg.

You’ve seen two different persistence implementations: the Gateway and Active Record patterns. From this point, you can implement an ActiveRecordBase abstract class to extend for all your classes that need persistence. This abstract class can use different gateways in order to persist data, and each implementation can even use different logic to fit your needs.

But this is just the tip of the iceberg. There are many other patterns, such as the Proxy Pattern, which are closely related to persistence; each pattern works for a particular situation. I recommend that you always implement the simplest solution first, and then implement another pattern when your needs change.

The Dependency Inversion Principle

It would be unjust to tell you that any one of the SOLID principles is more important than another. However, probably none of the others have such an immediate and profound effect on your code than the Dependency Inversion Principle, or DIP in short. If you find the other principles hard to grasp or apply, start with this one and apply the rest on code that already respects DIP.

A. High-level modules should not depend on low-level modules. Both should depend on abstractions.
B. Abstractions should not depend upon details. Details should depend upon abstractions.

This principle was defined by Robert C. Martin in his book Agile Software Development, Principles, Patterns, and Practices and later republished in the C# version of the book Agile Principles, Patterns, and Practices in C#, and it is the last of the five SOLID agile principles.

Before we start coding, I would like to tell you a story. At Syneto, we weren’t always so careful with our code. A few years ago we knew less and even though we tried to do our best, not all of our projects were so nice. We went through hell and back again and we learned many things by trial and error.

The SOLID principles and the clean architecture principles of Uncle Bob (Robert C. Martin) became a game changer for us and transformed our way of coding in ways that are hard to describe. I will try to exemplify, in a nutshell, a few key architectural decisions imposed by DIP that had a great impact on our projects.

Most web projects contain three main technologies: HTML, PHP, and SQL. The particular version of these applications we are talking about or what type of SQL implementations you use is irrelevant. The thing is, that information from an HTML form must end up, in one way or another, in the database. The glue between the two can be provided with PHP.

What is essential to take away from this, is that how nicely the three technologies represent three different architectural layers: user interface, business logic, and persistence. We will talk about the implications of these layers in a minute. For now, let’s focus on some odd but frequently encountered solutions to make the technologies work together.

Many times I’ve seen projects that used SQL code in a PHP tag inside an HTML file, or PHP code echoing pages and pages of HTML and interpreting directly the $_GET or $_POST global variables. But why is this bad?

html-php-sql-cross-dependencies

The images above represent a raw version of what we described in the previous paragraph. The arrows represent various dependencies, and as we can conclude, basically everything depends on everything. If we need to change a database table, we may end up editing an HTML file. Or if we change a field in HTML, we may end up changing the name of a column in an SQL statement. Or if we look at the second schema, we may very well need to modify our PHP if the HTML changes, or in very bad cases, when we generate all HTML content from inside a PHP file we will surely need to change a PHP file to modify HTML content. So, there is no doubt, the dependencies are zigzagging between the classes and modules. But it doesn’t end here. You can store procedures; PHP code in SQL tables.

html-php-sql-stored-procedures

In the schema above, queries to the SQL database return PHP code generated with data from the tables. These PHP functions or classes are doing other SQL queries which are returning different PHP code, and the cycle continues until finally all the information is obtained and returned… probably to the UI.

I know this may sound outrageous to many of you, but if you have not yet worked with a project invented and implemented in this manner, you surely will in your future career. Most existing projects, regardless of the programming languages used, were written with old principles in mind, by programmers who did not care or know enough to do better. If you are reading these tutorials, you are most likely a level higher than that. You are ready, or getting ready to respect your profession, to embrace your craft, and to do better.

The other option is to repeat the mistakes your predecessors made and live with the consequences. At Syneto, after one of our projects reached an almost unmaintainable state because of its old and cross-dependent architecture and we had to basically abandon it forever, we decided to never go back down that road again. Since then, we have striven to have a clean architecture which correctly respects the SOLID principles, and most importantly the Dependency Inversion Principle.

HighLevelDesign

What’s so amazing about this architecture is how the dependencies are pointing:

  • The user interface (in most cases a web MVC framework) or whatever other delivery mechanism there is for your project will depend on the business logic. Business logic is quite abstract. A user interface is very concrete. The UI is just a detail for the project, and it is also very volatile. Nothing should depend on the UI, nothing should depend on your MVC framework.
  • The other interesting observation we can make is that the persistence, the database, your MySQL or PostgreSQL, depends on the business logic. Your business logic is database agnostic. This allows exchanging persistence as you wish. If tomorrow you want to change MySQL with PostgreSQL or just plain text files, you can do that. You will, of course, need to implement a specific persistence layer for the new persistence method, but you will not need to modify a single line of code in your business logic. There is a more detailed explanation on the persistence topic in the Evolving Toward a Persistence Layer tutorial.
  • Finally, on the right of the business logic, outside of it, we have all the classes that are creating business logic classes. These are factories and classes created by the entry point to our application. Many people tend to think these belong to the business logic, but while they are creating business objects, their sole reason is to do this. They are classes just to help us create other classes. The business objects and the logic they provide are independent of these factories. We could use different patterns, like Simple Factory, Abstract Factory, Builder or plain object creation to provide the business logic. It doesn’t matter. Once the business objects are created they can do their job.

Applying the Dependency Inversion Principle (DIP) at an architectural level is quite easy if you respect the classic agile design patterns. Exercising and exemplifying it inside the business logic is quite easy also and can even be fun. We will imagine an e-book reader application.

We start developing our e-reader as a PDF reader. So far so good. We have a PDFReader class using a PDFBook. The read() function on the reader delegates to the book’s read() method. We just verify this by doing a regex check after a key part of the string returned by PDFBook‘s reader() method.

Please bear in mind that this is just an example. We will not implement the reading logic of PDF files or other file formats. That’s why our tests will just simply check for some basic strings. If we were to write the real application, the only difference would be how we test the different file formats. The dependency structure would be very similar to our example.

pdfreader-pdfbook

Having a PDF reader using a PDF book may be a sound solution for a limited application. If our scope was to write a PDF reader and nothing more, it would actually be an acceptable solution. But we want to write a generic e-book reader, supporting several formats, amongst which our first implemented version PDF. Let’s rename our reader class.

Renaming had no functional counter effects. The tests are still passing.

Testing started at 1:04 PM …
PHPUnit 3.7.28 by Sebastian Bergmann.
Time: 13 ms, Memory: 2.50Mb
OK (1 test, 1 assertion)
Process finished with exit code 0

But it has a serious design effect.

ebookreader-pdfbook

Our reader became much more abstract. Much more general. We have a generic EBookReader that uses a very specific book type, PDFBook. An abstraction depends on a detail. The fact that our book is of type PDF should only be a detail, and no one should depend on it.

The most common, and most frequently used solution to invert the dependency is to introduce a more abstract module in our design. “The most abstract element in OOP is an Interface. Thus, any other class can depend on an Interface and still respect DIP”.

We created an interface for our reader. The interface is called EBook and represents the needs of the EBookReader. This is a direct result of respecting the Interface Segregation Principle (ISP) which promotes the idea that interfaces should reflect the needs of the clients. Interfaces belong to the clients, and thus they are named to reflect the types and objects the clients need and they will contain methods the clients wants to use. It is only natural for an EBookReader to use EBooks and have a read() method.

ebookreader-ebookinterface-pdfbook

Instead of a single dependency, we have two dependencies now.

  • The first dependency points from EBookReader toward the EBook interface and it is of type usage. EBookReader uses EBooks.
  • The second dependency is different. It points from PDFBook toward the same EBook interface but it is of type implementation. A PDFBook is just a particular form of EBook, and thus implements that interface to satisfy the client’s needs.

Unsurprisingly, this solution also allows us to plug in different types of ebooks into our reader. The single condition for all these books is to satisfy the EBook interface and implement it.

Which in turn leads us to The Open/Closed Principle, and the circle is closed.

The Dependency Inversion Principle is one that leads or helps us respect all the other principles. Respecting DIP will:

  • Almost force you into respecting OCP.
  • Allow you to separate responsibilities.
  • Make you correctly use subtyping.
  • Offer you the opportunity to segregate your interfaces.

That’s it. We are done. All tutorials about the SOLID principles are complete. For me, personally, discovering these principles and implementing projects with them in mind was a huge change. I completely changed the way I think about design and architecture and I can say since then all the projects I work on are exponentially easier to manage and understand.

I consider the SOLID principles one of the most essential concepts of object-oriented design. These concepts that must guide us in making our code better and our life as programmers much much easier. Well-designed code is easier for programmers to understand. Computers are smart, they can understand code regardless of its complexity. Human beings on the other hand have a limited number of things they can keep in their active, focused mind. More specifically, the number of such things is The Magical Number Seven, Plus or Minus Two.

We should strive to have our code structured around these numbers and there are several techniques that help us do so. Functions with a maximum of four lines in length (five with the definition line included) so that they can all fit at once within our mind. Indentations not passing five levels deep. Classes with no more than nine methods. Design patterns that usually use a number of five to nine classes. Our high level design in the schemas above uses four to five concepts. There are five SOLID principles, each requiring five to nine sub-concepts/modules/classes to be exemplified. The ideal size of a programming team is between five and nine. The ideal number of teams in a company is between five and nine.

As you can see, the magical number seven, plus or minus two is all around us, so why should your code be different?

Interface Segregation Principle

The Single Responsibility Principle is about actors and high level architecture. The Open/Closed Principle is about class design and feature extensions. The Liskov Substitution Principle is about subtyping and inheritance. The Interface Segregation Principle (ISP) is about business logic to clients communication.

In all modular applications there must be some kind of interface that the client can rely on. These may be actual Interface typed entities or other classic objects implementing design patterns like Facades. It doesn’t matter which solution is used. It always has the same scope: to communicate to the client code on how to use the module. These interfaces can reside between different modules in the same application or project, or between one project as a third party library serving another project. Again, it doesn’t matter. Communication is communication and clients are clients, regardless of the actual individuals writing the code.

So, how should we define these interfaces? We could think about our module and expose all the functionalities we want it to offer.

hugeInterface

This looks like a good start, a great way to define what we want to implement in our module. Or is it? A start like this will lead to one of two possible implementations:

  • A huge Car or Bus class implementing all the methods on the Vehicle interface. Only the sheer dimensions of such classes should tell us to avoid them at all costs.
  • Or, many small classes like LightsControl, SpeedControl, or RadioCD which are all implementing the whole interface but actually providing something useful only for the parts they implement.

It is obvious that neither solution is acceptable to implement our business logic.

specializedImplementationInterface

We could take another approach. Break the interface into pieces, specialized to each implementation. This would help to use small classes that care about their own interface. The objects implementing the interfaces will be used by the different type of vehicles, like car in the image above. The car will use the implementations but will depend on the interfaces. So a schema like the one below may be even more expressive.

carUsingInterface

But this fundamentally changes our perception of the architecture. The Car becomes the client instead of the implementation. We still want to provide to our clients ways to use our whole module, that being a type of vehicle.

oneInterfaceManyClients

Assume we solved the implementation problem and we have a stable business logic. The easiest thing to do is to provide a single interface with all the implementations and let the clients, in our case BusStation, HighWay, Driver and so on, to use whatever thew want from the interface’s implementation. Basically, this shifts the behavior selection responsibility to the clients. You can find this kind of solution in many older applications.

The interface-segregation principle (ISP) states that no client should be forced to depend on methods it does not use.

However, this solution has its problems. Now all the clients depend on all the methods. Why should a BusStation depend on the state of lights of the bus, or on the radio channels selected by the driver? It should not. But what if it does? Does it matter? Well, if we think about the Single Responsibility Principle, it is a sister concept to this one. If BusStation depends on many individual implementations, not even used by it, it may require changes if any of the individual small implementations change. This is especially true for compiled languages, but we can still see the effect of the LightControl change impacting BusStation. These things should never happen.

Interfaces belong to their clients and not to the implementations. Thus, we should always design them in a way to best suite our clients. Some times we can, some times we can not exactly know our clients. But when we can, we should break our interfaces in many smaller ones, so they better satisfy the exact needs of our clients.

segregatedInterfaces

Of course, this will lead to some degree of duplication. But remember! Interfaces are just plain function name definitions. There is no implementation of any kind of logic in them. So the duplications is small and manageable.

Then, we have the great advantage of clients depending only and only on what they actually need and use. In some cases, clients may use and need several interfaces, that is OK, as long as they use all the methods from all the interfaces they depend on.

Another nice trick is that in our business logic, a single class can implement several interfaces if needed. So we can provide a single implementation for all the common methods between the interfaces. The segregated interfaces will also force us to think of our code more from the client’s point of view, which will in turn lead to loose coupling and easy testing. So, not only have we made our code better to our clients, we also made it easier for ourselves to understand, test and implement.

ISP teaches us to respect our clients more than we thought necessary. Respecting their needs will make our code better and our lives as programmers easier.

Liskov Substitution Principle

Single Responsibility (SRP), Open/Close, Liskov’s Substitution, Interface Segregation, and Dependency Inversion. Five agile principles that should guide you every time you write code.

Child classes should never break the parent class’ type definitions.

The concept of this principle was introduced by Barbara Liskov in a 1987 conference keynote and later published in a paper together with Jannette Wing in 1994. Their original definition is as follows:

Let q(x) be a property provable about objects x of type T. Then q(y) should be provable for objects y of type S where S is a subtype of T.

Later on, with the publication of the SOLID principles by Robert C. Martin in his book Agile Software Development, Principles, Patterns, and Practices and then republished in the C# version of the book Agile Principles, Patterns, and Practices in C#, the definition became known as the Liskov Substitution Principle.

This leads us to the definition given by Robert C. Martin:

Subtypes must be substitutable for their base types.

As simple as that, a subclass should override the parent class’ methods in a way that does not break functionality from a client’s point of view. Here is a simple example to demonstrate the concept.

Given a class Vehicle – it may be abstract – and two implementations:

A client class should be able to use either of them, if it can use Vehicle.

Which leads us to a simple implementation of the Template Method Design Pattern as we used it in the OCP tutorial.

template_method

Based on our previous experience with the Open/Closed Principle, we can conclude that Liskov’s Substitution Principle is in strong relation with OCP. In fact, “a violation of LSP is a latent violation of OCP” (Robert C. Martin), and the Template Method Design Pattern is a classic example of respecting and implementing LSP, which in turn is one of the solutions to respect OCP also.

To illustrate this completely, we will go with a classic example because it is highly significant and easily understandable.

We start with a basic geometrical shape, a Rectangle. It is just a simple data object with setters and getters for width and height. Imagine that our application is working and it is already deployed to several clients. Now they need a new feature. They need to be able to manipulate squares.

In real life, in geometry, a square is a particular form of rectangle. So we could try to implement a Square class that extends a Rectangle class. It is frequently said that a child class is a parent class, and this expression also conforms to LSP, at least at first sight.

SquareRect

But is a Square really a Rectangle in programming?

A square is a rectangle with equal width and height, and we could do a strange implementation like in the above example. We could overwrite both setters to set the height as well as the width. But how would that affect client code?

It is conceivable to have a client class that verifies the rectangle’s area and throws an exception if it is wrong.

Of course we added the above method to our Rectangle class to provide the area.

And we created a simple test by sending an empty rectangle object to area verifier and the test passes. If our Square class is correctly defined, sending it to the Client’s areaVerifier() should not break its functionality. After all, a Square is a Rectangle in all mathematical sense. But is our class?

Testing it is very easy and it breaks big time. An exception is thrown to us when we run the test above.

So, our Square class is not a Rectangle after all. It breaks the laws of geometry. It fails and it violates the Liskov Substitution Principle.

I especially love this example because it not only violates LSP, it also demonstrates that object oriented programming is not about mapping real life to objects. Each object in our program must be an abstraction over a concept. If we try to map one-to-one real objects to programmed objects, we will almost always fail.

LSP taught us why reality can not be represented as a one-to-one relation with programmed objects and how subtypes should respect their parents. We also put it in light of the other principles that we already knew.

The Open/Closed Principle

Single Responsibility (SRP), Open/Closed (OCP), Liskov’s Substitution, Interface Segregation, and Dependency Inversion. Five agile principles that should guide you every time you need to write code.

Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.

The Open/Closed Principle, OCP in short, is credited to Bertrand Mayer, a French programmer, who first published it in his book n Object-Oriented Software Construction in 1988.

The principle rose in popularity in the early 2000s when it became one of the SOLID principles defined by Robert C. Martin in his book Agile Software Development, Principles, Patterns, and Practices and later republished in the C# version of the book Agile Principles, Patterns, and Practices in C#.

What we are basically talking about here is to design our modules, classes and functions in a way that when a new functionality is needed, we should not modify our existing code but rather write new code that will be used by existing code. This sounds a little bit strange, especially if we are working in languages like Java, C, C++ or C# where it applies not only to the source code itself but to the binary also. We want to create new features in ways that will not require us to redeploy existing binaries, executables or DLLs.

As we progress with these tutorials, we can put each new principle in the context of the already discussed ones. We already discussed the Single Responsibility (SRP) that stated that a module should have only one reason to change. If we think about OCP and SRP, we can observe that they are complementary. Code specifically designed with SRP in mind will be close to OCP principles or easy to make it respect those principles. When we have code that has a single reason to change, introducing a new feature will create a secondary reason for that change. So both SRP and OCP would be violated. In the same way, if we have code that should only change when its main function changes and should remain unchanged when a new feature is added to it, thus respecting OCP, will mostly respect SRP also.

This does not mean that SRP always leads to OCP or vice versa, but in most cases if one of them is respected, achieving the second one is quite simple.

From a purely technical point of view, the Open/Closed Principle is very simple. A simple relationship between two classes, like the one below violates the OCP.

violate1

The User class uses the Logic class directly. If we need to implement a second Logic class in a way that will allow us to use both the current one and the new one, the existing Logic class will need to be changed. User is directly tied to the implementation of Logic, there is no way for us to provide a new Logic without affecting the current one. And when we are talking about statically typed languages, it is very possible that the User class will also require changes. If we are talking about compiled languages, most certainly both the User executable and the Logic executable or dynamic library will require recompilation and redeployment to our clients, a process we want to avoid whenever possible.

Based only on the schema above, one can deduce that any class directly using another class would actually violate the Open/Closed Principle. And that is right, strictly speaking. I found it quite interesting to find the limits, the moment when you draw the line and decide that it is more difficult to respect OCP than modify existing code, or the architectural cost does not justify the cost of changing existing code.

Let’s say we want to write a class that can provide progress as a percent for a file that is downloaded through our application. We will have two main classes, a Progress and a File, and I imagine we will want to use them like in the test below.

In this test we are a user of Progress. We want to obtain a value as a percent, regardless of the actual file size. We use File as the source of information for our Progress. A file has a length in bytes and a field called sent representing the amount of data sent to the one doing the download. We do not care about how these values are updated in the application. We can assume there is some magical logic doing it for us, so in a test we can set them explicitly.

The File class is just a simple data object containing the two fields. Of course in real life, it would probably contain other information and behavior also, like file name, path, relative path, current directory, type, permissions and so on.

Progress is simply a class taking a File in its constructor. For clarity, we specified the type of the variable in the constructor’s parameters. There is a single useful method on Progress, getAsPercent(), which will take the values sent and length from File and transform them into a percent. Simple, and it works.

This code seems to be right, however it violates the Open/Closed Principle. But why? And How?

Every application that is expected to evolve in time will need new features. One new feature for our application could be to allow streaming of music, instead of just downloading files. File‘s length is represented in bytes, the music’s duration in seconds. We want to offer a nice progress bar to our listeners, but can we reuse the one we already have?

No, we can not. Our progress is bound to File. It understands only files, even though it could be applied to music content also. But in order to do that we have to modify it, we have to make Progress know about Music and File. If our design would respect OCP, we would not need to touch File or Progress. We could just simply reuse the existing Progress and apply it to Music.

Dynamically typed languages have the advantages of guessing the types of objects at runtime. This allows us to remove the typehint from Progress‘ constructor and the code will still work.

Now we can throw anything at Progress. And by anything, I mean literally anything:

And a Music class like the one above will work just fine. We can test it easily with a very similar test to File.

So basically, any measurable content can be used with the Progress class. Maybe we should express this in code by changing the variable’s name also:

Good, but we have a huge problem with this approach. When we had File specified as a typehint, we were positive about what our class can handle. It was explicit and if something else came in, a nice error told us so.

But without the typehint, we must rely on the fact that whatever comes in will have two public variables of some exact names like “length” and “sent“. Otherwise we will have a refused bequest.

Refused bequest: a class that overrides a method of a base class in such a way that the contract of the base class is not honored by the derived class. ~Source Wikipedia.

This is one of the code smells presented in much more detail in the Detecting Code Smells premium course. In short, we do not want to end up trying to call methods or access fields on objects that do not conform to our contract. When we had a typehint, the contract was specified by it. The fields and methods of the File class. Now that we have nothing, we can send in anything, even a string and it would result in an ugly error.

A test like this, where we send in a simple string, will produce a refused bequest:

While the end result is the same in both cases, meaning the code breaks, the first one produced a nice message. This one, however, is very obscure. There is no way of knowing what the variable is – a string in our case – and what properties were looked for and not found. It is difficult to debug and to solve the problem. A programmer needs to open the Progress class and read it and understand it. The contract, in this case, when we do not explicitly specify the typehint, is defined by the behavior of Progress. It is an implicit contract, known only to Progress. In our example, it is defined by the access to the two fields, sent and length, in the getAsPercent() method. In real life the implicit contract can be very complex and hard to discover by just looking for a few seconds at the class.

This solution is recommended only if none of the other suggestions below can easily be implemented or if they would inflict serious architectural changes that do not justify the effort.

This is the most common and probably the most appropriate solution to respect OCP. It is simple and effective.

strategy

The Strategy Pattern simply introduces the use of an interface. An interface is a special type of entity in Object Oriented Programming (OOP) which defines a contract between a client and a server class. Both classes will adhere to the contract to ensure the expected behavior. There may be several, unrelated, server classes that respect the same contract thus being capable of serving the same client class.

In an interface we can define only behavior. That is why instead of directly using public variables we will have to think about using getters and setters. Adapting the other classes will not be difficult at this point. Our IDE can do most of the job.

As usual, we start with our tests. We will need to use setters to set the values. If considered mandatory, these setters may also be defined in the Measurable interface. However, be careful what you put there. The interface is to define the contract between the client class Progress and the different server classes like File and Music. Does Progress need to set the values? Probably not. So the setters are highly unlikely to be needed to be defined in the interface. Also, if you would define the setters there, you would force all of the server classes to implement setters. For some of them, it may be logical to have setters, but others may behave totally differently. What if we want to use our Progress class to show the temperature of our oven? The OvenTemperature class may be initialized with the values in the constructor, or obtain the information from a third class. Who knows? To have setters on that class would be odd.

The File class is modified slightly to accommodate the requirements above. It now implements the Measurable interface and has setters and getters for the fields we are interested in. Music is very similar, you can check its content in the attached source code. We are almost done.

Progress also needed a small update. We can now specify a type, using typehinting, in the constructor. The expected type is Measurable. Now we have an explicit contract. Progress can be sure the accessed methods will be always present because they are defined in the Measurable interface. File and Music can also be sure they can provide all that is needed for Progress by simply implementing all the methods on the interface, a requirement when a class implements an interface.

This design pattern is explained in greater detail in the Agile Design Patterns course.

People tend to name interfaces with a capital I in front of them, or with the word “Interface” attached at the end, like IFile or FileInterface. This is an old-style notation imposed by some outdated standards. We are so much past the Hungarian notations or the need to specify the type of a variable or object in its name in order to easier identify it. IDEs identify anything in a split second for us. This allows us to concentrate on what we actually want to abstract.

Interfaces belong to their clients. Yes. When you want to name an interface you must think of the client and forget about the implementation. When we named our interface Measurable we did so thinking about Progress. If I would be a progress, what would I need to be able to provide the percent? The answer is simple, something we can measure. Thus the name Measurable.

Another reason is that the implementation can be from various domains. In our case, there are files and music. But we may very well reuse our Progress in a racing simulator. In that case, the measured classes would be Speed, Fuel, etc. Nice, isn’t it?

The Template Method design pattern is very similar to the strategy, but instead of an interface it uses an abstract class. It is recommended to use a Template Method pattern when we have a client very specific to our application, with reduced reusability and when the server classes have common behavior.

template_method

 

So, how is all of this affecting our high level architecture?

HighLevelDesign

If the image above represents the current architecture of our application, adding a new module with five new classes (the blue ones) should affect our design in a moderate way (red class).

HighLevelDesignWithNewClasses

In most systems you can’t expect absolutely no effect on the existing code when new classes are introduced. However, respecting the Open/Closed Principle will considerably reduce the classes and modules that require constant change.

As with any other principle, try not to think about everything from before. If you do so, you will end up with an interface for each of your classes. Such a design will be hard to maintain and understand. Usually the safest way to go is to think about the possibilities and if you can determine whether there will be other types of server classes. Many times you can easily imagine a new feature or you can find one on the project’s backlog that will produce another server class. In those cases, add the interface from the beginning. If you can not determine, or if you are unsure – most of the time – simply omit it. Let the next programmer, or maybe even yourself, to add the interface when you need a second implementation.

If you follow your discipline and add interfaces as soon as a second server is needed, modifications will be few and easy. Remember, if code required changes once, there is a high possibility it will require change again. When that possibility turns into reality, OCP will save you a lot of time and effort.