Meaning Over Implementation

Conveying the meaning of your code is more important than conveying the implementation of your code.

This is a lesson I've learned that I thought was worth sharing.  It may sound pretty straight forward, but I've noticed it can be a hurdle when you're coming from a spaghetti code background.  Spaghetti code requires you to hunt down code to understand how the system works because the code is not 'meaningful.'

There are really two major components to this.  The first is, if you're used to spaghetti code, you've been trained to mistrust the code you're reading.  That makes it hard to come to terms with hiding implementation details behind well named methods, because you don't trust those methods to do what you need unless you know every detail of HOW they do it.

The second is in naming.  Generally I try to name methods so that they describe both what the method does as well as how it does it.  But this isn't always possible.  And when it's not possible, I've learned to always favor naming what the method is for as opposed to how the method is implemented.

There are a number of reason why this works out.  The first is that issue of trust.  If your code base sucks, you're forced to approach every class and every method with a certain amount of mistrust.  You expect it to do one thing, but you've been bitten enough times to know it's likely it does some other thing.  If this is the kind of code base you find yourself in, you're screwed.  Every programming technique we have at our disposal relies on abstraction.  And abstraction is all about hiding complexity.  But if you don't trust the abstractions, you can't build on them.

The other reason this works is the same reason SRP (the Single Responsibility Principle) works.  You can selectively pull back the curtain on each abstraction as needed.  This allows you to understand things from a high level, but then drill down one layer at a time into the details.  I like to think of this as breadth-first programming, as opposed to depth-first programming.

So when you're struggling with naming things, remember your first priority should be to name things so that others will understand WHAT it does.  They can read the implementation if they need to understand HOW it does it.

Abstraction: blessing or curse

Well this has been a fun week!  It's the first time I've ever done a series of posts like this, and all in one week.  But now it's time to wrap up and then rest my fingers for awhile.

All of this stuff has been about dealing with abstraction.  Seriously, think about what you spend most of your time doing when you're programming.  For me, I think it breaks down something like this:

• 50%: Figuring out why a framework/library/control/object isn't doing what I wanted it to do
• 30%: Deciding where to put things and what to name them
• 20%: Actually writing business logic/algorithms

So, yeah, those numbers are completely made up, but they feel right...  And look at how much time is spent just deciding where things should go!  Seriously, that's what "software engineering" really is.  What should I name this?  Where should I put this logic?  What concepts should I express?  How should things communicate with each other?  Is it OK for this thing to know about some other thing?

This is abstraction.  You could just write everything in one long file, top to bottom, probably in C.  We've all been there, to some degree, and it's a nightmare.  So we slice things up and try to hide details.  We try to massage the code into some form that makes it easier to understand, more intuitive.  Abstraction is our only way to deal with complexity.

But abstraction itself is a form of complexity!  We are literally fighting complexity with complexity.

I think this is where some of the art of programming comes into play.  One of the elements of beautiful code is how it's able to leverage complexity but appear simple.

All of our design patterns, principles, and heuristics (from Clean Code) can largely be viewed as suggestions for "where to put things and what to name them" that have worked for people in the past.  It's more like an aural tradition passed from generation to generation than it is like the strict rules of Civil Engineering.  This doesn't make any of these patterns less important, but it does help explain why good design is such a difficult job, even with all these patterns.

In this series of posts I've blabbed on about a number of different design topics, each time almost coming to some kind of conclusion, but never very definitively.  That's because this has been sort of a public thought experiment on my part.  I'm just broadcasting some of the topics that I have been curious about recently.  And I think where I would like to leave this is with one final look at the complexity of abstraction.

Abstraction, by definition, is about hiding details.  And anyone who has ever done any nontrivial work with SQL Server (or any database) will understand how this hiding of details can be a double edged sword.  If you had to understand everything in it's full detail, you could never make it work.  But sometimes those hidden details stick out in weird ways, causing things to not work as you were expecting.  The abstraction has failed you.  But this is inevitable when hiding details, at some point, one of those details with surprise you.

Indirection and abstraction aren't exactly the same thing, but they feel very related somehow, and I think Zed Shaw would forgive me for thinking that.  Indirection, just like abstraction, is a form of complexity.  And just like abstraction, it can be incredibly useful complexity.  I assert that every time you wrap an implementation with an interface, whether it's a Header Interface or an Inverted Interface, you have introduced indirection.

In Bob Martin's Dependency Inversion Principle article, he lays out 3 criteria of bad design:

1. Rigidity: Code is hard to change because every change affects too many parts of the system
2. Fragility: When you make a change, unexpected parts of the system break
3. Immobility: Code is difficult to reuse because it cannot be disentangled from the current application

I think there is clearly a 4th criteria which is: Complexity: Code is overly complicated making it difficult to understand and work with.  Every move in the direction of the first three criteria, is a move away from the fourth.  Our challenge then, indeed our art, is to find the sweet spot between these competing extremes.

And so I ask you, is it a beautiful code type of thing:

• To wrap every concept behind abstract objects (objects or data structures)?  
• To represent every behavior as a method of some stateful thing (service layers or oop)?
• To hide every class behind an indirection in the form of an interface (header interfaces or inverted interfaces)?
• To make every class reusable and plugable (DIP: loose or leaky)?

DIP: Loose or Leaky?

In the last post I looked a bit at the Dependency Inversion Principle which says to define an interface representing the contract of a dependency that you need someone to fulfill for you. It's a really great technique for encouraging SRP and loose coupling.

The whole idea behind the DIP is that your class depends only on the interface, and doesn't know anything about the concrete class implementing that interface.  It doesn't know what class it's using, nor does it know where that class came from.  And thus, we gain what we were after: the high level class can be reused in nearly any context by providing it with different dependencies.  And on top of that, we gain excellent separation of concerns, making our code base more flexible, more maintainable, and I'd argue more understandable.  Clearly, the DIP is awesome!

But!  I'm sure you were waiting for the But...  Since we now have to provide our object with it's dependencies, we have to know:

• Every interface it depends on
• What instances we should provide
• How to create those instances

This feels so leaky to me!  I used to have this beautiful, self contained, well abstracted object.  I refactor it to be all DIP-y, and now it's leaking all these details of it's internal implementation in the form of it's collaborators.  If I had to actually construct this object myself in order to use it, this would absolutely be a deal breaker!

Fortunately, someone invented Inversion of Control Containers, so we don't have to create these objects ourselves.  Or, maybe unfortunately, 'cause now we don't have to create these objects ourselves, which sweeps any unsavory design issues away where you wont see them...

What design issues?  Well, the leaking of implementation details obviously!  Are there others?  Probably.  Maybe class design issues with having too many dependencies?  Or having dependencies that are too small and should be rolled into more abstract concepts?  But neither of these is a result of injecting Inverted Interfaces, only the implementation leaking.

I do believe this is leaky, but I'm not really sure if it's a problem exactly.  At the end of the day, we're really just designing a plugin system.  We want code to be reusable, so we want to be able to dynamically specify what dependencies it uses.  We're not forced to pass the dependencies in, we could use a service locator type approach.  But this has downsides of it's own.

In the next post, I'll wrap this up by zooming back out and trying to tie this all together.

Header Interfaces or Inverted Interfaces?

Thanks to Derick Bailey for introducing me to this concept: Header Interfaces.  A Header Interface is an interface of the same name as the class which exposes all of the class's methods, generally not intended to be implemented by any other classes, and usually introduced just for the purposes of testing.  Derick compared these to header files in C++. In my earlier Interfaces, DI, and IoC are the Devil post, these are really the kinds of interfaces I was rallying against.  (I'll use the ATM example from Bob Martin's ISP article throughout this post)

public class AtmUI : IAtmUI {...}

One of the things I really found interesting about this was the idea that there are different kinds of interfaces.  So if Header Interfaces are one kind, what other kinds are there?

The first things that came to my mind were the Interface Segregation Principle and the Dependency Inversion Principle.  ISP obviously deals directly with interfaces.  It basically says that interfaces should remain small.  "Small" is always a tricky word, in this case it's about making sure that the clients consuming the interfaces actually use all the methods of the interface.  The idea being that if the client does not use all the methods, then you're forcing the client to depend on methods it actually has no use for.  Apparently this is supposed to make things more brittle.

I said "apparently" because I've never directly felt this pain.  I guess it really only comes into play if you have separate compilable components and you are trying to reduce how many things have to be recompiled when you make a change.  I use .NET, so Visual Studio, and it's not a picnic for controlling compilable components...  I think I have this requirement though, and just haven't figured out how to deal with it in .NET.  But for the moment, lets assume we agree that ISP is a good thing.

public class AtmUI : IDepositUI, IWithdrawalUI, ITransferUI {...}

The DIP leads to a similar place as ISP.  It tells us that higher level components should not depend on lower level components.  There is some room for uncertainty here around which components are higher than others.  For example, is the AtmUI a higher level than the Transaction?  I'll go with no, because the Transaction is the actual driver of the application, the UI is just one of it's collaborators.  Because of this, the DIP leads us to create separate interfaces to be consumed by each Transaction:

public class AtmUI : IDepositUI, IWithdrawalUI, ITransferUI {...}

So, maybe there are at least two types of interfaces: Header Interfaces, and what I'll coin Inverted Interfaces.  In the last post I talked about the "Service Layer" pattern.  It generally leads to the creation of what feel more like Header Interfaces.  But this is tricky, because the only difference I can really find here is based on who owns the interface.  An Inverted Interface is owned by the class that consumes the interface, and a Header Interface is owned by the class that implements the interface.

But sometimes the difference isn't really that clear cut.  If you're TDDing your way through an application  top-down in the GOOS style, the Service Layers are designed and created based on the needs of the "higher" level components.  So the component, and it's interface, both spring into existence at the same time.  So if the service only has one consumer right now, the interface feels very Header-y.  On the other hand, it was created to fulfill the need of a higher level component; very Inverted-y.

But if someone else comes around and consumes the same service later: well now we have some thinking to do. If we reuse the interface, then I guess we've made it a Header Interface.  Would Uncle Bob have said to create a new interface but make the existing service implement it?  The lines are blurred because the components we're dealing with all reside within the same "package" and at least right now don't have any clear call to be reused outside this package.

Sadly, the introduction of these interfaces brings us back to Dependency Injection.  So in the next post, I'll look at the Dependency Inversion Principle, and the consequences of these Inverted Interfaces.

Service Layers or OOP?

In the last post, I mentioned that my co-workers and I had settled on a design that was working very well for us, but that wasn't very "object-oriented," at least not in the way Bob Martin described it.

We evolved to our approach through a lot of TDD and refactoring and plain old trial and error, but the final touches came when we  watched Gary Bernhardt's Destroy All Software screencasts and saw that he was using pretty much the same techniques, but with some nice naming patterns.

I don't know if there is a widely accepted name for this pattern, so I'm just going to call it the Service Layer Pattern.  It's biggest strengths are it's simplicity and clarity.  In a nutshell I'd describe it by saying that for every operation of your application, you create a "service" class.  You provide the necessary context to these services, in our case as Active Record objects (NOTE: service does NOT mean remote service (ie, REST), it just means a class that performs some function for us).

So far so basic, the real goodness comes when you add the layering.  I find there are a couple ways to look at this.  The more prescriptive is similar to DDD's (Domain Driven Design) "Layered Architecture" which recommends 4 layers: User Interface, Application, Domain, and Infrastructure.  From DDD:

The value of layers is that each specializes in a particular aspect of a computer program.  This specialization allows more cohesive designs of each aspect, and it makes these designs much easier to interpret.  Of course, it is vital to choose layers that isolate the most important cohesive design aspects.

In my Demonstrating the Costs of DI code examples the classes and layers looked like this:

SpeakerController (app)
 > PresentationApi (app)
   > OrdersSpeakers (domain)
   > PresentationSpeakers (domain)
     > Active Record (infrastructure)
   > Speaker (domain)
     > Active Record (infrastructure)

This concept of layering is very useful, but it's important not to think that a given operation will only have one service in each layer.  Another perspective on this that is less prescriptive but also more vague is the Single Responsibility Principle.  The layers emerge because you repeatedly refactor similar concepts into separate objects for each operation your code performs.  It's still useful to label these layers, because it adds some consistency to the code.

Each of these services is an object, but that doesn't make this an object-oriented design.  Quite the opposite, this is just well organized SRP procedural code.  Is this Service Layer approach inferior to the OOP design hinted at by Uncle Bob?  Or are these actually compatible approaches?

The OOP approach wants to leverage polymorphism to act on different types in the same way.  Does that mean that if I have a service, like OrdersParties, that I should move it onto the Party object?  What about the PartyApi class, should I find some way of replacing that with an object on which I could introduce new types?

There is a subtle but important distinction here.  Some algorithms are specific to a given type: User.Inactivate().  What it means to inactivate a user is specific to User.  Contrast that with User.HashPassword().  Hashing a password really has nothing to do with a user, except that a user needs a hashed password.  That is, the algorithm for hashing a password is not specific to the User type.  It could apply to any type, indeed to any string!  Defining it on User couples it to User, preventing it from being used on any string in some other context.

Further, some algorithms are bigger than a single type.  Ordering the speakers on a presentation doesn't just affect one speaker, it affects them all.  Think how awkward it would be for this algorithm to reside on the Speaker object.  Arguably, these methods could be placed on Presentation, but then presentation would have a lot of code that's not directly related to a presentation, but instead to how speakers are listed.  So it doesn't make sense on Speaker, or on Presentation.

Some algorithms are best represented as services, standing on their own, distinctly representing their concepts.  But these services could easily operate on Objects, as opposed to Data Structures.  Allowing them to apply to multiple types without needing to know anything about those specific types.  So I think the Service Layers approach is compatible with the OOP approach.

In the next post I'll take a look at how interfaces fit into this picture.

Objects or Data Structures?

Here's a great (and old) article from Bob Martin called Active Record vs Objects.  You should read it.  I think it might be one of the best treatments of the theoretical underpinnings of Object Oriented design I've read, especially because it pays a lot of heed to what OOP is good at, and what it's not good at.

Here's some of my highlights:

• Objects hide data and export behavior (very tell-don't-ask)
• Data structures expose data and have no behavior
• Algorithms that use objects are immune to the addition of new types
• Algorithms that use data structures are immune to the addition of new functions
• Apps should be structured around objects that expose behaviors and hide the database

This all feels right and stuff, but it's all pretty theoretical and doesn't help me decide if my code is designed as well as it could be.  And that's what I'm going to be writing about.  In one post a day for the rest of the week I'll look at various elements of "good design," and try to fit the pieces together in some way I can apply to my code.

Good designers uses this opposition to construct systems that are appropriately immune to the various forces that impinge upon them.

He's talking about the opposition between objects and data structures in terms of what they're good for.  So apparently a good designer is a psychic who can see the future.

But that is the hard part, how do you know where you'll need to add "types" vs. where you'll need to add "functions"?  Sometimes it's really obvious.  But what I'm starting to think about is, maybe I need to get more clever about the way I think about types.  Because if Uncle Bob thinks apps should be structured around objects, that means he thinks there are lots of examples where you're going to need to add a new type.  Whereas, when I think about my code, I'm not really finding very many examples where I could leverage polymorphism to any useful effect.

This could simply be because the problems I'm solving for my application are simply better suited for data structures and functions.  Or it could be because I'm just not approaching it from a clever enough OO angle.

Recently, my co-workers and I had pretty well settled on a design approach for our application, and it has been working extremely well for us.  However, this article and it's clear preference for objects and polymorphism has me wondering if there may be another perspective that could be useful.  I'll talk more about this in the next post.

Demonstrating the Costs of DI

Here's a followup to my previous post "Interfaces, DI, and IoC are the Devil"

This time I want to demonstrate a little of what I'm talking about with some code samples.  I have contrived an example.  I wanted something that is real, so I took the code structure and the responsibilities from some real code in one of our applications.  But I changed the names to make it more easily understood.  I also removed some of the concerns, such as transaction handling and record locking, just to make it shorter.  I'm trying to be fair, so the example is not trivial, but it is also not complicated.

Note that I didn't compile any of this code, so please excuse typos or obvious syntactic errors I might have over looked.

Pretend we are writing an application to manage a conference.  It's an MVC 3 C# web app.  We have presentations, and presentations have speakers.  The speakers are ordered (1,2,3, etc).  As an optimization, we will cache some of the information about the speakers for a given presentation: how many are there and who is #1 ordered speaker.  This information will be cached on the PresentationSpeakers table.

The structure of the code is as follows.  The SpeakerController's Create action method is called to add a new speaker to a presentation.  This controller delegates the job to the PresentationApi class.  This class deals with coordinating the various domain services, and in real life would have dealt with database transactions and record locking/concurrency.  PresentationApi delegates to OrdersSpeakers (which obviously assigns the order numbers to the speakers) and PresentationSpeakers (which caches information about the speakers on a presentation).  Finally the Speaker and Presentation classes are active record objects.

This first example demonstrates the simplest way in which I would like to write this code.  There are no interfaces, there is no constructor injection, and it is using the Active Record pattern for database persistence.

The next example addresses the "problem" of the controller having a direct dependency on the PresentationApi class by adding IPresentationApi and injecting it through the controller's constructor. Notice that I also convert PresentationApi to be a singleton at this point and remove it's instance variables. This isn't strictly required, but it is typical. Notice how I now have to pass the presentationId into the LoadPresentationSpeakersRecord helper method.

In the third example, I remove PresentationApi's direct dependency on OrdersSpeakers.

Finally I eliminate ActiveRecord and replace it with the most evil pattern in the world, the Repository pattern. I chose to implement this in the way I've most commonly seen, with one interface per database table (give or take).

So, if you had your choice, which of these versions would you prefer to have in your code base? Which would you prefer to have to debug, understand, and maintain?

My answer, not surprisingly, is #1. Notice how each version adds more and more cruft into the code and obscurs what the code is actually trying to accomplish.  This is the cost of indirection.  It's really the same reason people like to fly the YAGNI flag.  And perhaps you've heard the old adage, "Do the simplest thing that could possibly work".  I desperately want as few constructs between me and what my code does as possible.  Which is why I yearn for the simple code in example #1.

PS.  If you actually read those code samples and studied the differences between them, pat yourself on the back!  I know this is very difficult to follow.  I actually wasn't intending to blog it, I just wanted to go through the motions myself and see how it turned out.  There have been quite a few times where I wrote something off because of my image of how it would turn out.  But then when I actually did it, it turned out much differently (MVC, MVP, and MVVM were all like that for me).  But in this case, it turned out just how I'd imagined it...  Crufty.