Who Deserves The Credit For Software Craftsmanship and Great Design?

Common SenseHow did people design great software before OO and how did we ever manage to run a successful project before Agile came along? Those are the questions young programmers undoubtedly ask themselves at some point in their career (I know I did, early on). You hear about and read horror story after horror story from the dark ages of software development (sometimes the dark ages are years ago, other times they include the project before this one) and then you hear about how things like OO or Agile are transforming (or have transformed) the industry. The truth is, projects didn't fail due to lack of Agile or OO (or any other 'revolutionary' practice) and there were plenty of successful projects even in the bad old days (it's just that you don't tend to hear about the good ones). Don't get me wrong, I believe in what Agile stands for and I haven't even seen non-OO software in my career :), but I don't subscribe to the buzzwords, I subscribe to what these concepts stand for, the core ideals if you like. Sometimes, you need to take a step back and consider that while the moniker may be shiny and new, the concepts it tries to package are often time honoured and fundamental (or as time honoured and fundamental as you can get considering the relative youth of our industry).

I was thinking about the Unix Philosophy the other day and it struck me that pretty much all of the ideas seem really familiar. So, I went and cracked open a copy of "The Art Of Unix Programming", just to make sure I wasn't getting myself mixed up. And what do you know, it was just as I remembered, but this time, I had a bit more experience to put it all in perspective for me. Here are a couple of quotes by Doug McIlroy:

"Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features."

Why, he is simply espousing making your software modular as well as giving a nod to separation of concerns. I dunno about you, but I keep something similar in mind when designing my classes, small, self-contained, I thought it was all OO principles :).

"Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them"

Release early, release often. Refactor and leave the code better than you found it.

"Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them"

Automate everything. Does it smell like agile in here or is it just me.

Let's keep going we don't even need to turn the page. Here is what Rob Pike had to say

"Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is.
 Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest."

Hmmm, avoid premature optimisation.

"Rule 4. Fancy algorithms are buggier than simple ones, and they're much harder to implement. Use simple algorithms as well as simple data structures."

YAGNI. Even more agile wisdom. You have to remember that this stuff is from the 70s, years before OO got any traction and decades before the agile movement was even on the horizon. And these quotes are a distillation of thought, after these people had time to mull it over; they were actually doing this stuff years before.

Eric S. Raymond, the author of "The Art Of Unix Programming", summarised the Unix philosophy in the following 17 tenets.

1. Rule of Modularity: Write simple parts connected by clean interfaces.

2. Rule of Clarity: Clarity is better than cleverness.

3. Rule of Composition: Design programs to be connected to other programs.

4. Rule of Separation: Separate policy from mechanism; separate interfaces from engines.

5. Rule of Simplicity: Design for simplicity; add complexity only where you must.

6. Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do.

7. Rule of Transparency: Design for visibility to make inspection and debugging easier.

8. Rule of Robustness: Robustness is the child of transparency and simplicity.

9. Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.

10. Rule of Least Surprise: In interface design, always do the least surprising thing.

11. Rule of Silence: When a program has nothing surprising to say, it should say nothing.

12. Rule of Repair: When you must fail, fail noisily and as soon as possible.

13. Rule of Economy: Programmer time is expensive; conserve it in preference to machine time.

14. Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.

15. Rule of Optimization: Prototype before polishing. Get it working before you optimize it.

16. Rule of Diversity: Distrust all claims for "one true way".

17. Rule of Extensibility: Design for the future, because it will be here sooner than you think.

Let’s pick out some choice tidbits. How about number 4, separate interface from implementation, I think we're all well drilled in that one these days :). I am really fond of number 8, this is the stuff that most agile developers live by, robust, simple and clear code (you can take robust to mean well tested if that is your preference). Number 15 is a good one, anyone do spikes when they code – I know I do? Number 16 is my favourite, I interpret it as "if something doesn't work, stop doing it and do something else", reminds me of Scrum (or as Conan the barbarian would say it "SCrom" :)). Much of it is about keeping it simple, i.e. the KISS principle, just like many of the concepts behind agile. But of course, just because we're keeping it simple, doesn't mean we are foolish about it (it's keep it simple stupid not keep it stupid stupid), which is where number 17 comes in.

So where am I going with all of this, did the dudes who wrote Unix invent agile (or at least the underlying ideals), or software craftsmanship, or great software design? It's unlikely, after all Don Knuth said:

"Premature optimization is the root of all evil."

He said this in 1974 (the link is pdf) – but was likely thinking it for years and there were probably people before him as well. It all had to start somewhere, of course, as I said; software is a relatively young industry. But, it doesn't really matter who the first person was that came up with these ideas, the fact that the ideas continue to figure prominently in newer concepts tells me that they are just basic common sense. When programming was born as a career, smart people came along, did the first project or two and quickly twigged on to what seemed to make sense and what didn't. All the nonsense, confusion and failure was mostly from a bunch of other people trying to fit software development into moulds it wasn't suited for – refusing to realise that this was an industry fundamentally unlike any that has come before and must have its own "rules" (rather than trying to inherit some from manufacturing or random other industries that "looked" similar). The saddest thing was that these people had power and could impose their will and make their ideas stick, which is why the whole industry went to hell for quite a while (even to this day), with signs of recovery only now beginning to emerge.

So Does Agile Get Credit For Anything?

Christianity and Islam are some of the most popular religions in the world. This should surprise you, since they are much younger than many other religions (especially Islam). Do you know why these religions are so popular? Because they were the first missionary religions, the first to actively seek out new converts for their faith. Most of the religions that came before, either did not actively welcome outsiders or were actively hostile to them. These religions are a testament to the fact that if you actively try to seek out converts and don't give up, even due to major setbacks, you will slowly gain more and more followers and will eventually spread your message far and wide.

The agile movement was the beginning of the "missionary work" for these common sense software development concepts that we have now all come to know and love. Up until then, there was no focus, no organisation, and even when there was organisation, there was no attempt to seek out converts (other developers) and actively spread the message about how software should be built. The agile movement changed all that, which just goes to show that developers are just people and are willing to buy into something that seems sensible and hits close to home. But, the message itself was never revolutionary, just like the religious messages were never revolutionary. It was simply a bunch of common sense concepts that many good developers have followed for years.

What I am trying to say (besides some curious factoids and a history lesson :)), is this. Don't faddishly follow "best practices", remember rule 16 – there is no "one true way". It doesn't matter what tech you're using, it doesn't matter what the configuration of your task board is. The key is to surround yourself with good people (this really is key and deserves a separate post) and to always remain aware of the situation – then figure out what makes sense and just do that (even if what makes sense is to follow best practice or to do the exact opposite :)). Good programmers have always known the score.

Image by RobertBasil

  • Al

    Very nice synthesis of ideas. Like many people, I read some of the Unix philosophy and guidelines when I started out, but I haven’t gone back to review them with years of personal experience to place them in a context. You’ve convinced me I should.

    I definitely get what you’re saying about the evangelical strength of Agile, but I think the Agile community also deserves some credit for taking these common sense ideas and applying them to the management of a project instead of the code. From my understanding of history, many good developers had these pre-Agile ideas, but their target audience was developers and technologists. With Agile, I think the original target audience was project managers and business people.

    For example, convincing a developer to work iteratively is quite easy. Convincing a manager to give up the foolish but safe-feeling of fixed-scope, fixed-time, fixed-price projects and instead work in 2-week iterations is quite difficult.

    • http://www.skorks.com Alan Skorkin

      You’re right, agile has been able to evangelise to include a wider audience than just developers. The thing to note is that the original evangelists were basically all developers. So it was developers evangelising to management, the fact that management has been listening just goes to show that if enough developers talk about something loud enough, management can’t afford not to listen.

  • http://leonardoluiz.blogspot.com Leonardo Luiz

    I like a lot this blog. It’s inspiring me to write down my own thoughts.

    This blog is not only about software development, it talks about to be developing software in a passionated way.

    Your analogy with religions was perfect. I think that Agile, like a movement, is something new. But the ideas behind are presents for long time.

    Raymond also wrote the classic The Cathedral and the Bazaar, that also talks about an smart way of doing software.

    • http://www.skorks.com Alan Skorkin

      Hi Leonardo,

      I am glad my writing has inspired you to share your own views. I think more developers should share their thoughts more regularly. There is so much knowledge out there sitting in peoples heads, we need some way to get at it :). I haven’t read “The Cathedral and the Bazaar”, but it’s been on my list along with dozens of other books, but I am still hoping to get to it eventually :).

  • Pingback: The inner buzz of a good find. « Sense the balance shift

  • David

    ARGH

    So many damn :)s.

    Kill them. Kill them all.

    • http://www.skorks.com Alan Skorkin

      You think that’s lots, you should see some of my e-mails. Seriously though, I’ve had people complain before, but smileys are on par with jar jar binks for me, most people hate him, but I love him. I guess it’s just part of my style, I do it automatically.

  • Harry

    When I first started reading on Agile, XP, etc I never really felt I was reading anything paradigm shifting, or even new or exciting! I’m both amused and angered whenever I see old wine being packaged in new bottles with the *sole* purpose of (intellectual) deception.

    One way to control this systemic problem (of the tendency to re-invention and re-branding of concepts) in our software industry would be to have some kind of industry *and* academic peer-reviews. Meaning, if a book or an article is not seriously ranked high, one need not bother about it. While this will not completely prevent cr@p from being seen as innovation, it will at least contain it to an extent… for, if I were to decide to promote some classic old method, practice, or concept via a shiny, new marketing term that will look good on my resume, I’d at least think twice which I need not today mainly due to the way things stand today.

    • http://www.skorks.com Alan Skorkin

      I do find that it is sometimes necessary to repackage the idea under a different name when it cops a lot of flack in it’s old form due to repeated incorrect implementation, but on the who you’re right, rebranding some ideas is a good way to make a quick buck.

      • Harry

        IMO, repackaging an original set of ideas under a different name is completely unnecessary, completely uncalled for… especially when these ideas were not just sound but *solid*! It gives the industry a false sense of (forward) progress, which the less insightful ones realize the long and hard and painful way. If the combined industry-hours and the brain-power that’s been spent to date on the Agile/XP cr@p — both by the duper and the duped — were to have been honestly spent on solving some *real* software engineering / computer science problems, who knows how much better our lives would have been today.

        Also, even when an old set of otherwise sound and solid ideas is “repeatedly” being “incorrectly implement,” I’d think that men of intellectual integrity and honesty would not just point it so, but point is so very explicitly… so that the original ideas and their innovators get the credit they are due and the learners see from Day 1 the full context in which the current idea is being introduced. I doubt if that was every done by Agile ‘innovators’ (read: promoters).

        There’s politics and deception and lies and what not even in software! May be, someone should dedicate a whole blog on this topic alone. If not a whole blog, may be at least a series of posts. I sometimes ‘feel’ that there was a lot less politics in the 80s and the 70s. People in the 90s and later, I guess, were under a lot of pressure to innovate. Since a genuinely original ideas and real innovations don’t come easy, people resort to all sorts hookery and crookery in broad daylight.

  • http://www.codemanship.com Jason Gorman

    I was not aware that OO design principles and refactoring were “Agile”, per se. They certainly predate it. I think it’s important to distinguish the two. You need clean code to be responsive to changing requirements. But clean code also tends to be more reliable and robust (and secure). Even if the requirements spec was frozen before development, I would still practice TDD and refactoring. And if we’re looking for Agile influences, I suspect precursors like Tom Gilb’s book on Evolutionary Project Management and the SmallTalk community loomed large.

    • http://www.skorks.com Alan Skorkin

      Hi Jason,

      You’re certainly right in that OO is not under the Agile umbrella per se, although I’d say refactoring is. What I was trying to say was that the principles that both OO and Agile espouse are actually predate both of those concepts, I focused on Agile more than I did on OO, but my point remains valid.

  • Pingback: Dew Drop – May 13, 2010 | Alvin Ashcraft's Morning Dew

  • http://www.thensys.com Jeremy Lowery

    This is a post I’ve been wanting to write for years. I found quite a lot of wisdom in The Art of Unix Programming that I try to apply in the systems I work in today. OOP allows you to apply these principles in a more fine grained manner. For example, if you start thinking about the isolation between UNIX processes in the same way as interfaces between objects, you start to look at OOP in a very different way (more “agile-like” if you prefer).

    I’ve been wanting to figure out a way to express all of the wonderful design advice in TAOUP
    in a way that wasn’t UNIX-centric. Design patterns come to mind, but might not be the best way.

    • http://www.skorks.com Alan Skorkin

      Hi Jeremy,

      I find, the best way to express something, is just to start putting your thoughts down, no need to get it perfect, just get it out there, you can always refine later.