Software As A Destination vs Software As A Journey

I am feeling a little existential and metaphorical today, so forgive me if this post has more than just a slight tinge of existential metaphor about it. I do hope it still makes some sense. 

Journey

There are two fundamental ways of looking at software development. One is all about the final product and the direct benefits you can get from it. The other is all about the lessons you learn from the process of building the software. I call them software as a destination and software as a journey. Historically speaking and even into the present day, the majority of companies that build any kind of software are 'software as a destination' companies. They want to extract some direct financial benefit from the software they build (which is fair enough), until this benefit is realised, the process of building the software is just a resource (time, money) drain. On the other hand, most individuals who build software for those companies would instinctively like to be (and in many cases ARE) 'software as a journey' people. We want to enjoy our work, we want to learn as much as we can, we want to interact with those around us, we want to build code that we can be proud of and we want to get paid. All of this will happen throughout the project rather than just at the end, and so it is all about the journey, the final success of the product is almost icing on the cake. It seems that there is a fundamental disconnect between what a company wants to get out of building software and what the developers will try to extract from the experience.

This is why the software development industry is in a sort-of screwed-up state, or at least it has been for many years (things are getting better). Companies see the destination and want to arrive there as quickly as they can, the sooner they get there the sooner they see the benefits from all that money they spend and the sooner they can start getting their money back, either directly or indirectly. Anything that stands in the way of this goal must be swept aside and this includes the developers who want to enjoy the journey and 'smell the roses' on their way to the final product. Of course, the companies hold all the cards (they employ you, pay you – which indirectly, feeds and clothes' your children) and so can usually get their way if they really want to. Developers can try to make noise about the value of paying attention to the journey, but if the company is dead set on doing things their way, that's the way things will be done, developers can either fall into line or there will be other developers to replace them. However, just like a real, physical journey, if you have your eyes set firmly on the destination and are not looking where you're putting your feet, there is a good chance you will trip over some roots, or a snake will bite you, or you won't notice that cliff which is right between you and where you're going. This is how we get failed projects.

Once the project and the company are at the bottom of the cliff, you can kinda tell that everyone is in trouble. But it is difficult to figure out exactly how you ended up where you are, you had your eyes firmly on the destination and forgot to pay attention to what was happening around you, so it is hard to pinpoint the mistakes that landed you at the bottom of a cliff. Companies will do a post-mortem, come up with some vague reasons for why things went to hell, everyone feels better that they have an explanation (leaving everyone none the wiser in reality). And so next time they go out and do the same thing all over again. All the while, the answer is staring everyone right in the face. If only they had listened to the developers who told them to pay attention to the journey and not worry so much about the destination things would have been different. Even novice programmers soon learn, from personal experience, that when we slow down and pay attention to what we are doing right now, we get a lot more out of it, we learn more and the final product is improved as a result. There are only benefits, no downsides, alright, you lose some time in the short term, but gain it all back with interest in the long term. But, at the macro level where committees make decisions, this same lesson takes years and decades to permeate the hive-mind. Not to mention the fact that seeing only the destination is much easier than constantly paying attention, and so even the occasional developer gets seduced by the 'software as a destination' mentality (it's really more than just the occasional), which sets everyone back by introducing conflicting opinion and muddying the waters unnecessarily.

There was a bunch of people who were all 'software as a journey' people who basically got sick of all the back and forth. They recognised the fact that the journey is what really matters not just at the personal level, where each individual developer is concerned, but at the team, project and company level as well. They decided to give a united voice to the message that as long as you take care and extract the maximum benefit along the way, the destination will take care of itself – you will arrive somewhere you want to be. It may not be exactly the same place where you thought you would end up, but it will be a nice place nevertheless and best of all, you will have a very good idea of precisely how you got where you are, despite not having a narrow focus on the final objective from the start. More like-minded people joined the initial bunch and the movement got more organised. Some of these people started companies of their own; others brought their message to larger organisation. And now that the 'software as a journey' message had a united voice (albeit still a comparatively small one), the success of looking at software development this way could be demonstrated, but more importantly, disseminated to the industry at large.

It is a major undertaking for a company to develop any significant piece of software. Not just from a financial perspective, but as far as airing out the dirty baggage and discovering the process inefficiencies that every significant company will have. After all, in order to put something logically into code you need to figure out exactly what that something is and how it works. This is a golden opportunity for the company to streamline and improve how their business works, figure out if some processes are unnecessary. You're building software, but you're paying attention to what you're doing not just at the technical level but at the organisational level as well. As a result, not only do you get more robust software that better fits your needs, but you also improve the day to day operations of the company; all because you were paying attention to the journey rather than focusing on the destination.

Destination

But, it is so much easier to simply look towards the goals and on paper it seems like it SHOULD be more efficient. If you're in management you want to seem more efficient and focused rather than being wishy-washy and touchy-feely, preaching self-awareness and a less-rapid pace just isn't strategic enough. And so companies create artificial deadline pressures (like end of financial cycle of some sort), to make everything neater and make the people in charge look good and it feels like for every step the agile movement takes towards running software projects better, some company somewhere, takes two steps back.

Robert A Heinlein once said:

"The supreme irony of life is that hardly anyone gets out of it alive."

Infact, he had many other quotes which I think are quite relevant to the software profession, maybe I'll do a post about that at some point. To me this quote says that it's the moment that really matters, enjoy the life you have now rather than constantly looking forward to something better. I am not sure if there is any irony to be found in software, but if you direct all your focus towards your goal without paying due attention to the nitty gritty of what you're doing every day, you're likely to not get any useable software out of it. As long as you have a reasonable idea of where you want to end up, you just need to get the details right and the bigger picture will tend to sort itself out. On the other hand, you can have the clearest possible goal in mind, but if you let the details slide, bad things will almost certainly happen.

Images by Norma Desmond and alicepopkorn

  • Amber Shah

    I know a senior developer who went to work at a company as their only programmer and built them a custom solution to a problem they were currently outsourcing or doing manually. The cost saved was in the high tens of thousands of dollars per month, so this was a huge win all around. While effective, the custom software solution was definitely bare-bones and needed a lot of fleshing out and future development (this was a large-scale project build by 1 super-coder). It was a long-term investment that was most definitely worth it for them.

    The business manager went up to the developer and asked – So what would happen if you leave the company? That’s a legitimate concern and of course the right answer is: you need to hire more programmers. Instead he was expecting to get the software to be at a point where he -could- leave and the software would continue to run indefinitely, supported only by non-programmers.

    The reality is that little-to-no software stands up to that kind of test. A piece of in-house software doesn’t have all the automation and trimmings that a product-for-sale would have. And even then, new versions and upgrades are shipped as a matter of course for products as well. Even if the product itself continued to function, bug-free, the technology it runs on and interfaces with would be changing. We’re in a world run by software and it’s an on-going thing. The idea of set-it-and-forget-it just doesn’t work.

    • http://www.skorks.com Alan Skorkin

      Hi Amber,

      I hadn’t even considered that aspect of my post. But you’re of course right, most companies would like nothing better than a piece of software that will do everything they need from it in perpetuity without having to spend any additional effort getting to support evolving business needs. We know that this is ridiculous, no software can meet this goal, but many less technical people don’t.

      And it doesn’t help that many large software vendors try to present their pre-canned software as just such a package. All you need is some configuration and it will always do everything you want it to do. I am gonna stop now before I start writing a post-sized comment :).

  • http://nikhilkardale.wordpress.com Nikhil Kardale

    Good post, especially – paragraph 3, the lines in bold – “just like a real, physical journey, if you have your eyes set firmly on the destination and are not looking where you’re putting your feet…..” – take a bow! Great analogy!

    It comes down to ‘thinking’ I guess, at every level (developer, manager, tester), sadly a lot of people do not think. There’s as much ‘heart’ involved in software development as there is the ‘head’, but people don’t realize it. Too much adherence to rules and age-old corporate strategies results in this sort destination-centric approach. Not that this adherence is bad but you sort of take things for granted and loose interest in changing and modifying the setup for good.

    Again, ‘adaptability’ is the keyword here and not so much ‘planning’, because you can always plan to reach your destination but have to adapt throughout the journey to actually reach there.

    • http://www.skorks.com Alan Skorkin

      Hi Nikhil,

      It is not by accident that one of the key agile tenets is that of embracing change :). Change will happen right now and you need to be able to deal with it, infact you need to plan for it to make sure it happens as painlessly as you can manage. As you said adaptability definitely helps. For many project, change is that thing that everyone hopes will pass them by, and everything will go according to plan. How likely is that.

  • Ant

    Holy wall-of-text Batman!

    I think part of the problem comes from developers not having the courage to push back on the business. Gutless in asking repetitively for extra time, doing the things that need to be done, not just have to be done.

    • http://www.skorks.com Alan Skorkin

      That is part of the issue, the other side, is the business not willing to listen to developers when they raise concerns, so developers learn to never raise concerns again and just make-do. Bad for the business and bad for developers.

  • Pingback: The Quality Imperative | Thought Clusters

  • http://www.michevan.id.au/ Evan

    It seems to me that you’ve got to keep an eye towards both the journey and the destination, otherwise you might have a really nice trip but not actually get anywhere.

    F. Scott Fitzgerald said, “The test of a first-rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function.” To some degree that’s what you have to do on a software project.

    I good team will have a mix of journey-oriented and destination-oriented contributors and an environment that cultivates vigorous discussion. A great team will have one or two contributors that can keep both in mind as the project progresses.

    E.

    • http://www.skorks.com Alan Skorkin

      Hey Evan,

      That is a very sensible approach, you definitely need to have some idea of where you want to end up. But historically speaking, journey orientation has been rare, whereas everyone has been on the destination band wagon. The issue is that destination orientation is relatively easy, at least in the superficial sense. Journey orientation on the other hand can be extremely difficult, dealing with all the ongoing issues and being able to adjust the course you take to the destination or even altering the destination itself as you go.

  • http://www.activelylazy.co.uk/ David Green

    I guess the problem is that the advantage for “software as a destination” people of taking care and paying attention every step of the way are all intangible things: if you do everything with love you’ll get solid, robust, testable, maintainable, reliable, supportable code.

    From a “software as a destination” mindset, if the product ships, does it matter if you’ve had problems testing it? If its hard to maintain? Hard to support? As long as its out there earning money: job done, right?

    • http://www.skorks.com Alan Skorkin

      Hey David,

      See, the “as long as it ships” attitude seems to be seen as the pragmatic approach, but in reality it can be quite the opposite. What if it doesn’t ship? Do you know why it didn’t ship, where did you go wrong, at what point could it have been salvaged and a myriad of other questions can arise. And if you have been destination oriented all throughout, you will only be able to make guesses in response to all those questions. And even if it does ship, can you do it again on the next project just as successfully? You can try doing everything you did exactly the same and the project might end up as a failure and you will once again be unable to answer why.

      • http://www.activelylazy.co.uk/ David Green

        Hi Alan,

        Very fair point – that only applies *if* software ships. But, I’ve seen enough projects that slip and slip and slip because of a “software as a destination” mentality. Technical debt is accepted in the name of pragmatism, gradually slowing down the project. Although the project probably *should* have failed, its slowly manhandled into a shape where at least it meets the basic business goals – with half hearted promises to come back and fix the mountain of technical debt in some mythical “phase 2″.

        You’re dead right, though – learning from these mistakes seems to be the hardest thing. I wonder if there’s a problem here as well of how we view success and failure. Its a natural human trait to praise ourselves for our successes and blame others for our failures. So when a project ships anything like on time, we’ll praise our amazing process and our own coding prowess. When a project gets canned, we’ll blame “the business”, “that other team” or “management” – rather than acknowledging the truth that maybe if we’d done a better job, either we would have known earlier that the project was doomed or the project might have actually shipped.

  • http://www.mostlymaths.net Ruben Berenguel

    There is a history that quite fits in here, taken from The Alchemist by Paulo Coelho (according to Wikipedia, Coelho is very happy about quoting him or even sharing his work by peer-to-peer, it seems). I’ll try to shorten it a little (because it is something like 8 paragraphs long). If you enjoy it, consider looking for the book, I recommend it highly.

    ******
    A boy looking for the secret of happiness meets the wise man he has been looking for an answer. The boy, after waiting for hours asks… And the answer is that he is too busy, and instructs him to go wander his palace and look around. But he asks him to do something: *carry a spoon with two drops of oil, without allowing the oil to spill.*

    The boy goes and wanders around for two hours, intently watching the oil in the spoon. When he comes back, the wise man asks if he has seen the Persian tapestries, the tame lion or the books in the library. The boy, embarrassed answers that he has not seen anything, he was too concentrated in the oil in the spoon. The master tells him to go and take a look at all these marvels.

    The boy picks the spoon and goes around again, enjoying the magnificent garden, peering through books, feeding the lion. When he is back, the wise man asks ‘But where is my oil?’ as he sees the spoon is empty. Then, the master says “The secret of happiness is to see all the marvels of the world and never to forget the drops of oil on the spoon”

    ******
    As programmers (mind you, but I guess being a mathematician shares the journey-destination metaphor… finishing your thesis feels like that) we should not only be aware of the beautiful journey we are in, but also keep in mind that we need to keep those two oil drops in our spoon.

    PS: Sorry for the long comment, I could not condense the history better than that.

    Ruben

    • http://www.skorks.com Alan Skorkin

      Hi Ruben,

      I really like that metaphor, I’ve been meaning to read The Alchemist for ages, but never seem to get around to it, too many books to read :(.

  • http://www.fredberinger.com Fred Beringer

    This is an excellent post ! Thanks for taking the time to write it. A successful Software Company will always help their people through their journey, whatever the journey might be. Focusing only on the bottom-line is not a sustainable model. If your company has a long term strategy for its products, people should be really high on the priority list.

    @Ruben: Thanks for sharing this metaphor !

    • http://www.mostlymaths.net Ruben Berenguel

      You are welcome! But the big thanks should go to Alan and Mr. Coelho, I just was the glue :)

  • Pingback: Dew Drop – May 11, 2010 | Alvin Ashcraft's Morning Dew

  • Pingback: Webs Developer » Alan Skorkin’s Blog: Software As A Destination vs Software As A Journey

  • Pingback: Webs Developer » Who Deserves The Credit For Software Craftsmanship and Great Design?

  • Russian

    “You’re building software, but you’re paying attention to what you’re doing not just at the technical level but at the organisational level as well. As a result, not only do you get more robust software that better fits your needs, but you also improve the day to day operations of the company”
    I like this fragment.

    I think that the process of software project development must be kept absolutely clear and predictable.

  • http://blog.dhananjaynene.com Dhananjay Nene

    Great read. As a developer and a manager, I think the trick is in to be able to appreciate the journey even as one doesn’t take the eyes of the ball in terms of ensuring a competitive software development organisation in terms of being able to meet the business and stakeholder expectations. A lot of issues arise when the business stakeholders cannot appreciate the journey, and I think it is just as bad when the journey dominates and the destination is forgotten.

    Having said that, I had once expressed my anguish on the same topic with similar sentiments but very different content here : http://blog.dhananjaynene.com/2010/01/software-development-is-about-the-middle-and-not-just-the-end/

  • Pingback: » links for 2010-05-26 (Dhananjay Nene)

  • Pingback: Means to an End « IT Primer