How The Quest For Efficiency Turns Small Problems Into Big Ones

PrioritizeWhen Sun released Java 1.5 (or Java 5, as they rebranded it) many people were eager to upgrade their projects to use this latest and greatest version. And why not, it offered some compelling features (arguably), the most visible of which was generics support. The project I was working on was no exception. Of course, in the enterprise world this upgrade path took a while (as enterprise things are wont to do :)), but eventually all obstacles were removed and the upgrade happened. But, along with all the "goodness", came the several thousand warnings telling you exactly how generics non-compliant your code was. There is nothing inherently wrong with this; it is good to know when there are issues with your code, even if these issues are relatively minor. The problem was that these warnings became a permanent fixture. You see, we do software development the efficient way these days, we prioritize everything and fixing trivial warnings, that don't actually affect the functionality, never had a chance of being prioritized anywhere except the bottom.

Initially I had no problem with this, I am as efficient as the next guy, and I believe in prioritization, but after a while something just started to bug me about the code (no pun intended :)). It just didn't look clean somehow and no amount of refactoring seemed to help. Plus, we just didn't seem to be as thorough – or maybe it was all my imagination. An insight into this issue didn't hit me until long after I was no longer on that project. You see it was all about those generics warnings! By upgrading and introducing several thousand new warnings into the code we turned our code into the ultimate house with broken windows (or at least cracked ones). And due to our well intentioned prioritization efforts, we never really had any intention of fixing this issue. You can probably guess how the story goes from here. Soon, a couple of legitimate warnings crept into the list, and before we knew it the couple turned into ten which quickly turned into several dozen. And you do try to fix the legitimate ones, but the psychological effect of staring into several thousand warnings will not be denied. Eventually you develop warning blindness (similar to ad blindness) and it doesn't matter if the warnings are legitimate or frivolous, you no longer bother, you just treat them as noise.

Prioritization And Timeboxing

Prioritize

Prioritization seems like one of those practices that brings nothing but good. It's pretty exciting because it is so easy to do and it has immediate benefits (kinda like standup meetings). We just move our features/tasks around, chop the least important ones off the bottom and all of a sudden we're making our deadlines, we're delivery focused. But, no practice is applicable universally. I am not trying to say that prioritization is bad; it is often a valuable and helpful practice as long as we don't apply it blindly. There will be situations where it will make sense to prioritize a less important task/feature higher, for reasons other than its relative business value. Like the thousand warnings problem. Even a minor daily/weekly effort to fix the issue would have kept it at the forefront and prevented a tiny problem from escalating in unexpected ways.

Maybe letting a few warnings slide is not such a big deal but the underlying issue of universally applying a seemingly benign practice can manifest itself in a more insidious fashion. Let's consider timeboxing, another one of those practices that brings nothing but good. If we timebox, we're forced to come to some sort of decision and we don't waste time in endless meetings and committees. Team retrospectives are a good example. During a standard retrospective every member of the team lists the issues that they consider important, the team then votes on all the issues to make sure the more important ones are discussed first (another example of prioritization). Why do we need to vote? Because, the retrospective is timeboxed, we only have X minutes and we'll only be able to get through so many things. The problem here, is that often the same issues crop up every time, but are never voted up to the top and so are never discussed. Does that mean the problem is less pressing – possibly? Does it mean the problem doesn't exist – not by a long shot?

If nothing gets done about an issue for long enough, people stop bringing it up. Why bother, it never gets discussed anyway, it's more efficient to just drop it. Now you have a problem in stealth mode that can grow and develop in all sorts of unexpected ways. Such as becoming an ingrained part of the culture (I've looked at aspects of culture before, it is an extremely diverse topic and I am planning to touch on it in several upcoming posts). People get used to it, and it takes on an element of tradition. Have you ever tried to change a tradition? It doesn't matter if it's helpful or detrimental, once the team has fully internalized it – good luck trying to budge it. And all this as a result of timeboxing and prioritization applied blindly – practices that are benign and helpful most of the time – causing a lot of harm.

You always need to adapt and adjust according to the situation; this is why we have people running projects and not machines. If a discussion seems to be leading somewhere good, don't worry about the timeboxing let it run, only cut it short if it stagnates. Retrospectives are the same, I do believe it is well worth spending as much time as necessary to go through all the problems the team is facing. It is a small price to pay for not allowing issues to fester. Keep an eye on what is bugging your team about the code. Prioritizing an unimportant feature higher can often deal with these issues cleanly and efficiently. It is not all about the business value, or rather, there is more to business value than just financial impact. The general health of the development team has intrinsic business value all of its own.

Images by Michael Valiant and hockadilly

  • http://lessonsoffailure.com Dave Rodenbaugh

    Timely post! I just posted this today, discussing The Law of False Alerts, which Java 5′s compiler suffered from as well… :) Good stuff, Alan.

    http://www.lessonsoffailure.com/software/stop-breaking-laws-software/

    • http://www.skorks.com Alan Skorkin

      Hi Dave,

      That’s an excellent post, I knew there had to be a name for what I was describing, I love it when I learn new things :). Not to mention the fact that reading you posts has given me 2 new ideas for posts to do in the future, cheers for that.

      • http://lessonsoffailure.com Dave Rodenbaugh

        Likewise.

  • Seif Sallam

    Wouldn’t this be great if its a web application.

  • Amber Shah

    The crux of the issue with warnings was that prioritization, by definition, meant NOT fixing the warnings (since almost everything else qualifies as more important). I can certainly understand this and I struggled with it at one time, until I realized that code quality is NOT something that shows up on any priority list…. nor should it.

    Code quality is just a standard that is set by the team. It is difficult to define, but it exists nonetheless. Is it ok to check something in that is not tested? Is it ok to ignore good OO design in the effort to get a feature out the door? Is it ok to check something in that is half finished. Is it ok to check something in that has compile warnings?

    One plausible solution would be to say that upgrading all of the data structures to use generics is part of the Java 5 conversion. That’s not what we did though. What we did was that each developer spent a little bit of time pre-check-in to do “clean up” activities. Whether it’s refactoring something that’s been bugging us, cleaning up warnings, fixing TODOs, clearing out checkstyle, etc. As the technical lead, I took the lead on this and spent a great deal of my own time doing it (as opposed to “assigning it” to someone else) and the other developers were happy to follow suit (developers just like good clean code).

    It would happen occassionally, where us developers would want something but the business would never prioritize it. Most times, when I really examined it, it was related to code quality. Then, it’s understandable that they wouldn’t prioritize it – that’s OUR job. When the business people say “Fix X bug” or “Implement Y feature” they have a certain implied “in a decent way”. Just like when I go to the mechanic and ask him to fix my car, I have an implied “don’t use duct tape”.

    So the point is that prioritization of business functionality does not preclude code quality activities, but they are not the same thing.

    • http://www.skorks.com Alan Skorkin

      Hi Amber,

      You make a very good point, I completely agree, a team should maintain an implied level of code quality regardless. Unfortunately this is often very hard to reconcile with being a business focused team. How much time do you ‘allow yourself’ to spend not working on features requested by the business. There is often a fine line between spending time doing worthy clean-up activities and being too pedantic about your code.

      The worst thing is that often with refactoring you have to prioritize as well, due to not being able to spend unlimited time on it and in this situation stuff like those warnings falls to the bottom yet again :(.

      I think there should be an explicit agreement between the business and the development team regarding what the team needs to do to maintain code quality and how they will do this.

      • http://rubyengineer.com Keith Hanson

        I’m a development manager (though I still code as much as I can, haha) of three other ruby programmers, working on core software that drives different sites for names like Mercedes Benz, Honda, and Southwest Airlines to name a few ;)

        Over the last few years, the core code has changed from PHP to Ruby, from Rails to Merb to Harbor. Changed from ActiveRecord to DataMapper. Needless to say, I feel the pain listed above about “warnings” and broken windows.

        The team has, in general, an incredibly pessimistic view about the code, and thus, about the company. I feel it just as much as they do. So here’s what I’ve done to handle it (this is likely a great post in and of itself, heh… if only I blogged more? :P)

        I first created a private chat room that no other higher manager would be a part of. I then made it clear that if they wanted to (excuse the language) bitch and moan, they’re not going to get reprimanded for it. I didn’t come right out and say it, but I let that culture develop on it’s own. Soon, I started hearing a flood of complaints. Many of them are minor and are simply bumps in the road. But a few of them… a few of them create a miserable work environment and are completely valid.

        It’s at that point that you need to take a realistic look at the company and determine just how many value detractors there are because of these valid complaints, and make an executive decision about that time spent. I work in a small company, so my time with the team is wholly left up to me (and my neck is on the line for it), but I’ve started to allow them to go through and pick a broken window every few days and fix it.

        We found that we had no real solid record of custom client expectations. So months down the road, a bug would appear, and no one knew about the original request. As such, a wiki-enabled “live spec” was created. As we get custom requests or work on a ticket/feature/bug, we update the living spec with the details of the functionality.

        Another dev found it to be particularly troublesome that our core software had so many configuration values for a single client. So he worked out, in a day, how to consolidate all of that.

        It’s small steps like these that change the view of your developers dramatically. Progress is being made, things are getting better! And for that 1 dev’s 1 day of time, do you really think a deadline would slip from it? And if it did, is it not worth it to go ahead and take the time out to make them more productive, happy coders? I’ll fight tooth and nail that it is :)

        • http://rubyengineer.com Keith Hanson

          Oh, and I’d like to add, my “prioritization” style is:
          Bring 2-3 quick, low-priority tasks to the top of the queue. From there, prioritize the mission critical stuff. In this way, you never end up having a task sitting around at the bottom of the queue for months on end while higher priority issues rise to the top over and over.

          Amber: I really love your ideas of “clean up before committing”. I may just try and implement that over in my neck of the woods :)

          Alan: The only way to win the maintenance argument is to start logging the amount of hours caused by frustrations in the codebase. A concrete example: Our codebase uses core templates, which can be overridden in a client site. Our designers often need to override them for HTML changes. Problem is, they never get pulled back in. So when we make an update to the core template, we have to check 5-6 other sites for overridden ones. Tallying that time up gave a staggering number of hours to present and argue for maintenance time :) I’m sure there’s something you could present in that vein of thinking as well :)

          • http://www.skorks.com Alan Skorkin

            You’re right logging hours helps, but I think it is often about changing attitudes as well. People always struggle to see how spending less time building new features (and instead fixing old ones) can help you with delivery. I really think that the concept of technical debt is highly under-emphasized.

        • http://www.skorks.com Alan Skorkin

          Hi Keith,

          I really like the ideas you present, it seems to me to be quite a pragmatic view. Whenever I read/listen to Agile thought leaders, there is always talk about these great teams where people felt sufficiently empowered to truly be critical in public and drive change. That is an ideal, but the reality is most people are a lot more likely to be honest if you remove the “career limiting” aspect of it :).

  • subramanian.g

    Hi Alan,

    While the flavor was slightly different in my case, the essence is the same. “Less visible” tasks like using the standard way to do a task( rather than the deprecated already existing way ) are shunned upon by almost every software developer in my part of the world. That according to me is the root cause for a codebase becoming a ponderous monster. I know of failed projects due to unwieldy codebases, where people just dont dare touching the cogs/gears ( read: the APIs that actually communicate with the system to get memory and so on ) and hence end up building inefficient stuff on top of something they only want to work around.

    PS: I really like your way of finding the right expressions to describe your thoughts on something without getting too verbose or boring.

    PPS: Just curious, ( i am a c++ developer on Visual Studio/Xcode never worked on java ), isnt there a pragma or project setting to suppress specific warnings in the IDE that you use?

    • http://www.skorks.com Alan Skorkin

      Hey,

      Thanks, there definitely is a way to suppress specific warnings in an IDE, the problem with doing that, is the fact that you might create a legitimate version of the warning you suppressed while coding. You don’t want to hide that. Suppressing stuff is just masking the problem, like taking medicine to relieve the symptoms rather than the cause of the medical issue. It might help in the short term but in the long term it may be very harmful.

      • subramanian.g

        Hi,

        But it sure helps in the specific case of having to spot the right warning amongst 100 other irrelevant ones. Some of our codebases are littered with #pragma disable: ### :)

        • http://www.skorks.com Alan Skorkin

          It does, except if you accidentally cause a warning that you’re ignoring, you will never see it at all – not nice. And I find once you open the ignore can of worms, people start to ignore all sorts of things, including things they shouldn’t – even less nice :).

  • Pingback: Blog 03/13/12 Time Tip: Prioritizing