The reason most great writers are great is that they work hard on their writing. The same goes for ideas: the reason most great researchers have great ideas is that they work hard on their ideas. What you typically see is not the 50-75% of ideas they had that were fundamentally flawed, but one of those remaining ideas that they have polished into a pearl. Nearly all of the remaining ideas except this pearl were in some sense wrong, but these ideas were somehow used to refine their understanding of the problem and the solution to it – without those ideas the final idea would not have materialized since their understanding at that stage was still incomplete. It is unfortunately not very likely that you will be able to observe this process directly from the masters: they are just as afraid as you of being wrong, and they probably also have a reputation of being incredibly smart that they believe they need to uphold to get future research funding.

I wish I could offer more information on how this refinement of ideas works, but I don’t understand it well enough for a meta-analysis. I do want to point out some tools though, and the first one is writing. Writing inevitably exposes the flaws in your understanding, and is perhaps our greatest tool. I realize your goal is not to do research, but there are so many similarities between the ideas you want to generate and research ideas that it would be silly of me to not point this out: look at the presentation for “How to write a good research paper” here (1 hour long, but well worth it).

Writing your ideas down for yourself when you have them is great advice. This means you need to have a recorder, phone, or a notebook and pen with you at all times. As a techie it is easy to go overboard with processes and ticketing systems and so on, but the guideline should be to keep it simple so you can jot down whatever thought you have at any time. Personally I go for a traditional space pen/moleskine combo. I keep the pen in my pocket at all times and it is roughly the same length as my house key.

Finally I want to point to a paper by Brian Martin about research productivity. It’s just a few pages so well worth your time.

Scott usually has very sensible stuff to say. That is always great advice especially when you’re a business, although you don’t always need to generate good ideas for business purposes. However you will never go wrong concentrating on issues which are a pain point for a bunch of people.

I love reading your blog. It seems obvious that besides having great ideas, you invest an amazing ammount of time in it.

Regards,
Droope

That’s definitely the right attitude, the other thing to remember as well is the fact that there really are no shortcuts. If you want something good to happen you have to put in the effort.