After about a year at the military contractor, I took a day off work to attend a computer conference to see if other companies worked the same messy way. There I found lots of CASE tools and tidy Methodology Manuals showing the “right way” to do thing. Strangely, one of the presenters was from the very company I was working for, with one of my colleague presenting the CASE tools and reliable methodology that “we use for all our projects”. Maybe I had been on unusual projects, but none of the dozens of people I was working with had ever used any of the tools of methodological steps we were presenting. I began to suspect that the same was true for many of the other tools and methodology vendors. I began to wonder if most projects had a much messier reality than their public face.
At this conference, I started chatting with a manager for a large American telecommunications company. It turned out we both had a common acquaintance, who recommended that the American company employ me, and a month later I was (joy of joys) in New York working in their research labs.
What bliss this research was: I could focus on technology without any worries about the dirty messy world of non-technical requirements
I had colleague who spent years working on speech recognition systems, and neural networks, just because these things seemed sexy. I had others working who tried to be at least “grounded” in their research and focused on virtual memory management performance optimization because it seemed potentially useful in some real world setting.
My own work, by the way, drifted accidentally into early research on high performance object oriented programming languages – simply because the dazzlingly intelligent guy heading up this group had a vacancy for which I was the only applicant. We worked for years coming up with all sorts of novel techniques for compilers, language runtime environments, hybrid programming languages (Lisp merged with Prolog, Smalltalk merged with C, etc.) and thought we were conquering the world.
As much as I enjoyed this indulgent lifestyle, I was constantly worried that the research was benefiting the intellectual curiosity of us researchers, but not much of it was really benefiting anybody else. My colleague reassured me that big businesses always had big research budgets, and most never expected most of it to come to anything useful, the underlying hope being that one day some of the geniuses would come up with something earth shatteringly great.
Not much later, the big bosses from head office looked at the research projects in our labs and said “clever, but irrelevant”. They shut down our whole division, putting hundreds of people out of work.
At first I was astounded at how short sighted these guys were; how could they not understand the staggering impact of our technical breakthroughs? Deep down, of course, I knew that all along we had been working on things that pleased us, but never gave much thought to what really mattered to business. Publicly, I blamed the “suits” for never telling us what they wanted. How could anybody produce anything relevant to the business, if the “suits” never gave us any requirements and just let us wander on our own technical whims? At the time, I was sure that this only proved that we were smart, and they were dumb. I decided to move to somewhere where even the suits were smart.