Tuesday, September 4, 2007

"Picking the Right Tool" is a Tautology

On any given system development project, the individual or team charged with "tool selection" is always going to pick "the right tool". No one is ever going to think "I'm intentionally picking the wrong tool for this job." They might knowingly opt to pick one that is technically inferior to others, but the justification for doing that comes as a result of considering the Political, Operational, Economic, and Technology (POET) factors, making the overall selection the "right" one.

(In hindsight it might be discovered that a choice was "wrong", but that's because some aspect of P, O, E, or T wasn't correctly understood, i.e. "it seemed like a good idea at the time". And it's also possible that those not in on the selection process may assert that a wrong choice was made, but well, then, make the case in the POET framework.)

My concern is that over the last several years the underlying technology is getting more and more precarious, more and more ad hoc, and getting less and less respect.

Many years ago I read an article espousing a similar concern. I don't remember the exact paragraph wording, but the gist of it was that there are people that constantly worry a great deal and think very deeply about very complicated and arcane things going on in the bowels of a data center whose failure could bring down the company, or even a chunk of the economy. What they do is not sexy, or cool, or cutting edge, and their work doesn't lead to startups and high-demand IPOs. But without their skills and commitment to do what they do the whole thing starts to fall apart.

That kind of work is at the very bottom of the POET stack, it's "sub-T" in fact. Someone is writing OSes and compilers and JIT JVMs and JDKs and web servers and fault-tolerant middleware and Perl interpreters and JavaScript engines and XML parsers and on and on. Everything in the technology stack, and the system development stack, rests on them.

And "picking the right tool" blithely assumes that these technologies are commodities and it's simply a matter of performing a suitably comprehensive trade study.

But I don't like what I'm seeing down here at the bottom of the stack.

I see programming languages getting hacked up to include new features that show little regard for maintaining a consistent and coherent language design, I see security being "patched in" to systems rather than being built in, and I see more and more features being packed into products that make them increasingly brittle. The systems are arguably "better" than what they were before, but I believe it is becoming a matter of plugging holes in dikes and building McMansions on landfills.

As Dan mentioned, technologists and developers almost always naively put their 'T' at the top of the stack, which sets them up for failure in a business goal oriented environment. Unfortunately, by kicking 'T' down the stack the acknowledgment of its criticality got demoted as well, so much so that technology evaluation becomes a matter of feature and price rather than quality engineering.

If you don't recognize the need to truly engineer a technology and acknowledge that technology choice actually does matter, and that tech is not a commodity, but is the foundation on which all your Service Oriented Architectures and Business Processes and Customer Relationship Management is based, your system is going to end up in pretty "POE" shape.


Dan said...

T might be last in the priorities, but it's still important. My point is it must support purpose, operations, and economic objectives, otherwise, why are we getting the technology to begin with? Technies often fall in love with their technology. This is normal.

What is interesting about working a technology effort is that it often ripples up to higher layers in an EA topology. Technology often affects business processes, data stores, and most certaining applications, software, and systems.


Lucretia9 said...

I disagree with your first sentence. A lot of the people doing the tool selection are management level and they don't know anything about this sort of stuff and end up picking C++ because it's the current fad. If/when D becomes the new C++, they'll start to jump on that bandwagon as well.

If you look, even places which were using Ada are now moving over to C/C++ which is scary.

C/C++ has it's place, but if you want something that hasn't been hacked up, don't use those languages.

Marc said...


The assertion of the first statement is from the perspective of the person (or committee) making the selection--note the second statement.

And unfortunately the "selecters" do too often equate "best known/most popular" with "best", just as you note with the past choosing of C++, even for those projects that originally went with Ada. And now the industry has gone with Java because it's got management mind share.

One can see how this happens, with today's first- and second-line managers who are tasked with making the tool and technology choices (versus the higher-level managers making the strategic choices) being yesterday's programmers, when Java was the hot technology, and so that now gets carried through.

Meanwhile today's edge programmers are hot with "scripting" and functional languages, e.g. Haskell, Erlang, Ruby, Python, Scala, etc. So guess what we'll be seeing calls for in 10 years? :-)

Fl'ame said...

I totally agree with your perspective on technology and your impression that our "stack" looks like a rag rug.
I write this comment to point your attention to the following article. I estimate, you will like it.


An ADA programer