Flexibility and Biology

In a previous post I have listed flexibility as one of the four pillars of agility. I defined flexibility very loosely with the following sentence: " the technical ability to cover a large span of business situations and processes with the existing set of IT services and capabilities."

This requires a little bit of further explaining. Especially, there are two ways to obtain this ability which are both relevant:

• The ability to implement change in order to achieve functional flexibility. This is foremost a matter of architecture. Hence it is the first thing that comes to mind, in the context of "SOA and Agility". This flexibility is expressed at the component level (the ability to parameterize: change the functional behavior though a change of parameters –cf. the 12^th chapter of my book on SOA), and at the "integration level", i.e. the ability to integrate a new component quickly and efficiently.
  
• The ability to adapt to a new situation "without change", that is without having to stop and modify one or more components. This kind of flexibility is a combination of "meta-data architecture", that is, the ability to change the behavior of information systems through the change of parameters and the ability to do so dynamically. This is the most important part, and the one that is most often overlooked. In many cases, changing parameters is still a new project that requires testing, synchronization and going live with a new version.
  

We can summarize by defining flexibility as the combination of functional flexibility (ease of change) and operational flexibility ( at run-time, without change to the existing binaries).

The first kind may be labeled "mechanical flexibility". The analogy makes sense because when you modify a mechanical device, you must (most often) shut it down before changing or adding a new piece. Most of what you can read about IT agility is actually of this kind. New technology, rule-based engines, meta-data parsers are all designed to extend the functional flexibility. This is definitely true of integration infrastructure technology that I have come across during the last few years. There are a few noticeable exceptions (such as using UDDI for dynamic discovery of services) but … surprisingly … they have not made it to mainstream IT yet.

I will call the second kind "organic flexibility" for obvious reasons: changing the SI without shutting it down (without the "go live" step) reminds us of living organisms which grow, change and adapt themselves without having to halt their biological processes. The boundary is subtle: everyday running systems are modified by the input of their users. So what? It turns out that for the same systems (at least those that I known of) there also exists "hidden"/admin parameters that cannot be changed without stopping and restarting the system. It also turns out that most major changes require writing new code, to re-compile, re-link or re-connect, hence a new "go live".

I must confess that I have been trained for twenty years to think about flexibility in a mechanical way. However, I now believe that the flexibility that matters the most is the organic kind. This will become truer and truer as the stringent requirements of real-time digital business will demand no shut-downs of any kind.

Organic flexibility is a matter of organization, people and processes as much as of technology. However, there is a technological part: one may not achieve this behavior (easily) with legacy technology. A key is obviously the availability of dynamic meta-data that is interpreted as parameters and modifies the behavior in a deep way. The challenge starts when different components must share a change of parameters to achieve a new behavior. This means and requires that propagating new parameters becomes a IT run-time service. The technology for this type of propagation is there, once again we find rule-based engines, scripting engine, automation tools such as processflows. If I reflect on the IT projects that I have been a part of, 80% of the energy is spent making sure that there is enough flexibility ("enough parameters") to make the component "future-proof", and 20% is spent thinking about the processes that are used to change these parameters. Designing an organic information system requires to change the ratio to the opposite.