The business value of code elegance in the digital age

Introduction

I was invited two weeks ago to a great event hosted by EPITA entitled “The Elegance of Algorithms”. The host of the debate was Cedric Villani and I really enjoyed this evening of thought-provoking speeches. I was delighted to be invited to join and participate, since the “elegance of programming & algorithms” has been a pet topic of mine for the last 30 years.

I got this invitation thanks to Akim Demaille who remembered that the title of the original web page of the CLAIRE programming language is “The Art of Elegant Programming”. I designed CLAIRE as an open source project in the 90s, together with François Laburthe and a support group including Akim and Stephane Hadinger, as a language that would be ideally suited to write operations research agorithms. CLAIRE is itself the result of 10 years of experience writing compilers for more complex languages such as SPOKE and LAURE. I decided that too much effort had been spent on sophisticated features that programmers did not use much, and that we should rather focus on what made the language versatile and elegant. Our new goal for CLAIRE was to write “executable pseudo-code”, with a language that could be used to teach algorithms, which I did with CLAIRE for a number of years at ENS and Jussieu. Our measure of elegance for CLAIRE was, for instance, the number of lines necessary to express – in an elegant form – the Hungarian matching algorithm. The challenge was to be as compact/efficient as possible – using a high level of abstraction, in the spirit of APL – while staying close to natural language (not the spirit of APL).

There were a number of reasons to focus on elegance 20 years ago. I was already aware that software systems are alive and that a good programming language should facilitate the constant grooming of the code. I had also learned the hard way that only what is simple and elegant survives when generations of programmers work onto a software system. However, what was only a vague intuition then is now blatantly obvious today in the digital age. Which is why I decided to jot down some of the key ideas that I developed in my speech at this EPITA event. Simply said, this post is about the monetary value of software elegance, from architecture to code.

The Business Value of Elegance

1. Software in the digital age is about flows, not assets

The key marker of software in the digital age is the continuous change of software systems; the consequence is that we need to love our code. This is very different from the “system engineering” culture of a few decades ago when we promoted a “black box” approach. If software maintenance required to edit and modify your code constantly, it should be nice to look at ! “Source code is back” and the aesthetics of programming becomes important. Elegance is what helps software systems to evolve into the hands of successive teams of programmers, when “ugly code” is encapsulated and “worked around”, producing hidden complexity and not-so-hidden cost explosion.

This world of constant adaptation is the world of agile software methods, based on iteration. However, iteration produces junk, accumulation of things that quickly becomes useless. It is a law of nature that is not specific to software. Iteration must come hand in hand with refactoring. As a metaphor, we need to tend the garden: clean up, sort and reorganize. Elegance becomes a philosophy and an architecture principle. Gardening means to grow simplicity and to let the “system’s potential” emerge. Digital gardening is similar to Zen gardening : to discover what is permanent in a changing flow.

This vision of systems as flows will intensify in the digital feature when systems are built through the collaboration of AI and human intelligence. Today already, more and more digital algorithms are grown from data through machine learning. This is why Henri Verdier told us that “data is the new code” – I refer the reader to the report on Big Data published by the National Academy of Technologies. What is developed with digital systems are no longer algorithms in the classical sense but processes that grow the necessary algorithms. However, these processes are meta-programmed with code that needs to be elegant for the very same reasons.

2. Digital software is the fruit of collaboration

The act of sharing is critical to building high quality software in a flow environment. This is why code reviews play such an important role and why pair-programming is at the heart of extreme programming. Digital software is the result of team efforts. Code reviews must be enjoyable to be sustained; elegance becomes a Darwinian virtue of code that survives these reviews. During this dinner, I went as far as saying that “elegance is the fuel of code reviews”, as an emotional, aesthetic and collaborative experience.

Digital systems leverage the “power of the crowd” through open source software. We now live in the world of software ecosystems that are built by communities. Software platforms are grown by weaving fragments that get reused through sharing. The elegance of the software fragments is the cement of the open source platforms. The more elegant the code, more eyeballs are reading and proofing it. A law of the digital age is that quality comes from the multiplication of reuse.

Elegance is a dynamic property revealed in a communication process, not a static or aesthetic judgment. This is not new, a few centuries ago, Boileau told us that “Whatever we conceive well we express clearly, and words flow with ease”. Elegance matters to make communication, hence collaboration, easier. An interesting parallel may be made with Cedric Villani’s book, “Birth of a Theorem: A Mathematical Adventure”: scientific creativity is the result of a network whose paths are the laborious efforts – mathematicians in front of their white pages or programmers in from of their screens – to progress along a task, while the nodes are the communication moments when collaboration occurs. Cedric’s book is a brilliant illustration of the importance of the collaboration network. It is, therefore, not a surprise if elegance has been considered a virtue in Mathematics for ages.

3. Today’s main challenge for software is the increasing word complexity

A key ambition of elegance is to reach simplicity. It means not only to search for simplicity of the first attempt or the first creative result, but also the constant quest for continuous simplification, in the scientific tradition of Occam’s razor. Gaston Bachelard told us that “simplicity is the result of a long process of simplification”. Simplicity received a lot of attention during our evening discussion, from a mathematical, musical or software point of view.

Simplicity is a north star of digital system because of its antifragile nature. This great concept from Nassim Taleb means to be reinforced (versus broken) by the constant flow of unexpected and adversarial events. For instance, biological live systems are antifragile whereas most mechanical systems are fragile. Simple code is antifragile – compared to sophisticated one – because it is much more prone to evolve at the same rate as the environment’s change. Digital systems require this behavior; successful platform are built through a constant flow of adaptation. There is much deeper systemic truth here, that you may start to grasp when reading “Simple Rules for a Complex World” by  Donald SullKathleen and M. Eisenhardt.

A cardinal virtue of simplicity in the digital age is the reduction of inertia. This is a core lean principle from Taiichi Ohno. There is much more here than the simple application of Newton dynamics (the lower the mass, the higher the acceleration). Since I was talking to mathematicians, I was able to refer to Pollaczek-Khinchine ‘s formula which I have quoted often in my blog. Without going into the complexity of Jackson networks, this formula expresses the virtue of simplicity (reduction of variation) to ensure a faster response time. This is one of the many case where the intuition of lean and the mathematical models coincide. This translates into a key principle for digital: to continuously clean up the technical debt, since the overall velocity is inversely proportional to the bulk of digital assets.

Conclusion

Throughout this post, I have used the word “elegance” to combine two qualities: simplicity – from a systemic perspective – and aesthetic value, measured though communication and collaboration. The three previous sections could be summarized as:

• Elegance is necessary to break the “time barrier” – allow long-lasting software improvements
• Elegance is necessary to break the “distance barrier” – enabling the collaboration of remote viewpoints to produce better software
• Elegance is necessary to break the “complexity barrier” – in order to design antifragile systems.

This is, on purpose, a conceptual and slightly pedantic way to express why elegance matters. I concluded with a much more forceful anecdote taken from a previous visit to Googleplex in Mountainview, where I have enjoyed reading a few good pages of tips about test-driven development, while standing in the restrooms. This is a vivid example of what a “code loving culture” can be. The silent drama, which is happening right now, is that the gap between those companies that have understood that “software is eating the world” and those who do not is growing. This is where we have come full circle to the main topics of this blog. I will refer the reader to Octo’s great book : “The Web Giants” where almost everything that I mentioned in this post resonate with the best practices of the most advanced software companies.

Code elegance is a fascinating topic and there is much more to say. For instance, it is interesting to try to characterize what makes a code elegant. Cedric Villani started his introduction by saying that elegance in mathematics was the combination of simplicity (concision), efficiency (something that really works) and surprise. The aesthetic value of surprise is a direct consequence of our emotional species-meta-learning strategy to value what is different from we expect (cf. Michio Kaku). I will end this post with a few ideas borrowed from Francois Laburthe, with an interesting reference to SOLID:

• Elegant design borrows heavily on abstraction and genericity. However, units (from functions or classes to module) have a clear and well defined purpose.
• Elegant design is mindful of waste, in the lean tradition. Everything is used, and constant refactoring tracks useless replication of features.
• Elegant design is geared toward self-dissemination because it “welcomes you” through a self-explanatory structure. This is greatly helped by reification / introspection when design elements are self-aware.
• Elegant design is open by nature – it embraces the philosophy of “fragment weaving” mentioned early on through facilitating (hence the importance of API) and selflessness (the design principle that the code will always be used by something more important than itself).

Exponential Information Systems

1. Introduction

I had the chance, two weeks ago, to listen to a dual talk by Norm Judah, the Service CTO from Microsoft, and Bernard Ourghanlian, Microsoft France CTO , when they visited the ICT commission at the NATF . The ICT commission will produce a full report about our investigation into « AI renewal and the explosion of machine learning » next year (similar to our Big Data report last year), but let me share a few random notes. The topic of these two lectures was Machine Learning, and they both started by stating the obvious: Machine Learning is already everywhere in our digital lives. We are using it implicitly all the time in most digital products and services that we use every day. What makes ML the « hot topic of the year … or decade » is both the constant progress and the increase in the progress that we have seen in the past few years. The following illustration shows the reduction of word error rate in speech recognition from Nuance (the Microsoft version of this curve that Bernard Ourghanlian showed is even more impressive).

Similar curves exist for many other ML domains such as image recognition and they show a similar pattern. Microsoft is investing massively in ML, notably through its Cortana platform (and I mean massively – Microsoft CEO, Satya Nadella, has made Machine Learning the #1 priority for the next decade). I will not cover all that I learned about Cortana today, such as « the more you use it, the better it works », but I was struck by two things.

First, we are still in the days of supervised ML, mostly, which means that data (samples for learning) collection, selection and curation are the fundamental skill of the next 10 years. I had heard this idea expressed earlier by Pierre Haren, former CEO of ILOG : IP (intellectual property) of most next-generation technology will be about data sets and learning protocols. The formalization and optimization of these protocols is going to be a wonderful discipline, from an abstract computer science point of view, but that will be the topic of another post. This is very much related to the “data is the new code” claim.

Second, the next wave is indeed the coming of really smart interactive assistants, mostly chatbots. There is so much hype about chatbots that this does not sound like much, but Norm Judah’s talk convinced me that the wave that is coming is bigger than what I thought. One of his most striking argument is that conversational assistants will give access to digital services to new categories of people such as:

• People who do not have a smartphone yet (in emerging countries), using SMS instead,
• People who have a smartphone but are not comfortable with apps (people over 80 for instance),
• People with smartphones and simple mobile application practice, but who lack the level of abstraction and education to feel at ease, or empowered, with many navigation systems proposed by corporations in their commercial apps.

According to Norm Judah, Bots are the revolution of the next 10 years. This revolution is coming faster than the next one: the revolution of business capabilities though general-purpose AI (the special purpose version is already there, as shown in fraud detection). He thinks that most digital applications will have conversational interfaces (such as chatbots) in the next 4 years. Bots will become a prevalent interface for many digital services (but not for everything, sometimes it is still faster to point and click than to tell). A good summary (in French) about the current state of chatbot may be found here, but the key message is that this is only the beginning and that text recognition and understanding is about to make spectacular progress in the years to come. Obviously, this is no way restricted to Microsoft / Cortana, this trend is everywhere : Facebook, Apple or Google for instance.

The upcoming revolution of Machine Learning and Artificial Intelligence is one of the key theme of the bestseller “Exponential Organizations”, a summary of which may be found here. I decided to call this blog post “Exponential Information Systems” because I believe that the general pitch from the book, that the exponential wave of technological progress that is coming demands a new organizational architecture, works for information systems as well. To be more specific, here are three key ideas from that book:

• The exponential rate of change of technologies, such as machine learning or IOT, demands a new kind of organization for enterprises, with more distributed control towards a network of autonomous cells, so that companies adapts continuously to their changing environment and the associated opportunities
• Such organizations – so called “exponential organizations” – exhibit a few SCALE-able traits: to use of on-demand resources, to leverage the power of communities, to master their own field of algorithms – I will underline this aspect in the conclusion - and to engage their customers and partners.
• The set of founding principles for such organizations is summarized with the IDEAS acronym: Interfaces (to attract external contributions), Dashboards (to support decisions from facts), Experimentation, Autonomy (which is the heart of agility and speed) and Social.

I strongly urge you to read this book, it really helps to understand the “exponential” changes of the decade to come, and how to adapt. Today’s post is focused on how this applies – logically - to information systems as well. Section 2 is a transposition of the first idea: information systems also needs to continuously adapt to their digital environment  (both from the outside – the boundaries – and the inside – the underlying technologies) which requires a rate of internal change that was unheard of two decades ago. Section 3 develops the importance of boundaries, interfaces and the enrolment of communities to create ecosystems. There is no surprise here, the networked aspect of the digital economy is rooted in API and cloud services ecosystems. Section 4 focuses on two key characteristics of the digital economy that we just mentioned earlier: the need to base decisions on facts – and re-evaluate them regularly as everything changes – and the need for “tinkering”, the name given by Nassim Taleb and Salim Ismail to experimentation. It should be obvious to anyone that these two characteristics are inherited from the information systems capabilities. The last section summarizes the principle of “capability architecture”, whose goal is to build a scalable playground for tomorrow’s algorithms. The new name of the enterprise architecture game is symbiosis, as is wonderfully explained in Erik Campanini and Kyle Hutchins new book “Darwinism in a consumer-driven world”.

To summarize, one could say that Exponential Information Systems are those whose architecture, software culture and lifecycle make ready to leverage the exponential wave of new digital technology. Let me end this introduction with a great quote from Pierre-Jean Benghozi in his article “Digital Economy: A Disruptive Economy? : “ … Eventually, it is the constant transformation of their models that grants companies a kind of stability and resilience … not simply the better use of their strategic resources or the answer to changes of their environment through innovation”.

2. Digital World Refresh Rate is a Game Changer

I have spent a decade experimenting – when I was the CIO of Bouygues Telecom – and theorizing the need for constant change and refresh of information system elements. This is a key focus of my lecture at the Ecole Polytechnique, of my second book and some of these posts. Information Systems must be seen as living organisms, which was one of the reasons to start this blog in the first place. I have developed techniques and formulas to build sustainable and cost-efficient information systems based on the proper refresh rate that ensures that the technical debt stays moderate and that the benefits of new technology (e.g. Moore’s Law) may be leveraged.

However, my 10-years-old thinking is now totally obsolete, as we need to increase the refresh rate by one order of magnitude. This is the key message from Exponential Organizations: get ready for waves after waves of technologies for building better web sites, better mobile apps, better big data systems, better machine learning algorithms, better artificial intelligence frameworks, smarter connected object, etc. I could reuse the modeling approach that I developed earlier but I will spare you the unnecessary formalization: the first consequence is that Information Systems must indeed increase their refresh rate to incorporate these technology waves without too much delay. The reason is not only that the exponential wave of new technologies open a wave of new opportunities – which they do – but also that new technologies allow to do things much cheaper and much faster, as is illustrated with Big Data technology, creating huge opportunities and risk of disruption. The second consequence is that, to maintain a high refresh rate while keeping the IT budget moderate, one needs to reduce the footprint. Modern information systems are required to move fast, constantly, and must, therefore, reduce their inertia. This is closely related to Klaus Schwab’s famous quote : “In the new world, it is not the big fish which eats the small fish, it’s the fast fish which eats the slow fish.“

Somehow these ideas were already true 20 years ago, it is just that the stakes have been raised and these principles have become mandatory. When I give a lecture about information systems in the digital age  the pivotal moment is the need to write systems that will change constantly, this is why continuous build & delivery is so important, this is why we need new ways of writing code. The mandate to keep IS small and as much free of technical debt as possible is a tough one. It is not a universal or homogeneous requirement throughout the information system: As in any living organism each part has its own renewal rate. For reasons that would be too long to explain here, refresh rate is expected to follow a power law; some hubs need to evolve constantly while some leaves may enjoy a peaceful life. Indeed, a good architecture is one that let each component evolve at its own rate – a key lesson from my tenure at Bouygues Telecom that was the topic of my first book and which I will address in section 4. This brings us to the quest for modularity, an architecture mandate which is more relevant than ever in the digital world.

An ideal information system, like an ideal organization as defined in BetaCodex is one that is built like a living cell (hence the illustration). The membrane and everything close to it evolves faster than the core of the cell. Thinking about changes from an edge/frontier perspective is a key insight from books such as “Exponential Organizations” – cf. “scaling the edge” in this  Deloitte summary - or system thinkers such as Idriss Abderkane  since adaptation in living systems usually starts from the edge, not the center. Here is another quote from “Complex Systems” by Par Terry R. J. Bossomaier and David G. Green : “Because systems near the edge have the richest, most complex behavior, such systems can adapt better and therefore have a selective advantage over other systems”. We will see this principle put to action in the capability map of Section 5. Sustainability and Systemic thinking is critical when designing information systems; otherwise massive amounts of refresh spending may be applied without changing much because the burden / mass of legacy is too large. I have seen too many times, in my days as CIO, people reasoning about change from the effort that was made to add new things without reflecting about the inertia of the technical debt.

3. Do Less, Hence Leverage Others' Work

This section will be short since I have already developed this idea in other posts : the only way  to churn out more software, with better quality without raising the cost is to massively reuse more from outside. The associated illustration is a living tissue as a metaphor of the cells working together in an ecosystem. In the same way that change happens both from the outside in and from within, software reuse also happens through the frontiers and from within. Without any attempt to be exhaustive, there are four major “engines” to churn software faster and I will focus today on the last since it is related to the previous section:

• The first pillar is the use of open source software . This is not the only way to reuse, but this is mandatory in the digital world because the open-source community is where a lot of the software innovation occurs. There is a Darwinian advantage for open-source software: the way it is build – incrementally, around communities, continuous testing and debugging with thousands of eyeballs, constant change, etc. -  produces the traits that are necessary to fit the digital environment. It is not a simple mandate since it requires strong changes in software architecture and culture for many companies.
• Reuse “at the frontier” is to develop APIs to propose and use web services. The importance of API does not need to be emphasized since the “Jeff Bezos memo”. There is also a culture dimension since API are attached to ecosystems, and since the best way to learn the API game is to participate to software communities.
• Delivering software faster requires full automation, hence the rise of DevOps and continuous build & delivery. There is more to DevOps than continuous delivery but the key point here is that modern software factories come with automation tools that may not be ignored. This is the first lesson of my four years building set-top boxes at Bouygues Telecom.
• Heterogeneous change-architecture with a focus on “agile borders” or “adaptive edge”. The most common pattern for this idea the the bi-modal architecture. Bi-modal is a very old idea in IS architecture. For instance, Octo and Pierre Pezziardi developed the empire/barbarian metaphor 15 years ago (the empire evolve more slowly than the barbarian outpost). Bi-modal is an obvious simplification since the core-to-edge differential design may be decomposed in many ways (cf. our Section 5).

Bi-modal is both a great and dangerous idea. Bi-modal defines two parts of the Information System realm, one where the legacy stands and one where the new digital frontier is being implemented. In his article, Bernard Golden develops similar reasons about the why and the importance of DevOps. It also leverages a key idea of the digital world: the best way to learn how to develop like the best digital companies is to use the same tools. Bi-modal is a great idea because it defines a new domain where the “new rules of the digital world” may be introduced. It makes “experimentation” much easier and support the introduction and development of a new software culture. Jay Fry blog post develop the same ideas, with a focus on the necessary disruption, which is precisely the same as the requirement for a new rate of change.

However, warnings started to be heard not late after Gartner re-introduced this idea in 2014. If taken literally, it may be seen as an excuse not to change the legacy, which would be wrong according to the previous section and according to Jason Bloomberg. In his article “Bimodal IT: Gartner recipe for disaster”, he proposed a caricatured version (slow versus fast IT) which he rightly criticizes.  He makes a thoroughly correct argument that the digital revolution need to embrace the whole information system.  In a more recent post entitled “Saying goodbye to bimodal IT”, Mark Campbell makes a harsh criticism about keeping the legacy untouched and mostly against a silo culture that will prevent global change. Change and new capabilities are indeed required everywhere; I will return to this point in the next two sections. 

This being said, I still believe that “bi-modal” is necessary because of speed requirement (back to the “rate of change” topic of the previous section). Everything mentioned in this section requires speedy transformation: the speed to adapt to a new open-source culture, the speed to adopt new software factory tools, the speed to learn how to play the API game. All these changes are mandates for the whole information systems, but if there is one common rate of change, it is mechanically too slow, because of the weight of legacy.

4. Fractal Capability Design

I started using the concept of capabilities in 2004 after reading the wonderful book of François Jullien “A Treatise on Efficacy”. Capabilities are the practical tool that one may use as a CIO or VP of engineering to develop a company’s situation potential. Very crudely summarized, François Jullien’s suggestion is to borrow from Chinese philosophy to develop strategies that are better suited to the complex and impossible to forecast world of our 21^st century. François Jullien opposes the Greek philosophy which is governed by goals and actions plans, where the player is applying his or her will to the world to change outcomes in a top-down manner, to the Chinese philosophy which is governed by situation potential and opportunities, where the player is using the environment as it comes and builds her or his potential in a bottom-up manner. In terms of IT strategy, it means that we stop building strategic plans where we forecast how the information systems will be used 5 years from now, but that we think about what opportunities could exist, what would be needed to best leverage these opportunities – the capabilities - and how to best use the next 5 years to grow the capabilities. If the difference is not clear, please read the book, it is simply the most relevant business book I have ever read.

There are many capabilities that are required in the digital world, but I would like to focus on three key capabilities:

• Mashup, that is, the ability to combine swiftly and deftly different services into new ones. The new term is composite applications, but I like this reference to the “old web days”. Being able “to mashup” is the first digital capability, derived from constant practice and open choices. The mashup capability is derived from many paradigms: source code integration, (micro) service-oriented architecture, tinkering culture, open source community adoption, etc.
• Complex event processing, which is the ability to collect, process and react to complex collection of events. CEP is also sitting on a large array of technologies, from event-driven architecture to rule-based automation. Complex event processing is the critical capability for conversations, and we know that in the digital world, “markets are conversation”. Obviously the arrival of massive amounts of artificial intelligence will make contextual and ambient intelligence a must for future digital services.
• Machine learning, as said in the introduction, plays a critical role everywhere. It is used to generate insights, to forecast and simplify usage, to personalize without becoming cumbersome, to detect threats and fraud. Machine learning is the critical capability for relevance, and no customer has time for irrelevant experiences in this new century.

The concepts behind these capabilities are not new, but the level of excellence that one may reach is constantly changing, hence the situation potential is changing. These capabilities intersect each other and reinforce each other. The consequence of the digital challenges is that information systems must develop these three capabilities in each of their components. I remember an old debate in 2007 when we discussed the ideal service architecture for Telcos. I came up with this motto “mashup everywhere” which meant that we had to build this “on-the-fly-composite-service” capability in the network, on our service delivery platform (SDP) and on the devices themselves – at the “edge” of the network.    This is truer than ever, the “exponential” wave of technologies means that these capabilities will have major impacts in all components of the information system. This leads to the concept of “fractal capabilities” which I have illustrated with the snowflake metaphor. “Fractal capability” means that the capability is implemented in a recursive manner, at different scales. To take another example from the previous three, I would propose today that "analytics capabilities must be everywhere" : the ability to take decisions based on facts is a given of the digital age, from core to edge.

The consequence, and the best illustration of this concept, is that EIP (Enterprise Integration Platform) is a concept more than a physical platform. As a system, it is a fractal architecture made of multiple integration patterns, at different scale. This is definitely not new, this was the topic of my first book more than 10 years ago and the lesson learned from many years of experience. Even in those years there were many reasons for the “fractal” design:

• There are too many constraints (different integration challenges for each different for each frontier) and its is hard to find a technology that matches all of them,
• Different components have different lifecycle; thus a fractal / recursive implementation (federated collection of platforms) is more flexible and adapts more easily to different “takt times”,
• Integration is about ecosystems: open integration is even more challenging, since each frontier comes with its own culture, not simply technical constraints,
• Resilience : fractal design supports the independence of sub-regions and reduces the possibility of a SPOF.

“Exponential times" amplify this set of constraints: seeing the EIP as a single platform becomes a juggling act. The successive waves and forms of integration technologies, patterns and ecosystems is much better suited to a fractal approach with the flexibility of organic thinking.

When I speak about system capabilities, it relates to people (teams) as much as technology. Capabilities are born from skills, and skills in an exponential world are developed through practice, through experimentation. Experimentation is a key pillar from the « Exponential Organizations » book : « Constant experimentation and process iteration are now the only ways to reduce risk. Large numbers of bottom-up ideas, properly filtered, always trump top-down thinking, no matter the industry or organization »;  « As Nassim Taleb explains, “Knowledge gives you a little bit of an edge, but tinkering (trial and error) is the equivalent of 1,000 IQ points. It is tinkering that allowed the Industrial Revolution.» In a world of constant exponential change, tinkering often beats expertise. A truly interesting example is given about a natural language contest where the most innovative team was a cross-functional team with diverse backgrounds and no true experts: « none of the winners had prior experience with natural language processing (NLP). Nonetheless, they beat the experts, many of them with decades of experience in NLP under their belts ». As I have expressed many times in this blog, "tinkering capabilities" are crucial to be able to import innovative technologies from the outside (you cannot buy a digital innovative platform that you do not understand, and understanding is achieved through practice).

5. Thinking Outside The Box: Four-Tiers Playground Landscape

I will end this post with a simple chart (below) which is a summary of the different ideas expressed so far, expressed as a “capability landscape”. It is a follow-up of a previous post entitled “software ecosystems and application sustainability”. There is a tension expressed here: on the one hand, many capabilities are “fractal” and must be developed everywhere; but on the other hand, it is always better to reuse what exists and organic design shows a way to deploy change faster. This landscape (not to be confused with a system architecture chart) is made of four domains, which are interwoven through the “fractal enterprise integration platform” that was just discussed:

1. Business / core capabilities: the traditional domain of IT as a function for business support. A key challenge is to make business capabilities easy to reuse and to share, hence the focus micro-service architecture and internal APIs. As expressed in the previous section, mashup, tinkering (CORE requires its own sandbox) or CEP must be grown capabilities in this domain.
2. Engagement capabilities:  a first frontier towards customers and partners, which is characterized by strong tinkering & integration capabilities. Since the first role of the engagement frontier is to “start conversations”, capabilities such as contextual intelligence or personalization are critical. This domain, called "matrix" on the schema, is the frontier with the outside world; it is the place where outside innovation gets mashed up with the enterprise own capabilities.
3. External engagement capabilities: where the state-of-the-art of advanced capabilities live. The topic of this whole post is to get ready to ingest massive amounts of innovation to come, and this is done by mashing up the “agile frontier” of the information systems with services delivered by advanced partners.
4. Last, I have represented the “Digital customer world”, with its own capabilities. There is a larger world outside, where our customer live, and it is also getting richer of advanced  capacities (such as smart assistants or machine learning) at exponential rate. Companies need to develop a symbiosis with this ever-richer digital environment and try not to compete with it.

To understand the value behind the decomposition of the “capability landscape” into four domains, it is interesting to look at the three following pairs. First, the matrix/edge distinction is important because the edge is bigger than the frontier, so companies must learn to become “parasites” of the software ecosystems that live on the edge. Leveraging the smartphone and the application stores is the most obvious example. Second, I have kept a bimodal core/matrix distinction because of the reasons exposed in Section 3, bit remember that the core domain needs to evolve and that it is also under disruptive attack from “Barbarians”. Last, I have emphasized a matrix/cloud difference because exponential technologies will grow – very fast – outside the enterprise and our challenge is to learn to use the capabilities before we master them (the mastery comes as a consequence of the practice, not as a prerequisite).

To conclude, I will rephrase the importance of algorithms, which is a cornerstone of the “Exponential Organizations” book, by borrowing three quotes. The first one tells about Allstate, shown as an example of a company that has learned to use algorithm competitions – similar to the ROADEF annual challenges – to improve their own state of the art: « It turned out that the Allstate algorithm, which has been carefully optimized for over six decades, was bested within three days by 107 competing teams. Three months later, when the contest ended, Allstate’s original algorithm had been improved 271 percent. And while the prize set the company back $10,000, the resulting cost savings due to better algorithms was estimated to be in the tens of millions annually”. The second quote tells about the UPS example of improving their telematics algorithm and shows that algorithms – which have always been important – are becoming mission-critical for many business as they become digital: « Given that the 55,000 trucks in UPS’s American fleet make sixteen million deliveries daily, the potential for inefficient routing is enormous. But by applying telematics and algorithms, the company saves its drivers eighty-five million miles a year, resulting in a cost savings of $2.55 billion. Similar applications in healthcare, energy and financial services mean that we’re entering a world of Algorithms R Us ». The last quote is a nice way to round the circle with my introduction on the importance and exponential progresses of Machine Learning: “ Machine Learning is the ability to accurately perform new, unseen tasks, built on known properties learned from training or historic data, and based on prediction”.