Saturday, November 26, 2011

Lean IT, Devops and Cloud Programming


I have become more and more interested with lean IT over the years. It started with the book "The Art of Lean Software Development" by Curt Hibbs. 
I enjoyed this book because its introduction of lean is 100% compatible of what I learned from Jeffrey Liker and other great authors about TPS. This simple book helped me draw the lines between good software development methods such as extreme programming or agile programming, and lean management. For those of you who are not familiar with lean or extreme programming, here is a very crude summary of some of the most salient points of lean software development :
  • No delays: avoid as much as possible work that is sitting between two steps of the development process (what is called WIP – work in process – in the lean jargon). This is true at the process level (the goal is to design a streamlines/ “single piece flow” development organization) and the developer level. A practical goal is to avoid switching tasks as much as possible: focus on one thing and do it right!
  • Quality in (right first time) :  this lean principle translates into testing as early as possible (a tenet from agile programming) but also to use all techniques that improve the quality of the code, even at the expense of source code productivity since we all know that it is cheaper not to produce a bug than to remove it later. Here comes, for instance, the practice of pair programming and code reviews, but also “good practices” such as programming guidelines and standards
  • Fast delivery: the lean principle is to reduce the “lead time” of the software development process, which requires working on all stages. Removing in-between delays (cf. earlier) is necessary but not enough. Continuous integration is a core technique to achieve this goal, such as fast deployment techniques.
  • Short deliveries: It is more efficient to produce small pieces of software at a high rate than bigger pieces at a lower rate. This is another key principle from lean (“small batches”), which is doubly true for software development: not only are l smaller batches easier to build (a well-known law of software engineering), but the continuous evolution of customers needs and environment makes it easier to adapt with a small batch approach.
  • Less code: this is the application of the KISS principle !  Lean software development tries to stay away from complexity (see later in this post).  Unnecessary code may pop up from many sources, lean applies a technique called VSM (Value-Stream Mapping) and a posture of “muda removal”.  Muda (waste) removal means to go through the process with the “eyes of the customer” and remove all that does not produce value from her perspective. VSM is a tool that tracks the value creation and assigns it to each step of the process. Lean software development aims at producing the right product, without unnecessary features. It is also an architecture principle (stay away from complexity) designed at simpler and faster maintenance over the years.
  • Customer participation: the best way to produce only what is necessary from the user’s perspective is to ask her frequently ! This is why end-user/customer participation is a tenet of agile programming. When the customer is not available, the principle mixes with “small batches” to become : deliver fast, fail faster to succeed sooner.
  • Regular work effort : the leveling of the effort is a key principle from extreme programming, the equivalent of “heijunka” from lean programming. A few years ago, when I was still a CIO, I started thinking about “extreme IT” (applying extreme programming at the information system level), and the sustainability of the effort is a crucial point. Regularity has a counterpart, which is discipline. Using methods and tools (such as configuration management, source code versioning, automated testing) is crucial. One should take a look at the wonderful talk from Mark Striebeck on “Creating a testing culture” at Google.
  • Minimize handovers: complex tasks, such as writing software, are better accomplished when the responsibility chain is not truncated into too many segments. Working as a team is the only alternative to deliver complex projects (this is the topic of my third book). This is another insight from the Agile Manifesto, this is why today’s best practice it to assemble multi-disciplinary teams including software developers, marketers and designers.
There are a number of interesting books on this topic. The most classical ones are those from Mary Poppendieck, such as “Implementing Lean Software Development: from Concept to Cash”.  I had the pleasure of presenting my last book at the Lean IT Summit a month ago and to meet with Mary Poppendieck (a number of great talks, including hers, Michael Ballé’s and Georges Striebeck’s, are now available online).
There are also books which are not about software development per se but very closely related. I am especially fond of two books which I have reviewed in my French blog:
Thanks to Guillaume Fortaine, I have come to learn about Devops. Quoting from Wikipedia, “DevOps" is an emerging set of principles, methods and practices for communication, collaboration and integration between software development (application/software engineering) and IT operations (systems administration/infrastructure) professionals.  The more I read about Devops (Devops.com is a treasure trove of great articles), the more I find that Devops is the missing link between Agile/Extreme Programming and lean management (Toyota-style, hence Lean-Startup-style).  Although Devops claims to “help finishing what Agile Development started”, there are different flavors in what one may read on the web: bridging the development /admin & operations gap, making lean IT happen, making collaboration happen (and also),  bringing agility to operations,  which in turns makes it easier to leverage the benefits of new computing resources such as cloud programming. My first obvious interest with Devops has been the practical deployment of lean IT, as a followup of what I just explained earlier. For instance, Devops promote a number of interesting tools related to deployment automation, such  as GLU. It turns out that the “Cloud and Devops intersection” is also quite promising. Indeed, leveraging the strengths of cloud programming requires a shift in development/architecture culture that requires a “Devops-like” approach.

It happens that I gave a talk last week at the Model-Driven Day about « Complexity, Modularity and Abstraction” (the talk is available in the “My Shared Document” box on the left). The talk is about, among other things:
  • Complexity (why it is important, how to measure it and to tame complexity – since avoiding complexity altogether is not an option at the IS level)
  •  Sustainability: how to transform enterprise architecture into a regular practice. This is related to the concept of “extreme IT” : avoid the “heroic struggle” to move towards the “continuous transformation of the information system”. This is a major reason why I have been advocating for SOA as a company-wide enterprise architecture practice for many years.
  • Architecture-oriented services” : I made this pun to emphasize the difficulty to produce the “right” services through SOA. “Architecture-oriented” means services that have the right level of abstraction, that are modular and “easy-to-compose”. To my knowledge, there is no easy recipe for this, but the wisdom and folklore of 40 years of software architecture design apply.
  •  Cloud computing: I have added a small addition about “cloud-ready architecture” in the 4th edition of my first book. I strongly believe that Information Systems will change in a spectacular manner when we learn how to exploit massively parallel architecture (using tools/approaches such as Hadoop/MapReduce). Using Cloud Computing to provide with a few Saas-based front-office services is nice and relevant, but the big change comes when cloud-computing is applied to back-office services (provisioning, billing, data mining). This requires an architecture change, but mostly requires a culture change.
The thought that prompted me to write this post is as follows. Devops is the missing link between the themes of my MDD talk: managing complexity, delivering agility/modularity and moving to the new century of massively parallel computing (including, but not restricted to, cloud computing). I speak of "Cloud Programming" in the title because I agree with a comment made by Georges Reese: the Cloud is a computing resources defined by its API, i.e., a resource that is managed through programming. The exact quote is:
Cloud is, for the purposes of this discussion, is sort of the pinnacle of SOA in that it makes everything controllable through an API. If it has no API it is not Cloud. If you buy into this then you agree that everything that is Cloud is ultimately programmable.
 Not only the cloud is more agile (up/down scalability), it may be controled by the piece of software which is using it. From a systemic perspective, it's a "whole new game" :)

47 comments:

Matthias Marschall said...

Hi Yves,

thanks a lot for putting this post together. It is a really great overview on lean, agile, and DevOps.

I like your positioning of DevOps as "missing link between Agile/Extreme Programming and lean management". That really puts it nicely.

I hope, more and more folks understand how agile, lean, and DevOps work together to create more flow in software development organisations.

Yves Caseau said...

A great follow-up is this article on the OCTO blog: http://blog.octo.com/en/devops-tips-and-tricks-on-the-ops-side/

Yves Caseau said...
This comment has been removed by the author.
Giom said...

Je remonte un vieux post parce qu'il y a un petit truc qui me turlupine à chaque fois que je le vois.
Tu dis souvent qu'il faut maintenant faire du "right first time". Autant je comprend très bien ce que tu dis sur le fait qu'il soit moins cher de faire bien tout de suite plutôt que de corriger plus tard, autant dans une vision plus macro je trouve que parfois la volonté de sortir des produits avec zéro défauts est une utopie qui fait perdre plus de temps qu'autre chose. Au niveau du développement de nouveaux services numériques j'avais plus l'impression qu'au contraire on migrait plutôt vers un fonctionnement "à la google" où l'on propose des versions beta aux clients le plus vite possible afin d'avoir des retours le plus tôt possible. Et maintenant que les gens ont compris qu'ils pouvaient mettre leurs appareils à jour on voit aussi sortir sur le marché des appareils non exempts de défauts mais qu'on corrigera avec une mise à jour ou dans une version 2...
Bref tout ça pour dire que le "right first time" d'accord au niveau des développements, mais pas au niveau de l'entreprise qui se refuserait à proposer à ses clients un produit non parfait.

Yves Caseau said...

Très bon point, qui mériterait un billet à lui tout seul. Il y a deux tensions dans le leanIT: (a) faire vite et faire zero défauts, (b) faire en juste à temps et anticiper/préparer/ cultiver. Pour faire une réponse rapide sur le premier point: il ne s'agit pas d'écrire du code avec toutes les exigences du premier coup, il faut en relacher certaines pour implémenter les petits lots (à la Google, à la Ries, à la prétoyping, etc.), en revanche, même pour un lot "beta" ou un MVP (minimum viable product), il faut viser l'écriture d'un code de qualité ! C'est vraiment un point clé: pour aller plus vite, on écrit moins de code, on remet des choses à plus tard, mais ce qu'on écrit est (1) propre et élégant (2) testé (3) architecturé/organisé. J'ai passé une grande partie de ma vie à écrire du code en mode ultra-rapide (ie, faire un POC en un week-end pour démontrer une idée), et je partage profondément l'idée lean que l'agilité n'est pas une excuse pour du code sale. Tout code sale (et souvent trop long) coûte incroyablement cher ... tot ou tard. Donc le slogan "quality first" s'interprete par "écrire toujours du code de qualité, et en écrire moins pour aller plus vite".

Unknown said...

Because your business is similar to others does not make it the same, and these often subtle differences in the way different businesses operate and are managed mean that the systems and software that suit one business will not meet the requirements of another company.We offer a wide range of Web Development Leeds services for mobile handheld PDA devices.

Unknown said...

This site is good because they give us a new thing and new ideas and new topic how good all of they are we should appreciate them because of these good thing.
seo company bangladesh

seema said...

very nice article.Thanks for sharing the Post...!


scrum method

Unknown said...

It's Very Informative Blog... Thanks for Posting...
DEVOPS

soumya said...

Thank you for sharing wonderful information with us to get some idea about that content. check it once through Devops Online Training Hyderabad

Unknown said...

Being new to the blogging world I feel like there is still so much to learn. Your tips helped to clarify a few things for me on Devops. Great Sharing keep up the good work.
Thanks & Regards
Devops and Its Significance

vepambattu chandu said...

very nice information about devops thanks for sharing..

DevOps Training in Hyderabad | DevOps Online Training

Sugantha Raja said...

I appreciate you for providing such a valuable content for us.Helped a lot in increasing my knowledge on DevOps. DevOps integrates tools for an advanced Software Delivery Pipeline emphasis on quality, automation and results.

DevOps Training in Chennai | DevOps Course in Chennai

Chandu Chinnu said...

Thanks for giving a great information about DevOps Good Explination nice Article
anyone want to learn advance devops tools or devops online training
DevOps Online Training
DevOps Online Training hyderabad
DevOps Training
DevOps Training institute in

Ameerpet

Unknown said...

DevOps training in Chennai will prepare you for a career in DevOps, the fast-growing field that bridges the gap between software developers and operations. You’ll become an expert in the principles of continuous development and deployment, automation of configuration management, inter-team collaboration and IT service agility, using modern DevOps tools such as Git, Docker, Jenkins, Puppet, and Nagios. DevOps jobs are highly paid and in great demand, so start on your path today.

DevOps Course Fees in Chennai | DevOps Training Credosystemz | DevOps Training in Chennai Quora | DevOps training in chennai with placement | Best DevOps training in chennai | Best DevOps training Institute in chennai | DevOps Certification Course in Chennai

IT Tutorials said...

Thanks for helping me to understand basic concepts. As a beginner in DevOps your post helps me a lot.

best devops training in chennai | DevOps training in Chennai omr | DevOps training in Chennai with placement

Augurs Technologies Pvt Ltd. said...

Looking for a SaaS App Development Company? Augurs Technologies is SaaS Application Development Company in India and develop custom applications for your growing business.

radha said...

Well said. Keep updating Devops Online Course

jyothi kits said...

best online training
Linux Training
Mysql Training
Office 365 Online Training

Praylin S said...

Great post! I really enjoyed reading your post. Hoping for more posts from you.
Microsoft Dynamics CRM Training in Chennai | Microsoft Dynamics Training in Chennai | Microsoft Dynamics CRM Training | Microsoft Dynamics CRM Training Courses

Unknown said...

This is really an amazing blog, thanks for sharing such a valuable post with us!
DevOps Online Training

DevOps Online Training in Hyderabad said...

Given very useful points Thanks for sharing this post.

Ramsawan said...

Many people put effort to look for Independent Villas and here comes our exclusive guide on how to choose the Best Independent Luxury Villas in Chennai even you can find Budget Cheap Villas in our most areas across ECR and OMR.

DevOps Online Training in Hyderabad said...

Really very interesting and useful information for DevOps learners. Thank you for posting this valuable information.

Praylin S said...

Really informative post. Thanks for sharing.
WordPress Training in Chennai | WordPress Training | WordPress Course in Chennai | Training institutes in Chennai with Placement | Tally Course in Chennai | Ionic Course in Chennai

Anonymous said...

Thanks for sharing info on Lean IT, Devops and SSAS Cloud Programming"

Asperina Technosoft said...

Thanks for sharing informative article. Asperina Technosoft is leading web development company offering professional service to customers across the globe at affordable price. Follow us at Facebook.

mist hyderabad said...

Thank u for this information
http://www.mistltd.com

zara said...

Principles of Devops Training in Hyderabad are shared very well.

Anonymous said...

Thanks for sharing this information. This is the best article which is useful to all Online training Learners Azure DevOps online training in hyderabad

Rajani said...

Thanks for sharing this information. This is the best article which is useful to all Online training Learners Azure DevOps online training in hyderabad

Unknown said...

Nice, useful piece of information
About Aditya Construction Company - Established in the year 2002, Aditya construction company is a renowned construction company in Telangana and Andhra Pradesh.
3 bhk Luxury Flats in Hyderbad
Visit Our Youtube Page For Aditya construction company

Unknown said...

I found your articles very informative that add more value. and Visit Our Flats reviews For Aditya Constructions Reviews
Visit Our Youtube Page For Aditya constructions

sureshbabus said...

Pretty! This was a really wonderful post. Thank you for providing these details
Angular 2 Training in bangalore
Angular 4 Training in bangalore
Angular 5 Training in bangalore
Angular 6 Training in bangalore
Angular 7 Training in bangalore
Angular 2 Institute in bangalore
Angular 4 Courses in bangalore
Angularjs Classes
Angularjs Training in Bangalore ,
Angularjs Training Institute Bangalore ,
AngularJS Classes in Bangalore
Python Training in Bangalore

rajani said...

Nice Article <a href="https://www.visualpath.in/Microsoft-Azure-DevOps-online-Training.html”> Microsoft Azure DevOps Training</a>

Infocampus said...

Python Training in Bangalore
Python Training in Marathahalli
Best Python Training Institutes in Bangalore
Python Training Center in Bangalore BTM
python class in Bangalore marathahalli
python courses in Bangalore

mytrainingsoline said...


Nice blog..! I really loved reading through this article... Thanks for sharing such an amazing post with us and keep blogging...
Devops online training
Best Devops online training
Devops online training in Hyderabad
Devops online training in india

rajani said...

Nice Blog!!
Microsoft Azure DevOps Training

marketing course in hyderabad said...

While DevOps is most often associated with automation and continuous delivery/integration tools, I believe the single most important tool that organizations need to properly adopt and use to make a transformation to DevOps is a monitoring system.

DevOps Training in Hyderabad

rajanibachina said...

Nice Article!! Thanks for sharing....
Azure DevOps Online Training
Microsoft Azure DevOps Training Courses
Microsoft Azure DevOps online Training in Hyderabad
Microsoft Azure DevOps Training
Azure DevOps Online Training in Hyderabad

Vivek Tb said...

Aniq Intreior said...

Thank you for sharing the post

hawaiistore said...

metaverse

Viswadhika said...

Im thankful for the blog post.Really looking forward to read more. Will read on

Best Tableau Online Training
Selenium Certification Online Training from Hyderabad
SAP ABAP Training Course Online
Best Artificial Intelligence Online Certification Training India
AWS DevOps Online Training

Muskan said...

I like your blog. "The Art of Lean Software Development" is a good book for learning methods of programming. You can also get more information from Software Testing Training in Greater Noida which helps you open doors of oppurnitites.

Muskan said...

I like your blog. "The Art of Lean Software Development" is a good book for learning methods of programming. You can also get more information from Software Testing Training in Greater Noida which helps you open doors of oppurnitites.

Anonymous said...

nice post..
Devops Consulting Cloud Services
iNoesis

 
Technorati Profile