What software development should NOT learn from manufacturing

In software engineering there have always been two schools of thought. One school feels that there is a lot to learn from manufacturing. The other school thinks that they are entirely different.

There have been 3 distinct phases in this debate:

  1. CMM Phase: Manufacturing has transitioned from craftsmanship to mass production – productivity and quality has improved many-fold. Software development can also benefit from such transition. CMM movement was born from this thought.
  2. Agile Phase: Manufacturing deals with machine, software development deals with people. Processes involving machines can be controlled precisely. People are inherently different and are not interchangeable. People communicate better face to face rather than through written documentation. From this realization agile movement was born.
  3. Lean Phase: Toyota revolutionized manufacturing through lean manufacturing system and dramatically improved quality and optimized cost. The core of lean manufacturing is empowered teams. Since agile movement also is based on self-organizing teams it must be possible to transplant the learning from lean manufacturing to software development. This lead to lean software development.

There is an apparent logic in all three reasoning. So, which advice should you follow? Are they compatible with each other? Before answering these questions you should look at the differences between manufacturing and software development.

Manufacturing Software Development
Repeatable:

Same item produced many-many time

Unique:

Software is written only when nothing similar exists

Well-defined:

Even before start, the specification of the output is clearly defined

Incomplete & Evolving:

Not only are requirements sketchy at the beginning, but also likely to change during the development cycle

Known:

The process of converting input to output is clearly known and can be repeatedly done

Unknown:

There always are unknowns – in form of new technology, new interfacing requirement …

Machine & Tool:

Any process efficiency is dependant mostly of the machines and tools used and less on the people operating it

People:

Knowledge, experience and skill of people can make huge difference in productivity sometime as much as 1:100 between best and worst

Will this gap ever be bridged? Will software development move closer to manufacturing?

I doubt it – here is why.

Repeatability vs. Uniqueness:

“…Every advance for the future state of the world requires the presence of software yet to be written…” – Grady Booch (here)

You are never going to write software which already exists!

Well-defined vs. Incomplete & Evolving:

The authors of the Agile Manifesto understood it when they wrote:

“…Responding to change over following a plan…”

“…Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage…”

Software is about advancement … software is about new idea … not all new idea succeed. Trial and error cannot be avoided.

Known vs. Unknown:

Todd Anglin in his very interesting post The Future of Software Development states:

“…There is no end in sight to the deluge of new technologies being delivered to market…”

“…If there is any one concept or idea that software developers must absolutely comprehend, it is this: it really is okay not to know it all…”

The rate of technology change is accelerating. Technology options are increasing. Current software, for all practical purpose, has infinite features! The world is becoming more and more interconnected. Hence, software developers of the future will need to handle more unknown, not less.

Machine & Tool vs. People:

Daiv Russell, in the article Hold On to Your Top Performers, states:

“…Most CTO’s realize that The Pareto Principle applied to IT staff means that 20% of your people are delivering 80% of your entire team’s bottom-line value…”

“…The difference between the extremes has been reported as high as 100:1…”

Unless we can find a method of completely automating the process of software writing, the dependency on people will remain. The chances of automating the process of software development in foreseeable future look remote.

What can we conclude from this?

Reject all idea which is derived or borrowed for the purpose of managing predictable processes.

Accept all idea which helps you to deal with unknown and uncertainty.

Will Microsoft VSTS 2010 be as good as the demo

Only once in a while you attend a product launch and come back with the impression that – yes – that is one feature I always wanted. (This happens frequently with Apple product launch.)

It was the Microsoft Tech-Ed event in Bangalore, end of day one and Amit Chatterjee, General Manager for the Visual Studio Test Business at Microsoft was talking about Visual Studio 2010. He talked about 3 features:

  1. How you can reverse engineering and generate design documentation from code
  2. How developer can recreate the exact scenario of a bug reported by the tester
  3. How agile project management aids have been incorporated

It is the second feature which caught my attention. A tester while reporting a bug can record the UI sequence leading up to the bug as a video and attach it to the bug. The developer can play the video, and see exactly how the bug happened. Developer can also replay the recorded test script to simulate the bug, examine the call stack when an exception is thrown and drill down to the code. It is called IntelliTrace. In addition to this you have the Test Impact Analysis Window which keeps developers informed about tests that need to rerun with every check-in.

This feature may be too good to be true in practice. There may be challenges in actually using these features. The process of recording may bring the testing process to a crawl. It may also not work in all circumstances.

But, I will be hoping that this feature delivers on the promise. How will this impact Test Driven Development?

Agile & VSTS

The second thought which occurred to me is that “Agile has really gone main stream”. There is no doubt about it because not only has Microsoft included support for agile project management but also thought it to be important enough to be listed in the top 3 features of VSTS 2010. Though, Scrum templates existed from 2005 onwards – it was never shown as a key feature.

Some time back I had done an analysis of Agile Tools Recommended by Experts based on a LinkedIn discussion. VSTS came in 5th. I suspect that if the same analysis is done a year from now VSTS will move up the ranking.

Microsoft understands developers

Vikram and Deepesh came back after attending the 3 day event with a gleam in their eye and a “Wow” expression. I suppose Microsoft understands developer better than anybody else.

They are also becoming more open to other platforms. For quite some time (2005-06) TFS had support for Eclipse, Linux, Unix and Mac OS. Did you know that Azure, their cloud computing platform, is not restricted to .Net? As long as your application works on MS SQL you should be able to migrate it to Azure. The application can be written in anything – PHP, Java – it only needs to run on Windows.

Software development is not like manufacturing

In last few months, I have met a lot of people who feel that lack of predictability in software engineering is because it is not mature enough as manufacturing or civil engineering – unfortunately I completely disagree with this view. For a change it was nice to meet somebody who agrees with me on this subject.

At the end of the day, there was a “CXO Panel Discussion” which was moderated by Tapan Acharya. Apart from me, the other panelists were:

  • Amit Chatterjee – Microsoft
  • D. R. Baghirathi – MindTree
  • Mahesh Prabhu – ITC infotech
  • P Sundar Varadaraj – Dell.
  • Raghavan  Subramanian – Infosys
  • Shankar Kambhapatti – Polaris.

During the “CXO Panel Discussion”, when we were discussion Amit said that “focus of manufacturing is to repeatable produce something” and the “focus of software development is to produce something which different from what already exists – otherwise there will be no need to create the software”.

When Is Reuse Feasible?

This post is triggered by a question from a friend. He is the CEO of a mid sized IT company based in India. What distinguishes this company is the fact that most of its revenue comes from IP created by them. He asked “Do you have any suggestions on how to create reusable assets and promote reuse among different product lines?”

I mumbled something about how reuse is difficult and how some of our large customers are also struggling with the idea. I doubt if I could convince him about the futility of attempt to reuse.

So I have decided to argue my case in this post.

  1. Effort required for reuse
    • Opportunity to use code without making any modification is very rare. There is always some architectural variation between different product line / project.
    • Effort required to modify an existing piece of code is much higher. If the change required is more than 20% then it may be easier to rewrite the code.
    • To write a piece of code which can readily be reused requires significantly more effort – factor of 3 is a reasonable estimate.
    • Therefore, to derive benefit the code needs to get used at least 4 to 5 times – so large number of product lines with identical technology is basic necessity for reuse.
  2. Pace of technology change
    • Even with same basic technology, software version has to match for effective reuse.
    • We live in a multi-tier world which uses a combination of different technologies – each of these technologies will have a significant new release every one to two years – therefore identical technology environment across product line is almost impossible.
  3. Open source software
    • Availability of open source assets further restricts scope of reusable code – what can easily be reused will probably be available as open source.

Managers think that reuse does not happen because of “Not Created Here” syndrome.

However, the truth is that the way code is written reflects the thinking of the creator. To reuse any piece of code, I need to start thinking like the creator. If my requirement can be so well encapsulated that I do not need to understand the code, I can probably find something better in the web.

So, I think the only effective way of reusing code is to either include one of the creators or somebody who have already used that code as a part of the team.