从奥兰多2010敏捷大会的氛围中获得灵感，Jurgen Appelo汇总了一份书籍清单，列举出100本最热门的敏捷书籍，它们对软件开发社区有所帮助。

Jurgen使用的方法是利用Amazon和GoodReads上的评级，以及书籍首次出版的日期来衡量书籍的热门程度。他也利用了Amazon上“同时购买了此书”的功能，书籍的知名度由评分人数得出，书籍的质量如何则基于其平均评级，结合一些其他因素得出，由此产生了他的书籍清单。

Jurgen最新的清单上，最热门的100本书籍中跻身前10的书籍包括：

去年，InfoQ发布过一篇新闻，列举了敏捷社区推荐的书籍。

在那份清单中，Mike Cottmeyer推荐了以下书籍，并给出了推荐原因：

• 《解析极限编程—拥抱变化》——作者：Kent Beck。书中提到XP的相关实践和成功秘诀，让所有的敏捷项目管理人员和领导者都念念不忘。
• 《敏捷软件开发—使用Scrum过程》——作者：Ken Schwaber。该书在解释如何使用Scrum做项目管理方面做得非常出色，而且对于刚刚接触敏捷的人来说也非常有价值。
• 《敏捷估计与规划》——作者：Mike Cohn。如果你已理解了基本理念，希望将规划加入到敏捷中，请阅读本书。
• 《用户故事与敏捷方法》——作者：Mike Cohn。将需求撰写成对客户有价值的功能线索，要理解这一点挺难的，本书可以帮你做到更好。
• 《敏捷软件开发》——作者：Alistair Cockburn。进阶敏捷实践人员的必读书，将软件开发描述为合作游戏，类似于音乐演奏者在舞台上的即兴演奏。
• 《软件项目管理与敏捷方法》——作者：Michele Sliger与Stacia Broderick。将PMP背后的流程映射到敏捷。是试图管理敏捷项目的PMP的必读书。

不久前，Mark Levison说过，没有以下书籍，他就无法启动一个敏捷项目。它们是：

• 《敏捷软件开发（原书第2版）》，作者：Alistair Cockburn
• 《敏捷估计与规划》，作者：Mike Cohn
• 《Collaboration Explained》，作者：Jean Tabaka

Mark还提到，以下书籍很重要：

• 《Agile Retrospectives: Making Good Teams Great》，作者：Esther Derby与Diana Larsen
• 《Fearless Change: Patterns for Introducing New Ideas》，作者：Mary Lynn Manns、Linda Rising
• 《用户故事与敏捷方法》，作者：Mike Cohn

根据Forrester Research今年第二季度的一份研究报告，在超过1000名专业开发人员中，采用敏捷模式进行软件开发的已经有10.9%采用了Scrum模式，在所有的敏捷开发模式中名列首位，而在所有的软件项目管理模式中，敏捷模式更是被35%的开发人员所采用。当然，研究报告为我们呈现的仅仅是一个统计学的观点，到底你的开发团队应该采用什么样的开发模式，这还是要根据各自不同的开发环境，人员构成，公司架构以及文化背景来决定。

图1：Forrester 关于敏捷模式的调查报告

Visual Studio 2010 是微软在2010年4月发布的全新一代的集成开发环境，配合同时发布的Team Foundation Server 2010（TFS——团队服务器） ，为开发团队提供了全面的应用程序生命周期管理（ALM）工具和平台。在2010这个版本中，对于敏捷，或者说Scrum模式的支持是前所未有的。虽然微软的Visual Studio Team System从2005年开始发布的时候就提供了敏捷流程模板（也就是MSF Agile）模板，但是2008版之前的这个敏捷流程模板都是基于MSF（微软解决方案框架）的；这个框架是微软针对自己的研发团队的最佳实践进行抽取总结出来的，与广大敏捷开发社区里面所流行的很多敏捷方法并不是很契合，造成了开发团队在实施的时候有很多不适用的地方。因此，微软在开发2010版本的过程中，大量的听取了敏捷开发社区中的声音，在自己的MSF Agile 5.0的模板中进行很多针对敏捷，更确切的说是Scrum开发模式的改进，使得2010版本中所集成的MSF Agile 5.0的模板非常适合我们来进行Scrum模式的开发组织。当然，微软的产品为了追求通用性，在MSF Agile 5.0的模板中并没有完全采用Scrum模式通行的名称和流程；同时，微软在两周前又发布了一个纯粹的Scrum流程模板以供那些希望完全使用Scrum模式的开发团队使用，当然这个模板现在仍然是Beta版。

我个人认为，开发团队采用哪一个模板并不是最重要的，重要的是我们需要在开发过程中不断地改进过程，并对这个模板进行定制，以便适合我们自己的开发流程。这也是为什么TFS所提供的是一个模板，因为它的目的就是希望我们在这个模板的基础上不断的改进，最终找到适合

自己开发团队的流程。其实这也很符合Scrum模式的理念；简单一点来说，Scrum模式是一种针对复杂项目的流程组织方式的框架，其目标是为了让我们开发出更高质量的软件产品。围绕的这个目标，Scrum模式为我们提供一个团队模型，一系列工具和一个简单的流程。在这样一个框架之下，Scrum模式要求我们不断地改进流程以达到适合团队的最佳状态，这种对改进的要求也是Scrum模式区别于其他开发流程的重要特点之一。

为什么Scrum模式适合软件开发？

软件行业至今已经有超过40年的历史，很多在软件工程中的管理方法都是在不断摸索中改进而来的。早期的软件行业由于规模有限，绝大多数属于作坊型，几个人在一起靠着自己的聪明才智创造出软件产品；但是当团队规模不断扩大的时候，开发人员开始需要一种模型来组织越来越庞大的团队，满足越来越复杂的需求。因为没有经验可循，软件开发团队将很多传统工业工程的方法借鉴到软件行业，因而出现像“瀑布式”的模型。“瀑布式”模型要求我们在实际的开发工作开始之前进行很多非常细致的设计和计划，力图将不可控的开发过程细化成可以控制的颗粒，以达到对复杂项目的总体控制目的。但是“瀑布式”模型忽视了软件项目的一个本质特点，那就是需求的不确定性；我们不可能像造汽车一样在上生产线之前把所有的零件都设计好，所有的流程都规定好，再进行装配；因为任何软件在实际进行编码之前都没有人知道这些代码应该如何实现，而且每一个开发人员的水平不同，习惯不同，写出的代码也是不同的；再加上客户对于软件的需求也是在不断变化的，一年之前的业务流程很可能在一年之后就产生的变化，如果还按照之前的需求进行开发，那么交付的时候肯定是无法满足要求的；更重要的事，在客户没有看到或者实际操作软件产品之前，他们永远也不能明确地告诉你他们要的到底是什么。因为这种种原因，造成了软件开发不可能采用传统的工程方法进行组织，因为其本身是一种需要依赖于开发人员智慧的探索性行为，也造成了我们的软件项目中有很大一部分是失败的。

Scrum模式的出现正是基于对于软件开发行为本质的认识，提供了一种松散的框架，让我们使用一种探索性的流程方法来组织本来就是探索性的开发过程；从根本上满足了软件开发本身对于流程的需求。这种方法论实际上是基于爱德华?戴明所提出的戴明环的管理方法；戴明环理论提出：人类在进行任何复杂活动时，获得成功的最有效过程要经过：Plan 计划– Do执行 – Check 检查– Act改进，四个子过程，并不停的迭代以便找到最佳的方法来解决问题。这个理论不是针对软件开发提出的，但是软件开发本身其实就是最典型的复杂活动。

图2：Scrum的流程

这里我们再回头看看Scrum的流程，Scrum的流程主要包含以下内容：

• (P) Release/Sprint Planning：发布/迭代计划
• (C&P) Daily Scrum：每日回顾
• (C&A) Sprint Review：迭代产品检查
• (A) Sprint Retrospective ：迭代流程检查

我们可以看到，Scrum模式的流程与戴明环仅仅相扣。有很多认为敏捷模式会弱化计划的作用，其实不然，敏捷模式更加强调计划，而且强调更加频繁的计划，比如：每日回顾这个流程就要求我们的团队每个成员每天早上用15分钟的时间来回答3个问题：

1. 你昨天做了什么？
2. 你今天计划做什么？
3. 有什么问题阻碍你的开发进程？

其实这正是对于之前开发内容的检查，同时也是对后续开发内容的计划过程。

Scrum模式需要怎样的工具来实现？

对于使用什么样的工具来实现Scrum模式，现在也有很多不同的观点。其实有很多人认为白板和即时贴就是最好的工具，其实对于小型团队来说这的确是最有效而且最经济的方法。但是如果考虑到软件公司的管理需求（工作量统计等），远程团队，开发工具集成，代码质量控制，发布后期支持等等；我们还是需要一个高度集成的平台和一整套工具来支持我们的开发团队。

图3：白板和即时贴

Visual Studio 2010所提供的集成开发环境可以满足我们以上的一系列需求，帮助我们的开发团队更好组织开发，帮助我们的管理层更好地掌控开发过程，帮助软件公司开发出更高质量的产品。

Scrum模式对于工具的要求，主要集中在以下一个方面：

1. 团队组织：满足PO （产品经理），Scrum Master （流程经理）和开发团队管理，以不同的权限访问团队项目并对不同角色提供个性化的信息支持的能力。
2. 产品需求记录和跟踪：对于Product Backlog Item （PBI 产品需求列表）的添加，编辑，优先级排序以及交付开发团队以后进行跟踪的能力。
3. 流程管理：满足Sprint Planning, Daily Scrum, Sprint Review和Sprint Retrospective这些流程中对于信息共享，信息转移和跟踪的能力。
4. 产品质量：在整个开发过程中，配合Scrum模式达到产出高质量代码和产品的能力。

下面我们就看看Visual Studio 2010系统在这4个方面如何满足Scrum模式的需求，并协助我们开发出高质量的产品。

Visual Studio 2010上的Scrum团队组织

一个完整的Scrum开发团队主要由以下角色组成：

1. Product Owner （PO 产品经理）：我喜欢把PO翻译为产品经理，因为PO的工作职责就是向客户和干系人收集产品需求，进行排序并保证开发团队按照干系人对需求优先级的要求进行交付。
2. Scrum Master （SM 流程经理）：对于Scrum Master我一直没有更好的翻译，将其译成为流程经理是因为这一角色要保证团队按照Scrum的方式来组织开发，并协助团队和PO进行有效的沟通，解决团队所遇到的问题。Scrum Master和项目经理的区别在于，他更加倾向于保证开发流程的完整性而不是倾向于满足客户/干系人的需求。
3. 开发团队：开发团队在Scrum模式中是作为一个整体出现的，一般来说团队的大小控制在3-7个人的规模；团队作为一个整体向PO负责，而不是每个人对于自己的任务负责。

在Visual Studio 2010 系统中，使用TFS服务器基于角色的权限控制，我们可以很方便地定义出不同的权限范围。当然，最简单的方法是把Scrum团队的角色和TFS的默认角色之间进行映射。

图4：TFS团队项目的默认角色

Scrum团队角色	TFS团队角色
Product Owner	Contributor
Scrum Master	Project Administrator
开发团队	Contributor Builders Project Administrator	根据团队不同人员的职责具体分配
项目干系人	Readers	如果客户愿意更直接的参与项目，可以允许他们直接访问TFS。

表1：Scrum团队和TFS团队角色映射

Visual Studio 2010系统中对需求记录和跟踪的支持

Scrum模式中的需求主要是采用Product Backlog Item（PBI产品需求列表）和Sprint Backlog Item （SBI 迭代需求列表）来进行管理的，在Visual Studio 2010系统中，直接提供了针对这两个列表的工作项查询，并且还提供了Agile Workbook （敏捷工作簿）帮助我们更好对工作量和任务分配进行调控。

图5：使用MSF Agile 5.0模板创建的TFS团队项目集成了对PBI和SBI的管理功能

图6：Product Backlog 查询结果

上图中就是使用TFS内置的Product Backlog查询获取的产品需求列表，这个列表是PO使用的主要工具，我们可以注意到这个列表已经根据Stack Rank列进行了排序，这也反映了产品需求列表的特性：需要根据客户/干系人对需求项的优先级向团队交付任务；而PO的除了需要不断完善这个列表，还需要不断和客户干系人进行沟通，一边确定这个优先级。

在Scrum模式中，对于优先级的定义决定于两个因素：需求的商业价值和紧迫程度；另外一个重要的指标就是Story Point，这个指标标志着当前需求项的相对大小，注意这里说的相对大小，很多人将这个值理解为人天或者人时，其实是不准确的，因为在PO准备产品需求列表的过程中，仅凭PO的经验是很难准确的判断出以时间为度量的工作量的，但是相对的大小是比较容易判断的。

另外，从State和Iteration Path两个列的值我们可以看到，已经有一些需求在迭代1-2中已经解决。根据这些信息，PO可以很容易的对工作进度和剩余需求进行管理。

另外一个重要的查询就是Iteration Backlog查询：

图7：Iteration Backlog查询结果

Iteration Backlog 中包含了团队在某个迭代中需要完成的需求以及针对这些需求细化出来的具体开发/架构/测试等任务。在Visual Studio 2010中，微软终于开始支持树形结构的工作项关联，从上图可以看出，每一个User Story的下面都挂接着相应Tasks，这些任务是在Sprint Planning Meeting中由团队成员自己根据PO对需求的阐述进行的细化，同时团队成员还需要根据经验对这些Tasks进行估算，给出基线估值（Original Estimate）。在开发过程中，团队成员在每天的Daily Scrum之前需要对前一天的任务更新状态（State），已完成工作量（Completed Work）和剩余工作量（Remaining Work）字段的内容；通过这些信息我们就可以使用TFS自带的燃尽图报表对进度进行查询和预测了。

实际上，纯粹的Scrum模式并不关心已完成工作量（Completed Work）也就是以完成工作量的值，但是对于使用人天/人时等信息来衡量团队工作量，甚至依赖这些数据想客户收取开发费用的咨询类公司来说，这些信息是非常重要的。

Visual Studio 2010对Scrum流程中重要事件的支持

Scrum模式中的几个重要的会议包括：

1. Sprint Planning Meeting
2. Daily Scrum Meeting
3. Sprint Review Meeting
4. Sprint Retrospective Meeting

这一系列的会议是真正体现Scrum模式对于开发流程控制的核心内容，在Scrum模式中另外一个非常重要的概念是：时间箱（Time Box），它要求我们对于流程中的事件进行非常严格的时间控制。很多人在开始进行Scrum模式开发的时候的一个普遍问题是：一个迭代（Sprint）的长度应该是多少？对于这个问题其实也没有标准答案，而必须根据团队的大小来进行判断。对于之前我所建议的3-7人大小的团队，我会建议采用2周的迭代长度。原因在于1周太短，团队还无法完成真正有商业价值并可以进行交付的需求；而3周的时间则太长，需求的变化所造成的风险会变得比较大。

采用迭代式开发的时候其实长度是越短越好，我们总是尽可能的缩短迭代以便可以通过给客户的交付获得更有价值的反馈以便对后续的开发进行调整，因此这个长度应该是团队刚刚可以完成可交付需求的最短时间。我们需要严格控制的是，迭代的长度应该是一个时间概念儿不是工作量的概念，也就是说如果2周的时间已经耗尽但是团队还没有完成当前迭代中的所有需求，那么也必须结束迭代进行交付，而不能选择延长迭代来完成未尽需求。这样做的结果有两个：1）当前的迭代会以失败告终；2）通过对已经完成需求的交付，我们可以获取客户的反馈。很明显，失败的迭代是我们不愿意看到的，但是客户对于已经完成需求的反馈比保持常胜将军的名誉更加重要，因为后者是保证我们软件质量（符合需求）的重要手段。

当然，这里隐藏着另外一个很重要的问题，在团队无法完全完成需求的情况下如何还能提供可交付的成果，这就要依靠我们对于需求定义方式的变化和Visual Studio 2010 中对持续集成和更加高效的测试支持来实现了。在需求定义上，我们需要采用业务导向的需求定义，保证每一个需求的完成都可以交付一定的商业价值。以往的需求往往是功能导向的，但是功能导向的需求对于用户来说不一定具备商业价值，但是业务导向的需求则可以保证这一点，比如：我们可以这样定义一个User Story，作为市场经理，我希望对客户数据进行查询以便可以找到本市的客户并和他们进行联系。使用这样的需求定义意味着只要我们完成这一需求对客户就是有价值的，因为它不是一个功能碎片，而是一个用户交互用例。如果在一个迭代中我们无法完成所有的需求，只要完成其中一个，那么都是可以向客户交付的。另外，借助Visual Studio 2010对持续集成和测试的支持，我们可以采用每日构建的方式保证所有完成的代码都符合质量要求，也就避免了在迭代后期进行集中测试而拖延交付的可能性。

Sprint Planning Meeting的支持

在Visual Studio 2010中提供了一个叫Agile Workbook的Excel模板，可以帮助我们很好地完成Sprint Planning Meeting。在这个会议中，最重要的任务就是将PBI转化成SBI，并且由团队给出完成这些SBI的承诺；团队要做出这样的承诺最重要的依据就是这些需求所涉及的工作量是否可以承受。Agile Workbook正是帮助我们回答这一问题的强大工具。从下图我们可以看到，当我们制定了迭代上的人员配备并将Task分配给每个开发人员以后，模板会给出非常直观的柱状图，帮助团队判断工作量是否可行。

图8：对迭代1-3上的工作量进行横向比较，根据历史数据判断后续迭代是否可行

图9：在当前迭代上对每个开发人员的工作量分配进行比较

Daily Scrum Meeting支持

这个会议非常简短，所以我们更加需要非常直观的图表以帮助团队对进度进行审核，在TFS中提供了燃尽图为团队提供这些信息。

图10：迭代燃尽图

根据每个开发人员对于工作量的更新，从上图我们可以很容易对完成时间进行预测，图中黑色实线和横轴的焦点就是当前迭代的可能完成时间。

Sprint Review & Retrospective Meeting 的支持

Sprint Review的支持更多地体现于Visual Studio 2010的持续集成能力，因为这个会议是对于需求完成情况的审核，如果我们能够保证需求是业务导向的并充分利用Visual Studio 2010的自动化构建和测试集成能力。那么我们就可以保证在这个会议上交付一定的商业价值。具体如何使用Visual Studio 2010来实现在后面做详细介绍。

Retrospective 会议其实非常简单，需要我们团队成员对当前迭代的运作进行总结，但为了使这些信息可以完整的保存以便后续使用，我们可以利用TFS提供的门户站点，定制一个SharePoint的列表分类的记录这些反馈以便团队查询。

Visual Studio 2010对于产品质量的保证

提高产品质量是Visual Studio 2010在设计阶段就确定的重要目标，在2010版本所添加的新特性中，已经想着这个目标形成了一套完整的解决方案。对于Scrum模式来说，交付高质量的产品也同样是其终极目标，而且我们需要在迭代时间很短的情况下仍然保证质量，这就更加需要依赖工具的支持。

自动化构建

之所以把自动化构建列在首位，是因为软件工程发展到今天，自动化构建已经是任何一个想要实现高质量的软件开发团队都必须采用的工程方法；另外，对于Visual Studio 2010系统来说，自动化构建也起着承上启下，贯穿全局的重要地位。当开发软件进入第一个迭代的开发时，所要进行的第一项工作并不是开始实际的编码，而是创建出符合团队需求的构建模板。这样做的目的在于团队在后期的实际开发中可以更加专注于需求的开发，而不必花费额外的时间和精力来集成开发人员的代码；开始阶段的代码量很少，团队可以有更加清晰的思路将迁入策略，架构验证，自动化测试列表设置好并保证构建可以正常运行；如果把这个工作放到迭代后期进行，往往会因为代码中的缺陷和不同开发习惯造成构建模板不能正常运行。

在Visual Studio 2010中，提供了更加便捷的模板创建工具，特别是添加了Gated Check-in 构建的触发方式，可以保证所有嵌入源代码库的代码都是经过验证的。

图11：Gated Check-in 构建触发器

Gated Check-in 触发方式和以往的触发方式所不同之处在于，开发人员执行迁入操作的时候代码并不会直接进入源代码库，而必须先经过构建的验证：保证编译成功和定义好的迁入验证测试可以成功运行，然后TFS才会把代码真正嵌入服务器。之前的持续集成（Continuous Integration） 方式也会在迁入的时候进行构建，但是这种构建是将代码先迁入，然后再运行构建，如果代码中已经存在了缺陷，那么在服务器上就会留下缺陷代码；Gated Check-in 借助TFS源代码管理中的“搁置”功能，先把代码搁置到服务器上临时存储中，在构建成功后才会正式迁入，所以缺陷代码不会进入服务器。

图12：构建参数配置

TFS的自动化构建可以集成测试列表，图中的上方的红色区域中就是要求构建从项目文件中的测试列表文件中提取单元测试并自动运行；另外一个在Visual Studio 2010种的重要改进就是下方红色区域中的架构验证参数。如果我们的项目文件中包含了架构层次图（Layer Diagram）的话，那么我们就是添加这个参数让构建自动的验证项目的代码是否符合架构设计的要求。

图13：Visual Studio 2010的层次架构图 Layer Diagram

Scrum模式开发中的架构设计给我们提出了非常大的挑战，由于我们采取业务导向的需求定义，开发人员必须从数据层一直实现到表现层；在这个过程中如何保证项目的架构仍然符合需求非常困难；而Visual Studio 2010的架构验证功能则可以帮助我们在每次迁入代码的时候都进行验证，保证违反架构规范的代码不会进入最终的交付产品。

消除无法重现的Bug

无法重现的Bug一直都是困扰开发人员的问题，开发环境，测试环境，生产环境的不同；开发人员，测试人员和最终用户的不同都是造成Bug无法被重现的客观因素。在Visual Studio 2010中，提供了很多强大的调试和测试工具来帮助我们解决这个问题。

• IntelliTrace（历史数据调试）
• 协作调试
• 测试管理器和手工测试（Test Manager）
• 实验室管理（Lab Manager）

IntelliTrace——历史数据调试器

IntelliTrace在开发过程中的名称就叫Historical Debugger （历史数据调试器），后来这个用来进行市场宣传的名称反而不能反映它的实质。IntelliTrace可以把程序运行过程中的所有历史数据都记录下来，使得程序员可以回滚到任何的历史点来查看程序状态，这对于开发人员调试复杂逻辑非常有用；之前我们在做同样工作的时候必须反复运行程序，以便找到问题，而现在则可以让程序反向运行。

图14：IntelliTrace调试器重所记录的程序历史数据

另外，IntelliTrace还可以把这些调试数据另存为tdlog文件；当开发人员A发现了B的一个问题的时候，他可以把自己调试环境中的tdlog发送给B，开发人员B就可以使用这个文件让Visual Studio恢复到开发人员A的调试状态，从而保证B可以有效的重现A所看到的问题。

协作调试

协作调试实际上解决多个开发人员在调试过程中的另外一些信息共享问题的方法，上面的IntelliTrace可以共享调试历史数据；但是用过Visual Studio 的开发人员都知道，像“断点”是不能保存到调试数据中，也不会被保存到项目文件中；所以协作调试就提供了开发人员共享断点信息，并且还可以让开发人员在断点信息上添加一些说明，以便帮助其他的开发人员理解问题。

测试管理器和手工测试（Test Manager）

测试管理器是Visual Studio 2010系统中为测试人员特意开发的可以独立运行的测试环境，它完全独立，不依赖于Visual Studio IDE，提供非常强大的测试录制等功能。在前面介绍构建的时候我曾经将单元测试集成到构建中去自动运行，但是单元测试只能针对后台逻辑进行，不能解决UI测试，或者叫黑盒测试问题。微软的测试管理器的出现，就是为解决UI测试的问题。

TFS 2010中专门提供测试用例（Test Case）工作项类型，这个工作项允许测试人员对具体的测试步骤进行设计，并且给出预测的结果；同时，借助测试管理器的录制功能，还可以把测试人员换的操作全部都录制下来，一边后来自动播放；或者生成Coded UI 测试，一旦有了Coded UI测试，我们就可以把这些针对UI的测试也集成到自动化构建中去。

图15：测试用例（Test Case）工作项

实际上，真正可以使用单元测试覆盖的测试仅占所有的测试的30%都不到，另外这70%的测试以往都是依赖于测试人员手工的进行；现在借助微软测试管理器的功能，我们可以将这些测试集成到高度自动化的开发流程中。可以帮助我们更加快捷的完成测试，为开发人员提供反馈。

在Scrum模式中，业务导向的需求也要求我们的测试团队可以更加快捷的给出测试结果，前一天完成的需求最好可以在第二天就将测试结果反馈给团队；依赖于每日构建，我们可以在每天晚上将前一天的代码生成一个新版本，共测试团队使用；测试团队在第二天就可以把测试结果反馈给开发团队，同时将可以自动化运行的测试继承到每日构建中；在第三天的时候我们的团队就可以利用这些已经自动化的测试来验证我们的程序了。

由于每天都进行测试，那么新增的代码量就非常有限，也就使得Bug的数量可以得到有效的控制，从这个方面上说，测试管理器所提供的手工测试，自动化测试录制和回放，并且和构建的继承为我们提供了一个非常高效的高质量的开发平台，从流程和工程技术上为质量提供了保证。

实验室管理（Lab Manager）

实验室管理是我在Visual Studio 2010系统中见过的最酷的功能，也是微软继承了自己的多项产品为开发团队提供的最完整的测试解决方案。在测试中一个非常难实现的问题，就是对于不同环境的创建，还原和状态的保存。如果同一个用例在不同的环境中运行，结果往往是不同的，而且我们客户的使用环境也往往很复杂，所以就要求我们的测试人员可以搭建很多不同配置的测试环境，以便验证应用程序可以适应他们要求。

微软借助自己的Hyper-V虚拟化平台，为测试团队搭建这样的测试环境提供了非常好的支持，比如：我们可以使用SCVMM和TFS协同工作，当TFS需要测试环境的时候，通过SCVMM部署一台符合要求的虚拟机，并把需要测试应用自动的部署到这个虚拟机中，最终在这个环境中运行指定的测试。这样的测试环境避免了测试人员自己的机器不干净而导致的结果偏差，而且还可以通过环境快照的方式吧虚拟机的某个状态直接交付给开发人员进行检查。

在上面所介绍的这些功能中我们可以看到，实际上我们解决了3个不同测试的不可重现问题：

1. 开发人员本机上的不可重现：IntelliTrace
2. 开发人员和开发人员之间的不可重现：IntelliTrace, tdlog和协作调试
3. 开发和测试环境之间的不可重现：微软测试和实验室管理器，Hyper-V

这些功能在工程技术上为团队保证了高质量，同时配合Scrum模式所推行的时间箱管理，业务导向的需求定义以及流程上的保证，Visual Studio 2010系统和Scrum一起帮助我们创建更好的产品和更好的团队。

结束

我使用Visual Studio Team System是从2005年开始的，最初的目的只是为了满足远程迁入代码的需要；但随着2008和2010版本的发布，对于流程定制和整体性的质量解决方案的需求越高。幸运的是，这个时候公司为我提供了到澳大利亚接受Scrum Master培训的机会，使我可以体系化的了解了Scrum模式的精髓，回来之后就对我们的开发团队进行了一系列的优化。

同时，作为Scrum Master我也同时获得了提供Professional Scrum Developer培训的机会，PSD课程是微软和scrum.org共同开发的一套基于实践的scrum开发人员培训课程，它使用Visual Studio 2010系统作为平台，将参训人员分为不同的团队，进行实际的开发工作，在开发的过程中让学员体会Scrum的妙处和Visual studio 2010的强大。目前我们已经在澳大利亚墨尔本和意大利米兰成功运作了这个课程。作为在亚洲去唯一向中国提供这一课程的提供商，我也希望能够和更多的开发人员分享这些内容。

1. Do some initial architecture envisioning.  At the beginning of a project, during Iteration 0 (see Figure 1), you want to do some initial architecture modeling to identify a potential vision for how your team will build the data warehouse.  As Agile Model Driven Development (AMDD) suggests, you do not need to create a comprehensive, detailed model up front, you only need a high-level vision at the beginning of the project and the details can be identified on a just-in-time (JIT) basis via model storming.  Sometimes a simple whiteboard sketch is all you need to understand your architectural vision.  If so, then just do that.  For a BI/DW project, the initial architecture views would likely be some form of deployment diagram capturing the technologies you intend to use and a high-level domain model overviewing the business entities and the relationships between them.

Figure 1. The Agile SDLC.

1. Model the details just in time (JIT).  The best time to model details isn’t at the beginning of a project but instead to model storm them throughout the project in a JIT manner.  There are several reasons for this.  First, like it or not, the requirements are going to change throughout the project.  Second, by waiting to analyze the details JIT, you have much more domain knowledge than if you had done so at the beginning of a project.  For example, if a requirement is to be implemented three months into a project, if you explore the details of that requirement at that point you have three months more domain knowledge than if you had done so at the beginning of the project, therefore you can ask more intelligent questions.  Third, if you’ve been delivering working software on a regular basis (see below) your stakeholders now have three months worth of experience with the system that you’ve built.  In other words, they can give you better answers.  Fourth, modeling everything up front appears to result in significant wastage.
2. Prove the architecture early.  Everything works in PowerPoint slides, on a whiteboard, or in CASE tool models but it isn’t until you prove it with code that you know that your architecture actually works.  Processes such as Rational Unified Process (RUP), Agile Unified Process (AUP), and OpenUP suggest that you build a working, end-to-end “skeleton” of your system to prove that all aspects of it work.  In the case of a DW, this would entail that you show that you can access the major legacy data sources, that your extract-transform-load (ETL) strategy works, that your database regression testing strategy works, and that your reporting tools can access your DW.
3. Focus on usage.  If you want to develop a system effectively, including a DW/BI system, then you need to understand how people will potentially use it to support their business objectives. This means that we need a usage-centered approach to development driven by use cases or usage scenarios, not a data-centered one driven by data models. Data is clearly an important part of the overall picture, but it’s only one of many parts. If we focus on data and not usage we run the risk of building something that nobody is interested in using, an all-too-common occurrence on traditional data warehouse efforts.
4. Don’t get hung up on “the one truth”.  The “one truth” philosophy says that it is desirable to have a single definition for a data element or business term, that there should be a common, shared definition for your master reference data and perhaps even your major business entities. To get to this “one truth”, when it is possible, often requires significant effort which often goes past the point of diminishing returns.  “One truth” is a nice vision to work toward, but don’t let it prevent your team from delivering important business value in a timely manner. The fact is that various portions of your organization have different ways of working, different priorities, and different constraints.  Seeking the one truth for a data element often proves to be an artificial constraint imposed by traditional data professionals, not by the actual business.  You can in fact take an Agile approach to Master Data Management (MDM).
5. Organize your work by requirements.  On agile projects we perform work based on prioritized requirements, not by technical issues such as source systems. Each iteration we do the work to fulfill the highest priority stakeholder requirements which fit into that iteration. During each iteration we get a little more data from system X, and some more from system Y, and some more from system Z, and so on.  If our iterations are two weeks in length, we pull two weeks worth of work from the top of the priority stack. By working in this manner we are always in a position where we are achieving the maximum benefit for our stakeholder’s IT investment, thereby reducing their risk.
6. Active stakeholder participation.   Stakeholder involvement is critical throughout your project, and better yet active stakeholder participation where stakeholders are not only involved with your project on a daily basis, they are also directly involved with the actual modeling effort itself.

DW/BI Lifecycle Best Practices

1. Take an evolutionary approach.  Requirements, or at least your understanding of them, will change throughout the lifecycle of your project for a variety of reasons.  The implication is that if you want to develop a solution which meets the needs of your stakeholders then you will need to take an evolutionary (iterative and incremental) approach to development. Envision the requirements and architecture at a high-level to start, but model storm the details just in time (JIT).
2. Embrace change.  A changed requirement late in the lifecycle is a competitive advantage as long as you can act on it.  Instead of adopting a strict change management process which for the most part is based on change prevention, you can instead adopt a more agile approach to change management where your stakeholders can easily change their minds as the progress progresses.  This of course requires the development team to adopt evolutionary database development techniques such as database refactoring, evolutionary data modeling, and database regression testing.
3. Deliver working software regularly.  Following short iterations, perhaps a few weeks in length, and providing working software at the end of each iteration, often results in stakeholders who are far more interested in getting more software than they are in getting more specifications. Effective DW/BI teams focus on high-value activities such as actually providing access to data and developing reports instead of merely documenting what you intend to deliver at some point in the future.  The only accurate measure of progress on a software development project is the delivery of working software, not the delivery of documentation or other non-executable work products which are nothing more than promised to deliver software at some point in the future.  A working system provides the concrete feedback to stakeholders that progress is being made, and regular updates to that software (at least internally) helps to reduce the feedback cycle and thereby decrease overall risk on your project.
4. Strive for iterations of one to two weeks.  Iterations of this length provide more opportunity to govern the project effectively due to the greater feedback provided by regular delivery of working software.  Short iterations motivate you to focus on high value activities, long iterations allow room for bureaucratic practices (and the inherent waste which goes along with them).
5. Test throughout the lifecycle.  One of the great blindspots in the data community is testing, which very likely explains the data quality challenges which we continue to suffer from in production databases.  It is common on agile projects to do significantly more testing than what typically occurs on traditional projects, and as you can see in Figure 2 agilists tend to test throughout the project lifecycle (in fact, many agilists prefer to take a test-first approach to development).  There is nothing special about databases — just like you should test your application code you should also test your database code.  If data is a corporate asset, a common refrain in the vast majority of organizations, should you not have a test suite in place which validates your data?  In short,  database regression testing is critical to your success.
6. Involve operations and support people early.  Operations and support staff a key stakeholders on any project, including DW/BI projects.  The sooner you involve them the sooner you find out what their requirements are (including pesky details such as deployment windows, archiving needs, hardware lead times, …).

Figure 2. Testing during development iterations.

Other Good Ideas

1. Adopt common development standards.  Just like when you’re building any other system, you should follow your organization’s common development guidance.  This includes modeling style guidelines, coding guidelines, data naming conventions, report design guidelines, and so on.  Consistency is an important contributor to quality, and guidance which is willingly followed by developers seems to be more effective in practice than guidance which is mandated and enforced by your governing bodies.  If such guidance doesn’t exist within your organization then you’ll want to develop some, or better yet, adopt existing ones from industry. It’s amazing what you can find on the Internet these days just by looking for it.
2. Use good tools.  DW/BI development is complex, and good tools will help to address that complexity. In addition to the standard tools for data modeling, extract-transform-load (ETL), and reporting you’ll also need tools which support evolutionary development techniques such as database refactoring, database testing, and database deployment.  The agile/evolutionary tools are currently emerging in the marketplace.
3. Don’t underestimate legacy data challenges.  Existing data sources are often a mess, as I discuss in The Joy of Legacy Data.  Ideally you’ll refactor the source to fix any data quality problems, but if that’s not an option then you’ll need to cleanse the source data as much as possible as you extract it from the legacy sources.  In my experience data cleansing should be seen as a process smell which indicates the need for legacy data source owners to become better at database evolution.
4. Travel light.  Serial approaches to development are typically documentation heavy, often in a naive attempt to counteract the inherent communication risks of waterfall lifecycles.  With the serial approach you often see teams create comprehensive logical and physical data models and detailed report specifications.  With an Agile approach where you develop working software each iteration, you quickly discover two things.  First, an evolutionary approach to development demands an evolutionary approach to data modeling and therefore you don’t need to create detailed data models up front.  Second, by having developers work closely with stakeholders you don’t need to create detailed report specifications, instead it is more effective to simply create a report and get feedback from your stakeholders which you then act on iteratively.
5. Adopt a lean approach to data governance.  Traditional, command-and-control approaches to data governance appear to work very poorly in practice.  The 2006 DDJ survey into the current state of data management practices showed that 66% of development teams will choose to “work around” their organization’s data group, and when they do so that 75% of the time it is because they find the data group too difficult to work with, too slow to respond, or that the data group doesn’t provide sufficient value to justify the effort of working with them. This is clearly problematic.  It is possible to take a lean/agile approach to data governance.

转载自：http://www.agiledata.org/essays/dataWarehousingBestPractices.html

Probably the most noticeable change to software process thinking in the last few years has been the appearance of the word ‘agile’. We talk of agile software methods, of how to introduce agility into a development team, or of how to resist the impending storm of agilists determined to change well-established practices.

This new movement grew out of the efforts of various people who dealt with software process in the 1990s, found them wanting, and looked for a new approach to software process. Most of the ideas were not new, indeed many people believed that much successful software had been built that way for a long time. There was, however, a view that these ideas had been stifled and not been treated seriously enough, particularly by people interested in software process.

This essay was originally part of this movement. I originally published it in July 2000. I wrote it, like most of my essays, as part of trying to understand the topic. At that time I’d used Extreme Programming for several years after I was lucky enough to work with Kent Beck, Ron Jeffries, Don Wells, and above all the rest of the Chrysler C3 team in 1996. I had since had conversations and read books from other people who had similar ideas about software process, but had not necessarily wanted to take the same path as Extreme Programming. So in the essay I wanted to explore what were the similarities and differences between these methodologies.

My conclusion then, which I still believe now, is that there were some fundamental principles that united these methodologies, and these principles were a notable contrast from the assumptions of the established methodologies.

This essay has continued to be one of the most popular essays on my website, which means I feel somewhat bidden to keep it up to date. In its original form the essay both explored these differences in principles and provided a survey of agile methods as I then understood them. Too much has happened with agile methods since for me to keep up with the survey part, although I do provide some links to continue your explorations. The differences in principles still remain, and this discussion I’ve kept.

From Nothing, to Monumental, to Agile

Most software development is a chaotic activity, often characterized by the phrase “code and fix”. The software is written without much of an underlying plan, and the design of the system is cobbled together from many short term decisions. This actually works pretty well as the system is small, but as the system grows it becomes increasingly difficult to add new features to the system. Furthermore bugs become increasingly prevalent and increasingly difficult to fix. A typical sign of such a system is a long test phase after the system is “feature complete”. Such a long test phase plays havoc with schedules as testing and debugging is impossible to schedule.

The original movement to try to change this introduced the notion of methodology. These methodologies impose a disciplined process upon software development with the aim of making software development more predictable and more efficient. They do this by developing a detailed process with a strong emphasis on planning inspired by other engineering disciplines – which is why I like to refer to them as engineering methodologies (another widely used term for them is plan-driven methodologies).

Engineering methodologies have been around for a long time. They’ve not been noticeable for being terribly successful. They are even less noted for being popular. The most frequent criticism of these methodologies is that they are bureaucratic. There’s so much stuff to do to follow the methodology that the whole pace of development slows down.

Agile methodologies developed as a reaction to these methodologies. For many people the appeal of these agile methodologies is their reaction to the bureaucracy of the engineering methodologies. These new methods attempt a useful compromise between no process and too much process, providing just enough process to gain a reasonable payoff.

The result of all of this is that agile methods have some significant changes in emphasis from engineering methods. The most immediate difference is that they are less document-oriented, usually emphasizing a smaller amount of documentation for a given task. In many ways they are rather code-oriented: following a route that says that the key part of documentation is source code.

However I don’t think this is the key point about agile methods. Lack of documentation is a symptom of two much deeper differences:

• Agile methods are adaptive rather than predictive. Engineering methods tend to try to plan out a large part of the software process in great detail for a long span of time, this works well until things change. So their nature is to resist change. The agile methods, however, welcome change. They try to be processes that adapt and thrive on change, even to the point of changing themselves.
• Agile methods are people-oriented rather than process-oriented. The goal of engineering methods is to define a process that will work well whoever happens to be using it. Agile methods assert that no process will ever make up the skill of the development team, so the role of a process is to support the development team in their work.

In the following sections I’ll explore these differences in more detail, so that you can understand what an adaptive and people-centered process is like, its benefits and drawbacks, and whether it’s something you should use: either as a developer or customer of software.

Predictive versus Adaptive

Separation of Design and Construction

The usual inspiration for methodologies is engineering disciplines such as civil or mechanical engineering. Such disciplines put a lot of emphasis on planning before you build. Such engineers will work on a series of drawings that precisely indicate what needs to be built and how these things need to be put together. Many design decisions, such as how to deal with the load on a bridge, are made as the drawings are produced. The drawings are then handed over to a different group, often a different company, to be built. It’s assumed that the construction process will follow the drawings. In practice the constructors will run into some problems, but these are usually small.

Since the drawings specify the pieces and how they need to be put together, they act as the foundation for a detailed construction plan. Such a plan can figure out the tasks that need to be done and what dependencies exist between these tasks. This allows for a reasonably predictable schedule and budget for construction. It also says in detail how the people doing the construction work should do their work. This allows the construction to be less skilled intellectually, although they are often very skilled manually.

So what we see here are two fundamentally different activities. Design which is difficult to predict and requires expensive and creative people, and construction which is easier to predict. Once we have the design, we can plan the construction. Once we have the plan for the construction, we can then deal with construction in a much more predictable way. In civil engineering construction is much bigger in both cost and time than design and planning.

So the approach for software engineering methodologies looks like this: we want a predictable schedule that can use people with lower skills. To do this we must separate design from construction. Therefore we need to figure out how to do the design for software so that the construction can be straightforward once the planning is done.

So what form does this plan take? For many, this is the role of design notations such as the UML. If we can make all the significant decisions using the UML, we can build a construction plan and then hand these designs off to coders as a construction activity.

But here lies the crucial question. Can you get a design that is capable of turning the coding into a predictable construction activity? And if so, is cost of doing this sufficiently small to make this approach worthwhile?

All of this brings a few questions to mind. The first is the matter of how difficult it is to get a UML-like design into a state that it can be handed over to programmers. The problem with a UML-like design is that it can look very good on paper, yet be seriously flawed when you actually have to program the thing. The models that civil engineers use are based on many years of practice that are enshrined in engineering codes. Furthermore the key issues, such as the way forces play in the design, are amenable to mathematical analysis. The only checking we can do of UML-like diagrams is peer review. While this is helpful it leads to errors in the design that are often only uncovered during coding and testing. Even skilled designers, such as I consider myself to be, are often surprised when we turn such a design into software.

Another issue is that of comparative cost. When you build a bridge, the cost of the design effort is about 10% of the job, with the rest being construction. In software the amount of time spent in coding is much, much less McConnell suggests that for a large project, only 15% of the project is code and unit test, an almost perfect reversal of the bridge building ratios. Even if you lump in all testing as part of construction, then design is still 50% of the work. This raises an important question about the nature of design in software compared to its role in other branches of engineering.

These kinds of questions led Jack Reeves to suggest that in fact the source code is a design document and that the construction phase is actually the use of the compiler and linker. Indeed anything that you can treat as construction can and should be automated.

This thinking leads to some important conclusions:

• In software: construction is so cheap as to be free
• In software all the effort is design, and thus requires creative and talented people
• Creative processes are not easily planned, and so predictability may well be an impossible target.
• We should be very wary of the traditional engineering metaphor for building software. It’s a different kind of activity and requires a different process

The Unpredictability of Requirements

There’s a refrain I’ve heard on every problem project I’ve run into. The developers come to me and say “the problem with this project is that the requirements are always changing”. The thing I find surprising about this situation is that anyone is surprised by it. In building business software requirements changes are the norm, the question is what we do about it.

One route is to treat changing requirements as the result of poor requirements engineering. The idea behind requirements engineering is to get a fully understood picture of the requirements before you begin building the software, get a customer sign-off to these requirements, and then set up procedures that limit requirements changes after the sign-off.

One problem with this is that just trying to understand the options for requirements is tough. It’s even tougher because the development organization usually doesn’t provide cost information on the requirements. You end up being in the situation where you may have some desire for a sun roof on your car, but the salesman can’t tell you if it adds $10 to the cost of the car, or $10,000. Without much idea of the cost, how can you figure out whether you want to pay for that sunroof?

Estimation is hard for many reasons. Part of it is that software development is a design activity, and thus hard to plan and cost. Part of it is that the basic materials keep changing rapidly. Part of it is that so much depends on which individual people are involved, and individuals are hard to predict and quantify.

Software’s intangible nature also cuts in. It’s very difficult to see what value a software feature has until you use it for real. Only when you use an early version of some software do you really begin to understand what features are valuable and what parts are not.

This leads to the ironic point that people expect that requirements should be changeable. After all software is supposed to be soft. So not just are requirements changeable, they ought to be changeable. It takes a lot of energy to get customers of software to fix requirements. It’s even worse if they’ve ever dabbled in software development themselves, because then they “know” that software is easy to change.

But even if you could settle all that and really could get an accurate and stable set of requirements you’re probably still doomed. In today’s economy the fundamental business forces are changing the value of software features too rapidly. What might be a good set of requirements now, is not a good set in six months time. Even if the customers can fix their requirements, the business world isn’t going to stop for them. And many changes in the business world are completely unpredictable: anyone who says otherwise is either lying, or has already made a billion on stock market trading.

Everything else in software development depends on the requirements. If you cannot get stable requirements you cannot get a predictable plan.

Is Predictability Impossible?

In general, no. There are some software developments where predictability is possible. Organizations such as NASA’s space shuttle software group are a prime example of where software development can be predictable. It requires a lot of ceremony, plenty of time, a large team, and stable requirements. There are projects out there that are space shuttles. However I don’t think much business software fits into that category. For this you need a different kind of process.

One of the big dangers is to pretend that you can follow a predictable process when you can’t. People who work on methodology are not very good at identifying boundary conditions: the places where the methodology passes from appropriate in inappropriate. Most methodologists want their methodologies to be usable by everyone, so they don’t understand nor publicize their boundary conditions. This leads to people using a methodology in the wrong circumstances, such as using a predictable methodology in a unpredictable situation.

There’s a strong temptation to do that. Predictability is a very desirable property. However if you believe you can be predictable when you can’t, it leads to situations where people build a plan early on, then don’t properly handle the situation where the plan falls apart. You see the plan and reality slowly drifting apart. For a long time you can pretend that the plan is still valid. But at some point the drift becomes too much and the plan falls apart. Usually the fall is painful.

So if you are in a situation that isn’t predictable you can’t use a predictive methodology. That’s a hard blow. It means that many of the models for controlling projects, many of the models for the whole customer relationship, just aren’t true any more. The benefits of predictability are so great, it’s difficult to let them go. Like so many problems the hardest part is simply realizing that the problem exists.

However letting go of predictability doesn’t mean you have to revert to uncontrollable chaos. Instead you need a process that can give you control over an unpredictability. That’s what adaptivity is all about.

Controlling an Unpredictable Process – Iterations

So how do we control ourselves in an unpredictable world? The most important, and still difficult part is to know accurately where we are. We need an honest feedback mechanism which can accurately tell us what the situation is at frequent intervals.

The key to this feedback is iterative development. This is not a new idea. Iterative development has been around for a while under many names: incremental, evolutionary, staged, spiral… lots of names. The key to iterative development is to frequently produce working versions of the final system that have a subset of the required features. These working systems are short on functionality, but should otherwise be faithful to the demands of the final system. They should be fully integrated and as carefully tested as a final delivery.

The point of this is that there is nothing like a tested, integrated system for bringing a forceful dose of reality into any project. Documents can hide all sorts of flaws. Untested code can hide plenty of flaws. But when people actually sit in front of a system and work with it, then flaws become truly apparent: both in terms of bugs and in terms of misunderstood requirements.

Iterative development makes sense in predictable processes as well. But it is essential in adaptive processes because an adaptive process needs to be able to deal with changes in required features. This leads to a style of planning where long term plans are very fluid, and the only stable plans are short term plans that are made for a single iteration. Iterative development gives you a firm foundation in each iteration that you can base your later plans around.

A key question for this is how long an iteration should be. Different people give different answers. XP suggests iterations of one or two weeks. SCRUM suggests a length of a month. Crystal may stretch further. The tendency, however, is to make each iteration as short as you can get away with. This provides more frequent feedback, so you know where you are more often.

The Adaptive Customer

This kind of adaptive process requires a different kind of relationship with a customer than the ones that are often considered, particularly when development is done by a separate firm. When you hire a separate firm to do software development, most customers would prefer a fixed-price contract. Tell the developers what they want, ask for bids, accept a bid, and then the onus is on the development organization to build the software.

A fixed price contract requires stable requirements and hence a predictive process. Adaptive processes and unstable requirements imply you cannot work with the usual notion of fixed-price. Trying to fit a fixed price model to an adaptive process ends up in a very painful explosion. The nasty part of this explosion is that the customer gets hurt every bit as much as the software development company. After all the customer wouldn’t be wanting some software unless their business needed it. If they don’t get it their business suffers. So even if they pay the development company nothing, they still lose. Indeed they lose more than they would pay for the software (why would they pay for the software if the business value of that software were less?)

So there’s dangers for both sides in signing the traditional fixed price contract in conditions where a predictive process cannot be used. This means that the customer has to work differently.

This doesn’t mean that you can’t fix a budget for software up-front. What it does mean is that you cannot fix time, price and scope. The usual agile approach is to fix time and price, and to allow the scope to vary in a controlled manner.

In an adaptive process the customer has much finer-grained control over the software development process. At every iteration they get both to check progress and to alter the direction of the software development. This leads to much closer relationship with the software developers, a true business partnership. This level of engagement is not for every customer organization, nor for every software developer; but it’s essential to make an adaptive process work properly.

All this yields a number of advantages for the customer. For a start they get much more responsive software development. A usable, although minimal, system can go into production early on. The customer can then change its capabilities according to changes in the business, and also from learning from how the system is used in reality.

Every bit as important as this is greater visibility into the true state of the project. The problem with predictive processes is that project quality is measured by conformance to plan. This makes it difficult for people to signal when reality and the plan diverge. The common result is a big slip in the schedule late in the project. In an agile project there is a constant reworking of the plan with every iteration. If bad news is lurking it tends to come earlier, when there is still time to do something about it. Indeed this risk control is a key advantage of iterative development.

Agile methods take this further by keeping the iteration lengths small, but also by seeing these variations in a different way. Mary Poppendieck summed up this difference in viewpoint best for me with her phrase “A late change in requirements is a competitive advantage”. I think most people have noticed that it’s very difficult for business people to really understand what they need from software in the beginning. Often we see that people learn during the process what elements are valuable and which ones aren’t. Often the most valuable features aren’t at all obvious until customer have had a chance to play with the software. Agile methods seek to take advantage of this, encouraging business people to learn about their needs as the system gets built, and to build the system in such a way that changes can be incorporated quickly.

All this has an important bearing what constitutes a successful project. A predictive project is often measured by how well it met its plan. A project that’s on-time and on-cost is considered to be a success. This measurement is nonsense to an agile environment. For agilists the question is business value – did the customer get software that’s more valuable to them than the cost put into it. A good predictive project will go according to plan, a good agile project will build something different and better than the original plan foresaw.

Putting People First

Executing an adaptive process is not easy. In particular it requires a very effective team of developers. The team needs to be effective both in the quality of the individuals, and in the way the team blends together. There’s also an interesting synergy: not just does adaptivity require a strong team, most good developers prefer an adaptive process.

Plug Compatible Programming Units

One of the aims of traditional methodologies is to develop a process where the people involved are replaceable parts. With such a process you can treat people as resources who are available in various types. You have an analyst, some coders, some testers, a manager. The individuals aren’t so important, only the roles are important. That way if you plan a project it doesn’t matter which analyst and which testers you get, just that you know how many you have so you know how the number of resources affects your plan.

But this raises a key question: are the people involved in software development replaceable parts? One of the key features of agile methods is that they reject this assumption.

Perhaps the most explicit rejection of people as resources is Alistair Cockburn. In his paper Characterizing People as Non-Linear, First-Order Components in Software Development, he makes the point that predictable processes require components that behave in a predictable way. However people are not predictable components. Furthermore his studies of software projects have led him to conclude the people are the most important factor in software development.

One wonders if not the nature of software development works against us here. When we’re programming a computer, we control an inherently predictable device. Since we’re in this business because we are good at doing that, we are ideally suited to messing up when faced with human beings.

Although Cockburn is the most explicit in his people-centric view of software development, the notion of people first is a common theme with many thinkers in software. The problem, too often, is that methodology has been opposed to the notion of people as the first-order factor in project success.

This creates a strong positive feedback effect. If you expect all your developers to be plug compatible programming units, you don’t try to treat them as individuals. This lowers morale (and productivity). The good people look for a better place to be, and you end up with what you desire: plug compatible programming units.

Deciding that people come first is a big decision, one that requires a lot of determination to push through. The notion of people as resources is deeply ingrained in business thinking, its roots going back to the impact of Frederick Taylor’s Scientific Management approach. In running a factory, this Taylorist approach may make sense. But for the highly creative and professional work, which I believe software development to be, this does not hold. (And in fact modern manufacturing is also moving away from the Taylorist model.)

Programmers are Responsible Professionals

A key part of the Taylorist notion is that the people doing the work are not the people who can best figure out how best to do that work. In a factory this may be true for several reasons. Part of this is that many factory workers are not the most intelligent or creative people, in part this is because there is a tension between management and workers in that management makes more money when the workers make less.

Recent history increasingly shows us how untrue this is for software development. Increasingly bright and capable people are attracted to software development, attracted by both its glitz and by potentially large rewards. (Both of which tempted me away from electronic engineering.) Despite the downturn of the early 00′s, there is still a great deal of talent and creativity in software development.

(There may well be a generational effect here. Some anecdotal evidence makes me wonder if more brighter people have ventured into software engineering in the last fifteen years or so. If so this would be a reason for why there is such a cult of youth in the computer business, like most cults there needs to be a grain of truth in it.)

When you want to hire and retain good people, you have to recognize that they are competent professionals. As such they are the best people to decide how to conduct their technical work. The Taylorist notion of a separate planning department that decides how to do things only works if the planners understand how to do the job better than those doing it. If you have bright, motivated people doing the job then this does not hold.

Managing a People Oriented Process

People orientation manifests itself in a number of different ways in agile processes. It leads to different effects, not all of them are consistent.

One of the key elements is that of accepting the process rather than the imposition of a process. Often software processes are imposed by management figures. As such they are often resisted, particularly when the management figures have had a significant amount of time away from active development. Accepting a process requires commitment, and as such needs the active involvement of all the team.

This ends up with the interesting result that only the developers themselves can choose to follow an adaptive process. This is particularly true for XP, which requires a lot of discipline to execute. Crystal considers itself as a less disciplined approach that’s appropriate for a wider audience.

Another point is that the developers must be able to make all technical decisions. XP gets to the heart of this where in its planning process it states that only developers may make estimates on how much time it will take to do some work.

Such technical leadership is a big shift for many people in management positions. Such an approach requires a sharing of responsibility where developers and management have an equal place in the leadership of the project. Notice that I say equal. Management still plays a role, but recognizes the expertise of developers.

An important reason for this is the rate of change of technology in our industry. After a few years technical knowledge becomes obsolete. This half life of technical skills is without parallel in any other industry. Even technical people have to recognize that entering management means their technical skills will wither rapidly. Ex-developers need to recognize that their technical skills will rapidly disappear and they need to trust and rely on current developers.

The Difficulty of Measurement

If you have a process where the people who say how work should be done are different from the people who actually do it, the leaders need some way of measuring how effective the doers are. In Scientific Management there was a strong push to develop objective approaches to measuring the output of people.

This is particularly relevant to software because of the difficulty of applying measurement to software. Despite our best efforts we are unable to measure the most simple things about software, such as productivity. Without good measures for these things, any kind of external control is doomed.

Introducing measured management without good measures leads to its own problems. Robert Austin made an excellent discussion of this. He points out that when measuring performance you have to get all the important factors under measurement. Anything that’s missing has the inevitable result that the doers will alter what they do to produce the best measures, even if that clearly reduces the true effectiveness of what they do. This measurement dysfunction is the Achilles heel of measurement-based management.

Austin’s conclusion is that you have to choose between measurement-base management and delegatory management (where the doers decide how to do the work). Measurement-based management is best suited to repetitive simple work, with low knowledge requirements and easily measured outputs – exactly the opposite of software development.

The point of all this is that traditional methods have operated under the assumption that measurement-based management is the most efficient way of managing. The agile community recognizes that the characteristics of software development are such that measurement based management leads to very high levels of measurement dysfunction. It’s actually more efficient to use a delegatory style of management, which is the kind of approach that is at the center of the agilist viewpoint.

The Role of Business Leadership

But the technical people cannot do the whole process themselves. They need guidance on the business needs. This leads to another important aspect of adaptive processes: they need very close contact with business expertise.

This goes beyond most projects involvement of the business role. Agile teams cannot exist with occasional communication . They need continuous access to business expertise. Furthermore this access is not something that is handled at a management level, it is something that is present for every developer. Since developers are capable professionals in their own discipline, they need to be able to work as equals with other professionals in other disciplines.

A large part of this, of course, is due to the nature of adaptive development. Since the whole premise of adaptive development is that things change quickly, you need constant contact to advise everybody of the changes.

There is nothing more frustrating to a developer than seeing their hard work go to waste. So it’s important to ensure that there is good quality business expertise that is both available to the developer and is of sufficient quality that the developer can trust them.

The Self-Adaptive Process

So far I’ve talked about adaptivity in the context of a project adapting its software frequently to meet the changing requirements of its customers. However there’s another angle to adaptivity: that of the process changing over time. A project that begins using an adaptive process won’t have the same process a year later. Over time, the team will find what works for them, and alter the process to fit.

The first part of self-adaptivity is regular reviews of the process. Usually you do these with every iteration. At the end of each iteration, have a short meeting and ask yourself the following questions (culled from Norm Kerth)

• What did we do well?
• What have we learned?
• What can we do better?
• What puzzles us?

These questions will lead you to ideas to change the process for the next iteration. In this way a process that starts off with problems can improve as the project goes on, adapting better to the team that uses it.

If self-adaptivity occurs within a project, it’s even more marked across an organization. A consequence of self-adaptivity is that you should never expect to find a single corporate methodology. Instead each team should not just choose their own process, but should also actively tune their process as they proceed with the project. While both published processes and the experience of other projects can act as an inspiration and a baseline, the developers professional responsibility is to adapt the process to the task at hand.

Flavors of Agile Development

The term ‘agile’ refers to a philosophy of software development. Under this broad umbrella sits many more specific approaches such as Extreme Programming, Scrum, Lean Development, etc. Each of these more particular approaches has its own ideas, communities and leaders. Each community is a distinct group of its own but to be correctly called agile it should follow the same broad principles. Each community also borrows from ideas and techniques from each other. Many practitioners move between different communities spreading different ideas around – all in all it’s a complicated but vibrant ecosystem.

So far I’ve given my take on the overall picture of my definition of agile. Now I want to introduce some of the different agile communities. I can only give a quick overview here, but I do include references so you can dig further if you like.

Since I’m about to start giving more references, this is a good point to point out some sources for general information on agile methods. The web-center is the Agile Alliance a non-profit set up to encourage and research agile software development. For books I’d suggest overviews by Alistair Cockburn and Jim Highsmith. Craig Larman’s book on agile development contains a very useful history of iterative development. For more of my views on agile methods look at the appropriate sections of my articles and blog.

The following list is by no means complete. It reflects a personal selection of the flavors of agile that have most interested and influenced me over the last decade or so.

Agile Manifesto

The term ‘agile’ got hijacked for this activity in early 2001 when a bunch of people who had been heavily involved in this work got together to exchange ideas and came up with the Manifesto for Agile Software Development.

Prior to this workshop a number of different groups had been developing similar ideas about software development. Most, but by no means all, of this work had come out of the Object-Oriented software community that had long advocated iterative development approaches. This essay was originally written in 2000 to try to pull together these various threads. At that time there was no common name for these approaches, but the moniker ‘lightweight’ had grown up around them. Many of the people involved didn’t feel this was a good term as it didn’t accurately convey the essence of what these approaches were about.

There had been some talking about broader issues in these approaches in 2000 at a workshop hosted by Kent Beck in Oregon. Although this workshop was focused on Extreme Programming (the community that at that time had gained the most attention) several non XPers had attended. One the discussions that came up was whether it was better for XP to be a broad or concrete movement. Kent preferred a more focused cohesive community.

The workshop was organized, if I remember correctly, primarily by Jim Highsmith and Bob Martin. They contacted people who they felt were active in communities with these similar ideas and got seventeen of them together for the Snowbird workshop. The initial idea was just to get together and build better understanding of each others’ approaches. Robert Martin was keen to get some statement, a manifesto that could be used to rally the industry behind these kinds of techniques. We also decided we wanted to choose a name to act as an umbrella name for the various approaches.

During the course of the workshop we decided to use ‘agile’ as the umbrella name, and came up with values part of the manifesto. The principles section was started at the workshop but mostly developed on a wiki afterwards.

The effort clearly struck a nerve, I think we were all very surprised at the degree of attention and appreciation the manifesto got. Although the manifesto is hardly a rigorous definition of agile, it does provide a focusing statement that helps concentrate the ideas. Shortly after we finished the manifesto Jim Highsmith and I wrote an article for SD Magazine that provided some commentary to the manifesto.

Later that year, most of the seventeen who wrote the manifesto got back together again, with quite a few others, at OOPSLA 2001. There was a suggestion that the manifesto authors should begin some on-going agile movement, but the authors agreed that they were just the people who happened to turn up for that workshop and produced that manifesto. There was no way that that group could claim leadership of the whole agile community. We had helped launch the ship and should let it go for whoever who wanted to sail in her to do so. So that was the end of the seventeen manifesto authors as an organized body.

One next step that did follow, with the active involvement of many of these authors, was the formation of the agile alliance. This group is a non-profit group intended to promote and research agile methods. Amongst other things it sponsors an annual conference in the US.

XP (Extreme Programming)

During the early popularity of agile methods in the late 1990′s, Extreme Programming was the one that got the lion’s share of attention. In many ways it still does.

The roots of XP lie in the Smalltalk community, and in particular the close collaboration of Kent Beck and Ward Cunningham in the late 1980′s. Both of them refined their practices on numerous projects during the early 90′s, extending their ideas of a software development approach that was both adaptive and people-oriented.

Kent continued to develop his ideas during consulting engagements, in particular the Chrysler C3 project, which has since become known as the creation project of extreme programming. He started using the term ‘extreme programming’ around 1997. (C3 also marked my initial contact with Extreme Programming and the beginning of my friendship with Kent.)

During the late 1990′s word of Extreme Programming spread, initially through descriptions on newsgroups and Ward Cunningham’s wiki, where Kent and Ron Jeffries (a colleague at C3) spent a lot of time explaining and debating the various ideas. Finally a number of books were published towards the end of the 90′s and start of 00′s that went into some detail explaining the various aspects of the approach. Most of these books took Kent Beck’s white book as their foundation. Kent produced a second edition of the white book in 2004 which was a significant re-articulation of the approach.

XP begins with five values (Communication, Feedback, Simplicity, Courage, and Respect). It then elaborates these into fourteen principles and again into twenty-four practices. The idea is that practices are concrete things that a team can do day-to-day, while values are the fundamental knowledge and understanding that underpins the approach. Values without practices are hard to apply and can be applied in so many ways that it’s hard to know where to start. Practices without values are rote activities without a purpose. Both values and practices are needed, but there’s a big gap between them – the principles help bridge that gap. Many of XP’s practices are old, tried and tested techniques, yet often forgotten by many, including most planned processes. As well as resurrecting these techniques, XP weaves them into a synergistic whole where each one is reinforced by the others and given purpose by the values.

One of the most striking, as well as initially appealing to me, is its strong emphasis on testing. While all processes mention testing, most do so with a pretty low emphasis. However XP puts testing at the foundation of development, with every programmer writing tests as they write their production code. The tests are integrated into a continuous integration and build process which yields a highly stable platform for future development. XP’s approach here, often described under the heading of Test Driven Development (TDD) has been influential even in places that haven’t adopted much else of XP.

There’s a great deal of publications about extreme programming. One area of confusion, however, is the shift between the first and second edition of the white book. I said above that the second edition is a ‘re-articulation’ of extreme programming, in that the approach is still the same but it is described in a different style. The first edition (with four values, twelve practices and some important but mostly-ignored principles) had a huge influence on the software industry and most descriptions of extreme programming were written based on the first edition’s description. Keep that in mind as you read material on XP, particularly if it was prepared prior to 2005. Indeed most of the common web descriptions of XP are based on the first edition.

The natural starting place to discover more is the second edition of the white book. This book explains the background and practices of XP in a short (160 page) package. Kent Beck edited a multi-colored series of books on extreme programming around the turn of the century, if forced to pick one to suggest I’d go for the purple one, remember that like most material it’s based on the first edition.

There’s a lot of material on the web about XP but most of it is based on the first edition. One of the few descriptions I know of that takes account of the second edition is a paper on The New XP (PDF) by Michele Marchesi who hosted the original XP conferences in Sardinia. For discussion on XP there is a yahoo mailing list.

My involvement in the early days and friendships within the XP community mean that I have a distinct familiarity, fondness and bias towards XP. I think it’s influence owes to marrying the principles of agile development with a solid set of techniques for actually carrying them out. Much of the early writings on agile neglected the latter, raising questions about whether the agile ideas were really possible. XP provided the tools by which the hopes of agility could be realized.

Scrum

Scrum also developed in the 80′s and 90′s primarily with OO development circles as a highly iterative development methodology. It’s most well known developers were Ken Schwaber, Jeff Sutherland, and Mike Beedle.

Scrum concentrates on the management aspects of software development, dividing development into thirty day iterations (called ‘sprints’) and applying closer monitoring and control with daily scrum meetings. It places much less emphasis on engineering practices and many people combine its project management approach with extreme programming’s engineering practices. (XP’s management practices aren’t really very different.)

Ken Schwaber is one of the most active proponents of Scrum, his website is a good place to start looking for more information and his book is probably the best first reference.

Crystal

Alistair Cockburn has long been one of the principal voices in the agile community. He developed the Crystal family of software development methods as a group of approaches tailored to different size teams. Crystal is seen as a family because Alistair believes that different approaches are required as teams vary in size and the criticality of errors changes.

Despite their variations all crystal approaches share common features. All crystal methods have three priorities: safety (in project outcome, efficiency, habitability (developers can live with crystal). They also share common properties, of which the most important three are: Frequent Delivery, Reflective Improvement, and Close Communication.

The habitability priority is an important part of the crystal mind-set. Alistair’s quest (as I see it) is looking for what is the least amount of process you can do and still succeed with an underlying assumption of low-discipline that is inevitable with humans. As a result Alistair sees Crystal as requiring less discipline that extreme programming, trading off less efficiency for a greater habitability and reduced chances of failure.

Despite Crystal’s outline, there isn’t a comprehensive description of all it’s manifestations. The most well described is Crystal Clear, which has a modern book description. There is also a wiki for further material and discussion of Crystal.

Context Driven Testing

From the beginning it’s been software developers who have been driving the agile community. However many other people are involved in software development and are affected by this new movement. One obvious such group is testers, who often live in a world very much contained by waterfall thinking. With common guidelines that state that the role of testing is to ensure conformance of software to up-front written specifications, the role of testers in an agile world is far from clear.

As it turns out, several people in the testing community have been questioning much of mainstream testing thinking for quite a while. This has led to a group known as context-driven testing. The best description of this is the book Lessons Learned in Software Testing. This community is also very active on the web, take a look at sites hosted by Brian Marick (one of the authors of the agile manifesto), Brett Pettichord, James Bach, and Cem Kaner.

Lean Development

I remember a few years ago giving a talk about agile methods at the Software Development conference and talking to an eager woman about parallels between the agile ideas and lean movement in manufacturing. Mary Poppendieck (and husband Tom) have gone on to be active supporters of the agile community, in particular looking at the overlaps and inspirations between lean production and software development.

The lean movement in manufacturing was pioneered by Taiichi Ohno at Toyota and is often known as the Toyota Production System. Lean production was an inspiration to many of the early agilists – the Poppendiecks are most notable to describing how these ideas interact. In general I’m very wary of these kinds of reasoning by analogy, indeed the engineering separation between design and construction got us into this mess in the first place. However analogies can lead to good ideas and I think the lean ideas have introduced many useful ideas and tools into the agile movement.

The Poppendiecks’ book and website are the obvious starting points for more information.

(Rational) Unified Process

Another well-known process to have come out of the object-oriented community is the Rational Unified Process (sometimes just referred to as the Unified Process). The original idea was that like the UML unified modeling languages the UP could unify software processes. Since RUP appeared about the same time as the agile methods, there’s a lot of discussion about whether the two are compatible.

RUP is a very large collection of practices and is really a process framework rather than a process. Rather than give a single process for software development it seeks to provide a common set of practices for teams to choose from for an individual project. As a result a team’s first step using RUP should be to define their individual process, or as RUP calls it, a development case.

The key common aspects of RUP is that it is Use Case Driven (development is driven through user-visible features), iterative, and architecture centric (there’s a priority to building a architecture early on that will last the project through).

My experience with RUP is that it’s problem is its infinite variability. I’ve seen descriptions of RUP usage that range from rigid waterfall with ‘analysis iterations’ to picture perfect agile. It’s struck me that the desire of people to market the RUP as the single process led to a result where people can do just about anything and call it RUP – resulting in RUP being a meaningless phrase.

Despite all this there are some very strong people in the RUP community that are very much aligned with agile thinking. I’ve been impressed in all my meeting with Phillippe Kruchten and his book is best starting point for RUP. Craig Larman has also developed descriptions of working with RUP in an agile style in his popular introductory book on OO design.

Should you go agile?

Using a agile method is not for everyone. There are a number of things to bear in mind if you decide to follow this path. However I certainly believe that these methodologies are widely applicable and should be used by more people than currently consider them.

In today’s environment, the most common methodology is code and fix. Applying more discipline than chaos will almost certainly help, and the agile approach has the advantage that it is much less of a step than using a heavyweight method. Here the light weight of agile methods is an advantage. Simpler processes are more likely to be followed when you are used to no process at all.

For someone new to agile methods, the question is where to start. As with any new technology or process, you need to make your own evaluation of it. This allows you to see how it fits into your environment. As a result much of my advice here follows that I’ve given for other new approaches, bringing back memories of when I was first talking about Object-Oriented techniques.

The first step is to find suitable projects to try agile methods out with. Since agile methods are so fundamentally people-oriented, it’s essential that you start with a team that wants to try and work in an agile way. Not just is a reluctant team more difficult to work with, imposing agile methods on reluctant people is fundamentally at odds with the whole notion of agile development.

It’s valuable to also have customers (those who need the software) who want to work in this kind of collaborative way. If customers don’t collaborate, then you won’t see the full advantages of an adaptive process. Having said that we’ve found on several occasions that we’ve worked with customers who didn’t want to collaborate, but changed their mind over the first few months as they begun to understand the agile approach.

A lot of people claim that agile methods can’t be used on large projects. We (ThoughtWorks) have had good success with agile projects with around 100 people and multiple continents. Despite this I would suggest picking something smaller to start with. Large projects are inherently more difficult anyway, so it’s better to start learning on a project of a more manageable size.

Some people advise picking a project with little business impact to start with, that way if anything goes wrong then there’s less damage. However an unimportant project often makes a poor test since nobody cares much about the outcome. I prefer to advise people to take a project that’s a little bit more critical than you are comfortable with.

Perhaps the most important thing you can do is find someone more experienced in agile methods to help you learn. Whenever anyone does anything new they inevitably make mistakes. Find someone who has already made lots of mistakes so you can avoid making those yourself. Again this is something true for any new technology or technique, a good mentor is worth her weight in gold. Of course this advice is self serving since ThoughtWorks and many of my friends in the industry do mentoring on agile methods. That doesn’t alter the fact that I strongly believe in the importance of finding a good mentor.

And once you’ve found a good mentor, follow their advice. It’s very easy to second guess much of this and I’ve learned from experience that many techniques can’t really be understood until you’ve made a reasonable attempt to try them out. One of the best examples I heard was a client of ours who decided to trial extreme programming for a couple of months. During that period they made it clear that they would do whatever the mentor said – even if they thought it was a bad idea. At the end of that trial period they would stop and decide if they wanted to carry on with any of the ideas or revert to the previous way of working. (In case you were wondering they decided to carry on with XP.)

One of the open questions about agile methods is where the boundary conditions lie. One of the problems with any new technique is that you aren’t really aware of where the boundary conditions until you cross over them and fail. Agile methods are still too young to see enough action to get a sense of where the boundaries are. This is further compounded by the fact that it’s so hard to decide what success and failure mean in software development, as well as too many varying factors to easily pin down the source of problems.

So where should you not use an agile method? I think it primarily comes down the people. If the people involved aren’t interested in the kind of intense collaboration that agile working requires, then it’s going to be a big struggle to get them to work with it. In particular I think that this means you should never try to impose agile working on a team that doesn’t want to try it.

There’s been lots of experience with agile methods over the last ten years. At ThoughtWorks we always use an agile approach if our clients are willing, which most of the time they are. I (and we) continue to be big fans of this way of working.