There is an intriguing question that pops up frequently in organizations developing software in projects: when is a project successful? For sure, one of the most (mis)used resources on the subject is the Standish Group. In their frequently renewed CHAOS Report they define a project successful if it delivers on time, on budget, and with all planned features. For a number of reasons this is, in my opinion, a rather vague definition.
One of the key and often very much underestimated assets of working in agile teams, whether working on products or projects, is the idea of sustainable pace. In my view, sustainable pace targets at making sure that, even under time pressure, which is not rare in software development, the team remains it’s cool. For those of you who have been part of agile teams, you will have noticed that achieving sustainable pace is not always easy.
While writing this post, I am on an airplane to Helsinki. Nice sunny city. And lucky for me it is about 50 degrees Celsius warmer than the last time I visited it. In Helsinki I will have look at a couple of projects that use a distributed agile approach, but fail to deliver on-time and on-budget.
When I get back to the Netherlands next week, I will pick up coaching in three different organizations and three very different projects. The first project is all about continuous software product development, with business developers, quarterly releases and multiple configurations of the products delivered. The second one is building a public available service, both for mobile and web, using .NET and Mono technology. And the third project is rebuilding an old customer-facing client-server application in Java, whilst moving towards a service oriented architecture, complete with document management and business process automation. So in merely two weeks I will look at five very different projects, in different organizations, with different targets, and very different issues. But there is one common denominator: they are all trying to be as agile as they possibly can.
All projects great and small
In only a few years agile has moved from guerilla warfare to mainstream. The popularity of everything agile is reaching new peaks. From agile practices such as continuous integration, pair programming, and test driven development, to agile approaches, such as Scrum and Kanban to even open space sessions, agile thinking, agile gaming and a seemingly never ending stream of agile conferences.
Despite the fact that no-one in the agile communities actually claims that agile, or Scrum, or Kanban, is a silver bullet for all projects great and small, many organizations these days move towards agile, based on a firm believe that agile will solve all their problems. And it just doesn’t.
Dealing with x
In my experience, agile approaches only cover a small subset of the topics and issues that projects in the real world will need to address. Simply because most agile approaches merely provide a minimalistic framework to guide your projects, and simply because it is virtually impossible for the authors of agile approaches to provide fitting solutions for each and everyone’s problems.
Very often project teams ask me: how do you deal with x in agile? And feel free to replace x with every issue that your projects have suffered from in the past years. This month x’s for me? Estimating bug fixing, velocity for starting teams, agile and service oriented architecture, agile and building frameworks, distributed functional testing, documentation in Finnish, while developers and testers can’t read Finnish, agile work breakdown, estimating user stories, DBA’s that have too much control, or even how to shorten the time that is required to move from development to test (which is currently two weeks).
So how do you deal with x in agile? If I was a Scrum Master (with or without the dash) my default answer would probably be: it depends. It’s an easy way out. Your audience will continue to believe all wisdom is converged in Scrum, even without offering a real solution. But then again I’m not a Scrum Master. So my answer is a bit different: agile doesn’t cover x.
No tailored solution
Basically, agile doesn’t exist. And even worse, there’s no such thing as the agile approach or methodology. At most, there’s a set of approaches that adhere to a set of principles. And although these agile principles and the approaches that adhere to them will give you a sense of direction on how to solve your specific project issues, none of them will provide you with a tailored solution.
Moving towards agile does never mean that you should stop thinking and go on automatic pilot. Each and every one of your projects is different. They start different, use a different set of techniques, have different teams with different people, serve different clients with different demands, you have different roles in your organizations, apply different techniques and technologies, and projects have different endings.
More than meets the eye
Projects are all different, and thus there is no one-size-fits-all agile approach. You will always have to tailor agile to your organization and to your project. Find out what it is that your project requires, besides implementing the basic agile principles. Figure out how you want to deal with requirements, as user stories could well be an over-simplified technique. Identify how estimation works best for you. Don’t throw out old techniques that your organization has been using for years and years just because they are not mentioned in the Scrum Guide. If your environments don’t match on-click deployment, identify how it can best match agile projects. Identify how to report out to your management best. And try to establish how the roles in your organization could match the way you would like to deliver. What is it enterprise architects can contribute? Or functional analysts? And foremost, make sure your testers now how important they can be in agile projects. Even though most agile approaches don’t mention any of these roles specifically.
To some, these warnings may well be an open door, in that case: carry on. But in my experience, most organizations and projects taking the plunge into agile, Scrum, Kanban, XP, please be aware that there’s so much more than meets the eye.
In my previous post, I explored how offshore Agile software development offers many benefits over more traditional, Waterfall style approaches, but only if some of the obvious difficulties in communication, overheads and language issues are addressed. So how do organizations overcome those difficulties to make offshore Agile work?
Over many years at Capgemini, we have gained experience with distributed Agile projects, whether onshore or offshore, and have learned a great deal about the dynamics of Agile software development teams. Based on our experience, this article outlines a number of key recommendations for making Agile work across distributed teams.
It is highly recommended to facilitate a cultural exchange of the people involved in the project. Prior to starting any implementation, it is good practice to ensure a preliminary stage where the basics for the project, such as an overall model of the requirements, estimates, plan, and a baseline architecture are set. This is an ideal moment to have the client and everybody in the team meet face-to-face, and get acquainted. Despite the obvious costs of arranging this meeting, teams will connect much more easily and collaborate more smoothly later on in the project. This is especially beneficial for long-running Agile projects.
Facilitate continuous communication
In Agile projects, communication is key. Distributed projects need to facilitate the ability to continuously communicate. With the distance between team members involved in offshore projects, online communication methods are essential. Where possible, it is important to use phone, but preferably video conferencing for kick-offs, retrospectives and stand-up meetings. Many teams also use simple chat programs for asking questions and sharing knowledge.
Solve language issues early
Many organizations, especially in public service, rely on communication and documentation in their native language. Even code is often written in the native language. To the offshore team members these languages are new and awkward. Even with team members sent to language courses, non-native languages leave room for misinterpretation. It is vital to offshore projects, especially when using Agile approaches, to set up a workflow for translating documentation and code before coding starts. Solving language issues should even be part of the contract.
User stories are an immensely popular technique to gather requirements in Agile projects. However, as with traditional use cases, user stories can appear at different levels of granularity and suffer greatly from ambiguity. In many of our projects we therefore successfully apply smart use cases, a more standardized technique for defining requirements. By nature, smart use cases are defined at the same level of granularity, and take a much more standardized approach, facilitating easier distributed communication on individual work items.
Standardize work item workflow
At the start of iterations, many Agile projects spend a lot of time breaking down user stories into individual tasks, and on estimating the required effort in hours, with the goal of being able to negotiate the amount of work that can be handled during each iteration. Iteration kick-off workshops are costly, and even worse, require the whole team to be present. It is good practice to minimize these kick-offs. Work item breakdowns and estimates are required much less if across work items work is aligned to a standardized work item workflow – with steps such as design, coding, developer testing, testing and acceptance.
Visualize work item workflow
Once a standardized number of steps in the work item workflow are defined, the actual status of the individual work items can be visualized easily on a dashboard. Where co-located teams usually stick post-its on a whiteboard, distributed teams will need a distributed dashboard. Usually this is a website that is accessible to all team members, including the client, or aligned with bug tracking or source control tools.
Work item teams
Rather than the traditional divide between analysis and design, onshore and development, and testing offshore, there are great benefits in working in teams that consist of team members on either side of the line working jointly to implement individual work items. By operating in such work item teams, or feature teams, there is a much more implicit focus on getting the work done. Work item teams tend to be more coherent and much more motivating and stimulating.
Standardize architecture and technology
A much-heard complaint in offshore development is that “they” don’t understand the software architecture and the technology that is used. However, it is important to realize that aspects such as software architectures, frameworks, complex domains and service oriented architectures are complicated by nature, and that for any team, whether co-located or distributed, it will take time to get used to any proposed solutions.
Offshore projects have a bad reputation for team instability. There are sometimes cases where members of the offshore team quit over the course of a weekend, or are replaced by new members who don’t have the required skills and knowledge. It is vital to keeps teams stable, especially in long-running projects. As working in Agile teams is experienced as much more motivating and pro-active, Agile helps to reduce team instability.
In conclusion, whether offshore Agile projects can be as successful as onshore Agile projects depends on a great number of factors in addition to the ones outlined above. But once the obvious difficulties in communication, overheads and language issues are reduced, offshore Agile projects can actually work particularly well, given a collaborative and standardized approach.
Due to the ever-rising demand for seasoned software developers in the nineties, offshore software development became a compelling alternative to in-house development for many organizations. Despite the cultural, language and time differences and the geographical distance involved, more and more projects were executed with offshore development and testing, benefiting from lower rates of cost and the high availability of people, and where necessary, the ability to have teams working around the clock.
Particularly during the early years of offshore software development, the majority of projects were executed using rather traditional, Waterfall-style approaches. Projects were characterized as fixed-price and fixed-date. The requirements and the design were compiled onshore, after which coding and testing was done in another part of the world, be it Eastern Europe, India or South America.
The Waterfall model typically involves executing each of the activities in software development consecutively, and only once – requirements, analysis, design, code, test and deployment – to deliver a product based on the completion of each of these project milestones. Interestingly, this model has led to high failure rates in projects, even without offshoring some of the activities. At each milestone knowledge is lost and testing is executed very late – only when development is done – with exponentially rising costs of repairing bugs. Moreover, achieving completeness in requirements early on in a project is difficult and changes to requirements are costly.
Oddly enough, even with these anomalies, and even with failing local projects, many organizations still ventured into offshore software development applying the Waterfall model. Not surprisingly, many of these projects failed to deliver on time or on budget and did not deliver the required functionality. Despite the hoped-for benefits of lower costs and high availability of skilled people, offshore projects add another level of complexity due to more complex control and coordination, and because of language, cultural and time zone discrepancies.
Is Agile more suited for offshore than Waterfall?
So with Agile approaches, such as Scrum and Kanban, reaching the peaks of their popularity, an interesting question is: can Agile approaches and techniques overcome some of the shortcomings of offshore Waterfall development?
In short, Agile approaches are characterized by working in short iterations, where during each iteration a number of continuously re-prioritized work items are fully realized by a multi-disciplinary team, usually applying a similar life cycle per work item – requirements, analysis, design, code, test and deployment – as Waterfall uses over a whole project.
At the start of an Agile project the requirements are only identified, and not compiled into full detail. This list of requirements is known as the backlog, and is not designed to be complete. Rather, items from the backlog are elaborated on during iterations (also known as sprints). So all the real work is done during iterations. As a consequence, Agile teams are required to be multidisciplinary, and work together on a daily basis to implement functionality work item by work item. As such, co-location of teams, quite often at the client, works best in Agile.
So is Agile more suited than Waterfall for offshore software development? Clearly there are huge benefits. Requirements, analysis and design needn’t be finalized until a work item is actually implemented. So there is no grand, fixed, inflexible design decided upfront. Testing and deployment also take place immediately after coding the individual work items, not only at the end of the project. The total life cycle of a work item is usually no longer than a couple of days, instead of the whole project duration. That is why Agile works well in domains where it is accepted that requirements are never complete and might change, which is the case in the vast majority of projects worldwide.
So what happens when Agile approaches and techniques are applied to offshore software development to overcome Waterfall shortcomings? Apart from the apparent benefits, applying Agile to offshore also comes with consequences. Applying an Agile approach will involve close collaboration between all roles, whether on-site or offshore. It will also involve daily communication, and the ability to work on the same work item at the same time. Communication is therefore key in Agile projects, and as we all know, distance makes communication harder. Therefore offshore Agile teams need to be able to rely on other means of communication than on-site teams. Moreover, it is key that information is clear and unambiguous, which is difficult as this is exactly the bottleneck that Agile is trying to overcome, as work items are not elaborated on until implemented.
But despite some of the difficulties involved in offshore Agile projects, particularly in highly complex domains or around regulatory sensitivities, Agile approaches have the potential to offer many benefits. In my opinion, it almost goes without saying that offshore Agile is going to be more effective to most organizations than offshore Waterfall, but only when key issues around communication, overheads and language issues are overcome. I will explore some of the key strategies for making offshore Agile software development work in my next post.
This post was also published on the ISD Connect website at:
For as long as I can remember I have been evangelizing, promoting, practicing, coaching, and training agile. For me as a developer the goals for applying agile approaches and techniques are pretty clear. I want to make better software. Higher quality, better suited for use, and possibly also faster. And from my own empirical evidence I can certainly state agile helps.
Het besparen van kosten is een veelgenoemde aanleiding voor Business Intelligence (BI) projecten. Zo wilde een bekende overheidsinstantie weten hoe effectief de bestrijding van uitkeringsfraude was. Het onderzoeken van mogelijke fraude kost de instantie geld, maar het vinden van fraudeurs levert echter direct geld op. En dus ging zoekt de instantie naar de optimale verhouding tussen het aantallen onderzoeken en het aantal opgespoorde fraudes. Kortgezegd wilde men met zo min mogelijk onderzoeken zoveel mogelijk fraudeurs vinden.