Successfully Creating Value Added Documentation Using Agile Software Development in Heavily Regulated Environments
When technology supports heavily regulated industries, comprehensive documentation is viewed as a critical IT deliverable that provides evidence of compliance, assists in maintaining continuity of operations, and demonstrates transparency to key stakeholders. However, requirements for comprehensive documentation can present a major challenge for adoption of Agile principles because they require resources to be diverted from the Agile team’s primary focus: delivering software that achieves end user adoption. Does this mean that Agile methodologies are incompatible with IT development, engineering, and integration in heavily regulated environments? We don’t think so.
While the Agile Manifesto specifically states that working software should be valued over comprehensive documentation, it doesn’t claim that documentation is not useful, and it certainly doesn’t suggest that documentation should be completely removed from an Agile team’s standard process. We interpret “working software over comprehensive documentation” as guidance for Agile teams to document what is required to achieve end user adoption and acceptance by your customer in a manner that can pass muster from a compliance perspective.
Thus in heavily regulated environments where working software alone may not be enough to achieve end user adoption/customer acceptance and compliance, service providers may actually need to produce significant documentation in order to achieve these goals. As such, Agile teams in these environments should think of documentation requirements in the same light as they do functional system requirements (i.e., work products that must be delivered to meet customer needs) and find ways to incorporate documentation development and delivery into their standard process seamlessly and organically.
To incorporate appropriate documentation into your team’s standard Agile process, we recommend the following high-level approach. First, Agile teams in heavily regulated environments should capture documentation requirements as stories and add them to the product backlog. For large documentation update efforts, the team may need to create an epic that includes multiple stories, which will be addressed over the course of multiple sprints. For example, if the team is responsible for creating a design document, they may create an epic for the overall design document and then the epic may include stories for each of the main sections within the document. In such a manner, the regulatory documentation needs can be built into the Agile process and not an ad hoc activity.
Once epics and stories have been created, the Agile team can then groom the story to identify the priority of the story for the customer, define acceptance criteria, and capture a high-level estimate for the work involved. (Keep in mind that the level of effort estimate for the story should not exceed the length of the sprint timebox.) Once the story includes the comprehensive information, the team can proceed with its typical process of selecting stories from the product backlog to create the sprint backlog for the upcoming sprint in the Sprint Planning Meeting. Once the story is included in the sprint backlog, one of the team’s resources can begin working on the story so that it will be ready for review by the end of the sprint timeframe. If possible, Agile teams in heavily regulated environments that have extensive documentation requirements should hire/assign dedicated resources for these documentation stories.
We’ve employed the above approach on a number of IT programs with great success. On our longest running Agile program, our client is highly satisfied with the documentation delivered using our approach and we’ve never missed a major documentation deadline in over seven years. We achieved our success rate by providing the customers with multiple opportunities for review and feedback. We also paired the documentation review cycles with actual demonstrations of functionality when appropriate, which helped to cultivate trust between our customers and our approach. Today, these solutions and their subsequent benefits continue to be critical to the sustained success of our team.
Although heavily regulated industries are increasing their support of Agile development methodologies documentation requirements are likely to remain a primary challenge for Agile teams in these environments. For more information on how to manage documentation in heavily regulated IT environments while using Agile, read this white paper, Agile in the Federal Space: Keeping Documentation Lean.