Is there a role for the BRD in Agile?

My answer is “yes” but not the old type of Business Requirements Document (BRD). I think there is a role for a new type of on-line BRD – one that can easily evolve during the project. Some argue that TDD is replacing this need. But I think there’s a place in the Agile process for capturing the business requirements/information in parallel with the development tasks that ends up on-line and useful/readable by all. It happens DURING the process, NOT up-front.

I think anyone who believes in Agile knows that the biggest problem with a big fixed documentation up-front, especially in Word, is that it gets out-of-date quickly and then is just a pile of often-misleading words. OR it’s held as the “bible” and then people become possessive of it and build some big process about how to any requirements mismatches, design flaws, etc. become change request-type issues and there we go – Waterfall all over again.

I believe the BAs/PMs are more likely to feel they can retain project oversight if there’s an on-line tool to track the product backlog (in high-level terms) and bring forward when time to implement it and where you can expand/refactor/capture details as you go along. Where stories can evolve into an on-line knowledgebase/repository that people (like the Call Center, Services group, new hires) can read and reference.

Of course, all projects, Agile or not, have requirements that are (hopefully) based on business needs. Some teams are struggling with their PM/BAs wanting to create a large document at the beginning of the project containing detailed use cases with every possible variation on every requirement. Some are having trouble convincing the PMs/BAs that a Product Backlog with user stories instead of fully documented use cases is a suitable method, with detailed analysis taking place during a sprint.

I think the hesitancy from BAs/PMs is a feeling of a loss of control when they are told to just write down “I want this …” stories, put notes on a board and work “in the air” (verbal communication mainly, not written). Some may argue that is a “lack of trust” – but I argue that PMs are also builders/designers who want to end up with “their” product at the end, their BRD. That still can be a valuable artifact. As long as it’s something that evolves with the product seamlessly and isn’t a wall blocking the project.

That’s where Software 2020 fits in beautifully. The prior blog, “Automatic Documentation – How we use Software 2020 to build Software 2020” describes the automatic way to use Software 2020 for tracking tasks, communicating between the team members (PMs and Developers) resulting in both an outstanding product and nice on-line knowledge base repository (aka BRD).

Software 2020 provides rich text online storage that is:

Easy to maintain

Easy to track

Easy to priortize

Searchable and sortable

Automatic Documentation – How we use Software 2020 to build Software 2020

To better describe how Software 2020 can help you create your As-Built documents as a by-product of your software development process, automatically, here’s how we used it when we built Software 2020. Of course we couldn’t use it from start-to-finish like we recommend you use it because it wasn’t built yet ☺ But we proved how great a tool it is and will continue to use it as the product evolves.

I first wrote a spec to capture my thoughts and ideas since there was no Software 2020 to put my ideas in but we used it as a guide, not as firm requirements. Freeman is so creative that when he doesn’t need to follow a spec exactly we get better products. Similar to why Agile proposes fuzzy stories which evolve into the final firm specification. We work best as a team when we can iterate and evolve the product as we go along. We’re agile.

So we started fuzzy (like you would with Agile stories). I want a “Task Tracker” tool like our prior SD Tracker (proprietary) tool. I want another document-type, Word-like “view” of the subset of tasks that were requirements or “specs”.

I’ve built enterprise application software for years and managed those efforts, using a combination of tools and proven processes that allowed us to build products quicker, better, and with less people than other software companies could in our field. Our techniques and approaches have withstood the test of time. I knew how I’d like one tool to solve the holes and issues I’m seeing in how companies use tools today: Agile Story Managers, Task Trackers, Requirements Management Tools – and definitely better than what is used in Agile teams that don’t like tools: Word, Excel, Story Boards. I had a good idea of what I wanted. Freeman had great ideas about the new kind of architecture he wanted to build.

The Tracker module was completed first since we had a clear starting place, improving the SD Tracker tool we built at our prior company. We started our project in October and by December the Tracker module was functional enough to use it to enter all of the development tasks in. I took the list of spec/tasks we’d been working from and entered them in the Tracker and from then on we worked as we always have in the past: Nothing gets coded unless there’s a Tracker Task for it and the task is assigned “open-fix” for development, set “in-progress” when work begins, “resolved” when on our integration system ready to test and “closed” when I’ve tested it.

Since by this point were already screens built for the Tracker, we started working to make sure those matched my ‘ideas’ and had some minor corrections. For example, the first task I entered in the Tracker, #1001, was a clean-up task:

“On all screens, in the white top banner, there shouldn’t be a title/words between the logo and tabs (now shows dash + Page Title + User’s Name). Instead the Title should be white and centered…”

As we did the clean-up, we evolved this task to capture the final requirement for screens in the Description of the task:

“There are three types of screens: Summary Screens that present batched sets of information, Detail Screens (Task, Person, Client), and Pop-Ups (attach a New Attachment, Set Defaults, etc.)

“On all screens, in the white top banner, there’s the Software 2020 log on the left and Navigation (Nav) buttons on the right. The first Nav link goes to the user’s Profile. The user’s name should be shown as ” Profile” in the tab bar. That way tech support knows who the user is but the screen is cleaner.

“Below the white banner is a blue Title Bar. The Title should be white and centered in the blue title bar.”

Once we had an “Organize View” feature in February 2012, I selected tasks that represented requirements, like the one above, #1001, and linked them to the appropriate spec until all tasks that represented requirements were in organized specs, by module. Here’s the current Software 2020 Spec List:

Or closer up view:

When I drill into the Look & Feel spec I see this:

Note that it looks like a nice word document (embedded graphics, etc.). That first task, #1001 is there. If you look at the entire screen you can view task information to the right so it’s not just a Word Doc but is a nice way to view, and update spec tasks:

Now I have all of the information about what was built, each and every task, in a nice readable set of specs. This is so much easier and better than trying to maintain a separate Word document or even update requirements in a linked tool. And definitely an advantage over having no documentation at the end except your pile of yellow stickies and code, regardless of how well commented and structured the code is.

Some advocate the Test Driven Development (TDD) approach. I agree there is benefit to TDD and would like to have a Test View on the same set of completed tasks to create test plans from. But disagree that TDD alone would replace real business-oriented requirements specs.

There’s a “Download” button if you want to save versions as Word docs, Excel spreadsheets, or an HTML page that you could store on a central server after each release to have a set of documents reflecting exactly how that version was built.

Then, as the Product Owner, I can add new features for the next release as Stories or requirements right in the Organize View (which is how I like to do it because it’s organized and logical, like a spec or word document) and they show up as tasks in the tracker. I could also, for new features entirely, create a Story Spec and put Themes / Epics / and Stories there.

New spec tasks and any other tasks (bug fixes, maintenance tasks, etc.) assigned to a release can have Story Points or time estimates entered and can be easily prioritized using the Ranking Tool:

Once ranked, the team can assign to Sprints and there’s no re-entering of tasks needed, no extra work.

Prior to Software 2020, we spent 10 years developing robust, secure enterprise application software using the prior SD Tracker tool and the Teleogic/IBM/DOORs Requirements Management System. Our software competed with the big guys – Siebel, Salesforce, SAP, Oracle – and won. Our process and toolset worked effectively but had manual steps and we didn’t have Agile story management tools.

Using Software 2020, the “One Tool” to do it all, is so much cleaner, easier, more efficient, and because of the automatic linkage, clearer for the developers than having to use a tracker then open a requirements spec, look for the links, etc. The task has the link to the spec – one click and they see this task in context with other related information (tasks related to the same function, screen shots, etc.)

For Agile teams, for the first project you would enter Stories grouped under Themes and Epics in the Organize View. Software 2020 provides nice Agile story management tools (charts, ranking tool, story point calculator) to help organize the sprints. As the story evolves, if the PO simply keeps the task/story description up-to-date, and, because it IS the task the developer is working on, there’s no mis-understanding. Because that description is on the task the developer is working on, both the PO and developer can easily view/agree that the description does match the code. There isn’t a separate “Requirements Spec” to maintain. Nothing to get out-of-sync. It’s automatic.

With Software 2020, the company obtains the benefits of having good systems documentation without any added work.

How to Document Software in the 21st Century – “Organize” View

On a recent LinkedIn post, people were saying “we still struggle … when it comes to testing and documentation” and not having good design documentation “creates difficulty for the software testers to come up with tests …” and that “In regulated environments, documentation is a must … and most important tracing from requirements to design and tests.”

I think you can have both. I believe having traceability from the design specification to tests and ending up with an accurate “As-Built” Spec is a “must” and valuable even in Agile environments. What “isn’t” Agile is expecting everything to be completely designed up-front. Rather I believe in evolving the requirements (or fuzzy up-front stories) but at some point the task must become clear enough for implementing.

My friend and I were VP Engineering and CTO respectively at my software company. She and I had both come out of the Aerospace industry and wanted to take what we’d learned and throw out the unnecessary steps to create a light-weight, high-quality process for start-ups. We called our process “Practical”. People who saw how we worked said we were agile. One key was we integrated an on-line requirements management tool tightly into our process. I’d update the new features in the on-line specs. Some would be complete up-front if the change was well-know and researched. Other times there’d be a placeholder – “We want a configurator”. Developers worked from these on-line specs. We were iterative – sometimes the specs evolved with the prototypes and the inventiveness of the developers but before testing started, the specs had to be finalized. Testers tested against the spec updates. I had a lot of manual work to link tasks to specs and make sure the wording was always up-to-date as the tasks evolved. But well worth it. The whole company leveraged those on-line specs. Extremely valuable.

But I always yearned for a better integrated tool where the story was automatically a task so everything was in one place. And as the task was split, refined it stayed organized, like in a spec so at the end I had accurate, on-line description of the actual deliverable product.

Companies now embracing Agile (completely or some hybrid) start with an Agile Story Manager tool and often need to re-enter tasks into their tracker. In companies that used to have good, integrated specs as part of their process like ours, those specs are becoming inaccurate, aren’t used by developers to code from and aren’t tested. Worthless.

We built Software 2020 because of these needs. It’s the tool I’d always wanted and we’ve been using it to build the tool itself (how’s that for iterative 🙂 There’s an “Organize” view that lets you structure your stories in a word-like doc (you could start with themes and epics but if you had an existing organized view “spec” you’d add the new features in their proper place) with screenshots, etc. and each specification is automatically a developer task. You can update the description of the specification as design is clarified either in the “Organize” view or regular task Tracker screens. Later we want to add another “view” for the test center. That gives the linkage the regulatory projects need – automatically. But even for non-regulatory, even on our little tool-building project, my architect/developer is already finding the value of looking at the document view of my design spec, in-context, to better understand the new features I’m designing/asking for. So I think we can be Agile and have nice documents too.

If you need an Up-Front Estimate

Someone said in a recent discussion thread “Estimating with the poker and the story points is very difficult to translate to time and money”. I’ve decided that story points are a great tool even for client-type estimation.

Before Agile, when estimating a large number of features or customer requirements, I’d first decide large-medium-small. Small would be 1 day, medium 5 and large 10 unless I thought it was REALLY large then would compare it to something I knew we’d done before. For example, anything that affected an import always seemed to take 4 weeks so I’d give them 20 days. Or a new module = 3 months. That’s all the story points are doing – giving relative sizing to projects too big for a developer to realistically know how to estimate it.

Story point work nicely because they are a series (I like the Fibonacci Series) which avoids wasting time with decisions like “Is it 5 days or 6 days?” Or worse, “Is it 30 days or 35 days?” I find that developers are pretty good at estimating 1, 2, 3, and maybe 5 day tasks. Anything larger needs to get broken down to obtain accurate estimates. But since up front clients want an initial approximate estimate, tell the developers to think of the story point as 1 day (I think most do anyway) and play poker. If you want more than just a “ballpark” estimate, then for anything > 5 points, split the stories. Stick with Story points initially – split 8 point tasks into a 3 and a 5 point story, etc. It helps if you have a tool that lets you easily split tasks and update story points.

After the project kicks off, have the developers convert points to hours (not directly points to hours but to update with real estimates like if some thing take 4 hours (we don’t user ½ story points) or 32 (between the 3 point and 5 point) – to get more accuracy). If you have a tool that tracks hours and part of your process is the developers continue to update the remaining hour estimates, that works well. We have a good date estimation tool then that shows the end date based on developer “load” (similar to estimating based on “confidence” but “load” is more explainable to development managers). I’ve found through the years it’s a mistake to assume that when a developer says it will take them 2 days that the code will get checked into the code management system in 2 calendar days. Two days in the developer’s thought process is time spent coding. Most employees typically have other things they do besides straight coding: meetings, discussions with the product owner/manager, client issues, bug fixing unrelated to the new efforts. So our tool lets the manager put in the ‘load’ and even a % by person (if you have a developer on your project only half time for example). With that type of tool, I and my development managers successfully kept projects on-track release after release. Our Board of Directors was very impressed year after year 🙂

Sometimes even with good up-front task management, during the process you may encounter an estimation error or something that was missed that will extend the timeline. Having a tool to keep checking release date projections helps identify such problems earlier than later. Typically projects will take as long as they will take. The earlier clients are aware that something ended up bigger than initially anticipated, the better.

And having broken the project down into manageable tasks, you can review the more granular features and functions with your client in the event that the revised estimates are not acceptable and together determine if some features should be postponed or the schedule/cost increase accepted.

Software 2020’s Date Estimator and other features make it easy to estimate and work effectively.

We don’t need separate tools

Aren’t new enhancement requests, stories, bugs, and requirements all just “Tasks”? And instead of all the different tools today — Agile Project Backlog Management Tools, Bug Tracker Tools, Requirements Management Tools — shouldn’t those be just different views to manage these (or subsets of these) “Tasks”? I think so.

Story Boards, Requirements Specs, Bug Tracking Tools are all just different views of “Tasks”.

Given that perspective, a modern robust Tracker would manage all “Tasks” (whether they are the new Agile Stories, bugs, client issues, etc.) consolidated for team tracking and estimation to complete reports considering the team’s vacation calendar and all of their work combined.

Then modern Agile tools could help plan new releases taking into account information about the developers’ other work (hence their percent availability) and provide a better estimate of completion for the current sprints and releases. Useful tools then would help manage the current release (in Agile, that’s Story Boards and organized hierarchical views – grouping tasks into epics, stories and their children). And facilitate the team’s management of these Tasks as they move from concept to delivery.

But keep them in the bigger database of “Tasks” so managers can manage the team’s full effort, not just the current release.

How to Document software in the 21st Century – Part 1

(Written originally for Duck Pond Software Blog – December 2011)

Something has been bothering me for a long time. I wrote “Smooth Sailing” a few years ago when I was astonished that people didn’t recognize the value of having good documentation – moreover they thought it wasn’t possible. I’d used DOORs successfully for years and to me, having developers work against a spec, give feedback, collaboratively develop the spec and code together, then have QA test against the spec assured a final spec that matched code that Tech Support could use to know, decisively, how the code was supposed to work.

That let us clearly specify customer complaints as Bugs (don’t match the spec), Enhancement Requests (not in the spec) or “Gaps” (maybe should have been in the spec in the customer’s eyes but were missed. Or spec ambiguities.) That saved us a lot of discussion time in our SDRB (Software Design Review Board) meetings where we reviewed the issue clients and internal support people had with the product. Bugs were things developers missed – the onus was on the Development Managers. “Bad Developers”. Enhancements were things we put in the next major releases. Obviously not negotiable for maintenance releases. Gaps were negotiable but there was no blame on the developers – so yep, good idea, let’s move on it. Meetings were clear. No contention.

So why didn’t everyone want good, ‘as built’ specs?

I was surprised when some companies thought they had ‘as built’ specs – only to find what they were referring to were documents an external tech writer group wrote after the software was released. Those aren’t “as built” specs as much as they are “what we think was built” specs. Since the developers didn’t code to them and QA didn’t test to them in the initial release, they are, always suspect. Those documents are never valid, correct. I would question if any company devotes enough time and money to guarantee that kind of ‘after-the-fact’ documentation is correct.

Whereas it’s so clean and clear if the spec is used as part of the development process and then as part of the QA process that it “IS” the “bible” of the code. How the code was developed. What the code does do. If you have specs like that, you know what’s in the code. Tech Support knows what the product does. Everything is clean. Clear.

But I’ve come to realize that there are roadblocks in companies to why this isn’t as easy as I assumed. But I think I have a solution and am working on it. It’s very cool. I’ve realized that there are real reasons why people feel it’s hard to maintain real as-built DOORS-type specs. I, as a manager, was willing to do the extra work to have as built DOORs specs because I saw the value. But not everyone is as dedicated to the end result.

There has to be a better way.

I think there is. We’re calling it Software 2020 – a software tool for the 21st century “:-)”

Why Software 2020?

In the ’90s we found the perfect software process and the right toolset. Our process was practical and results amazing. Leveraging managers with years of software management expertise together with very talented practical software architects, the Azerity product demonstrated that software can be done right. Azerity competed with the big gorillas (SAP, Siebel, Oracle) and won every time. And it didn’t cost the clients millions to install and millions more to upgrade. High quality, intuitive. Some said it was nearly perfect software.

What did we learn?

• One. How to build a practical architecture. One that delivers what clients want – intuitive, useful, useable. How to avoid the pitfalls which often occur when engineers aim for software elegance but end up with an architecture that is overly complex, bulky, and slow.
• Two. How to develop a software plan that includes all aspects including deployment and upgrades. We think software companies should charge for value-added services but not for avoidable deployment and upgrade costs.
• Three. 
  How to use the right processes and tools to make sure the software meets spec, is a high quality release (no bugs, that’s right, none) and completed on-time. The right software development tool to improve the process and software quality was SD Tracker.

Our process was practical

• Adopted what worked from the government SEI/CMM and 2167A
• But eliminated unnecessary process steps
• Eschewed organizational silos and waterfall ‘walls’
• Encouraged developer creativity and feedback (communications)
• Fostered accountability and ownership (“buy-in”)

Our management tools were SD Tracker and DOORs

• SD Tracker for bug tracking, enhancement requests/backlog, internal projects and our Call Center
• DOORs for Requirements Management – our Specs

Results

• Amazing productivity, good quality, happy support team, satisfied customers

Others who saw our process said we were “Agile”

• I said “Yes, we’re very agile” (little “a”)
• Later I was trained in “Agile”. I thought “humm” some good things, but some bad. Could be great with more focus on the bigger picture.

Duck Pond Software was created to provide support to software companies on architecture, product, and process.  As the 2000s have progressed and more and more teams are moving to Agile, I’m finding that Agile is helping teams move to better software development methodologies and how the ’90s processes can be updated to help support Agile teams.

But I’m also finding the current Agile tools are lacking.  That’s why we’ve started software2020.org and are developing Software 2020. A demo will be available soon. It combines the best of the Azerity processes and tools plus Agile. There’s more to come . . .