How to Measure Agile Development

There’s the right way to do something … andhow to measure agile development then there’s the wrong way. Unfortunately, measuring agile development is often approached in the latter method. Too many managers measure the development of an Agile project through lines of code produced or through team velocity without realizing that these measurements lead to negative consequences. In order to make sure that your Agile development project is as successful as possible, don’t make those mistakes. Follow these tips and measurement practices instead.


How to Measure Agile Development

1.    Don’t measure what’s easy

Measuring the lines of code that your star programmer has produced this past week, or team A’s velocity is easy. You count the lines of code or the number of story points that have been finished. The problem, however, is that while this data is easy to extract, it doesn’t mean that it’ll help you to determine whether or not you’re on time, on budget, or within scope.

You need to make sure that you’re delving into the data and looking at other statistics (as will be outlined below), to make sure that you are producing the quality project that your client deserves. Only by analyzing solid data will you be able to determine patterns that will ultimately allow you to take action and make your project successful.


2.    Don’t use measurements that could affect performance

To some extent, all measurements affect performance. By that, we simply mean that you have a goal, and that you take measurements to monitor your progress towards that goal. If your team isn’t where you want them to be, you push them harder to get to where they should be, right?

While it is important to stay on schedule, using measurements such as team velocity or an individual’s lines of code to judge progress is not going to help. In fact, it hurts you more often than not because it alters your programmers’ performance. Instead of writing quality code, they start producing a large quantity of lackluster code. It’ll get ugly quickly.

So, while you may need your team to work a little harder, don’t use these measurements as justification unless you want your team’s quality of work to decrease.


3.    Use measurements that will tell you about the project

If you shouldn’t being using measurements that will cause your team to change their performance, what sort of measurements should you be using? Easy. It’s all about using measurements that will tell you about the quality, time schedule and value of the project, as well as the quality of your team’s productivity.



I. Measuring Project Quality


1.    Automated Tests

Running automated tests, and counting the number that your program passes over time will tell you about the quality of code. The higher the number of passed tests, the better the code. Note however, that for this metric to be reliable, the tests themselves have to be good. If you are using bad test cases, then the number of passes will be high, even though your code isn’t great. Make sure that you’re using good test cases if you’re looking to use test code pass rate as a project quality metric.

If the program isn’t passing these  quality tests, you know that your team needs to work on the quality of their code in order for it to meet user requirements as well as project deadlines. Talk to them about code quality and improving it if this is the case.



2.    Defect Counts

Track the number of post-sprint defect arrivals and the number of post-release defect arrivals. How these numbers increase or decrease over time will tell you how your team is working to improve program quality. If they’re succeeding in getting rid of these bugs, the project quality is improving.



II. Measuring Time Schedule


A Burndown Chart

On a “Burndown Chart,” the amount of work left at the beginning of each iteration is illustrated with story points on the y-axis, and iterations on the x-axis. While the ideal trajectory would be straight-downwards, this isn’t always the case. Increases in project scope may cause a team to start an iteration with more story points than they had at the end of the previous iteration.

At the end of the day, a Burndown Chart illustrates how quickly a team is moving, and allows management to estimate how many deliverables the team will be able to present to their client within a given time. The chart gives tangible evidence as to scope of the project, including scope creep, and allows for a better estimate as to whether the project will meet it’s requirements within the given time frame. In short, it measures whether the project will be on time or not.



III. Measuring Project Value


It’s hard for a client to know the ROI of a project that is yet completed. That’s why it’s important to know how valuable the project is to the client – whether or not it’s meeting their expectations or not.

One way of measuring the project value is through a customer satisfaction survey. Asking your clients to fill out a questionnaire allows you to understand where you need to improve your team/the project. A customer satisfaction survey allows you to understand whether or not your software development is on track with client expectations.


IV. Measuring Team’s Productivity


1.    Measuring Technical Debt

Technical debt is the amount of work that needs to be done before a section of code can be considered done. Java Sonar explains measuring this technical debt by looking at duplications, violations, coverage, complexity, and comments and by then assessing costs as it finds these errors in the codebase. This cost assessment gives you a measurement of technical debt.

If technical debt shrinks over time, then your team is writing quality code and quickly addressing code issues. You have a great team! If technical debt only increases, however, your team is writing inferior code and is unable to keep up with the debugging process. Their code writing and rate of debugging needs to be addressed.


2.    Counting the Number of Stories in Progress

Collaboration is essential in Agile, so as an Agile team, your members should be (with some leeway) working together until a story is done. This collaboration allows for a higher output, which is why you need to keep track of the number of story points your team is working on.

High numbers show that your team is failing to collaborate, so you’re shooting for low numbers. Many scrum masters will shoot for a number lower than 2 story points in progress.



In order for Agile development to work for you, you need to make sure that you’re measuring more than just the easy stuff, the kind of data that can change your programmers’ performance for the worse. Instead, you need to focus on analytics that tell you about your project’s quality and ability to produce results, as well as about your team’s productivity levels. If you do those things, then you’ll be able to gain insight from the analytics that will ultimately allow you to make the best decisions. And that, my friend, is how you measure Agile Development.


How do you measure software development in an Agile environment? Let us know in the comment section or join the conversation on Facebook, Twitter, Google+ and LinkedIn.


Looking for more information like this? Check out other blog posts on this topic by clicking on the buttons below:

IT Consulting


Thanks to Ben+Sam for the use of their respective photograph. Information for this article from TechRepublic, ScrumSense, and Java Sonar.