Every Engineering VP and Portfolio Manager that I've worked with has had their preferences on what they want tracked on their projects. This is fair -- every organization is unique and each has its own areas that need more and less attention. I've known some who only care about one number: the delivery date. Others have gone into minute detail, tracking changes in lines of code per project hour across their portfolio.

There has to be a balance, of course. Status reports need to be lightweight enough that project managers will provide them consistently. But they need to be thorough enough to highlight problems before they occur. So what do you really need to track?

Here is a list of the seven essential considerations that executives need to track, at some level, in their software projects. For each of these, you will need to judge the depth of tracking that your projects require. But even if you only need a one-line response, you must ask these seven questions regularly.

#1: Scope

The first is scope: Will the team deliver what they planned? This is difficult to quantify, so it can be hard for executives to get a quick overview of which requirements in a project will deliver. If a project has many small requirements, one option that I've found to work well is to track the number of requirements completed versus the number outstanding. For a fewer number of large requirements, we either reported each requirement individually, or decomposed the requirements into smaller, more measureable units. In both of these cases, we typically compared what has been accomplished to what was planned by this point in the project.

#2: Schedule

Will the team deliver on time? Project schedule typically gets a lot of attention in status meetings. However, the behaviour I've frequently seen is that the project manager reports that the project is on schedule, until near the end of the project, when the plan starts slipping out of control. Why does this happen? Usually, the truth was that the project was never really on schedule, but the team didn't realize this until it was too late. The project team needs to re-evaluate its estimates on a regular basis. That way, when problems start to happen, they are exposed, and you can respond immediately.

#3: Cost

Will the team deliver on budget? In teams' rush to meet their schedules, project cost is an item that often gets neglected. After all, the easiest thing to do when a project starts slipping is to put more people on it. But before you do this, you need an honest assessment of whether future revenues justify increasing the investment, and whether the investment would be better spent on a higher return project.

#4: Quality

Is the team meeting their quality goals? This is an easy one to measure, but the challenge here is to define what their quality goals should be. I worked with one team that established an aggressively high quality target, but that target was an arbitrary number, and wasn't based on what the customer really needed. The result was a continually slipping release, even though the release would have been good enough for the customer. As with project scope, understand what your customer really needs, and target that.

The objectives above are traditional objectives that most project managers track. The considerations that follow are less intuitive, but are critical for software projects.

#5: Risk

Has the team identified and mitigated their greatest risks? Most software projects fail because of unforeseen problems. Risks come from many sources, including assumptions that turned out to be incorrect, and unexpected tasks that were required but not planned for. Even a single unmitigated risk, if its impact is serious enough, can shut down the largest projects. Risk management is a very large topic in itself. However, in the very least, make sure your project managers spend time, on an ongoing basis, identifying and mitigating their projects' highest exposure risks.

#6: Dependencies

Has the team integrated other teams' deliverables? Dependencies on other teams is the other key source of failure in software projects. The Project Management Institute's Guide to the Project Management Body of Knowledge (PMBOK) calls this objective, "Procurement". The danger here isn't just that the other team may not deliver on time. A much more common occurrence is that the deliverable doesn't behave quite as expected, causing significant delays during integration. Agile teams that I've worked with mitigate this problem through "continuous integration", where builds are integrated throughout the development cycle, rather than simply integrating releases near the end of the project.

#7: Culture

How did the team achieve their other project objectives? Was the team motivated to excel or was there fear and resentment? This defines the culture of your team and of your company. This is crucial, because in a knowledge industry, the real intellectual property isn't located in source code repositories, it's located in people's heads. Training new staff to replace employees who resigned is a tremendous cost for a project to absorb. As well, younger Generation Y employees tend to place a higher value on a company's culture than their older colleagues. The groups I've seen with a strong, positive culture have consistently outperformed those who worked in a more negative atmosphere. Despite this, how the team met their objectives is the most overlooked project objective. Not only is it difficult to measure, but the impact of a negative culture is often not felt until the next project. Identify the behaviours your company values, and reward those project managers who consistently display them.

Let me know what you think...

I hope this gives you some useful ideas on how to get better oversight of your projects. In my newsletter, we'll investigate these and other software project management techniques in greater detail.

Let me know what you think! There is never a single solution to any problem, and I've seen some truly unique ways to consistently deliver successful projects. How did you do it? Of course, if you have any questions, or if you would like more information on how to implement these or other software development processes in your organization, please feel free to contact me at Charles@CharlesConway.com.

If you know of someone who may find this article of interest, please forward it on!

Good luck!
Charles