Home Page     Contact Dave Thompson Consulting    Site Map

Dave Thompson Consulting
408 Rose Drive
Allen, Texas 75002
(Dallas/Ft Worth area)

(972) 727-5670

Software Project Development Process
(A discussion for small and medium-sized business owners and managers)

 

INTRODUCTION

What is a project?

Project is a general term, used in many settings,  such as construction (building a bridge, a road, a house), education (class project), and others. Developing software is similar to construction (something is created) and the language used often borrows from the construction industry.

Programming to create software is often spoken as "building" software. And the entire effort - from start to finish - to create (build) a software product, whether custom or for commercial resale (packaged software) is frequently called a project.

Software projects may be small (limited time and cost) such as one developer/programmer working on something for a week or two, or very large (time and cost) such as multi-programmer development projects where effort is measured in programmer-months.

Categories of Software Programming Effort

Programming for business applications typically falls into categories such as:

Development/Programming Category Description
New Development Create a new application to support a business need
Enhancement Modify an existing application to add functionality or features
Maintenance Fix non-critical defects; change user interface to more accurately reflect work flow; add new lookup and reporting capabilities; minor changes to forms/user screens
Fixes Remove software defects from existing application
Conversion (version upgrade) Convert software to newer release of underlying application software such as converting an Access 97, 2000, 2003, 2007 application to Access 2010, 2013, or 2016.
Conversion (legacy software) Write a new software application, using a newer (sometimes much newer) programming language to replace an application written some years ago (typically 10 or more years old)

Thinking of software development along lines of categories is useful since this tells you something about risk, schedules, and cost. For example, new development, or converting legacy software, is a higher risk activity than enhancement or maintenance.

There is another category ... recovering from failed efforts. When software projects fall off the rails (missed milestones, costs are far beyond budgets, program managers/developers cannot coherently discuss a plan to reach completion), the decision maker can either terminate the project, or define a set of corrective actions to implement before continuing the effort.

While organizations with IT staffs typically have dealt with runaway projects in the past, the small to medium-sized business owner likely has not. Typically the project is halted.

The decision maker should make the effort to determine what went wrong, and seek to restart the project. 

Restarting a project often involves finding a new resource for programming effort.

Reality of Software Development

Writing software remains a creative process and while subject to rational management practices, some software projects do fail. It is important to keep firmly in mind that  some does not mean all. Some projects fail - but not all. Some projects are completed late and over-budget - some, but not all.

The failures that get reported do tend to be a bit spectacular (Note: These examples are from more than 10 years ago. Hint: Things haven't changed much: spectacular failures still happen.)

Published Thursday, January 13, 2005 Los Angeles Times
New FBI Software May Be Unusable
A Central Feature of the Agency's $581-Million Computer Overhaul Aimed at Coordinating Anti-Terrorism Efforts is Reportedly Inadequate
by Richard B. Schmitt


A new FBI computer program designed to help agents share information ... may have to be scrapped ... The bureau recently commissioned a series of independent studies to determine whether any part of the Virtual Case File software could be salvaged. Any decision to proceed with new software would add tens of millions of dollars to the development costs and render worthless much of a current $170-million contract...

Published March 9, 2005 New York Times (website)
F.B.I. Ends a Faltering Effort to Overhaul Computer Software
By ERIC LICHTBLAU 

WASHINGTON, March 8 - The Federal Bureau of Investigation declared an official end Tuesday to its floundering $170 million effort to overhaul its computer software and said it would take at least three and a half years to develop a new system...

Consider the frank assessment by a veteran Microsoft project manager:

Users and clients might complain if a project is delivered one month, three months, or six months late, if it’s hard to use, or if it lacks a few critical functions. But if the bulk of the planned software product is delivered at all – at any cost – ever – most software consumers will consider that project to be a success. We have suffered so many failed projects that the only outcome we really consider to be a failure is a total collapse.

Software Project Survival Guide, Steve McConnell, Microsoft Press, 1997  (Page 4)

Part of the profession for IT staffs is to deal with software development and implementation issues. However, if you're a small to medium-sized business owner, with an information technology need but without your own IT staff, the process of implementing IT solutions represents a learning curve that should be approached with some seriousness. 

Patterns of Success and Failure

Why software projects succeed or fail is widely studied. Most of the work on this topic focuses on large projects either for business applications or government use. Such projects employ teams of programmers with associated project management and effort is measured in programmer-years.

For single-developer projects such as for small business use, or large corporation, department-level applications, we can draw on the large-system studies for some general understanding. 

Two (among many) factors studied and reported on by Capers Jones  (Software Productivity Instituterelate project size and programming source (who is doing the programming/software development) to success/failure rates. Figure 1 shows this relationship in a graph adapted from the Capers Jones book, Applied Software Measurement, Second Edition, (c) 1997, 1991 by The McGraw-Hill Companies, Inc.

Graph of project success rate versus complexityClick to enlarge - opens in new window

Figure 1. Project Success versus Complexity and Programming Source

In the environment of PC workstations and desktop office suites, many end users - "power users" - may be very adept at low to medium complexity projects, but their success rate falls precipitously once a threshold complexity is reached (Complexity 100 in Figure 1 above : the power user break point).

The complexity axis in Figure represents relative numbers ranging form the very simple and obvious (Complexity = 1)  to very large, multi-year, multi-team projects at Complexity = 100K.

Independent developers (Outsource) operate in the mid-range of the Complexity axis, and larger technology firms (Outsource) operate at the top end of the Complexity scale.

Corporate IT departments operate across the full range. At the low end of complexity, end users tend to be supported by help desk persons. Beyond the capabilities of individual power users, projects move to the more formal processes for project approval and scheduling as set by the IT department. And while the IT department generally makes internal versus outsource recommendations, department managers often ask for outside consultants/developers for department level projects when the IT department is backlogged and just not able to make timely responses to department-level requests.

The small business owner, when the power user break point is reached, typically looks to outside developers/consultants.

A Medium Complexity, Multi-User, MS Access Project

As the complexity of projects increases, the amount of time spent programming falls as a percent of the total project hours. Project management time (scheduling, client meetings, formal status reporting, planning) and program quality assurance time (testing) can be - and frequently are - more important to project  success than the technical skill of the individual programmer.

But one must be cautious here - as with most generalizations - for while the lack of technical skills of the programmer can absolutely prevent a project from succeeding, the  reverse is not always true: expert programming skills do not always translate into successful projects.

The time and cost to complete a project obviously varies with complexity, but also varies by project type and the client's experience level with the process of developing custom software. Figure 2 below shows the project hours to implement a multi-user, MS Access Client Information System for a financial consulting firm.

Chart of project hours


Fig. 3. Network Architecture

 

Stage 1 (40 billed hours)

The purpose of Stage 1 was to reach a go/no go decision based on a projected cost and schedule. But to provide this, it was first necessary to reach a consensus on:

As Stage 1 progressed, and the client understood the need for focus and direction before proceeding. Before continuing,  the client decided to conduct an internal survey of their user community for suggestions and requirements.

Stage 2 (0 billed hours)

The client conducted an internal audit of their user community. Results were consolidated, categorized (must have, nice to have, future capability), prioritized and distilled into a written document.

Stage 3 (159 billed hours)

The primary programming effort spanned about two months. This stage included interim releases sent to client for review. Also the client began to load data,  such as their customer base, into the application.

Note: One of the issues for new software systems is front-end-loading. For example, a client information system would not be useful for some time if every time you needed to look up an existing client, you had to stop and enter them into the system. The data for most businesses falls into two groups: static and dynamic. Static doesn't imply that the data never changes, but it changes rarely. Thus it is practical to load existing client data (names, addresses, etc) into a new application in the weeks preceding cutover (or the day that the new software will be put in use for all users).

Dynamic data will begin flow into the system once it goes into production. For a client information system clearly contact notes would represent dynamic information, and one must consider that even basic client data (mostly assumed static) can change in the interval between entering data into the system and the date that the new system goes live (cutover into production).

Implementation issues are important, and should be considered early in the project cycle.
Dave Thompson

 

Stage 4 (0 billed hours)

With the release of version 1.0, the software was "in production" and used by the client for daily operations.

Note: As a developer, my primary responsibility is the development if the application to meet the client's expressed business needs. For multi-user environments (such as LAN's or wide area networks), I look to the client to provide network and technical support to install the application,  and to support the application user community for network and hardware issues.  I expect the client to deal with installing the commercial version of MS Access, and keeping up with Microsoft's interim releases of service packs. My job is to write the software; I look to the client (and most would have it no other way!) for network support issues.
Dave Thompson

 

Stage 5 (39 billed hours)

For a new software system, the requirements will not be completely known until the users have used it.

"requirements uncertainty principle" at pages 313-314 by Watts S. Humphrey, A Discipline for Software Engineering, (c) 1995 by Addison-Wesley Publishing Company, Inc.

During the first month of use, the system coordinator kept a list of user requests for changes and new reports. These changes were implemented in two groups during the Stage 5 follow-up. Note that in the split-mdb model (a program mdb that links to a data mdb), if the data model doesn't change (new tables, new fields, etc), then program changes can be made by the developer and the program "slip streamed" into the production environment with no downtown for the application.

 

Summary

Programming effort can be categorized by activity and this categorization affects the costs, schedules, and risks of developing, or modifying, custom software.

For new development, success follows a rather known path of developing a shared consensus between client and developer, distilling this to a written agreement, and proceeding only when client and developer have a clear view of the project and its goals. This consensus should include a written (summary format as a minimum) description of the system requirements, delineation of responsibilities between developer and client, and a specific agreement that changes to requirements once a project starts affects both schedule and costs.

It is an efficient development model to spread the effort over time, advancing from one stage to another only when ready. In the client information system described here, a large measure of the success was due to the pause (Stage 2) between the initial meetings and the start of programming. By conducting an internal user survey, the client was able not only to identify the necessary elements of a solution, but also to build a support base among users by involving them in setting the requirements for the new  software.