POL00093654
POL00093654
BIRD & BIRD
90 Fetter Lane
London EC4A 1JP
Our Ref: HRS/msp/BPOCL-001
Your Ref:
4 May 1999
Legally Privileged & Confidential
119 London
Web Page
Ms Patricia Kelsey ww.wobirds.com
BA/POCL Programme
Third Floor oes
Terminal House GE Camps
52 Grosvenor Gardens OM Getheie
LONDON SW1W0AB tee
\ “DW Byam-Cook
‘ G JH Smith
Ker
Macdonald
PROJECT MENTORS’ SUPPORTING REPORTS OMCSine
Cw Rees
As previously promised, Project Mentors has now completed three supporting Reports ha Sondten
which I now attach. These support their Management Summary Report, which I eae
circulated last week. caren mate
NT honking
RM Bickersat
As in the past, the attached Reports are legally privileged on the basis that they were Skvooning
commissiond by us as the Joint Programme Lawyers. Accordingly, they should be TCC Tether
given the most limited possible circulation within your organisation on a need to know HE Pearson
: VSACroak
basis. TROD Asserson
J Stannard
. CiR aren
brane A teva ty DCI Cook
ta IMGyngei
MR Hae
C Fowl
AJ Sanderson
HJ Rubin
wBaker
PR Brownlow
HAMISH SANDISO!
10 Hunter
FAReeve
1 Sires
PC Dally
Encls RH Butteworh
NS ® Blundett
RW Faween
ce. Vince Gaskell, BA IP Hordies*
Sarah Graham, DSS A Maqua®
Dave Miller, Horizon cores
Ron Powell, DSS Canes stants
Paul Rich, POCL SNI Chalton
Jeff Triggs, Slaughter and May P Dana
RE Fawcett”
Dri N Walden”
KAB\BPOCLWOOlLETTERS\KELSEY £48"
fs
} “not a solicitor
Ms et EC Office 15 rue de la Loi, 1040 Brussels, Belgium Telephone
INVESTOR IN PEOPLE Hong Kong 20/F Printing House, 18 Ice House Street, Central, Hong Kong Telephone {
Pawocat registered
in Brussels
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
REPORT
PATHWAY SYSTEMS DEVELOPMENT PROJECT
SYSTEMS DEVELOPMENT APPROACH
Ref: A.41.06 V1.9
Status: Provisional
Prepared: April 1999
Prepared by Project Mentors Lid., acting as sub-contractors to Bird & Bird, Solicitors
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Pathway Systems Development Project - Systems Development Approach
SECTION
1 INTRODUCTION
1.1 Context
1.2 Scope of this Report
1.3 Purpose
1.4 Constraints
1.5 Structure of Report
2 MANAGEMENT SUMMARY
2.1 Good Practice
2.2 Pathway’s Approach
2.3 Conclusions & Impacts
3 THE SYSTEM DEVELOPMENT LIFE CYCLE
3.1 Introduction
3.2 Components of SDLC
4 SYSTEM DEVELOPMENT METHODS AND SSADM
4.1 Background
4.2 SSADM
4.3 SSADM Life Cycle Coverage
4.4 SSADM Principles
4.5 Overview of SSADM
4.6 SSADM Project Management
5 QUALITY ASSURANCE
5.1 Quality Assurance Methods
5.2 Program Testing
5.3 System Testing
5.4 Acceptance Testing
5.5 Live Running
6 PATHWAY'S APPROACH
6.1 Introduction
6.2 SSADM Stage 3 Primary Products
6.3 Detailed Required System Logical Data Model
6.4 Detailed Required System Data Flow Model
PAGE
oo & aw
S
10
10
10
12
12
13
14
14
15
18
19
19
19
19
20
21
22
22
23
24
25
Prepared by Project Mentors Ltd., acting as sub-contractors to Bird & Bird, Solicitors
i
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contempiation of litigation.
Pathway Systems Development Project - Systems Development Approach
SECTION PAGE
6.5 Detailed System Functions 26
6.6 Processing Specification 27
6.7 User Job Specifications 28
6.8 Enhanced Requirements Catalogue 29
6.9 Quality Assurance 30
A. APPENDIX - SSADM DELIVERABLES 33
DOCUMENT CONTROL 36
Change History 36
Changes Forecast 36
Distribution 36
References 37
Prepared by Project Mentors Ltd., acting as sub-contractors to Bird & Bird, Solicitors
ii
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
1 INTRODUCTION
as Context
This report is one of a set of related reports commissioned to support
potential litigation and negotiations for settlement of that litigation with respect
to the Card Payment Programme. The structure of this set of related reports
is illustrated in figure 1.1 below, which highlights the position of this report
within that structure.
Pathway Systems Development Project
Management
Summary
Ref A.41.08
Project Systems
Management Development
Approach
Ref A.41.05 Ref A.41.06
Requirements
Review
Ref 4.41.07
Figure 1.1_- Structure of Related Reports
This report describes our findings with respect to Pathway’s development
approach, while the “Project Management’ report [Ref. A.41.05] sets out our
assessment of their management of the project. The “Requirements Review’
report [Ref. A.41.07] provides additional technical information. The overall
findings of our review are set out in “Pathway Systems Development Project -
Management Summary’ [Ref. A.41.08].
It should be noted that there are links between the management and
technical factors, and ideally this system development approach report
should be read in conjunction with its companion papers.
Prepared by Project Mentors Ltd., acting as Ret: A.41.06 V1.9 Date: Aprit 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 1 of 87
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
1.2. Scope of this Report
The Card Payment Programme calls for the provision of a wide range of
services by the CONTRACTOR (Pathway Ltd.) for the Benefits Agency (BA)
of the Department of Social Security (DSS) and Post Office Counters Limited
(POCL). Three related agreements define the rights and obligations of the
parties in providing these services. The agreements are between Pathway
and the DSS and POCL jointly (the Authorities agreement) and between
Pathway and the DSS and between Pathway and POCL.
The services to be supplied under the agreements are Development
Services, Rollout Services, Steady State Services, Management Services,
Contingency Services and Transfer Services. This report, and_ its
companions, are concerned only with the development services, which
include:
_~ « PAS - Payment Authorisation Service;
« CMS - Card Management Service;
« BES - Benefit Encashment Service;
« APS - Automated Payments Service;
« EPOSS__ - Electronic Point of Sale Service.
A number of other development services are identified in the relevant
agreements or schedules to those agreements as contingency, optional or
additional services. Apart from the Order Book Control Service (OBCS),
which is a POCL optional service, none have been covered in this review.
The services could not be supplied in any way which met the expressed
requirements of the Authorities without the support of computer based
systems. Therefore the systems are fundamental to the services provision.
The related agreements clearly place responsibility with Pathway for the
provision of these systems, together with the necessary interfaces between
them and other DSS and POCL systems. Appendix A of our report “Pathway
Systems Development Project - Project Management” [Ref. A.41.05] lists the
relevant clauses and schedules supporting this allocation of responsibility.
Under the related agreements, Pathway has the authority to determine
whether each system can be an existing (packaged) system, a modified
package or be developed from scratch. However, whichever route is chosen,
the end results must meet the requirements of the Authorities and support
the solution defined by Pathway.
We describe the provision of these systems as the development project,
shortened to the project, in contrast to the overall provision of the services
which depend on them. The systems which the project will provide we have
collectively called the system. We have not reviewed the service provision
aspects of the Card Payment Programme because the system has yet to be
completed.
Prepared by Project Mentors Ltd., acting as Ref.: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 2 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
POL00093654
POL00093654
1.3 Purpose
This report has been commissioned to support potential litigation and
negotiations for settlement of that litigation. It therefore aims to provide
negotiators, who may not be wholly familiar with IT developments, with an
assessment of Pathway’s performance in this area.
Because people without a systems background may use the report, we have
striven to provide a view that we trust will be readily comprehensible. We
have as far as possible avoided the use of IT ‘jargon’. However, the
concepts described in this report need to be understood if the scale and
impact of failures in following good systems development practice are to be
recognised.
1.4 Constraints
Currently (April 1999) we do not have access to Pathway documents or to
their staff. In consequence, our findings are based on:
* relevant items in the Authorities possession;
* the absence of material which it might reasonably be expected Pathway
would have prepared and passed to the Authorities;
* our assessment of the causes behind various events in the project’s
history.
1.5 Structure of Report
The report is structured into the following chapters:
* Chapter 1, this chapter, presents background information;
¢ Chapter 2 is the management summary;
* Chapter 3 describes the general steps in the life of IT systems - the
System Development Life Cycle;
* Chapter 4 describes how IT professionals have designed methods for
successfully developing IT system. Particular focus is given to SSADM;
* Chapter 5 describes the principle stages in assuring the quality of
software developed during an IT project;
« Chapter 6 compares Pathway's approach against that of SSADM, and
reviews Pathway’'s approach to quality assurance;
* Appendix A provides an outline description of the primary products from
an SSADM project.
Prepared by Project Mentors Ltd., acting as Ref: 4.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 3 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
24
2.2
MANAGEMENT SUMMARY
Good Practice
Most Information Systems use a similar development and maintenance
Process, usually known as a System Development Life Cycle (SDLC). They
are not all developed and operated in the same way, but they mostly pass
through the similar phases in their lifetime. Managing the development of an
IT system is inherently difficult. Maximising the possibility of a successful
outcome requires a clear understanding of the requirements to be met and
the process and techniques to be used during the project. The IT industry
has evolved and developed, sometimes through painful experience, a general
approach that is considered ‘good practice’. This approach has been
embodied in a number of ‘Structured Systems Development Methods'
(Structured Methods).
Structured methods consist of two basic elements:
1. A default structure of steps and tasks which the project team should
consider following.
2. A set of techniques to be applied in each step that provide structured
definitions of user requirements and system components.
Structured methods move the project forward through analysis and design,
using techniques that gradually transform user requirements into a system
specification, from which the system's software can be developed and tested.
In our opinion, it is generally accepted that IT projects following good practice
will adopt a structured method.
Good practice also demands that the quality of the developed system is fit for
purpose. Formal Quality Assurance should be an integral part of a project's
development approach. Documents will be reviewed and approved by
interested parties, while software will be tested against its specification.
Testing will be performed by both developer and business users to
demonstrate correctness and acceptability.
Pathway's Approach
2.2.1 Evaluation Framework
In examining Pathway's approach to the development of the Card
Payment system we have had the following questions foremost in our
minds:
* what structured method has Pathway used during the project?
¢ what system development activities have been performed and
what have been the products of those activities?
POL00093654
POL00093654
Prepared by Project Mentors Ltd., acting as Ret: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 4 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
* what quality assurance approach has Pathway adopted to ensure
that the quality of documents and software meets the Authorities
needs?
We would have reviewed Pathway's own description of their
development approach. However, Pathway has not provided the
Authorities with such a description. We have therefore had to infer
their approach from the documentation we have examined.
In illustrating Pathway's approach we felt it appropriate to compare
their development steps and products with one of the well-accepted
commercially available methods. Structured Systems Analysis and
Design Method (SSADM) is the standard Information System
development method for UK government projects, and has become a
de facto standard for much of the UK private sector. We therefore
chose SSADM as our comparison template. However, it must be
emphasised that another structured method would have served
equally well.
SSADM covers most of the system life cycle from feasibility study
through to system design. We compared stages of the Card Payment
project with those of SSADM identifying the relevant responsibilities of
Pathway and the Authorities. The following diagram illustrates the
SSADM stages and responsibilities
Authorities Pathway’s
Primary
Responsibilities
Stage 0
Feasibility
Construct
Products
Stage 1
Investigation of Current Environment
Provide Business Understand
Knowledge & Stage 2 Requirement
Select Solution Business System Qotions & Propose
Solution
Provide
Detailed { Stage 3 } Analysis
Business Definition of Requirements Leadership
Knowledge
Review Stage 4 Stage 5
Technical Systems OptionsI Logical Design
=e = Construct
Products
Stage 6
Physical Design
Figure 2.1 - SSADM Stage Responsibilities
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 5 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
2.2.2 Stages0&1
The work performed by the Authorities during the period up to the
publication of the Statement of Service Requirement (SSR) “Bringing
Technology to Post Offices and Benefit Payments: Statement of
Service Requirement - March 1995 Final version 6” [Ref. 3] was
equivalent to SSADM stages 0 and 1 and was their sole responsibility.
2.2.3 Stage 2
In broad terms, this was the period of demonstration, ITT production,
vendor selection and contract production. The vendors each
proposed a solution and the Authorities selected Pathway's. During
this period Pathway had ample opportunity to clarify requirements to
the degree necessary to select and design an appropriate solution
and to plan its development. However, there is little evidence that
Pathway attempted to understand the Authorities requirements
beyond the level expressed in the SSR and contract. In particular it
appears that Pathway did not begin to build the foundation for the
detailed analysis that should have followed contract signing.
It should be noted that although Pathway's solution matched the
contract requirements catalogue point for point, the catalogue
requirements are not at a level of detail from which a system
specification and design can be developed.
2.2.4 Stage 3
The equivalent of SSADM Stage 3 began immediately after contract
signing. This stage should have focused upon agreeing, in precise
detail, what the Pathway service was to deliver. Pathway’s
responsibility was to ensure that it had a clear and precise definition of
what the Authorities required. The Authorities responsibility was to
provide ail necessary information in response to Pathway's requests.
Leadership should have come from Pathway since this stage uses
techniques which require IT specialist skills, rather than the business
skills and knowledge held by the Authorities.
Pathway did not produce a detailed analysis of the Authorities
requirements. lsolated problem areas, such as _ Foreign
Encashments, have received particular attention. However, there is
no coherent detailed requirements specification. In particular the
following critical products have not been submitted for agreement:
° an agreed detailed definition of the Authorities' business data;
* an agreed detailed definition of the Authorities’ business rules;
* an agreed detailed specification of the Pathway solution and how
this meets the Authorities' business requirements.
Prepared by Project Mentors Ltd., acting as Ret: A.41,06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 6 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
2.2.5
2.2.6
Stages 4,5&6
While SSADM Stage 3 is concerned with what has to be delivered,
Stages 4, 5 and 6 are related to design and development, and thus to
how the service is to be delivered. The how is Pathway's
responsibility and Pathway has been most protective of their right to
design and develop their solution. The Authorities responsibilities
should have been limited to the review and approval of detailed
design where it has a direct impact upon the Authorities - for example
the appearance of the counter clerk's screen.
Quality Assurance
Since there is no requirements specification, Pathway's testing of the
system can only demonstrate conformance with the contract
requirements catalogue and with Pathway's own _ internal
specifications. It cannot demonstrate conformance with the
Authorities’ detailed requirements, since these have not been
documented.
2.3 Conclusions & Impacts
2.3.1
Conclusions
From our analysis we conclude that Pathway made no attempt to
undertake requirements analysis in accordance with normal industry
practice. This despite their having access to the SSR and subsequent
requirements since April 1995. Much of this work could, and should,
have been done during the demonstrator period.
In more specific terms, we conclude that:
« Pathway failed to satisfactorily analyse the Authorities’
requirements during the procurement process and as a result
significantly underestimated the effort and time required to
develop their solution;
« in the period since contract signing Pathway has failed to analyse
satisfactorily the Authorities’ detailed requirements. As a result
Pathway has designed and partially built a system without knowing
whether it fully meets the Authorities' requirements;
« Pathway has failed to employ ‘good practice’ techniques for
establishing detailed requirements, apparently in breach of Clause
702 of the Authorities Agreement.
POL00093654
POL00093654
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 7 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
2.3.2 Impacts on the Programme to Date
Without having completed an analysis of the business requirements
set out in the contract, Pathway can never have been in a position to
understand the detail of the those requirements. Without that detail it
is not possible to develop a system level specification to bridge the
gap between requirements and program specifications. Without such
a system specification it is difficult to design programs in a way which
ensures that all system requirements are catered for.
“Optimisation”
Failure to complete requirements analysis and the consequent lack of
detailed understanding of what is required is, we believe, at the heart
of Pathway’s complaints that the Authorities are “seeking to optimise
the system”. What they see as optimisations are in reality the
detailing of the business requirements. A competent analysis would
have identified these:
* more comprehensively;
* earlier in the project, giving all parties more opportunity to
consider and agree options;
* at a point where they could have been incorporated into a
coherent design at minimal cost.
Estimating and Planning
Requirements analysis is fundamental to preparing estimates of the
effort required to develop software. Without estimates it is not
possible to establish resource requirements, nor to develop a soundly
based schedule. The lack of requirements analysis meant that
Pathway's estimates had no firm foundation. The continuing delays to
the project schedule reflect this.
Other Elements of the System
We have performed a high level analysis of the requirements for the
BPS and EPOSS system elements, not for other parts of the system.
Nevertheless, we have grave concerns that the lack of professional
analysis we found in the BPS and EPOSS elements would also be
found in other areas if the necessary work were undertaken. These
concerns are supported by a number of interviews with Authorities’
staff, from which it is apparent that Pathway are loath to release
design documents to BA/POCL. While they have on occasion cited
Intellectual Property Rights as a reason for refusal, we suspect that
the right level of documentation has not been developed.
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
rovisional Page 8 of 37
sub-contractors to Bird & Bird, Solicitors Status:
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
2.3.3
Impacts on the Programme in the Future
Our experience of systems where requirements have not been
analysed satisfactorily is that the system fails to support users’ needs.
An effective acceptance test will identify many such failings,
necessitating rework. The result is an extension of the time and cost
to complete the system and roll it out. The alternative is to allow
unacceptable processing in the operational environment, with
unpredictable and potentially damaging results.
In our opinion, Pathway’s failure to analyse the requirements
satisfactorily presents a high risk that users’ needs will not be met fully
by the system so far developed. This may necessitate significant
rework as testing continues with a consequent, but unpredictable,
extension of time-scales.
POL00093654
POL00093654
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 9 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
3 THE SYSTEM DEVELOPMENT LIFE CYCLE
3.4 Introduction
Most information systems share a common development and maintenance
process - usually known as a System Development Life Cycle (SDLC). This
is not to say that they are all developed and operated in the same way, but
that they will all pass through the same basic phases in their lifetime.
Strategy
Planning
Af
Feasibitity
ee
Analysis
AE
Design
Ewe
Software
Development
SE
limplementationI
3
Maintenance
oa
Figure 3.1- Basic System Development Life Cycle
Figure 3.1 above illustrates one version of this life cycle. There are many
variants, some sub-dividing these phases into smaller units, others merging
them. However, all follow a similar structure.
3.2 I Components of SDLC
3.2.1 Strategy Planning
In most organisations there will be a formal mechanism for deciding
which areas of the business require new or enhanced computer
systems. This may be referred to as strategy planning or something
similar, but will always involve assessing the relative priorities of
different areas, with a view to initiating one or more development
projects.
Prepared by Project Mentors Ltd., acting as Ref: A41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 10 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
3.2.2
3.2.3
3.2.4
3.2.5
3.2.6
3.2.7
Feasibility Study
Before system development begins in earnest it is often appropriate to
establish its feasibility, although sometimes there is no choice in the
matter; organisations may have to provide computer support to
remain competitive or to comply with legislation. The ideas that come
out of strategy planning are often vague and untried. Some
assessment of their feasibility needs to be carried out before too
much time and effort is spent on projects that cannot be cost justified
or that are technically impossible.
Systems Analysis
Once the project is underway the first task is to establish the
requirements of users, and hence of the business. At this point some
idea of the eventual shape of the system may exist. However, the
primary aim is to concentrate on what it should deliver, rather than
how it should deliver it.
System Design
This is the translation of the user requirements gathered during
systems analysis into a computer system design. It details exactly
how the requirements will be satisfied.
Software Development
The system design provides a blueprint for developing and testing the
new system's software, whether this is a modification of an
established package or a bespoke development.
Implementation
It is during implementation that programs and hardware are installed,
and users trained and assisted in moving over to using the new
system.
Maintenance
This is often referred to as the production or operational phase, and
covers the period when the system is up and running in support of the
business. It is the period where the system will need to be enhanced
in response to changing business requirements. Any errors
discovered in the system will also need to be rectified.
POL00093654
POL00093654
Prepared by Project Mentors Ltd., acting as. Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 11 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
44
SYSTEM DEVELOPMENT METHODS AND SSADM
Background
In the early days of systems development (1960s and 1970s) each individual
developer or project team would devise their own method of moving through
the life cycle, often influenced by hardware and software considerations.
This did not always lead to the best system design or to the easiest of
systems to maintain. Most of the problems with systems developed in this
way were due to poor communication of ideas, between users and
developers, or between a system's designers and its maintainers, and to lack
of rigour which led to errors and omissions.
Initial efforts in overcoming these problems were directed at the programming
or implementation end of the life cycle. Traditional programming methods
tended to result in code that was understood by the person who wrote it, but
was virtually incomprehensible to anyone else. This meant that it was difficult
to maintain or to identify and remove residual errors (debug). Structured
Programming methods were developed to overcome this problem by
providing a series of steps and diagrammatic program design techniques to
ensure that code was:
e well structured and easy to follow;
° based on consistent and effective design principles;
* accurately specified.
Their structured nature also meant that programming activities could be
managed and quality controlled more effectively.
While this resulted in greatly improved program quality, it did not address the
whole development life cycle. There is no point in producing well designed
and coded programs that fail to provide the facilities that users need to
support the business. So attention turned to the earlier phases of the life
cycle and in particular systems analysis and design. The result has been a
number of 'Structured Systems Development Methods’ (Structured Methods).
Structured methods consist of two basic elements:
1. A default structure of steps and tasks which the project team should
consider following.
2. A set of techniques to be applied in each step that provide (largely
diagrammatic) structured definitions of user requirements and system
components.
Many claims have been made about the effectiveness and benefits of
structured methods. The main claim is that systems more closely match the
needs of users because user requirements have been better understood and
communicated from the outset. This is achieved by applying rigorous
techniques that force analysts to examine the business problem thoroughly,
and by improving communication through the use of structured diagrams that
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 12 of 37
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
are easily understood and less ambiguous than text. By providing a firm
structure of steps and checkpoints throughout the life cycle, projects can be
carefully managed and personne! skills can be used effectively.
Structured methods move the project forward through analysis and design,
using techniques that gradually transform user requirements into a system
specification, interrelating and cross-referencing with each other. This
ensures that information is not lost.
It is also claimed that structured methods reduce the costs of a system, but it
must be stressed that this is over the entire life of the system and not just in
the analysis and design phases. It is precisely because more effort is spent
during analysis and design that the resulting system should require less
maintenance (due to higher quality, greater flexibility and fewer errors).
Bearing in mind that maintenance may account for 70% of the lifetime cost of
many systems, savings in this area are likely to outweigh increased costs in
other phases.
Finally, most structured methods are self-documenting. They produce a
comprehensive description of the system and how it operates as part of the
analysis and design process. This removes the need to set up a separate
documentation task once the system is in production (a notoriously onerous
task), and again ensures consistency.
The increasing need for the development of systems with sophisticated
screen interfaces has led to the development a number of structured
methods which focus upon evolving the optimum ‘user interface’. Although
known as ‘Rapid Application Development’ (RAD) methods, they still follow
the basic system development life cycle. They demand early development of
‘screen’ software that is reviewed by potential users, modified to take their
comments into account, and presented for further review. This cycle may be
repeated several times. RAD is also used for developing systems to support
new business ventures where detailed requirements are not yet understood.
However, where the new system is to support an established business
environment, the detailed business requirements must still be formally
analysed prior to software development.
The UK Government has adopted structured methods as a standard for all IS
projects. Structured Systems Analysis and Design Method (SSADM) is the
standard Information System development method for UK government
projects, and has become a de facto standard for much of the UK private
sector. It also forms the core of numerous courses at HND, BSc and MSc
levels.
4.2 SSADM
SSADM is a well-tried and comprehensive method. It starts by building a
high level picture of system requirements that are gradually refined so that a
detailed and rigorous system design can be produced. This is achieved
through the careful and staged application of a range of standard techniques.
The structural framework of SSADM controls the whole process.
Prepared by Project Mentors Ltd., acting as. Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 13 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
4.3 SSADM Life Cycle Coverage
SSADM covers most of the system life cycle from feasibility study to system
design (see Figure 4.1 below).
Strategy
Planning
Feasibilty Study
Requirements Analysis
‘Requirements Specification
Logical System Specification
ZUran
Physical Design
Figure 4.1 - SSADM Life Cycle
4.4 SSADM Principles
4.4.1 User Involvement
On too many occasions in the past users’ requirements have been
lost or overridden by IT analysts and designers. It is a prerequisite for
SSADM that user commitment and involvement are agreed right from
the start. Most of SSADM's techniques can be taught to users without
any prior experience of IS development. They can then participate
fully in the project.
4.4.2 The Three Views of SSADM
SSADM looks at a system from three different, but highly
interdependent perspectives:
¢ The first is that of functionality or processing. This looks at the
way in which data is passed around the system and the processes
or activities that transform it, i.e. it sets out the functions provided
for users by the system.
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 14 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
e The second is that of data. An Information System exists only to
store and act upon an organisation's data. By understanding the
true nature and structure of the data we get to the real heart of the
system. Data structures are far more constant than processing or
functions, which tend to change fairly frequently; so it is the data
view that forms the backbone of SSADM. In this sense SSADM
belongs to the family of structured methods referred to as ‘data
driven’.
«The final view looks at the effects of time and real world ‘events'
on the data held within the system. Whereas the function and
data views are rather ‘snapshot’ in nature, the event view is
dynamic; it is specifically designed to model system behaviour
over time.
4.4.3 Separation of Logical and Physical Models
One extremely important concept in SSADM (as in most rigorous
methods) is the distinction between logical and physical views of
system components.
Physical components are those that actually physically exist within the
real world (or will exist in the future). They represent things, as they
are, warts and all, complete with constraints imposed by
organisational, political or technical factors. For exampte in the
physical world, there may be two processes associated with producing
an invoice; filling in the invoice details and calculating the invoice cost.
The reason for the separation of the two tasks may be purely
organisational, ie. the job descriptions or policy in the accounts office
dictate that one job holder (for example clerk) carries out the
description task, and another (for example accounts supervisor)
carries out the other. The fact is that there are two distinct physical
processes.
Logical components are those that represent a picture of what
underlies the physical components. In a sense they give a picture of
what physical components would look like in an ideal environment,
free from real world constraints. Using the invoice creation example
from above, the logical view would be one of a single activity: creating
the invoice. The fact that the process is split in reality is irrelevant; we
are only carrying out one ‘business’ process.
Another example of physical and logical views of a system component
might be the invoice itself. In physical terms we have a single object:
the invoice. In logical terms we have information about several
objects: the customer being invoiced; the items detailed in the invoice;
the quantities and costs of each invoice line and of the total invoice.
Prepared by Project Mentors Ltd., acting as Ref: A.41,06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 15 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
45
Overview of SSADM
SSADM consists of three main components:
e The default structure or framework of an SSADM project;
e A set of standard analysis and design techniques - The techniques are
used to develop the products of an SSADM project;
e The products of each technique - Most Tasks create or enhance a
standard SSADM product. At the end of an SSADM project the new
system will be described by the sum of these products. An outline
description of the products from each stage will be found in Appendix A.
This review has focused upon the structure and products of an SSADM
project since the techniques are of secondary interest.
At the most detailed level, SSADM describes a large number of Tasks that
may be undertaken during the project. These are structured into groups so
that they can be planned and managed successfully and their progress
monitored accurately. The lowest level grouping of interest to this review is
the Stage. This structure is illustrated in Figure 4.2 below.
Stage 0
Feasibility
Stage 1
Investigation of Current Environment!
Stage 2
Business System Options.
Stage 3
Definition of Requirements
Stage 4 Stage 5
Technical Systems Options Logical Design
Stage 6
Physical Design
Figure 4.2 - The Stages of SSADM
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors ‘Status: Provisional Page 16 of 37
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
Stage 0 - Feasibility: The scope of the proposed IS project is defined using
some of SSADM's core techniques to produce a high level overview of
Processing and data. Several options for taking the project forward to a full
SSADM study are looked at and a single option is selected on the grounds of
benefits versus costs. A decision to abandon the project might be taken if
the project is found to be infeasible.
Stage 1 - Investigation of Current Environment: in many cases the new
computerised information system will be intended as a replacement or
extension of existing systems (which may be fully computerised, entirely
manual, or a combination of the two). In this situation, the full analysis of
requirements is begun by modelling the current system with a view to drawing
out existing problems and new requirements.
Stage 2 - Business System Options: Stage 1 will have Produced a
reasonably comprehensive statement of user requirements. This is now
examined and options for solving the business problem (or a subsection of
the business problem) are assembled. Although some generic physical or
technical aspects need to be taken into account, attention will be directed
towards defining business (or logical) solutions, and not towards describing
any specific technical environment. A single option will be selected as
providing the shape and direction for detailed requirements specification.
Stage 3 - Definition of Requirements: Stage 3 lies at the heart of an
SSADM project; it is where user requirements are transformed and refined
into a detailed and precise specification of what the system is required to do.
Stage 4 - Technical System Options: Stage 4 is carried out in parallel with
Stage 5 (Logical Design). The specification resulting from Stage 3 will
provide the project team with enough information to propose alternative
technical environments to implement the system design upon. In many cases
the team will have no discretion in the choice of hardware and software, but
where there is discretion the team can draw up several options for hardware,
software and development platforms, and help management to select a
single option for use in physical design.
Stage 5 - Logical Design: In Stage 5 the system design process is taken as
far as is possible, without reference to a particular technical environment.
The resulting design will be logical in nature and so capable of
implementation on a variety of platforms. It will also act as a more or less
permanent model of how the system satisfies user requirements. The logical
Nature of the design means that it should reflect underlying business rules
and activities rather than physical constraints.
Stage 6 - Physical Design: The logical system design from Stage 5 is now
translated into a physical design based on the technical environment selected
in Stage 4. In many areas physical design is dependent directly on technical
issues specific to the chosen environment. Consequently SSADM is only
able to provide generic guidelines in these areas. In this way it covers most
but not all of the systems design phase.
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 17 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
4.6 SSADM Project Management
SSADM provides no specific project management activities of its own, but it
does require the use of effective management procedures. At the end of
most steps and stages SSADM will output a number of products for quality
assurance and project control purposes. A structured project management
method, such as the UK Government standard PRINCE/2, offers appropriate
structure for the orderly management of projects. As part of the project
management structure, SSADM assumes that a Project Board will exist to
control and agree the decisions of the project team. This board should
ideally be drawn from user and iS management.
Prepared by Project Mentors Ltd., acting as Ref.: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 18 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
54
5.2
5.3
QUALITY ASSURANCE
Quality Assurance Methods
The purpose of quality assurance is to verify that the system is of acceptable
quality. Most of the products from stages 0 to 6 of an SSADM project are
recorded as documents. Review and comment is the main quality assurance
method employed during these stages. The majority of Pathway documents
we have examined appear to have been through an appropriate process of
review.
However, the principle method used to ensure the quality of the software
developed from the design specifications of stage 6 is testing. There are four
distinct stages of testing:
* Program Testing
« System Testing
e Acceptance Testing
* Live Running
Program Testing
Each program and program component will be tested against its specification
to ensure that it behaves according to that specification. A large number of
test cases will be executed to ensure that each program handles not only the
standard conditions and values that need to be processed but also the
extreme and unusual conditions. It is particularly important to show that a
program that interacts with the users of the system will deal successfully with
data and conditions that are outside the expected range - it should reject the
data indicating the reason for rejection rather than failing with a program
error. When programs fail their tests, the code will be modified and the
Programs resubmitted for testing. Only when the programs have been
successfully ‘program tested’ should they be submitted for inclusion in the
next stage of testing.
System Testing
Sets of programs are assembled (a process often known as integration). The
full system is then tested against the system's specification. Although
individual programs may have been shown to work correctly through program
testing, when working together they may behave in an unexpected way. This
may be because the programs have been incorrectly coded, or more likely,
the system design did not take certain conditions or data values into account.
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 19 of 37
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
A substantial number of test cases will need to be constructed, and extensive
planning performed, before system testing can begin. Where there is a
significant on-line workload i.e. users at terminals or connected PCs, it is
often necessary to use special software testing packages to simulate the
large number of transactions expected during live running.
Errors detected at system testing are more expensive in time and effort to
correct than those detected at program testing. More than one program may
be affected by the change in design, with each having to be modified and re-
tested. In addition, the system will need to be integrated again with system
test cases - already executed successfully - re-run to ensure that changes
made to the programs have not introduced new errors. Only when system
testing has demonstrated that the system is of the required quality should it
be released for acceptance testing.
5.4 Acceptance Testing
Unlike earlier testing, acceptance testing involves business users. Its primary
aim is to demonstrate that program and system testing has been rigorous.
To this end, a form of statistical sampling is often employed to select test
cases. The test cases must be defined at the finest level of detail to ensure
that business rules are followed precisely. If errors are discovered, additional
test cases will be constructed to determine the extent to which system testing
has failed in error detection.
Acceptance testing also provides the first opportunity to exercise, and
possibly refine, business processes dependent upon the new IT systems
facilities and to ensure that the processes and IT facilities integrate
successfully.
Sometimes acceptance testing is used to demonstrate that the system
successfully meets the business requirements as expressed in the
requirements catalogue. However, if proper requirements analysis and
specification has been performed prior to system design, this use of
acceptance testing is not necessary. Business users will have ensured that
the requirements analysis is a true expression of the business requirement
and that the system's logical design meets the business' needs. Thus, if
system testing has demonstrated that the system specification has been
correctly implemented, acceptance testing should be no more than a short
quality assurance exercise. However, if requirements analysis has been
inadequate, acceptance testing must of necessity be an extended and
expensive exercise with much system re-design, modification or addition of
Program code with all associated program and system testing repeated.
When a system is being procured from an outside supplier, acceptance
testing is the usual method to ensure that the supplier has met the terms of
the contract.
On successful completion of acceptance testing, the system can be released
for live running.
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 20 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
5.5 Live Running
Once the business starts to use the new system to support its processes,
unexpected conditions come to light. These are circumstances that were not
foreseen by the users or IT staff during the analysis and design of the
system, despite the application of rigorous methods. Thus although not
generally seen as such, live running forms the final cycle of testing.
Errors found during live running may only be corrected through a complete
cycle of requirements definition, modification to the system design and the
necessary program modifications. These changes must be followed by the
appropriate program, system and acceptance tests to prove the changes
work as intended, and have not caused other errors. The speed at which this
must be done will depend upon the criticality of the error detected.
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status; Provisional Page 21 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
6 PATHWAY'S APPROACH
64 Introduction
Figure 6.1 below shows the SSADM stages illustrated earlier. It also shows
how the responsibilities for delivering the products of those stages were
shared between the Authorities and Pathway.
Authorities Pathway’s
Primary Primary
Responsibilities Responsibilities
Stage 0
Construct Feasibility
Products
Stage 1
Investigation of Current Environment
Provide Business = pacerstand
Knowledge & Stage 2 z Propose
Select Solution Business System Options Solution
Provide
Detailed Stage 3 Analysis
Business Definition of Requirements Leadership
Knowledge
Review Stage 4 Stage 5
— Systems Options} Logical Desi
2 = a Construct
Products
e 6
Physical Design
Figure 6.1 - SSADM Stage Responsibilities
The work performed by the Authorities during the period up to the publication
of the Statement of Service Requirement (SSR) “Bringing Technology to Post
Offices and Benefit Payments: Statement of Service Requirement - March
1995 Final version 6” [Ref. 3] was equivalent to SSADM stages 0 and 1 and
was their sole responsibility.
Stage 2, in broad terms, was the period of demonstration, ITT production,
vendor selection and contract production. The vendors each proposed a
solution and the Authorities selected Pathway's. During this period Pathway
had ample opportunity to clarify requirements to the degree necessary to
design an appropriate solution and plan its development. Indeed, Pathway
should have constructed, at a minimum, a first draft of the System Logical
Data Model and the System Data Flow Model. This activity would have
ensured that where Pathway did not understand the Authorities requirements
it could have raised the appropriate questions. Without this definition of what
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 22 of 37
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
was required it would not have been possible to identify the work required or
produce a plan for completing the project.
The equivalent of SSADM Stage 3 began immediately after contract signing.
This stage should have focused upon agreeing, in precise detail, what the
Pathway service was to deliver. Pathway's responsibility was to ensure that it
had a clear and precise definition of what the Authorities required, whereas
the Authorities responsibility was to provide all necessary information in
response to Pathway's request. Leadership should have come from Pathway
since this stage requires techniques to be used which are IT specialist skills,
rather than the business skills and knowledge held by the Authorities.
SSADM Stages 4, 5 and 6 are related to design and were, and still are,
Pathway's responsibility since they are concerned with how the service is to
be delivered. Pathway has been most insistent upon this point. The
Authorities’ responsibilities should have been limited to the review and
approval of detailed design where it has a direct impact upon the Authorities -
for example the appearance of the counter clerk's screen.
Successful completion of Pathway's design responsibilities was, and is,
dependent upon the quality of the requirements specification produced during
the period of the project equivalent to SSADM stage 3. We therefore focus
upon that period, together with an assessment of the Quality Assurance
applied by Pathway.
6.2 SSADM Stage 3 Primary Products
The primary products from this stage are:
Detailed Required System Data Flow Model
Detailed Required System Logical Data Model
Detailed System Functions
User Job Specifications
Processing Specification
Enhanced Requirements Catalogue
Because many IT development projects are tasked with replacing an existing
IT system with a new one, the two systems are distinguished by the terms
current system and required system. Stage 3 is focused upon the required
system with its products named to reflect this distinction where necessary.
In the following sub-sections we present for each product:
« Adescription of the SSADM product and the main reasons for producing
it. Each product can be composed of a number of sub-products. We
have only presented this detail where it aids comparison.
e An outline of what has actually been produced during the project and how
well this supports the underlying purpose of the SSADM product;
e The implication for the project of what has actually been produced.
Prepared by Project Mentors Ltd., acting as Ret.: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 23 of 37
POL00093654
_ POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
6.3 Detailed Required System Logical Data Model
6.3.1 Description
A Logical Data Model (LDM) describes in detail:
* the main objects of interest to the business, for example Outlets
and Counter Clerks. These are termed business entity types;
e the attributes of interest for each entity type - for example the
Name & Address of an Outlet and the /dentifier of a Counter
Clerk. The definition for each attribute includes the type of data -
numeric, character, the length of the attribute, the range of values
it can take, the meaning of coded values etc;
« the nature of relationships between entity types. For example an
Outlet may have one or more Counter Clerks working at the Outlet
and a Counter Clerk must work at one and only one Outlet.
Information on expected volumes of data is also held with the LDM.
Once again, this will be important during the design process in
ensuring performance targets are achieved.
The Detailed Required System LDM is a critical product of the
development process. Indeed, it is generally considered to be the
single most important product for ensuring a successful project
outcome.
6.3.2 Actual Products
We can find no evidence of a Detailed Required System LDM, or
equivalent, for the complete system. The POCL Reference Data LDM
has been produced by POCL. Some part of this is implicitly included
in the EPOSS logical data model. There is no LDM of the remainder
of EPOSS. The Horizon library does not contain an LDM for BPS. It
may be that CMS and PAS projects have developed one, however the
documentation of those projects is not visible to the review team.
The full required system logical data model should have been
reviewed and signed-off by the Authorities. This has not occurred.
6.3.3 Implications
Pathway will almost certainly have produced physical data models
ie. specifications of file and database records (products of later
stages). However, these are not substitutes for an LDM since:
« Without a single reference description of each entity type and its
attributes, it is likely that designers will produce different
representations of the same entity types. This can lead to failure
during formal testing or, if not detected, failures during live
running;
Prepared by Project Mentors Ltd., acting as Ret: A.41.06 V1.9 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 24 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
The nature of relationships between entity types is not explicitly
described. This makes it difficult for programs to police them and
ensure that only valid relationships are recorded e.g. each Post
Office may be the nominated Post Office for_one_or more
Customers. However, each Customer must have one and only
one nominated Post Office;
Most systems hold more than one copy of information about each
business entity; for example details of POCL products are held on
the POCL Reference Data system, the Pathway Reference Data
System and the EPOSS counter system. Without the LDM as a
foundation for the physical design, it is difficult to identify where
data is replicated and how consistency between multiple copies is
being maintained.
Business changes will occur that demand modifications or
additions to data held on the system. Without the LDM the
process of identifying the set of files and programs to be modified
becomes significantly more difficult, and thus extends the time
taken to implement the required business changes.
6.4 Detailed Required System Data Flow Model
6.41
Description
A Data Flow Model (DFM) illustrates the way in which data (or
information) is passed around the system, and how it is transformed
and stored within the system. A DFM is a set of diagrams and
supporting text describing the following:
External Entities - The people, organisations and other computer
systems that act as sources of data to, or recipients of data from,
the system - for example Counter Clerks, Help Desk staff and the
CAPS system;
Processes - These represent business activities carried out upon
and triggered by data. They should not be confused with
computer programs. A process may sometimes equate directly
with a program, but even then will be defined in user, rather than
computer, terms. In other words, it should reflect the business
activity it supports;
Data Stores - These are, as their name suggests, stores of data
within the system. They may be computerised data stores or
manual data stores (filing cabinets etc.);
Data Flows - These are arrows which represent the flows of data
to, from and within the system.
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 25 of 37
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
The expected frequency of each type of event will also be recorded.
This will be important during the design process in ensuring
performance targets are achieved.
Stage 3 should produce a fully expanded data flow model. The lowest
level (elementary) functions will be identified.
6.4.2 Actual Products
No Required System Data Flow Model or equivalent is recorded in the
Horizon library. !f Pathway had produced one it should have been
reviewed and agreed by the Authorities to be a proper foundation for
the following design stages. Indeed, it couldn't be produced without
the business knowledge of Authorities staff. We have found no
evidence that Pathway produced a Required System Data Flow
Model.
6.4.3 Implications
Data Flow Modelling is a widely used and mature analysis technique,
and can be found in most structured methods. A DFM and the related
Logical Data Model are the foundation upon which a system is
designed. Without this analysis, the designers are, at best, working in
an inefficient manner, gathering detailed requirements from users in
an ad-hoc way. At worst they are ‘working in the dark’, with their
understanding - or more probably lack of understanding - not being
revealed until testing of the system by the users.
6.5 Detailed System Functions
6.5.1 Description
Defines the logical functions that will be used when recording
business 'events' (for example - Customer Presents New Card Pick
Up Notice, Introduce New POCL Product, Balance Stock Unit etc.).
Each functional description defines the logical data associated with
the event type that will be passed across the user interface. The
precise techniques used to describe the functions will depend upon
the sophistication of the user interface (simple screen, windows
interface etc.). The user roles applicable to each function are also
defined. This is the first step in defining access rights for users.
The logical interfaces to other IT systems are also described (for
example between CAPS and PAS).
Prepared by Project Mentors Ltd., acting as Ret: A41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 26 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
6.5.2
6.5.3
Actual Products
There are Pathway produced physical design specifications of
system interfaces. These contain a mixture of the required logical
information together with design detail not relevant to SSADM
Stage 3. None of the design documents cross-reference documents
containing purely ‘logical’ views. We conclude that the latter have not
been produced.
Implications
Failure to consider the logical user interface prior to detailed design
leads to:
* Decisions about the style of user interface being made before a
complete understanding of what information is to be passed. As a
result, design re-work is likely to be required (extending
time-scales);
¢ Difficulty in separating the logical definition of user interfaces from
the physical definition i.e. separating the ‘what is to be done’ from
the ‘how it is to be done’. This raises the risk of users missing
important detail that will not be discovered until testing (extending
project time-scales), or possibly not until live running (with
potential business impact).
6.6 Processing Specification
6.6.1
Description
This is a precise and complete definition of the business rules for
maintaining each entity type defined in the logical data model in
response to each event type that drives the system. The specification
must take into account all possible states of data and how it will be
manipulated to comply with business rules. Computer programmers
will encode these rules during the detailed design stages of the
project. If there are gaps and inaccuracies in the specification, then
these will be reflected in the implemented computer system.
Since the specification is the route by which the business users
communicate the detailed business rules to computer programmers, it
must be expressed in a way that is understandable to both groups.
SSADM uses a range of diagrammatic techniques to define the
processing specification. Some other methods use Structured
English to the same effect.
Prepared by Project Mentors Ltd., acting as Ref. A.41,06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 27 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
6.6.2
6.6.3
Actual Products
We have found no evidence that processing specifications have been
produced. Only the SADD and a number of functional description
documents (such as Foreign Encashments) have been produced by
Pathway. Most have been signed-off by the Authorities. However,
none of them are at a level of precision that achieves the objectives of
a Processing Specification.
Implications
Pathway has produced computer programs without having an agreed
logical processing specification in place. Which business rules have
been implemented cannot be determined except by:
¢ Examining program specifications where they exist (Pathway do
not make this information available to the Authorities);
« xamining program code where specifications do not exist or are
inadequate detail (Pathway do not make this information available
to the Authorities);
e Testing programs to see what they do.
Only the last option is open to the Authorities (see the following
Chapter on the subject of Pathway's approach to Quality Assurance),
6.7. User Job Specifications
6.7.1
6.7.2
Description
A new IT system is usually developed hand-in-hand with changes to
business processes. The User Job Specifications describe the
required processes and the use made of the required IT system
functions.
Actual Products
POCL Business Process Descriptions describe in diagrammatic form
the process to be followed in outlets when handling ‘business events’.
However, as there is no definition of the IT system's functions
available to us, we have no way of knowing if the POCL business
processes will be adequately supported by the Pathway system.
The majority of changed BA business processes use the CAPS
system for support, and their User Job Specifications are therefore
not within the scope of this review.
Prepared by Project Mentors Ltd., acting as Ref.:
41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 28 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
6.7.3
Implications
When the new business processes are tested with the IT system
there is a significant risk that the two will be incompatible to some
degree. Either the Authorities desired business processes will need
to be modified to fit with the IT system or Pathway will need to change
the IT system to properly support the new business processes. In
either case, further delays and additional costs are likely.
6.8 Enhanced Requirements Catalogue
6.8.1
6.8.2
6.8.3
Description
Additional or clarified business requirements are often identified
during the Requirements Specification stage of a project. These are
added to the catalogue produced in the previous stages. The
requirements are a mixture of what the new system must achieve in
business terms and constraints on how the functionality is to be
delivered (performance, reliability, availability etc.)
The catalogue will be used to check that the products produced in
stages 3, 4 & 5 adequately address the business requirements.
Actual Products
The Authorities produced the Contract Requirements Catalogue.
Pathway produced matching solutions, also recorded in the Contract.
Many of the requirements are about the service rather than the
system. We have focused upon system requirements.
The contract requirements catalogue contains a mixture of system
requirement types. Some would normally be found in the SSADM
catalogue. Others are of a more detailed nature and define business
rules and would appear in other products from this SSADM stage.
Overall, the catalogue is good at defining what is required without
constraining Pathway in how the solution is to be delivered.
Controlled changes to the catalogue continue to be made as the
Authorities requirements are clarified or amended, but there have
been relatively few that affect the development of the system.
Implications
The few changes that have been needed to add to or clarify
requirements related to the development of the system supports the
view that the original statement of requirements was satisfactory.
This refutes the Pathway allegation of changing requirements.
Prepared by Project Mentors Ltd., acting as Ref.: A.41.06 V1.9 Date: Aprit 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisionat Page 29 of 37
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
6.9 Quality Assurance
6.9.1 Testing Stages
Testing the software to see if its behaviour corresponds with the
relevant specification is the primary means of assuring quality. This
applies to the individual programs that make up an IT system, as well
as the complete IT system. There are four distinct stages of testing -
program testing, system testing, acceptance testing, and live running.
The testing terminology used within the Card Programme does not
correspond precisely with this definition. From examination of the
relevant documentation we conclude that the following table
represents the equivalence between stages:
Typical Testing Stage
Card Programme Testing
Stage
Program Testing Program Testing
System Testing Acceptance Testing
Acceptance Testing (1) Model Office Testing
Acceptance Testing (2) Live Trial
Live Running Live Running
Table 6.1 - Testing Stages
6.9.2 Program Testing
We cannot comment directly upon Pathway’s program testing since
the relevant documentation is not open to our inspection. However,
from documentation we have seen, we can infer what has happened
in the past and predict what is likely to happen in the future.
Without a detailed requirements analysis, program specifications will
contain errors and omissions. Therefore, no matter how good
Pathway's program testing has been it is likely that programs will not
meet the Authorities business requirements. This will not come to
light until Acceptance Testing (Model Office & Live Trial). Further
modification of Pathway's software will be required with resultant
extension to time-scales.
6.9.3 System Testing - Card Programme Acceptance Testing
Acceptance testing is being used to demonstrate that the Pathway
system meets the Authorities requirements as expressed in the
contract requirements catalogue. Pathway has defined a series of
Business Threads, the individual steps of which are used to exercise
the system. The Threads represent potential business scenarios and
Ref: A.41.06 V1.9
Status: Provisional
Prepared by Project Mentors Ltd., acting as
sub-contractors to Bird & Bird, Solicitors
Date: April 1999,
Page 30 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird * Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
test that the system successfully handles them. There is a separate
acceptance test specification for each major functional area - BES,
OBCS, APS etc. Each specification lists the set of requirements
catalogue items relevant to the particular functional area and identifies
the business threads that will be executed to demonstrate that each
requirement has been successfully met.
The catalogue is not, however, a detailed system specification. This
lack of detail means that acceptance testing against the catalogue
alone cannot reveal the ‘correctness’ of the Pathway design. This is
evidenced by the detail of the Model Office Testing, which goes far
beyond the detail contained in the contractual requirements
catalogue. As is the case for program testing, even exceptionally
good system testing by Pathway will not, indeed cannot, detect a
significant number of errors and omissions.
6.9.4 Acceptance Testing (1) - Card Programme Mode! Office Testing
We quote from the Horizon Test Team's Model Office Low Level Test
Plan for Nile Release 2.0 [Ref. Horizon Test Team, Model Office Low
Level Test Plan, BA/POCL Nile Release 2.0, Version 1.1, 5" August
1998):
"The Model Office test phase will:
« Focus on testing the alignment between the operating procedures
and systems functionality
e Prove the interaction of procedures across all systems involved in
Nile Release 2.0 for the first time
¢ Give assurance that the training provided to operators will be
sufficient to use the Horizon system
¢ React and resolve incidents as per live operating procedures (i.e.
via the various supporting Helpdesks)
Conversely, it will not:
« Attempt to prove functionality that has not already been proved in
a prior phase of testing
e Focus on excepiional or contrived tesis
° Attempt any excessive performance or volume testing"
Model Office Testing is thus performing the classical role of
acceptance testing by ensuring that the IT system will integrate with
the new business processes and that selected users are well trained
in its operation. It is also clear that Model Office testing pre-supposes
that program and system testing has been rigorous and will therefore
have included "exceptional" and "contrived" tests i.e. tested that the
software can correctly handle all valid business conditions and,
conversely, reject all invalid conditions.
Prepared by Project Mentors Ltd., acting as Ref.: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 31 of 37
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
6.9.5
6.9.6
However, as we describe earlier, program and system testing will not
have achieved this objective since no full analysis of the Authorities
business rules has been produced. Thus testing the system's
capability for handling exceptional conditions will not be periormed
until Live Trial and Live Running.
Acceptance Testing (2) - Card Programme Live Trial
Whilst still part of the contractual acceptance process, Live Trial is, in
reality, Live Running, albeit at a limited number of offices for a limited
set of benefits.
Live Running
The great majority of transactions performed during Live Running wilt
be ‘run of the mill' transactions which have been thoroughly exercised
in earlier phases of testing. These should be processed correctly.
However, Live Running, because of its scale and the diversity of
business situations, will reveal errors and omissions within the
Pathway system with unpredictable results.
Prepared by Project Mentors Ltd., acting as Ret.: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 32 of 37
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
A. APPENDIX - SSADM DELIVERABLES
SSADM stage Primary SSADM Products
0. Feasibility * Overview Data Flow Model - an outline description
Study of the selected functional areas of the business.
Defines the scope.
¢ Overview Logical Data Model - the equivaient logical
data model. Main business objects identified but
attributes not defined. Once again, defines scope.
« 1" Cut Requirements Catalogue - Gathers together
the views of business users on what a new system
should address, for example what is missing from
current facilities, what is good about current
facilities, expected changes to business activity etc.
¢ Feasibility Evaluation - Cost benefit evaluation of
possible solutions. Will contain an outline
development plan. May recommend abandonment if
project is found to be unfeasible.
1. Investigation + Business Activity Model - A more detailed
of Current investigation of the activities performed in the
Environment business. Allows the scope to be clarified.
* Work Practice Model - identifies who does what in
the organisation.
e Enhanced Requirements Catalogue - detailed
investigations will reveal additional catalogue items.
© Current Physical Data Flow Model - what goes on
now in terms of the main processes
° Current Environment Logical Data Model
¢ Current Services Description
2. Business « Business System Options
System Options Selected Business System Option
3. Requirements e Detailed Required System Data Flow Model - a fully
Specification expanded data flow model. The lowest level
(elementary) functions are identified. Shows how all
the required processes will interact with each other
and which external events drive the system.
« Detailed Required System Logical Data Model -
provides a complete definition of every entity,
attribute and relationship in the business data model
to be supported by the new system.
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors te Bird & Bird, Solicitors Status: Provisional Page 33 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
* Detailed System Functions - defines the functions
that will be made available to users, and the user
roles applicable to each function. It also describes
the interfaces to other systems. It is the logical view
of the all interfaces to the system
e User Job Specifications - the new IT system will be
developed hand-in-hand with changes to business
processes. These specifications describe the
required processes and the use made of the
required system functions.
e Processing Specification - a precise and complete
definition of the rules for maintaining the data
defined in the logical data model in response to each
event that drives the system.
« Enhanced Requirements Catalogue - additional or
clarified business requirements are often identified
during this stage. These are added to the catalogue
produced in the previous stage.
4. Technical « Technical Systems Options - the Requirements
System Options Specification stage will have revealed additional
detail such that a number of options for physical
design will need to be considered and one selected
to be carried forward.
* Outline Technical System Architecture - describes
how the selected system will operate.
» System Description - describes the functionality in
the Requirements Specification that is to be
implemented in the selected system.
* {Impact Analysis - explains the impact of the selected
option on the organisation.
* Outline Development Plan - an overview plan for the
remainder of the project.
¢ Cost/Benefit Analysis - will allow for the justification
or rejection of various components of the selected
option.
¢ Workload Model - An assessment of how the
selected system will support the expected workload.
Allows capacity planning to be performed.
« Application Style Guide - defines standards for the
user interface so that a consistent look and feel will
be implemented across the system.
Prepared by Project Mentors Ltd., acting as Ret: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 34 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
5. Logical Design + Dialogue Design - defines the detailed logical
structure of the user interface dialogues i.e. the
structure of the interaction between user and
system.
e Conceptual Process Models - only required for
complex update and enquiry processes.
6. Physical e Application Development Standards
Design « Physical Data Design
« Function Specification
« Space & Timing Estimates
Prepared by Project Mentors Lid., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Status: Provisional Page 35 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
DOCUMENT CONTROL
Change History
Version Date Status Purpose
1.4 19/2/1999 Draft An incomplete draft for review by Project
Mentors team.
1.2 25/2/1999 Draft A complete draft for review by Project
Mentors team.
1.3 5/3/1999 Draft A complete draft incorporating comments
from previous review. For further comment
by Project Mentors team.
1.4 11/3/1999 Draft Incorporates comments from review by
Project Mentors Team. Includes additional
detail on quality assurance. For review by
team.
1.5 22/3/1999 Draft Further draft incorporating comments from
review on 17/3/99.
1.6 30/3/1999 Draft Management Summary added plus additional
comment incorporated. For review by team
on 31/3/99.
1.7 1/4/1999 Draft Introduction modified to be consistent with
other reports. Changes to layout made for
consistency. Minor modifications following
review comments.
1.8 9/441999 I Provisional I Incorporates minor modifications to text and
layout.
1.9 27/4/1999 I Provisional I Incorporates further minor modifications to
text and layout.
Changes Forecast
This document version will be presented to Bird & Bird for review. This may result in
further modifications.
Prepared by Project Mentors Ltd., acting as Ret: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 36 of 37
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Systems Development Approach
Distribution
Project Mentors Bird & Bird Benefits Agency POCL
Andrew Davies Hamish Sandison
Andy Wing
References
4. Report: Pathway Systems Development Project - Requirements Review, Project
Mentors document A.41.07
2. Report: Pathway Systems Development Project - Project Management, Project
Mentors document A.41.05
- 3. Bringing Technology to Post Offices and Benefit Payments: Statement of Service
Requirement - March 1995 Final version 6
4. Horizon Test Team, Model Office Low Level Test Plan, BA/POCL Nile Release
2.0, Version 1.1, 5" August 1998
Prepared by Project Mentors Ltd., acting as Ref: A.41.06 V1.9 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 37 of 37
POL00093654
POL00093654
Project Mentors Orchard Farm Cottage Tel:
Meadie Fax:
Aylesbury E-mail: mentors
Bucks HP17 9UD Mobi
April 28, 1999
Mr. Hamish Sandison
Partner
Bird & Bird
90 Fetter Lane
LONDON EC4A 1JP
Dear Hamish,
Independent Consultant Review of BA/POCL Payment Card programme
Privileged and confidential. Prepared in contemplation of litigation
We have now completed production of the full set of reports on the Pathway Systems
Development Project, as described in my letter of 23rd March. I therefore enclose 1
bound and 1 unbound copy of the following:
A41.05 — Project Management
AA41.06 Systems Development Approach
A41.07 Requirements Review
The summary report, A.41.08 Management Summary, was sent to you last week.
Our team will complete the indexing and filing of documentation this week and next, at
which time they will have no further work to carry out for the programme. Unless I hear to
the contrary, they will then be assigned to other projects.
We will, of course, be available to present our conclusions to you and the Authorities.
I will contact you to discuss arrangements for such presentations.
-Mours. sincerely.
GRO
“Professor Andrew Davies
Director
Project Mentors Limited
Registered in England no.3193018 Registered Office: 30 Upper High Street Thame Oxon
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
REPORT
PATHWAY SYSTEMS DEVELOPMENT PROJECT
PROJECT MANAGEMENT
Ref: A.41.05 Version 2.1
Status: Provisional
Prepared: April 1999
Prepared by Project Mentors Ltd., acting as sub-contractors to Bird & Bird, Solicitors
POL00093654
POL00093654
Privileged and confidential.
Bird & Bird
Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
SECTION PAGE
1. INTRODUCTION 1
1.1. Context 1
1.2. Scope of this Report 1
1.3, Purpose 2
1.4. Constraints 3
1.5. Definitions 3
1.6, Document Structure 3
2. MANAGEMENT SUMMARY 4
2.1. Pathway’s Responsibilities 4
2.2. Good Practice 4
2.3. Assessing Pathway’s Approach 4
2.4. Conclusions 5
2.5. impacts - To Date 6
2.6. Impacts - In the Future 6
3. CHARACTERISTICS OF A PROJECT 8
3.1. What is a Project? 8
3.2. Why Manage Projects? 8
3.3, Key Elements to be Managed 9
4. THE PRINCE/2 METHODOLOGY 10
4.1. Introduction 10
4.2. Organisation Wt
4.3, Planning 13
4.4, Controls 18
4.5, Management Review and Decision Points 16
4.6. Risk Management 17
4.7. Quality Management 18
4.8. Configuration Management 20
4.9. Change Control at
5. ASSESSING PATHWAY’S MANAGEMENT OF THE PROJECT 22
5.1. Introduction 22
5.2. Limitations of Our Review 22
5.3. Programme Management versus Project Management 22
5.4. Pathway’s Responsibility for Development 23
Prepared by Project Mentors Ltd., acting as sub-contractors to Bird & Bird, Solicitors
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
SECTION
5.5. Pathway’s Responsibility for Project Management
5.6. Our Approach
a
. PATHWAY’S PROJECT PLANNING
6.1. Introduction
6.2. Summary of Findings
6.3. Phasing of Delivery and Delays
6.4. Up to Award of Contract
6.5. Award of Contract to First Replan
6.6. The Replan of February 1997
6.7. The Replan of July 1997
6.8. Delay and Replan - November 1997
6.9. Delay and Replan - April 1998
6.10. Current Position (April 1999)
7. OTHER ASPECTS OF PATHWAY’S PROJECT MANAGEMENT
7.1, Introduction
7.2. Project Controls
7.3. Risk Management
7.4. Quality Management
APPENDIX A - DEVELOPMENT SERVICES
APPENDIX B - DEFINITIONS OF TECHNICAL TERMS.
DOCUMENT CONTROL,
Change History
Changes Forecast
Distribution
References
PAGE
23
25
26
26
26
27
28
31
32
35
37
39
40
44
44
44
47
47
50
53
56
56
56
56
57
Prepared by Project Mentors Ltd., acting as sub-contractors to Bird & Bird, Solicitors
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
1.41.
1.2,
INTRODUCTION
Context
This report is one of a set of related reports commissioned to support
Potential litigation and negotiations for settlement of that litigation with respect
to the Card Payment Programme. The structure of this set of related reports
is illustrated in figure 1.1 below, which highlights the position of this report
within that structure.
Pathway Systems Development Project
Management
Summary
Ref A.41.08
4,
Project Systems
Management Development
Approach.
Ref A.41.05 Ref A.41.06
Requirements
Review
Ref A.41.07
Figure 1.1 - Structure of Related Reports
This report describes our findings with respect to Pathway’s management of
the project, while the “Systems Development Approach” (Ref. A.41 -06] and
“Requirements Review’ [Ref. A.41.07] reports set out our assessment of the
technical aspects of Pathway's development of the system. The overall
findings of our review are set out in “Pathway Systems Development Project -
Management Summary’ (Ref. A.41.08].
It should be noted that there are links between the management and
technical factors, and ideally this project management report should be read
in conjunction with its companion papers.
Scope of this Report
The Card Payment Programme calls for the provision of a wide range of
services by the CONTRACTOR (Pathway Ltd.) for the Benefits Agency (BA)
of the Department of Social Security (DSS) and Post Office Counters Limited
(POCL). Three related agreements define the rights and obligations of the
parties in providing these services. The agreements are between Pathway
and the DSS and POCL jointly (the Authorities agreement) and between
Pathway and the DSS and between Pathway and POCL.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 1 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
1.3.
The services to be supplied under the agreements are Development
Services, Rollout Services, Steady State Services, Management Services,
Contingency Services and Transfer Services. This report, and its
companions, are concerned only with the development services, which
include:
« PAS - Payment Authorisation Service;
e CMS - Card Management Service;
« BES - Benefit Encashment Service;
e° APS - Automated Payments Service;
* EPOSS _ - Electronic Point of Sale Service.
A number of other development services are identified in the relevant
agreements or schedules to those agreements as contingency, optional or
additional services. Apart from the Order Book Control Service (OBCS),
which is a POCL optional service, none have been covered in this review.
The services could not be supplied in any way which met the expressed
requirements of the Authorities without the support of computer based
systems. Therefore the systems are fundamental to the services provision.
The related agreements clearly place responsibility with Pathway for the
provision of these systems, together with the necessary interfaces between
them and with other DSS and POCL systems. Appendix A to this report lists
the relevant clauses and schedules supporting this allocation of responsibility.
Under the related agreements, Pathway has the authority to determine
whether each system can be an existing (packaged) system, a modified
package or be developed from scratch. However, whichever route is chosen,
the end results must meet the requirements of the Authorities and support the
solution defined by Pathway.
We describe the provision of these systems as the development project,
shortened to the project, in contrast to the overall provision of the services
which depend on them. The systems which the project will provide we have
collectively called the system. We have not reviewed the service provision
aspects of the Card Payment Programme because the system has yet to be
completed.
Purpose
This report has been commissioned to support potential litigation and
negotiations for settlement of that litigation. It therefore aims to provide
negotiators, who may not be wholly familiar with IT developments, with an
assessment of Pathway’s performance in this area.
Because the report may be used by people without a systems background,
we have striven to provide a view which we trust will be readily
comprehensible, and have, as far as possible, avoided the use of IT jargon.
This applies particularly to Chapters 3 and 4 which outline, respectively, why
project management is necessary and what is accepted good project
management practice. The concepts described in this report need to be
understood if the scale and impact of failures in following good project
management practice are to be recognised.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 2 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
1.4,
1.5.
1.6.
Constraints
Currently (April 1999) we do not have access to Pathway documents or to its
staff. In consequence, our findings are based on:
¢ relevant items in the Authorities possession;
* the absence of material which it might reasonably be expected Pathway
would have prepared and passed to the Authorities;
* our assessment of the causes behind various events in the project's
history.
Definitions
There are a number of terms used throughout this document and its
companion reports which can have different meanings in different contexts.
We have defined our use of these terms at Appendix B.
Document Structure
The next Chapter summarises our overall findings on Pathway’s project
management performance. Following the Summary, we set out an overview
of what constitutes a project, how project based activities differ from on-going
activities, and why it is so crucial that effective management is applied to
Projects.
We have then presented a very brief overview of one particular project
Management approach, known as PRINCE/2. This is offered as a
representative and well documented standard against which Pathway’s
performance can be assessed.
Next we have described the way in which we have compared Pathway’s
project management against good practice, also providing more details of the
context of this review. The following Chapter sets out the findings of our
review of Pathway’s project planning, relating these to the various replanning
exercises which have taken place during the life of the project. The final
Chapter sets out our findings on Pathway performance with respect to project
controls, risk and quality management.
The body of the report is supported by two Appendices. The first lists the
clauses and schedules in the Authorities, DSS and POCL agreements (the
related agreements) which set out Pathway’s responsibility for the
development project and its management. The second contains definitions of
technical terms.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 3 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
241.
2.2.
2.3.
MANAGEMENT SUMMARY
Pathway’s Responsibilities
At the outset of our work we reviewed the contractual documentation to
determine whether Pathway was responsible for:
* developing the software for the Card Payment programme;
* managing that development.
In our opinion Pathway was responsible for both aspects. We further
identified that under the related agreements (clause 702.2), Pathway was
obliged to apply good industry practice to all work in managing and effecting
development of the software. Pathway, in its response to the Authorities
Statement of Service Requirements, proposed the use of the PRINCE
methodology for project management.
Good Practice
Managing projects in software development is one of the key factors in
achieving the benefits expected from them. From the early days of
computing, increased understanding gained from successes and failures has
led to formal approaches to project management. While different approaches
lay emphasis on different aspects, all recognise the need for projects to be
properly organised, planned and controlled. They also endorse the necessity
for managing the risks associated with development, and for ensuring that an
appropriate degree of quality is maintained. From the late 1970’s to the
present day, formal approaches to project management have evolved from
the use of standard checklists to the availability of methodologies. In our
opinion, current good practice in software development demands the
application of a methodology.
Assessing Pathway’s Approach
Having established that Pathway was responsible for managing the software
development in accordance with good industry practice, we chose the
PRINCE/2 methodology as the standard against which to measure its
performance. PRINCE/2 is the successor to PRINCE, which Pathway had
Proposed to use. It is in the public domain, and was available before the
award of contract in May 1996. PRINCE/2 is less prescriptive than PRINCE,
allowing the project manager to be flexible in choosing how its eight
fundamental components are adopted. However, the requirements of those
eight components must be met.
We had no direct access to Pathway’s documents, nor the opportunity to hold
discussions with its staff. Because of this we based our assessment on
documents in the Authorities possession, supplemented by discussions with
staff from BA and POCL. This has restricted our review to planning, controls,
quality and risk management, four of the eight PRINCE/2 components. We
were not able to assess organisation or the sub-division of the project into
management stages. We could not directly assess configuration
management or change control, although we consider that appropriate
measures were in place for these aspects.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 4 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
2.4.
We reviewed the events of the project from the issue of the Statement of
Service Requirements through to and including April 1999. Our work
concentrated on the planning component, the most critical aspect of
PRINCE/2, and indeed of other methodologies. For planning we formed a
view on what actually happened, as opposed to what might have been
expected under PRINCE/2, at each point where the project was replanned.
For the other three components we have drawn our conclusions from an
overall project view.
Conclusions
It is our view that for the first eighteen months of the Card Payment
programme Pathway failed to meet its obligations to use good industry
practice in managing the software development project. In particular we
conclude that Pathway:
* had no plan even approximating to the requirements of PRINCE/2 prior to
the award of contract;
« failed to develop a realistic plan shortly after award of contract;
* again failed to develop realistic plans during the replanning activity of
February 1997;
e did not prepare plans consistent with good project management practice
until early in 1998;
* failed to control the project from its commencement until Pathway’s project
management improved early in 1998;
* did not address the need for risk and quality management in the first 18
months of the project.
We believe most of the project management problems stem from Pathway’s
failure to undertake a detailed analysis of the Authorities requirements. This
lack of requirements analysis is set out in more detail in our report “Pathway
Systems Development Project - Systems Development Approach” [Ref.
A.41.06]. However, had Pathway attempted to develop a good project plan
before or even shortly after contract award, the need for requirements
analysis would have become apparent.
From late in 1997 or early in 1998, Pathway’s project management approach
started to improve. We believe, but would need access to Pathway’s records
to confirm, that Pathway is now applying most of the principles of PRINCE/2
to its management of the project. We do however remain concerned that
there appear to be no detailed plans in place for the development of software
deliveries after New Release 2.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 5 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
2.5. Impacts - To Date
Pathway’s early failure to plan and control its software development project,
together with its lack of application of risk and quality management
measures, have had severe and adverse impacts on the Authorities. The
impacts are those resulting from delays, and those resulting from the phasing
of software delivery.
2.5.1. Delays
It now seems possible that New Release 2 may start rollout on a
national basis in August 1999. This is just over two years later than
envisaged when the contract was awarded. It must also be borne in
mind that New Release 2 will not contain all the functions originally
intended. The balance is to be made up in New Release 2+, which it
is hoped will be available for national use from February 2000. This
represents a delay of two years and seven months from the date
expected at contract award, and a full two years later than was agreed
during the replanning of February 1997. We have not yet seen
detailed plans for the development of New Release 2+, making it
difficult to place reliance on its being delivered for use in February
2000.
These delays have cost the Authorities substantial sums. The
benefits expected from the system have not been achieved, and the
Authorities have had to keep their own programme management and
technical staff in place for much longer than anticipated.
2.5.2. Phasing of Software Delivery
Apart from an initial, very limited, release, the software was to be
supplied as a single delivery in July 1997, followed three months later
by a further delivery of additional functionality. Pathway encountered
development problems towards the end of 1996 and proposed that
the software be delivered as a series of releases. The Authorities
accepted this proposal. Each extra release has and will require the
Authorities to provide staff and facilities to test it, increasing their
costs substantially.
2.6. Impacts - In the Future
As we have stated in 2.4 above, Pathway’s management of the development
project seems to have improved. However, we are concerned that there are
no detailed plans available for New Release 2+ or for later releases. While
we understand there are commercial issues around the content of New
Release 2+, we would have expected there to be quite detailed plans for its
development, albeit hedged with commercial caveats.
Assuming the present release does commence national rollout in
August 1999, we believe there are risks that:
* planning for the next and subsequent releases may be obscured by the
desire to rollout New Release 2;
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 6 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
* if, as is quite possible, technical problems arise as the number of outlets
using the system grows, Pathway may focus on the need for technical
‘fixes’ to keep rollout on schedule. Without firm project management
disciplines, Pathway may lose sight of the need to manage development of
subsequent releases.
To help manage these risks we recommend that the Authorities ensure
Pathway apply good industry practice in its management of the project.
Clause 5 of schedule A2 to the related agreements appears to give the
Authorities the opportunity to monitor Pathway’s Quality Management System
through third party assessment. They may wish to consider making more
active use of this provision.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 7 of 57
Privileged and confidential. Bird & Bird I Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
3.1.
3.2.
CHARACTERISTICS OF A PROJECT
What is a Project?
Projects originate with a desire for change. This desire may arise from
competitive, regulatory or legislative pressure, from a wish to offer better
customer service, to achieve savings in costs or a combination of some or all
of these.
A project may be defined as an endeavour to achieve a specific goal in a pre-
determined time. When that goal is achieved, or proves unachievable (at all
or within the time allowed), the project terminates.
By nature, projects are ‘one off’ exercises. In this they may be contrasted
with commercial activity based on continuing operations. For example, a
manufacturing company will be concerned with taking in raw materials and
possibly sub-assemblies, applying industrial processes to these, and
producing finished products for sale to its customers. While the volume of
such activity may rise and fall with the company’s fortunes and general
economic cycles, the principal business remains unchanged from one year to
the next.
Both projects and on-going operations may be said to take inputs, apply a
process and generate outputs. However, the nature of projects are such that
each one will have different inputs and produce a different result. This
requires that the process is well chosen and offers the best chance of
success. Without a good process there can be no certainty that the desired
result is achieved. This is illustrated in table 3.1 below.
POL00093654
POL00093654
Inputs Process Outputs
On-going I Real things Clearly defined Clearly defined (finished
(raw materials, (machine/chemical products for sale, sub-
sub-assemblies) I processes, labour assemblies)
operations)
Project I Intangibles Good practice Variable
(concepts, ideas) I (Varies depending on I (depend on inputs, technology,
inputs and outputs) cost)
Table 3.1 - On-going versus Project Activity
Why Manage Projects?
Projects do not exist in a vacuum. Each is initiated by an organisation
(company, Government Department) which funds the work of the project
team in order to achieve the requisite results. In these circumstances the
initiating organisation is the customer, and the project team is the supplier.
The customer will want to know from time to time what progress is being
made on the project he or she is paying for. After all, if problems are
encountered, it may well be better to abandon the work and write off the costs
incurred to date than to proceed and incur further costs. This decision is the
customer's prerogative.
Prepared by Project Mentors Ltd., acting as Ref: A.41,05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 8 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
The supplier needs to know what progress is being made, whether the project
is going to be completed on time and within budget, and if not what the
overrun (or underrun) is going to be. The supplier also needs to know if the
customer's requirements change during the course of the project for
whatever reason.
The process of supervising all the activities on a project is project
management. To ensure that such activity is both focused and accountable,
a project manager is appointed. This person must have the experience and
skills appropriate to the size and complexity of the undertaking.
3.3. Key Elements to be Managed
The fundamental role of the project manager is to deliver the required product
to the customer at the agreed time. To do this he or she needs to put in
place:
* a suitable project organisation;
¢ plans for the project;
* controls to monitor progress against the plans;
* management review and decision points;
* procedures to identify and control the risks to the project;
* ways to make sure that the quality of the final result is satisfactory;
* management of the products which the project produces, including both
interim and final products. These are the assets that the project builds,
and their control is generally termed configuration management;
* controls to ensure that any necessary changes are documented and
controlled.
These factors are common to the major project management methodologies.
We set out in the next Chapter why these factors are important, and explain
how the PRINCE/2 methodology provides for them.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 9 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
4.1.
THE PRINCE/2 METHODOLOGY
Introduction
There are a number of methodologies available as standard approaches for
project management. We have elected to use PRINCE/2 to illustrate good
practice in this area for three main reasons:
* itis widely used in public and private organisations within the UK;
¢ the methodology is in the public domain (although a registered trademark
of the Government's Central Computer and Telecommunications Agency
(CCTA)), and thus freely available;
¢ Pathway proposed the use of PRINCE, the forerunner to PRINCE/2, in its
document “Pathway Response to OJEC Notice 94/S 165-58937/EN” (the
Pathway Proposal).
PRINCE/2 is a development of PRINCE, the project management approach
established in 1989 by CCTA. PRINCE itself was based on a commercially
developed methodology named PROMPT which was created in 1975. Thus
at the time the Card Payment Programme was starting, PRINCE/2 and its
forebears had been in existence for 20 years.
PRINCE/2 is designed to support the management of the specification,
design, development, testing and implementation phases of projects, to
deliver a defined set of ‘products’. For systems development projects, these
phases are known collectively as the Systems Development Life Cycle
{SDLC). The SDLC is preceded by the ideas/concept/feasibility stages and
followed by usage and eventual discontinuance of the products.
PRINCE/2 has eight major components, as shown in figure 4.1 below, and
the following sections set out an overview of each component.
Change
Organisation
Control 9
Configuration Planning
Management
Quality
Management Controls
Risk
Stages
Management 9
Figure 4.1 - PRINCE/2 Components
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 10 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
4.2. Organisation
4.2.1. Why have a Project Organisation?
A formal organisation structure provides clarity of responsibility within
the group charged with the work, and accountability to those for whom
it is being carried out. A suitable organisation:
* provides clear lines of communication between groups;
¢ allows individuals to fit into roles appropriate for their experience
and abilities;
« enables those who have an interest in the outcome of the project to
be involved with it without being full time members of the project
team.
4.2.2. PRINCE/2 Project Management Structure
The project management organisation structure recommended by
PRINCE/2 is illustrated in figure 4.2 below.
Corporate
Management
—T—
Project
Board
Project
Manager
Project
Support
Team
Leaders
Figure 4.2 - PRINCE/2 Project Management Structure
Prepared by Project Mentors Ltd. acting as Ref: A.41.05 V2.1 Date: Aprit 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 11 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
The methodology recognises that the organisation must contain a
number of key roles. These roles can be assigned to specific
individuals or groups. On smaller projects, one individual may
undertake more than one role. The key point is that the roles are
needed and must be fulfilled. The following points explain the roles in
more detail.
Corporate Management - is outside the scope of PRINCE/2. It
represents the senior management level in the organisation which
determines the benefits required from the project, and which
authorises the Project Board to expend the necessary resources to
achieve them.
Project Board. The purpose of the Project Board is to represent the
interests of the overall business, the users of the system and the
supplier of the system. The Board has the responsibility for ensuring
that the project stays on course and delivers the requisite products at
an acceptable level of quality.
Members of the Project Board will be senior managers with the
authority to commit resources to the project. They are most unlikely
to be full time members of the project team. For this reason they
must manage by exception, responding to deviations from plan and
business and project issues as they are advised by the project team.
There are three key roles which must be fulfilled within the Project
Board:
* executive;
* senior user;
e senior supplier.
The executive bears final accountability for the project. The role
maintains a balance between the business, user and supplier
positions during the project, and ensure that a ‘value for money’
approach is maintained. A further responsibility is ensuring that the
integrity of the original business case for the project is maintained.
The senior user role, which may well require one or more individuals
to represent a number of different groups of users, is concerned with
making sure that the final product is fit for its intended purpose. The
role extends to allocating user resources to the project, and to making
sure that interim products are of sufficient quality.
The senior supplier role is there to ensure that the design and
development approaches are realistic, and to represent those
concerned with the design, development, testing and implementation
of the product. Where the supplier side of the project has its own
business case, the senior supplier role is also responsible for
maintaining the integrity of that case.
Project Manager. Projects start with the production of a Project
Initiation Document (PID) which defines the scope of the project. This
is approved by the Project Board and day to day management of all
the activities required to complete the project are assigned to the
project manager.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 12 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
The project manager's role is to ensure that the work results in the
agreed product on time and budget to an acceptable level of quality.
This encompasses responsibility for making sure the project achieves
the benefits originally foreseen.
Team Leaders. For all but the smallest projects, the project manager
requires assistance. It is normal therefore to appoint team leaders to
manage specific areas of the project. Team leaders are responsible
for making sure specific products, as allocated to them by the project
manager, are developed on time, to budget and to an agreed level of
quaiity.
Project Assurance. The project manager has day to day
responsibility for the project, including time, cost, resource and quality
matters. Without independent verification, the Project Board has no
way of telling whether or not reports from the project manager
accurately represent the status of the project. The role of the project
assurance group is to provide an independent channel of verification
to the Project Board that all is well with the project.
Project Support. The project manager has responsibilities in a large
number of areas. Monitoring these soon becomes time consuming
and burdensome. The role of project support is to undertake the
routine administration of such areas as maintaining the project plans,
recording changes and managing the configuration of the products.
4.3. Planning
4.3.1.
Why Plan?
There are two key aspects to project plans, the activity of planning
and the plans themselves. During the World War II preparations for
the “D” Day invasion of France, General Eisenhower is said to have
remarked “Plans are useless, Planning is essential”.
The second part of this remark (“..Planning is essential) stresses that
the actual process of defining a plan is vital to an enterprise, because
it makes the planning team think through every activity in detail.
Having thought through the endeavour, a plan of how things should
happen can then be drawn up to communicate with all involved what
they have to do and when.
The first part of the remark (“Plans are useless”) reflects the fact that
things go differently in the event from the way they were planned.
However, without the initial plan, it is impossible to determine how far
from that plan events have strayed.
The plan provides a route map through the project. By looking at
signposts along the way, the project team, principally the project
manager, can determine how far off-track they are, and from this work
out the best route to getting back on track. Without a plan the team
never knows whether it is on the right route.
Prepared by Project Mentors Ltd., acting as Ref; 4.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 13 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
Project plans are absolutely fundamental to achieving success with
the project because they afford the means of implementing progress
measurement and corrective actions. They are absolutely central to
any methodical approach to managing projects. A plan may be
defined as a document defining how a specified target is to be
reached, when it will be reached and who will be needed to reach it.
By defining the how, when and who, the elements of cost, time scale
and quality can also be addressed.
4.3.2. PRINCE/2 Planning
A plan sets out unequivocally:
« all the products to be produced to reach the desired aim, including
intermediate products along the route;
e for each product, both the direct work activities needed to develop
that product and the indirect activities needed to ensure that it is of
acceptable quality;
* the estimated work involved in each direct and indirect activity;
e the resources required to complete the work, together with the
skills required by the people involved;
« dependencies between activities within the project;
e dependencies on the delivery of such things as information,
products and services from outside the project (external
dependencies);
the timing of the start and completion of each activity;
* points at which progress can be monitored and controlled;
« risks to the project and measures to avoid, mitigate or eliminate
them.
A PRINCE/2 plan is more than a schedule of activities. It provides the
reader with key information in a readily understandable way so that it
can be assessed, questioned and confirmed.
4.3.3. PRINCE/2 Planning Levels
PRINCE/2 has two primary levels of planning, a project plan which is
maintained at a higher level (less detail) than stage plans.
A project plan is needed from the outset, but there will normally be
insufficient certainty to prepare a detailed plan at the beginning. It
must therefore present the best available estimates of the likely future
course of the project, and must state any assumptions made in its
preparation. The project plan is the standard by which the Project
Board assesses progress.
Under PRINCE/2, all projects are divided into manageable stages. A
stage plan is required before work commences on each stage, and
must be prepared at a very detailed level. The stage plan is the
standard against which the project manager assesses progress.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 14 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
The project and stage plans can be further sub-divided where the size
or complexity of the project warrants this. PRINCE/2 has the concept
of team plans which are used by team managers to monitor progress
of their teams’ activities. Where team plans are used, the stage plan
is a consolidation of its subsidiary team plans.
PRINCE/2 has one further, very important, type of plan, the exception
plan. These are to be produced when it becomes apparent that a
team, stage or project plan is not going to be achieved within its
constraints. The exception plan, which has the same format as the
plan it replaces, uses actual data on the activities of the original plan
up to the point where the exception plan takes over, and then
re-forecasts the details needed to complete the activities. It will also
explain the reasons for the deviations, what will happen if nothing is
done, any other options which may be available, and the impact on
the project and business case. Exception plans must, if they impact
the project plan, be approved by the Project Board.
4.4. Controls
4.4.1. What are Controls?
Once plans are in place, progress can be assessed and reported in
terms of deviations from those plans. Any activities which are in line
with their planned schedules can quite rightly be assumed to be of no
danger to the achievement of the desired result on time. If the plan
had a particular activity starting within a range of dates and
completing within a later range of dates, then provided that activity is
still within those boundaries it gives no cause for concern.
Controls are concerned with measuring how close the project is to its
planned schedule. They are necessary to enable management to be
sure that the correct products, with the right level of quality, are being
produced on schedule and within the agreed cost and resource
constraints, and that the business case remains viable. They provide
the information by which each level in the project management
hierarchy can:
¢ monitor progress;
* compare achievement with plan;
detect problems and initiate corrective actions as early as possible;
authorise further work.
.
The controls used must also monitor external events which impact on
the project.
4.4.2. Controls in PRINCE/2
Most of the controls within PRINCE/2 are driven by events, such as
the start or end of a stage, the delivery of a product or something
similar. There is also provision for time driven controls, such as
regular progress reports.
Prepared by Project Mentors Lid., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 15 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
At the Project Board level, controls are exercised on the basis of
management by exception. That is the future work is planned, and
most reporting is based on exceptions against that plan, for example
the late completion of a stage or increased resource requirements.
As long as the project progresses within its planned parameters, the
Project Board need not intervene.
The main controls available to the Project Board are:
* the project initiation process, which is determines the organisation
of the project, provides plans for the project initiation phase and
starts to develop a formal project brief;
* an assessment of the status of the project at the end of each
stage. This is to determine if the stage has been successful, if the
project is still on course and will meet the needs of the business
case, and if risks are being contained. This offers the opportunity
to reorganise, or even cancel, the later stages of the project;
¢ regular ‘highlights’ reports on progress, issues and risks;
¢ exception reports which provide the earliest possible warning of
deviations and put forward the options available for continuance;
* where deviations are foreseen, a mid stage assessment may be
made to enable the Project Board to consider what actions can be
taken;
¢ at the end of the project, a project closure process to check that
everything required has been delivered, to determine if any
follow-on actions are required, and to record ‘lessons learned’ as
input to the planning process for future projects.
The Project Board is also responsible for monitoring events external
to the project, and ensuring that the project team are advised of any
changes arising from such external factors. Such items are normally
communicated via the project manager.
The project manager exercises day to day control of the project within
an authorised stage of the work. Provided they do not exceed the
constraints set by the Project Board and the business case, the
project manager can make detailed adjustments to the project as
necessary. A large part of the manager's control mechanism is work
package authorisation. This is a mechanism used to allocate work to
individuals or teams in a way which clearly sets out the controls they
are expected to apply on quality, time and cost, and the requirements
for progress reporting and hand over of completed products.
4.5. I Management Review and Decision Points
4.5.1.
Why have Management Review and Decision Points
It is all too easy for projects to take on ‘a life of their own’. They can
become introverted, with the project team focusing on solving the
project's problems, and ignoring the very business context which
brought the project into being.
Prepared by Project Mentors Ltd., acting as Ret: A
.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 16 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
4.5.2.
To avoid this, it is necessary for the sponsors of the project to insist
that its achievements and future plans are reviewed at pre-set
intervals, and not allowed to proceed unless they are satisfied the
project remains worthwhile.
PRINCE/2 Stages
PRINCE/2 divides a project into a series of management stages,
which identify management decision points. Such management
stages should not be confused with technical stages such as the
specification of a system or the development of the software, as such
technical stages do not necessarily correspond to business decision
points. Dividing the project into management stages gives control to
the business by allowing the Project Board to assess progress and
determine the future at key decision points according to the business
needs.
By reviewing the project at the end of each stage, the Project Board
can, indeed is forced to:
* assess the viability of the project at regular intervals, and thus
avoid its continuance if it is not going to meet the business needs;
* ensure that key decisions are made before the detailed work
necessary to implement them is committed;
« clarify for the project what the impact of external events, such as
legislation or budget constraints, will have.
As each stage is completed, more detailed knowledge of what is
needed for the next stage becomes available. Thus more detailed
plans for the next stage can be developed with greater certainty, and
any impact on the viability of the overall project clarified.
4.6. Risk Management
4.6.1. Why Manage Risks?
All projects have risks associated with them. By their very nature
projects are instruments of change, and there can be no guarantee at
the outset that the change is achievable at ail or within time and cost
constraints. There are particular risks with IS projects which plan to
use advanced technology. The technology may not work as
expected, or performance may prove inadequate for the task.
Outside factors may also present risks to a project, including such
things as;
¢ the actions of competitors;
¢ legislative change;
* evolution (or revolution!) in the marketplace;
« changes in customer or supplier management or strategic
objectives.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 17 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of fitigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
4.6.2.
The aim of risk management is to attempt to identify such factors, and
determine and implement actions which can either eliminate them
altogether or reduce the impact of such risks if they do occur.
Risk Management under PRINCE/2
The methodology defines risk as “the chance of exposure to the
adverse consequences of future events’. It accepts that controlling
and containing risk is a key factor in making a project successful, and
is thus a major element in project management.
PRINCE/2 identifies two main types of risk:
e business risks, which are the threats which can prevent the
project from delivering the products or services which will achieve
the planned business benefits;
* project risks are those which threaten the orderly management of
the project, and in consequence the achievement of the project's
end results within the agreed time and cost constraints.
Managing these risks is a key part of the Project Board and project
manager roles. It is for the project manager to identify, record,
periodically review and propose planned actions to ameliorate the
risks. The Project Board must notify the project manager of risks
external to the project, make decisions as to which of the project
manager's proposed risk avoidance options are to be pursued and
balance the level of risk against the potential benefits of the project.
The management of risk is split into two processes:
« risk analysis, which is the identification of risks, the evaluation of
their likely impact and the definition of actions which can be taken
to minimise their effects;
e risk management, which addresses the monitoring and controlling
of the actions identified to reduce the impact of risks, and thus
improve the chances of the project achieving its objectives.
4.7. Quality Management
471A.
What is Quality Management?
The end result of the project must not only satisfy the requirements,
but do so in a way that is sufficiently robust to provide continuous
support for the needs of the organisation. Software systems are
made up of many thousands, often millions, of instructions which
operate on one set of data to produce another set of data. To ensure
that these transformations are handled correctly on every occasion,
and can detect and not perpetuate errors in the data they are
operating on, each step of the process of turning requirements into a
finished system must be quality checked.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 18 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
It is a function of project management to agree with the client
organisation the level of quality which is appropriate to the system,
and then to devise, implement and monitor controls to ensure that
level is maintained.
4.7.2. Quality Management under PRINCE/2
The methodology uses the ISO 8402 standard definition of quality as
“the totality of features and characteristics of a product or service
which bear on its ability to satisfy stated and implied needs’. For
projects this means those features of the project's end products or
services which make them fit for purpose in satisfying the objectives
of the project. Within a project environment, satisfying implied needs
introduces a good deal of uncertainty, and is avoided by the analysis
and specification of requirements in sufficient detail to remove that
uncertainty.
Quality Management is the process of ensuring that all interim and
final products and services built during the project enable the results
to achieve the level of quality required. Quality management has four
main components:
¢ a Quality System. This provides an organisation structure and the
processes and procedures to implement quality management.
Where the customer and supplier have quality systems, one or the
other or a pre-agreed blend of the two should be used;
* Quality Assurance. This is the vehicle for setting up the quality
system, and, through a process of regular reviews and audits,
ensures that it operates in a way which meets the quality needs of
the organisation;
¢ Quality Planning. Identifying the quality objectives and
requirements for a particular project is achieved by quality
planning. This is a key part of the project initiation stage, and
defines the quality approach for the whole project in the Project
Initiation Document;
¢ Quality Control. This activity is concerned with examining
products as they emerge to ensure that they meet their pre-
determined requirements.
There are published standards for Quality Management Systems.
These have evolved over the years, and currently the most universally
accepted standard for design and development in the UK and Europe
is BS EN1S09001:1994. Many organisations are certified in their
conformance with this standard by independent certification firms. It
is accepted that use of PRINCE/2 can help an organisation to meet
this standard.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 19 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
4.8.
Configuration Management
4.8.1,
4.8.2.
What is Configuration Management?
Like any organisation, a project must manage its assets. In software
development, these are not stock, plant and machinery, but the
documents which specify the software, the software itself, and the
management documentation (such as plans) which support
communication within the project team and externally. The full set of
these products is termed the configuration and thus their management
is known as configuration management.
Configuration management is essential if there is ever to be more
than one version of a product, and it is virtually unknown for any
project not to have several versions of each product.
To take a smail subset of a typical development project, a system
design leads to a number of program specifications and a system test
specification. If a problem is found during the system test, this may
lead to revision of the systems specification, to a number of the
dependent program specifications and to the programs. When the
new version of the system is re-tested, it is essential that the right
versions of the programs are run for that test. Without configuration
management it is all too easy for old versions to be used, and of
course the system test will fail again.
Configuration Management under PRINCE/2
The methodology identifies the four major functions required for
configuration management:
¢ identification of all the components which go to make up the final
product. This is essentially the inventory of the project's assets;
* control of the products. It would be impractical to place a product
in development under configuration management, because the
development is a process of constant build and change. There
must however be the ability to state that, at a certain point, the
Product is complete, otherwise it cannot be used in later work
which depends on it. Thus a completed product becomes a
controlled product. If changes are later needed to it, a formal
authorisation process is required to make the changes, and to
identify all ‘down stream’ products which may be affected;
* status accounting provides a history of the evolution of each
product and makes that evolution clearly visible. This enables the
process to be tracked, and offers an audit trail of how and why
changes were made;
¢ verification, to enable checking that a set of related products is
consistent in terms of their versions.
Configuration management is closely associated with change control,
the topic set out in the next Section.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 20 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
4.9,
Change Control
4.9.1.
4.9.2.
Why Control Changes?
For all but the simplest projects, change is inevitable. The time span
of large IS development projects ranges from a minimum of eighteen
months to many years. During that time much may change in the
customer's organisation. New business areas develop, old ones
cease to be viable, competitors launch new products and regulatory or
legislative changes occur.
However, change to specification or scope can ruin any project if it is
allowed to happen without proper control. The impact of any potential
change may be additional time or cost through new work or rework.
In each case the impact needs to be weighed against the perceived
benefits, and the decision to proceed or not made by the customer's
management. Where changes are approved, the corresponding
increase or decrease in the project’s budget and time scale must also
be authorised.
PRINCE/2 and Change Control
PRINCE/2 demands that each change considered be put forward as a
change request. The relevant parties then assess the impact on the
project, and provide estimates of the impact on time and cost which
would be incurred if the change is authorised.
These estimates are then reviewed, by the Project Board or by a body
(the change authority) set up by the board. If the impact is beyond the
remit of the Project Board, approval must be sought from higher
management. If a change request is approved, then a change contro!
notice is authorised. This states precisely what is to be changed and
the corresponding increase or decrease in time and cost budgets.
Where changes have a serious impact, it may well be necessary to
replan the remainder of the project to accommodate them, almost
certainly leading to development of an exception plan.
POL00093654
POL00093654
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 21 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
5.
5.1.
5.2.
5.3.
ASSESSING PATHWAY’S MANAGEMENT OF THE PROJECT
Introduction
In Chapter 3 above we set out the reasons why projects must be managed,
and followed this in Chapter 4 with a brief outline of one well accepted project
management methodology; the one which Pathway proposed to use. In this
Chapter we describe the way in which we have performed our assessment of
Pathway’s performance in this area, the assessment itself being set out in
succeeding Chapters.
As set out in Chapter 1, our review is limited in scope to the systems
development project. It does not cover the broader provision of the services
which form the overall Card Payment Programme. To provide a firm
foundation for the review and its findings, we have found it necessary to:
* clarify the limitations to our review imposed by lack of access to Pathway
people and documents;
¢ define the differences between management of the overall Card Payment
Programme and management of the development project;
* establish the degree to which Pathway was responsible for developing the
necessary software;
* substantiate the view that Pathway was responsible for managing the
project.
The following sections describe these factors in more detail, and are followed
by a description of our approach to the assessment.
Limitations of Our Review
Pathway’s own records are not currently available to us, and we have not had
the opportunity of discussions with its staff. This restricts our work to those
elements of project management which are clearly documented on the
Authorities side of the contractual boundary, or where the effects of failures by
Pathway are clearly visible to the Authorities .
Programme Management versus Project Management
There is a clear distinction between Programme Management and Project
Management. The programme incorporates all the activities required to
manage the business change expected to provide the benefits the Authorities
were seeking. The project, as we have defined il, is restricted to the
development of the computer systems on which the programme depends, and
good project plans are a key input to the programme plans.
5.3.1. Programme Plans
Developing the plans for the programme was the responsibility of the
PDA until April 1998, thereafter this role was transferred to the Horizon
Programme team. To develop these plans, inputs were taken from the
sponsors (POCL, DSS (including the Benefits Agency, the BA’s CAPS
Programme, and the Social Services Agency for Northern freland)) and
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 22 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
5.4,
5.5.
from Pathway. A series of programme plans, called Master Plans were
produced. Two versions of these plans have contractual significance in
that they define contractual milestone dates:
© version 1.0, approved at the PDA Board meeting of 13 August 1996,
which reflected the original milestones set out in schedule B07 of the
Authorities agreement;
* version 3.0, which was approved at the PDA Board meeting of
20 March 1997 and incorporated the revised milestone dates which
emerged from the replanning done in February 1997.
Since April 1998 the programme plans have been produced by the
Horizon programme team.
5.3.2. Project Plans
We believe (see 5.4 below) that Pathway was responsible for managing
the project. Because provision of the new systems was critical to the
programme, software delivery dates were in turn critical elements of the
programme plan.
Pathway’s Responsibility for Development
This topic was briefly discussed in section 1.1 of this report. We believe that
Pathway's responsibility for the development of systems and interfaces is
unequivocally supported by the clauses and schedule references from the
related agreements set out at Appendix A.
While Pathway was free to select the development approach of its choice, it
remains bound by Clause 702.2 of each of the related agreements. This
states that “the CONTRACTOR shall discharge its obligations under the
[AUTHORITIES/DSS/POCL] Agreement with aif reasonable skill, care and
diligence including but not limited to good industry practice and (without limiting
the generality of this Clause) in accordance with the best of its own established
internal procedures?
This clause, calling for the use of good practice, applies not only to the
technical development of the system (see our report “Pathway Systems
Development Project - Systems Development Approach’, [Ref. A.41.06]), but
also to the management of the development project.
Pathway’s Responsibility for Project Management
The Card Payment Programme was procured under the Government’s
Private Finance Initiative (PFI). An essential aspect of such procurements is
the transfer of risk from the purchaser to the supplier. in essence the
purchaser contracts for a set of services. These are paid for by the
purchaser as a series of payments for services provided over the life of the
contract. It is for the supplier to build the necessary infrastructure to provide
the services (for example a bridge or computer system) at its own risk, and
then recoup the cost, and make a profit, by charging according to some
pre-determined schedule (by toll for a bridge, by transaction charges for a
computer system for example).
Prepared by Project Mentors Ltd., acting as Ref 4.41.05 V2.1 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisionat Page 23 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
As the supplier is taking the risks, it is reasonable for the supplier to have
complete control of the infrastructure building phase without interference from
the purchaser. After all, it is for the supplier to develop a solution which
meets the specified requirements in a way which best enables that supplier to
meet time and cost constraints, and, hopefully, to make a profit from
subsequent charges. If the services are delivered late, or do not meet the
purchaser requirements, then the supplier will not achieve the expected
revenue streams on time, or possibly at all.
There are however dangers to both parties in this reliance on the supplier,
because:
* failure to apply good practice leads to delay in the introduction of the
service;
* poor specification, development and/or testing of the software will lead to
frequent errors, and increase the cost of maintenance. While the cost
may be borne by the supplier, errors, and any slowness in clearing those
~ errors, will impact the purchaser's customers with consequent monetary or
reputation costs;
¢ it may prove difficult or impossible to persuade an alternative supplier to
bid for the services as the end of the contract period, leaving the customer
with no supplier or subject to predatory pricing by the incumbent.
For these reasons it is normal for the purchaser to insist on the application of
good practice to all the activities which go into building the services. As cited
in the previous Section, Clause 702.2 of the related agreements requires
Pathway to use good industry practice in discharging its contractual obligations.
The use of formal project management is recognised as good practice
throughout the computer industry, and indeed in most project based activities,
For this reason we interpret Clause 702.2 as requiring that Pathway manage
the project in accordance with good industry practice.
Pathway, in its document “Pathway Response to OJEC Notice 94/S 165-
58937/EN” submitted on 8 June 1995, (the Pathway Proposal) undertook to
use the PRINCE project management methodology. The precise reference is
om Pathway Proposal, Annex 2, Section 2.2.5 paragraph 3. We understand that
this proposal is included within the agreements by virtue of clause 706
(Statements and Representations), which states that “The CONTRACTOR
warrants and represents that all statements and representations made to the
AUTHORITIES in connection with tendering for and entering into the
AUTHORITIES’ Agreement are, to the best of its knowledge, information and
belief, true and accurate at the time of making such statements and
representations and that, from the date of execution hereof, it will advise the
AUTHORITIES of any fact, matter or circumstance of which it may become
aware which would render any such statement or representation to be false or
misleading.”
Prepared by Project Mentors Ltd., acting as Ret: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 24 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
5.6.
Pathway accepted the need to apply good practice, and stated positively that it
would use the PRINCE methodology, which is generally accepted within the
software development industry as being good practice. In our view, Pathway’s
acceptance of the need for good practice and adoption of PRINCE as its
project management methodology reflects a clear understanding that Pathway
held full responsibility for the management of the development process. At no
stage has Pathway advised the Authorities to the contrary, or of any intention to
use any alternative approach.
We therefore view the management of the development project as being
Pathway’s responsibility.
Our Approach
We have taken the PRINCE/2 methodology as the basis for our comparison.
It is the successor to PRINCE, the methodology Pathway proposed to use. In
our view the only significant difference between PRINCE and PRINCE/2 as
they apply to the Card Payment Programme is that PRINCE/2 is less
prescriptive. It allows the project more flexibility in the way the methodology
is implemented, and has better provision for managing projects based on
packages or modified packages.
As set out in Chapter 4, PRINCE/2 has eight major components. We have
been able to review Pathway’s performance against four of these (planning,
controls, risk management and quality management), our findings on each
factor being set out in the following Chapters.
We have not been able to review performance with respect to organisation or
management review and decision points (project stages). This is because we
do not have the access to Pathway’s own records or to its staff which would be
required to prepare a reasoned assessment of these two areas.
With respect to configuration management and change control, again we are
not in a position to review Pathway’s records, and cannot comment on its
performance. However, from our experience of similar projects it seems to us
that:
¢ without at least a reasonable configuration management process, Pathway
would not have been able to achieve even the limited functionality already
delivered. This view is reinforced by the amount of detail provided with each
proposed release control document;
* the change control process appears to have been set up, and maintained, in
accordance with good industry practice. While there may be disagreements
as to the precise content, responsibility and charging for individual changes
or proposed changes, the process itself seems well founded. We
understand that the Authorities played a substantial part in defining and
implementing this process.
Of the four components we are able to assess, planning is the critical factor.
We have reviewed this up to award of contract and then at each point where
project delays have occurred. We have reviewed controls, risk and quality
management across the duration of the project.
Prepared by Project Mentors Ltd. acting as Ref: .41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 25 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
6.
6.1,
6.2.
PATHWAY’S PROJECT PLANNING.
Introduction
In this chapter we have addressed Pathway’s approach to project planning.
Our key conclusions are summarised in the next section, followed by an
overview of the introduction of the concepts of phasing of software delivery
and a brief history of the delays so far to the project.
There then follow a series of sections dealing with various replanning
exercises, setting out for each what happened in the project, how we would
have expected PRINCE/2 to have been applied to the replanning and our
conclusions on Pathway’s use of the methodology at that time.
Summary of Findings
From our review of the material available, we believe Pathway failed to follow
good practice in its planning of the project. This was a critical failure both in the
pre-award period, and from contract award until the replanning done in
November 1997. From that point, Pathway’s planning seems to have
improved, but even so subsequent plans have not been achieved.
Our key conclusions are that Pathway:
« commenced its work on the Card Payment programme with inadequate
plans, and possibly without any plans at all;
e was the major cause of the multiple project delays which occurred in the first
18 months after contract award, through its lack of planning;
e has caused the Authorities to incur additional direct costs through the need
to test multiple releases of the system, rather than the one major delivery
followed by a second delivery a few months later as originally agreed;
* has forced the Authorities to defer functionality to achieve a system which
can be rolled out nationally. This is evidenced by the late introduction of
New Release 2+, and by the creation of Release 3.0, a release who’s
scope has yet to be defined.
In our view, the most significant single cause of Pathways failure to produce
adequate plans was its failure to perform detailed analysis of the
requirements. This topic is discussed more fully in “Pathway’s Approach to
Systems Development” [Ref A.41.06]. Without a detailed requirements
specification, Pathway lacked firm knowledge of the work required to develop
the system.
Without a detailed knowledge of what is to be done, it is not possible to
develop detailed plans of how to do ‘it’. Work break down (task) lists cannot
be prepared, detailed estimates cannot be developed, and activities cannot
be scheduled.
Prepared by Project Mentors Ltd., acting as Ret: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 26 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
6.3.
Phasing of Delivery and Delays
6.3.1.
6.3.2.
Phasing of Software Delivery
The Related Agreements envisaged three software deliveries. These
were:
* ‘Initial Go Live’ (IGL). This was to have very limited functionality for
a few post offices;
e the full system. This was to provide all the functionality defined in
the related agreements. Rollout was scheduled to start in July
1997;
* a further release of software three months later to introduce
additional functionality.
Early in the course of the project, once Pathway realised it was not
going to provide the software in time for rollout to commence in July
1997, the system was sub-divided into a series of releases. In the
interests of keeping the programme moving forward, and to enable
the Pathway system to align with the CAPS programme release
schedule, the Authorities agreed to this ‘phasing’. The first proposed
phasing was not adhered to, and there has been regular redefinition
of the contents of each release, with functionality moved forwards into
later releases and, occasionally, back into earlier ones.
The phasing has forced the Authorities to spend more time on testing.
While the overall effort needed to define all test cases may not have
been greatly affected, the number of test executions has increased
substantially. This has added to the costs and complexity of the
programme.
Delays and Replanning
From the award of contract until the present time, the project has
suffered continuous delays, and rollout of the full system has still not
started. Delays have been formally announced by Pathway in:
November/December 1996;
May 1997;
July 1997;
November 1997;
April 1998.
The most recent Pathway plan suggested a software delivery date
which would have enabled the start of national rollout at the end of
March 1999. Further delay means that this has had to be deferred.
The current expected date for starting national roll out is August 1999,
more than two years after July 1997 as envisaged under the related
agreements.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 27 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
Not only have these delays occurred, but the software which may be
rolled out in August 1999 does not possess all the functionality
contracted for. This functionality will be added later, at a date yet to
be confirmed but currently expected to be February 2000. If this last
date is achieved the delay over original expectations will exceed 2%
years.
6.4. Up to Award of Contract
6.4.1.
What Actually Happened
During the period between the notice requesting expressions of
interest published in the Journal of the European Union on
30 August 1994 and the award of contract on 15 May 1996, Pathway:
* received the first draft of the Statement of Service Requirements
(SSR) published in 19 December 1994;
* received the final SSR (Version 6.0) which was published on
13 April 1995;
* submitted “Pathway Response to OJEC Notice 94/S 165-
58937/EN” on 8 June 1995, (the Pathway Proposal);
¢ took part in the Demonstrator Phase which ran from
28 August 1995 to 31 January 1996;
* received the Requirements Catalogue which was published on
6 February 1996;
* submitted its solutions to those requirements by 23 February 1996;
* submitted its tender (financial bid) for the contract on 21
March 1996;
* submitted a revised tender on 22 April 1996;
From the issue of the initial SSR at the end of 1994 to contract award
in May 1996, Pathway had 16 months to assess and understand the
requirements before committing to satisfying them. For the period
from September 1995 to January 1996, the Demonstrator Phase
offered five months with ready access to the business experts of the
OSS and POCL to help all three bidders understand and confirm the
requirements. With this experience, Pathway felt confident enough to
submit its solutions to the requirements towards the end of February
1996.
In its proposal, Section 7.3.1.3, Pathway estimated the time required
to develop the system as nine months. Pathway further stated that
“To achieve this [the roll-out programme] we will start the design and
development of the systems during June 1995, using the
demonstrator phase to confirm requirements and to demonstrate to
BA/POCL a viable solution and our readiness to roll out......”. (Section
7.3.1.5).
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 28 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
The Pathway proposal provides no detail of the tasks involved or the
estimated effort required to achieve the quoted dates. Neither in its
tender of 21 March 1996 nor in the revised tender submitted on
22 April 1996 did Pathway set out detailed plans for the development
and implementation of the services. None of these documents
contains any detailed set of resource requirements.
The schedule adopted for the programme, and that to which Pathway
agreed, was that set out in Schedule B7 to the Authorities Agreement.
This schedule is reproduced in Table 6.1 below.
EVENT DATE(S)
May 1996
Date of execution of Related Agreements
Development of Services pre-Operational Trial I March 1996 to August 1996
Operational Trial comprising the
CONTRACTOR's Integration and
Performance Testing, the AUTHORITIES’
Acceptance Testing including the Live Trial of
BES, CMS, PAS, APS, EPOSS, OBCS, POCL
Infrastructure Services
Limited Go-Live
September 1996 to June
1997
From 23rd September 1996
Implementation planning for Roll-out
May 1996 to March 1997
Roll-out of POCL Infrastructure Services
July 1997 to December
1998
Roll-out of DSS Services
July 1997 to July 1999
Roll-out Review, Validation and Rectification
August 1999 to February
Period (terminating on Roll Out Completion
Date)
2000
Table 6.1 - Timetable at Contract Award
(Note: Where specific days of a month are not detailed in Table 6.1
above, we have assumed that start dates relate to the
beginning of the month and end dates to the end of the
month.)
From the schedule it can be seen that development work for the pre-
Operational trial system was to start in March 1996 and be finished by
August 1996. Allowing sufficient time for testing, this represents six
months development effort.
Beyond this, schedule B7 called for completion of the operational trial
by the end of June 1997. Backtracking from this date:
¢ the operational trial was to take three months, and thus needed to
be started by the beginning of April 1997;
* contractor's testing was to take between four and six weeks, and
thus would have had to start at the beginning of March (or by the
middle of February if six weeks were required).
Prepared by Project Mentors Ltd., acting as
sub-contractors to Bird & Bird, Solicitors
Ref: A.41.05 V2.1
Status: Provisional
Date: Aprit 1999
Page 29 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
On this basis, development work would have had to be completed, at
the latest, by the end of February 1997. This would be:
¢ six months from the completion of pre-Operational trial software in
August 1996, or:
* some 9% months from the award of contract.
Even if extra elapsed time could have been found by overlapping
some development and testing activities, it is hard to see how more
than an absolute maximum of 10% months development time could
have been available.
At the start of the period available for development, no detailed
specification of the requirements was at hand. It might have been
possible to define requirements for one or two areas in sufficient
depth to start system design for those elements in one or two months.
Requirements definition for other areas would have needed more time
but could perhaps have been conducted in parallel with the
development of the first one or two areas. It is our view that at most
nine months was available for the design, development, program and
system testing of the overall system. This is the time originally
Proposed by Pathway, and we conclude that at the time of award of
contract Pathway believed this to be sufficient.
6.4.2. What We Would Have Expected
To bid for a contract, particularly one of this magnitude and
complexity, any supplier must recognise the need to analyse
requirements to a level at which the costs involved can be estimated
and an appropriate bid submitted.
Given that tew bids are single tender, there is always the probability
that any one bid will not be successful and the costs expended in
Producing it cannot be recovered. This is entirely normal in the
software and services industry. The costs of unsuccessful bids
should be recovered in those which do succeed. In our view, the fact
that the Card Payment Programme was a PFI procurement makes no
difference to the need for pre-bid requirements analysis and planning.
To ensure it was proposing a realistic project, and one on which it
could reasonably expect to make a profit, we would have expected
Pathway to produce a series of plans during the course of its bid for
the Card Payment Programme. We believe Pathway had ample
opportunity to do so through the extensive documentation of
requirements provided by the Authorities, the opportunities for
elucidating and clarifying requirements and the overall length of the
procurement.
6.4.3. Conclusions
We conclude that no plan even approximating to a PRINCE/2 plan
was prepared or used by Pathway prior to award of contract. We
believe this view is supported by the lack of evidence of planning in
the Pathway proposal.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 30 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
6.5. I Award of Contract to First Replan
6.5.1. What Actually Happened
The first software delivery to support IGL was delivered on time. The
IGL service commenced with one office on 23 September 1996, and
four weeks later had been extended to a further nine offices. IGL
offered a very restricted service for paying Child Benefit (ChB) by
card. ChB is one of the least volatile DSS benefits,.
At the PDA board meeting of 21 November 1996, Pathway advised
that it had done some replanning in order to maintain the live trial
commencement date as 1 April 1997 and thus keep the start of
national rollout to July 1997. Pathway introduced the concept of
phasing delivery of the software as part of its revised plans.
The revised approach was considered by all parties, and, at the
following PDA Board meeting (19 December 1996) the PDA
Presented reasons why it was felt the new Pathway phasing approach
was unacceptable. Further consideration was given to the situation by
all parties, and at the next PDA Board meeting (22 January 1997,
agenda item 3., ref. 17), it was agreed that “a solution based on a
phased release of complete systems (application segmentation) was
the best way forward’. This decision led to a replanning exercise the
following month, known as the ‘February 97 Replan’, from which
emerged a revised set of milestones.
6.5.2. What We Would Have Expected
We wouid have expected one of the earliest activities after award to
be the development and agreement of a detailed requirements
specification. At the same time we would have expected Pathway to
use its increasing understanding of the detailed requirements to
develop a project plan. As the detailed requirements were identified
and agreed, so the corresponding work breakdown structures,
estimates and resource implications could be built up.
We would have expected a detailed plan to the standard demanded
by PRINCE/2 to be available within two to four weeks of completion of
requirements analysis.
Given Pathway’s responsibility for project management and the PFI
environment of the project, there would have been no contractual
obligation for Pathway to review or agree such a plan with the
Authorities. However, the project plan is a critical element of the plan
for the overall programme, which is clearly the Authorities’
responsibility.
Pathway appear to have accepted the need for its project plan to fit
into the master plan. This is supported in the notes of the PDA board
meeting held on 13 August, under item 4.6, which states that “Mr.
Bennett (Managing Director of Pathway) confirmed that ICL Pathway
were anxious to progress from drawing up plans to actually using
them and were content therefore to sign off this version for that
purpose. He did however want it placed on record that the plan
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 31 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
6.5.3.
quoted dates which were a mixture of agreed baseline, planned
targets and hoped for dates and this weakened the effectiveness and
the process for review’. The plan referred to is Master Plan
Version 1.0.
Conclusions
By November 1996, barely six months into the development, Pathway
became aware that it was going to have difficulty meeting the
contracted dates. From this we conclude that Pathway started the
development without a clear understanding of what was to be done,
and therefore without a sound plan of how the work was to be carried
out.
With hindsight it is clear the project could never have been
accomplished by Pathway in the time scale originally proposed and
accepted. Following PRINCE/2 principles in preparing a project plan
would not have avoided some delay but would have revealed the
problems much earlier, enabling them to be addressed more quickly
and significantly reducing later delays.
6.6. IThe Replan of February 1997
6.6.1.
What Actually Happened
Following Pathway’s announcement that it could not meet the
contracted dates, and the phasing of the project, the February 1997
teplan started. One factor in the replan was that, from the preceding
October, the CAPS programme had realised that it would not be able
to meet its own plan. Two independent consultancy reviews of CAPS
were initiated by BA, and their reports used as part of a wide-ranging
teplan of the programme. The result was some delay to the previous
CAPS planned dates. These revised dates were incorporated into the
Card Payment Programme replan of February 1997.
A revised programme plan was endorsed by the PDA Board meeting
on 20 March 1997 as Master Plan Version 3.0. Pathway had
substantial input to the dates in this plan, as did CAPS. The revised
milestones from that plan were incorporated into the related
agreements on a ‘no fault’ basis. Master Plan Version 3.0 recognised
a series of software delivery dates for different releases, as set out in
table 6.2 below.
POL00093654
POL00093654
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 32 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
Release I For Date Planned
1.b__ [Restricted live use (OBCS only) 28 April 1997
i.c___I Restricted live use (1.b + BPS for ChB) I30 June 1997
1.d__ {To link to CAPS 2.2 1.c +3 weeks
1.e__ [Live Trial (1.c + EPOSS + APS) 8 September 1997
1.0 __I Full use in national roll-out. 21 November 1997
2.0 I National Use (Further functionality) 26 January1998
3.0 _INational use ( « y April 1998
4.0 __jNational use ( “ ) October 1998
Table 6.2 - Release Dates from February 1997 Replan
When the February 1997 replan was agreed on 20 March 1997, there
were essentially four major software releases planned. These were:
e Release 1.0, comprising sub-releases 1.b, 1.c and 1.e. Release
1.0 was to provide sufficient facilities to enable the programme to
commence its national rollout on 21 November 1997. This would
provide the OBCS, BPS, EPOSS and APS services with some
restrictions in functionality, and with limited security;
e Release 2.0, to comprise a single software release, that is with no
sub-releases, was to provide virtually all outstanding system
functionality with a higher level of security;
e Releases 3.0 and 4.0 are not fully defined in the documents
available to us, but from scrutiny of the release content
descriptions of Release 1.0 and 2.0 it appears reasonable to
suppose that Release 3.0 would complete all functionality and
security features, and 4.0 would provide additional POCL functions
as required and be the “second release of the Software
approximately three (3) months after the July 1997 implementation.
This will enable the CONTRACTOR and the AUTHORITIES to
introduce additional functionality at the counter and in the
supporting systems which is identified after the Functional
Specification has been approved by the AUTHORITIES’ envisaged
in Schedule BO7 to Authorities agreement.
It is worth comparing the situation at the February 1997 replan with
that envisaged 10 month earlier at award of contract. Table 6.3 below
compares anticipated software delivery dates (rather than start of
rollout dates) for the two plans.
Original Replanned Delay
Release Date Release Date (Months)
IGL 23/9/96 Completed On Time None
1.0 Feb. 1997 3.0 Apr. 1998 14
2.0 May 1997 4.0 Oct 1998 7
Table 6.3 - Comparative Software Delivery Dates
Prepared by Project Mentors Ltd., acting as
sub-contractors to Bird & Bird, Solicitors
Ref: A.41,05 V2.1
Status: Provisional
Date: April 1999
Page 33 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
Release 1.b (which provided the Order Book Control System) was
achieved almost on time. All subsequent releases were delayed, and
as at April 1999 only Release 1.c is in operational use.
6.6.2. What We Would Have Expected
Once Pathway had decided on a phased release of the system, we
would have expected it to have prepared detailed plans for the
development and testing of each release. Such plans would have
been necessary to determine:
* the work involved in developing the components of each phase,
and therefore the phase contents;
* delivery dates for the software in each phase;
* any additional work occasioned by revising a software component
from one phase to another;
« the resources required to work on each phase;
« what degree of overlap was possible in developing and testing the
phases. Overlap would minimise the overall duration of the project,
but would need to be balanced against the risks of reworking
earlier development as new functions were added at later releases.
A clear knowledge of the dependencies between tasks is required
to make this judgement.
The software development plan would be one of the major inputs to
the revised programme master plan, the accuracy of which would
depend to a very high degree on the accurate prediction of software
delivery dates. Because of the strong links between the two, we
would have expected Pathway to ask for assistance from the
Authorities in preparing the development project plan, both to improve
its accuracy and to obtain the Authorities’ support for it.
6.6.3. Conclusions
We conclude that Pathway again failed to develop realistic project plans
during the February 1997 replan. We believe this view is supported by
the following factors:
* the three slippages of Release 1.c (June to August, then August to
October and finally to November). Taking the original time from the
February replan date (20 March) to the then planned date of 30 June
is 14% weeks. The time actually taken after replan was 321 weeks,
an overrun of 125% on a 14% week schedule;
¢ the very short notice given of each slippage. A well constructed
project plan, properly monitored, would have given much more timely
warning of the true impact of delays to early tasks on later,
dependent, activities.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: Aprit 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 34 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
It was at the point of the February 1997 replan that Pathway identified
the need to deliver the software as a series of releases. In our opinion
this demonstrates that, prior to this point, Pathway lacked a clear
understanding of the size of the task it had undertaken. Planning
carried out professionally much earlier in the procurement/
development cycle would have made this obvious, perhaps in time to
address the issues.
6.7. The Replan of July 1997
6.7.1.
What Actually Happened
Release 1.c, due on 30 June 1997, was delayed by Pathway, citing
problems with the Riposte software element. This delay was formally
announced at the PDA Board meeting on 8 May 1997, just seven
weeks after the previous board meeting had agreed the new plan, and
less than eight weeks before software delivery was due. Pathway
wished for a six week delay to 1.c, which would have moved the
30 June date to 11 August. At the PDA board meeting of 4 June 1997,
it was recognised that Pathway could not meet its June date for 1.c, and
18 August was accepted for planning purposes for rollout to 200 post
offices. At the same time it was recognised that Release 1.e would also
be late, and it was deferred to 5 January 1998.
At the PDA Board meeting of 15 July 1997 Pathway presented a
revised plan which it had drawn up following an intemal review. This
moved 1.c still further back, to 13 October 1997. At the meeting it is
recorded that “Mr. Crahan reported that it had appeared that
everything had been moving forward toward achieving Congo 4
(Release 1.c) by the 18 August until PDA received information from
MOR [Model Office Rehearsal], Integration Testing and Security
Testing which identified real problems. On 3 July ICL Pathway
informed the PDA that, following an internal programme review, they
had concluded that they would not be able to meet the 18 August go-
live date and proposed 13 October as the revised date for Congo 4.
Since that time the PDA had been working with ICL Pathway to
assess the detail. To date the PDA had not been given visibility of the
low level detail supporting the high level plan for 13 October date, but
it was hoped that this would be made available at a meeting currently
taking place. It was essential that PDA were apprised of the detailed
evidence supporting ICL Pathway proposals before any analysis of
the achievability could be done and a plan built for Congo 4 to bring
with confidence to the Board’. [Ref. BA/POCL Programme Delivery
Authority Board, Minutes of the Meeting 15 July 1997, item 3.1].
As part of the internal review mentioned above, Pathway had drawn
up a presentation entitled “ICL Pathway Programme Review
July 1997". This showed very high level plans for the original and
then newly proposed delivery plans for 1.c. It also proposed a
schedule of future releases and outlined the contents of those future
releases. The proposed schedule is shown in table 6.4 below.
Prepared by Project Mentors Ltd., acting as Ret: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 35 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
Release Purpose Date
CONGO 4 [Pathway 1.c] Link to CAPS 2.1 13/10/97
CONGO 4.1 [Pathway 1.c] Link to CAPS 2.2 03/11/97
CONGO 4.2 [Pathway1c+] Link to CAPS Multi ACC IJan. 1998
NILE 1.0 [Pathway 1c++] Link to CAPS 3.0 Feb. 1998
NILE 2.0 [Pathway ‘New’ 2.0] Link to CAPS 3.0 Mar. 1998
NILE 3.0 [Pathway ‘New’ 3.0] Link to CAPS 4.0 Jul. 1998
Table 6.4 - Future Releases as proposed by Pathway (July 1997)
(Note: ‘CAPS Multi ACC’ refers to the CAPS systems running at
multiple Area Computer Centres.)
The proposal effectively replaced 1.e with New Release 2.0 (NR2),
and deferred the delivery of the Automated Payments ‘SMART’
service, Electronic Funds Transfer at the Point of Sale (EFTPOS) and
other items originally scheduled for Release 2.0 (not NR2) into
Release 3.0. Releases CONGO 4 Minus and CONGO 5 Minus were
also dropped. It is not absolutely clear what these last two releases
were, but it seems likely they were interim releases originally designed
to link Pathway 1.e to different versions of CAPS.
At the next meeting of the PDA Board (21 August 1997), it was
accepted (Item 2.1.4.6) that “Pathway’s plans for Release 2 were
potentially deliverable but tight. Prime options and fallback options
were being discussed. The problems were EVP [Extended
Verification Procedures] and security. POCL also had problems with
testing especially Electronic Point of Sale System (EPOSS). Pathway
reported that their testing strategy was under review and agreed to
pay particular attention to EPOSS".
At the PDA Board meeting on 23 September, the anticipated release
date for 1.c moved back yet again, this time to 3 November 1997,
following problems in testing the software. This date was achieved.
6.7.2. What We Would Have Expected
By the time this replan was undertaken, Pathway had proposed, and
the Authorities accepted, that development of the system would be in
phases. The application of PRINCE/2 principles suggests Pathway
should have used the time since the previous replan (February 1997)
to develop more detailed plans for the development and testing of
Releases 1.e onwards.
Given that this replan was announcing further delays, Pathway would
have been wise to share its detailed level planning with the Authorities
to validate its conclusions and to gain the Authorities understanding of
Pathway’s difficulties. This was not done in a formal sense, although
we understand there was at least one planning meeting with the PDA
staff. Detailed plans however require considerable time to understand
and verify, and more than a single meeting would have been needed.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 36 of S7
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
6.7.3. Conclusions
There is some evidence in Pathway's decision to abandon 1.e, and to
defer some functionality from NR2 to Release 3, that it was beginning
to recognise:
« the complexity of the problem and the overall development effort
which was going to be required;
e that too many releases would stretch Pathway’s development and
testing resources.
However, the subsequent slippage against all the dates forecast in
July 1997 indicates quite clearly that Pathway’s planning was
inadequate. Without access to Pathway documents it is a matter of
conjecture, but we believe that at this stage Pathway still did not have
detailed plans for development. Had such plans been available, it
seems unlikely that the July 1997 schedule would have presented,
only for it to be radically revised four months later.
6.8. Delay and Replan - November 1997
6.8.1. What Actually Happened
The PDA Board meeting of 5 November 1997 returned to the issues
of planning. A report was given by Mike Coombs, who had been
appointed by Pathway as its Programme Director in July 1997. In
relation to Release 2.0 he stated that “there would be no work on
forecast dates until the baseline had been signed off. Indicative dates
had been given to assist with the PDA’s planning, giving a window of
24 August 1998 to 9 November 1998, with 14 September 1998 being
the indicative date. No account had been taken of a CAPS’ window’.
[Item 3.3, bullet point 3 of the minutes of the meeting refers].
We have sympathy with the approach of not planning until it is known
what has to be planned, that is, once the ‘baseline’ definition of
required functionality is agreed. In our view this marks the start of
Pathway’s attempt to bring some degree of professionalism to its
management of the project. However, in this case it must be
remembered that Pathway:
¢ was first advised of the requirements some 2% years earlier, when
they received the SSR on 13 April 1995;
« had been engaged on the development of the system for 1% years
since the award of contract;
¢ had just announced a ‘best case’ delay of 6 months to Release 2
(March 1998 to September 1998) a mere 2 months after its
previous date had been accepted as “potentially deliverable but
tight” [PDA Board 21 August 1997},
e was still offering fess functionality than had originally been
promised in the software delivery expected in February 1997.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 37 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
At the same meeting (5 November 1997), Mr. Coombs was also able
to report that “since the last Board meeting, a series of structured
reviews had taken place with the PDA on the plan for Release 2.
Weekly review meetings were taking place on the plan-for-plan,
issues management and key activity reviews. Formal checkpoint
meetings had taken place on 13 and 31 October 1997’. [Item 3.3,
bullet point 1 of the minutes of the meeting refers]. We see this as a
most welcome sign that Pathway was beginning to involve the PDA in
its planning.
Following the above meeting, and continuing into early December, a
series of joint Pathway / PDA ‘workshops’ were held. On
25 November Pathway made a presentation entitled “Pathway New
Release 2 Replan Review” to a number of PDA staff. On the third
slide of this presentation Pathway state that its internal plan date for
the start of live trial of Release 2 was 14 September 1998. Pathway
also stated that “Sensitivity analysis shows 5th October 1998 as
80%ile - therefore this date is offered for planning purposes’. We
understand that no explanation was given of the process used in
performing this sensitivity analysis nor any detail of the plan or model
on which it was performed.
The presentation also introduced the concept of a ‘New Release 2+’
(NR2+). [Slide: “Release Strategy’] This was apparently to cover
those items previously expected within Release 2 which would not
now be provided until later. It also maintained references to Releases
3 and 4, and added a new Release 5. No dates were attributed to
Releases NR2+, 3, 4 or 5.
At the next PDA Board meeting (11 December 1997), it was stated
that 5 October 1998 would be accepted as the planning assumption
for delivery of Release 2 software to enable the commencement of
live trial in January 1999. [Ref. 3.1].
The PDA Board met for the last time on 26 March 1998. At that
meeting it was agreed that the planned date for NR2+ should be July
1999. From 1 April 1998, the PDA ceased to exist and its functions
devolved to the BA and to POCL. Much of the information reviewed
in the remainder of this Chapter is derived from the minutes of
Horizon checkpoint meetings from 2 June 1998 onwards. This forum
essentially replaced the PDA Board meetings.
6.8.2. What We Would Have Expected
This replan was announcing still further delays. However, in this case
we believe that Pathway had more detailed plans than those
presented to the Authorities on 25 November. Our view is based on
the higher quality of what was presented and the revised release
strategy, both of which indicate more careful and professional
planning.
Again we would have expected Pathway to review its detailed plans
with the Authorities to gain their validation and acceptance. We
understand that this was not done.
Prepared by Project Mentors Ltd., acting as Ref: A.41,05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 38 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
6.8.3. Conclusions
In our view it was not until July 1997 that Pathway started to
understand the full scope of what it had undertaken. Even by
November of that year it still had not understood the full implications
of its contracted commitments. Pathway responded to the problems
identified by strengthening its management and project team,
although too late to keep Release 1.c on target.
While the “Release 2 High Level Plan” which formed part of Pathway’s
presentation of 25 November 1997 does show dependencies between
activities, there is insufficient information in the plan to evidence that it
is the summary of lower level, more detailed, plans.
From this we conclude Pathway’s planning by November 1997 was
still some way from a level which could be considered as good
industry practice.
6.9. Delay and Replan - April 1998
6.9.1. What Actually Happened
By early April 1998 it had become clear that Pathway would not be
able to deliver NR2 in time to start national rollout by October of that
year. A new programme plan was prepared by the Horizon team, and
resulted in the issue of the “Horizon Programme Replan Summary” by
the Authorities in July 1998. This document states that “The replan
activity was initiated by the Sponsors in conjunction with Pathway in
April 1998....”. (Ret. Horizon Programme Replan Summary, Section 2.,
para. 2].
There was some debate between Pathway and the Authorities in
drawing up this plan as to the amount of time required for testing.
Pathway’s position was that one logical day’s testing (the testing for
example of the events on a single business day) could be
accomplished in a single actual day. Despite the BA’s insistence that
this was not achievable, Pathway insisted on a plan based on this
level of time utilisation. The BA’s view was proved correct when the
testing overran.
This replan called for delivery of NR2 software by 3 August 1998, and
which, following testing, could lead to the start of national rollout in
July 1999. However, NR2 would not contain all the contracted
functionality, in particular it would not.have on-line enquiries, soft EVP
or additional POCL services such as the Automated Payments Smart
Card facilities. These were expected in NR2+, which was expected to
be delivered for testing by 1 October 1999.
6.9.2. What We Would Have Expected
We would have expected Pathway to discuss with the Authorities very
detailed plans for NR2 and NR2+, and fairly detailed plans for
subsequent phases. These plans would have included details of the
estimates of time required for development and testing activities, the
Prepared by Project Mentors Ltd., acting as Ref: 4.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 39 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
resources required for each task, and the dependencies between
them. All these factors should have been informed by Pathway’s
development experience on the project, so that estimates could be
realistic and resource shortfalls clearly identified.
It would then be reasonable to expect the Authorities to comment on
those plans, and, where there were disagreements, for example on
testing estimates, to develop consensus or at least compromises.
The result would have been a revised plan in which all parties could
have had reasonable faith. It would also have provided a firm
foundation for the Authorities’ own programme planning work.
6.9.3. Conclusions
It is our view, confirmed in discussions with CAPS personnel, that
Pathway tended to subordinate planning to commercial issues.
Pathway tried to fit the timescales in its plans to dates of commercial
significance. We believe this is the reason Pathway insisted that one
logical day’s testing could be achieved in one physical day.
Even by April 1998 Pathway did not seem to plan on the basis of
realistic estimates of the work required to complete all the tasks
necessary to deliver a release. While conscious that the commercial
pressures were high, such an approach was based on hope, not
reality, and did nothing to speed implementation of the system nor
provide accurate dates on which the Authorities could plan their own
activities.
Overall we conclude that from November 1997 onwards, Pathway’s
project planning had continued to improve. Although Pathway
appeared to have the capability to plan properly, it continued to
subordinate realistic plans to commercial considerations.
6.10. Current Position (April 1999)
6.10.1. What Actually Happened
Pathway submitted NR2 software for the start of the first testing run
on 4 August 1998. There were effectively two sets of tests run in
parallel, end to end (E2E) tests and model office rehearsals (MOR) in
preparation for model office test (MOT). Both are to be completed to
the Authorities satisfaction as a pre-requisite for live trial.
End to end testing is designed to prove that all the functions of the
system work within the Pathway application, and successfully link to
other POCL and BA systems. It was originally intended to be three
cycles of testing, but has been extended to provide four cycles:
« E2E cycle 1, which completed on 22 September, having overrun by
two weeks in a five week schedule;
e E2E cycle 2, which commenced on target on 19 October 1998. It
initially ran behind schedule, but was completed on time by
20 November;
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 40 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
* an additional cycle, known as pre-proving. This comprised two
sets of tests aimed at demonstrating that certain errors had been
cleared and to get ready for the final end to tend tests;
« E2E Final, the final set of end to end tests. Preparatory work for
this cycle started on 10 February 1999 and the tests completed on
17 March.
Model office rehearsals and tests are intended to check how the
Pathway system functions work in post offices, including those
functions related to BA transactions. Four cycles are used:
« MOR1 is the first cycle of model office tests. This was started on
3 August 1998 and completed on 27 August 1998;
« MOR2, the second cycle, started on 14 September 1998. It
appears to have been completed on time;
¢ MORS was due to start on 19 October 1998. Although set up
activities commenced on time, a problem delayed the start of
counters testing until 22 October. Testing was completed by 12
November;
¢ MOT started on time by 15 February, and completed, also on time
by 14 March 1999.
Further testing, designated ‘targeted testing’ aimed at demonstrating
that software incidents have been resolved started on 24 March, and
was completed by the beginning of April 1999.
Authorities reviewed the results, and a Release Authorisation Board
(RAB) meeting was held on 7 April to determine whether or not NR2
software should be approved for use in the live trial. The meeting was
unable to agree whether this was possible. POCL and Pathway
considered that sufficient testing had been performed, but the BA did
not concur.
Once approval for NR2 software to go into live trial is given, some
time will be needed for:
« Pathway to install the software, and bring the computer sites and
ancillary equipment up to the necessary level of readiness;
* migration of those offices on Release 1.c to NR2.
Had the RAB meeting on 7 April 1999 authorised release, live trial
could have started on 10 May 1999, with the commencement of
national rollout scheduled for the middle of August 1999. On this
timetable, rollout of the necessary equipment to post offices would
start in mid-June. It is not yet clear what impact the RAB’s decision
not to authorise NR2 for live trial will have on the rollout schedule.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 41 of S7
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
This still leaves the NR2+ release to be dealt with. This will be a
software only release, and is expected to be installed from the end of
February 2000. As this release is expected to contain all the
contracted functionality, it is equivalent to Release 1.0 as envisaged in
the related agreements. Full functionality may thus commence rollout
2 years and 7 months later than expected at the outset, and 2 years
later than expected at the February1997 replan.
6.10.2. What We Would Have Expected
Again, to conform with PRINCE/2 principles, we would have expected
to see Pathway maintaining detailed plans for the completion of NR2,
the design and development of NR2+ and at least medium level plans
for Release 3.0.
We have not seen such plans, but review of the Horizon checkpoint
meeting minutes clearly points to Pathway sharing high level plans for
NR2 with Horizon. We do not believe Pathway could have done this
successfully, nor have got the likely date of rollout to within one month
of that forecast a year earlier, without having detailed plans.
6.10.3. Conclusions
It is clear from the discussions at, and reports submitted to, the
Horizon checkpoint meetings, that from the middie of 1998 Pathway
was becoming ever more realistic in its planning. Pathway are now
sharing plans, albeit the high level ones, with Horizon, and indeed is
helping to ensure that its project plans align with the Horizon
programme plans.
We believe that, in relation to NR2, Pathway has moved a long way
towards planning at a level which now represents good industry
practice. We have not had the opportunity of reviewing its detailed
plans.
We determine a sense within the Horizon programme that NR2 will be
rolled out at least broadly in line with the dates given above. We
would however caution that:
* there is as yet no detailed plan for the design and development of
NR2+. For obvious commercial reasons we believe Pathway will
concentrate on getting NR2 installed, and there is a risk that it may,
albeit temporarily, lose sight of the importance of NR2+;
« the rollout programme itself is likely to yield technical difficulties. It
would be quite in line with the experience of similar projects for
there to be performance issues as the number of users increases.
Such problems are usually amenable to technical solutions, but
these can be costly and may significantly slow the rate of rollout at
times;
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: Aprii 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 42 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
* once a substantial part of the rollout is complete, it would be
extremely difficult, in terms of reputational risk, for the Authorities,
and in particular for POCL, to stop, let alone reverse the process.
For this reason, progress on NR2+ and Release 3.0 must be
monitored very carefully.
Prepared by Project Mentors Ltd., acting as Ret: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 43 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
7. OTHER ASPECTS OF PATHWAY’S PROJECT MANAGEMENT
7.1. Introduction
In this Chapter we set out our views of Pathway’s performance with respect
to the remaining three factors of project management defined by PRINCE/2,
namely project controls, risk management and quality management.
Managing these factors in an active, rather than re-active, manner depends
on planning for them. We have shown in the preceding chapter that planning
was lacking, particularly in the early, formative, days of the project. One
result of this poor planning is that there is little material visible on the
Authorities’ side of the contractual boundary to enable a detailed assessment
of Pathway’s project controls or its risk or quality management.
However, there is sufficient material to form an opinion, which should be
tested and verified were access to Pathway’s own documeniation and staff to
become available.
7.2. Project Controls
7.2.1. Why Have Project Controls?
One of the key objectives of having project controls is to enable the
progress of the project to be measured. By putting a sound project
plan in place, these controls can be applied, and progress stated, in
terms of deviations from that plan. In this way, progress reports are
on a basis of common understanding.
With a proper plan, delays to tasks early in the schedule can be
detected and evaluated to determine whether they will delay later
tasks, and eventually delay the project, or whether the slippage can
be corrected. Even if overall slippage proves unavoidable, there is
early warning of the problems, and other parties may be able to adjust
their plans to minimise the damage. Without proper plans, delay may
only be recognised late. There will then be little opportunity to
recover.
Figure 7.1 below illustrates a simple dependency network. Each box
represents a task, and the lines between them represent the order in
which they must be completed.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 44 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
: ~?i- 35 days i:
: t— todays EE todays PE 15 cays
Task $
(12 days)
Task 6
(15 days)
Extemal Action 1
Task 3
(19 days)
Figure 7.1 - Illustration of Task Dependencies
In figure 7.1 above:
e task 4 cannot start until tasks 1 and 2 have been completed;
¢ task 5 cannot start until task 4 has completed and external action 1
has been satisfied;
¢ task 6 cannot start until tasks 3 and 4 have been completed.
The elapsed time to complete all the tasks is expected to take 35 days
(Task 1 + Task 4 + Task 6). This is the critical path through the
network. Any delay to these tasks will automatically delay the ‘Finish’
point. Tasks 2, 3 and 5 all have some ‘slack’ time. Task 2 could be
delayed by two days without affecting the finish, task 3 has one days
slack and task 5 three days. It should be noted that ‘external action 1”
must be completed before task 5 can start. This could, for example,
by the agreement of a specification.
If task 1 was delayed, say by two days, this would be picked up by the
project control system, at worst as soon as the delay occurred, but
perhaps earlier. There would still remain 23 days (35 minus 12), and
this might be sufficient to, for example, assign additional resources to
task 6 so that it could finish two days earlier. Thus the network would
complete on time. If the delay was not picked up until day 35, there is
of course no time for remedial action.
External action 1 is required no later than day 23 (35 minus 12), and if
this were not to happen, then completion of the network would be
delayed by one day for every day the external action went
uncompleted. This would be a key dependency for the project to
manage, as, being external, the project would have less control over
it.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 45 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
7.2.2. What Actually Happened
The project has been subject to continuous delays, as set out in
Chapter 6. A number of these delays were announced by Pathway at
very short notice, and this is in our view prime evidence that it was not
controlling the project in accordance with good industry practice.
Pathway submitted a paper “Selection of Examples of Problems
Facing Pathway as set out in the Pathway Position Paper Dated 6
March 1998’. This undated paper, of 53 pages and marked “Without
Prejudice” contains a large list of complaints by Pathway, many of
which allege delays by the Authorities in supplying information which
Pathway needed to continue development of the system. None of
these allegations contain any statements of the form ‘Pathway needs
this information by [date] if the project is not to be delayed’. In our
earlier reviews of the project [April 1998 and August 1998] we also
found no evidence of such statements. In our view this points firmly to
there being no project plan and no project control.
This lack of control we find quite perplexing. Where software
suppliers undertake fixed price software development it is very much
the norm for any delay by the customer, for example in supplying
information, to be cited as a reason for delay and to support the
recovery of increased costs. In the environment where a plan has
been agreed between the parties, this is a reasonable, if often
contentious, approach, particularly as it works both ways.
The software development for the Card Payment programme
incorporates a fixed price software development contract, albeit one
where the development is paid for out of future revenue. We would
have expected Pathway to take a ‘hard nosed’ attitude toward any
delay by the Authorities in meeting planned dates.
Planning by Pathway has improved since the end of 1997, and we
would expect that project controls would improve in parallel. Our
review of the minutes of the Horizon checkpoint meetings from April
1998 confirms that such improvement has indeed taken place.
7.2.3. Conclusion
We conclude from the short notice of delays provided by Pathway,
and its inability to define firm dates by which actions by the Authorities
are required to avoid delays, that for the first 18 months of the project
Pathway:
e had no project controls in place;
¢ was driven by events as they unfolded, not by careful monitoring of
the on-going project, and thus had no means of either making up
lost time or of providing early warning of delays. Pathway was not
in control of the project.
Such a situation does not provide the level of project control which
good industry practice demands or which is offered by the PRINCE/2
methodology.
Prepared by Project Mentors Ltd, acting as Ref: A.41,05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 46 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
7.3.
7.4.
Risk Management
It is a fundamental requirement of project management that the risks to the
project are identified, assessed and measures to control them put in place.
While we do not have access to any risk register which Pathway may
maintain, it is clear that there were risks to the project from external sources,
notably from the Authorities. There is no doubt that the drop down process
and the Contracting Authority Responsibilities (CARs) represented risks to
Pathway’s system development project.
Pathway recognised this in its paper “Selection of Examples of Problems
Facing Pathway as set out in the Pathway Position Paper Dated 6 March
1998, in which it cites the drop down period {Item 7.] and Contracting
Authority Responsibilities [Item 10.]. In both cases the paper complains of
increased costs and delays, and with respect to CARs, of disruption.
We have found no evidence in our reviews of the PDA Board meeting
minutes, nor in the Core Negotiating Team (CNT) minutes for the period to
April 1998 of any attempt by Pathway to add these two matters to a risk
register. Yet from the award of contract these, together with Agreements to
Agree (A2As), represented external activities which impinged to some extent
on the development of the software. It should be noted that the majority of
issues during drop down, the CARs and the A2As relate to the provision of
the services under the contract. Only a relatively small minority have an
impact on the development of the software.
We would have expected that, under a PRINCE/2 project approach, Pathway
would have established a risk register, and shared this with the Authorities so
that a common understanding of the size, impact and priority for resolution of
the risks could be established. The risk register should have been the
subject of regular review with the Authorities so that all parties could agree
the responsibilities and actions necessary to reduce or eliminate those risks.
We find no evidence that this was done at any appropriate level of reporting.
We conclude that Pathway failed to manage risk in accordance with good
industry practice.
Quality Management
Even before the award of contract it is clear that Pathway was well aware of
the need for quality management. Its proposal [Annex 2. Pathway Quality
Policy] states at item 2.1 that:
‘2.1. GOALS
Pathway’s goal is to meet or exceed BA and POCL expectations
throughout the life of the contract. This goal will be achieved by:
{a) Adopting the Strategic Quality Mode! (SQM) within Pathway for
Total Quality Management, driven by customer needs.
(b) Establishing and implementing a Quality Management System
and relevant processes, e.g. Subcontractor Performance
Management, Service Level Management, auditing and
performance analysis and improvement.
Prepared by Project Mentors Ltd., acting as Ret: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 47 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
(c) Gaining !SO9001 or equivalent accreditation via Third Party
assessment, and ensuring aif sub-contractors meet this
standard,”
Later in Annex 2, item 2.4 states that:
"2.4 QUALITY MANAGEMENT SYSTEM DOCUMENTATION
The Quality Management System will be supported by a
comprehensive set of documentation describing appropriate
processes and procedures, in accordance with the requirements of
109001. This will consist of:
(a) Quality Manual, which will state Pathway’s policy, define
end-to-end processes and provide a documented set of
managerial instructions.
(b) Control procedures, which will define detailed methods and
controls used to assure Quality, for example design
procedures and standards, subcontractor standards, change
procedures, audit procedures.
(c) Work instruction which will define procedures for specific
tasks, for example test specification, upkeep of
documentation, meeting minutes.”
At award of contract, Clause 5 to Schedule A2 of the related agreements
calls for:
no CONTRACTOR'S POLICIES AND STANDARDS
$1 Quality Management System
5.1.1 The CONTRACTOR shall operate a quality management system
which complies with BS EN 1SO 9001:1994 for all its activities within
the scope of the AUTHORITIES’ Agreement.
5.1.2 The quality management system shall be applied to all aspects of the
delivery of Services hereunder.
5.1.3 The quality management system shall be audited and certified by a
BSI accredited auditor, who is independent of both the
CONTRACTOR and of the AUTHORITIES:
(a) in any event, at intervals of not longer than twelve (12)
months, and
(b) in addition, within twenty (20) Working Days of any such
request.
5.1.4. The CONTRACTOR shail within one (1) month of each audit (i)
provide the AUTHORITIES with copies of all reports produced by the
auditor on the quality management system, and (ii) notify the
AUTHORITIES of and carry out the CONTRACTOR's proposed follow
up actions where required.”
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 48 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
Given this proposed and contracted commitment to quality management, we
would have expected Pathway to use the four main components necessary
for effective quality management:
¢ a quality system;
© quality assurance;
quality planning;
© quality control.
The last of these, quality control, is the only aspect which we have had the
opportunity to review in terms of the design documents passed by Pathway to
the Authorities. The technical documents we have reviewed appear to have
passed through quality control procedures. However, many of them required
several reviews by the Authorities before they could be accepted, with many
comments on each version. This suggests to us that quality control
procedures concentrated on the form rather than the substance of the
documents. We regard this as a weakness in quality control.
We have had no opportunity to review directly Pathway’s quality management
system. However, quality control comes from the planned application of a
quality system which in itself is defined and maintained by quality assurance.
We therefore believe that the weakness of quality control points firmly to
inadequacy, and possibly non-existence, of some or all of these components.
We also understand from the Joint Contracts Team that in the event Pathway
has not gained ISO9001 or equivalent accreditation, although it is not known
whether or not it has attempted to do so. 1SO9001 is not strictly required
under the contract as clause 5.1.1 to schedule A2 states, “The
CONTRACTOR shail operate a quality management system which complies
with BS EN ISO 9001:1994 for ail its activities within the scope of the
AUTHORITIES’ Agreement” (our underlining). However, if Pathway's quality
management system truly does comply with 1SO9001, then, we find it
surprising that it has not been so accredited. If it does not, then Pathway is in
breach of the requirements of clause 5.1.1.
We are also a little surprised that the Authorities have not availed themselves
of their rights under clauses 5.1.3 and 5.1.4 above in relation to the software
development project. At least two annual audits under 5.1.3 should have
taken place already (May 1997 and May 1998) with another imminent.
Our conclusions are based primarily on the uneven quality of documents
presented to the Authorities and the amount of time necessary to review and
revise them before they are acceptable. We believe that Pathway has failed
to apply quality management to the development at a level in accordance with
good industry practice.
Prepared by Project Mentors Ltd., acting as Ref: A.41,05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 49 of 57
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
APPENDIX A - DEVELOPMENT SERVICES
In determining that Pathway is responsible for developing the software required to
support the Card Payment Programme and for managing the project to develop that
software, we reviewed the contracts for the programme and the relevant schedules
to those contracts.
There are three contracts involved, known as the Related Agreements. One is
between Pathway and both DSS and POCL. The other two are agreements between
Pathway and the DSS and between Pathway and POCL respectively. We have used
the following acronyms to distinguish the agreements in this Appendix:
JNT: The ‘Joint’ Authorities Agreement
DSS: The Benefits Agency Agreement
POCL: The Post Office Counters Agreement
All: All three agreements
Relevant Clauses
201(DSS) Contractor to meet the requirements specified in Schedule A15 in
accordance with solutions specified in Schedule A16 by performing
the Basic DSS services referred to in clause 201.2:
DSS Development services in clause 402
Roll out services in clause 404
DSS Steady State services in clause 405
Management services in clause 602
DSS Contingency services in clause 409
Transfer services in clause 906
201(POCL) Contractor to meet the requirements specified in Schedule A15 in
accordance with solutions specified in Schedule A16 by performing
the Basic POCL services referred to in clause 201.2:
POCL Development services in clause 402
Roll out services in clause 404
POCL Steady State services in clause 405
Management services in clause 602
POCL Contingency services in clause 410
Transfer services in clause 906
401(JNT) Contractor to develop:
Service Architecture Design Document
All necessary interfaces to create the Service Architecture.
402(DSS) Contractor to develop:
Optional DSS Services (Schedule C1)
Card Management Service (Schedule E1)
Payment Authorisation Service (Schedule D1)
Service Architecture Design Document
DSS Contingency Services (Schedules D9 & E9)
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 50 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
402(POCL)
601(All)
602(All)
602(DSs)
602(POCL)
605(DSS)
605(POCL)
702(All)
706 (All)
Contractor to develop:
Optional POCL Services (Schedules C1 & H1)
Benefits Encashment Service (Schedule D1)
Automated Payment Service (Schedule E1)
EPOSS (Schedule F1)
POCL Infrastructure Services (Schedule G1)
Service Architecture Design Document
POCL Contingency Services (Schedules D9,E9,F9,G11 &
{optionally)H9)
Authorities entitled to monitor Contractor performance per Schedule
A4 procedures.
Contract management performed per Schedule A4
DSS Services to be managed per Schedules D5 & E5
POCL Services to be managed per Schedules D5, E5, F5 & G7.
DSS undertakes to provide all information, services, facilities and
responses designated as DSS responsibilities in Schedules D3 & E3.
DSS to use all reasonable endeavours to perform these to any agreed
timetable per Schedule B9.
POCL undertakes to provide all information, services, facilities and
responses designated as POCL responsibilities in Schedules D3, E3,
F3 & G5. POCL to use all reasonable endeavours to perform these to
any agreed timetable per Schedule B9.
If it cannot be resolved by the Group within 14 days, it shall be
referred to Expert resolution or the English courts.
Performance of Services. Pathway undertake to use appropriately
skilled staff, and use good industry practice in the discharge of its
obligations. Sub-clause 702.2 is of particular relevance to this report.
Statements and Representations.
Relevant schedules
AA(All) Contract Management:
1 Objectives
(JNT) To monitor and manage the relationship between the three parties
and to authorise actions affecting the interests of both Authorities.
(DSS) To monitor and manage the delivery of the DSS services and to
authorise actions which improve those services.
(POCL) To monitor and manage the delivery of the POCL services and to
authorise actions which improve those services.
2 Organisation
(JNT) A Contracts Steering Group to be established to co-ordinate the
agreement.
(DSS) A Contract Administration Group to be established to co-ordinate the
agreement.
Prepared by Project Mentors Lid., acting as Ref: A.41.05 V2.1 Date: Aprit 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 51 of 57
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
(POCL) A Contract Administration Group to be established to co-ordinate the
agreement.
D3, E3 (DSS) DSS Responsibilities
Lists specific DSS responsibilities
D3, E3, F3, POCL Responsibilities
G5 (POCL)
Lists specific POCL responsibilities
Prepared by Project Mentors Ltd., acting as Ref: A.41,05 V2.1 Date: April 1999
Page 52 of 57
sub-contractors to Bird & Bird, Solicitors Status: Provisional
Privileged and confidential. Bird & Bird Prepared in contempiation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project_ Management
APPENDIX B - DEFINITIONS OF TECHNICAL TERMS
This Appendix sets out a number of the technical terms used in this report, together
with our definitions of those terms.
BA
B.2
B.3
Requirement(s)
A business need which is to be provided, made possible or not prevented by
the system. Requirements may be defined at significantly different levels of
detail:
« high level, which encompasses a very broad business need. If for
example the aim was to produce a new billing system for a telephone
company, one high level requirement might be “the system must be able to
accept regular and variable payments by direct debit”;
e detailed level, which, building on the example above might state “the
system must generate a customer advice setting out details of the
payment to be directly debited to their account” AND “the system must be
able to correctly account for direct debits returned unpaid by a Bank” AND
“the system must notify the account manager and customer in the event of
a direct debit being unpaid”.
It is quite possible for requirements to be stated at levels between the high
and the detailed. Such intermediate stages provide a better understanding of
the complexity of the higher level requirements as they are analysed (see
“Analysis’). This increased understanding can provide early insights into how
the eventual system should be organised (perhaps as linked sub-systems
rather that as a single system), or indeed whether the requirements can be
met at all.
Specification
A specification is a document which contains sufficient information to enable
an item to be built entirely from that specification without further explanation.
For example, an engineering part specification might state “take a bar of
10mm diameter EN8M steel, face the end flat to a surface finish of +
10microns, centre drill BS1, turn down to 8mm + 0.05mm for a distance of
20mm + .05mm and part off”.
Such a specification would enable any reasonably experienced turner with
access to the right tools, raw materials and measuring equipment to make
the part with no reference to anyone else.
For IT systems, specifications are produced for the detailed requirements, for
the system itself, for each program which makes up the system, and for the
processes of testing the system and ensuring the system meets the users’
requirements.
Analysis
The process of breaking down requirements at a particular level into more
detailed requirements at a lower level, a very critical process in IT systems
development.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 53 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
B.4 Design
In this document “Design” refers to the process of system or program design:
* System Design is the process of producing a specification for each of the
major components which will comprise the full system. Using the example
of a telephone billing system, there might well be sub-systems to cater for:
* the maintenance of information about customers, such as adding
new customers changing names and addresses, altering bank
details, deleting customers who decide not to use the service;
* generating bills from call information, despatching those bills,
reconciling them with cheques or other payments received,
handling direct debits;
* accounting for payments and receipts;
* providing analyses of calls and payment patterns or other
information useful to the direction of the enterprise.
e Program Design results in a specification for each program in each
system component. Such a specification details:
* the output data the program must produce;
* the input data it can use to produce the output;
» the processing rules which determine how the outputs are
formulated from the inputs, for example “if the input billing marker
is “direct debit” then generate a direct debit advice and direct debit
record for the Bank, otherwise generate an invoice and a request
for payment’;
= various technical parameters such as what to do with errors, the
programming language to be used and links to the operating
system. (This is not an exhaustive list).
B.5 Quality
The quality of an IT system may be defined as its “fitness for purpose”. A
telephone billing system might be deemed to be fit for purpose if only, say,
one bill in ever 10,000 was wrong (although overbilled customers might not
agree!). An air traffic control system could not be deemed fit for purpose if it
allowed one crash in every 10,000 landings. Although both systems require
that they be built to a certain level of quality, that level, and the cost of
achieving it, is very different.
A computer system is built from a series of specifications of increasing detail.
From these specifications, software is developed, and then tested, first each
program in isolation, then as groups of programs and finally as a complete
system. If quality is built into each specification and program as it is
developed, the end result will be a system fit for purpose, and testing
becomes the final quality assurance operation.
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.4 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 54 of 87
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
B.6
If the desired level of quality is not built into any system from the outset, the
only way of determining if it is fit for purpose is to test every combination of
data through every possible path through the system. Such a task is
essentially impossible for any but the most trivial system.
Testing
Testing should really be a particular case of quality assurance, in that at each
of the levels set out below assurance is sought that the system or component
performs to its specification, a specification which has itself been quality
checked. The levels are:
« Acceptance Testing, which is normally specified and conducted by the
users of the system. Acceptance testing ensures that the system does
what the users need it to do, and does so in a way which fits into their
normal business processes;
e System Testing, is normally specified and conducted by the system’s
developers, although often in conjunction with user representatives. Its
purpose is to ensure that system satisfies all the requirements set out in
the requirements and systems specifications. This will include a series of
technical, performance and resilience tests;
¢ Integration Testing, may be used as an intermediate step between
system testing and program testing, where the complete system is made
up of a number of inter-dependent sub-systems. Integration testing is
used to ensure that each sub-system operates correctly on its own before
testing its interfaces to other areas;
e Program Testing. Each program is tested to ensure it performs in
accordance with its program specification. Such tests are usually
specified by the author of the program design (preferably as part of that
design process) and either conducted by the programmer who wrote the
program with results checked independently or conducted independently.
Prepared by Project Mentors Ltd., acting as Ret: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 55 of S7
POL00093654
POL00093654
Privileged and confidential.
Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project. Management
DOCUMENT CONTROL
Change History
Version Date Status Purpose
1 11 Feb 1999 Draft Reviewed by PML 22/2/99
1.2 15 Feb 1999 Draft For review by PML on 26/2/99
1.3 9 Mar 1999 Draft For review by PML on 9/3/99
1.4 9 Mar 1999 Draft Report Development only
1.5 18 Mar 1999 Draft For review by PML on 23/3/99
1.6 23 Mar 1999 Draft Minor changes from review on 23/3/99
1.7 25 Mar 1999 Draft Major restructuring of Chapter 6
1.8 31 Mar 1999 Draft Changes from review on 31/3/99
1.9 8 Apr 1999 Provisional_IChanges from review on 8/4/99
2.0 19 Apr 1999 I Provisional_I Further changes from review 8/4/99
2.1 28 Apr 1999 I Provisional _IChanges from review 26/4/99
Changes Forecast
The review scheduled for 28 April 1999 is expected to yield no more than
changes to correct typographical and grammatical errors. The next version is
expected to be submitted for client review.
Distribution
Project Mentors Bird & Bird Benefits Agency POCL
ADD, JP, AJW Hamish Sandison — _
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: April 1999
sub-contractors to Bird & Bird, Solicitors
Status: Provisional
Page 56 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Project Management
References
No. [Reference
1 Pathway Systems Development Project - Systems Development Approach
[Ref. A.41.06]
2 Pathway Systems Development Project - Requirements Review
[Ref. A.41.07]
3 Pathway Systems Development Project - Management Summary
[Ref. A.41.08]
4 Contract between Pathway and the DSS and POCL, the ‘Authorities
Agreement’
5 Contract between Pathway and the DSS, the ‘DSS Agreement’
6 Contract between Pathway and POCL, the ‘POCL Agreement’
7 PRINCE/2 Manual - CCTA
8 PDA Master Plan Version 3.0
9g Horizon Master Plan Version 4.0 / Integrated Programme Plan
10__I Horizon Programme Replan Summary
11 IPathway Response to OJEC Notice 94/S 165-58937/EN (The Pathway
Proposal)
12__IStatement of Service Requirements Final Version 6.0 of 13/4/95
13__IMinutes of the PDA Board Meetings
14 __I Minutes of Horizon Checkpoint Meetings
15 {ICL Pathway Programme Review July 1997
16__IPathway New Release 2 Replan Review
17 {Selection of Examples of Problems Facing Pathway as set out in the
Pathway Position Paper Dated 6 March 1998. An undated paper of 53
pages, marked ‘Without Prejudice’
18 __I Minutes of the meetings of the Core Negotiating Team (CNT)
Prepared by Project Mentors Ltd., acting as Ref: A.41.05 V2.1 Date: Apri! 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 57 of 57
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
REPORT
PATHWAY SYSTEMS DEVELOPMENT PROJECT
REQUIREMENTS REVIEW
Ref: A41.07 V1.5
Status: Provisional
Prepared : April 1998
Prepared by Project Mentors Ltd., acting as sub-contractors to Bird & Bird, Solicitors
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
SECTION PAGE
1 INTRODUCTION 1
1.1 Context 1
1.2 Scope of The Report 1
1.3 Purpose 2
1.4 Constraints 2
1.5 Structure of Report 2
2 REVIEW APPROACH 3
2.1 Introduction 3
2.2 Analyse Requirements 4
2.3 Enhance LDM & DFD with Pathway Design Details 4
2.4 Stages of Analysis 4
2.5 Time Scale 5
3 REQUIREMENTS ANALYSIS 6
3.1 Introduction 6
3.2 Payment Card System Outline 6
3.3 Diagram Notation 7
3.4 BPS 8
3.5 EPOSS 7
4 OBSERVATIONS 18
4.1 General 18
4.2 EPOSS 19
DOCUMENT CONTROL 21
Change History a1
Changes Forecast 2t
Distribution at
References 2t
Source Documents 22
Prepared by Project Mentors Ltd., acting as sub-contractors to Bird & Bird, Solicitors
i
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
1
1.2
INTRODUCTION
Context
This report is one of a set of related reports commissioned to support
potential litigation and negotiations for settlement of that litigation with respect
the to the Card Payment Programme. The structure of this set of related
reports is illustrated in figure 1.1 below, which highlights the position of this
report within that structure.
Pathway Systems Development Project
Management
Summary
Ref A.41.08
Project aueme
Management evelopment
s Approach
Ref A.41.05 Ref A.41.06,
Requirements
Review
Ref A:41.07-
Figure 1.1 - Structure of Related Reports
Our report “Project Management’ {Ref. A.41.05] sets out our assessment of
Pathway's management of the project, while the “Systems Development
Approach" report [Ref. A.41.06] describes our findings with respect to
Pathway’s development approach. This report - “Requirements Review” [Ref.
A.41.07] - presents our analysis of certain areas of the Authorities
requirements. The overall findings of our review are set out in “Pathway
Systems Development Project - Management Summary“ [Ref. A.41.08].
ft should be noted that there are links between the management and
technical factors, and ideally this report should be read in conjunction with its
companion papers.
Scope of The Report
In our initial report [Ref. Expert Review of BA/POCL Payment Card
Programme - Initial Report August 1998, Project Mentors.] we stated:
“It is our impression that ICL Pathway have been primarily responsible for the
delays to the programme by seriously under estimating the effort and time
needed to develop the services and, as a result, not allocating sufficient
resources to complete their contracted obligations within the agreed
timescale.”
Prepared by Project Mentors Lid., acting as Ref: A.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 1 of 22
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
As a common cause of such a failure is poor analysis of business
requirements, we have performed a review of Pathway's approach to this
analysis. We have examined the business requirements expressed at the
time of contract signing and compared them against the current
understanding of the Authorities' requirements.
This work was based on documents from both the Authorities and Pathway,
together with very limited informal discussion with BA staff at Terminal
House.
1.3 Purpose
This report has been commissioned to support potential litigation and
negotiations for settlement of that litigation. It therefore aims to provide
negotiators, who may not be wholly familiar with IT developments, with an
assessment of Pathway’s performance in this area.
Because people without a systems background may use the report, we have
striven to provide a view that we trust will be readily comprehensible. We
have as far as possible avoided the use of IT “jargon”. However, the concepts
described in this report need to be understood if the scale and impact of
failures in following good systems development practice are to be
recognised.
1.4 Constraints
Currently (April 1999) we do not have access to the great majority of Pathway
documents or to their staff. In consequence, our findings are based on
relevant items in the Authorities possession.
1.5 Structure of Report
The report is structured into the following chapters:
¢ Chapter 1, this chapter, presents background information;
* Chapter 2 describes our approach to the review;
e Chapter 3 presents the results of our analysis;
e Chapter 4 presents a number of observations that result from our
analysis.
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 2 of 22
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
21
REVIEW APPROACH
Introduction
We were surprised to discover that no detailed analysis of the requirements,
an essential process for successful IT development, appears to have been
performed. To allow us to compare current requirements with the original
requirements, the review team therefore initially selected the BPS (Benefits
Payment System) element of the system as a sample, and assembled a
draft requirements analysis. This allowed us to consider Pathway's claims
regarding DSS requirements. We then performed an equivalent exercise on
POCL's EPOSS (Electronic Point of Sale System) requirements.
Requirements analysis is the vital first step in tuming high level business
requirements into systems specifications from which software can
successfully be developed. We had anticipated therefore that our comparison
would be between a requirements analysis post-contract and the current
version of that analysis. We could find no evidence in either the Horizon
library list or DSS libraries of such an analysis. Discussions with DSS &
POCL staff at Terminal House and DSS staff at Norcross and Longbenton
failed to identify any unrecorded but relevant documents.
A direct comparison between the requirements expressed in the catalogue
(text) and those implied in Pathway's design documents (mostly text) was
not feasible because of different structure, inconsistent naming, etc. To
enable us to compare like with like, we analysed the requirements in the
catalogue using Logical Data Modelling and Data Flow Diagrams and then
compared the functions and data expressed in Pathway's design documents
with the analysis. This approach is illustrated in figure 2.1 below.
_—
—
Analyse I_I Analysis
Requirements I “I. DM &DFDI[J
Enhanced
LDM &DFDj
Structured
English
Contract
Requirements
Catalogue/SSR}
Enhance
LDM & DFD
with Pathway
Design Details I
Documents }F
List of
Differences
Figure 2.1 - Comparison Approach
Prepared by Project Mentors Ltd., acting as Ref: 4.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 3 of 22
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
2.2
2.3
24
Analyse Requirements
This work was based initially on the final version of the Authorities’ Statement
of Service Requirements (“SSR”). This document was issued to potential
suppliers in April 1996. While not a contractual document, in our view this is a
well produced and thorough document which would have given any potential
supplier the opportunity to gain a thorough understanding of the system
through analysis and questioning.
We then examined the Contract Requirements Catalogue to ensure we had
complete coverage of the Authorities requirements. The output from this
process was a draft Logical Data Mode! and Data Flow Diagrams
Enhance LDM & DFD with Pathway Design Details
The documents reviewed at this stage were principally those produced by
Pathway, although we also considered a number of CAPS and POCL
definition documents. The principal aim of the activity may be expressed as -
"Is there any entity, attribute or functionality described in the Pathway
documents that is not present in the Authorities requirements"?
We derived the implied business requirements from the Pathway design
documents, discarding design detail not relevant to the Authorities business
requirements. We then matched them to the Authorities requirements in their
analysed form. Because of inconsistencies in naming of entities and
attributes in different documents it was necessary to make some
assumptions about equivalence.
We also produced definitions of the principal DSS functions. These are
written using indentations to show a logical structure, often described as
structured English.
The results of this second stage of analysis were:
* anenhanced LDM and DFD;
* structured English descriptions of those DSS business rules where
Pathway had raised the issue of ‘optimisation’;
« a list of the differences between the Authorities’ requirements and their
requirements as implied by the Pathway design documentation.
Stages of Analysis
Our initial analysis focused upon the DSS' requirements. This enabled us to
consider the following Pathway claims of ‘optimisation’. The results of this
evaluation were recorded in our Position Paper on requirements optimisation
[Ref. Position Paper - Requirements Optimisation, Project Mentors document
A.42.07]
The second stage of analysis focused upon EPOSS. This area of
functionality was chosen for the following reasons:
« EPOSS is the foundation of POCL's business requirement;
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 4 of 22
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
* The BES component of the DSS' requirements runs as an application
within the EPOSS environment. It was therefore important to understand
how the two interacted;
« EPOSS is controlled, to a large extent, by POCL's Reference Data.
Reference Data has been an issue raised by Pathway. We wished to
understand the issue better.
2.5 Time Scale
Our high level analysis of the BPS requirements took four weeks effort from a
single consultant, spread over a period of two months.
Requirements analysis is an iterative task, which makes it difficult to be
precise about the amount of time spent on each individual activity. However,
an approximate break down would be:
¢ Initial (SSR) data model - 2 days
¢ Current data model - a further 3 days
« DFDs- 1 week
e Structured English - 2 weeks
The analysis of the EPOSS requirements took some six weeks effort. This
reflected the more extensive requirements of POCL..
We would like to stress that we have not produced a fully detailed analysis of
requirements for BPS or EPOSS. We have taken the analysis only to that
level which enabled us to determine whether Pathway’s claims of optimisation
and lack of clarity were valid. It is our view that a detailed requirements
analysis could have been produced in no more than nine months, and
probably less, given:
* the right number of appropriately skilled staff;
* correct and speedy resolution of queries by the Authorities.
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 5 of 22
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
3 REQUIREMENTS ANALYSIS
3.1 Introduction
This chapter presents the results of the analysis in the form of Data Flow
Diagrams (DFDs) and Logical Data Models (LDMs). The details of the
analysis have been recorded using the System Architect CASE (Computer
Aided Software Engineering) tool and the diagrams are an output from the
tool. Greater detail has been recorded than is presented here. In particular,
the names of all the attributes of the business objects have been entered in
the CASE tool.
3.2 Payment Card System Outline
The following text and diagram is reproduced from the SSR:
Figure 3.1 below “shows the component services (in boxes) within the
procurement service boundary (the large oval) that are to be provided by the
Service Provider. Outside the procurement service boundary are the
Services (in boxes) and users (in small ovals) with which and whom the
Service Provider will interface. The dotted box around Value Added
Processing (VAP) is to indicate that the Transaction Management Service
(TMS) is part of VAP”.
Procurement Service Boundary
Client
ryan Systems
Added I
Processing I
Sanat ‘Transaction
5
Attn Senge yeaa
CAPS = ‘Gparaional “Finance
POCL STRATEGIC ‘Support ~ Distribution
Card INFRASTRUCTURE SERVICE Seraces = Value Added
Management Provessing
Benet Senice Conier
Offices interface
POCL
Users
Benet
Customers
Bost Office
Clerk
Post Office
Customers
Figure 3.1 - The Service Architecture
We have used the Procurement Service Boundary to define the scope of our
analysis. The analysis is divided into the following components:
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 6 of 22
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
« BPS - The end-to-end service defined by the Authorities and represented
in the above diagram by Payment Authorisation Service (PAS), Card
Management Service (CMS), and that part of Counter Interface that is the
Benefit Encashment service (BES). The separation of BPS into these
components is a constraint upon the technical solution, however it is not a
business requirement of the Authorities. For our analysis of business
requirements, the separation has been excluded;
e EPOSS - The major part of the Counter Interface.
3.3 Diagram Notation
The analysis is represented by a series of diagrams. The notation used is as
follows.
3.3.1 Data Flow Diagrams
A Data Flow Diagram (DFD) illustrates the way in which data (or
information) is passed around the system, and how it is transformed
and stored within the system. A diagram shows the following:
External Entities - Rectangles at the edge of the diagram. The
people, organisations and other computer systems that act as
sources of data to, or recipients of data from, the system - for
example Counter Clerks, Help Desk staff and the CAPS system;
Processes - Soft cornered rectangles. These represent business
activities carried out upon and triggered by data. They should not
be confused with computer programs. A process may sometimes
equate directly with a program, but even then will be defined in
user terms rather than in computer jargon. In other words, it
should reflect the business activity it supports;
Data Stores - Narrow rectangles marked with the letter 'D'. These
are, as their name suggests, stores of data within the system.
They may be computerised data stores or manual data stores
(filing cabinets etc.);
Data Flows - These are arrows which represent the flows of data
to, from and within the system.
3.3.2 Logical Data Mode! Diagrams
A Logical Data Model (LDM) uses the following notation:
Boxes - represent the main objects of interest to the business, for
example Outlets and Counter Clerks. These are also termed
business entity types;
Lines - represent the nature of relationships between entity types.
For example an Outlet may have one or more Counter Clerks
working at the Outlet and a Counter Clerk must work at one and
only one Outlet.
Prepared by Project Mentors Ltd., acting as Ref A.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 7 of 22
POL00093654
POL00093654
Privileged and cont
fidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
3.4 BPS
3.4.1. BPS Functionality
In
.
outline, BPS provides facilities for:
Maintaining customer details;
Providing and validating encashment tokens (payment cards and
temporary tokens);
Entering customer payments;
Validating and recording encashment of payments;
Recording details of customer sessions at Post Offices;
Maintaining details of Post Offices and other standing data.
3.4.2 BPS Diagrams
The following diagrams provide an overview of the analysis:
1.
BPS High Level Data Flow Diagram: The principle functions of the
BPS requirement. These functions are performed by a
combination of PAS, CMS and BES.
2. BPS Logical Data Model: The business objects of interest to the
Benefits Agency.
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: April 1999
sub-contractors to Bird
& Bird, Solicitors Status: Provisional Page 8 of 22
POL00093654
POL00093654
Privileged and confidential.
Bird & Bird
POL00093654
POL00093654
Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme: Pathway Systems Development Project - Requirements Review
Torelt cone
ancioniad PD chtogtyp]
coment
owe
Posi Oba?
Terai acer
Tine tae
arenes
change nonteead PO
elie?
peat daa fom
vomes
"SY caioger sou
unre dla or
‘ye
pad payne ale
are
abe
f
vena I Toni commas
‘Salons
Fi
5 ik
igure 3.2 - BPS Data Flow Diagram
ry
Prepared by Project Mentors Ltd., acting as sub-contractors
to Bird & Bird, Solicitors
Ref; A.41,07 V1.5
‘Status: Provisional
Date: April 1999
Page 9 of 22
}
Privileged and confidential.
Bird & Bird
POL00093654
POL00093654
Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme: Pathway Systems Development Project - Requirements Review
post
pen post pos
office office I,
he
(Tosed
post
office ai
old post offig nominat)
(change
nominated
post office
Q
the beneficiary
benefit
type
previous paymest
token type]
<)
authorised
payment
possibl
d
% (authorised
Sf payment
tokens
cashment office
post office
ater
iss office
Issudd by
post office
S post office
S token
post office
g od
c 4 O—oed
(encashment
clerk
tempor ary
token
Payment card 0)
t A
Counter
the bendgiciary of
horised as
31
(encashmen:
Qether
ncashment
the encash
(casual agent
session
token for
5 O)
IS)
payge sesslo
(customer
sion
encashment
Prepared by Project Mentors Ltd., acting as sub-contractors
to Bird & Bird, Solicitors
Ref; 4.41.07 V1.5
Status: Provisional
Date: April 1999
Page 10 of 22
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
3.5 EPOSS
3.5.1 EPOSS Functionality
In outline, EPOSS:
* provides post office counter clerks with support facilities in the
sale of Post Office/ Client products and services (BES and APS
being examples of services);
* provides stock ordering and management facilities
* provides administrative and reporting facilities
¢ is the accounting system for an outlet
POCL provides Pathway with details of all POCL's outlets, products
and services via a file interface. Pathway then ensures these details
are distributed to all Post Offices.
Pathway, in turn, provides POCL and POCL's clients with details of all
EPOSS transactions performed at Post Offices on their behalf.
3.5.2 EPOSS Diagrams
The diagrams reproduced below are:
¢ EPOSS High Level Data Flow Diagram: The principle functions of
EPOSS including the maintenance of POCL Reference Data.
« EPOSS Logical Data Model Subset - Stock Items: The business
objects that describe the products and services sold and provided
at an Outlet.
¢ EPOSS Logical Data Model Subset - Scales: The data relevant to
calculating charges for objects weighed on counter scales.
* EPOSS Logical Data Model Subset - Stock Details: Information
gathered during stock balancing activities.
« EPOSS Logical Data Model Subset - Sessions: Information
gathered about counter sessions and the types of session.
« EPOSS Logical Data Model Subset - Transactions: Information
gathered about transactions performed within sessions.
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: April 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 11 of 22
Privileged and confidential.
i
Bird & Bird
POL00093654
POL00093654
Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme: Pathway Systems Development Project - Requirements Review
‘rairenonanI
Poorer
F
igure 3.4 - EPOSS Data Flow Diagram
To ear
Prepared by Project Mentors Ltd., acting as sub-contractors
to Bird & Bird, Solicitors
Ref: A.41.07 V1.5
Status: Provisional
Date: April 1999
Pago 12 of 22
I
I
POL00093654
POL00093654
} .
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme: Pathway Systems Development Project - Requirements Review
accounting I
node
No fiiont —
reporting
perlod
Qf
(stock Item
—~od
heck digit
method
itemlink type “84
item additional
fleld
Figure 3.5 - EPOSS Logical Data Model - Stock Items
Prepared by Project Mentors Ltd,, acting as sub-contractors Ref: A.41,07 V1.5 Date: April 1999
to Bird & Bird, Solicitors Status: Provisional Page 13 of 22
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme: Pathway Systems Development Project - Requirements Review
scales service
) class
geographical
area
I weight band
) service type
) d
service
Figure 3.6 - EPOSS Logical Data Model - Scales
Prepared by Project Mentors Ltd., acting as sub-contractors Ref: A.41.07 V1.5 Date: April 1999
to Bird & Bird, Solicitors Status: Provisional Page 14 of 22
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme: Pathway Systems Development Project - Requirements Review
ftock unit
portion
I
citent
a
I declaration
oS hugieevats
I Gtock croup
a cb erapancy — — .
point _ Gtoek Rem
- stock _ I }
rt —.__J
a porno) diook Rem
—~
————-f denomination Ip
Sine aassmaaatee tare denomination
declaration hi
—
Figure 3.7 - EPOSS Logical Data Model - Stock Details
Prepared by Project Mentors Ltd., acting as sub-contractors Rel: A.41.07 V1.5 Date: April 1999
to Bird & Bird, Solicitors Status: Provisional Page 15 of 22
Privileged and confidential.
Bird & Bird
}
POL00093654
POL00093654
Prepared in contemplation of litigation,
Expert review of BA/POCL Payment Card Programme: Pathway Systems Development Project - Requirements Review
enittance in
session
i
ary
yj
setion
de type
7
‘Sahustment
Sassioi
fer
er ‘Session
jechare neoative (ee
‘alscrepancy
tye customer
Session
Fi
igure 3.8 - EPOSS Logical Data Model -
Sessions
Prepared by Project Mentors Ltd., acting as sub-contractors
to Bird & Bird, Solicitors
Ref: A.41.07 V1.5
Status: Provisional
Date: April 1999
Page 16 of 22
POL00093654
POL00093654
} j
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme: Pathway Systems Development Project - Requirements Review
Stock item
fession
ffransaction
(fransaction tink
(method of paymen’ — I — se
ayment in (Stock unit
ky balance item
(iscoun other
transaction
Figure 3.9 - EPOSS Logical Data Model - Transactions
Gcales
transaction
Date: April 1999
Rof: A.41.07 V1.5
Page 17 of 22
Prepared by Project Mentors Ltd., acting as sub-contractors
Status: Provisional
to Bird & Bird, Solicitors
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
4 OBSERVATIONS
44 General
4.1.1 Changes in Requirements
We found no changes of substance in requirements between those
expressed in the contract and those ‘implied’ by Pathway design
documentation. However, as would be expected:
* there is more detail in the Logical Data Model derived from
physical design documentation, but noticeably only in terms of
their being additional attributes to each of the data entities. There
are no new data entities;
« the functionality is described in more detail in detailed design
. documents than it is in the contract. However, we found no
additional primary functions in the design documentation.
It is of interest to note that, without specifically attempting to, the
process also identified one or two BPS exception cases in terms of
business process. While Pathway and / or the BA may have identified
these cases, they are not documented in the programme’s technical
library.
4.1.2 Inconsistent Naming
There are a significant number of instances of business objects
relevant to both BPS and EPOSS but known by different names in
documents describing the requirements and/or design. There is thus
scope for misunderstanding which may become apparent during
testing.
During the analysis we have made assumptions on correspondence
of names. The following table presents these assumptions:
BPS EPOSS
Post Office Outlet
Post Office Counter Terminal
Counter Session User Logon
Customer Session Serve Customer Session
(sub-type of Session)
Encashment sub-type of Transaction
These assumptions have not been verified with any Authorities staff.
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 18 of 22
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
4.2 EPOSS
4.2.1
4.2.2
4.2.3
Requirements Match
The POCL business requirements are more extensive and complex
than those of the Benefits Agency. However, the EPOSS
requirements in the contract and the SSR are not as well structured or
detailed as those for BPS. Nevertheless, the requirements catalogue
does identify all the elements that are represented in the derived
model.
Stock Portion
When shared stock units are balanced, each counter clerk using the
stock unit has to enter appropriate stock declarations relating to that
part of the stock unit that they manage. They are required to enter a
‘portion id’ to uniquely identify their part. The portion id is not
registered with the EPOSS system. It is, in effect, an unchecked text
field. This appears to be a fudge. if EPOSS was properly supporting
shared stock units, the stock unit parts (Portions) would be properly
registered using administrative functions. Stock Portions are business
entities and we have therefore included them on the LDM, however
their details do not appear to be stored in the EPOSS ‘database’.
POCL Reference Data
POCL Reference Data is used to control the behaviour of the EPOSS
system as well as provide details of products, services and accounting
structures. The data is passed to Pathway who transform it into a
special coded form which is then distributed to all Post Offices.
POCL have produced a logical data model for the reference data.
Pathway only describes the data in its coded form. We therefore used
the POCL logical data model as primary source for our analysis.
The data model for POCL reference data describes both:
* a generalised network structure that allows any ‘node’ to
participate in multipte hierarchical structures;
* specific business structures such as an Outlet’s relationships to
such things as contract type, outlet default opening times, outlet
language etc.
This is compact in space terms, but difficult for business users to
understand and relate to their own requirements. It also hides the
difficulty at implementation time of combining attributes together from
different entities (e.g. combining address details from Organisational
Unit together with Post Office Outlet) or keeping the physical
implementation close to the logical model with implied additional costs
at implementation and run-time.
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: Aprif 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 19 of 22
POL00093654
POL00093654
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
Business functions tend to deal with fixed, not generalised, structures.
IT system designers implement databases as generic structures
because business changes can be reflected speedily when the
business re-organises (which most do regularly!). However, business
users need to talk about the functionality in the current structural
terms.
It is not clear how much of the POCL maintained reference data is
relevant to EPOSS. Some part of the generalised accounting and
organisational structures do not look to be supported by EPOSS. The
documentation does not clarify this point.
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: Aprit 1999
‘sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 20 of 22
POL00093654
POL00093654
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
DOCUMENT CONTROL
Change History
Version Date Status Purpose
11 5/3/1999 Draft Incorporates working papers on BPS
and EPOSS. For review by Project
Mentors team.
1.2 30/3/1999 Draft Version for review by Project Mentors
team.
1.3 2/4/1999 Draft Version for review by Project Mentors
team. Structure of document modified.
1.4 9/4/1999 Draft Version for review by Project Mentors
team. Contains minor amendments.
1.5 27/4/1999 I Provisional I Version for review by Bird & Bird.
Changes Forecast
No changes are expected other than for issues raised by Bird & Bird
Distribution
Project Mentors Bird & Bird Benefits Agency POCL
Andrew Davies Hamish Sandeson
Andy Wing
References
™ 1. Expert Review of BA/POCL Payment Card Programme - Initial Report August
1998, Project Mentors.
2. Selection of Examples of Problems Facing Pathway as set out in Pathway
Position Paper dated 6 March 1998
3. Pathway's System Development Approach, Project Mentors document A.41.06
Position Paper - Requirements Optimisation, Project Mentors document A.42.07
5. Review of BA/POCL Security Requirements, PA Consulting Group, 13” March
1998
6. Selection of Examples of Problems Facing Pathway As Set Out in The Pathway
Position Paper Dated 6 March 1998, undated Pathway paper.
Prepared by Project Mentors Ltd., acting as Ref: 4.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 21 of 22
Privileged and confidential. Bird & Bird Prepared in contemplation of litigation.
Expert review of BA/POCL Payment Card Programme
Pathway Systems Development Project - Requirements Review
Source Documents
The following documents provided the information from which the draft analysis was
performed.
DSS Documents
CAPS to PAS/CMS Data Interface Definitions and Validation Rules (Post Nile 2)
CAPS to PAS/CMS Codes Files Definition {Release 3)
DDS Agreement Schedule 15 - Requirements
DSS Agreement Schedule 16 - Solutions
DSS Agreement Schedule 01 - Interpretations
POCL Documents
Reference Data Project - Data Model, POCL document RDP/TEC/005 Issue 2,
5° January 1998
POCL Agreement Schedule 15 - Requirements
POCL Agreement Schedule 16 - Solutions
Pathway Documents
Functional Specification Version 6.0
Service Architecture Design Document, CR/FSP/0004 Version 5.1
Foreign Encashments CR/FSP/0009 Versions 4 and 5
CCN117 - Supporting Documentation, CRAON/CCN117
CCN 0083 - One Payment Receipt and One Signature Required for each
Transaction (PDA Change Request BO005)
CCN 220 — Restricted PO Indicator Operation
CCN 204a - Generate Card Stop following CMS End of Interest
CAPS Access Service High Level Design, SU/DES/0001
EPOSS Functional Description, BP/FSP/004 Version 3.6
EPOSS Solution Proposal, (no reference) Version 0.5, 07/09/1998
EPOSS Business Rules, EP/DES/008, Version 0.8, 14/10/1998
Prepared by Project Mentors Ltd., acting as Ref: A.41.07 V1.5 Date: April 1999
sub-contractors to Bird & Bird, Solicitors Status: Provisional Page 22 of 22
POL00093654
POL00093654