RESTRICTED - COMMERCIAL Prepared in contemplation of litigation
Notes from reading the Bird & Bird/Project Mentors Report
We have no evidence to disagree with the underlying message in this report. Our
experience supports the view that Pathway have failed to follow “best practice” ina
number of areas in their development of the service
Therefore we are not in a position to refute the statements made, however due to the
lack of visibility of Pathway’s process and documentation, we are also not able to
fully substantiate them. However, our experience and contact with Pathway,
together with the significant problems which we know they have had (as indicated
by results of testing, of major delays and slippages), would point to similar
conclusions.
The paper makes a number of very valid points regarding the development lifecycle,
including:
© “effective business requirements analysis is required to achieve a satisfactory,
comprehensive business design. This can then be used as the basis for the technical
design of the high resilience, high volume system.....”
We know that Pathway have paralleled up much of ther development lifecycle,
including designing & building much of the infrastructure in advance of
performing analysis and design at an application level.
¢ that it unlikely that a formal requirements analysis has been performed
“continuing problems experienced in the development of the system”
I would extend this to saying “formal top down system design process”
¢ = “without having complete an aalysis of the buisness requirement... Pathway can never
have been in a position to understand the detail of these requirements. Without this level
of detail, it is not possible to develop a system level specification...”
Certainly we know that they were “caught out” by the complexity - this is
supported by statements from Pathway staff, the massive slip, etc.
Our view over the past 2+ years of experiencing Pathway development has been::
¢ Horizon denied visibility, especially of application design (oddly, we have
managed to get some visibility of the ‘risk areas’ such as the Riposte
middleware).
=> no opportunity for Horizon to perform independent V&V
=> no evidence that Pathway have performed a V&V
e Pathway use the “IPR” card, however it appears that this is used to cover a
deficiency in their documentation (and therefore their methodology)... When we
do get exposure, documentation is patchy and to variable standards. Absence
of “high level” design - eg for BPS, aware of some detailed design documents for
Oracle, Agent and Counter parts of BPS, but not of overall BPS sysystem (nor, for
WITNO5970121
WITNO5970121
RESTRICTED - COMMERCIAL Prepared in contemplation of litigation
instance, the messaging etc).
[See problems at Ric with duplicates, with Recall and Reissue, etc]
[eg when trying to determine operation of BES, Pathway’s designers did not
appear to have access to formal design documentation, and appeared to have
difficulty in providing answers to questions]
e Pathway adopting an “end of pipe” approach to quality and performance - a “fix
on fail” in testing approach, rather than putting in the effort to get it right first
time.
e BPS was fairly well specified in SSR terms (overspecified) - likely that other
products, esp EPOSS, where considerably more requirements analysis needed,
will be worse.
e SADD, as the only officially visible “specification” - is a high level document -
does not cover how the service operates in detail. Patchy in detail, PAS/CMS
detailed, BES fairly weak, EPOSS very weak, hence need for “fill in” documents
such as the EPOSS FS.
e We have had to resort to documenting OUR understanding of the design, based
on discussions with Pathway and of using prototypes of the counter software,
and play these back to Pathway for confirmation.
¢ — Little evidence of thorough consideration of exception conditions or failure [qv
issues on EPOSS in LAN failure, etc]. The behaviour of the system in various
exception conditions is not documented and does not appear to be known to
Pathway.
[eg Incomplete Transactions at Ric, some ~18 failure conditions have been
identified, a number of which have required urgent fixes - why were these not
“designed out” from the start?]
¢ Pathway have tried to apply RAD - or claim to apply RAD - for totally
inappropriate stages of design. Even major supporters of the RAD
method[ology] freely admit that it is best suited to front end, user interface
development, rather than system design.
e Pathway have generally shown an inability or unwillingness to understand or
recognise the complete requirements set - qv failure to follow the (contracted)
security standards, some whole requirements missed (eg timeout back at Ric) etc.
Have tended to think they can develop a system without being bothered by the
detailed requirements or their meaning. Failure to use the “clarification”
mechanisms.
We would certainly agree that, from our perspective, we have no evidence to
support a view that Pathway have followed “best practice” in the development of the
Horizon system, and in particular the Horizon business applications.
WITNO5970121
WITNO5970121
RESTRICTED - COMMERCIAL Prepared in contemplation of litigation
Comments on the paper:
e little confused between “Payment Card System” - this appears to refer to the
entire programme - and BPS, which is “predominantly a BA system element”.
* concentrates on Requirements Analysis, however this is only one part of the
lifecycle. I see the weakness as going throughout, into Design etc.
e Circumstantial evidence mostly - due to the lack of access, Pathway can and are
likely to claim they do have all of this, it’s just that in a PFI it is inappropriate for
them to share this with the programme.
© Project Mentors have been restricted to the “input” (SSR and Requirements
however they don’t seem to mention the Requirements Catalogue particuarly) and the
only “output” they can find, the SADD, of what they consider to be the Analysis
process. This is a valid comparison, although given that the Authorities have
“signed off” the SADD in some way (ok, grudgingly at times), have we not in
some way underwritten it? Need to think on this one
WITNO5970121
WITNO5970121