POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
Bringing Technology to Post Offices and Benefit Payments
HORIZON PLAN FOR ACCEPTANCE DURING THE ICL PATHWAY
OPERATIONAL TRIAL
Reference: I BA/ACC/001
Author: Tain Kerr
Classification: Horizon controlled document
Version: 2.0
Date: 12/01/99
Status: Revisions to Approved Version
Authority: John Meagher, Horizon Acceptance Manager
Document Summary: Provides a high-level description of the Horizon Plan for the
management of Acceptance of the Operational Trial of ICL Pathway. This has been developed
as a PRINCE style Stage Plan. The paper indicates the source of detailed Horizon
documentation dealing with specific aspects of the task. It includes outline details of the
Horizon Acceptance organisation and resource requirements and the overall Acceptance Plan.
The procedures to be used and the tasks expected of those to be actively involved in the
process are covered in those supporting documents.
Contact Area Copy
J Freeman Horizon Library
J Meagher Horizon Product Assurance Manager and
Horizon Acceptance Manager
Dave Miller Horizon Programme Director
Peter Crahan CAPS Director
Paul Wootton DSS Acceptance Manager
Dave Smith POCL Automation Manager
Ruth Holleran POCL Business Assurance
Horizon Acceptance Test Managers
Horizon Acceptance Team
ICL Pathway Acceptance Manager (for
information)
att00002.dat Page I of 48 12/01/99
RESTRICTED COMMERCIAL
POL00029137
POL00029137
PWY REF: BP/PLA/016
Approved:
Approved John Dicks Paul Wootton John Meagher
ICL Pathway DSS Acceptance I Horizon Acceptance
Acceptance Manager Manager
Manager
Signature
Date
Nothing contained herein shall be deemed or construed as affecting existing
contractual obligations or creating new contractual obligations between ICL
Pathway, the DSS and/or POCL.
att00002.dat
Page 2 of 48
12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
DOCUMENT CONTROL
0.1 Version History
Version Date
0.1 23/09/98 First rewrite by Iain Kerr based on an earlier draft
paper and comments from a review of that paper.
0.2 25/09/98 Revision to include Peer Review comments.
1.0 30/10/98 Consolidating comments from Sponsor Review.
For Approval.
2.0 12/01/99 Revisions to Approved Version to include process
diagrams and to reflect results of tripartite audit of
Technical Testing. Some renumbering of sections
and paragraphs has been made to make the layout
more logical.
0.2 Related Documents
(Not all documents listed are referred to in this paper)
Ref__I Document Reference Ver__I Date
1 Contract Schedule A07 - Acceptance BA/POCL 72 22/05/97
Procedures
2 ICL Pathway Nile Release 2 High Level I ICL Pathway I 5.0 10/09/98
Plan PM/PLA/031
3 Horizon Programme Replan - Level 0 Horizon 5? 2/09/98
Plans
4 Pathway New Release 2 Contents ICL Pathway I 2.0 24 Feb
Description 98
5 Pathway New Release 2 Contents ICL Pathway I 0.6 28 Aug
Description 98
6 Service Architecture Design ICL Pathway I 4
Documentation
7 Contract Schedule C03 BA/POCL 5.6 I 24/09/96
8 Principles of Acceptance Trials for Pathway 2.0 8/12/97
Pathway Release 2
9 Acceptance of the Operational Trial PDA 1.0 22/05/97
PDA/REL/A
CC/001
10 Acceptance Scoping Study Horizon 1.0 12/06/98
Ul Live Trial Scope and Management PDA A 11/08/97
12 Role and Tasks of the Horizon Horizon 2.0 2/12/98
Acceptance Test Managers BA/ACC/007
13 Horizon Programme Quality Review Horizon 0.4 31/03/98
Procedures
14 Horizon Acceptance Management Horizon 2.0 I 2/12/98
Organisation BA/ACC/004
15 Witnessing of Acceptance Trials (and Horizon 0.2 25/09/98
Acceptance Reviews) examining the BA/ACC/002
approach
att00002.dat Page 3 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
16 Horizon Process for Witnessing Horizon 1.0 9/12/98
Acceptance Trials BA/ACCO005
17 Horizon Process for Handling Acceptance I Horizon 30/11/98
Incidents BA/ACC/003 I 2.0
18 Horizon Acceptance Management Tools I Horizon 0.1 4/01/99
BA/ACC/008
19 Process for contractual Acceptance of the I Horizon 0.1 to be
Operational Trial [based on Reference 9] _I BA/ACC/006 issued
20 Horizon Critical Path Plan Horizon 9.0 28/10/98
21 Nile Release 2.0 End to End Interface and I PDA 1.0 12/02/98
Model Office Testing Approach TSTMOSTS
TRO2.DOC
22 Terms of Reference for tripartite audit of I Horizon TOR-I 3.0 25/11/98
Technical Testing 30
23 Review of Technical Testing (Draft) Tripartite 0.2 16/11/98
Team
ROO4VDAT.
DOC
24 Horizon Process for Conducting BA/ACC/009 I 0.1 4/01/99
Acceptance Reviews
25 I Release Authorisation of the NR2 (Child I Horizon 2.2 I 9/11/98
Benefit) Release RANR2
(ChB)
att00002.dat Page 4 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
Table of Contents
Contents Page
0. DOCUMENT CONTROL... 3
DS see
RELEASE AUTHORISATION OF THE NR2 (CHILD BENEFIT) RELEASE....sssessseesees
RANR2 (CHB)...
2.2,
O/IT/ 98.4,
1, 1. MANAGEMENT SUMMARY...
2.1. DEFINITION OF ACCEPTANCE, 8
2.2. OBJECTIVES OF ACCEPTANCE....coconcseen . 10
2.3. ASSUMPTIONS... son ssn son - onsen LO
2.4. ACCEPTANCE TESTING TIMETABLE o.ssnnnnnnnnnnnnnnn seonnsnmnnnnnnee V2
2.5. OVERVIEW OF ACCEPTANCE PROCESS. 13
3, ACCEPTANCE TEST METHOD.....
3.1. ACCEPTANCE PROCESS. . son 15
3.2. ACCEPTANCE IN STEPS. 15
3.3. TECHNICAL TEST.. 15
3.4. MODEL OFFICE TEST. sons ose . snnenenenneee . : vonenee 16
3.5. END T0 END INTERFACE TEST 16
3.6, LIVE TRIAL, soe soseee 16
3.7. DECISION MAKING - ACCEPTANCE OF THE RELEASE. 17
3.8. CONTRACT ACCEPTANCE, 17
3.9. AUDIT OF TESTING AND WITNESSING STRATEGY...... ee vn _ 7
4, ACCEPTANCE TEST ACTIVITIES...
4.1. ACCEPTANCE TRIAL,
4.2. ACCEPTANCE REVIEW.
4.3. ACCEPTANCE SPECIFICATIONS.
4.4. THE ACCEPTANCE IMPLICATIONS OF REPLAN...
4.5. THE ACCEPTANCE PROCESS.
4.6, HORIZON TASKS DURING ACCEPTANCE.
5. PHASE 1 - PREPARATION FOR ACCEPTANCE. seseseeneeneenseeseeneen: . 25
5.1, ACCEPTANCE SPECIFICATIONS REVIEW AND AGREEMENT. 25
5.2. ACCEPTANCE SPECIFICATIONS IN DISPUTE. 28
5.3. REVIEW FOR COMPLETENESS... sssnsnnnneotnsnsesee sustntnnnnntnnninnnnneniatnnnnnsee sssseenenennennennnnennennenesnee 29
5.4. ACCEPTANCE SPECIFICATION SIGN-OFF 00s 29
5.5. ACCEPTANCE PLANNING. . “ “ “ 29
5.6, DEVELOPMENT OF ACCEPTANCE, MANAGEMENT TooLs. . ae esses ae 30
5.7, RESOURCING FOR ACCEPTANCE. ..scso:cisinssinninnnnsisnnininninnnninnininaninininanenanannninnnnnanannnnnenenananesste evo BL
5.8. DSS Work PACKAGE DRAFTING... somemnenninnnnennenennnennnnnse 3
5.9. MODEL OFFICE TEST AND END TO END INTERFACE ~ “AC EPTANC ‘CRITERIA 31
6. PHASE 2 - ACCEPTANCE TESTING...
6.1. INTRODUCTION. 0.00000 .
6.2. ACCEPTANCE AND RELEASE AUTHORISATION.
7. ACCEPTANCE TEST TASKS.
7.1, ACCEPTANCE TRIALS. 33
att00002.dat Page 5 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
7.2, ACCEPTANCE REVIEWS ..ssosssssssnenninsnnnneinnnee
7.3. ACCEPTANCE INCIDENT MANAGEMENT...
8, PHASE 3 - CONTRACTUAL ACCEPTANCE
9, HORIZON ACCEPTANCE ORGANISATION...
9.1. STRUCTURES INVOLVED IN THE ACCEPTANCE PROCESS. 36
9.2. HORIZON ACCEPTANCE MANAGER. 36
9.3. HORIZON ACCEPTANCE TEAM... - - - ssn sone 36
9.4. ACCEPTANCE TEST MANAGERS 37
9.5. IMPACT ANALYST(S)....cscesccssceseessessessessessessesseessessessessesseeseeseesessnseieesssussuesseeseesesuesissssuessessessessesneeseee 37
10. HORIZON ACCEPTANCE - RELATIONS WITH SPONSORS.......sesssesseeseesees
10.1, RELATIONSHIPS BETWEEN HoRIZON AND DSS.
10.2. RELATIONSHIPS BETWEEN HORIZON AND POCL.
11. HORIZON ACCEPTANCE MANAGEMENT FORUM...
12. HORIZON RELATIONSHIPS WITH ICL PATHWAY...
12.1, ICL PArHWay TEST MANAGER:.
ANNEX A - HORIZON ACCEPTANCE GLOSSARY...
ANNEX B - DETAILED TIMETABLE.
ANNEX C - NOTES ON TESTING.........
1.
att00002.dat Page 6 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
1. MANAGEMENT SUMMARY
1d Acceptance is the act of accepting a ICL Pathway Service or Deliverable on
confirming that it meets contracted obligations in accordance with the relevant
Acceptance Criteria. This stage plan indicates the assumptions and broad principles
used for the management of the Horizon Acceptance of Products and Services during
the ICL Pathway Operational Trial. The plan has been developed on PRINCE lines
from the Horizon perspective. The plan specifically deals with Acceptance of the
Operational Trial Service and Deliverables, and the processes to confirm that it meets
contracted obligations.
1.2 Following the present Replan proposals, which are yet to be agreed and contractually
finalised, it is assumed by all parties that the Operational Trial system will be delivered
in three Steps (referred to in the Replan Level 0 as “stages”). It is further assumed
that each Step will be subject to a separate Acceptance activity, culminating in
eventual Acceptance of the Operational Trial Services and Deliverables as a whole,
when all contracted obligations have been met.
1.3 In the Acceptance process both Authorities, that is DSS and POCL, the Sponsors of
Horizon, have sovereign acceptance rights. Horizon is tasked with the management
of the Acceptance process on behalf of both Authorities.
14 The Horizon plans for the management of Acceptance are outlined at the higher level.
The paper goes on to outline the various management organisations and covers the
relationships between Horizon and the Authorities and with ICL Pathway during
Acceptance.
1.5 The purpose of the document is to provide high level guidance for Horizon’s
management of Acceptance activities generally. The paper shows and explains the
various processes involved
1.6 The paper indicates the subordinate Horizon documents which provide detailed
guidance on Acceptance practice, organisation and activities. D.N. This paper, and
the other Horizon Acceptance documents are living documents which will be updated
to reflect any changes in policy and processes which arise.
att00002.dat Page 7 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
2.0.1
2.0.2
2.1.
2.1.1
PWY REF: BP/PLA/016
INTRODUCTION
The purpose of the document is to provide high level guidance for Horizon’s
management of Acceptance activities generally. This paper describes the overall
principles and practice of Horizon’s management of the Acceptance of the ICL
Pathway Operational Trial Services and Deliverables on behalf of DSS and POCL
The paper then sets out the PRINCE style stages of the Horizon Acceptance Plan and
offers the principles for the relationships between Horizon and its Sponsors and
between Horizon and ICL Pathway during Acceptance. The paper includes a number
of process diagrams and accompanying text which explain the practical steps involved
in all phases of Acceptance.
The definitions given in the Glossary at Annex A are those which were offered and
agreed by the Sponsors in Reference 9. They are intended to emphasise aspects
which relate to the Acceptance this process and so do not always exactly equate to
those given in the Related Agreements. The source of those definitions not directly
from the Agreements is indicated at Annex A.
Definition of Acceptance
Acceptance is the act of accepting a ICL Pathway Service or Deliverable on
confirming that it meets contracted obligations in accordance with Acceptance
Criteria [Reference 1]. This paper specifically deals with Acceptance of the
Operational Trial System, and the process to confirm that it meets contracted
obligations. Consistent with the present Replan proposals [References 2 and 3],
which are yet to be agreed and contractually finalised, it is the current working
assumption by all parties that the Operational Trial services and products will be
delivered in three Steps (referred to in the Replan Level 0 as “stages”). It should be
noted that the Horizon three Step approach is not accepted by ICL Pathway. Each
Step will be subject to a separate Acceptance activity, culminating in eventual
Acceptance of the Operational Trial Services and Deliverables as a whole, when all
contracted obligations have been met.
The contents of Pathway New Release 2 are fully defined at Reference 4.
D.N. The assumptions cited here and the breakdown of Acceptance into three Steps
may be altered as a result of the replanning and contract renegotiations which are
ongoing.
The working assumption is that Acceptance Step (1) for ICL Pathway New Release 2
(NR2) Single Benefit will take place once ICL Pathway has demonstrated:
. that ICL Pathway NR2 working against a single benefit Child Benefit (ChB)
meets contracted obligations;
. the Proof of Concept of mobile counter configuration (for outlets at which
Horizon equipment cannot be installed in a permanent position or are installed
in vehicles);
. satisfactory progress in developing NR2+;
. that a satisfactory stage has been reached in preparing for Acceptance Step 2.
att00002.dat Page 8 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
2.1.4 Acceptance Step (2) for ICL Pathway NR2 Multiple Benefits will take place once the
ICL Pathway has demonstrated:
. that ICL Pathway NR2 meets contracted obligations when working against
multiple benefits [ChB, Income Support (IS), Job Seeker’s Allowance (JSA)
and Retirement Pension (RP)]; and
. the proof of concept for NR2+.
2.1.5 The contents of Pathway New Release 2 are fully defined at Reference 5.
2.1.6 Acceptance Step (3) for ICL Pathway NR2+ will take place once ICL Pathway has
demonstrated that NR2+ operates against multiple benefits in accordance with
contracted obligations.
2.1.7 All Acceptance activities are directed to Contractual Acceptance. Contractual
Acceptance of the Operational Trial Services and Deliverables has significant
implications to the Programme as payment guarantees come into force, rights of
termination are effected and service levels and remedies apply. Note however, the
working assumption that on Acceptance of Step 1, the Authorities’ termination rights
for failed Acceptance of the Operational Trial System for Step 2 will not be
surrendered.
2.1.8 In the development of the project, the Authorities’ Agreements have been divided
into 24 Acceptance Specification documents. Although the DSS has argued that
Acceptance of the Operational Trial System should be considered against a single
logical Acceptance Specification for each of the three Acceptance Steps, this is not
the contractual position. Acceptance Tests against an Acceptance Specification may
begin as soon as that Specification has been signed-off as agreed by the Authorities
and the Contractor. The Acceptance Specifications for Step I have been developed by
ICL Pathway against the Service Architecture Design Document (SADD) version 4
[Reference 6] to cover NR2. The Acceptance Specifications for the Operational Trial
include Operational Trial Criteria derived as required by Authorities Agreement
Schedule C3.
2.1.9 The drafting of the current Acceptance Specifications had assumed that NR2 would
be operating against multiple benefits. Step 1 NR2 Technical Tests and End to End
(E2E) Tests will be conducted against multiple benefits. Model Office Test and Live
Trial will not test against multiple benefits. Since the Step 1 Model Office Test and
Live Trial acceptance conditions will differ from those for Step 2, modified and
additional Acceptance Specifications will be required for the Step 2 testing of NR2.
2.1.10 The process for the development, review and sign-off of those amended and
additional Acceptance Specifications will follow those used earlier and managed by
Horizon to minimise delays by all parties.
2.1.11 These and other definitions used during Acceptance are listed at Annex A. The
definitions are intended to reflect the usage in the Horizon Acceptance processes. and
complement the definitions in use in the Related Documents.
att00002.dat Page 9 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
2.2. Objectives of Acceptance
2.2.1 The strategic Horizon Acceptance Objectives, on behalf of the DSS and POCL, are:
. To fully recommend acceptance of the ICL Pathway Deliverables and Services
only if key contracted obligations are met.
. To provide the means by which the Contracting Authorities may agree with ICL
Pathway that they are capable of delivering the Services contracted for in the
Related Agreements.
. To ensure that the Authorities termination rights and other remedies in the
event of Failed Acceptance are not diluted.
. In the event of Failed Acceptance, to ensure that the provisions allowing for a
cure period may be properly followed. (For example, in the context of the
Operational Trial, failure to cure could lead to termination of the contract).
. In the case of less than complete Acceptance to ensure that the Authorities are
able to accept such part of the Operational Trial System as the Authorities may
decide and to provide the mechanisms where pro-rated charge or otherwise
arrangements may be agreed.
. To maintain synchronisation with the Release Management processes which run
in parallel with Acceptance.
2.3. Assumptions
D.N. The assumptions cited here and the breakdown of Acceptance into three Steps
may be altered as a result of the replanning and contract renegotiations which are
ongoing.
2.3.1 In producing this plan the assumptions outlined in the following paragraphs have been
made in addition to those made about the three Step Acceptance process detailed
above.
2.3.2 Acceptance must follow the procedures defined in the Related Agreements as
amended by formal change control. This plan is based on the assumption that the
evolution of Acceptance of the Operational Trial into a 3 Step process is agreed by all
parties and the Related Agreements are amended under change control. Failing that
agreement, this plan will be amended where applicable..
2.3.3. The Related Agreements are contractually binding and any proposed changes, before
becoming valid, must be formally agreed by the relevant parties via formal change
control.
2.3.4 Contractual Acceptance of the Operational Trial Services and Deliverables constitute
confirmation by the Authorities to ICL Pathway that contracted obligations for the
Operational Trial system have been met. Acceptance at the end of Steps 1 and 2 of
the Operational Trial constitutes confirmation to ICL Pathway that they have met
their contracted obligations for that Step of the Operational Trial, and that they have
made satisfactory progress in the outstanding Step(s).
att00002.dat Page 10 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
2.3.5
2.3.6
2.3.7
2.3.8
2.3.9
2.3.10
2.3.11
2.3.12
2.3.13
PWY REF: BP/PLA/016
ICL Pathway are responsible for producing a Testing and Integration Strategy for the
Operational Trial, the Operational Trial Acceptance Specifications and the
Acceptance Criteria contained within them.
The Authorities are responsible for providing Acceptance Test Conditions (although
in the event, those for the Technical Test have been have been produced by ICL
Pathway). (It should be noted that Acceptance Test Conditions have been developed
for Model Office Test and End to End Test and are being developed for Live Trial. It
is expected that this process will be repeated for the changing scope of Steps 2 and 3.)
ICL Pathway are responsible for the production of Business Threads, High Level Test
Plans and Test Scripts to meet the Test Conditions.
The Acceptance Specifications, High Level Test Plans and Test Scripts are based on
the ICL Pathway design for Release NR2. Thus:
. The Acceptance Specifications are Release specific (and may be Acceptance
Step specific) and may not represent the contracted obligations;
. The Acceptance Specifications, and all subordinate documents will have to be
checked before use in later Steps of Acceptance. Where required, this will
entail a repeat of the Acceptance Specification review and agreement process
managed by Horizon.
Horizon are responsible for managing the agreement of the Acceptance
Specifications.
ICL Pathway are responsible for writing all Test Scripts and for designing and
organising the majority of tests which occur during the Operational Trial. Outline
details of those tests which involve Acceptance [extracted from Reference 7] and
notes on Joint Testing [Reference 8] are at Annex B. ICL Pathway are also
responsible for supporting the Model Office Test
Either Authority may at any time request that additional Acceptance Tests be
performed. The conditions under which additional tests might be sought would be
where the tests conducted by ICL Pathway failed to provide adequate evidence that a
Test Condition had been met.
Horizon is responsible for ensuring suitable attendance for the Acceptance Trials and
Reviews.
During the course of the Acceptance Trials, effective management procedures will be
in place to ensure that Acceptance Incidents are managed to resolution in known
versions of software, and for effective retesting and configuration management. The
procedures will produce an audit trail through which it is possible to track from
incident to resolution, identifying each test and software version involved. Up to date
management information, and the audit trail will be made available to the Horizon
Test Manager, Horizon Acceptance Manager, DSS Acceptance Manager, DSS Audit,
the Horizon Acceptance Incident Manager and/or relevant Horizon Acceptance Test
Manager.
att00002.dat Page 11 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
2.4, Acceptance Testing Timetable
2.4.1 The outline high level timetable based on the ICL Pathway and Horizon Programme
Replan documents [Reference 3] is shown at Figure 1 below. D.N. This timetable
will be refined to reflect the outcome of the current contractual negotiations and the
agreed Horizon Critical Path Plan [Reference 20].
1999 2000
/ 4 / 4 A 4 s Oy N 10 / 4 F 4 M I A pM J i J 1 A 1 Ss 4 ° 4 N 1 D ! 4 F 4 M 4
Prepdenrb Nrambdidee I oT oT PT yo T PT PT rT ft tT TT TT 7 od
‘Acceptance! (ChBICAP3‘0} i H : :
Specicications 5 : : i
5j---> Releasé
« L\ Bitoraton
NR2 End to End po
I (CRBICAPS.0) :
<> i Start National Rollout : i
, FS pct migration < — —»
Technical Testing
step 1. + -
it it Pré-Proofing NR2/2
Single Benefit 1é-Proofing
BIT Regression Testing
Step2 il
_ BoE 1 Paymentstops dtc. CAPS foo: # Card follout
Multiple Benefit NR2 E 1 Pré-proving Muli Benefit Model Offige <> >
ae ¢BRUICAPSOUSNPSCS Sy Senetts
i : i into trial
5 A. CAPS3.5 Release
BoE i ‘Into Model Office with = i
I : Fomine enquities i :
: Poi jull-Benefit Live Trial
: Technical Testing
Brepare!NR2+
i <=>,
Acceptance : po PPNR2+ Modd! I ,
Specicidations i 1 *OffeaTest_—_, nroduetotinto
oS : : <—_ fe
> A : eet
Do - FpccePTaNCE Pb A.
: Po ACCEPTANCE
from 1 Nov £996 ‘Operational Trial
Figure 1 - Overall Timetable for Operational Test
2.4.2 It should be noted that the definitions of the contents of NR2 and NR2+ (References
4 and 5] include a number of specific exclusions which will not form part of
Acceptance Tests.
2.4.3. More detail of the timetable is held in the Horizon Master Plan [Reference 3]. An
extract giving detailed timetable for Acceptance activities is at Annex B (D.N. To be
added after replanning is completed.)
2.5. Overview of Acceptance Process.
2.5.1 The structures involved in the Acceptance process and the outline responsibilities of
the various groups involved are summarised at Figure 2 below. The processes
involved are developed further in this paper.
att00002.dat Page 12 of 48 12/01/99
RESTRICTED COMMERCIAL
POL00029137
POL00029137
PWY REF: BP/PLA/016
JAB members advise
‘Sponsor Boards and
solicit their input to
‘make the final
Acceptance decision.
CAPS & Cards Horizon
Business Delivery Director Programme Director
CAPS & Cards
Programme Board
Chief Executive's Committe:
Programme
DSS Acceptance Manager
through CAPS & Cards
Business Delivery Birector
maiters
JAB
convened to
consider and
to recommend
Joint Acceptance Board (JAB) - bipartite
Horizon Programme Director (Chait)
CAPS & Cards Business Delivery Director
Horizon Acceptance Manager
ICL Pathway (in attendance)
[Contract Signatory (as required)]
Horizon Acceptance Manager
‘manages the Acceptance of the Horizon Acceptance Manager ‘major
overall contract supported by the Acceptance
Horizon Acceptance decisions
Process Manager
ict Pathway Acceptance Manager
Acceptance
Management
Forum
Acceptance Test Managers
‘manage review and sign-off
of individual Acceptan
Specifications.
Florizon
Co-ordination
Horizon Acceptance Team Leader
(Note 1) (ote 1)
Horizon Acceptance Manager
PSS Acceptance Manager Incident Review Panel (IRP)
Horizon Acceptance Incident Manager (Chait)
Horizon Acceptance Manager
Acceptance
Review
Review Teams
Panel
convened
weekly
DSS Acceptance Manager
POCL Business Assurance
Panel members from: Product Assurance, Service
Management, Implementation, Security,
(and as required, representatives from J
Commercial)
ICL Pathway Acceptance Manager (for Part 1).
Note 1
The AME could be two-part with Horizon/DSS co-ordination
as Part 1 and the tripartite business at Part 2,
Figure 2 - Overview of the Structures and Responsibilities for the Acceptance Process
att00002.dat Page I of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
3. ACCEPTANCE TEST METHODS
3.1. Acceptance Process
3.1.1 Acceptance is a process for assessing the functionality and performance of the ICL
Pathway Services and the Service Infrastructure prior to National Rollout. Each Step
of Acceptance has three parts: Technical Test, Model Office Test (MOT), End to End
Interface Test (E2E) and Live Trial, save that Step 3 ends with Introduction into Live
Service, rather than Live Trial.
3.1.2 Acceptance Specifications produced by ICL Pathway in fulfilment of a contractual
obligation, but to be agreed by the Authorities, refer also refer to Technical/Security
Testing, Business Integration Test (BIT) and System Test. These are ICL Pathway
owned tests which have their own High Level Test Plans, and form part of
Acceptance Trials. Horizon will seck visibility of the HLTPs and LLTSs for
Technical Tests on behalf of the Sponsors and to confirm that these test documents
are under firm change control.
3.1.3. The ICL Pathway approach to business and technical testing is outlined within the
Principles of Acceptance Trials for ICL Pathway Release NR2 [Reference 8].
3.2. Acceptance in Steps
3.2.1 Step 1 Acceptance Tests will be conducted on the MOR 3 version of Release NR2
and include a Technical Test which is common to Steps I and 2. Step 1 also includes
stand-alone MOT, E2E Test and Live Trial. MOT and Technical Testing can overlap
but should finish in sequence.
3.2.2. Step 2 Acceptance Tests will be conducted on a later version of Release NR2 and
include the Technical Test which is common to Steps 1 and 2. Step 2 also includes its
own stand-alone MOT and E2E Test and Live Trial of Multi-Benefits. MOT and
Technical Testing can overlap but should finish in sequence.
3.2.3. Step 3 Acceptance Tests will be conducted on Release NR2+ and include Technical
Test and stand-alone MOT, E2E Tests and Introduction into Live Usage. MOT and
Technical Testing can overlap but should finish in sequence
3.3. Technical Test
3.3.1 Technical Test is the first phase of each Step to ensure that the ICL Pathway solution
for that stage meets the performance, volume, resilience and operability requirements
of the BA/POCL programme. The objective of the Technical Test in all three Steps is
to confirm that the underlying elements of the Services and the Service Infrastructure
function correctly and to confirm their readiness for the MOT.
3.3.2. The E2E Interface Trial was originally ex-contractual but, having been included in the
Replan proposals, it is assumed that it will be included in the Contract under change
control.
att00002.dat Page I of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
3.3.3
3.4,
3.4.1
3.4.2
3.6.
3.6.1
PWY REF: BP/PLA/016
If that is the case, it is assumed that the E2E element of Integration Test will be
conducted in association with Model Office Rehearsal (MOR). The assumed overall
objectives for the performance of E2E centre on the bringing together of the IT
functionality across all the participating systems, with the operational and business
procedures, to form an overall business system. E2E and MOR activities are planned
and scripted together, run on the same environment, and involve the same personnel.
E2E forms a natural extension of Direct Interface Test (DIT). It is run in parallel
with, but separate from, MOR.
Model Office Test
Model Office Test (MOT) is the second phase of each Step of Acceptance. MOT is a
further test of the Pathway system to provide evidence as to whether the contracted
obligations to both parties have been met. It uses simulated data and as near lifelike
arrangements in a production like environment using actual. Tests are conducted to
ensure the system and manual business processes interact through the use of training
and procedures to meet the overall business needs of the BA/POCL programme.
MOT is designed and run by Horizon in co-operation with ICL Pathway.
The objective of the MOT is to test the Services and the Service Infrastructure in an
environment which simulates (as far as possible without using live operational data)
the live operation of the end to end Services and the Service Infrastructure, and to
confirm their readiness for the Live Trial. [Reference 21]
End to End Interface Test
The End to End Interface Test has been designed to ensure that the integrity of data
remains intact when all systems, including, Pathway, are linked together throughout
the End to End environment. The tests are also required to test some of the lengthier
business procedures across the business domains. User procedures will be used for
tests which cannot be covered during Model Office Rehearsal and Model Office Test.
[Reference 21]
The End to End Interface Test has been designed jointly by representatives of both
systems and business processes working jointly with ICL Pathway and other external
system representatives.
Live Trial
Live Trial, outlined at Reference 11, is the third stage of the first and second Steps in
Acceptance and runs over a core period of 11 weeks. The main elements are:
. the implementation and operation of the Live Trial services, that is the full core
services to be implemented which support the assessment of other areas
including POCL implementation, Service Management, Transaction Timings
and Invoice Production;
. reporting success to Release Management in order for them to authorise the
next release processes;
att00002.dat Page 2 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
. identification of improvement opportunities for the ICL Pathway and Sponsor
services and support for Sponsor’s benefit realisation work.
3.6.2 The objective of the Live Trial shall be to test the Services and the Service
Infrastructure with operational data, Users and Customers, to confirm that the
Services and the Service Infrastructure function in the live environment, and to
confirm their readiness for national Roll-Out.
3.6.3. The number of Outlets involved in the Live Trial is defined in the controlled document
entitled “Delivery and Implementation during the Operational Trial Period”. At
present the Live Trial is to involve the 204 existing Outlets plus 100 new Outlets.
3.7. Decision Making - Acceptance of the Release
3.7.1. Each Step will have a final phase which will be the decision making processes of
Horizon and the business Sponsors regarding the Acceptance of the Release. This is
dealt with in the document dealing with the third Step of Acceptance - Contract
Acceptance [Reference 19]. D.N. This paper will be developed and issued following
the outcome of the replan.
3.8. Contract Acceptance
3.8.1 At the end of Step 3, the formal Contract Acceptance processes will include:
. the negotiation with ICL Pathway of any service elements requiring partial
acceptance;
. formal Contract Acceptance of the Operational Trial.
3.9, Audit of Testing and Witnessing Strategy
3.9.1 At the request of its Sponsors, Horizon initiated a tripartite audit of the ICL Pathway
testing process under Terms of Reference approved by the Acceptance Management
Forum [Reference 22].
3.9.2. The audit team have recommended in their report [Reference 23] that witnessing in
relation to contract Acceptance should be defined with its objective and scope as
follows.
. Objective: As part of the contract Acceptance process, to utilise an opportunity
available to the Authorities to obtain independent and objective information
from the performance and results of Pathway testing that the underlying
elements of the Service and Service Infrastructure function correctly.
. Scope: Operating within the Technical Testing stage of Acceptance Trials.
3.9.3 The audit team recommended that the Witnessing strategy to be adopted is selective
witnessing of priority Acceptance areas in services and/or functionality that the
Authorities believe to be of crucial importance, or where there are known or
perceived problems in ICL Pathway’s ability to deliver the functionality. The resulting
approach to witnessing being a combination of “Process and Content Witnessing”
att00002.dat Page 3 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
involving a mixture of forward looking and retrospective examination.
3.9.4 In the event, the Acceptance Management Forum have agreed to concentrate on the
“Content” witnessing of remaining Acceptance Trials, of any Acceptance elements of
continuing ICL Pathway testing (such as the BIT Regression Tests) and of course
during the Model Office and End to End Interface Tests and the Live Trial.
3.9.5 The detailed processes for witnessing of Acceptance Trials and the conduct of
Acceptance Reviews are detailed at References 16 and 24 respectively.
att00002.dat Page 4 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
4.1.
4.
4.2.
421
4.3.
43.1
44,
441
PWY REF: BP/PLA/016
ACCEPTANCE TEST ACTIVITIES
Acceptance Trial
Acceptance Trial is one of two methods of testing the acceptability of a Deliverable or
Service (the other being Acceptance Review). Descriptions of one or more
Acceptance Trials will be included as part of each Acceptance Specification. These
Acceptance Trials can consist of one or more inspections, demonstrations, running
real or simulated operational workloads, monitoring the service or presentations.
Acceptance Review
Acceptance Reviews cover any Acceptance scrutiny of the ICL Pathway products,
deliverables and services that are not an actual Acceptance Trial. They include
demonstrations witnessed by Horizon and Sponsor staff, site visits by Horizon and
Sponsor staff, presentations to Horizon and Sponsors, etc., in addition to the actual
review of Deliverable documents and other items. See Reference 24.
Acceptance Specifications
The format for an Acceptance Specification is given in the Related Agreements and
includes:
. a definition of its purpose and scope;
. procedures for raising and resolving Acceptance Incidents. This will probably be
a reference to a standard process and details of any deviations;
. the Acceptance Period e.g. Live Trial;
. the Deliverables and Services covered;
. the acceptance method(s) to be used and thus the events to be organised e.g.
Acceptance Reviews (Document Quality Reviews) or Acceptance Trials;
. the Acceptance Criteria for each Deliverable or Service (the requirements), and
the associated Acceptance Test Conditions;
. guidelines for analysing Acceptance Incidents;
. test data for Acceptance Trials;
. Authorities responsibilities and resources required for each Acceptance
Trial/Acceptance Review.
The Acceptance Implications of Replan
As outlined earlier, the Replan has effectively divided Acceptance into three Steps.
The implications of this are:
att00002.dat Page 5 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
. that Acceptance will be a much longer activity and will require additional
resources;
. the Acceptance Specifications, HLTPs and other supporting documentation
may have to be revised, reviewed and the revisions signed-off for the second
and third Steps, prolonging the Acceptance preparation activities, in parallel
with execution on earlier Steps;
. there may be further adjustments if and when the contracted Acceptance
obligations are contractually altered under current change control activity and
commercial negotiations.
44.2 The creation of two additional Steps requires additional work in revising and
reviewing documents, obtaining witnesses and other resources and completing
reports. The Acceptance Management Forum [defined in Reference 14] is now in
existence and will direct the co-ordination of these new activities.
45. The Acceptance Process
4.5.1 The essential process of each Step of Acceptance is considered to occur in three
phases illustrated at Figure 3 below. There are other associated processes not shown
in the interests of simplicity.
Acceptance Preparation Acceptance Testing Contractual Acceptance
i The
Produce and Reviews" ¢ MOT,
Revise Entry and Exit oy
(Pwy) teria for MO
E H Produce report] [Joint
i for Joint Acceptance
Dangr RESET Pertom acceptance I I Accent
Testand reduce] I aor acceptance AS Yes [Plan Schedule Acceptance Board “a
‘cca I I onsen wa tHenzon& [Lo] Reviews ang ete] (Horizon bf riorizon
peste ‘al Pw acceptance Thal Accetance [I rector and
[Pwy) {Honzon, OSS, rh) team in Director
= Pool 8 PW) suppor ot I I Gaps io)
Note 3 i AMFL
Preduce an
Acceptance
eet intent Report
[Horaon, DSs 7
Series) Img
vise Specitcato
and ower ofan
for later Stage?
es
Resolve
Problem
Revise Pw]
Pwr) \
3 Can tbe
resolved?
Produce an
Impact Analysis. [J
fal)
IROUSe ACCOpTaNE
Speciation
OR ot Acceptance
and HiTPs 1 4! Speciteation ete Yeo
ete ‘all
Pw
q
Note 3
Some of these
activites may
be open to auait
by Horizon, DSS
contl or hough a
“Secession over or POC.
the nature of the
requirement
Figure 3 - The Horizon Acceptance Process
4.5.2 These phases have been used as the basis of the Stage Plan for the Acceptance of ICL
Pathway which is shown at Figure 4 below.
att00002.dat Page 6 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
Horizon Acceptance - Horizon
Stage Plan Acceptance Plan
Horlzon Horizon .
fp] Acceptance [I Acceptance Kennan
Preparation Testing
Acceptance Identification of
Specification °
I__ ‘ests to be
Review & Witnessed
‘Agreement
Review
Acceptance
Acceptance Resource Reports
I» Specification wi tanning
‘sign-off jinessing Trials
and Reviews
Contractual
Negotiations with
ICL Pathway
Acceptance Test Witnessing
Planning Acceptance Trials:
Agree
Development of Acceptance
Acceptance auinessing
* Management Renews
Tools
Sign-Off
Resolution Acceptance Acceptance
‘Ownership P
— Incident
Acceptance Management
Criteria 9
Develop Consolidated
Acceptance - a
Structures epenng I Procedures
Develop Strategy Raising i
for Auditing Acceptance i I Miferagement
Acceptance * Incidents Hi
Process from MOT
I Develop Strategy I rT a
for Evaluating Auditing FORSaRS
Acceptance Test Acceptance j
Reports and I ——_____ Process
Acceptance Develop and Horizon
Incidents Agree with DSS Cconfiguraton
pF, MT and Management
I witnessing
Drafting Horizon Strategy
——— + DSS Work I /—_—" Horizon Risk
Packages Management
Figure 4 - Horizon Acceptance Stage Plan
4.5.3 Each Step of Horizon Acceptance of the Operational Trial of ICL Pathway will be
conducted in the phases described below, each of which has associated Horizon
Acceptance Products as shown at Figure 5 below. The overall view of the main
processes involved is shown at Figure 6 - the bold entries indicating those Horizon
processes described in this paper.
att00002.dat Page 7 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
Horizon Acceptance -
Product Breakdown Horizon
Acceptance Plan
v v
Horizon Horizon Horizon
Acceptance Acceptance Contractual
Preparation Testing Acceptance
Horizon Contractual
Acceptance Plan Logs of ls » Release
and working _____.. Witnessed Trials Acceptance
papers and Reviews Report
f
Signed-off
Contract Changes
Acceptance Closed + under Change
Specifications +! Acceptance Control
(including Criteria) Incident Reports ©
Agreed Pathway I¢ Operational Trial
+I High Level Test Unresolved + Acceptance
Plans etc. [——*I_ Acceptance Report
Incident Reports
Horizon and DSS.
Acceptance Quality Reports
Management + Acceptance
Structure, TORs Reviews
etc
Acceptance Acceptance
+I Management * Incident Reports
Database
Acceptance Audit Reports
Issues Database
I Horizon DSS. 5 > Progress Reports
Work Packages
Agreed Appoach Consolidated
and Procedures Acceptance
to Audit of Report
Acceptance
‘Agreed
Model Office I. Acceptance
Criteria and Test
Conditions for
Model Office Test
Test Scripts
Figure 5 - Horizon Acceptance Product Breakdown
att00002.dat Page 8 of 48 12/01/99
RESTRICTED COMMERCIAL
POL00029137
POL00029137
Pathway Test Design
and Planning
so
oe
a“
Pathway Draft
Acceptance
Specification etc
Pathway & Horizon Test
Schedule Planning
Horizon Test Design
and Planning
(Model Office & E2E)
HLTPs LTS, —_—_——
Pathway Acceptance
OF Database
Horizon and Sponsors
Acceptance
‘Specification Review,
Revision and approval
WITNESS.VSD
__——Test Conditions ete
“signed-off AS
PWY REF: BP/PLA/016
Pathway Test
Management
PinlCL and PIAT
Processes
Horizon Acceptance
Database
WITNESS.VSD
Horizon Trial/Review
Witnessing
Management
ACCREPT.VSD
>I Witness Planning
Resourcing
Horizon & Sponsor
l-
Horizon & Sponsor
Management) = fwe—__ Acceptance
Reporting/Escalation 'ssues
and
Figure 6 - Overall View of Horizon Acceptance Processes
Al Forms
Al Analysis Forms ———»@_ Acceptance Incident
KEY
---- Pathway
Horizon
Acceptance
AIPROC.VSD
I
Horizon & Sponsor
Handling
v
To follow
Contractual
Acceptance
Horizon Issues
att00002.dat
Page I of 48
12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
4.6. Horizon Tasks During Acceptance
4.6.1. As explained above, each Step in the Acceptance process involves three phases of
work; Preparation, the actual Testing and Contractual Acceptance activity. The next
three sections detail the processes in those phase in some detail.
5. PHASE 1 - PREPARATION FOR ACCEPTANCE
5.1. Acceptance Specifications Review and Agreement
5.1.1 During this stage the Acceptance Specification for each Acceptance Test is produced
by ICL Pathway and then delivered to Horizon [formerly the PDA] for review and
agreement by the Authorities. The processes involved are shown at Figure 7 below.
att00002.dat Page I of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
veemen — Horizon and ATM
Le ‘Acceptance I Review of Acceptance
—— ‘Specification ‘Specification
ra
Acceptance Criteria I_—~ ~
re 7 Consolidated
‘Sign-off by DSS, review comments
POCL and submit
Horizon - forreview
Pathway respond ath
Pass into to comments. prance:
‘Change Control comme Specification
7
So Updated
7 Acceptance
for use Speciation
» _
4 v
ra
- Horizon and ATM
7 Revise for later coordinate Quality
Tests Steps?” ¥°* ——) Review of Acceptance
~ Specification
Sign-offby DSS,
POCL and Horizon
Consolidated
review comments
submit
Pass in to Change for review
Contre!
Pathway revise
Pathway respond
Undated Pathway
Acceptance response to
Speeification review comments
Y
Figure 7 - The Acceptance Specification Review Process
att00002.dat Page 2 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
5.1.2 The initial stages are the development by ICL Pathway of the Acceptance
Specification, together with other associated documents from the Agreements and the
Acceptance Criteria.. In some cases new Acceptance Criteria have been developed to
represent new requirements of the changes in the solution which have been mutually
agreed between Horizon and ICL Pathway. The Acceptance Specifications are
delivered to the Horizon Team for recording in the Horizon Library and for the
formal review process.
5.1.3. Horizon conducts a formal quality review using a panel of reviewers from relevant
parts of the Sponsor organisation or organisations; their activities and the collation of
comments being the responsibility of the nominated Horizon Acceptance Test
Manager. DSS reviews and comment collation are co-ordinated by the DSS
Acceptance Manager. POCL reviews and collation are the responsibility of POCL
Business Assurance. The review, consolidation of comments and subsequent
negotiations with ICL Pathway are managed by a nominated “document guardian”,
more correctly the Acceptance Test Manager.
5.1.4 If there is consensus that the Specification is fit for purpose in the Acceptance process
the Acceptance Test Manager and their ICL Pathway opposite number will notify
those concerned that the Specification is ready for “sign-off? and submit the
document into Change Control. The agreed document will then be published for use.
5.1.5 If the DSS, POCL and Horizon reviewers believe that the Specification is incomplete
or inadequate they will complete quality comment sheets spelling out the deficiencies,
the reasons for the comments and, if possible, suggested changes. The reviewer's
responses are channelled through the Acceptance Test Manager and passed back to
ICL Pathway in a consolidated format
5.1.6 ICL Pathway then analyse the consolidated comments. ICL Pathway can then accept
the comments and make appropriate changes or respond to the reviewers comments,
explaining reasons for rejecting the comments. At the same time, ICL Pathway will
produce and issue a revised version of the Acceptance Specification and an updated
consolidated review sheet indicating their responses. The ATM will then distribute
the new version and review sheet.
5.1.7. The review process is usually conducted in two cycles before ICL Pathway issue the
final version of the Acceptance Specification for approval by the Authorities and entry
into the Change Control process.
5.1.8 I The process for agreeing these Acceptance Specifications is based on the standard
Horizon Quality Review Process [Reference 13].
5.1.9 Products in this stage include:
. Acceptance Specifications which are agreed by both ICL Pathway and the
Sponsor or Sponsors and ready for submission into Change Control for “sign-
off’ and use in Acceptance Testing.
. agreed supporting Acceptance documents under ICL Pathway configuration
management (High Level Test Plans, Low Level Test Scripts, SADD etc.)
. Agreed Acceptance Criteria
att00002.dat Page 3 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
5.1.10 This stage has the following external dependencies:
. timely preparation and completion of Acceptance Specifications and supporting
documents by ICL Pathway;
. competent, timely and comprehensive staged reviews of Acceptance
Specifications by DSS and POCL reviewers;
. timely preparation of acceptable supporting documentation (HLTPs, BTs etc.)
under configuration control for the Acceptance Tests by ICL Pathway;
5.2. Acceptance Specifications in Dispute
5.2.1. Some Acceptance Specifications have not been completely agreed and signed-off by
the agreed start date for Acceptance Tests in mid October 1998. Faced with the delay
in obtaining agreement to the Specifications, the contracting Authorities have
indicated that they would be willing to agree to ICL Pathway commencing
Acceptance Testing without approval of the relevant Acceptance Specification or
Specifications. As part of that agreement, ICL Pathway would consider including any
additional Acceptance Tests identified after the start date within a subsequent stage of
regression testing or fitting outstanding Acceptance Tests as part of Acceptance
Reviews.
5.2.2 At a meeting between the Horizon Acceptance Manager and ICL Pathway
Acceptance Manager on 2 September 98, it was agreed that, wherever possible, the
Acceptance Tests should be conducted against complete and signed-off Acceptance
Specifications. Failing that, the Acceptance Tests and Reviews should be conducted
against caveated Acceptance Specifications and the test results include the
requirement for retesting against any missing, deficient or disputed Acceptance
Criteria.
5.2.3. The procedure for dealing with those issues and Acceptance Criteria omitted from the
caveated Acceptance Specifications will be:
. ICL Pathway to continue to accept comments from the ongoing review of
Acceptance Specifications. But this is not a “licence” for Horizon or its
Sponsors to reopen those Acceptance Specifications which have been “agreed”
or “published for approval”.
. ICL Pathway will add additional criteria to the Acceptance Specifications and
map to HLTPs where appropriate tests exist.
. If tests do no exist, ICL Pathway will suggest alternative means for Sponsors to
gain the necessary evidence - possibly through Acceptance Reviews.
. In the event that no appropriate existing Acceptance Trial or Review is
available then ICL Pathway will create a test which can be executed during the
Business Integration Test (BIT) regression phase, following completion of the
MOT but before commencement of the Live Trial. It should be noted that the
extensions to BIT may not be accomplished before the start of Live Trial;
. It may also be possible to run deferred tests in the later Steps.
att00002.dat Page 4 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
5.3. Review for Completeness
5.3.1. The Acceptance process consists of detailed analysis which, when amalgamated, will
allow the sponsors to form an overall view of proposed Acceptance Tests. The
Acceptance Database [Reference 18] will be the main tool to check the completeness
of the proposed tests, using full set of agreed Acceptance Specifications in order to
identify gaps.
5.3.2. Work is in progress, under Reference 10 and follow-on gap analysis, to identify gaps
through the gap analysis, but only when all of the Acceptance Specifications have
reached maturity may suspected omissions be confirmed. Horizon are maintaining an
ongoing dialogue with ICL Pathway throughout the preparation stage in order to
minimise the number of gaps which may be identified late. However, some issues will
only be confirmed towards the end of the Acceptance preparation stage, and others
may be revealed as the Acceptance process is under way.
5.4. Acceptance Specification Sign-off
5.4.1 Once each of the Acceptance Specifications are approved by all three parties, the
comments will be consolidated at Horizon and then reported up the DSS and POCL
Acceptance hierarchy, with a recommendation to submit to Change Control for “sign-
off” for each of the 23 specifications.
5.4.2 Products in this stage include:
. Signed off Acceptance Specifications.
5.5. Acceptance Planning
5.5.1 This stage involves:
. the establishment of the Horizon Acceptance Team, including the Acceptance
Test Managers;
. the preparation of this document, the overall Acceptance Stage Plan;
. drafting of Stage Plans and other documents for the remaining Acceptance
processes, including Acceptance Incident Management;
. development of the Horizon Acceptance organisation including Terms of
Reference for the Horizon Acceptance Boards and Job Descriptions for all of
the main Acceptance roles;
. development of a Horizon and BA body dealing with Acceptance issues of
mutual interest;
. the development of the Horizon organisation to include all Horizon related
Acceptance activities;
. development of the Horizon Acceptance management tools, including
. Horizon Acceptance Database;
att00002.dat Page 5 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
. Horizon Acceptance Forms;
. Horizon Acceptance Files for subsequent audit purposes.
5.5.2 Products of this stage are:
. Complete and Approved Horizon Stage Plan for the Management of
Acceptance [this document];
. Horizon Acceptance Management Organisation [Reference 14].
5.6. Development of Acceptance Management Tools
5.6.1 The Horizon Acceptance Team will develop the following Acceptance Management
Tools:
. The Horizon Acceptance Management Plan (this document);
. A set of subordinate plans and procedures to cover all three phases of
Acceptance;
. The Horizon Acceptance Database (using both Access 2 and Access 95
applications) outlined in Reference 18;
. An Acceptance Incident element of the Horizon Acceptance Database also
outlined at Reference 18;
. The Horizon Acceptance Database, described at Reference 18.
5.6.2 Products of this stage are:
. Terms of Reference for all Horizon Acceptance Management bodies [Reference
14];
. Horizon Acceptance Database [Reference 18];
. Horizon Issues Database[Reference 18].
5.7. Resourcing for Acceptance
5.7.1 This stage requires the Horizon Acceptance Team, together with the BA Acceptance
Manager and contacts in the POCL Business and Technical Assurance groups to
identify the resources required to carry out the witnessing of Acceptance Trials and
Reviews, whose principles are discussed at Reference 15. Since the resources
available are known to be limited, there may a need to prioritise the Acceptance Trials
in order to target those offering greatest benefit to the Acceptance effort.
5.7.2 The priorities for witnessing and the co-ordinated resourcing of the witnessing tasks
will be managed by the Horizon Acceptance Management Team on behalf of the
Acceptance Management Forum.
5.7.3. This stage has the following external dependencies:
att00002.dat Page 6 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
. Adequate supplies of staff with appropriate technical and/or business skills from
Horizon and its Sponsors;
. Timely provision of the ICL Pathway detailed timetable for conducting
Acceptance Tests.
5.7.4 Products of this stage are:
. Confirmed resources for the conduct of all Horizon Acceptance Trials and
Reviews
. Working timetable for the witnessing of Trials and Reviews.
5.8. DSS Work Package Drafting
5.8.1 This stage requires Work Packages to be drafted in the DSS standard for DSS
approval for all Horizon Acceptance work on behalf of the DSS.
5.8.2 Products of this stage are a set of completed and accepted DSS Work packages.
5.9. Model Office Test and End to End Interface - Acceptance Criteria
5.9.1 Responsibility for Acceptance Test for the MOT and E2E Interface lies not with ICL
Pathway, but with Horizon. This includes the need for clear definition of Test
Conditions and Acceptance Criteria, including entry and exit criteria for the MOT.
5.9.2. MOT and E2E Interface do not involve the use of a contractually controlled
Acceptance Specification. Acceptance Incident reports may still be raised during
MOT and E2E, but they will not include a reference to an Acceptance Specification.
6. PHASE 2 - ACCEPTANCE TESTING
6.1. Introduction
6.1.1 This main execution phase will consist of a number of Acceptance Tests, documented
in the Acceptance Specifications, which can consist of Acceptance Trials or
Acceptance Reviews or in some cases a combination of the two. In general the
Acceptance of Services will be by Acceptance Trial and the Acceptance of
Deliverables will be by Acceptance Review. Details of the Horizon process for the
handling of Acceptance Trials is given at Reference 16. The conduct of Acceptance
Reviews is detailed at Reference 24.
6.1.2. The underlying principle is that Acceptance Testing (performing Acceptance Trials
and Acceptance Reviews) happens only once unless an Acceptance Incident occurs
that requires a change and retest.
6.1.3. All Acceptance Trials or Acceptance Reviews undertaken in collaboration with ICL
Pathway will be listed in one of the agreed Acceptance Specifications or will be part
of a subsequently agreed change to an Acceptance Specification.
6.1.4 The outcome of an Acceptance Trial or Review is either a pass or a fail (a deficiency);
att00002.dat Page 7 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
the definitions of which take into account the severity of Acceptance Incidents which
have occurred during the Trial or Review. If no Acceptance Incidents are raised to
record deficiencies, then the Service or Deliverable will be deemed to have been
accepted. The outcomes of these events need to be carefully recorded on the
Acceptance Database by the Horizon Acceptance Support Team.
6.1.5 In the case of the MOT, where there is no ICL Pathway Acceptance Specification,
Acceptance Incident Reports may be raised in the event of any incident occurring
during the test which identifies a failure to meet any obligation under the Authorities’
Agreements.
6.1.6 More details on the processes for handling Acceptance Incidents are given at
Reference 17.
6.2. Acceptance and Release Authorisation
6.2.1. Whereas the Acceptance decision focuses on assessing whether the ICL Pathway
delivered services meet the contracted obligations, Release Authorisation takes a
broader view to assess the business viability of the delivered services. The key
question for Release Authorisation is whether to proceed with the implementation of
releases into the BA/POCL service environment (i.e. including appropriate POCL or
DSS system releases). The decision to authorise the release of services into the
defined environment requires a consideration to be given to the readiness of all parties
to act as both supplier and customer in supporting and implementing these services in
the prescribed environment.
6.2.2 It should be understood that although DSS retains joint responsibility for making the
Release Authorisation decision with POCL, the Horizon Programme PID identifies
the Horizon programme as having the responsibility for managing the Release
Authorisation process. The procedures for Release Authorisation are detailed at
Reference 25.
6.2.3 In this context it is envisaged that the Acceptance and Release Authorisation processes
will be run in parallel and that although the basis for decision-making within each
stream may utilise much common evidence, the two processes need to remain separate,
coming together at specific points to influence decision making.
7. ACCEPTANCE TEST TASKS
TA, Acceptance Trials
7.1.1 As defined in the Authority Schedules, an Acceptance Trial consists of one or more of
the following:
. inspection of any system used to provide a Service;
. demonstration of any system used to provide a Service;
. running a real or simulated operational workload on any system used to provide
a Service;
att00002.dat Page 8 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
. monitoring provision of a Service over a trial period to verify performance to
the Service Levels specified for the Service.
7.1.2 The scope and conduct of any Acceptance Trial is documented in the Acceptance
Specification. It is contractually agreed that an Acceptance Trial may not commence
until the Acceptance Specification has been approved by the Authority. However, in
the case of caveated Acceptance Specifications, testing will be carried out against the
agreed parts of the Specification.
7.1.3 Acceptance Trials can take many formats. The majority of them will be walk-
throughs of the service and deliverables under examination type conditions, from a
test not necessarily designed to satisfy a single Acceptance Specification and usually
conducted on ICL Pathway premises. Those Trials may be witnessed as desired by
Horizon and its Sponsors. Any other type of activity during Trials can be linked to an
Acceptance Criteria if it in some way provides assurance to Horizon and its Sponsors.
7.1.4 The Horizon and Sponsor observers of Acceptance Trials are referred to as
"witnesses".
7.2. Acceptance Reviews
7.2.1 An Acceptance Review consist of one or more of the following:
. inspection of a Deliverable (e.g. a document) by one or more competent
reviewers nominated by the Authority;
. inspection by one or more competent reviewers nominated by the Authority of
documentary evidence that a Service has been provided;
. a meeting between the Authority and ICL Pathway at which one or more of the
following occur:
. Deliverables or documentary evidence are inspected;
. a presentation is given by ICL Pathway;
. the results of prior inspections, visits or presentations are reviewed and
the outcome agreed;
. a demonstration of a Service or Deliverable.
7.2.2. The scope and conduct of any Acceptance Review proposed shall be documented in
an Acceptance Specification, and the Acceptance Review shall not commence until
the Acceptance Specification has been approved by the Authority.
7.2.3. The process for the conduct of Acceptance Reviews is detailed at Reference 24.
Acceptance Reviews of Deliverables will use the standard Horizon Quality Review
process where this is relevant e.g. for a document review.
7.2.4 The Horizon, POCL and DSS personnel conducting Acceptance Reviews are also
referred to as "witnesses".
att00002.dat Page 9 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
7.3. Acceptance Incident Management
7.3.1. An Acceptance Incident is any unusual or undesirable occurrence, or any request for
advice and guidance, which has been raised in writing, using the procedures set out in
Reference 17, by Horizon, DSS or POCL witnesses to ICL Pathway in respect of an
Acceptance Trial or Review. Acceptance Incidents may be raised by Acceptance Test
Managers, by Horizon and Sponsor witnesses of Acceptance Trials or Reviews, or by
ICL Pathway staff associated with Acceptance, by Horizon Service Management staff
or others, such as Model Office testers.
7.3.2. Details of the Acceptance Incident Management process and procedures are given at
Horizon Process for Handling Acceptance Incidents [Reference 17].
8. PHASE 3 - CONTRACTUAL ACCEPTANCE
8.0.1 Contractually the Joint Acceptance Board (which reports to the Boards of DSS and
POCL) must confirm Acceptance of the Operational Trial to ICL Pathway within five
working days of the end of the Acceptance Period (the end of Live Trial). [Details of
this process and the management bodies involved, are still under review by the
Acceptance Management Forum. When clarified, this section will be enlarged to
reflect the agreed processes.
8.0.2. The process to be used in this third phase of the plan has still to be designed based on
the principles set out in Reference 12. But essentially the phase is to enable the Joint
Acceptance Board to make an informed decision as whether to formally accept the
ICL Pathway Service and Products. The Horizon Acceptance Manager will produce
a management summary for the Joint Acceptance Board which is extracted from the
acceptance reports produced by each Acceptance Test Manager with the assistance of
the Horizon Acceptance Team. Details of the impact assessment for any outstanding
Acceptance Incidents will also be included in the management summary.
8.0.3. The Joint Acceptance Board will consider the Acceptance recommendations made to
them and taking into account any other matters known by them, will make
appropriate recommendations to the DSS CAPS& Cards Programme Board and the
POCL Chief Executives’ Committee. Those recommendations are likely to be either
rejection, acceptance or partial acceptance of the ICL Pathway Services and Products.
The decision of the two Sponsor Boards will then be passed to the Release
Authorisation Board.
8.0.4 In this context, partial acceptance means where ICL Pathway have failed to meet the
criteria for contractual Acceptance but that a negotiated route to the achievement of
full service and contractual remedies can be agreed between the Authorities and ICL
Pathway.
att00002.dat Page 10 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
HORIZON ACCEPTANCE ORGANISATION
9.1. Structures Involved in the Acceptance Process
9.1.1 The figure below is an overview of the structures and responsibilities for the
Acceptance process. These are elaborated in Reference 14.
take te fia
CAPS & Cards
Programme Board
CAPS & Cards Horizon
Thsiness Delivery Director Progr same DIRTOP
JAB members advise POCL
Sposa Bourisand €hiet Executive's Commits
DSS Acceptance Manager
‘through CAPS & Cans
Business Delivery Director
Hotizon Progranme
JAB
convened 19
consider and
torecommend
major
Acceptance
decisions,
Horiron Acceptance Manager
manages the Acceptance of the forizon Acceptance Manager
Joint Acceptance Board (JAB) - bipartite
Horizon Programme Director (Chait)
CAPS & Cards Business Delivery Dtector
Horizon Acceptance
Process Manager
ICE Pathway (in atendance)
{Contract Signatory (a required)]
Kcceptane Acceptance Management Forum (AME)
Horizon Accepiance Manager (Chair)
DSS Acceptance Manager
Acceptance Test Managers
jan-oft ‘Management
Forum
Frorizon
Co-oeination
Note 1)
Horizon Acceptance Manager
SS Acceptance Manages
TCL Pathway Acceptance Manager
Horizon Acceptance Team Leader
ote 1)
Incidem Review Panel (RP)
Horizon Acceptance Incident Manager (Chait)
Horizon Acceptance Manager
DSS Acceptance Manag
POCE Busines Assurance
Panel merabers fren: Product Assurance, Service
Management, Implementation, Securit, Testing
(and as required, representatives from JCT and
Conmersal)
ICL Pathway Acceptance Manager for Part 1)
‘weekly
Note 1
‘The AMF could be two-part with Horizon DSS co-ordination
ss Part 1 and the tripartite business at Prt 2.
Figure 5 - Overview of the Structure and Responsibilities for the Acceptance Organisation
9.1.2 An outline of the Horizon Acceptance management organisation is given below.
9.2. Horizon Acceptance Manager
9.2.1. The Horizon Product Assurance Manager has the additional role of the Horizon
Acceptance Manager. His task is to oversee the planning, co-ordination and
facilitation of the Horizon Acceptance Process and ensure that all the business
requirements in the Related Agreements are fully covered in a complete and
comprehensive way. In addition he is charged with regular liaison with his fellow
Acceptance Managers at the DSS and ICL Pathway. He is a member of several of the
Horizon Acceptance boards. His duties are detailed at Horizon Acceptance
Management Organisation [Reference 14].
9.3. Horizon Acceptance Team
9.3.1 Horizon have established a small Acceptance Team within the Product Assurance
Group to support the management of the Acceptance process. Under the Horizon
Acceptance Team Leader, the Horizon Acceptance Team includes the Acceptance
Incident Manager, the Acceptance Process Manager and Acceptance Process support
staff. More details of the roles and responsibilities of these staff are provided at
att00002.dat Page 11 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
Horizon Acceptance Management Organisation [Reference 14].
9.4, Acceptance Test Managers
9.4.1. Horizon has appointed Acceptance Test Managers for each Acceptance Test. They
are usually the former “document champion” for the respective Acceptance
Specification. More information is at Roles and Tasks of the Horizon Acceptance
Test Managers [Reference 12].
9.5. Impact Analyst(s)
9.5.1 The Impact Analyst(s) is/are responsible for carrying out an Impact Analysis on a
specific Acceptance Incident where ICL Pathway claims that the Acceptance Incident
cannot be resolved.
9.5.2 There may be several Impact Analysts, and they may work on one or more
Acceptance Incidents. An Acceptance Incident may require Impact Analysts from
one or both of the Sponsor(s), from Horizon, or from ICL Pathway.
10. HORIZON ACCEPTANCE - RELATIONS WITH SPONSORS
10.1. Relationships Between Horizon and DSS
10.1.1 It has been agreed that Horizon will hold Acceptance co-ordination meetings with the
DSS, especially the BA COBAPS staff, on a regular basis, possibly as a precursor to
the tripartite meetings of the Acceptance Management Forum.
10.1.2 In addition the Acceptance Test Managers have established but informal networks
used in the review of Acceptance Specifications and other documents. These will
become firmer during the witnessing of Acceptance Trials.
10.2. Relationships between Horizon and POCL
10.2.1. Horizon co-ordination with POCL occurs both formally and informally at several
levels from that of the Programme Director, Horizon with the POCL Chief
Executive’s Committee to similar links at lower levels.
1. HORIZON ACCEPTANCE MANAGEMENT FORUMS
11.1.1 Horizon Acceptance Management requires the establishment of three forums whose
roles and responsibilities are shown in outline at Figure 5 above and are detailed at
Horizon Acceptance Management Organisation [Reference 14]. That document
covers the following:
. Acceptance Incident Review Panel (IRP), responsible for day-day management
and resolution of Acceptance Incidents. Meeting formally on a weekly basis to
confirm decision and priority reports and to otherwise review progress and to
develop reports to the next level, the Acceptance Management Forum. The
IRP will begin its meetings as soon as Acceptance Testing begins.
att00002.dat Page 12 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
12.
PWY REF: BP/PLA/016
The Acceptance Management Forum (AMF) which is tasked with the co-
ordinated management between POCL, ICL Pathway and the DSS of the
Acceptance process within the agreed programme milestones and to resolve any
issues escalated to it. It will formally meet at least formally on a fortnightly
basis. Any Acceptance Incidents which are not resolved at the Forum are to be
escalated to the Joint Acceptance Board, part of the Horizon Delivery Group.
The AMF began its work at the first tripartite meeting on 1 October 1998.
The Joint Acceptance Board (JAB), which is tasked with formal confirmation of
the three steps of Acceptance during the Operational Trial. The Board is the
final arbiter in the case of Acceptance Incidents and other programme issues
which are referred to it.
att00002.dat
Page 13 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
HORIZON RELATIONSHIPS WITH ICL PATHWAY
12.1. ICL Pathway Test Manager:
12.1.1 The ICL Pathway Test Manager:
. may initiate an Acceptance Incident report;
. acknowledges the receipt of an Acceptance Incident form;
. commissions the analysis of the Acceptance Incident, and completes the
Acceptance Incident Analysis form;
. manages the resolution of the cause of the Incident;
. advises the Acceptance Incident Manager that the amended Deliverable or
Service is ready for re-testing.
12.1.2 The ICL Pathway Acceptance Manager may also be needed to help provide undertake
Impact Analysis where necessary, by providing Impact Analysts as described above.
12.1.3 The ICL Pathway Incident Manager (if one is established - to be verified with ICL
Pathway) may take on some or all of the responsibilities of the ICL Pathway
Acceptance Manager.
att00002.dat Page 14 of 48 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
ANNEX A - HORIZON ACCEPTANCE GLOSSARY
The definitions given below are those offered and agreed in Reference 9. They are
intended to provide definitions to be used in this process and so do not always exactly
equate to those given in the Related Agreements. The source of the definition is
included when not directly from the Agreements.
Term Definition
Acceptance Acceptance is the act of accepting a ICL Pathway Service or
Deliverable on confirming that it meets contracted obligations in
accordance with Acceptance Criteria [Reference 1].
Acceptance The criteria for Acceptance of Deliverables and Services as
Criteria specified and formally agreed either in the Related Agreements
or in the Acceptance Specifications.
Acceptance A comprehensive management tool for use during all three
Database phases of Acceptance. The main purpose of the Database is to
support Acceptance management and reporting. Other uses
include: provision of the means of checking ICL Pathway
Acceptance Specifications, Acceptance Criteria, and their
supporting documents (such as Business Threads and High
Level Test Plans) against the current version of the Contract and
its Schedules; support for the management of the Review and
Sign-off of Acceptance Specifications issued by ICL Pathway;
provision of a basis for gap analysis; support for the work of
Acceptance Test Managers during Acceptance Trials and
Reviews in identifying the impact of test failures or Acceptance
Incidents on the objectives of Horizon and its Sponsors;
provision of the means of recording the Acceptance Incident
management process. [Reference 18]
Acceptance Any unusual or undesirable occurrence, or request for advice
Incident and guidance, which is reported in writing by Horizon to ICL
Pathway during an Acceptance Test. An Acceptance Incident
can be either where an Acceptance Criterion is not met; where
there is a substantive fault; or in other circumstances.
[References 1,9 and 17]
Acceptance Issue I A problem occurring during the three phases of Acceptance or
the aftermath of an event in any aspect of the Acceptance
process, which critically threatens to damage, hinder or stop the
Acceptance process or to affect the time or other resources
allocated to Acceptance. In general, Acceptance Issues are
referred to and managed by the Acceptance Management
Forum. [References 14 and 19]
Acceptance An Access 2 database containing all details of Acceptance Issues
Issues Database referred to and managed by Horizon Acceptance Manager or by
the Acceptance Management Forum. [Reference 18]
Horizon A central post within Horizon that is responsible for co-
Acceptance ordinating the Acceptance activities and liaison with the ICL
Manager Pathway Acceptance Manager. [Reference 14]
Horizon A manager answering to the Horizon Acceptance Manager who
Acceptance is responsible for the management of the reporting of
att00002.dat Page A-1 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
Incident Manager
Acceptance Incidents and the process of resolution. [Reference
14]
Acceptance
Group
A separate Acceptance Group will be formed for each
Acceptance Test and may also, in some instances, differ for
individual Acceptance Reviews and/or Acceptance Trials. The
activities of the Acceptance Group will be managed by the
Acceptance Test Manager. Members of the Acceptance Group
will come from Horizon or the sponsor organisations and will be
chosen for their experience and specialist business knowledge of
the subject being accepted. [Reference 17]
Acceptance Test
Manager (ATM)
The former “document champion” for on Acceptance
Specification; the manager of the Acceptance Group of
witnesses and others and the Horizon “owner” of a specific
Acceptance Test. This person will have an in-depth knowledge
of the particular business area that they are to manage.
[Reference 12]
Acceptance The period of time for the completion of an Acceptance Test.
Period
Acceptance One of two methods of testing the acceptability of a Deliverable
Review or Service (the other being Acceptance Trial). Descriptions of
one or more Acceptance Reviews will be included as part of
each Acceptance Specification.
Acceptance One of 23 comprehensive documents produced by ICL Pathway
Specification specifying a Deliverable or Service (as defined in the
Agreements).The document, also describes the relevant
Acceptance Test. [This document]
Acceptance Step
One of the three phases of the Acceptance process comprising
Step (1) for ICL Pathway Nile Release 2 Single Benefit; Step (2)
for ICL Pathway NR2 Multiple Benefits; and Step (3) for ICL
Pathway Nile Release 2+. [This document]
Acceptance
Support Team
A support team managed by the Horizon Acceptance Team
Leader. [Reference 14]
Acceptance Test
A procedure for determining whether a Deliverable or Service
meets the relevant Acceptance Criteria. It may include one or
more Acceptance Trials or/and Acceptance Reviews. ICL
Pathway have chosen to map some Acceptance Tests onto
Services e.g. BES, OBCS.
Acceptance Test
Conditions
The conditions used to determine whether or not each
Acceptance Criteria has been met.
Acceptance Trial
One of two methods of testing the acceptability of a Deliverable
or Service (the other being Acceptance Review). Descriptions
of one or more Acceptance Trials will be included as part of
each Acceptance Specification. These Acceptance Trials can
consist of one or more inspections, demonstrations, running real
or simulated operational workloads, monitoring the service or
presentations.
Business Thread
Documents developed by ICL Pathway. Business Threads which
att00002.dat
Page A-2 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
(BT) describe very high level business driven test scenarios which will
form the backbone of the System Test stage.
contractual The third phase of the Acceptance process, during which the
Acceptance Joint Acceptance Board is contractually required to confirm
Acceptance of the Operational Trial to ICL Pathway within five
working days of the end of the Acceptance Period. [Reference
9]
Deliverable
An item which ICL Pathway is required to provide under the
terms of the Related Agreements.
Direct Interface
Tests to ensure that each of the system interfaces conform to the
Test (DIT) Application and Technical Interface Specification and support
the business, technical and security requirements of both parties
on either end of the interface. [Reference 8]
End to End Testing to ensure that the full end to end business processes,
Interface Test within the BA/POCL scope, are satisfied by each of the systems
(E2E) involved, whether existing or new. [Reference 8]
Failed The outcome of an Acceptance Test where one or more high or
Acceptance medium severity deficiencies remain unresolved at the end of the
Acceptance Period.
High Level Test I Documents developed by ICL Pathway against the requirements
Plan (HLTP) of the Acceptance Test Specifications. The HLTP cover each of
the stages of testing, detailing the specific scope, coverage,
objectives and success criteria that apply. In this high level test
planning stage, the test objectives have been formally
documented and agreed by both Horizon and by ICL Pathway,
and combined to form a joint set of test objectives which serve
to drive all subsequent testing activity.
Low Level Test
A detailed test script developed by ICL Pathway from the HLTP
Script (LLTS) and the BT which details the actions to be carried out during
each Test. [This document.]
Live Trial The third and last phase of the Operational Trial
Model Office Test I The second phase of each Step of Acceptance; and a further test
(MOT) of the Pathway system to provide evidence as to whether the
contracted obligations to both parties have been met. It uses
simulated data and as near lifelike arrangements in a production
like environment using actual. Tests are conducted to ensure the
system and manual business processes interact through the use
of training and procedures to meet the overall business needs of
the BA/POCL programme. [This paper]
Operational Trial
A process for assessing the functionality and performance of the
Services and the Service Infrastructure prior to National
Rollout. Each Step of the Operational Trial has three phases:
Technical Test, Model Office Test and Live Trial. Contractual
Acceptance is linked to the completion and outcomes of this
trial.
Product
Acceptance Test
ICL Pathway tests to confirm their acceptance of products from
both ICL Pathway development teams and from ICL Pathway
sub-contractors. [Reference 8]
Release A documented collection of software and/or data provided by
the Contractor to deliver a Service.
att00002.dat Page A-3 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
The ICL Pathway Release Policy further refines the above
definition into: “Software Releases”; “Reference Data Releases”
(of which there are five classes); and “Maintenance Releases”.
[Release Management paper]
Related The Authorities Agreement, the DSS Agreement and the POCL
Agreements Agreement.
Service This document provides the design of the service architecture in
Architecture accordance with Clause 401.1 of the Authorities’ Agreement,
Design Document I and Requirements 928 and 951.
(SADD)
The purpose of the SADD is to describe the design of the
service architecture (R951). It must as a minimum specify:
The major components used in providing the services;
The functionality within major components;
The interfaces between components and with Authorities’ and
other parties’ systems;
The service levels required across interfaces. [Reference 6]
Services All the Services to be performed by, and all other obligations of,
ICL Pathway under the Related Agreements.
Security Test
Tests conducted to ensure that the ICL Pathway solution meets
the security requirements of the BA/POCL programme. Some
of these tests will be achieved through tests conducted in other
phases. [Reference 8]
System Test
The testing to ensure that the ICL Pathway solution functionally
conforms to the requirements of the BA/POCL programme. In
addition, these tests will ensure that the Human Computer
Interface for the combined products meet the HCI requirements.
[Reference 8]
Technical Test
The first phase of the Operational Trial to ensure that the ICL
Pathway solution meets the performance, volume, resilience and
operability requirements of the BA/POCL programme.
Witnessing
The observation by staff from Horizon, the DSS and POCL of
Acceptance Trials carried out by ICL Pathway to provide an
opinion on the reliability and sufficiency of information
contained in the Pathway Acceptance Test Closure Reports and
to assess the ICL Pathway management of those tests. Witnesses
should have either technical or business expertise relevant to the
tests or have experience in the technical audit of complex system
tests. Witnessing may be process and/or criteria related. Process
witnessing entails examination of aspects of management control
over the testing process as applied to Acceptance Trials. Criteria
witnessing involves verification of the output of particular
service or product operating successfully within the Acceptance
Test environment. Both aspects of witnessing may be applied
whilst Acceptance Testing is underway by means of observation.
Similarly both aspects may be applied retrospectively by
examining the supporting documentation e.g. Test Results, ICL
Pathway Closure Reports and PinICLs and Acceptance Incident
reports raised during the Trial.
att00002.dat
Page A-4 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
ANNEX B - DETAILED TIMETABLE
1. The following GANTT chart is the detailed time table for Acceptance activities in
Step I
To be added
att00002.dat Page B-1 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
ANNEX C - NOTES ON TESTING
System Test Streams
1. System Testing is an ICL Pathway managed set of tests which covers the following
Pathway systems:
Automated Payment Service (APS);
Electronic Point of Sale Service (EPOSS);
Benefit Payment Service (BPS) which covers Automated Payment Service,
Card Management System Payment Authorisation Service;
Order Book Control Service (OBCS);
Transaction Processing System (TPS);
Management Information System (MIS);
Training;
Audit.
Integration Test Streams
2. Integration Test covers a wide range of complementary test streams. The test streams
where Acceptance Testing will be carried out are:
DIT - Direct Interface Test;
T&S - Technical & Security Test (including Security Test; Performance Test;
Integrity Test and Systems Management Test);
BIT - Business Integration Test;
E2E - End to End Interface Test;
MOR - Model Office Rehearsal.
3. The phased approach to test execution is the “3 pass model’ formalised for Release
NR2:
Ist Pass - stabilise;
Main Pass - iterative defect removal;
Final Pass - audit trail, which then pass used for Acceptance Trials.
att00002.dat
Page C-1 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
E2E and MOR
4. MOR is an integral part of Integration Test which prepares the way for Model Office
Test. MOR involves Joint Business Test Team, and Horizon testing (including POCL
counter staff) play a major role in it, but it must be emphasised that MOR remains a
Pathway test activity with specific Pathway test objectives.
5. The aims of E2E and MOR test streams are to bring the IT system together with the
business procedures, to trial the training activities and to deploy the target live
support channels, running all the contributing systems in unison and exercising all
major end to end business flows. In effect to trial the overall business system.
6. The Ist and Main passes of both E2E and MOR are recognised as Pathway owned
test phases, an integral part of the Integration Test stage, operating within the Joint
Testing agreement. MOT remains an independently owned PDA test stage. The
Final Pass of E2E is proposed likewise to be an independently owned PDA test stage.
The start of E2E Final Pass should be aligned with that of MOT to facilitate baseline
control.
Live Trial
7. Live Trial is a stage of testing, and that there are certain aspects of the service which
it will not be possible to demonstrate prior to actual live operation. Hence the
contractual acceptance of the service cannot be concluded until the Live Trial has
been completed.
BIT Maintenance and Regression Testing
8. With the completion of BIT, ICL pathway will switch to a continuous programme of
BIT regression testing to sustain fault rectification and system software modifications.
The regression tests may be used for delayed Acceptance Trials.
Test Programme
9. The sequence of test passes which involve Acceptance is detailed at Annex B.
Joint Testing
10. Joint Testing is a set of activities which input to ICL Pathway and will operate under
the direction of ICL Pathway and will not, therefore, have any assurance aspect such
as Acceptance. It is possible that the joint testing resource may have involvement in
Acceptance Testing on behalf of either ICL Pathway or the Horizon.
1. In Joint Testing, Horizon resources will be deployed to work with ICL Pathway,
within this agreement, on System Test, BIT, DIT, End to End (E2E), Security and
Technical testing streams. The work on E2E is separate from the Horizon activities
on E2E and Model Office Test (documented in the Nile Release 2 End to End
Interface Model Office Testing Approach). The resource will operate within two
Horizon teams, the Joint Business Test Team at Feltham, and the Joint Technical Test
Team at Bracknell.
12. The Joint Business Test Team will provide resource to work with ICL Pathway on
att00002.dat Page C-2 12/01/99
POL00029137
POL00029137
RESTRICTED COMMERCIAL
PWY REF: BP/PLA/016
System Test and BIT. In addition, a DSS and a POCL resource will work with ICL
Pathway to support DIT and E2E tests, in particular to aid ICL Pathway with analysis
on data reconciliation and accounting.
13. The Joint Technical Test team is a Horizon team which works with The Solution
Centre, Bracknell, on ICL Pathway’s Security and Technical testing.
att00002.dat Page C-3 12/01/99