Acceptance of the Operational Trial
FUJ00119567
FUJ00119567
ACCEPTANCE OF THE OPERATIONAL TRIAL
Reference: PDA/REL/ACC/001
Author: Mary Reade
Classification: I PDA Agreement
Version: 1.00
Date : 22/05/97
Status: Authorised
Authority: QR Sign off & PMT approval
Document Summary: Defines the PDA acceptance process for use in the Operational Trial.
Circulation
Contact Area Copy
M.Trotter PDA Library
PDA Team I PDA
Leaders
J Dicks ICL Pathway
J Morley ICL Pathway
P Wootton BA Sponsor
R Wood BA Sponsor
K Davenport I BA Sponsor
M Rego POCL Sponsor
M Burdett POCL: Sponsor
B Kerr POCL Sponsor
Nothing contained herein shall be deemed or construed as affecting existing
contractual obligations or creating new contractual obligations between ICL
Pathway, the DSS and/or POCL.
Vesion 1.00 Authorised Page I of 16
Date 22/05/97
FU.
Acceptance of the Operational Trial
0. Document Control
0.1 Version History
Version Date
1,00 22/05/97 _I First authorised version
Contents Page
1, MANAGEMENT SUMMARY ....0.......:0sceeee sesesnesseeecneesasecssssseeesesstassraserasseatseaeersssseersesaeesseesied
2. GLOSSARY,
3. PURPOSE OF THIS PAPER.....
4, ASSUMPTIONS
5. THE ACCEPTANCE PROCEDURE.......
5.1. Overview...
5.2. Acceptance Preparation.
5.3. Acceptance Testing...
5.4. Contractual Acceptance
6. ROLES AND RESPONSIBILITIES
6.1. The Acceptance Board
6.2. The Acceptance Test Managers
6.3. Acceptance Groups
6.4. The Acceptance Manager.
6.5. The Acceptance Incident Manage:
6.6. The Acceptance Process Support Team..
7. ACCEPTANCE DEFICIENCIES
8. THE NEXT STEPS
ANNEX A - PATHWAY ACCEPTANCE SPECIFICATIONS
ANNEX B - ACCEPTANCE SPECIFICATION - DELIVERABLE DESCRIPTION,
Vesion 1.00 Authorised Page 2 of 16 Date 22/05/97
FUJ00119567
1J00119567
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
1. MANAGEMENT SUMMARY
1. Contractual Acceptance is the process for agreeing with ICL Pathway that the
business needs of the DSS and POCL as stated in the Related Agreements are met.
This paper describes the approach to accepting from ICL Pathway the Operational
Trial (speciifcally release le and the functionality of release 2).
1. The Acceptance of the Operational Trial has very significant implications to the
Programme as payment guarantees come into force, right of termination are effected
and service levels and remedies apply.
1, The process is divided into three stage. They are as follows:
April May June — July Aug Sept Oct Nov
Acceptance Preparation
Acceptance Testing
Contractual Acceptance
1. ICL Pathway are responsible for designing, planning and conducting Acceptance
Tests in accordance with the guidelines in the Related Agreements. The PDA are
responsible for reviewing their proposals and witnessing the Acceptance Tests.
1. Acceptance Criteria have already been produced by ICL Pathway and agreed with the
PDA. These relate directly to the requirements in the Related Agreements.
1. ICL Pathway have divided the Acceptance Criteria into Acceptance Tests, one for
each service, these describe the Acceptance Trials (demonstrations, site visits etc. ) or
Acceptance Reviews (QR document reviews) that will be undertaken.
1. The structure of the PDA needs to include an acceptance team and individuals with
knowledge of the business from the PDA and the sponsor organisations must be
available to witness the acceptance tests.
Vesion 1.00 Authorised Page 3 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
1. GLOSSARY
1. The definitions given below are intended to provide definitions to be used in this
process and so do not always exactly equate to those given in the Related
Agreements.
Acceptance Is the procedure for contractually accepting a Deliverable or Service
provided by ICL Pathway.
Acceptance Criteria I The criteria for Acceptance of Deliverables and Services as specified
and formally agreed either in the Related Agreements or in the
Acceptance Specifications.
Acceptance Incident I Any unusual or undesirable occurrence, or request for advice and
guidance, which has been reported in writing by the PDA to ICL
Pathway during an Acceptance Test
Acceptance A central post within the PDA that is responsible for co-ordinating the /
Manager Acceptance activities and liaising with the ICL Pathway acceptance
manager.
Acceptance Group I A separate Acceptance Group will be formed for each Acceptance Test
and may also in some instances differ for individual Acceptance
Reviews and/or Acceptance Trials. The activities of the Acceptance
Group will be managed by the Acceptance Test Manager. Members of
the Acceptance Group will come from the PDA or the sponsor
organisations and will be chosen for their experience and specialist
business knowledge of the subject being accepted.
i
Acceptance Test I The manager of the Acceptance Group and PDA owner of the I
Manager Acceptance Test. This person will have an in-depth knowledge of the
particular business area that they own.
Acceptance Period I The period of time for an Acceptance Test. I
Acceptance Review I This is one of two methods of testing the acceptability of a Deliverable
or Service (the other being Acceptance Trial). Descriptions of one or
more Acceptance Reviews will be included as part of each Acceptance
Specification.
Acceptance A document, produced by ICL Pathway, that describes an Acceptance
Specification Test.
Acceptance Support I A support team managed by the Acceptance Manager.
Team
Acceptance Test A procedure for determining whether a Deliverable or Service meets
the relevant Acceptance Criteria. It may include one or more
Acceptance Trials or/and Acceptance Reviews. ICL Pathway have
Vesion 1.00 Authorised Page 4 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
chosen to map these onto Services e.g. BES, OBCS.
Acceptance Test I The conditions used to determine whether or not each Acceptance
Conditions Criteria has been met.
Acceptance Trial This is one of two methods of testing the acceptability of a Deliverable
or Service (the other being Acceptance Review). Descriptions of one or
more Acceptance Trials will be included as part of each Acceptance
Specification. These Acceptance Trials can consist of one or more
inspections, demonstrations, running real or simulated operational
workloads, monitoring the service or presentations.
Deliverable An item which ICL Pathway is required to provide under the terms of I
the Related Agreements.
Failed Acceptance IThe outcome of an Acceptance Test where one or more high or
medium severity deficiencies remain unresolved at the end of the
Acceptance Period.
Live Trial The third and last phase of the Operational Trial
Model Office Test I The second phase of the Operational Trial
Operational Trial A process for assessing the functionality and performance of the
Services and the Service Infrastructure prior to National Rollout. The
Operational Trial has three phases: Technical Test, Model Office Test
and Live Trial. Contractual Acceptance is linked to the completion and
outcomes of this trial.
Related Agreement I The Authorities agreement, the DSS agreement and the POCL
agreement.
Services All the Services to be performed by, and all other obligations of, ICL
Pathway under the Related Agreements.
Technical Test The first phase of the Operational Trial.
1. PURPOSE OF THIS PAPER
1. This paper sets out the approach for the contractual Acceptance of the ICL Pathway
Services at the end of the Operational Trial. It is intended to bring together
discussions between the sponsors, the PDA and ICL Pathway following the issue of
the working paper “The Acceptance of Pathway Service RWP1”.
1. In terms of releases this Acceptance relates to ICL Pathway Release le plus the
requirement to accept the ICL Pathway release 2 functionality during the Operational
Trial [see CCN105].
Vesion 1.00 Authorised Page 5 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
0.1.
The Acceptance processes, plans and activities will be clarified in subsequent papers
and the process for accepting future releases will also be produced at a later date.
ASSUMPTIONS
In producing this paper the following assumptions have been made:
that Acceptance must follow the procedures defined in the Related
Agreements;
Acceptance is the process for contractually agreeing with ICL Pathway that the
Services meet the business needs of BA and POCL as stated in the Related
Agreements;
the Related Agreements have already received sponsor approval and any
subsequent changes will seek sponsor approval via the PDA Change Control
Process;
ICL Pathway are responsible for designing the Acceptance Tests and producing
the corresponding Acceptance Specifications.
the PDA are responsible for agreeing the Acceptance Specifications;
ICL Pathway are responsible for designing and organising the Acceptance
Tests;
the PDA may at any time request that additional Acceptance Tests be
performed, but these requests should be used with restraint;
the PDA is responsible for ensuring suitable attendance for the Acceptance
Trials and Reviews;
that after Acceptance Trials have commenced all incidents and upgrades to the
Services must be controlled and access to this information must be available to
the Acceptance Manager and/or relevant Acceptance Test Manager.
THE ACCEPTANCE PROCEDURE
Overview
Acceptance has been divided into three stages. A simple diagram of these stages
follows at figure 1.
Vesion 1.00 Authorised Page 6 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
The Stages of
Acceptance Revise
em
‘Desgn Acceptar * f
es.gn Acceptance) Perform “™ Produce report
Teg ane prcuee I I Rot Acceptance Acceptance “ ~ forPDA Boars Decision
acres Specification Aged >¥em) pu a boa) Reviewsand I» Pass ‘Yom Acceptance Ip} of, Contact!
Speatat wero secre Tra” J ain rear
Procuce an
Acceptance
Revs tncient Report
POs)
Resolve Fcanitbe
Probiem — lae< esoec? >
Praduce an
Impact Anaysis
(POA BPN]
Acceptance Preparation Acceptance Testing Contractual Acceptance
April, May & June 97 June to November 97 November 97
Figure I
1. Each of the stages is described in further detail in the subsequent sections. Further
information can also be found in the Related Agreements.
1. The dates given in figure 1 are planning estimates. The planning details including the
dates for each Acceptance Trial or Review in the acceptance testing stage will be
planned with ICL Pathway during the acceptance preparation stage.
0.1. Acceptance Preparation
1. Basically during this stage the Acceptance Specification for each Acceptance Test is
produced by ICL Pathway and then delivered to the PDA for agreement. A list of the
Acceptance Tests is given at Annex A.
1, The format for an Acceptance Specification is given in the Related Agreements and
includes:
. a definition of its purpose and scope;
. procedures for raising and resolving Acceptance Incidents. This will probably
be a reference to a standard process and details of any deviations;
. the Acceptance Period e.g. Live Trial;
. the Deliverables and Services covered;
Vesion 1.00 Authorised Page 7 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
0.1.
. the acceptance method(s) to be used and thus the events to be organised e.g.
Acceptance Reviews (Document Quality Reviews) or Acceptance Trials ;
. the Acceptance Criteria for each Deliverable or Service (the requirements), and
the associated Acceptance Test Conditions;
. guidelines for analysing Acceptance Incidents;
. test data for Acceptance Trials;
. Authorities responsibilities and resources required for each Acceptance
Trial/Acceptance Review.
The process for agreeing these Acceptance Specifications will be to use the standard
PDA Quality Review Process.
Acceptance Criteria
High level Acceptance Criteria have been agreed and are defined for each service in
the Related Agreements.
More detailed Acceptance Criteria have been produced by ICL Pathway who have
taken the requirements from the Related Agreements and broken them down into
sections that equate to the requirement paragraphs.
These Acceptance Criteria may relate to one or more Acceptance Test Conditions
that are used to determine whether or not the Acceptance Criteria have been met.
Acceptance Methods
The Acceptance Methods for each Acceptance Test will consist of one or more
Acceptance Trials and one or more Acceptance Reviews. In general the Acceptance
of Services will be by Acceptance Trial and the Acceptance of Deliverables will be
by Acceptance Review.
A point to be remembered is that the scope of Acceptance Trial is very wide and can
encompass such activities as site visits, inspections, demonstrations, presentations,
monitoring the service and running real or simulated operational workloads.
Acceptance Testing
The underlying principle is that Acceptance Testing (performing Acceptance Trials
and Acceptance Reviews) happens only once unless an Acceptance Incident occurs
that requires a change and retest.
All Acceptance Trials or Acceptance Reviews undertaken in collaboration with ICL
Pathway will be listed in one of the agreed Acceptance Specifications or will be part
of a subsequently agreed change to an Acceptance Specification.
Vesion 1.00 Authorised Page 8 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
0.1.
0.1.
Acceptance Reviews are normally used for documents and they will use the standard
PDA Quality Review process and can be completed whenever the document is
delivered.
Acceptance Trials can take many formats. The majority of them will be walkthroughs
of the Service under exam type conditions usually on ICL Pathway premises. Any
other type of activity can be linked to an Acceptance Criteria if it in some way
provides assurance to the PDA.
The outcome of an Acceptance Trial or Review is either a pass or a fail (a
deficiency). If no Acceptance Incidents are raised, to record deficiencies, then the
Service or Deliverable has been accepted. The outcomes of these events need to be
carefully recorded on the Acceptance database by the Acceptance Support Team.
The process for managing Acceptance Incidents is described at section 7.
Contractual Acceptance
Contractually the Acceptance Board (see section 6) must confirm Acceptance of the
Operational Trial to ICL Pathway within five working days of the end of the
Acceptance Period (the end of Live Trial).
The process has still to be designed but to enable the Acceptance Board to make an
informed decision the Acceptance Manager will produce a management summary to
summarised the acceptance reports produced by each Acceptance Test Manager.
Details of the impact assessment for any outstanding Acceptance Incidents will also
be included.
The Acceptance Board will either reject, accept or partially accept the ICL Pathway
Services. This decision will then be given to the Release Authorisation Board.
The significance of this Contractual Acceptance is that:
. the payment guarantees come into force;
. the rights to termination at the end of the Operational Trial period are lost;
. service levels and remedies apply.
ROLES AND RESPONSIBILITIES
The Acceptance Board
For each Acceptance Specification, after the Acceptance Trials and Reviews have
been completed and the Acceptance Incidents that can be resolved have been (and
retested), then a Acceptance report will be produced by the Acceptance Test
Manager. These reports will be summarised and issued to the Board who will
formally accept Release 1 and will notify the Release Management Board of this
action.
Vesion 1.00 Authorised Page 9 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
0.1.
0.1.
0.1.
This Acceptance Board will probably be based on the Core Negotiation Team with
additional representation where necessary.
The Acceptance Test Managers
The Acceptance Test Managers for each Acceptance Specification have been
identified and are shown in Annex A.
Their responsibilities will be to:
. manage the quality review and agreement of the ICL Pathway Acceptance
Specification for their Service(s) using the PDA Quality Review process;
. liaise with their opposite numbers in ICL Pathway;
. decide on a suitable composition for the Acceptance Group and negotiate the
required time with the persons concerned;
. to co-ordinate and manage the Acceptance Group and organise their attendance
at Acceptance Tests and/or Reviews;
. liaise with the Acceptance Manager on progress and return information on the
Acceptance Trials and Reviews;
. produce a signed off report for each Acceptance Test within the timescale given
in the detailed procedure;
. raise Acceptance Incidents and submit them to ICL Pathway and the
Acceptance Manager;
. decide if an Acceptance Trial is no longer required because a previous action
e.g. technical test has adequately covered the topic.
Acceptance Groups
Each Acceptance Test Manager will need resources from the PDA and the Sponsors
to contribute to their Acceptance Test(s). For the purpose of this paper these
resources have been called a group because in reality they will mostly belong to other
teams or organisations and there will be no line manager responsibilities. For
example members of the PDA from the Implementation, POCL Infrastructure,
Security or Testing Teams may all contribute to an Acceptance Test and the Sponsors
may also choose to be represented.
The Acceptance Groups will be formed for each type of event to ensure that the most
suitable business representatives are present. For example the Sponsors may choose
which type of Acceptance Trial for each Acceptance Test they need to witness.
The Acceptance Manager
The role of the Acceptance Manager is to co-ordinate and facilitate the PDA
Acceptance Process and ensure that all the business requirements in the Related
Vesion 1.00 Authorised Page 10 of 16 Date 22/05/97
FU,
Acceptance of the Operational Trial
Agreements are fully covered in a complete and comprehensive way. The Acceptance
Manager will be managed by the Release Authorisation Manager
1. The Acceptance Manager will manage the Acceptance Support Team which includes
the Acceptance Incident Manager and the Acceptance Process Support Team.
1. The responsibilities include:
definition of the process, ensuring all are able to fulfil their roles;
ensure that the process fits with that required by release authorisation;
monitoring and recording the progress of Acceptance Incidents;
Acceptance planning;
managing the Acceptance Support Team;
interfacing with the activities that are running in parallel, such as Release
Authorisation;
monitoring progress and producing management information for the PDA and
Sponsors;
providing a focal point for ICL Pathway, the PDA and Sponsors;
monitoring and recording the sign off of Acceptance Trials and Tests;
ensuring that all the Services and Deliverables in the Related Agreements are
included and are complete and that all interested parties have been consulted;
checking on probity of the procedures;
manage the Acceptance Support Team in their tasks of incident management,
producing management reports on progress, managing a PDA database of
information on the status of Acceptance Criteria.
0.1. The Acceptance Incident Manager
1. The Acceptance Incident Manager will be part of the Acceptance Support Team and
will be responsible for managing the acceptance incident process by:
maintaining the Acceptance Incident database;
managing any Acceptance Groups deemed necessary;,
producing management reports;
planning;
liaising with the Testing Team;
Vesion 1.00 Authorised Page 11 of 16 Date 22/05/97
FUJ00119567
1J00119567
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
0.1.
. monitoring the resolution of Acceptance Incidents;
. co-ordinating and distributing information.
This post should be an full time Higher Executive Officer grade post. The candidate
would need an understanding of the Related Agreements and of processes.
The Acceptance Process Support Team
This team will support the Acceptance process by:
. maintaining the Acceptance Database;
. distributing and managing documentation;
. minuting meetings where necessary;;
. co-ordinating diaries.
This post should be an Executive Officer grade post and will be the equivalent of one
whole time person.
ACCEPTANCE DEFICIENCIES
Errors that occur during Acceptance will be classified as Acceptance Incidents and
will be analysed by ICL Pathway in accordance with the process described in the
relevant Acceptance Specification.
The Acceptance Incident Manager will log all the Acceptance Incidents that have
been raised onto a database and monitor the progress. Incidents that have failed to be
resolved or are causing concern will be entered in to the standard PDA Issues
Management Process by the Acceptance Manager.
A more detailed process for managing Acceptance Incidents will need to be
developed and could be based on that used in for Model Office Testing.
THE NEXT STEPS
This paper has been intended to provide an overview of Acceptance. Some of the
major tasks in the next steps are given below:
. planning and preparation for the ongoing monitoring of ICL Pathway’s plan
(Acceptance Manager);
. producing a deliverable description for an Acceptance
Specifications(Acceptance Manager);
. designing a process and procedures for Managing Acceptance Incidents
(Acceptance Manager and Acceptance Incident Manager);
Vesion 1.00 Authorised Page 12 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
. consolidating the available information on acceptance criteria, requirements,
responsibilities etc. into a database (Acceptance Process Support Team);
. designing a process and procedures for sign off (Acceptance Manager);
. further work on the membership and responsibilities of the Acceptance Board
(Acceptance Manager);
. liaison with ICL Pathway (Acceptance Manager and Acceptance Test
Managers);
. accepting Acceptance Specifications (Acceptance Manager, Acceptance Test
Manager and the Acceptance Groups).
Vesion 1.00 Authorised Page 13 of 16 Date 22/05/97
Acceptance of the Operational Trial
FUJ00119567
FUJ00119567
ANNEX A - PATHWAY ACCEPTANCE SPECIFICATIONS
The following table lists ICL Pathway’s Acceptance Tests and the ICL Pathway Test
Managers, the Acceptance Test Managers.
Acceptance Test PWY Test Manager I PDA Acceptance Test
Managers
APS/HAPS David Cook Bob Booth
Audit John Dicks Tom Patterson
BES David Hollingsworth Mike Hannont
CAPS Interface Tony Hayward Mick Clarke
End to End Service Barrie Proctor Colin Oudot
EPOSS John Dicks Graham Seedall
Help Desk Denise Ivermee Colin Standring
MIS Peter Robinson David McLaughlin
OBCS David Cook Mike Hannon
PAS/CMS Tony Hayward Kerry Hartley
POCL Infrastructure David Cook Richard Hill
POCL/DSS Implementation - Part
A User Training and.
Implementation
Pamela Barlow
Trevor Rollason
POCL/DSS Implementation - Part
B Marketing
Pamela Barlow
Debra Channon
POCL/DSS Implementation - Part
C Services
Denise Ivermee
Karen Rogers
POCL/DSS Implementation - Part
D Rollout
Pamela Barlow
Douglas Craik
Reconciliation John Dicks Tom Patterson
I Ref Data John Dicks Richard Hill
I Securit Barrie Proctor Gareth Lewis
TIP Interface John Dicks _ John Cottam
Vesion 1.00 Authorised
Page 14 of 16
Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
ANNEX B - ACCEPTANCE SPECIFICATION - DELIVERABLE
DESCRIPTION
PROGRAMME DELIVERY AUTHORITY
DELIVERABLE DESCRIPTION
TITLE Acceptance Specification
OWNER ICL Pathway
Deliverable Description written by Mary Reade - Acceptance Manager
PURPOSE
An Acceptance Specification is produced by ICL Pathway to give a detailed description of an
acceptance test and all its component parts.
COMPOSITION
This document must follow the standard defined in the Related Agreements, which is as
follows:
(a) the purpose and scope of the acceptance test;
(b) the detailed procedures for raising and resolving acceptance incidents for the
acceptance test;
(c) the acceptance period for the acceptance test;
(d) the deliverables and services which are subject to acceptance, and the acceptance
method to be used to accept each. The acceptance method can be by one or more acceptance
trials or one or more acceptance reviews.
(e) the acceptance criteria for each deliverable or service, where appropriate a reference
to the Related Agreements, and for each acceptance criterion the acceptance test conditions
to be used for establishing that the acceptance criterion has or has not been met;
(f) the guidelines to be applied during the analysis of acceptance incidents, in order to
establish the severity of such acceptance incidents;
(g) the test data necessary for any acceptance trials;
(h) the Authorities responsibilities, including resources required to carry out each
acceptance trial or acceptance review.
FORM/FORMAT
A4 Word document
DERIVATION
The Related Agreements
Acceptance Criteria
SADD and other documents that are referenced form the Related Agreements
ICL Pathway “Standard for documenting Acceptance Specifications” 19/9/96
ICL Pathway “Standard for Specifying Acceptance Criteria” 9/9/96
ICL Pathway “Standard for Raising and Progressing Acceptance Incidents” 19/9/96
EXTERNAL DEPENDENCIES
ICL Pathway are responsible for producing acceptance specifications in accordance with
their plan.
Vesion 1.00 Authorised Page 15 of 16 Date 22/05/97
FUJ00119567
FUJ00119567
Acceptance of the Operational Trial
QUALITY CRITERIA
Does the document comply to the ICL Pathway standards and to those in the Related
Agreements?
Is it complete?
Do all the references to the service or deliverable in the Related Agreement link to an
acceptance criteria?
eg. requirements
solutions
policies and standards
service levels
Operational Trial
service definitions
software
hardware
Are the acceptance criteria complete?
Are there any references to the service or deliverables in the related documents e.g. SADD,
and if so are they included?
Are all types of acceptance trial included e.g.
monitoring;
meetings;
reference site visits;
MOT;
Live Trial;
presentations;
testing.
Are the test conditions appropriate to the acceptance criteria?
Are you be satisfied that this acceptance test covers the whole of the topic?
Will you be reassured that contractual acceptance can be agreed after completion of this
acceptance test.
QUALITY MEASUREMENT METHOD
Quality Review by the PDA (and Sponsors?).
Note: Check the PDA requirements and their owners spreadsheet to find the main interested
parties and include the Acceptance Manager.
RELATED REQUIREMENT NUMBER(S)
All requirements
RELATED STRATEGIC MILESTONE NO
Vesion 1.00 Authorised Page 16 of 16 Date 22/05/97