POL00393874
POL00393874
Ref: PD/TRP/001
benefits
hal
agency
Bringing Technology to Post Offices and Benefit Payments
JOINT TESTING AGREEMENT Controlled Copy No
Reference: PDA/TST/jtagrmnt
Author: C. McCrohan
Classification: Reference document
Version: 0.4
Date : 30/01/98
Status: Issued
Authority: PDA Mitchell Leimon
ICL Pathway John Wooler
‘Document Summary:
The document sets out the agreement for joint testing, between ICL Pathway’s Testing &
Business Integration Group and PDA’s Testing & Integration Group. The scope and purpose
lof the agreement are described, as are the organisations’ roles and responsibilities within the
agreement .
Circulation
Contact Area Copy
30 January 1998 Draft 0.4 Page I
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
0. Document control
0.1 Version History
Version Issue Change Details
Date
Version A dd/12/97 initial draft reviewed by Simon Rilot
draft
Version B dd/12/97 I second draft reviewed by PDA T&I
draft
Version C 22/12/97 third draft reviewed by POCL, DSS, PDA and
draft ICL Pathwar
Version 0.4 30/01/98 final draft for review by POCL, DSS, PDA and
issued ICL Pathwa:
0.2 Related documentation
1.
ICL Pathway document by John Hunt: Revisions to the Testing & Integration Approach
for ICL Pathway Release 2, Version 2.0, 24/10/97 (VI/STR006)
. PDA document by Keith Hall: Nile Release 2 End to End Interface Model Office Testing
Approach
. PDA document by Paul Duff/John Murray: Release 2 Stage Management Plan
. PDA document by Catherine McCrohan: Joint Business Test Approach
. PDA document by Don Dougherty: Joint Security & Technical Test Approach
c:\new_folder1\mikel.doc Page ~2 of 11
30 January 1998 Draft 0.4
POL00393874
POL00393874
benefits
TESTING & INTEGRATION
GROUP.
Contents Page
1. PURPOSE OF DOCUMENT...
2. BACKGROUN!
3. EXCLUSIONS.....
4. REVIEW...
5. ICL PATHWAY AIMS OF JOINT TESTING...
6. PDA AIMS FROM JOINT TESTING AGREEMENT........cccsesseeseen
7. MANAGEMENT APPROACH.....c.ecsssseesseeneseeenenee
8. EMPOWERED DECISION-MAKING.
9. JOINT TESTING ACTIVITIES...
9.1 JOINT TECHNICAL TESTING (JTT).
9.2 JOINT BUSINESS TESTING (JBT)..
10. RESPONSIBILITIES...
10.1 ICL PATHWAY. . 10
10.2 PDA Test & INTEGRATION. 10
11. CONFIDENTIALITY...
12. LIMITATIONS.....ccssesseeseeeee
c:\new_folder1\mikel.doc Page ~3 of 11
30 January 1998 Draft 0.4
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
1. PURPOSE OF DOCUMENT
This is a PDA paper which sets out, at a high level, the Joint Testing Agreement
between ICL Pathway Testing & Business Integration Group (which includes the
Security & Technical testing at the ICL Solution Centre) and the PDA Testing &
Integration Group.
The agreement is described in terms of scope, roles, responsibilities and working
practices. This document is supported by documentation describing detailed local
arrangements which operate within the framework of this agreement.
2. BACKGROUND
For background, see John Hunt’s paper “Revisions to the Testing & Integration
Approach for Pathway Release 2”, Version 2.0, 24/10/97. In the latter part of 1997, a
review of testing activity related to the BA/POCL Programme was carried out by PDA,
ICL Pathway and sponsors. A key conclusion of the review was to ratify ICL
Pathway’s recommendation to operate a form of joint testing. This paper describes what
“joint testing” means.
3. EXCLUSIONS
This agreement excludes arrangements relating to all other interactions between ICL
Pathway and PDA/sponsor testing organisations, eg support arrangements during test
phases; PDA assurance of ICL Pathway test material; PinICL Management; or progress
reporting.
The agreement also excludes other ICL Pathway and PDA/sponsor interactions, such as
those relating to assurance or acceptance.
The agreement does not address Acceptance. At the time of writing, agreement has yet
to be reached on the conduct and management of Acceptance Testing, so this document
does not anticipate that agreement. It is possible that the joint testing resource may
have involvement in Acceptance Testing on behalf of either ICL Pathway or the PDA,
and in the event of such involvement being agreed, that involvement will be
documented.
4, REVIEW
This agreement is being put in place for ICL Pathway Release 2. At this time it is not
known whether a similar arrangement will be required for ICL Pathway Release 3, but
depending on the success of the arrangement and its relevance to ICL Pathway Release
3, it is likely that the agreement for Joint Testing will be extended.
It is planned to review the effectiveness of the Joint Test Agreement in May 1998. The
c:\new_folder1\mikel.doc Page ~ 4 of 11
30 January 1998 Draft 0.4
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
review is planned so that ICL Pathway and sponsors can be satisfied that the investment
of resource is justified. The review will also help to inform planning for ICL Pathway
Release 3.
ICL PATHWAY AIMS OF JOINT TESTING
ICL Pathway has three key objectives for joint testing:
(a) enhanced quality of test;
(b) speedier decision-making from PDA/sponsors;
(c) _ earlier resolution of scope and coverage discrepancies resulting in a more robust
product, more representative of the real requirements, and with reduced rework
in the later stages of testing.
This document proposes that the first objective will be achieved by the joint testing
resource working within ICL Pathway, that the second will be served by improved
processes in relation to access to the existing decision-makers and that the third will be
served by a combination of this agreement and actions taken for (b).
PDA AIMS FROM JOINT TESTING AGREEMENT
PDA and sponsors support ICL Pathway’s aims and have committed to providing
resource to ICL Pathway for Joint Testing. The return to sponsors from this agreement
lies mainly in the aspiration that investing in the early stages of testing will pay dividends
in later stages. As a more direct and immediate return, the PDA and sponsors require
improved reporting from ICL Pathway and increased visibility of progress and activity.
Accurate, comprehensive and timely reporting from ICL Pathway is critical in
supporting PDA and sponsor activity.
The reporting requirement is a by-product of this agreement rather than a direct
deliverable from it. Status, progress management and reporting remain the responsibility
of ICL Pathway management, and will formally be communicated by ICL Pathway via
agreed routes, such as the fortnightly Checkpoint meeting. Improved reporting
processes are being actively discussed at present between PDA Release Management
and ICL Pathway Programme Office. The revised reporting arrangements will be
documented in the PDA Release 2 Stage Management Plan.
c:\new_folder1\mikel.doc Page ~5 of 11
30 January 1998 Draft 0.4
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
(a) MANAGEMENT APPROACH)oint Testing is undertaken to inject business
knowledge from PDA/Sponsors into ICL Pathway for the purpose of enhancing
ICL Pathway’s test preparation and fault management.
(a) Joint Testing is a set of activities which input to ICL Pathway. The resource
provided to ICL Pathway will have business operational experience, and may
additionally have some testing experience, but the emphasis is on providing ICL
Pathway with that which it cannot provide itself: business knowledge.
(a) Joint Testing will operate under the direction of ICL Pathway and will not,
therefore, have any assurance aspect. ICL Pathway retains responsibility for
management and adequacy of the testing approach, and for the quality of the test
documentation. PDA Product Assurance (formerly PDA Service Development)
remains the appropriate PDA body for the provision of business, security and
technical assurance.
(a) The PDA/Sponsor resource working with ICL Pathway on Joint Testing will work
within Pathway teams under the direction of ICL Pathway team leaders. The PDA
will provide management to work with ICL Pathway in managing this resource, to
provide line management (rather than day-to-day management) and to ensure that
the operation of the agreement is effective.
(a) The PDA line management will also report back into the PDA in relation to the
effectiveness of the agreement and resource utilisation. PDA and sponsors need
regular assurance that the Joint Testing resource is adding value. The PDA Joint
Test Managers (Catherine McCrohan and Don Dougherty) will provide weekly
progress reports on the joint testing activity to the PDA Test & Integration
Manager (Simon Rilot), and will copy the ICL Pathway Test & Business
Integration Manager (John Wooler).
(a) Should testing issues arise over progress, reporting or quality, the proper
escalation path for the PDA Joint Test Managers will be within ICL Pathway, via
the following ICL Pathway managers: Test & Business Integration Manager (John
Wooler), the Development Director (Terry Austin) and finally the Programmes
Director (Mike Coombs).
(a) Concerns will be raised by the PDA Joint Test Managers within PDA only after
discussion with ICL Pathway. Any such concerns would then be discussed by the
PDA Test & Integration Manager (Simon Rilot) and PDA Testing Co-ordinator
(Mitchell Leimon) with their ICL Pathway counterparts in advance of any wider
escalation.
c:\new_folder1\mikel.doc Page ~ 6 of 11
30 January 1998 Draft 0.4
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
EMPOWERED DECISION-MAKINGThose working for ICL Pathway under this
agreement will be able to advise of likely PDA/sponsor priorities and impacts based on
agreed documentation and operational experience of the sponsors’ businesses, but will
not have decision-making authority. Decision-making authority rests with PDA staff in
PDA Product Assurance (formerly known as PDA Service Development, comprising
Product Management, Technical Assurance, Business Assurance and Fraud & Security
Group) and backwards into sponsors.
The diagram below documents how incident analysis will operate.
Relationship between Pathway (incl. Joint Test Resource) and Product Assurance Activities
SADD.
RCD
Specifications
TS I
oz HLDS
==
r LLDS
Stress
Note 1: The 70/30 ratio above is an estimate.
SPONSORS
Product
Assurance
Agreed SADD, RCD & referenced doc’n are
detailed in specifications and designs A
© 70% oF incicents should be followed up by
Pathway by reference to document set, and
operational knowledge from Joint Testing
©
30% of incidents
will get passed
to PDA
‘product
experts’ for
resolution
Incidents
Testing
produces
incidents
Testing
available to support the testing teams is key.
Note 2: PDA Product Assurance participates in test preparation
Level Test Plans in advance of their use in test execution.
misapprehensions about how the services are intended to operate.
Note 3: In line with Section 3, Acceptance is not covered by the diagram.
The extent to which authorised documentation is
vorkshops, and reviews High
This reduces the possibility of
c:\new_folder1\mikel.doc
30 January 1998
Page ~7 of 11
Draft 0.4
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
JOINT TESTING ACTIVITIESThis section provides an overview of the applicable
joint testing activities. Resource will be deployed to work with ICL Pathway, within this
agreement, on System Test, BIT, DIT, End to End (E2E), Security and Technical
testing streams. The work on E2E is separate from the PDA activities on E2E and
Model Office Test (documented in the Nile Release 2 End to End Interface Model
Office Testing Approach). The resource will operate within two PDA teams, the Joint
Business Test Team at Feltham (managed by Catherine McCrohan), and the Joint
Technical Test Team at Bracknell (managed by Don Dougherty).
6.1 Joint Technical Testing (JTT)
The Joint Technical Test team is a PDA team which works with The Solution Centre,
Bracknell, on ICL Pathway’s Security and Technical testing. Due to the nature of these
test streams, the emphasis for Joint Testing in these stages is on technical as well as
business issues, and the input will be weighted accordingly.
(a) The PDA Security & Technical Test Team will work with ICL Pathway on test
preparation pertaining to security and technical aspects of the services. This work
will comprise writing of, and documentation review for, release test strategies,
High Level Test Plans and Low Level Test Scripts. It is possible that the PDA
Security & Technical Test Team skills may be more usefully deployed in relation
to the review of documentation, and if so, this will be agreed.
(a) The PDA Security & Technical Test Team will review all incidents to assign an
initial view of business impact.
(a) The PDA Security & Technical Test Team will, depending on available resource,
execute test scripts and raise PinICLs as appropriate.
More detail about the Joint Technical Tests may be found in the Joint Security &
Technical Test Approach, see reference 5.
6.2 Joint Business Testing (JBT)
6.2.1
The Joint Business Test Team will provide resource to work with ICL Pathway on
System Test and BIT. In addition, a DSS and a POCL resource will work with ICL
Pathway to support DIT and E2E tests, in particular to aid ICL Pathway with analysis
on data reconciliation and accounting. More detail about the Joint Business Tests may
be found in the Joint Business Test Approach, see reference 4.
Prioritisation
It is proposed to concentrate the team’s effort onto areas of greatest likely return, and to
further prioritise in relation to the areas of most significance to sponsors in the solution.
c:\new_folder1\mikel.doc Page ~8 of 11
30 January 1998 Draft 0.4
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
Accordingly, it is proposed that the JBT will concentrate on Test Preparation, then
Incident Analysis and finally Execution, depending on available resource. This order
reflects where the JBT is likely to be of greatest value to support the joint testing aims.
Additionally, it is proposed that the JBT concentrates on EPOSS, APS, Migration, BPS
and then OBCS. It is intended to cover all of these areas, but in the event of a need to
prioritise, this is the order in which the services will be considered. This order of
priority has been proposed after discussion with ICL Pathway and PDA Product
Assurance, and it reflects both the relative complexity and visibility of services as well as
historical concerns.
6.2.2 System Test & BIT Activities
(a) The JBT will work with ICL Pathway on test preparation pertaining to EPOSS,
APS, BPS, OBCS, migration, RDMC, TPS and possibly Audit. This work will
comprise writing/documentation review for Business Threads, High Level Test
Plans and Low Level Test Scripts. It is possible that the JBT skills may be more
usefully deployed in relation to the review of documentation, and if so, this will be
agreed.
(a) The JBT will review all System Test & BIT incidents to assign an initial view of
business impact.
(a) The JBT will, depending on available resource, execute test scripts and raise
PinICLs as appropriate.
6.2.3 Review Exercise
As this agreement is being effected after a considerable volume of work has already been
produced by ICL Pathway, it is proposed that an initial review exercise be carried out to
consider the test materials already produced, so that ICL Pathway can have some
retrospective business input. Where traceability is not explicit, the JBT will attempt to
provide it. The approach to this exercise will be set out in another document.
6.2.4 DIT & E2E
6.3 The DSS and POCL resource will work with ICL Pathway, mainly during test execution,
and will concentrate on incident analysis. The focus of the work will be on data reconciliation
and aiding ICL Pathway in test analysis to facilitate more speedy resolution of incidents. This
work is within the Joint Test Agreement and is not related to the PDA management activities
at Borough High Street, eg the DIT Co-ordination function, or indeed sponsor project
activities. RESPONSIBILITIESICL Pathway
c:\new_folder1\mikel.doc Page ~9 of 11
30 January 1998 Draft 0.4
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
(a) Retains responsibility for the testing approach and quality of test
documentation;
(b) To ensure that the PDA is fully apprised of progress and quality of test, through
the normal reporting channels;
(c) To ensure that Joint Testing resource is used to add value rather than as a
necessary part of the head count;
() To ensure that the resource supplied as part of Joint Testing is supplied with
equal access to documentation (with the exception of commercially sensitive
documentation), as well as appropriate accommodation and associated support
(eg PC, email, access to PinICL).
(e) To provide management information sufficient to enable the joint test managers
to plan and manage resource levels. This information will need to be sufficient
for the PDA Joint Testing management to ensure that the agreement is
operating effectively;
(f) To describe and gain agreement for revised management reporting processes.
6.4 PDA Test & Integration
(a) To provide appropriate resource in line with levels agreed with ICL Pathway for
the purpose of Joint Testing,;
(b) To allow those resources provided to ICL Pathway to fully support ICL
Pathway in line with this agreement;
(c) To ensure that the Joint Test Agreement is being adhered to by resource
provided to ICL Pathway, in particular that value is being added and that the
resource is providing input to ICL Pathway.
(d) To ensure that PDA management provided for Joint Testing will brief PDA on
issues relating to the provided resource and required resource, and will restrict
reporting on progress, issues and quality of test to that information published by
ICL Pathway for that purpose or as otherwise set out in the Management
Approach section of this document.
c:\new_folder1\mikel.doc Page ~ 10 of 11
30 January 1998 Draft 0.4
POL00393874
POL00393874
TESTING & INTEGRATION
GROUP.
CONFIDENTIALITYThe PDA staff provided to ICL Pathway in line with this
agreement shall not disclose material identified as ICL Pathway confidential
information or material to the PDA or sponsors. Any information or material obtained
in the course of Joint Testing which is indicated by ICL Pathway personnel as not open
for disclosure will be regarded as confidential.
Material identified as confidential which is disclosed by the Joint Testing staff to ICL
Pathway will not be disclosed by ICL Pathway to external parties.
7. LIMITATIONS
The following limitations are set out to avoid the possibility of a subsequent
misunderstanding by PDA/sponsors or ICL Pathway.
e Information passed between the Joint Testing staff and ICL Pathway staff, ie within
the ICL Pathway Testing & Business Integration Group, will not be regarded as
formally passed from one organisation to another.
¢ The Joint Testing staff may not: act on behalf of PDA/sponsors; perform a
reporting role for ICL Pathway to PDA/sponsors: or respond on behalf of
PDA/sponsors, eg requests for clarification of requirements.
e The PDA and sponsors are providing the Joint Testing resource in line with this
agreement and in the expectation that value will be added by their involvement.
The Joint Testing resource has business knowledge based on operational experience
and will endeavour to support the aims of this agreement. The PDA emphasises
that input from the Joint Testing resource will be based on experience and by
reference to available documentation, but that any information/clarification formally
required of PDA/sponsors will need to be requested of PDA/sponsors or ratified by
them.
c:\new_folder1\mikel.doc Page ~ 11 of 11
30 January 1998 Draft 0.4