POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Document Title:
Document Reference:
Document Type:
Release:
Abstract:
Document Status:
Author & Dept:
External Distribution:
Security Risk
Assessment Confirmed
Approval Authorities:
Approach to Testing for the Horizon Online™ system
TST/GEN/STG/0906
Strategy
Horizon Online™ Release 5
This is a joint Fujitsu Services/Post Office contract controlled
document describing the strategic approach to be applied for all
testing and integration activities performed for the Horizon Online
system, post Release 2
APPROVED
James Brett (POL)
(Specify those individuals outside of the Royal Mail Group Account
who required approved version only. For RMGA Document to
distribute following approval)
YES, confirmed, see section 0.9
Signature
Tonnvane Wiswell POL Test Manager See Dimensions for record
Debbie Richardson RMGA Test Manager ‘See Dimensions for record
See HNG-X Reviewers/Approvers Matrix (PGM/DCM/ION/0001) for guidance on who should approve.
© Copyright Fujitsu Services
Limited & Post Office Limited 2011
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
CONFIDENCE) Version. 2.0
UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 1 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
0 Document Control
0.1 Table of Contents
i} DOCUMENT CONTROL
0.1 Table of Contents
0.2 Document History
0.3 Review Details.
0.4 Associated Documents (Internal & External)
0.5 Abbreviations
0.6 Glossary.
0.7 Changes Expected . 10
08 Accuracy....
0.9 Security Risk Assessment
4 INTRODUCTION... 1
1.4 Document Purpose....
41.2 Related Documents and Processes
41.3 Document Structure
1.4 Assumptions.
1.5 Exclusions
2 MANAGEMENT SUMMARY
2.4 Scope of Testing
2.2 Joint Working ..
2:3 Testing Approach
24 Environments
2.5 Test Tooling
2.6 Key Planning
3 BACKGROUND... 16
4 APPROACH.
44 Considerations
42 Testing Stages.
43 Joint Working
44 Testinvolvement with Development
5 TEST REFERENCE DATA... 19
& TEST ENVIRONMENTS +20
6.1 Live Support Test (LST).
62 sval....
6.3 Development Rig:
6.3.1 Integration ..
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 2 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
7 GOVERNANCE PROCESS...
7.
7.4 Organisational Structure ..
7.2 Reporting ...
Test Strand Reporting .
Programme Level Test
Test Status Reporting .
Defect Reporting ..
7.2: Acceptance Status Reporting
7.3 Meetings
7.4 Risks Management Approac!
75 Entry and Exit criteria...
8 TOOL SETS.......... 27
9 TEST AUTOMATION............ 29
10 ACCEPTANCE...... 31
B Release acceptance...
4
10.2 User involvement (Customer Experience)
RESPONSIBILITIES, STAFFING, AND TRAINING NEEDS
People and Roles
Support Roles .
Staffing and Training Needs 33
3.1 Skills. siseeeeseseee 33
A RELEASE 5 SPECIFIC TESTING CONSIDERATIONS .........csssesesesereees +36
A.1 CT916: Operator Self Funded..
A.2 CT918: Front Office of Government — IT Enhancements .
A.3 CT910: Repositioning of Partner Bank Withdrawal Prompt.
A.4 CT908/922: Test Tool Automation
Al 6 CT951: PAF Replacement
A.7 CT979: APOP Enhancements.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 3 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
0.2 Document History
VersionNo. Date Summary of Changes and Reason for Issue Associated Change -
CP/PEAK/PPRR
Reference
0.4 First Draft — informal review to identify gaps and dependencies
02 21/09/10 Second Draft — for formal review
03 28/10/10 First Issue — in response to comments
1.0 11/1110 First Approved version
14 11/05/11 Updates to;
+ Remove Release 3 and 4.
* Document the approach to Maintenance releases
testing
* Cover Release 5
1.2 26/05/11 Updates following formal review
1.3 15-JUN-2011 Further updates relating to comments received from Steve
Porter, Tonnvane Wiswell, Gareth Jenkins, and Nick Battle
2.0 23-Jun-2011 Approval version
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 4 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
0.3 Review Details
See HNG-X Reviewers/Approvers Matrix (PGM/DCM/ION/0001) for guidance on completing the lists below. You
may include additional reviewers if necessary, but you should generally not exclude any of the mandatory reviewers
shown in the matrix for the document type you are authoring.
* = Indicates retumed comments on previous version
Review Comments by
Review Comments to James brett.
& RMGADocumentManagemen
Mandatory Review
Role Name
POL Principle Test Manager Tonnvane Wiswell!
POL Release Manager Graham Bevan
POL Design Authority lan Trundell
FS Programme Test Manager Debbie Richardson
FS Release Test Manager Sheila Bamber
FS SV&l Test Manager Chris Maving
FS LST Test Manager Mark Ascott
FS Development Manager Graham Allen
FS Development Manager David Harrison
FS Development Jon Hulme
FS Development Nick Battle
FS Integration Manager Vijesh Pandya
FS Requirements Manager David Cooke
FS Design Authority (Stock Ordering) Duncan MacDonald
FS Design Authority (Pay Pal) Allan Hodgkinson
FS Design Authority (Pay Pal / FDG / PAF) ‘Andy Thomas*
FS Design Authority (IT Enhancements) Sarah Selwyn *
FS Design Authority (IT Enhancements) Chris Bailey
FS Design Authority (Operator Self Funded / Banking Gareth Jenkins*
Withdrawals / Client File Delivery)
FS Design Authority (Operator Self Funded / Banking Ged Griffiths
Withdrawals)
FS Design Authority (Client File Delivery / FOG) Pete Jobson
FS Design Authority (Test Tool Automation) Steve Porter
Role Name
FS Release Manager David Hinde
FS Programme Director David Court
FS CS Manager Sarah Bull
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TSTIGEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY Date’ 23-Jun-2011
STORED PageNo: 5 of 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system ~
FUII TSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
FS Deployment delivery manager Martin Brett
POL IT Projects manager Chris Taylor
POL Chief Architect Peter Stanley ¢
Issued for Information ~ Please
distribution list to a minimum
Position/Role Name
POL IT Enhancements Project Manager Tom Basquille
POL PayPal Project Manager Noel Beaton
POL Operator Self Funded Project Manager Phil Norton
POL PAF Replacement Project Manager Noel Beaton
POL Test Manager Michele Gilliver
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TSTIGEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY Date’ 23-Jun-2011
STORED PageNo: 6 0f 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system ~
FUII TSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
0.4 Associated Documents (Internal & External)
Referen Version Date Titl Source
PGM/DCM/TEM/0001 I 4.0 21-Nov-2008 RMGA HNG-X Generic Document Dimensions
(DO NOT REMOVE) Template
PGM/CHM/PRP/0001 HNG-X Release Approach Options Dimensions
Paper
TST/GEN/STG/0001 HNG-X Testing Strategy Dimensions
TST/GEN/STG/0002 Approach to testing for HNG-X Dimensions
TST/GEN/STG/0009 I draft HNG-X Post release 1 Test Rig Dimensions
Strategy
DEV/GEN/SPE/0007 Platform Hardware Instance List Dimensions
PGM/PAS/PRO/0002 HNGxDBM Requirements and Design I Dimensions
Process
PGM/PAS/PRO/0003 HNGxDBM Code, Build and Dimensions
Component Test Process
PGM/PAS/PRO/0004 HNGxDBM Test Planning and Dimensions
Preparation Process
PGM/PAS/PRO/0005 HNGXxDBM Test Execution Process I Dimensions
TST/MGT/STG/0001 HNG-X Risk Management Plan Dimensions
TST/GEN/REP/0006 HNG-X TESTING TOOLS Dimensions
REQUIREMENTS
RD/PRO/OOS Reference Data Handling For Test PVCS
Rigs
TST/GEN/STG/0007 HNG-X TEST STREAM Dimensions
REFERENCE DATA PROVISIONING
STRATEGY
Release 5 Specific
Documents
QAS/REQ/001 QAS interface Replacement for PAF I Dimensions
Service Description Summary
DES/APP/DPR/1312 PAF Replacement Service Design Dimensions
Proposal
REQ/CUS/CDE/1309 I 2.1 27 January 2011 Automated Test Tool Requirements Dimensions
Specification
DES/APP/DPR/1047 ‘Automation Test Tool Design Dimensions
Proposal
DEV/APP/SPG/1208 Automation Test Tool Support Guide I Dimensions
REQ/CUS/CDE/1183 I 1.1 FiTE Requirements Catalogue Dimensions
REQ/CUS/CDE/1315 I 1.1 FiTE Requirements Catalogue Dimensions
DES/APP/DPR/1311 Generic Web Service Design Dimensions
Proposal
DES/APP/DPR/1257 Script from Script Design Proposal Dimensions
DES/APP/DPR/1256 Data Persistence Design Proposal Dimensions
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 7 of 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system ~
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Reference Version Date Title Source
DES/APP/DPR/1308 Encrypt/Decrypt Design Proposal Dimensions
DES/APP/DPR/0012 Operator Self Funded Design Dimensions
Proposal
DES/APP/DPR/1175 Client File Delivery Design Proposal Dimensions
DES/APP/DPR/1352 Two Factor Authentication for TESQA I Dimensions
Design Proposal
TSTISYT/HTP/1349 Maintenance Release 4.34 HLTP Dimensions
Unless a specific version is referred to above, reference should be made to the current approved
versions of the documents.
0.5 Abbreviations
Abbreviation Definition
ADC/ AP-ADC Automated Payments Additional Data Capture
ADSL Asymmetric Digital Subscriber Line
AEI Application, Enrolment and Identification Service
APOP Automated Payment Out Pay
BA Business Analyst
BAL Branch Access Layer
BAU Business As Usual
BPD Business Process Diagram
BRDB Branch Database
CBA Counter Business Application
cIT Component integration Test
coB Close Of Business
COTs Commercial Off the Shelf software
cs Customer Service
csc Computer Science Corporation
DR Disaster Recovery
E2E End to End
Fu/FS Fujitsu Services
FRES First Rate Exchange Services
FX Foreign Exchange
GPRS General Packet Radio Service
HMRC. Her Majesty's Revenue and Customs
HP Hewlett Packard
Is Infrastructure Services
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 8 of 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system ~
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Definition
Integrated Services Digital Network
Independent Test Unit
JPM JP Morgan
LST Live Support Test
NPS Network Persistant Store
NRT Near Real Time (ADC Messaging)
os Operating System
OSR Online Service Routing
PA Permanent Agent
PAH Primary Account Holder
PAN Primary Account Number
PC Personal Computer
PCI Payment Card Industry
PHU Portable Horizon Unit
PIN Personal Identification Number
POca Post Office Card Account
POL Post Office Limited
PUN Pick Up Notice
ac Quality Centre
RDDS Reference Data Distribution Service
RDMC Reference Data Management Centre
RDS Reference Data System
RDT Reference Data Team/Test environment
SPTS Service Provision Test Support [Team]
sval Solution Validation and Integration [Test]
Tc Travellers Cheque
Tco Total Cost of Ownership
TPoS Travel Point of Sale
UL User Interface
VAT Value Added Tax
VPN Virtual Private Network
WSDL Web Services Description Language
XML Entensible Markup Language
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 9 of 65
FUJITSU
Approach to Testing for the Horizon Online™ system
FUJITSU RESTRICTED (COMMERCIAL IN
POL00039120
POL00039120
CONFIDENCE)
0.6 Glossary
Term
Definiti
HNG-X
The name of the project to migrate Horizon to the new infrastructure and replace the
counter facing data centre components
Horizon Online™
The name of the system providing IT support for the Post Office Counters
IRE14 Fujitsu’s Primary Data Centre for HNG.
IRE19 Fujitsu's Secondary / DR / Test Data Centre for HNG
MDM Post Office Reference Data Management System
PEAK Fujitsu’s incident management System
POLMI Post Office Management Information System
POLSAP Post Office Financial System
POLVAT The project implementing VAT conformance for certain counter transactions
SYSMAN2/3 Fujitsu's System Management systems
TPoS Phase 2
The project implementing the redesigned Bureau de Change transactions at the
counter
VSAT
Satellite connectivity solution for Horizon Online
0.7 Changes Expected
This document will be updated to describe the approach for future releases of Horizon Online.
0.8 Accuracy
Not Applicable
0.9 Security Risk Assessment
Security risks have been assessed and it is considered that there are no security risks relating specifically to this
document.
© Copyright Fujitsu Services
Limited & Post Office Limited 2011
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
CONFIDENCE) Version. 2.0
UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 10 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
1. Introduction
This document describes the strategy and testing approach for the Horizon Online™ system. This system
was implemented by the IT migration project HNG-X, which is referred to as HNG-X Release 1. To
distinguish between the migration project and subsequent enhancements to Horizon Online™, the next
release of HNG-X is referred to as HNG-X Release 2 in this document. It is assumed that future releases
will follow the same naming convention.
1.1 Document Purpose
This document is an addendum to the contractually controlled document (CCD) entitled Approach to
testing for HNG-X [TST/GEN/STG/0002] which was the testing strategy for HNG-X (Release 1), distilled
from the generic strategy document [TST/GEN/STG/0001].
As such this document is only intended to describe the differences and any additions to the testing
strategy for future releases of HNG-X. Where the contents of future releases are known, appendices
within this document will provide specific considerations. The document and its appendices will need to
be reviewed for subsequent releases.
1.2 Related Documents and Processes
The following documents defined the existing approach to testing and the related processes. Those items
in bold text are of particular interest as they relate to the scope of this document.
TST/GEN/STG/0001 = HNG-X TESTING STRATEGY
TST/GEN/STD/0002 = Approach to testing for HNG-X
TST/GEN/STG/0010 Approach to testing for HNG-X Release 2
PGM/PAS/PRO/0001 ~=HNGxDBM Overview
PGM/PAS/PRO/0002_ HNGxDBM Requirements and Design Process
PGM/PAS/PRO/0003_ HNGxDBM Code, Build and Component Test Process
PGM/PAS/PRO/0004 HNGxDBM Test Planning and Preparation Process
PGM/PAS/PRO/0005 ~HNGxDBM Test Execution Process
PGM/PAS/PRO/O006 +HNGxDBM Implementation Planning and Release Process
PGM/PAS/PRO/0007_ +HNGxDBM Support Documentation Process
PGM/PAS/PRO/0008 HNGxDBM Generic Acceptance Process
PGM/PAS/PRO/0009_ ~HNGxDBM Integration Process
TST/MGT/STG/0001_ HNG-X Risk Management Plan
These were produced for Release 1 and should be reviewed to capture any changes to the processes.
and to identify where potential changes are required or would be recommended.
1.3 Document Structure
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 11 of 65
FUJITSU
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
2 Management Summary
3 Backround
4 Approach
5 Test Reference Data
6 Test Environments
7 Governance Process
8Tool Sets
9 Test Automation
10 Acceptance
11 Responsibilities,
Staffing, and Training
Needs
Appendices
A- Release 3
considerations
B - Release 4
considerations
This section provides a summary of main changes and the overall testing approach
for Release 2.
This section provides a brief summary of the Release 1 and 2 testing strategy.
This section defines the testing approach for Release 2 and arrangements for joint
working.
This section describes the approach to the provision of test reference data
This section describes the test environments required to support testing of Release
2
This section lists the toolsets to be used.
This section lists the tools used to support testing
This section described the use of automation and its expanded use.
This section defines how user acceptance will take place.
This section lists the test roles and responsibilities.
Describes what level and scope of testing required for each specific change within
the future releases known to date.
© Copyright Fujitsu Services
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
CONFIDENCE) Version: 20
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 12 of 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system ~
FUII TSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
1.4 Assumptions
. : oo : —
vib : oe La
Assumption 1 No changes to branch hardware or counter O/S
Assumption 2 LST plus 1 additional existing test rig as-is will support future release testing — implies no
changes required to the rig designs.
Assumption 3 Testing will be in-house and not outsourced or subcontracted.
Assumption 4 The HNG-X hardware in IRE11/19 is reserved for HNG-X use only.
Assumption 5 The scope of testing remains as independent testing of systems components
delivering as integrated, installable packages. It covers system validation and
integration and live support testing.
Assumption 6 Future releases will preserve the fixes previously made and any additional fixes made in the
maintenance delivery of the current release
Assumption 7 Future release changes will be separate from release of BAU patches.
Assumption & Major Re-Accreditation testing will not be required unless the changes identify a
new 3" party..
Assumption 9 The continued use of Doors by both POL and FJ for requirements capture is
assumed.
Assumption 10 No requirement for any Volume or Integrity testing in future releases (of which the content is
currently known)
1.5 Exclusions
Exclusion Id Lo Description _ Co
Exclusion 1 Large scale Infrastructure changes are excluded from future releases.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 13 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
2 Management Summary
This paper defines the approach for the ongoing testing of the Horizon Online™ system.
Rather than focus on a specific release, the bulk of this document is intended to define the ongoing
approach to testing of the Horizon Online™ system, which is generic and consistent across releases.
Appendices will consider the specific testing requirements of each change contained within the coming
releases.
2.1 Scope of Testing
The likely profile of releases going forward can be surmised as:
e Anumber of small maintenance releases soon after the current release goes live
e Further releases affecting both the counter and data centre with significant business focussed
change
e Patch releases - monthly operating system and other third party product patches that must be
applied to maintain conformance
At time of writing, only regular maintenance and patch releases can be expected and planned for; the
scope of future releases is volatile and therefore subject to change. To prevent the need of a separate
test approach document for each release of HNG, this document will contain details of each Release,
and considerations that need to be made when determining the appropriate test approach. These
considerations will be listed as appendices, and the document will be revised on a regular basis.
2.2 Joint Working
Based on the success of Joint Working it is intended to continue joint ownership of testing. The combined
skills and experience of POL and Fujitsu staff have provided a well balanced team across the various
roles. Post HNG-X Release 1, the overall team size will be significantly smaller so it will be essential that
roles are filled appropriately.
This will need to include the involvement of POL staff during testing covering the “customer experience”.
2.3 Testing Approach
The approach will continue to be based on the following principles:-
e Risk based testing
e Progressive, incremental development and testing
e Early involvement of test staff in project requirement review and development activities
e Adherence to gateway criteria such as test stage entry criteria
« Cyclic testing in parallel to development to optimise scheduling / timescales
e Progressive acceptance as tests are completed
The testing approach for future releases differs from that adopted for HNG-X Release 1 as future
releases are likely to consist mainly of business change to the existing solution without the introduction of
additional hardware or COTs software. As a result, no signification accreditation, volume or integrity
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 14 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
testing will be required. This means that testing can be supported within Development through
component integration (CIT) and integration testing. Functional business testing of the changes to the
Solution, plus the appropriate level of regression testing, will be achieved by the Solution Validation
(SV&l) test team using a single test environment.
Testing of regular patch updates and maintenance releases will be undertaken on the LST environment,
Preproduction testing remains unchanged with the release being checked/rehearsed via LST prior to
implementation in the live system.
Unit testing > Component Integration Testing > Integration (build package) Testing > SV&I > LST > Live
Although the fundamental approach for future releases follows the same waterfall based approach used
in the initial releases, once the grouping and delivery schedule is understood for the coming releases, a
hybrid approach could be adopted which allows for some early releases of subsets of changes and a
number of iterations to build up the final set of released functionality. This will lead to some lifecycle
phases overlapping.
2.4 Environments
To support the testing the following test environments are required (small Development kit is not listed):-
e CIT test rig -for continuous CIT testing
« Integration Test rig — for testing packaging and builds
e Solution Validation & Integration test rig —- This will be the SV&l test rig as-is from Release 1
testing
e Live Support Test (LST) will continued to be used for final pre-production proving
« RDT test rig — for Reference Data Validation and Verification
2.5 Test Tooling
As part of the objective to reduce TCO and improve delivery timescales, use of test automation will be
overhauled to make use of existing development automation capabilities. After an initial proof of concept,
the development derived test automation framework will be deployed onto SV&I along with each release.
This automation framework will offer the benefits of unattended execution, and will allow the expansion of
the automation suite to encompass a larger share of the regression test overhead.
Testing will continue to use the same test management (Quality Center) and defect management (Peak)
tools as well as the same management processes and procedures.
2.6 Key Planning Constraints
e Only 1 test environment (in addition to LST) will be used. Project planning will have to be
based around this limitation and is likely to affect test scheduling, number of testers required,
and the shape of future releases.
e Although there is likely to be technical testing in future releases, such as testing changes to
data centre components, it is assumed that future releases will not require extensive non
functional test, including volume, integrity, scalability, disaster recovery and security.
e SV4&l testing will be planned into test cycles; however the length of these cycles and number
of iterations will be driven by contents and requirements of the release.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 15 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
3 Background
The development and testing of HNG-X release 1 was a large and complex migration project. The testing
strategy for release 1 was based on Objective Driven Testing and incorporating Progressive Integration
and testing [see TST/GEN/STG/0002].
The Independent Test Unit (ITU) was organised into a number of test streams:-
e System Test - Test against Design Specifications — usually the first time complete subsystems
were tested as an integrated unit
e System Validation and Integration - Testing against Requirements - functional and non functional
covering business and infrastructure and based on testing the complete integrated solution
e Release Migration Testing — Tested the migration steps and plan
e Release Accreditation Testing — Tested interfaces with external systems
e Volume and Integrity Testing — Performance and stress testing
e Live Support Testing — release and fix deployment testing only
To support the test stream, each used a separate test environment based on hardware based in IRE19
(for the data centre components) and BRA0O1 (for testers’ workstations, business workstations, remote
servers, emulators and branch / counter systems).
The current HNG-X contract (defined in requirements 5493, 5489, and Schedule B3.3 ) requires that only
the LST rig exist on an ongoing basis plus the ability to create / maintain additional rigs if required for
more ‘in depth’ testing. For the second HNG release, functional testing was deemed required, therefore
the SV&l test rig was retained, whilst other rigs used at Release 1 were decommissioned. The test
approach employed at Release 2 is defined in TST/GEN/STG/0010.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 16 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
4 Approach
4.1 Considerations
Future release development and testing must be planned alongside any residual testing being
undertaken on the current release, Maintenance release(s), delivering business as usual fixes, Antivirus
updates, and security patches, will be subjected to functional and pre-production testing. Where
practical, future releases will be deployed in between maintenance releases. Special consideration needs
to be given if the scope of a future release overlaps with changes being deployed via the maintenance
stream — i.e. if a single platform is due to receive concurrent updates from maintenance and future
releases, testing may have to be sequenced accordingly, and in some circumstances, alternative options
may need to be considered to mitigate the risk or parallel releases under test at the same time. As a rule,
major releases will be subjected to functional/non-functional validation on SV&I followed by deployment
testing on LST. Maintenance Releases will validated on LST only.
The overall testing approach will be based on risk based testing (or objective driven testing) as defined by
the HNG-X Testing Strategy. In brief this means that test cases will be assessed regarding business
impact and importance which will be combined to derive a test priority. Test schedules will then take
these priorities into account wherever practicable to ensure the highest risk tests are run as early as
possible.
The size, content and project timescales of future Releases will have a direct impact to the scheduling
and execution of testing. Once the content for a given release is known, the joint test team will need to
consider;
e Grouping for optimal Dev scheduling,
« Grouping for optimal Test scheduling,
e what this means in terms of a delivery schedule,
e number of test streams and cycles and hence test rig configurations,
* whether the release can be split or needs to be a single release,
e need for accreditation testing
need for any non functional testing
The overall approach will be based on evolution rather than revolution — introduction of change only
where beneficial and cost effective.
4.2 Testing Stages
As HNG stabilises, the ‘big-bang’ of various testing stages employed at Release 1 can be scaled down,
as the requirement for large scale infrastructure and performance testing recedes. Assuming most future
HNG releases will be focussed mainly on development to the Counter Business Application (and
supporting BAL / BRDB components), the requirement to undertake the various forms of non-functional
testing (e.g. Security, Volume, Integrity, DR, etc.) is lessened. Future testing will therefore go through the
following stages;
Component Testing > Integration Testing > Solution Testing > LST > Live
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 17 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
As in earlier releases, testing overall will still follow a waterfall model, however recognition needs to be
given that future HNG releases will now deliver one or more desired changes to system functionality (i.e.
Change Request). The deployment of these releases may overlap in order to meet the needs and
constraints of each change, therefore a degree of parallelism may be required, but as a minimum, each
change should be taken through the above stages, .
To support the testing the following test environments are required (small Development kit is not listed):-
e CIT (including slight extensions from the CIT Release 1 environment)
«Integration (for build package testing)
e SV4&lI test rig (for Solution Validation & Integration testing. This includes integration with wider
E2E systems)
e LST test rig. (for deployment validation)
4.3 Joint Working
The aim is to continue with Joint working approached developed and built upon in the first releases of
HNG, involving both POL and Fujitsu staff, bringing a valuable synergy and greater skills / experience
mix.
Staff should continue to be co-located and integrated into specific teams. Nominally a ratio of approx 30%
POL staff is suggested.
4.4 Test involvement with Development
Timely involvement of test analysts with Development staff both helps the development area understand
the needs and plans of the testing to follow; and helps the test area to understand the scope of the
development activity and test needs. The lead designers in the development area should also define any
special test requirements and to review the testing plans.
It is expected that POL and SV&I Test analysts will continue to be engaged with development areas —
particularly on each periodic Counter release — to verify their high risk tests initially in the CIT
environment, thus addressing risk early. When required, SV&l testers will work with the Counter
development team in CIT to identify issues, enabling bugs to be identified and fixed within the
development cycle prior to a release being issued onwards onto Integration. The CIT environment also
enables SV&l testers to have early sight of new functionality which helps build familiarity of functionality
early in its delivery lifecycle.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 18 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
5 Test Reference Data
HNG-x testing has a significant dependence on the provision of Reference Data to allow and support
effective and thorough testing of Horizon Online business functionality. Where possible, live reference
data will be utilised to eliminate data discrepancies between live and test. Live reference data will be
provided on a regularly (normally weekly) data feed from Fujitsu Customer Services. Along with this data
feed, the CS Reference Data team will also provide data specific to coming changes. As much as
possible, this data will be as-live, but not yet enabled in the live estate — it will be enabled early by
CSRDT, on the test data stream only, to enable testing to commence. The test data stream will support a
number of test rigs (CIT, SV&l and LST) so changes should not be rig-specific. Any rig-specific data will
be applied directly onto the required rig, once the CS RDT data stream has been successfully applied.
Care should be taken to maintain the integrity of the CS RDT data stream, so that test-specific data does
not preclude the future delivery of CS RDT data.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 19 of 65
FUJITSU
CONFIDENCE)
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
FUJITSU RESTRICTED (COMMERCIAL IN
6 Test Environments
Contractually, Fujitsu Services are obliged to provide support for two test rigs on an ongoing basis; LST
and SV&I. The following Test Environment Configurations needs to be provided and supported for this
project.
6.1 Live Support Test (LST)
There is no change to the role of LST. The LST test rig will continue to be used to test all HNG-X fixes
and releases prior to deployment to Production systems. LST will also be the primary test environment
used to validate the contents of any maintenance releases The LST environment is a close logical
representation of the live environment, and is defined in the following documents:-
Reference Name Content
Allocation of platform instances to
DEV/GEN/SPE/0007 Physical Hardware Instance List hardware (plus other information
such as VM mappings)
DEV/INF/ION/0003
Network Database
VLan and IPaddresses
DEV/INF/LLD/0112
HNG-X LST TEST SERVICES LOW
Low level network design
LEVEL DESIGN
Data Centre SAN Storage Layout
DEV/INF/LLD/0043 and racking
HNG-X SAN STORAGE MAPPING
Branch equipment and communications are not listed in the above. The LST rig has every type of counter
(PC, PHU 1.5) and comms (ADSL, ISDN, GPRS, VSAT).
LST also has Disaster Recovery capability for the non-data centre equipment. The data centre Disaster
Recovery components are limited to a DAT server with the main LST databases being replicated in
IRE11. The LST Disaster Recovery rig is a facility that enables LST to continue to function, and not a
representation of the Live Disaster Recovery facilities, i.e. we can not use these LST Disaster Recovery
facilities to test Live Disaster Recovery functionality in any way.
6.2 SV&l
This test environment will be used as the primary test rig for all functional / non functional testing for all
future HNG-X releases.
SV&l is primarily focussed on the validation of the solution within the Horizon domain, however the
environment will flex depending on the requirements of each release. For example;
e During Release 2 : connectivity was established to VocaLINK, A&L and Streamline
e During Release 3 : Connectivity was established to AEI
Where end to end connectivity is not available, SV&I testing will prove up to the boundary of the domain —
i.e. outbound files will be validated for format and content manually.
The key assumption here is that for future release testing there will be none of the following:-
e Integrity testing since new platform types are not required
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 20 0f65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
e Volume testing as none of the new requirements cause significant increases in transaction
volumes.
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TSTIGEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY Date’ 23-Jun-2011
STORED PageNo: 21 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Reference Name Content
Allocation of platform instances
to hardware (plus other
DEV/GEN/SPE/0007 I Physical Hardware Instance List information such as VM
mappings)
DEV/INF/ION/0003 Network Database VLan and IPaddresses
HNG-X TEST SERVICES LOW
DEV/INF/LLD/0032 LEVEL DESIGN Low level network design
Data Centre SAN Storage
DEV/INF/LLD/0043 HNG-X SAN STORAGE MAPPING Layout and racking
Branch equipment and communications are not listed in the above. The SV&I rig has every type of
counter (PC, PHU 1.5) and comms (ADSL, ISDN, GPRS, VSAT"). Various reference data configurations
are supported to ensure that a wide selection of live office configurations are represented. This is not
intended to be an exhaustive configuration test, instead the most likely configurations will be simulated.
There is no DR available for SV&l. Also, in the event of IRE19 for live being used in a DR situation (or for
DR testing), the SV&l rig would be unavailable.
6.3 Development Rigs
6.3.1 Integration
The Integration rig is used to compile the software baselines from development into workable packages
that can be deployed to other test rigs or ultimately to live. This rig is dynamic. The configuration of the
Bladeframe depends on the system components being packaged and built.
6.3.2 CIT
The CIT rig is a counter-orientated rig comprising major component units integrated to enable early
discovery of defects, and exposure corresponding code and reference data for the first time. The CIT rig
comprises a set of NT Counter desktops, plus a number of discrete servers some hosting BAL/OSR
instances, some hosting Branch Database, NPS and APOP databases, some hosting a number of virtual
machines with Authorisation Agents and Web Services and one providing a reference data platform
(RDMC, RDDS). The various instances are used to concurrently maintain different versions of Counter &
BAL/OSR releases.
6.3.3. RDT
Also required is test reference data which is provided by the Reference Data Team (RDT). The process
for provision of Reference Data is defined in RD/PRO/005 and the method for managing test rig
reference data is described in TST/GEN/STG/0007.
1 For development and test there are 2 VSAT links available.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 22 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
7 Governance Process
7.1 Organisational Structure
Programme management and governance of HNG-X will be undertaken through the following programme
roles:
e HNG-X Release Board — has overall responsibility for managing and directing the HNG-X
releases. It will be the final point of escalation and arbitration for the joint test team. It will also
have a right of veto regarding authority being given to start a test stage.
e FJS HNG-X Release Manager — reports to the HNG-X Release Board and has overall
responsibility for managing the Fujitsu Services delivery of the programme.
« POL Release Manager — reports to the HNG-X Release Board and has overall responsibility for
managing the POL delivery of the programme.
e FS RMGA Head Of Testing — has overall assurance responsibility for testing across Royal Mail
Group Account projects
e POL Principle Test Manager — has overall assurance responsibility for testing across Post
Office Ltd projects and programmes. In relation to the HNG-X releases, the POL Principle Test
Manager is responsible for:
o Assuring that the POL test strategy is implemented within the HNG-X programme.
o Approving Post Office Ltd test resource allocations.
e FJS HNG-X Release Test Manager — reports to the FS RMGA Head Of Testing and has overall
responsibility for the Fujitsu Services strands testing each HNG release...
e FJS HNG-X Test Manager — reports to the FIS HNG-X Release Test Manager, with a line to the
FJS HNG-X Release Test Manager and jointly manages the delivery of HNG-X testing with the
POL HNG-X Test Manager. The main responsibilities of the FS HNG-X Test Manager are:
o Jointly with the POL HNG-X Test Manager, developing and maintaining the FIS HNG-X
level 2 and level 3 testing plans.
o Developing the HNG-X testing resource budget for FJS-POA and obtaining FJS-POA
testing resources.
o Reviewing and approving HNG-X test artefacts on behalf of FJS-POA.
o Jointly with the POL HNG-X Test Manager, directing HNG-X testing activities.
o Reporting on HNG-X test status through the FIS-POA management line.
o Jointly with the POL HNG-X Test Manager, managing testing risks and issues.
o Liaising with the development team regarding component level testing.
« POL HNG-X Test Manager — reports to the POL Principle Manager and jointly manages the
delivery of HNG-X testing with the FIS HNG-X Test Manager. He also has a reporting line to the
POL Release Manager. The main responsibilities of the POL HNG-X Test Manager are:
o Jointly with the FIS HNG-X Test Manager, developing and maintaining the FJS HNG-X
level 2 and level 3 testing plans.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 23 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
o Developing the HNG-X testing resource budget for Post Office Ltd and obtaining Post
Office Ltd testing resources.
o Reviewing HNG-X test artefacts on behalf of Post Office Ltd.
o Jointly with the FS HNG-X Test Manager, directing HNG-X testing activities.
o Reporting on HNG-X test status through the Post Office Ltd management line.
o Jointly with the FJS HNG-X Test Manager, managing testing risks and issues.
o Liaising with the POL Acceptance Manager regarding contractual acceptance of HNG-X.
7.2 Reporting
There are various levels of reporting outlined in the following sections. This reporting will be supported
by joint test team core roles which prepare, maintain and deliver report statistics and supporting
information.
7.2.1 Test Strand Reporting
Each test stream manager will produce a brief highlight report each week (using an agreed template),
with bullet points covering:
e Test strand status for time, cost and quality — flagged as red, amber or green, with a brief
comment where the status is not considered to be green.
e Key activities completed this week (bullet points including what went well and challenges)
« Key activities planed next week (bullet points)
e Risks and issue
« Milestone status against plan
Test strand reports will be submitted to the Joint HNG-X Test Managers by agreed time (e.9.12:00 each
Monday). They will be used as a basis for upward reporting and status reporting at the Joint Test Team
meetings.
Joint Test Team meetings will be held in Bracknell on an agreed day (e.g. every Tuesday). They will be
chaired by one of the Joint HNG-X Test Managers and will be attended by the lead representatives
covering Test Architect, Test Environments and the Test Stream Managers. Minutes of these meetings
will be held in PVCS Dimensions and in the POL HNG-X Programme Library.
7.2.2 Programme Level Test Reporting
The POL HNG-X Test Manager and the FJS HNG-X Test Manager will report progress up through their
respective programme management reporting lines.
The POL HNG-X Test manager will submit a highlight report to the POL HNG-X Programme Manager
(copied to the POL Test Manager) by COB each Wednesday. This will include:
« Overall test status for time, cost and quality — flagged as red, amber or green, with a brief
comment where the status is not considered to be green.
e Key activities completed this week (bullet points including what went well and challenges)
« Key activities planed next week (bullet points)
e Risks and issue
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED Page No: 24 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
e Milestone status against plan
The reporting of milestone status will be agreed between the joint HNG-X Test Managers. Any major
disagreements on the status of key testing milestones will be referred up to the Programme Board.
7.2.3 Test Status Reporting
Each test strand will report on test status as follows:
e During a test execution cycle, a weekly test status report will be produced at the end of each
week.
e During critical stages of test execution, daily test status reports may be produced. In general,
however, authorised personnel will be able to do ad-hoc test status reporting using the testing
dashboard or reporting tools within QC.
e Atthe end of each cycle, an ‘End of Cycle Test Report’ will be produced. This will be produced
from Quality Center, supplemented by text for explanation.
e At the end of a test stage, and ‘End of Stage Test Report will be produced. This will be a text
based report, with key statistics drawn from QC.
7.2.4 Defect Reporting
Defect status reporting will either be undertaken through Quality Center (QC) and/or the FIS PEAK
system. The joint test team will raise new defects in QC, which will then be fed into PEAK if the defect is
deemed to be a fault that needs fixing in the FS domain. QC will be used as the hub for supporting test
reporting data and acceptance reporting information.
Please see separate documentation for details on defect workflow and classification.
7.2.5 Acceptance Status Reporting
A separate document defines how testing supports the contractual acceptance process. In summary
acceptance is reported through references to functional use cases and non functional acceptance criteria.
It is assumed that the evidence required to support acceptance will be collated and reported through QC..
7.3 Meetings
During the preparation and execution phases of testing regular meetings will be held between joint test
team and relevant programme level representatives to ensure all parties can provide updates on
progress and discuss any concerns that may cause problems related to achieving the testing objectives.
The format will be combination of face-to-face meeting and telephone conferencing.
The frequency of meeting should be commensurate to the phase of the programme and tasks; it is
expected that fortnightly meeting will be sufficient during the earlier phase of the programme moving to
daily as we approach and enter later testing phases.
7.4 Risks Management Approach
Post Office Ltd testing risks and issues will be recorded on the Testing Risks and Issues Log. Any risks or
issues that could have an impact on the programme will be escalated to the Fujitsu and Post Office Ltd
Programme Managers and will be included in their Programme Risks and Issues Register.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 25 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
7.5 Entry and Exit criteria
Entry and exit to each defined test phase or cycle will be governed by entry and exit criteria. The criteria
will be defined in the High Level Test Plans.
Once the new automation tool is in place, a suite of automation scripts will be developed that can be used
to assure quality of deliveries from development — i.e. a set of automated scripts will be developed by the
Joint Test Team and run on the development environment prior to the dispatch of code. SV&I test will
only accept the code if all tests pass (or can be otherwise explained) in the development environment.
The automation suite will be repeated once the code is delivered to SV&I, to ensure no degradation
during Integration and Release Management processing.
Entry and Exit readiness meetings prior to the planned test phase or cycle dates will be used to review
and agree the readiness to progress through the entry or exit point. These meetings require data for
each defined criteria to be provided before the meeting and the meeting attendees should include
relevant test stream managers, FJS and POL test mangers, representatives from programme
management, development and those providing any environments, deliveries or support to the test
phase.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 26 of 65
FUJITSU
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
8 Tool Sets
The following tools are required to support testing. Please refer to TST/GEN/REP/0006 HNG-X
TESTING TOOLS REQUIREMENTS,
Tool / Product
Origin
Usage
Dimensions
COTS
Controlled Document repository
Platform configurations and
Product usage
Rig Configuration parameters
Quality Center
COTS
Requirements (replicated from
DOORS)
Test plans and scripts
Requirements Traceability
Defect tracking
Test Progress Reporting
Doors (POL and FJ)
COTS
Requirements Management
Doors — QC interface software
COTS
Peak
FJ Bespoke
Product Defect management
Lexcel
COTS
Banking Authorisation Emulator
In house emulators — these need to be
maintained and updated if required
FJ Bespoke
APOP
Track and Trace
MA Simulator (Streamline)
DVLA simulator
ETU Simulator
Network Banking Multiplexor
POCA Card Issuing Emulator
GWS Endpoint emulator
Connect Direct (remote)
COTS
COTS Product
Interstage
Fujitsu
Interstage development
environment - used for Counter
development smoke and
regression tests.
FUJITSU RESTRICTED (COMMERCIAL IN Ref:
© Copyright Fujitsu Services
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY Date’
CONFIDENCE)
STORED
TST/GEN/STG/0906
Version: 2.0
23-Jun-2011
Page No: 27 of 65
FUJITSU
CONFIDENCE)
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
FUJITSU RESTRICTED (COMMERCIAL IN
Tool / Product
Origin
Usage
In house Counter UI Automation Tool
FJ Bespoke
Counter UI Automation Too! — this
is a Java based tool developed by
the Counter development team to
enable automated testing of
Counter application by injecting
input behind the User Interface
(Ul). This is to overcome limitations
with commercial tools like
WinRunner that are heavily
dependent on the UI and require
much maintenance when the UI
changes. This tool is used for all
automated Counter smoke and
regression testing and is a
fundamental component of CIT
testing.
JProfiler
ej-
technologies
JProfiler — used for Counter
development performance and
memory leak tests
© Copyright Fujitsu Services
Limited & Post Office Limited 2011
FUJITSU RESTRICTED (COMMERCIAL IN Ref:
CONFIDENCE)
UNCONTROLLED IF PRINTED OR LOCALLY
STORED
TST/GEN/STG/0906
Version: 2.0
Date: 23-Jun-2011
Page No: 28 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
9 Test Automation
Test automation was introduced as part of HNG-X Release 1 testing to provide a standard capability to
support regression in CCIT and SV&I and counter performance testing. This automation testing method
continued into Release 2, with further manual regression tests being converted to automated.
A review is being undertaken to look at the automation utilised across Releases 1 and 2, Current
automation is constrained in the fact that an proportion of the automated tests are attended — i.e. requires
user involvement to enter details at the PIN pad or check receipts output to the printer. A proof of concept
has been commissioned to investigate the adoption of the bespoke automation tool currently in use by
the development team. Making reuse of existing development offers the following benefits;
e Incorporate existing development UI Automation peripheral emulators. This avoids the need for
human intervention whilst running the scripts, and should improve speed of test execution
considerably.
e Incorporate existing development UI Automation routines which can inspect peripheral output.
This has obvious benefits to increasing the scope of the automated tests, reducing human testing
time, etc. For example, this would cover test scenarios that cannot be done in CIT because the
components are not present within CIT.
e Potentially provide precise code coverage measurement and reporting of SV&l manual and
automated testing by instrumentation of the Counter.
« Potential side-effect of providing a more rapid means of delivering Counter updates to SV&lI.
e Ability to whitebox test database and Counter
e Migrate from current unsupported automation tool (WinRunner) to a java-based toolset that is not
dependant on target platform Operating Systems
The proof of concept will demonstrate the automation of a small range of counter transactions, running on
a single SV&I counter. This approach differs from the current development automation, whereby
automation is run directly on developers local machines. In SV&l, the automation scripts will be
developed on non-standard machines, but will be executed on the standard counter build
The proof of concept will have several limitations; automated transactions will not be written to the Branch
Database to preserver integrity. Automation will only be delivered for one version of the Counter Business
Application. A pre-defined set of tests will be delivered along with the Automation framework; it will not be
possible to develop further automation.
If successful, the proof of concept will then be expanded into a full-blown test automation suite. The
automation framework will be delivered along with each CBA software release, and will come with a pre-
defined set of ‘action’ commands. Automation scripters, (Joint Test Team test analysts with no specific
automation training), will tailor these action commands to undertake the desired counter activity. The goal
will be to develop an automation pack that initially matches current automation capability, but with a view
to expand to take on more manual regression testing, and to be easily updateable following each
software release. Automation testing should be able to complete entirely unattended, in a matter of
minutes rather than days. Interaction with the Branch Database will be enabled, allowing a wider impact
of automation across the target infrastructure. Supporting code coverage reports will help assess the
effectiveness of automated regression testing, and provide a comparison across each release.
Aspirationally, the automation framework will be developed to allow it to be controlled by the Joint Test
Team via the existing QC interface. This will simplify the scheduling and execution of automated tests,
and provide a direct link to business requirements and the defect management lifecycle.
Anew Automation HLTP (TST/SOT/HTP/0976) is being developed to restructure the planned automation
testing to be more targeted and efficient in identifying target areas of likely defects.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 29 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Where possible, the automation tool will be utilised to streamline and assist the testing of changes that
are currently tested by manual means — an example of this is using the tool to process bulk volumes of
PAF transactions to enable comparative assessments of performance of the existing and repleacement
PAF Service at HNG Release 5.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 30 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
10 Acceptance
10.1 Release acceptance
Independent testing will support both contractual acceptance of HNG-X and business acceptance of the
HNG-X solution.
e Contractual acceptance - will be managed by the Post Office Ltd Acceptance Manager, who will
define and agree the Acceptance Process with the relevant Fujitsu test and acceptance
authorities. This will be a progressive process through contractually agreed quality gates.
Evidence for contractual acceptance will be produced from document reviews, process
walkthroughs and the independent test strands undertaken by the joint test team. The evidence
will collated in QC and presented to the POL Acceptance Manager for formal sign-off
e Business acceptance — will be managed by the Release Authorisation Board. Evidence to
support business acceptance will be built up progressively through the various test strands within
independent testing. Again, evidence from independent testing will be assembles in QC.
Progressive acceptance will continue to be used to reduce any bottleneck of acceptance activity just
before any formal acceptance gateway. Progressive acceptance is supported by traceability and test
status information from Quality Center to determine when sufficient evidence has been collected against
acceptance criteria for it be agreed in principle. This then allows a simpler check at the acceptance
gateway for formal acceptance of the criteria.
10.2 User involvement (Customer Experience)
Customer experience is a key success factor in future POL projects. As such this area needs to be
covered throughout the project lifecycle including forming part of the business acceptance.
Ata high level this area includes:
e Usability trials of any customer facing solution elements early enough to be able to influence
development
e Where possible involvement of customers in aspects of the business functional testing
e Close working with business change teams to inform and test communication, training and
support materials
e Setting of test metrics to inform programme of progress towards desired customer experience.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 31 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
11 Responsibilities, Staffing, and Training Needs
This section presents the required resources to address the test effort outlined in the Test Strategy—the
main responsibilities, and the knowledge or skill sets required of those resources.]
The rationale for the proposed staff levels is also discussed in HNG-X Post release 1 Test Rig Strategy
[TST/GEN/STG/0009]}.
11.1 People and Roles
This table shows the staffing assumptions for the test effort.
Human Resources
Role Minimum Resources Specific Responsibilities or Comments
Recommended
(number of full-time roles
allocated)
Test Manager 2 Provides management oversight.
(1 for Fujitsu, 1 for Responsibilities include:
POL) e planning and logistics
e agree mission
e identify motivators
* acquire appropriate resources
* present management reporting
* advocate the interests of test
¢ evaluate effectiveness of test effort
Test Architect 1 Defines the technical approach to the
implementation of the test effort.
Responsibilities include:
e define test approach
¢ define test automation architecture
e define the test environment requirements
e verify test techniques
e define testability elements
Test Team Manager 1 for LST Plans and manages the detailed testing.
1 for SV&l Responsibilities include:
e Coordinate updates to the test environment
¢ Establish the detailed test schedules
e Allocate the testing resources to the tests
e Assess readiness before each test phase
starts
e Monitor and report progress
e Produce the test report
Test Analyst 6 for LST Identifies and defines the specific tests to be
10 for SV&I(6 for conducted.
Fujitsu, 4 for POL) Responsibilities include:
e identify test ideas
e define test details
¢ determine test results
« document change requests
e _ evaluate product quality
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY Date’ 23-Jun-2011
STORED PageNo: 32 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Human Resources
Role Minimum Resources Specific Responsibilities or Comments
Recommended
(number of full-time roles,
allocated)
Implements and executes the tests.
Responsibilities include:
e implement tests and test suites
* execute test suites
e log results
* analyze and recover from test failures
¢ document incidents
Rig Runner 2 SV&l & LST Ensures test data (database) environment and
assets are managed and maintained.
Responsibilities include:
e support the administration of test data and
test beds (database).
SPTS 1 SV&l only Provides local build support. Responsibilities
include:
Build the counter systems and local comms
Take backups of counters
Install counter comms
Assist with builds on other non-data centre
platforms
11.2 Support Roles
Other units will need to provide support to the test unit.
e Release Management (to coordinate upgrades to the test rigs)
« 1S/ Network Support (to implement changes as defined and on instruction from the test manager
11.3 Staffing and Training Needs
11.3.1 Skills
The following table lists the business and technical areas and the recommended skill level (hence
training). In some cases the skill level only applies to those job holders who are expect to acquire / have
specialist knowledge (e.g. only a few testers will need SYSMAN3 hands on experience)
Table 1 Skills Profile Matrix
4 = expert 5 z ee = s
2=user 5 BS se I] 8 5 g
= = gs 3 S &
3 = aware z 3 ee 8 4
Quality Center 2 2 2 2 3 3
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TSTIGEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 33 of 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system ~
FU] ITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
1 = expert g g gs z 5
2=user Fa s 2 Hy 2 é 5
3 = aware 3 3 g= / B 2
Peak 2 2 2 2 2 2
Dimensions 2 2 2 2 2 2
Project Web / Collaboration 2 2 2 2 2 2
oc 2 2 2 3 3
HNG-X overview 4 4 4 4 4 4
Unified process 3 2 2 2
HNG-X Platform design 3 2 3 2 2
HNG-X Storage design 3 2 3 3 2
HNG-X Network design. 3 2 3 2 2
HNG-X on line functionality 3 2 2 1 3 3
HNG-X batch functionality 3 2 2 1 1 3
HNG-X branch functionality 3 2 2 1 3 3
SYSMAN2 3 3 3 2 2
SYSMAN3 3 2 3 2 2
Security 3 2 3 2 2
Integrity 3 3 3 2 3
Branch rollout 3 2 3 3 2 2
Unix 2 2
Windows 2 2
Solaris 2 2
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 34 of 65
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
POL00039120
POL00039120
A Release 5 Specific Testing considerations
At the time of writing, Release 5 contains the following changes;
e CT916 - Operator Self Funded
e CT918 - Front Office of Government IT Enhancements (FiTE)
e CT910 - Repositioning of Partner Bank Withdrawal Prompt
* €T908/922 - Test Tool Automation
« CT951—PAF Replacement
* CT979-—APOP Enhancements
* CT928 - Client File Delivery (CFD)
e CT895 — Fujitsu Data Gateway (FDG)
Along with the following Changes proposed internally by Fujitsu;
© CP504 - DX! Get Well
e CP595 — DXI SSL Scanning
e CP570-—EMDB Versioning
© CP571-—VSAT
e CP555— TWS Monitoring
The following changes are considered out of scope for Release 5;
e CT924a - Stock Ordering
« CT920 - PayPal (pending confirmation from PayPal)
«© C1776 - 2 Factor Authentication for TESQA
The changes listed above are likely to be delivered in a number of phases, rather than a single Release 5
deployment;
« Release 4.34 Maintenance Release : Will contain the POLVAT receipt rounding change,
Partner Bank Prompt and the counter aspect of the Generic Web Service part of FiTE.
« ‘Early’ Release 5 Data Centre only Release : to deliver the PAF replacement within the QAS.
contract expiry timeframe.
* Release 5 Phase 1 : to deliver FiTE changes, PayPal (if in scope), Operator Self Funded, APOP
enhancement and the bulk of the internal CP's
e Release 5 Phase 2 : to deliver Client File Delivery and FDG
The following sections will consider the testing implications of each of the external changes contained
within Release 5 — it is assumed that the standard testing approach described within the main element of
this document will be followed for the internal CP's listed above..
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 35 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A separate update to this document will be provided to detail the test approach for Client File Delivery
and Fujitsu Data Gateway once more aspects of the solution design are known.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 36 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A.1 CT916: Operator Self Funded
A.1.1 Background
Post Office Limited want to introduce a new business model that will allow external Operators to fund and
manage a reduced range of Post Office Services. A key component is for the Operator to inject cash into
the Post Office business that they undertake, To allow the Post Office to distinguish the cash injected by
the Operator, changes are required to the way Cash is defined and managed within Post Office
Reference Data and Financial Systems.
Self funded Operators will be remunerated under a different commercial arrangement to existing, Post
Office funded Sub-postmasters, Franchises and Agents.
A.1.2 Scope
Operator Self Funded will be implemented via a combination of code change and reference data. Code
changes are required as followed;
1. to the Horizon counter application and supporting databases\applications
2. to the POL SAP financial accounting system.
Reference data will be used to distinguish Operator Self Funded offices from other office types, and
remunerate these offices based on transactions performed.
The following diagram depicts all of the systems and interfaces potentially impacted by this change.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 37 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Clients
HQ functions Outputs
Reports Vendors
MDM Ref. Data POL SAP PDF Reports
Trx.
Ref. Data
TPS Suracts
Ref.
RDMC/RDDS pata ™™
BRDB
Ref data} Tr. t por Reports
BAL
Ref. Data, Trx. PDF Reports
Counter
To describe the above diagram, in terms of components and interfaces;
Compone:
Function Description of changes
Counter Point Of Sale e Process new POE settlement reference data products
e Complete Rem In / Out of Self Funded Cash
« Complete restricted set of transactions
e Restrict acceptance of Cheques for settlement
« Amendment to Transaction Correction settlement
options (removing cheque)
« Produce new ‘Office Weekly’ report
e List available PDF reports
e Print/Preview selected PDF report
BAL Branch Access Layer e Pass through PDF report requests
— service routing
BRDB Branch Database — e Process new POE settlement reference data products
data/transaction .
repository ¢ Store Self Funded transactions
e Store Overnight Cash Holdings entries
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 38 of 65
FUJITSU
Approach to Testing for the Horizon Online™ system
FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
POL00039120
POL00039120
Component ription of changes
Batch process to collect and store PDF Report files
TPS Transaction Process new POE settlement reference data products
Processing Service —
preparation of data
for external
processing
Process new Alternate Product to Article mapping
reference data
Process and aggregate transactions from BRDB,
using Alternate mappings as appropriate
Output Branch Ledger Entry (BLE) files containing
Self Funded transactions — for loaded into POLSAP
Output existing extracts to HRSAP and POLMI
(containing Self Funded transactions)
RDMC/RDDS_ I Reference Data
Management Centre
Reference Data
Distribution Service
Process new POE settlement reference data products.
Process new Alternate Product to Article mapping
reference data
Generate reference data views for TPS /BRDB
Distribute Self Funded Products to appropriate
counters
Store and Distribute restricted Menu Hierarchy to
appropriate counters
MDM Master Data
Manager - Post
Office Reference
Data system
Deliver new POE settlement reference data products
Deliver new Alternate Product to Article mapping
reference data (non-core links)
POLSAP. Post Office SAP
financial accounting
system
Process new Article to Account mapping reference
data
Generate PDF reports and manage transmission to
filestore within Fujitsu domain, including error handling
Generate Group PDF reports for distribution to P&BA.
HQ Functions Product & Branch
Accounting (P&BA)
staff in Chesterfield
Financial reconciliation
Settlement with Vendors / Clients
Undertake Dunning Process
Authorise and account for Emergency Payments to
Self Funded Offices
PDF Report management and summarisation
Vendors / Self Funded
Clients Operators
Receive Settlement information
Reconcile financial transactions
Receive / Make payment for transactions performed.
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 39 of 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system ~
FUII TSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Component Function ription of changes
POL MI Post Office « Receive transaction details from TPS
Management
Information system e Provide environment for transaction query, analysis
and data mining
HR SAP Post Office « Receive remuneration data from TPS
Remuneration
system ¢ Settle with Postmasters
A.1.3 Test Basis.
The Post Office has specified a set of requirements for the Operator Self Funded change. These specify
the changes in a set of Use Case definitions and Non-Functional statements. These statements will for
the basis for design, development, testing and acceptance.
Fujitsu Services have produced a Design Proposal [DES/APP/DPR/0012] documenting the changes
required to the Fujitsu elements of the overall solution. It is expected that a lower level design document
will also be made available to detail the development required on POLSAP and the Horizon Online
counter. The Design Proposal will be cross-referenced to the requirements catalogue once baselined.
A.1.4 Test Approach
The CIT environment will be used to prove the fundamental aspects of the counter changes, but
environmental constraints limit the scope of testing in CIT. There is no capability to generate BLE files or
process PDF reports, so testing will focus on the counter application change, ensuring the menu
hierarchy restrictions and other business rules are correctly applied. CIT will test up to BRDB with
simulated input of pdf reports.
Similarly, LST is constrained by the fact that there is no connectivity between POLSAP and the LST
environment, therefore E2E testing is not possible. Testing will focus on the migration activities within the
Horizon domain; i.e. launching Operator Self Funded outlets. It will also focus on the changes to TPS and
BRDB in support of the OSF changes, but will not interface BLE of PDF reports to\from POLSAP.
It is proposed that changes to the Horizon Online counter application will be tested on the SV&I
environment.. Changes to the POLSAP financial accounting system will be tested on a POLSAP test
environment independently, before an E2E Assurance test is completed on the overall solution. The
previous diagram has been overlaid with the boundaries of each test phase, which is described in detail
below;
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 40 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
POL SAP Testing
Clients
HQ functions Outputs
Vendors
Reports
Ref. Data
PDF Reports
MDM POL SAP
Ref. Data
} Existing
extracts
Ref.
RDMC/RDDS Data__™ tf
Ret. Data] Tix
BAL
Ref, Data}, Tx. PDF Reports
Counter
Horizon Testing
Testing of the Fujitsu elements of the overall solution will be undertaken on the SV&l test environment
alongside the testing of other Release 5 changes. Testing will be undertaken by members of the Joint
Test Team, and will focus on;
e Functional validation of the counter changes; namely;
o Process new POE settlement reference data — this is an existing interface but new
‘Type’ standing data is being introduced, so testing will focus on the continued operation
of reference data once the new type data is made available.
o Remittance of Self Funded Cash — Cash is not currently supported in Rem IN/OUT
to/from Client mode, so testing will focus on the availability and support available for new
products in this existing mode.
o Availability of a restricted set of transactions — Testing will focus on the modifications
to the Menu Hierarchy to only display appropriate transactions and administration
services.
o Restricting the acceptance of Cheques for settlement — by removing the buttons and
references to Cheques from the system. Testing will validate that acceptance of Cheques
will be effectively disabled at the point of sale, and will also look for hidden dependencies
on other related changes (I,e the Policing MOP change introduced as part of HNG R2.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 41 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
co Production of a new ‘Office Weekly’ report — validation of report format and content
co The listing of available PDF reports, and the subsequent Print and/or Preview of
any selected reports — This is a totally new Use Case/function, so will be tested
comprehensively.
NOTE: Some of the above changes are not delivered by code, instead they are BAU reference data
activities that will be made prior of in conjunction with the code changes delivered fro OSF. Defects
determined as a result of this reference data change will be routed to the BAU ref data teams
accordingly.
* non-functional testing of the supporting systems and databases as follows;
o Reference Data — All existing interface specifications are expected to remain. However
there will benew standing data, signifying the alternative POLSAP mappings to be used
for Operator Self Funded branches, and there will be new processes developed to
extract relevant data for supporting systems such as TPS and BRDB. It is assumed that
a feed of live reference data will be intercepted (as per current test process), and fed into
the test environment to enable testing of the down stream processing.
o Transaction Extraction and Amalgamation — Existing Interfaces to POL accounting
and Management Information systems are expected to remain unchanged. Testing will
ensure this is the case, and will also focus attention on the new scheduled task that of
maps product to various POLSAP Article depending on the Office Type that performed
the transaction. Connectivity to external systems will not be available during this phase,
‘so test outputs will be validated locally.
o PDF Reports - changes to support receipt storage and transmission of PDF files from
POLSAP. This is a new interface, and will require testing of file receipts as well as testing
of error-handling routines. A direct interface will not be available within this phase of
testing, therefore the PDF payload may well be emulated, by the manual copy of PDF
data to the specified filestore. Testing will be undertaken on the new scheduled
processes that collect the reports and load them into the Branch Database.
o Migration —i.e. the phased implementation of Data Centre components followed by
subsequent Counter updates then eventual activation via reference data.
Tests for this phase will be documented in the R5 HLTP (TST/SOT/HTP/1051). The test plan will list all
Operator Self Funded requirements relevant to the Fujitsu domain, and will provide an analysis of test
coverage against requirements. During testing a view of requirements status may be provided assuming
baselined requirements have been loaded into Quality Centre. On completion of testing, test results and
details of any outstanding issues will be logged in the R5 Test Report (TST/SOT/REP/???7).
Volume and Performance testing is considered out of scope of this test phase — the majority of change
falls within existing, un-changed interfaces and it is assumed that existing capacity is sufficient. Testing of
the new PDF report process is deemed (within the Design Proposal) to be fairly insiginificant.
POLSAP Testing
Testing of the POLSAP elements of the overall solution will be undertaken on the PLE [?] test
environment. Testing will be undertaken by members of the SAP Basis development team, with
input/assurance from members of the POL Test Team. Testing will focus on;
¢ Functional validation of the POLSAP changes; namely;
o Reference Data - The continued ability to process Article to Account mapping reference
data
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 42 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
o PDF Reports — The ability to generate PDF reports in the correct format with the correct
content. Also the generation of Group PDF reports for distribution to P&BA
o Financial reconciliation — the ledgering of summarised transaction data to Self Funded
Operator accounts.
o Undertake Dunning Process — to prompt Self Funded Operators for settlement.
o Settlement with Vendors / Clients - the process of clearing accounts once settlement
is confirmed as received.
o Emergency Payments - Authorise and account for Emergency Payments to Self
Funded Offices.
e non-functional testing of the supporting processes;
o Report Management - PDF transmission to filestore within Fujitsu domain, including
acknowledgement and error handling
o Migration — from current process to new.
It is assumed that this phase of testing will be documented in a test plan that will be made available to
POL / Joint Test Team for assurance. It is further assumed that project requirements pertinent to
POLSAP will be covered in this phase
It is assumed that volume and performance testing is out of scope for this phase of testing.
End to End Testing
The objective of this phase will be to integrate the individual components into an overall solution. This will
only occur once each changed component is proven to be robust and largely error-free. A readiness.
review will be held to confirm that this is the case before End to End (E2E) testing commences. Individual
component test plans will be reviewed by members of the Joint Fujitsu/POL test teams to develop a
Master Test Plan that will detail the E2E testing to be undertaken.
Any defects identified during E2E test will be jointly review by Horizon and POLSAP test teams to ensure
the root cause is correctly identified.
An integrated E2E test environment no longer exists, therefore it is proposed that outputs from individual
components are manually transferred (via email, for example), between systems for onward processing.
An example of a pseudo-E2E test is described below.
1. Ensure that all the required ‘standing’ reference data changes have been keyed into live MDM.
2. Produce ‘standing’ reference data extracts for all receiving systems (Horizon, POLSAP, POLMI
and HRSAP)
Intercept live ‘standing’ reference data extracts onto relevant test environments.
Identify test office requirements and develop ‘non-core’ reference data for testing purposes.
Load ‘non-core’ reference data to all appropriate test environments.
Log on to Operator Self Funded offices and undertake a broad variety of transactions
Collect BLE files once produced by the Horizon host systems and email to XI team.
XI team load idocs into SAP test environment
POL test team check SAP against expectations from counter.
0. POL test team run reconciliation, dunning and settlement processes.
1. POL test team test production of periodic PDF reports for Operators. These get emailed to the
Joint Test Team.
12. POL test team verify PDF report acknowledgements.
13. Joint Test Team load PDF files into Branch Database
14. Joint Test Team validate reporting at counter based on PDF report content.
15. POL test team validate existing outputs to POLMI and HRSAP.
ASL ONOMAY
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 43 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A full schedule of E2E activity will be developed and documented in the Master Test Plan. Upon
completion of testing, an E2E Test Report will be issued documenting test status and listing any
outstanding defects. At this stage, it is likely that any outstanding defects will be taken through the
deferral process, for resolution in subsequent releases. It is also at this point where a complete view of
requirements coverage can be obtained and a decision for Release and Project requirements acceptance
can be made.
It is assumed that volume and performance testing is out of scope for this phase of testing.
A.1.5 Test Environment
Horizon testing will make use of the existing SV&l test environment. Source PDF reports may be
emulated, however if viable outputs from the POLSAP system are available, these will be used in
preference.
POLSAP testing will make use of the PLE [?] test environment. Source BLE files will be emulated,
however if viable outputs from the Horizon system are available, these will be used in preference.
End to End testing will use elements of both test environments.
A.1.6 Dependencies
Live-like ‘standing’ reference data must be keyed into the live reference data system MDM, and extracts
must be sent to test representatives from all receiving systems (Horizon, POLSAP, POLMI and HRSAP).
The Operator Self Funded Menu Hierarchy must be defined and made available to the Fujitsu reference
data team for encoding.
The Application and Technical Interface Specifications for the PDF Report interface must be available
and sufficiently detailed to allow valid testing to occur.
A.1.7 Constraints
All testing should be considered for impact on the Payment Card Industry regulations — as no customer
sensitive data is held relating to this change, no specific PCI testing will be undertaken.
A.1.8 Limitations
As previously stated;
e Performance and Volume testing are excluded from all test phases.
e End to End testing will be simulated through transmission of data by manual means. As the
formal interface will not be used, there is a small risk that capacity issues will not be captured
during E2E testing — however the low volumes of OSF office numbers and frequency of reports
make capacity issues unlikely.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 44 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A.2 CT918: Front Office of Government — IT Enhancements
A.2.1 Background
As part of the Post Office Ltd (POL) initiative on Front Office of Government (FiTE) an IT Enhancement
project has been raised to deliver capability at the Horizon counter that will enhance the current AP-ADC
toolset.
The changes affect the AP-ADC behaviour at the counter; the resultant transaction is the same as current
AP-ADC transactions and will be treated the same for accounting, reconciliation etc.
The enhanced capability is seen as a key enabler for PO Ltd to be able to provide services to
government in a timely and cost efficient manner. The capability will be added generically to the AP-ADC
toolset therefore making it available to all transaction designers, regardless of client.
This changes fall into 4 distinct categories, these are considered sub-requirements of the overall Front
Office of Government - iT Enhancement (FiTE) project;
FiTE1 - Generic Web Service Capability - Ennancements to the existing GenericOnline datatype
to allow access to a generic web service at the data centre which is capable of interacting with a third
party web service. The third party service must adhere to an agreed interface and be compatible with the
Horizon Generic Online interface (i.e GWS will not provide a browser capaibility on the counter). The
new generic web service is introduced at Rel 5 and that this enables future GWs agents to be introduced
using a client take on activity independent of release.. This will allow POL to interface to new third party
web services via introduction of reference data that defines the service offering for the client — i.e. no
large-scale code or infrastructure changes are required (it is assumed that individual service configuration
may be defined as Java script, but this is handled as BAU development-sourced reference data),
FiTE2 - Data Encryption / Decryption — Introduction of new Encrypt and Decrypt datatypes to
allow the encryption/decryption of customer sensitive data contained within an AP-ADC transaction. This
will allow POL to transmit customer sensitive data to third parties in an encrypted form. Third parties will
provide a Public key which will be used along with a symmetric key as part of the encryption process and
the Client, in possession of the corresponding Private key will be able to access the symmetric key and
then decrypt the message. Should they need to send back encrypted data they will use the symmetric
key that has been securely passed which the Counter will retain.
FiTE3 — Subscripting — Introduction of a new Call Script datatype that allows one ADC script to
call out to another script. The intention is to allow for common data capture routines (such as customer
name and address, or customer identification capture) to be created as a subscript, which can then be
called by any other ADC transaction. This will benefit POL by enforcing consistency across the various
ADC transactions, and also simplify change when required (e.g.if a new form of ID is made available,
such as Identity Cards, only the ID capture sub-script needs to be modified, rather than every ADC.
transaction that captures identification details).
FiTE4 - Data Persistence — Introduction of a facility to retain information captured as part of an
ADC transaction within memory, for the duration of the Customer Session. This will allow for subsequent
ADC transactions within the same Customer Session to make reference to previously captured data. For
example, if, as part of Transaction A, a customers name and address are captured, Subsequent
Transaction B may inherit previously entered data as a default value if similar information is required.
A.2.2 Scope of Testing
All of the enhancements described above will be delivered through enhancement of existing ADC
functionality or through development of new ADC functionality within the counter application, along with a
generic web service framework added within the datacentre. Therefore functional testing of each FiTE
component will be undertaken.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 45 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
FITE1 (Generic Web Services) will also require the following additional testing;
e Non-functional testing — of the new generic Web Service, considering deployment, operational,
audit, monitoring and supportability aspects. It is assumed that volume/performance testing is not
required at this stage.
e E2E testing — It is assumed that elements of FiTE 1 will be incorporated in the development of
the solution for CT920 (Pay Pal). If this is the case then FiTE 1 will be exercised end to end as
part of testing with Pay Pal. If not, then testing will be limited to internal emulation of third party
end points through the provision of a number of endpoint emulators.
FiTE2 (Encryption/Decryption) will require an element of non-functional testing to ensure correct
adherence to the agreed encryption protocol, and possibly some performance testing of
encryption/decryption time at the counter. There is also an implication to counter reboots, which may take
longer due to additional libraries being loaded and initialised after a reboot.
FITES originally detailed a requirement for Script Branching facilities - has been dropped from the
requirements and is considered out of scope for testing. FITE 5 now refers to the requirements for a
Security Assessment to be undertaken — this assessment is not appropriate to testing
A.2.3 Test Basis
The Post Office will specify their requirements for all of the IT enhancements as a series of ‘shall’
statements, each with an acceptance criteria and method. These requirements will be formally issued as
a requirements catalogue (see document references to REQ/CUS/CDE/1315 and REQ/CUS/CDE/1 183) .
Once baselined, the requirements will be imported into POL Doors, and transferred to Fujitsu via the
existing interface. In turn, the requirements will be made available within Quality Centre via the existing
Doors>QC integration software. All testable requirements will be associated to appropriate tests. Quality
Centre may also be used to track acceptance of non-testable requirements.
As the majority of the enhancements are to be delivered via the ADC datatype toolset, there will be no
updates to the Business Use Cases that describes counter functionality. The functionality and User
Interface impact of ADC datatypes is described in the AP-ADC Reference Manual (DES/GEN/MAN/002),
which will be updated as a result of the FiTE enhancement development.
Fujtsu will respond to the baselined requirements in the form of a series of design documents, which will
vary depending on the complexity of each component. This is summarised as follows;
Component High Level Design Statement Detailed Design Proposal
FITE1 — Generic Web Service Y — Due 14/01/11 Y — Due 28/02/11
FiTE2 - Encryption Y — Due 28/01/11 Y — Due 11/03/11
FITES - Subscripting Y — Due 14/01/11 Y — Due 14/02/11
FITE4 — Data Persistence Not Planned Y — Due 14/02/11
A.2.4 Dependencies
Detailed design proposals and updates to the AP-ADC Reference Manual must adequately describe the
FITE changes prior to testing. In order to test each component, the following additional deliverables are
required.
FITE1 — Generic Web Service
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 46 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A detailed description of the interface between the Generic Online datatype and the Generic Web service
must be available.
Ideally, a third party end-point should be available to connect to during testing. However, to limit the costs
of the introduction of the GWS framework capability, POL have chosen to defer third party end point
proving until the first live clients are ‘taken-on’ to the new GWS service. Testing of the first few live clients
will be enhanced to ensure the GWS offering functions correctly across the E2E service.
In the absence of third party end points being available, Fujitsu will develop an emulator to allow testing
of the Generic Web Service to be completed without the need for an external end-point. If this emulator is
provided to support development and early development testing, it is assumed that it will also be made
available and utilised by CIT, SV&l and LST test phases.
A detailed description of the interface between the Generic Web service and the emulated end points
must be available. This is assumed to take the form of a Service and Technical Specification for each
emulated service. This will include the Service Name and target URL as a minimum, but should also
specify heartbeat mechanisms and message timeout parameters.
FiTE2 — Data Encryption/Decryption
A definition of the agreed encryption protocol must be available - see REQ/CUS/CDE/1315.
POL must provide a means to verify the encryption performed at the counter — it has been suggested that
Cogent have a test tool that could be made available for this purpose.
FS must provide a tool to simulate the encryption of incoming online messages (using FIPS140-2
certified libraries) to the required standard, in order to verify decryption capabilities.
If the Cogent test tool is not available, Fujitsu must demonstrate that the message structure prior to
encryption, the message structure after encryption, and the post-decryption message structures are all
compliant with the standards laid out in the FiTE Encryption Requirements catalogue
(REQ/CUS/CDE/1315). This inspection will be dependant on the availability of the FiTE Solutions
Architect and Security Architect both of whom will need to be involved - see REQ/CUS/CDE/1183..
FiTE3 — Subscripting
POL/FS should agree the mechanism that defines and polices the creation of a library of pre-defined sub-
routines.
POL must provide relevant examples of suitable sub-routines, for example; a series of data capture
prompts to capture a Customer Name and Address.
FiTE4 — Data persistence
No specific dependencies.
A.2.5 Constraints
The solution to FiTE2 — Data Encryption implies that Customer Sensitive data will be encrypted. If
Customer Sensitive data includes Primary Account Number information, then this would need to be
tested to ensure it is compliant with PCI requirements. It is assumed that as the Customer Sensitive data
portion will be encrypted at source at the counter, the message will be suitably obfuscated to prevent
PAN replay if information is compromised. This is within the control of POL via APADC scripts.
Other components (such as FiTE1 — Generic Web Services) may make use of encryption facilities
provided by FiTE2, but will not have any PCI implications in their own right.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 47 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A.2.6 Limitations
The testing of the solution to FITE1 — Generic Web Services will be tested within the Horizon domain
only. i.e. testing will make use of internal emulators that act as the client end point for the Generic Web
Service. As a result, testing client connectivity methods (such as Secure Session (SSL) encryption) will
not be tested within SV&I or LST— no SSL capability is expected from the test emulator. It is understood
that Development will be using a copy of the DXI software on a test platform and the GWS emulator to
test the use of SSL
Client connectivity testing will form part of the Client Take On process as and when live services are
developed.
End to End testing on the SV&I test environment will not be carried out for each subsequent service that
makes use of the Generic facilities — this activity will be undertaken on the RDT environment as part of
BAU Client Take On (although it is noted that additional testing focus will be given to the first few clients
going live through the RDT phase) .
There is a similar limitation on FiTE-2 testing, in that there are no online encrypting services available in
RS, except in emulation by the Crypto Proxy. Client File encryption can only be tested by the Cogent tool
(if available).
A.2.7 Test Environment
The following components are ideally required to test all of the FiITE changes — however, as noted in the
limitation section above, third party emulators will not be available during R5 testing so scope of testing
will remain within the Fujitsu domain;
Client End End point routing dependant
= on client availability.
4
Service / Technical =
Specifications per End Point Dx!
GWS Emulator
_ . multiple Emulators
f
I Scope of 4
I Testing
I
I Internal BAL Ref Data
I AIS/TIS f Certificate
Counter
The following table maps FITE components against system Components, and describes if the system
component is to be modified as a result of the FiTE changes;
TE Counter B. Ref Data GwWs DXI Client End
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY Date’ 23-Jun-2011
STORED PageNo: 48 of 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system ~
FU] ITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Component Branch Acess Genere Web Server Data Exchange Point
1-Generic I No—although I Possibly - to No Yes — New No No
Web Service I new Service / I handle many Generic Web
Routing data I clients routed Service
in existing through one
Generic service
Online
datatype
2-Data Yes—new I No-however Yes — No No No
Encryption / Enerpyt I used to route I provision of
Decryption Decrypt Encrypt / certificates
Datatypes Decrypt
messages
& Certificates I and process
certificates
3 = Sub- Yes — new No No No No No
Scripting Call
Datatypes
4—Data Yes — new No No No No No
Persistence I logic to detect
persistent
data
Both SV&I and LST will have all of the above counter/data-centre components. Both environments will
also have the capability to connect to a third party end-point if required, however this is not expected
within the RS5 testing timeframe.
A.2.8 Test Approach
This section will detail the test approach to FiITE IT enhancements. It will consider;
« The specifics of each component
e = Test Planning
° Test Execution
e Test Reporting
« Test Governance
Although the later aspects (planning, execution, reporting and governance) are generic to the release as
a whole, they are briefly summarised here to allow this appendix to fully cover testing of the FiITE
changes and to be read in isolation.
SPECIFIC TEST CONSIDERATIONS OF EACH FiTE COMPONENT
The following sections detail the testing approach for each test FITE component;
FiTE1 — Generic Web Services
Ideally, the best approach to testing this change would be to utilise the new Generic Web Service
capability as part of a new client take on, i.e. have a real client test system as the end point, connect the
Generic Web Service to it, and undertake end-to-end testing with the client test system. At the time of
writing, there are some candidate clients that may be used for this testing (namely Paypal or Student
© Copyright Fujitsu Services
Limited & Post Office Limited 2011
FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
UNCONTROLLED IF PRINTED OR LOCALLY
STORED
Ref TSTIGEN/STG/0906
Version: 2.0
Date: 23-Jun-2011
Page No: 49 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Loans), however, the availability of these clients cannot be guaranteed, therefore an alternative approach
should be considered in the absence of a client system to test with.
If a client system is available, then a test plan will be developed to thoroughly test the counter transaction
and interface to client. The approach for this testing will be documented in the relevant section of this
document. Test Coverage against requirements relating to FiTE1 will be achieved through linking the
client based tests to the FiTE1 requirements alongside any requirements generated by the client-specific
project.
Some FiTE1 requirements relate to the operation of the Generic Web Service handling multiple web
services. So it is likely that testing will need to create other Generic Web Services, on top of any that are
planned to go live in R5 timescales, in order to test concurrent operation of multiple web services via the
Generic Web Service. If no ‘real’ web services are available within R5 testing timescales, then multiple
‘test’ web services will be needed to test concurrency. It is proposed that a ‘test’ web service will in fact
make use of an existing web service that currently uses a Defined Web Service interface, such as Kahala
(for Postal Guaranteed Delivery Date look-ups), PostcodeAnywhere (for Bank Sort Code validation) or
BT (for Broadband availability checking).
In order to utilise existing Defined Web Services, Client Take On support packs will be created to enable
the configuration of the Generic Web Services — i.e. define the Service Name, the target url (of the web
service endpoint), and the intended service-monitoring mechanism (i.e. heartbeat or other mechanism) .
New ADC transactions will be created, using GenericOnline calls to the Generic Web Service. The
Generic Web Service will be configured to direct appropriate calls to the target end-point. Test
transactions will then be undertaken to ensure that the Generic Web Service routes information to the
correct endpoint and converts responses into the correct ADC format. Tests will also be undertaken to
ensure the service-monitoring mechanism is operating correctly. A benefit of this approach is that the
Defined Web Service can be used as a comparison when verifying the operation of the Generic Web
Service. A potential drawback is that most of the current Defined Web Services offer only basic lookup
facilities (i.e. the counter sends up a telephone number and the web service responded with a maximum
potential line speed). Testing will therefore be limited to the types of interaction supported by the existing
defined services. There is unlikely to be any heavy-duty database interaction, but that should be
acceptable, considering that this is not the intended use of the Generic Web Service offering, It is not
considered practical to make use of APOP to simulate database interaction, as this would require
additional development to allow web service access to APOP.
An alternative approach to testing generic capabilities via existing Defined Web Services would be to
make use of an internally developed End-Point Emulator. If this is made available to testing, then it will be
utilised to validate ADC scripts, GWS configuration and web service operation within the Fujitsu domain.
A consequence of using an emulator, which connects directly to the Generic Web Service rather than via
the DXI, means that testing will not be truly representative, nor will testing of SSL capabilities be possible.
Consideration will be given to the message handling capabilities of the Generic Web Services, looking at
the maximum allowable data content to ensure that there are no constraints to perceived future use. For
example, if biometric data were to be sent to a third party via the Generic Web Service, it is likely that the
data packet size would exceed the 4K character limit imposed by AP, but not that of the GenericOnline
call to the Web Service.
INTERDEPENDANCIES WITH OTHER FiTE COMPONENTS
One specific area of consideration is the use of encrypted data over Generic Web Service - it is likely
that the transmission of customer sensitive data across the Generic Web Service will be required, so this
mechanism should be tested. It is assumed that no decryption is required at the Generic Web Service —
instead, encrypted data should pass through unaltered. Encryption over Generic Web Services will be
achieved through combination of Encrypt datatype and GenericOnline calls within an ADC transaction
script. Similarly, tests should be undertaken to ensure that encrypted data received from a Generic Web
Service client and be successfully decrypted using the standard Decrypt datatype.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 50 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
RIG COVERAGE
As the Defined Web Services are already available to all test rigs, it is assumed that all test streams (i.e.
CIT, SV&I and LST) could make use of this test approach, assuming availability of Generic Web Service
configuration information and appropriately modified ADC scripts. Although it may be possible to
introduce ‘test’ generic web services to LST (and Model Office for that fact), it is not desirable to introduce
test data into live (or live-like in the case of LST) streams. Therefore it is proposed that ‘test’ Generic Web
Services remain the focus of CIT and SV&l, and LST/Model Office only test FiTE1 as and when a live
Generic Web Service is enabled. Instead, LST will focus on the deployment of code to the platform that
will host the Generic Web Service, and will monitor operation of the Service in a benign state (i.e. the
functionality is available, but is not called in any ADC transaction scripts).
MIGRATION
It is assumed that no specific migration considerations need to be considered; Generic Web Services will
be enabled in live through the delivery of amended ADC transaction scripts and endpoint configuration
parameters. This will only happen once the Generic Web Service components have been installed on the
TCOM web server as part of the Release 5 data centre upgrade. All testing (CIT, SV&l and LST) will
follow this deployment path.
FiTE2 — Data Encryption/Decryption
The encryption/decryption mechanism to be delivered must be compliant with the Post Office defined
protocol. This is the same standard used by Cogent in the AEI (Application, Enrolment and Identity)
programme, and is currently being used in the transmission of customer sensitive data between the
Biometric capture kiosks in Post Offices to the UK Borders Agency and DVLA. By utilising the same
encryption standard, future integration between Horizon Online and the AEI system may be possible. As
well as these strategic benefits, some opportunities in the testing of encryption may be realised. Cogent
have developed a tool that can be used to decrypt messages (assuming the required encryption key
information is provided). POL will secure use of this tool to ensure that the messages encrypted within
the Fujitsu domain are compatible with other sources. Availability of this tool is a key dependency on the
testing that can be completed. Due to Intellectual Property Rights concerns, it may be that the tool
remains within POL control.
Fujitsu are expected to develop a similar tool to allow the encryption and decryption of messages within
the Horizon domain.
INTERDEPENDANCIES WITH OTHER FiITE COMPONENTS
As stated above, encryption/decryption capabilities should be tested across the Generic Web Service,
against an emulated end-point. Testing against a real client will occur during Client Take On for the first
client utilising GWS and encryption capabilities.
RIG COVERAGE
CIT and SV&I testing will focus on encryption/decryption capabilities of the new ADC datatypes, using
test transactions developed by the CIT/SV&I team. If required, POL will provide outputs from the Cogent
tool to the CIT/SV&I team, who will validate that messages leaving the Fujitsu data-centre domain are
compliant with the Post Office specified standard.
LST only receive live reference data, and therefore will not be in a position to test encryption capabilities
until such time that live reference data is available that calls the encryption facility. LST will however,
ensure that any changes required to the Counter cryptographic libraries do not affect operational service.
MIGRATION
It is assumed that encryption capabilities will be introduced by the combination of;
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 51 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
a) installing a new CSP (cryptographic libraries) on the Counter — no datacentre changes are
required to support encryption/decryption operations
b) changes to the ADC modules held within the counter code to make new encryption/decryption
capabilities available. This change will be applied as part of the R5 counter delivery.
c) changes to ADC transaction script data to call new encryption/decryption datatypes. This change
will be controlled by the POL reference data team, and will only occur once the data-centre
migration to R5 functionality has been confirmed.
It is assumed that existing script ‘downgrade’ functionality will utilised to ensure that encryption-enabled
ADC scripts are handled gracefully in the scenario where reference data is made available at outlets yet
to receive the encryption/decryption code modules delivered in the R5 counter.
CIT/SV&I Testing will ensure that the appropriate downgrade rules are in place to ensure that encryption
capabilities cannot be invoked until such time that all counter components are in place.
FiTE3 — Subscripting and FiTE4 — Data Persistence
APPROACH
The remaining 2 FiTE changes are expected to be delivered via amendments to the ADC code modules
held within the counter code, and will be called by appropriately modified ADC reference data scripts.
There are no data-centre or third-party components, therefore the test approach for these changes is
considered compatible.
One of the key requirements of the FiTE IT enhancements is that ADC script changes are only made on
a fix-going-forward approach -— i.e. existing ADC scripts will not be modified to make use of Subscripting,
or Data Persistence capabilities. This is to prevent additional change to existing interface specifications,
and negate the need for extensive regression testing. With this approach in mind, it is expected that
Subscripting or Data Persistence capabilities will only be called by new ADC transactions, generated post
R5 data-centre migration and counter roll-out. Therefore it is unlikely that there will be any live ADC
scripts available for use in testing. In order to test functionality a set of ADC scripts will be developed by
the test team for test purposes only.
INTERDEPENDANCIES WITH OTHER FiTE COMPONENTS
It is assumed that test ADC scripts will prove each component in isolation, however scripts should be
created to ensure that each component interacts correctly with each other. For example, a Cail script will
be created to simulate a ‘Customer Address’ sub-routine to capture several data items as an address.
Checks will also be undertaken to ensure that data captured in a sub-routine are available to subsequent
scripts as part of the Data Persistence change. Alternatively, we may write a script to update persisting
data items can be called rather than including in each script. It is expected that a matrix of
interdependencies will be developed as part of the HLTP analysis.
RIG COVERAGE
CIT and SV&I testing will focus on the functional aspects of each new counter component, using test
ADC scripts developed by the test team.
LST will ensure that the introduction of the new counter components remain benign and do not affect the
operation of the counter. It is not expected that live ADC transactions will be deployed to LST within the
R& testing window.
MIGRATION
As described in the encryption section above, it is assumed that existing script ‘downgrade’ functionality
will be utilised to ensure that Caj/l, and Data-Persistence ADC scripts are handled gracefully in the
scenario where reference data is made available at outlets yet to receive the appropriate ADC code
modules delivered in the R5 counter.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 52 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
CIT/SV&I Testing will ensure that the appropriate downgrade rules are in place to ensure that the
remaining FiTE ADC functions cannot be invoked until such time that all counter components are in
place.
TEST PLANNING
A formal test plan will be developed by members of the Joint Test Team covering the testing of each FITE
component within the SV&I test phase. It is assumed that testing during CIT and LST phases will be more
dynamic and reactionary, therefore testing documentation will be less formal. The SV&I High Level Test
Plan (TST/SOT/HTP/1051) will detail functional, non-functional and E2E tests (where applicable). The
test plan will be issued under formal change control, with a view to having an approved version available
prior to the commencement of test execution.
The Test Plan will contain a mapping to source requirements, reflecting the mapping held between tests
stored in Quality Centre and requirements held in DOORs.
TEST EXECUTION
The Release 5 deployment model has not yet been confirmed; however it is likely that Release 5 will
follow a model frequently adopted post Release 1. This means that code is likely to be delivered in a
series of incremental ‘drops’, determined by the progression of each projects requirements, design and
development. Each delivered drop will undergo approximately one week of CIT, where individual
development streams’ code baselines will be merged and installed on target infrastructure for the first
time. Tests will be undertaken to ensure the code integrates successfully, and that the counter is stable
enough to warrant further exposure on SV&I. Results of this testing will be captured in a CIT Wiki page,
detailing test outcomes, known limitations and defects identified. Note: early GWS framework deliveries
are not expected to pass through CIT.
Prior to commencing SV&I test execution, a readiness review will be held to ensure that development is
as expected, that CIT has been completed and that the SV&I test environment is in a suitable state. If the
readiness review indicates that it is appropriate to do so, SV&I test execution will then commence. SV&I
test execution is cyclic in nature, normally with a 2 or 3 week test cycle planned to follow each delivery
drop/CIT test. Each FiITE component will be scheduled for at least two cycles of SV&I testing, and may be
subject to more if defects are identified. Due to development continuing in parallel once testing
commences, defects identified during cycle 1 will not be formally delivered until cycle 3 at the earliest.
Therefore the scope of each test cycle will fairly fluid, depending on the coverage already achieved and
the fixes expected for a given change. SV&I will not plan to undertake full Test Plan coverage every
cycle. The SV&lI test plan will be split by component, so if Generic Web Services are to be tested via
another change within the release, the test plan will document planned testing across each component.
Once all component ‘drops’ of Release 5 have undergone sufficient testing within SV&l, a readiness
review will be held to determine suitability to start LST. If the review determines that it is right to progress,
then all component drops will be merged to form a single ‘release’. This will be delivered to LST through
formal routes, i.e. deployment plans developed by the live support teams. This release will undergo a
cycle of LST testing (normally 2-3 weeks) before the LST team will provide formal sign-off to commence
live implementation, normally commencing with roll-out to the Model Office.
It should be noted that ‘releases’ are often split into component chunks — i.e a release to the central
systems housed in the data-centre, and a release changes to be applied to the counter. If this is the case
for Release 5, then a data-centre release and a counter release will be subject to SV&l and LST sign-off.
TEST REPORTING
CIT Testing will be documented in the form of a Wiki page, which will be extracted as a PDF document at
regular intervals (normally to coincide with the test readiness review of each drop of code.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 53 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Planned SV&I Testing will be documented in the Release 5 High Level Test Plan (TST/SOT/HTP/1051).
Upon completion of testing, test results, requirements coverage and residual defects will be documented
in the Release 5 Test Report (reference TBC). Multiple iterations of the test report can be expected if the
release is split into data-centre and counter components.
LST testing outputs will be provided in the form of formal sign-off of release notes.
TEST GOVERNANCE
Handover between test stages will be in the form of regular Test Readiness Reviews, attended by
empowered representatives from testing, implementation, development, design and project management.
SV&I and LST testing will be monitored during test analysis and execution, with management review
meetings and test metric reporting provided on a regular basis each week.
If required, defect review meetings will be held between test, development and design to discuss issues
and agree resolution priorities.
A test governance meeting will be held on a weekly basis to review testing progress across the test
strands and cycles, and inform higher management within POL and Fujitsu. Outputs of the Governance
meeting will form the basis of the weekly test highlight report and will also feature in the Release Board
presentation which is reviewed on a fortnightly basis.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 54 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A.3 CT910: Repositioning of Partner Bank Withdrawal
Prompt
This change moved the ‘Ask the customer if they want cash today’ prompt that appears during
debit\credit card payments for cards that are also supported for online banking. On the original Horizon
system, this prompt appeared early enough in the transaction flow to allow the clerks to prompt the
customer accordingly, but on HNG, the prompt moved to later in the transaction flow, which meant the
card had been removed from the PIN Pad, and put away by the customer. Banking transactions for
Partner Banks declined significantly as a result.
This change followed the standard approach as in CIT>SV&l >LST and is being deployed as a
maintenance release between R4 and R85 and is covered in detail in TST/SYT/HTP/1349.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 55 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A.4 CT908/922: Test Tool Automation
A.4.1 Background
The Joint Test Team developed a Test Automation capability during the development of Release 1. This
capability was based upon the Winrunner tool, and allowed for automation or semi-automation of a
variety of counter-based activities.
Ongoing support for Win-runner, licence costs and limitations with the capability of the automation tool
led to POL launching an initiative to improve automation capabilities. Investigation discovered that the
Development team had a Java-based tool which could potentially be extended to support functional
testing on the SV&I environment. A Proof of Concept exercise successfully demonstrated a version of the
development tool running on the target environment. Accordingly, POL raised a Business Case to fund
development of the existing tool to support the automation requirements
A.4.2 Scope
The Automation Test Tool will be developed for use on the SV&I test environment only, although future
use on other test environments should not be precluded.
Although not Release-dependant, the Automation Test Tool will be formally issued in alignment with the
baseline Release 5. Functionality available in the Release 5 baseline will be supported by the first formal
release of the Automation tool.
A.4.3 Test Basis.
The Post Office has specified a set of requirements for the Automated Test Tool. These specify the
changes in a set of ‘shall’ statements. These statements will for the basis for design, development,
testing and acceptance.
Fujitsu Services have produced a Design Proposal [DES/GEN/DPR/1047] documenting the design of the
automation test tool. The Design Proposal is cross-referenced to the requirements catalogue.
Fujitsu Services will also produce a Support Guide [DEV/APP/SPG/1208] which defines the functions
supported by the tool.
A.4.4 Test Approach
Rather than develop test cases linked to requirements for the automation tool, the approach to
acceptance of the test tool is slightly different. The important consideration is that the tool supports a
defined set of tests that need to be in place to allow automated regression testing to transition to the new
tool. A candidate set of 415 tests have been identified as the core automated regression test pack. These
tests must be developed and proven against the automation test tool as part of Release 5 testing.
Requirement acceptance will be largely based on completion of these tests, along with documentary
evidence supported by the Design Proposal and Support Guide. To monitor progress and allow
incremental acceptance, the following staged milestones were agreed.
Milestone Planned
Start
Working counter on an SSC Workstation 31/01/11
Infrastructure in place for script repository 31/01/11
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 56 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
oe
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Milestone Planned
Start
Final version of tool delivered 28/02/11
10% of the candidate Regression test converted. 25/02/11
30% of the candidate Regression test converted. 31/03/11
Documentation for the script repository available 07/03/11
60% of the candidate Regression test converted. 25/04/11
90% of the candidate Regression test converted. 31/05/11
Automated Test Work Instructions available 20/05/11
100% of the candidate Regression test converted. 30/06/11
Regular review meetings will be held to assess coverage and update requirements status accordingly.
A.4.5 Test Environment
The automation tool is deployed alongside the Counter Business Application. During test execution, the
automation tool will run on the standard-build black counters on the SV&l environment within in the main
hall. Minor network modifications will be required to allow the automation tool to access the Branch
database directly when installed on a standard counter.
To support script development, the CBA and automation tool will be installed onto standard-build SSC
workstations, which are deployed on testers desk. Minor network modifications will be required to allow
the automation tool to access the Branch database directly when installed on the SSC workstation..
To allow the PIN Pad emulation characteristics with the automation tool, a minor change to the banking
agent deployed on banking platforms NAA, NAL, NAC needs to be deployed.
A repository file-store has been established on the Delivery Server to store completed tests. Automation
scripts will be copied to automation counters to enable independent test execution.
A.4.6 Limitations
The automation tool will be deployed to the SV&I test environment. Further change requests may be
made to extend the use of the tool to CIT and LST.
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 57 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A.5 CT776: Two Factor Authentication for TESQA
Now deemed out of scope for Release 5 — if CP is agreed it is likely that this will be delivered as a
release-independent change, subject to exposure in the standard CIT>SV&I>LST test cycle (where
appropriate).
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 58 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A.6 CT951: PAF Replacement
A.6.1 Background
Post Code lookup and address validation to the Horizon Online counter application is currently provided
by the QAS Name Tracer tool. The contract for this tool expires shortly, therefore the Post Office have
requested that Fujitsu develop a replacement solution that offers a reduction in licencing costs and
improves the quality of address matching.
A.6.2 Scope
The PAF Replacement Service will be implemented as a new service within the Fujitsu data centre. This
may reside on the existing PAF Web Service (PWS) or may be located on the Branch Database — detail
to be determined in the Design Proposal. Whichever method is uses, new processes are expected in
order to handle the weekly update from Royal Mail
No counter changes are expected to support the replacement PAF service.
The following diagram depicts all of the systems and interfaces potentially impacted by this change.
Royal Mail POL
Existing Quarterly CD Updates Monthly CD Updates
Adhoc CD Updates
BRDB
__f
Existing PAF lookups New PAF lookups
BAL
t PAF lookups
Counter
To describe the above diagram, in terms of components and interfaces;
Component Function scription of changes
Counter I Point Of Sale Capture Post Codes and / or Address information to
I
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 59 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Component ription of changes
pass to Postal Address File lookup.
e Return validated Postal Addresses and present
options for selection
BAL Branch Access Layer « Pass through PAF requests to existing QAS running
— service routing on PWS.
e Pass through PAF requests to replacement
architecture (assumed to be BRDB)
PWS PAF Web Service e Receive PAF requests and respond with validated
Postal Address options
BRDB Branch Database — e Receive PAF requests and respond with validated
data/transaction Postal Address options
repository
Royal Mail Updates to Postal e Monthly CD to refresh Postal Address File
Address File
Post Office Updates to Postal e Adhoc updates to Postal Address File.
Limited Address File
A.6.3 Test Basis.
Requirements are defined in QAS Interface Replacement for PAF Service Description Summary
QAS/REQ/001. Requirements will be imported into POL DOORS and transferred to Fujitsu DOORS and
Quality Centre.
The solution design will be specified in PAF Replacement Service Design Proposal DES/APP/DPR/1312.
A.6.4 Test Approach
The standard approach of CIT>SV&I>LST testing is deemed appropriate for this change. Looking at each
stage individually;
CIT Testing
CIT will focus on the integration of the main components (i.e. counter, BAL and BRDB/PWS). Testing will
aim to ensure responses from the Replacement PAF Web Service are returned, but will not validate
specifically that the correct responses are returned. No specific consideration will be given to the
transition of the service from the existing QAS service to the replacement — it is assumed that the
replacement service will be deployed with data available. No testing of update mechanisms will be tested
during CIT.
Tests for this phase will be documented on the CIT Wiki page. Where possible, tests from SV&I will be
run on CIT to pre-prove them prior to commencing SV&I
SV&l Testing
Once CIT testing has demonstrated that the main components hang together, SV&I testing will then
commence, covering functional and non-functional aspects of the replacement service.
FUNCTIONAL TESTING.
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED Page No: 60 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Tests will be undertaken to ensure that the replacement service provides a broadly equivalent service. It
is acknowledged that the replacement service will not provide a ‘like for like’ match to the existing QAS
service, therefore the following approach will be adopted;
« Post Code Lookups - It is assumed that the replacement service may provide more results, or
match more Post Codes than the current QAS software. In these instances, testing will look to
validate that the correct address is available, and that the clerks’ interaction with the system is
not adversely impacted. Any discrepancies between the existing and replacement service will not
necessarily be deemed faults with the replacement service; instead they will be logged and
categorised against a pre-defined set of ‘difference classifications’. These ‘difference
classifications’ will have been presented to representatives from within the PAF replacement
service project. If a discrepancy fits within a known ‘difference classification’ it will be noted
accordingly. If the discrepancy is outside of the known ‘difference classifications’ it will be logged
and reviewed with the PAF replacement project — if deemed unacceptable, a defect will be
raised.
« Address Matching - The existing QAS service provides ‘sounds like’ matching logic to return
responses, e.g. ‘Morpath’ would return results for ‘Morpeth’. This functionality is unlikely to be
replicated exactly by the replacement service; therefore it is likely that address matching will
return different results. Again, these differences will not automatically be assumed as defects, but
will be assessed against a set of known ‘difference classifications’. If the differences are deemed
significant, a briefing to clerks may be drafted to notifying them of potential differences in
advance.
The ‘difference classifications’ mentioned above will be defined and agreed by POL project stakeholders
prior to testing commencing.
A broad selection of English and Welsh addresses will be processed through both PAF lookup
mechanisms, with results being compared and discrepancies being reviewed.
The solution will also be tested at least once through the transition period from existing QAS to the
replacement service.
Lookups will be attempted following an update of the PAF file from both Royal Mail and Post Office
Limited. Reversion to a previous update will also be tested — i.e. if the update process fails, the system
should regress back to a previous valid version of PAF data.
Tests for this phase will be documented in R5 SV&/ HLTP (DES/TST/SOT/HTP/1051) and will be
reported in the R5 SV&/ Test Report (TBS).
NON-FUNCTIONAL TESTING.
The following areas will be considered;
« Deployment of the replacement service — installation and activation of the new service.
e Migration — cut-over from the existing service to the new service.
e Supportability of the replacement service — i.e. logging and reporting of events generated during
service operation.
e Ongoing maintenance — inclusion of the replacement service in scheduled housekeeping and
PAF updates.
e Decommissioning of the existing service — this will occur at a later date, under a separate change
proposal. However, address data will be removed from the PAF Webserver in order to appease
QAS that no ongoing licence infringements will occur.
LST Testing
FUJITSU RESTRICTED (COMMERCIAL IN Ref: TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 61 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
LST will build upon the non-functional aspects of SV&I testing, and will aim to prove that the replacement
service can be deployed, supported and updated in a live-like environment.
LST will undertake a limited sample of PAF lookups, comparing results against those returned in
functional testing on SV&l.
Modifications to the PAF service reporting will be tested in LST. The new service will not report the stats
to the data warehouse as the current PAF service. The stats will be available via the data collected by the
BRSS tables - this new functionality is being introduced by CP639 - Capacity Management Reporting
A.6.5 Test Environment
The CIT, SV&I and LST test environments will receive the PAF replacement service components as part
of the R5 data-centre upgrade — this is assumed to be via delivery of new OSR/BAL and BRDB
components
It is assumed that each test environment has sufficient capacity to handle the increased capacity required
on the BRDB.
It is assumed that there is no counter component for this change.
A.6.6 Dependencies
CD's containing updates from Royal Mail and similar updates from Post Office Limited must be made
available for loading onto the test environment, prior to test execution.
It is assumed that there is sufficient environmental space available on the BRDB to support the
replacement service data tables.
The design team have provided a spreadsheet which details a days worth of live transactions, the
responses that PAF returned, along with the anticipated response from the replacement service. This
spreadsheet should be made available to testing to act as an expected result when validating Horizon
outputs.
A.6.8 Limitations
No volume testing will be undertaken on CIT, SV&l or LST — assurance of performance characteristics
will be obtained from a load generation exercise undertaken within a development environment
SV&I do not have a Standby Branch Database, so will be unable to fully test Service Reporting — this will
be undertaken on LST.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 62 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
A.7 CT979: APOP Enhancements
A.7.1 Background
The APOP service has been operational for a number of years, and support numerous voucher-based
services such as Postal Orders and OutPay. As more services and higher transaction volumes have
made use of the APOP service, concerns were raised that the system was frequently operating above
design limits and was vulnerable in the following areas;
e Capacity limits are being breached in the overnight batch processing, whereby the volume of
data being extracted for customer reports is significantly higher than the design limit.
« Processing of client data into the system is constrained and large campaigns can be required to
be split over a number of days processing.
e The current simple query/response model of APOP limits the capability of the service as this
does not support RAC (request/acknowledge/confirm) banking style financial transactions.
« As data is archived all traces of the records are removed from the main database. This is
causing issues when follow-up enquiries/encashments occur, the volume of which cannot be
supported by the archive retrieval process.
e There is no system failover solution in place. Should the specific node through which APOP
transactions are routed fail, then all APOP services would go down until the node could be
restored.
This change aims to address these vulnerabilities to provide a resilient service capable of supporting the
Post Office's current and future needs.
A.7.2 Scope
Batch Processing issues will be resolved by retiming and reprioritising TWS schedules.
RAC transaction processing will be provided through development to the existing Near Real Time
APADC message processing — new processes will run to collect NRT messages from the counter and
post them to the APOP database.
The APOP database will be modified to retain history records rather than archive as current.
Additional NPS nodes will be provided to provide resilient routing.
A.7.3 Test Basis.
A set of requirements have been identified to support this change; these will be loaded into Doors and
Quality Centre and will form the basis of acceptance and testing.
Asolution Design Proposal is available in DES/APP/DPR/xxxx.
A.7.4 Test Approach
Testing of the APOP enhancements will vary across the test environments.
CIT TESTING
Testing is constrained by the scope of the CIT environment.
There is no TWS schedule in operation, therefore batch processing changes cannot be tested.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 63 of 65
POL00039120
POL00039120
Approach to Testing for the Horizon Online™ system ~
Fe)
FUJITSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
Amendments to ADC scripts to introduce NRT confirmations can be proven, however the process that
collects these NRT confirmations and posts them to APOP cannot be tested.
Enhanced record retention cannot be tested due to lack of APOP database availability.
Resilience cannot be tested due to lack of NPS nodes.
SV&l TESTING
Changes to TWS schedules will be monitored with regression testing of existing APOP services being
undertaken to ensure no adverse impact to operations.
Near Real Time Confirmation functionality will be tested on SV&I. POL will provide amendments to the
Postal Order Sale and Recovery ADC scripts, to one Payout script and possibly a new RAC-compliant
APOP service such as Christmas Club or Business Mails. NRT Confirmation messages from counter to
APOP will be tested, and POL testers / P&BA staff will validate the impact of the interim-authorised
transaction via the APOP admin service. Note: this testing will prove the confirmation process but not
necessarily prove the change to each service to the point that the changes can be made in live — this is
considered to be a comprehensive change which would require project management and coordinated
deployment, possibly within the BAU environment.
To test the archiving change, metadata will be amended to test this against the Postal Order Service.
The voucher archiving parameter will be amended this change can be tested within SV&I timescales.
Counter transactions will be generated to enable the demonstration of retained history records to POL
P&BA staff via the remote link to APOP.. The result of this test will enable a decision to make the
required meta-data changes to the live Postal Order Service.
Resilience will not be tested on SV&I due to environmental constraints — only one NPS node exists.
LST TESTING
LST will test:
(1) Failover from NPS1 to NPS2 whilst APOP counter transactions and APOP TESQA sessions are in
progress. We should choose an APOP update transaction which inserts and then updates the same
voucher in the same counter session and failover between the insert and update. Ideally failover whilst a
batch input file is being processed should also be tested but we can't do that in our environment as it
would require breakpoints to be put in the code. It is assumed that development testing will cover this
aspect.
(2) APOP regression test whilst APOP is running on NPS2. This should include performing all APOP
counter transactions, exercising all APOP TESQA functionality, and all APOP batch processing including
file input, report production, and backup, archiving & housekeeping processes. This would have to
include the weekend batch schedule as there are Saturday and Sunday only APOP schedules. We
should also consider restoring and rolling forward an APOP database dump taken prior to failover.
(3) Failback from NPS2 to NPS1. Unlike failover, failback will be done in a controlled manner so it is
probably not necessary to be performing transactions and using the TESQA server whilst it’s being done.
(4) APOP regression test whilst APOP is back on NPS1. This should repeat all the testing done in (2)
above.
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 64 of 65
POL00039120
POL00039120
oO Approach to Testing for the Horizon Online™ system
FUII TSU FUJITSU RESTRICTED (COMMERCIAL IN
CONFIDENCE)
FUJITSU RESTRICTED (COMMERCIAL IN Ref TST/GEN/STG/0906
© Copyright Fujitsu Services CONFIDENCE) Version: 2.0
Limited & Post Office Limited 2011 UNCONTROLLED IF PRINTED OR LOCALLY _ Date: 23-Jun-2011
STORED PageNo: 65 of 65