FUJ00232714
FUJ00232714
oO TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
Document Title: TESTING & QA REPORT
Document Reference: COM/MGT/REP/4166
CP/CWO Reference: NIA
Abstract: Explanation of how Testing & QA is performed for Horizon
Applications.
Document Status: APPROVED
Author & Dept: Fujitsu
External Distribution: Restricted. See section titled Information Distribution.
Information See section 0.8
Classification:
Approval Authorities:
Name
Fujitsu Horizon Audit Team (POA) See Dimensions for record
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
Version 1.0
UNCONTROLLED WHEN PRINTEDOR _Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 1 of 32
FUJ00232714
FUJ00232714
(oe) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
Table of Contents
0 DOCUMENT CONTROL. .....c.scscscsesscsssssssesssesssssssseseseesensarsrscenssserscsseseneneeeacaeeeesees 4
0.1. Document History. 4
0.2 Review Details... 4
0.3. Associated Documents (Internal & External, 4
0.4 Abbreviation: 4
0.5 Glossary... 5
0.6 Changes Expecte: 6
0.7. Accuracy...... 6
0.8 Information Classification... 6
1 EXECUTIVE SUMMARY........cccecscsssesesscssssssssesesessssssesessesenssrersceeeseeteceseesseseseasasens 7
2 PURPOSE & SCOPE..........scssssssssssssessssssssssesssnsssscasscsescseseneecsessensnsesecaseeseeseaseesenee 8
3 BACKGROUND & INTRODUCTION.........ccccesesssssessssseseseensssstseneteensstetesenentseasacees 8
4 TESTING & QA... csessssssssesssesessesenssssesessesssssssssseesescasasssarsstacarsceetsensaneressseenensnenee 9
44 Testing Overview & Approacl
4.2 Defining the Test Approach
4.2.1 Test Planning......
4.2.2 Test Preparation
4.3. Utilisation of environments ('
4.4 Major & Maintenance Release Flo
4.41 Test automation coverage.
4.5 HNG Test Environments...
4.5.1 Component Integration Test (CIT) environment
4.5.2 Integration (INT) environment
4.5.3. SV&I environment.
4.5.4 LST environment
4.5.5. RDT environmen
4.6 Platform differences.
4.7. Transaction types.
4.8 Model Office — Liv
NILCMBAGARRORASOO
5 FORMAL AUDIT REPORTS.........csssssssssssssseseseeeesessseseecsesesesceseseneneeeesaseeaeneas 22
6 —- CONCLUSIONG.........ssssssssessesesserssseesssecnsensscessesssnsatsuestsuceresceeasensasensesessneeseneanter 22
7 RECOMMENDATIONS.
8 INFORMATION DISTRIBUTION.........:c:scscsssssssessssssesesseessestecseeeesteceeessssesesesasaseees 23
APPENDIX A — EXAMPLE TEST READINESS REVIEW MAIL.........ssesssssssesssesesenees 24
APPENDIX B — EXTERNAL ENDPOINTS
APPENDIX C — ENVIRONMENT PLATFORMG..........cssecsssssssssssessesessesstseesseeeseeseaeenseee 29
APPENDIX D — MASTER RECOMMENDATIONS LIST.........sscsssesessssesseeesseeseneeeetees 31
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS
Page No: 2 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
0 Document Control
0.1 Document History
Version No. Date Summary of Changes and Reason for Issue Associated Change
CWO, CP, CCN or
Peak Reference
1.0 29/01/2021 Approved for release N/A
0.2 Review Details
Mandatory Review
Role Name
Horizon Audit Team Fujitsu
0.3 Associated Documents (Internal & External)
Reference Version Date Title Source
COM/MGTIREP/4161 I Latest Latest Expanded Table of Contents forthe I Dimensions
Testing & QA Report
COM/MGTI/REP/4168 I Latest Latest SDLC Report Dimensions
0.4 Abbreviations
APADC Automated Payment - Advanced Data Capture
APOP Automated Payment Out-Pay
BAL Branch Access Layer
BAU Business As Usual
BCMS Branch Change Management System
BEX Belfast EXit
BMX Branch Access Monitor via JMX
BRAO1 Bracknell Fujitsu Office
BRDB Branch Database
BRSS Branch Database Standby Server
CBA Counter Business Application
cDP Common Digital Platform
cit Component Integration Test Environment
CTR Counter
cTT Counter Transaction Type
cws Collections Web Service
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 3 of 32
re)
FUJITSU
TESTING & QA REPORT
FUJITSU CONFIDENTIAL
FUJ00232714
FUJ00232714
DEV Development
DR Disaster Recovery
E2E End to End
EBMS Europe Business Management System
EPOSS Electronic Point Of Sale System
FSC Forward Schedule of Change
GDC Global Delivery Centre
GWS Generic Web Services
HBS Horizon Business Server
HIH Horizon Integration Hub
INT Integration Test Environment
LST Live System Test Environment
MDM Master Data Management
NCR Suppliers of Self Service Kiosk hardware and associated application
PODG Post Office Data Gateway
POL Post Office Limited
POLSAP A POL financial system
RAID Risks, Assumptions, Issues, Dependencies
RDDIV Reference Data Delivery Interim Verification
RDDOV Reference Data Delivery Outlook Systems Group Verification
RDDPL Reference Data Delivery Pseudo LIVE
RDDT Reference Data Delivery Terminal Verification
RDT Reference Data Test Environment
RPOS Remote Point of Sale
SDLC Software Delivery Life Cycle
SME Subject Matter Expert
SOAPUI Test tool
SSK Self Service Kiosk
SV&l Solution Validation and Integration test environment
T&V Test and Validation organisation within Fujitsu
TRR Test Readiness Review
TwWsS Tivoli Workload Scheduler
UAT User Acceptance Testing
0.5 Glossary
BEX Project Project to migrate all systems out of the Fujitsu Belfast data centres
HORIce A tool used to interrogate the Branch Database for certain reports
Hot Fixes Software patches
JIRA Software delivery and project management software platform
© Copyright Fujitsu 2021
FUJITSU CONFIDENTIAL, Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS PageNo: 4 of 32
FUJ00232714
FUJ00232714
o TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
REFDATA Reference Data used by the Horizon application
Service Improvement Activity to make an improvement to how the service is delivered
0.6 Changes Expected
en
0.7 Accuracy
Fujitsu endeavours to ensure that the information contained in this report is accurate but, while every effort is made
to ensure the accuracy of such information, it accepts no liability for any loss (however caused) sustained as a
result of any error or omission herein.
0.8 Information Classification
The author has assessed the information in this document for risk of disclosure and has assigned an information
classification of FUJITSU CONFIDENTIAL. This report is also subject to the Information Distribution statements
in Section 8.
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 5 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
1. Executive Summary
The purpose and scope of this report is to explain the current Fujitsu application Testing & QA approach
This Testing & QA Report explains the current processes, procedures and controls that Fujitsu operate to
manage the testing of Horizon and to support Horizon’s systems and applications. The report explains
the different environments and shows how each environment supports the end-to-end Software Delivery
Life Cycle.
On 20 August 2020, POL requested an audit of Horizon by sending a letter to Fujitsu titled “Horizon
Audit’. Following a number of discussions between POL and Fujitsu, it was agreed by POL that Fujitsu
would prepare a set of reports on key topic areas identified by POL.
The spirit of the discussions between POL and Fujitsu was to share content that would allow both
organisations to confirm the efficiency of the current ways of working together, and to identify ways to
make meaningful improvements. Fujitsu believe in collaboration and welcome constructive suggestions
from POL.
This report explains the current Fujitsu Testing & QA approach (“Testing & QA Report’). It follows the
“Expanded Table of Contents for the Testing & QA Report” (COM/MGT/REP/4161) which was shared
with POL on 01 December 2020.
This Testing & QA Report covers the Test activities and complements the “SDLC Report”
(COM/MGT/REP/4168), which covers the lifecycle from Requirements to Release.
The current Fujitsu Testing & QA process, with the associated test rigs, used as part of the full SDLC,
provides a high quality test service for the validation of changes for a safe deployment into LIVE.
Testing & QA and been the topic of numerous collaborative discussions between POL and Fujitsu over
the years and during recent meetings. Fujitsu and POL recognise that there are constraints imposed by
using the current approach and that there is a need for both POL and Fujitsu to focus on improvements.
Accompilation of recommendations for improvement arising out of these meetings has been collated into
a single list (set out at Appendix D to this report) so that POL and Fujitsu can work jointly to improve the
Testing & QA capabilities. The recommendations should be prioritised and the most relevant ones
actioned as promptly as possible.
Fujitsu has endeavoured to ensure that the content of this report is correct as at the date of issue. Fujitsu
reserve the right to make changes to the way we work in the ordinary course of its operations and
business without obligation to update this report. POL should verify the position with Fujitsu before
relying upon any information or content from this report, as well as bearing in mind the requirements set
out in “Information Distribution” at Section 8 below.
The author has assessed the information in this report for risk of disclosure and has assigned an
information classification of FUJITSU CONFIDENTIAL. This report is also subject to further Information
Distribution statements at Section 8 in this report.
POL is invited to comment on this report to seek any additional clarifications it needs. Fujitsu will
endeavour to respond to any comments or clarifications requested and may, if it deems necessary,
provide an updated version of this report.
Fujitsu welcome the opportunity to provide this report.
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS
Page No: 6 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
2 Purpose & Scope
The purpose and scope of this report is to explain the current Fujitsu application Testing & QA approach
This Testing & QA Report explains the current processes, procedures and controls that Fujitsu operate to
manage the testing of Horizon and to support Horizon’s systems and applications, consistent with
Fujitsu's obligations set out in the Horizon Contract. The report explains the different environments and
shows how each environment supports the end-to-end Software Delivery Life Cycle.
POL is invited to comment on this report to seek any additional clarifications it needs. Fujitsu will
endeavour to respond to any comments or clarifications requested and may, if it deems necessary,
provide an updated version of this report.
Fujitsu welcome the opportunity to provide this report and looks forward to a constructive dialogue with
POL.
3 Background & Introduction
On 20 August 2020, POL requested an audit of Horizon by sending a letter to Fujitsu titled “Horizon
Audit’. Following a number of discussions between POL and Fujitsu, it was agreed by POL that Fujitsu
would prepare a set of reports on key topic areas identified by POL.
The spirit of the discussions between POL and Fujitsu was to share content that would allow both
organisations to confirm the efficiency of the current ways of working together, and to identify ways to
make meaningful improvements that would enhance the working relationships and hopefully ensure a
better experience for the POL branches and their postmasters. Fujitsu believe in collaboration and
welcome constructive suggestions from POL.
This report explains the current Fujitsu Testing & QA approach (“Testing & QA Report’). It follows the
“Expanded Table of Contents for the Testing & QA Report” (COM/MGT/REP/4161) which was shared
with POL on 01 December 2020.
This Testing & QA Report covers the Test activities and complements the “SDLC Report”
(COM/MGT/REP/4168), which covers the lifecycle from Requirements to Release.
As a general comment, Fujitsu is only one supplier involved in the overall delivery of end-to-end
services to POL in relation to Horizon. The Horizon application also relies on the working partnership
between POL and its chosen partners — such as Verizon, Computacenter and Atos — as well as external
service providers such as banks and affiliated organisations. This applies to both the IT systems and the
operational processes in Horizon.
Although every effort has been made to avoid confusing jargon in this report, the very nature of this
aspect of the service delivered to POL necessitates the use of many acronyms and phrases that may
need expanding upon to ensure the correct understanding. Fujitsu accepts that further explanation may
be necessary and encourages POL to seek these clarifications.
The topic of Testing & QA is a current open and active topic that results in continued communications
between POL and Fujitsu. This report aims to collate those interactions and communications in order to
provide a common baseline for any future discussions.
Fujitsu has endeavoured to ensure that the content of this report is correct as at the date of issue. This
report has been prepared with the input of numerous Fujitsu individuals and attribution of any statements
made in this report should be made to Fujitsu only. In preparing this report, the authors have collectively
characterised and summarised many internal Fujitsu documents. They have also described processes
and procedures which have been established over many years and may not be in written form. Many of
the documents, processes and procedures described in this report are continuously updated and Fujitsu
reserve the right to make changes to the way it works in the ordinary course of its operations and
business without obligation to update this document. POL should verify the position with Fujitsu before
relying upon any information or content from this document in the future as well as bearing in mind the
requirements set out in “Information Distribution” at Section 8 below.
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS
Page No: 7 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
4 Testing & QA
4.1 Testing Overview & Approach
Fujitsu has a Test and Validation (T&V) organisation who are the main providers of testing resource.
Fujitsu also has a corporate testing process within the Fujitsu Europe Business Management System
(EBMS). This includes a Testing Policy, Organisational Testing Strategy and Testing Methodology.
« The Testing Policy states the expectations Fujitsu has for the way testing will be conducted.
e The Organisational Testing Strategy defines the high level test approach to be adopted for all
programmes and projects.
e The Testing Methodology supports the Testing Policy and Organisational Testing Strategy by
identifying the supporting testing methods to facilitate achieving the defined goals.
The EBMS testing process provides document templates which the Post Office Account (POA) has
implemented where appropriate.
For POA this testing process impacts Solution Validation and Integration (SV&I) Testing and Live
System Testing (LST).
Development (DEV), Component Integration Testing (CIT) and Integration Testing (INT) all form part of
the Development phase of the Software Delivery Life Cycle and follow their own processes.
Section 4.5 describes in detail the environments used on POA. A summary is provided below:
e DEV - Used for developing code and performing Unit testing and code review. There is an
environment located in BRAO1.
e CIT-An extension of the DEV environment based in BRAO1 where components of code can be
Integrated for the first time e.g. Counter, Branch Access Layer (BAL) and Branch Database
(BRDB).
e INT - Used for packaging and testing the deployment of the code baselines and the regression
path of those baselines that make up change to individual platforms, which in turn make up a
release.
e SV&l - Used to test change from a solution perspective bringing together many components of a
change for the first time to validate the full requirements of a change.
e LST - Used to support the LIVE environment focussing on the patching and maintenance of the
individual components that make up the Horizon solution.
« RDT - Used to validate the REFDATA associated with the Horizon solution with the associated
tooling.
4.2 Defining the Test Approach
The Test Approach on POA is risk-based where changes are analysed by Subject Matter Experts (SME)
and as a result, tests, including regression tests, are derived.
The Horizon changes fall mainly into two streams: “Major” release or “Project” releases; and
“Maintenance” releases.
Major releases are those derived from POL Change Requests, which typically impact multiple platforms.
These go through the full SDLC and all test environments:
e ~=Test Planning
e Test Preparation
* Test Execution
o Including defect management
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 8 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
e —Reporting/Monitoring
e Test Completion
Maintenance releases are incremental updates to the LIVE environment.
4.2.1 Test Planning
At this stage, based on the details of the change, the required test environments will be identified. If the
change will follow the Major release path then Test Plans for DEV/CIT/SV&I/LST will be produced
describing what will be tested as part of a change for that test environment.
The key sections are:
e The scope of testing — both in and out of scope features
« RAID items
e Environment description with any constraints
° Test tools to be used
° Test resource
e Test schedule
e Requirements traceability.
These plans will use the POA document template and are subject to the document management process
for review and approval using Dimensions.
Test Planning will also involve attendance by key test resources at impacting and planning sessions to
assess the change against current workload.
An example Test Plan can be found in document reference TST/SOT/HTP/4072 which has been shared
with POL separately.
4.2.2 Test Preparation
Test preparation is where the Test team prepare for the testing activities.
4.2.2.1 Test Environment Preparation
The Test environments each have their own purpose and capability and the environments may need
specific preparation activities. For example, the SV&I environment is the first environment that has a
working Tivoli Workload Scheduler (TWS). TWS manages jobs that run at certain times during the day
(e.g. reconciliation of Counter data). TWS has a customised configuration as SV&I is not identical to
LIVE.
Test environment preparation follows a process of understanding the change and also the activities that
need to be completed during the preparation phase.
4.2.2.2 Test Cases
The SME for each area of change uses all relevant documentation (requirements/designs), design
review sessions and ad hoc conversations to determine what Test Cases need to be generated to
validate the change and test the agreed business requirements.
Test Cases will then be written as test scripts detailing:
e The definition of what is being tested
e Step by step process describing:
e Data input
* On-screen messages
e Keystrokes
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 9 of 32
FUJ00232714
FUJ00232714
ee) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
e Expected results
e Test data / cards / branches / pinpad combinations.
4.2.1.3 Test Readiness Review
For Major releases, a Test Readiness Review (TRR) will be performed, and chaired, by the Test Lead
with assistance from some of the other delivery leads. The TRR will gather all relevant information to
confirm that the Test phase is ready to accept the change and start test execution.
Each point will be assigned a “RAG” (Red Amber Green) status where: Green indicates the point is
complete; Amber indicates the point has a plan to “Go Green”; and Red indicates that the point requires
actions to plan.
A risk-based approach will be used to determine which activities are “showstoppers” to the process and
which activities have enough containment to move forward.
An example TRR mail can be found in Appendix A — Example Test Readiness Review mail.
4.3 Utilisation of environments (within the Change &
Software Delivery Lifecycle)
The following diagram represents the current Horizon Software Delivery Lifecycle. The Test
environments are highlighted as “blue” for context.
Datacentre
‘Upgrade I
Model Full
elk office [7] Met Rollout
Customer { Design & Development [ Deployment
Change
4.4 Major & Maintenance Release Flow
The Horizon changes fall mainly into two streams: (1) “Major” release or “Project” releases; and (2)
“Maintenance” releases.
Major releases are those derived from POL Change Requests typically impacting multiple platforms.
These go through the full SDLC and all test environments.
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR _ Date 29-Jan-2021
STORED OUTSIDE DIMENSIONS
Page No: 10 of 32
FUJ00232714
FUJ00232714
ie) TESTING & QA REPORT -
FUJITSU FUJITSU CONFIDENTIAL
Maintenance releases are focussed changes for individual components that make up the Horizon
solution e.g. Counter, BAL, and BRDB.
These releases go through a subtly different process where changes “skip” the SV&l environment but get
“topped up” once a month. “Topped up” is the process whereby any changes that were deployed, tested
and signed off by LST are delivered back into the SV&I environment. These include Maintenance
releases, security patches and Hot Fixes.
The following diagram (4.4.A) represents the flow of a Major release in context to the Test environments.
Each row represents the next step in the progression of a release through the environments.
R1/R2/R3 are fictitious release numbers. Green represents delivery of new change to an environment.
Pink shows where change is yet to be deployed. Red starts the cycle again. So, in diagram 4.4.A below,
we can see that new release “R2” starts in Dev, then moves to CIT, then moves to INT etc.
The overlay bubble comment relates to the point in time where we do not have an environment which is
representative of LIVE as RDT and LST have the new release deployed ready to go to LIVE. At this
point, environments become out of sync with LIVE. The POL Live Service team is aware of this (see
Recommendation 1).
4.4.A - Major release deployment flow
o 3
DEV R2 [ }I I I }f
Ce
eee
DEV R2 CT R2 INT N/A
oS Ae een fs I
~
This is where
PRODUCTION
becomes out of sync
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR _ Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 11 of 32
FUJ00232714
FUJ00232714
ie) TESTING & QA REPORT -
FUJITSU FUJITSU CONFIDENTIAL
This diagram 4.4.B represents a Maintenance release deployment flow.
The Maintenance release is shown as a “point” release and uses the same colour scheme as described
for Major releases above.
Again, each row represents the next step in the progression of a release through each environment. The
overlay bubble comment highlights the sequence of deployments which “misses” out SV&l as per the
current process.
4.4.B — Maintenance release deployment flow
as
DEVR21 INT NA ASTR24
Sr ee Si Wy
SORTED)
SV&I take a “Top up” of any
DEVE, ‘ ROT R2
fs I fees I MntRel / security patches /
Hotfixes on a monthly basis a
ae
o
ao
[
I
I
te Gi Gee
XK
Oo
)
I
4.4.1 Test automation coverage
4.4.1.1 Counter Business Application (CBA) Test Tool
The CBA test tool is an in-house developed tool that was originally used by the Development team as a
way of simulating Counters to verify changes.
The Test team use this tool to automate tests across each area of functionality to formulate a regression
test pack which shows a stable Counter delivery, but would also “seed” data into the Data Centre for
those apps to consume, which enables a wider set of tests to be performed.
For each Major release there is potential for Counter changes and therefore enhancements to the CBA
test tool. If a change to the CBA test tool forms part of that release it would follow the SDLC for the
relevant environments (and the Testing process where appropriate).
The approach for this area of testing is to run the current test pack against a new version of the Counter
as part of the testing cycle. The tests are run, failures are investigated, defects are raised and changes
are made to the tests, if needed, to get an outcome which is a “clean” run of passes and therefore a
known state.
Note: Testing of REFDATA is a POL responsibility. REFDATA is fundamental to the operation of the
CBA. See Recommendation 13 in Appendix D.
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR _ Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 12 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
A joint review by Fujitsu and POL is currently underway of the CBA test tool, test pack and process
output as part of CWO0223.
4.4.1.2 SOAPUI
Although POL and Atos are responsible for testing the NCR Self-Service Kiosk hardware and
application, Fujitsu have responsibility for testing the interface of the SSK to the Horizon solution.
SOAPUL is the tool used to simulate these SSK messages across the Fujitsu interface. A regression
pack has been formulated and is run when specific changes are made to the Fujitsu interface or as part
of a wider solution regression pack.
This tool and scripts are used across the CIT/SV&I & LST test environments.
4.5 HNG Test Environments
4.5.1. Component Integration Test (CIT) environment
The Component Integration Test environment (CIT) is used by the Fujitsu Development team. It was
initially in place to integrate the Counter Business Application with the Data Centre BAL/BRDB to ensure
a stable Counter/Data Centre interface.
The environment capability has now been increased to cover more services.
This environment does not have:
e Tivoli Workload Scheduler (TWS) so batch scheduling cannot be tested
e Estate Management capability
e REFDATA Supporting systems
e Any Computacenter software distribution.
The environment components are physically located in the BRA01 office. The team that manage and
use this environment are located in the Fujitsu GDC India Bangalore office and BRAO1.
4.5.1.1 Scope
The current scope of testing is to validate Counter/BAL changes in isolation with the flexibility of
incorporating changes swiftly as and when issues arise.
The testing is driven by JIRA where the testers allocate verification test(s) for the changes delivered by
the development team.
Elements of the testing process used in SV&I are being implemented for CIT including expanded
resource in the UK to give focus to testing with representative hardware/peripherals.
4.5.1.2 External Users/Use cases
There are currently no external users of this environment.
4.5.1.3 External End points & Simulated services
There are no external end points. Counters are simulated along with the branch peripherals.
4.5.1.4 Interfaces to Third Parties
There are no interfaces to POL third parties.
4.5.1.5 Testing from a Counter perspective
There is currently no use of physical Counter/peripherals from the Fujitsu GDC India Bangalore office
due to a constraint imposed by GDPR/PCI. However, all Counter variants and peripherals can be
simulated.
4.5.2 Integration (INT) environment
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 13 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
The Integration environment is a “technical” rather than “functional” test environment with servers based
in the BRA01 office.
In this instance, the technical testing relates to how a packaged piece of code is proven to be delivered
successfully onto a Data Centre platform and the regression steps for this package. There is no
functional test capability in this environment.
The resources are based in BRAO1 and the Fujitsu GDC India Bangalore office.
4.5.2.1 Scope
This environment is used for packaging and testing the deployment of the code baselines that make up
changes to individual platforms which make up a release.
Each baseline needs a particular ordering and verification to determine how the change is best delivered
to the individual platforms in the Data Centre.
Some baselines come with accompanying deployment steps/details which need formulating and
packaging along with code.
The environment is made up of servers with the different operating systems representative of the LIVE
environment. Integration has at least one instance of each Data Centre platform type. They are on a co-
ordinated rig and in cases where a product is deployed to multiple platforms they are tested together.
4.5.2.2 External Users/Use cases
This is an “unmanaged” environment. It does not have any platforms located in the Belfast Data Centres.
It is used specifically by the Integration Team with members based in the UK and our Fujitsu GDC India
Bangalore office.
4.5.2.3 External End points & Simulated services
This environment does not have any external end points or simulated services. Its purpose is to test the
delivery of code rather than Horizon functionality.
4.5.2.4 Interfaces to Third Parties
This environment has no interface to third parties.
4.5.2.5 Testing from a Counter perspective
This environment does not have Counter testing capability. Its purpose is to test the delivery of code
rather than Horizon functionality.
4.5.3 SV&l environment
The Solution Validation and Integration environment has been specifically tailored / designed to support
functional testing to validate customer business requirements for project related change.
It is representative of LIVE in that it has many of the capabilities that operate in the LIVE environment:
“Real” Counter and peripheral hardware supplied and managed by Computacenter, with Verizon
managed Branch networking integrated with a fully managed Data Centre and real interfaces to third
parties — albeit with test end points.
It is used by many user groups and is seen as the “busiest” of test environments servicing multiple
requests on a regular basis.
The resources that make up this test team are all based in the UK.
4.5.3.1 Scope
The purpose of SV&I is to provide an environment that can test change from a solution perspective
bringing together many components of a change for the first time.
It has a Counter estate based in BRAO1 (Fujitsu Test) and Winnersh (Atos Test) interfacing with a
Belfast Data Centre. This Counter estate is representative of LIVE for hardware and Branch networking
including peripherals (Scanner/PinPad/weighscales/printers etc.)
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 14 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT “
FUJITSU FUJITSU CONFIDENTIAL
It is the first environment where TWS changes are implemented. TWS run scheduled jobs to replicate
the processing of data through many Data Centre components with reconciliation processes which
consolidate the data for onward processing to external POL systems.
Its capabilities and constraints include:
+ Data Centre components fully managed Monday-Friday (this refers to all code deliveries)
* TWS schedules running Monday-Friday
+ PODG, GWS and HBS Services
+ Representative hardware
* Customised REFDATA delivery mechanism
+ Simulated end points with other systems
+ External endpoint connectivity
+ Extends to Winnersh (Counters)
+ No clustered BRDB
+ Not scaled to LIVE
+ No BRSS platform
+ No HORIce capability
+ Usage managed by a Forward Schedule.
This highlights a number of areas that have been discussed between POL and Fujitsu where
improvements can be considered. See Recommendations in Appendix D.
4.5.3.2 External SV&l Users/Use cases
Over a number of years the SV&I environment has proved capable of supporting other user groups’
testing and development activities to enable better utilisation of its resources to support POL change.
The current user groups and their typical activities are described in the following table:
User Group Description of usage
Atos + Path of a transaction / data from the Fujitsu “Boundary” to POL third
parties including Banks / Online services and also POLSAP
+ Testing of NCR changes to the SSK, using physical SSK
+ Support of Computacenter testing Build changes / patches
* POL UAT — Hosting and / or performing
+ — Integration Testing of CDP changes
+ REFDATA changes - APADC (Automated Payment Advanced Data
Capture) / new products
NCR + Develop changes to their SSK application using SV&I (interface to
HBS) as a stub
Accenture + — Supporting Atos testing
Ingenico * — Providing integration to environments to support the testing of PCI
changes
HIH Project + Multiple test environments integrated with SV&I (INT 1 & 2,
Performance, RPOS)
+ Accessed by POL team in Moorfields
+ Focus on end-to-end testing
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS
Page No: 15 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
+ More recently Performance Testing
Horizon Development + Test fixes prior to formal delivery (under guidance from the SV&l
Teams support team)
These user groups are all managed using a simple Forward Schedule of Change workbook which is
updated on a weekly basis where details of who is using the environment and for what purpose are
noted. This includes details of any scheduled outages (maintenance release top ups, DR rehearsals) that
may impact the access or availability to these groups.
There is also a local Support Guide used by all users that describes how the environment is supported
and how user groups can raise observations, request evidence etc. See document reference
TST/SYT/PRO/3862 (currently v0.3), which has been shared with POL separately.
4.5.3.3 External End points & Simulated services
In order to validate some of the Horizon functionality/requirements it is necessary to simulate a number
of POL’s other third party end points to provoke responses or send information.
The detail of these external endpoints and simulated services can be found in Appendix B — External
Endpoints.
4.5.3.4 Interfaces to Third Parties
SV&I is the only environment that is used to connect externally and this is only relevant for Atos/POL
testing of Payment and Banking changes.
Where third parties have a Test end point, these are managed via requests to the SV&l Rig Support
team to implement a change to a configuration file where a “switch” is made to change from the use of a
simulator for traffic to the desired Test end point.
4.5.3.5 Testing from a Counter perspective
The approach to the testing of Counter Business Application changes in this environment follows the
approach detailed in section 4.2.
The Counter estate has the following configuration capabilities:
+ Single Counter outlets
+ Multi Counter outlets
+ Multi language
+ Counter Training Office
+ Global Office.
The Counters are not managed by the Computacenter software distribution tool. This allows for the
flexibility, during a Test phase, to make rapid deliveries of the application as and when required.
Additionally, the Computacenter compliance check is not enabled in order to further support this
approach.
Note. The compliance check is a tool that Computacenter use to determine which is the most valid
version of the CBA and if “new” versions are deployed for Test purposes. This check will remove that
version and rollback to what it believes is the most valid.
4.5.4 LST environment
The Live System Test environment is in place to support the LIVE environment, focussing on the
patching and maintenance of the individual components that make up the Horizon solution.
As part of the SDLC, this is represented best in Section 4.4 where these changes by-pass the SV&I
environment.
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 16 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
For Project change, LST is used primarily to rehearse the deployment/migration activities with the same
tooling/teams and process that would be used for LIVE.
The resources that make up this team are based in the UK and our Fujitsu GDC India Bangalore office.
4.5.4.1 Scope
The LST environment is a representation of the LIVE environment with Counter and peripheral
components located in the BRAO1 offices and the Data Centre capability in the Belfast Data Centre.
There is DR capability for Counters at the Fujitsu Stevenage office.
Monitoring of the supporting Data Centre systems is done using the Systems Management Monitoring
toolset to look for new expected and unexpected events/alerts. Checks are also made on the systems
that manage how the Counter estate is monitored when branches and Counter positions within branches
are added or removed (Estate Management database and Branch Change Management System
(BCMS)). BAL / BMX logs are also checked.
The following list gives a high-level view of the LST environment capability:
+ Data Centre platforms fully integrated
+ Data Centre components managed 24/7
+ TWS schedules running 24/7
+ Real Counters / peripherals
* Counters managed by Computacenter
+ Formal REFDATA delivery mechanism
+ Simulated end points with other systems
+ No external endpoint connectivity
* Clustered BRDB environment
+ PODG and HBS services
+ Scaled where appropriate
+ BRDB Standby Server (BRSS)
+ Goldengate replication from BRDB
+ HORIce — Software Support Centre tooling
+ Independent systems monitoring by SMC team
+ — Subject to full change control.
4.5.4.2 External Users/Use cases
User Group Description of usage
HIH Project + — Test Environments integrated with LST
+ Accessed by POL team in Moorfields
+ Focus on end-to-end testing
+ More recently Performance Testing
4.5.4.3 External End points & Simulated services
LST simulates all external end points. Appendix B — External Endpoints provides further detail.
4.5.4.4 Interfaces to Third Parties
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 417 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
The only external interface is to Postcode Anywhere to be able to run tests that require address
information.
4.5.4.5 Testing from a Counter perspective
The Counter estate in LST has the capability of testing with the following configurations based on the
specific requirement for the change and what is required to be tested in the LIVE estate:
+ Single Counter outlets
+ Multi Counter outlets
+ Multi language
* Counter Training Office
+ Global Office
+ Computacenter monthly patch testing
* Computacenter build and regression testing with current LIVE Counters
+ HBS/PODG services.
The Counters are managed by the Computacenter software distribution tool in order replicate the
mechanism used in LIVE. Additionally the Computacenter compliance check is enabled.
4.5.5 RDT environment
RDT is the Reference Data Test environment. In the Software Delivery Life Cycle it is the final
environment before LIVE. This is only in terms of keeping its Counter and Data Centre components up to
the most recent versions of code. It is not to test the actual reference data (REFDATA) itself.
4.5.5.1 Scope
This environment has been established to validate the REFDATA associated with the Horizon solution
with the associated tooling. This REFDATA is used to “drive” much of the functionality that the CBA can
perform but has also been used to deliver “code” e.g. updates to the HBS service.
The following table reflects the various types of REFDATA that are currently processed.
Ref Data I File type Data
(Type)
A MDM Reference Data (POCL) e Branch Data
« Product Data
* Accounting Calendar
¢ POL Clients and Client Accounts:
e Automated Payment Tokens
e Banking Schemes
© CTT Mappings
e End of Session Prompts
e Transaction Prompts
Cc APOP Service definition reference data
only, EPOSS reference data
F Counter specific reference data: Logo e Various logos and bitmaps
bitmaps, or other non-xml data
G HNG-X specific ref data, shared to BRDB Ie Supports BAU deliverables such as
and other platforms like CWS encryption certificates and Generic
Web Service definitions
Xx Postal Services/Transaction scripts/Menu e Pin Pad Reference data
definition/Miscellaneous other reference e Postal Services
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR _ Date 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 18 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT '
FUJITSU FUJITSU CONFIDENTIAL
data Transaction Scripts
APADC Scripts
Menu Definitions
Restricted Product Curricula
Logon Curricula
Various Other Reference Data Objects,
Including:
o Message Definitions
o Postal Service Tokens
o Postal Service Administration
Processes
e POL Delivery Agreements
¢ POL Client Accounts
Help data I Online Help Pages accessed by the
Counters from the central HBS Server
FirstRate I Bureau margins / Spot Rates / Branch &
Price Profile
PAF Postcode Address Data
RDT has Counter and Data Centre capability. However, its focus is to consume the Data Centre services
and verify REFDATA processes correctly to the Counter / Data Centre components.
The Data Centre components are hosted on the same infrastructure as LIVE.
There are 4 streams of work that are hosted within the RDT environment. The first 3 in this list are used
by Atos information services, the fourth by Fujitsu.
1. RDDT —- Terminal Verification for Reference Data Validation, integrity checking, and automated
testing
2. RDDIV - Interim Verification (Code Changes). Each change individually (Product, token menu
hierarchy rules scripts)
RDDOV - POL Outlook Systems Group verification of full packages of REFDATA
4. RDDPL (Pseudo LIVE REFDATA environment) for the investigation of any Live Reference Data
issues.
Some elements of the environment are managed locally e.g. the Tivoli Workload Scheduler that manage
the batch processing.
Data Centre components are managed by the Fujitsu Belfast teams, with Counter software being
managed by Computacenter.
4.5.5.2 External SV&l Users/Use cases
Outside of Fujitsu resource, the user groups using one or more of the RDT “Streams” are Atos
Information Services and Atos Test.
The Atos Information Services Group “key” the REFDATA based on POL requirements which is “fed”
into the REFDATA tooling for processing before going through the delivery process.
The Test Team are currently used to generate and test APADC scripts again, before being “fed” into the
REFDATA tooling.
4.5.5.3 External End points & Simulated services
LST simulates all external end points. Appendix B — External Endpoints provides further detail.
4.5.5.4 Interfaces to Third Parties
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 19 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
The only external interface is to Postcode Anywhere to be able to run tests that require address
information.
4.5.1.5 Testing from a Counter perspective
Each of the four streams has two physical Fujitsu Counters which are both Single Counter Outlets
(SCOs). The only peripherals being used are the back office and tally printers to verify REFDATA
changes on receipts and reports.
N.B. Atos have a number of Counters:
e 2SCO in RDDT
* 1SCO in RDDIV
* 6SCO in RDDOV.
4.6 Platform differences
Fujitsu maintain a Platform Hardware Instance List (PHIL) which lists the available platforms for
LIVE/LST/SV&I/RDT. DEV/CIT/INT are a consolidated entry.
Not all platforms are available in all environments. This is based on the intended use of the environment
and the test purpose, and has evolved over years and through numerous changes.
There are a number of recommendations made which refer to opportunities to increase Test
environment capability. See section 7.
Appendix C — Environment Platforms shows the differences between the environments at a Data Centre
level.
4.7 Transaction types
For Counters to transact, they need the Fujitsu Counter Business Application and REFDATA delivered
by POL. They also need the hardware and peripherals.
The REFDATA determines which transactions a Counter can complete. This is based on many factors
including geographical location.
The Counter functionality is split into several main areas as defined below. These are further broken
down into 82 sub categories, which are not listed here due to the size but can be provided to POL
separately, if requested.
= BAC - Branch Accounting
= BAD - Branch Admin
= BDC - Bureau de Change
= BNK - Banking
= BOF - Back Office
= CSM - Cash and Stock Management
=» DVLA-DVLA
= ETU - Electronic TopUp
= EUM - Enhanced User Management
= GLB - Shared
= _GWS - Generic Web Services
= IOP - In and Out Payments
= PAF - Address Find
* POCA - Card Issuing
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 20 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
= POS - Postal Services
= RSS - Retail and Stock Sales
= SDB - Settlement by Debit or Credit Card
= SM - Smart Metering.
SV&lI is the only environment capable of testing this full set of transactions (see section 4.5.3).
CIT/LST/RDT can perform some of these transactions but is limited (see earlier sections).
4.8 Model Office - Live Proving
There is currently no formal engagement with the Model Office team by the Fujitsu Test organisation.
Model Office for LIVE proving is conducted by the POL project team.
5 Formal Audit Reports
POL has commissioned an ISAE3402 audit as well as quarterly PCI Prioritised Approach audits on POA.
Both audits examined the topics discussed in this report. Furthermore, POA are periodically requested to
contribute to internal Fujitsu corporate audits to support Fujitsu UK in attaining and maintaining a variety
of certifications such as 15027001, 1S09001 and 1SO22301.
6 Conclusions
The current Fujitsu Testing & QA process, with the associated test rigs, used as part of the full SDLC,
provides a high quality test service for the validation of changes for a safe deployment into LIVE.
It is, however, recognised by both Fujitsu and POL that there are constraints imposed by using the
current approach, and that there are areas for improvement. It is therefore acknowledged that there is a
need for both POL and Fujitsu to focus on improvements. (see Recommendations).
7 Recommendations
Testing & QA has been the topic of numerous collaborative discussions between POL and Fujitsu over
the years and during recent meetings. A compilation of recommendations for improvement arising out of
these meetings are contained in Appendix D. This list in Appendix D is aimed at collating the various
action items into a single list so that POL and Fujitsu can work jointly to improve the Testing & QA
capabilities. The recommendations should be prioritised and the most relevant ones actioned as
promptly as possible.
8 Information Distribution
This report and any enclosed materials (the “Audit Materials") are being provided to Post Office Limited (
“POL”) pursuant to POL’s request for an audit of Horizon (the “Audit"). The Audit Materials comprise
work product prepared by Fujitsu pursuant to requests from POL. Fujitsu has confined this report to the
specific requests from POL and accepts no responsibility for any other matters. The Audit Materials
relate to the current HNG-x platform.
The Audit Materials are confidential and provided to POL for the sole purpose of the Audit. The Audit
Materials may only be shared by POL with KPMG, the external auditors appointed by POL in connection
with the Audit. POL shall take all necessary precautions to ensure that any Audit Materials are: (i) not
used for any purpose other than the Audit and; (ii) not disclosed to any third party (apart from KPMG),
without Fujitsu’s express consent in writing. In particular, it should be noted that:
(i) the Audit Materials may contain highly confidential and sensitive information which, if disclosed,
is likely to significantly increase the risk of cyber and engineering attacks on the Horizon system;
(ii) the Audit Materials may contain personal data within the meaning of the General Data Protection
Regulation (“GDPR"); and
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 21 of 32
FUJ00232714
FUJ00232714
re) TESTING & QA REPORT
FUJITSU FUJITSU CONFIDENTIAL
(iii) any system architectural content may be subject to copyright and/or other intellectual property
rights and cannot be shared or disseminated.
Prior to making any permitted disclosure of the Audit Materials (or any part thereof), POL shall provide
Fujitsu with reasonable advance notice of such intended disclosure and shall permit Fujitsu the
opportunity to redact information including but not limited to any privileged information, personal data
and/or other commercially sensitive or proprietary content.
This report refers to various documents that are confidential and internal to Fujitsu. Such confidential
documents are proprietary to Fujitsu and are not intended for sharing outside of Fujitsu. Fujitsu in no way
waives or intends to waive confidentiality in these documents by describing, referring to, reproducing
extracts of, or in any way referencing these documents in this report. Where extracts of such documents
are reproduced in this report, redactions have been applied to protect personal and sensitive information.
The Audit Materials, or any part thereof, may not be altered or amended without Fujitsu’s express
consent in writing. Under no circumstances shall any Fujitsu personne! be named or identified in any
reports or other documents created by POL based on information from the Audit Materials (or any part
thereof). Attribution of any Audit Materials shall be to Fujitsu only.
Unless agreed specifically in writing to the contrary Fujitsu does not accept any duty of care or any other
legal responsibility whatsoever to any person or entity in relation to this Report, any related enquiries,
advice or other work. Any person who receives a draft or copy of this Report (or any part of it) or
discusses it (or any part of it) or any related matter with Fujitsu, does so on the basis that he or she
acknowledges and accepts that he or she may not rely on this Report or any related information given by
Fujitsu for any other purpose.
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS
Page No: 22 of 32
FUJ00232714
FUJ00232714
ie) TESTING & QA REPORT -
FUJITSU FUJITSU CONFIDENTIAL
Appendix A — Example Test Readiness Review mail
a POL Ten ange
SVE Tet Manage
=
‘egraon sed Rea Manage
Sat Erronment Readiness
Exiting counter
+ Ready -R70287
+ RAMOS nee tobe plugged in > Emulator, hie wil et be connected to Axis
+ R030- Sprint 9
+ WPOS Package needs tobe vaste
* ohn ickman’s inrvction: needs to be captures
© TPV~Should be Automatic (Nee to check wt
© Currently iis sorted marty in SV
In IST Weshovd be testi mstomtieay
+ G19 pINPADts be used
‘SB New PINPAD C3 (S-9} are received (SVB ad LST)
the PINPAD from RAMEDS to C1 (S819)
isnot ready fer S18 /S+19 to RAMEDE
fo esting nigrtion of he PINPAD,
Fa
20 PINPAOS— SB lt
‘© UT aol these to make 8 to RAMEOR
Acceptance Env (ingen) -SVBI
1 Enveonment sorting needs tobe done Bs previous the woviroament was mapped to otegraton Ew. Key ston
Mike —toensute provisioning onthe acceptance en
tntegration ~ Devetopment only
Certcation env. Acereditaton ene
Pre Production -?
+ 990.30 Sprit version - ETA 04/08
fscxend
20.92 ~ Sprint S— antine packages to Be installed - 04/00
4 conten the penton of release
000 networe comnectnvty to ingeica
Done
cca ~ ETA 07/095
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR _ Date 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 23 of 32
FUJITSU
TESTING & QA REPORT
FUJITSU CONFIDENTIAL
FUJ00232714
FUJ00232714
Tame [Scggested deliverable [Team [owner Fore steraied frp ‘RAG
ive. doc name, kcomoteion} status
tatement ete) boate
Whats the curear stata [seatement rex [SvaI Test Manager RO W complete, bs
the Testing in SVBI? npected
What are the code fist af deiverabies rom [DEV [Development Manager lost 04/08 Kava
componeatstor delivery Ithe development teams iam unter 08/09
for this eyele? ukrsancyrcit/csRerosTifiorine
la there anything new aris
it just PEAK fines?
Ot testing Discusion to understand OE Development ales open
Wht testing hase being [how the CIT testing went anager (Crt Test Anahystfaformal Foxes
done? report showing the sop lebwere)
tne results of teats rest Stata: %
= Serie a
functionatty
i breating
Sorint S— Remaining
fremene ISCREFOATA [SSC REFDATA~ Technical ISG eh data
levered
When ist chertued for
every to SEI?
mae REFOATA?
Does SVEI need to take [Statement SCREFDATA ISSCREFDATA™ Technical ASG ef. data
REFDATA reset? speciaat etvered
Has any TEST ONLY DATA
been vied adi being
managed inthe
sparoprite folder
What the curene Ssatementaxto te [EV evelogment Manager
debvery status of these [delivery status into tom
components? Dimensions and whether luxysmerrcnsesreroar
What has been delivered? Iehey ate.on trpet
Depending onthe resists of
n
Whatls the progres ofthe [Intof delverabies [It fetegration Manager
deliverables though feompared to that in item
Integration? a
te there ay rok tothe
current dates
What are the iat of document bev [Development Manager R7030- Release
documentation eterences tr each ates needs be
etverables fortis Jonange updated vis the Lodstes
release Development desiverate Jt: Other
soreadsheet und thei
Including Support guides [otatus
lionart/appRove}
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL, Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR _ Date 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 24 of 32
FUJ00232714
FUJ00232714
ie) TESTING & QA REPORT -
FUJITSU FUJITSU CONFIDENTIAL
‘Ave there any additional Statement of the release [i itajor Release op ups are moved tl
deliverables fer this leonters. rasciager integration
telease? rae any Maintenance Patanage
Top-ug's, and another —_Iielease signed off by IST
emonth Security patches Ithe deployment plan és
mate avaiiale to verity
any “enceptionat”
lpssetines
‘Build hom integration [ist ofrelease spechc [INT frecgration Manager
—
Report of faitures
Date at the very toast
Deployment Plan available IStaternent oa Phtajor Retease Manager Pieployment Plan
for review patsy need slightly
frnodified for LST
Date at the very feast Jome questions are
pen. Okay tor Svat
the counter goingtn be IStatement/Updated [INT tegration Manager (MA iA ry
deployed using BIGEN® __Ipeacess and timescale
‘Does the test taol require a [Staternant loev [Development Manager [CBA Test Tao!
new version a this teed delivery
Cycie/Retease 3 Simulator
Ro
‘re the baseline’ [statement Pow [POL Test Manager Jum testabe
requirements available for jequiresments
OSV poate,
Others ~ingenico or
fray fal for SVE
Pietion on Steve € te
come back
Mave allthe HTP been [TST/SOT/MTP/ TI? [Test [Svai Tes Manager [fest Approach @
produced and formally poroved
reviewed?
‘Are SVEI Test Resources [Statement fre [Sve Test Manager iesources ave
‘are available for the anined
plmned test execution?
1s SV Test preparation [Test Sevipts in QE rest low 10% camnplote
complete?
‘Ace datacentre deplayment [Statement JreaNOUNT [SVEI Tex Manager [Resources are igned
resqurces available?
statement fre [svi Text Manager irsaurees are aligned]
element of Test Rigs?
Mave the defect waging [Statement bre POL Text Manager need
Ao8
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR _ Date 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 25 of 32
FUJ00232714
FUJ00232714
(oe) TESTING & QA REPORT .
FUJITSU FUJITSU CONFIDENTIAL
Appendix B — External Endpoints
Correct as at: 14 December 2020
uve vai ist ROT
Banking Payments
nk ISimulated/3rd Panty [Simulated IN
caro ‘Simulated/3rd Party [Simulated Iv
SANTANDER, ‘Simulated/3rd Party [Simulated Iv
IGlobalPay IPayment Files (inc. AMEX) ISimulated/3rd Party ISimulated IN
Epay IETU confirmation. ISimulated/3rd Party ISimulated IN
res User Hes Transactions v lv Iv
POL Reconciliation Reports / TES Reports Simulated/PODG _ISimulated/PODG IN
crs POLSAP oad file ‘Simulated/PODG —_Isimulated/PODG _IN
[cor Sanitised EMIS [Simulated/PODG Isimulated/PODG __IN
[On _line Services
[credence [EWS Branch Full detail PODG Daemon running —_[¥ IN
Royal mail [cwS- Branch Lookup / Space IREFDATA from PODGIREFDATA from PODG [N
PostCode Anywhere laWS- Postcode Lookup ard Party rd Party rd pany
IQuadrient (Neopost) IBWS - Postal Delivery data [3rd Party [3rd Party. IN
[caro [BWS~ Card Activation ‘Simulated/3rd Party [Simulated Iv
DVIA lDWS- License Data ‘Simulated/3rd Party [Simulated Iv
First Rate laws Simulated/ard Party [Simulated simulated
Health Lottery laws: ‘Simulated/3rd Party [Simulated Simulated
National Express laws ‘Simulated/3rd Party [Simulated Simulated
Home Phone DVIA laws SSimulated/3rd Party [Simulated Simulated
ISettlement_Logistics_APOP-
Pou
FRITS RDT use
[ap Cliente Pons
[APOP Clients capobiityss
IOther Clients ‘These end points are for the PODG solution to send files to. In SV&I/LST test routes
[Arrow have been established to simulate the various route "definitions" to mimic how the ameansto
transfer
Royal Mail different file types are distributed. ae
The Yellow items are feeds into the Horizon solution and although the capability to I"
testis there, these are not testedas amatterof regression only hen changes have I 4 ang
been made that we ae informed need testing or rgeression testing via change process CorPor™te
reports, NBS
Service
details for
[Track and Trace XCS Emulator IXCS Emulator uve
[COGENT JAE 3rd Party, [N/A Supporting
POL [TWs, Test W/S [Test W/S-
REFDATA
nS [Branch Data PODS kes bes mo
mom Help and REFDATA iv
ee ne saarmamgar SVB / {ST consume the output ofall these FX
deliveries as a weekly REFDATA fileset
Ssc/DEV [PODG Routes and other config ly
[aPOP client Retail prices ly
HBS
ingenico DC Smart Metering Test Sewvice rest Service o
[cop [coP Data ed Party lath ary Iv
British Gas Smart Metering Test Service rest Service Iv
Understand thatthe HIH service has been
ceased (from a Test perspective), howeverI
ie smart Metering and basket settlement the connectivity isstillthere __IN
Estate Management Support
[customer Services Service Level Reports N iv iN
ssc estate Management Reporting via HORICE_[N lv Iv
POL Postshop/Payzone Branch Configs N iv Iv
ssc [Branch Data N iv Iv
[agent Portal ror querying N ly Iv
Retail Integrators [CSR Validation N iy Iv
BCS lEstate Configuration ly LY (10 Dys Full Cycle) _[N
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL, Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR _ Date 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 26 of 32
FUJ00232714
FUJ00232714
ee) TESTING & QA REPORT -
FUJITSU FUJITSU CONFIDENTIAL
(eas
Ieaeon arm [pry aot ‘arr 00
ne ins Fat 77 stat
aw laws ia sv wa
ier ions oo iva
Now laws atonigre va
Poo ws [isto valet Retox
ss ts [fartshess va
iow em oo Fo
fc ax [fossa iva ‘sander
ie i Rect Ivar FOSSA
image)
ig Commarea wan co iar
ba name™ fateveenate iar me
Intedio nc
ins
Seay le oar (sail ina
epeon input aarsae arene
a eb op [arn emip ina aa Soop
ca Kaan fa Wek Voronoi iar
i i as codon a
forte coor on jake bay nase
UeTEEE fa loa (aah a
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR _ Date 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 27 of 32
FUJ00232714
FUJ00232714
foe) TESTING & QA REPORT .
FUJITSU FUJITSU CONFIDENTIAL
Appendix C — Environment Platforms
Correct as at: 14 December 2020
ACA iY IN IN IN IN IN IN Shared
ACO iY IN iY \v Yy i iY
Kove ly IN i i IN IN IN Not Required - DEV
ARO? ly IN YY. lv \Y \Y Y
Auwwe ly IN iY Vv \Yy Y Y
AWS iY IN iY Y i i iY
BAL? ly IN iY i \Y \Y iY
Bo ly. IN iv iv IN IN IN [Not Required - DEV.
BA ly. IN ly. ly. IN IN IN [Not Required - DEV
Btev2 ly IN iY lv ly Y iY
Bosv2 ly IN iY \v ly i iY
Brew? ly IN iY ta \Y iY iy
Bly iN iY. Vv iy ia iy
BRSv2 ly IN iY lv \y y Y
Bsiv2 ly IN iY lv ly i iY
Bsivs ly IN iy ta IN IN IN Not Required - DEV
BS iy. IN YY ly \Y \Y Y
BSNS ly IN ly. ly. IN IN IN [Not Required - DEV
BSS iY IN iY lv ly Y iY
BSW iY IN iY v IN IN IN Not Required - DEV
Bs iY IN iY Vy \Y Y y
CANS ly IN iY. lv Y G Y
cA ly IN iY Vv IN IN IN Not Required - DEV
AS iY IN iY ly IN IN IN Not Required - DEV
coev2 ly IN iy ta \Y Y i
cosv2 IN IN iy. lv IN IN IN Not Required - DEV
cosv3 IN IN ly lv IN IN IN Not Required - DEV
cow ly IN IN IN y Y. iY Shared
ows iy IN iy iv Y \Y iy
DAT? ly IN iY lv ly i iY
owe ly IN i ta ly \Y iY
Dave ly IN iy. ta \Y iY Y
DoE? ly IN ly Vv \Y y Y
Dene iY IN iY lv ly y iY
cad iY IN iY i ly Y iY
ONS i IN iy i \Y iY iY
ows ia iN Lv Vv iy iy ia
xe iY IN iY lv y y iy
ox iY IN IN IN IN IN IN Not Required - DEV
oT iY IN IN IN IN IN IN Not Required - DEV
EAS iY. IN YY. lv \Y \Y Y
BAS? ly IN ly \v Yy Y Y
=s iY IN iY IN IN IN IN Not Required - DEV
iY IN i i i Y iY
Ss ly IN iY i IN IN IN Not Required - DEV
= Y IN ly lv G Y Y
Ely IN iy lv IN IN IN Not Required - DEV
eS iY IN iY \v i Y. iY
Lila i IN iY i Y Y iY
BO? ly IN iY. Vv IN IN IN Not Required - DEV.
Ba iY IN ly Vv ly Yy iy
Baw? ly IN iY ly IN IN IN Not Required - DEV
ou iy IN i i Y \Y iY
aad i IN iy lv \y Y iY
5S iY IN ly IN G Y iY Not Required - PD
Essv2 ly IN iY lv IN IN IN Not Required - DEV
ST iY IN iY ta \Y. Y iY
Lad i IN iY lv Y iY iY
FRG IN IN iY. IN \y ia y Emulator
FSS iY IN iY \v IN IN IN Not Required - DEV
ows iY IN iY Ly Y \Y iY
esv2 ly IN iY ta \y Y. ul
sw iY IN IN IN IN IN IN Not Required - DEV
pee Y IN IN IN IY y iY
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL, Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 28 of 32
FUJ00232714
FUJ00232714
(oe) TESTING & QA REPORT .
FUJITSU FUJITSU CONFIDENTIAL
ow IN IN IN IN IN IN (Not Required - DEV
Keely IN Mi Y ly YY \v
KSN IN IN ly IN Y ly ly [Decommissioned
Ksw2 ly IN Mi \Y IN IN IN Not Required - DEV
aa 2 IN IN IN IN IN IN Not Required - DEV
Weve v iN v iY IN IN IN Not Required - DEV
Owe IN IN Mi \Y IN IN IN Emulator
ww IN IN ly IN IN IN IN [Not Required - DEV
wave ly IN Mi iY YY YY \v
Nave ly IN ly iY ly ly ly
Mal? ly iN Y iy. YY YY v
oly IN ly IN IN IN IN Not Required - DEV
ney IN ly IN IN IN IN [Not Required - DEV
how LY IN Mi \Y iY ly Ml
nos ly IN ly iY IN IN IN [Not Required - DEV
Nez ly IN IN IN N IN IN Not Required - DEV
nec IN ly IN IN IN IN Not Required - DEV
NPOve v IN v IN IN IN IN Not Required - DEV
Neva ly IN ly Y. N IN IN Not Required - DEV
we ly IN ly IN IN IN IN [Not Required - DEV
iss iy IN lv Y N IN IN Not Required - DEV
wey IN IN IN IN IN IN [Not Required - DEV
Ovi v IN v iY \Y ly v
ws ly IN Mi \Y 1 ly Mi
Or IN IN a iN IN IN IN Not Required - DEV
ows LY IN Mi Y y ly. Ma
PAN ly IN IN IN ly ly ly Shared
iad ly IN LY iY. y YY v
PRG IN IN ly IN ly ly ly [Decommissioned
RAW v Y. IN iN IN IN IN Not Required - ROT
Few iM \Y IN IN IN IN IN Not Required - ROT
REA Ly iY IN IN IN IN IN [Not Required - ROT
FOM v IN LY Y ly ly ka
RB? ly \v IN IN IN IN IN Not Required - RDT
RGN? Lv iY. IN iN YY YY v
Rew ly LY IN IN \Y ly Ma
Rez ly iy IN IN IN IN IN (Not Required - ROT
RLIVe Y LY IN IN ly ly ha
Laat ly \Y IN IN IN IN IN Not Required - ROT
REve LY LY IN IN IN IN IN Not Required - ROT
Rive ly ly IN IN IN IN IN Not Required - RDT
ROW v iY. IN IN IN IN IN Not Required - ROT
RsWw2 ly LY IN IN ly ly. Ma
Rw? ly lv IN IN IN IN IN Not Required - RDT
Rxsv2 ly LY IN IN IN IN IN Not Required - ROT
SO ly IN ly \Y ly ly ly
SPN ly IN IN iN y YY v Spreadsheet
SPs ly IN ly N N IN IN Infrastructure
heseill (7 IN ly \Y ly ly ly
SN ly IN Mi Y ly ly Ma
hese (1 IN ly \Y IN IN IN Not Required - DEV
SRS IN IN Md iN YY YY ly
ssw IN IN Mi IN ly ly ly
Se ly IN ly \Y IN IN IN Not Required - DEV
wT iy IN IN N N N IN lAppliance
sys2 ly IN ly \Y ly ly ly
sve ly IN Mi \Y IN IN IN Not Required - DEV
Lind ly IN Mi LY ly ly ly
waa ly IN ly \Y IN IN IN Not Required - DEV
wes ly IN Mi LY ly ly ly
wea hy IN ly IN IN IN IN Not Required - DEV
xs IN IN ly \Y. YY lv ly Emulator
XCV2 IN IN Ma LY IN IN IN Emulator
BO OFFLE
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL, Ref: COM/MGT/REP/4166
Version: 1.0
UNCONTROLLED WHEN PRINTEDOR _Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS Page No: 29 of 32
re)
FUJITSU
FUJ00232714
FUJ00232714
TESTING & QA REPORT .
FUJITSU CONFIDENTIAL
Appendix D — Master Recommendations List
We set out below the list of Recommendations, the majority of which have been the topic of
conversations between POL and Fujitsu over the years and during recent meetings. This Report
presents an opportunity to collate all of the action items into this single list so that POL and Fujitsu can
work jointly to improve the Testing & QA capabilities. Recommendations should be prioritised and the
most relevant ones actioned as promptly as possible.
Ref I Recommendation Additional Notes
1 Review options to ensure there I As described in section 4.4 above, there are times when there is no
is always a test environment single environment which is at the LIVE version of code / REFDATA.
representative of the LIVE This poses a risk of issues being found in LIVE with no guaranteed Test
environment rig available to retest/verify the issue. Although this situation rarely
exists for more than 2 weeks, consideration is needed to build/support
a dedicated replica of the LIVE environment.
2 Review options to keep the « As described in the SV&I environment section above, the SV&I
SV&I environment “in step” with environment is not “in step” with LST/RDT as monthly security
LST/RDT even when monthly updates/maintenance releases/Hot Fixes do not get deployed until ~1
security updates/maintenance month after they have been signed off by LST.
releases/Hot Fixes are being Increase the rig capability so that all change can be managed through
deployed the full SDLC e.g. additional BALs, clustered databases, full Estate
Management capability
e Plan for all change to go through the full SDLC.
3 Review options to extend SV&l « — SV4&lI's ability to replicate LIVE is decreased as there is no out of
support hours — perhaps to 24/7 hours/weekend support. TWS schedules are not run at the weekend
and the rig is effectively shut down.
«The LST environment currently has a fully functioning TWS schedule
and is supported 24/7
¢ _SV&I would be an addition to this.
4 Review options to provide « — Early testing of Counter changes in the CIT environment are restricted
counters/peripherals within the because physical Counters/peripherals are not available to the GDC
CIT environment India development team
« Implement a CIT UK team where this capability could be established.
« Implement full testing in the SV&l environment.
5 I Review and agree options to «Early testing of change is not always possible in the CIT environment
enhance the scope of the CIT because it lacks solution capability e.g. no TWS schedules run,
solution capability environment is not “managed”, not all end points are simulated.
« Fund a review to determine what this better capability looks like.
e Implement full testing in the SV&l environment.
6 I Review viability and value in * Ifany non-functional testing is required, there is no dedicated
creating a test environment able environment scaled to a LIVE level.
to truly simulate LIVE levels of I ¢ — Build/support a dedicated replica of the non-functional
activity test environment.
* Scale up SV&l on a change by change basis.
7 I Review SV&l to identify options I This is a topic of regular discussion between POL and Fujitsu
for less conflict of use and « There are multiple user groups using the SV&I environment and we
hence higher throughput of have seen instances of project timescales being impacted due to
testing clashes with maintenance schedules (downtime).
¢ — Build an environment reflective of the needs of the current non-Fujitsu
user groups.
8 Expand scope of Counter * Some sets of functionality are not regression tested e.g. APADC, APOP
regression testing to always due to dependency on Atos.
include Atos Atos is not regularly involved in the Counter regression test phase.
* POL to engage with Atos for every release to confirm whether there is
scope for additional effort.
9 Review the approach to « A specific defect retesting cycle is not always included when planning
impacting/testing to include how testing in SV&I. This is done partly to improve the view of the downtime
defects are managed in terms required to maximise throughput through the environment.
© Copyright Fujitsu 2021
FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS PageNo: 30 of 32
re)
FUJITSU
FUJ00232714
FUJ00232714
TESTING & QA REPORT .
FUJITSU CONFIDENTIAL
of time and effort
+ Anad hoc approach to delivering fixes is used. This impacts the ability
to plan remaining test tasks.
applicable findings from the
audit they commissioned with
EDGE at the start of 2019
10 I Update the roles and © Clarify the roles and responsibilities of POL third parties who directly
responsibilities of all involved impact Fujitsu — e.g. Atos/Computacenter.
parties * Knowledge sharing from Atos/Computacenter/Verizon/POL Model
Office.
11 I Review the number/variety of Ie The number/variety of Branch configurations (including Global
Branch configurations Office/CTO and the different languages) are not available on a
(including Global Office/CTO consistent basis.
and the different languages) to I ¢ — Fujitsu and POL to engage with POL and POL third parties -
determine what the availability Computacenter/Verizon.
requirements are
42 I Investigate options with * GlobalPayments UK do not supply “Test” files for Fujitsu processing
GlobalPayments UK to supply This became a caveat for 20.92 and at present will not change for
"Test" files for Fujitsu 20.94
processing « POL should engage GlobalPayments so that this gap is filled.
13 I Enhance Fujitsu visibility of «There are potentially changes to Counter functionality made via the
Atos REFDATA testing and BAU REFDATA process (by Atos) which Fujitsu do not have sight of
output results which affect the maintenance of existing tests. (An example would be
APADC scripts).
14 I Review and improve interaction I ¢ Review interaction with Model Office. Although Model Office / LIVE
with Model Office Proving is a POL responsibility, this activity has "dwindled" to the point
of being invisible as part of a delivery process.
* Task(s) added to plans for engaging Model Office through the "journey"
and sharing information about the changes to determine whether a
LIVE proving task would add value.
15 I Review the OBC19 process and I « _OBC19 does not currently apply for PODG route definitions going into
the delivery of route definitions test environments, leaving the potential for gaps in test capability.
and their other configurable « It is already agreed that any PODG route associated with a change will
items to ensure appropriate test form part of projects going forward and NOT rely on the OBC19
coverage process.
16 I Clarify the "boundary" to Fujitsu [Ie Clarify the "boundary" to Fujitsu testing. The SV&l / LST environments
testing specifically relating to use a Counter estate made up of Computacenter Counter hardware
the Counter estate hardware (including peripherals) and have tested with this setup for many years.
(including peripherals) However, that has built an expectation that Fujitsu have the latest
versions of hardware and that that is what is tested which leads to
confusion.
« A formal RACI should be created to determine the scope of testing for
Fujitsu highlighting potential gaps.
17 I Conclude the Service Work is already underway and being tracked in weekly meetings.
Improvement activity to produce
a full list of the transactions
Fujitsu is able to test
18 I POL to input to the list with the I ¢ Duplication is likely but maintaining a consolidated list would be
preferred.
© Copyright Fujitsu 2021
FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4166
VersionI 1.0
UNCONTROLLED WHEN PRINTED OR Date: 29-Jan-2021
STORED OUTSIDE DIMENSIONS PageNo: 31 of 32