POL00030528
POL00030528
foe] THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
Document Title: THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
Document Reference: COM/MGT/REP/4169
CP/CWO Reference: Not applicable
Abstract: A report to Post Office Ltd describing the 29 Bugs, Errors and
Defects as identified by Fraser J.
Document Status: APPROVED
Author & Dept: Fujitsu
External Distribution: Restricted. See section titled Information Distribution.
Information Classification: See section 0.7
Approval Authorities:
Name Role Date
Fujitsu Horizon Audit Team (POA) I See Dimensions for record
oxer ji
opyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COMI/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 1 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
0 Document Control
0.1 Table of Contents
0 DOCUMENT CONTROL..
0.1 Table of Contents...
0.2 Review Details...
0.3 Associated Documents (Internal & External)
0.4 Abbreviations
0.5 Glossary
0.6 Accuracy
0.7 Informatio
1 EXECUTIVE SUMMARY 10
2 PURPOSE & SCOPE 10
3 BACKGROUND AND INTRODUCTION.............0+ 10
4 TERMINOLOGY... 12
5 THE 29 BEDS 13
5.1 Receipts and Payments Mismatch bug .
5.1.1 Horizon System Affected
5.1.2 Description .
5.1.3 Dates...
5.1.4 What happened.
5.1.5 How it was spottec
5.1.6 Howit was actioned.
5.1.7 Impact ........
5.2. Callendar Square/Falkirk bug
5.2.1 Horizon System Affected
5.2.2 Description .
5.2.3 Dates...
5.2.4 What happened..
5.2.5 How it was spotted
5.2.6 How it was actioned.
5.2.7 Impact... we
5.3. Suspense Account bug
5.3.1 Horizon System Affected
5.3.2 Description .
5.3.3 Dates...
5.3.4 What happened.
5.3.5 How it was spotted
5.3.6 How it was actioned.
5.3.7 Impact .
5.4 Dalmellington bug/Branch Outreach Issu 18
5.4.1 Horizon System Affected 18
5.4.2 Description . 18
5.4.3 Dates...
5.4.4 What happened...
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 20f 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.4.5 How it was spotted
5.4.6 How it was actioned .
5.4.7 Impact...
5.5 Remming In bug
5.5.1 Horizon System Affected
5.5.2 Description .
5.5.3 Dates...
5.5.4 What happened.
5.5.5 How it was spotte
5.5.6 How it was actioned .
55.7 Impact ........
5.6 Remming Out bug
5.6.1 Horizon System Affected
5.6.2 Description .
5.6.3 Dates...
5.6.4 What happened..
5.6.5 Howit was spotted
5.6.6 How it was actioned .
5.6.7 Impact.........
5.7 Local Suspense Account issue
5.7.1 Horizon System Affected
5.7.2 Description .
5.7.3 Dates...
5.7.4 What happened.
5.7.5 How it was spotted
5.7.6 How it was actioned .
5.7.7 Impact .
5.8 Recovery Issues.
5.8.1 Horizon System Affected
5.8.2 Description .
5.8.3 Dates...
5.8.4 What happened..
5.8.5 How it was spotte
5.8.6 How it was actioned .
5.8.7 Impact .
5.9 Reversals
5.9.1 Horizon Sy:
5.9.2 Description
5.9.3 Dates...... : 28
5.9.4 What happened.. 28
5.9.5 Howit was spotted 28
5.9.6 How it was actioned . 28
5.9.7 Impact .
5.10 Data Tree Bi Failure discrepancies
5.10.1 Horizon System Affected ...
5.10.2 Description .
5.10.3 Dates...
5.10.4 What happened..
5.10.5 How it was spotted
5.10.6 How it was actioned
5.10.7 Impact
Girobank discrepancies
1 Horizon System Affected
2 Description .
3 Dates
.4 What happened.
5.11.5 How it was spotted
5.11.6 How it was actioned .
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 3 of 80
POL00030528
POL00030528
THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
o
FUJITSU FUJITSU CONFIDENTIAL
5.11.7
5.12
5.12.1
5.12.2
5.12.3
5.12.4
5.12.5
5.12.6
5.12.7
5.13
5.13.1
5.13.2
5.13.3
5.13.4
5.13.5
5.13.6
5.13.7
5.14
5.14.1
5.14.2
5.14.3
5.14.4
5.14.5
5.14.6
5.14.7
5.15
5.15.1
5.15.2
5.15.3
5.15.4
5.15.5
5.15.6
5.15.7
5.16
5.16.1
5.16.2
5.16.3
5.16.4
5.16.5
5.16.6
5.16.7
5.17
5.17.1
5.17.2
5.17.3
5.17.4
5.17.5
5.17.6
5.17.7
5.18
5.18.1
5.18.2
5.18.3
5.18.4
5.18.5
5.18.6
5.18.7
5.19
Post & Go/TA discrepanci
Impact...
Counter-replacement issues ..
Horizon System Affected ...
Description .
Dates...
What happened.
How it was spotted
How it was actioned .
Impact...
Withdrawn stock discrepancies
Horizon System Affected
Description .....
Dates...
What happened.
How it was spotted
How it was actioned .
Description ..
Dates...
What happened.
How it was spott
How it was actioned .
Impact...
Phantom Transactions..
Horizon System Affected
Description .
Dates...
What happened..
How it was spotted
How it was actioned .
Impact .
Reconciliation issues
Horizon System Affected
Description .
Dates...
What happened......
How it was spotted...
How it was actioned .
Impact .
Branch Customer discrepancies ..
Horizon System Affected
Description .
Dates...
What happened..
How it was spottec
How it was actioned .
Impact...
Concurrent login:
Horizon System
Description .
Dates...
What happened.
How it was spotted
How it was actioned .
Impact ........
in POLSAP
© Copyright Fujitsu 2021 Ref: COM/MGT/REP/4169
FUJITSU CONFIDENTIAL
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 4 of 80
POL00030528
POL00030528
THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.19.1 Horizon System Affected
5.19.2 Description .
5.19.3 Dates...
5.19.4 What happened..
5.19.5 How it was spotte
5.19.6 How it was actioned .
5.19.7 Impact...
5.20 Recovery Failure:
5.20.1 Horizon System
5.20.2 Description .
5.20.3 Dates...
5.20.4 What happened.
5.20.5 How it was spotted
5.20.6 How it was actioned .
5.20.7 Impact.
5.21 Transaction Correct n Issue:
5.21.1 Horizon System Affected ...
5.21.2 Description .
5.21.3 Dates...
5.21.4. What happened..
5.21.5 How it was spotted
5.21.6 How it was actioned .
5.21.7. Impact...
5.22 Bugs/errors/defects introduced by previously applied Peak fixes ..
5.22.1 Horizon System Affected
5.22.2 Description.
5.22.3 Dates
5.22.4 What happened..
5.22.5 How it was spotted
5.22.6 How it was actioned .
5.22.7 Impact... .
5.23 Bureau de change.
5.23.1 Horizon System Affected 66
5.23.2 Description .
5.23.3 Dates......
5.23.4 What happened..
5.23.5 How it was spotte
5.23.6 How it was actioned
5.23.7 Impact...
5.24 Wrong branch customer change
5.24.1 Horizon System Affected
5.24.2 Description .
5.24.3 Dates...
5.24.4 What happened..
5.24.5 How it was spotted
5.24.6 How it was actioned .
5.24.7. Impact...
5.25 Lyca Top Up..
5.25.1 Horizon System Affected
5.25.2 Description.
5.25.3 Dates......
5.25.4 What happened..
5.25.5 How it was spotted
5.25.6 How it was actioned .
5.25.7 Impact.
5.26 TPSC 250 Report.
5.26.1 Horizon System Affected
5.26.2 Description ..
© Copyright Fujitsu 2021
pyrig 1) FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 5 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.26.3 Dates...
5.26.4 What happened.
5.26.5 How it was spotted
5.26.6 How it was actioned.
5.26.7 Impact.
5.27 TPS...
5.27.1 Horizon System Affected
5.27.2 Description .
5.27.3 Dates...
5.27.4 What happened..
5.27.5 How it was spotted
5.27.6 How it was actioned .
5.27.7 Impact.
5.28 Drop & Go.
5.28.1 Horizon System Affected
5.28.2 Description .
5.28.3 Dates...
5.28.4 What happened.
5.28.5 How it was spottec
5.28.6 How it was actioned.
5.28.7 Impact...
5.29 Network Banking
5.29.1 Horizon System
5.29.2 Description .
5.29.3 Dates...
5.29.4 What happened.
5.29.5 How it was spotted
5.29.6 How it was actioned.
5.29.7 Impact....
6 INFORMATION DISTRIBUTION Td
©C ight Fujitsu 2021
‘opyright Fujitsu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 6 of 80
POL00030528
POL00030528
eo THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
0.2 Review Details
Issued for Information
Position/Role Name
Horizon Audit Team Fujitsu
0.3 Associated Documents (internal & External)
Reference Version Date Title Source
Judgment Appendix 2 Bugs, Errors, Defects. I
0.4 Abbreviations
Definition
ADSL Asymmetric Digital Subscriber Line
AP-ADC Automated Payment-Advanced Data Capture
APS Automated Payment Service
BAL Branch Access Layer
BAU Business As Usual
BEDs Bugs, Errors and Defects, as documented in Appendix 2 of the Horizon Issues
Judgment handed down by Mr Justice Fraser on 16 December 2019 in the Post
Office Group Litigation.
BIMS Business Incident Management System
BRDB Branch Database
BTS Branch Trading Statement
CAP Cash Accounting Period
cDP Common Digital Platform, hosted by Accenture
CoB Close Of Business
cts Client Transaction Summary
FAD Finance Accounting Division. Unique identifier allocated by POL to individual Post
Office branches
FI Financial institution
FSM Fujitsu Service Management (an informal abbreviation)
HNG-X The Business Capabilities and Support Facilities provided to POL by Fujitsu following
CCN1200
HSD Fujitsu Horizon Service Desk
HSH Fujtsu Horizon System Helpdesk. Service Desk provided by Fujitsu until 02-Jul-
ISDN Integrated Services Digital Network
KEL Known Error Log (Knowledge Base Article repository)
MAC Major Account Controllers (Fujitsu team that receive SPM Incident calls from POL, or
previously Atos)
Msu Management Support Unit (Fujitsu team that analysed Reconciliation reports and
issued BIMS during Legacy Horizon)
NBSC POL New Business Support Centre
© Copyright Fujitsu 2021 FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 7 of 80
oO
FUJITSU
POL00030528
POL00030528
THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU CONFIDENTIAL
Abbreviation
Definition
OBCS Order Book Control Service
OLS Online Service
OPS Office Platform System
Peak Not an acronym; the name of a Fujitsu Incident and Release Management database
PinICL The predecessor system to Peak
PG Post & Go
POL Post Office Limited
POLFS POL Finance System
POLSAP POL SAP system
PON Post Office Network
P&BA Product and Branch Accounting. The department within POL responsible for back
end accounting functions
RDT Reference Data Team (Fujitsu)
Rem, Rem in, Rem out,
Remmed in, Remmed out,
Remming
Post Office shorthand for “remitting”. See also entries for Remittance, Remming in
and Remming out in Glossary.
RNM Regional Network Manager
sco Single Counter Office
SMC Systems Management Centre (Fujitsu Systems Management Service)
SPM Subpostmaster
ssc Software Support Centre (Fujitsu Third Line Support Service). Referred to as “Fujitsu
ssc’
SU Stock Unit
To POL Transaction Correction
TIP Transaction Information Processing system
TP Trading Period
TPS Transaction Processing System
0.5 Glossary
Term
Automated Payment-
Advanced Data Capture
(AP-ADC)
Det
Allows a script defined in reference data, typically written by POL or its agents, to
define a series of steps to execute on the counter in order to perform a transaction.
Each step uses a Data Type (the equivalent of a function call) and associated
parameters to define the processing performed by the step
Branch Access Layer
(BAL)
The interface between Counters and the Data Centre — a set of Web Service end
points. The servers in the HNG-X Data Centre that provide the service end-points
accessed by the Counter. The BAL is responsible for authentication and
authorisation of service requests from outside the Data Centre.
Business Incident
Management System
(BIMS) report
The Business Incident Management System (BIMS) reports the progress towards the
resolution of a Business Incident to allow Post Office Ltd. to complete an accurate
reconciliation or settlement with their clients.
Client Transaction
Summary (CTS)
Report delivered each day by Fujitsu to POL, via PODG. The Client Transmission
Summaries are produced automatically from the Horizon host systems, and
represent summaries of the data sent to the AP Clients.
© Copyright Fujitsu 2021
FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 8 of 80
oO
FUJITSU
POL00030528
POL00030528
THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU CONFIDENTIAL
Term
Definition
Energis
A third party network services provider.
Error Notice
The method used by POL to correct branch accounts, prior to the Transaction
Correction method.
Escher
Escher Group Ltd, 260 Franklin Street, Suite 400, Boston, USA, MA 02110. The
third party provider of the Riposte application software used in Legacy Horizon
Horizon Issues Judgment
“Horizon Issues Judgment’ refers to the judgment (Judgment (No. 6) Horizon issues)
handed down by Fraser J on 16 December 2019 with neutral citation number: [2019]
EWHC 3408 (QB)
Horizon Online
The HNG-X software application.
Legacy Horizon
The Horizon application prior to HNG-X.
Lyca
A mobile phone network operator. Lyca pay-as-you-go accounts can be topped up in
Post Offices.
Post Office Group
Post Office Group Litigation refers to the long-running legal dispute between a group
Litigation of claimants, who are all either former or serving subpostmasters (SPM), and the
Post Office.
Powerhelp ‘An application used by the Fujitsu helpdesk (HSH) between 1998 and 2007.
Remittance: A consignment to or from a Branch of cash, Stock, or other value items to be brought
to account.
Remming in: To log a remittance of items incoming into the office via a Horizon transaction.
Remming out: To log a remittance of items leaving the office (normally returning stock/cash back to
the Post Office central distribution centres) via a Horizon transaction.
Riposte Proprietary software product from Escher Group that was used on Horizon (pre HNG-
X) to exchange messages between Counters and Data Centres.
Smartpost Name given to the Mails application in the Fujitsu implementation of Escher’s Mails
product, which provided a service to allow mail rates to be looked up for parcels and
letters.
0.6 Accuracy
Fujitsu endeavours to ensure that the information contained in this report is accurate but, while every effort is made
to ensure the accuracy of such information, it accepts no liability for any loss (however caused) sustained as a result
of any error or omission herein.
0.7
Information Classification
The author has assessed the information in this document for risk of disclosure and has assigned an information
classification of FUJITSU CONFIDENTIAL. This report is also subject to the Information Distribution statements in
Section 6.
© Copyright Fujitsu 2021
FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 9 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
1 Executive Summary
On 20 August 2020, POL requested an audit of Horizon by sending a letter to Fujitsu titled “Horizon
Audit”. Following a number of discussions between POL and Fujitsu, it was agreed by POL that Fujitsu
would prepare a set of reports on key topic areas identified by POL.
This report explains how the BEDs identified during the Horizon trial were addressed by Fujitsu (“BED
Report”). It follows the “Expanded Table of Contents for the BED Report” (COM/MGT/REP/4 164) which
was shared with POL on 01 December 2020. It was subsequently agreed that the sections proposed in
COM/MGT/REP/4 164 which were to deal with “Current Processes, Procedures & Controls” would be the
subject of a separate report, “The BED Current Process Report” (COM/MGT/REP/4184). This “BED
Report” therefore covers only the sections listed in COM/MGT/REP/4164 under the heading “Historical —
Response to the ‘29 Bugs”.
Fujitsu operates the Horizon environment to the service standards contractually required by POL. No.
complex IT system, such as Horizon, will ever be completely free of bugs, errors and defects. Fujitsu’s
monitoring systems and processes seek to proactively identify faults, log them as Incidents, and then
work to resolve them promptly following the agreed Incident management processes. Fujitsu also relies
on Incidents being reported by SPMs directly, or by POL. These reported Incidents are also logged and
worked on to identify their cause and resolution options. Thousands of Incidents have been logged since
the inception of Horizon, as would be expected of a system of this complexity and size. Mr Justice Fraser
highlighted 29 bugs, errors and defects in the Horizon Issues Judgment. This report provides information
relating to all 29.
This document is based on limited information available to Fujitsu. Fujitsu was not a party to the Post
Office Group Litigation, and does not have access to evidence or other materials available to the court
and parties in that case. Accordingly, as a non-party to the Post Office Group Litigation, Fujitsu is not in a
position to disagree with Mr. Justice Fraser's findings to the extent that they may be relevant to this
report.
Fujitsu does not have the information which POL and/or the claimants used as the basis for their
submissions to the Court, nor the evidence on which Mr Justice Fraser based his judgment and various
findings, including Appendix 2 of the Horizon Issues Judgment relating to Bugs, Errors and Defects. As a
party to that case, POL has detailed knowledge regarding the matters at issue, having participated in the
entire proceedings, and having had complete access to such evidence and documents. In preparing this
report, Fujitsu has relied on the limited information in its own records.
POL is invited to comment on this report to seek any additional clarifications it needs. Fujitsu will
endeavour to respond to any comments or clarifications requested and may, if it deems necessary,
provide an updated version of this report.
Fujitsu welcomes the opportunity to provide this report.
2 Purpose & Scope
The purpose of this report is to explain the actions taken by Fujitsu to address each of the 29 Bugs,
Errors and Defects (BEDs) contained in Appendix 2 of the Horizon Issues Judgment handed down by Mr
Justice Fraser on 16 December 2019 in the Post Office Group Litigation. This report provides POL with
information to understand how Fujitsu addressed each of the identified BEDs.
POL is invited to comment on this report to seek any additional clarifications it needs. Fujitsu will
endeavour to respond to any comments or clarifications requested and may, if it deems necessary,
provide an updated version of this report.
Fujitsu welcomes the opportunity to provide this report and looks forward to a constructive dialogue with
POL.
3 Background and Introduction
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 10 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
On 20 August 2020, POL requested an audit of Horizon by sending a letter to Fujitsu titled “Horizon
Audit”. Following a number of discussions between POL and Fujitsu, it was agreed by POL that Fujitsu
would prepare a set of reports on key topic areas identified by POL.
This report explains how the BEDs identified during the Horizon trial were addressed by Fujitsu (“BED
Report’). It follows the “Expanded Table of Contents for the BED Report” (COM/MGT/REP/4 164) which
was shared with POL on 01 December 2020. It was subsequently agreed that the sections proposed in
COM/MGT/REP/4164 which were to deal with “Current Processes, Procedures & Controls” would be the
subject of a separate report, “The BED Current Process Report” (COM/MGT/REP/4184). This “BED
Report” therefore covers only the sections listed in COM/MGT/REP/4164 under the heading “Historical —
Response to the ‘29 Bugs”.
Fujitsu operates the Horizon environment to the service standards contractually required by POL. No.
complex IT system, such as Horizon, will ever be completely free of bugs, errors and defects. Fujitsu's
monitoring systems and processes seek to proactively identify faults, log them as Incidents, and then
work to resolve them promptly following the agreed Incident management processes. Fujitsu also relies
on Incidents being reported to them by SPMs directly or by POL. These reported Incidents are also
logged and worked on to identify their cause and resolution options. Thousands of Incidents have been
logged since the inception of Horizon, as would be expected of a system of this complexity and size. Mr
Justice Fraser highlighted 29 bugs, errors and defects in the Horizon Issues Judgment. This report
provides information relating to all 29.
This document is based on limited information available to Fujitsu. Fujitsu was not a party to the Post
Office Group Litigation, and does not have access to evidence or other materials available to the court
and parties in that case. Accordingly, as a non-party to the Post Office Group Litigation, Fujitsu is not ina
position to disagree with Mr. Justice Fraser's findings to the extent that they may be relevant to this
report.
Fujitsu does not have the information which POL and/or the claimants used as the basis for their
submissions to the Court, nor the evidence on which Mr Justice Fraser based his judgment and various
findings, including Appendix 2 of the Horizon Issues Judgment relating to Bugs, Errors and Defects. As a
party to that case, POL has detailed knowledge regarding the matters at issue, having participated in the
entire proceedings, and having had complete access to such evidence and documents. In preparing this
report, Fujitsu has relied on the limited information in its own records.
When Fujitsu became aware of an issue, it would be logged as an Incident and the relevant Fujitsu team
would investigate the Incident to understand the cause and any required actions needed to resolve it.
Fujitsu would seek to determine the sequence of events that had occurred, and then, where appropriate,
would refer the matter back to POL for guidance on the preferred resolution actions. Since the inception
of Horizon in 1999, a huge amount of data and information has been provided to POL through extensive
reporting processes and numerous interactions and meetings, as is normal for a system of this size and
materiality.
As a general comment, it should be noted that Fujitsu is one of many suppliers involved in the overall
delivery of end-to-end services to POL in relation to Horizon. The Horizon application also relies on the
working partnership between POL and its chosen partners — such as Verizon, Computacenter and Atos —
as well as external service providers such as banks and affiliated organisations. This applies to both the
IT systems and the operational processes in Horizon.
Although every effort has been made to avoid confusing jargon in this report, the very nature of this
aspect of the service delivered to POL necessitates the use of many acronyms and phrases that may
need expanding upon to ensure the correct understanding. Fujitsu accepts that further explanation may
be necessary and encourages POL to seek these clarifications.
Fujitsu has endeavoured to ensure that the content of this report is correct as at the date of issue. This
report has been prepared with the input of numerous Fujitsu individuals and attribution of any statements
made in this report should be made to Fujitsu only. In preparing this report, the authors have collectively
characterised and summarised many internal Fujitsu documents. They have also described processes
and procedures which have been established over many years and may not be in written form. Many of
the documents, processes and procedures described in this report are continuously updated and Fujitsu
reserves the right to make changes to the way it works in the ordinary course of its operations and
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 11 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
business without obligation to update this document. POL should verify the position with Fujitsu before
relying upon any information or content from this document in the future.
4 Terminology
The names used in this Report to describe each of the 29 BEDs are those used by Mr Justice Fraser in
Appendix 2 of the Horizon Issues Judgment: Summary of Bugs, Errors, Defects.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 12 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5 The 29 BEDs
5.1 Receipts and Payments Mismatch bug
5.1.1 Horizon System Affected
Horizon Online.
5.1.2 Description
The bug caused a problem with balancing, resulting in a mismatch between the Payments and Receipts
which meant they did not match correctly when moving discrepancies into Local Suspense, which
resulted in discrepancies being ‘lost’.
5.1.3 Dates
This bug occurred in 2010, with the majority of Incidents recorded as occurring between August and
October 2010.
5.1.4 What happened
Branches must rollover into a new Trading Period (TP) every month, and may not rollover with an
unresolved discrepancy. If discrepancies were found when rolling over a stock unit (SU), Horizon
prompted the user to decide to either:
a) move the discrepancies into the Suspense Account (which aggregated all discrepancies into a
single gain or loss for a TP, and which then allowed rollover to complete), or,
b) cancel the rollover.
The bug however allowed SPMs to rollover with an unresolved discrepancy. The problem only arose in a
branch where all the following conditions were true:
e The branch had an unresolved discrepancy, and,
e The branch cancelled (option b above) the completion of the (TP), i.e. did not transfer the
discrepancy to the Suspense Account (option a above), and,
« Within the same session the branch then continued to rollover to a new Balance or TP.
A bug in the code meant that when Cancel was pressed (option b) the discrepancy was cleared, and as it
was not moved into the Suspense Account, an accounting error occurred. If the user did not check the
Final Balance Report then they may not have been aware that there was now a receipts and payments
mismatch, although the discrepancy would re-appear and be recorded on the next TP.
5.1.5 How it was spotted
Some branches reported to NBSC that they had received the following error message “MSG31318 Office
balancing error there is a non zero trading position on rollover of branch by user WMC002 to trading
period 8”. The Incident was passed to Fujitsu, and Fujitsu investigated to identify which branches had
been affected.
5.1.6 How it was actioned
Fujitsu corrected the bug using a code fix so that discrepancies were no longer ‘lost’.
This was done in two parts:
1. Acorrection to reference data was applied on 13 October 2010.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 13 of 80
POL00030528
POL00030528
THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU CONFIDENTIAL
oO
FUJITSU
2. The reference data correction was effective once counters received the 02.12 counter upgrade
which completed roll out to all branches on 18-Oct-2010.
The software issue that caused the discrepancy was monitored (as in PC0204263) and all instances
were reported to the POL Duty Manager.
Fujitsu identified and advised POL of the branches affected, and the amounts of each mismatch.
5.1.7 Impact
Fujitsu identified that 62 branches were impacted by a receipts and payments mismatch which
corresponded to the value of the ‘lost’ discrepancies. The total loss across the 62 branches, of which
Fujitsu is aware, was £9,029.22, with some branches experiencing gains.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time.
© Copyright Fujitsu 2021
FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 14 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.2 Callendar Square/Falkirk bug
5.2.1 Horizon System Affected
Legacy Horizon
5.2.2. Description
A software bug in Riposte could cause Horizon to not recognise transfers between different SUs so that
information recorded on the sending terminal was not passed to the receiving terminal. If the SPM re-
entered the transfer on a different terminal then the transfer was registered twice resulting in a
discrepancy in the branch accounts.
5.2.3 Dates
2000 to 2010.
According to the findings of Justice Fraser in the Horizon Issues Judgment, the effect of this bug
continued till 2010. In Appendix 2 of the Horizon Issues Judgement, Justice Fraser states: “It is agreed
that this bug occurred between the years of 2000 and 2006, although there is an issue about when it
stopped. In my judgment, the period when the effects of this occurred are 2000 to 2010.” See also,
paragraphs 149 — 150 of the Technical Appendix to the Horizon Issues Judgement.
As anon-party to the Post Office Group Litigation, Fujitsu is not in a position to disagree with Mr. Justice
Fraser's findings. Fujitsu does not have the information which POL and/or the claimants used as the
basis for their submissions to the Court, nor the evidence on which Mr Justice Fraser based his judgment
and various findings, including Appendix 2 of the Horizon Issues Judgment and the discussion of the
Callendar Square Bug. As a party to that case, POL has detailed knowledge regarding the matters at
issue, having participated in the entire proceedings, and having had complete access to such evidence
and documents.
5.2.4 What happened
A bug in the Riposte software sometimes had the effect of preventing a counter from writing messages,
either those being replicated to it, or those generated on that counter.
1. When the SPM completed the ‘Transfer In' on one counter, the confirmation message indicating that
this had been processed was not replicated to the other counters.
2. The problem was not always immediately obvious to the SPM / counter user, and could result in them
attempting to re-enter the transfers which they believed they had done but could see were missing.
Since the transfer had not been replicated, the SPM was then able to repeat the same ‘Transfer In’.
4. When the counter was re-started, all repeat instances of the Transfer In would become visible which
could cause errors in the accounts.
5. Attempting to balance the branch when a counter was in this state could also result in errors.
In the Callendar Square case, the issue was that the recording of the cash being transferred into the SU
was not being made visible to other terminals, and so it was possible to repeat the Transfer In on another
terminal. The Riposte issue would mean that the terminal on which the first Transfer In was recorded
was no longer operable until it was restarted.
5.2.5 How it was spotted
The KEL JBallantyne5245K records that an error message produced as a result of this bug was reported
by an SPM in 2000, and individual occurrences of the problems arising from the bug, as described in
5.2.4, were sometimes apparent to SPMs and reported to the Fujitsu helpdesk (HSH) and POL helpdesk
(NBSC).
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 15 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
Instances of duplicate ‘Transfers In' with no corresponding ‘Transfer Out’ between counters were
automatically detected by Fujitsu via a nightly batch process which produced a report, TPSC256. Fujitsu
checked this report each morning and raised a Peak for any exceptions noted. Errors were reported to
POL via the Business Incident Management System (BIMS) mechanism.
KEL JBallantyne5245K additionally records that events that resulted from the Riposte bug were also
detected by Fujitsu Systems Management Centre (SMC) monitoring, and it noted that if the event was
detected during working hours, then SPMs should be contacted and advised to reboot.
The Callendar Square occurrence was reported by the SPM.
5.2.6 How it was actioned
During the period 2000-2005, manifestations of this Riposte bug were dealt with in the following ways:
1) Ifan SPM noticed and reported an error with the counter, or if events triggered by the bug were
detected during the same day by Fujitsu SMC monitoring, SPMs were advised to restart the
counter which would fix the issue, and in any case the issue with the counter would clear itself
overnight.
2) Instances in which a duplicate Transfer In had taken place (as happened at Callendar Square)
would be identified by Fujitsu's BAU monitoring of the daily automated “Host Detected Cash
Account Lines Comparisons Report” TPSC256 as a receipts and payments mismatch. This
would result in a Peak being raised and the problem investigated. It would then be reported to
POL using the BIMS process, so that they could contact the branch to resolve the mismatch.
Such instances would also appear in the daily automated “Host Detected Cash Account Line
Mismatches Report” TPSC268.
At Release S80 in August 2005, Horizon moved from using a one week Cash Account Period (CAP) to a
monthly Trading Period (TP). During the migration to S80, report TPSC268 was discontinued at the point
of data centre migration, followed by TPSC256 after the switch to TP. It is possible that the duplicate
Transfers In may have been detectable by POL through their POLFS system, of which Fujitsu had no
visibility.
After the reporting and investigation of the issue at Callendar Square in September 2005, Escher
provided a fix which Fujitsu included as part of the S90 release in February and March 2006.
5.2.7. Impact
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on Fujitsu monitoring, and/or on SPMs reporting the issue when it arose.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 16 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.3 Suspense Account bug
5.3.1 Horizon System Affected
Horizon Online.
5.3.2 Description
Historic data from 2010 reappeared in some branches' accounts for the same monthly TP in 2011 and
2012. SPMs then re-settled those same entries to clear their Local Suspense accounts in 2011 and 2012
despite having settled them in 2010. Some branches benefitted from the same gain three times or
suffered from the same loss three times.
5.3.3 Dates
The effects of this bug were between 2011 and 2013. Although branch account data from 2010 was
involved, the bug arose in 2011.
5.3.4 What happened
The issue was caused by a bug in the changes made to the archiving strategy relating to SUs on 3 July
2011. Some records associated with deleted SUs that should have been archived were ignored by the
archiving process, leading to those records becoming visible in the same monthly TPs in 2011 and 2012.
As a result, some SPMs re-settled incorrect entries in order to clear their Local Suspense accounts,
despite those entries already having been settled in 2010.
14 branches were identified by Fujitsu Software Support Centre (Fujitsu SSC) as having old erroneous
data in their accounts, which had an impact on Suspense Accounts.
19 additional branches were identified as having old data, although in these cases, and so far as Fujitsu
was aware, its investigations identified that the old data did not affect the Suspense Accounts, and was
unlikely to have an ongoing impact on branch accounts.
In affected Branch Trading Statements, it was found that the sum of the two ‘Discrepancy Transferred’
lines did not match the total of the two ‘Discrepancy Resolved' lines.
In affected Suspense Account reports, it was found that the 'B/Fwd' figure on the report did not match the
‘C/Fwd' figure on the report from the previous TP.
5.3.5 How it was spotted
An SPM raised an Incident with the POL helpdesk (NBSC) in February 2013 and NBSC passed the issue
to Fujitsu SSC for investigation. Fujitsu determined a method of detecting affected branches and notified
POL so that POL could contact the 14 branches identified.
5.3.6 How it was actioned
The old data that kept reappearing was deleted from the Branch Database (BRDB).
A software fix was released in October 2013 so that if the same conditions arose again, affecting the BTS
or the Local Suspense account as described above in section 5.3.4, the SPM would be alerted of the
discrepancy.
5.3.7 Impact
Of the 14 branches at which Local Suspense accounts were affected, Fujitsu understands that four were
Crown and ten were SPM branches. Fujitsu understands that five experienced losses, seven gains, and
two both a loss and a gain.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 17 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.4 Dalmellington bug/Branch Outreach Issue
5.4.1 Horizon System Affected
Horizon Online.
5.4.2 Description
This affected core and outreach branches only. The bug allowed an SPM to inadvertently Rem In cash to
the outreach branch multiple times using the same barcode, so that a multiple of the actual cash transfer
would be recorded at the outreach branch, causing a discrepancy.
5.4.3 Dates
2010 to 2016.
5.4.4 What happened
The effects of this bug arose only as a result of particular events, such as at Dalmellington:
1. The user logged into Horizon to make a cash declaration; following this the SU timed out and
logged off due to inactivity.
2. The user logged back in to the SU and carried out the remittance delivery transaction (i.e. the
Rem out to the outreach branch). Two delivery slips were printed, the user pressed Enter, which
then printed the Rem In slip.
3. Instead of the Remittances and Transfers Home screen being displayed, the Pouch Delivery
Screen continued to display, with the Enter button enabled.
4. The user then pressed Enter three times, with the result that as well as the £8,000 originally
Remmed out of the core branch, an additional £24,000 (3 x £8,000) was Remmed out.
As a result the outreach branch showed a £24,000 discrepancy; the Dalmellington core branch showed
no discrepancy.
On investigation it was found that the ability to Rem in duplicate barcodes had existed since Release 1 of
HNG-X in 2010 and that other branches had been affected (see below in 5.4.5).
5.4.5 How it was spotted
The bug was identified following the SPM at Dalmellington noticing the discrepancy and reporting it to
POL helpdesk (NBSC). POL then reported it to Fujitsu. The discrepancy was visible to the user as
separate receipts were printed for each transaction, and the additional Rem outs would have been listed
in the transaction log reports available to the user.
Fujitsu's investigation identified 112 instances at 88 branches of duplicate pouch barcode IDs being
Remmed in, of which 47 were originally identified as being related to the particular Branch Outreach bug
identified at Dalmellington. 43 of those 47 instances were known to have been corrected either by POL
issuing Transaction Corrections (TC), or by the SPM reversing the duplicate Rem In. Of the four
remaining, two were for small values, £0.01 and £1 respectively, and two were for larger amounts,
£25,000 and £2,500. These larger amounts occurred at branches that did not have outreach branches.
POL stated that in the case of these two branches, there were no shortages or TCs recorded, and no
contact from those branches during the relevant period.
5.4.6 How it was actioned
A software update was released in January 2016 to prevent users of outreach branches from being able
to Rem in the same barcode multiple times.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 18 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.4.7. Impact
In the case of the Dalmellington branch, Fujitsu understands that POL allocated the affected outreach
branch a customer account and then issued a TC for £24,000.
Fujitsu informed POL of the other affected branches and the extent of the discrepancies, but is unaware
of the details of any actions taken by POL as a result of that information.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 19 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.5 Remming In bug
This term covers two separate bugs, although both related to Remming in cash, and both resulted in a
cash pouch being recorded twice in error.
5.5.1 Horizon System Affected
Horizon Online.
5.5.2 Description
5.5.2.1 Issue 1
When Remming in cash pouches, an SPM was able to Rem in the same pouch more than once by
scanning the same barcode. The scenario is described in 5.5.4.1.
5.5.2.2 Issue 2
When Remming in cash pouches, an SPM was able to Rem in the same pouch more than once by
scanning the same barcode. The scenario is described in 5.5.4.2.
5.5.3 Dates
Issue 1: July 2010.
Issue 2: February 2010.
5.5.4 What happened
5.5.4.1 Issue 1
The system was designed to check if a barcode had already been scanned, and to reject it if it an attempt
was made to enter it again. However, the barcode would only be added to the Remmed in list once the
Rem process was fully complete.
In this instance, the SPM scanned a pouch on counter 1, but did not complete the Rem in process before
moving to counter 2, logging in with different credentials, and then scanning the same barcode again.
The Rem in process completed on counter 2, but the SPM then returned to counter 1 and completed the
incomplete Rem (of the same barcode) on counter 1. Because the barcode had already been scanned
but not cancelled on counter 1, it did not need to be scanned again, and it was possible to complete the
Rem in. Had the SPM cancelled and restarted the process on counter 1, Horizon would have prevented
this second Rem from the same barcode, as it would already have been added to the list of scanned
barcodes as a result of the completed Rem on counter 2.
As a result the same Rem in was recorded twice, causing two lots of the same cash to be added to the
branch accounts, creating a shortfall
5.5.4.2 Issue 2
The SPM started Remming in a pouch by scanning its barcode, but then pressed "PREV" which moved
back one screen, and the SPM then scanned the same barcode again. Horizon Online only printed one
receipt which would make it appear to the SPM that there had been only one Rem in.
The duplicate Rem would have been recorded in the branch transaction records, resulting in a shortfall.
It was also possible to return to the barcode scan screen by pressing “PREV” on the print screen when
attempting to reprint the receipt, and this would also present an opportunity for the same barcode to be
scanned again.
5.5.5 Howit was spotted
© Copyright Fujitsu 2021
pyrig 1) FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 20 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
Both of these issues were visible to the two SPMs who reported them to the POL helpdesk (NBSC).
Fujitsu then created a report to detect instances of same issue.
5.5.6 How it was actioned
5.5.6.1 Issue 1
A software update was released on 23-Jan-2011 which modified the process so that a barcode would be
added to a temporary check list immediately after being scanned, rather than it only happening once the
process had completed.
5.5.6.2 Issue 2
A software update in April 2010 disabled the “PREV” button at the appropriate point in the Rem in
process, eliminating the risk of SPMs moving back to the barcode scan screen, and so being able to
duplicate the scan.
While implementing the fix, Fujitsu created a report to monitor for further occurrences of the same
problem.
It is possible that the remittance errors were detectable by POL through POLSAP Reconciliation
processes, but Fujitsu had no visibility of this process, and the monitoring that was put in place was
implemented by Fujitsu using Horizon Online.
5.5.7 Impact
5.5.7.1 Issue 1
Fujitsu understands there were 46 recorded instances of this bug. The duplicate Rem would become
part of the branch accounts, causing a shortfall. As SPMs are not able to reverse a completed Rem in,
Fujitsu understands that a TC, issued by POL, would be needed in each case.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose.
5.5.7.2 Issue 2
Fujitsu understands there were 19 recorded instances of this bug. The duplicate Rem would become
part of the branch accounts, causing a shortfall. As SPMs are not able to reverse a completed Rem in,
Fujitsu understands that a TC, issued by POL, would be needed in each case.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 21 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.6 Remming Out bug
This term covers two issues included in Mr Coyne’s report. Although both relate to Remming out cash,
they have different causes and symptoms.
5.6.1 Horizon System Affected
Legacy Horizon.
5.6.2 Description
There were two separate bugs reported by Mr Coyne, and identified by Fraser J as Bug 6(i) and Bug 6(ii).
5.6.2.1 Issue 1
Identified by Fraser J as Bug 6(i).
On a Monday following a weekend counter release (T30 INC1), branches reported imbalances when
Remming out cash or stock. The imbalances occurred when Remming out multiple identical items
separately, instead of Remming them out as a single batch using the quantity button.
5.6.2.2 Issue 2
Identified by Fraser J as Bug (ii).
While Remming out some coins, an SPM scanned an incorrect pouch barcode. On being prompted to
Cancel or Retry with the correct barcode, the SPM pressed Cancel, but before the reversal operation
completed, the SPM was able to press the Home button, which would normally be inactive, leading to the
remittance being moved to the Suspense Account.
5.6.3 Dates
Issue 1: February 2007.
Issue 2: May 2005.
5.6.4 What happened
5.6.4.1 Issue 1
SPMs Rem out cash to be collected and returned to the POL cash centre. Each bag can hold only one
denomination (£1 coins, £5 notes etc.), and multiple bags are placed in a pouch. The branch recorded in
Horizon the amount in each pouch, by the number of bags, and the value and the denomination in each
bag. Once Remmed out, the cash was removed from the branch cash holdings in Horizon, but was
recorded in a temporary “cash in pouches” line, where it remained until the pouch was physically
collected. The collection team scanned the pouch barcode which removed it from the “cash in pouches”
line in Horizon.
When an SPM had, for example, two bags of 500 x £2 coins, Horizon expected this data to be entered in
a single entry as 2 x 500 x £2 coins, with the quantity button used to indicate 2 bags. As a result of this
bug it became apparent that many SPMs would instead, in this example, enter 1 x 500 x £2 coins, and
then repeat the entry to record the second bag.
The bug led to only one of the two separately entered bags being recorded as leaving the branch cash
holdings, although both bags would be recorded on the “cash in pouches” line. When both bags were
collected, only one bag was recorded as removed from the branch cash holdings, although two bags had
been physically removed.
If the branch spotted the shortfall created in the branch cash holdings and tried to reverse the Rem of one
bag, then the branch cash holdings would be corrected, but the amount would remain in the “cash in
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 22 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
pouches” line within the Suspense Account, something which could not be adjusted by the branch, and a
TC would be required.
5.6.4.2 Issue 2
It should be noted that this issue 2 is not related to issue 1, and the only reason they were grouped
together is due to Mr Coyne mistakenly believing them to be symptoms of the same bug.
The SPM in this case was trying to Rem out £100 of 5p coins, but on scanning the stock pouch barcode,
Horizon displayed the message “Incorrect Pouch Type”, presumably because the barcode was of the
incorrect type to be used for coins. On then being presented with the option to Cancel, or Retry scanning
another barcode, the SPM pressed Cancel, which caused Horizon to begin reversing the Rem out.
During the reversal process, the Home button was briefly displayed and was pressed by the SPM,
resulting in the remittance of £100 being transferred to the Suspense Account, instead of being
cancelled. The £100 now in the Suspense Account had no correlating pouch ID and so could not be
removed.
The underlying issue is that the Home button should have been disabled at all times during this process,
but it is believed (see 5.6.6.2) that the counter on this occasion must have been running unusually slowly
for the Home button to have been active for long enough to be pressed.
5.6.5 How it was spotted
5.6.5.1 Issue 1
Six SPMs noticed the issue and reported it to the POL helpdesk (NBSC). A further 13 instances were
identified from an automated payments mismatch report.
5.6.5.2 Issue 2
This issue was reported by one SPM to the Fujitsu helpdesk (HSH). The Fujitsu helpdesk informed the
POL helpdesk (NBSC), and advised the SPM to contact NBSC.
5.6.6 How it was actioned
5.6.6.1 Issue 1
The T30 INC1 Release was regressed overnight to immediately remove the potential for recurrences.
The code was corrected and released at a later date.
5.6.6.2 Issue 2
Fujitsu attempted to reproduce the error but were unable to access the Home button in similar
circumstances; only by an analysis of code was it concluded that the Home button may have been active,
and it was presumed that it was only displayed for a long enough period to allow it to be pressed because
the counter was running unusually slowly. The Peak (PC0120937) records that a check was made for
any similar Incident and that none was found. KEL GMaxwell3853P was raised in case any other
instances were reported in future.
As the issue had only been reported once and had not been reproducible during investigations, and as it
was deemed that a change to the code may risk introducing further issues, it was decided that no fix
would be made unless further instances of the problem were reported. The information from Peak is that
there were no further reported occurrences to Fujitsu.
The Peak was closed as “Published Known Error” and the closure response is recorded as having been
sent back via the Fujitsu helpdesk (HSH) to the POL helpdesk (NBSC) on 14-Jun-2005.
5.6.7 Impact
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 23 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.6.7.1. Issue 1
As at the week of the release and regression, Fujitsu understands 49 branches had been affected.
Fujitsu understands that cash correction of those affected branches was completed the following week.
A further one branch was identified as being affected in April, on a counter where the regression of the
release in February had failed.
5.6.7.2 Issue 2
Fujitsu understands only one branch was identified as having experienced this issue on one occasion,
and understands that no loss was experienced. Fujitsu understands that POL issued a TC or Error
Notice to fix the accounting issue.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 24 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.7 Local Suspense Account issue
This issue first occurred during the Horizon Online Live Pilot in April 2010, at a time when there were only
60 branches using Horizon Online. There were subsequently reoccurrences of the problem in August
and September 2010.
5.7.1. Horizon System Affected
Horizon Online Live Pilot, and Horizon Online.
5.7.2. Description
When completing a SU rollover, having experienced a discrepancy, SPMs were repeatedly asked how
they wanted to settle the discrepancy and were unable to rollover.
5.7.3 Dates
April to September 2010.
5.7.4 What happened
When undertaking an SU rollover, the normal process is:
SPM clears the Suspense Account and presses 'confirm' to complete the rollover.
SPM returns to the screen asking how the discrepancy is to be cleared.
ens
SPM then selects one of the settlement options to make good the discrepancy.
4. Once the settlement option is selected, the SPM receives confirmation of the SU rollover.
The effect of this bug was that after step 3 above, instead of moving on to step 4, Horizon Online reverted
back to again asking the SPM to select the settlement option.
The underlying problem was that the interface between the counter and the BAL was not working
correctly in certain circumstances; the BAL would send a message to the counter that could not be
understood.
5.7.5 How it was spotted
A number of SPMs reported to the POL helpdesk (NBSC) that they were unable to complete the process
as described in 5.7.4, and that Horizon displayed “System error” messages 'MSG 0783' or 'MSG 0784'.
5.7.6 How it was actioned
Initially a KEL was raised detailing a workaround, which was that SPMs were to cancel the operation and
contact the Fujitsu SSC with certain details. It was recognised that SPMs could potentially rectify the
issue themselves if they were given the correct advice, and Peak PC0204396 records that many did so
successfully.
The underlying bug that caused the intermittent error was corrected by a software release in September
2010.
5.7.7 Impact
Fujitsu understands 95 branches were affected in total: 33 during the pilot in April 2010, and a further 62
in August and September. Fujitsu sent a BIMS report to POL with full details of all the affected branches
that had been unable to resolve the issue.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 25 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.8 Recovery Issues
This term covers two separate issues reported by Mr Coyne, one of which was a bug and occurred during
the Horizon Online Live Pilot, the other being multiple instances of failed recoveries which were
nevertheless detected.
5.8.1. Horizon System Affected
Horizon Online Live Pilot, and Horizon Online.
5.8.2. Description
5.8.2.1 Issue 1
Auser experienced a failed transaction on one SU, then concurrently logged into another SU to invoke
recovery for the failed transaction; it recovered on the correct SU, but the two SUs were in different TPs.
This scenario lead to a loss in one TP, then a gain in the next. This arose during the HNG-X Live Pilot
and was fixed prior to full rollout.
5.8.2.2 Issue 2
Mr Coyne discusses various failed transaction recovery Peaks and KELs, but they are examples of failed
recoveries that were successfully detected via routine monitoring and then notified to POL so that they
could correct the discrepancies.
5.8.3 Dates
Issue 1: April 2010.
Issue 2: 2010 to 2018.
5.8.4 What happened
5.8.4.1 Issue 1
The Incident leading to identification of this bug occurred in a Crown branch on 14-Apr-2010:
1. Aclerk added a banking withdrawal to the basket on counter 9, but did not settle the basket.
2. The same clerk then logged on to counter 10 without logging off counter 9, receiving the
message that if they continue the log in on 10, they would be forced to log out from 9.
3. Counter 9 session was terminated and 3 disconnected session receipts were printed showing
that there was an outstanding session that required recovery.
4. A different clerk then logged in to counter 9 while simultaneously logged on to counter 1, and
invoked the recovery process against that incomplete basket.
5. The basket was recovered but written into an incorrect TP.
It was determined that in this case the effect of this bug occurred due to the sequence of events involving
the clerks being logged in simultaneously on different SUs.
5.8.4.2 Issue 2
Mr Coyne references many different Peaks and KELs which are not necessarily related to each other,
and are not related to the bug in issue 1. The main KEL referred to is Acha959T, which is the KEL that is
to be referred to when a “State 4” failed recovery is identified via automatic monitoring.
Recovery processes are part of Horizon Online and are designed to mitigate against the risk of
transactions failing due to interruptions such as power, network, communication or hardware failures,
which are risks that cannot be completely eliminated. The Peaks that Mr Coyne identifies include
© Copyright Fujitsu 2021
pyrig 1) FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 26 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
instances of the recovery process failing, but as this is also a Known risk that cannot be completely
eliminated, automated failed recovery reports are run each day as part of the Reconciliation Service, to
identify such failures. The reports are checked each day by Fujitsu and the BIMS process is used to
report any failures to POL so that TCs can be issued to branches.
Therefore, although the KEL and the Peaks referred to record instances where recoveries failed, they are
also a record of the Reconciliation Service that Fujitsu provides to POL working correctly to detect,
investigate, and then report to POL such failures, leading to their subsequent successful correction.
5.8.5 How it was spotted
5.8.5.1 Issue 1
The bug was detected after Crown branch staff reported the issue to the POL helpdesk (NBSC) as
described in 5.8.4.1. This Incident was recorded in PC0197769.
The second Peak (PC0198352) was detected by the monitoring put in place by Fujitsu to detect further
instances of the bug following the initial report (PC0197769).
5.8.5.2 Issue 2
The instances of recovery failures cited were detected by Fujitsu’s BAU daily monitoring as part of the
Fujitsu Reconciliation Service.
5.8.6 How it was actioned
5.8.6.1 Issue 1
Initially this issue was mitigated by the implementation of a report to detect other branches being
affected.
A software fix was released in June 2010.
5.8.6.2 Issue 2
The various Incidents covered by issue 2 were subject to the routine detection and correction by the
Reconciliation Service and the BIMS process that is in place to protect against such recovery failures.
As this issue was not an identifiable bug, there was no fix to be made.
5.8.7 Impact
5.8.7.1 Issue 1
In relation to the first Incident (PC0197769), Fujitsu understands there was no loss because the
transactions offset each other.
In the case of the Incident recorded in PC0198352, a BIMS report was issued to POL so for the impact to
be corrected.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose.
5.8.7.2 Issue 2
Fujitsu uses the BIMS process to report any failures to POL so that TCs can be issued to branches as.
necessary.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 27 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.9 Reversals
5.9.1 Horizon System Affected
Legacy Horizon
5.9.2 Description
A software error resulted in an issue with reversals of Remming in transactions. Instead of reversing the
transaction, the amount of the transaction doubled. This bug was introduced as a result of another fix.
5.9.3 Dates
24-Apr-2003 to 02-Jun-2003.
5.9.4 What happened
An SPM was trading on one SU and Remmed in £13,910 of cash, but having failed to move to another
SU to continue trading, attempted to reverse all of the transactions on the first SU. Instead of the balance
returning to zero, the Rem in doubled to £27,820. On calling the POL helpdesk (NBSC), the SPM was
advised to attempt the reversal again, resulting in an error message indicating that first attempt at
reversal had completed successfully.
The issue was caused by a software error that had been introduced as a result of the fix for another Peak
(PC0083954). The bug was that Horizon applied the wrong mathematical symbol when reversing Rem in
transactions, applying the same mathematical operator (+ or -) as the original Rem, instead of the
opposite.
5.9.5 How it was spotted
The SPM who first reported the effects of this bug was able to see the error in the transaction report, and
reported it to POL helpdesk (NBSC) on 24-Apr-2003, who referred it to Fujitsu.
5.9.6 How it was actioned
Fujitsu SSC identified that that the software had incorrectly calculated the reversal value and quantity.
Fujitsu SSC collected the evidence from the counter. Fujitsu SSC passed the Incident to the Fujitsu
Management Support Unit (MSU) team to liaise with NBSC to arrange a correction for branch, and
contacted the SPM to inform them that this was to happen.
Fujitsu SSC created KEL PSteed2847N and then routed the Peak to Fujitsu Fourth Line Support who.
identified the root cause. The Fujitsu Development team made a fast track fix available and passed it to
the Fujitsu Test team.
By May 2003, the fix was initially released to a subset of branches as a Pilot release in the first instance.
A few days later, the fix was applied to 2,178 branches, and rolled out to all branches by 2 June 2003.
The Incident closure was agreed with the SPM who had originally raised it.
5.9.7 Impact
Fujitsu has no record of the total number of branches affected, although the Service Review Book for
April 2003 records that there were four known instances.
POL would have needed to issue Error Notices in order to avoid any long lasting effects on branch
accounts.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 28 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.10 Data Tree Build Failure discrepancies
As identified by Mr Coyne, this term covers two issues, although the second of the two is itself actually
two unrelated issues, one of which occurred in Test only.
5.10.1 Horizon System Affected
Legacy Horizon.
5.10.2 Description
The effects of these issues were that Horizon displayed incorrect information on screen. The data tree
was the means used in Legacy Horizon to construct a record of the transactions within given Balancing
Periods. The underlying problem was one of error handling, in that Legacy Horizon had no means of
creating a warning if any errors occurred during the creation of the data tree.
The Peaks related to this issue relate to errors in constructing the data tree, due either to problems in
reading the relevant transactions from the disks (potentially hardware issues), or in the Reference Data
that was used to define the data tree.
5.10.2.1 Issue 1
An SPM reported a £43,000 discrepancy due to a temporary error (possibly a disk error) causing an
erroneous data tree to be built without any warning. Effectively, the bug is the lack of warning that there
had been an error.
5.10.2.2 Issue 2
Two Peaks were cited by Mr Coyne:
5.10.2.2.1 Issue 2(i) PC0121925
A Test SU on a Fujitsu Test Rig experienced a gain following a cash declaration and rolling over into the
next TP, the discrepancy being the value of transactions performed on the SU after roll over.
5.10.2.2.2 Issue 2(ii) PC0132133
An SPM reported erroneous and inconsistent discrepancies appearing in daily cash report previews.
5.10.3 Dates
Issue 1: November 1999 to late 2000.
Issue 2(i): June to July 2005 (in Test only).
Issue 2(ii): February to January 2008.
5.10.4 What happened
5.10.4.1 Issue 1
In November 1999, after balancing SUs and doing an office snapshot, an SPM found that the snapshot
showed a £43,000 discrepancy which was known to be wrong. The effect of the bug was to cause an
incomplete data tree to be built without any warning being displayed, thereby creating an incorrect office
snapshot. The underlying transaction data was not affected, and if the snapshot had been re-run it may
have produced a correct snapshot.
The SPM followed the normal procedure in rolling over and made good any discrepancies in the process.
However, since this roll over contained incorrect data (as a result of the incorrect office snapshot), this
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 29 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
meant that a false discrepancy was accepted into the branch account and the incorrect shortfall was
committed to the branch account as a loss.
5.10.4.2 Issue 2
5.10.4.2.1 Issue 2(i) PC0121925
This issue was initially raised by POL when a Test SU being operated on a Test Rig experienced a gain
of £45.05 following a cash declaration and roll into Branch Trading. The amount of the discrepancy was
the cash value of the transactions performed on the SU after rollover. The issue occurred as a result of
Riposte failing to notify the data tree of new transactions occurring.
5.10.4.2.2 Issue 2(ii) PC0132133
The SPM reported that, while running previews of daily cash reports, a discrepancy was reported, which
increased and then disappeared on re-running the report over the course of a twenty minute period.
The error was due to transactions carried out after a SU rollover not being added to the data tree. The
underlying cause was that the mechanism to notify the data tree of new transactions was switched off as
result of a bug when Cancel was pressed on a certain message during a SU rollover.
5.10.5 How it was spotted
5.10.5.1 Issue 1
The SPM recognised a branch snapshot as being incorrect, and reported it to the Fujitsu helpdesk (HSH).
It was then passed on the Fujitsu SSC resulting in the Peak being raised.
Further branches were identified as suffering from similar issues in March, April and June 2000, although
the root cause had not yet been identified.
5.10.5.2 Issue 2
5.10.5.2.1 Issue 2(i) PC0121925
The error was recognised and reported to the Fujitsu Test team by the POL Test team during testing of a
future release.
5.10.5.2.2 Issue 2(ii) PC0132133
The inconsistent and erroneous daily cash report previews were recognised and reported to the Fujitsu
helpdesk (HSH) by the SPM.
5.10.6 How it was actioned
5.10.6.1 Issue 1
The first recognised instance was addressed by the branch manager and POL agreeing to amend the
balance figures manually.
The Peak relating to that first instance was held open by Fujitsu for monitoring purposes in case similar
Incidents were reported, which they were in March, April and June 2000.
The root cause was identified on 28-Jun-2000 and a fix was scheduled, then rolled out, as part of a
release (C14) in late 2000.
5.10.6.2 Issue 2
5.10.6.2.1 Issue 2(i) PC0121925
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 30 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
Fujitsu investigated this error that had only occurred in Test by running test simulations, but the same
error was replicated only once in over 150,000 attempts. The issue was passed to Escher, but as it was
not possible to describe a scenario in which the error could be reproduced, no further action was taken by
Escher.
5.10.6.2.2 Issue 2(ii) PC0132133
Fujitsu SSC investigated and gathered data, including contacting the SPM again.
Fujitsu Fourth Line Support diagnosed the root cause and created a software fix which was released in
January 2008.
5.10.7 Impact
5.10.7.1_ Issue 1
Peak records suggest that other branches experienced related errors. BIMS are recorded as having
been issued to POL so that Error Notices could be issued by POL to correct the errors.
5.10.7.2 Issue 2
5.10.7.2.1 Issue 2(i) PC0121925
There was no known impact on SPMs as the issue was only known to have occurred in Test.
5.10.7.2.2 Issue 2(ii) PC0132133
There was no lasting impact on the one branch account which was thought to have been affected by this
problem. The Peak records that the SPM raised concerns about the appearance of a discrepancy;
however, Fujitsu SSC clarified that no transaction was missing and explained why the error had occurred.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 31 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.11 Girobank discrepancies
This term includes several issues identified by Mr Coyne, around the common thread of Girobank. These
issues date from 2000, and due to changes since that time, the records are relatively incomplete.
5.11.1 Horizon System Affected
Legacy Horizon.
5.11.2. Description
5.11.2.1 Issue 1
Giro transaction reversals carried out after the report cut-off time were not included on the daily Giro
report for that day, nor in the same report for the next day. Despite this, the transaction reversals would
have been recorded correctly in the weekly office Cash Account report. The system was working as
designed, so this was not regarded as a bug, error, or defect.
5.11.2.2 Issue 2
The same Giro deposit was included on two successive daily reports as a result of being committed
during a particular window of time on a shared SU. It was noticed by Fujitsu while investigating Issue 1
above, and a fix was applied to prevent reoccurrence. It appears to be another instance of the bug
recorded in PC0052575, which is included in Issue 3.
5.11.2.3 Issue 3
A discrepancy occurred between the daily Giro report and the office daily report due to a transaction
being carried out on a shared SU by one user while another user was printing and cutting off the daily
report. The bug described in first Peak included in this issue (PC0052575) appears to be identical to that
described in Issue 2. It is not clear if the second Peak (PC0052704) is evidence of a bug.
5.11.2.4 Issue 4
A discrepancy occurred between the daily Giro report and the office daily report due to a cut-off being
performed on one counter despite a failed attempt to print a transaction on another counter.
5.11.2.5 Issue 5
Two branches noticed discrepancies between the daily Giro report and the office daily report, although
weekly reports were unaffected.
5.11.2.6 Issue 6
A branch believed that two Giro deposits did not appear on the reports run either on the same day, or the
next day. However, the SPM was mistaken, and this was not a bug, error, or defect.
5.11.3 Dates
Issue 1: May 2000.
Issue 2: May 2000.
Issue 3: August 2000.
Issue 4: July 2001.
Issue 5: February — April 2002.
Issue 6: May 2000.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 32 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.11.4 What happened
5.11.4.1. Issue 1
Fujitsu PinlCLs (an earlier form of Peak) record several instances of Girobank reporting differences in the
value of Giro transactions between the daily Giro reports and the weekly Cash Account reports at certain
branches. These differences were the result of a Giro transaction being entered and reversed, with the
reversal being performed after the report had already been cut-off. The reversal would then not be
included in a further run of the daily report for that day, nor on the following day.
The Peak notes that the system was working as designed, and there was no reason to believe that data
was not being recorded correctly. The Horizon OPS (Office Platform System) Report and Receipts
Design document [SD/DES/005] confirms the behaviour: a transaction and its subsequent reversal would
be suppressed from the report, and if a transaction was reversed after a report was cut off, then only the
reversal would be suppressed (because the transaction has already appeared on the report prior to cut-
off and reversal). It is not clear to Fujitsu what method POL would have advised SPMs to use in the
event that it was necessary for such reversal transactions to be carried out after the report cut-off.
5.11.4.2 Issue 2
An £81 Giro deposit was included on two consecutive daily reports. This was because the transaction
was entered in a very small window of time between two system calls being undertaken, resulting in the
duplication. It was identified that the problem occurred when using a shared SU to display and cut off a
daily Giro withdrawal or deposit report on one terminal at the same time as performing a Giro transaction
to the same SU on another terminal. The weekly reports were unaffected. This bug appears to be
identical to that recorded in PC0052575, in Issue 3.
5.11.4.3 Issue 3
PC0052575 records that an SPM reported £20 and £628.25 discrepancies between the Counter Daily
Giro Deposit Report and the Office Daily Giro Deposit Report. The issue was diagnosed as arising out of
the use of a shared SU. There was a window of time between a user printing and cutting-off a report,
during which if another user was to perform a transaction, that transaction may not show on the report.
The weekly reports were unaffected. This bug appears to be identical to Issue 2.
In the other Peak included in this Issue, an SPM attempted to re-enter two transactions that were
incorrectly believed to be missing, but did so before cutting off the report. The weekly reports, when
checked, were found to be correct. This Incident was not the result of the bug described in PC0052575
of Issue 3 nor Issue 2, but appears to be the same as those in Issue 1.
5.11.4.4 Issue 4
A branch Cash Account showed two Giro deposits of £1,503, but the reports showed only one. The
discrepancy was caused under specific circumstances:
1. An attempt was made to print a report on one counter, but the print script did not complete.
2. Ona separate counter, a transaction was entered which should have appeared on the still
incomplete report.
3. Back on the first counter the report print was retried, but the print failed (for example due to the
printer being switched off).
4. Despite the print failure, the report that failed to print was still cut off, missing out the transaction
from the other counter.
Note that overall accounts were unaffected, but the transaction was missing from the reports.
5.11.4.5 Issue 5
In one instance, the Office Balance Snapshot figures were double the figures on the Cash Account
snapshot, although the Office Balance report and the Cash Account figures were unaffected.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 33 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
In another instance, an SPM found that a Giro Daily Report printed the whole of the previous week's
deposits.
Both issues related to the same underlying problem although the KEL raised for this purpose was only
referenced in one further Incident.
5.11.4.6 Issue 6
An SPM reported that two Giro deposits had not appeared on the Giro Deposits report run on the same
day as the transaction, nor on the next day's report. Following investigation by Fujitsu SSC, it transpired
that the transactions were in fact included on the report the following day.
5.11.5 How it was spotted
5.11.5.1. Issue 1
Most of the Peaks in this issue record that the Incidents were raised by SPMs calling one of the
helpdesks (POL NBSC or Fujitsu HSH) to report that Girobank had contacted them by phone or letter to
inform them that differences had been detected in the figures for the Daily Giro Reports and the weekly
Cash Account reports. In each Peak it is recorded that the total of the Daily Giro Reports was higher than
the weekly Cash Account Report.
5.11.5.2 Issue 2
This issue was noticed by Fujitsu while investigating Issue 1, and is recorded on the same Peak
(PC0044232). It was not reported by the SPM, who had called the helpdesk to report an instance of
issue 1.
5.11.5.3 Issue 3
The two instances of this issue were reported by SPMs to the Fujitsu helpdesk (HSH).
5.11.5.4 Issue 4
The SPM had received an Error Notice from Girobank, but believed that the discrepancy it corrected had
been created by Horizon erroneously duplicating a transaction. The discrepancy was reported to the
Fujitsu helpdesk (HSH) by POL on behalf of the SPM.
5.11.5.5 Issue 5
The two instances of this issue were reported by the SPMs to the Fujitsu helpdesk (HSH).
5.11.5.6 Issue 6
The SPM reported the apparent problem to the Fujitsu helpdesk (HSH).
5.11.6 How it was actioned
5.11.6.1 Issue 1
There was no fix in Horizon as the system was working as designed; this was not a bug, error, or defect.
5.11.6.2 Issue 2
This bug was fixed by new ReportProcessor, ReportBroker and DataServer files, and Fujitsu released the
fix between 30 May and 23 June 2000.
5.11.6.3 Issue 3
The fix was exactly the same as applied for issue 2.
© Copyright Fujitsu 2021
pyrig 1) FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 34 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.11.6.4 Issue 4
This fix was to ensure that the flag used to determine whether a cut off was allowed was reset whenever
the dataserver tree was rebuilt. Fujitsu released the software fix in December 2001.
5.11.6.5 Issue 5
Fujitsu were unable to replicate the issue and were therefore unable to issue a specific fix, but a new
version of the component was released with extra tracing code to enable Fujitsu to gather more evidence
in the event of the issue reoccurring. A KEL (DRowe440r) was created, but only referenced in one
further Incident.
5.11.6.6 Issue 6
No fix was necessary as this reported Incident was not a case of a bug, error, or defect. The situation
was explained to the SPM.
5.11.7 Impact
5.11.7.1I Issue 1
Fujitsu's records show that, in some instances, Error Notices had been issued to SPMs (in one case it is
stated to have been issued by Girobank) to correct the accounts of the branches concerned. Fujitsu
understands these Error Notices were issued to correct the discrepancy in the daily reports.
5.11.7.2 Issue 2
The Peak records that the issue only affected daily reports, and not the weekly Cash Account report. It
does not record what specific actions were taken to support the SPM on this particular issue, or if any
actions were necessary.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose
5.11.7.3. Issue 3
It is not known what specific actions were taken to support the SPM in the case of PC0052575.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug (the same bug
as Issue 2) was present at the time, or whether POL relied on SPMs reporting the issue when it arose.
In the case of PC0052704, the problem had occurred and been reported while a relief SPM was working
at the office; the Peak was closed following further contact with the regular SPM, who re-ran the report
and agreed with the Fujitsu SSC representative that there were no problems.
5.11.7.4 Issue 4
The SPM raised the Incident to question why the Cash Account had shown a single Giro deposit twice; it
is recorded at the start of the Peak that an error notice had already been issued to the SPM by Girobank
to clear the discrepancy.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose.
5.11.7.5 Issue 5
Although the Incident was raised by the SPM as a result of being presented with reports that did not
match, it was noted in the Peak (PC0073855) that the weekly Cash Account Report and the SU Balance
Report were unaffected, and therefore the SPM should not be prevented from rolling over correctly.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 35 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
Fujitsu SSC passed back to the Fujitsu HSH that if advice was needed on how to send the weekly report
to PON then the SPM should contact POL NBSC. In PC0073855 it is recorded that the situation was
explained to the SPM, in the other it is not clear what action, if any, was taken with regards to the SPM.
5.11.7.6 Issue 6
There was no impact.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 36 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.12 Counter-replacement issues
The term covers one main issue identified by Mr Coyne, and two other issues which were unrelated to
the main issue.
5.12.1 Horizon System Affected
Legacy Horizon.
5.12.2 Description
5.12.2.1 Issue 1
This is the main issue identified by Mr Coyne. Following the replacement of a counter hard drive at a
single counter branch, messages were overwritten, resulting in a receipts and payments mismatch.
5.12.2.2 Issue 2
A branch had two counters removed, leaving it as a single counter branch. As the remaining single
counter did not have a mirror disk, the branch had no replication of data if it was not connected to the
datacentre.
5.12.2.3 Issue 3
Riposte failed to index messages, which resulted in some items being missed from the receipts side of
the Office Balance Report.
5.12.3. Dates
Issue 1: November 2000 to September 2002.
Issue 2: March 2006.
Issue 3: February 2008.
5.12.4 What happened
5.12.4.1 Issue 1
An SPM reported a receipts and payments mismatch: receipts were £50,191.70, payments were
£50,024.58, resulting in a £167.12 discrepancy. The discrepancy would have been flagged to the SPM
on the Cash Account when attempting to rollover.
This occurred in a single counter branch where the hard drive had been replaced, but the replacement
appeared to cause two messages related to an Order Book Control Service (OBCS) transaction to be
overwritten, so that a transaction of £167.12 was not added to the Cash Account.
The root cause was Riposte coming online from recovery mode too early, resulting in messages being
overwritten because they had not been committed to the datacentre.
5.12.4.2 Issue 2
A branch had two counters removed, leaving it as a single counter branch. As the remaining single
counter did not have a mirror disk, the branch had no replication of data if it was not connected to the
datacentre; six messages on the counter had not been replicated to the data centre.
5.12.4.3. Issue 3
Riposte failed to index four messages which resulted in some items being missed from the receipts side
of the Balance Report. It is unknown how common this issue was. The Peak notes that the branch did
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 37 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
not experience a discrepancy as a result because this was a reporting issue only; indexes are not used
when replicating data and so cash/stock were unaffected.
5.12.5 How it was spotted
5.12.5.1 Issue 1
The issue was visible to the SPM who reported it to the Fujitsu helpdesk (HSH) in November 2000.
It is believed that similar instances would have been apparent to SPMs.
5.12.5.2 Issue 2
This issue was detected by Fujitsu in March 2006 as a result of an automated daily report (TPSC250).
5.12.5.3 Issue 3
This issue was detected by Fujitsu in February 2008 as a result of an automated daily report (TPSC250)
resulting in a Peak (PC0153660), but was also separately reported to the POL helpdesk (NBSC) by the
SPM; the NBSC called the Fujitsu helpdesk (HSH) who then passed the Incident to Fujitsu SSC, resulting
in another Peak (PC0153851). It was recognised by Fujitsu that it was the same issue.
5.12.6 How it was actioned
5.12.6.1 Issue 1
To fix the individual error in this particular instance, and to correct similar errors when reported, Fujitsu
was able to find the overwritten transactions by inspecting the Riposte mirror messagestore. This
involved comparing the relevant messages in the neighbour's messagestore with the node reporting the
problem. The information was passed to Fujitsu MSU who issued a BIMS report to POL.
The root cause was identified as “the SCO mirror issue” and was fixed by changing the way that the
system came out of recovery to eliminate the scenario under which transactions could be lost. The fix
was released in September 2002.
5.12.6.2 Issue 2
To address this particular Incident, Fujitsu extracted the six messages which had not been replicated to
the data centre, and issued a BIMS to POL.
5.12.6.3 Issue 3
Fujitsu SSC contacted the branch directly and explained the situation, holding the Incident open until the
branch rolled into the next TP. When that had happened, Fujitsu SSC instructed the Fujitsu helpdesk
(HSH) to then check that the SPM was happy before closing the Incident. A BIMS was issued so that
POL were aware that there was a receipts and payments mismatch, but it is unlikely that an Error Notice
was needed as there was unlikely to be a lasting effect once the branch rolled into the next TP.
5.12.7 Impact
5.12.7.1. Issue 1
. Following the reporting of such discrepancies, a BIMS would have been issued by Fujitsu to POL,
resulting in an Error Notice being issued by POL to the SPM.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose.
5.12.7.2 Issue 2
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 38 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
The Peak records that engineers had visited the branch to attempt to resolve the issue through hardware
changes, and it was necessary for Fujitsu to issue a BIMS to POL, resulting in an Error Notice being
issued by POL to the SPM in order to correct the branch accounts.
5.12.7.3 Issue 3
This instance was a reporting issue only, and so cash/stock were unaffected. The situation was explained
to the SPM, who agreed to close the Incident, and there was no other impact.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 39 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.13 Withdrawn stock discrepancies
5.13.1 Horizon System Affected
Horizon Online.
5.13.2 Description
Discrepancies could occur when an SPM declared that they held stock of a withdrawn product and then
accepted the resulting discrepancy on the system. The system would re-introduce the withdrawn product
into the system, and the discrepancy would continue to rollover.
5.13.3. Dates
July 2010 to September 2011.
5.13.4 What happened
An SPM returned some withdrawn stock to POL after the reference data change relating to that stock had
been made, without performing the Rem out. This meant that the value of the items remained in the
branch account, resulting in a shortfall to the value of the withdrawn stock.
The next TP balance showed the shortfall, prompting the SPM to make good, resulting in a TC being
issued by POL to balance the shortfall in the next TP.
The effect of the bug was that the withdrawn stock was re-introduced in November 2010 as part of the
branch stock after the subsequent rollover, and this occurred again in the following two months. The re-
introduction was not noticed by the SPM in December (because there were some unrelated losses that
were partially offset by the TC), but noticed in January 2011 because combined with the reintroduction in
December 2010, the stock value of that item had doubled.
5.13.5 How it was spotted
The discrepancy became obvious to the SPM who reported it to the POL helpdesk (NBSC) on 19-Jan-
2011, who then passed it to Fujitsu.
5.13.6 How it was actioned
To address the immediate issue in the branch, on 14-Feb-2011, NBSC advised the SPM to declare the
withdrawn product as zero and accept the resulting discrepancy. Fujitsu records indicate that this course
of action had been followed by 02-Mar-2011.
A reference data fix was put in place to prevent SPMs from encountering the problem particular to this set
of withdrawn stock again.
The root cause was that the Declare Stock picklist included all products known to a branch even if there
was no ability to transact that product, thereby allowing the possibility for branches to make erroneous
declarations of withdrawn products. It was recognised that the check that prevented this in Legacy
Horizon was not present in Horizon Online. The software fix involved ensuring that the counter moved
into the correct mode when the call was made to compile the list of declarable products, so that only valid
products could be included.
The fix to prevent recurrences of the same issue was released on 21-Sep-2011.
5.13.7 Impact
The main Peak records that around 60 branches may have declared withdrawn stock over the period that
the bug existed.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 40 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 41 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.14 Bureau discrepancies
This term covers two separate issues reported by Mr Coyne. Both the root cause and the resolution of
both issues occurred outside the Fujitsu domain.
5.14.1 Horizon System Affected
Horizon Online.
5.14.2 Description
5.14.2.1 Issue 1
An SPM tried to pre-order two currencies. A network timeout occurred after the second was added to the
basket. The SPM attempted to cancel the whole order, but only the first currency was successfully
cancelled, leaving a shortfall for the amount of the second currency.
5.14.2.2 Issue 2
This issue involved a one-sided transaction; the branch had a record of a sale of Euros (€4,500) but this
was not reflected on the POLSAP system.
5.14.3 Dates
Issue 1: August 2017 — November 2017.
Issue 2: December 2017 — May 2018.
5.14.4 What happened
5.14.4.1. Issue 1
An SPM attempted to pre-order two currencies for a customer: £1,000.07 in Indonesian Rupiah; and
£204.59 in Singaporean dollars. The order for Rupiahs was successful; however, at the point the dollar
order was added, there was a network timeout. When the system came back online, a warning message
stated that the second order may not have succeeded, although the basket and transaction log were
showing both orders.
As the customer's receipt showed only the Rupiah order, the SPM attempted to cancel the whole order of
both currencies. The cancellation only succeeded for the Rupiah order, leaving a £204.59 shortfall in
respect of the dollars.
5.14.4.2 Issue 2
This issue related to a one-sided transaction in which a branch had a record of a sale of Euros, but it was
not recorded in POLSAP, leading to a discrepancy.
5.14.5 How it was spotted
5.14.5.1 Issue 1
Reported by the SPM to the Atos helpdesk on 17-Aug-2017, after which it was escalated to the Fujitsu
MAC team, then to Fujitsu SSC.
5.14.5.2 Issue 2
The Atos helpdesk reported the issue to the Fujitsu MAC team.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 42 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.14.6 How it was actioned
5.14.6.1 Issue 1
The discrepancy within the branch was referred by Fujitsu SSC to POL in order for them to decide which
reconciliation or TC was required to balance at the branch.
Although the immediate cause of the issue was an interruption to the network connection, the root cause
of the system failing to recover correctly from that interruption was identified by Fujitsu as being an issue
with an AP-ADC (reference data) script. The fix to the script would have been carried out by Atos who
were responsible for AP-ADC scripting at this time.
Fujitsu had no further involvement in the fix and therefore has no further information.
5.14.6.2 Issue 2
The root cause of the issue was outside of the Fujitsu domain. Fujitsu’s records are that the issue was
dealt with by Atos and Accenture, and that the fix involved amending the MMBE files (SAP Transaction
code for Stock Overview) for US$ and Euros to reflect the currency holdings in the branch. A comparison
of data between Fujitsu and Accenture identified no discrepancies, and a visit to the branch by a POL
trainer confirmed that the branch holdings matched the figures held in Horizon. This information is
recorded in the Peak (PC0265443) but POL and / or Accenture would need to confirm if any other action
was required.
5.14.7 Impact
Fujitsu has no information on impact for either of the issues described.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 43 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.15 Phantom Transactions
This term includes several issues. Issue 1 as described below is itself a collection of separate but
possibly related Incidents; issues 2 and 3 are two Incidents reported by one SPM on one day, but are not
related to issue 1.
5.15.1 Horizon System Affected
Legacy Horizon.
5.15.2 Description
5.15.2.1 Issue 1
Some SPMs reported that items and transactions appeared and disappeared without any user input.
After extensive monitoring, it was concluded that user error could not be ruled out in any of the known
circumstances. It is worth noting that this collection of issues arose and then ceased within a few months
of the branches first going live with Horizon in 2001. Ultimately, the issues appear to relate to insufficient
training, and, in some cases, equipment failures which were then resolved.
5.15.2.2 Issue 2
An SPM reported that a receipt containing three transactions printed without any user input. This was,
however, standard functionality that occurred if a user was automatically logged off due to inactivity after
having entered, but not completed, transactions.
5.15.2.3 Issue 3
Two counters in the same branch had certain differences in the icons displayed on screen. One counter
had received the latest Riposte release (which changed the button text), the other had not, so the two.
counters were one version apart.
5.15.3 Dates
Issue 1: April 2001 — November 2001.
Issues 2 and 3: August 2000.
5.15.4 What happened
5.15.4.1 Issue 1
An SPM reported that items and transactions would appear and disappear from the screen without any
user input. The Peak (PC0065021) contains multiple reports by the SPM of separate instances of this
type of error, but also reports of errors in various other branches.
The Peak (PC0065021) opens with a record that the SPM had made a previous complaint that had been
closed without his agreement, and that the SPM had already had to pay to cover losses incurred as a
result of the problems. It records the SPM had previously had disagreements with Post Office Counters
Limited (POCL) and Fujitsu over whether the problems were caused by user error, or issues with
Horizon
Mr Coyne in his report and Fraser J in his Judgment both included entries from this Peak referring to
statements that the Romec engineer had witnessed “phantom transactions” while at the branch, although
Fujitsu has no direct evidence from Romec to confirm this.
5.15.4.2 Issue 2
An SPM reported that a receipt containing three transactions printed without any user input.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 44 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
Horizon automatically logs a user out after a period of inactivity, a standard security measure of many IT
systems. If the user has recorded, but not completed, some transactions in the stack, then when the
systems automatically logs off, it will complete these transactions and assume payment was made by
cash. The receipts are then printed to make it clear to the SPM that it has happened.
Fujitsu's investigation showed that the transactions were held in a suspended session hence they were
not complete when the user was logged out. The assumption was that they were inadvertently placed in
suspense by the SPM.
This was not a bug, but functionality intentionally designed into the system, based on POL's requirement.
5.15.4.3 Issue 3
The SPM that reported issue 2, also reported at the same time that there were differences between the
icons displayed on the two counters in the branch.
5.15.5 How it was spotted
5.15.5.1 Issue 1
All Incidents relating to this issue were reported by SPMs to both Fujitsu and to POCL.
5.15.5.2 Issue 2
Reported by an SPM directly to the Fujitsu helpdesk (HSH) in August 2000.
5.15.5.3 Issue 3
Reported directly to the Fujitsu helpdesk (HSH) in August 2000 at the same time as Issue 2, by the same
SPM.
5.15.6 How it was actioned
5.15.6.1 Issue 1
Fujitsu carried out extensive work and investigation in attempting to resolve the issues reported by the
principal SPM in Peak PC0065021, starting on 17-Apr-2001. To understand how PC0065021 was closed
out with the closure category “No fault in product”, it needs to be read in conjunction with two other Peaks
from 2001, PC0062561 which relates to the same branch, and PC0066391 which catalogues apparently
similar issues at other branches.
PC0062561 was opened two months before PC0065021, on 15-Feb-2001, one month after the branch
went live with Horizon on 11-Jan-2001. It records that, since go live at Old Isleworth 111025, the SPM
had raised 75 Incidents, mostly for advice and guidance. The SPM then requested additional training,
and arrangements started to be made to provide this. The number of Incidents was being monitored, and
that although decreasing, the number was still high.
No further calls from the SPM were added to PC0065021 after 30-Sep-2001, and on 12-Nov-2001, the
Fujitsu specialist who had carried out the monitoring closed the Incident, concluding “Phantom Txns have
not been proven in circumstances which preclude user error. In all cases where these have occurred a
user error related cause can be attributed to the phenomenon.” (Note that the use of italics here and in
the remainder of this section indicates text that is directly quoted from Peaks.)
As stated above, to understand the context within which PC0065021 was closed with that conclusion, we
have included information from PC0062561, which summarises the actions taken up to 17-Aug-2001:
Update on problems at outlet from FSM (Fujitsu Service Management):
"This outlet has reported continual phantom transaction problems causing us to exhaust every possible
course of action in trying to solve them. We have:
1) spent limited time at the outlet watching how he operates the system — no evidence of swipe head was
found
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 45 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
2) collected a weekly detailed list of when the phantoms are happening, what products, which counter
etc. - no relevant trends were noted
3) we have replaced all kit and cables on both counter positions
4) installed a test rig with Comtest software for three weeks to track interference on the screens - nothing
conclusive was found
5) installed a resistive monitor at counter position 1 and sent the old screen to Pat Carroll for testing
(waiting to speak to Pat on his return from holiday)
Atfter all this the PM is still experiencing Phantom transactions but they are mainly on counter positon 1
and this is always used by Robert Parker (PM). I have asked Robert if I can spend some time at the
outlet with him so I can be present when the phantoms occur but he is not keen for this to take place as
he feels the outlet is too small and gets too heated as it is.
I spoke with HSH this moming and she advises that since Powerhelp was last archived, Mr Parker has
logged 34 calls to the helpdesk and a vast amount are advice and guidance. My personal feeling is that
Mr Parker could do with some further training and I feel that this should be our next course of action. The
only other option we have open to us is to change the ISDN line, which is the old style, but myself and
HSH feel that this is an expensive option to go down when it may be user error at fault.
PON to look at training issues. pursued with PON 17/08/01.
The Peak records that the number of Incidents raised by the branch appeared to be falling off to between
two and four per week. The final entry in PC0062561 is dated 20-Sep-2001, noting that this was several
weeks before the master Peak PC0065021 was closed:
PON have written to the RNM to address the training issue, see text below: "From RNM - I spoke to
training and Dev. this afternoon and arranged 2 days training for next week, when I rang Mr Parker he
told me that he did not need the extra training so i have now cancelled it. He also told me that the
Phantom transactions have stopped.
PON to RNM: "There seems to be no issues at this outlet if you are happy with the postmasters
response.
Js there anything else that needs investigating at the outlet proven to be directly linked with phantom txns
(discrepancies?) as there are none recorded? If not I would like your agreement to close down this
problem as now resolved. I would like to make you aware though that the postmaster does seem to be
making quite a few calls still to the HSH helpdesk, mainly around simple things such as reversals.
RNM to PON: Thanks for making aware about the number of calls your still receiving, i don't think we will
ever stop him from making these. I see no reason why this call cannot not be closed. as i said the
Postmaster said he is no longer getting these transactions.
Calls have actually reduced in September, there are currently only 4. I have agreed with PON that there
is little else which can be done. The PM is not making errors with his work and the call volume has
improved. I have agreed to close the problem.
F} Response :
final- completed
[END OF REFERENCE 27645229]
...Defect cause updated to 39:General - User Knowledge
As well as including the same branch, Peak PC0066391 also refers to reported instances at other
branches which were investigated in 2001, as follows:
e Gamesley 279432:
o June: System monitoring put in place for six weeks, SPM reported only one incident in that
time.
o August: ComTest has shown no results at Gamesley.
o September: Gamesiey has reported an incident of a confused transaction during their
balancing. However, investigation from SSC showed that this was a result of user error, not
following the correct balance sequence. There is now no clear evidence of any phantom
transactions. Those that have been resolved have been attributed either to faulty equipment
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 46 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
or to user error. There is no evidence of any software error or extemal interference giving rise
to phantom transactions.
e¢ Wawne 250321:
o June: ComTest diagnostic monitoring equipment [same as for Old Isleworth 111025] has
been installed at the site to test for environmental problems affecting the monitors. So far, no
Significant readings recorded. No significant readings.
* Trecynon 441611:
o June: “Postmaster replaced faulty fluorescent strip lights and starters, since when the
problem has not recurred. This problem is now closed.” Note, no evidence was recorded in
the Peak to indicate that this was related to any problems with Horizon.
« Riddings 376311:
o June: “Keyboards and Monitors have been swopped with no improvement. On site
investigation showed significant variations in the users methods of operation. One incident
observed, and under investigation by 3” line.”
o August: “Riddings is no longer reporting problems since user errors were highlighted, and is
considered closed by agreement with PON Problem Office manager Paula Astles.”
e Cefn Glas 316611: June: “Chomeric collars and mains filters fitted; Romec fixed faulty alarm box,
since when no further incidents. This problem is now closed.”
5.15.6.2 Issue 2
This was not a bug. The system was functioning correctly so no fix was necessary. The situation was
explained to the SPM and it was agreed the Incident could be closed.
5.15.6.3 Issue 3
The second counter, which had not yet received the latest Riposte release, would have been
automatically upgraded during the software rollout, but Fujitsu upgraded it sooner in order to bring it into
line with the other counter.
5.15.7 Impact
5.15.7.1 Issue 1
The SPM who raised the initial complaint that resulted in Peak PC0065021 stated that he had had to pay
£1500 in losses that he said were related to the problems. There is no evidence recorded of any
financial impacts to the other branches referred to in 5.15.6.1.
5.15.7.2 Issue 2
The SPM agreed to the Incident being closed once the situation had been explained by Fujitsu SSC, so
there was no apparent impact.
5.15.7.3 Issue 3
Fujitsu was unable to identify any financial or operational difficulties resulting from the differences
between button texts.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 47 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.16 Reconciliation issues
This term includes several issues.
5.16.1 Horizon System Affected
Issues 1, 2, 3, 4 affected Legacy Horizon.
Issues 5 and 6 affected Horizon Online.
5.16.2 Description
5.16.2.1 Issue 1
Discrepancies were reported by automated BAU reconciliation reporting, related to two transactions
(£8.06 and £0.08, totalling £8.14) that were brought forward values from the previous week's Cash
Account.
5.16.2.2 Issue 2
A false discrepancy of 1p was detected by automated BAU reconciliation reporting; investigation showed
no actual discrepancy in the branch.
5.16.2.3. Issue 3
Related to a Test system, not Live, therefore does not affect SPMs.
5.16.2.4 Issue 4
Automatic detection of a receipts and payment mismatch of £4,464.46, related to the data tree issues
described above in section 5.10.
5.16.2.5 Issue 5
Data was detected as missing from the Client Transaction Summary (CTS) Report due to new products
with start dates not on the midnight boundary. Although this system was operating as designed, a fix was
made so that this did not reoccur.
5.16.2.6 Issue 6
A difference was found between a branch's figures and the CTS report.
5.16.3 Dates
Issue 1: March 2000 — August 2000.
Issue 2: April 2002 — August 2002.
Issue 3: July 2000 — August 2000.
Issue 4: May 2000.
Issue 5: September 2014 —- December 2014.
Issue 6: September 2010.
5.16.4 What happened
5.16.4.1 Issue 1
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 48 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
At the end of each working day, the counter calculated information for the Cash Account in two
independent ways and ensured that they matched, then a report was generated of any instances of
mismatches. The Peak was raised as a result of that process which showed a discrepancy of £8.14.
The root cause was identified in a related Peak (PC0040243) as being code that, in the event of a
particular sequence of operations being carried out by the SPM, would cause transactions to be brought
forward from the previous week's Cash Account. The apparent discrepancy was therefore a reporting
issue, and although the scenario in which it arose was thought to be relatively unusual, it was treated as
a bug to be fixed.
5.16.4.2 Issue 2
Daily BAU reconciliation reporting identified discrepancies between totals produced at the counter (1p)
and those produced at the data centre (zero). Fujitsu's investigations identified that the discrepancy was
due to the way that the host re-calculated the totals from the branch data when carrying out the
reconciliation check: values generated as 0.01 (1p) were being stored as 0.0099 which was then treated
as zero and therefore ignored, and not recorded for the purpose of the report. This bug was fixed in
order to avoid further instances of false mismatches appearing on report TPSC268A.
5.16.4.3 Issue 3
The Peak was raised by a tester explicitly checking for issues in a test environment. A report showed a
difference between the number of files recorded as being transferred to TIP and the number of files
actually transferred. The root cause was that when the report was run it was not accounting for the files
already counted. There is no suggestion in the Peak of any issues with the underlying data being
transferred to TIP, only with the report.
5.16.4.4 Issue 4
Areceipts and payment mismatch was automatically detected. Investigation identified the immediate
cause of the imbalance in the Cash Account being a result of a failure in the EPOSSDataserver. The root
cause was related to a data tree build failure (documented in section 5.10) that occurred following a hard
disk failure in the branch.
5.16.4.5 Issue 5
Data was detected as missing from a CTS Report. It was identified that when products with a start date
of 01-Aug-2014 were traded on that date, the metadata for those products did not yet exist on the
Automated Payment Service (APS) database when the CTS report was written. The root cause was
identified as products being introduced with start dates / times that were not on a midnight boundary; in
this particular case the start time was 00:00:02.
5.16.4.6 Issue 6
An SPM found there was a difference between their branch figures and the CTS report and reported it to
the Fujitsu helpdesk (HSD). Investigation by Fujitsu SSC found that the root cause was an issue with
POLSAP, so the Peak was closed as POLSAP was not supported by Fujitsu SSC.
5.16.5 How it was spotted
5.16.5.1 Issue 1
Detected by Fujitsu's automated BAU reconciliation reporting.
5.16.5.2 Issue 2
Detected by Fujitsu's automated BAU reconciliation reporting.
5.16.5.3 Issue 3
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 49 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
Detected when explicitly checking for issues in a Test environment. Not an issue in Live.
5.16.5.4 Issue 4
Detected by Fujitsu's automated BAU reconciliation reporting.
5.16.5.5 Issue 5
Detected and raised by POL as a result of checking a CTS report.
5.16.5.6 Issue 6
Reported to POL by one of POL’s vendor clients, and then reported by POL to Fujitsu.
5.16.6 How it was actioned
5.16.6.1 Issue 1
A software fix to prevent any recurrence was released by early August 2000.
5.16.6.2 Issue 2
The fix involved a change to the code for the report and was released in August 2002.
5.16.6.3 Issue 3
This issue was spotted during testing, and the report was fixed before becoming Live
5.16.6.4 Issue 4
As this issue was identified as relating to the data tree build issues, the fix was the same as detailed in
section 5.10.6.1: the root cause was determined on 28-Jun-2000 and a fix was scheduled, then rolled out
as part of a release (C14) in late 2000.
5.16.6.5 Issue 5
A fix was released in December 2014 so that the latest version of a product for the current day would be
retrieved, removing the restriction that allowed only those products that became effective at the start of
the day to be retrieved.
5.16.6.6 Issue 6
The issue was within POLSAP, and therefore no fix was necessary within Horizon.
5.16.7 Impact
5.16.7.1 Issue 1
Fujitsu understands the reported discrepancy of £8.14 was a reporting issue only. There was no financial
impact on the branch accounts.
5.16.7.2 Issue 2
Fujitsu understands the only effect of this bug was to cause the appearance of false mismatches in
reconciliation reporting. It did not have the potential to affect branch accounts.
5.16.7.3 Issue 3
This issue occurred in Test, not in Live, and Fujitsu understands no SPMs were affected.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 50 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.16.7.4 Issue 4
There was a branch mismatch of £4,464.46. The Peak records that the Incident was passed to Fujitsu
MSU to report the mismatch to POL so that it could be settled with the SPM. Fujitsu does not hold any
other records relating to this.
5.16.7.5 Issue 5
This issue affected reporting between POL and third parties and did not cause any discrepancies for
SPMs.
5.16.7.6 Issue 6
Fujitsu understands this is unlikely to impact branch accounts unless POL erroneously raised a TC. The
Peak shows that POL had identified both the amount involved and the branch, so it is unlikely that there
would have been an impact on branch accounts.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 51 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.17 Branch Customer discrepancies
5.17.1 Horizon System Affected
Legacy Horizon
5.17.2 Description
A counter crashed while the user was processing a cash withdrawal transaction. The SPM started the
recovery process but did not complete it. The customer did not receive the money as the transaction did
not complete, but the bank did record the withdrawal.
5.17.3. Dates
March 2008.
5.17.4 What happened
A branch counter crashed while processing a cash withdrawal transaction. As a result Horizon did not
record a completed cash withdrawal. Since the transaction did not complete, the SPM should not have
handed over the cash to the customer, and it appears from the record that the SPM did not do so.
However, the customer's bank recorded the transaction as having taken place, hence the customer
would suffer the loss of the withdrawal from their account, but without receiving the cash.
The branch did not log back in to the counter after the crash, so the automatic recovery process was not
carried out. (If the recovery process had been completed properly, it would have recorded a reversal of
the failed cash withdrawal, resulting in the customer's bank returning the money to the customer's
account, and no transaction being registered on Horizon, i.e. it would be as if the transaction had not
happened at all).
Fujitsu detected the problem transaction via the automated NB102 report, where it appeared as a state 4
transaction (an incomplete transaction), which reports discrepancies between the Financial Institution's
(Fl) view of what has happened and Horizon’s. In this case, the discrepancy was that the Fl had
authorised a payment/withdrawal on the assumption that the cash payment had been made, whereas
Horizon recorded that there had been no payment. This report caused the creation of the first Peak
(PC0156174).
Having noted that the SPM had not logged back into the counter since the event, Fujitsu contacted the
branch and advised the SPM to log in to the counter and allow the writing of the recovery messages to
complete. Within around 30 minutes it was confirmed that the messages had been written, and later that
day a BIMS was issued to POL.
However, the next day the transaction again appeared on the NB102 report, this time as a state E37
(uncleared exception), leading to the creation of the second Peak (PC0156246). It appeared that on
attempting the recovery process, the SPM had not allowed it to complete, and had declined the recovery.
The Peak also records that POL had contacted Fujitsu to report that the FI (Citibank) had contacted POL
to ask why the transaction was showing on their system as an exception. It was thought that the branch
account was in balance, but the customer's account had been debited, and therefore needed rectifying.
Fujitsu advised POL to check with the branch to confirm that the money had not been paid to the
customer, and if that was the case, to notify the bank of the discrepancy so that the customer account
could be corrected.
5.17.5 How it was spotted
Detected by Fujitsu automated BAU reconciliation reporting (NB102). See section 5.17.4 for full details.
5.17.6 How it was actioned
No fix was required to Horizon.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 52 of 80
POL00030528
POL00030528
THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU CONFIDENTIAL
oO
FUJITSU
This issue relates to the correct operation of the recovery process, and did not require a software fix. The
attempt at recovery failed due to the SPM declining the recovery part way through the process; the Peak
suggests that this was due to the SPM not having handed the money over and therefore thinking that the
recovery was not necessary.
5.17.7. Impact
The Peak shows that Fujitsu informed POL, and that a BIMS was issued to POL, for the customer's
account to be corrected by their bank.
© Copyright Fujitsu 2021
FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 53 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.18 Concurrent logins
This term identified by Mr Coyne includes two separate and unrelated issues.
As an operational rule, a user should not be allowed to log on to two terminals concurrently in order to
maintain a clear audit trail of who processed which transactions. As such, SPMs and Branch staff should
not share user credentials.
5.18.1 Horizon System Affected
Legacy Horizon.
5.18.2 Description
5.18.2.1 Issue 1
A.user managed to log on to two counters simultaneously when the first counter was unable to respond.
5.18.2.2 Issue 2
Areceipts and payments mismatch was caused by two operations being carried out with simultaneous
logins, although there was no overall loss to the branch.
5.18.3 Dates
Issue 1: 1999 to 2001.
Issue 2: 2000.
5.18.4 What happened
5.18.4.1 Issue 1
While printing a final Cash Account on counter 2, the system froze for an hour. Having called the Fujitsu
helpdesk (HSH), the SPM was asked to logon to counter 1 to re-try printing out the final Cash Account,
which was successful. Counter 2 was still showing ‘printing report’ as the SPM was logging on and
printing the report on counter 1. Counter 1 then started to show “no entry sign” icons when the SPM.
logged out. POL NBSC requested Fujitsu to investigate the fact that the SPM had apparently been
logged on to two counters concurrently. There was no evidence of any financial discrepancy.
It was confirmed after investigation by Fujitsu that the SPM had been able to log himself onto the system
at both counter 1 and counter 2 at the same time.
5.18.4.2 Issue 2
Areceipts and payments mismatch was reported by an SPM to the Fujitsu helpdesk (HSH). The SPM
reported that they had transferred £9,250 out from counter 8 by mistake, when it should have been from
counter 1, with £5,590 being transferred in to counter 3 and £3,660 to counter 4. This had occurred while
counter 8 was in the process of rolling over, but 3 and 4 had not yet rolled. As a result, the cash left
counter 8 in Cash Accounting Period (CAP) 18 and arrived in counters 3 and 4 in CAP 17.
CAP 18 was therefore in surplus, because £9,250 was transferred out and never back in within the same
CAP, while CAP 17 had a gain, because £9,250 was transferred in but there was no corresponding
transfer out without the same CAP. The SPM then rolled over the office for CAP 17, accepting the
discrepancy.
Fujitsu's investigation indicated that the immediate cause of the error was that the user rolled counter 3,
while logging in and processing a transfer out on counter 4, so was logged in to both counters
concurrently.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 54 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
It was identified that the root cause was a failure in the logon checks within EPOSS SU, and it was later
recognised that this failure had been possible due to there being what is described in the Peak as, “a
short period on live” when two pieces of code were out of step. This was an interim problem that had
already been fixed by a release.
5.18.5 How it was spotted
5.18.5.1 Issue 1
The Peak records that POL were concerned that it appeared that the SPM had been able to log on to two
counters simultaneously and asked Fujitsu to investigate. It is not clear to Fujitsu why the issue was
originally raised with POL.
5.18.5.2 Issue 2
SPM reported a receipts and payments mismatch to Fujitsu helpdesk (HSH).
5.18.6 How it was actioned
5.18.6.1 Issue 1
Fujitsu identified the issue as being a bug in the code supplied by Escher, and asked Escher to fix it. The
Peak states that the code containing the fix was applied in September 2000. However, it was noted later
in the same month that the ability to log onto two counters simultaneously remained after the date of that
release, and this was reported to Escher. Escher's response was recorded in October as being “We
believe that the login process is functioning as designed and will not be addressing this issue.” The Peak
records that there was discussion about creating a workaround via EPOSS.
The Peak was eventually closed, with the observation that the original SPM concerned no longer worked
for POL, and that if the issue arose again then the Peak could be reopened. There is no record of the
Peak being subsequently re-opened nor of any similar Incidents being raised. There is no evidence of
the bug having led to any account discrepancies.
5.18.6.2 Issue 2
It was recognised that the possibility of this event recurring was about to be removed by a software
release which was already in the pipeline and which subsequently went live.
5.18.7 Impact
5.18.7.1 Issue 1
Fujitsu understands there was no record of any impact on the SPM, and the reported issue did not
involve any discrepancy.
5.18.7.2 Issue 2
There were receipts and payments mismatches over three CAPs, but as they netted to zero there was no
evidence of financial impact on the branch.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 55 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.19 Post & Go/TA discrepancies in POLSAP
5.19.1 Horizon System Affected
Horizon Online. Only branches with Post & Go terminals could have been affected.
5.19.2 Description
This occurred in a scenario in which Horizon was receiving Post and Go (PG) data for six separate PG
tills at a branch, but only four of the tills had associated SUs. This caused the entire subfile for the
branch to be Held, and the transaction data was not sent to POLSAP.
5.19.3 Dates
2012.
5.19.4 What happened
A branch was found to have many entries in the “Subfiles_on_hold” report. Horizon was receiving PG
data for six separate Post & Go tills at the branch, but only four of them had associated SUs. This
caused the entire subfile for the branch to be Held, and the transaction data was not sent to POLSAP.
It was noted by Fujitsu SSC that other branches were experiencing similar issues, although the branches
themselves would not notice the issue because it did not affect the transfer of data from PG terminals to
Horizon and therefore did not affect branch accounts. It only affected the transfer of data outside of
Horizon to POLSAP.
5.19.5 How it was spotted
POL’s POLSAP team had noticed some balance mismatches in data from certain branches and raised an
Incident with Fujitsu
5.19.6 How it was actioned
Fujitsu's investigation showed that the issue could arise when some SUs within a branch were not being
used for cash transactions, as a result of which the SPM would not be prompted to create the necessary
associations.
An MSC (a controlled operational change) was raised to implement a pre-existing script which created a
zero value transaction for the P&G terminals that were missing them, thereby prompting the branch to
create the required SU association. Once the associations existed, the held transaction files were then
able to pass through to POLSAP.
This form of fix was applied to at least one other branch, before an estate-wide permanent fix was applied
in November 2012.
The Subfiles_On_Hold report to detect such issues was already in place prior to this occurrence. Fujitsu
reminded POL to monitor the report, which was sent to them daily, so that any other external terminals
with problems could be investigated quickly in case similar corrections were needed.
5.19.7 Impact
Fujitsu understands this issue affected the reporting of data from Horizon to POLSAP, and did not cause
discrepancies at branches.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 56 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.20 Recovery Failures
This term covers three issues.
5.20.1 Horizon System Affected
Horizon Online.
5.20.2 Description
5.20.2.1 Issue 1
An SPM raised an alleged discrepancy with POL and Fujitsu 9 months after the events in question.
Fujitsu believe the Incident to have been a genuine accounting issue, and not caused by any fault in
Horizon. Mr Coyne made no reference to the nine month delay in his report.
5.20.2.2 Issue 2
Asession failed due to a communications / network problem. On restoration of the service, a Health
Lottery transaction failed to recover correctly.
5.20.2.3 Issue 3
This was a BAU reconciliation Incident following a standard process with no evidence of any failure.
5.20.3 Dates
Issue 1: September 2012.
Issue 2: February 2015.
Issue 3: April 2010.
5.20.4 What happened
5.20.4.1 Issue 1
An SPM rolled her office over with a discrepancy of less than £1, then the following day performed a daily
cash declaration that showed a shortfall of £191.48. The next day one of the counters suffered a memory
dump issue, and the following day the counter was replaced.
A few days later the SPM reported a shortfall of £300 and asked the POL helpdesk (NBSC) if it might be
related to the base unit fault or replacement. NBSC passed the Incident to Fujitsu, and Fujitsu SSC
investigations found no evidence of any system errors, and suspected that the cash shortfall that existed
before the memory dump issue was the same shortfall that was now being reported, in other words it
predated the memory dump and the base unit replacement. The issue was passed back to NBSC for
investigation as an operational accounting issue.
Eight months later the SPM opened another Incident with the POL helpdesk alleging that the counter
replacement had caused the £300 loss. Fujitsu responded to POL by providing information from the
original Incident earlier in the year, and about the memory dump and the base unit swap. The Peak also
noted that it was not clear what, if any, investigation had been carried out by NBSC, and that no further
Incidents had been raised at the time regarding the discrepancy. Fujitsu advised POL that if further
investigation was required then POL would need to submit an ARQ request in order to retrieve financial
transaction data from the audit server.
5.20.4.2 Issue 2
A user reported that they were unable to complete a recovery on a counter. The settlement of the
session failed due to a network communications problem. On logging back in, the recovery process was
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 57 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
initiated but the recovery of one of the two transactions in the session failed, and the user was then
unable to log in on that counter.
The root cause was the failure of the AP-ADC recovery script to handle Lottery transactions fully.
5.20.4.3 Issue 3
The Peak cited by Mr Coyne relates to a routine reconciliation Incident which was raised as a result of the
automated overnight reporting, and there is no suggestion that it involved any recovery failure.
The original issue in the branch leading the reconciliation issue appeared to relate to a cash withdrawal,
but the exact details are unknown.
5.20.5 How it was spotted
5.20.5.1 Issue 1
The SPM raised the discrepancy with the POL helpdesk (NBSC) who passed the Incident to Fujitsu.
5.20.5.2 Issue 2
Reported by an SPM to the POL helpdesk (NBSC) in the first instance who then raised it with Fujitsu.
5.20.5.3 Issue 3
This was a routine reconciliation Incident automatically raised as a result of Fujitsu’s BAU automated
reconciliation reporting.
5.20.6 How it was actioned
5.20.6.1 Issue 1
No fix was made nor shown to be needed.
Fujitsu advised POL that if further investigation was required then POL would need to submit an ARQ.
request in order to retrieve financial transaction data from the audit server. The Peak was closed and
Fujitsu has no record of an ARQ request for this branch.
5.20.6.2 Issue 2
The workaround to get the counter through recovery so that it was operational again was to mark the
Recovery Data for the lottery transaction as complete rather than deleting it. This was done in the
transitory Recovery Data Table (not in a Transaction Table) using an SQL statement which required
Privileged User (“APPSUP") access. This was done only on receipt of confirmed Authorisation from POL.
5.20.6.3 Issue 3
No fix was made nor shown to be needed.
The issue was dealt with under standard BAU processes by Fujitsu issuing a BIMS report to POL.
5.20.7 Impact
5.20.7.1 Issue 1
Fujitsu understands the issue was passed back to POL NBSC. Fujitsu has no record of the outcome.
5.20.7.2 Issue 2
Fujitsu issued a BIMS report to POL to enable POL to arrange any corrections for the branch
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 58 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.20.7.3. Issue 3
Fujitsu issued a BIMS report to POL on the same day the Incident was raised to enable POL to arrange
any corrections for the branch.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 59 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.21 Transaction Correction Issues
This term covers three issues identified by Mr Coyne, one of which only occurred during testing.
5.21.1 Horizon System Affected
Issues 1 and 2: Legacy Horizon.
Issue 3: HNG-X.
5.21.2 Description
5.21.2.1 Issue 1
The four Peaks identified by Mr Coyne were raised during the testing of TC functionality for the then
future S80 release. These Peaks were not in Live, so could not have affected branch accounts.
5.21.2.2 Issue 2
The screen froze when selecting Transaction Correction (TC). This bug was identified on 51 sites, due to
formatting of the TCs, soon after the functionality was introduced.
5.21.2.3 Issue 3
TCs more than 40 days old were not showing on the TC report which it was possible to request for up to
60 days after the TC was issued. This was due to the retention period being only 40 days.
5.21.3 Dates
Issue 1: January — May 2005.
Issue 2: December 2005 — January 2006.
Issue 3: September — October 2010.
5.21.4 What happened
5.21.4.1 Issue 1
The Peaks identified were raised by Fujitsu's SV&I team and related to issues which occurred during the
testing of TC functionality for the then future Release S80. Three Peaks (PC0114154, PC0118562,
PC0121331) related to the appearance of TC option buttons on the screen, while the other (PC0120459)
related to the TC pick list freezing when a TC was received.
5.21.4.2 Issue 2
In December 2005, Fujitsu received calls from branches reporting that their screens would freeze when
selecting a TC from the picklist, preventing the branch from dealing with the outstanding TC. The issue
was reported by four branches in relation to a TC for Premium Bond Sale, and a further 48 branches
relating to a Camelot Lottery TC (KEL LKiang2837P). The issue impacted branches as they were
prevented from rolling over, due to being unable to clear their outstanding TCs.
The problem was found to be caused by the code being unable (and not designed) to deal with some of
the formatting used by the POL TC team in preparing some of the TCs. The code to render the text of
the TC on screen attempted to split the blocks of text at a suitable space between words, but some of the
TCs contained long strings of around 80 concatenated characters causing the code to fail.
5.21.4.3 Issue 3
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 60 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
An SPM contacted the Fujitsu helpdesk (Service Desk) to report an £80 cash loss and claimed it was
caused by a system error. In evidence, the SPM provided a processed TC report generated on 10
September 2010, with a date range between 12 July 2010 to 10 September 2010 (i.e. 60 days). The
report showed a TC accepted on 08 September 2010, but not the TCs accepted on 13 July 2010 and 21
July 2010.
Fujitsu requested (by phone) some additional information from the SPM in order to further investigate the
£80 loss, but no further information was available. The Peak records that a check was made by NBSC
that confirmed that the TCs missing from the report had been processed correctly, and that information
was passed on to the SPM.
Fujitsu Development ascertained that the cause was a bug introduced as part of HNG-X. Through the
picklist on screen, Horizon allowed SPMs to select a backwards date range of up to 60 days when
generating a processed TC list, but the relevant database only retained the data for up to 40 days. It was
confirmed that the original use case stated that the TCs should be retained for 60 days, and that because
the setting on the database table was incorrect, this was a bug.
5.21.5 How it was spotted
5.21.5.1 Issue 1
The issues were identified by the Fujitsu SV&l team during Testing.
5.21.5.2 Issue 2
The issue was reported by SPMs to the Fujitsu Helpdesk (HSD).
5.21.5.3 Issue 3
An SPM reported an apparent £80 loss to the POL helpdesk (NBSC), and mentioned that when running a
report of processed TCs that it was missing two TCs that were known to have been accepted. NBSC.
raised an Incident with Fujitsu helpdesk (Service Desk) to investigate, following which the root cause was
determined.
5.21.6 How it was actioned
5.21.6.1 Issue 1
These Peaks all record that the issues identified during testing were resolved by Fujitsu. In one case
[PC0114154 / PC0118562] following the application of reference data and counter code fixes, the
functionality was retested and confirmed to be working correctly. In the other two cases [PC0121331 /
PC0120459] the problems were recognised as having been caused by work packages being missed from
the counter build that was being tested; once this was corrected, the Peaks were closed
5.21.6.2 Issue 2
As an interim measure Fujitsu advised SPMs to rollover all other SUs except one, pending the fix. A
software fix was released, completing rollout in January 2006. Additionally, the issue was raised formally
with POL via the Fujitsu HSH SDM, to request the introduction of some controls over the formatting of
TCs.
5.21.6.3 Issue 3
The fix made by the Fujitsu Development team in January 2011 was to extend the TC retention period in
the database table (TPS_TXN_CORRECTION_DETAILS) to 62 days so exceeding the counter picklist
and the original use case. Records show that 62 days was chosen as branches needed to rollover the
TP within 62 days.
5.21.7 Impact
© Copyright Fujitsu 2021
pyrig 1) FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 61 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.21.7.1 Issue 1
There would have been no impact on SPMs as these issues did not occur in the Live system, only in test.
5.21.7.2 Issue 2
SPMs were unable to process TCs and experienced the counter freezing for a long period when they
attempted to do so. SPMs were advised on how to deal with the problem as an interim, and then a fix
was released and it was confirmed that branches were then able to process the previously frozen TCs.
5.21.7.3 Issue 3
Acheck was made that confirmed that the TCs missing from the TC report had been correctly processed.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 62 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.22 Bugs/errors/defects introduced by previously applied
Peak fixes
This term covers three issues identified by Mr Coyne.
5.22.1 Horizon System Affected
Legacy Horizon.
5.22.2. Description
5.22.2.1 Issue 1
If incorrect keys were pressed while generating a certain report, the screen could freeze.
This was identified on a Training Counter in test, and it was determined that it could also occur in the live
system, but the Peak records no evidence that the issue ever did occur in live.
Mr Coyne incorrectly identified this as a BED which was introduced by previously applied fixes. We
understand Coyne based this on the use of the word “regression” in the Peak (PC0053160). However,
the term “regression” was mistakenly used by the tester who seemed to suspect that the fix of the original
problem that led to the Peak being raised had regressed the counter application to an earlier version.
This suggestion was then contradicted by a developer who pointed out that the test rig had been
incorrectly set up by adding individual work packages instead of using an approved combination of work
packages.
5.22.2.2 Issue 2
An SPM reported a discrepancy between the system cheque figure and the declared figure. It was
thought to be due to code regression caused by a fix for another bug.
5.22.2.3 Issue 3
The two Peaks identified by Mr Coyne were raised in Test, not in Live.
Mr Coyne may have believed this to be an issue in Live because a FAD code is recorded in the Peak, but
the FAD in question (206511 Danby House) was included in the test environment, and the Peak is
categorised as B for Business Integration Testing, rather than L for Live.
5.22.3 Dates
Issue 1: August — September 2000.
Issue 2: January — September 2004.
Issue 3: June — August 2000.
5.22.4 What happened
5.22.4.1 Issue 1
A Fujitsu test team that was testing a future release found a bug in the training environment, described as
"Training Counter freezes using transaction log. This manifests itself in training, if a delegate mis-hears or
miskeys a keying sequence in doing a transaction log report."
Tests on the software that was currently in use in live determined that if the same sequence of events
was performed, the same problem might arise. There is no evidence recorded that it had ever arisen in
the live system, but it was decided to produce a fix to ensure that it could not.
5.22.4.2 Issue 2
© Copyright Fujitsu 2021
pyrig 1) FUJITSU CONFIDENTIAL Ref: COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS PageNo: 63 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
POL NBSC reported to Fujitsu that an SPM had negative figures in his Cash Account which the SPM
could not have entered. Fujitsu contacted the SPM to obtain further information so that the issue could
be fully investigated.
A discrepancy occurred between the cheque figure entered on the system and the declared cheque
figure; instead of the discrepancy being cleared it was doubled, and cash was also found to be wrongly
adjusted.
It was acknowledged that this was a bug introduced as a result of code regression occurring when fixing
an earlier bug (PC097081).
5.22.4.3 Issue 3
Within the test environment only, a new bug (PC0049702) that caused a discrepancy to appear on the
Counter Detected Reconciliation Errors (TPSC252) report was introduced as a result of fixing an issue in
a previous test Peak (PC0047518).
5.22.5 How it was spotted
5.22.5.1 Issue 1
Identified by a Fujitsu Test team during Testing.
5.22.5.2 Issue 2
An SPM reported the original issue to POL NBSC who passed it on to Fujitsu.
5.22.5.3 Issue 3
Identified by the Fujitsu Test team during Testing.
5.22.6 How it was actioned
5.22.6.1 Issue 1
A fix was rolled out to live on 6 September 2000 in order to prevent any such occurrence in live, although
there is no evidence that it ever had occurred in live.
5.22.6.2 Issue 2
Fujitsu SSC spoke to the SPM and advised how to declare the cheques in the light of the newly
introduced software error. The SPM followed the procedure and agreed that the error had then gone
away. The SPM was advised that the method the SPM had used previously would no longer work.
A software fix was included in release S60 (released August — September 2004).
5.22.6.3 Issue 3
The new issue (recorded in both PC0049702 and PC0052776) that arose in Test as a result of a previous
fix (PC0047518) was itself fixed by an updated Report Processor file. The Peak records that this fix was
then tested and no recurrence of the problem was found.
5.22.7 Impact
5.22.7.1 Issue 1
There was no evidence that the bug caused any events in the live system.
5.22.7.2 Issue 2
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 64 of 80
POL00030528
POL00030528
oo THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
As described in 5.22.6.2, the immediate problem that the SPM had reported was resolved following
advice. No other records of this particular issue have been found.
5.22.7.3 Issue 3
There would have been no impact on SPMs as the issue occurred in Test only, not in Live.
©C ight Fujitsu 2021
‘opyright Fujitsu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 65 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.23 Bureau de change
This term covers three separate issues identified by Mr Coyne, only connected by their relation to Bureau
de Change. Two of the issues were the result of users not following the correct procedure.
Fujitsu noticed that in the Written Closing Submissions at the Horizon Issues Trial, the Incidents identified
below as Issue 3 were not considered by POL, due to identifying one instance of Issue 1 as Issue 2, and
then identifying the Peak included below as part of Issue 2, as relating to Issue 3. Due to this
misidentification, POL ascribed all of the Incidents in this BED to user error, whereas Fujitsu believes only
Issues 1 and 2 were caused by user error, while the Incidents collected under Issue 3 required various
fixes to be made, as described below.
5.23.1 Horizon System Affected
Issue 1 and Issue 2: Legacy Horizon.
Issue 3: Horizon Online.
5.23.2. Description
5.23.2.1 Issue 1
An SPM reversed a currency transaction but did it incorrectly. Notable for the fact that the SPM wrote to
The Subpostmaster magazine about it. There was a second instance of a similar error a few months
later.
5.23.2.2 Issue 2
An SPM reported a discrepancy after Remming out currency, reversing it, then re-Remming them out.
This was identified as being possibly caused by the SPM not physically counting the cash when declaring
it.
5.23.2.3 Issue 3
Mr Coyne identified a series of Peaks relating to issues with the bureau rate board display of exchange
rates. Fujitsu believe none of the Incidents had the potential to affect branch accounts. Three were
coding fixes and two were hardware replacements.
5.23.3 Dates
Issue 1: December 2005 — July 2006.
Issue 2: November 2007 — December 2007.
Issue 3: June 2010 — April 2011.
5.23.4 What happened
5.23.4.1 Issue 1
The SPM had attempted to reverse a transaction for the sale of €1,000 (worth £750), but reversed the
settlement instead. The SPM then attempted to compensate for the resulting discrepancy by adjusting
the stock, leaving the margin for the transaction as a loss of £30.
Fujitsu identified that the SPM did not reverse the transaction correctly, thereby causing the loss. Fujitsu
SSC made several suggestions in the Peak: that the POL Operations Manual might be reviewed for
clarity; that it was important that the POL helpdesk (NBSC) understand how the process worked so that
they could diagnose such issues when they arose; and that a warning might be added if a transaction
being reversed is cash.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 66 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
The Peak records that the issue was escalated to POL, but it was also noted that Fujitsu were unable to
implement any changes without guidance from POL.
A further instance of the same error being made was identified six months later.
5.23.4.2 Issue 2
An SPM reported a discrepancy of £907.97 on the main SU which was believed to relate to currency
transactions; the currencies had been Remmed out, reversed, then re-Remmed out.
Investigation by Fujitsu SSC indicated that the SPM was making incorrect cash declarations, and POL.
NBSC were advised of this.
Analysis showed instances of cash declarations, followed by a Rem in or a transfer in that would affect
the cash figure, followed by a further cash declaration for the same amount as the first. This would create
a discrepancy and suggests that the SPM was not actually counting the cash when making the second
declaration.
5.23.4.3 Issue 3
There was a number of instances relating to the rates displayed on the rates board.
PC0200042 records that a software error led to a rate being rounded down within HNG-X so that the rate
displayed at a branch using HNG-X (9.269) was fractionally different from that displayed at a Horizon
branch (9.2691). The same rate would have been applied by the two systems to any transactions carried
out, so would not have affected branch accounts.
PC0200090 records a similar issue in which the Model Office rate boards were not showing "trailing
zeroes" (so 10.1 displayed instead of 10.100). This could not have affected branch accounts.
PC0201340 records that the rates showing on the SPM's rate board were different from those on
Horizon. This was a known issue post-migration to HNG-X. The issue could not have affected branch
accounts.
PC0209240 records two issues raised by the SPM. Firstly, the rates on the rate board differed from
those on the counter; this was due to the rate boards having a 6-character display limit, so that currencies
for which the exchange rate was over 10 to £1 were rounded up or down to fit into the available space.
There was no fix for this, it was a limitation of the rates board.
The second issue related to an actual difference in some rates between the rates board and Horizon,
beyond that explained by the rounding issue. This issue was intermittent and could be resolved
temporarily by forcibly refreshing the board from the counter. It was determined that the issue had arisen
following a data centre change (R4 Data Centre), after which the file feeds from First Rate that supplied
the spot rate and margin values for the day, started to arrive with a gap of only five seconds or less
between each other, instead of with a gap of around a minute, as they had previously. It was suspected
that the rates board was failing to refresh from values as the files arrived in rapid succession. The fix was
that First Rate started sending the files a few minutes apart.
Neither of the issues in this Peak had an impact on branch accounts.
5.23.5 How it was spotted
5.23.5.1 Issue 1
Fujitsu were alerted to the SPM's letter published in Subpostmaster magazine. The SPM stated that they
had raised the issue with the helpdesk when it happened. Fujitsu searched for, but found no record of
any related call to the Fujitsu helpdesk (HSD) from the SPM at the time the Incident was alleged to have
happened, and do not know if POL had a record of any such call to the POL helpdesk (NBSC).
5.23.5.2 Issue 2
The SPM reported the discrepancy to the POL helpdesk (NBSC) who passed it on to Fujitsu.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 67 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.23.5.3 Issue 3
The majority of these various issues were reported by SPMs to the Fujitsu helpdesk (Service Desk) or to
the POL / Atos helpdesk. In two cases the issues were detected by Fujitsu and POL testers respectively.
5.23.6 How it was actioned
5.23.6.1 Issue 1
No fix was applied as the issue was that the SPM did not follow the proper processes. A KEL
(AChambers2252R) was created as a result of this Peak in case there were further instances of such
errors.
5.23.6.2 Issue 2
No fix was applied as the issue was that the SPM was incorrectly making cash declarations. Fujitsu
explained to POL NBSC the type of error that the SPM was making.
5.23.6.3 Issue 3
PC0200042: a coding fix was issued by Fujitsu in September 2010.
PC0200090: a coding fix was issued by Fujitsu in September 2010.
PC0201340: an engineer replaced a node in the rate board which had failed to migrate to HNG-X.
PC0209240: there was no fix for the rounding that was applied to certain currencies, as it was a limitation
of the specified rates board. The intermittent issue of the rates board failing to refresh correctly in the
morning was fixed by First Rate sending the files a few minutes apart.
5.23.7 Impact
5.23.7.1 Issue 1
Fujitsu understands the receipt printed would show that it was the wrong entry that had been reversed.
5.23.7.2 Issue 2
Having passed the results of the investigation back to NBSC, the Peak was closed. Fujitsu has no record
of any action taken by POL in respect of the SPM, and any decision as to whether action should be taken
was wholly the responsibility of POL.
5.23.7.3 Issue 3
Fujitsu understands that none of the issues in this category had the potential to affect branch accounts as
they all related only to the rates being displayed, not to the actual rates used in Horizon.
Fujitsu understands that the amounts displayed to the public on the rate board appeared slightly different
to what they actually received, although the rates were the same.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 68 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.24 Wrong branch customer change displayed
5.24.1 Horizon System Affected
Legacy Horizon
5.24.2 Description
When a quantity greater than 1 was entered for a Smartpost Transaction, the quantity was not reset back
to 1 when the user moved on to the settlement screen. This scenario could lead to discrepancies due to
subsequent items in the session being multiplied by whatever quantity remained.
Although a fix was made, a further instance was then reported, leading to a second release of the fix.
5.24.3 Dates
November to December 2005.
5.24.4 What happened
An SPM noticed that incorrect quantities appeared on the settlement screen after entering stamps or
postage labels on Smartpost.
This could result in subsequent items in the session being multiplied by that incorrect quantity and so
affect further items being sold, or the amount being tendered towards settlement.
On this first report of the error a Fujitsu engineer attended the branch on the same day and changed the
keyboard and screen which did not rectify the issue, so it was referred to Fujitsu SSC as a suspected
software issue. Fujitsu SSC spoke to the SPM to obtain further information and it was identified that the
issue was likely to have started following changes made to Smartpost which were implemented as part of
a release on 24 October 2005.
The issue was passed to the Development team who provided a fix via reference data, which went live
two weeks after the original Peak was opened.
Another instance of the same problem was reported by another SPM in December 2005. It was realised
that the fix had only been released to one group of active branches while another group had not received
it. The situation was explained to the SPM by the Fujitsu SSC staff investigating. The fix was then
released to the remainder of branches on the next day. It was also known that non-polling branches
would only obtain the fix the day after they next ran Cleardesk overnight.
5.24.5 How it was spotted
It was reported by SPMs, as the incorrect quantity was displayed on the settlement screen.
5.24.6 How it was actioned
The fix is described above in 5.24.4, the fix first being made live via reference data on 18 November
2005. Following the second Incident it was determined that not all branches had received the fix, and so
the second tranche of branches received the fix on 8 December 2005.
5.24.7 Impact
Fujitsu understands that this bug only affected Smartpost for a relatively short duration (43 days in total).
Fujitsu has no records of any effects on branch accounts or actions that POL may have taken as a result.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 69 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.25 Lyca Top Up
5.25.1 Horizon System Affected
Horizon Online.
5.25.2 Description
Having entered a Lyca phone Top Up transaction at the counter, the counter was unable to process the
authorisation response returned from E-Pay, and the SPM would have to recover the transaction. If that
recovery was done incorrectly, then a branch account discrepancy would result, but if it was done
correctly, then a discrepancy between POL and E-Pay would result.
5.25.3 Dates
August 2010.
5.25.4 What happened
When entering a Lyca mobile phone Top Up transaction, the counter was unable to process the
authorisation response returned from E-Pay. This resulted in the on screen error message "unable to
connect to data centre", followed by the user being logged out.
This would lead to one of two discrepancies, depending on the actions carried out in the branch following
the above error:
1) If the user recovered the transaction and confirmed that it had been successful by pressing 'Yes', a
shortfall would be created for the branch to the value of the E-Top Up transaction, as a successful
transaction would have been recorded in the branch accounts, indicating that money had changed hands.
However, no money would have been taken from the customer.
2) If the user recovering the transaction confirmed that it had not been successful by pressing 'No’, then
no discrepancy would be created for the branch, as the transaction would be recorded as ‘zero value’ in
the branch accounts, and a reversal generated for the Top Up. Instead, it would create a mismatch
between the data held by POL and E-Pay. The discrepancy would have been flagged in the daily
automated NB102 report. Fujitsu would then issue a BIMS Incident report to POL, summarising the
particular occurrence and explaining the potential discrepancy to enable POL to reconcile the position
with the branch.
A KEL (ballant}020J) was raised to document the fix and allow any further instances to be identified and
investigated.
5.25.5 How it was spotted
More than one SPM noticed and reported the issue, resulting in Peaks PC0202925 and PC0203108.
It was also detected by the BAU automated NB102 reconciliation report, resulting in Peaks PC0203215
and PC0203284.
5.25.6 How it was actioned
From analysing the log of the failed Lyca transaction Fujitsu determined that incorrect reference data was
the immediate cause of the error. The root cause of the reference data error was found to be due to an
issue in Fujitsu's RDT tooling that was used to generate the Reference Data from the original data
provided by POL.
The error in the live system was fixed by a corrected Reference Data release on 19 August 2010. The
underlying problem to the RDT tooling was fixed by a simple change by the Fujitsu Reference Data
Team.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 70 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.25.7 Impact
Fujitsu understands several branches were impacted by this bug. As explained in 5.25.4, it is unclear
whether the recovery impacted branch accounts.
PC0203284 and PC0203215 are examples of BIMS being issued to POL by Fujitsu to reconcile
discrepancies caused by this bug.
Fujitsu has no record as to whether POL issued a general warning to SPMs that the bug was present at
the time, or whether POL relied on SPMs reporting the issue when it arose.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 71 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.26 TPSC 250 Report
This term covers five issues identified by Mr Coyne which all relate to reconciliation report TPSC250.
5.26.1 Horizon System Affected
Legacy Horizon.
5.26.2 Description
5.26.2.1 Issue 1
Acounter code bug caused a failure to check that a prepaid amount was not greater than the total
amount, resulting in incorrect exceptions in reconciliation reports.
5.26.2.2 Issue 2
Five records were incorrectly written after recovery from a Data Centre communications issue resulting in
incorrect exceptions being recorded in reconciliation reports (i.e. not real exceptions in branches).
5.26.2.3 Issue 3
The TPS total for one branch was higher than the counter total due to the totals for day 0 and day 1 being
rolled into one.
5.26.2.4 Issue 4
Exceptions in a reconciliation report for a branch led to identification that an individual counter had errors
and it was replaced.
5.26.2.5 Issue 5
A particular Smartpost transaction type produced intermittent exceptions on reconciliation reports.
5.26.3 Dates
Issues 1, 2, 3, and 4: 2005.
Issue 5: 2008 - 2010.
5.26.4 What happened
5.26.4.1 Issue 1
This relates to an issue arising from Smartpost code failing to check that a prepaid amount is not greater
than the total amount, and resulted in Host system reconciliation errors. These false errors were then
detected and reported on by automated reconciliation reports.
The issue is limited to a reconciliation reporting issue and is separate to branch accounts. The effect
would have been to create unnecessary work for Fujitsu SSC investigating false reconciliation issues.
5.26.4.2 Issue 2
This was a related but separate bug to issue 1 above that also caused incorrect reconciliation exceptions
to be reported, and, again, there was no impact on branch accounts.
The fault was found to be that a message with an incorrect date format could be written to a database
object. This occurred when the End of Day Agent failed to communicate with Riposte, probably due to a
temporary communications issue, causing the code to follow a recovery path resulting in the incorrectly
formed attribute.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 72 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
The issue only affected the reconciliation reports which effectively took a copy of the branch accounts for
onward progression to POL's back-end systems.
5.26.4.3 Issue 3
Peak PC0122664 raised as a result of the TPS Total in the TPSC250 report being higher than the
Counter Total for a single branch. It was confirmed that the files/data sent to POL were unaffected by the
issue, and that this was another case of a false error appearing on the report.
The error was that the TPS Total rolled the totals for day 0 and day 1 into one, meaning that the Counter
Total was correct for day 1, but didn't match the TPS Total, which was in itself also correct.
5.26.4.4 Issue 4
PC0122766 was opened as a result of the TPS Total in the TPSC250 report showing as higher than the
Counter Total for a single branch.
Investigation showed that the error was caused by Riposte errors on counter 1 at the branch. The branch
counter was replaced
5.26.4.5 Issue 5
Asingle branch showed that the TPS Total and Counter Total Values for the Number and Absolute
Quality columns were the same, but with a difference in the Absolute Value, which is greater than the
TPS value. This was a mismatch between the TPS Absolute Value and Counter Absolute Values.
KEL AChambers253L was referenced in the Peak, but in this case the problem with Bulk Mails was that
the Credit/Debit tags were written as double that of the Sale Value. This does not cause an impact upon
the branch as these values are only used in the calculation of the reconciliation report at the data centre,
and did not affect the information sent to POL or the branch accounts.
The relevant KEL (Ballantj2547K) documents that related Incidents had occurred where 1) the session
did not net to zero, in which case an incomplete summaries and receipt and payments mismatch would
result, or 2) in which the transaction message was not written but the balancing message was written.
This was then picked up in the datacentre counter measures (TPSC250 and TPSC257), and would cause
areceipts and Payments mismatch at the branch.
5.26.5 How it was spotted
All the identified example Peaks were detected by Fujitsu, due to incorrect exceptions (i.e. "false alarms",
and not reflecting real exceptions in branches) appearing in reconciliation report TPSC250, automatically
raising Peaks which would then require investigation by Fujitsu SSC. If, following investigation, it was
determined that the exception was not real, then no reconciliation would be required, and therefore no
BIMS would be issued to POL, and the Incident was closed.
5.26.6 How it was actioned
5.26.6.1 Issue 1
The root cause was the counter code bug writing values that caused issues with the reconciliation
reports. A fix was created and released in March 2005.
A KEL (AChambers253L) was created to allow related instances to be identified and understood by the
support teams.
5.26.6.2 Issue 2
A software fix was created to ensure that the date formats would be correctly applied if the circumstances
arose again.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 73 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
The record in PCO125210 shows that the fix was applied but regressed due to an unrelated problem, but
then reapplied on or after 06-Sep-2005.
5.26.6.3 Issue 3
The fix was exactly the same as applied for issue 2.
5.26.6.4 Issue 4
The branch counter identified as being faulty was replaced. A check was made that confirmed that
Counter Totals then matched the TPS total as reported in the TPSC250 report, so no further
reconciliation was required.
5.26.6.5 Issue 5
As the office did not appear on the incomplete summaries report, this meant that all amounts matched.
There was therefore no need for any reconciliation, so the Peak was closed.
5.26.7 Impact
5.26.7.1 Issue 1
These were not real exceptions and Fujitsu understands there was no impact on SPM accounts.
5.26.7.2 Issue 2
These were not real exceptions and Fujitsu understands there was no impact on SPM accounts.
5.26.7.3 Issue 3
These were not real exceptions and Fujitsu understands there was no impact on SPM accounts.
5.26.7.4 Issue 4
Fujitsu understands no transactions had been done on the affected counter 1 that day and that the totals
on the host matched all the transactions done on the unaffected counter 2. Fujitsu understands that it is
therefore unlikely that there would have been an impact to the SPM.
5.26.7.5 Issue 5
Fujitsu understands there was no issue in terms of the branch appearing in the incomplete summaries
report. Fujitsu understands all the amounts matched, and therefore there was unlikely to have been an
impact on branch accounts.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 74 of 80
POL00030528
POL00030528
fe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.27 TPS
5.27.1 Horizon System Affected
Legacy Horizon
5.27.2 Description
Smartpost wrote slightly corrupt transactions which led to incorrect exceptions reporting, but did not
impact branch accounts.
5.27.3 Dates
Peaks were opened in relation to this issue between 2006 and 2010.
5.27.4 What happened
A Peak (PC0141445) was opened following the detection of a Smartpost transaction in which the Credit
attribute doubled against the sale value. The attribute concerned was used to calculate the
‘EPOSSDailyRecon' absolute values, which in turn triggered the TPSC250/257 exceptions in the case of
a mismatch with the Host generated Totals. A check of the messagestore for non-zero sessions found
none, so it was concluded that there was no balancing problem for the branch concerned.
A KEL (BallantJ2547K) was raised as a result. This bug affected Smartpost transactions which were
either missing the grammar attribute and / or had a corrupted grammar attribute. The grammar attribute
is used to calculate the value for the reconciliation reporting, meaning that as the attribute was either
missing or corrupted then reconciliation reporting errors were produced.
Of the 40 Peaks identified that reference the KEL, the majority (36) had SaleValues that netted to zero
and therefore required no reconciliation. The remaining four required corrections for small amounts to be
made to the TPS_POL_FS_Summaries_Incomp table, which were carried out under OCRs.
5.27.5 How it was spotted
As stated above, the original problem was detected as part of Fujitsu's daily BAU monitoring of
reconciliation reports, when the branch appeared on two automated reports, TPSC257 (POLFS
Incomplete Summaries Report), and TPSC250 (Host Detected Transaction Control Errors). Investigation
over a number of months (carried out under PC0141145) failed to determine the underlying cause.
5.27.6 How it was actioned
Reported Incidents were investigated as part of the BAU reconciliation reporting process, with any
genuine mismatches being corrected. Despite investigation, Fujitsu were unable to determine the root
cause of the erroneous reporting, and this bug ceased to have an effect when Legacy Horizon was
replaced by Horizon Online.
5.27.7 Impact
Fujitsu understands as most cases resulted in a reconciliation reporting issue only, there was no effect on
branch accounts. Fujitsu understands where there was a mismatch, the amounts were small and were
corrected.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 75 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.28 Drop & Go
5.28.1 Horizon System Affected
Horizon Online.
5.28.2 Description
A Drop & Go transaction failed to credit the customer's card, so was retried, resulting in the branch being
debited twice, hence creating a shortfall.
5.28.3 Dates
June 2017, August 2018.
5.28.4 What happened
Acclerk initiated a Drop and Go transaction for £100 which failed due to timeouts, but then a success
message was displayed, so the clerk settled the transaction. The customer checked their balance and
stated that the top up had not been processed. The clerk then performed another Drop and Go
transaction that was successful.
This resulted in a £100 shortfall for the branch as Horizon recorded the top-up twice.
Reconciliation between the Horizon feed and the Accenture Common Digital Platform (CDP) system
identified that only one top-up had been received by Accenture CDP, but two top-ups were present in the
Horizon Batch Feed. The second Horizon transaction matched the CDP transaction, confirming the
problem was with the first transaction.
5.28.5 How it was spotted
The SPM reported the issue to the POL / Atos helpdesk who passed the Incident to Fujitsu. Fujitsu
passed the Incident back to Atos on the same day, having identified the problem as being either with an
AP-ADC Drop & Go script, or as a possible user error. AP-ADC scripts were supplied and maintained by
Atos.
5.28.6 How it was actioned
Fujitsu SSC identified the issue as relating to an Atos script, so the Incident was passed back to Atos for
investigation. Fujitsu are unable to comment further on what occurred after the Incident was passed to
Atos.
An apparently similar Incident occurred in August 2018, and again, the Incident was passed to Atos.
5.28.7 Impact
As stated in 5.28.4, the error resulted in a £100 shortfall for the branch as Horizon recorded the top-up
twice. Fujitsu has no further information on how this was dealt with by POL.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 76 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.29 Network Banking
This term includes two issues identified by Mr Coyne. These are not bugs, errors, or defects within
Horizon, but are instances of problems caused by communications failures outside of Horizon.
In the context of these issues it should be noted that as part of the Fujitsu Reconciliation Service,
automated reports detected discrepancies between the Horizon records of transactions and the Fis’
records. Such discrepancies were reviewed by Fujitsu MSU, routed to Fujitsu SSC for investigation
where necessary, and BIMS issued to POL where appropriate.
5.29.1 Horizon System Affected
Legacy Horizon.
5.29.2 Description
5.29.2.1 Issue 1
An intermittent fault, most likely attributed to the line provided by BT and Energis, caused a transaction to
not complete. A shortfall in the branch account was caused by the SPM handing over the money despite
failure.
5.29.2.2 Issue 2
Intermittent breaks in OLS (Online Services) due to problems with a BT line. Although this issue meant
that no Network Banking activities could be carried out while the service was down, no specific
transactions are referred to in the related Peak.
5.29.3 Dates
Issue 1: October — November 2004.
Issue 2: January 2010.
5.29.4 What happened
5.29.4.1_ Issue 1
An SPM reported that their ISDN line (operated by Energis and BT) was down and had been having
connectivity issues for two weeks. The SPM also reported issues with two customers’ pensions
transactions for £90 and £50, in which the transactions had been declined. The two customers had
returned to the branch the following day stating that the money had been taken from their accounts,
despite the transactions having been declined the day before.
Fujitsu SSC was able to make a voice call on the line, and also undertook analysis of the network
banking messages recorded in the messagestore.
The root cause was identified as being an intermittent fault with the BT ISDN line, and this update was
passed to the SPM. NBSC were also informed of the investigation.
Further checking with the SPM confirmed that one of the transactions (£90) had been correctly refunded
to the customer; the other transaction (£50) was refunded but there was a delay in it appearing in the
customer's account, which resulted in them complaining to the SPM. As a result, the SPM handed the
customer £50 which lead to a shortfall for the branch.
5.29.4.2 Issue 2
An SPM reported a temporary communications issue, potentially caused by adverse weather. The issue
was investigated by Fujitsu, but, after a short period, the communications were restored and the SPM
agreed the Incident could be closed.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 77 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
5.29.5 How it was spotted
5.29.5.1 Issue 1
The SPM noticed and reported the issue, recognising the connectivity issues and reporting the specific
problem with the two pension transactions.
5.29.5.2 Issue 2
The SPM reported issues of Online Services (OLS) unavailability.
5.29.6 How it was actioned
5.29.6.1 Issue 1
Having established that the fault was most likely to be with the Energis / BT ISDN line, Fujitsu SSC.
requested the branch be upgraded to an ADSL line, following which there was an improvement in the
branch's connectivity. Some minor breaks continued however, and this resulted in a call to Energis to
investigate.
The Peak (PC0109642) cloned for investigation includes a record that since the reported issue in
November 2004, the branch in question had been migrated to an ADSL service and was therefore no
longer suffering ISDN line breaks. It was also noted that the “rest of the estate is also migrating towards
one or other of the “always connected" services, so again this particular issue will have a reducing (to
nothing) impact on the service.” The Incident was then closed.
The issues with the pension transactions were also investigated. The SPM confirmed that the £90
transaction had been refunded to the customer, and asked for action to be taken to resolve the £50
transaction. The £50 transaction had been authorised by the bank but the message had not reached the
counter in time due to the line problems. Following authorisation, the bank ring fenced the money, then
after receiving the cancellation, the bank refunded the money to the customer's account. The SPM
stated that they had also paid the £50 back to the customer. This part of the Incident was routed to
Fujitsu MSU for reconciliation purposes.
5.29.6.2 Issue 2
The Peak (PC0142872) record of the investigation shows that the SPM was contacted by Fujitsu SSC
several times to provide additional information in an attempt to diagnose what was causing the service to
go down.
The SPM then reported that the problems had ceased, but requested that the situation be monitored for a
few days. Four days later another check was made with the SPM, and the Incident was closed. It was
suspected that severe weather may have been triggering the outages, and it was noted that any further
occurrences at the branch should be referred to BT / POL to investigate the line itself.
5.29.7 Impact
5.29.7.1 Issue 1
The £50 transaction issue was routed by Fujitsu MSU to POL for reconciliation purposes.
5.29.7.2 Issue 2
Fujitsu understands the impact on the SPM in this case was transitory as the connectivity issues ceased
and the SPM had not complained of any problems with specific transactions or the branch accounts.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 78 of 80
POL00030528
POL00030528
(oe) THE BED REPORT (29 BED AS IDENTIFIED BY FRASER J)
FUJITSU FUJITSU CONFIDENTIAL
6 Information Distribution
This report and any enclosed materials (the “Audit Materials”) are being provided to Post Office Limited
(“POL”) pursuant to POL's request for an audit of the HNG-X services Fujitsu provides (the “Audit”). The
Audit Materials comprise work product prepared by Fujitsu pursuant to requests from POL. Fujitsu has
confined this report to the specific requests from POL and accepts no responsibility for any other matters.
The Audit Materials relate to the current HNG-X environment.
The Audit Materials are confidential and provided to POL for the sole purpose of the Audit. The Audit
Materials may only be shared by POL with KPMG, the external auditors appointed by POL in connection
with the Audit. POL shall take all necessary precautions to ensure that any Audit Materials are: (i) not
used for any purpose other than the Audit and; (ii) not disclosed to any third party (apart from KPMG),
without Fujitsu’s express consent in writing. In particular, it should be noted that:
(i) the Audit Materials may contain highly confidential and sensitive information which, if disclosed,
is likely to significantly increase the risk of cyber and engineering attacks on the HNG-X
environment ;
(ii) the Audit Materials may contain personal data within the meaning of the General Data Protection
Regulation (“GDPR’); and
(ili) any system architectural content may be subject to copyright and/or other intellectual property
rights and cannot be shared or disseminated.
Prior to making any permitted disclosure of the Audit Materials (or any part thereof), POL shall provide
Fujitsu with reasonable advance notice of such intended disclosure and shall permit Fujitsu the
opportunity to redact information including but not limited to any privileged information, personal data
and/or other commercially sensitive or proprietary content.
This report refers to various documents that are confidential and internal to Fujitsu. Such confidential
documents are proprietary to Fujitsu and are not intended for sharing outside of Fujitsu. Fujitsu in no way
waives or intends to waive confidentiality in these documents by describing, referring to, reproducing
extracts of, or in any way referencing these documents in this report. Where extracts of such documents
are reproduced in this report, redactions have been applied to protect personal and sensitive information.
The Audit Materials, or any part thereof, may not be altered or amended without Fujitsu's express.
consent in writing. Under no circumstances shall any Fujitsu personnel be named or identified in any
reports or other documents created by POL based on information from the Audit Materials (or any part
thereof). Attribution of any Audit Materials shall be to Fujitsu only.
Unless agreed specifically in writing to the contrary Fujitsu does not accept any duty of care or any other
legal responsibility wnatsoever to any person or entity in relation to this Report, any related enquiries,
advice or other work. Any person who receives a draft or copy of this Report (or any part of it) or
discusses it (or any part of it) or any related matter with Fujitsu, does so on the basis that he or she
acknowledges and accepts that he or she may not rely on this Report or any related information given by
Fujitsu for any other purpose.
© Copyright Fujitsu 2021
pynignt Fu FUJITSU CONFIDENTIAL Ref COM/MGT/REP/4169
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 22-Feb-2021
STORED OUTSIDE DIMENSIONS Page No: 79 of 80