POL00358213
POL00358213
THE POST OFFICE GROUP LITIGATION
Claim Nos HQ16X01238,
HQ17X02637 and HQ17X04248
IN THE HIGH COURT OF JUSTICE
QUEEN'S BENCH DIVISION
BEFORE THE HONOURABLE MR. JUSTICE FRASER
BETWEEN:
ALAN BATES & OTHERS
Claimants
- AND -
POST OFFICE LIMITED
Defendant
CLAIMANTS’ PROVISIONAL/OUTLINE DOCUMENT IN
RELATION TO THE HORIZON ISSUES
This provisional outline document is served in compliance with paragraph 15 of the Third
CMC Order and has been prepared without the benefit of full disclosure and the exchange of
evidence. It should be read with the Claimants’ Generic Particulars of Claim and Reply.
POL-BSFF-0183934
POL00358213
POL00358213
BUGS, ERRORS AND DEFECTS IN HORIZON
Accuracy and integrity of data
a)
Ll
To what extent was it possible or likely for bugs, errors or defects of the nature alleged
at §§23 and 24 of the GPOC and referred to in §§ 49 to 56 of the Generic Defence to
have the potential to (a) cause apparent or alleged discrepancies or shortfalls relating to
Subpostmasters’ branch accounts or transactions, or (b) undermine the reliability of
Horizon accurately to process and to record transactions as alleged at §24.1 GPOC?
The Claimants’ outline case is that it is clear that bugs, errors and defects have affected
both Subpostmasters’ branch accounts and transactions and have undermined Horizon’s
reliability. The fact that these bugs, errors or defects could arise, and that they have in
fact arisen, is consistent with, if not probative of, the possibility and likelihood that other
similar bugs, defects and errors have arisen which would similarly affect
Subpostmasters’ branch accounts and/or transactions. The Defendant admits that certain
bugs or errors in Horizon have occurred and that some of them resulted in discrepancies
and thus shortfalls in branch accounts (paragraph 56(1) of the Generic Defence and
Counterclaim).
It is common ground that the Horizon system is not perfect and it is not asserted by the
Defendant that it has never had any errors or bugs. It is also agreed that the volume of
transactions effected through Horizon is very large. The admitted imperfections in
Horizon (which the Defendant maintains result in a low probability of errors arising) are
entirely consistent with the levels of errors asserted by the Claimants, when regard is had
to the millions of transactions processed by Horizon.
The bugs, errors or defects admitted by the Defendant, include:
(i) the Calendar Square/Falkirk bug — which led to Horizon failing to recognise
transfers between different stock units, thereby affecting Branch accounts;
(ii) the Payments Mismatch defect — which affected at least 62 branches and was
related to the process of moving discrepancies into the local suspense account,
POL-BSFF-0183934_0001
POL00358213
POL00358213
thereby affecting Branch accounts. This defect was not capable of being
identified by Subpostmasters, who would have believed from Horizon that their
account was balanced when it was not; and
(iii) the Suspense Account bug — which erroneously replicated suspense account items
for 14 branches from 2010 in the same monthly trading periods in 2011 and 2012,
thereby affecting Branch accounts.
It is alleged by paragraph 56 of the Generic Defence and Counterclaim that no
Subpostmasters were “ultimately” held responsible for the resultant shortfalls caused by
these bugs, errors or defects. (The evidence shows, for example, that the Payments
Mismatch document shows that for a significant period of time, Subpostmasters were not
informed of a known defect affecting their accounts.) Even taking the Defendant’s case
at its highest, it is therefore self-evident that bugs, errors or defects in Horizon did have
the potential to (a) cause apparent or alleged discrepancies or shortfalls relating to
Subpostmasters’ branch accounts or transactions, or (b) undermine the reliability of
Horizon accurately to process and to record transactions.
Other bugs, errors and defects which have been identified thus far and which caused
discrepancies affecting Branch accounts, and therefore had the potential to do so,
included:
(i) Known Error Log achal233J — causing a discrepancy between branch cash
declarations and the amount received by the Cash Management system;
(ii) I Known Error Log achal1717T — caused by an “unknown system problem” causing
a difference between the cash declared by the branch and the cash which the
system considered that the branch should have had;
(iii) I Known Error Log acha621P — caused the wrong screen to be presented leading to
duplication and/or incorrect entries of Remittance In transactions (see also
paragraph 4.1 below);
(iv) I Known Error Log acha 194L - resulting in cash declarations from kiosk not being
automatically passed into Horizon;
(v) Known Error Log LKiang3014S — after making multiple cash declarations and
then running a trial balance report, the calculation for the trial balance was
incorrect;
POL-BSFF-0183934_0002
(2)
2.1
G)
3.1
POL00358213
POL00358213
(vi) I Known Error Log MScardifield2219S — cached data was not updated via Riposte,
resulting in incorrect data being presented in discrepancy, variance and balance
reports;
(vii) Known Error Log wbra5S353J — a customer was charged twice for the same
transaction, as a likely side effect of bad reference data within Horizon; and
(viii) Known Error Log jsim5530K — reference data controls would not allow the
reversal of cheque rem-out mode transactions.
Did the Horizon IT system itself alert Subpostmasters of such bugs, errors or defects as
described in (1) above and if so how.
The Claimants are not aware of any automated system within the Horizon system to
inform Subpostmasters of bugs, errors or defects, either in relation to matters set out in
paragraph 1 above or at all. The Claimants infer and invite the Court to infer that there
is no such automated system. The way in which the Defendant dealt with the Payments
Mismatch defect (paragraph 1.3(ii) above) demonstrates that errors known to the
Defendant were not automatically communicated to Subpostmasters by Horizon and that
the Defendant (a) did not do so for a significant period and (b) then considered whether
and how to do so. Given that the Defendant has been unable to identify any
communication by which Subpostmasters were informed of the Payments Mismatch
defect, the Claimants will invite the Court to draw the inference that they were not so
informed, unless the Defendant can demonstrate otherwise.
To what extent_and in what respects is the Horizon System “robust” and_extremel:
unlikely to be the cause of shortfalls in branches?
[GPOC §23 and 24; Defence §§49 to 56]
There is no single, authoritative definition of “robust” in the context of a system such as
Horizon, which appears to be more commonly used in public relations than as an
objective performance standard. Pending any clarification of its objective meaning, it
appears to relate to the ability of a system to perform correctly in any scenario, including
where invalid inputs are introduced, namely, to have in place effective error repellency.
POL-BSFF-0183934_0003
3.2
POL00358213
POL00358213
It is clear that, on a sensible construction of the term “robust”, Horizon did not meet this
standard because:
(a)
(b)
(d)
(c)
(9)
it contained bugs, errors and defects as set out in paragraph I above which created
discrepancies in the branch accounts of Subpostmasters;
it suffered failures of internal mechanisms which were intended to ensure
integrity of data. Examples include Known Error Log dsed4733R (causing
multiple failed recoveries as a result of a wrongly named recovery script) and
Known Error Log obenge5933K (transaction only recovered partially following
a loss of communications);
the system did not enable such discrepancies to be detected, accurately identified
and/or recorded either reliably, consistently or at all;
the system did not reliably identify ‘Mis-keying’, which is inevitable in any
system with user input, and did not reliably have in place functionality to restrict
users from progressing a mis-key;
it required numerous processes and workarounds to be in place to allow Fujitsu
to modify and correct data already recorded by Horizon, which would not be
required in a “robust” system; and/or
there were weaknesses and risks of errors and other sources of unreliability within
Horizon, of which the Defendant was well aware from internal or third party
reports, examples of which the Claimants have only recently identified in Stage
3 Disclosure, as follows:
(i) POL-219218 — Ernst & Young report to the Defendant on, inter alia,
weaknesses relating to the control environment operated by the Defendant’s third
party IT suppliers for year ended 27 March 2011;
(ii) POL-217341 — Defendant’s report (Review of Key System Controls in
POLSAP) dated November 2012 highlighting the potential for errors not being
identified or resolved leading to discrepancies in client balances and the risk of
inappropriate access being assigned to users, leading to inappropriate activities
being undertaken within the system;
POL-BSFF-0183934_0004
POL00358213
POL00358213
(iii) POL-217378 — Minutes (redacted) of a meeting on 18 September 2013 of the
Defendant’s Risk and Compliance Committee recording a decision not to mitigate
risks identified by Ernst & Young relating to the communication by Fujitsu of
changes made to the Horizon system; and
(iv) I POL-0216106 — Report (draft — 1 October 2013) by Detica (BAE Systems)
‘Fraud and Non-conformance in the Post Office’; Challenges and
Recommendations’ concluding, inter alia, (a) ineffective process, policy and
working practice across the various Post Office central teams to gather
information, prioritise and act in a coordinated manner; (b) technology available
to central teams are not fit for purpose; and (c) Post Office systems are not fit
for purpose in a modern retail and financial environment.
3.3. However, as noted above, whether Horizon is ‘robust’ plainly depends upon the
definition given to that word. Even the small chance of errors, bugs and defects admitted
by the Defendant and/or supplemented by those underlying the reports above, would be
likely to produce the result alleged by the Claimants. Therefore, as noted above, even if
the overall probability of a bugs or errors affecting Branch accounts is small, the sheer
number of transactions undertaken by the Horizon system is consistent with the level of
discrepancies arising from bugs and errors in issue in these proceedings.
Controls and measures for preventing / fixing bugs and developing the system
(4) To what extent has there been potential for errors in data recorded within Horizon to arise
in (a) data entry, (b) transfer or (c) processing of data in Horizon?
4.1 There were inadequate controls incorporated as part of Horizon to prevent and/or remedy
data entry errors, as evidenced by the ability to select and enter different methods of
payment (Known Error Log allend1645p) and the non-appearance of the correct screen
to successfully process a cash pouch, which caused a clerk to inadvertently double up the
amount of cash recorded (Known Error Log acha621P).
4.2 There was the potential for errors with the transfer of data in Horizon, as evidenced by:
POL-BSFF-0183934_0005
43
(5)
5.1
POL00358213
POL00358213
(a) a transaction authorised by a bank but cancelled by pin pad, which was not reversed
(Known Error Log carde219R);
(b) a successfully recorded transaction failing to appear in the PODG file and
consequently not being transferred to the relevant third party (the Environment
Agency) (Known Error Log jharr1323L); and
(c) _ the erroneous recording of the same transaction twice after a session transfer to a
different counter, which was recognised as a possibility for both NS&I and
Network Banking transactions (Known Error Log Marris34331).
There was the potential for errors in the processing of data, as evidenced by:
(a) _ the system bug which led to both the old value and the amended value being stored
and used in a transaction when a counter level correction was made via the
“Previous Key” (Known Error Log CharltonJ2752T);
(b) a declined network banking transaction that still resulted in money being taken
from the customer’s account (Known Error Log SSur343P);
(c) E-pay transactions crediting the customer account twice, although only one
payment had been taken (Known Error Log LKiang3526R and Known Error Log
SSur5310P);
(d) POLSAP outage — 25 & 26 January 2016 causing £90m of inward remittances from
branches failing to be processed (see document POL-216412); and
(e) ‘Accuracy of the FAT’ where figures recorded on the R&B and Tier 2 FAT have
not been accurate and have not agreed with the figures on Horizon (POL-0216223).
How, if at all, does the Horizon system itself compare transaction data recorded by
Horizon against transaction data from sources outside of Horizon?
It is presently understood that the process by which the Horizon system compares
transaction data recorded by it against transaction data derived from sources outside of it
is known as Reconciliation. This comprises a large number of complex, electronic
systems to monitor transactions but there is also the ability for manual intervention to
POL-BSFF-0183934_0006
(6)
6.1
POL00358213
POL00358213
establish and correct erroneous data. Those systems presently identified by the Claimants
are listed at Appendix 1. The scope for error in Reconciliation depends, in part, upon the
interface between these systems. The Claimants believe that manual interventions could
be undertaken by at least the POL Finance Department, the Fujitsu Management Support
Unit and the Fujitsu Third Line Support. How Horizon’s functionality allowed these to
be made is still being considered, but errors occurring in the electronic interface between
systems and the exchange of data (e.g. erroneous or duplicate data) appears to be a
significant risk area for generating errors which may affect Branch accounts.
To what extent did measures and/or controls that_existed_in Horizon prevent, detect.
identify, report or reduce to an extremely low level the risk of the following:
a. data entry errors;
b. data packet or system level errors (including data processing, effecting, and
recording the same);
c. a failure to detect, correct and remedy software coding errors or bugs;
d. errors in the transmission, replication and storage of transaction record data; and
e. the data stored in the central data centre not being an accurate record of
transactions entered on branch terminals?
[GPOC §§5, 14-15, 24.1, 24.1A, 94A, 95;
Defence §§35(2), 36, 38(1), 50(1), 52-54; Reply $41]
The Claimants refer to the matters set out above under issue (4) and find it difficult to
identify any such measures and controls, let alone measures and controls which were
effective to prevent, detect, identify, report or reduce to an extremely low level the risk
of the errors referred to above. On the basis of the Claimants’ investigations to date, such
measures and controls as were incorporated were insufficient and inadequate for the
purposes referred to in the question, as evidenced by:
(a) the inadequate controls incorporated as part of Horizon to prevent and/or
remedy data entry errors, as referred to in paragraph 4.1 above;
(b) the issuing of a Transaction Correction rendering the Subpostmaster liable
following a reversal initiated by Horizon (Horizon data Lepton SPSO 191320);
POL-BSFF-0183934_0007
POL00358213
POL00358213
(c) the absence of a formal reconciliation produced between the POLSAP system
and the Credence transaction scheme; and
(d) the absence of an instantaneous or prompt fix for bugs, errors and defects
detected or identified. Such issues once discovered were often deferred and
dealt with only on a cost/benefit basis.
OPERATION OF HORIZON
Remote Access
(7) Were Post Office and/or Fujitsu able to access transaction data recorded by Horizon
remotely (i.c. not from within a branch)?
[Defence §7; Reply §9]
7.1 The Defendant has publicly stated that it was unable to access transaction data and edit
or delete transactions recorded by branches (paragraph 26 of the Generic Particulars of
Claim).
7.2 The Defence now admits that this was incorrect (paragraphs 56-58 of the Generic
Defence and Counterclaim): such remote access was possible.
7.3 The Claimants are still investigating the scope of the functionality by which this was
possible.
Availability of Information and Report Writing
(8) What transaction data and reporting functions were available through Horizon to Post
Office for identifying the occurrence of alleged shortfalls and the causes of alleged
shortfalls in branches, including whether they were caused by bugs, errors and/or defects
in the Horizon system?
[Defence §7; Reply §9]
8.1 It is believed that the following data and reporting functions were available to the
Defendant:
(a) the TPS Report Set;
POL-BSFF-0183934_0008
8.2
(9)
9.1
POL00358213
POL00358213
(b) the APS Report Set;
(c) the DRS Report Set;
(d) an ARQ report (which would provide substantial information on a branch
account but which would only be made available to the Defendant on request to
Fujitsu on commercial terms and/or at a cost not presently known to the
Claimants);
(e) reports and data obtained by means of the Business Incident Management
process; and
(f) reports and data obtained by means of the Problem Management Procedure.
The Claimants are still investigating what other data was available to the Defendant.
Atall material times, what transaction data and reporting functions (if any) were available
through Horizon to Subpostmasters for:
a. identifying apparent or alleged discrepancies and shortfalls and/or the causes of
the same; and
b. accessing and identifying transactions recorded on Horizon?
[GPOC §§14.2-14.3, 17 and 19.3;
Defence §§38(2)(b), 38(3), 46(2); Reply §15.2-15.3]
Subpostmasters were able to access limited reports and receipts produced by the
Counter (i.e. the terminal in Branch). These reports provided basic information of the
individual transaction as it appeared on Horizon but would not in themselves enable
apparent or alleged discrepancies and shortfalls and/or the cause for such faults to be
identified. Further, transactions listed on Trading Statements were presented as totals
so that any individual errors could not be identified. In addition, transactions involving
third parties (eg., the National Lottery) could be aggregated for periods that did not
align with the trading periods used by the Defendant. If and insofar as information was
available, this would not enable Subpostmasters to identify any error which occurred
beyond their Branch and even errors made in the Branch were difficult to trace or
identify.
10
POL-BSFF-0183934_0009
POL00358213
POL00358213
Access to and/or Editing of Transactions and Branch Accounts
(10) Whether the Defendant and/or Fujitsu have had the ability/facility to: (i) insert, inject.
edit or delete transaction data or data in branch accounts; (ii) implement fixes in Horizon
that had the potential to affect transaction data or data in branch accounts; or (iii) rebuild
branch transaction data:
a. atall;
b. without the knowledge of the Subpostmaster in question; and
c. without the consent of the Subpostmaster in question.
10.1 The Claimant refers to issue (7) above. The Defendant has admitted that it and/or Fujitsu
had the ability and facility to do what is set out in sub-paragraphs (10) (i) — (iii).
10.2 Investigations undertaken by the Claimants to date suggest that this was possible by
means of the use of:
(i) “global branches’ (with branch codes such as 999998 and 999999), which would
enable the input of transactions within Horizon as though it had come from an
actual Branch;
(ii) the Branch transaction correctional tool; and/or
(vy) the Transaction Information Processing repair tool.
10.2 The Claimants have found no functionality by which use of these facilities would be
visible to, known by or communicated to Subpostmasters either prior to or after their use.
On this basis, the Claimants infer that these facilities could be used without the
knowledge or consent of Subpostmasters and will invite the Court so to find.
(11) Ifthey did, did the Horizon system have any permission controls upon the use of the
above facility, and did the system maintain a log of such actions and such permission
controls?
[GPOC §§21.3, 23, 25; Defence §§48(3), 50, 57]
POL-BSFF-0183934_0010
1
(12)
(13)
POL00358213
POL00358213
The Claimants do not yet have this information. However, the system for the insertion
of corrective Transactions by the Reconciliation Service required authorisation from the
Defendant before such a change was made. The Claimants are unable to confirm
whether such actions were logged and/or whether such permissions were obtained in
practice and/or whether the requirement for authorisation was applied generally. It would
be normal for any such exceptional changes made in an IT system to be logged or some
form of audit trail kept of them. No such log or audit trail has yet been identified, but the
Claimants will continue to investigate.
If the Defendant and/or Fujitsu did have such ability, how often was that used, if at all?
The Claimants refer to issue (11) above. In the circumstances, the answer to this question
is not yet known to the Claimants, but the Claimants will continue to investigate.
To what extent did use of any such facility have the potential to affect the reliability of
Branches’ accounting positions?
[GPOC §§21.3, 23, 25; Defence §§48(3)(c), 57]
The very purpose of such facilities was to be able to affect Branch accounts and/or
transaction data upon which such accounts depend. Their use therefore had this potential.
The Claimants are continuing their investigation.
Branch trading statements, making good and disputing shortfalls
(4)
How (if at all) does the Horizon system and its functionality:
a. enable Subpostmasters to compare the stock and cash in a branch against the
stock and cash indicated on Horizon?
b. enable or require Subpostmasters to decide how to deal with, dispute, accept or
make good an alleged discrepancy by (i) providing his or her own personal
funds or (ii) settling centrally?
c. record _and_reflect_the consequence of raising a_dispute_on_an_alleged
discrepancy, on Horizon Branch account data and, in particular:
POL-BSFF-0183934_0011
14.1
14.2
14.3
POL00358213
POL00358213
i. does raising a dispute with the Helpline cause a block to be placed on
the value of an alleged shortfall; and
ii. is that recorded on the Horizon system as a debt due to Post Office?
d. enable Subpostmasters to produce (i) Cash Account before 2005 and (ii) Branch
Trading Statement after 2005?
e. enable or require Subpostmasters to continue to trade if they did not complete a
Branch Trading Statement; and, if so, on what basis and with what
consequences on the Horizon system?
[Defence §§42-46; Reply §§17.1-17.2, 21]
When a Subpostmaster performs a Daily Cash Declaration, this would show any
discrepancy between cash held by the branch and the amount of cash which Horizon
stated should be in the branch. When a Subpostmaster undertakes a mandatory Monthly
Trading Period Rollover, this requires all discrepancies (including any discrepancies
which had been moved into a suspense account in the intervening period) to be resolved.
A Branch Trading statement is produced at the conclusion of this process
Prior to 2005, branches were required to undertake a weekly balance and produced a
Horizon generated ‘Cash Account’. Discrepancies identified were placed in the Branch’s
cash account (in the “suspense section”). The discrepancy would cither result in the issue
of an error notice or remedial action by the Subpostmaster to balance the account (eg. by
adding money into the Branch).
After 2005, when a discrepancy was identified, a Transaction Correction was sent to the
relevant Branch through the Horizon system, and appeared to Subpostmasters as if it was
generated by the system. The range of options available to a Subpostmaster for
processing the Transaction Correction in Horizon was: (1) Write off to P&L; (2) Assign
to Nominee; (3) Seek Evidence; (4) Stock Write Off; (5) Cancel; (6) Accept Now. The
Horizon system therefore did not acknowledge the possibility of error arising other than
by the fault of the Subpostmaster and there was no ability to challenge a Transaction
Correction using the Horizon system itself. If the option to “Seek Evidence’ was selected,
additional evidence would be provided only once and entirely at the discretion of the
Defendant. A new Transaction Correction would be then issued, but to which there would
POL-BSFF-0183934_0012
POL00358213
POL00358213
be no option for the Subpostmaster to “Seek Evidence” again, even if the evidence sent
by the Defendant had not provided the information sought by the Subpostmaster. The
Claimants understand, from an internal email sent by Mark Dinsdale to Nigel Allen dated
24 June 2010 (at 17:37 hours), that it was not the Defendant’s practice to supply ARQ
data to Subpostmasters in response cither to the “Seek Evidence” option or at all
(although there may have been very occasional exceptions to the Defendant’s practice in
this regard).
14.4 Transaction corrections were communicated to Subpostmasters via Horizon and were
presented to Subpostmasters as being part of the Horizon system. It has transpired,
during the course of the present proceedings, that Transaction Corrections were generated
by the manual entry of data by employees or agents of the Defendant.
14.5 Discrepancies (shortages and surpluses) were, with authorisation from the Defendant,
placed in the Branch "suspense section" of their cash account. This discrepancy was held
until enquiries into the discrepancy were concluded (whether or not the cause of the
discrepancy was identified) and was then removed by the issuing of an error notice (now
known as a Transaction Correction) by the Defendant and / or by the Subpostmaster
putting the money into the branch (or settling centrally) to cover the loss or removing the
value of the gain from the branch to balance the account.
14.6 After 2005, a discrepancy had to be resolved prior to the required date for the Monthly
Trading Period Rollover (unless varied) as Subpostmasters are not permitted to continue
trading until the Branch Trading Statement process is complete.
14.7 A loss is recorded as a debt to the Defendant in the event that the discrepancy is not
resolved in favour of the Subpostmaster.
Transaction Corrections
(15) How did Horizon process and/or record Transaction Corrections?
[Defence §§12, 39-40, 45-46; Reply §21]
POL-BSFF-0183934_0013
POL00358213
POL00358213
15.1 The Claimants refer to issue (14) above. Whilst it is now understood that Transaction
Corrections were generated by manual entries, the Claimants are not aware of any error
repellency measures which were in place or adopted in order to safeguard the accuracy
of this process, whether as to errors in the data from which the manual entries were made
or in the course of making the manual entries themselves.
15.2 Transaction Corrections, if successfully processed, would be the subject of a
confirmatory message. The total volume of Transaction Corrections processed during a
month would appear on the Branch Trading Statement.
15.3 In summary, the Horizon system made it difficult for Subpostmasters to challenge
discrepancies the cause of which was unknown to them.
Dated 17 August 2018
Served by Freeths LLP, Floor 3 West One, 100 Wellington Street, Leeds, LS1 4LT
POL-BSFF-0183934_0014
POL00358213
POL00358213
Appendix 1
a. Banks (LINK A&L, CAPO) for online authorisation of banking transactions (DCS for
debit and credit card authorisations) and transaction data used for reconciliation
b. Online Clients (e-pay, Streamline, DVLA) for online authorisation of transactions and
(for e-pay and Streamline) data used for reconciliation
c. SAP ADS - A Post Office system that handles cash and Foreign Currency logistics. Data
includes cash on hand statements from each branch, planned orders, replenishment deliveries
and delivery/collection data
d. HR SAP - A SAP system that handles remuneration to the branch franchises and
‘multiples’ such as Tesco.
e POL MIS - An Oracle based system to provide MI reporting to Post Office.
f. First Rate - Provides bureau rate information. It is also passed all bureau transactions to
allow First Rate to undertake MI.
g. Siemens Metering - Provides Rates and Customer data for Quantum gas pre-payment
card.
h. Camelot — Provides National Lottery services.
16
POL-BSFF-0183934_0015