POL00112816
POLO00112816
IN CONFIDENCE
This is a judgment to which the Practice Direction supplementing CPR Part 40 applies. It will be handed
down on a date which will be identified to the parties during the week of 2 December 2019 in a court also
to be identified. The draft of Part M will also be provided during that week. This draft is confidential to
the parties, their legal representatives and two members of the steering committee on the same terms as
were imposed for the draft of Judgment (No. 3) and complied with by email to the court of 11 March 2019
from the Claimant’s counsel, and accordingly neither the draft itself nor its substance may be disclosed to
any other person or used in the public domain. The parties must take all reasonable steps to ensure that
its confidentiality is preserved. No action is to be taken (other than internally) in response to the draft
before judgment has been formally pronounced. A breach of any of these obligations may be treated as a
contempt of court. The official version of the judgment will be available from the Courts Recording and
Transcription Unit of the Royal Courts of Justice once it has been approved by the judge and will also be
published on www.bailii.org.
The court is likely to wish to hand down its judgment in an approved final form. Counsel should therefore
submit any list of typing corrections and other obvious errors in writing (Nil returns are required) to the
clerk to The Honourable Mr Justice Fraser, via email at angela.fraser@justice.gov.uk, by 6.00pm on
Thursday 5 December 2019, so that changes can be incorporated, if the judge accepts them, in the handed
down judgment. All corrections to be in separate word documents, and “nil” returns are required.
Case No: HQ16X01238, HQ17X02637 and HQ17X04248
Neutral Citation Number: [2019] EWHC xxx (QB)
THE POST OFFICE GROUP LITIGATION
IN THE HIGH COURT OF JUSTICE
WUEENS BENCH DIVISION
Rolls Building
Fetter Lane
London, EC4A INL
Date: November 2019
Before :
THE HONOURABLE MR JUSTICE FRASER
Between :
Alan Bates and Others Claimant
- and -
Post Office Limited Defendant
Patrick Green QC, Kathleen Donnelly, Ognjen Miletic and Reanne MacKenzie
for the Claimants (instructed by Freeths LLP)
Antony de Garr Robinson QC, Simon Henderson, Owain Draper and Rebecca Keating
for the Defendant (instructed by Womble Bond Dickinson LLP)
Hearing dates: 11, 12, 13, 14, 18, 19, 20, 21 and 27 March;
3,9 and 12 April; 4, 5, 6, 7, 11, 13 and 14 June 2019;
1 and 2 July 2019
Draft distributed to parties on 28 November 2019
POL00112816
POLO00112816
Mr Justice Fraser:
1.
This judgment is part of the long-running legal dispute between the claimants, who are
all either former or serving sub-post masters and sub-post mistresses (“SPMs”), and the
Post Office. All the SPMs in the litigation had contracted with the Post Office at
different times to run branch Post Offices in different locations across the country. The
Post Office introduced a computer system called Horizon in 2000 across all its
branches. That was changed in 2010 to an online version called Horizon Online or
HNG-X, and the former version is now called Legacy Horizon. The claimants’ case is
essentially that both Legacy Horizon and Horizon Online (which used many elements
of the existing system) were or are unreliable, and this led to unexplained shortfalls and
discrepancies in their branch accounts. The Post Office denies this. There are numerous
different causes of action brought against the Post Office, and the Post Office
counterclaims against the claimants, including seeking damages for fraud. This is
Group Litigation under CPR Part 19.
This judgment concems the operation and functionality of the Horizon system itself. It
follows the lengthy Common Issues judgment, which is Judgment (No.3). That
judgment was extremely long, for two main reasons. It concerned six different
claimants, each of whom had contracted with the Post Office at different times, and
each of those six claimants’ cases concerned very different facts. Very few of the
material facts relating to contract formation were agreed for any of these six, and
therefore factual findings had to be made for each of them in this respect.
Approximately 500 paragraphs were necessary to do this. Although first instance
judgments must be reasoned so that parties understand the outcome (and so that any
judgment can be subject to appropriate review), very lengthy judgments can be difficult
to follow. Ifa judgment becomes too long, or too technical, it can be counter-productive
to wider understanding on the part of those not immersed in the fine detail of every
aspect of the case.
Ihave therefore included technical detail about the Horizon system, its operation, and
some aspects of the technical evidence, in a Technical Appendix to this judgment. The
contents of that appendix should not be seen as being of subordinate effect to the
contents of the judgment itself; this is done simply for the convenience of readers. It
also is intended that readers who do need to immerse themselves in technical matters
to a very fine level of detail may not find it necessary to study the appendix, and reading
the judgment alone may be sufficient. Notwithstanding that approach, however, this
judgment too is extremely long. This is due to the complexity of the Horizon system,
which even for computer systems is extraordinarily complicated for the reasons
explained below; the period of time over which the complaints range (which starts with
the introduction of Horizon in 2000); and due to the way that the litigation has unfolded.
There are some matters in this judgment that go to issues affecting the group litigation
going forwards, such as disclosure, which have increased its length. It is also the case
that this litigation is being very strongly contested on both sides. I have endeavoured to
provide substantial detail to explain my findings on the Horizon Issues to assist both
sides. Finally, the only findings that are made in this judgment are those necessary to
come to conclusions on the Horizon Issues. All other matters in all the claims of the
claimants and the counterclaims by the Post Office remain for decision in later
judgments.
Fal
ZO AT TO TOMMOOWP>
POL00112816
POLO00112816
This judgment is in the following parts:
Paragraph no.
Introduction 5
The Horizon Issues 18
Features of this Group Litigation 57
Evidence of Fact: The Claimants 76
Evidence of Fact: The Post Office 202
Documents and Available Information 552
The Experts’ Agreements 647
The Period 21 March 2019 to 4 June 2019 699
The Expert Evidence 724
Conclusion on Expert Evidence 863
Audit Data 898
Overall Conclusions 918
Final Section
Technical Appendix
Introduction
These proceedings are being conducted pursuant to a Group Litigation Order (“GLO”)
made on 22 March 2017 by Senior Master Fontaine, and approved by the President of
the Queen’s Bench Division. A more comprehensive introduction to the issues
generally between the parties is contained at [2] to [43] of Judgment (No.3) “Common
Issues”, which is at [2019] EWHC 606 (QB). Other judgments concerning procedural
rather than substantive issues, are the first Judgment at [2017] EWHC 2844 (QB) and
Judgment (No.2) “Strike Out” at [2018] EWHC 2698 (QB). During the Horizon Issues
trial which is the subject of this judgment, the Post Office issued an application that I
recuse myself as Managing Judge in this Group Litigation, stop the Horizon Issues trial,
so that it could be recommenced at some later date in the future (before a replacement
Managing Judge). That application was refused and the judgment is at Judgment (No.4)
“Recusal Application” at [2019] EWHC 871 (QB). Permission to appeal was refused
by the single Lord Justice on 9 May 2019. There is also Judgment (No.5) “Common
Issues Costs” at [2019] EWHC 1373 (QB) which made various orders in respect of the
costs of the Common Issues trial, a hearing in respect of that having taken place on 23
May 2019.
This trial concerns what the parties referred to at the case management stage as the
Horizon Issues. The intention behind the case management of this litigation was that
the contractual issues (which were called the “Common Issues” and affected all the
claimants) and the computer issues relating to the operation, functionality and reliability
of the Horizon system (which were called the “Horizon Issues”, and also affected all
the claimants) would be resolved first. The parties needed time to prepare for these two
trials, which took place in late 2018 (for the Common Issues) and into the spring of
2019 (for this one), in particular to perform disclosure, and prepare and serve evidence.
For the Horizon Issues, this included expert evidence, for which I gave permission,
from two IT experts, one for the claimants (Mr Coyne) and one for the Post Office (Dr
Worden).
The recusal application caused significant delay and disruption to the Horizon Issues
trial. Originally, that trial had been programmed to finish (including delivery of oral
POL00112816
POLO00112816
closing submissions) by 11 April 2019, a date which became unachievable as a result
of that application. Following Judgment (No.4) the remaining witness of fact for the
Post Office to be called in person, Mr Parker, a senior Fujitsu employee, was called on
11 April 2019.
The two IT experts could not be called until 4 June 2019 onwards. The two experts
therefore had a period of some weeks, following on from the evidence of fact, before
they were called to give their oral evidence. There was therefore an interval between
11 April 2019 and 4 June 2019 when the experts were called. Their evidence
collectively took two court weeks.
The effect of this interruption upon the trial timetable meant that there was therefore
time available for each of the experts to consider the full scope of the cross-examination
of the factual witnesses that had taken place some weeks before, and reflect (with more
time than is usual for experts whilst a trial is underway) upon whether that impacted
upon their views. Experts are supposed to consider the factual evidence advanced by
both sides in litigation neutrally (although whether that occurred fully in this case is a
topic to which I shall return) and if witnesses are cross-examined, then — at least
potentially — other evidence that emerges ought, if it is important, to be considered by
those experts in arriving at their opinions given in evidence. In a conventional trial,
expert evidence will usually follow on almost immediately after the evidence of fact
has been heard. This means that, in practical terms, most experts will only have a very
short time in which to consider any potentially important factual developments in any
case before they are themselves cross-examined. Here, there was no such restriction.
The Horizon Issues trial timetable was therefore different to a conventional trial in this
respect, because the vast majority of the oral factual evidence, with the single exception
of Mr Parker’s evidence, was completed by 21 March 2019. That is 75 days, or almost
11 weeks, before Mr Coyne was called by the claimants, and even longer before Dr
Worden was called by the Post Office. The approach of the experts to the opportunity
presented by that interval further to assist the court with their evidence is notable, in
my judgment. I will deal with this further in Parts H and I of this judgment. For a more
detailed description of the architecture of the Horizon system (both Legacy Horizon
and Horizon Online) reference should be made to the Technical Appendix. For a
general understanding of the issues in this litigation, an overview is sufficient.
Horizon is both an accounting system, and also supports a large number of what are
called business applications. Some transactions that a customer might wish to carry out
in their local Post Office branch are what are called retail activities, such as buying a
Post Office product. One example of this is buying a book of stamps. However, the Post
Office has a large number of business customers whose products are offered to the
public through branch Post Offices. The Post Office calls these companies “clients”.
For every kind of activity which a customer wishes to transact in their Post Office
branch, Horizon needs to have functionality to perform it. This functionality means
supporting the counter activity of carrying out the transaction, which is another way of
saying Horizon enables the SPM or their assistant to transact the business over the
counter. That business may be a combination of Post Office retail activity, and
purchasing services or products offered by the Post Office on behalf of its clients.
However, in conjunction with the ability of a SPM to serve a customer (what is called
counter or front office), there is the associated “back office activity” of settling with
POL00112816
POLO00112816
Post Office's 'client' organisation, who has provided some service to the customer - such
as the DVLA, or a bank. Accounting is a theme or thread that runs through all of these
business requirements of Horizon, but it is only a part of them. The number of services
provided by Post Office branches is large and has increased steadily from 1998 to the
present day. The number of clients has almost certainly increased, and the functionality
of Horizon has expanded in line with the growth in service, both on the counter and in
the back office.
For those activities where the Post Office branch is acting like a retail outlet (such as
selling stamps), both the hardware and software is provided by Horizon to support this
activity. This is the Electronic Point Of Sale Software component of Horizon, that is
referred to as “EPOSS”. EPOSS allows the SPM or assistant to record that some goods
have been provided to a customer, compute the price of those goods, and allow the
customer to pay the money required for all their purchased goods, using either cash or
a credit/debit card. Often, a customer may wish to carry out two or more different
activities in one visit to the counter - for instance, to settle a utilities bill and to buy
some stamps. This can be done in the same activity and so Horizon has the concept of
a customer carrying out a ‘basket’ of activities and settling the total amount due for the
basket in several ways - by one credit card transaction, by a cheque, by cash, or by a
mixture of these.
Baskets of Post Office activities and non-Post Office activities are not supported by
Horizon. Often, a local Post Office branch will be a retail outlet too, selling non-Post
Office goods such as groceries, newspapers or even (as with Mr Bates at Craig-y-Don)
a haberdashery. If a customer wishes to buy a newspaper and some stamps, the
newspaper is not sold by the Post Office; it is simply sold by the associated, though in
accounting terms separate, retail outlet run by the SPM which uses the same premises.
So, the customer has to settle in two parts. In some premises, a customer may queue up
to purchase (say) greetings cards and pens (from the retail side of the branch Post
Office). If they then wish to perform some activities with the Post Office, they may then
need to queue up at a separate position to do that; that separate and second activity
would be transacted through Horizon. In this respect, the National Lottery is an
exception and spans the two businesses.
The Post Office’s clients include high street banks, Camelot, gas and electricity
companies (for paying utility bills), DWP (for paying benefits and pensions) and DVLA
(for paying road fund tax). Because of the different nature of the services provided
through the Post Office for all of these many hundreds of client organisations, the
services provided through the Post Office will be different from the service provided
for other clients. There is therefore the need for some unique software functionality
within Horizon. This must be provided both in the branch and the back office to support
the activities for that client. This is a part of what makes Horizon such a large and
complex system. The other reason that Horizon is so complex is because it has evolved
over a long period. As will be seen below, the original concept and design for Horizon
was for a joint Post Office/DWP project so that welfare benefits could be paid to benefit
claimants in a certain way. That changed and the DWP withdrew from the project, but
the need to offer other and an increasing number of products has led to many additions
being added on to the system, increasing its complexity over time.
POL00112816
POLO00112816
As well as the counter activities necessary to run a branch Post Office, Horizon also
supports the periodic process of balancing and rollover for each branch; essential
elements in the accounting to the Post Office performed by the SPM. Every branch
operates in Trading Periods, which are either four or five weeks (according to a
timetable published periodically by Post Office). At the start of each Trading Period the
branch is supposed to be 'in balance’. This means that the physical stock and cash in the
branch agrees with the data regarding the stock and cash held in Horizon. Then, during
the Trading Period, Horizon records all customer transactions made at the branch, so it
records the changes in cash and stock. It is also required to record any replenishments
or remittances of cash or stock in the branch. Thus, Horizon records all changes in cash
and stock held at the branch during the Trading Period, and should be able to compute,
from the starting amounts and the changes, the expected amounts of cash and stock at
the end of the period.
Without reciting the entire background to the group litigation between the Post Office
and the claimants, it is the accounting and functional accuracy of Horizon that is at the
heart of the current disputes, which have run for a great many years. The claimants
maintain that the Horizon system in operation threw up numerous discrepancies and
shortfalls in their branch accounts, for which the Post Office unfairly held them
responsible. The Post Office dispute this, and maintain that these occurrences are
explicable by fault or criminality on the part of the claimants.
Horizon, whether in its first incarnation as Legacy Horizon until 2010, or what is called
Horizon Online since then (originally HNG-X, now HNG-A running on a different
windows system), is at the centre of this group litigation. It is also important to
remember that it is not a system that is operated by the Post Office. It was originally
designed, “rolled out” and operated by ICL, a company that was partly owned by
Fujitsu, and which is now fully absorbed within Fujitsu, and has been for many years.
There is a contract between Fujitsu and the Post Office in respect of Horizon, and
perhaps in its current form it is rather different than the original contractual relations
between those two entities. The contractual relationship between the Post Office and
Fujitsu is not of direct relevance to the Horizon Issues, but arises tangentially in respect
of what is a sub-issue, namely the charging structure operated by Fujitsu for what are
called ARQ requests for audit data and whether this inhibited the Post Office in this
respect. The operation, functionality and accuracy of Horizon is an extremely thorny
issue (or bundle of issues), although given the breadth of allegation and counter-
allegation in the group litigation they are not the only issues. Providing the answers to
the Horizon Issues will not lead to complete resolution of the litigation. It should
however resolve one of the central issues about Horizon’s accuracy and functionality.
The Horizon Issues
The Horizon Issues are as follows. In the 3“' Case Management Order of 1 March 2018,
Leading Counsel for the parties were ordered to meet and seek to agree the Horizon
Issues to be tried in March 2019. This was done, the issues were agreed by the parties
and approved by the court. I take the following from the Case Management Order dated
23 March 2018, which was the next order made, and was a Consent Order. The
paragraph references to the pleadings were included in the list of Horizon Issues, which
were themselves appended to the Order of 23 March 2018 itself. These include at the
beginning an agreed definition of “the Horizon System”.
POL00112816
POLO00112816
“AGREED LIST OF HORIZON ISSUES
definitions for the purpose of this list of issues
“the Horizon System” shall for the purposes of this list of issues mean the Horizon
computer system hardware and software, communications equipment in branch and
central data centres where records of transactions made in branch were processed, as
defined in the Generic Particulars of Claim (“the GPOC”), at §16 and as admitted by
Post Office in its Defence at §37.
BUGS, ERRORS AND DEFECTS IN HORIZON
Accuracy and integrity of data
(1) To what extent was it possible or likely for bugs, errors or defects of the nature
alleged at §§23 and 24 of the GPOC and referred to in §§ 49 to 56 of the Generic
Defence to have the potential to (a) cause apparent or alleged discrepancies or shortfalls
relating to Subpostmasters’ branch accounts or transactions, or (b) undermine the
reliability of Horizon accurately to process and to record transactions as alleged at §24.1
GPOC?
(2) Did the Horizon IT system itself alert Subpostmasters of such bugs, errors or defects
as described in (1) above and if so how?
(3) To what extent and in what respects is the Horizon System “robust” and extremely
unlikely to be the cause of shortfalls in branches?
Controls and measures for preventing / fixing bugs and developing the system
(4) To what extent has there been potential for errors in data recorded within Horizon
to arise in (a) data entry, (b) transfer or (c) processing of data in Horizon?
(5) How, if at all, does the Horizon system itself compare transaction data recorded by
Horizon against transaction data from sources outside of Horizon?
(6) To what extent did measures and/or low controls that existed on Horizon prevent,
detect, identify, report or reduce an extremely low level of risk of the following:
a. data entry errors;
b. data packet or system level errors (including data processing, effecting, and recording
the same);
c. a failure to detect, correct and remedy software coding errors or bugs;
d. errors in the transmission, replication and storage of transaction record data; and
e. the data stored in the central data centre not being an accurate record of transactions
entered on branch terminals?
POL00112816
POLO00112816
OPERATION OF HORIZON
Remote Access
(7) Were Post Office and/or Fujitsu able to access transaction data recorded by Horizon
remotely (i.e. not from within a branch)?
Availability of Information and Report Writing
(8) What transaction data and reporting functions were available through Horizon to
Post Office for identifying the occurrence of alleged shortfalls and the causes of alleged
shortfalls in branches, including whether they were caused by bugs, errors and/or
defects in the Horizon system?
(9) At all material times, what transaction data and reporting functions (if any) were
available through Horizon to Subpostmasters for:
a. identifying apparent or alleged discrepancies and shortfalls and/or the causes of the
same; and
b. accessing and identifying transactions recorded on Horizon?
Access to and/or Editing of Transactions and Branch Accounts
(10) Whether the Defendant and/or Fujitsu have had the ability/facility to (i) insert,
inject, edit or delete transaction data or data in branch accounts; (ii) implement fixes in
Horizon that had the potential to affect transaction data or data in branch accounts; or
(iii) rebuild branch transaction data:
a. at all;
b. without the knowledge of the Subpostmasters in question; and.
c. without the consent of the Subpostmaster in question.
(11) If they did, did the Horizon system have any permission controls upon the use of
the above facility, and did the system maintain a log of such actions and such
permission controls?
(12) If the Defendant and/or Fujitsu did have such ability, how often was that used, if
at all?
(13) To what extent did use of any such facility have the potential to affect the reliability
of the Branches’ accounting positions?
Branch trading statements, making good and disputing shortfalls
(14) How (if at all) does the Horizon system and its functionality:
20.
21.
POL00112816
POLO00112816
a. enable Subpostmasters to compare the stock and cash in branch against the stock
and cash indicated on Horizon?
b. enable or require Subpostmasters to decide how to deal with, dispute, accept or make
good alleged discrepancy by (i) providing his or her own personal funds or (ii) settling
centrally?
c. record and reflect the consequence of raising a dispute on an alleged discrepancy,
on Horizon Branch account data and, in particular:
(i) does raising a dispute with the Helpline cause a block to be placed on the value of
an alleged shortfall; and
(ii) is that recorded on the Horizon system as a debt due to Post Office?
d. enable Subpostmasters to produce (i) Cash Account before 2005 and (ii) Branch
Trading Statement after 2005?
e. enable or require Subpostmasters to continue to trade if they did not complete a
Branch Trading Statement; and if so, on what basis and with what consequences on the
Horizon system?
Transaction Corrections
(15) How did Horizon process and/or record Transaction Corrections?”
Given the nature of the proceedings, and the disputes about the Horizon system which
were apparent on the face of the pleadings, I had indicated to the parties at an early
stage that the generic disputes about the operation of the Horizon system, which would
need expert evidence, would be resolved next after the contractual issues (which
became called the Common Issues, and led to Judgment (No.3)). The issues were not
foisted on the parties by the court. The court approved the wording of the issues agreed.
by the parties during the case management stage of the litigation. There was no
difficulty about this at the time.
However, the meaning of issues I and 3 in particular proved, at the Horizon Issues trial
itself, to be controversial. This is regrettable. Nor was any controversy aired with the
court prior to the actual trial. That too is regrettable. The court is well used to parties
who disagree over the answers to certain issues in litigation generally. It is
unsatisfactory when they also disagree about what any particular issue itself actually
means, or the question that is being posed (including being posed to experts) by an
issue.
Here, the two main areas of dispute over the meaning of the Horizon Issues were as
follows (there were others, but these are the most important ones). The claimants
approached Issue 1 as requiring consideration of whether bugs, errors or defects had
the potential to cause discrepancies or shortfalls in SPMs’ branch accounts or
transactions. In other words, this issue was not limited to consideration of whether bugs,
errors or defects had in fact actually caused discrepancies or shortfalls. The Post Office,
on the other hand, approached Issue 1 as requiring consideration of whether bugs, errors
22.
23.
24.
25.
26.
POL00112816
POLO00112816
or defects had in fact actually caused discrepancies or shortfalls, and by reference to the
claimants’ accounts specifically. This approach by the Post Office sought, in my
judgment, to narrow the scope of the Horizon Issues. This was not broached at the case
management stage.
The second main area of dispute concerned Issue 3. This includes consideration of the
concept of “robustness”. This word has been at the heart of the Post Office’s defence
of the Horizon system for many years. The Post Office has said publicly, and on many
occasions, that no computer system is 100% accurate and/or perfect but Horizon is
“robust”. This approach by the Post Office pre-dated the commencement of the group
litigation and was at the heart of the Post Office’s response to the increasing criticisms
from different quarters about Horizon. I deal with this important concept below in more
detail at [36] below. What the Post Office effectively means by this (in outline terms
only) is that the Horizon system can be relied upon by the Post Office for the purposes
for which it is designed and intended. The parties — and this is clear on the pleadings,
as well as in the terms of Horizon Issue 3 — disagree over whether Horizon is robust. In
the Reply, the claimants challenged this in the following terms:
“Tt is therefore denied that Horizon 'is robust and [...] is extremely unlikely to the cause
of losses in branches' (paragraph 16). In fact, the relatively small chance of errors
admitted by the Defendant, would be likely to produce the very picture reflected in the
Claimants’ case.”
Mr de Garr Robinson for the Post Office was somewhat critical at the trial of the
drafting of both Issues 1 and 3. He was not involved in the drafting of the Horizon
Issues. However, that criticism overlooked that the Post Office had agreed to the
wording of all the Horizon Issues through its other leading counsel — there have been a
total of three altogether instructed for the Post Office thus far - and the wording had
also been approved by the court. The Horizon Issues themselves were appended to the
Case Management Order itself.
Lists of issues, whether in group litigation or generally, are very important. I can do no
better than quote Longmore LJ in Scicluna v Zippy Stitch Ltd [2018] EWCA Civ 1320,
who said at [14] “ever since the Woolf reforms, parties in the High Court have been
required to agree lists of issues formulating the points which need to be determined by
the judge. That list of issues then constitutes the road map by which the judge is to
navigate his or her way to a just determination of the case.” In this litigation, the
Horizon Issues are not only important, but are vitally important. The whole of the
proceedings concerns the operation and reliability of the Horizon system. It could quite
easily be called the Horizon Post Office Case. Although there are a great number of
different causes of action pleaded in both directions, both by the claimants and by the
Post Office, against one another, the central core of the case is about Horizon. I consider
that the wording of both Horizon Issues 1 and 3 — indeed, all the Horizon Issues — to be
clear and had they not been, I would not have approved the wording.
However, some explanation is necessary given the disagreement to which I have
referred.
The phrase “bugs, errors or defects” is sufficiently wide to capture the many different
faults or characteristics by which a computer system might not work correctly. The
27.
28.
29.
30.
POL00112816
POLO00112816
parties in this litigation, usually but not always for convenience, would often refer
simply to “bugs”, and one part of the 2 Experts’ Joint Statement became known as
“the Bug Table”. Computer professionals will often refer simply to “code”, and a
software bug can refer to errors within a system’s source code, but “software bugs” has
become more of a general term and is not restricted, in my judgment, to meaning an
error or defect specifically within source code, or even code in an operating system.
Source code is not the only type of software used in a system, particularly in a complex
system such as Horizon which uses numerous applications or programmes too.
Software bug means something within a system that causes it to cause an incorrect or
unexpected result. During Mr de Garr Robinson’s cross-examination of Mr Roll, he
concentrated on “code” very specifically and carefully. There is more to the criticisms
levelled at Horizon by the claimants than complaints merely about bugs within the
Horizon source code.
The wording of Horizon Issue 1, “to what extent was it possible or likely for bugs,
errors or defects.....” is therefore very wide. Although that wording was proposed and
agreed by the parties, I considered at the case management stage when the Horizon
Issues were ordered, and still consider now, that these words clearly cover the whole
range of criticisms levelled at Horizon by the claimants, and the matters that could
potentially be wrong with the Horizon system without restriction. Bugs, errors or
defects is not a phrase restricted solely to something contained in the source code, or
any code. It includes, for example, data errors, data packet errors, data corruption,
duplication of entries, and a very wide type of different problems or defects within the
system. “Bugs, errors or defects” is wide enough wording to include a wide range of
alleged problems with the system.
J also consider that the words “possible or likely” are also wide enough to cover both
ends of the spectrum of what the parties would have the court decide. In other words, it
is a neutrally worded issue of wide effect which does not, by its phrasing, indicate any
particular starting point or end point. “Possible” means could something happen.
“Likely” means that something is, on the balance of probabilities, more likely to have
happened than not. The issue uses both terms, and therefore poses a wide question so
that the answer provided by the court will be of maximum utility in the group litigation.
Finally, the word “potential”, in the clause “have the potential to (a) cause apparent or
alleged discrepancies or shortfalls relating to Subpostmasters’ branch accounts or
transactions, or (b) undermine the reliability of Horizon.....” clearly means, on its
express terms, that this issue is dealing with possible or prospective effects, rather than
whether bugs, errors or defects have in fact caused actual or alleged discrepancies or
shortfalls. The way the wording of this issue was dealt with in the expert evidence will
be dealt with below.
If there were any ambiguity in Issue 1, and I do not consider that there is, the issue
specifically refers to certain passages in the pleadings. It would therefore be of
assistance to consider the pleadings. The relevant passages from the Particulars of
Claim as amended are as follows:
“23. However, the Claimants aver that there were a large number of software coding
errors, bugs or defects which required fixes to be developed and implemented. There
were also data or data packet errors. There was a frequent need for Fujitsu to rebuild
31.
POL00112816
POLO00112816
branch transaction data from backups, giving rise to the further risk of error being
introduced into the branch transaction records. The Claimants understand that Fujitsu
maintained a 'Known Error Log' relating to some or all of these issues which was
provided to the Defendant but which has not been disclosed.
24. Further, the Claimants aver and rely upon the following:
24.1. Insufficient error repellency in the system (including sufficient prevention,
detection, identification and reporting of errors), both at the data entry level and at the
data packet or system level (including data processing, effecting and reconciling
transactions, and recording the same);
24.1A bugs and/or errors and/or defects in Horizon and any data or data packet errors
had the potential to produce apparent shortfalls which did not represent a real loss to
the Defendant;
24.2. Horizon is imperfect and has the potential for creating errors (as the Defendant
has admitted in pre-action correspondence, in the Letter of Response, dated 28 July
2016, at paragraph 1.3);
24.3. bugs and/or errors have on some occasions produced discrepancies and/or
apparent shortfalls (as the Defendant has admitted in pre-action correspondence, in the
Letter of Response, Schedule 6) and such shortfalls may also have arisen from data or
data packet errors; and, further
24.4. the Defendant sought and/or recovered such alleged shortfalls from
Subpostmasters (as is presently understood to be admitted by the Defendant in the
Letter of Response, Schedule 6, paragraphs 4.1 to 4.5).”
The passages in the Defence which are identified in the Issue are paragraphs 49 to 56
This is a lengthy series of paragraphs but as all are referred to expressly within Issue 1,
I will reproduce them all. These state the following:
“Bugs, errors or defects in Horizon
49. As to paragraph 22:
(1) If and to the extent that the Claimants wish to assert that any of the shortfalls for
which they were held responsible were Horizon-generated shortfalls, it is for them to
make that distinct allegation and seek to prove it. Post Office notes that they do not
make the allegation in the GPoC. It further notes that, in paragraph 20 of their solicitors’
letter to Post Office's solicitors dated 27 October 2016, the Claimants make it clear that
they do not allege that there is a systematic flaw in Horizon or indeed any flaw which
has caused any Claimant to be wrongly held responsible for any shortfall.
(2) It is denied that Post Office has unreasonably or otherwise failed to provide
“obviously relevant disclosure" in relation to bugs, errors or defects in Horizon. There
has been no order or application for disclosure and, in the premises set out above, there
appears to be no basis for providing such disclosure.
POL00112816
POLO00112816
50. Paragraph 23 is embarrassing for its lack of particularity, in that (amongst other
things) it does not identify the errors, bugs or defects on which the Claimants rely or
how "large" their number was or the period in which they are said to have occurred and
nor does it identify the transaction data that Fujitsu is alleged to have rebuilt, how
"frequent" was the need to rebuild it or the extent of the "risk of error" which is said to
have been introduced. In the premises, Post Office cannot plead to the first three
sentences of this paragraph. However:
(1) AIL IT systems experience software coding errors or bugs which require fixes to be
developed and implemented. As is noted in paragraph 53 and 54 below, there are robust
measures in place in Horizon for their detection, correction and remediation.
(2) AILIT systems involving the transmission of data over the internet experience data
or data packet errors during transmission and such systems routinely have protective
measures in place to prevent such errors creating any difference between the data
transmitted and the data received and retained by the recipient. Horizon has robust
controls making it extremely unlikely that transaction data input in a branch would be
corrupted when being transferred to, and stored in, Post Office's data centre in a manner
that would not be detected and remedied.
(3) Like all IT systems, Horizon has backups to guard against any loss of data due to
local hardware failure. Where hardware fails, the data on that hardware is recovered
from the backup. Post Office takes the term "rebuild" to refer to the situation before the
introduction of Horizon Online where a new terminal was introduced to a branch and
the data stored on the other branch terminals (or on a disc where it was a single counter
branch) was restored to the new terminal. In this context, Post Office does not accept
that there was a "frequent" need to "rebuild" data from back-ups.
(4) It is admitted that Fujitsu maintain a "Known Error Log". This is not used by Post
Office and nor is it in Post Office's control. To the best of Post Office's information and
belief, the Known Error Log is a knowledge base document used by Fujitsu which
explains how to deal with, or work around, minor issues that can sometimes arise in
Horizon for which (often because of their triviality) system-wide fixes have not been
developed and implemented. It is not a record of software coding errors or bugs for
which system-wide fixes have been developed and implemented. To the best of Post
Office's knowledge and belief, there is no issue in the Known Error Log that could
affect the accuracy of a branch's accounts or the secure transmission and storage of
transaction data.
51. In paragraph 24, the Claimants again bundle many ambiguous and/ or misleading
allegations together. Post Office separates out and addresses those allegations in
paragraphs 52 to 56 below.
52. As paragraph 24.1 does not explain what is meant by "error repellency", what sorts
of errors are referred to, what is meant by "data entry level", what would constitute
"sufficient" prevention, detection, identification or reporting of these errors, or in what
respects the error repellency of Horizon was insufficient, Post Office cannot plead to
this paragraph. However, the general thrust of paragraph 24.1 is denied and the robust
controls, procedures and practices pleaded in paragraphs 53 and 54 below are noted.
POL00112816
POLO00112816
53. As to paragraph 24.1A, it is a truism that errors or bugs in an IT system and data or
data packet errors have the potential to create errors in the data held in that system.
However, Horizon has at all material times included technical control measures to
reduce to an extremely low level the risk of an error in the transmission, replication and
storage of the transaction record data. These have varied from time to time and they
currently include the following:
(1) Horizon creates, transmits and stores trans action data in the form of "baskets". A
basket is a complete transactional session between a customer and Post Office and may
include one, several or many individual transactions taking place within the same
session. Horizon will not accept a basket of transactions that does not net to zero (i.e.
the value of any sales is set off by the value of any payment made or received). This
reduces greatly the risk of any error in the data entered within any given basket.
(2) If a basket of transactions fails properly to complete its transmission to the central
database (because, for example, of a power loss), the system rejects any partial
transmission and requests the full basket from the branch terminal. This reduces greatly
the possibility of baskets of transactions failing to be recorded.
(3) At the point of a basket being accepted by Horizon, it is assigned a unique sequential
number (a "JSN") that allows it to be identified relative to the other baskets transmitted
by that branch. This reduces greatly the risk of recording duplicate baskets or there
being a missing basket.
(4) Each basket is also given a digital signature, i.e. a unique code calculated by using
industry standard cryptography. If the data in the basket were to change after the digital
signature was generated, this would be apparent upon checking the digital signature.
(5) Initial data integrity checks are undertaken when baskets are received at the Post
Office data centre from a branch. Baskets are then copied from the central database to
the Audit Store where a digital seal is then applied (the "Audit Store Seal"). If the
baskets and/ or the data within the baskets were altered after the application of the Audit
Store Seal, this would be apparent when the baskets are extracted from the Audit Store.
(6) Horizon and the above controls are themselves subject to various audits and checks
including audits carried out by third parties.
54. Further as to paragraph 24.1A, in addition to the technical controls referred to
above, there are several operational procedures and practices conducted by Post Office
and Subpostmasters that serve to increase the reliability of the data stored in the central
data centre as an accurate record of the transactions entered on branch terminals. These
currently include the following:
(1) For many transaction types, Post Office compares its own transaction record against
the corresponding records held by Post Office clients. If an error in Horizon were to
result in the corruption of transaction data, this should be revealed by the comparison.
(2) There are detailed procedures in place to address the risk of data loss resulting from
interrupted sessions, power outages or telecommunications failures in branches. These
are set out in the "Recovery - Horizon Online Quick Reference Guide" and Horizon
POL00112816
POLO00112816
guides the system user through the recovery process (which include completing any
transactions that are cut short). These procedures should prevent any data errors arising
from interrupted sessions, power outages and telecommunications failures.
(3) The display of the transactions being effected on-screen at the branch terminal
allows the user of the system to identify any inconsistency between the information
shown on the screen and the transaction that the user has keyed into the system. If, for
example, a hypothetical bug in the terminal were to cause a key-strike on number S to
be recorded as an input of number 6, this would be detected rapidly by system users,
given the large number of system users and the huge number of transactions effected
on Horizon.
(4) The accounting and record-keeping obligations placed on Subpostmasters reduce
the risk of any errors going undetected. For example, there is an obligation for each
branch to count and declare to Post Office the cash it holds on a daily basis, which
increases the likelihood of promptly detecting any overstatement or understatement of
the cash position on Horizon. If a Subpostmaster detects that an error has been made at
an early stage, its cause is more likely to be identified.
(5) Fujitsu operates industry standard processes for developing and updating Horizon
and for investigating and resolving any identified potential system errors.
55. As to paragraph 24.2, Post Office admits that, like all other IT systems, Horizon is
not a perfect system which has never had any errors or bugs. However, as indicated in
paragraphs 53 and 54 above, it has robust systems in place to identify them, fix them
and correct their consequences (if any).
56. As to paragraphs 24.3 and 24.4:
(1) There have been occasions on which bugs or errors in Horizon have resulted in
discrepancies and thus shortfalls or net gains in some branch accounts, as outlined in
Schedule 6 of the Letter of Response. It is denied (if it be alleged) that such bugs or
errors have affected any of the Claimants.
(2) On each occasion, both the bugs or errors and the resulting discrepancies in the
relevant branch accounts were corrected. Post Office took steps to ensure that it had
identified all branches affected by the bugs or errors and that no Subpostmaster was
ultimately held responsible for any resultant shortfalls.
(3) Paragraphs 4.1 to 4.5 of Schedule 6 to the Letter of Response relate to the so-called
Suspense Account Bug. Without prejudice to the burden of proof, none of the branches
affected by the Suspense Account Bug are branches for which the Claimants were
responsible.
(4) None of the Subpostmasters whose branches were affected by the Suspense Account
Bug were ultimately held responsible for the shortfalls that it generated. The Claimants
are therefore wrong to understand Post Office as having admitted that it "recovered
such alleged shortfalls from Subpostmasters". Where Subpostmasters in the affected
branches had made good or settled centrally shortfalls that were later corrected, those
Subpostmasters received a payment or credit in the amount of the shortfall.”
32.
33.
34,
35,
POL00112816
POLO00112816
It can be seen that the word “robust” is used throughout these passages of the Defence,
and “robust systems” in paragraph 55. Given “robustness” forms part of Issue 3 there
is no need separately to consider it at this stage when analysing the differences between
the parties about what Issue 1 means.
The passages in both parties’ pleadings which I have reproduced above show that it is
the potential, in general terms, of the Horizon system to have the effect complained of
that is the subject matter of Horizon Issue 1. In order to read Issue I in the manner
contended for by the Post Office, one would have to delete entirely the phrase “to have
the potential” and also substitute the term “Subpostmasters’ branch accounts or
transactions” with “the claimants’ branch accounts or transactions”. That is not what
Issue I states, it is not what the parties agreed the issue would be, and it is not the issue
that the court approved and ordered. In my judgment, the approach of the Post Office
to what Horizon Issue I actually means is too narrow.
In my judgment, the correct construction of Horizon Issue I is that contended for by
the claimants. In other words, it involves a two-stage process. Firstly, consideration of
whether there were, or are, bugs, errors or defects in the Horizon system as alleged by
the claimants. Secondly, if the answer is that there were, or are, such bugs, errors or
defects, the second stage is to consider whether these have (or did have previously) the
potential to cause apparent or alleged discrepancies in SPMs’ branch accounts
generally. The issue is not whether such bugs, errors or defects did in fact cause such
discrepancies or shortfalls in the claimants’ accounts specifically. That separate or
different issue — the effect upon claimants’ branch accounts - is a more claimant specific
one. It will have to be determined at some stage, for any of the claimants whose
individual claims come to trial in the future. It may require expert forensic accountancy
evidence. It was not ordered to be dealt with in the Horizon Issues trial. The Horizon
Issues were intended to be, and in my judgment on their wording are, generic issues
relating to Horizon and its operation.
The passages in the pleadings quoted above do show that the Post Office had, in
Schedule 6 to its Letter of Response, accepted the existence of two bugs or errors in
Horizon, one of which was called the Suspense Account Bug, but denied that the latter
had affected any branch accounts of any of the claimants. The passages in paragraph
56 of the Defence that explained that no SPMs had been “ultimately held responsible
for the shortfalls that it generated” reads a little differently with the hindsight provided
from evidence in two lengthy trials. This is because the Post Office’s Defence
effectively accepts that “Subpostmasters in the affected branches had made good or
settled centrally” the sums in question — that is the shortfalls in their branch accounts
that Horizon showed -- but states that these were /ater corrected. Both the expressions
“making good” and “settled centrally” means that the Post Office had originally held
the SPM responsible for the losses. These terms mean that the SPMs had either paid the
money (making good) or been given time to pay sums which the Post Office treated as
a debt, even if they were disputed sums (settling centrally), meanings given to these
terms that were confirmed by Ms Van Den Bogerd in her evidence for the Common
Issues trial. However, the pleading means that the Post Office had then, on the Post
Office’s case, subsequently corrected this which was why it pleaded no SPMs were
“ultimately held responsible”. Therefore there might prove not to be as much between.
the parties on this point in reality, as the Group Litigation unfolds, as there appears to
POL00112816
POLO00112816
be on the pleaded case. The Suspense Account Bug undoubtedly had an initial impact
upon SPMs’ branch accounts.
The meaning of “robustness”
36.
37.
38,
39.
40.
Turning to the disagreement about Issue 3, given the parties disagree about whether the
Horizon system is (or was) “robust”, it is a fairly elementary step to consider the
meaning of that term, and how it is being used by the parties. Context is important so
far as the meaning of the word “robust” is concerned. If someone is in robust health, it
usually is taken to mean that they are healthy. A robust exchange of views can be a
polite way of referring to an argument. Given the importance of the concept to the
Horizon system, its prominence in the Post Office’s defence of the system, and its
express inclusion (admittedly in inverted commas) in the Horizon Issues, I asked each
side in the litigation during oral closing submissions for a reference from their pleadings
or submissions for the meaning which they ascribed to the word. I referred to this as
their benchmark definition. Robustness was referred to by both sides in the litigation in
numerous places, but not always in the same precise terms, and clarity is to be
welcomed.
The claimants answered this by reference to the remainder of Issue 3, namely
“extremely unlikely to be the cause of shortfalls in branches” and explained that the
claimants had found the word robust “difficult to define” other than by reference to this.
This would mean therefore that it had no separate independent meaning other than as a
summary of the longer second part of Issue 3. In other words, a robust system would
be one that is extremely unlikely to be the cause of shortfalls in branches. The claimants
also implicitly, if not expressly, criticised use of the term both by the Post Office in its
pleadings and written submissions as being more aligned to public relations than as a
performance standard.
The Post Office asked for some time to provide the reference that I requested. Given
the meaning of “robust” is so central in the Post Office’s defence of the Horizon system,
I granted the Post Office the time that was requested.
The Post Office subsequently, after the trial ended, submitted a short document entitled
“the Post Office’s case on the meaning of robustness”. This was not what was intended
when I sought a reference from the Post Office to their definition, and the document
submitted went rather further and made wider ranging submissions. The document did
state, so far as the meaning of the word is concerned, the following:
“In Post Office’s submission, the meaning of robustness is a matter for expert opinion.
Robustness is a well-established concept in the IT industry and is the subject of
academic study: see para. 361 of Post Office’s written closing”.
I do not consider that the meaning of words is a matter for expert opinion. The two
experts in this case are IT experts, not experts in linguistics or the meaning of language.
However, the meaning of robustness within the field of IT is, arguably, a matter upon
which the experts’ opinions should be considered, not least because they were applying
that term to their expert exercise. The Post Office also relied upon the 1“ Experts’ Joint
Statement which in respect of Issue 3 stated the following as agreed:
41.
POL00112816
POLO00112816
“There are different dimensions of robustness, such as robustness against hardware
failure, software defects and user error. The robustness of the system also depends on
the processes around it.
Robustness does not mean perfection; but that the consequences of imperfection must
be managed appropriately. If the extent of imperfection is too high, this would be very
difficult to do which would imply less robustness.
Horizon has evolved since its inception. Therefore, its robustness may have varied
throughout its lifetime. The level of robustness may have increased or decreased as the
system was changed.
The existence of branch shortfalls is agreed. The experts do not agree at this point as to
whether this indicates any lack of robustness.”
In the areas of disagreement in this Joint Statement, each expert provided the following.
Mr Coyne stated (inter alia):
“For the purposes of addressing the robustness of Horizon, I have applied the following
definition of robustness:
‘The ability to withstand or overcome adverse conditions, namely, the ability of a
system to perform correctly in any scenario, including where invalid inputs are
introduced, with effective error handling.’ ”
In consideration of the likelihood of Horizon to be the cause of shortfalls in branches,
Horizon is not determined to be robust in this regard because:
(a) it contained high levels of bugs, errors and defects as set out under Issue 1 above
which created discrepancies in the branch accounts of Subpostmasters;
(b) it suffered failures of internal mechanisms which were intended to ensure integrity
of data;
(c) the system did not enable such discrepancies to be detected, accurately identified
and/or recorded either reliably, consistently or at all;
(d) the system did not reliably identify ‘Mis-keying’, which is inevitable in any system
with user input, and did not reliably have in place functionality to restrict users from
progressing a mis-key;
(e) it required numerous processes and workarounds to be in place to allow Fujitsu to
modify data already recorded by Horizon, which would not be required in a “robust”
system; and/or
(f) there were weaknesses and risks of errors and other sources of unreliability within
Horizon.
(italics present in original)
42.
43.
44,
POL00112816
POLO00112816
Dr Worden stated in the same Joint Statement:
“The definition of 'robust' proposed above by Mr Coyne is not adequate, for reasons
given below. The term 'robust' is not, as implied in para 3.1 of the outline, either ill-
defined or a piece of IT public relations. Robustness (which is closely related to
resilience) is an engineering objective, and large parts of project budgets are devoted to
achieving it. It receives its meaning in the phrase 'robust against... [some risk or threat]',
and there are a large number of risks that business IT systems need to be robust against
- such as hardware failures, communications failures, power cuts, disasters, user errors
or fraud. These are the dimensions of robustness.
In all these dimensions, robustness does not mean 'be perfect’; it means ‘address the
risks of being imperfect’. The extent of robustness is to be interpreted as: in how many
dimensions was Horizon robust? and: in each dimension, how large were the remaining
risks?
In my report I shall survey the evidence I have found that Fujitsu paid sufficient
attention to the dimensions of robustness, and that they did so successfully. I shall also
address evidence from Mr Coyne implying that Horizon fell short of its robustness
objectives.
In my current preliminary opinion, Horizon is a highly robust system, and this has
important implications for the other Horizon issues, notably issue 1.”
It can be seen therefore that Dr Worden in the Joint Statement did not agree Mr Coyne’s
definition, and expressly said it was not adequate. In any event, the meaning of any
word — even “robust”, or “robustness” - ought to be capable of description by the parties
themselves. Although on its face it did not appear that Dr Worden agreed with Mr
Coyne’s definition, a footnote in the Post Office first set of post-hearing submissions
suggested that Dr Worden was not disagreeing with the first part of Mr Coyne’s text,
in other words that part of the text that contained his definition of robustness (which
was in italics in the 1‘ Joint Statement). Obviously if the parties (or their experts) could
agree the definition to be applied so far as the Horizon System is concerned, that ought
to be identified. I therefore asked the Post Office via e mail whether it agreed with the
definition adopted by Mr Coyne, and if not, what its alternative definition was.
This led to a further document being received from the Post Office dated 18 July 2019.
It referred to the passage in the 1“ Joint Statement (which is quoted at [40] above) as
“the agreed definition”. That rather overlooks that Mr Coyne identified the definition
of robustness which he was applying, and Dr Worden expressly disagreed with this in
the same Joint Statement under the heading “Areas of Disagreement”, and stated “the
definition of ‘robust’ proposed above by Mr Coyne is not adequate, for the reasons
given below”. It also overlooks that in the 3“ Joint Statement, paragraph 3.1 had an
agreed entry which stated the following:
“Trrespective of how you define the detail of robustness, in line with most other large-
scale computer systems, Horizon's robustness has generally improved.
45.
46.
47.
48.
49.
POL00112816
POLO00112816
From our experience of other computer systems, Horizon is relatively robust. We agree
that 'robust' does not mean infallible and therefore Horizon has and will continue to
suffer faults. Robustness limits the impact of those faults and other adverse events.
This increase in robustness has, in part, developed from Post Office discovering
bugs/errors and defects in live use and then applying fixes and improving monitoring.”
(emphasis added)
Later in the same document of 18 July 2019 the submission was made by the Post Office
that “the robustness of a system is the effectiveness of the system in managing the risks
of imperfections (which are inevitable in any system) and their consequences”. It was
also submitted that “As Post Office understands it, this is what Mr Coyne meant when
in his comments in [the 1“ Joint Statement] he defined robustness as “the ability to
withstand or overcome adverse conditions, namely, the ability of a system to perform
correctly in any scenario, including where invalid inputs are introduced, with effective
error handling”.
This was precisely the definition which Dr Worden, in his areas of disagreement on the
1“ Joint Statement, described as “inadequate”. The end position therefore is as follows.
The claimants found “robustness” difficult to define in the abstract and tied it in with
the other wording of Horizon Issue 3; a robust system would be “extremely unlikely to
be the cause of shortfalls in branches”. That however is a consequence of how a robust
system would operate, not a definition of what robustness means.
The Post Office defined it as follows: “the robustness of a system is the effectiveness
of the system in managing the risks of imperfections (which are inevitable in any
system) and their consequences”. The Post Office was also prepared to accept Mr
Coyne’s italicised definition in the 1* Joint Statement, namely ‘The ability to withstand
or overcome adverse conditions, namely, the ability of a system to perform correctly in
any scenario, including where invalid inputs are introduced, with effective error
handling”.
Mr Coyne applied the definition he set out in italics in the 1‘ Joint Statement, quoted
in the immediately preceding paragraph of this judgment and at [41] above.
Dr Worden’s definition was as follows:
“Robustness (which is closely related to resilience) is an engineering objective, and
large parts of project budgets are devoted to achieving it. It receives its meaning in the
phrase 'robust against... [some risk or threat]', and there are a large number of risks that
business IT systems need to be robust against - such as hardware failures,
communications failures, power cuts, disasters, user errors or fraud. These are the
dimensions of robustness.
In all these dimensions, robustness does not mean 'be perfect’; it means ‘address the
risks of being imperfect’. The extent of robustness is to be interpreted as: in how many
dimensions was Horizon robust? and: in each dimension, how large were the remaining
risks?”
52.
53.
54.
55.
POL00112816
POLO00112816
The Post Office also submitted that Mr Coyne’s definition was not “materially
different” to that of Dr Worden.
Ido not accept the submissions made by the Post Office in paragraph 3(b) of the written
submissions dated 18 July 2019 on robustness concerning how Mr Coyne approached.
the concept of robustness and countermeasures, a detailed topic to which I return when
dealing with countermeasures. This is because some of the countermeasures considered
by Dr Worden are not parts of the Horizon System at all, such as SPMs noticing adverse
entries in their branch accounts, and the manual issuing of TCs by the Post Office
(which both parties agree are outside of the Horizon System).
I do however accept the Post Office’s submissions that there is not a great or material
difference in the definitions of robustness adopted by the parties’ experts. I do not
accept the claimants’ submission that robustness is difficult to define. Dr Worden
defined robustness by using what he termed as “the dimensions of robustness”. It is
rather circular to describe the meaning of robustness as being “robust against” some
particular risk. Although Mr Coyne provided his definition in the 1‘ Joint Statement,
the statement by Dr Worden that this was “inadequate” may only have been aimed at
the entirety of Mr Coyne’s entry in the areas of disagreement, as effectively accepted
by the Post Office in their most recent written submissions on the subject. Whether that
is an explanation of the lack of agreement in the Joint Statement, I agree with the Post
Office that Mr Coyne’s definition is not materially different to that used by Dr Worden.
Robustness is indeed an engineering concept. It means the ability of any system to
withstand or overcome adverse conditions. A robust system is strong and effective in
all or most conditions. The robustness of a system is the effectiveness of the system in
managing the risks of imperfections (which are inevitable in any system) and their
consequences; this is the same meaning as how robustness was described in the Post
Office’s written submissions dated 18 July 19. Robustness does not mean perfection
The exercise necessary above, to arrive at the definition of robustness in [54] above, is
not judicial pedantry. Given the central importance of robustness to the disputes about
the Horizon System, and the Horizon Issues, it is in my judgment essential. It is
surprising, given how central the assertion of robustness has been to the Post Office’s
defence of the Horizon System, that Dr Worden’s interpretation of the term has been
relied upon so heavily by the Post Office, given the term was used by the Post Office
for some years prior to his involvement.
However, regardless of that observation, I find that both experts correctly understood
what robustness in fact means, and applied the definition at [54] above in considering
their expert evidence. I will return to the expert evidence in some detail later in the
judgment, including in the Technical Appendix.
Features of this Group Litigation
There are many different ways of managing Group Litigation. The subject matter of
such litigation is different from group to group, and what is appropriate in one set of
proceedings will not necessarily be the best approach in another set. Group litigation
may involve many claimants — even tens of thousands, in some cases —all with what is
58.
59.
60.
POL00112816
POLO00112816
essentially the same type of claim, governed by a number of common issues that apply
across all or most of the cases. Some group litigation may involve similar numbers of
claimants, all with similar claims that are factually different. In this case, there are 584
claimants, which in the context of some group litigation, is not many, although it is still
in the many hundreds, and the period of time over which the relevant events are said to
have taken place is about 15 or 16 years. There are different aspects to each individual
case, but to deal with the litigation efficiently, cost-effectively and proportionately it is
simply not feasible for the same judge to try all the claims, one after the other, in full
on each of their respective merits. Such a process would take several years. It is not
what group litigation is intended to achieve.
The concept of group litigation is that the Managing Judge, whoever he or she may be,
with the assistance of the parties insofar as that may be available, selects the most
suitable mechanism for that particular set of proceedings in order to achieve compliance
with the over-riding objective. That will — or should — lead to cost and time saving. That
does however require a high degree of co-operation from the litigants. Here, my
intention of holding a substantive trial each judicial term onwards to resolve as much
of this Group Litigation as possible, as quickly as practicable, became simply
unachievable as a result of the issuing of the recusal application by the Post Office,
explained at [5] to [9] above. A further trial in the autumn of 2019 also became
undesirable as a result of the parties wishing to have time to consider mediation. Over
the numerous hearings and two full substantive trials that I have conducted, I have
gained the distinct impression that the Post Office is less committed to speedy
resolution of the entire group litigation than are the claimants, but it is not possible to
state with finality whether that is correct.
All litigation is important to those involved in it. In this Group Litigation, all the
claimants are of the view summarised at [11] in Judgment (No.3) in terms of the
behaviour of the Post Office. Recitation that the claimants are of that view does not
mean that any similar views are held by the court, or that any decisions have yet been
made by the court on that point one way or the other. The total sums claimed by way
of liquidated sums (excluding heads of general damages not yet quantified) are
approximately £18.7 million. The Post Office counterclaims for certain sums against
the claimants and also alleges fraud. Reputations are plainly at stake on both sides. That
is the case in many types of proceedings, particularly high-profile ones that attract press
interest. However, that sum of money is not large, in the context of large scale and
protracted litigation. The parties’ joint costs are approximately now £27 million, and
because of the notification provisions in one of the earlier Case Management Orders I
made, regular notifications of the total sum of costs are made to the court. The Post
Office alone spent over one million pounds in little more than a month earlier this year;
the notification letter of 13 May 2019 from its solicitors stated that its costs were in
excess of £12,800,000; by the time of its letter of 25 June 2019, the Post Office’s costs
were in excess of £13,900,000. Both this level, and rate, of expenditure is very high.
Not all of these will be recoverable costs regardless of the outcome, not least due to the
existence of two Costs Management Orders, but the total figures still represent real
expenditure of actual money.
These proceedings have the additional feature of criminal convictions on the part of
some SPMs. As also explained at [11] in Judgment (No.3), there is a Criminal Cases
Review Commission (“CCRC”) review underway in respect of the convictions of some
61.
62.
63.
64.
POL00112816
POLO00112816
claimants. I have been told that these are subject to a stay pending judgment of the
Horizon Issues. I have explained this before in open court, but matters such as criminal
convictions are no part of this Group Litigation. This court has no jurisdiction over such
matters.
At [23] to [27] of Judgment (No.3) I explained the applications that had been made by
the Press, the majority of which were granted by consent, with the sole exception of a
journalist being permitted to record personally the proceedings. Although I refused this,
the need for any separate recording effectively became redundant because the parties
agreed that he could be provided with a copy of the transcript being performed at their
joint expense.
Although criminal convictions are no part of these proceedings, examples of alleged
software bugs, errors and/or defects affecting branch accounts, which led to certain
consequences for SPMs, did form part of the subject matter of the Horizon Issues trial.
Given the impact (actual or potential) upon branch accounts, with the potentially
adverse implications for SPMs generally in the background, and given the way that the
SPM witnesses in this trial were cross-examined (as with the Common Issues trial,
some being accused expressly of criminal offences), criminal convictions were not part
of the trial, but were part of the background. In the claimants’ oral closing submissions,
Mr Green QC for the claimants identified a high level chronology of what he called
“doubling up”, by which he meant bugs, errors and/or defects that led to certain entries
in branch accounts being “doubled” incorrectly. He did so by specific reference to
allegations by a particular SPM who had been convicted of a criminal offence.
Mr de Garr Robinson QC for the Post Office objected to this, complaining of what he
said was jury advocacy. In the course of what became an increasingly intemperate
exchange between counsel at the very end of the trial he stated, by reference to what he
said were the rules of commercial litigation, that “one of those rules is that one doesn't
say things incautiously that might have an impact on evaluations being done in another
place in relation to different proceedings.”
Prior to considering the “rule” to which he was referring — it is not one of the Civil
Procedure Rules, so far as I am aware — it was therefore necessary to identify exactly
what “different proceedings” were being referred to in this disagreement between the
parties. There are five elements to this matter.
(1) Firstly, it had never been raised by any party at any stage during this Group
Litigation that there were any criminal prosecutions currently underway such that it was
necessary to consider any reporting restrictions for the Group Litigation. I expressly
raised this with the Post Office on the occasion referred to at [62] and [63] above, who
confirmed there were no such prosecutions underway. The “different proceedings” to
which Mr de Garr Robinson referred were the proceedings already before the CCRC.
(2) Secondly, the possibility of fisture (as opposed to current) criminal prosecutions, or
the potentially criminal impact upon individual SPMs, did more than hover in the
background to the Horizon Issues trial. Some claimants who gave evidence in this trial
were expressly accused by the Post Office of criminal offences in cross-examination in
this trial, something which had also occurred in the Common Issues trial. It should be
clearly understood that any findings I make in respect of any witnesses do not determine
65.
66.
POL00112816
POLO00112816
with finality any issues to be tried in any particular trial to follow, including the trial(s)
of their individual claims.
(3) Thirdly, the rule to which Leading Counsel for the Post Office referred was not
identified. Journalists must be alive to the risk of serious prejudice when criminal
proceedings are live, as a result of the Contempt of Court Act 1981, and are thereby
restricted in reporting in that respect. That statutory provision replaced what had been
referred to as the swh judice rule, but this did not relate to statements made in evidence
in open court. The principle of open justice is a most important one and there were no
restrictions imposed on any reporting of the Horizon Issues trial (nor the Common
Issues trial) at all, nor were any sought by the claimants or by the Post Office.
(4) Fourthly, there was no specific restrictions imposed on the parties by the court in
terms of the evidence they could adduce and what could be said by way of submissions
(written or oral), due to potential impact upon any “different proceedings”. It was also
not argued before me that the CCRC proceedings, which are subject in any event to a
stay pending at least part of the outcome of the Group Litigation, are in any way of such
a character that reporting of the Horizon Issues trial ought to be restricted in any way,
or such that they could be prejudiced by the group litigation. Indeed, the presence of a
stay of those proceedings pending the outcome of some of the group litigation issues
would suggest directly to the contrary. The objection therefore by the Post Office to the
way in which the claimants sought to make closing submissions on the actual impact
upon different SPMs of what were said to be bugs, errors and defects in the Horizon
System was, so far as it could be understood, not a well-founded one.
(5) Finally, the claimants by their counsel were trying to draw parallels between what
had happened to one particular SPM regarding “doubling up”, and numerous entries in
internal Fujitsu and Post Office documents prior to then of similar, if not identical,
occurrences being caused by bugs, errors or defects. Those parallels are obvious on the
face of the documents. The degree to which the CCRC find such parallels of assistance,
if at all, is a matter entirely for them. The Post Office submitted most strongly that these
different references were not to one single bug, but were references to a number of
different issues or bugs that had been experienced in Horizon over the years. Whether
that makes the claimants’ point for them or not, it is important to remind all the parties
that the issues in this litigation are not going to be decided with sympathy, or lack of it,
coming into account in the analysis in any respect whatsoever.
Mr Green for the claimants submitted that the Post Office was “trying to distract from
the stinging nature of the underlying documents by objecting” to the submissions and
that the Post Office may be finding it “uncomfortable” to have its own documents, in a
particular date range, highlighted by reference to particular criminal cases, prosecutions
mounted by the Post Office, and the experiences of SPMs in those cases. Certainly the
Post Office and Fujitsu have been exposed to a degree of scrutiny in this litigation which
does not appear to have occurred before; however, there has not been litigation of this
type on these issues before.
The claimants and the Post Office doubtless know this already — certainly their legal
representatives will - but given the high profile nature of this dispute and the fact that
this judgment may be read by those other than the parties themselves, I will make the
position very clear:
67.
68.
POL00112816
POLO00112816
1. The Senior Courts of England and Wales (who acquired this name by reason of the
Constitutional Reform Act 2005) comprises the Court of Appeal, the High Court and
the Crown Court.
2. The Court of Appeal has two divisions, Criminal and Civil. Appeals from the High
Court are to the Civil Division. Appeals from the Crown Court are to the Court of
Appeal Criminal Division, also known as the CACD.
3. The High Court is the highest court of first instance in civil cases
4. The Crown Court is the highest court of first instance in criminal cases. Some
criminal matters are dealt with by the High Court (by way of Divisional Court) but these
are narrow in scope and do not arise in any respect concerning the Group Litigation.
All appeals against both convictions and sentence from the Crown Court are dealt with
by the CACD.
5. The Criminal Cases Review Commission, or CCRC, is a statutory body responsible
for investigating alleged miscarriages of justice in England, Wales and Northern
Ireland. It was established by section 8 of the Criminal Appeal Act 1995. It is an
independent non-departmental government body, and has the power to refer cases to
the CACD.
6. This Group Litigation is concerned only with the issues arising in the civil claims
being brought against the Post Office by the claimants, and the Post Office’s
counterclaims It will result in a series of judgments on those issues which are public.
What, if anything, the CCRC do in any respect following any of the judgments is
entirely a matter for the CCRC and forms absolutely no part whatsoever of the Group
Litigation.
7. This court has no jurisdiction in respect of any of the convictions of those SPMs who
were successfully prosecuted by the Post Office. Although the presence of criminal
convictions does has evidential effect in respect of individual claims by individual
claimants who have been convicted of false accounting, these have not arisen in either
of the two substantive trials held to date (Common Issues and Horizon Issues) nor will
they arise in either of the next two (the principles governing Heads of Loss, and then
some individual claims).
There has been no restriction imposed on any party, or any witness, in this group
litigation by the court in terms of the evidence that could be adduced, or submissions
that could be made, with one exception. Mr Henderson of Second Sight was subject to
a restriction upon his evidence by reason of the terms of an agreement that he had
entered into with the Post Office when Second Sight were engaged in what was
originally intended to be consensual resolution of the claims. This was called the
Second Sight Mediation Scheme. This feature of the evidence of Mr Henderson is a
matter which is dealt with in more detail in Part D of this judgment. It was a restriction
in which the court was not involved.
Turning to a different matter, I have already made certain criticisms of the Post Office
in Judgment (No.3) in terms of how it had conducted itself in the litigation, and I had
69.
70.
71.
72.
POL00112816
POLO00112816
also made criticisms of some of its witnesses. One of those witnesses, Ms Van Den
Bogerd, was also called as a witness in the Horizon Issues trial. So far as I am
concerned, she came to the court for the Horizon Issues trial with a clean slate in terms
of whether her evidence on the Horizon Issues would be accepted or not. A different
way of putting the same point is that simply because certain findings had been made
concerming the evidence she had given in the Common Issues trial, this did not mean
that I adopted any particular starting position so far as her factual evidence was
concemed for this trial. The court was entirely neutral in terms of starting position.
Although it was certainly not a point that went to her credit that I had already made
adverse findings of the accuracy of her evidence in the Common Issues trial, that did
not mean that I started with any fixed view of the likely accuracy of her evidence in the
Horizon Issues trial. Her evidence in the Horizon Issues trial is dealt with at [203]
below.
Further, before turning to the detail of each side’s evidence, an approach was adopted
by the Post Office on occasion of seeking to adduce what was (or should have been) in
reality evidence of fact, but by way of submission, or points made “on instruction”.
Sometimes, depending upon the nature of the subject matter, such an approach is
understandable or unavoidable, and may be unobjectionable. It is not therefore sensible
to state that this should never be done in any conceivable circumstance in any trial.
However, on important points that have been dealt with by a particular witness in their
evidence of fact, it is not a suitable device to adopt. This was particularly done in terms
of the cross-examination of Mr Coyne concerning evidence already given by Mr
Godeseth in his cross-examination, about alteration by Fujitsu of a particular branch
account. I deal with that in detail at [376] to [379] below.
This was also done in Appendix 2 of the Post Office’s Closing Submissions, where
(sometimes detailed) factual explanations were given in respect of bugs in the Bug
Table. That appendix was compiled by different teams of solicitors and counsel,
something explained by the Post Office when, some months after it was submitted, they
discovered that three pages were missing and sought permission to serve them rather
late. I granted permission for them to be added, as they had been prepared before the
deadline for service and omitted due to an administrative oversight.
Submissions should not contain evidence, or positive evidential assertions, that are not
present in the evidence served in the trial. This is a fundamental point. I provide some
examples in the Technical Appendix by reference to specific entries for specific bugs.
Blurring (or ignoring) the lines between submission and evidence is entirely unhelpful.
Evidence is something that comes from a witness (lay or expert) and which the
opposing side is entitled to test by way of cross-examination. It is not appropriate for
detailed factual assertions to be made in closing submissions that are not directly
referable to evidence in the case. There is no way such factual assertions can be tested;
if they come in closing submissions, there is no way that the opposing party can deal
with those assertions in their own evidence, or even put relevant points to witnesses for
the other party in cross-examination.
Further, this is not a case that is being tried in a Specialist List, such as the Technology
and Construction Court - it is a general Queen’s Bench Division case - but it readily
could have been tried in such a list. It contains a great deal of technical subject matter,
particularly in this trial dealing with the Horizon Issues. The two IT experts have each
73.
74.
POL00112816
POLO00112816
given evidence in other computer litigation before. Such subject matter, and such expert
evidence, is readily suited to analysis by the parties and precision, which is the usual
approach of courts generally. In my judgment, this is particularly important in technical
matters such as these. Bluster and unfounded assertion in cross examination and
submission are rarely helpful.
Finally, there has been a vast amount of highly detailed material deployed by both sides,
not simply evidence of fact, and expert evidence, but also reference to a great many
documents. The use of an electronic bundle has made this easier to manage than
otherwise, but closing submissions alone were in excess of 800 pages in total from both
sides. It is neither possible nor desirable to recite in this judgment, or the Technical
Appendix, or resolve, every single disputed item, or every single disputed fact, no
matter how minor. I only make findings in this judgment that are necessary to enable
me to resolve the Horizon Issues themselves. Simply because I do not specifically refer
to a particular submission or piece of evidence, it should not be thought that I have not
had regard to it. I have considered all the material, evidence (both factual and expert),
submissions, and passages in contemporaneous documents, multiple times. The
experts’ agreements in particular have been of great assistance, but everything has been
considered. This judgment, together with the Technical Appendix, will be of substantial
length, and to recite everything would very probably make it of unmanageable
proportions.
In adopting this approach, I have borne very much in mind the over-riding objective in
the Civil Procedure Rules, the need for proportionality, but also the obvious need to
provide a reasoned judgment. I have also taken specific account of the dicta of Males
LJ in Simetra Global Assets Ltd and another v Ikon Finance Ltd and another [2019]
EWCA Civ 1413. That case concerned foreign exchange trading, and claims for
dishonest assistance and damages for deceit and conspiracy against a total of 12
different defendants, both personal and corporate. Although it concerns those matters,
very different to the Horizon Issues, it does state generally what ought to be included
in a judgment. The requirement for a judge to give adequate reasons in a judgment is
analysed at [37] to [46], with the expression “the building blocks of the reasoned
judicial process” (used by Henry LJ in Glickman v Redbridge Healthcare NHS Trust
[2001] EWCA Civ 1097) approved at [42] by Males LJ. Four points are summarised at
[46] of Simetra:
“Without attempting to be comprehensive or prescriptive, not least because it has been
said many times that what is required will depend on the nature of the case and that no
universal template is possible, I would make four points which appear from the
authorities and which are particularly relevant in this case. First, succinctness is as
desirable in a judgment as it is in counsel's submissions, but short judgments must be
careful judgments. Second, it is not necessary to deal expressly with every point, but a
judge must say enough to show that care has been taken and that the evidence as a
whole has been properly considered. Which points need to be dealt with and which can
be omitted itself requires an exercise of judgment. Third, the best way to demonstrate
the exercise of the necessary care is to make use of "the building blocks of the reasoned.
judicial process" by identifying the issues which need to be decided, marshalling
(however briefly and without needing to recite every point) the evidence which bears
on those issues, and giving reasons why the principally relevant evidence is either
accepted or rejected as unreliable. Fourth, and in particular, fairness requires that a
75.
76.
77.
POL00112816
POLO00112816
judge should deal with apparently compelling evidence, where it exists, which is
contrary to the conclusion which he proposes to reach and explain why he does not
accept it.”
Jam acutely conscious that the first of those points, succinctness, is not likely to be
achieved in this judgment. This is not only due to the nature of the subject matter, the
Horizon Issues, but also the fact that both Legacy Horizon and Horizon Online are
involved, and the system was brought in some time ago, namely the year 2000. Due to
the nature of the claims brought (and the limitation issues certain to arise), the
functionality of the Horizon system(s), robustness, and the other Horizon Issues had to
be dealt with at an early stage in the group litigation. I will return to the other three of
Males LJ’s four points after my review of the evidence, as well as his comments on the
importance of contemporaneous documents at [48] and [49] of Simetra. I do this at
[931] and following below.
Evidence of Fact: The Claimants
The claimants originally served statements from a greater number of witnesses of fact
than were in fact called at the trial. This was because, just before the Pre-Trial Review
on 14 February 2019, the Post Office objected to the evidence of two particular
individuals. It may be remembered that prior to the Common Issues trial the Post Office
issued an application to strike out a significant number of passages in the six Lead
Claimants’ witness statements, and this application was dismissed in Judgment (No.2)
“Strike Out”. On this occasion, although no application to strike out was issued, the
Post Office sought a ruling at the PTR in respect of the evidence of Mr McLachlan and
Mr Henderson.
Mr McLachlan had been called as an expert witness in the criminal trial of Mrs Seema
Misra, a SPM at West Byfleet in Surrey who was charged with both theft from her
branch and also false accounting. He was called for the defence. The sums in question
were approximately £74,000. Mrs Misra pleaded not guilty, and her defence was that
the Horizon system was to blame. She was convicted by a jury after a trial in late 2010
at Guildford Crown Court, and sentenced to a term of imprisonment of 15 months. She
was pregnant at the time of her conviction and imprisonment. Expert evidence was
called by the Post Office at her trial from Mr Gareth Jenkins from Fujitsu. Mr Jenkins
was not called as a witness by the Post Office in the Horizon Issues trial before me, but
a large amount of the evidence from the Fujitsu witnesses was attributed to information
directly given to them by Mr Jenkins. This was a controversial matter between the
parties at the Horizon trial. The Post Office did not proffer an explanation for Mr
Jenkins’ non-appearance as a witness; they were not obliged to do so. However, in their
closing submissions, they did so. The explanation provided was by way of submission
and not evidence. This is a matter which I deal with further in at [509] to [513] below.
Prior to the PTR, the Post Office objected to Mr McLachlan’s evidence on the basis
that it contained opinion evidence and the claimants did not have permission for it,
which is something required under the CPR. This was resolved by the claimants
deciding not to call Professor McLachlan, and explaining at the PTR itself that they
would not call him. I have not therefore considered the evidence of Mr McLachlan
which forms no part of the case.
78.
79.
80.
81
82.
POL00112816
POLO00112816
Mr Henderson is a Director of a company called Second Sight Support Services Ltd
(“Second Sight”). Second Sight was appointed by the Post Office to conduct a review
into problems with the Horizon system in July 2012. A number of reports were
produced by Second Sight, and the Post Office responded to these. These reports were
as follows:
1. Interim Report dated 8 July 2013;
2. Briefing Report Part One dated 25 July 2014;
3. Briefing Report Part Two version I dated 21 August 2014;
4. Briefing Report Part Two version 2 dated 9 April 2015.
The Post Office submitted at the PTR that I should give a ruling concerning those parts
of Mr Henderson’s witness statement upon which the Post Office were required to
cross-examine, and those upon which it was not. This was said to be justified by a
concer the Post Office had, namely that if it were required to cross-examine upon the
correctness or accuracy of the contents of the Second Sight reports themselves, the
Horizon Issues evidence would not be capable of being completed within the time
estimate at that stage, which was 16 days. Mr de Garr Robinson also expressed concern
that any failure by the Post Office specifically to cross-examine upon any particular
point in any of the accompanying Second Sight material, rather than the actual witness
statement of Mr Henderson, would lead to the claimants submitting that such a point
was not challenged and would therefore be taken by the court as being formally agreed
by the Post Office. I declined to give a ruling directing which parts of the statement
should be cross-examined upon by the Post Office. This was for three reasons.
Firstly, the answers to the different Horizon Issues would not be determined based on
the accuracy or correctness of the contents of the Second Sight reports themselves. The
court was to hear detailed expert evidence from two IT experts whose evidence would
go directly to the Horizon Issues. Whether the Second Sight reports were, or were not,
correct in their conclusions (which were generally critical of Horizon) was not even a
sideshow. Certainly their contents would not be determinative of the Horizon Issues in
the Group Litigation. I have had no regard to the contents of the different Second Sight
reports, nor to the Second Sight conclusions, in arriving at the answers to the Horizon
Issues. I am aware that all of the Second Sight conclusions were challenged by the Post
Office, and there was no technical route available to the claimants whereby a failure to
cross-examine upon these conclusions or reports, whether due to the length of the trial
or otherwise, would lead to a conclusion by the court that some aspect of the Second.
Sight reports was agreed by the Post Office, when it was not.
Secondly, it is not for the court in civil litigation to identify in advance to any litigant
how it should cross-examine upon the evidence of fact of the opposing party. This is
particularly so in a time-limited trial such as this one, but in my judgment is a general
point of principle. The court was effectively being asked to direct the Post Office as to
how it should conduct its case. This is not the function of the court. I was aware that
Mr Henderson’s evidence was challenged by the Post Office. It is for this reason that
he was to be cross-examined.
Thirdly, it would in any event be wholly unconventional in a time-limited trial dealing
with the subject matter of this Group Litigation for the court to give any weight to any
submissions by the claimants that particular points of criticism or detail in such detailed
83.
84.
POL00112816
POLO00112816
documentation as the Second Sight reports were essentially “accepted” by the Post
Office because they had not been challenged in cross-examination. There is never
sufficient time, in any time limited trial, to cross-examine upon everything. A time
limited trial in the 21“ century is not conducted in the same manner as a trial would
have been many years ago, particularly in a detailed technical dispute such as this one.
The period in question in the group litigation spans some 16 years. If attention is to be
paid to other court users, the over-riding objective and the Civil Procedure Rules, it is
simply not possible for a trial judge to permit the parties to have unlimited time to cross
examine upon everything. The quid pro quo of that is a party cannot be expected to
cross examine upon everything. There should have been no concern on the part of the
Post Office that the Second Sight report(s) would be taken as not challenged by the Post
Office unless their contents were subject to cross examination. Given there was such
concem, I explained the position and the claimants also expressly said at the PTR that
no such point would be taken. That latter element was unnecessary, as whether the point
was taken by the claimants or not, it would not have been accepted.
The position was therefore clarified or explained at the PTR and the Post Office,
notwithstanding that no formal ruling was made on those specific parts of Mr
Henderson’s statement that had to be challenged in cross-examination, appeared to be
content. The matter was explained in open court at the PTR and a transcript of this
hearing is available. The contents of the Second Sight reports were not cross-examined
upon when Mr Henderson gave evidence, but in my judgment there was no need for
them to be. I have not taken any account of the contents of the Second Sight reports in
deciding the Horizon Issues.
The claimants therefore called the following witnesses of fact. For the reasons that I
explain below in relation to Mr Roll for the claimants, and Mr Godeseth for the Post
Office, I consider these two witnesses in particular to have been of great assistance in
resolving the Horizon Issues. The specific experiences of the claimant witnesses who
gave evidence about what occurred in their branches are, if I accept them, specific
examples of the working of the Horizon system at the branch Post Office end in
practice. I will deal with each of the witnesses in turn.
Mr Latif
85.
86.
Mr Adrees Latif was the SPM at Caddington Post Office in Caddington, Bedfordshire,
since 2001 until late September 2018. His appointment with the Post Office therefore
ended after the litigation commenced, and he was subjected to an audit in September
2018. He was cross-examined by video link from Islamabad, as it had been necessary
for him to travel to Pakistan as a result of a family bereavement. He left the UK on 19
February 2019, and he travelled from Kashmir to Islamabad to give his evidence.
He gave evidence about two specific incidents. One occurred in July 2015 and related
to the transfer of £2,000 from the AA stock unit to the stock unit designated SP1.
Originally his statement said it was the SJ] stock unit but this was a typographical error
which he readily accepted. He successfully transferred the £2,000 from AA, but when
he went to the SP1 unit, the same sum had not transferred in to that unit successfully.
There was no explanation for this that he could come up with, including having checked
his own CCTV, and he was sure he had carried out the transaction correctly — it was
not an unusual transaction. He described the sum of £2,000 as having “simply
87.
88.
89.
90.
POL00112816
POLO00112816
disappeared from Horizon” and explained that this would lead to a shortfall in the
branch account for that sum.
The second incident occurred in January 2018 in relation to Camelot and scratch cards.
His heading for this he corrected to “Transaction Acknowledgement Issue”. He
explained that certain information was sent by Camelot to Horizon twice, and the Post
Office sent out a notice stating that due to this mistake by Camelot, a Transaction
Correction or TC would be issued. The correction occurred and he accepted it, but the
stock figure for scratch cards in his branch on Horizon remained unchanged. This
therefore showed his branch as having £1,000 more in scratch cards than was actually
present in the branch. This issue was still outstanding as at the date of his amended
statement which was I March 2019, and remained outstanding as at the date of his
cross-examination which was 12 March 2019. This is, obviously, a period well in
excess of one year.
The Post Office’s case on the £2,000 was put methodically to Mr Latif. That regarding
the stock transfer is best summed up in the evidence of Ms Van Den Bogerd, who stated
that provided certain steps or action were carried out correctly, what Mr Latif had said
happened simply would not occur. She said, inter alia “providing these two actions are
completed, the stock unit from where the cash is transferred should not show a
discrepancy" (emphasis added). As she put it, her “strong belief is that Mr Latif has
recalled these events incorrectly”. She also said that “the records that Post Office has
reviewed do not support what Mr Latif has said and I believe that he may have mis-
recollected events from 3 years ago”. The Post Office’s case was essentially that Mr
Latif had not done the steps correctly, because had he done so, what he said happened
could not have happened.
Mr Latif’s response to that was both consistent, considered and credible, and was best
summarised in one of his answers:
“A. I'm experienced -- I have been running a post office for 17 years, sir. I have also
worked for the Post Office on training other offices how to run a post office. I was also
involved in running and introducing the new Horizon software changes in 2006
onwards, where I went to several offices on behalf of the Post Office to give them
training. So I'm an experienced, trained subpostmaster and I ran my business
successfully for 17 years. So I may have been a bit brief in the statement but obviously
I can run through those -- exactly those steps that we would take to make sure that there
is no operator error on our behalf.”
The conflicting evidence on this particular point is a good illustration of the “poles
apart” position to Horizon by the Post Office and the claimant SPMs in this litigation.
Because of the Post Office’s position on Horizon, almost all and any of the criticisms
or accounts of factual events which the claimants made, or make, about how this system
worked in practice are attributed to fault or carelessness by the SPM or their assistants.
Indeed, without fault or carelessness by an SPM, the Post Office simply cannot explain
these occurrences. The Post Office’s position is therefore to challenge the factual
account — which it is entitled to do — because if the factual account by an SPM is
accepted as truthful and accurate, then the Post Office would have to accept that there
must be a fault or faults within Horizon. Therefore, the Post Office cannot accept that
91
92.
POL00112816
POLO00112816
the factual account is truthful and/or accurate. Thus the dispute goes around and around
in endless circles. This litigation is aimed at breaking that deadlock.
It is also the case that Mr Latif had the following point positively put to him about why
he had checked the CCTV that he had within his branch. “So you now say you looked
at the CCTV because your colleagues were concerned that you hadn't done the
transaction properly?” even though Mr Latif had said no such thing. Mr Latif was
subject to a fairly robust attack, not only on his account, how it matched up with other
records which the Post Office said contradicted it, what he and his assistants had or had
not been doing, and indeed upon the full scope of his evidence and his credibility — as
shown by the question I have reproduced. That question was framed as though even his
own colleagues had concerns about what he had done. It was positively put to him by
the Post Office that he had not even complained to the Post Office, although he provided
the name of his Area Manager Mr Navjot Jando and said he had complained to him
many times. The Post Office did not call Mr Jando to rebut this. One exchange will
suffice as an example of the type of attack upon Mr Latif:
“Q. You don't say anywhere in your witness statement that the £2,000 physical cash
also somehow disappeared, but that seems to be what you are now saying, is that right?
A. Well, the system gave a shortfall of £2,000 and that's been my statement all the way
through, sir, so I don't know what you're trying to confuse me, but there's a shortfall of
£2,000 in stock unit AA and there should not be a stock shortfall. The money is
physically there.”
Mr Latif’s evidence had never been, so far as his witness statement and evidence orally
before the court, that £2,000 in cash had disappeared. His evidence was that there was
a shortfall of that amount shown in Horizon as a result of what he had done. There were
some aspects of the cross-examination of Mr Latif which were simply unhelpful.
Firstly, extensive spreadsheets were put to Mr Latif which he had not seen before. They
are plainly not in chronological order, and had in any event been what was called
“filtered” by the Post Office legal team; they were not agreed by the claimants, nor was
the “filtering” process explained. There is a limitation on the degree of assistance to be
obtained by such an exercise. Further, there was no agreed exercise by the experts
whereby the two of them had gone through all the records directly relating to this
specific instance and agreed what the records did, or did not, show. The Post Office’s
case amounted, literally, to a pure challenge of fact that what Mr Latif said had
occurred, simply did not occur in fact.
“Q. On the basis of that, Mr Latif, Post Office says there was no failed transfer such as
that described in your witness statement and that you are simply wrong about that, it
never happened.
A. So you are calling me a liar?
Q. Mr Latif, you may be mistaken or you may be lying. I put the question that it didn't
happen.
A. Well, I state that they did.”
93.
94.
95
96.
97.
POL00112816
POLO00112816
So far as the other issue experienced by Mr Latif in January 2018 was concerned, the
Post Office accepted, as Ms Van Den Bogerd’s original written evidence (prior to later
correction) put it, that its data showed “that the branch received two TAs on 18 January
2018. However, due to an error by Post Office, instead of increasing the scratch-card
stock, the TAs decreased the stock. To be clear, this was a data entry error by Post
Office and not an issue with Horizon. Horizon processed the TAs accurately. I note that
the TAs were accepted by the branch, which could have been challenged at that point
if the user had noticed that the TAs were not for a positive number, as they should have
been.”
(emphasis added)
However, this is notable for the following reasons. The first was that the blame is shifted
back to Mr Latif for not spotting the mistake and challenging the TA. His evidence was
that it was not possible to challenge a TA. They simply have to be accepted. This was
accepted by Ms Van Den Bogerd when she came to give evidence orally. Therefore the
notable point made by Ms Van Den Bogerd, a director of the Post Office, to shift the
blame back onto Mr Latif is simply wrong in fact. TAs cannot be challenged, they have
to be accepted.
The second is that the Post Office evidence entirely omits any reference to the “memo
view” — a type of communication used by the Post Office on Horizon which leads to a
“pop up” on the terminal the next time each user logs on. There was simply no reference
to this at all by Ms Van Den Bogerd, and Mr Latif said that this was sent out by the
Post Office “to everyone” saying the error on the Lottery would be corrected by means
of the issuing of a TC. Finally, Ms Van Den Bogerd corrected this passage in her
evidence in chief by way of printed corrections prepared before she was called. She
said that these corrections had been handed to her solicitors prior to the trial. If that is
correct, it is surprising that Mr Latif was cross-examined on the basis of the original
evidence in her statement, as that was not going to be her evidence in chief. However,
regardless of that, she readily agreed that there is no choice regarding the acceptance of
TAs — they simply have to be accepted by the SPM. In those circumstances, I am
puzzled as to how the emphasised sentence in [93] above came to be in her statement
at all. Also, the call logs (about which Ms Van Den Bogerd was cross-examined)
concerning this incident entirely support Mr Latif’s evidence.
Finally, although this is a minor point compared to the ones in the preceding paragraph,
the tenor of her witness evidence that Horizon processed the TAs in this respect gave
the impression that there were no problems with the Lottery and TAs so far as Horizon
were concerned.
Not only did the steps taken by the Post Office — the TAs - not correct the issue that
occurred in Mr Latif’s branch, but in cross-examination Mr Latif explained further:
“A. Can I just confirm, there was an audit done in September of this year, an audit by
a Post Office trained auditor, and my stockholding was still showing negative. Anda
Jane Lawrence is the auditor and she has — still could not resolve this matter, so the
problem hasn't gone away, the problem is still there. And there have been a number of
calls to the helpline to resolve that negative stock and it hasn't worked. They haven't
come back with a response.”
98.
99.
POL00112816
POLO00112816
Mr Latif was very clear about this:
“A. I still state that an independent audit was done by the Post Office, a Mrs Jane
Lawrence, in September 2018 the stockholding was still negative and as the branch was
handed over to another subpostmaster that is going to be investigated, it's going to be
investigated. If you are now coming back with this evidence, I still say that there is a
problem somewhere and I don't know what's happened but we have still got a negative
stock figure within our branch.”
He returned to this again, in the light of being pressed yet further on the point that it
was human error at the branch that had caused the discrepancy or shortfall.
“So the fact remains there was an independent audit happened by your Post Office --
by Post Office's auditor, her recommendation was that they look into it as is (inaudible)
the strategy is still there. And there were a number of calls to the helpline pleading
with them to resolve this issue before the audit and there will be a complete trail of that,
sir.”
He also referred to the what had happened as being “a glitch”.
I accept the evidence of Mr Latif, who struck me as a reliable and careful person, and
who had personally been the one who had tried to perform the transfer from one stock
unit to another. He had personally experienced what he explained to the court in this
respect. I accept his evidence on this in preference to that of the Post Office, which
effectively was from people who were not there, who maintained, more or less, that it
simply could not have happened, and who had nothing to substantiate or corroborate
the challenge made to Mr Latif’s primary evidence. I find as a fact that it did happen
as Mr Latif explained. I find that Mr Latif performed the required steps correctly in
respect of the stock transfer between units, as one would expect of someone who had
17 years of experience, and was sufficiently skilled at his role such that the Post Office
had, prior to the litigation, been sufficiently satisfied of his competence that he was
used by the Post Office as a trainer for training other SPMs. I also accept his evidence,
which is direct primary evidence of the state of the accounts at this branch, that neither
he nor the auditor for the Post Office whom he named, have resolved the issue
conceming the Lottery, the incorrect TAs and the effect of the TC. Ms Van Den
Bogerd’s evidence is a number of steps removed from the branch, and is little more
sophisticated than assertions that there must have been other matters to blame,
alternatively reliance upon records which did not, due either to their contents or to the
way that they were deployed in cross-examination of Mr Latif, demonstrate the points
that the Post Office maintained they demonstrated. I make further findings in respect of
Ms Van Den Bogerd’s evidence in the section below dealing with the Defendant’s
Evidence in Part E. Mr Latif’s branch was subject to an audit, and his appointment as a
SPM ended then or shortly afterwards.
In due course Mr Latif will have his own individual claim, and the Post Office with
have its individual counterclaim against him, tried. The only findings I am making in
respect of his evidence are those necessary for me to resolve the Horizon Issues. All
other issues remain to be tried in those later proceedings. Those later proceedings may
explore in some detail not only the two specific matters in respect of which he gave
POL00112816
POLO00112816
evidence in the Horizon Issues trial, but any others which are relevant to both claim and
counterclaim. They will do so by reference to other documents, as explained in the
section of this judgment Part K Audit Data. The degree to which the findings of fact
that I make affect my conclusions on the expert evidence will be dealt with in Part L
Overall Conclusions.
Mr Tank
100.
101.
103.
Mr Jayesh Tank was the SPM at Fleckney Post Office, in Fleckney Leicestershire, from
4 May 2006 to 15 March 2017 when it was closed as part of the Network
Transformation Programme. He gave evidence regarding certain issues with Horizon
that he had experienced, which included the effects of a complete power failure to the
building, which occurred mid-transaction. This occurred whilst a customer was making
a withdrawal in the branch from her Post Office card account. On his evidence, this led
to a shortfall in his accounts of about £600. Mr Tank paid this sum to the Post Office,
and the letter from the Post Office stating how it came to be owed described it as a
“branch discrepancy”; it was in fact £660. He paid it by way of a deduction from his
remuneration.
The second issue was one concerning mail labels, when Horizon would (as he put it)
“jump ahead” to the end of the transaction and no label (which is what would or should
be affixed to the packet to be posted) would be printed. This would cause a loss in the
branch accounts.
Ms Van Den Bogerd again gave evidence about these specific instances. Her witness
statement led Mr Tank to correct the date when he remembered the power failure
occurring (he accepted it was 2014, not earlier in 2010 or 2011 as he had initially
thought), and he corrected this in a supplemental statement. Her statement had helped
him pin down the date. This incident became the subject of a PEAK at Fujitsu, although
Mr Tank did not know this at the time. This supplementary statement also exhibited to
it old “posts” he had made contemporaneously on an internet forum on the website
Yahoo in respect of another loss, £195, which he had suffered in 2011.
Mr Tank had been interviewed by the Post Office investigators in 2015 and in the
Horizon Issues trial it became clear that certain matters extremely critical of his conduct
were going to be put to him in cross-examination. He was therefore given the warning
against self-incrimination under the Civil Evidence Act in the same way that the two
claimants in the Common Issues trial, who were accused in cross-examination of
criminal offences, were given it. Mr Tank answered all the questions that were put to
him. Mr Tank had, eventually, used the icon on Horizon in his branch for “Official
Postage” incorrectly, and his explanation for this was that this was deliberate and was
done in order to get the attention of the Post Office. He said at his 2015 interview that
all his attempts by way of phoning the helpline and contact with his area manager had
simply not resolved the numerous complaints he had made. He frankly accepted that he
had not sent the post by official means which he had entered on the Horizon system as
official postage, doing what he did by way of protest. It was put to him that he had
“helped himself” to official postage, which on the evidence available was a suggestion
that did not seem to have any basis in fact, if he had been doing what he said he was
doing.
104.
106.
107.
POL00112816
POLO00112816
In his interview on 5 November 2015 by the Post Office investigator Mr Bridges, it had
been accepted by Mr Bridges (who interviewed him) that Mr Tank had reversed the
entries for official postage in any event. This acceptance at the time by the Post Office
investigator was directly contrary to the way that the points were put to Mr Tank in his
cross-examination, which were that he had “helped himself” to postage and “improved
his financial position” by acting as he did. Detailed findings on this will have to wait
for the full trial of his claim and counterclaim, but it is notable, in my judgment, that
the attack on the credibility of this witness in the Horizon Issues trial, and positive
allegations to him of criminality, were not consistent with the contemporaneous record
of the acceptance by Mr Bridges for the Post Office in the investigation interview in
2015. Mr Bridge clearly accepted that the entries had been reversed. Indeed, Mr Bridges
introduced this subject in that interview as follows:
“Okay thank you. Let's move on to the postage claims for the moment. So in my letter
I gave details of the claims and the reversals that you completed since 25 August. I
think my first question would be that I know that you reversed them but why undertake
them in the first place bearing in mind these are, in effect you are stating the transactions
which took, well say the transactions have taken place which you have used official
postage for which in effect did not take place.”
The questions in cross-examination, which must have been put on instruction, did not
seem to take account of what the Post Office had accepted at that time. Another feature
of the interview is that Mr Tank was told by the interviewer “there are no issues with
Horizon”. That exchange is as follows:
“KB (Mr Bridges) I can confirm that there are no issues with Horizon and as my letter
stated there is no issue with the production of labels.
Jay (Mr Tank) Then obviously you conducted your investigation into my concems
without asking me for my evidence. I've got the evidence here to show you. I'm happy
for you to take copies but I'm going to hang on to it but how can you say something is
fully investigated if you've not looked at all of the evidence.”
The Post Office relied upon evidence from Ms Van Den Bogerd. Mr Tank was cross-
examined by reference to this and also by reference to a technical explanation
conceming the fact, it was said in cross-examination, that the power outage must have
occurred at a particular point in time. The cross-examination was as follows:
“MR HENDERSON: So what appears to have happened was a transaction from the
Post Office card account was in the middle of being processed and so it was in the
stack presumably.
A. Yes.
Q. But had not yet been posted to Horizon. So you hadn't cashed out on that
transaction, you hadn't completed everything to do with that transaction?
A. On the stack —
Q. It's on the stack.
POL00112816
POLO00112816
A. It's on the stack, but the stack has a balance of zero, so to clear the stack you just
press "enter" and it goes straight —
Q. But you hadn't got to the point of clearing the stack?
A. I'm not sure.
Q. Okay. My suggestion is that there was probably an outage at just the point where
the money had been taken from Post Office card account but had not been processed
onto Horizon. That's my suggestion to you.
A. Okay.
MR JUSTICE FRASER: Well, is the witness going to be in a position to agree or
disagree?
MR HENDERSON: Well, he might be if he recalled.
MR JUSTICE FRASER: Do you recall that happening when there were outages?
A. No. I cannot recall.
MR JUSTICE FRASER: Were you aware of when outages would occur like that?
A. Not all the time.
MR JUSTICE FRASER: Do you want to put the question again?
MR HENDERSON: Yes. What I'm suggesting is that the cause of this problem was
that an outage occurred at a particular point in time.
A. Yes.
Q. You were in the process of effecting a transaction from POCA?
A. Yes.
Q. It was in the stack and it had cleared from POCA?
A. Actually you mentioned the word "outage". I'm not -- was there a power outage?
Q. I'm not sure if it was a power outage, but I think it may have been a problem with
the system.
A. Ah, okay.
Q. The system went down in some way.
108.
109.
110.
POL00112816
POLO00112816
A. So -- yes, because you said that if there's a power outage then there's evidence
when you have to log back in, so did that happen on this occasion?
Q. Okay, I want to come to all this and I'm doing this clumsily. What I'm suggesting
is that what may have happened -- and if you don't recall, you don't recall, but what
may have happened is that the transaction was in the stack, the money had been taken
from the Post Office card account and before you cleared the stack there was an
outage.
A. Possibly.
Q. Okay.”
(emphasis added)
T have reproduced this passage because it demonstrates the following. The Post Office
put a possible explanation, based on technical grounds, to the witness who did not have
either at the time, or at the trial, the technical expertise, or the relevant technical records,
or the data, to enable him to agree or disagree. Even that explanation itself accepted
“there may have been a problem with the system” and “the system went down in some
way”. The witness’ answer, which in my judgment is the only sensible answer that
could sensibly be given in these circumstances, was “possibly”.
This type of evidence obtained on cross-examination is of no assistance in resolving
the Horizon Issues, other than an implicit acceptance by the Post Office (in this
occasion) that outages could, potentially, in technically terms lead to what Mr Tank
experienced. In any event, the Post Office’s explanation accepted that “there may have
been a problem with the system” which is equally consistent with the claimants’ case,
as it is with the Post Office’s case, if not more consistent with the claimants’ case than
with that of the Post Office. Power outages do happen, and there is a process to be
followed. Given Mr Tank was using Horizon Online, he was then in his cross-
examination taken to Version 5 of a HOL quick reference guide. The version to which
he was taken was not, however, the version available to him at the time, as that was a
single A4 double-sided version in his branch, and the one put to him at the trial was
longer than two pages and available online. The date attributed in the electronic trial
bundle to Version 5, the version put to him in cross-examination, is 30 July 2015. His
forum posts and his supplementary witness statement made it clear that the incident
occurred in September 2014, a later date admittedly than given in his first statement,
but still somewhat earlier than July 2015. This was however the version of the guide
that Ms Van Den Bogerd relied upon in paragraph 78 of her 2" witness statement to
challenge Mr Tank’s account. I find the version that he would have had available at the
time was not Version 5, and was the single A4 version that he had available to him in
his branch in September 2014. However, and regardless of this, Mr Tank accepted that
recovery receipts would or should be generated and set out the procedure. He also
accepted that giving all the receipts to the customer was not the proper procedure.
In the PEAK dealing with the shortfall of his accounts of £195, which is actually headed
“Failed Recovery Transaction(s)” Fujitsu recorded:
11.
112.
113.
POL00112816
POLO00112816
“Date: 14-Dec-2011 10:43:53 User: Wayne Bragg
Summary:
The banking transaction had completed (A3 rec'd and authorised @ 13:33:37),
including the receipt print (13:33:42), and money should have changed hands.
The basket settlement failed from 13:35 with 'No response received from data centre’,
and the two retries also failed, and the attempt CANCELLED at 13:37.
The Disconnected Session receipts show "Cash TO CUSTOMER 195.04" so the
customer's account should be correct but the branch will have a shortage (for a
withdrawal) because the session hasn't been recorded.”
In my judgment this entry clearly supports Mr Tank’s evidence about how Horizon
operated on the occasion to which he referred. The heading of the PEAK was “failed
recovery transactions”. The PEAK clearly states that the customer’s account would be
correct — they were given the cash of £195.04 as they expected, but “the branch will
have a shortage (for a withdrawal)” as the session was not recorded. A transaction
correction was issued. There were related PEAKS, and also there was an entry “KEL
acha959T may be relevant” on the PEAK, and Anne Chambers was referred to. The
PEAK also recorded “Sending to SSC for investigation.” That KEL was raised by Anne
Chambers on 28 February 2010, and the title was “HNGX banking reconciliation — state
4”. This is an important KEL, and I deal with its further detail in the Technical
Appendix.
There was a dispute between Mr Tank and counsel for the Post Office about whether
this was an example of “Horizon working as it is supposed to”, which Mr Tank did not
accept, and also whether he would have been refunded the sums (which he accepted
had occurred) had he not phoned in and reported the problem. It was put to him that he
would have been refunded even had he not called in to report, although how this would
have occurred given the way PEAKS are initiated was not explained to him. In any
event, the PEAK demonstrated that what Mr Tank had said occurred, had indeed
occurred. Transaction Corrections are issued outside the scope of the Horizon system.
The position regarding the procedure for spoiled labels was, again, a situation whereby
the Post Office’s explanation was this simply could not have happened. Ms Van Den
Bogerd’s evidence was that there was a procedure available that Mr Tank should have
used, yet the document put to Mr Tank that was said to support this stated expressly
that “the label could only be spoiled if the label was on hand” and Mr Tank’s point was
that no label was “on hand”, given the problem he had was that the label did not print
at all. Using the procedure in the document therefore would not be possible, and indeed
would be contrary to the Post Office instructions that it could only be used if the label
was on hand. These instructions also required the SPM to write “spoilt” on the label
and keep it with the Horizon receipt for two years. This could not be done if the label
had never printed. The explanation by Ms Van Den Bogerd, put to Mr Tank, was also
directly contrary to the contents of a letter dated 7 September 2015 from the Post Office
to Mr Tank which stated:
114.
116.
POL00112816
POLO00112816
“Your enquiry has been investigated and I can confirm there are no issues with Horizon
online or the production of postage labels which would cause the situation you
described. However if a user either pressed the yes or return key quickly before screen
messages appear this can lead to a user confirming a postage print has happened when
in rare circumstances it may have failed and put the cost of the failed print into the
basket. In this circumstance Subpostmasters should contact Network Business Support
Centre to arrange a credit for the spoiled postage.”
This means the SPM must make a telephone call to the Helpline, to correct the fact that
the branch account would show a charge for the postage even though no label had been
printed and hence could not be provided to the customer. This letter suggests not that
Mr Tank had not followed a particular procedure for dealing with spoiled (rather than
non-existent labels) but that the system would, depending upon how quickly a user
pressed a particular key, not print a label, even though the cost of the failed print would
go in the basket. By going “in the basket” this means the branch accounts would include
the cost of that label as a debit to those accounts.
The Post Office in cross-examination put the following to Mr Tank:
‘Q: There were procedures built into Horizon to cater for the situation that you
explained -- I have to say in the vaguest of terms, but as I understand what you are
saying, there were procedures in place which ensured you could deal with the
situation, weren't there?
A. No.
Q. We will have to differ.”
I would express this rather differently. If the speed of pressing a key could lead to the
cost of a printed label being added to the basket (and hence branch accounts) even
though that label had not printed, it is difficult to see that there is a “procedure built
into Horizon” as defined to cater for this. On the contrary, it appears from the Post
Office’s own letter in 2015 that there was no such procedure. For the avoidance of
doubt, I find that there was no such procedure built into Horizon. The evidence
demonstrates to the contrary.
Further, it is difficult to see how the explanation proffered by Ms Van Den Bogerd.
regarding how to deal with a spoiled label can apply to Mr Tank for two reasons. Firstly,
I accept that as a matter of language a label cannot be “on hand” if it was never printed.
Mr Tank did not consider he could properly use the procedure in the guide for the
problems he experienced, and I agree with him. Secondly, the Post Office’s own
explanation to him in 2015 was very different to what he was told in cross-examination
in 2019 that he should have been doing. Even were I to assume (in the Post Office’s
favour) that there must have been instructions from the Post Office somewhere
regarding how a SPM should behave when a label failed entirely to print, as print
failures (which would not produce a label “on hand”) are surely not entirely unheard
of, it remains to be seen what those instructions were, and whether those instructions
conflicted with what he was told in the letter in 2015. No such instructions have been.
produced by the Post Office in any event.
I consider Mr Tank to have been a credible witness. I find that Mr Tank’s experience
in branch was not an example of Horizon working “as it was supposed to”. I find that
117.
118.
119.
120.
POL00112816
POLO00112816
the creation both of a PEAK, and reference to a KEL within that PEAK, to be consistent
with my conclusion. The proposition that Horizon worked as it was supposed to is a
flawed one. Indeed, that proposition is also inconsistent with common sense. PEAKS
and KELs are not created for situations where Horizon is working “as it is supposed
to”. They are created to deal with errors. The full title of the KEL is Known Error Log.
Nor are matters referred to the SSC for investigation when Horizon is working
correctly. Whether what occurred afterwards was an example of Horizon working
correctly depends on the categorisation of Transaction Corrections and whether they
are part of the Horizon system or not. Mr Tank also gave evidence, which I accept —
and this is made out in the contemporaneous documents — that he had some difficulty
in advancing this matter through the correct channels. The helpline operator with whom
he was dealing initially refused to “pass up” the matter to a more senior person. Whether
this was an isolated instance of unhelpfulness in this single case, or a more generally
obstructive approach across the board, will have to wait for future trials in this litigation,
and I recite it for completeness only in respect of Mr Tank’s experience. It does not
form part of my consideration of the Horizon Issues.
Mr Tank was accused of criminal offences and it was said that he had “helped himself”
to official postage and had “taken official postage”. These accusations were not put to
him at his interview in 2015 and indeed the text of that interview shows that the Post
Office interviewer expressly accepted that he had reversed the transactions, which is
not consistent with Mr Tank “helping himself”.
All the other issues between Mr Tank and the Post Office remain to be tried in the later
proceedings. The only point of difference between them on one of his two issues
concerning Horizon was whether what occurred when a transaction correction was
issued could properly be described as “Horizon working as it is supposed to”, and
certainly the KEL referred to in the PEAK appeared during the Horizon Issues trial
more than once. The creation of a PEAK, and the KEL to which reference was made
within it, and whether TCs such as this did demonstrate Horizon working as it should,
or not, are wider matters that are dealt with both by the experts and later in this
judgment. I reject the suggestion that Horizon was working as it was supposed to on
this occasion. So far as Mr Tank’s experience with the £195 shortfall is concerned, there
was a Horizon Online failure; three identical receipts were printed, which should not
have happened; the receipts all showed a disconnected session; and there was a loss in
the branch accounts that evening for the amount paid out to the customer. This was
even though Mr Tank had settled with the customer for the amount specified on the
receipt. There was no record of the transaction at all on the transaction log that was
produced for the period of the Horizon Online failure. None of these, in my judgment,
are examples of “Horizon working as it was supposed to”.
The operation of the helpline is not part of the Horizon Issues, so it is not necessary to
consider and make findings on what Mr Tank said the helpline told him, which he
explained in his posts was “that the loss is mine unless I can sort out with customer
directly.”
The later proceedings concerning Mr Tank’s claims (and any counterclaim) will
probably hear further evidence which are relevant to both claim and counterclaim. They
will do so by reference to other documents, as explained in the section of this judgment
POL00112816
POLO00112816
Part K Audit Data. The degree to which the evidence of fact affects my conclusions on
the expert evidence will be dealt with in Part L Overall Conclusions.
Mr Anup Patny
121.
122.
123.
124.
Mr Anup Patny became the SPM of Spencefield Post Office in Leicester in October
2014. He is currently suspended by the Post Office, having been suspended on 17
August 2016. His son, Mr Aakash Patny, also gave evidence. I shall refer to Mr Anup
Patny as Mr Patny Senior for this reason. I shall also deal with his evidence, and that of
his son, together for reasons that will become apparent.
There was what he described as a “major system outage” on 9 May 2016 in his branch,
which was a single counter branch run in conjunction with his retail business in the
same premises. This led to the closure of his branch and he believed that this had
affected the whole network. At the end of the next trading period, which was on 11 May
2016, there was a shortfall of almost £17,000, predominantly made up of a shortage of
16,000 coins of £1 denomination. This was a very large number and he knew he would
never have such a large amount of £1 coins in his branch. He was also the person who
had dealt with the cash delivery that took place on the relevant morning, which is called
“remming in” the cash. This means accepting cash into the branch from the cash
delivery from the Post Office to the branch, and entering the cash into Horizon. His
son, who also gave evidence, followed this matter up with the Post Office. On 19 May
2016 the Post Office contacted him about a discrepancy in respect of stamps, which so
far as he was concerned related to the £16,000 regarding coins.
The Post Office accepted that there had been an outage but disputed that this either did,
or could have, resulted in the discrepancy. Ms Van Den Bogerd gave evidence about
this, and the Post Office’s position is best summarised in the explanation put to Mr
Patny Senior:
“Q. Well, what I'm suggesting is that the most natural explanation for this, whatever
adjustments were made, is that at some point on 11 May someone hadn't counted a big
pile of £10 notes. They had been put in a safe and forgotten about, which is
understandable, and that they were found the next day, or located, and there was an
accurate cash declaration on 12 May.
A. I don't know about that, sir.
Q. It's perfectly plausible, isn't it?
A. I can't say anything to that.”
This suggestion is, in my judgment, somewhat fanciful. A “big pile of £10 notes” is not
the sort of thing that ordinarily gets “put in a safe and forgotten about”, and then
suddenly “found”. It was also put to the witness that “things were pretty chaotic in your
branch when it comes to these sorts of things” and “the cash declarations look like they
are all over the place”. Mr Patny Senior did not agree with this characterisation. He
certainly did not agree with the “big pile of £10 notes” theory. His evidence was also
that it was his son who had contacted the helpline, and who had made adjustments to
Horizon that he was advised to do by the helpline. Initially the Post Office sought to
POL00112816
POLO00112816
challenge this account through cross-examination of Mr Patny Senior only. Counsel for
the Post Office did not initially intend to put questions to his son, even though he was
plainly the correct witness to answer the detailed questions, as he was the person who
said he had done certain things, and he was about to be called as a witness. The
reluctance of the Post Office to put questions to the person who was the plainly relevant
person to ask about this was said to be because “the records are the records” and the
Post Office did not intend to repeat cross-examination that had already been done. I was
not prepared to permit this. I required questions about what Mr Patny had, or had not,
done, to be put to him directly, and not have the cross-examination conducted by proxy
through his father. Mr Patny Senior’s son had given a witness statement in respect of
these matters, and was plainly the correct witness to whom the questions should be put.
Mr Aakash Patny
125.
126.
127.
128.
Mr Aakash Patny is the son of Mr Patny Senior, and worked in the branch and retail
premises with him, as did Mrs Patny, the wife of Mr Patny Senior and mother of Mr
Patny. He is not a claimant. He was given the warning against self-incrimination under
the Civil Evidence Act. He did however answer all the questions put to him.
Mr Patny assisted his father, particularly with balancing. He would usually arrive at
lunch time, and so was not present when the national outage occurred on the morning
of 9 May 2016, which had been resolved by the time he arrived that date. He gave
evidence concerning the same amount of money, which both he and his father
considered a shortfall, and which the Post Office had claimed was due to it and was a
branch discrepancy. It had been settled centrally but had not been paid to the Post
Office.
So far as 11 May 2016 was concerned, when he arrived at the branch he became aware
of the shortfall of over £17,000, which was a “complete surprise” to him and which he
considered could not be explained. He had contacted the helpline, and had followed the
steps he was advised to take, which he thought had resolved the issue. These steps were
things he did on the Horizon terminal, and sequences of entries he was talked through.
The documents available supported his evidence that he had made such a call, although
some of the entries did not match what he said he had told them. There was no record
of what he was told to do.
The Post Office’s case on the shortfall was that the outage on 9 May 2016 did not lead
to any discrepancy. It was not put to Mr Patny that he had not made key strokes as
advised over the telephone, or that he made different keystrokes to those advised. All
that was put, somewhat unclearly, was that “the problem” was resolved, cash had been.
physically “found” that had been missed at the time of the previous cash declaration,
and also that a figure showed on a cash declaration of “plus £17,000 odd” had the effect
of “cancelling out” the previous discrepancy and showed that “there was no longer a
cash problem’. Mr Patny denied any money had been found that had been lost; denied
that if a discrepancy had been cancelled out the figure would have been plus £17,000,
rather than zero; and said that although he thought following the key strokes he was
told to input had resolved the issue, it had not. He was told to “readjust” the cash stock
figure, something he had never done before, and relied upon the instructions he was
given by the Helpline. The Post Office’s cross-examination was done by reference to
the cash management report, not the audit data. I will return to audit data later in this
129.
130.
131.
132.
133.
POL00112816
POLO00112816
judgment, but the audit data would have showed what keys were pressed, in what
sequence and when. Such data would have been of great assistance.
On 19 May 2016 the branch was contacted by the Post Office, in respect of an over-
declaration of stamps. The amount of this was approximately the same value as the
amount of the discrepancy of cash a few days earlier. He had done the declaration for
stamps. A person called Debra Lambley phoned the branch. He was struck by the co-
incidence in the amount of the stamp discrepancy, and told Ms Lambley the stamp
declaration was impossible due to the number of stamps this represented. To assist a
reader of this judgment on that latter point, but something that was not explored in the
evidence, to obtain a broad idea of how many stamps that would represent, the
following points arise. On 28 March 2016 the Royal Mail announced an increase that
week in the price of stamps. 1° class stamps became 64p each, and 2" class stamps
became 55p each. This shortfall would have represented 25,000 1‘ class stamps, or over
29,000 2"4 class stamps. It goes without saying that this is a great deal of stamps.
He was instructed to re-adjust the stamp stock figure, followed her instructions as to
how to do this, and the system then showed the £16,000 shortage. It was by now past
7.00pm, he phoned the Helpline, they contradicted Ms Lambley and he then had to re-
declare the following day, being unable to do so that day as it was after the 7.00pm cut
off.
The Post Office’s case on this, as it was put to Mr Patny, was firstly that he had made
a mistake in declaring the stamps. He denied this. The circularity of the Post Office’s
case can be seen from a lengthy passage of evidence, which I will not reproduce, from
page 26 on that day’s transcript to page 35. Mr Patny explained, basically, that the
helpline gave instructions to resolve the cash discrepancy; this resulted in the figure for
stamps being out by the same amount; Ms Lambley was not prepared to have this, and
following her instructions resolved the stamps situation, but the cash discrepancy
reappeared. Then the sequence happened a second time. Mr Patny was prepared to
accept he may have made a mistake once, but denied he could have made exactly the
same mistake the same way a second time.
It was also then positively put to him that one way of disguising a shortfall in cash
would be to over-declare the number of stamps. A positive allegation of dishonesty was
put to him, which he rejected.
The premise behind this suggestion of dishonesty was that it was said by the Post Office
that if the stamps figure went up, the cash figure would automatically go down by the
same amount. He did not agree with this. It was put as a positive statement in the
following terms:
“Q. I suggest finally, Mr Patny, it is rather surprising, given that you do all the
balancing, or did all the balancing in the branch, that you don't understand that the effect
of increasing the declaration of stamps is to have a corresponding effect on the
declaration of cash?
A. I wasn't aware of that, no.”
134.
135.
136.
POL00112816
POLO00112816
There was no direct evidence to support the proposition that by increasing a declaration
of stamps, it would have a corresponding effect upon the declaration of cash held in the
branch. In any case, in the Common Issues trial, there was evidence that making a cash
declaration required specifying how many notes of each denomination was physically
held in the branch. An SPM had to declare both the cash and the stock that was held in
the branch. It is difficult to see how the number of stamps held in a branch would or
could affect the physical counting and declaration of the amount of cash held in a branch
by way of specific denominations. However, and in any event, the allegation of
dishonesty against Mr Patny was a line of cross-examination that was not supported by
evidence from Ms Van Den Bogerd, who was expressly asked about it, and who
accepted that she had not made an allegation of dishonesty in her witness statement and
said she did not offer an opinion on whether Mr Patny was dishonest. That means that
there was no evidence from any witness called for the Post Office to support the
allegation of dishonesty.
The second problem in relation to which Mr Patny gave evidence was a MoneyGram
transaction in February 2016. This related to a failed payment by card by a customer,
who tried to send £3,100 by MoneyGram but whose card was declined twice. However,
the branch accounts still showed two debits in that amount in the branch accounts that
led to a loss of £6,200. A TC was issued but only for one of these, which led to a £3,100
loss. Mr Patny accepted that he had only cancelled the transaction at the time, and not
reversed it as well (which was required) until later, in the evening, when advised to do
so by the Helpline. As he put it, the amount had “doubled up”, in that the MoneyGram
transaction, which was cancelled and reversed later on (which was for £3,100) had led
to a loss showing of £6,200. If that is correct, then it is indeed double the value of the
transaction. Ms Van Den Bogerd said that this was “not supported by the data” and also
relied upon the fact that the cash shortfall on the day was greater than £6,200, namely
about £500 greater than the cash shortfall which Mr Patny said had been caused by
MoneyGram. She could not proffer an alternative explanation, however, and in her
witness statement said:
“On 23 February, the branch declared cash holdings of £25,803.87, a net downward.
movement of £8,601.59. The net value of transactions during this period resulted in a
£1,806.71 decrease in cash. The Moneygram transaction described above would
account for a further £3,100, bringing the total explainable cash movement to a
£4,906.71 decrease. However, this leaves £3,694.88 of cash movement unaccounted
for. L cannot say for certain what caused this additional loss of cash but there is nothing
in the accounts that suggest a problem with Horizon. It appears more likely to me to be
a problem with cash handling in the branch or a user error when making cash
declarations.”
(emphasis added)
This statement does not answer the evidence of Mr Patny or the point, and the more
that counsel for the Post Office tried to press the point(s) in cross-examination, the more
stark it became that it was no answer. Firstly, it assumes that the MoneyGram
transaction was only responsible for £3,100, not the figure of £6,200 which Mr Patny
stated. It assumes that £3,694.88 does not include the lower figure of £3,100, which in
my judgment, on the documents deployed, it plainly does. Secondly, Ms Van Den
Bogerd cannot say what caused the additional loss of cash. Thirdly, it states that that
137.
138.
139.
140.
POL00112816
POLO00112816
there is “nothing in the accounts” which suggests a problem with Horizon (emphasis
added). That is rather stating the obvious. If all the problems complained of by the
claimants with Horizon could be seen in the accounts, this litigation would not be
occurring. Fourthly, she retreats to criticism of the way cash is handled in the branch,
or other mistakes — “user error”. It is a perfect summary of the Post Office’s case, but
it does not provide an answer.
Additionally, Mr Patny explained that he had fully explained all of this to the area
manager Mr Irwin, who agreed with him that the MoneyGram transaction had appeared
to double up. Further disclosure by the Post Office when the trial of Mr Patny Senior’s
claim comes to be tried, together with evidence from Mr Irwin if he is called, will
provide further information. Finally, the exercise undertaken by the Post Office in
cross-examination failed to take account of the fact that one had to consider both the
cash figure, and the shortfall, from day to day to make sure that one is comparing apples
with apples. I consider the exercise undertaken in cross-examination to be flawed.
I cannot resolve this purely on the factual material put before the court by the Post
Office. The audit data showing the actual key strokes inputted will be required in order
to do that, together with findings on the expert evidence and my findings on the
existence of bugs, defects and errors within Horizon. However, I do find that Mr Patny
came across as a careful person, a credible witness, and I accept that he took the steps
he did as explained in his evidence. He accepted that he had not reversed the transaction
until later, rather than immediately as he ought to have done. The resolution of the
movement of the cash position, and what if any explanation is available, simply cannot
be done with finality at this stage on the basis of the factual evidence before the court
alone. The Post Office had, according to Mr Patny, failed to provide the Credence data
which would make it clear what had happened and how. The helpline logs show that
Mr Patny had chased this on numerous occasions, and also the entries suggested that a
Credence report was being prepared. The audit data showing what key strokes he had
undertaken and when, was not produced.
There was no evidence before the court to support the allegation of dishonesty that was
expressly made against him, and it was not supported by Ms Van Den Bogerd. On the
material before the court in this trial, I find the allegation of dishonesty against him not
to be made out and I reject it. That is not to say that the subsequent trial dealing with
Mr Patny Senior’s claims will automatically accept all the evidence given by his son, if
he calls him as a witness. All future issues in individual claims will be dealt with, and
resolved, on the evidence led by the parties in those subsequent trials. I am only
deciding matters in this judgment that are necessary to resolve the Horizon Issues.
However, in circumstances where positive allegations of criminality are expressly put
to a witness in a trial such as this, it is only fair to deal with those allegations. They
cannot as a matter of fairness be left hanging in mid-air, particularly where there is no
evidence in support from the party making the allegation in any event.
The later proceedings concerning Mr Patny Senior’s claims will explore in some detail
not only the specific matters in respect of which he and his son gave evidence in the
Horizon Issues trial, but any others which are relevant to both his claim and
counterclaim. They will do so by reference to the evidence advanced in that later trial
and the other documents, as explained in the section of this judgment Part K Audit Data.
POL00112816
POLO00112816
The degree to which the evidence of Mr Patny Senior and his son affects my
conclusions on the expert evidence will be dealt with in Part L Overall Conclusions.
Mrs Burke
141.
142.
143.
144.
145.
Mrs Angela Burke had worked for the Post Office in various roles since she was 16.
Most recently, she worked in the Newport Post Office in Brough where her husband.
was the SPM. She was a Branch Assistant and had worked as an assistant for about 15
years, and had also previously been a SPM herself. Her and her husband had an
associated greeting cards and stationery business, and the branch was closed in October
2017 as part of Network Transformation. The card business has also closed.
Her evidence related to events of 9 May 2016 when the national outage occurred, the
same date as the occasion about which both Mr Patnys had given evidence. She gave
evidence about the impact upon the branch business, the way she was serving customers
and how Horizon was being very slow, with a sand timer appearing on the screen for a
very long time. She served one customer, who was making a cash withdrawal, as she
obtained the relevant messages and approvals on the screen. However, after they had
left, a receipt printed saying “Recovery failed” and the withdrawal of £150 was not
shown. She then later studied the transaction log and this latter transaction did not
appear.
She realised what was happening and decided to close her branch that morning. She sat
down and worked through all the transactions. She was given an explanation by the
Helpline which she did not accept, as she knew and could remember that the payment
to the customer had been authorised, which is what led to her handing over that sum.
She also was engaged in a contemporaneous exercise whilst it was very fresh in her
memory. She said there was no alert from Horizon, and “there was no means through
the Horizon system for the discrepancy to be identified or for its cause to be established
in my situation”. The transcript of this call was available — she and her husband had
obtained it using a subject access request from the Post Office.
Mrs Burke then went to extraordinary lengths. She also proved herself very tenacious,
as many people may well have simply given up on the sum of £150. She identified the
customer, and she tracked him down. She went to his house and explained. He still had
the receipt from the transaction at her Post Office. It entirely matches her account. She
went with the customer to the customer’s bank, which was the TSB in Goole. She
explained with the customer to the bank cashier, who printed out the bank statement
and showed that the sum had been withdrawn from the customer’s bank account. The
customer permitted Mrs Burke to have this.
She and her husband then pursued this through the Helpline and their call was escalated.
They did receive a TC for £150, but actually that referred to an amount to Lloyds Bank
and not to TSB. This was pointed out to the Post Office who claimed there was no code
for the TSB. If that is correct, I find it highly surprising that there should be no separate
code for the TSB. The bank currently known as Lloyds used to be known as Lloyds
TSB — there are cases on bailii that use that name — but the TSB was split from Lloyds
TSB, which then became simply Lloyds Bank. This split occurred some years before
the events of which Mrs Burke gave evidence. TSB and Lloyds Bank had been separate
for some time by May 2016.
146.
147.
148.
149.
150.
POL00112816
POLO00112816
Mrs Burke’s evidence was explored in very considerable detail in cross-examination,
including the fact that she chose to keep the stack open in order to serve more than one
customer at once, how she had served them and the other details of what had occurred.
She was challenged on what was said to be a “relatively small” point, of whether she
would have received a TC without having done all the work that she had done, in
tracking down the customer and going to his bank. I reject that categorisation of the
point. It is not a small point. It is, in the context of the overall litigation, a major point.
It is difficult to see, given the evidence collected by Mrs Burke which was provided to
the Post Office, how they could have failed to issue a TC. Whether this TC would have
issued absent the work that she did to demonstrate the fault and its effect is not a “small
point”.
There are really two points that are relevant. The first is a broader one; is the process
for issuing TCs part of the Horizon system? The second is a case specific one: would,
on the balance of probabilities, the Post Office have issued a TC in her case without the
evidence that Mrs Burke herself collected and provided to the Post Office?
So far as the first point is concerned, the process for issuing TCs is not part of the
Horizon system. So far as the second point is concerned, the documents put to Mrs
Burke demonstrate that Fujitsu had worked out what had occurred; that the internal
document attached to an internal e mail of 12 May 2016 did not accept that Newport
branch had in fact paid the money to the customer, even though Mrs Burke had told
them this on 9 May; and by 16 May 2016 the decision had been taken that the branch
would receive a TC anyway. Her response to the “small point” was “possibly, yes”.
The conclusion that I draw from this is that the Post Office required something more
from Mrs Burke than her word that the money had been paid out by her branch to a
customer — the internal documents refer to “not knowing” if the money had been paid,
even though she had already told the Post Office this. I find that the fact that this is a
Fujitsu document rather than a Post Office document does not matter for this purpose.
That something more that was required, in her case, was the evidence she had
personally compiled that was of great weight, and was accepted by the Post Office, as
the TC was issued. Whether evidence of lesser weight would have been accepted by
the Post Office is not possible to say, and is entirely hypothetical. Fujitsu had, though,
identified this particular problem experienced at the Newport branch by the national
outage. The information of what occurred, and the problem, was not visible to the SPM
or their assistants in the branch.
Counsel’s “relatively small point” put to Mrs Burke, that the Post Office would have
issued her with a TC anyway, is in fact not entirely consistent with the Post Office’s
own evidence. Mrs Van Den Bogerd stated in paragraph 110 of her witness statement
that “Once Post Office was presented with evidence that the customer had received the
cash and the customer's bank had recorded the withdrawal, a transaction correction was
issued to bring the branch.” She makes it clear, in my judgment, that it was the work
that Mrs Burke did that had led to the issuing of the TC. She also makes it clear that
she believes that what happened occurred as a result of what Mrs Burke did:
“Following Mrs Burke's investigation, Post Office generated a transaction correction
for the £150 withdrawal that had not been recovered at the time. If Mrs Burke had not
1S]
152.
POL00112816
POLO00112816
done this herself, there is a process built into Horizon for flagging non-recovered
transactions which would have prompted an investigation and I'm sure would have led
to the same outcome.”
She explains that had this not been done, more would have been required from the Post
Office — namely an investigation — before the Post Office would have reached the
decision to refund the £150, although she does say that she is sure the same outcome
would have been reached. This is rather different to the way the case was put to Mrs
Burke, which did not include that any suggestion of any further investigation by the
Post Office was required. The impression sought to be given was that on the Fujitsu
documents already available, a TC would have been issued anyway to Mrs Burke a few
days later. I do not accept that. Something more at the Post Office end would have been
plainly required, as Ms Van Den Bogerd explains. Mrs Burke was the person who had
in fact provided proof of payment out, and so no investigation was undertaken.
It is in my judgment likely, on the basis of all the evidence in the litigation to date, that
the sum of £150 would have showed up as a shortfall at the end of the next branch
trading period, as any further investigation was not likely to have been completed by
the end of that period. However, if Mrs Burke had not acted as she had, and if an
investigation had been done by the Post Office, if that had been resolved in her favour,
and if that had led to a TC being issued, then the shortfall in the branch accounts would
have been corrected in a later branch trading period, namely the period during which
the TC was issued. No loss flowed to Mrs Burke in respect of this incident, as a result
of her own investigation. In any event, this is not a point of difference between the
parties that it is necessary to resolve in order to resolve the Horizon Issues. There was
plainly a potential impact to her branch accounts in any event, and that potential impact
was caused by the Horizon system.
Mr Roll
153.
Mr Richard Roll had worked at Fujitsu between 2001 and 2004. This was during the
days of Legacy Horizon and he had no experience of Horizon Online. He provided two
witness statements, the first of which was dated 11 July 2016. This was obviously very
much in the early days of the group litigation. The second statement addressed certain
factual matters in Dr Worden’s first expert report. I consider Mr Roll to be an important
factual witness in this group litigation. He has no personal interest in the litigation and
is not a claimant. He has never worked for the Post Office, although whilst at Fujitsu
working on Legacy Horizon he obviously had a great amount of involvement in the
Horizon system. However, given the expert evidence and particularly the degree of
agreement between Mr Coyne and Dr Worden on the number of accepted bugs in
Horizon, which even on Dr Worden’s position was 11 different ones, in addition to Mr
Roll’s evidence there was ample other evidence in relation to my findings on the
Horizon Issues. Mr Roll’s career has started in the Royal Air Force, which he joined in
1976 and left in 1989. Whilst in the RAF, which he had joined as an avionics engineer,
he worked on mainframe computer systems and was selected for a software
development team working on aircraft control and attack systems. After he left the
RAF, he worked in various roles in development and support, and joined Fujitsu in
January 2001. There, he worked in the Software Support Centre, or SSC, in Fujitsu at
Bracknell in 3“ line support.
154.
155.
156.
157.
158.
POL00112816
POLO00112816
After he left Fujitsu he changed career direction entirely and attended the University of
Southampton, from where he obtained a BSc in podiatry. He worked in the NHS until
2011 and then went into private practice, where he remains, now with his own clinic.
The claimants described Mr Roll as a “whistleblower”, effectively making public
allegations about how Fujitsu dealt with Legacy Horizon and the access which was
available to Fujitsu to SPMs’ branch accounts. This was access of a far wider nature
than admitted to by the Post Office or Fujitsu publicly.
It was Mr Roll’s evidence that was substantially behind the acceptance by Fujitsu
witnesses of the ability to obtain remote access without SPM knowledge or permission,
and the injection of messages into the counter. In particular Mr Parker had, in his first
statement, described that Fujitsu lacked this “power” and had said that Mr Roll’s
evidence was inaccurate and misleading. Mr Parker later corrected his 1‘ statement,
and I deal with his explanation about this change — and what he said he meant in his 1*
statement — in my summary of Mr Parker’s evidence below at [464] to [498].
Mr Roll described himself as an IT specialist, a description with which I agree, and one
which is demonstrably correct given the number of years he performed his role at
Fujitsu in the SSC in 3“ line support. Problems or issues would only reach 3“ line
support if 1% or 2" line support did not resolve them. His IT experience includes, in my
judgment, a considerable degree of expertise in software. Indeed, although I am critical
of Fujitsu elsewhere in this judgment, that criticism does not extend to any suggestion
that they employed personnel within SSC who were not experienced and sufficiently
expert in software matters to perform that role. Had Mr Roll not been sufficiently
qualified to work in 3" line support, he would not have been there for the period that
he was.
The Post Office in cross-examination attacked both his recollection, and seemed also
to challenge his expertise. This the Post Office is entitled to do. Given he was 3“ line
support, in my judgment he obviously had a high degree of expertise. There was a
debate at one stage about whether he was “elite” or “super elite”, and he put himself in
the former category. The Post Office also concentrated on a narrower band, or sub-set,
of the issues regarding which he gave evidence, focussing more on potential problems
with code and Mr Roll’s experience of those, rather than the wider issues generally in
respect of Horizon of which he gave evidence. He had made clear very early in his
cross-examination that, so far as he was concerned, data corruption was an issue in
Legacy Horizon as well as software issues which were related to the code.
Mr Roll had left Fujitsu in 2004 and there will inevitably be difficulties in recollection
by any witness of detailed points after such a passage of time. He was willing to indicate
when he could not remember something, and would agree with points put to him by the
Post Office when relevant. These were often on the basis of hypotheses which he was
asked to accept, often in relation to a spreadsheet which Mr Parker had prepared
showing how relatively few software issues Mr Roll was said to have worked upon. Mr
Roll also, again inevitably, was not submerged in the details of the subject matter of the
litigation, in the same way as some of the Fujitsu witnesses such as, for example, Mr
Parker himself, who is the Head of Post Office Application Support and the SSC, and
who has worked on the Post Office account at Fujitsu since 1997, before Horizon was
even introduced. Given Mr Roll left Fujitsu in 2004, it is 15 years since he worked on
Horizon.
159.
160.
POL00112816
POLO00112816
His two statements were fairly short (4 and 8 pages respectively) and given at a high
level. One example can be provided from his cross-examination. He was of the view
that problems referred to 3“ line support, which by definition would mean that 1° and
2"¢ line support had not been able to deal with them, would not include problems caused
by mistakes by SPMs. Mr Parker disagreed with this and this was put to Mr Roll.
“Q: Then the next unnumbered paragraph, Mr Parker says:
"If NBSC were unable to identify the cause of a discrepancy they would often fall back
ona default statement along the lines of 'this looks like a software issue’ so that the SSC
would investigate it. However, Mr Roll's statement that 'if an error was referred to us
then it was extremely unlikely to be due to a mistake made by a postmaster’ is not
correct. The vast majority of discrepancies investigated by the SSC as pseudo 'software
issues' were (and are) not caused by software issues."
I would like to suggest to you, Mr Roll, that that's true, isn't it?
A. The way that I remember it, it was issues to do with the software that were causing
the problems. Whether that was the programme that had been written or data
corruption, that's what I remember as our problems being.
Q. You remember there being problems with data corruption?
A. Yes.
Q. I'm not asking you about data corruption, Mr Roll, I'm asking you about software
issues. The claim I would like to put to you again is the last sentence of that paragraph:
"The vast majority of discrepancies investigated by the SSC as pseudo ‘software issues'
were (and are) not caused by software issues."
Are you in a position to agree to that?
A. From my recollection I would disagree with that, but it was a long time ago.
Q. I'm grateful.”
This passage of cross-examination, and there are a number of other similar examples,
shows the following:
1. Mr Roll’s conclusion was a general one, with which Mr Parker disagreed. Assertion
and counter-assertion are not, of themselves, of assistance to resolving the Horizon
Issues. His conclusion, as with that of Mr Parker, is simply their point of view.
2. Ifa problem made it to 3 line support, then by definition it had not been capable of
being resolved by 1° and 2" line support.
3. Fujitsu would, even at the 3“ line support level, attribute some problems which they
could not understand or resolve to SPM error, and Mr Parker’s evidence is consistent
161.
162.
POL00112816
POLO00112816
with this approach by Fujitsu. This is made clear in the text of PEAKS put to Mr Parker.
There is a very stark example in one of the PEAKs where the Post Office’s own auditors
have specifically ruled out user error, yet Anne Chambers chose to close the PEAK
with the defect cause being user error.
4. The Post Office would try to keep the evidence in cross-examination of Mr Roll
within very narrow bounds. This passage shows that Mr Roll expressly gave data
corruption as one of the problems. Horizon Issue 1 included “bugs, errors and defects”.
In my judgment data corruption is included within that phrase. Mr Roll was kept very
close in his cross-examination to evidence concerning code. Bugs, errors and defects
are far wider than simply code. The Horizon Issues were not specifically restricted to
software issues as narrowly re-defined by the Post Office.
5. Mr Roll would sensibly concede that the events of which he gave evidence were a
long time ago.
6. As shown by the answer “the way that I remember it, it was issues to do with the
software that were causing the problems”, even at this remove of time, Mr Roll had a
specific recollection that there were “issues with the software” that were “causing the
problems”. This is as long ago as 2001 to 2004.
Some of the cross-examination was of no assistance in resolving the Horizon Issues,
for example an exercise with a spreadsheet which Mr Parker had been involved in
preparing, showing the percentage of calls to 3“ line support and the broad categories
into which they had been placed (by Mr Parker). Mr Roll was in no position to give any
helpful evidence in respect of this, and the categories were in any event misleading as
headline descriptive terms. A witness who had left Fujitsu’s employment some 15 years
ago would not, sensibly, be in a position to agree or disagree with such detailed points
or collation of data in any way that would provide the court with useful evidence,
particularly when it emerged (as it later did when Mr Parker was cross-examined) how
misleading the Fujitsu headings were.
Mr Roll had looked at Mr Parker’s statements, although he said only briefly, and he did
not know how many statements Mr Parker had made. He could not remember some of
the details, such as the detailed descriptions of 1° and 2" line support at SSC, and was
frank about this. He could not recall the split between 1“ line support by Fujitsu and the
Post Office’s own 1“ line help desk called the National Business Support Centre (or
NBSC). He acknowledged that he worked in 3 line support, rather than “third/fourth
line support” as he had put it in his witness statement. He did not however over-state
his abilities; for example he said some members of 3 line support were “super elite”,
but he would not count himself as falling within that description. Part of his cross-
examination which I consider fairly encompasses his approach to giving evidence is the
following, where parts of Mr Parker’s witness statements were being put to him.
“Q. And if we can pick it up at paragraph 28 of Mr Parker's witness statement, you
worked in third line for between -- well:
"Between I January 2001 and 31 December 2014 ...” and those are the years that you
worked for Fujitsu"... the SSC received a total of 27,005 calls, meaning that on average
563 calls per month were dealt with over this 4-year period."
POL00112816
POLO00112816
And he refers to a spreadsheet setting that out and he then analyses the data in that
spreadsheet. Would you accept that that is a fair reflection of the amount of calls
coming in, the amount of incidents coming in to SSC when you were there, third line?
A. I can't really remember. I know there were periods when it was very busy and
periods when it wasn't so busy. Sometimes we had three or four jobs on the go at once,
other times we were given other work to do from the manager.
Q. And paragraph 29, Mr Parker says:
"Transferred calls (ie those not resolved by the SSC) are of interest." He says:
"A very small proportion of calls transferred to 4th line support would have concerned
software errors requiring resolution ...”
Stopping there, Mr Roll, that's true, isn't it?
A. Yes.
Q. So he then says:
"... it would be interesting [therefore] to know the number of calls transferred to fourth
line.”
Would you agree to that? It would give some indication of the extent to which incidents
coming into the SSC properly, genuinely represented software areas that required
fixing? That would be a useful way of -- a touchstone of trying to work out --?
A. Yes.
Q. Thank you. He then says that, unfortunately:
"_. while the SSC have records of the volume of transferred calls, we do not retain
records of where they are transferred to and it is not the case that all of these would
have been transferred to 4th line support. For example incidents would often arrive at
SSC from internal teams for routing back to help desks."
Do you remember that?
A. I don't remember that particular ...
Q. But would it be right to say that of the calls coming into third line support, a
significant proportion would go -- of calls that would then be transferred out, would go
to places other than fourth line support? Would that be fair? Does that accord with
your recollection?
A. The way I recollect it is that calls would come in and we would work on them, either
fix them, in which case they would go back to the originator, or we would pass them
on.
163.
164.
165.
POL00112816
POLO00112816
. To other people?
. To other people.
. Depending on the nature of the problem?
> 2 > ©
Yes.
Q. So if there was an infrastructure problem you would probably pass the call on to the
infrastructure team, it wouldn't go to fourth line support, would it?
A. Probably.
Q. And there are a number of other teams that calls coming into third line would be
passed to who would not be fourth line support?
A. Probably. I can't remember the full details.”
This can be seen all to be at a very high level. Mr Roll is not likely to have known the
total number of calls that came into SSC during a 4 year (or indeed any) period; the
average number of calls per month; or of how these were dealt with. He would also not
be likely to be able to recall these details, even had he known them at the time. The
analysis which it was put to him would have been useful to know — the percentage, or
even total number, of calls transferred from 3 line support to 4" line support — would
not have been something he would known at the time either. Interestingly, not only
would such information have been something that Fujitsu could have readily recorded
at the time, as both 3“ and 4" line support at SSC are both Fujitsu functions, and not
only was this plainly accepted as something that would be of assistance, but Fujitsu
“did not retain records of where they are transferred to”. Thus the absence of a useful
record in this respect is a result of how Fujitsu retained its records.
Mr Roll was then taken to an analysis of calls coming in, and calls going out, which
had been done by Mr Parker. This related to some 3,764 calls. Mr Roll was never going
to be in a position to dispute that figure; dispute the percentages given for how the calls
were dealt with; or provide any meaningful evidence about Mr Parker’s analysis at all.
Sometimes unreasonable or unhelpful witnesses take issue with matters of which they
have no knowledge; Mr Roll did not do so. If he did not know, or was not in a position
to answer substantively, he would simply say so.
Another passage of his evidence was as follows, when a passage of Mr Parker’s
evidence was put to him that stated "From the SSC, only a tiny proportion of incidents
were escalated to the 4th line support team. It follows that only a tiny fraction of
incidents raised actually needed to be looked at by the only team who might potentially
effect changes in software." He was asked if he accepted the conclusion:
“A. When you take it as an average then yes I suppose so, but the system evolved from
-- I started there in 2000 I think it was and left in August 2004. The nature of the work
changed over time and the ones that stick in my mind are the ones where there was sort
of the fire-fighting efforts where it was -- there were difficult periods where there were
software issues, so ...
POL00112816
POLO00112816
Q. Yes, Mr Roll. We could by all means go back to the graphs because you will have
seen from the live PEAKs into and out of the SSC it actually gave monthly figures, but
what I'm suggesting to you, Mr Roll, is that -- although you may well have been busy
on all sorts of things, the fact of the matter is that software problems requiring a
software fix represented a tiny fraction of the work that was handled by the SSC third
line support.
A. Yes.”
166. This theme of the way that the Post Office put its case carried on into the evidence of
Dr Worden and his Section 8 statistical exercise, which I address below. The
expressions “tiny proportion”, “tiny fraction” and other terms used sought to
demonstrate, basically, that Horizon worked very well most, if not all, of the time.
Given the use to which the Horizon system was put by the Post Office, namely monthly
balances which governed the branch accounts of SPMs, with shortfalls and
discrepancies being “made good” — which means paid — by the SPMs in question, this
approach can be seen to have limited utility. Firstly, these are subjective terms.
Secondly, given the PEAKs show that a SPM may report a single unexplained
discrepancy for (say) £25,000, or even £1,000 or £500, the fact that this may be only a
tiny fraction of the number of calls the SSC worked on that month might come as cold
comfort to that specific SPM. The number of calls, and how they were dealt with, is of
some relevance to robustness, but such high level and subjective points were not likely
to be of enormous assistance, one way or the other, in challenging Mr Roll’s evidence.
167. It was put to Mr Roll that his role, which he described as “product specialist”, was “the
junior level of people working” in SSC, which he disputed. He explained that most
people in SSC were “product specialists”. 25 of the 30 or so total number of people
were, according to Mr Roll, all product specialists.
“Q. -- the SSC? Of that 30 people how many people were at your level?
A. When I worked there I -- it was two or three people were senior levels, I think Mr
Parker was one, a couple of others, then there was Mike Peach and I believe the rest of
us -- 25 or so.
Q. The figures I would like to ask suggest to you, Mr Roll, are about 25 people had the
junior level and there were about five people who were true specialists who I think you
fairly described as specialist earlier on in your evidence. Would you accept that that
might be the case?
A. Junior level seems to -- it doesn't fully explain the complexity of the system or the
knowledge of the system required, but yes, I suppose ...
Q. Well, let's agree on ordinary -- I'm not seeking to cast any imputations and it is right
that you should — words do have implications. So the ordinary level was your level —
A. Base level.
168
169.
POL00112816
POLO00112816
Q. --and there was this perhaps five or so people above -- the senior people above you
and what did they do? Did they do the more challenging work?”
The Post Office attempted to dilute Mr Roll’s experience and cast him as someone who
was not sufficiently skilful to understand, or give evidence about, the matters he
explained in his witness statement. It is clear to me, and this was effectively confirmed
by the documents put to Mr Parker, that everyone at 3“ level support in SSC was a
specialist. The notion that these 30 people, in a department separate from the normal
Fujitsu work areas, to which access was security restricted, were all at “the junior level”
or “ordinary” with 5 people managing them who were “the true specialists” is
misleading. There were a handful of people at the senior level, and the rest of 3" line
support were product specialists. A Fujitsu internal document, put to Mr Parker but not
to Mr Roll, explained it rather more comprehensively. This is from a 2011 document
but there was no evidence to suggest that there was a wholesale reorganisation of SSC
3" and 4" line support between Mr Roll’s time there and this document.
“3"4 line support
3* line support groups within RMGA include:
SSC — 3 line support for RGMA written application code.
MSS ~ 3 line support for software distribution and event management
3 line support staff apply analytical skills to the symptoms and evidence gathered by
1“ and 2™ line and undertake in-depth investigation into incidents. They have detailed
knowledge of the system based on documentation and source code inspection.
Trained on operating systems, COTS packages that underlie the application and the
coding languages used within the application. They are also expected to self train by
examination of support guides, design documentation written for the components of the
end user application. They will also have access to development _and_package
management tools to allow the production of specialised diagnostic code, scripts and
It is incumbent upon the 3“ line support unit to produce a work around and on 4" line
to produce the final code solution to any software problem. This does not preclude the
production of a workaround by other units or negate the requirement for 4" line to
provide assistance in the generation of a workaround.
The SSC are responsible for the implementation of any workarounds that require data
changes to the live system. They are the only unit with authorisation and sufficient
physical security controls to perform this function.”
(emphasis added)
This gives a more comprehensive picture of the type of expertise required to be in 3%
line support, which is where Mr Roll worked whilst he was at Fujitsu. This is of more
assistance than the terms deployed, such as “the five true specialists” (for the senior
managers) and “the junior level” for Mr Roll. It shows that 3° level support have a high
170.
171.
POL00112816
POLO00112816
level of necessary expertise. They are the only unit with the necessary authorisation and
security controls to be permitted to implement workarounds that require data changes
to the live system. This is not the role of junior personnel.
This area of questioning also focused on code, and introduced another concept, the
“core specialist team”.
“Q. I'm really trying to grope towards what the five true specialists, what kind of work
they were doing. They were doing the more challenging kind of work, weren't they?
A. I find it difficult to answer that from what I remember of the way we worked. There
were some areas where some of the senior people where I would perhaps have been
more -- had more experience because of previous work and previous programming that
I would have been better off and had more knowledge than they would have done, but
in other areas then they would be far superior to me.
Q. Well, let me suggest one area where they would have been far superior to you, Mr
Roll, or at least it was perceived within the organisation -- I'm not trying to have a
debate with you about your own perceptions of yourself, that would be completely
unfair, but the perception at the SSC was that were there software errors or potential
software errors that required large amounts of code to be examined, the people who
would generally do that examination would be those five people, people like Mr Parker.
Would you accept that?
A. Generally I suppose, yes, although —
Q. And it would be relatively rare for someone outside that core specialist team to be
doing that kind of work -- not impossible but relatively rare, yes?
A. Several of us looked at the code on occasions. I was a C programmer in that and
other languages before I moved to Fujitsu so there were areas of knowledge we had
from previous areas, but I suppose most of it would then be passed on to those people.
Q. Isee. So would you accept from me then that generally it would be them that would
look at lots of code but occasionally there might be occasions when someone else would
look at lots of code?
A. From a PinICL perspective yes, but some of us looked at code more often, just out
of interest.”
This series of questions introduced another subjective variable, “large amounts of
code”. Mr Roll, in his answer above, explained that he had been a C programmer before
he moved to Fujitsu. C is a general purpose programming language which is widely
used. Mr Roll, as someone who had been a programmer both in C and other languages
before he went to Fujitsu, had both written code himself (as a C programmer), and I
find would have been very familiar with code in its different guises. Indeed, given the
Fujitsu documentation describing the expertise of 3“ line support in [168] above, which
stated 3" line support would have “detailed knowledge of the system based on
documentation and source code inspection”, it is clear to me, and I find, that Mr Roll
was amply experienced to give evidence to this court on the matters that he did
173.
174.
175.
176.
177.
POL00112816
POLO00112816
concerning the Horizon system (which was at that point Legacy Horizon), the problems
that were experienced with it, and what occurred at SSC. Source code is a term used to
describe a version of software, or a description, as it is originally written. There are
different definitions of source code in different computer authorities (such as the Linux
Information Project) but the precise definition is not important for the purposes of the
Horizon Issues trial.
What is important, in my judgment, is that Mr Roll used to programme in C programme
language and other languages before he went to Fujitsu; he was required to understand
and have “detailed knowledge of the system based on documentation and source code
inspection” by virtue of being a member of 3“ line support at SSC; all the evidence
demonstrates that he did his job effectively and competently; and he did not leave
Fujitsu under any sort of a cloud. It was effectively accepted by the Post Office that he
had a responsible position and a responsible role, and was well trained.
Mr Roll’s evidence was that errors made by SPMs were relatively easy to pick up at 1°
and 2" line support level; that most errors he dealt with were coding errors or data
corruption; that issues were identified that required software “fixes” to be written by
Development; and — importantly in my judgment - that the type of issues that were
routinely encountered at SSC could and did cause financial discrepancies to branch
accounts. He also stated that:
“If we were unable to find the cause of the discrepancy then this was reported up the
chain and it was assumed that the postmaster was to blame.”
He said that even if software fixes were developed the problem would sometimes
reappear several weeks later. He also stated that remote access to branch systems was
possible; the ability was extensive; that this was done without the SPMs being aware;
that data and transaction information could be changed by Fujitsu; and that sometimes
SSC would log into a branch system whilst it was switched on but not in use.
The final paragraph of his 1* statement said:
“In summary, the issues with coding in the Horizon system were extensive.
Furthermore, the coding issues impacted on transaction data and caused financial
discrepancies on the Horizon system at Branch level. It was those issues that I, and
other colleagues at Fujitsu, were routinely working on daily. Furthermore, remote
access to the Horizon system at Branch level was extensive, as was the ability to change
data and change transaction information, even while the postmaster was working,
without the postmaster being aware of this.”
He had also said that “during the course of resolving the software issues, we would
frequently access a Post Office counter IT system remotely”.
His use of “frequently” and “routine” are, in my judgment, subjective, and as explained
above in terms of “tiny fraction”, subjective terms are not entirely helpful. What he
meant by this is it was not unusual for this to occur. It is difficult to judge, at the remove
of 15 years from when Mr Roll left Fujitsu, just how often something that he remembers
as frequently or routine in fact occurred. He sensibly accepted that the more regular or
mundane matters in which one is involved tend not to be remembered, with rather more
178.
179.
180.
181
POL00112816
POLO00112816
unusual events sticking in the memory. In terms therefore of how often this occurred
whilst he was there, it is not possible to come to a concluded view. However, I consider
it important that the occasions to which Mr Roll refers were not isolated incidents on
his evidence, nor were they unusual, and that financial discrepancies on the Horizon
system at branch level was something which he recalls working on daily.
He also gave evidence about hardware failures, and gave specific responsive evidence
in respect of Dr Worden’s expert report.
I accept Mr Roll’s evidence, which is supported by the contemporaneous documents.
Notwithstanding the limitations on his evidence due to the passage of time, I found Mr
Roll to be a reliable and helpful witness. I also found his evidence to be very important.
The Post Office set out to demonstrate that he did not have the expertise that he claimed.
to have; that he was at too low a level in 3“ line support to have been involved in the
matters which he described, certainly so far as software issues were concerned; and that
Mr Parker’s evidence in particularly about what he had been doing should be preferred.
On those two former points the Post Office failed, in my judgment. The success or
failure of the latter point also, but not exclusively, depended upon the evidence of Mr
Parker. After Mr Parker was cross-examined, it was clear to me that Mr Parker’s
exercises put to Mr Roll could not be relied upon to demonstrate what they sought to
demonstrate. I prefer the evidence of Mr Roll to that of Mr Parker and the other Fujitsu
witnesses (with the exception of Mr Godeseth) by some considerable margin. I find that
during the years when Mr Roll worked at Fujitsu, 2001 to December 2004, SSC were
not infrequently involved in attempting to remedy unexplained shortfalls and
discrepancies in branch accounts reported by SPMs. If they were unable to find the
cause of the discrepancy then the assumption would be made that it must be the SPM
to blame. I will return to this subject when I have dealt with the bug table, and my
findings on the number of bugs present in the system.
There is one area of Mr Roll’s evidence, again a wholly subjective one, to which I do
not attribute any weight, not that I disbelieve him. This was concerning the pressure of
work within the SSC and the fact that members of the SSC were under pressure to
“close calls”, which basically means record them as having been completed. The
relevant passages are as follows:
“15. During my time at Fujitsu I know that there were budget pressures and
redundancies which impacted system development and testing. The test team felt they
were under enormous pressure to complete the testing within certain timescales which
negatively affected the test regime. Meanwhile, the development team had to balance
time spent on fixes, with time spent on developing new features for Legacy Horizon
and time spent developing a new system which I believe later became Horizon Online.
16. In my first statement, I refer to the pressure that the SSC team and Fujitsu were
under generally due to an awareness of the financial penalties imposed by the service
level agreements between Post Office and Fujitsu (paragraph 12 of my first statement).
I believe that although individual penalties were quite modest, when applied across
multiple counters/post offices the cumulative figures involved were very high,
potentially amounting to tens of millions or more. I disagree with Stephen Parker’s
statement that these potential financial penalties were not a factor for the SSC
POL00112816
POLO00112816
(paragraph 43 of Stephen Parker’s witness statement) as we were aware of them and
often commented on them, e.g. “That’s saved Fujitsu another £25 million”.
182. I do not attach much weight to this as pressure in the workplace is such a subjective
matter. Some members of SSC may have felt under great pressure; others less so. This
is a subjective account of what it was to work at SSC at that time from the point of view
of Mr Roll. The type of statement to which Mr Roll refers in terms of millions of pounds
saved does not advance matters one way or the other. Given the type of problems that
3* level support was involved in attempting to resolve, there would have been an
inevitable pressure involved in attempting to resolve such issues speedily. Mr Roll
chose to leave Fujitsu, and therefore he left those pressures behind. Putting it at its
mildest, it would be extraordinarily disappointing if PEAKs were closed, attributing
fault to a SPM, simply because that was the easiest and quickest way for a SSC product
specialist to keep on top of their workload. Without separate cross-examination of each
product specialist who chose to close PEAKs where prior entries suggested user error
had been ruled out, it is not possible to determine the degree to which this was their
motivation. However, it is not relevant. It is no part of the Horizon Issues to determine
why, or whether, individual personnel at SSC acted in a particular way — budgetary
concerns for their employer; exercises of judgement; even laziness; or any of the other
many possibilities. Mr Parker gave evidence that “the possibility of financial penalties
was never a factor for the SSC”, and Mr Roll said that it was. Mr Parker’s evidence on
this suffers from a wholesale attribution of this lack of motive to every member of SSC,
as indeed does Mr Roll’s, in the other direction. Everyone is different, and there will
have been a range of different reasons operating on each member of SSC each day. Any
sensible business will, in any event, always have at least part of its attention on financial
performance and this is understandable. The Horizon Issues trial is not an inquiry into
how Fujitsu manages its personnel, or its business. I do question the allocation of
Category A, B or C to some of the PEAKs that were used in the trial, but this is
something that was pursued (though not very far) with Mr Parker and I deal with it
there.
183. The way that Dr Worden was simply not prepared to accept Mr Roll’s evidence at face
value, and set out to disprove his factual evidence, something which in my judgment
an independent expert should never do, is dealt with in the section of this judgment
where I deal with Dr Worden’s evidence. The degree to which the evidence of Mr Roll,
and his experience of Legacy Horizon between 2001 and 2004, affects my conclusions
on the expert evidence will be dealt with in Part L, Overall Conclusions.
184. The fact that I find Mr Roll reliable does not mean that I automatically accept all his
conclusions. I have given a specific example at [181] about financial pressures
However, Mr Roll’s evidence was supportive of the claimants’ case and it was
necessary for the Post Office to render his evidence unreliable, insofar as they could, in
order to damage the claimants’ case and bolster their own. In my judgment, this the
Post Office failed to do. Mr Roll’s evidence was also what directly led to the true picture
emerging of remote access. However, of itself this does not provide an answer to the
Horizon Issues.
Mr Henderson
POL00112816
POLO00112816
185. As explained in [78] and [79] above, Mr Henderson is a director of a company called
Second Sight which was involved in a mediation scheme, until the Post Office withdrew
from it. Initially Second Sight was engaged to perform an investigation, but after its
interim report its scope of involvement changed. It ran from 2012 to 2015, when Second
Sight’s appointment was terminated. The investigation and subsequent mediation
scheme were set up in the light of disquiet about the complaints of various SPMs, the
operation of Horizon, and concerns that were publicly discussed, not least by Members
of Parliament. All of the documents and e mails which Second Sight had at that time
were returned to the Post Office. Mr Henderson estimates these as 16,500 e mails and
18,500 other documents, which were sensibly organised in 1,700 folders. They were in
good order.
186. The documents in that set that were provided by the Post Office for this litigation were
not, according to Mr Henderson, provided in the same way. He criticised them for
having no folder structure, having had metadata removed, and lacking the original date
and time stamps which were present on the original documents that Second Sight had.
been provided with. Some important documents, such as the Fujitsu XML transaction
reports, were described by the Post Office as “unreadable”. He had however reviewed
some, but not all, of the XML data when inspection was provided of some of the
unreadable documents. He did not believe that the documents provided in the Horizon
litigation were the complete set of data provided to Second Sight which he had returned.
I accept his evidence on this. He was clear in his recollection about how the documents
were organised, and approximately how many there were and the type. I do not know
why the claimants were not given, in disclosure, the same documents, prepared and
collated in the same way, as the Post Office themselves had received them from Second
Sight.
187. He described Mr Jenkins, whom he had met, as the Fujitsu lead engineer on the Post
Office contract. He had met him in September 2012. He explained in his witness
statement that he had been given sample XML data by Mr Jenkins, and had used this to
see a level of detail that neither SPMs nor, interestingly, the Post Office, had available
to them to view. He had also been told at the same meeting that remote access was
“occasionally used” by Fujitsu. He asked for complete e mail records for 2008 for the
7 employees known to be working at Bracknell at the time, so he could satisfy himself
as to whether such remote access was done without the SPMs permission or knowledge.
He was not given these, and the partial records he was given were inconclusive. His
reverse engineering of the XML data he was given gave him grounds for concern but
Second Sight’s engagement was terminated before he could reach any concluded view.
188. He had also identified what he considered to be particular difficulties that Horizon had
with foreign currency — it is (or was) a single currency system and only used pounds
sterling — and also the National Lottery. In view of the evidence of Ms Van Den Bogerd.
in the Common Issues trial and the Ping fix, that latter point may not be particularly
controversial as this litigation progresses. Foreign exchange is the subject matter of
bugs 14 and 23 in the bug table. Bug 14 is now accepted by the Post Office in Appendix
2 of its Closing Submissions as one of a number of “bugs with lasting impact (although
they were resolved)”.
189. A working group was involved in the Second Sight scheme, and the progress of cases
all the way through the mediation scheme was slower than the working group wanted.
190.
191.
192.
POL00112816
POLO00112816
Second Sight’s appointment was terminated before Mr Henderson could reach finalised
conclusions on most of the issues he said that were uncovered. I have already stated the
approach that I adopt in relation to the technical nature of the Horizon issues, and how
the evidence of the experts in IT who gave evidence before me effectively takes
precedence over observations by other witnesses on technical issues, which must
include Mr Henderson.
Mr Henderson explained in his oral evidence that he considered his evidence to the
court not to be to entirely unrestricted. He considered it was subject to a restriction that
was imposed upon him. There was a letter of engagement in this respect, which was
dated 1 July 2014 and accompanied by a document which was signed by someone
called Chris Aujard for the Post Office. The letter of engagement set out several pages
of terms governing Second Sight’s engagement. Clauses 2.1 onwards set out what
Second Sight were to do:
“2.1 The Scheme has been set up to try to achieve the mutual and final resolution of a
Subpostmaster's concerns about Horizon and any associated issues.
2.2. Second Sight Support Services Limited ("Second Sight" or "you") has agreed to be
a member of the "Working Group" whose role it is to oversee the Scheme and to assist
in investigating individual Subpostmaster complaints.
2.3. This letter and its schedules will form the basis of the terms of Second Sight's
engagement by Post Office Limited ("Post Office") to provide Services to the Working
Group in relation to the Scheme.”
Clause 6.2 stated:
“6.2 Second Sight will not, and will ensure that the SS Directors and any SS Personnel
will not, act, directly or indirectly, in any capacity (whether for any former or current
Subpostmaster or a competitor of Post Office or otherwise) against Post Office or any
of its officers, directors or employees save to the extent a) that it is expressly agreed in
writing by Post Office that the work proposed to be undertaken will not have a material
adverse effect on Post Office's commercial or financial interests or reputation, or b) as
required by applicable law or by the mandatory rules or requirements of any regulatory
authority, government department or agency to which Second Sight is subject or c) as
required by an order of a court of competent jurisdiction.”
Clause 6.3 imposed this for a period of 15 months.
The shorter one page document which Mr Henderson and his co-director were asked to
sign and return contained different terms. The letter of engagement set out the terms
upon which Second Sight would act in the mediation scheme. The shorter document
stated the following:
“T will not act, directly or indirectly, in any capacity (whether for any former or current
Subpostmaster or a competitor of Post Office or otherwise) against Post Office or any
of its officers, directors or employees save to the extent a) that it is expressly agreed, in
writing by Post Office that the work proposed to be undertaken will not have a material
adverse effect on Post Office's commercial or financial interests or reputation, or b) as
193.
194.
196.
197.
198.
POL00112816
POLO00112816
required by applicable law or by the mandatory rules or requirements of any regulatory
authority, government department or agency to which Second Sight is subject or c) as
required by an order of a court of competent jurisdiction.”
The period of that was stated to be 15 months, corrected by hand from the period
initially typed in of 12 months. This obviously therefore, as signed, represented if not
the same then almost identical terms as those contained in paragraph 6.2 of the letter of
engagement.
It is a curiosity that the detailed terms in the letter of engagement were produced— as
Mr Henderson explained — about half way through the process. It is also the case that
the services to be provided by Second Sight were to be provided to the Working Group,
which also included members of the Justice for Subpostmasters Alliance or JFSA,
although the services were to be paid for by the Post Office. I have outlined these terms
above to make it clear, on a transparent basis, what restrictions were imposed upon
Second Sight. The Post Office’ wish to avoid any “material adverse effect on Post
Office's commercial or financial interests or reputation” is expressed in very wide
terms.
The restriction to which Mr Henderson referred in his oral evidence, which he
considered limited his evidence, was an agreement between the Post Office and the
claimants about what Mr Henderson could state in evidence in this litigation, which he
referred to as the “protocol agreement”. The Post Office was therefore in the position
that it could, to a certain extent, control the scope and extent of Mr Henderson’s
evidence. He also thought that there was a restriction on the length of the witness
statement he could provide. For clarity, there was no such restriction imposed by the
court.
The Post Office, in its closing submissions, submitted the following:
“Mr Henderson was asked whether confidentiality restrictions had caused him any
inhibition in answering the questions put to him in cross-examination. He said that he
had the issue in the back of his mind and that he had tried to make sure that his answers
did not infringe the protocol. It is understandable that Mr Henderson would wish to be
careful, but the idea that he was restricted in answering Post Office’s questions is
bizarre.”
The submissions then went on to deal with the questions put to Mr Henderson, and how
there was no restriction that could have impacted upon his answers.
The actual question posed to Mr Henderson was not as set out in the above extract of
the submissions at [195] above, it was rather wider. It was posed by me and it was at
the end of his cross-examination. It was in the following terms:
“J just want to be clear: is it your evidence therefore that because of that protocol
agreement your evidence of fact to this court is narrower in scope than it would be
absent the protocol agreement?”
His answer was “yes, it is”. He was then asked by counsel whether it had inhibited him
in answering questions, and he said it did not. I do not consider that the closing
submissions are an accurate summary of both questions put to him about this, or the
restrictions he consider had an impact upon his evidence. The restriction was wider than
199,
POL00112816
POLO00112816
impacting upon the questions he was asked, and his answer showed that his evidence
as a whole had been affected.
The mediation scheme was brought to an end by way of notice given by the Post Office
to Second Sight on 10 March 2015. The cross-examination on this was as follows:
“Q:.... If we could just look at it very briefly, the termination letter to which you have
just referred is at {F/13/24.1}. That's the termination letter I think to which you at least
indirectly referred.
A. That's correct.
Q. If we could look then to {F/1324.2}. This is a much longer letter from Post Office
that came to you on the same day. Do you recall this longer letter? If I maybe tell you
broadly what it is to do with. It's a letter in which Post Office sets out a plan for how
Second Sight could finish its outstanding work. Do you remember that letter?
A. Yes, Ido.
Q. And the second paragraph makes clear that Second Sight was expected to continue
working during the notice period and that even beyond the notice period there would
be a proposed future role for Second Sight. That's what this document dealt with, isn't
it? It's a fairly long document.
A. Itis, but there's another document of that date or very close to that date which you
haven't mentioned which was the press release from Post Office announcing the
winding up with immediate effect of the mediation scheme itself. That I understand
was the primary decision and our termination was a consequence of that, not a separate
issue.
Q. Well, I think when you say mediation scheme it's fair to say, isn't it, that this winding
up process, as you describe it, did not put an end to mediations; in fact —
A. It put an end to the mediation working group with, as I understand it, no consultation
and the announcement was made I think the day immediately before the next planned
meeting of the working group, so it was a considerable sort of shock to everybody.”
The following points can be made in respect of the evidence from Mr Henderson:
1. Second Sight was engaged to act in a mediation process. Mediation is consensual
dispute resolution, and such processes require a high degree of confidentiality within
them if they are to be effective.
2. The termination of that process unilaterally by the Post Office, with or without
consultation, is, so far as this group litigation is concerned, something that may or may
not in the future call for further consideration in terms of costs. It is not at this stage
relevant to the answers to the Horizon Issues.
200.
201.
POL00112816
POLO00112816
3. Reasonable and conventional terms of confidentiality were to be expected in any
terms of engagement agreed between the Post Office and Second Sight, given the
scheme was a mediation scheme.
4. Terms seeking to protect the “Post Office's commercial or financial interests or
reputation”, which were included, whether justified or not in all the circumstances of
the role being performed by Second Sight, are not relevant to the answers to the Horizon
Issues.
5. In Farm Assist Ltd (in liquidation) v Secretary of State for the Environment, Food
and Rural Affairs [2009] EWHC 1102 (TCC) a mediator, who had been issued with a
witness summons compelling her attendance to give evidence, failed in her bid to have
the summons set aside. Ramsey J gave a judgment refusing to do so, having considered
the position on confidentiality, without prejudice privilege and other privilege. He
stated, in his conclusions at [44]:
“The court will generally uphold that confidentiality [of mediation proceedings] but
where it is necessary in the interests of justice for evidence to be given of confidential
matters, the Courts will order or permit that evidence to be given or produced.”
The point has not, however, been argued in this case in any respect, and it may never
arise.
6. In this case the restriction has been imposed by the Post Office and agreed by the
claimants. It is regrettable, in my judgment, that any witness of fact feels their evidence
to be restricted by any existing agreement with a party to that litigation. Apart from
anything else, it is something of a contradiction for a witness, who swears or affirms
that their evidence is “the truth, the whole truth and nothing but the truth” to then add
that there is such a restriction. This appears to contradict the requirement to tell the
whole truth. However, the court has never been asked to become involved in resolving
any dispute between the parties in this respect.
7. 1 do not consider that any such restriction — the scope of which I am in any case
unaware - will have had any effect upon my consideration of the correct answers to the
Horizon Issues, or the answers themselves.
Mr Henderson seemed to me to be a careful and honest witness. Although his
involvement in the Second Sight scheme may have led to his having particular personal
views about Horizon, he was measured about how he expressed these. His views about
concems over issues in particular affecting the National Lottery and foreign currency
exchange are, to a certain extent, corroborated both by the Ping fix and the position of
the Post Office on bug 14 in the bug table. However, whether they are corroborated or
not, it is not necessary to come to a concluded view on Mr Henderson’s evidence about
these, as it is the two IT experts whose evidence will be of most assistance in this
respect.
The degree to which the evidence of fact by the claimants affects my conclusions on
the expert evidence will be dealt with in Part L, Overall Conclusions.
202.
POL00112816
POLO00112816
Evidence of Fact: The Post Office
The Post Office called the following witnesses. For the reasons explained further below,
I consider Mrs Van Den Bogerd and Mr Godeseth to be the most important witnesses
called by the Post Office.
Mrs Van Den Bogerd
203.
204.
205.
206.
207.
Mrs Van Den Bogerd gave evidence in the Common Issues trial, and indeed her witness
statement for the Horizon Issues trial was served during that trial in November 2018,
and shortly before she was called as a witness. She had become the Business
Improvement Director of the Post Office shortly before the Common Issues trial. She
is a senior director at the Post Office and is, so far, the most senior member of Post
Office personnel to have given evidence in the group litigation. I had made certain
adverse findings about her evidence in Judgment (No.3) on the Common Issues, which
was sent out to the parties in draft form under the usual embargo shortly before the
Horizon Issues trial started, and was formally handed down at the end of the first week
in the that trial, just after the claimants’ evidence of fact had been completed.
I have already explained that Ms Van Den Bogerd’s evidence was considered
completely afresh in this trial, and that simply because I had made the findings that I
had about her previously, did not mean that I would adopt the same conclusions in this
trial. Her written evidence for the Horizon Issues trial was still substantially of the same
tenor in relation to individual SPMs in terms of widespread attribution of fault to SPMs
as a default setting, but I did detect a change of approach in Ms Van Den Bogerd in this
trial, compared to the Common Issues trial.
Originally, for example, in her statement Mrs Van Den Bogerd had specifically
attributed Mr Patny’s loss on Money Gram to user error, based on what she said the
data showed. This passage, in paragraph 72, was corrected by her on a sheet of
corrections of a number of the Post Office’s witnesses, and in that correction (there
were a number of others) she explained Mr Patny had cancelled the transaction but had
not reversed it. She therefore accepted that the transaction had been cancelled.
She generally demonstrated in her cross-examination a more realistic approach to the
accuracy of her evidence than she originally demonstrated in the Common Issues trial.
By the time she was cross-examined she would have had the benefit of reading
Judgment (No.3), and that might explain her new approach.
She had also amended, in her sheet of corrections, certain statements that had been
included in her Horizon Issues witness statement that were simply not sustainable on
the facts. One example of this was in relation to Mrs Stubbs (a claimant and witness in
the Common Issues trial, though not in the Horizon Issues trial) whom Mrs Van Den
Bogerd had said “chose to settle centrally” items that were, in fact, obviously and
plainly disputed by Mrs Stubbs; and in respect of which no SPM had no real “choice”.
Their choice, such as it was, was either paying immediately or settling centrally, which
meant not paying immediately and seeking time to pay. Mrs Van Den Bogerd’s
explanation for this was that she had not known these sums were disputed by Mrs
Stubbs. Quite how that could be, given the extended saga in relation to these sums, the
involvement of Mrs Stubbs’ MP on her behalf (Sir John Redwood, a former Cabinet
208
209.
210.
211.
212.
POL00112816
POLO00112816
Minister) the Post Office’s promises to Mrs Stubbs and her MP of an investigation (the
results of which, if one was ever done, have still not emerged so far as I know), and
indeed Mrs Stubbs’ own evidence in the Common Issues trial, is not at all clear.
She also explained that in relation to Mrs Burke “I have looked at other evidence in
relation to Mrs Burke and what was very clear to me is that Mrs Burke had done
absolutely nothing wrong in that situation.” This was not at all how Mrs Burke had been
cross-examined. Mrs Van Den Bogerd said that the corrections she had made were
communicated to the Post Office’s solicitors before the Horizon Issues trial had started.
If that is true, I do not see how counsel for the Post Office would have cross-examined
on the basis of her un-corrected statement.
She gave evidence about out of hours transactions and so-called phantom sales, the
latter of which she explained (in her written evidence) as follows. “I am informed by
Post Office’s solicitors that in the course of investigating this matter, Fujitsu have
advised that ‘phantom sales’ were reported in around 2000 which appeared to be caused.
by hardware issues”. There is a master PEAK in relation to this from 2001, and even
though Mrs Van Den Bogerd was very closely involved in the issues on Horizon, she
had not known about this until some time later. Indeed, she could not remember the
approximate year when she had become aware of it. She did not even recall, in the
witness box, having seen the master PEAK before.
I am most surprised that Mrs Van Den Bogerd could not remember seeing this PEAK
before she was shown it in cross-examination. It is a very important PEAK. It is PEAK
number 0065021, dated 17 April 2001. The reason it is important is as follows. It relates
to multiple branches. It concerns phantom transactions. It identifies dissatisfaction from
more than one SPM as to how phantom transactions are being investigated and
resolved, or more accurately, how they are not being. It shows calls being “closed” — ie
brought to an end - without the permission of the SPM, even though that should not be
done. It also shows at least one SPM threatening not to comply with their contractual
obligations due to lack of confidence in the system, and also threats of legal action.
Further, in one branch, where items were the subject of phantom transactions
(according to the SPM) ROMEC, the Royal Mail’s own engineers, attended that branch
to fit suppressors and other equipment in an effort to rectify this.
The PEAK plainly records the involvement of ROMEC, the Royal Mail’s own
engineering personnel, as follows. “ROMEC have been to site and state that they have
actually seen the phantom transactions, so it is not just the PM's word now.” (emphasis
added). The significance of this entry is obvious, and notable. Mrs Van Den Bogerd.
agreed that this was “independent site visit corroboration of the problem by Royal
Mail’s own engineers at the branch”, and she also agreed that this was “clearly not user
error any more”. I do not understand how the master PEAK containing such important
information could not have been at the forefront of Mrs Van Den Bogerd’s mind. It is,
in my judgment, important corroboration from those with experience of Horizon (the
Royal Mail’s own engineers) who state they have seen the phantom transactions.
However, the conclusion reached by Fujitsu and recorded in the PEAK was as follows:
213.
214.
216.
POL00112816
POLO00112816
"Phantom transactions have not been proven in circumstances which preclude user
error. In all cases where these have occurred a user error related cause can be
attributed to the phenomenon"
The PEAK also concludes “No fault in product".
This conclusion by Fujitsu is simply not only not made out on the factual evidence,
including the contemporaneous entries in the PEAK itself, but it is, in my judgment,
entirely unsupportable. It wholly ignores the independent support of the ROMEC
engineers, who have reported that “they have actually seen the phantom transactions”
and it arrives at a conclusion that, in my judgment, entirely contradicts the evidence
available to Fujitsu at the time and indeed contradicts common sense. Given the entry
that “‘it is not just the PM’s word now”, this conclusion ignores two entirely different
sources of actual evidence. One, what the SPM reported. Two, what the ROMEC
engineers visiting the branch actually saw.
Another PEAK Mrs Van Den Bogerd was asked about demonstrated the lack of
accuracy in Fujitsu’s characterisation of the type of problems that made their way to
the SSC (and which formed the basis of the Post Office’s cross-examination of Mr Roll
based on the Fujitsu/Parker spreadsheet). This was PEAK 0208335 of 14 February
2011, also headed “Phantom Stock Declaration”. This related to withdrawn stock
discrepancies, and was summarised as "Branches will be forced to declare stock when
they don't want to. Apparent reappearance of withdrawn stock may cause spurious
discrepancies." It was recorded in the PEAK that this could affect 10 branches per week
over the next few months. The PEAK showed that a SPM was told to declare the correct
stock, which that SPM disagreed with, and which Mrs Van Den Bogerd agreed would
mean the SPM entering an account into the Horizon system with which the SPM
disagreed. Mrs Van Den Bogerd had not seen this PEAK either before she was asked
about it in the witness box. She agreed that this PEAK appeared to be a software
problem, and also that it was in a sense easier for the SPM because the phantom
transaction related to stock that was not even any longer in use. She simply could not
have dealt in that stock.
In any event, I found this PEAK of great assistance, not only due to the content I have
summarised above. A fix was developed by Fujitsu but Anne Chambers, a Fujitsu
employee who had also given evidence in legal proceedings in court before (in at least
one case, the Castleton case) had entered the following in the PEAK which stated that
“this fix and the MSC already applied doesn't remove all old declarations”. Further,
Fujitsu chose to categorise this PEAK as “Administrative Response”. That term does
not begin, in my judgment, properly to describe or summarise the problems to which
the body of the PEAK referred. I find that this PEAK clearly related to a software
problem, regardless of the misleading way that Fujitsu chose to categorise it as
“Administrative Response”. I consider that anyone reading this PEAK at the time would
conclude that this was a software problem.
Mrs Van Den Bogerd was taken through a number of examples of real-world situations,
recorded in a variety of contemporaneous documents, where a wide variety of SPMs
reported a very wide range of problems. In one an internal Post Office e mail reported
that a SPM had “found sensitive issue with Horizon when the system put a phantom
cheque on the cheque line in July 2013. Claims to have evidence to support his claim.
POL00112816
POLO00112816
Although he himself did not suffer a loss, thinks that Horizon is flawed. Did not ask to
be contacted about this. Just wanted to say that he had this information and threatened
to go to MP asa result.”
217. The question was posed internally at the Post Office:
"Given the current media and in particular the BBC's attention on Horizon, do you
think it is worthwhile looking into this alleged flaw' with Horizon that this SPMR has
highlighted to pre-empt any enquiries from his MP?”
218. The ultimate response from Andrew Winn of the Post Office was that the claim could
not be investigated without further details and Fujitsu involvement, that Mr Winn did
not understand the purpose of the call by the SPM, and also stated:
“My instinct is that we have enough on with people asking us to look at things.”
219. Mrs Van Den Bogerd agreed that this was an inadequate response. She said it would
have been very easy for Mr Winn to have contacted the branch and obtained further
details, and he should have done so. In my judgment, the stance taken by the Post Office
at the time in 2013 demonstrates the most dreadful complacency, and total lack of
interest in investigating these serious issues, bordering on fearfulness of what might be
found if they were properly investigated. The SPM, whose branch was known to the
Post Office, should obviously have been asked for further details (if further details were
required for an investigation), and the Post Office and/or Fujitsu should plainly have
investigated the matter as a matter of importance. By 2013 Horizon was an
extraordinarily controversial subject; there can simply be no sensible excuse for the
Post Office failing to try and understand this particular subject. This is particularly
reprehensible given that an internal Post Office document in August 2013 showed that
Mr Winn’s involvement in this was because his area of responsibility was as follows:
“also responsible for resolving specific branch accounting issues.” It was his specific
job to resolve specific branch accounting issues, yet he decided “we have enough on”.
T agree with Mrs Van Den Bogerd that this is inadequate — that is putting it at its most
favourable for the Post Office. Somewhat stronger terms are also justified.
220. Mrs Van Den Bogerd had only learned of the ability of Fujitsu in terms of remote
access, namely the insertion of transactions at the counter under Legacy Horizon
“within the last year or so”, which given she is central to the Post Office’s position in
the group litigation shows, in my judgment, a remarkable situation. It is not necessary,
in order to resolve the Horizon Issues, to go further into why she has found this out so
belatedly, or why that might be.
221. Mrs Van Den Bogerd would accept changes to her evidence in cross-examination where
these were justified. One example was in relation to Mr Latif and corrections issued in
respect of the lottery. Her written statement had said:
“However, due to an error by Post Office, instead of increasing the scratch card stock,
the TAs decreased the stock. To be clear, this was a data entry error by Post Office and
not an issue with Horizon.”
POL00112816
POLO00112816
Actually, given the process of issuing TAs for the lottery was entirely automated, and
done in conjunction with the Camelot system and by Horizon, she accepted that the
passage should be corrected. The cross-examination was as follows:
“Q: So would it be fair to correct that part of your statement —
A. It would be actually, yes.
Q. -- to say "To be clear, this is an issue with Horizon and not a data entry error by
Post Office"; is that a fair correction to make?
A. Ihave made a mistake in that the way I have worded that, absolutely, yes. So,
yes, it is fair.
Q. Would you agree with the formulation I have given, or would you prefer
something slightly different? What would you want the court to note as your
evidence?
A. So the automated -- my understanding is the transaction -- the TA is the
information that comes from Camelot to us and then it is passed through into
Horizon, so in that respect Horizon just conveys it, is my understanding, and the
information that's come from Camelot in that respect would be incorrect.
Q. So the point is that either way, it is not a manual data entry by Post Office?
A. No, it's not.
Q. Whatever it is, it's definitely not that, you agree with that?
A. Yes, absolutely not. Agree, yes.
Q. What it could be is some problem with the information somewhere between the
terminal in the branch —
A. The lottery terminal, yes.
Q. The lottery terminal in the branch and the matters showing up on the face of the
Horizon terminal in the branch?
A. Yes, l agree that.
. Somewhere there?
Yes.
. So it's definitely not a user error, is it?
. No, that's not user error.
QP > 2 > 0
. Right. And on the face of it, it at least suggests some doubt as to the robustness
222.
223.
224.
POL00112816
POLO00112816
and integrity of the Camelot data coming through in that automated system, doesn't
it?
A. I would say yes.”
It can therefore be seen that her written evidence - “To be clear, this was a data entry
error by Post Office and not an issue with Horizon” — could more accurately be stated,
after she answered these questions, as “to be clear, this was not a data entry error by
Post Office and was an issue with Horizon”. There is a world of difference between
those two statements, which are, in my judgment, poles apart both in their content and
effect. Indeed, the accurate statement is the exact opposite of how it had been put in her
witness statement.
She also accepted that whereas her written evidence relied, so far as Mr Latif’s
experience was concerned, upon the fact that TAs had been “accepted by the branch,
which could have been challenged at that point”, in fact a branch had no choice but to
accept TAs. They simply had to be accepted by the SPM. She also accepted that there
were problems with TAs not just for the Lottery, but for Post and Go, and also
Paystation, including in Crown Offices (which were not branch Post Offices but which
also used Horizon).
She had also said that Mr Coyne had relied upon extracts from the Rose Report which
were “taken out of context and mistakenly claim[s] that the relevant reversal was issued
in error by Horizon and not the” SPM. She then went on to explain why this was not
correct. The Rose Report is explained in more detail in the Technical Appendix, and
also referred to in the part of this judgment at [934] onwards where I consider
contemporaneous documents. However, in her cross-examination, she accepted that
this part of her statement was not correct. The Rose Report, in fact and in express terms,
stated that the reversal was done by the system, something Mrs Van Den Bogerd
accepted; she said that it was “done by the system, absolutely yes”, that it was
“definitely generated by the system” and there was “no question” it was generated by
the system. She stated she was not “making herself clear” in her witness statement.
Mrs Van Den Bogerd was re-examined on this specific point and the question and
answer are as follows:
“Q: The extracts taken from the report by Helen Rose referred to by Mr Coyne are
taken out of context and mistakenly claim that the relevant reversal was issued in
error by Horizon not the subpostmaster.”
I just want to give you an opportunity -- Mr Green was pressing you to accept that
actually what you had said in that sentence was wrong. I would like to give you an
opportunity to explain to the court what you actually meant.
A. What I meant was that the actual reversal was part of that recovery and it had
actually taken place as it should have taken place, which is what I meant in that. So it
wasn't a failed reversal because it actually had happened as it should have happened,
but I accepted in there that the -- it wasn't obvious to the postmaster at the time that
what had happened -- that he hadn't -- because it didn't show that he had actually -- it
showed that he had done it and he knew he hadn't done what we referred to earlier
was an explicit reversal. That's what I meant in that.”
226.
227.
POL00112816
POLO00112816
I do not accept Mrs Van Den Bogerd’s characterisation of the evidence she had
originally given in her witness statement. She had clearly stated that Mr Coyne, having
taken passages out of context, “mistakenly claim[ed] that the relevant reversal was
issued in error by Horizon, not the Subpostmaster”. In fact she accepted that the reversal
had been generated by the system, which in my judgment it plainly had. Not making
oneself clear is a curious way of describing that her own statement had said the exact
opposite. In my judgment her witness statement had, on the face of the statement, stated
the exact opposite to this part of the Rose Report.
In brief terms, the Rose Report dealt with a situation whereby the data being used made
it appear that a reversal had been done by the SPM, when in fact it had been done by
the Horizon system. Mrs Van Den Bogerd was asked about one particular feature of the
Rose Report (named after its author, Helen Rose), which dealt with the fact that no
separate code was used such that this could be identified. This arose as a result of an
occurrence at Lepton when the SPM engaged a forensic accountant as although he paid.
the shortfall (about £80) he was adamant that he had not done the reversal that the
system showed he had done. In other words, a reversal could appear in the Credence
data as though it had been done by the SPM, yet in fact it had been the system that had
done the reversal. I consider this to be important, given that the Credence data was used
by the Post Office in any investigations. An extract from the report was put to her in
her cross-examination (because she had given the evidence above about Mr Coyne, the
Rose Report) together with a quotation from an e mail exchange with Mr Jenkins:
"Q:
“Just one question from my part - if the reversal is system created but
shows as an existing reversal, could this not be reflected with a
different code ie SR (system reversed) to clear up any initial challenges.
My feelings at the moment are not questioning what Horizon does as I
fully believe that it is working as it should, it is just that I don't think
that some of the system based correction and adjustment transactions
are clear to us on either Credence or ARQ logs."
That's what she is saying, yes?
Yes.
. And that was a fair observation, wasn't it, by her?
. That was a fair observation, yes.
. About the shortcomings of Credence and ARQ logs, yes?
Yes.
2 > 2 > © >
. And Gareth Jenkins' answer:
"I understand your concerns. It would be relatively simple to add an
extra column into the existing ARQ report spreadsheet, that would make
it clear whether the reversal basket was generated by recovery or not.
228
POL00112816
POLO00112816
I think this would address your concern. I'm not sure what the formal
process is for changing the report layout. Penny, can you advise as to
the process: is this done through a CR?"
Do you know what a CR is?
A. Change request.
Q. Change request, okay. Then at the bottom:
"Ldo believe that the system has behaved as it should and I do not see
this scenario occurring regularly and creating large losses. However, my
concerns are that we cannot clearly see what has happened on the data
available to us and this in itself may be misinterpreted when giving
evidence and using the same data for prosecutions.
My recommendation is that a change request is submitted so that all
system created reversals are clearly identifiable on both Fujitsu and
Credence."
Do you know if that change request was acted on after that?
A. I don't believe it has been acted on.
Q. You don't believe it has been?
A. I don't believe so.”
(emphasis added)
Another express passage or conclusion in the report was also put to her:
Q. "The reversal was due to recovery (Counter Mode Id = 8 118) so this was not an
explicit reversal by the clerk. This scenario is fairly rare so it is certainly quite easy for
the clerk to have made a mistake and either he or the customer could be in pocket/out-
of-pocket (depending on exactly what happened!).”
Then this:
"The system is behaving as it should.”
So there were issues, weren't there, where a problem could arise for a subpostmaster by
design of the system; were you aware of that? There was a whole category of PEAK
codes for faults which are agreed between Fujitsu and Post Office to just stay like that
as part of the design?
A. Okay.”
These two questions were then put individually:
“Let's take it in stages. You can see there it says "The system is behaving as it should"?
229.
POL00112816
POLO00112816
A. Yes.
Q. That would be problematic for a subpostmaster, wouldn't it?
A. The way Gareth describes it here would be, yes, because what he says is that it
would have printed the session receipt but it doesn’t seem as if it did, when actually
disconnection transaction receipts were actually printed in this example and a recovery
receipt was printed. But that's not referred to in here.
Q. Well, let's just have a look at that. Let me just ask you my second question and then
we will go on to probe that with more care. The second point is are you aware of a
closure code for Fujitsu for PEAKs which refers to faults which are known in the
Horizon system but agreed between Post Office and Fujitsu to stay there?
A. I'm not aware of a closure code.
Q. You didn't know about that?
A. No, I don't know about that.”
The following important points arise in respect of this:
1. Mr Jenkins thought that “the system had behaved as it should”. Given Mrs Van Den
Bogerd accepted that this situation could lead either to an SPM, or a customer, either
being in pocket or out of pocket, I disagree.
2. The conclusion was that the scenario might not occur regularly and create “large
losses”. The size of the potential losses was plainly relevant to Fujitsu. Given how the
Post Office used the Horizon system vis a vis its SPMs, sums that were not “large” to
Fujitsu does not mean that smaller losses would not still be appreciable sums to
individual SPMs, who would have to pay for them if there were branch shortfalls. There
is also no way of discerning what Mr Jenkins meant by “regularly” in terms of the risk
of re-occurrence. Both regularity, and whether potential losses were large, are
subjective. The Lepton incident showed that the Post Office required the SPM initially
to pay the shortfall, which he did.
3. The Credence data was inadequate to show what had actually happened. Indeed, not
only were this data inadequate, it made it look as though the SPM had done something
that he or she had not done, and which the system had done.
4. It was accepted that more was required in terms of the required accuracy for “giving
evidence and using the same data for prosecutions”. It was concluded that system
created reversals had to be identifiable, which they were not as at the date of the Rose
Report and these e mails.
5. A change required a formal Change Request, and that was not initiated or acted upon.
6. Finally, and in my judgment also importantly, any risks that Fujitsu and/or the Post
Office “cannot clearly see what has happened on the data available to” them and “this
in itself may be misinterpreted when giving evidence and using the same data for
prosecutions” is a serious matter. I do not understand how a report containing such a
230.
231.
232.
233.
234.
POL00112816
POLO00112816
reference to such a serious matter could be mis-summarised by Mrs Van Den Bogerd.
in her witness statement dealing with Mr Coyne’s analysis of this.
A large number of documents, including operational documents, were put to her, many
of which it appeared she had not seen before. For example, she was shown documents
in respect of MoneyGram dealing with a software release in January 2016 which
showed, to use the IT speak of the document, “multiple instances of system latency” —
or, to use more normal language, problems in the operation of the system. These
persisted through 2016 until nearly the end of that year. The internal summary of the
problem was:
"For the last several months Post Office has experienced a live operational issue with
MoneyGram transactions across the branch network. In the event of a transaction
timing out at the counter, a system error message is displayed to the user ... and the
transaction is aborted. This leaves no record of the transaction at the counter and the
transactions and funds may or may not have been committed in the MoneyGram
domain. This causes significant issues for Post Office and MoneyGram and for
customers."
(emphasis added)
Mrs Van Den Bogerd said that she had not seen the internal document setting all this
out until she gave evidence in the witness box. One of Mr Patny’s issues in his evidence
was about MoneyGram.
She had also been involved in a meeting on 17 May 2010 with James Arbuthnot MP,
Oliver Letwin MP, and others from the Post Office, including the Chairman and the
Chief Executive of the Post Office. The documents, including the briefing pack,
demonstrate that the MPs were told that the Post Office wished to be “open and
transparent”. This was not accurate. The MPs were told that the Horizon system had
undergone an upgrade in 2010 and had the full support of the NFSP. In reality, its
functionality for SPMs had not been upgraded, and the NFSP had privately expressed.
concern to the Post Office about Horizon. There is nothing open and transparent about
telling these MPs information to the contrary.
A large number of different issues with both Horizon Legacy and Horizon Online were
also put to her, only some of which she knew about. She has been closely involved with
the defence of Horizon for some years. To be fair to Mrs Van Den Bogerd, she has also
been responsible for some changes over time which have made Horizon easier to use,
more reliable and (to use the wording of Horizon Issue 1), more robust. Horizon as at
the date the experts were looking at it is a more robust system than it was, say, 5 or
even 3 years ago. A great deal of this is likely to be due to the efforts of Mrs Van Den
Bogerd.
Equally, however, the contents of some of the internal Post Office documents that were
put to her were, in my judgment, very damaging to the Post Office’s case on the Horizon
Issues. In an “Extracts from Lessons Learned Log” document of 11 November 2015,
which was heavily redacted (including the redaction of single words), one entry under
“issues identified” was as follows regarding the Post Office:
235.
236.
237.
POL00112816
POLO00112816
"Failure to be open and honest when issues arise eg roll out of Horizon, HNGx
migration issues/issues affecting few branches not seemingly publicised."
(emphasis added)
This frank expression in a document authored by Mrs Van Den Bogerd herself’ as
recently as 2015 is consistent with the general tenor of the claimants’ case and their
criticism of the Post Office. This entry in this contemporaneous internal Post Office
document is, in my judgment, inconsistent with the picture which the Post Office
continually seeks to portray, namely that it wishes to be open and transparent about
issues with Horizon, and is as interested in getting to the bottom of any problems with
Horizon as anyone else.
There were two periods that she was asked about which clearly showed the problems
experienced by Horizon almost from the outset. These were the introduction of Legacy
Horizon, in late 1999 and 2000 (the national roll-out); and the migration to Horizon
Online in 2010 (including the pilot scheme in 2009). Mrs Van Den Bogerd gave some
evidence in her statements about both of these, in complimentary terms, although she
was far more involved in the latter than the former. At the time of the introduction of
Legacy Horizon she was only responsible for a smaller number of branches in her then
role.
In PEAK PC0033128 an entry in November 1999 shows that the branch at Dungannon
experienced a discrepancy of £43,000. The PEAK states:
“PM - Dugannon PO £43k discrepancy
Outlet has a discrepancy of £43,000 after balancing SUs and doing office snapshot.
Phil Turnock POCL BSM has advised outlet on this weeks balance. Steve Warwick
development is investigating why this mis-balance occurred.
Immediate impact of this weeks balance has been addressed but POCL are concerned.
that the cause is still unknown and this will affect this and other outlets.”
After some investigation the following entry appears:
"L have talked with development ref this problem. It is seen as a one-off. No fault can
be found and developments do not expect to be able to find a fault with the evidence
available. There is no additional information available as evidence. I suggest this call
be placed on monitor for 1 month."
The approach mirrors so much of the case. A discrepancy occurs; Fujitsu cannot find a
fault; and say they do not expect to be able be find one “with the evidence available.”
By February 2000 Fujitsu state “counter 5 looks suspect” and continued
“Further examination of the event logs for these two counters indicate that counter 5
looks suspect (C drive nearly full and big gap of no messages).
POL00112816
POLO00112816
Calls from PO into HSH for period between 30-Oct and 10-Nov indicate a reboot
(counter not specified, but would tie in with counter 5 event log) on Saturday 30-Oct-
1999.
The evidence in the message store was that messages continued to be written to the
message store but that all the 'Payment' transactions which should have been recorded
in the rollover trailer messages failed to appear (although others did, such as the Rem
OUT and Transfer OUT totals).
This indicates that the problem was not one of running out of Disk space but of failing
either to retrieve, or write out, transaction totals for one particular node in the node
hierarchy.
Given that there were known problems with corrupted Persistent Object indexes at
about this time, it is possible that an update to an EPOSSNodes object failed to be
registered correctly at the outlet, causing the node accumulation to fail.
It was decided to prove this out by deleting the 'Payments' node in the node Hierarchy
and then running the SU balance, to attempt to identify the root cause of the problem.
Call passed to testing to be scheduled.
Update 18th Feb 2000
The test was carried out on 16th February as follows: delete the Payments EPOSSNodes
object before producing a SU balance, on a version of the current live system (CI2_2R).
When trying to print the Payments part of the SU balance, the missing node is detected
by the system and an error tablet with message "A system error has occurred whilst
printing. Please ring the helpdesk. Error at 67640." is generated. So the balance could
not be finished.
This type of error trapping error trapping was introduced at the end of last year when
resolving A1298 issues and we are investigating if the outlet did not have such error
handling when the problem occurred.
Certainly, with the current system, a missing Payments node now would not go
undetected.
The problem is currently back with development for further investigation.”
238. On 17 March 2000 there is another entry that says there has been another incident “of
a very similar nature” at another office. This can therefore be seen to be something that
was not, as recorded initially, a “one-off”, even if that were acceptable given the
discrepancy of £43,000. Mrs Van Den Bogerd agreed that this showed that the
underlying information was not checked until February 2000, even though the
discrepancy occurred in November 1999. She was asked about a later entry in April
2000 that referred to another occurrence, this time in Appleby for £9,000
approximately. The cause for this was recorded in July 2000 as “data trees failing to
build” and that entry states “there have been a number of calls relating to this kind of
issue. A fix has been put in at C14 which will prevent this happening”. Mrs Van Den
Bogerd accepted that this was an example of an error occurring, and the system itself
failed to spot that the error had occurred.
239. She was shown PEAK PC027887 which was dated July 1999 and related to “receipts
and payments misbalance”. This showed an escalating misbalance that was increasing,
in my judgment, in dramatic steps up to over one million pounds:
“On week 9 receipts and payments misbalance of £1337.05, week 10 misbalance of
£24000, week 11 misbalance of £12000, week 12 misbalance of £1051111.48 and week
240.
241.
POL00112816
POLO00112816
13 misbalance of £17426.05, she has a difference on week 11 of balance due to post
office and balance brought fwd on week 12 of £1082544.32
Overall these weeks net out a difference of £ 27343.84. she needs business support
(reconciliation) to look into this.”
She agreed that £1.05 million was a large amount for any branch and could not be a
“real amount of cash” that “the SPM has put in her pocket”. A later entry in the same
PEAK of 27 July 1999 was:
“Balance brought forward was multiplied twice due the known software error. The
initial balance brought forward for this CAP was £1196622.72. This was multiplied
twice to give a total BBF of £2279189.04.
The discrepancy was therefore £1082540.28. This was due a known software error
which has no been resolved.”
She did not know if “no resolved” meant “now resolved” or “not resolved”. That PEAK
was not closed until 31 August 2000, and was categorised as “Administrative
Response”. One of the latest entries in this PEAK, which was not put to Mrs Van Den
Bogerd but in my judgment is relevant to Fujitsu’s approach generally, in particular to
categorisation of PEAKs (as this had already stated there was a “known software
error”), was dated 31 August 2000 (one year after the original issue occurred) and it is
most convenient to reproduce here in this judgment. This stated:
“I have now taken over analysing this problem from Steve Warwick who requested
additional information on 13/10/1999 17:49:21. I have looked at the information and I
am not convinced it can possibly be complete.
For example he said he wanted information from 20th May until 26th May. I opened.
the spreadsheet mc20may.xls and observed it only had 6 counter 32 transactions in - is
this correct?
In any case my team is not used to dealing with data in this form (spreadsheets). We
have developed techniques to look at problems using a full message store plus audit
and event logs from the failing counter. Even if all the relevant spreadsheets could be
obtained I do not think it would be worthwhile me getting to grips with this new method.
of analysing problems.
I see this is a very old problem (21/07/1999) and there have been many software updates
since then.
May I suggest we discontinue investigation of this particular problem but that if a
similar problem occurs again you send full message store plus audit and event logs from.
the failing counter.”
And
“Closing call on basis of insufficient evidence. As this is such an old call I have not
contacted the call originator. I suggest that this call remains closed!
INSUFFICIENT EVIDENCE - ADVICE GIVEN”
242.
243.
244.
245.
246.
POL00112816
POLO00112816
(block capitals present in original)
The number of different incidents recorded in the monthly incident review increased in
November and December 2000, which is shortly after (for example) the date when
Horizon was installed in Mr Bates’ branch office at Craig-y-Don, as set out at [106] of
Judgment (No.3). By then the Horizon System had been rolled out nationally. As the
ICL Management Summary that was put to Mrs Van Den Bogerd states:
“The most frequently occurring incidents in November were both types of Receipts and
Payments Incidents (Migration and Post Migration), with 31 incidents per category.
The Migration incidents have remained at the same level, whereby the Post Migration
occurrences have increased. This was followed by 17 Transactions Polled by TIP but
not by HAPS, these were due to delayed transactions as reported on APSS 2133c. These
transactions are added back into normal processing.”
Mrs Van Den Bogerd was not aware of these problems going on; she was a Regional
Network Manager and did not have a national role at the time. No background
information was provided to her about problems branches other than her own were
experiencing, which were being passed back to ICL at the time. She therefore had no
knowledge of the type of problem experienced by Mr Bates in June 2001, which was
recorded in a Post Office audit document of his branch that was put to her which stated
“A correct assessment of cash holdings could not be made because the Horizon system
intermittently adds the previous days cash holdings to the daily declaration.”
By 2010 and Horizon Online, she did have a national role and broader responsibility.
She agreed that Horizon Online did not upgrade functionality, and documents were put
to her that showed, as Mr Green put it “a litany of issues” with Horizon. The pilot
scheme went sufficiently badly that Fujitsu put it on “red alert” and the high volume
pilot was suspended in April 2010. Mrs Van Den Bogerd had not mentioned this in her
witness statements and did not know this had occurred. The NFSP are recorded as
“raised concerns but remain supportive”. All migrations were suspended as from 25
March 2010. There were database errors recorded in a Post Office document at F/614
called “Horizon Online Programme Update” which required an Oracle patch set to be
supplied by Oracle, and on one occasion for a limited period 50% of MoneyGram
transactions failed.
A Network Functional Report of May 2010 was put to her. Over 5 pages of the 6 pages
are redacted. Mrs Van Den Bogerd could not recall if she had seen this document, and
said that given the extent of redactions, it would be hard to recognise it, an observation
with which I agree. She said that she was not aware of there being issues with Horizon
over Easter 2010, which the document recorded.
Redactions had been incorrectly applied by the Post Office and/or its solicitors to some
of the relevant documents. This can be seen from the following example. I asked
Leading Counsel for the Post Office to review certain redactions and having done so,
an unredacted version was disclosed of a document dated 25 June 2014 entitled “Branch
Support Programme”. This had been co-authored by Mrs Van Den Bogerd. The purpose
of the paper was to “Update the Post Office Executive Committee on the progress of
the Branch Support Programme.” Each one of the 6 key performance indicators against
which the programme would be tracked had initially been redacted by the Post Office,
and following the review they were unredacted. They were as follows:
247.
248.
249.
250.
POL00112816
POLO00112816
“* Reduction of operating costs by £3m per annum
* Reduction in net agent debt by £1m
* Reduction in subpostmaster suspensions as a result of audit shortages to a level of 60
per year
* Reduction of calls into NBSC by 25%
* Reduction of audit losses of £10k of over by 50%
* Satisfaction with on-line training models of 95%”
Apart from the first bullet point, three of the other five are consistent with an aim by
the Post Office to make improvements to reduce the amount of debt incurred by, and
suspensions, of SPMs, and reduce the audit losses. The other two are related to training.
This is not consistent with a view that the debt/suspensions/audit losses are incurred by
carelessness on the part of SPMs or criminal activity. It is also hard to see how these
could justify having been redacted originally.
So far as her evidence about Mr Latif was concerned, after Mrs Van Den Bogerd had
been taken through the features of his experience with the failed stock transfer and the
similarities with accepted bugs in Horizon, namely the printing out of two receipts very
close to one another (in his case, 4 seconds apart), and asked if this worried her, she
said she “would want to go back and have a look at” it. In my judgment, part of the way
through the Horizon Issues trial in 2019 is rather too late for that. I conclude that there
was no real answer available to Mrs Van Den Bogerd for the points put to her
conceming Mr Latif’s experience. Had there been, I have no doubt she would have
provided that answer in evidence.
Mrs Van Den Bogerd was in the witness box for in excess of one day, the longest period
of any of the witnesses of fact for either the claimants or the Post Office. Her cross
examination led to a far greater understanding of the Horizon Issues on the part of the
court, although her written evidence was, as originally drafted, extraordinarily one
sided. She minimised any reference to problems or issues with Horizon, and reverted
to potential user error whenever possible as a potential explanation, an approach which
she explained in her written statement as providing “plausible” explanations. Her
witness statement also stated, in terms, the exact opposite of what the reality of the
situation was, and I have given examples at [221] [223] and [226] above. Witness
statements are supposed to be factually accurate, and care must be taken in future
rounds of this group litigation that they are drafted in accordance with the rules. Making
statements that are the exact opposite of the facts is never helpful, to put it at its mildest.
However, during her cross-examination she readily conceded a number of points which
were put to her. Her accuracy was also more satisfactory than it had been during the
Common Issues trial. She accepted her written evidence suggesting a causative user
error by Mr Tank was not correct, for example. I do not consider that her written
evidence had provided plausible explanations. It provided explanations that the Post
251
252.
253.
POL00112816
POLO00112816
Office wished to advance, as these explanations would, if accepted, provide a defence
to the claimants’ factual case. These explanations were not based on the facts.
There were no evident attempts on this occasion to mislead me in her oral evidence, or
avoid uncomfortable points through a claim of ignorance (with the possible exception
of the PEAK referred to at [210] above) or of not having seen a document before. When
she told me that she had not seen a particular document before, I accept that. However,
she also told me that she had been assisted in preparing her evidence by a team of ten
people, and the Post Office had devoted “high level resources” to her evidence, as one
would expect. It is therefore very surprising that she had not seen many of the important
documents that Mr Green put to her in cross-examination. Some, such as the master
PEAK on phantom sales, one of the recurring issues on Horizon, would have been the
obvious place for anyone, still less a team of ten, to start when considering preparation
of evidence for a witness statement. She explained that there was some pressure of time
in terms of how long was available to her to prepare the statement, and this was further
explained in supplementary re-examination, but I do not accept insufficient time as a
valid explanation for her lack of knowledge on such important points. For example, she
told me that at the time of preparing her witness statement, she had not even heard of
the Callendar Square bug, one of two bugs that the Post Office accepted some time ago
had been present in the Horizon system. This is an extraordinary gap in her knowledge.
She did not know that there was a KEL dealing with failed recoveries, originally raised
by Anne Chambers in Fujitsu as long ago as 2010, which is called KEL acha959T and
which was updated most recently in 2017. This described failed recoveries, and seemed
on its face to accept that these would recur, and was very close to the experience of
both Mr Tank and also Mrs Burke. I do not see how Mrs Van Den Bogerd (assisted by
her team of ten, and with the benefit of the Post Office’s considerable resources) could
seek to give accurate evidence in the Horizon Issues trial without referring to this KEL,
still less without even knowing about it. I am also disappointed — putting it at its very
best for the Post Office — that a team of ten could have assisted Mrs Van Den Bogerd
in preparing a witness statement that was so inaccurate on such important points as I
have identified above.
Although my findings on her evidence in the Common Issues trial cannot be ignored, I
am of the view that her approach in the Horizon Issues trial to answering questions was
far more constructive and aligned to what is expected of any witness giving evidence
in court, particularly a senior witness of an organisation such as the Post Office. I do
however consider that this litigation, and indeed her cross-examination, is a very
expensive way for a senior director at the Post Office to become educated about the
myriad issues contained in the documents that were put to her. Either the team of ten
people assisting her with her evidence had the aim of producing entirely one-sided
evidence in chief, or they were unaware of all the documents relied upon by the
claimants. Either alternative is highly regrettable.
Finally, and this is a point in Mrs Van Den Bogerd’s favour, there is no doubt that the
Post Office is not as sufficiently close to the detail of what has occurred over the years
on Legacy Horizon and Horizon Online as Fujitsu. As will be seen from my analysis of
the Fujitsu evidence of fact, I have certain views about the lack of accuracy on the part
of Fujitsu witnesses in their evidence. If that lack of accuracy has also been included in
reporting to the Post Office by Fujitsu, then that goes someway to explaining the Post
Office’s lack of grasp of so much material that is consistent with the claimants’ case.
254.
POL00112816
POLO00112816
Now, the Post Office also has the added benefit of the views of both the IT experts in
the litigation, their four joint statements, the agreed number of bugs in the bug table
(12, if one takes the number accepted by Dr Worden), the total number of bugs Mr
Coyne says he has discovered (which so far as Horizon Issue I is concerned became
21), and collectively my findings on the Horizon Issues. These are not restricted solely
to the number of bugs which I have found to exist which relate to Horizon Issue 1, but
also those that relate to Horizon Issue 4. There are multiple bugs, errors and defects in
both Legacy Horizon and Horizon Online in its HNG-X form.
The degree to which the evidence of Mrs Van Den Bogerd sits with my conclusions on
the expert evidence will be dealt with in Part L, Overall Conclusions.
Ms Phillips
255.
256.
257.
258.
Ms Phillips is a Team Leader for Agent Accounting and Santander Banking. She was
originally engaged by the Post Office in 1999 through an employment agency, and has
worked in a number of roles since then. In her current role, she oversees the processes
of recovering losses from SPMs that they have declared in their branches. The
department used to have “debt” rather than “accounting” in the name, but its function
remained the same.
Ms Philips confirmed what had been in dispute for so long during the Common Issues
trial, namely that SPMs had no option but to accept the figures provided to them, even
though they may have been “settled centrally”. This is notwithstanding that her
terminology in her witness statement was that SPMs “chose to accept”. This
terminology was used throughout the witness statements for the Post Office, and it has
already been seen that Mrs Van Den Bogerd accepted TAs had to be accepted and there
was no choice in this respect, as Mr Latif had explained. SPMs clearly did not have a
choice, and the wording “chose to accept” is not an accurate description. It obviously
states or suggests that SPMs had a choice, when no such choice was available to them.
The only choice they had was whether to pay the amount immediately, or settle
centrally which means obtaining time to pay. It had been necessary for me to make
findings on what “settling centrally” meant in the Common Issues trial, and the Lead
Claimants in that trial had been directly challenged on this very point in their cross-
examination.
There is no reference in the letters sent to the SPMs in question relating to any ability
to dispute the sums which the Post Office treated as debts. Ms Phillips’ evidence went
to the way that the Post Office seek to collect sums from SPMs who have settled
centrally, even though they may have earlier raised a dispute (or sought to raise a
dispute) with the Helpline. Sometimes amounts above £5,000 would be checked by
reference to the Helpline call logs, but the amount was not a set threshold and it
depended upon how many sums had been settled centrally as to whether this happened.
This checking did not therefore always occur, even above that figure.
Her evidence demonstrated that there was no dispute function within Horizon.
Essentially, her task was debt collecting, as the Post Office saw amounts “settled
centrally” as debts. Her concern was to obtain payment, or agreements to pay, and if
not, then to escalate the matter to a SPM’s contract adviser. Usually, she said, the first
or even second letter would lead to payment by an SPM. Neither of those standard
259.
260.
261.
POL00112816
POLO00112816
letters referred to disputes. Mr Godeseth confirmed that the lack of any dispute button
or function on Horizon was, as he put it, “by design”. This shows that the Post Office
had been advancing a case, at least for a substantial part of the Common Issues trial,
which was directly contrary to the evidence of its own witnesses of fact in the later
Horizon Issues trial. I find this difficult to understand or explain. However, Mrs
Phillips’ role was fairly clear and she explained the process of branch dispute forms
which would be used when a SPM raised a dispute directly with her team. This
elementary system seems sensible, but it was only introduced in early 2018, and it
would lead to a seven day period after which (if the form was not returned) the “block”
placed on the amount would be lifted. She estimates that it would be used at least 5
times a month.
Ms Phillips gave useful evidence on the system used by her team, and I found her
credible. The evidence did however go more to the Post Office’s accounting functions,
and how it pursued sums it considered to be due from SPMs. The reforms introduced
in 2018 in terms of the branch dispute form are part of that process, but are not part of
the Horizon system.
She had also provided some evidence to Mr Smith (who was also called as a Post Office
witness) in terms of the number of Santander TCs that were issued, which had not been
described correctly in his witness statement and which had been misunderstood by both
the experts. This was clarified, and also she explained that of the number of TCs issued
in one year in her particular area, 2016-2017, which were 3,968, the Post Office could
not say how many were disputed because that data was not kept. This would appear to
bea fairly elementary piece of information for the Post Office to monitor or record. The
failure to do so is surprising.
Ms Phillips evidence in her witness statement did not directly impact upon the Horizon
Issues to any appreciable degree. Her evidence went to wider issues in the group
litigation. Her figures for Santander TCs, which had been given in evidence for some
reason by Mr Smith and not by her, were considered by the experts.
Mrs Mather
262.
263.
Mrs Tracy Mather is a Team Leader of the Finance Service Centre or FSC. She started
working for the Post Office in 1987 and has been in the FSC throughout, becoming a
team leader in 1999. She has managed teams in Cheques, Postal Order, Pay-out and
MoneyGram. Her witness statement was served in response to Mr Coyne’s 1‘ Expert
Report, and explained why the Post Office used Credence to investigate discrepancies.
She explained the difference between Credence, which records the transactional data,
and POLSAP, which is what is called a “back end” accounting system. She explained
the latter is on a higher level. Credence is used by her team to investigate differences
between what a branch says has happened to a transaction through Horizon, and as she
put it, what a different source of information might say. Credence was adopted in 2009.
She confirmed that Credence does not record key strokes, it records “transactional data
as in sales and non-sales”. Mrs Van Den Bogerd had stated that Credence records actual
key strokes by stating in her written evidence that Credence “records all key stroke
activity performed in that branch by the user ID, date and time.” This was not correct
and was corrected by Mrs Mather in chief. Her team does not have access to the ARQ
264
265.
266.
267.
POL00112816
POLO00112816
data, which has to be obtained from Fujitsu. Credence is not used for MoneyGram. I
accept her evidence on this point and find that Credence does not record all key stroke
activity.
Doubts expressed in the Ernst & Young Management Letter for the year ended 27
March 2011 about the integrity of data in Credence were put to her, but she was unaware
of any such doubts having been expressed and did not know about them. She had not
been in a team that had requested audit data, and said that would be the fraud or security
team who would do this. Passages from internal documents suggesting the audit data
came at a high cost to the Post Office were not something of which she seemed to be
aware. She started using Credence in 2016.
She gave written evidence about what was called the Rose Report, although she did not
actually know about it herself, and she also was unaware of anyone being deterred from
making ARQ requests due to the cost to the Post Office (charged by Fujitsu), which
was a point Mr Coyne had raised. Given her team did not make ARQ requests, her
evidence on this was unsurprising. She was aware that a certain amount of requests
were allowed, above that amount a charge was raised by Fujitsu; however she did not
know what the charge was.
There was one passage of her witness statement which went plainly beyond any direct
evidence she could usefully give. It was in paragraph 20 and gave vague evidence about
her understanding of some confirmation given by Fujitsu to the Post Office’s solicitors
about the contractual limit on ARQ requests, when or if that had been exceeded, a “new
commercial term” between Fujitsu and the Post Office about what was agreed and the
cost of these. Her conclusion was “any requests above the agreed limit are chargeable
but I understand that the terms depend on the details of the requirement.” She was not
cross examined about this, and given her answers in cross examination about her
understanding of ARQ requests (which was minimal) and her experience of having
made such requests (which was never), that is understandable. However, in terms of
providing guidance to the parties on the contents of witness statements, to assist the
efficient conduct of these proceedings going forwards, witnesses should not have
paragraphs of this nature in their witness statements about matters in which they have
had no involvement at all. Mrs Mather— and there is no criticism of her in this — has
simply never been involved either in making ARQ requests, or in the contractual
arrangements between Fujitsu and the Post Office, or the “new commercial term”
between those two entities in relation to the extra data required for the litigation. She
stated that she had “no idea” about the charges that would be raised.
I found Mrs Mather to be a credible and helpful witness. Her evidence was very useful
regarding Credence, and otherwise of less than central relevance to the Horizon Issues.
The degree to which the evidence of fact supports my conclusions on the expert
evidence will be dealt with in Part L Overall Conclusions.
Mr Smith
268.
Mr Paul Smith is the Operations Support Manager at the FSC and has been since
November 2016, having started at the Post Office in 1996. Originally he started in
pensions and then moved to human resources and payroll, until in 1999 he moved to
the Network Business Support Centre Helpline, or NBSC. In 2000 he moved to work
269.
270.
271.
272.
273.
POL00112816
POLO00112816
as a Problem Manager, dealing with problems that came through the Helpline. He led
the team that updated the knowledge base for the Helpline, and was Lottery Manager
in 2004. After performing different roles he became a Change Analyst in 2011, and
then the Santander manager. He returned to Lottery within the Post Office in 2013 and
worked there as a Team Leader. His existing role has a number of different activities
including handling specific product based problems. His evidence dealt with volume of
TCs and a suggestion made in Mr Coyne’s report that TCs for a branch in Potters Bar
were initially made in error.
One passage in his statement is of particular importance in resolving the Horizon Issues.
It states:
“TCs are issued by FSC. I understand from Post Office’s solicitors that the processes
by which FSC determines whether a TC is required is outside the scope of the Horizon
Issues trial.”
The reason that this is important is as follows. Dr Worden relies upon TCs as one of his
“countermeasures” that goes to the robustness of the Horizon system. Mr Coyne does
not, and the claimants submit that the whole process of TCs is entirely outwith the
Horizon system. This evidence from one of the Post Office’s own witnesses makes it
clear that the claimants’ point of view is shared (or at least was when the statement was
prepared) and the decisions regarding whether TCs are issued are outside the scope of
the Horizon Issues. This evidence by Mr Smith supports the claimants’ position on this,
and also supports the approach of the expert evidence of Mr Coyne, and is contrary to
the position adopted by Dr Worden.
A case management system was introduced by the Post Office in September 2018 to
record each individual challenge to a TC. This is called the Dynamics System, and is
still in what is called “roll out”. Individual challenges to TCs were not recorded prior
to that. Further information about how this system was working was obtained in cross-
examination, and leads me to the conclusion that the way that TCs are recorded now is
far improved from what it was before the Dynamics System was introduced. That does
not however take one very far. Mr Smith knew of reports that were available from that
system but he had not produced any. Those reports would be the best evidence of the
type of points Mr Smith was seeking to make for the most recent periods at least, but
given he had not used them, it is not possible to speculate on what they might show.
Mr Smith had obtained almost all the data contained in his statement from others, and
some was plainly incorrect, and other data so vague as to be wholly unhelpful. For
example, for Santander, originally he had provided a table with a column stating that
2,890 disputes were received from branches in the 2016/2017 financial year. In fact,
that column should have been disputes received by Santander from the Post Office.
This was corrected in his second witness statement but Mr Smith was unnecessarily
combative, in my judgment, about the need for the correction, maintaining that “the
facts and figures were correct” in the first statement when they plainly were not. The
figures were correct as figures but the descriptions of what the figures were, were
plainly wrong.
Even after the correction, the various figures extracted by him from information
provided by others was not particularly useful. For example, the figures for the numbers
274.
275.
POL00112816
POLO00112816
of TCs disputed, and compensating TCs, for BOI Retracts and the Lottery, were both
estimated at 1,500 and 500, for both line entries. This estimate was plainly of the most
vague type, and even Mr Smith could not comment on the reliability of either of those
estimated figures for either line entry, although he maintained the figure for issued TCs
would be reliable. This evidence from him, which was at a very high level, was not of
great or indeed any assistance. For example, in respect of MoneyGram his evidence
was that one or two disputes a month were received but they were rarely if ever
accepted. Given he had no idea at all of the overall total, this evidence does not assist
me in resolving the Horizon Issues one way or the other.
His explanation for the TCs at Potters Bar was that a cheque that was deposited for
£90,000 was mis-keyed, and entered for £900,000. Accordingly TCs were required to
correct this at the branch, and the need for two was because the payment was made by
cheque.
I found Mr Smith’s evidence, with three exceptions, to be of no assistance. The two
exceptions are his evidence I have summarised in [269] which is relevant to the scope
of the Horizon Issues and Dr Worden’s countermeasures, and his explanation about
Potters Bar and the two TCs. I accept his explanation about the Potters Bar TCs. Both
of those exceptions are relevant to the Horizon Issues. The third exception is his
evidence about the Dynamics System. This goes to the robustness of Horizon as of 2019
and is therefore relevant. However, the absence of such a system also goes to robustness
prior to its introduction in 2018, and is also relevant. However, his wider evidence is of
limited relevance at best. His knowledge of the figures contained in his statement, and
the vagueness of his evidence generally, were such that the utility of the exercise which
he presented in his statement is doubtful. In any event, the substance of that evidence
went to an attempt to demonstrate that some TCs are challenged by branches, and fewer
challenges are upheld than are made. The figures vary inevitably from year to year, and
this case is not about general statistical trends in any event. In some areas of business
TCs have fallen to a very low level, for example the DVLA, which Mr Smith explained
was only 4 in the last year, none of which have been upheld. However, the records that
were put to him show that there were 2,717 in the period 2010/2011. All this evidence
does is show that far fewer TCs are now issued by the Post Office in relation to the
DVLA business, which as Mr Smith explained, has fallen considerably in any event.
None of these points are of particular relevance to resolving the Horizon Issues.
Mr Johnson
276.
Mr David Johnson is a Training and Audit Advisor at the Post Office. He started his
career at the Post Office in 1984 as a counter clerk in a Crown Branch, and worked his
way up until he was the Branch Manager at the Barry Crown Office. He was in that
post when Horizon Online was introduced in 2010. He moved to his current role in
2012, and his role used to be called Field Support Adviser. Two thirds of his time is
spent training SPMs; the other third is spent auditing them. He had also been involved
in training some of the legal representatives involved in the group litigation, and the
experts, on the Horizon system. As well as his first witness statement he had provided
a supplementary one dealing with some points in Mr Henderson’s witness statement
which was served for the claimants.
277.
278.
279.
280.
281.
POL00112816
POLO00112816
He was cross-examined about screen layout and design, the risk of mis-keying (which
did not seem to concern him, although there were internal Post Office documents
showing it was a known problem) and also the reports and functions available to SPMs
to investigate shortfalls and discrepancies, rather than those which the Post Office itself
used such as Credence. He agreed that the ways available to SPMs to investigate in
branch were “not the most user-friendly way of investigating”.
He agreed with a statement in a 2016 internal Post Office document entitled “Network
Development Programme Operation Simplification” that stated:
"There are a number of branch operations processes, especially around branch
accounting and reconciliation, which operate using legacy processes. They are
unnecessarily complex and detract Post Office branch resources from serving
customers. Stock Unit Management and accountability is very poorly controlled and is
operated on very complex business rules. The lack of accountability and visibility of
cash and stock transfers between Stock Units can lead to errors, rework and provides
opportunities for fraud.”
(emphasis added)
He also stated that the following entry was “not completely unreasonable” although he
said that his experience of suspense accounts in branches was that they worked and did
what they were supposed to do:
"Similarly, Suspense Accounting is based upon legacy cash accounting practices, ill-
defined and out of date processes. Inefficiencies lead to poor utilisation of resources
both in Post Office branches and Support Services."
Mr Johnson is a highly experienced person, as one would expect given that the Post
Office chose him to train the legal teams and experts on Horizon. All his experience,
so far as using Horizon is concerned, is in Crown Offices, which are not the same as
branch Post Offices in that the Post Office operates Crown Offices itself. This means
that a SPM in a branch Post Office is personally responsible for any losses in that
branch; whereas a Crown Office manager is an employee of the Post Office. This
therefore gives him a somewhat different angle of approach to the use of Horizon, but
I accept him as an accurate and helpful witness.
His evidence was very useful in terms of how Horizon works. The Horizon Issues are
not about the actual design of the Horizon terminals in branch, and whether (say) this
could be improved by having larger buttons or the screen layout adapted differently. In
some of Mr Johnson’s evidence, the claimants’ counsel strayed into the areas of
improvements to functionality in branch and design. These are not part of the Horizon
Issues. The degree to which the evidence of fact affects my conclusions on the expert
evidence will be dealt with in Part L Overall Conclusions.
Mr Dunks
282.
Mr Andy Dunks is at IT Security Analyst at Fujitsu Services Ltd, and was the first
Fujitsu witness to be called for the Post Office. He has been employed at Fujitsu on the
Post Office account since 2002. He has working knowledge of the Horizon System. His
283.
284.
285.
286.
POL00112816
POLO00112816
evidence went to audit data extraction and the integrity of the data during that process.
His evidence related to the process as it was when he made his statement, although
given he had been involved for some 17 years his experience also related to Legacy
Horizon and his evidence about data extraction covered that period too. As emerged in
his cross-examination, he had also given a witness statement in the criminal trial of Mrs
Seema Nisra.
He agreed with the definition of data integrity put to him, which is that data integrity is
the overall completeness, accuracy and consistency of that data which you can measure
by comparing between sources. He said the 12 controls that he had listed in his
statement explaining these he had done from his recollection, and that he had not
worked off a document to come up with this list.
Mr Dunks gave every indication, in the first part of his evidence, of being helpful and
frank, and explained the process of data extraction and answered questions put to him
openly. However, this encouraging start came to an end when he was asked about two
curiously worded paragraphs in his statement. These stated as follows:
“8. There is no reason to believe that the information in this statement is inaccurate
because of the improper use of the system. To the best of my knowledge and belief at
all material times the system was operating properly, or if not, any respect in which it
was not operating properly, or was out of operation was not such as to effect the
information held within it.
9. Any records to which I refer in my statement form part of the records relating to the
business of Fujitsu Services Limited. These were compiled during the ordinary course
of business from information supplied by persons who have, or may reasonably be
supposed to have, personal knowledge of the matter dealt with in the information
supplied, but are unlikely to have any recollection of the information or cannot be
traced. As part of my duties, I have access to these records.”
Before I come to his evidence about this, it is obvious that the wording of paragraph 8
is almost that of a legal disclaimer (or a legally worded claim of accuracy, to be more
precise), rather than a witness’ actual evidence. It would be very curious for a witness
of fact to decide to put such a formally (and rather clumsily) worded paragraph in their
witness statement. However, he was asked about these passages. He was asked what he
meant by “the system”. The transcript records the following:
“Q: What is the "system" there? Is that the system of the process of extracting audit
data, or is it something else?
(Pause).
A. Good question. There's no -- I'm not sure what I was meaning by that, "There is no
reason to believe ..."”
He was asked if, in relation to the part which states “the system was working properly
or, if not, any respect in which it was not operating properly, or was out of operation”
the following:
287.
288.
289.
290.
291.
POL00112816
POLO00112816
“Q: Again, pausing there, it is slightly confusing. Are you aware or not aware of any
instances where that system was not operating properly?
A. No, not really, no.”
He was asked if that part of the statement had been written for him by someone else,
and he said he could not remember. He was also asked if it was a “Fujitsu party line”
to put this in statements when it comes to extracting data and he said no, and that he
was not aware of a Fujitsu party line. He accepted that gaps and duplicates in audit data
were problematic, and he explained that they had to be investigated by audit support. It
was clear to me that Mr Dunks considered audit data that had gaps and duplicates should
not, in that form, be used. He said he would contact audit support if these occurred.
However, some of Mr Dunks’ evidence, in particular what he had to say about
paragraph 8 of his own statement, proved in my judgment to be somewhat misleading.
He was shown a witness statement from Mr Gareth Jenkins from the Misra criminal
trial in 2010. He said he had not seen this statement before. The statement put to him is
some 8 years older, approximately, than his own statement. That stated at the end:
“There is no reason to believe that the information in this statement is inaccurate
because of the improper use of the computer. To the best of my knowledge and belief
at all material times the computer was operating properly, or if not, any respect in which
it was not operating properly, or was out of operation was not such as to effect the
information held on it. I hold a responsible position in relation to the working of the
computer.”
The similarities between this passage, and the one in paragraph 8 of his witness
statement almost 9 years later, are somewhat striking. Given he had not seen the
statement before, there was no obvious explanation. His answer was as follows:
“Q. And you're not sure why it would largely replicate your paragraph 8?
A. No, I mean, we do have a standard witness statement that we produce for ARQs.
When we supply ARQs we are sometimes asked for a witness statement to go through
the process and verify as far as I'm aware that the data I supplied is accurate. Now, we
use that quite a lot and it may actually be in that statement.
Q. Isee. So when I asked you earlier whether it was something of a Fujitsu party line
you said you didn't think it was, but actually it looks as though it is on the basis of what
you have just told me and on this document as well
A. It could be. It may be part of our standard witness statement that we supply.”
Mr Jenkins had given evidence in his witness statement from 2010 about duplicate
entries in the audit data that had had to be removed. Mr Dunks was then taken to the
witness statement of another Fujitsu witness in the Misra trial, called Penelope Thomas.
She had given evidence about data extraction and listed a number of controls, which
were remarkably similar to the ones that Mr Dunks gave evidence about (although Ms
292
293.
294.
POL00112816
POLO00112816
Thomas had omitted one). She also had the identical passage in her witness statement
as Mr Dunks’ paragraph 8.
Mr Dunks also said in respect of this, which in my judgment painted a rather different
picture to that portrayed in his witness statement, the following:
“A. The statement again is to verify the integrity of the data once extracted and given.
We don't control what's in the data. Our process is about extracting it and securely
passing it over to the Post Office. It's not our concern of what's in the data.
Q. It's not your concern what's in the data?
A. No, it's what we're -- we're process-driven to extract certain types of data for certain
requests.
Q. And so whether or not that might match another record or not, or replicate or
duplicate or have gaps, that's not part of your remit, that's not really part of your
concer?
A. No, it's not.
Q. I see. So reading that and reading the paragraph 8 as it was in your witness
statement, it doesn't give much comfort to somebody that's then trying to rely on ARQ
data as being a gold standard to compare and investigate anomalies, does it?
A. Possibly not.”
The reference to “gold standard” was the expression that had been used by Dr Worden
in his report to describe the quality of the audit data that was held by Fujitsu. It should
be a very great concern to anyone tasked with extracting audit data, should that data
prove to have gaps and duplicates in it. This is because gaps and duplicates in the data
affects the accuracy of that data. I consider that such a cavalier approach to whether
audit data has gaps and duplicates, as evidenced by Mr Dunks’ saying that such things
were not part of his concern, to be contradictory to a statement verifying the accuracy
of such audit data.
My findings in relation to this are as follows:
1. Mr Dunks expressly sought to mislead me by stating that there was no “Fujitsu party
line” when it came to the contents of drafting witness statements about audit records
for legal proceedings. There plainly is; it was used in the statements in 2010 and it was
used by him in his statement for the Horizon Issues trial.
2. The passage included in paragraph 8 of his statement is plainly a standard form of
words, and reads as though it is a legally drafted assertion of accuracy. The witness’
assertion that the system was at all material times working properly, or if it wasn’t, this
did not affect accuracy, does not sit consistently with Mr Dunk’s acceptance of gaps
and duplicates in audit data.
295.
296.
POL00112816
POLO00112816
3. That Mr Dunks did not really know very much about paragraph 8 of his own witness
statement was confirmed by the way he was puzzled when he was asked about it.
4. However, Mr Dunks’ acceptance of standard wording provided by others (probably
within Fujitsu) is less important than the fact that he originally tried to mislead the court.
5.1 do not accept that he identified the 12 controls in his statement from his recollection.
He must have been working off another document, which he was not prepared to
identify. I draw this conclusion because of the marked similarity with the controls
included in Mrs Thomas’ statement from 2010; the wording of the controls themselves;
and the fact he was prepared to mislead me about the paragraph 8 point.
6. In re-examination he asserted that the statements in his paragraph 8 were true, and
that he would not have signed his statement had this not been the case. I find that he
cannot have known whether they were true or not. There is no evidence of any steps he
took to check whether or not they were true or not. I find they were simply standard
sentences supplied to him and he signed his statement without any independent
knowledge of whether they were true or not.
7.1 find that passage in his witness statement simply to be meaningless assertion.
He said that he was unaware of Post Office internal concerns that there was “a lack of
system audit trail” in a document of 22 October 2016. I find his lack of knowledge
about this surprising, given his role.
I found Mr Dunks very unsatisfactory as a witness. He was both plainly aware of the
Fujitsu “party line”, or corporate position, regarding the words asserting accuracy of
audit data, and he was very anxious to keep to it, whilst initially denying that there was
one. The degree to which the evidence of fact affects my conclusions on the expert
evidence will be dealt with in Part L Overall Conclusions.
Mr Godeseth
297.
298.
Mr Torstein Godeseth is employed by Fujitsu and is the Chief Architect on the Post
Office account. He had made three witness statements, as had Mr Parker. After
graduating from Oxford with a degree in physics in 1974, he worked for Rolls Royce
as a combustion engineer and then joined the Royal Navy in 1977 as an Instructor
Officer. Whilst in the Royal Navy, he started his career in IT working in systems
programming. It will be remembered that in the 1970s computing and IT was a very
young field, and Mr Godeseth has therefore been involved in that industry since its
relative infancy. He left the Royal Navy in 1981 and worked in systems programming
and technical support for the IT at Forward Trust Ltd. He joined the Post Office IT
department in November 1987. Initially this was to work on a project to introduce
technology into Post Office branches.
He was working with the Post Office as a technical advisor when the Post Office and
Benefits Agency procured the Horizon system. He remained involved in different roles,
including being outsourced from the Royal Mail IT department to Xansa in 2003, and
contracted to the Post Office to act as the technical adviser interfacing with Fujitsu
amongst others. Although an independent contractor, he worked between 2005 and.
2010 with the Post Office as the technical advisor on IT projects including the change
299.
300.
301.
302
303.
304.
POL00112816
POLO00112816
from Legacy Horizon to Horizon Online. He joined Fujitsu as a full employee in 2010.
He described himself in cross-examination as the designer of Horizon.
He had a greater knowledge of Horizon Online than of Legacy Horizon, although given
his involvement in Horizon from its early days he had a considerable degree of
knowledge about both. Horizon was initially, in the late 1990s, called the Pathway
project and was a joint effort between the Post Office and the Benefits Agency,
including a way to computerise the payment of benefits to those entitled to them,
through payments to benefits claimants which were to be made through the Post Office
by means of a swipe card method. This project ran from approximately 1996 to 1999
and was to be provided by ICL (which Fujitsu partly owned and later fully acquired).
At some point during the development of this project the Benefits Agency withdrew,
and what had been the Pathway project, essentially a tri-partite venture (the parties
being ICL, the Post Office and the Benefits Agency) became Horizon, now called
Legacy Horizon.
Although this litigation is not about the way that the Horizon system was designed, it
is about its functionality and robustness in use, Mr Godeseth said that it was a specific
Post Office decision not to have any dispute button/function for SPMs built into the
Horizon system.
He said:
“T think the basic argument was that disputes -- we wanted the flow of data through the
system as quickly as possible because that keeps our books tidy and it was an inference
that there was always the -- you had to press a button to take things through, but then
you would pick up the phone to NBSC and say that wasn't right.”
(emphasis added)
I accept this factual evidence. It is, however, directly contrary to a major part of the
Post Office’s case in the Common Issues trial, namely that by reason of an SPM
accepting items at the end of a branch trading period, that gave the branch trading
statement the effect in law of being what is called a “settled account”. I rejected that
argument by the Post Office then, unaware when I did so of later evidence of fact that
would emerge of the specific decision taken at the time by the Post Office that expressly
did not include any feature within the Horizon system for a SPM to dispute items with
which they disagreed. My finding in the Common Issues trial is consistent with Mr
Godeseth’s evidence on this point.
I deal with Mr Godeseth’s evidence within this judgment in more detail than any of the
other witnesses of fact from either the claimants or the Post Office. This is because his
evidence was considerably more detailed, and of more direct assistance in resolving the
Horizon Issues, than any other witness of fact. He has been involved in Horizon in both
its iterations for 20 years and has a vast amount of knowledge of its operation. I found
the majority of his evidence reliable. I provide my conclusions regarding him as a
witness at the end of this section below at [453] to [463].
In Legacy Horizon, a messaging system was responsible for storing all data in Post
Office branches and replicating it to data centres. This was called Riposte. Mr Godeseth
decribed himself as being “on the other side of the fence” during this period and he had
306.
307.
308.
309.
POL00112816
POLO00112816
consulted Mr Jenkins for the section of his evidence dealing with when Riposte was in
use. However, Mr Godeseth did describe himself as “having a pretty good knowledge
of Riposte”.
When it was brought in, Horizon Online was aimed, not at improving functionality of
Horizon, but at reducing cost, and re-used (the term put to Mr Godeseth was “recycled”)
many application components of Legacy Horizon. The data was no longer to be held at
the branches (on the counters), it was to be held centrally in the branch database, or
BDRB. Accordingly, harvesters were required to extract transactions from the BDRB.
The branch database receives information from different sources. Data in Horizon
Online is obtained from different sources.
As an example with the Lottery, if a person buys some lottery tickets in their branch
Post Office, the information about a particular transaction starts at the lottery terminal
in a branch; goes directly from there to Camelot; from Camelot it goes to Credence;
and then from Credence it goes to the branch database, which is effectively that
information arriving in Horizon. It is then transmitted to the terminals (in the branch)
and the SPM would see a series of TAs on the terminal in the branch, which that SPM
would have to accept; these TAs would relate to the lottery tickets sold at the beginning
of this short history. The transaction of the purchase of the lottery tickets by this route
then enters the branch accounts, and the data is captured by the audit system. This is
how lottery transactions work after what was called the Ping fix. Prior to that it was
somewhat less streamlined, although given the process I have described is the post-Ping
fix streamlined version, it is obvious that pre-Ping fix, the route for the data was even
more convoluted.
Other data for other transactions for other products goes into the branch accounts from
what is called the basket. That basket is compiled by, as an example, a customer at the
counter purchasing various Post Office and other client products; the SPM or assistant
serving that customer and pressing an icon on the screen (for example for stamps, and
other purchases); or for other products, from a PIN pad, scanning a barcode or weigh
scales. The cost of a book of stamps is taken from the reference data table and the
terminal shows the SPM or assistant how much money the total number of transactions
are worth and what the cost to the customer is to be. It is basically the shopping basket
which is put together by the SPM (or assistant) serving the customer and adding, by
means of pressing buttons on the screen, different items to the basket as the customer
goes through the items or products they wish to buy. At the end of this hypothetical
transaction, if the cost of the basket is (say) £21.98, the customer will pay that amount
(say by cash) which will be taken by the SPM and put into the cash amount held in the
branch.
In Legacy Horizon, the data was held in the branch on the counters in a message store.
The message system called Riposte was how the replicating of the data to data centres
occurred. The data was then stored on the correspondence server message store. This
was explained by Mr Godeseth in the following way.
All counter data was held in a bespoke message store (which was part of the Riposte
product supplied by another company called Escher Inc.). This data was replicated
within each branch to all counter positions and from each branch to the data centres
where it was held in the correspondence server message stores. Similarly, any data
310.
311.
312.
313.
314.
POL00112816
POLO00112816
inserted into the message store at the data centre (for example reference data or
authorisations for banking transactions) would be replicated back to the branch
counters. Selected data was then extracted from the correspondence servers to update
Post Office's back end systems.
Users with sufficient access permissions could inject additional messages (i.e. data) at
the correspondence server. Any additional messages injected at the correspondence
server by users with sufficient access permissions included information including the
identity of the user. That information would not be visible in the standard audit extracts,
but it would be visible in a detailed examination of the raw audit data. An SPM did not
have access to audit extracts.
There was some disagreement between Mr Godeseth and internal Post Office
documents about whether the audit data from Riposte was held in Riposte attributable
language (his view) or whether an audit conversion tool was required to convert existing
audit data from Riposte to another readable/searchable format. This ultimately may not
matter, and it may be that the author of the internal document suggesting otherwise did
not understand that Riposte attributable language was a readable/searchable format.
This difference in view does not matter for the purpose of resolving the Horizon Issues.
The Riposte software was provided by another entity called Escher. There was no
evidence about the contractual relationship between Fujitsu and Escher, and that does
not matter for the purpose of resolving the Horizon Issues.
Mr Godeseth had studied a number of problems in Horizon, and knew there were
problems with Riposte, having seen numerous PEAKs referring to this. He had also
been responsible for some testing (for example potential failures of controls, such as a
basket not balancing to zero). He was not aware of items with the same Journal
Sequence Number or JSN having been committed to the BDRB, and agreed that this
should not be possible. This is because the whole way the BDRB works (or one of the
principles of it, at least) is that each item committed there should be given a unique
JSN. It should not be possible for two different items to be given the same JSN.
Mr Godeseth explained that “the legacy version of Horizon was far more susceptible to
communication glitches” and agreed that there were two improvements made when
Horizon Online was introduced in that respect. These were that the susceptibility to
communication glitches was reduced; and also the quality of the communications
infrastructure was improved. It should also be added that because the storage of data
was moved to somewhere not in branch (somewhere which had not existed in Legacy
Horizon, namely the BDRB), and because Horizon Online was an online system, the
communications infrastructure had to change substantially in any event.
One internal document from as long ago as 2008 stated: “In addition, in common with
many elderly systems that have been subjected to a succession of major changes, it [ie
Legacy Horizon] has become increasingly difficult to make those changes, and
expensive to operate." This was put to Mr Godeseth:
“Q: Now, that's a fair description of how the system originally was perhaps designed
jointly with the DSS at the beginning and launched and then over the years, between
then and 2008, there have been lots of sort of bolt-ons and additional things that have
been changed on the system, haven't there?
315.
316.
317.
POL00112816
POLO00112816
A. Yes, I think that's fair. The major one probably in my mind would be banking.”
The reference to the DSS meant the Benefits Agency. In other places this was referred
to as DWP (or Department of Work and Pensions) but all three references — DSS,
Benefits Agency, DWP — meant the department involved in paying welfare benefits.
Different documents and references would use different acronyms to refer to the same
entity.
He explained that a small group of Fujitsu users from the SSC (about 30 users) had the
ability to inject additional transactions into a branch's accounts in Horizon Online, using
a designed piece of functionality called a Balancing Transaction. This was
approximately the whole number of personnel who worked in SSC.
He had given information in his first witness statement, namely that in Legacy Horizon,
any transactions injected by SSC would have used the computer server address as the
counter position which would be a number greater than 32, so it would be clear that a
transaction had been injected in this way by someone other than the SPM. This is
important because it would be consistent with the case advanced by the Post Office that
any such injections would be entirely visible as having been done externally (ie, not
within the branch) due to the counter number used.
However, this important information was simply incorrect, and was corrected both by
Mr Godeseth and Mr Parker in subsequent statements before they were called, and as a
direct result of Mr Roll’s evidence. The information that was incorrect, and therefore
had to be corrected, had come directly from Mr Jenkins. This shows that Mr Jenkins
did, in at least one very important respect, give Mr Godeseth directly incorrect
information about the visibility of injected transactions, which not only could have an
effect on branch accounts, but would have shown (or rather, not shown) that the impact
on those accounts had come from injections made outside the branch. Mr Godeseth
only found out the true position when Mr Parker was preparing his subsequent witness
statement in the weeks prior to the commencement of the Horizon Issues trial, in other
words in 2019. He had not known that before. His explanation about this was as follows.
“Q. You were finding out a detail that you didn't know before in quite a controversial
area, weren't you?
A. It was clearly an area that was going to be of interest because of the fact that we
were inserting transactions into Riposte. It was an operational necessity and it was done
in a controlled way. I had believed that the way that transactions were being injected
would give them a counter position greater than 32 because the correspondence servers
basically had nodes or addresses which were above 32, there was a special address for
the gateway server, there was a special address for the extra disc in a single position
branch and I had basically expected messages to be introduced using a different counter
position and having read a whole number of PEAKs, I can quite clearly see that the
standard practice in Fujitsu was to label something which was being inserted into
Riposte so as to make it as clear as possible that it was not being done -- it was being
done as something out of the ordinary, it was being inserted because of a problem.
So we had techniques for doing that. You could put in an attribute because this wouldn't
be visible to a subpostmaster, I fully understand that, but it would be visible in the audit
318.
319.
320.
POL00112816
POLO00112816
trail, when you ever come back to pull out the audit trail, you could put in an attribute
to say "This was done under PEAK 75". Subpostmasters would never see that. They
would not see it in their account in their branches and I'm fully aware that that is the
case. It was a better audit than Mr Roll was alluding to when he said that it was left in
a PINICL, because that would have been an audit which is separate from the actual data
that we would be looking at should we ever need to pull stuff out of the audit trail and
the intention was always to make it as clear as possible that this had been done under
exceptional circumstances.
The techniques used to make it as visible to the subpostmaster as possible would be to
put in references which referred to a counter that didn't exist in the branch, such as -- I
saw a technique described in a number of cases which said put in a -- you know, if you
are correcting something for counter 1, call it counter I 1; if you're correcting something
for counter 2, call it counter 12. These things would have been visible to a
subpostmaster and the reason that you had to do it that way was to make sure that these
transactions also got picked up and dealt with because these were legitimate counter
numbers.
If I start to put in data with a number which is not a legitimate counter number then it's
going to be ignored by systems further down the track.
Q. So which were legitimate counter numbers?
A. Up to 32.”
(emphasis added)
He also said in relation to Riposte that “Riposte was responsible for actually wrapping
the message that we were looking to insert at the counter, and in doing that, Riposte
will tell you the counter ID, or technically it was a stream, it would pick up the user ID,
it would pick up the time, so this was effectively the envelope which wrapped the
payload that we were looking to inject. If there was no user logged on at the counter
then Riposte would introduce a blank user ID and that would be picked up in later
processing.” This therefore meant that if, say, a SPM (or an assistant) was logged on at
the time that the message was inserted at the counter, it would appear next to that SPM’s
user ID (or that of the assistant). It would look as though the SPM had been responsible.
This was made crystal clear in different passages of cross examination:
“Q: You realised for the first time that it was possible to inject or insert a transaction
with a counter position less than 32 when Mr Parker was preparing his second
statement?
A. Correct.
Q. And you knew that that was a contentious issue in this litigation, yes?
A. Yes.”
And further:
“Q. Now, it would be possible, would it not, to use a counter number of 1, or 2, or 3?
321.
322.
323
POL00112816
POLO00112816
A. It would.
Q. And if that counter number was a counter number actually in use by the SPM, it
would appear to the SPM, from the records they could see, that it was a transaction
which had been done in their branch, by them or their assistants?
A. Yes.”
(emphasis added)
I consider this to be extremely important evidence, both in resolving the Horizon Issues,
and indeed in the whole group litigation. Its import is obvious. It means that Fujitsu
could remotely insert a transaction into the accounts of a branch using a counter number
which was the same as a counter number actually in use by the SPM (or an assistant).
This would appear to the SPM from the records that they could see as though the
inserted transaction had been performed in the branch itself. This information was only
disclosed by Fujitsu (and therefore the Post Office) in January and February 2019. Even
Mr Godeseth, a very senior person in Fujitsu so far as Horizon is concerned, said that
he did not know this before.
Mr Godeseth was also shown a Fujitsu document which was "Fujitsu restricted" and
"Copyright Fujitsu Limited 2017". The title of the document was "Post Office Account
— Customer service problem management procedure" and its purpose is described as
"To describe and document the customer service problem management process." He
was not sure he had seen the document before, and he was certainly not familiar with
its contents. Both experts had seen the document and proceeded on the basis that what
it recorded had in fact been adopted. “The customer” for the purpose of the document
was the Post Office.
He sensibly accepted that to have a robust system it was important to make informed
assessments of where problems lie, based on the relevant information that was
available, and also that it was important to capture and track that information in a way
that could be readily analysed. Indeed, Mr Godeseth’s role included awareness that
changes being made to the system were implemented without prejudicing the continued
operation of the system. Indeed, the document defined “problem” as the unknown
underlying root cause of one or more incidents. That, to me, seems a sensible definition.
An unknown root cause of even one incident is a problem in a system such as this one.
This is accepted by the document’s definition of “problem”.
The document identified the metrics that were to be used within Fujitsu to measure this.
Mr Godeseth agreed that they were professional and sensible metrics. They are stated
as follows:
“The following metrics, to be reported monthly, will be used to measure effectiveness
of the process and drive performance of the process and overall service in general:
Number and impacts of incidents occurring before root problem is identified and
resolved
325.
POL00112816
POLO00112816
Number of repeat incidents following corrective action
© Number of problem records arising from pro-active actions and trend analysis
Number of changes arising from proactive actions
“ Percentage of problem records without an action plan
“I Average length of time to resolve problems
Number of incidents closed without a KEL
“ The number of Problem Records arising from Managed Changed (MSC, CP)
activities
The number of Problem Records arising from the implementation of new
services/major releases.”
However, when the claimants had sought to obtain reports that would be expected to
exist, which included or demonstrated the reporting against these bullet points which
Fujitsu’s own document stated were required, and which were supposed to be prepared
every month, Fujitsu stated (through the Post Office’s solicitors) that “Fujitsu believes
that it does not record problems in such a way that would allow this to be determined
without retrospectively carrying out detailed analyses” and that it would require “a
disproportionate effort and cost” to provide these. Mr Godeseth also gave written
evidence that the reporting system identified in Fujitsu’s own document had not been
implemented and that the records did not exist. The passage in his statement said:
"[ have spoken to my colleague Steve Bansal, Fujitsu's senior service delivery manager,
who has informed me that the Post Office account customer service problem
management procedure document was introduced by Saheed Salawu, Fujitsu's former
Horizon lead service delivery manager and that Saheed Salawu left the Fujitsu Post
Office account in around February 2013, before the new procedure had been
implemented. J understand from Steve that Saheed Salawu's replacement did not wish
to_implement the changes and therefore the records referred to by Mr Coyne in
paragraphs 5.157 to 5.159 of his report do not exist, as we continued to follow the
previous existing reporting methodology.”
(emphasis added)
Mr Godeseth did not know who Saheed Salawu’s replacement was, although he thought
it might be a Mr Wicks. He did not know what the “previous existing reporting
methodology” to which he referred in his own witness statement in fact was. He could
not explain why the report which was said to have been abandoned, or at least not
pursued after February 2013 when Saheed Salawu had left Fujitsu, was issued for
approval in July 2014, over one year later. Indeed, he could not really explain why the
records sought did not exist at all. He thought the reporting might be done at meetings
with ATOS, but he could not remember going to one. In fact, his evidence on this was
so very vague as to be somewhat alarming, given his role within Fujitsu. He did not
327
328.
329.
330.
331.
POL00112816
POLO00112816
know if there were any records of the particular type apart from between the years 2014
to 2017, and he only knew that because Mr Bansal had told him.
In another document, he was shown an entry which recorded that First Rate, a joint
venture between the Post Office and the Bank of Ireland dealing with foreign exchange
(also called bureau de change or bureau services), had identified an anomaly over the
way that Horizon reversed transactions were recorded and polled through to them. This
was fixed in something called counter release 9, but although he said this happened “on.
his watch” this had not been brought to his attention.
In the 2015 Problem Management Problem Review, the Dallmellington bug (which is
an acknowledged and agreed software bug) was referred to, but without identifying how
many branches were affected, for how long they were affected, or the amounts by which
their branch accounts were impacted. He accepted that it was fair to describe the report
as “not a particularly rigorous or robust treatment of recording the problem, its extent
and duration and effect.”
The following conclusions can be drawn from this evidence by Mr Godeseth. Fujitsu
should, had it been interested in providing a robust system to the Post Office, been
collating and reporting problems, which should obviously have included software bugs,
errors and defects, against the metrics contained in the Customer service problem
management procedure document, or very similar ones. On the Fujitsu evidence before
the court, it did not do so. The discrepancies in the dates of distribution and issue of the
document entitled "Post Office Account — Customer service problem management
procedure” and its "Copyright Fujitsu Limited 2017" do not tally with the account in
Mr Godeseth’s witness statement that this process was abandoned or not pursued when
Mr Salawu left Fujitsu in 2013. The dates just do not match up. The document was
issued for approval one year after that gentleman left Fujitsu. Even the evidence that it
was not pursued or implemented after Mr Salawu left was second hand from Mr Bansal,
the source of Mr Godeseth’s knowledge.
Had Fujitsu done what its own document copyrighted in 2017 states was required, the
experts in this litigation could, and probably would, have used those reports. Those
reports would have included comprehensive records of precisely the sort of matters both
experts had to investigate in order to provide their evidence to the court on the Horizon
Issues. It is, in my judgment, a deficiency in the robustness both of Legacy Horizon and
Horizon Online that such records were not kept by Fujitsu. This deficiency must
inevitably also affect the quality of information that the Post Office itself would have
had.
In the Post Office’s closing submissions, an explanation was given in submission in
relation to the explanation given by Mr Godeseth in his witness statement which I have
set out in [325] above about why the records did not exist, that explanation being that
the procedure was not implemented. The Post Office submitted that it was only section
1.4 of the document that was not implemented, and not the whole document. This was
in paragraph 147 of its written closing submissions, which dealt with what the Post
Office submitted were “errors” made by Mr Godeseth which it submitted (and which
the Post Office accepted) were “significant” and which it also stated had taken the Post
Office “by surprise”. It sought to correct these errors, by way of submission rather than
evidence, and one of those corrections was in relation to the implementation of the
332
333.
POL00112816
POLO00112816
document in question. The claimants objected to this, and said the explanation that was
provided was nowhere in the evidence and the Post Office should not be permitted to
give evidence by way of submissions.
Leading Counsel for the Post Office was asked about this specific part of the
submissions at the end of oral closing submissions and said that the explanation in
paragraph 147.4 of the closing “is my instructions but, my Lord, it is based upon a
previous version of the document and then an amended version of the document. And
I will undertake to give your Lordship the two references.” In an e mail of 10 July 2019,
which provided some references, further explanation was given which were said to
support the submission that it was only section 1.4 of the document that was not
implemented. The Post Office relied upon the following four points in support of that
submission:
1. The passage in Mr Godeseth’s witness statement I have reproduced at [325] above.
2. That there were different versions of the document that were disclosed, but only one
had been put in the trial bundle. The first version containing section 1.4 was dated 6
June 2010 and was authored by Mr Salawu.
3. This indicated therefore that the change was introduced by him. Most of the
procedure document had been in existence for a number of years before that.
4. Earlier versions had been disclosed, none of which contained section 1.4, and none
included section 1.4 and none were distributed to or even mentioned Mr Salawu.
The following points can be made about this.
1. Witness statements are expected to be factually correct.
2. Each party in this litigation who has submitted witness statements is given the
opportunity to ensure those statements contain accurate evidence before they were
served. Further, each witness is asked, at the beginning of their evidence when they
confirm the contents of their statements in their evidence in chief. Apart from
qualifications that were sometimes made, in this case (and in the great majority) the
statements are usually stated to be true to the best of that witness’ knowledge and belief.
3. If a witness’ written evidence is shown to be incorrect (as Mr Godeseth’s, along with
other of the Fujitsu witnesses, was) then the appropriate way to correct this is either in
re-examination or by way of supplementary evidence. Here, Mr Godeseth was asked
about this document and gave evidence in respect thereof on 20 March 2019. Mr Parker,
also a senior witness from Fujitsu, was not called to give evidence until 11 April 2019.
That was ample time for this self-contained point to have been dealt with by
supplementary evidence from Mr Parker, if that was thought by the Post Office to be
important. In any event, there is no reason for it having been incorrect in the first place.
Whether a procedure was, or was not, adopted by Fujitsu internally, is not something
that ought to be capable of misinterpretation in any event, let alone on a subject as
important as recording problems in the Horizon system.
334.
335
336.
POL00112816
POLO00112816
4. I do not accept a matter such as this is suitable to be corrected by submission “on
instruction” in any event.
5. However, even if it were, I am afraid that my view of information emanating from
Fujitsu is, based on numerous other instances of such information being wrong in the
actual evidence, to be somewhat sceptical of its reliability. It would also be potentially
unfair to the claimants to allow this type of “correction” as they would be deprived of
the opportunity of testing the new explanation. Absent some corroboration of this type
of information coming from Fujitsu that supported it, I would not accept it in this case
on this issue.
6. Further, and even if I were to take a contrary view of the points at 1 to 5 above, and
accept that the Post Office should be given the opportunity of correcting Mr Godeseth’s
evidence in this way, the explanation given by way of closing submissions does not
assist the Post Office, for the reasons explained in the following paragraphs, [334] and
[335] below.
This is for the following reasons. Mr Coyne’s point, to which Mr Godeseth was
responding, was that the Fujitsu document identified metrics and KPIs to
measure/control and reduce the risk of failure to detect, correct and remedy Horizon
errors and bugs. He was basing this on section 1.4 of the document. His conclusion was
that “From the above, it is my opinion that Post Office should be aware of all recorded
bugs/errors/defects in addition to those previously acknowledged by them, from the
process metrics compiled above.” These records were sought by the claimants and the
Post Office explained that Fujitsu’s position was that the records did not exist. They
were not therefore provided to the experts. Mr Godeseth’s cross-examination showed
that the Fujitsu explanation was wrong (and the Post Office’s closing submissions
accepted it was wrong). The records however were not produced.
Finally on this, after receiving the e mail of 10 July 2019 to which I refer at [332] above,
I asked in an e mail of 12 July 2019 for hard copies of the documents referred to in the
relevant paragraph of that e mail (and hence relied upon by the Post Office in this
respect) to be delivered to me. This was done, and a file served on the court containing
four versions of the Post Office’s Customer Service Problem Management Procedure.
They are version 0.1 dated 13 November 2007; version 2.0 dated 22 April 2008; version
2.1 dated 6 June 2010; and the version put to Mr Godeseth which was in the trial bundle,
dated 5 September 2017. Given the additional versions produced all pre-date Mr Salawu
leaving Fujitsu in 2013, this means that the points I have identified at [329] and [330]
are still valid ones, and are still in the claimants’ favour.
Even on Dr Worden’s evidence and his views on the lower number of bugs in the Bug
Table (he accepts that there are 11, although Mr Coyne’s figure is somewhat higher),
there are plainly more bugs in Horizon than Fujitsu itself was aware of. Over the years
of both Legacy Horizon and Horizon Online, the total number of software bugs, defects
and errors in Horizon considered by the experts is far greater than the number to which
Fujitsu have admitted. The total number of software bugs, defects and errors in both
versions of Horizon is very important information. Why Fujitsu chose not to collate and
report these in the manner that even a Fujitsu internal document stated would (or
should) be done, is wholly unclear. A comprehensive record, whether using the metrics
identified by Mr Salawu or other similar ones, would surely only assist both Fujitsu and
337.
338.
339,
340.
POL00112816
POLO00112816
the Post Office in its administration of Horizon. Given that such a comprehensive
record has not been produced to the experts for their consideration, notwithstanding the
request made on behalf of Mr Coyne, the court is entitled to conclude that no such
comprehensive record exists.
Mr Godeseth was also asked about the BDRB transaction correction tool. This applies
to Horizon Online. The original design document for this stated that it was to allow the
SSC to correct transactions by inserting balancing records to transactional/accounting
or stock tables directly into the BRDB system, and also to audit the changes made. It
was not designed to delete or update the records. However, in cross-examination the
claimants’ counsel analysed, and put to him, some careful points on how this tool had
been used in practice.
The tool works by means of an SQL insert into a variety of tables in the BDRB. SQL.
is an abbreviation for Structure Query Language, and is a domain-specific language
used in programming and designed for managing data. It is very widely used. Although
Mr Godeseth would refer simply to “Oracle” as the programme, Oracle Database or
Oracle is a database management system which is produced and marketed by the US
corporation which is also called Oracle. Although that is the name of the company, and
SQL was developed (or invented) before the Oracle Corporation was founded, in this
trial the terminology sometimes became a little shorthand. The Oracle Corporation is
very well known, not only in the computing field, but through its co-founder Mr Larry
Ellison who, amongst other achievements, has backed and competed in various
America’s Cup races, including winning the 33 America’s Cup in 2010 in the trimaran
USA 17 with Oracle BMW Racing. Mr Godeseth, and the experts, are all highly
experienced in the field and shorthand references to well-known languages or
applications such as SQL and Oracle are to be expected. This use of shorthand did not
lead to any lack of clarity in the evidence, however.
Information provided by the Post Office’s solicitors prior to the trial stated that the
transaction correction tool had been used 2,297 times. However, all save one of these
usages were said to be “Type 1” which were for the most part used to unlock stock units
and which had no impact on branch accounts. There was one admitted instance of what
was called “Type 2”, which was where a Balancing Transaction was inserted where it
changed transaction data in the main transactional tables. This will have affected branch
accounts. Mr Godeseth had not been involved in drafting that information supplied by
the Post Office’s solicitors, but he knew about it and it matched his view of the intended.
use of the transaction correction tool. The 10 database objects, or tables, and sequences
into which the tool was permitted to write are identified in a table at 2.4.1 of the low
level design document for the tool. They are identified in terms of their object names,
and were the BRDB operational exceptions table; the system parameters table; the FAD
hash outlet mapping table; the process audit table; the process audit sequence; the
transaction correction tool journal table; the FAD hash current instance table; the
transaction correction tool control table; the branch information table; and finally the
branch operators (or operational) exception sequence.
The design document made clear that:
“The following transaction tables have been granted INSERT privileges to
OPS$SUPPORTTOOLUSER.
341,
342.
343.
344.
POL00112816
POLO00112816
The transaction correction statement is only allowed to insert into these tables.”
These 9 tables were then listed. The process would use a “"Transaction file containing
an SQL INSERT statement that creates the required balancing transaction.", which is
an Oracle command, and the SQL INSERT statement would provide a missing half of
a transaction, where only one half was present. As it was put to Mr Godeseth, “the "SQL.
INSERT" statement effectively goes in and puts in the missing other side of that
transaction?”, a point with which he agreed. The SQL INSERT puts that missing other
side into the records.
The SSC members would log into their own UNIX user, and then change directory and
place their transaction file in to the sub-directory. The document then records “They [ie
the SSC member using the tool] will then invoke BRDBX015 manually. The shell
”
script module will be owned by the UNIX user 'supporttooluser’.
However, the explanation that was provided by the Post Office’s solicitors, which Mr
Godeseth said he was fully aware of, about the use of the tool was as follows:
"Each document is associated with a single SQL statement which made a database
correction. There are two different types of correction shown in the files - the SQL
statements for each are of the form:
"1, Update OPS$BRDB.brdb_rx_recovery_transactionsSET_ settlement_complete
_time stamp = ..." And then the "INSERT INTO" command.
The explanation of Type 1 was "Type I reflects the action taken to reset the recovery
flag on a transaction. This will have no effect on branch accounts....”. It was also stated
expressly that "Type 2 reflects the action taken to insert a Balancing Transaction where
it changes transaction data in the main transactional tables. This will affect branch
accounts."
Mr Godeseth expressly accepted, however, that this showed that the command used for
what was being called Type I was no/ an insert command, it was an update command.
Mr Godeseth described himself as not expert on Oracle, but he had a working
knowledge of it, and he is plainly more experienced with Oracle than most people. I
find his experience in Oracle more than sufficient for him to be able to answer questions
on this subject. Mr Godeseth’s evidence shows that Type I was an update — something
which the low level design document expressly said the tool would not be used for. The
design document did not contain the necessary database object fields table for
performing the unlocking function, which the Post Office’s solicitors said represented
all of the occasions, save one, when the tool had been used. This was all put carefully
and clearly to Mr Godeseth in his cross-examination, and he agreed with all the points
put.
This was further confirmed — the mandatory use of insert, and the fact that an update
command was not included in the design intent of the tool — elsewhere in the design
document where it stated under method “The module will read the contents of the input
transaction file, which will be in the form of an SQL insert statement. Only a single
insert statement is allowed and (after an optional introductory comment) it must start
346.
347.
348.
349.
POL00112816
POLO00112816
with the ‘insert into' clause." The insert would be in the SQL script, and Mr Godeseth
expressly confirmed this. This is another example of Fujitsu failing to observe its own
design intent for the use of a tool that can have an impact upon branch accounts, and in
my judgment is a factor that goes to the robustness of Horizon as well as the accuracy
of data. As was put to Mr Godeseth, the use of the tool had gone “way beyond” the
intent in the document, and he agreed. He said that “there is tooling which is based on
this” — by which he meant based on the transaction correction tool - “which has two
aspects to it, certainly, so I think I’m agreeing with you”.
Mr Godeseth was taken, very carefully, through a specific use of the transaction
correction tool in 2010. In PEAK 0195561, a problem was reported to the SSC on 4
March 2010 where a SPM had tried, on 2 March 2010, to transfer out £4,000 (referred
to in the PEAK as 4,000 pds, which means either pounds (plural) or pounds sterling)
from an individual stock unit into the shared main stock unit when the system crashed.
The SPM was then issued with 2 x £4,000 receipts. These two receipts had the same
session number. The PEAK, as one would expect, records various matters in note form
and also uses informal shorthand. However, the main thrust is that when the SPM did
the cash declaration, although the main stock unit (into which the £4,000 was being
transferred) “was fine”, the unit from which the cash was taken “was out by 4000
pounds (a loss of 4000 pds)”. This is very similar to what Mr Latif said had happened
to him, although the transfer in July 2015 to which he referred was £2,000. The PEAK
related to Horizon Online and was the admitted occasion when the Balancing
Transaction tool had been used.
Mr Godeseth had obtained the information in his witness statement dealing with this
from Mr Parker. He also said he thought it was puzzling, and he was not sure that the
PEAK “would be a totally accurate reflection of what happened” and was “an
interpretation put together by a developer who is investigating the problems.” He
suggested the original log files would be better (by which he meant better sources of
information) and still exist, but they were not before the court. These log files would be
held by Fujitsu, so any failure to have them before the court was no fault of the parties
in this litigation. If a senior Fujitsu witness thought they would be better records, they
should have been accessed and evidence about them could have been given directly.
The PEAK records the following:
“After discussion with Gareth Jenkins, the suggested correction is to negate the
duplicate transfer out by writing 2 lines to the BRDB_RX_REP_SESSION and
BRDB_RX_EPOSS_TRANSACTIONS tables, with:
1) Product 1, Quantity 1, Amount 4000.00, Counter mode id 7 (TI)
2) Product 6276, Quantity -1, Amount -4000.00, Counter mode id 7 (TT)
This should be done using the Transaction Correction tool. An OCP approved by POL.
will be needed.”
This concerns Horizon Online, due to the references to OCP and OSR, and shows
extensive efforts being made at Fujitsu to try to understand why this has occurred, and.
general doubt that the efforts to reproduce the fault in testing has worked. Some at
350.
352.
353.
POL00112816
POLO00112816
Fujitsu wanted the priority to be downgraded; other entries resisted this, shown by some
of the entries such as:
“What is missing from this Peek is an explanation of the events in terms of the requests,
how they were ordered and when any was committed. Only then can we qualify the
priority. The assumption is that we have a fix. The facts are —
1: A settlement request to timed.
2: A retry of request timeout occurred.
3: According to the DB entries both later succeeded.
Now unlike other reconciliation Peeks this stands out.....
We can’t reduce the priority unless we understand what is going on. ”
The PEAK is lengthy and demonstrates a degree of frustration within Fujitsu at their
own failure to get to the root cause. One entry on 24 March 2010 states inter alia
“....this shouldn’t have happened”. A later one on 25 March 2010 states “advised on
the latest update in the call she states she was going to check if it was the same thing
that happened before. [The Post Office] are want to know why this has happened? -
why does it keep happening? - can you advise on this.”
(emphasis added)
One of the reasons or concerns expressed in the PEAK is that JSN entries — which
should be unique and in respect of which no duplicates should be permitted in the
journal table - appear to have occurred. Mr Godeseth confirmed that this should not
have happened. Duplicate entries should not be permitted in the database. The
documents showed that this did occur. Mr Godeseth said that the “JSN is part of the
primary key into the message table, I cannot see how an Oracle database would allow
that to happen” and “that would have to be a bug in Oracle, and it was certainly not the
bug that I was looking — it was not the bug that I was looking at for the red alert”. He
also said: “If this in fact happened as written down here [ie in the PEAK] then it would
have had to have been a bug in Oracle and I certainly don't remember any such bug.”
The PEAK also shows computer script being used, both in an attempt to correct, and
understand and analyse, what occurred. On 23 April 2010 an entry states:
“T think we have done as much as we can on this one. In conclusion although we haven’t
been able to totally explain the behaviour, the risk of this type of PEAK occurring again
has been minimised in live due to a change of behaviour in the BAL with respect to
transactions. .... This may now be marked as a duplicate of PC0194893”.
Rather concerningly, the following entry appears that same day in another entry stating
that the “Defect cause” is “updated to 40 — General — User”.
However, later the same day the following entry is made:
“T am sending this call back with Response Rejected.
354.
355.
356.
357.
POL00112816
POLO00112816
Closing a call as Duplicate Call’ results in a black mark against me. It basically means
that I should not have sent the call over since the same problem has already been sent
over in a previous call.
PC0195561 (duplicate transfer of 4000.00 cash) may have been caused by the same
underlying fault as PC0194893 (banking reconciliation), however I could not have been
reasonably expected to link the 2 calls and take the decision that it was not necessary
to send PC0195561 over for further investigation.
Please close this call with category 'Advice After Investigation’
[End of Response]
Response code to call type L as Category 52 -- Pending -- Response Rejected
Response was delivered to Consumer.”
Eventually KEL cardce262s was updated with the information in the PEAK on 4 May
2010 and the call was closed. One of the final entries states that day “Category 60 —
Final — S/W Fix Released to Call Logger”. “S/W fix” means software fix. The call is
closed the same day. This is 2 months after the incident. The PEAK also referred to
other PEAKs. Mr Godeseth was asked about those too. PEAK 0195962 records that the
transaction tool has “been used in live”. That led to an Operational Change Process or
OCP 25882 and stated:
“Due to a system fault, the branch did a Transfer Out of £4000 and a corresponding
Transfer In of £8000
Justification: Correct a loss of £4000 at the branch due to a system fault
When: Planned for 10/03/2010 16:00 with a duration of 30 minutes
Extra detail: The Transfer In details were incorrectly doubled up when they were
written to the BRDB. This needs to be corrected using the Transaction Correction tool.”
(emphasis added)
The actual printed address of the OCP document is as follows:
http://deathstar/SSC2/SSC_OCP/viewocp.jsp? OCPRef=25882&S1D=f630 192836137
This may show that the folder in which the document was kept was named by someone
with an interest in a particular series of well-known films, or it may not. It does however
show that it was plainly generated or kept by the SSC.
Mr Godeseth agreed that the OCP showed at least one use of the transaction correction
tool for one balancing transaction. Part of Mr Godeseth’s explanation for what had
occurred, even though there had been two duplicate JSN entries, was as follows:
358.
359.
360.
361.
POL00112816
POLO00112816
“In this situation the symptoms, as I'm reading them, are that because there was a bug
in Horizon and this is in the new system, it was pretty early days of the Horizon Online,
because there was a problem, we had something coming through which got through the
journal filter but then failed at the branch database and so therefore, as far as the branch
database is concerned, it has not happened.
MR JUSTICE FRASER: I understand that. I think what Mr Green is putting to you is
that this shouldn't have got past the journal filter.
A. It's a bug, so certainly the way that the system should have worked, a JSN -- the
same JSN coming up would be just a simple repeat of the message for -- because there
was some sort of glitch.”
(emphasis added)
His final evidence in cross-examination on this was the following:
“A. Somebody may have been trying to look for a duplicate JSN entry. I can't really
comment on what the guy was doing at the time who was trying to investigate. I am
simply asserting that barring an Oracle bug, which would have been huge, you cannot
have two entries with the same JSN.”
In my judgment, the Horizon Issues are sufficiently wide and costly to resolve
concerning the Horizon System, both Legacy and Online, without widening them to
include some attempt at including specific bugs within Oracle too. The Oracle
Corporation is not even a party, and has had no opportunity to provide evidence or
submission. I simply recount Mr Godeseth’s evidence as he attempted to explain the
PEAK. He did, as can be seen, refer to potential bugs in Oracle as an explanation for
something that he believed to be puzzling. He accepted it was a bug and/or “a glitch”.
In any event, other than Mr Godeseth’s evidence expanded in re-examination that he
was involved in 2010 when establishing himself in Fujitsu (and in this he referred to an
Oracle bug, which involved nodes in the database going down) there is no evidence
before me that there were any bugs in Oracle in respect of this doubling up, and I reject
that suggestion. What this PEAK shows, in my judgment, is that Fujitsu itself, with a
great deal of time to prepare for the Horizon Issues trial, simply could not explain then,
and cannot explain now, what caused this event to occur. That is relevant to my
consideration of the Horizon Issues. It also shows that there were those in Fujitsu who
wished to close the call with the error put down to the user, as shown by the “Defect
cause” being “updated to 40 — General — User”. There is nothing in any of the material
on this incident to suggest it was in any way caused by the user.
He was taken to another PEAK, PC0175821 which was dated 19 February 2009. This
is during the period of Legacy Horizon.
PEAK 0132674 was in relation to an incident on 21 December 2005 which showed a
SPM, ata five counter site with seven stock units, had rolled over a particular stock unit
called BB. He rolled it over in what was called “an effectively empty state” but having
declared the correct amount of cash, and also adjusting the stock levels to the correct
volumes, his branch account showed “a gain of approximately £18,000”. This was an
362.
363.
364.
POL00112816
POLO00112816
obvious impact upon his branch account, and it was investigated. One entry in the
PEAK stated:
“We are unable to correct the system figures safely. We can however provide accurate
figures for what should have been in the Final Balance for BB, to enable POL [ie the
Post Office] to make the correction perhaps by using a Transaction Correction.
POL need to make a decision on whether they are able to correct the problem in this
way, however we do not see any other alternative. Corrective action should be taken
before 11th January when the branch is due to roll into TP10.
The cause of the problem is unknown and is under investigation.”
(emphasis added)
This accepts there is a problem, and accepts that its cause is unknown. The suggestion
by Fujitsu later in the PEAK on 3 January 2006 was to generate a negative figure
effectively by changing the relevant data in the messagestore to try and cancel out the
£18,000 which the SPM had showing as a positive figure in his branch accounts. It is
obvious to me that this would only remedy the effect of the problem, rather than what
caused the discrepancy in the branch accounts in the first place. However, the entry is
illuminating in any event:
“If we get to the problem before the office is rolled we are able to change objects in the
messagestore to reset the stockunit back to the CAP (TP) rollover trailer. The PM can
then rollover. PM should get a large shortage which cancels out the large gain.
We don't want to be having to do this as making manual changes to the messagestore
is open to error and each time we have to seek authorisation from POL to make the
changes.
If we get to the problem after the office is rolled (as in this call) then we are unable to
correct the system figures safely. Its not been decided how we get the PM sorted out.
All in all, we want this fixed asap.”
(emphasis added)
Mr Godeseth accepted that generating the artificial negative figure in the message store
could be done, but said he was “not involved interfacing with Horizon” at the time and
was not at the operational level shown in the PEAK. He was not involved in obtaining
authority from the Post Office for such matters.
Another PEAK 0152014 dated 10 December 2007 showed a problem that was
described in the PEAK as “POLFS Incomplete Summaries Report” which related to a
foreign currency transaction. At this time, £484 sterling could buy $1,000 (the PEAK
did not specify US or Canadian, but the associated OCP identified US$ specifically)
and the PEAK recorded that:
366.
367.
368.
POL00112816
POLO00112816
“This is due to a single SC line written for $1000 (£484) with no settlement in the
middle of two RISP transactions.
On call PCO151718 the harvester exception was corrected and now the transaction for
the day don't zero, hence this issue with the incomplete summaries report.
Am currently retrieving the messagestore for this branch, we will then be inserting a
new message on the counter to remove the effects of this. OCP 17510 has been raised.”
Again, this was being done to correct the effect of what had occurred, rather than
discovering what had led to the issue in the first place. In two places in the PEAK, the
following entry appeared:
«** Again, this may also have caused a receipts and payments error, can EDSC please
confirm whether this is a gain or loss at the counter and the amount.**”
Mr Godeseth had been asked to look at this PEAK in advance (his evidence went over
two days and this was agreed by the parties), and therefore had been given a chance to
familiarise himself with its contents, and those of an associated KEL. The PEAK stated.
"Worth noting that the branch did not have any issues with the mismatched transactions
because this was fixed before they did the roll. The branch is not aware of this and it's
best that the branch is not advised." (emphasis added)
When it was put to Mr Godeseth that this showed that on not all occasions were SPMs
advised of impacts on their branch accounts of particular problems, he said that was “a
fair inference”. An inference is a common sense conclusion, and I agree with Mr
Godeseth that it is indeed fair to draw that inference. He did however then add “I think
I would say that Post Office were well aware of this and I would argue that it's a Post
Office decision whether or not to tell a subpostmaster.” This shows that Mr Godeseth
was not immune from arguing the case or making suggestions not supported by the
evidence. Nothing was produced by either side to show that the SPM in this case was
told. Certainly the person at Fujitsu who wrote “....it’s best that the branch is not
advised” cannot have shared Mr Godeseth’s view at the time. He did however also say,
in my judgment again importantly, that “in the background to this there was a dialogue
with [the] Post Office” and he agreed with counsel who put to him that the “Post Office
would have been aware of what was being done by Fujitsu”.
The associated OCP however, whose subject was “write corrective bureau message for
FAD 183227” — bureau meaning foreign currency — gave a great amount more detail.
It stated:
“A single SC message 183227-7-1101211 was written in error on 26th November at
12:43:17, selling 1000 US dollars, with no corresponding settlement line. To remove
the effects of this message at both the branch and on POLFS, we will insert a new
message to negate the effects of the original message.
Justification: If the change is not made in the counter messagestore (before the stock
unit is balanced on Wednesday), the branch will have an unexpected gain of £484 (or
thereabouts - depends on exchange rate), and a receipts and payments mismatch. This
gain would have to be resolved at the branch. There would also be an inconsistency
369.
370.
371.
POL00112816
POLO00112816
between the branch and POLFS to be resolved. By correcting the problem locally, the
branch may not be aware of the problem, and there will be no inconsistency between.
the branch and POLFS.”
And:
“The message will include a comment to show it has been inserted to resolve this
problem (this will not be visible to the branch).
This change will first be applied to a copy of the messagestore within the SSC
environment, and the stock unit then rolled over to make sure there are no unexpected
consequences.
Neither the new nor the old message will be included in data sent to POLFS.
Gary Blackburn (POL) is already aware of this issue.”
(emphasis added)
This shows that the Post Office did know, as Mr Godeseth had said. It also shows that
the branch would not be able to see what was done. It also showed that the intention
was to reverse, by means of a specific insert into the messagestore by Fujitsu, an entry
or data in order to correct a discrepancy in the branch accounts caused by an error or
problem within Horizon Legacy. This was approved by the Post Office as shown by a
later entry. The OCP had been raised by Anne Chambers. The “extra detail” —
effectively what the message needed to contain or achieve — was:
“Extra detail: The original message had ProductNo:5129, Qty:1, SaleValue:484,
PQty:1000. The new message will have Qty:-1, SaleValue:-484, PQty:-1000, with other
attributes (including exchange rate) as before.”
The specific message that was inserted, as shown in the OCR, was as follows:
“Extra detail: This OCR is being raised so that EDSC is authorised to amend the txn
details for branch 183227 and insert these into tps_pol_fs_summaries_incomp table on
the host.
Comments
Andy Keil (POA SSC Support) wrote at 12/12/2007 15:07: Updated POLFS feed for
branch 183227 product 5129 mode SC with SaleValue=1014.73 and PQty=2080”
Mr Godeseth agreed that this showed “just over $2,000 being inserted in the Post Office
system”.
However, the PEAK demonstrated that the fix or corrective action that had been applied
had simply made matters worse. This is because it stated for 14 December 2007, two
days after the message had been inserted by Andy Keil of Fujitsu, the following:
“The counter problem which caused the first issue has been corrected by inserting a
message into the messagestore, for equal but opposite values/quantities, as agreed with
POL (OCP 17510).
372.
374.
POL00112816
POLO00112816
As aresult of this corrective action, the net effect on POLFS is zero, and POLFS figures
are in line with the branch. POLMIS received both the original message and the
corrective message.
Once the problem was corrected, there should have been no impact on the branch.
However it has been noted that the stock unit BDC had a loss of $1000, which was
generated after the correction was made. We have already notified Gary Blackburn at
POL (email attached). This appears to be a genuine loss at the branch, not a
consequence of the problem or correction.”
(emphasis added)
On 17 December Anne Chambers also stated in the PEAK:
“Summary for development:
A single SC line was written for $1000 (£484), with no settlement, in the middle of two
RISP transactions.
The line was missing some AdditionalData so it wasn't harvested properly, but the main
problem was the lack of settlement. POL authorised us to insert an equal but opposite
message, to prevent a discrepancy (in theory anyway) and to avoid problems on POLFS.
Please note that this is exceptional and must not be seen as a convenient avoidance in
place of a fix.
Subsequent investigations showed that a second packed pouch barcode had been
scanned, before the receipt for the first had finished processing. See the audit log /
messages on counter 7 26th Nov at 12:43 / 12:44.
PC0147357 is already with development for what I think is a similar problem - Mode
reverts to what it was prior to RISP.
Please also see update on that call.
Routing to EPOSS-Dev via QFP.
[End of Response]
Response code to call type L as Category 40 -- Pending -- Incident Under
Investigation.”
(emphasis added)
The PEAK was finally closed on 17 April 2008, the same day that the “defect cause”
was “updated to 14: Development — Code”.
There are two important points to make in relation to the entry in this PEAK which I
have quoted at [371] above. Firstly, the discrepancy in the branch accounts, which was,
after the “corrective action”, in deficit, was broadly in the amount of the difference
376.
377.
378.
POL00112816
POLO00112816
between the two corrections that were made, namely the one by the OCP and the OCR.
These corrections should, in my judgment, have been in the same amount, as one relates
to “front end” (or the branch messagestore) and the other “back end” of the accounts so
far as Horizon is concerned. The statement in the PEAK that “this appears to be a
genuine loss at the branch, not a consequence of the problem or correction” is simply
insupportable. That is a statement made by Anne Chambers, which in my judgment
flies in the face of the documents. I am supported in my conclusion by two things.
Firstly, plain and obvious common sense. Secondly, Mr Godeseth’s evidence, when it
was put to him that this was a fair possibility when he said “having read this PEAK in
more detail overnight then yes, clearly that is what appears to have been the case.” I
accept that evidence. It is supported by the documents and by the text of the two
different corrections.
The second important point is that the PEAK makes it clear this, or something very
similar, has happened before. It states “PC0147357 is already with development for
what I think is a similar problem - Mode reverts to what it was prior to RISP.” The
reference to “development” is the department within Fujitsu where software is written
to correct issues with the system. This is supported by the closure of the PEAK and
defect cause stating “Development — Code”.
There is an associated issue that arises on this PEAK, the number of which I will again
recite for clarity, PEAK PC0152014. I will deal with this here, rather than in the section
dealing with Mr Coyne’s evidence or within the Technical Appendix (although the
latter place is a more suitable home for it). This is because it goes to the weight to be
given to Mr Godeseth’s evidence.
A detailed explanation was put to Mr Coyne on this very matter, which occurred on the
third day of his evidence on 6 June 2019 and went from page 103 of the transcript, ran
until page 115 when the OCP was then put (which went to page 120) and then the OCR
which was put between page 120 and 126. Leading Counsel for the Post Office stated
“T can tell you on instruction what that figure means”; “So on instructions I can tell you
than the sale value of £1,0114.73 and the quantity of 2,080...... ” and “I appreciate that
I’m telling you this on instructions because I have to say nobody knew that this
suggestion was going to be made until it was put to Mr Godeseth....” It was made clear
that there was what was being put to Mr Coyne as a detailed explanation of what had
occurred, was “on instruction”, and it was nowhere in the evidence.
Mr Godeseth was cross-examined on 20 and 21 March 2019. When Mr Coyne was
cross examined, about 2 % months later, a great deal was put to him “on instruction”,
as I have identified. It was a detailed factual explanation that was not included in any
evidence at all. Firstly, the OCR that went with it showed that “This OCR is being
raised so that EDSC is authorised to amend the txn details for branch 183227 and insert
these into tps_pol_fs_summaries_incomp table on the host.” This was required
precisely because Mr Godeseth was correct, in my judgment, in his understanding of
what had occurred. I accept his evidence on this. Secondly, the explanation put “on
instruction” was (broadly) that there were two problems. A careful attempt was made
to explain to Mr Coyne how (basically) Mr Godeseth had misunderstood, and to
downplay the effect of what had occurred.
379.
380.
381.
POL00112816
POLO00112816
I do not accept that this subject was capable of being corrected — even if that is what
the documents showed, which in my judgment they plainly did not — in the way adopted
by the Post Office by giving evidence by way of submissions “on instruction”. This
deprived the claimants of any ability to challenge the explanation, if explanation it was,
by cross-examination. Further and in any event, there was ample time for the Post
Office to have provided a short witness statement from a Fujitsu witness, such as Mr
Parker (who was not called until 11 April 2019) dealing with this, if that was the
evidential explanation. If that had been done, permission for that statement would have
been required, but given the matter arose from something Mr Godeseth was asked
about, that application could have been dealt with at the time and if this genuinely was
evidence that the Post Office could not have known in advance would be required, it
would have been allowed in. It simply is not procedurally acceptable, or fair, for
evidence of this nature to be given by way of submission “on instruction”. However
and in any case, I have considered the Post Office’s case on this notwithstanding these
points, and I find that the documents do not substantiate the explanation given on Day
16 to Mr Coyne when he was being cross-examined.
The explanation did not make sense on the face of the documents, and they did not
support the points that were being put “on instruction”. Firstly, as Mr Coyne pointed
out, the PEAK identified that a message was to be inserted into the messagestore. None
of the documents put to Mr Coyne “on instruction” identified what that message was.
This point was made by Mr Coyne during the cross-examination:
“Q. Yes. So what they are talking about here, what this change here is, is a change
being made by using the TIP repair tool into the TPS, correct?
A. Right, so this doesn't relate to the creation of the message then.
Q. It doesn't relate to the branch accounts, Mr Coyne, does it? This is an OCR which
involves an exercise -- well, the use of the TIP repair tool to change data that is in the
TPS system, yes?
A. Yes, but the PEAK refers to the insertion of a message into the messagestore.”
The critical point, that the explanation skipped over, was what the message that was
inserted in fact was. The second point is that the PEAK refers to a KEL, KEL
obengce3120K. At the beginning of the trial, I asked for hard copies of all PEAKs and
KELs put to witnesses to be prepared in hard copy too, so that I had a working file that
I could use during the trial for PEAKs and KELs already referred to (in addition to the
three OPUS trial bundle screens that I had to work from). This was done. That KEL is
at tab 21 of volume 2 of the file entitled “PEAKs and KELs referred to in Days 5-8 and
12”. The KEL states the following, and I will reproduce the text in full, including the
relevant table references.
“KEL obenge3120K
Title:
Not harvested: Could not update database: Updating table TMS_RX_BDC_
TRANSACTIONS, ORA-02290: check constraint
(OPS$TPS.BT23B_BUREAU_REGION_CHK) violated.
382.
POL00112816
POLO00112816
Summary:
Harvesterd did not harvest message: Could not update database: Updating table
TMS_RX_BDC_ TRANSACTIONS, ORA-02290: check constraint
(OPS$TPS.BT23B_BUREAU_REGION_CHK) violated
Symptoms
The Harvester tried to insert into the BdC table TMS_RX_BDC_TRANSACTIONS a
message with missing ‘Blackbox’ data. This caused Oracle [ORA-02290] to detect a
check constraint (OPS$TPS.BT23B_BUREAU_REGION_CHK) violation, hence the
appearance in the TPSC254.<br><br>Harvester exceptions do not normally cause
entries on TPSC250, but this one seems to. <br><br>TPSC257 displays the contents of
the TPS Incomplete Summary table where the summarisation program detects that the
total harvested transactions for the day don’t NET to zero.
Problem
Due to some data in the ‘Blackbox’ or AdditionalData of the messagestore not being
populated, the bureau product was missing key data
including<br>BUREAU_REGION<br>MARGIN<br>MARGIN_PRODUCT<br>EF
FECTIVE_EXCHANGE_RATE<br>and so when the Harvester attempted to insert the
message into the BdC table TMS_RX_BDC_TRANSACTIONS, however Oracle
[ORA-02290] detected a check constraint
(OPS$TPS.BT23B_BUREAU_REGION_CHK) being violated.
Solution — ATOS
Warning: Check the AMOUNT (or sum of AMOUNTS) for the branch on the TPSC254
report.<br><br>If the total AMOUNT is NOT ZERO, and the branch is NOT on
TPSC257 (same day), <br>or the total AMOUNT IS zero and the branch IS on
TPSC257 (same day) then there is probably a problem with the messages written on the
counter — a bureau SC message with no corresponding settlement. Use KEL
<ahref=kel._view_kel.jsp?KELRef=acha3 159Q>acha3 159Q</a>.<br><br>Check the
messages anyway, just to be on the safe side. If the session was settled
properly. ...<br><br>MSU must raise an OCR so that SSC can use the TIP Repair Tool
to populate the missing columns, using values from another transaction for the same
currency, same branch, same day if possible.<br><br>NOTE: By repairing the txn, the
TPS_POL_FS_SUMMARIES_INCOMP will be corrected automatically after that txn
has been successfully harvested. <br><br>*** In the past, we have just repaired these.
on the assumption that they are genuine SC transactions which have been corrupted.
However I think it may not be a genuine sale, but is related to an attempted pouch
reversal (this is certainly the case for the more extreme instances where the settlement
is missing). If the SaleValue and the PQty have opposite signs, and there is no reversal
(mode ER) for the transaction, this may have caused a loss or gain at the branch which
they can’t resolve themselves. Let Anne know if there is another occurrence like this.
(emphasis added)
This KEL shows that there was an impact on branch accounts — it states “this may have
caused a loss or gain at the branch which they can’t resolve themselves”, which is very
383.
384.
385.
386.
387.
POL00112816
POLO00112816
clear. It shows that the explanation put to Mr Coyne “on instruction” is, in my judgment,
completely wrong. It also shows that it has happened before, as “in the past we have
just repaired these”. I would also note for completeness that this KEL is not dealt with
at all by Mr Parker in the table accompanying his 1 witness statement. Two KELs
from Ms Obeng, namely CObeng1123 at number 24 in the table at E2/11/40; and
obenge5933K at number 31 in the table at E2/11/43.
J accept Mr Godeseth’s evidence on this point, which is wholly consistent with the
documents, in particular the PEAK and the KEL. The text of those documents shows
that a message was intended to be written into the messagestore; the actual message
used is nowhere available; the effects of all this clearly being an impact on branch
accounts.
A further PEAK was 0175821, which was 22 February 2009. This also related to
foreign currency transactions. This PEAK related to two problems, or a problem with
two elements to it. It is at F/485/1 in the trial bundle, and it clearly and expressly refers
to the same KEL, obenge3120K.
The two aspects were explained by Mr Godeseth as a change using the transaction
repair tool to “get the feed into POLFS correct” and also “a change to the messagestore
to get the branch aligned.” The repair “had to go two ways” because the branch and the
POLFS had to be aligned. If they were not, there would be a discrepancy in branch
accounts. This showed the two problems, or two sides to the same problem. It was stated
as follows in the PEAK:
“The first is where all five SC transactions missing core data as described in the above-
mentioned KEL." This means obenge3120K, referred to at the beginning of the PEAK.
And the “Second is absence of equal but opposite 2 (ie settlement) lines. See
PC0152014 for a similar problem and how problem was resolved."
“There are two sides to the problem relating to these txns. The first is where all five SC
txns missing core data as described in the above mentioned KEL. Second is absence of
equal but opposite [i-e. settlement] lines. See PCO152014 for a similar problem and how
problem was resolved.
For the first problem, I have used the TRT [the transaction repair tool] to insert the
missing data i.e. Region, Margin, Margin Product and EffectiveExRate.”
The reference to “txns” means transactions. PC0152014 is the very PEAK referred to
in [376] above.
Mr Godeseth agreed with both of those points in the PEAK as they were put to him by
the claimants. Basically, half the data was missing. The transaction repair tool or TRT
was used for the first problem, which was also called the TIP repair tool.
Numerous other PEAKS and some KELs were put to him with very specific details.
Both Gareth Jenkins and Anne Chambers were involved in raising, or investigation, a
great many of them and they are referred to in the text of both classes of documents. In
all of them, Mr Godeseth’s answers to questions were, in my judgment, consistent with
388.
389.
POL00112816
POLO00112816
the entries within the documents, and also consistent with the claimants’ case on the
Horizon Issues. In particular, those in the PEAKS dealing with foreign currency which
I have explained above was clear, and settlement lines were missing, which Fujitsu
sought to correct by injecting lines into both the messagestore and into POLFS.
Finally, a series of questions were put to him demonstrating what are called APPSUPP.
permissions. This is a very powerful permission (which really means level of access)
and PEAK 0208119, which runs from February 2011 onwards into 2012 and beyond
(the final entry being in 2015), made the following relevant comment:
“As per the previous PEAK comments, the role 'APPSUP' is extremely powerful and
should only be used under extreme circumstances and under MSC supervision. As such
the Branch Database design was that 3rd line support users should be given the 'SSC’
role, which is effectively read access, ie. 'select_any_table + select_catalogue'. SSC
team members should only have to [access] BRSS for normal support investigations,
unless the information has not replicated in time. SSC should only given the optional
role 'APPSUP' temporarily (by Security Ops authorisation/emergency MSC) if required
to make emergency data amendments in BRDB Live.
It is a security breach if any user write access is not audited on Branch Database, hence
the emergency MSC for any APPSUP role activity must have session logs attached
under the MSC. Host-Dev previously provided scripts, such as the Transaction
Correction Tool, are written to run under the SSC role and also write to the audit logs.
SSC users created on BRDB should only have the SSC role, and the user creation script
should be amended by Host-Dev to reflect this. A separate script giving/revoking
emergency MSC access via APPSUP can be delivered, logging this to the hostaudit
directory. In parallel Host-Dev should investigate any Host-Dev delivered script to
ensure they are all executable by the SSC role. SSC should investigate any of their own
scripts to ensure they have sufficient permissions under the SSC role, taking into
account they should primarily perform their work on BRSS.
Any day to day scripts should not access BRDB directly. Any scripts needing more
than the SSC role should be questioned, except those that would run under MSC
APPSUP. Once the investigation is complete, all BRDB SSC users with APPSUP
should have the role removed by ISD (via MSC) and ensure they do have the SSC role.
If anyone is in disagreement with the above course of action, then I'll set up a meeting
with yourselves and Security when I'm down in BRAOI next week.”
(emphasis added)
This entry was from Andy Beardmore, Senior Software and Solution Design Architect
Application Services. The experts are agreed that the APPSUPP role would, effectively,
permit anyone who had that permission to do almost anything. It was available to 3°
line support at SSC, the level at which Mr Roll was employed by Fujitsu. This PEAK
further substantiates the evidence of Mr Roll and is consistent with it. APPSUPP was
described by Mr Parker as “the more technically correct name for a type of privileged
access to the BRDB”. It is a very powerful permission.
390.
391.
392.
POL00112816
POLO00112816
A later entry in the same PEAK states:
“The Business Impact has been updated:
1. Cost: There is currently no "cost" to this issue. The users affected have more access
than is required.
2. Perceived Impact: The customer is not aware of this problem or change.
3. Scope: No actual impact/incidents of problems relating to this issue have been
experienced yet (and not expected)”.
(emphasis added)
The customer was the Post Office. The “issue” referred to was Fujitsu SSC users having
had far greater access, namely the powerful APPSUPP permissions, on more than the
intended basis, which was supposed to be in extreme circumstances only and
temporarily. This is shown in an entry for 1 February 2011 by Mark Wright who stated
“"..SSC users have the APPSUPP role” and “When we created SSC users for
BDB/BRS etc. we used “appsup” as that is what SSC have always been and what they
migrated as on Horizon databases.” (emphasis added)
It is clear that all the members SSC had always APPSUPP, but were not supposed to
have this powerful role all the time.
Anne Chambers said, in an entry on 1 February 2011:
“Unfortunately development write their scripts explicitly to use SSC. So I think we're
stuck with it unless they deliver new scripts (which would not be a popular or quick
option).
When we go off piste we use appsup. Can we have both??”
(emphasis added)
Mr Godeseth said that by “off piste” he was very confident that she meant “having to
fix a problem that was not catered for by a script that is available to these people” by
which he meant SSC. It was clear that SSC were using APPSUPP and wished to
continue to do so. It was equally clear that all the members of SSC were not supposed
to have it.
The Post Office’s auditors for the year ended 27 March 2011 were Ernst & Young
(“E&Y”) a very well-known global firm. In the Management Letter for that year, E&Y
had identified the following:
“The main area we would encourage management focus on in the current year is
improving the IT governance and control environment.
Within the IT environment our audit work has again identified weaknesses mainly
relating to the control environment operated by POL’s third party IT suppliers.
394.
395.
396.
397.
POL00112816
POLO00112816
Our key recommendations can be summarised into the following four areas:
4 Improve governance of outsourcing application management
Improve segregation of duties within the manage change process
“ Strengthen the change management process
_ Strengthen the review of privileged access”
(emphasis added)
The “third party IT suppliers” referred to were Fujitsu. Whether it was the work
performed by E&Y that had led to the decision at Fujitsu that the APPSUPP
permissions were not being sufficiently restricted or controlled, or something else, does
not much matter. A company such as E&Y would not lightly refer to “again identified
weaknesses” in the Management Letter for a particular year unless that conclusion had
been reached after a high degree of professional and in depth investigation, of a type
that even a lengthy adversarial trial such as this one cannot hope to replicate.
Indeed, the Post Office’s solicitors confirmed in a letter in this litigation dated 11
February 2019 that Fujitsu SSC users who were identified as PERSON-POA UNIX
(also sometimes referred to as POA Unix users) had a range of privileges and were
granted the UNXADM role. This may be a descriptor for Unix (hence UNX)
Administrator (hence ADM) but this does not matter. The UNXADM role contains, to
quote from the letter of 11 February 2019 itself, the “DBA role” which is stated to be:
“This is an Oracle supplied role for use by database administrators (DBAs). Lots of
privileges are granted to this role so users have the ability to update/delete/insert into
any of the Branch database tables.”
(emphasis added)
POA Unix users also are:
“Granted the ability to execute the following executable functions:
(a) OPS$SUPPORTTOOLUSER.PKG BRDB TXN_CORRECTION —framework to
allow the user to insert fully audited balancing records into a BRDB transaction table
(made against node ID 99).”
Both of these are powerful roles. Mr Godeseth was asked specifically about the DBA
role explained in [395] above. His specific answers merit quotation verbatim:
“Q. You would agree that those people have the role which allows them privileges to
update, delete, or insert into branch database tables whether they are using the
correction tool or not?
A. Those people could log on to the database and do an awful lot of damage.
398.
399.
400.
401.
402.
POL00112816
POLO00112816
Q. And the only audit of that that we have prior to 2015 was log on and log off; that's
correct, isn't it?
A. Correct.”
(emphasis added)
Mr Coyne had given evidence that he had identified 2,175 occasions when it had been
used. Mr Parker in his 3° witness statement had said that figure was wrong, but also
said:
“J have not examined the privileged user logs, but based on my experience my
expectation is that these uses of APPSUP, or at least the vast majority, are for support
work that does not involve changes to transaction data. I cannot recall any cases in
which it has been used to change transaction data, but I cannot state unequivocally that
there are no circumstances in which it has ever happened.”
Mr Godeseth said in his written evidence that “Privileged users can, theoretically,
inject, edit or delete transaction data in Horizon Online. As far as I am aware, this has
never happened.” His evidence above about prior to 2015 makes it clear that he could
not work out the number of times this may have happened in any event. Therefore the
fact that he was not aware it had ever happened is neither here nor there, and does not
assist the Post Office.
In his re-examination Mr Godeseth was asked about an entry in an OCP in relation to a
fix that stated UNIX user brdb. He said he did not know the detail, it meant logging on
to the database, and the “UNIX user” referred to “would be one of the guys in Ireland”.
These were not members of the SSC. When asked to clarify “who are the ‘guys in
Ireland’ exactly?” he said “the people who support the hardware, so UNIX is an
operating system so they work at a pretty low level on the systems” and by “low level”
he meant “they have pretty powerful user rights but they are very much driven by
process as to how they use them”. He also said there was complete and utter control on
the process they go through before they did any of this type of activity, which in that
re-examination example was "Logon to BRDB Node I as UNIX user 'brdb'."
He was also cross-examined about some of the bugs, which appear in the Bug Table
considered and partly agreed by the experts. The main admitted bug by the Post Office
(in distinction to other bugs admitted by its expert, Dr Worden) is called the Callendar
Square bug. Mr Godeseth gave evidence in his witness statement about this, partly in
response to a statement that had been given by Professor McLachlan, who was not in
the event called by the claimants. However, Mr Godeseth’s evidence about Callendar
Square on this was still called by the Post Office. It is a Legacy Horizon issue. Mr
Godeseth had no first hand knowledge of this but had reviewed documents and spoken
to Mr Jenkins.
One of the KELs to which he was taken, with the reference JSimpkins338Q, related to
problems with Riposte and what was sometimes called the Riposte lock or unlock
problem. This was when an unexpected error occurred in Riposte while attempting to
insert a message. The KEL shows it ran from 2002, with other events occurring in 2003,
2004 and 2005. The entry for September 2005 expressly states that “this problem is
403.
404.
POL00112816
POLO00112816
occurring every week, in one case at the same site on 2 consecutive weeks”. A PEAK
was sent to development. Under “solution” in the KEL the following entry appears:
“SMC: If the event is seen at a multi-counter office during the working hours of the
PO, or up to 18:00 on a weekday (in case they are balancing out-of-hours), RING THE
OFFICE AND GET THEM TO REBOOT the eventing counter. If they are in the
process of balancing, it is strongly advised that they reboot before continuing with
balancing as they are at risk of producing an incorrect balance. Warn the PM that if
transactions appear to be missing, they should not be re-entered - they will become
visible after the counter has been rebooted.
If a reboot/Cleardesk does not resolve this problem, send the call over for further
investigation - SSC can rebuild the messagestore on the affected counter.”
(Block capitals in original; emphasis by underlining added)
The risk of a branch producing an incorrect balance as a result of this problem with
Riposte is clearly recorded. An incorrect balance is another way of referring to
discrepancies in branch accounts, a central issue in this litigation.
The Callendar Square issue or consequences (now admitted to be a bug) were
discovered in 2005 at the Callendar Square branch (hence the name). It is also
sometimes called the Falkirk bug, but I shall refer to it as Callendar Square. It
undoubtedly caused discrepancies in the accounts. The SPM in question had support
from their Area Manager and the documents demonstrate the following, from the Area
Intervention Manager Visit Log:
“This office had severe problems balancing on Wk 25, resulting in a shortage of
£6,414.46. After checking various reports I am satisfied that the error is made up of:
£3,489.69 — Transfers
£2,870 — Giro Deposits
£54.52 — unidentified (however due to all the coming and going with re- keying entries,
then this could come back as an error).
The Spmr claims that there was a Horizon software problem on 14.09.05 from 15.30
onwards. This was picked up when a member of staff noticed that a transaction, which
had been taken by another member of staff, had not been entered onto the system, so
therefore she put the transaction through. She checked at the time with her colleague
who said that she thought she had put it through already however she accepted that she
could have made a mistake. Following on from that, it was picked up that other giro
business deposits that had been entered had not come up on the system, so they were
re-keyed.
There was also a problem with transfers from one stock to another, in that they had
doubled up. The Spmr made several telephone calls to the NBSC, telling them about
his problems and he was advised to carry on with balancing and produce his Cash
405.
406.
407.
POL00112816
POLO00112816
Account. Whilst doing this a warning came up, however the NBSC told the staff to
continue to roll over. The result was that the office balanced £6,414.46 short.
The Spmr spoke to the Horizon Support Centre (ref E0509150123) who investigated
and agreed that there had been a navigation problem that had now been rectified. They
told the Spmr that they would report to NBSC that they had identified and rectified the
problem and that the amount could be held in the suspense account. However, as part
of the shortage relates to transfers, and no error notice will be issued, then the Suspense
Account Team are not prepared to authorise the entry.
I telephoned The Suspense Account Team (Ann Wilde), who told me that checks could
be made with Girobank after next Wednesday, and if that shows that duplications have
been made, then they will authorise the amount to be moved to the suspense account,
until the office receives an error notice. However, Ann stands by what she said about
the transfer problems, and that they would not move this amount to The Suspense
Account.
I went back to the Horizon Support Centre and spoke to a supervisor (Ken). He said
that the call had now been closed as the problem had been rectified. I asked what was
to happen about the resulting shortage and he referred me back to NBSC, who they said
would do various checks. I then contacted NBSC, spoke to Rob Hughes and told him
the story — he said he would put a call through to Service Support. No follow up was
received from Service Support regarding this call.”
(emphasis added)
The important points from this are as follows. This problem tumed out to be an admitted
bug. At the time, the SPM noticed the problem — and could only identify it as a “problem
with Horizon” and reported it. The branch accounts were plainly affected by a sizeable
amount, over £6,400. Horizon Support Centre, after investigation, agreed that there was
a “navigation problem” — this is a problem with the Horizon System, and was caused
by a bug. The Suspense Account Team would not authorise the shortfall in the branch
accounts being moved to the Suspense Account; in other words, they were not prepared
not to hold the SPM responsible. Mr Godeseth also accepted that “the genesis of it
appears to go back to February 2003 and similar lock agent problems back in November
2000”, a point that he accepted as “that’s fair”. The history of the problem was therefore
somewhat longer than something that just occurred in 2005 for the first time.
Mr Godeseth admitted that it was “a horrible position” for the SPM to have been in. He
said “Service Support” was not Fujitsu, it was the Post Office. The story at [404] and
[405] read like a mini-summary of many of the factual issues in the litigation generally,
with the exception that this is an admitted bug by the Post Office.
The subsequent documents in this respect show that the matter went on for some
considerable time. In November 2005 the following is stated:
“1/ This problem is the root cause of the reconciliation error closed in PC0126042
2/ Presumably the root cause is deemed to be software not hardware
POL00112816
POLO00112816
3/ The Postmaster has a workaround in place which is not to duplicate transactions (e.g.
Transfer In) just because the original attempts were successful but not showing on all
nodes
4/ POA CS MSU have a workaround in place which is that if 3/ above is not followed
& PC0126042 reoccurs, a BIMS will be issued advising POL to issue a Transaction
Correction
5/ There is no SLT for software fixes as they are delivered based upon the priority or
severity of the issue and could remain open until both businesses decide a fix is
necessary or the work around is adequate.”
This shows that the entry in the PEAK identifies this as a software problem; that is
effectively now accepted, but this entry from 2005 did not lead to any public acceptance
of a bug in Legacy Horizon either then, shortly afterwards, or for some years after that.
408. The acronym SLT means Service Level Target. What this entry means is that this
problem had occurred before, and was known to have occurred before; there was
something called “a workaround” in place, but that required the SPM physically
deciding not to duplicate transactions if the original transaction did not show up as
successful on all the nodes; if that were not done, TCs would be required; both Fujitsu
and the Post Office had to decide a software fix was required in order for this to be
corrected by fixing the software. No such decision had been taken at this point.
409. At Callendar Square itself, in January 2006 the SPM phoned one of his superiors. The
record of that call in the log states the following:
“Issue Raised: PM WANTS TO SPEAK TO SANDRA MCKAY URGENTLY
REGARDING SERIOUS PROBLEMS WITH HORIZON IT IS REPEAT
PERFORMANCE AS TO WHAT WAS HAPPENING BEFORE AND HSH FAILING
TO PICK IT UP
Response by SPM: Telephoned the office and Allan said that he was having problems
come back to him to say that there is no system problem and that he should contact
NBSC. He did this and from what I can understand the NBSC have told him that he is
trying to balance on two different terminals. Allan disputes this and is adamant that
there is a system error.”
(Block capitals in original; emphasis by underlining added)
410. The statement that “there is no system problem” which is told to the SPM evidently
conflicts with what the PEAK at [407] had recorded, that “presumably the root cause”
was the software.
411. Emails in February 2006 to and from Anne Chambers stated expressly the following:
“.... notice that in the early guise of this problem in the call it states the PM as Female:
412.
POL00112816
POLO00112816
Wed 12 October 2005 17:39 by UK956078 / HSH1 Saved: Wed 12 October 2005 17:39
Pm was trying to transfer £2490 from node 7 onto node 2. She states that she has
accepted the transfer on node 2 but the system is not showing this. On node 7 it is
showing pending transfer but it is not showing on node 2. It appears on her transfer
sheet as completed
At the bottom of this email re a magical £43k appearing and disappearing the PM is
Male He reports:
You may recall that in September the above office had major problems with their
Horizon system relating to transfers between stock units.
The Spmr has reported that he is again experiencing problems with transfers, (05.01.06)
which resulted in a loss of around £43k which has subsequently rectified itself. I know
that the Spmr has reported this to Horizon Support, who have come back to him stating
that they cannot find any problem.
Clearly the Spmr is concerned as we have just spent a number of months trying to sort
out the first instance and he doesn't want a repeat performance. He is convinced that
there is something wrong with his Horizon kit. I would be grateful if you could
investigate this and give him any support that you can. I'm due to visit the office
tomorrow to have a look at his paperwork and discuss the situation with him.
So apologies for the long windedness but I have been given this by Liz as a problem
so:
1. Is there a problem at this branch? is it Horizon kit or is there an issue with staff there?
2. If there is an issue is this $90 release the cure? how confident are you/we it will fix
the problem?
3. $90 counter release due week 4th March. Getting Sarah to check if this site is in the
pilot 24th or just part of the general release 4th March.
Appreciate your comments please.”
(emphasis added)
In the answer to this question posed in February 2006, Anne Chambers stated the
following:
“Haven't looked at the recent evidence, but I know in the past this site had hit this
Riposte lock problem 2 or 3 times within a few weeks. This problem has been around
for years and affects a number of sites most weeks, and finally Escher say they have
done something about it. I am interested in whether they really have fixed it which it
why I left the call open - to remind me to check over the whole estate once $90 is live
- call me cynical but I do not just accept a 3rd party's word that they have fixed
something!
413.
414.
415.
416.
POL00112816
POLO00112816
What I never got to the bottom of, having usually had more pressing things to do, was
why this outlet was particularly prone to the problem. Possibly because they follow
some particular procedure/sequence which makes it more likely to happen? This could
still be worth investigating, especially if they have continuing problems, but I don't
think it is worthwhile until we know the $90 position.
Please note that KELs tell SMC that they must contact sites and warn them of balancing
problems if they notice the event storms caused by the held lock, and advise them to
reboot the affected counter before continuing with the balance. Unfortunately in
practice it seems to take SMC several hours to notice these storms by which time the
damage may have been done.”
(emphasis added)
This shows the following important points. At least Anne Chambers in early 2006, and
all those with whom she was corresponding, knew that this problem — now admitted to
be a software bug — had been around “for years”. Horizon Support were telling the
SPM, whose branch accounts were affected by discrepancies, that “they cannot find
any problem”. The SMC ~ the part within Fujitsu responsible for providing corrective
action for the “event storms” — would not always notice these had occurred in time and
by then “the damage may have been done”. The damage referred to can only mean
impact upon branch accounts.
Mr Godeseth gave evidence about S90, the software release intended to remedy this,
which was distributed in March 2006. This was 6 years after Legacy Horizon became
in use. His evidence was that this release was one year before what he called “the
relevant period” for Ms Misra’s branch, one of the SPMs convicted of criminal
offences. He also gave evidence that the Callendar Square bug affected 30 branches,
and had caused mismatches in the branch accounts at 19 (corrected in supplementary
evidence in chief from 20) of them. He said in his witness statement that this
information had come from Matthew Lenton, Fujitsu’s Post Office Account Document
Manager and that “Fujitsu has established this for the purposes of this statement using
the event logs described above.” He accepted that this part of his witness statement was
not correct. The figures that he had provided had not come from an exercise specifically
for the purposes of his evidence, or for the Horizon Issues trial. It had actually come
from “the spreadsheet [which] is something that was done at the time, as I understand
it by Anne Chambers, to actually investigate what had happened when we had had these
locks.”
This was an important correction by Mr Godeseth, for reasons that I will now explain.
It is a matter of public record that Anne Chambers gave evidence before the High Court
in the case of Post Office Ltd v Castleton [2007] EWHC 5 (QB) before HHJ Havery
QC. Her evidence in that case is summarised at [23] of that judgment. Callendar Square
is referred to, but as a single branch affected. The reason that I have referred to this case
is because it specifically referred to by Mr Jenkins in an e mail of 8 March 2010 which
was referred to by Mr Godeseth in paragraph 14 of his witness statement. Mr Godeseth
said that he agreed with that e mail, which stated:
417.
418.
POL00112816
POLO00112816
“ve been asked about the issue at Callender Square, Falkirk (I’m not quite sure about
the spelling) that came up at the Castleton Trial.
I thought I'd better keep you in the loop on this.
T’ve now dug back into the archives to provide the following summary:
1. The problem occurred when transferring Cash or Stock between Stock Units. Note
that West Byfleet does operate multiple Stock Units so the issue could have occurred.
2. It manifests itself by the Receiving Stock Unit not being able to “see” the Transfer
made by the “sending” Stock Unit and is compounded by attempting to make a further
transfer. Note that such transactions usually reappear the next day.
3. It is clearly visible to the User as a “Receipts and Payments mismatch” at the time
that one of the Stock Units is Balanced. This usually results in the Branch raising a call.
There are no such calls in Andy Dunks’ Witness Statement which summarises the calls
raised by West Byfleet. Also this can be checked on any Balance Reports or Branch
Trading Statements that are available from the Branch which should show that Receipts
and Payments do match and that the Trading Position is zero.
4. The problem is also visible when looking at system events associated with the
Branch. The System events from 30/06/2005 to 31/12/2009 for West Byfleet have been
checked and no such events have been found.
5. The problem was fixed in the $90 Release which went live in March 2006 and so
would not have been relevant at the time of the detailed Transaction Logs obtained for
West Byfleet between December 2006 and December 2007.
Therefore I can conclude that the problems identified in Calendar Square, Falkirk are
not relevant to West Byfleet.”
This e mail does not provide anywhere near the same degree of information about the
Callendar Square bug as was available at the time of the Horizon Issues trial, or as
recorded in the PEAKs above. There has obviously been further investigation at Fujitsu
into this specific issue since then, as made clear by Mr Godeseth’s evidence about the
information coming from Mr Lenton and Ms Anne Chambers. The e mail does not use
the term “software bug”. It does not refer to the fact that it is accepted by Fujitsu,
following an investigation, that the admitted bug had affected at least 30 branches. Nor
does it refer to the fact that the Callendar Square bug caused mismatches at 19 of those
30 branches.
The spreadsheet to which Mr Godeseth referred as having been “prepared at the time”
by Anne Chambers was also put to him. It included textual notes and a list of call
references and branches affected by approximate date, which were all in 2004 and 2005.
The notes were:
“NB many other branches had multiple events, preventing replication, but these are the
majority of those which came to Peak, having either reported a problem or it caused a
reconciliation report entry.
419.
420.
421.
POL00112816
POLO00112816
From Sept 2005, cash accounts were replaced by branch trading statements and the
TPSC256 report was no longer populated. I can't remember how we then knew about
receipts and payments mismatches and if we would have picked up on further issues.”
(emphasis added)
These notes make clear that there were other branches potentially affected, and the list
of 30 branches compiled in the spreadsheet were “the majority”. It does not state that it
is an exhaustive list and the notes suggest it is not. The summary for each branch listed
gives a variety of consequences or effects. They include:
1. Receipts and Payments Mismatch.
2. Receipts and Payments Error.
3. Host Generated Cash Account Line.
4. Unable to roll Stock Units.
5. Reconciliation Report.
6. Slow running counter.
7. Pension and Allowance Report printouts.
8. Screen freezes.
9. Problems with declarations.
10. Unable to create AA.
11. Loss due to double transfer.
12. Transfers not showing on nodes.
13. CAC Lines not matching.
The spreadsheet was disclosed to the claimants on 27 February 2019, or very shortly
before the Horizon Issues trial, and the properties shown on the electronic document
identify Anne Chambers as the author, with a date of 22 December 2015.
Mr Godeseth also, entirely accurately in my judgment, accepted that the Callendar
Square bug had been present for a considerable amount of time prior to 2005. He said
“T would agree that the underlying bug had been there for a considerable time, probably
since the Horizon went in”. This is made clear by, amongst other documents, Anne
Chambers’ entry in February 2006 stating that the problem had “been around for years”.
In another of the many PEAKS that dealt with this issue, the entry for 22 September
2005 states “Note - a few of these errors seem to occur every week at different sites.”
422.
423.
424.
425.
POL00112816
POLO00112816
Mr Godeseth was taken to the contents of two letters from the Post Office’s solicitors
to the claimants’ solicitors. The first was dated 28 July 2016. This was the Post Office’s
solicitors’ substantive response to the claimants’ letter of claim, is very lengthy (as was
the letter of claim itself) and ran to 99 pages. The second letter was dated 11 January
2019. He agreed that certain important elements of the paragraphs in those letters
explaining Callendar Square were incorrect. They were:
1. That the bug was only discovered in 2005.
2. That the bug had only affected one branch. The second letter stated in very clear
terms that “The Falkirk/Callendar Square issue was only known to have affected that
one branch.”
Not only were both those statements in the solicitors’ correspondence completely
wrong, but the numerous documents at [398] to [421] above demonstrate that there were
a great number of references, including in both KELs and PEAKs (documents of a type
that had been at one stage claimed to be wholly irrelevant to the issues in this litigation)
to the direct contrary.
Mr Godeseth’s witness statement about this, prior to his cross-examination, presented
a very different picture to the one that eventually emerged. Indeed, I would go further,
and I find that his witness statements omitted some very important headline points in
respect of the Callendar Square bug, presented a chronology very different to the real
one, and had the effect (whether intended or accidental) of giving a misleading
impression of the Callendar Square bug and its impact.
I find that the headline points omitted by Mr Godeseth are as follows:
1. The Callendar Square bug had existed “probably” (Mr Godeseth’s word in cross-
examination) since the inception of Horizon in 2000. There were numerous incidents
of it occurring prior to 2005.
2. By February 2006, Anne Chambers and others at Fujitsu knew that this bug “had
been around for years”.
3. It had an impact “most weeks”. Preparing a software fix was not seen as a priority.
4. The bug had directly impacted the branch accounts of at least 19 branches, and
possibly more, as the Anne Chambers’ spreadsheet listed the 30 branches as “these are
the majority”.
5. There were numerous effects of this bug, and at least 13 different ones identified in
the spreadsheet.
6. Even in 2006, Horizon Support was telling at least one SPM that they “could not find
any problem” with Horizon when the bug’s impacts were reported to it, and this was
recorded in documents linked by Fujitsu directly with the Callendar Square bug.
7. Fujitsu had not established the total number of branches affected by consulting the
events logs for the purposes of preparing Mr Godeseth’s witness statement, or
426.
427.
428.
POL00112816
POLO00112816
otherwise. To represent the spreadsheet prepared by Anne Chambers as being the
totality of branches affected by the bug is to ignore the text of the notes.
Finally on this point, the Post Office’s solicitors’ letters were obviously not factually
inaccurate. Nor can the Post Office’s own legal team have known about the Anne
Chambers’ spreadsheet until very early 2019, otherwise disclosure of such an important
document would surely not have been given only on 27 February 2019, only seven
working days before the Horizon Issues trial began. I do not know why the Anne
Chambers’ spreadsheet of 22 December 2015 was only disclosed to the claimants a
mere 7 working days before the start of the Horizon Issues trial, and it is not necessary
to speculate.
Although the Callendar Square bug in Legacy Horizon could, amongst other effects,
cause a receipts and payments mismatch, there is another bug in Horizon Online that
also had the same effect. This was called the Receipts and Payments Mismatch bug and
is the first entry in the bug table.
Mr Godeseth gave evidence about the Receipts and Payments Mismatch bug. This was
a problem at the cache. His information came both from Mr Jenkins and his own
investigations. He did not know about it at the time in 2010, and he did not in his witness
statement refer to what is in my judgment a very important document conceming this
bug, namely the Receipts/Payments Mismatch issue notes dated in the trial bundle index
17 October 2012, although it is likely to be a 2010 document as an associated document
from Mr Jenkins (trial bundle reference F/1000/1 and F/1001/1) is dated 29 September
2010. I will reproduce certain parts of the issue notes, as I consider them to be highly
relevant to the Horizon Issues. There were 10 attendees, including Mr Jenkins. The
document was marked Commercial in Confidence and is, according to the trial bundle
index, a memo of a meeting. Another document with the same date is headed
“Correcting Accounts for “lost” Discrepancies” and states “any branch encountering
the problem will have corrupted accounts”. The issues notes document stated:
“What is the issue?
Discrepancies showing at the Horizon counter disappear when the branch follows
certain process steps, but will still show within the back end branch account. This is
currently impacting circa 40 Branches since migration onto Horizon Online, with an
overall cash value of circa £20k loss. This issue will only occur if a branch cancels the
completion of the trading period, but within the same session continues to roll into a
new balance period.
At this time we have not communicated with branches affected and we do not believe
they are exploiting this bug intentionally. (emphasis added)
The problem occurs as part of the process when moving discrepancies on the Horizon
System into Local Suspense.
When Discrepancies are found during Stock Unit rollover into a new Transaction
Period, then the User is asked if the discrepancy should be moved to Local Suspense.
If the branch presses cancel at this point the Discrepancy is zeroed on the Horizon
System.
POL00112816
POLO00112816
Note at this point nothing into feeds POLSAP and Credence, so in effect the
POLSAP and Credence shows the discrepancy whereas the Horizon system in the
branch doesn't. So the branch will then believe they have balanced.
Tfat the next screen the rollover is completely cancelled, then no harm is done. However
if the Rollover is re-attempted at this point, the rollover will continue without any
discrepancy meaning Horizon doesn't match POLSAP or Credence.
This has the following consequences:
+ There will be a Receipts and Payment mismatch corresponding to the value of
Discrepancies that were "lost"
Note the Branch will not get a prompt from the system to say there is Receipts and
Payment mismatch, therefore the branch will believe they have balanced
correctly.
+ When the Branch begins the new Branch Trading period the discrepancies will show
at Zero, however the Receipts and Payment mismatch will carry over into the next
period.
Impact
+ The branch has appeared to have balanced, whereas in fact they could have a loss or
a gain.
* Our accounting systems will be out of syne with what is recorded at the branch
+ If widely known could cause a loss of confident in the Horizon System by branches
+ Potential impact upon ongoing legal cases where branches are disputing the integrity
of Horizon Data
+ It could provide branches ammunition to blame Horizon for future discrepancies.
Identifying the issue and forward resolution
The Receipts and Payment mismatch will result in an error code being generated which
will allow Fujitsu to isolate branches affected this by this problem, although this is not
seen by the branches. We have asked Fujitsu why it has taken so long to react to and
escalate an issue which began in May. They will provide feedback in due course.
Fujitsu are writing a code fix which stop the discrepancy disappearing from Horizon in
the future. They are aiming to deliver this into test week commencing 4" October. With
live proving at the model office week commencing 11'" October. With full roll out to
the network completed by the 21“ of October. We have explored moving this forward
and this is the earliest it can be released into live.
The code fix will on stop the issue occurring in the future, but it will not fix any current
mismatch at branch.
Proposal for affected Branches
POL00112816
POLO00112816
There are three potential solutions to apply to the impacted branches, the groups
recommendation is that solution two should be progressed.
SOLUTION ONE - Alter the Horizon Branch figure at the counter to show the
discrepancy. Fujitsu would have to manually write an entry value to the local branch
account.
IMPACT - When the branch comes to complete next Trading Period they would have
a discrepancy, which they would have to bring to account.
RISK- This has significant data integrity concerns and could lead to questions of
"tampering" with the branch system and could generate questions around how the
discrepancy was caused. This solution could have moral implications of Post Office
changing branch data without informing the branch.
SOLUTION TWO - P&BA will journal values from the discrepancy account into the
Customer Account and recover/refund via normal processes. This will need to be
supported by an approved POL communication. Unlike the branch "POLSAP" remains
in balance albeit with an account (discrepancies) that should be cleared.
IMPACT - Post Office will be required to explain the reason for a debt recovery/ refund
even though there is no discrepancy at the branch.
RISK - Could potentially highlight to branches that Horizon can lose data.
SOLUTION THREE - It is decided not to correct the data in the branches (ie Post Office
would prefer to write off the "lost"
IMPACT - Post office must absorb circa £20K loss
RISK - Huge moral implications to the integrity of the business, as there are agents that
were potentially due a cash gain on their system.”
(bold present in original)
429. I find this to be a most disturbing document in the context of this group litigation. It is
a 2010 document and issues between the Post Office and many SPMs concerning the
accuracy of Horizon had, for Legacy Horizon, gone on for a decade (2000 to 2010) and
these continued under Horizon Online (introduced in 2010). Under “Impact”, some of
the bullet points incorporate a summary of these issues.
“s The branch has appeared to have balanced, whereas in fact they could have a loss or
again.
+ Our accounting systems [ie Horizon or the Post Office’s] will be out of syne with what
is recorded at the branch
+ If widely known could cause a loss of confident in the Horizon System by branches
+ Potential impact upon ongoing legal cases where branches are disputing the integrity
of Horizon Data
+ It could provide branches ammunition to blame Horizon for future discrepancies.”
(emphasis added)
430. The attendees at this meeting included at least one member of Post Office (rather than
Fujitsu) personnel, Andrew Winn of POL Finance. There were obviously legal cases
going on at the time, hence the reference in the underlined bullet point to “ongoing legal
cases”. If these were criminal cases, the Post Office would be the prosecuting authority,
431.
432.
433.
POL00112816
POLO00112816
with certain important duties. If these were civil cases, the Post Office would be a party
with disclosure obligations. An affected branch would believe it had balanced its
accounts correctly; it would not have done so. There is an evident concern amongst
those at the meeting which is recorded in this document that this issue should not
become “widely known” in order to avoid causing “a loss of confidence in the Horizon
System”. Fujitsu do not seem to have been particularly prompt in either identifying the
problem or reacting to it.
Of the three solutions considered, the one that was adopted was issuing a debt
recovery/refund even though there is no discrepancy at the branch. Given one of the
Common Issues was the status of a Branch Trading Statement, which the Post Office
argued should legally have the status of a settled account with the SPM as agent, I note
the suggestion adopted with more than passing interest. The solution explains “even
though there is no discrepancy at the branch”. This means the Branch Trading
Statement would be correct, or in balance. The discrepancy would not be in the Branch
Trading Statement, it would be in POLSAP or Credence. That this is correct is shown
in an associated document from Mr Jenkins (trial bundle reference F/777/2) where he
states that “the data used for the BTS will also have a zero value for Discrepancies at
the end of the period.” BTS means Branch Trading Statement. A similar entry is in the
document at F/1000/1 which states at /2 “Note that if the bug was not present, then the
Discrepancy would have been transferred to Local Suspense and that would have been
cleared, so there are a number of things wrong with the BTS.” (emphasis added)
Further, an associated document entitled “Receipts and Payments Mismatch” states:
“1. The purpose of this note is to document a request that we have had from Post Office
in terms of presenting details of what happened as a result of a bug in HNG-X in
September 2010 which caused a Receipts and Payments mismatch and also resulted in
Discrepancies being lost.
The background to this is the fact that there was a BBC documentary broadcast_on
Monday 7" February 2011 reporting on postmasters being unhappy about being
pursued for losses by Postmasters on Horizon.
It should be noted that the issues described here relates to HNG-X (Horizon Online)
and that the implementation of the accounting mechanisms in the two systems is totally
different (but they do produce the same reports and support the same business
process).”
(emphasis added)
The two references — one to “ongoing legal cases”, the second to a BBC documentary
~ show that there was a distinct sensitivity within both the Post Office and Fujitsu about
keeping this information to themselves in order to avoid a “loss of confidence” in
Horizon and the integrity of its data, or a less complimentary way of putting it would
be to enable the Post Office to continue to assert the integrity of Horizon, and avoid
publicly acknowledging the presence of a software bug. The solution adopted was the
issuing of TCs to the relevant branches. There was no publicity given to the SPMs at
the time about the presence of this software bug in Horizon. An entry at F/1000/3 states:
434.
435
POL00112816
POLO00112816
“It should be noted that as Discrepancies are normally Losses, then a Lost Discrepancy
would normally work in the Branches favour and so there is no incentive for the branch
to report the problem. Also if we do amend the data to re-introduce the Discrepancy,
this will need to be carefully communicated to the Branches to avoid questions about
the system integrity.”
(emphasis added)
Mr Godeseth was asked about the lack of communication of this.
“Q. Are you surprised that this had not been communicated so people would just be
warned about it?
A. There was obviously a fear that subpostmasters may be looking to exploit this
because it gave -- there was a fear that people could see this as a way of defrauding the
Post Office.
Q. So concealing it from SPMs who were honest was justified because of the
expectation of dishonesty of subpostmasters in the network, in a nutshell?
A. In my view, this was a decision made by Post Office on how to manage this
particular bug. You could interpret it the way that you have put it.
Q. Do you agree with the way that I have put it to you?
A. I think I'm agnostic. I can see the -- I can see a rationale for not broadcasting this,
but equally, if the objective is to be totally open and honest and take the risk of causing
more chaos in the network then yes, I would have to agree with that.”
Indeed, the approach of the Post Office to notification of the existence of bugs to SPMs
is an illuminating one. The Post Office communicates with SPMs generally in a number
of ways. Methods that have been referred to in both of the Common Issues trial and
also the Horizon Issues trial include “Branch News”, a type of newspaper that was used,
and also “MemoView”, a pop up message that appears on a Horizon Terminal. During
the preparation of this judgment, in view of the fact that judgments are required to be
factually accurate, because dates sometimes become lost in the midst of very detailed
submissions, and also because the simplest way to arrive at a date when something has
been done is (sometimes) simply to ask the parties for a date or a reference, on 7 October
2019 the parties were jointly asked the following:
For each of the Dalmellington and Callendar Square bugs, could the parties please
indicate (by means of an agreed fact but only if that is possible, if not each party’s
contended for answer) their answers to the following two questions:
1. The date when the existence of each of these was communicated by the Post Office
to SPMs;
2. A trial bundle reference to a contemporaneous document, e mail or communication
from the Post Office demonstrating question 1.
436.
437.
438.
439.
440.
441.
POL00112816
POLO00112816
The parties could not agree on a date when the existence of these two bugs — Callendar
Square being an acknowledged bug in the bug table dealt with below (from the 2™ Joint
Statement of the two experts) and Dalmellington now accepted to be a bug — was
communicated to SPMs. The claimants’ answer to the two questions was the same for
each bug, namely that there was “no date in any contemporaneous document from the
Post Office in the trial bundle showing that such communication in fact took place and
identifying the actual content of any such communication.” The Post Office’s answer
was as follows:
For Callendar Square: “It has never been Post Office’s case that, once it was discovered,
the existence of the Callendar Square bug was communicated by Post Office to all
SPMs or communicated to SPMs who had been affected. Post Office's case is that it
was unnecessary and would have been inappropriate to do so.” A number of references
and further explanation then followed, stating that when “an SPM was affected by the
bug, this was detected and corrected in the ordinary course” and also that “In some
cases, SPMs who were affected by the Callendar Square bug may have been made
aware that there was a problem in Horizon which had caused an error in their accounts
which required correction.”
For Dalmellington: “It has never been Post Office’s case that, once it was discovered,
the existence of the Dalmellington bug was communicated by Post Office to all SPMs
or communicated to the SPMs who had been affected. Post Office's position is that it
was unnecessary and would have been inappropriate to do so.” It was also stated that
“when an SPM was affected by the bug this was detected and corrected in the ordinary
course” and that the Dalmellington bug was prevented, by effective counter-measures,
from causing even a single impact on branch accounts”.
The Post Office also stated that there were real disadvantages to notifying SPMs of this.
The Post Office relied upon passages in Dr Worden’s report that summarise reasons
why users of an IT system should not be given information about parts of that system
which they did not encounter in their daily work; that there is “no point in trying to
educate all the users in details and terminology of the system which will never concern
them”; that the best thing to do is try and fix a bug or defect, rather than “create some
new error message to the users”; that “automated messages from the system are only of
limited help to users”; and other points in similar vein. The Post Office also provided
documentary references to instances where “an SPM might have been made aware”
although these documents were not deployed in the evidence.
I will reproduce two entries from one of these documents, a PEAK with reference
PC0103864. This shows that on 3 June 2004 an SPM reported “that he had a problem
with some transfers yesterday, he was transferring stock and cash between the aa main
stock unit and the bb shared stock unit and although only one transaction shows for the
transfer out the transactions were transferred into the bb stock unit twice giving the pm
a discrepancy.”
The discrepancy in the cash account was £22,290 in total (shown in the entry for 8 June
2004) although the reconciliation error reported by the Host was £44,580. This was
finally resolved on 5 August 2004 with the issue of an error notice. There is nothing to
suggest that the SPM was told that this had been caused by a software bug.
442.
443.
444.
POL00112816
POLO00112816
The Post Office’s approach to this, in my judgment, entirely to miss the point. Fujitsu
knew, to take Callendar Square as an example, that this bug existed in Horizon. They
knew that it had affected branch accounts. It was not, as the Post Office puts it
“unnecessary and inappropriate” to notify SPMs of this. I have listed the points on this
bug at [425] above omitted by Mr Godeseth from his written evidence. Those same
points all lead to the same conclusion in my judgment, namely that the Post Office
ought to have notified, at the very least, all those SPMs whose branch accounts had
been impacted by this bug that this had occurred, and that it had occurred as a result of
a software bug. The fact that the integrity of Horizon data was a live issue at this time
should not have influenced the decision to notify SPMs of a software bug. Further, the
Post Office’s explanation in its submissions that SPMs had their accounts “corrected in
the ordinary course” is not a suitable phrase, unless by “ordinary course” one means
keeping the cause or reason for the correction hidden from the other party in the
accounting transaction, namely the SPM. Also, one is not educating users in the details
and terminology of the system (as suggested by Dr Worden) if one informs them that
there is a software bug in the system and the symptoms are as follows. This is relevant
to Horizon Issue 1(2), namely whether the Horizon IT system itself alerted SPMs of
such bugs, errors or defects as described in Issue 1(1) above and if so how, and also
Horizon Issue 9, which concerns transaction data and reporting functions available to
SPMs. It is also relevant to Dr Worden’s consideration of countermeasures, which he
considers includes vigilance by users (which means SPMs).
Mr Godeseth gave express evidence in his witness statement that the Receipts and
Payments mismatch bug occurred in September 2010. That date too was incorrect. The
issue notes refer to Fujitsu knowing about it far earlier, and Mr Godeseth accepted he
had seen this document before his cross-examination. That document even records
Fujitsu being taken to task by the Post Office about how long it has taken to react, as in
“We have asked Fujitsu why it has taken so long to react to and escalate an issue which
began in May. They will provide feedback in due course.” Not just the impression, but
the express text in Mr Godeseth’s witness statement, was to the effect that the bug was
discovered in September 2010 and almost immediately dealt with. That was far from
the case.
Mr Godeseth gave evidence about another bug, called the Local Suspense Bug, also in
Horizon Online. His witness statement quoted almost word for word from a
contemporaneous document prepared by Mr Jenkins. Almost all his information on this
bug came from Mr Jenkins. The bug caused entries from local suspense accounts in
2010 to be reproduced in two successive years. What was supposed to be only
temporary data was retained, and not deleted as it ought to have been, and the system
then used it again. It affected certain tables in the branch database and the archiving
strategy of deleted Stock Units. This affected 15 branches. The effects of the bug were
brought to the attention of the Post Office by two SPMs “with the largest discrepancies”
and Fujitsu became aware of it in January 2013. However, the note by Mr Jenkins of
15 May 2013 summarising it states that the problem was known about earlier than that:
“In April 2011 a problem was found with the archiving strategy related to Stock Units
that have been deleted in a Branch. A consequence of this is that some changes were
made to the archiving strategy on 3“ July 2011. An unintended consequence of this
change was that any Branch that deleted a Stock Unit at the end of 2010 which had
local suspense transaction in that Stock Unit before it was deleted were left in the table
445.
446.
POL00112816
POLO00112816
used for constructing the BTS. This meant that as Trading Periods cycle around each
year, these BTS records became visible in 2011 when the same Trading Period was
reached.
The effect of these old records was that after the BTS was produced an incorrect figure
was generated for the Opening Balance of the Local Suspense Account for the
following period. This amount corresponded to the value of the historical record. These
orphaned records were created between 16‘ November 2010 and 9" December 2010.
When the next Trading Period was balanced, then this incorrect Opening Figure would
result in the total value for Local Suspense being calculated incorrectly and the SPMR
being asked to make good an incorrect amount. It is at this point that transactions would
be generated into the audit trail reflecting the fact that the SPMR had cleared the Local
Suspense account for an incorrect amount. The audit trail operated correctly in the sense
that it accurately recorded the transaction on the system.
This problem was not reported to Fujitsu in 2011/12 and only affected a small number
of Branches and only for a single Trading Period. However the two branches with the
largest discrepancies did report the issue to Post Office Ltd who could see the impact
of the problem in their back end system and wrote off the loss or gain for the branch
but did not ask Fujitsu to investigate further.
At the same Trading Period in 2012/13, the problem re-occurred and this time one of
the affect Branches reported the problem to Fujitsu on 25" February 2013 (Peak
223870) resulting in a detailed analysis of this issue and finding the orphaned BTS
records. The root cause was determined by 28" February 2013 and a preliminary report
was sent to Post Office Ltd. A further update was sent on 14" March 2013 with a full
analysis of the issue and all the affected branches.”
(emphasis added)
Another software bug which Mr Godeseth gave evidence about what called the
Dalmellington Bug. This is also known as the Branch Outreach Issue or Bug, as it
affected what are called Outreach branches, which are those that have a core branch
(such as a rural or small branch post office) and an outreach branch, such as a mobile
post office van, or village hall somewhere else (where post office services would be
provided, say, one day a week). This “outreach” element is used, particularly in very
tural areas, to provide post office services to remote communities. A SPM needs to scan
or record a pouch from one branch (the core) into the outreach branch (the van or village
hall) to record that the contents of the pouch are not in the core branch but are in the
outreach branch. This is a branch to branch cash remittance, if the pouch contain cash.
Mr Godeseth had no first hand knowledge of the bug. As the claimants put it in opening,
this is a useful way of examining certain features in Horizon as the same person, the
SPM, controls “both ends” of the transactions.
The Dalmellington Bug related to Horizon Online and actually included two potentially
separate issues, and the combination of these. One related to what is called Forced Log
Out, when the Post Log On script was not correctly closed down and was left on the
stack of incomplete processes. The other related to the way the Pouch Delivery script
operated; because the Post Log On script had not correctly closed down meaning the
447.
448.
449.
450.
POL00112816
POLO00112816
stack was not empty, the Pouch Delivery script thought it had not finished and
attempted to repeat the last part of the script. This had the effect of recording the
remittance transactions and printing of receipts. This would lead to duplicate pouch
IDs. Pouches have bar codes that are scanned and each is supposed to have a unique
ID. The duplicate pouch ID would have a value attached to it — the sum in the original
pouch. It would however be recorded twice as a result of this.
What happened at Dalmellington was the SPM scanned £8,000 from her core branch
into her outreach branch. She was lucky in this sense, in my judgment, in that she
“controlled” both ends of the transaction. The transfer replicated four times. She had
Horizon receipts in her outreach branch of £32,000 (4 times £8,000) and therefore a
discrepancy of £24,000, as she had only transferred £8,000 in cash out of her core
branch into it. This was investigated by SSC and she was issued with a TC a few weeks
after it had occurred. This was caused by a software bug, and is now admitted by the
Post Office, although it describes it as having “transient impact”. That simply means it
was corrected.
When investigating this Fujitsu found 112 occurrences affecting 88 different branches
in the previous 5 years. They were as follows, taken from an internal Fujitsu
presentation dealing with this:
Feb 2010 to Jan 2011 65 incidents
2011 6 incidents
2012 9 incidents
2013 7 incidents
2014 9 incidents
2015 16 incidents
The same document showed fixes applied in April 2010, January 2011 and January
2016. It also showed only one call to SSC at Fujitsu in 2015, and none for any of the
years 2010 to 2014. The presentation also showed that some branches that were affected
were so-called “mediation branches” — in other words the SPMs would have been in
the aborted Second Sight mediation scheme — but that none of the dates when those
branches were affected matched the “mediation dates”, which means the periods for
which ARQ data was to be used in the mediations to analyse exactly what had happened
at each branch. There is nothing in the Fujitsu document used by Mr Godeseth for his
evidence to suggest that this analysis led to each branch being told of these software
bug occurrences outside the mediation date range, and the items were usually corrected
by means of TCs.
Mr Godeseth said that “Unfortunately this particular error is not subject to receipts and
payments problems because it could be -- it could be user doing something twice, it's -
- the bug had the effect of making it look as though a user was simply doing something
multiple times.” He said that “the issue was not raised with Fujitsu until 2013”, it was
put to him it was 2015 (based on the slides) but regardless of which of these is right,
451.
452.
453.
454.
455.
POL00112816
POLO00112816
the Fujitsu slides show only one call to Fujitsu in 2015, none for the other years,
although the reference to fixes in 2010 and 2011 suggests it was reported earlier. If
those fixes were intended to correct the bug they cannot have worked. Fujitsu
maintained that they investigated and found all occurrences, and the Post Office
maintained that they corrected any financial impact of this bug. Whether this is correct
or not in all cases, the evidence demonstrated that the effects of this bug were
experienced for the period from 2010 until 2016.
In his re-examination, some of which I have already referred to in the context of UNIX.
user and privileges above, Mr Godeseth also said that having seen the OCPs and OCRs
in cross-examination he accepted data had been deleted but did not consider it to be
transaction data. Initially he described it as “operational data” but said that was too
wide, and:
“There possibly ought to be a different definition for it because equally data in the
database which is telling you which stock unit, you know, what -- which TP [ie trading
period] you're in, whether a stock unit is in use, all of those are operational data which
do not go down and impact the transaction data, so for me, the definition of transaction
data -- I should be able to get from one opening -- one set of opening figures to the end
opening figures by going back to the transaction data in its absolutely raw form.”
He said it was what one went through to bring different totals together “to get to your
end position in a trading statement.” Regardless therefore of whether the data which he
accepted was being deleted is, or is not, properly described as “transaction data”, it
would feed into the end position in a branch trading statement. It would therefore, in
my judgment, have an impact upon branch accounts.
By the end of Mr Godeseth’s evidence, and by virtue of the way he answered questions,
the reality of the development and operation of both Legacy Horizon and Horizon
Online was far clearer than it had been before the Horizon Issues trial. I found Mr
Godeseth to be a truthful, credible and helpful witness. His evidence was of
considerable assistance; he did not always answer questions the first time he was asked,
but the final answer would be reached. Given his role throughout the whole of the life
both of Legacy Horizon and Horizon Online, I found his evidence in cross-examination
to be of central relevance to the Horizon Issues. His evidence in cross-examination was
also, in my judgment, broadly supportive of the claimants’ general case.
There are however two aspects of Mr Godeseth’s evidence — though not his
performance as a witness in the witness box - that were highly unsatisfactory and cannot
pass without comment. The first aspect relates to his written evidence. The totality of
his evidence, after his cross-examination, bore very little semblance to the picture that
had been portrayed in his witness statements. On some extremely important factual
matters, such as the dates when Fujitsu had become aware of a particular bug, or the
spreadsheet exercise by Anne Chambers prepared in 2015 (and disclosed in 2019), his
written evidence was simply directly wrong. On others, such as the headline points
omitted on the Callendar Square Bug that have been set out in [425] above, very
important central elements detrimental to Fujitsu were simply omitted.
Legacy Horizon had, as has been explained, started life originally as something rather
different to what it became, having initially been intended as a tri-partite project
456.
457.
458.
459.
POL00112816
POLO00112816
involving payment of benefits. It did not unfold in this way and became rather different.
Horizon Online also did not have a happy birth. The pilot for it had to be stopped, and
Fujitsu put it on what was called “red alert”. Mr Godeseth described this as “very
serious”. The biggest issue was with Oracle, which was what Mr Godeseth was working
on and hence knew the most about, but he explained that there were other problems
going on at the same time. Some of these problems were put to him — and it must be
remembered that this was a pilot scheme, with some problems to be expected — and
they included cash being short on one day by £1,000 because a transaction for £1,000
was not showing up on the online report facility; cash withdrawals being authorised on
screen yet the printed receipt being declined (the customer brought the cash back next
day having noticed); a similar problem with a cash deposit; and remming in figures all
being doubled up. These are all somewhat — and indeed markedly - similar to some of
the problems alleged by the different claimants’ witnesses in this litigation.
Not one of these problems was referred to in Mr Godeseth’s witness statements.
Witness statements are supposed to be accurate, and in a case such as this one with such
centrally important issues, accuracy is clearly important. Quoting only selectively from,
or wholly ignoring, contemporaneous documents prepared by (say) Mr Jenkins, who
was the extensive source of much of the evidence, is not only unhelpful, but it presents
an entirely misleading evidential picture. It is not necessary to consider further how
many personnel at Fujitsu may have assisted Mr Godeseth in producing such
documents. Their content was wholly misleading in their original written form.
Fortunately the cross-examination of Mr Godeseth led to a far clearer picture in so far
as his evidence is concerned.
The second unsatisfactory aspect which arose from his evidence is the approach of
Fujitsu as demonstrated in various documents, including the PEAKs and KELs, but also
in particular in the Receipts/Payments Mismatch issue notes. To see a concern
expressed that if a software bug in Horizon were to become widely known about it
might have a potential impact upon “ongoing legal cases” where the integrity of
Horizon Data was a central issue, is a very concerning entry to read in a
contemporaneous document. Whether these were legal cases concerning civil claims,
or criminal cases, there are obligations upon parties in terms of disclosure. So far as
criminal cases are concerned, these concem the liberty of the person, and disclosure
duties are rightly high. I do not understand the motivation to keep the type of matters
recorded in these documents hidden from view. There can be no proper explanation for
keeping the existence of a software bug in Horizon secret in these circumstances.
The degree to which either or both of Fujitsu and/or the Post Office, expressly or
constructively, knew exactly what and when, is for future trials in this litigation, and I
make no findings in that respect. They are not necessary in order to resolve the Horizon
Issues and I do not speculate.
In my judgment, however, there are sufficient entries in the contemporaneous
documents to demonstrate not only has Fujitsu been less than forthcoming in
identifying the problems that have been experienced over the years, but rather the
opposite. The majority of problems and defects which counsel put to Mr Godeseth, and
which were effectively admitted by him, simply would not have seen the light of day
without this group litigation.
460.
461.
462.
463.
POL00112816
POLO00112816
I do not know if this is because of concerns about the future of Horizon going forwards;
or for Fujitsu to protect its corporate reputation; or for some other reason. It is
unnecessary to speculate, and I do not do so. An internal Post Office IT risk
management document from 2017 stated that “the HNG-X platform is end of life and
is running on unsupported Windows software”, that it needs replacing, and also that the
"Branch counter technology is aged and unreliable, with frequent hardware failures,
resulting in branch disruptions." Mr Godeseth agreed with all of this. The unsupported.
platform is Windows NT4. As Mr Godeseth put it, “any technologist would tell you
that was too old”. It now runs on Windows 10 and is called HNG-A rather than HNG-
X. This change to Windows 10 occurred in February 2017 and the roll out was
administered by Computacentre, with Fujitsu providing the software. The Horizon
system of 2019 is a vastly different system than both Legacy Horizon and the earlier
version of Horizon Online, HNG-X.
The original Horizon Online system was introduced in 2010, and was even then
substantially based on the existing system, Legacy Horizon, itself at least 10 years old.
That had been introduced in 2000, and therefore it should not be controversial to
describe Horizon Online in its HNG-X form as old technology. There have of course
been upgrades and other improvements since 2010, and other software additions and
add-ons, but Horizon will by necessity require replacement at some point. I detected a
degree of sensitivity in the litigation generally about any plans for replacing Horizon
Online, as though the Post Office feared that any hint that it might be replaced would
be seized upon by the claimants (and/or assumed by the court) as accepting that the
criticisms of Horizon Online by the claimants would be corroborated by any Post Office
plans for its replacement. Such fears, if there are any, are groundless, at least from the
court’s point of view. It is inevitable that technology systems, particularly complex
systems from as long ago (in technology terms) as 2010 will need replacing at some
point. Technology moves at an increasingly rapid pace. There is reference in some of
the documents in the trial bundle, but not in the evidence, to something called “Thin
Client”, about which Dr Worden said he had no knowledge. He thought it was a cloud
based system. It is unnecessary to speculate about this either. It is sufficient to state
that, in my judgment, any plans there may be for replacement of Horizon Online in the
ordinary course of the Post Office’s wishes to improve its business are not something
that I need to consider in order to resolve the Horizon Issues.
The degree to which Mr Godeseth’s evidence affects my conclusions on the expert
evidence will be dealt with in Part L Overall Conclusions.
Very slightly before the end of Mr Godeseth’s cross-examination — about 30 minutes
or so — the Post Office sought to bring the Horizon Issues trial to an end by issuing their
recusal application seeking to remove me as the Managing Judge, and to have the trial
re-started at some indeterminate point in the future before another judge. There was
therefore a substantial interruption in the trial, and a period of 22 days before the
Horizon Issues trial resumed and the next factual witness for the Post Office, Mr Parker,
was called.
Mr Parker
464.
Mr Stephen Parker is also a Fujitsu employee, and is now the Head of Post Office
Application Support. He first started work on what was then called the Royal Mail
465.
466.
POL00112816
POLO00112816
Group Account in 1997, which was before the introduction of Horizon. He has
continued to provide support to the Post Office Account in the various roles he has
occupied at Fujitsu throughout the whole of Horizon’s life, by which he meant both
Legacy Horizon and Horizon Online.
He was originally a support consultant within the SSC, and stated that he was
effectively the deputy manager of the SSC, providing 3" line support. He was lead
designer and part of development team for the internal website providing the support
knowledge database (which is also called KEL), technical documentation management
and operational change control. He assisted the SSC manager in the provision of the
support service and operational management. He became the Manager of the SSC in
March 2010. The SSC later expanded and provided support services to other Fujitsu
customers, but the Post Office Horizon system is still the largest. He has between 25
and 40 staff in the SSC whom he manages. He said that he was deputy manager of the
SSC when Mr Roll worked there, although he did not have that title, and that people in
the SSC would know this because he would stand in for the manager in his absence and
make decisions on approving actions for him.
He had provided three different witness statements for the Horizon Issues trial, and his
2°4 and 3" statements responded to and ultimately corrected his earlier evidence, which
arose in respect of what Mr Roll said had been done whilst he was at the SSC, including
injecting transactions during the time of Legacy Horizon. These were done using
Riposte, the language then in use. During the Horizon Issues trial and just before the
issuing of the recusal application, the Post Office’s solicitors wrote a letter to the
claimants which stated the following. This letter was dated 20 March 2019, and was
sent the night before Mr Parker was originally due to be cross-examined, his evidence
then being delayed due to the recusal application:
“We understand from Fujitsu that the SSC has been carrying out further work to identify
any Peaks that show transactions being injected at the counter in Legacy Horizon in
addition to those referred to in paragraphs 29 and 30 of Mr Parker's second statement.
On Monday [ie the 6" day of the Horizon Issues trial] we learned that an SSC technician
has:-
+ searched for all KELs that mentioned "RiposteMessageFile", "Ripostelmport" and
"RiposteMessage';
+ collated the responsive KEL references (AChambers2226M, CObeng1029162824,
DSeddon822M, MYoung5043M, RColemanl250R, acha2340K, _ ballantj498J,
dsed344J, AHolmes3343J, DSeddon1753N, GMaxwell46141, PCarroll12541,
RKing5S135L, 81111, pear847S, wbra7 16s);
+ re-searched the Peak system for any Peaks which contained those KEL references;
and
+ identified the following Peaks: PC0105560, PC0106885, PC0063599, PC0063871,
PC0065796, PC0066061 and PC0083998 (which cross refers to PC0076029).
We understand that the Peaks referred to above relate to either:-
* correcting configuration data after a PinPad change; or
+ reference data.
We also learned that during this exercise the SSC technician has also identified three
examples of the marooned transaction scenario described in paragraph 38.2 of Mr
Parker's second statement: PC0068495, PC0099141 and PC0079196.”
467.
468.
469.
470.
POL00112816
POLO00112816
The search terms identified in the letter were different to the ones that Mr Parker had
used in his 2 witness statement. These were "RiposteMessageFile",
"RiposteMessage", "LPO Delete", "Marooned" and "RiposteObject put". The reason
this is important is the command “Ripostelmport”, used by the SSC as a search term
during the week commencing 18 March 2019, but not used for the purposes of Mr
Parker’s 2" witness statement somewhat earlier, is actually one of the commands in
fact used to inject transactions. In other words, the exercise carried out for the purposes
of the witness statement was wholly deficient, in that it did not search for one of the
specific terms used to inject transactions.
When asked to explain this in his cross-examination, and the corrections that he made,
Mr Parker stated the following:
“A I notified the legal team that Mr Simpkins had done some more work and as a
result of that we changed -- we -- this letter was actually generated.
Q And you didn't think it was important to correct paragraph 29 of your second witness
statement accurately to reflect the situation as it would have been on the day you were
due to give evidence, or indeed today?
A_ That wasn't a choice I made personally. I was advised that we generated this letter.”
(emphasis added)
He did not say who made the choice to which he referred, not to correct his witness
statement.
It certainly seems to me rather curious — at best - that the exercise performed for Mr
Parker’s 2" statement, by personnel at SSC who are expert in such things, should have
omitted using the search term “RiposteImport”. Given the purpose of paragraph 29 and
30 of Mr Parker’s 2"¢ witness statement was to identify how many times data had been
in fact injected, a search exercise that did not use “RiposteImport” was never going to
give the full picture. It is also rather curious that, having done that, the exercise that
was described in the letter of 20 March 2019 from the Post Office’s solicitors that
correctly used “RiposteImport” as a search term should have been done so very closely
before Mr Parker was originally due to be called, namely 21 March 2019. In re-
examination, Mr Parker said that “the thought occurred to him that he could add some
other search terms into the work that he had done previously” and “he came to me with
the data”. Why this thought should have occurred to the person it occurred to, who was
not a witness, and at the time that it did, could not be pursued. Mr Parker denied that he
was trying to conceal anything, and said he was just trying “to get the right data for the
court which can be difficult sometimes when you are going back fifteen years.” This
explanation overlooks that the decision not to use “RiposteImport” was one taken for a
statement prepared in November 2018, not one going back fifteen years.
Mr Parker had also given evidence in his 1“ witness statement in respect of Mr Roll’s
written evidence that Fujitsu could inject and affect branch transaction data without the
knowledge of the SPM. He had said the following:
471
472.
473
474.
POL00112816
POLO00112816
"As I explain below, those suggestions are incorrect and Mr Roll's account of Fujitsu's
actions and powers is inaccurate and misleading".
He explained this paragraph as follows in his cross-examination.
“A. In that paragraph I am trying to make the point that the suggestion that we
frequently changed branch transaction data without informing the branches that such
actions were being actually taken is not correct. "Frequently" is a subjective term but I
would not have described the rate at which we were changing branch transaction data
as "frequently".
One has only to read the original paragraphs as drafted to see that Mr Parker was not
originally taking issue with frequency, or how often such actions were taken, he was
stating in express terms that this could not be done by Fujitsu at all. For someone who
has effectively spent the best part of 20 years in SSC, rising to a very senior post, such
evidence must have been — and I find was - quite obviously incorrect. His explanation
of this was as follows:
“Q Now, given the change of your evidence in your later statements about the ability
remotely to access and inject transaction data, can you explain to the court what you
were saying you believed about that sentence? What should the court now read for those
words in the light of the three statements that it has before it?
A_ If 1 take the last sentence in isolation, which is what I think you are asking me to
do, then I don't understand how I apply it, because I am -- I have been simply trying to
say there that the frequency was not high, and that we would always involve the
Subpostmaster wherever possible if that sort of action was actually being taken. That's
what I'm trying to say by that.”
This was a very poor attempt at explaining away quite directly incorrect factual
evidence in a witness statement on a very important area in the litigation, namely
Fujitsu’s ability to inject transactions remotely. Mr Parker’s only recourse was to claim
he had not been saying what he plainly had been saying in his 1“ witness statement —
that Fujitsu did not have the power to do it, and that Mr Roll’s evidence to the contrary
was inaccurate and misleading.
Mr Parker also accepted that at the time he signed his 1‘ witness statement he had
known that Fujitsu had the power to insert transactions into individual branch counters
by using the correspondence server, a process Mr Roll described as “to piggy back
through the gateway”. It was put to him that he had been giving his evidence initially
from a position of not being very well informed about what could or could not be done.
He denied that, said “that would be wrong” and that in general terms he was confident
of the information he gave. The obvious conclusion therefore that I draw from this is
that Mr Parker chose specifically to give the impression in his 1“ witness statement that
Fujitsu did not have the power (the word Mr Parker expressly chose) to inject
transactions into the counter at branches, even though he knew that it did. This paints
him in a very poor light as a credible witness. There is also no adequate explanation for
why Mr Parker’s colleague should have decided, in the middle of the trial, to use the
“Ripostelmport” search term; equally, there is no adequate explanation for why this
search term was not used for the exercise for Mr Parker’s witness statement. Injection
POL00112816
POLO00112816
of messages into the counter has been an extremely central issue in the long running
dispute between the Post Office and its SPMs for a long time, and Mr Parker’s reliance
in his re-examination upon the passage of time and the difficulty of “getting the right
data” for the court is, in my judgment, a poor excuse.
475. A series of propositions was put to him about the degree of expertise expected of all the
members of SSC 3” line support. He did not agree with all of the points put, and in
particular did not agree that all those in the 3“ line support role would be expected to
have detailed knowledge of the system, based on both documentation and the inspection
of source code; or that they would be trained in the coding languages used within the
application. He was, however, left rather exposed in terms of the accuracy of this
evidence when it was shown that the points that he had been addressing — some of
which he would not accept - were taken from a Fujitsu internal document, entitled “End.
to End Application Support Strategy” marked Fujitsu Restricted and Commercial in
Confidence. Indeed, page 1 of that document identifies Mr Parker as the author of the
document, and it was approved by the Head of Application Services at Fujitsu, an
obviously very senior role. This refusal to accept his own previously drafted points in
what I consider to be an important contemporaneous document also paints him in a very
poor light as a credible witness.
476. He accepted that Mr Roll was conscientious, skilled and experienced, and that he had
given him a reference.
477. He, and others not identified in his statement (although he said who they were when he
was asked, Mr John Simpkins and Mr Mark Wright), had prepared a spreadsheet that
had been put to Mr Roll showing the total number of calls to SSC between 2001 and
2004. Mr Simpkins, for what it is worth, appears to have been the person behind the
exercise referred to in the letter at [466] above, so he was obviously involved both in
preparing evidence and doing searches. Mr Parker agreed that SSC during Mr Roll’s
time needed to address coding issues, even if they did not necessarily have an estate
wide impact, and he also agreed that the list of PEAKs that had been identified as
involving Mr Roll was only those where he had been the person who put the final
response on the PEAK, rather than where he had worked on the PEAK at all.
478. He also accepted that in categorising “software errors” he had excluded from the count
those that were categorised as Avoidance Action, Administrative Response and
Solicited Known Error.
479. He was taken to a PEAK in 1999 where an initial balance was multiplied twice “due to
a known software error” in the amount of a discrepancy of over one million pounds -
£1,082544,32 to be exact. The PEAK ran to 14 pages. 10 people worked on it including
those in Development as well as the SSC. He accepted that this would have involved
people looking into code. It was, however, categorised as “Administrative Response”.
Other PEAKs to which he was taken showed reconciliation errors, and a note by the
developer identifying that what were “up to 10 unnecessary reconciliation errors each
week” and that the PEAK was a regression of another PEAK; the investigation into it
led to the conclusion that a subsequent software release had not caught a fix. It stated
"Risks (of releasing and of not releasing proposed fix): Without this fix, there will be
possibilities of system errors at counter and while doing reversal transaction". This
software fix was “released to live”, which means the fix was actually written and
480.
481.
482
483.
484.
POL00112816
POLO00112816
introduced into the system. Mr Parker had not included this in his software category
either, because the final person working on it had categorised this as Administrative
Response.
In another PEAK, it was recorded that “the system is still playing up in that the screen
is hanging in the middle of transactions -- PM did transaction ... but left office for 1
hour -- when he came back the monitor had 141 first-class stamps on screen totalling
£38.07". Mr Parker accepted that this was not how the system was supposed to work,
but the same PEAK also recorded the SPM becoming upset because the SSC were
advising it was user error. The SPM wished to make a complaint about the person at
SSC and the PEAK records “we both feel that this PM is complaining unjustly”.
The same PEAK recorded that ROMEC engineers had been to the site, and had actually
seen the problem themselves, which were phantom transactions. It recorded “it’s not
just the PM’s word now”. Indeed, this PEAK is particularly concerning, in terms of the
accuracy of PEAK closures by Fujitsu. It records that one of the Fujitsu entries states:
“T now have pressing evidence to suggest that unwanted peripheral input is occurring,
the likely source being the screen. This has been seen at Old Isleworth ...When the PM
has been asked to leave the screen on overnight I have observed system activity
corresponding to screen presses happening with no corresponding evidence of
eitherroutine system activity or human interference. The way forward now is to
correlate this with Microtouch complied monitoring software and to this ends Wendy
is arranging for installation of the kit on Friday..."
However, notwithstanding this, the subsequent entries include “Phantom transactions
have not been proven in circumstances which preclude user error" and “In all cases
where these have occurred a user error related cause can be attributed to the
phenomenon"?
The use of the phrase “in all cases” in respect of attributing user error wholly ignores
that the ROMEC engineer is said to have seen the problem themselves, and it is not
simply the word of the SPM. It ignores the activity observed overnight, when the branch
was not even open. Also, there is no reason, given the lengthy entries and studying the
text throughout the PEAK in detail, why user error should be reached as a conclusion
at all. This appears to be used simply as an excuse to avoid an adverse conclusion to
Fujitsu.
Notwithstanding the detailed entries in the PEAK itself that in my judgment plainly
point to the contrary, the PEAK was still categorised as No Fault in Product.
In another, dealing with phantom transactions, one office was described as “a problem
office” that was accordingly being “monitored”. The PEAK recorded that “there have
been incidents which showed a possible correlation between system activity and
phantom transactions. These pointed to a touch screen problem and as a result the
screen was replaced with a resistive model. As this produced no measurable
improvement it has to be assumed that the problems were user related". This was also
closed as No Fault in Product.
486.
487.
488.
489.
POL00112816
POLO00112816
AKEL on phantom transactions stated "There have been several calls over the last few
months where Postmasters have reported phantom sales. Items appear by themselves
for which the PM has not pressed an Icon. These may be individual items or several of
the same item. Sometimes when no one has been near the screen items may appear".
These are similar problems being reported by a number of different SPMs. That alone
should have suggested, not as a matter of computer expert evidence but simply basic
common sense, that something was amiss. If a number of different and unconnected.
people all report the same symptom, then that could well suggest a common problem
to them all. However, the KEL, which ran from 2000 to 2004, suggested that the cable
between the screen and the base unit was the root cause. The dates on this KEL show
that, even on the face of the document, the problem was manifested for some years.
Another PEAK that showed a problem described as “non-zero trading position on office
rollover” was put to Mr Parker. This had been given a priority C status by Fujitsu, a
status he accepted meant it was not taken account in the Service Level Agreement with
the Post Office in liquidated damages thresholds (a type of financial penalty), which
was a feature of those software PEAKS given higher priority status A and B. Status C
also was non-critical. The description of the problem was described as “PM states that
he has rolled over but the system is telling him that he hasn't -- PM states that he is in
balance period 7 and he states he is getting the message 'wrong trading period MSG
31318 office balancing error and "Non-zero trading position ... on rollover of branch
by user WMC002 to trading period 8".
Mr Parker described himself as “hesitant” to describe this as a payments mismatch
issue, and said it was an “office balancing error”, even though later in the PEAK senior
personnel at SSC recorded “NBSC have ruled out user error” and also that "The
problem occurred on 15/09/10 when stock unit 02 rolled over. This was originally
reported, as per KEL, BALLANTJL759Q, in call PC0204537 ... but for some reason
the call was closed without being investigated. There is a known problem with the use
of the Cancel button during the stock unit rollover. This is fully described in KEL
WRIGHTM33145J".
(emphasis added)
He said he had not recognised the KEL number as being the payments mismatch KEL.
He accepted that in another entry in the PEAK the matter was stated to be “Severity:
Critical” — which meant it could not be status C, defined as non-critical — and also
included the entry “the branch accounts will need to be corrected.” This PEAK was
however, on 1 November 2010, closed as category “Avoidance Action Supplied”. Mr
Parker’s spreadsheet did not include “Avoidance Action Supplied” as constituting a
software issue. He claimed that he had used “the definition my experience dictated
rather than that in the document” and that “I forgot that that was exactly what I had
done in that circumstance when I prepared that a few months ago.” Finally, after the
question was put a third time, he accepted that it was a deliberate decision on his part
to depart from the definition in Fujitsu’s own documentation.
Another PEAK also identified as category C, non-critical, related to pouches being
remmed in twice, the duplicate pouch being £25,000 in value. The PEAK stated “this
might be a software issue” but also “PM is unsure if this is user error if it is a software
fault. NBSC have not confirmed if it was user error or if they can reverse it.” One entry
490.
491.
492.
POL00112816
POLO00112816
stated "POL have been informed of the error. Hopefully they'll issue a TC to correct
loss at the branch. The underlying problem caused by using previous button during or
just after scanning pouch barcodes, is still under investigation". This was closed as a
Solicited Known Error.
One PEAK which I found most illuminating, in terms of Mr Parker’s response to it
when he was asked questions, is PEAK 0229446. This is dated 17 November 2013,
hence is Horizon Online, not in the period of the pilot project and sufficiently after the
introduction of it in 2010 for it not to be put down as the sort of issue that might occur
in very early days of a system whilst minor issues were ironed out. The problem was
described in the PEAK as “PM doing cash declarations every now and again has major
loss.” Entries in the PEAK included "PM has had cash declaration problem throughout
the year and is losing a lot every now and again"; "He phoned up helpline told him can't
of declared properly. He states that he loses £2,000 then jumps suddenly to £4,000, one
point they lost £8,000 and is always losing money. PM stated that he has three post
offices, only happens on this site"; and then "Done a declaration this morning and had.
a £6,000 also. It shows no error message when doing it. No report prints out only print-
out of cash declarations". This passage of his cross-examination followed this:
“Q. Pausing there, if the PM is correctly reporting that, then that would be very serious
for the Postmistress or Postmaster, wouldn't it?
A. If it is being correctly reported, yes.
Q And it would not be the system working as it should.
A If we attribute it as a system fault, yes.”
This again was categorised by Fujitsu as Category C, non-critical. In a sense, those two
questions and answers sum up a large part of the case. The reason that this plainly, in
my judgment, does demonstrate a system fault is Fujitsu’s own PEAK recorded:
“Voiced NBSC quoted ref H18174172 to see what checks they have done themselves
before transferring call to horizon.
They stated they had trainers come into the office and ruled out user error.”
And
“NBSC states the user error checks were carried out by Auditors at the site and not over
the phone.”
(emphasis added)
This matter was assigned to Anne Chambers. The final part of her conclusion was
(having said that “there were no known issues that would result in the variance being
incorrect):
“T can't tell why the declared cash doesn't match the expected cash figure, the branch
need to make sure that what they have recorded on the system is correct, and investigate
the anomalies.”
493.
494
POL00112816
POLO00112816
(emphasis added)
This is simply passing the buck back to a SPM in a branch. How an SPM could
“investigate the anomalies” given they appeared to be generated by the Horizon system
is not explained, nor could it sensibly be.
She also closed the response with the category “Final — no fault in product” and closed
the call by defect cause “General — User”. I consider this PEAK to be an ideal
illustration of what I consider to be the most extraordinary situation at SSC.
1. Fujitsu routinely assigned non-critical Category C to matters that were really very
important in their own right in any event, but of extreme importance to SPMs whose
branch accounts were being directly affected. Mr Parker accepted that only Category A
or B attracted financial penalties. It is not possible, on the evidence before the court, to
conclude one way or the other whether this affected the categorisation adopted by
Fujitsu personnel.
2. Fujitsu would ignore information directly from the Post Office that demonstrated that
a SPM was not at fault. ROMEC engineers observing specific matters occurring, or in
this case the Post Office’s auditors ruling out user error, were simply ignored. This
effectively amounted to the Fujitsu SSC personnel positively over-ruling the Post
Office’s own non-SPM personnel, who were giving them essential information about
failures in the Horizon system.
3. Fujitsu would also close PEAKs in a variety of ways that entirely mischaracterised
the issue. This particular one was “no fault in product” and defect cause “general —
user”, in other words, caused by the SPM. This had been specifically ruled out by the
Post Office’s own auditors who had visited the branch.
Fujitsu do not, on the face of these documents shown to Mr Parker, appear to me to
have properly and fully investigated these myriad problems, nor do Fujitsu appear to
have categorised such incidents correctly. They also seem to have glided away, in their
investigations, from concluding that there were any issues with the software wherever
it was possible for them to do so, regardless of evidence to the contrary, an approach
that has been carried into the Fujitsu evidence for the Horizon Issues trial. In re-
examination Mr Parker was asked about the way he had done his exercise, and also the
point was put to him that the PEAKs to which he was taken were outwith the period
when Mr Roll was employed at SSC. The implication of this was that Fujitsu may, or
would, have been more accurate in the period in question rather than in later periods.
Mr Parker said he could not possibly read all the PEAKs, that he considered his data to
be accurate, and although “you may find a few Peaks where the response code does not
tally with the document” given the number of PEAKs involved “human beings will
make those errors”.
I reject that explanation, such as it is. I do not consider that Mr Parker was interested in
accuracy in any of his evidential exercises, and I do not consider that he was objective
in the way he presented his evidence, although he sought to give the impression that he
was. I consider that Mr Parker, and the team who assisted him, sought to portray the
Horizon system — Legacy Horizon and Horizon Online — in a light as favourable to
496.
497.
498.
POL00112816
POLO00112816
Fujitsu as possible, regardless of its own internal evidence to the contrary. Mr Parker’s
spreadsheet plainly did not calculate all the categories that were used by Fujitsu for
problems with the system, and sought to give the impression that Mr Roll had been.
involved in far fewer incidents of this type than were the case.
This conclusion about Mr Parker’s lack of accuracy, is supported by the changing story
of Fujitsu about remote injection of messages; the need for correcting supplementary
statements, including from Mr Parker himself; his admission that he knew Fujitsu had
the power to do this, even though his 1* statement expressly said otherwise; the attempt
to shift the blame for this on to Mr Roll; the assertions in his subsequent statement that
Mr Roll had been unclear; and also by his attempt to portray his original evidence as
dealing with frequency, not power or ability. My conclusions are also supported by his
“deliberate decision” to depart from Fujitsu’s definition in its own internal document,
which I have explained at [488] above. I also consider that the fact that someone such
as Mr Parker had been on the Post Office Account for such a very long time, in his case
throughout the whole life of Horizon, means he was hardly best placed to be objective
in this type of evidence.
In my judgment, the exercise done by Mr Parker to demonstrate the number of PEAKs
that Mr Roll worked on involving software, included in the spreadsheet of
categorisation he and others prepared, is of no evidential value whatsoever. It relies
upon the categorisation of the PEAKs performed by the Fujitsu personnel themselves,
which in the vast majority of documents used in the trial were simply wrong or
misleading; and it also used a different definition to the one included in Fujitsu’s own
documents such as the PEAKs above. He accepted that he had made a deliberate
decision to use a definition other than the one in the Fujitsu documents, which he
justified by saying was based on his experience. I find that he was, by means of the
exercise he was involved in, trying to downplay the extent of Mr Roll’s involvement in
software issues, and also trying to downplay the extent of software problems that were
experienced on Legacy Horizon during Mr Roll’s employment at SSC.
Although Mr Parker agreed, as it was put to him more than once, that accuracy is
important, I do not consider his evidence in his witness statements to have been
remotely accurate, even though he stoutly maintained that it was. He continued to
maintain this in his re-examination, even though by then he can have been in no doubt
that he had departed from Fujitsu’s own definition — which he had said he had forgot
about. I found him a very unsatisfactory witness, who presented in his witness
statements a misleading and one-sided sanitised version of actual problems and events
that Fujitsu had experienced in his time at SSC. Although he provided a table with one
of his witness statements that provided a detailed commentary on a large number of
PEAKs and KELs, that commentary appearing in a column headed “Fujistu’s
Comments” with two columns, “Response to Mr Coyne” and “Financial Impact on
branch accounts” — that latter almost always “no impact” the entries in those columns
are almost entirely self-serving by Fujitsu and I find them to be of no evidential value.
The text of the actual PEAKs and KELs themselves are far more useful, obviously
entered at a time when the Fujitsu/Post Office position in the litigation was not at the
forefront of the different authors’ minds, and these are what for the most part the experts
have considered. Mr Parker’s table is of no assistance.
499.
POL00112816
POLO00112816
The degree to which the evidence of fact affects my conclusions on the expert evidence
will be dealt with in Part L Overall Conclusions.
Mr Membury
500.
S01.
502.
503
504.
Mr William Membury is another Fujitsu employee, namely a Fujitsu Central Quality
Partner and he is specifically focused on the Post Office account. He was taken ill
during the early part of the Horizon Issues trial, and when (after dismissal of the recusal
application) the trial resumed with outstanding factual evidence on 11 April 2019, he
was not well enough to be called in person. His witness statement was dealt with as
hearsay evidence and given the circumstances the claimants agreed to extend time for
the relevant notice to be given under CPR Part 33.2.
Mr Membury has worked for Fujitsu since 1998 and became the Quality Risk and
Compliance Manager for the Post Office account in 2011. He has overseen multiple
audits of the Horizon system. Given the timing of this, these would have all been on
Horizon Online. In 2014 he became a Central Quality Partner specifically focusing on
the Post Office as well as the Payment Card Industry (PCI) Security Standards Subject
Matter Expert (SME) for Fujitsu UK and Ireland. In 2015 he became the PCI SME for
the Europe, Middle East, India and Africa regions. He returned to the Fujitsu Post
Office Account as Head of Quality and Compliance in October 2018.
He described his statement as an overview of the auditing and development regimes in
place in relation to Horizon to the extent that those points were not covered by the
disclosed technical documents. The statement was said to provide a high-level overview
of the audits to which Horizon had been subject since 2000; Fujitsu’s audit
methodology and his role in the audits; and to describe the development of changes to
Horizon and how that tied in with or into the audit process.
Mr Membury’s statement was very brief; it ran to only 5 pages of text. It also omitted
some very material matters. For example, he stated that the Post Office and Fujitsu
agreed in 2010 that Horizon Online would be audited against the International Standard
on Assurance Engagements (ISAE) No. 3402 assurance standard (Horizon ISAE 3402
Audit), entitled "Assurance Reports on Controls at a Service Organization", which was
issued in December 2009. This was developed to provide an international assurance
standard for allowing public accountants to issue a report for use by user organisations
and their auditors (user auditors) on the controls at a service organisation that are likely
to impact or be a part of the user organisation's system of internal control over financial
reporting. It provides assurance that the solutions in place to manage financial
transactions are appropriate.
However, he completely omitted any reference to the Ernst & Young Management
Letter for the year ending 27 March 2011 which has been referred to in [264], [393]
and [524] above, which included concerns about privileges and other critical matters in
relation to IT. Indeed, he made only two references to Emst & Young. One was to state
that:
“Ernst & Young have carried out the Horizon ISAE 3402 Audit since the 2012/2013
financial year (preparations for the audit began with Post Office in 2011)”
506.
507.
POL00112816
POLO00112816
The second was a reference to how Ernst & Young performed the ISAE 3402 audit for
the period 1 April 2014 to 31 December 2014. This is, in my judgment, a serious
omission.
He could not be asked any questions about his evidence; although that was not his fault,
this goes to the weight which I attach to his evidence in any event.
I consider that Mr Membury’s evidence is of limited, if any, assistance in resolving the
Horizon Issues. It does however continue the very one-sided picture presented by all
the Fujitsu witness statements, which was to omit any reference to important
contemporaneous documents that criticise or demonstrate deficiencies with Horizon.
The absence of Mr Gareth Jenkins
508.
509.
It is entirely a decision of the parties which witnesses they choose to call in any
proceedings in respect of any evidence. The position of one person, however, who did
not appear in the Horizon Issues trial, must be considered in more detail than would be
usual, as the claimants make considerable complaint about this. The person in question
is Gareth Jenkins, a senior Fujitsu employee who, although he retired recently, was
obviously widely available to the Post Office and the fount of a great amount of
information to both the Post Office’s witnesses of fact, and also to Dr Worden (although
he was not separately identified in Dr Worden’s “sources of information” paragraphs
in his 1“ Report). The fact that he provided information to Dr Worden emerged during
the latter’s cross-examination. Mr Jenkins had previously given expert evidence for the
Post Office in some of the criminal prosecutions of SPMs, in particular that of Ms
Misra, to whom I have referred above, who was convicted in Guildford Crown Court
in 2010.
When the Post Office served its evidence of fact, there was no witness statement from
Mr Jenkins, although many of their witnesses relied upon him as their source of
information, he was referred to very often, and he obviously knew a great deal about
Horizon. The extent and way in which Mr Jenkins had been closely involved was
explained by Mr Godeseth in his cross-examination. Mr Godeseth had, in respect of the
receipts and payments mismatch, originally stated in paragraph 42 of his 2° witness
statement that 60 branches were affected. He had corrected this to 62, a factual
correction that was specifically made by him. The following passage of evidence is
relevant to Mr Jenkins’ involvement in this.
“A. No, no, I didn't do a calculation to come up with the 60. I was quoting from other
people.
MR JUSTICE FRASER: Someone just gave you the 60, did they?
A. I thought I was quoting from other people. I -- Gareth even said to me that in my
statement I had said "approximately 60", so I was not -- clearly I didn't because the
statement here doesn't contain that word. I had rather hoped it had when this was first
brought to my attention, but no, I certainly did not do any specific calculation to come
up with the 60 that I put into my original statement.
MR GREEN: Did Gareth explain the change to you from 60 to 62?
510.
S11.
POL00112816
POLO00112816
A. No.
Q. But how did he come to tell you what was in your original statement? What was
that conversation?
A. I picked up the number -- to be -- my objective in this was to explain to the court
the symptoms of the bug and how -- the technical aspect of it. I did not pay particular
attention to getting the detail on how many branches were affected, correct.
Q. Okay. So you have spoken to Gareth since your statement?
A. I don't think I have spoken to him about this in particular. I was -- as I say, when
Gareth had said I had originally said "approximately 60" I was thinking that was quite
neat, but that's not the case.
Q. Well, you said "Gareth even said to me that in my statement I had said
‘approximately 60", so he must have said that to you after your statement had been
filed?
. It was a comment in a document that we were exchanging.
. But you hadn't spoken to him about remote access since your first statement?
No.
Qo > 2 >
. Why have you stayed off that topic with him?
A. Oh, sorry, this was just acomment. We have been exchanging documents, we have
been commenting on documents, so it was not a particular conversation. It is merely a
case of Gareth had commented on this when it was pushed back to us that I had
originally said 60 and actually the answer was different.”
There is nothing wrong with Mr Godeseth correcting the number of affected branches
from 60 to 62. There is also nothing wrong with him reproducing a number given to
him by someone else, as long as it was properly identified that the information came
from someone else, who ought to be identified (which it wasn’t in his statement, but
was in at least outline terms in his cross-examination). The reason for reproducing these
passages is simply to identify the extent to which Mr Jenkins was so closely involved
in the litigation, a point relied upon by the claimants. There were a great many
references throughout Mr Godeseth’s evidence, written and oral, about information he
had obtained from Mr Jenkins.
When the Post Office served their evidence of fact, the claimants had asked the Post
Office why there was no statement from Jenkins, whether Mr Jenkins was available to
give evidence, and also whether he was involved as one of the “shadow experts” whose
name was, or names were, never disclosed, but the sizeable costs of whom had
originally appeared in the Post Office’s costs budget (prior to its removal as its own
counsel recognised there was no basis for it to be included as recoverable costs in the
512
S13
514.
POL00112816
POLO00112816
litigation). No explanation was given for Mr Jenkins’ absence in response to these
requests, or in evidence in the trial.
There the matter might have rested. However, in the Post Office’s written closing
submissions, an explanation of sorts was for the first time provided. This was in the
context of two matters: firstly, by way of explanation of Mr Godeseth’s evidence, and
potentially to downplay its impact; secondly, in relation to the claimants’ complaints
about the second hand nature of some of the Post Office’s factual evidence because in
large part this had emanated from Mr Jenkins. This explanation by the Post Office
included the following passages in its submissions:
“144. [The claimants] understandably complain that Mr Jenkins and the other source of
Mr Godeseth’s information could have given some of this evidence first hand.
However:
144.1 Taking into account that Professor McLachlan’s evidence specifically addressed
things said or done by Mr Jenkins in relation to the Misra trial, Post Office was
concerned that the Horizon Issues trial could become an investigation of his role
in this and other criminal cases.
144.2 Moreover, Post Office was conscious that if it only adduced first hand evidence
in the trial, it would end up having to call more witnesses than could be
accommodated within the trial timetable.
144.3 Furthermore, so far as Post Office was aware, the relevant parts of Godeseth 2
were most unlikely to be controversial. For example, the Misra trial was a matter
of public record, the four bugs were covered by contemporaneous
documentation and Post Office had no reason to doubt Fujitsu’s account of the
documents it held.”
In a footnote to paragraph 144.2 of the closing submissions, the Post Office added
“.....As noted above, had its witnesses only given first hand evidence, Post Office
estimates that some 34 additional witnesses would have been required.”
The following are relevant in my judgment:
1. Of primary importance is the principle that it is for each party to decide whom to
call, and what evidence they seek to obtain from any particular witness.
2. Mr Jenkins is an important and central person so far as the operation, efficacy and
robustness of Horizon is concerned, and also in respect of the number of incidents over
the years that have led to PEAKS, KELS, problems and fixes. The point in the footnote
about a potential further 34 witnesses is not relevant to whether he was called by the
Post Office. The Post Office did not at any stage apply to the court and explain that
further time was required for the length of the Horizon Issues trial for this specific
reason. However, given the primacy of point (1) above, it was entirely for the Post
Office to decide whether to call him or not, how many witnesses to call and who those
witnesses were to be.
S15
516.
POL00112816
POLO00112816
3. The Post Office chose to proffer a reason for his absence in closing submissions.
They were not obliged to explain. However, the reason in paragraph 144.1 is not a valid
reason for his absence. The claimants would have been entitled, in cross-examination,
to put to Mr Jenkins prior inconsistent statements of his, if there were any, including
statements he may have made in earlier proceedings (if they were inconsistent with his
evidence in the Horizon Issues trial). That is permitted as part of cross-examination if
a witness’ evidence is in issue. However, the Horizon Issues trial would not have
become “an investigation of his role in this and other criminal cases” — the Horizon trial
was about the Horizon Issues. Also, this type of cross-examination would only have
arisen if his statement(s) in the Horizon Issues trial were not consistent with those he
had made in support of the prosecutions in which he had been involved.
4. Regardless of the validity of the explanation given by the Post Office, he was not
called as a witness, and it merits repetition that it is entirely a matter for the Post Office
which witnesses it called, and which it did not. No speculation is permitted as to what
evidence he might have given in the Horizon Issues trial.
5. There is authority that in certain circumstances the court can draw adverse inferences
from the absence of a witness. The claimants do not invite me to draw adverse
inferences from his absence, and I do not do so.
6. The fact that the Post Office chose to advance certain evidence in the Horizon trial,
that emanated from Mr Jenkins, by means of another witness saying “Mr Jenkins told
me that.....” means that the claimants were deprived of the opportunity properly to test
that evidence. It also emerged in cross-examination, although not in his written report,
that Dr Worden has also obtained certain information from Mr Jenkins. By dealing with
the material in this way, and having deprived the claimants of the opportunity to cross-
examine Mr Jenkins, the weight to be given to evidence emanating from him is less
than it would be otherwise.
There are two consequences of the Post Office choosing to adduce such evidence in
this way. The first is that Mr Jenkins was protected from any cross-examination on such
evidence; the second is that such evidence would not be given the same weight by the
court as though Mr Jenkins had given it himself. The Post Office would have known
this in advance, as this is wholly conventional. How these different factors were
weighed by the Post Office and its legal advisers in deciding whether or not to call him,
is solely a matter for the Post Office. The end result is that Mr Jenkins did not appear
and was not cross-examined.
However, I reject the suggestion that Mr Godeseth was insufficiently knowledgeable of
the matters upon which he was cross-examined for his evidence to be of significant
evidential weight. The Post Office’s Closing Submissions sought to dilute the effect of
Mr Godeseth’s cross-examination by, at the same time, maintaining that Mr Jenkins
would have been a far better person to have answered those questions; yet explaining
why he had not been called. These submissions, though understandable from the Post
Office’s point of view in forensic terms only, have little force. A clear choice was made
by the Post Office not to call Mr Jenkins as a witness; that was their right. They are
entitled to call such evidence as they deem necessary. They cannot, however, have
matters both ways, and try to downplay Mr Godeseth’s evidence because he was called
to deal with certain matters, and Mr Jenkins was not. They have to abide by the
POL00112816
POLO00112816
consequences of their choice of witnesses, in terms of the evidence now before the court
to resolve the Horizon Issues trial.
Inaccurate Statements by the Post Office
S17
518.
519.
Litigation in this jurisdiction is adversarial. What that means is each side advances their
own case, and challenges or rebuts the case of the other side, and the court assesses the
evidence before it put there by the parties, makes necessary findings on the facts and
applies the law. Parties in civil litigation will usually advance their own evidence of
fact, and although mechanisms are available for compelling other witnesses to attend
and answer questions, there are pitfalls in doing so and issuing a witness summons
(what used to be called a subpoena) is a relatively rare step. The court can only therefore
resolve the case on the evidence before it, although it can draw inferences, that is to say
common sense conclusions, on the evidence.
The Post Office has no obligation to assist the claimants in advancing their case against
it. However, the Post Office has maintained publicly that it was seeking to be
“transparent” about Horizon, and prior to the litigation it made certain public statements
in relation to the increasing disquiet on the part both of SPMs, and others who became
involved either on their behalf (such as some Members of Parliament) or in an
investigative way (such as the BBC Panorama programme and other journalists). These
statements by the Post Office routinely and strongly insisted that there was nothing in
the criticisms being levelled at the accuracy of Horizon, and that losses that were shown
in SPMs’ branch accounts were caused either by carelessness or dishonesty on the part
of the different SPMs who experienced what they considered to be unexplained
discrepancies and losses. In 2015 a Parliamentary Select Committee held hearings into
the mediation scheme that had been set up to attempt to address the claims by SPMs in
respect of Horizon. I have not considered any of the evidence submitted to the Select
Committee, or any details of the mediation scheme (which was being conducted by
Second Sight) and refer to them here simply as a matter of chronological record.
The claimants rely upon a public statement released by the Post Office in 2015 after the
BCC Panorama programme. Part of the statement was headed “The Horizon System”
and stated:
“The Horizon Computer System
Horizon is robust and eff ective in dealing with the six million transactions put through
the system every day by our postmasters and employees at 1 1,500 Post Office branches.
It is independently audited and meets or exceeds industry accreditations. There have
been 500,000 users of the system since it was introduced.
Nevertheless, rigorous re-investigations were undertaken into claims made by 136
mainly former postmasters that the system caused losses in their branches.
There is overwhelming evidence that the losses complained of were caused _by user
actions, including in some cases deliberate dishonest conduct. The investigations have
not identified any transaction caused by a technical fault in Horizon which resulted in
a postmaster wrongly being held responsible for a loss of money.
520.
521.
522.
523.
POL00112816
POLO00112816
There is also no evidence of transactions recorded by branches being altered through
‘remote access’ to the system. Transactions as they are recorded by branches cannot be
edited and the Panorama programme did not show anything that contradicts this.”
(emphasis added)
Another document from 2015 upon which the claimants rely is an internal e mail chain,
which originated from Paula Vennells, the Chief Executive of the Post Office, on 30
January 2015. This was prior to her appearance before the House of Commons Select
Committee in February 2015. She posed the following question in an e mail sent
internally to Mark Davies and Lesley Sewell, both of the Post Office:
“Dear both, your help please in answers and in phrasing those answers, in prep for the
Sc:
1) "is it possible to access the system remotely? We are told it is."
What is the true answer? I hope it is that we know this is not possible and that we are
able to explain why that is. [need to say no it is not possible and that we are sure of this
because of xxx and that we know this because we have had the system assured.”
(emphasis added)
Ms Vennells obviously needed to know whether the answer matched her understanding,
which was ~ as she put it, both “I hope” and “I need” -- that it was not possible to access
the system remotely. This query was passed on through various people, including at
one stage from James Davidson who has both a Fujitsu and Post Office e mail address,
who answered to Mark Underwood:
“As discussed, can you hook up with Kevin to review what answers have already been
provided to second sight as this should form the Post Office response.”
The answer was provided by Mark Underwood on 30 January 2015:
“Can Post Office or Fujitsu edit transaction data without the knowledge of a
Subpostmaster?”
Post Office confirms that neither it nor Fujitsu can edit transaction data without
the knowledge of a Subpostmaster.
There is no functionality in Horizon for either a branch, Post Office or Fujitsu to edit,
manipulate or remove a transaction once it has been recorded in a branch's accounts.
The following safeguards are in place to prevent such occurrences:”
(various matters are then listed in the remainder of the e mail)
(bold present in original)
This then was subject, in the e mail chain, to a degree of refinement. Kevin Lenihan
forwarded the e mail onwards to Mark Underwood and others, and stated:
524.
POL00112816
POLO00112816
“Mark / Mel,
James has had a look at your answer to Q1. And thinks there’s too much detail for Paula
~ this was written for a different type of audience. He has captured the same points but
in a more appropriate format :-
He states:-
Having looked again at the request from Paula, it appears that the fundamentals around
this question (remote access) are not understood. I suggest that Paula is briefed along
the lines of the following.
1) No transaction data is held locally in any branch. Transactions are completed and
stored in a central database and copies of all data is sent to a secure audit database.
2) Sub-postmasters directly manage user access and password setting locally so system
access (to create transactions) are limited to approved local personnel only who are
responsible for setting their own passwords. Users are only created following an
approval process which requires authorisation by the sub-postmaster. All subsequent
transactions are recorded against the id used to log on to the system.
3) Once a transaction has been completed, there is no functionality (by design) for
transactions to be edited or amended. Each transaction is given a unique number and
‘wrapped’ in a digital encryption seal to protect its integrity. All transactions are then
posted to a secure and segregated audit server.
4) On approval, there is the functionality to add additional transactions which will be
visible and have a unique identifier in the audit trail. This is extremely rare and only
been used once since go live of the system in 2010 (March 2010)
5) Support staff have the ability to review event logs and monitor, in real time, the
availability of the system infrastructure as part of standard service management
processes.
6) Overall system access is tightly controlled via industry standard ‘role based access’
protocols and assured independently in annual audits for ISO 27001, Ernst and Young
for IAS 3402 and as part of PCI audits.”
(emphasis added)
T am unaware of the final answer Ms Vennells received internally to her fairly
straightforward question, or of what she told the Select Committee. None of the people
through whom this e mail chain passed were called as witnesses in the Horizon Issues
trial. However, the point in bold in the e mail quoted at [522] above is now, as a result
of this litigation, known to be simply not correct. Such editing can take place without
the knowledge of the SPM. All and any previous statements by the Post Office stating
that this cannot be done are simply factually wrong. The more detailed points at (1) to
(6) of the e mail quoted at [523] provide only selected information and, in my judgment,
are off the point. They have the effect of obscuring what ought to be a simple answer,
with a level of detail which makes the actual answer highly unclear. The answer at (1)
applies only to Horizon Online. For Legacy Horizon, transaction data was held locally
in the first instance, on what was called the counter. This was explained very clearly by
the Post Office in its opening, including in its oral opening submissions. The first
sentence of (3) in the list in the e mail is not correct. The statement in (4) — which may
refer to the Transaction Correction tool — is correct (or at least, not incorrect) but only
so far as it goes, and in my judgment crucially omits the ability to edit transactions
without visibility to the SPM. It also fails to consider the existence of APPSUPP
permissions at all, which every member of SSC had. The experts are agreed that the
APPSUPP permission was very powerful and enabled a SSC employee to do pretty
525.
526.
527.
POL00112816
POLO00112816
much whatever they wanted, to paraphrase the Post Office’s own expert. It can also be
seen, particularly by the evidence that emerged in the cross-examination of Mr
Godeseth, that in a great many cases the PEAKs show that user error was attributed to
problems that were not user error at all, and that others within the Post Office (for
example its own auditors and ROMEC engineers) had either witnessed for themselves
and/or satisfied themselves that user error was not the cause of specific issues, but
Fujitsu decided they were user error in any event.
The extent of permissions enjoyed by SSC was the subject of some express concern by
Ernst and Young (the Post Office’s own auditors) in 2011, and although the position
may have been regularised by 2015, certainly prior to 2011 the situation was very
different. The e mail also states “he has captured the same points” yet the most
important point to which an answer is given, or certainly the answer given in bold in
the answer at [522] above, is not corrected, even though it is plainly wrong. The overall
tenor of what Ms Vennells received, if it reflected the full content of this e mail chain,
would have been that the Post Office and Fujitsu could not edit transaction data without
the knowledge of the SPM. The evidence in this trial is directly to the contrary, and
Fujitsu could. It is important to bear in mind the distinction between what is called
Legacy Horizon, and Horizon Online. The term “counters” is a Legacy Horizon term.
This is a term for the way the data was held at the branch; these were discs (including
mirror discs) at the branch which contained the data. The discs were contained in
terminals on the counter. In Horizon Online, the data was held at a central database or
server (the phrase used by the Post Office in its oral opening).
This tenor of these public statements made by the Post Office was maintained into, and
as part of, the group litigation. For example, certain statements were made in the
Generic Defence in these proceedings. A party’s Defence is the formal pleading
document which sets out a party’s legal position in the litigation. Since the reforms in
civil litigation adopted after what are called the Woolf Reforms, which led to the Civil
Procedure Rules in 1999 which govern all civil litigation in the jurisdiction, such
pleadings must be accompanied by a Statement of Truth. These are, for larger
companies, usually signed by directors or very senior personnel, which occurred in this
case. That statement attests to the truth of the matters stated in the pleading.
This issue was dealt with in the pleadings in the following way. By this point the Post
Office had accepted that there was some limited ability to do that which it had
previously stated was not possible. In the Generic Statement of Case, the claimants set
out the following:
“25. Further, the Defendant was, by itself and/or via its agent Fujitsu, able to alter
branch transaction data directly and carry out changes to Horizon and/or transaction
data which could affect branch accounts.
26. However, the Defendant has made public statements in the following terms:
26.1. "Horizon does not have functionality that allows Post Office or Fujitsu to edit or
delete the transactions as recorded by branches" (Defendant's published reply to Second
Sight's Briefing Report Part Two, concerning a review of the Horizon system); and
528.
POL00112816
POLO00112816
26.2. "Transactions as they are recorded by branches cannot be edited" (Defendant's
published reply to the BBC Panorama documentary in relation to Horizon).
27. These statements were untrue, as the Defendant now admits.”
These paragraphs were pleaded to by the Post Office in its Generic Defence in the
following terms:
“57. Paragraph 25 appears to be concerned with the editing or deletion of transaction
data input by or on behalf of a Subpostmaster without his or her consent. Accordingly,
Post Office assumes that it is not concerned with transactions such as Transaction
Corrections which are sent to branches but must be accepted by or on behalf of the
Subpostmaster before forming part of his or her branch account. As to the
circumstances in which such transaction data can be edited or deleted without the
consent of the Subpostmaster:
(1) Neither Post Office nor Fujitsu_has the ability to log on remotely to_a Horizon
terminal in a branch so as to conduct transactions.
(2) A Post Office employee with "global user" authorisation can, when physically
present at a branch, use a terminal within the branch to add a transaction into the
branch's accounts. The purpose of "Global User" authorization is to allow access to the
systems for during training and/ or audits. Any transactions effected by a Global User
are recorded against a Global User ID and are readily identifiable as such.
(3) Fujitsu (and not Post Office) has the ability to inject transactions into branch
accounts (since the introduction of Horizon Online in 2010, transactions of this sort
have been called "Balancing Transactions"). These transactions do not involve any
removal or amendment of the transactions entered at the branch. Their intended purpose
is to allow Fujitsu to correct errors or bugs in Horizon by cancelling the effect of an
error or bug on a branch's data. They may be conducted only by a small number of
specialists at Fujitsu and only in accordance with specific authorisation requirements.
They are rarely used. To the best_of Post Office's information and belief, only one
Balancing Transaction has ever been made so as to affect a branch's transaction data.
and this was not in a branch operated by a Claimant. A Balancing Transaction is readily
identifiable as such.
(4) There are a small number of Fujitsu specialists who have certain privileged user
access rights which they could in theory use to amend or delete the transaction data for
a branch. The intended purpose of privileged user rights is system support, not the
alteration of branch transaction data. To have abused those rights so as to alter branch
transaction data and conceal that this has happened would be an extraordinarily difficult
thing to do, involving complex steps (including the writing of sophisticated computer
programmes and circumvention of sophisticated control measures) which would
require months of planning and an exceptional level of technical expertise. Post Office
has never consented to the use of privileged user rights to alter branch data and, to the
best of its information and belief, these rights have never been used for this purpose.
(5) Post Office cannot conceive of a reason why any Fujitsu personnel would have
sought to add, inject, amend or delete any transactions in any branch accounts so as to
529.
531.
POL00112816
POLO00112816
create a false shortfall. It would for all practical purposes be impossible for any of them
to generate significant shortfalls without detection and, even if they were able to do so,
they would be unable to take the benefit of such shortfalls for themselves.”
(emphasis added)
This stance was maintained by the Post Office in the evidence served on its behalf for
the Horizon Issues trial, until service of Mr Roll’s second witness statement. To be fair
to the Post Office, its origin was the witness statements served by Fujitsu employees,
rather than Post Office employees. The position within the Fujitsu witness evidence,
prior to its correction by the later statements from Mr Parker, was that what Mr Roll
said was possible on Legacy Horizon, and what he had himself done, was simply not
possible. Indeed, Dr Worden considered it sufficiently clear that as an IT expert he felt
able confidently to assert in his 1°‘ Expert Report that he, Dr Worden, had “established”
that Mr Roll’s evidence of fact in this respect was wrong. After service of Mr Roll’s
second witness statement, Fujitsu finally came clean and confirmed (via Mr Parker)
that what Mr Roll said was correct. Data could be altered by Fujitsu on Horizon as if at
the branch; under Legacy Horizon, transactions could be inserted at the counter in the
way Mr Roll described. This could be done without the SPM knowing about this. Mr
Godeseth also confirmed that it would appear as though the SPM themselves had
performed the transaction. This is directly contrary to what the Post Office had been
saying publicly for many years.
Therefore some of the earlier public statements made by the Post Office, and the
important one contained in the Post Office’s pleaded Defence to which I have referred
above, were factually untrue in at least one highly important respect. This concerns the
ability of Fujitsu to insert transactions into a branch’s accounts remotely, without the
SPM being aware of this, and without the transaction being identifiable in the
transaction data as having been inserted remotely in this way. The term “remotely”
means from a location elsewhere than within the branch itself.
Although the phrase “remote access” was used during the Horizon Issues trial, the
parties did not always use the expression in the same way as one another. The Post
Office’s definition was explained in oral opening in the following way:
“When I talk about remote access I'm talking about action taken remotely to either
inject new transactions or to edit existing transactions or to delete existing
transactions in a way that could change the accounting position of the relevant
branch.”
For the purposes of this judgment, I too use the term “remote access” in this way.
The true picture is now that, following the Horizon Issues trial, there is evidence of
transactions recorded by branches being altered through remote access by Fujitsu; and
transactions as they are recorded by branches being edited, that this can be done without
a SPM’s knowledge or permission, and without being identifiable as having been done
remotely. Dr Worden was asked about the 2015 Post Office statement in response to
Panorama, which he said he had not read, and he explained that it was the audit records
in the audit store that could not be edited. He was not prepared to say the 2015 statement
was factually wrong. The BRDB records could be edited, and he also referred to the
532
533.
534.
POL00112816
POLO00112816
joint statement in which the two experts had agreed that they could not say that anything
could not be done.
In my judgment the 2015 Post Office statement plainly is wrong. I find that the 2015
statement was not true. Again, to be fair to the Post Office, this was accepted in the
litigation and at the hearing before Senior Master Fontaine. The quotation from the
transcript of that hearing when this was addressed by the Post Office in oral submissions
is as follows:
“Master, first of all, could I just deal with the remote access point? The letter to which
my learned friend took you was, as you might expect, written by people who thought it
was correct. The Horizon system is a very complicated system. It involves lots of
departments in ... both in Fujitsu and in the Post Office. And the people who are
responsible for the correspondence didn't know that, in fact, there were these two other
routes. Very few people at Post Office knew that there were these two other routes.
They were ... they were routes that are under ... essentially under the control of Fujitsu
who's the expert independent contractor that is involved in the operation of the system.
And it is a matter of enormous regret that the people who wrote that correspondence
and made those submissions weren't aware of that but, you know, we are where we are;
the point is that, the point having been discovered, the Post Office wasted no time in ...
in bringing the truth ... the accurate ... and accurate set of facts to the knowledge of the
claimants.”
(emphasis added)
The claimants rely upon the way this sequence unfolded, and the admission by the Post
Office that its previously public statements were untrue, as making it particularly
important that the true and accurate position was provided by the Post Office. Indeed,
the submissions I have quoted at [532] above make it clear that the court and the
claimants were being told that the Post Office had wasted no time in bringing the truth,
“the accurate set of facts”, to the knowledge of the claimants. As the analysis of the
Defence and the evolution of the Fujitsu evidence of fact on remote access
demonstrates, the accurate set of facts did not emerge at that time either. The truth, the
accurate set of facts, has only emerged as a result of the final supplementary statements
for the Horizon Issues trial were served, and only finally so far as Fujitsu is concerned,
after their witnesses had initially stated clearly in their witness statements that Mr Roll
was wrong, before eventually accepting that he was right. The supplementary witness
statements that accepted this were dated 29 January 2019. Prior to that date, the opposite
position had been maintained by the Post Office.
The Post Office in its opening submissions stated the following:
“Taking remote access as an example, the need for remote intervention affecting branch
accounts will obviously be rare. On any view, the occasions on which privileged users
at Fujitsu have exercised their ability to remotely inject, edit or delete branch
transactions or accounting entries will represent_a tiny percentage of the relevant
transactions/accounting entries. And the occasions on which they have done so
negligently or dishonestly will, in turn, represent_a very small percentage of those
occasions. So, compared with the volume of business recorded in branch accounts, the
number of cases in which false data will have been remotely introduced will be
535.
536.
537.
538.
POL00112816
POLO00112816
extremely small (multiplying a small chance by a small chance). This is a “second order
effect” (a small proportion of a small proportion) which is, by definition, extremely
unlikely to have any significant impact on the robustness of Horizon.”
(emphasis added)
That submission misses the point, in my judgment. It elides two matters, namely
whether something is technically possible, and the number of times that it has in fact
been done. The former, whether it was possible, was denied, and that denial is now
shown to be wrong. The latter, the number of times it was done and with what effect,
is a different matter. There is also very little, if any, evidence that is relied upon by the
Post Office to justify the assertion that it was a “tiny percentage” of times.
I consider the significance of the previously factually untrue statements to be
considerable. The statement was made publicly by the Post Office, turned out not to be
factually correct, and the Post Office gave an explanation and said the full set of facts
was now available. The situation was pleaded to by the Post Office in its Generic
Defence. That too was not correct, and the true position has only emerged in the
Horizon Issues stage of the litigation as a result of the evidence of Mr Roll, which I
have dealt with above. It was only following his written evidence that Mr Parker, and
Mr Godeseth — both senior Fujitsu employees - prepared their supplementary witness
statements correcting their first statements. These first statements, as I have explained
above, were simply untrue in that important respect. These witnesses had previously
stated that this was not possible. Mr Parker said Fujitsu did not have the power to do
this.
I find as a fact that Fujitsu do have the ability to insert transactions into branch accounts
ona remote basis (in other words, remote access as the Post Office defined it in opening
submissions existed) and this could be done without these being visible to the SPM in
that branch in question, either at the time or subsequently. It also follows that this can
be done without an SPM in this position having knowledge of this, and therefore
without consenting to it. Someone who does not know something is happening cannot
give permission for it. It is not possible to say, in blanket terms, that permission was
not sought and/or given in some cases. Indeed, Dr Worden accepted that the APPSUPP
permissions meant that — as he put it — Fujitsu personnel could do “pretty much
anything”. On Day 18 he said in cross-examination the following:
“Q. Dr Worden, you knew that a central issue, not only a central issue legally but a
very high-profile issue in the case, was the extent to which Post Office had remote
access to the counters, didn't you? You knew that?
A. Yes, and what I'm talking about, what I was talking about was the extent to which
this could happen without the knowledge of the subpostmaster.
Q. And that's the —
A. And we agreed in the joint statement that more or less Fujitsu or Post Office could
do anything.”
He was asked about this answer two days later.
539.
540.
POL00112816
POLO00112816
“Q. It is Day 18, page 67. I think it is lines 5 and 6 actually. It says: "Answer: And
we agreed in the joint statement that more or less Fujitsu or Post Office could do
anything."
A. Yes. We can see precisely what we agreed in a minute, obviously.
Q. In relation to the —
A. We agreed the experts couldn't demonstrate that they couldn't do everything. I
mean, that's sloppy wording by me there. I think the joint expert statement says it
better.”
The exact wording in the joint statement in respect of repairing transactions is as
follows at 10.2 of the 4"" Joint Statement:
“Certain facilities and procedures used by Fujitsu to repair the more common issues
which arose in Horizon were standardised, and evidence of them persists. However, to
repair less common issues which arose from time to time, standard tools and procedures
might not have been sufficient, and evidence might not persist of what was done at the
time. Even when evidence does persist, it may be extremely difficult for the experts to
interpret it today, because of the scale and complexity of Horizon.
Therefore, it is usually difficult for the experts to make categorical negative statements
of the form: ‘X or Y never happened’.”
This litigation is between the claimants and the Post Office. Fujitsu is not a party.
Although one of the contracts between the Post Office and Fujitsu was uploaded to the
electronic trial bundle on the penultimate day of evidence (by the Post Office’s
solicitors) the litigation is not concerned with the details of that contractual relationship,
other than (tangentially only) the cost to the Post Office of making ARQ requests, as
the ARQ data is maintained by Fujitsu and charges are raised to the Post Office in this
respect. However, Fujitsu’s involvement does cast something of a shadow. That
contract has been addressed in terms of whether certain documents were, or were not,
in the control of the Post Office. I have already dealt with the issue of PEAKS and
KELS, and how the Post Office maintained initially it was not obliged to disclose these.
Fujitsu personnel who were not witnesses were involved in providing information to
the Post Office’s expert, Dr Worden. It was the Fujitsu witnesses who originally, and
in my judgment unequivocally and directly, flatly contradicted what Mr Roll said could
be done on the subject of remote access. I have also identified that in my judgment, the
majority of the Fujitsu witnesses were more interested in following some sort of Fujitsu
party line than they were in answering questions in cross-examination wholly frankly,
although I exempt Mr Godeseth from that criticism. From what was said by the Post
Office’s Leading Counsel to Master Fontaine at the very first case management hearing,
those within the Post Office had relied upon Fujitsu in obtaining relevant information
for its earlier public statements, and it plainly relied upon Fujitsu in terms of evidence
of fact in the Horizon Issues trial, because it called a number of Fujitsu witnesses. I am
unaware of the degree to which the Post Office has depended upon Fujitsu in the
background to the Horizon Issues trial, and there is no reason why I should be, and I
make it clear that I do not speculate on this. This information must have originated from
541
542.
544.
545.
546.
POL00112816
POLO00112816
Fujitsu at some point, but the Post Office must bear some responsibility for such
incorrect statements having been made before, both publicly and in its pleadings.
Nor could it be said, in the Post Office’s and/or Fujitsu’s defence, that remote access
was a side issue. It has been a very important central element of the whole dispute
between SPMs and the Post Office over the Horizon system for many years. Mr Green
for the claimants adopted a non-computing analogy, namely that of “a back door”. The
origin of this was a phrase of Mr Roll, who referred to the back door to the system. The
branch accounts can be likened to a locked room, to which the SPM has the key. This
development in the case is comparable, after many years of the Post Office and Fujitsu
maintaining that only the SPM has the key, that only the SPM could perform anything
within the locked room, and if the Post Office and/or Fujitsu needed access to the room
the SPM would know they had borrowed the key, to now being an admission by the
Post Office (and Fujitsu) (and a finding of fact by the court) that there is a back door
which they could, and did, use without the SPM even knowing. This analogy is not
entirely apt, because software and remote access to edit transactions is not completely
the same as physical access through a door, but it is an attempt to capture the general
idea in lay terms.
The truth concerning remote access has now emerged, in litigation that started in 2017.
I find it notable that the truth did not emerge in the Fujitsu witness statements that were
originally served for the Horizon Issues trial. Such statements stand as the evidence in
chief of witnesses of fact. They are supposed to be accurate. Minor corrections are not
unusual and indeed are almost expected, as a trial approaches, as witnesses either
research further or remember (when preparing for trial) other minor details. This topic,
however, did not undergo that type of correction, and is a subject far more central and
important than that. The truth only finally emerged in later statements, which were
required to correct what I find were directly inaccurate statements in the first witness
statements of Mr Godeseth and Mr Parker. There has been no adequate explanation for
the contents of those first statements, which not only omitted this important fact, but
contained evidence directly to the contrary. Those first witness statement were
misleading. The statement in the Defence was misleading too. It ought also to be noted
that the truth did not emerge internally within the Post Office in the e mail answers
provided to internal inquiries in 2015 by senior Post Office personnel, such as the Chief
Executive, who posed the specific question in preparation for providing evidence to a
Select Committee and asked: “What is the true answer?”
She also said in the same e mail “I hope it is that we know this is not possible and that
we are able to explain why that is”. The true answer is that, contrary to her aspiration,
it was possible.
She also stated “I need to say no it is not possible and that we are sure of this because
of xxx and that we know this because we have had the system assured.”
The true answer to that was also “yes, it is possible.”
It is also difficult to believe that the signatory of the Statement of Truth in the Generic
Defence would have signed that statement if she did not believe the contents of that
pleading to be true. There is another relevant feature in this case, which is that the
Generic Defence also includes a counterclaim seeking damages against the SPM
547.
548.
549.
550.
POL00112816
POLO00112816
claimants, including damages for fraud. Fraud is the most serious allegation that can be
brought in civil litigation and there are special rules in relation to pleading it, which
means that a pleading containing a fraud allegation should be subject to particular
scrutiny before it is served
It may therefore be that the Post Office itself fell into error as a result of information
provided to it by Fujitsu on this important matter. It may be that some within the Post
Office were themselves surprised by these revelations prior to and during the Horizon
Issues trial. There is no need for me to speculate on this, and I do not do so. Certainly
Mr Godeseth did not appear to have known about this for very long. Whatever the origin
of this, and whether it came from Fujitsu internally being less than frank with the Post
Office or not, the effect is that the Post Office has made specific and factually incorrect
statements about what could be done with, or to, branch accounts in terms of remote
access without the knowledge of the SPM. The evidence in this trial has made it clear
that such remote access to branch accounts does exist; such remote access is possible
by employees within Fujitsu; it does exist specifically by design; and it has been used
in the past.
It follows that the previously stated public position of the Post Office to the contrary,
in the statements to which I have referred above, is specifically wrong in fact.
Further, and as part of the Post Office serving supplementary witness statements from
both Mr Godeseth and Mr Parker in respect of Mr Roll’s evidence, the Fujitsu witnesses
adopted a position in the litigation, the rationale of which is somewhat difficult to
understand. Responsibility for the incorrect and potentially misleading evidence of Mr
Godeseth and Mr Parker (in their first statements) was effectively laid at the door of Mr
Roll for giving what was said to be vague evidence, or evidence that, it was said, had
not been fully understood by Fujitsu. Infallibility is a rare commodity, and everyone is
capable of making mistakes. However, it is how one reacts to mistakes that is telling.
In this instance, the initial reaction of the Fujitsu witnesses was to seek to shift the
blame for their own misleading written evidence upon someone else. In this case, that
“someone else” was their former Fujitsu colleague whose very evidence was
responsible for exposing the full picture. This is not something that could have been
arrived at lightly. To be fair to Mr de Garr Robinson, it was not a point he adopted on
the Post Office’s behalf during the Horizon Issues trial. In my judgment, he was entirely
sensible in not doing so. That it was done at all in the written Fujitsu statements speaks
volumes, in my judgment, about Fujitsu’s reaction to being caught out.
Mr Parker’s supplementary statement blaming Mr Roll said the following:
“In paragraph 20 of Roll 2, Mr Roll describes a process by which transactions could be
inserted via individual branch counters by using the correspondence server to piggy
back through the gateway. He has not previously made this point clear. Now that he
has, following a discussion with colleagues who performed such actions I can confirm
that this was possible. I did not mention it in my first witness statement because, when.
faced with a less clear account in Mr Roll's first statement, my recollection was that if
it was necessary for the SSC to inject a transaction data into a branch's accounts, it
would have been injected into the correspondence server (injecting via the server was
the default option which was followed in the vast majority of cases).”
552.
553.
554.
555.
POL00112816
POLO00112816
The recusal application was issued the day after Mr Godeseth’s cross-examination
made it clear not only that this remote access existed, but after he was taken in careful
cross-examination through specific examples of Fujitsu personnel manipulating branch
accounts, and leading to discrepancies in branch accounts. I am aware that criticism of
the Post Office and Fujitsu in this respect may prove to be controversial, as earlier
criticism of the Post Office in Judgment (No.3) was not well received by it. However,
if criticism is justified, I consider it would be detrimental to proper resolution of the
Group Litigation if that criticism were to be withheld because it may lead to a further
negative reaction by the Post Office. It is also an inherent part of the judicial function
in any litigation to make findings, which may include criticisms where justified, that
may be contrary to a litigant’s own view of the merits of their case. Some litigants are
so convinced of the righteousness of their own position that they consistently refuse to
accept any possible view of the litigation other than their own. Such a blinkered view
is rarely helpful, and would be particularly unhelpful from a publicly owned institution.
I consider that criticism is justified of the Post Office’s incorrect previous statements —
which included public and high profile statements such as the response to the BBC
Panorama programme, whatever was said on the subject to the Select Committee, as
well as in its own pleadings - about remote access without the knowledge of SPMs. The
Post Office should have done its best to discern whether such remote access was
possible, and whether it had occurred, before these statements were made. Fujitsu
should have been frank and unequivocal, internally, with the Post Office, so that there
could be no possibility of incorrect statements on this important point being made
publicly by the Post Office. The Fujitsu witnesses should not, in their first witness
statements, have made the incorrect statements that they did. Had these initial
statements been factually accurate, there would have been no need for the
supplementary statements from them that eventually led to the true position being
accepted by Fujitsu, and therefore by the Post Office. It is highly regrettable that such
a situation as this should have developed. I am not making any findings on any future
issues, whether of fact or law, yet to be tried.
There is however considerably more to resolving the Horizon Issues than simply
making findings about remote access to branch accounts.
Documents and Available Information
There are certain categories or descriptions of classes of documents that have featured
heavily in the evidence at the Horizon Issues trial. The path to disclosing them has not
always been smooth. The majority, if not all, of the technical documents that relate to
how Horizon was actually operating in fact in IT terms are in the possession of either
the Post Office or (more usually) Fujitsu. The two most important categories, in my
judgment, are Known Error Logs (also known as “KELs”) and PEAKS. The first of
these records or logs known errors, which means errors with the Horizon system. The
latter is a browser-based software incident and problem management system used by
Fujitsu for the Post Office account, in other words for incidents and problems associated
with Horizon that occur.
Disclosure had previously been a troublesome topic between these parties. I had in
Judgment (No.3), dealing with the Common Issues, been somewhat critical of some
aspects of the Post Office’s approach to the disclosure of some documents. These
556.
557.
558.
POL00112816
POLO00112816
observations were not well received by the Post Office and some had been categorised,
incorrectly, by Mr Parsons in his 14"" witness statement as “critical invective”. Whether
the contents of Judgment (No.3) encouraged the claimants to mount, or continue to
advance, criticisms of the Post Office’s disclosure in the Horizon Issues trial (in the
sense that they may have felt they had a “following wind”) is not clear. The Post Office,
in the Horizon Issues trial (predominantly in solicitors’ correspondence, but also in
some submissions), also criticised the claimants’ disclosure. For the sake of clarity, I
wish to make it clear that the claimants had no such following wind.
The Horizon Issues are decided in this judgment based on all the evidence and the
submissions. I deal with disclosure in the subsequent paragraphs of this judgment in
order to assist and guide the parties concerning the future conduct of the group
litigation. Some issues, such as whether certain Fujitsu documents were within the Post
Office’s control, are likely to crop up again in respect of other types of documents in
the future. The court’s interest in disclosure is to promote compliance with the rules,
and also to ensure cost effective progress of the litigation. The answers to the
substantive Horizon Issues are no different because, for example, the experts were not
given particular documents relating to the specific bugs in good time. For the most part,
large numbers of KELs and PEAKS were provided to them, and the experts had an
abundance of material. Some 5,000 KELs emerged from Fujitsu well after the trial
ended, in October 2019. The experts did not consider these as they were not given them.
Mr Parker had given evidence (at paragraph 61.9 of his 1“ witness statement) that 1,491
KELs had been deleted at the time of writing that statement, but there was no reference
to this far greater number that emerged after the trial was over.
The Post Office’s written closing submissions also included a separate section on
disclosure as follows, which I will quote verbatim:
“16. The Court’s interventions on disclosure
1145. Post Office is concemed that the Court may have lost sight of the nature of the
Orders made and of the approach which the Court ordered. It is striking that Cs have
not themselves made any applications for specific disclosure nor have they advanced.
any complaint that particular disclosure orders have not been complied with.
1146. The recent judgment of the Court of Appeal in Serafin v Malkiewicz & ors [2019]
EWCA Civ 852 @ para 118 is relevant:
“We are also highly troubled by the repeated demands and criticisms by the Judge
regarding the Claimant’s disclosure, in circumstances where pre-trial disclosure had
been completed by both sides at a time when both the Claimant and Defendants had
been represented by solicitors and counsel, and no application for further disclosure had
been made by the Defendants...”
These submissions identified what were said to have been “the Court’s interventions on
disclosure” in the Horizon Issues trial. The authority identified by the Post Office, and
stated to be relevant, concerned a defamation case in which the judge had demanded
disclosure of a litigant in person, and made criticisms of him, both during the trial of
that litigant’s libel claim, and in the judgment itself. In some instances the court in that
case had, of its own volition, demanded production of certain documents from the
559.
560.
POL00112816
POLO00112816
litigant in person, and within a very short timescale. This had been part of findings
made by the Court of Appeal in respect of procedural irregularity and the successful
appeal against the judgment.
There were no “interventions on disclosure” by the court during the Horizon Issues
trial, nor were there “demands and criticisms”. When asked during oral closing
submissions about this passage in the Post Office’s written submissions citing Serafin,
Mr de Garr Robinson did not demonstrate a great deal, if any, enthusiasm to advance
any claims of procedural irregularity having occurred during the Horizon Issues trial.
He submitted that “this is not that case” — which does raise the question of why the
passages from Serafin were included in the Post Office’s submissions in the first place.
There were, however the following matters that arose in this litigation during the
Horizon Issues trial, which given paragraphs 1145 and 1146 of the Post Office’s closing
submissions, should be recorded:
1. Redactions. In Judgment (No.3) I had identified the issue of redactions made by the
Post Office of some contemporaneous documents that were used in the Common Issues
trial. During the early part of the Horizon Issues trial some documents were referred to
which again had redactions, which due to their non-redacted parts did not seem to me
to be of a type that would obviously attract privilege. In view of the previous history of
this subject, I asked leading counsel for the Post Office to perform his own review of
the redactions that had at that point been made of documents deployed in the Horizon
Issues trial. This review resulted in some of the redactions in some of the documents
being removed, and unredacted versions of those documents becoming available to the
claimants, some of which were then used in cross-examination. Redactions of 2
documents were maintained on the grounds of legal professional privilege, which is
entirely conventional. He also provided a helpful two page note explaining the approach
that had been adopted. Asking counsel to do a review of this nature in these
circumstances is entirely conventional. There was no short timescale imposed for this
to be done. I asked for a summary of the position on 4 June 2019, which was after the
interval which occurred as a result of the recusal application, and about 2 /% months
after I had asked him to perform the review. It was an outstanding matter, of a type
sometimes referred to in a trial as “housekeeping”, which had not been entirely
forgotten, but which had been overtaken in the latter weeks of March when the trial was
temporarily halted.
2. A witness statement was ordered from the Post Office to explain the express, and
incorrect, submissions made to the court by the Post Office about the Royal Mail’s
refusal to produce, at the Post Office’s request, audit documents. The Royal Mail had,
contrary to what the court had expressly been told by the Post Office, not even been
asked by the Post Office for these. The Post Office’s leading counsel accepted that he
had, entirely unwittingly, and on instruction, provided misleading information to the
court and explained and apologised as soon as he discovered this. A witness statement
from the Post Office, which was provided by Mr Parsons, was entirely justified in these
circumstances.
3. A witness statement was also ordered from the Post Office in respect of the
production of over 2,000 documents in disclosure by the Post Office in the interval in
the trial period between 11 April 2019 (completion of the factual evidence), and the
S61.
562.
563.
POL00112816
POLO00112816
experts commencing their oral evidence on 4 June 2019. Given the timing and quantity
of these, in the circumstances a witness statement (again, provided by Mr Parsons) was
justified. The first explanation given to the court, again on instruction, again proved to
be wrong. These documents were originally said to have all been of some age (ten years
or so) and discovered at Fujitsu on a long-forgotten server. One deployed the day after
this explanation in cross-examination turned out to be dated 21 August 2018. A witness
statement was again called.
None of the above, in my judgment, fall into the category of the court losing sight of
the nature of the orders that had been made concerning disclosure, or of demands or
criticisms by the court. Further, the claimants made numerous complaints of the Post
Office about disclosure. Indeed, the factually incorrect submissions made to the court
about the audit documents and the Royal Mail came at the end of a court day when the
claimants, by their leading counsel, expressly attempted to make a specific disclosure
application. I was not prepared to hear that application at that point. Royal Mail was
the correct respondent to such an application, not the Post Office. Royal Mail was not
represented, nor even present in court, nor had any notice been given by the claimants
to the Royal Mail of such an application. Each of these features is a far from promising
start. I did however make directions for the making of a third party disclosure
application against the Royal Mail, but this did not ultimately prove to be required.
Once those at the Royal Mail were actually asked for these documents, the Royal Mail
readily produced them. The claimants also made regular complaints about disclosure
and would often point out that documents had been produced during the hearing. This
is because documents were often produced during the hearing. This is something that
had also occurred during the Common Issues trial.
The Post Office is, however, correct to point out that there have been no specific
disclosure applications mounted by the claimants, with the exception of the one to
which I refer in [561] above in respect of the Royal Mail. General complaints about
disclosure only take a party so far, and that is not very far, if deficiencies are known
about in advance of a trial, and no applications are made.
The parties had also voluntarily adopted Model C of the Disclosure Pilot, which I
approved. This was done prior to the coming into force of the Pilot, the full title of
which is Disclosure Pilot for the Business and Property Courts (“BPC”) which became
mandatory in all BPC cases where the first CMC is held after 1 January 2019. This
group litigation is not a BPC case and the first CMC was held before that date in any
event. The Pilot is designed to run for a two year period. The Pilot is found at Practice
Direction 51U, which is made under CPR Part 51.2. The features of the Pilot that are
particularly relevant here are the Principles included in paragraph 2 of the Pilot. These
state:
2. Principles, “document”, “adverse” and “known adverse documents”
2.7 Disclosure extends to “adverse” documents. A document is “adverse” if it or any
information it contains contradicts or materially damages the disclosing party’s
contention or version of events on an issue in dispute, or supports the contention or
version of events of an opposing party on an issue in dispute.
564.
565.
566.
567.
POL00112816
POLO00112816
2.8 “Known adverse documents” are documents (other than privileged documents) that
a party is actually aware (without undertaking any further search for documents than it
has already undertaken or caused to be undertaken) both
(a) are or were previously within its control and (b) are adverse.”
3.1 A person who knows that it is or may become a party to proceedings that have been
commenced or who knows that it may become a party to proceedings that may be
commenced is under the following duties (“the Disclosure Duties”) to the court—
(1) to take reasonable steps to preserve documents in its control that may be relevant to
any issue in the proceedings;
(2) once proceedings have commenced against it or by it and in accordance with the
provisions of this pilot scheme, to disclose, regardless of any order for disclosure made.
known adverse documents, unless they are privileged;”
(emphasis added)
The fact that the parties chose Model C does not go to answer the complaints that the
claimants have made about the Post Office’s disclosure. The Post Office had an
obligation to disclose known adverse documents, as do the claimants as well. “Adverse”
as defined plainly includes any documents that refer to bugs, errors or defects, or the
operation of the Horizon system that led to potential impact upon branch accounts. This
obligation is “regardless of any order for disclosure made”, in other words it exists
whether any specific classes of documents are, or are not, already the subject of any
order. The rational for this is obvious. Reliance upon the adoption of Model C is not
therefore the answer to the claimants’ complaints in this respect. However, the fact that
the claimants did not make applications for disclosure is relevant to the criticisms made
at the Horizon Issues trial
The Post Office has, essentially, submitted that the breadth of disclosure requests has
been too wide. Particular criticism is that “Freeths’ [the claimants’ solicitors] did not
control Mr Coyne’s requests”. I am not persuaded that it is the correct approach to
matters of independent expert evidence to seek to “control” the documents that an
independent expert wishes to see. The claimants did not issue any opposed applications
for disclosure of documents which their expert wished to see, so a filter of some sort
was applied in that sense to the documents Mr Coyne asked for, and was not given.
Finally, disclosure is a particular feature of litigation in this jurisdiction. However, there
was a clear perception in commercial litigation generally that the then-current
disclosure regime had become inadequate and also that standard disclosure was often
excessive in scale, cost and complexity. This is why the Disclosure Working Group was
created in May 2016 at the initiative of Sir Terence Etherton, then the Chancellor of the
High Court and now the Master of the Rolls. This led to the Disclosure Pilot.
It would be highly regrettable if disclosure in this case were to become yet another
battleground between the parties. There are more than enough points in issue between
the parties without adding endless disagreements about disclosure to that list. There are
the following areas, however, of disclosure which should be recorded. In my judgment,
POL00112816
POLO00112816
the background to the evolution of the following categories does throw light on the
parties’ different approach to the litigation.
The Known Error Log or KELs
568.
569.
570.
S71.
572.
KELs are sometimes referred to in the singular — for example, in some places reference
is made to the Known Error Log, as though it were a single Log. In practice, the Log as
a whole includes multiple KELs, as they were referred to during the trial. If an incident
or problem occurs, a PEAK is raised, and if the problem or incident refers to an existing
issue which is known to have occurred before, reference will be made to that KEL entry,
or more usually, it is said that the PEAK will refer to the KEL. Both PEAKS and KELs
have reference numbers; the former are usually all numbers after the prefix PC, whereas
KELs have letters (taken from the identity of the employee who raised the KEL
followed by four digits and a letter). Therefore, when a problem (which I shall refer to
generally as an example as problem x) occurs for the first time, a KEL will be raised
which will be given a reference. Subsequent PEAKS which identify the same problem
x again, over time, afterwards will all refer to the same KEL. Sometimes a PEAK will
be identified as including a problem which is similar to problems identified in more
than one KEL, in which case both the KELs will be referred to within that PEAK. Any
creation of a new KEL, or updating of an existing KEL, must be authorised by SSC
before it can be seen by all users.
The experts agreed the following in the 2" Joint Statement about both PEAKs and
KELs: “KELs and Peaks together form a useful source of information about bugs in
Horizon but are a limited window on what happened. It is sometimes necessary to use
evidence from both to try to understand, but even so they are not a comprehensive
picture. It is to be expected that both KELs and Peaks are incomplete in various
respects.” It is also agreed that KELs will often give information about the impact of a
bug or user error and they may also provide information about causes. There are other
documents referred to as OCPs and OCRs. They are Operational Change Proposals and
Operational Change Requests.
The subject matter of this litigation spans many years, starting from the first
implementation of the Horizon system (what is now being called Legacy Horizon) in
2000. In the Letter of Claim from the claimants dated 28 April 2016, the Known Error
Log was sought from the Post Office, that letter stating:
“We understand that Fujitsu maintained a 'Known Error Log' for Horizon and that such
reports will have been provided to Post Office. Please see the list of the categories of
documents relating to Fujitsu referred to below, that we request disclosure of.” Item 22
in the list of documents sought was “The 'known error log’ kept by Fujitsu and provided
to Post Office as referred to above, and all correspondence relating to the same.”
The answer in a letter from the Post Office’s solicitors against the specific item 22 was:
“In circumstances where you have not particularised any factual basis on which
Horizon is defective, disclosure of these documents (if they exist) is not relevant,
reasonable or proportionate.”
The suggestion in that letter that the Known Error Log was not relevant, is simply
wrong, and in my judgment, entirely without any rational basis. The further suggestion,
S73
S74.
575.
576.
S77
POL00112816
POLO00112816
viewed with the hindsight now available, that the “known error log” may not exist, is
disturbing. The claimants’ request used the precise title — “known error log” — and this
clearly did exist. To suggest in an answer “if they exist” is somewhat misleading.
Item 23 in the same list of documents sought was “Internal memoranda from Fujitsu
and POL referred to by Second Sight as identifying a “Horizon bug’ within Horizon
Online.”
The answer against that item was:
“We do not recognise the document to which you refer. Please provide further details.”
In my judgment, the documents sought in that entry must clearly include any PEAKS
that identified a bug within Horizon Online. “Internal memoranda” is a plural reference,
yet it was interpreted by the Post Office’s solicitors as though it were singular, and the
request was for a single document, or a document with the title “internal memoranda”.
This is, in my judgment, obstructive.
The claimants were not to be dissuaded, and sought the Known Error Log or KELs
again. A reply from the Post Office’s solicitors on 13 October 2016 is relied upon by
the claimants as showing that the Post Office was denying the relevance of the Known
Error Log. This reply stated:
“The claims which you have particularised concern errors with the Core Audit Log.
Following a review of the Known Error Log, Fujitsu have confirmed that there have
been no logs in respect of Core Audit Log. The remainder of the Known Error Log does
not relate to the claim which you have particularised and as such disclosure of this
document is not relevant.”
(emphasis added)
Existence of the Known Error Log was at that stage accepted, but its relevance to the
proceedings was now challenged. The Post Office’s solicitors stated that its contents
“did not relate to the claim” and that “disclosure of this document is not relevant”.
Disclosure of it was plainly resisted. The claimants did not therefore have it when the
Generic Particulars of Claim was pleaded on 6 July 2017. In the Generic Defence,
which is dated 18 July 2017, the Post Office changed its position, and now pleaded that
the Known Error Log was not in its control. At paragraph 50(4) of the Generic Defence,
the Post Office stated:
“It is admitted that Fujitsu maintain a "Known Error Log". This is not used by Post
Office and nor is it in Post Office's control. To the best of Post Office's information and
belief, the Known Error Log is a knowledge base document used by Fujitsu which
explains how to deal with, or work around, minor issues that can sometimes arise in
Horizon for which (often because of their triviality) system-wide fixes have not been
developed and implemented. It is not a record of software coding errors or bugs for
which system-wide fixes have been developed and implemented. To the best of Post
Office's knowledge and belief, there is no issue in the Known Error Log that could
affect the accuracy of a branch's accounts or the secure transmission and storage of
transaction data.”
(emphasis added)
579.
580.
581.
POL00112816
POLO00112816
This was expressly challenged in paragraphs 3 and 4 of the Generic Reply.
The two main points to consider within that passage in the Defence are that the Post
Office stated that it did not have control of the Known Error Log, and that entries within
it related to minor/trivial matters and could not affect the accuracy of a branch’s
accounts. This position was maintained, notwithstanding the terms of the Post Office’s
skeleton argument for the first case management hearing on 19 October 2017. That had
stated:
“Cs? response to the criticism of their case on Horizon is to argue that they do not have
the access to information and documents that would allow them to plead any properly
articulated case as to the “bugs” that they wish to blame for the shortfalls in their
branches. Without prejudice to its position that the case is not properly pleadable, even
at a generic level, Post Office is prepared to take steps to help Cs investigate Horizon.
take an informed view as to whether they really wish to maintain their claims and, if
so, decide how to proceed with them. Post Office wishes to put an end to the speculative
attacks on Horizon and the related allegations as to its own conduct. It puts forward its
proposals in paragraphs 2 to 8 of the Draft Order as a pragmatic compromise between
the parties’ competing interests and concerns.”
(italics present in original, emphasis by underlining added)
At the first case management hearing itself, upon specific enquiry by the court, this
position was further maintained.
Disclosure of the Known Error Log was resisted by the Post Office on the grounds both
of control, and content. So far as control is concerned, Mr Parsons had provided his 4"
witness statement for that hearing, the first before me as the managing judge, which
expressly stated at paragraph 35 “Despite Post Office explaining the irrelevance of the
Known Error Log and that it was not within Post Office’s control...” and also at
paragraph 37 “Due to the large amount of information involved and the fact that the
Known Error Log is not in Post Office's control....”. This position was confirmed by
the Post Office through its leading counsel in the following exchange with the court at
that hearing, the transcript of that showing the following exchange.
“Mr Justice Fraser: Do you still maintain it is not in your control?
A: My Lord, yes. It’s a Fujitsu document.
Mr Justice Fraser: No. Just because it’s a Fujitsu document doesn’t mean it’s not in
your control
A: That’s right. At no point has Post Office ever suggested that it’s in its control so far
as I’m aware in any of the letters that I —
Mr Justice Fraser: Okay.
A: (Reading from pleading)
582.
583.
584
586.
POL00112816
POLO00112816
"To the best of Post Office's information and belief the known error log is a knowledge
based document used by Fujitsu which explains how to deal with or workaround minor
issues that can sometimes arise in Horizon for which often because of their triviality
system-wide fixes have not been developed and implemented. It is not a record of
software coding errors or bugs for which system-wide fixes have been developed and
implemented. To the best of Post Office's knowledge and belief there is no issue in the
known error log that could affect the accuracy of branch accounts or the secure
transmission and storage of transaction data.”
This was the relevant passage in the Generic Defence that was read out and Post
Office’s leading counsel then continued:
“Tt will be clear from what I've just read to your Lordship that Post Office thinks that
this is a complete red herring.”
The position was then, again, expressly confirmed as follows.
“Mr Justice Fraser: I think it's going to be useful for me, certainly, for you to define all
the different grounds upon which you currently resist disclosure of the known error log.
Number (1) is you say it's not in your control.
A: Control.
Mr Justice Fraser: Number (2) you say the subject matter is not relevant. Is that right?
A: My Lord, yes. An order that Post Office disclose documents relating to bugs and
errors causing the branch account errors, if Post Office had control of the document and.
it dutifully complied with that order it would not be disclosing the known error log
because, as far as it is aware, the known error log is not that kind of document.”
So far as the content of the Known Error Log is concerned, when asked by the court
“does it contain any errors at all?” the following answer was given by the Post Office’s
leading counsel:
“It contains things like there's a problem with printers. There's a printer. You have to
kick it on the left-hand side to make the printer work. I mean there's a vast range of
hardware problems of that sort and maybe some software problems .... but not the kind
of bugs, errors and defects that the claimants are wishing to pursue in their particulars
of claim so far as Post Office is aware.”
This exchange with the court then continued.
“Mr Justice Fraser: Well, that's the rider which slightly might concern---
A: Well, my Lord, unfortunately, that's all that Post Office can say because it's not Post
Office's document. It's Fujitsu's document. Fujitsu are the experts.”
Given there was no witness statement from anyone at Fujitsu available on the subject
at that hearing, that was as far as the matter could be taken on that occasion in terms of
making any specific order relating to the Known Error Log. The Post Office had made
587.
588.
589.
590.
POL00112816
POLO00112816
clear that any order for disclosure that did not specify the Known Error Log, but was
more general requiring disclosure of documents in relation to bugs and errors causing
branch account errors, would not lead to disclosure of the Known Error Log in any
event. A practical way forward, without requiring a fully argued separate application
which would have included Fujitsu, was adopted whereby the IT experts were permitted
to inspect the Known Error Log. The status of the Known Error Log as a disclosable
document was resisted both on the grounds of control and relevance. It has to be
acknowledged that this practical solution was one suggested by Mr De Garr Robinson.
It must therefore have come as a surprise to the Post Office, given the contents of its
pleadings and the express submissions that it had made to the court, that both experts
considered the Known Error Log to be highly relevant. This should also have led to the
Post Office beginning to doubt what it was being told by Fujitsu, given the source of
what the court was told about this was what Fujitsu had told the Post Office, as set out
above. The explanation of what the Known Error Log was, what it contained, and its
lack of relevance, was not remotely accurate.
Although a certain number of entries in the Known Error Log, which led to a significant
number of different entries which are each called KELS being deployed in the trial and
examined closely, were thereafter available, the issue of control of the Known Error
Log and the Post Office’s earlier position resisting disclosure of this, did not go away.
An entire appendix to the claimants’ opening submissions was devoted to criticism of
the Post Office on disclosure, including its shifting position on the Known Error Log.
In closing submissions, at the very end of the final day of the Horizon Issues trial, the
claimants handed up a document entitled “Claimants” Points in Reply”. Part F of that
was a short document headed “KELS said not to be in the Post Office’s control”
together with a variety of references. It ought also to be recorded here that some 5,000
KELs were later disclosed by the Post Office in October 2019, well after the trial ended,
once the Post Office was told by Fujitsu that previous entries, which Fujitsu had
previously told the Post Office were not retained, were in fact retained.
The subject of whether the Post Office had control of the Known Error Log therefore
remained somewhat live. It simply was not, therefore, going to go away. The Post
Office’s legal team had loaded version 12 of its contract with Fujitsu (“the Fujitsu
contract”) onto the electronic bundle database on the last day of evidence, Day 20, for
re-examination of its expert witness, Dr Worden. The contractual situation between the
Post Office and Fujitsu was therefore before the court, which it had not been for the
first case management hearing on 19 October 2017. It is also the case that “control” of
documents (other than the KELs) held by Fujitsu could potentially crop up again in this
litigation going forwards. It is cost effective and efficient to resolve that now, for the
assistance of the parties.
I therefore gave both parties the opportunity to lodge further written post-hearing
submissions restricted to whether the Known Error Log was in the Post Office’s control.
It was a point relied upon by the claimants and it has to be resolved. There were two
reasons for this. One is that in group litigation of this type, the parties’ approach to
disclosure goes beyond a single document, or type of document. Guidance as to the
court’s general approach will be of assistance in the litigation going forwards. There
may well be other documents held by Fujitsu that need to be disclosed later, and the
S91
592.
593.
594.
595.
POL00112816
POLO00112816
issue of control needs resolving. Secondly, the court is entitled to expect accurate
evidence from parties on interlocutory matters, and accurate submissions.
The Fujitsu contract that was loaded onto the electronic bundle was a recent one. This
document has a version history at the front which shows that it runs, with amendments
therein identified, from 31 August 2006 to version 12 on 3 July 2017. Version 12 pre-
dates the date of the Generic Defence, but in any event, clause 25 (which appears below
at [597]) does not appear against any of the later version numbers in the version history
of the Fujitsu contract. It is likely therefore that clause 25 in substantially (if not
identical) form appeared in all the earlier versions of the Fujitsu contract prior to
Version 12.
The claimants made the requested post-hearing written submissions based on the
meaning of ‘control’, for the purposes of CPR Part 31.8, which is uncontroversial.
That rule provides as follows:
“(1) A party’s duty to disclose documents is limited to documents which are or have
been in his control.
(2) For this purpose a party has or has had a document in his control if—
(a) it is or was in his physical possession;
(b) he has or has had a right to possession of it; or
(c) he has or has had a right to inspect or take copies of it.”
Sub-paragraphs (a) to (c) are not exhaustive, but collectively those two sub-paragraphs
mean, in this case concerning this document or documents, that if the Post Office has,
or had, a right to possession of the Known Error Log, and/or a right to inspect or take
or be provided with copies of it, it is within the Post Office’s control. Control will be
established if a party has a contractual right to inspect or take copies of the document
or documents in question. Given the terms of the Fujitsu contract, the claimants
submitted that the Known Error Log is in the Post Office’s control and has been at all
material times.
The Post Office written submissions on this were also received. These conceded, as it
was put in paragraph 3, that the KEL was in its control. That paragraph stated at sub-
paragraph (1):
“As at July 2017, Post Office understood the KEL to be a relatively unimportant internal
working document produced and used by Fujitsu to assist in the performance of some
services under the contract. On that basis, the KEL would not properly have been
characterised as a “record...relating to the performance” of Fujitsu’s services under the
Fujitsu Contract and would thus not have been within Post Office’s control. But, on the
facts now known to Post Office, it would not contend the KEL to be outside its control.”
(emphasis added)
The title of the log — the Known Error Log, or known error log — includes rather
obviously the two words “known errors”. The Horizon Issues use the expression “bugs,
errors and defects”. The presence of the word “error” in both might give an early
indication of likely relevance of the log. Further, as shown in [598] below, in the Fujitsu
contract the term records includes the phrase “full and accurate records” (emphasis
S97
POL00112816
POLO00112816
added). Although records is used in the definition of Records — a point correctly
identified by the Post Office, which says is largely circular — it is clear that “full and
accurate records” are included.
Clause 25 in version 12 of the Fujitsu contract states the following:
“25.8 In addition to its obligations under Clauses 25.2 and 25.3, Fujitsu Services shall
provide the Court Case Support Services to Post Office in relation to prosecutions and
other disputes (whether civil or criminal) with any third party including but not limited
to any fraud, theft, breach of contract or impropriety (the “Court Case Support
Services”). The Court Case Support Services shall include any matters whether they
relate to Horizon, HNG-X or any other system provided by or on behalf of Fujitsu
Services to Post Office, its agents or its subcontractors (including Post Office Service
Integrator and any Tower Contractor). Fujitsu Services shall provide the Court Case
Support Services within the timeframes required by Post Office or the relevant court or
other authority.
25.9 Without prejudice to Clause 25.3, the Court Case Support Services shall comprise:
25.9.1 the provision of copy reports;
25.9.2 the provision of data (including transaction data, event logs, helpdesk call logs.
non-polled data and remuneration data) where such data is held by or in the control of
Fujitsu Services;
25.9.3 the compilation of data (including transaction data, event logs, helpdesk call
logs, non-polled data and remuneration data);
25.9.4 the interpretation of data (including transaction data, event logs, helpdesk call
logs, non-polled data and remuneration data):
25.9.5 the provision of technical reports regarding technical aspects of any system
(whether Horizon, HNG-X or otherwise);
25.9.6 live witness evidence at Court if any of the information provided (including
without limitation that provided pursuant to Clauses 25.9.1 to 25.9.5) is challenged to
the extent to which Fujitsu Services provided said information; and
25.9.7 the right of access to Records, including but not limited to information, reports
and data, held by or in the control of Fujitsu Services, and the assistance of Fujitsu
Services’ personnel with appropriate knowledge of the applicable Records (to the extent
any such personnel remain employed or contracted to Fujitsu Services) for any
independent experts and/or legal advisors instructed by Post Office and/or any other
claimant(s) or defendant(s) and the Prosecution in any mediation, arbitration tribunal,
court case or dispute in which Post Office is involved in relation to the Horizon and
HNG-X or any other system provided by or on behalf of Fujitsu Services to Post
Office.”
(emphasis added)
598.
599.
600.
601
POL00112816
POLO00112816
Records is defined in the Fujitsu contract in Schedule 1 “Interpretation”, in the
following way:
“Records: means the full and accurate records relating to the performance of the
Services.”
I consider that this term plainly includes the Known Error Log, as that relates to the
performance of the Services, and in any event this contains data and hence also falls
within clauses 25.9.2 and 3. I reject the lengthy analysis in the Post Office’s
submissions on this point that the KEL is not properly characterised as being a record.
It plainly is.
In my judgment the Post Office clearly has and had a contractual right to be provided
with the Known Error Log by Fujitsu, given these are civil proceedings for (inter alia)
damages sought by the claimants for breach of contract, and also fraud being alleged
against the claimants by the Post Office by way of counterclaim. I do not accept that
the KEL is a type of document covered by the authority cited by the Post Office at
paragraph 11 of its submissions on this point, namely working papers prepared by
professionals for their own assistance in carrying out expert work for their clients;
Hanley v JC & A Solicitors [2018] EWHC 2592 (QB) at [42]. Given the terms of the
contract between the Post Office and Fujitsu, it is not necessary to consider that point
further.
I consider it verging on entirely unarguable, given the express terms of the Fujitsu
contract which is now available to the court, that the Known Error Log was not in the
control of the Post Office. Mr Parson’s witness statement had merely stated “it was not
within Post Office’s control” and I simply cannot understand that statement, given the
terms of the contract that the Post Office had with Fujitsu. It plainly is in the Post
Office’s control given the terms I have reproduced above, and that point is now
conceded.
The fact that the Post Office submits that in July 2017, on its understanding then, the
KEL was not a “record” that “related to the performance” of the Fujitsu services
demonstrates a worrying lack of knowledge on the part of the Post Office about both
Horizon, and Fujitsu’s record keeping. It also means, when this is put together with
what the Post Office, by its counsel, submitted to the court at the 1‘ case management
conference, that Fujitsu were extraordinarily inaccurate about the information it
provided to the Post Office at that stage of these proceedings. As at 2017, the Horizon
system (both Legacy Horizon and Horizon Online) had been in use for nearly 17 years.
KEL appears in a table of Abbreviations/Definitions at F/87/3 in a document entitled
“CS Support Services Operations Manual” that is dated 29 January 2001. That
document is referred to in footnote 34 of Mr Coyne’s 1" report, and three passages
within it state the following:
“4.5.1 Maintaining the Known Error Log on the SSC intranet site
The SSC generates and maintains a Known Error Log (KEL) system that uses
searchable documents in HTML format. The mechanism for searching is a query entry
in an intranet site. The KEL system is available to first, second, third and fourth line
support units as well as SSC staff.
4.5.2 Transferring knowledge between support units
602.
603.
604.
605.
POL00112816
POLO00112816
The SSC intranet site has KEL search facilities and other useful diagnostic data,
documents and tools.
SSC and SMC staff raise KELs based on customer-observed symptoms.
KELs are further maintained once the fault has been resolved.”
“4.7.1 Known Error Logs (KELs)
The intranet site holds known error details in Microsoft Word format, the contents of
which may be searched for, in full text form. Documents are created to a defined
template wherever possible. An application has been generated which limits the
properties of the document to a subset of possible values, for clarity and ease of search.
This application is made available to all support units.
The process for creating KEL entries outside of the SSC has not yet been formulated,
but it is expected that no KEL will be allowed onto the system before it has been
authorised by SSC staff.”
(bold present in original)
The Post Office plainly did not know this when making the submissions that were made
in 2017. The Services Operation Manual had been in existence for 16 years by then,
and this ignorance about a document clearly explained within that document is most
surprising.
Even without such an express obligation in the Fujitsu contract itself, I find it very
difficult to believe that Fujitsu could mount any sort of coherent or legitimate objection
to producing it at the request of the Post Office. The assertion that the Log was not in
the control of the Post Office is, in my judgment, not a valid one.
Similarly unsustainable is the submission that was made in 2017 that its contents were
not relevant to the Horizon Issues, and that it did not contain references to bugs and
errors having an impact on branch accounts. On that latter point, the Post Office’s
submissions were based upon what the Post Office had been told by Fujitsu. Fujitsu
were, again, being clearly inaccurate about what they had told the Post Office was
contained in the Known Error Log.
Nor is the passage quoted above in the Generic Defence an accurate description of the
Known Error Log. The Known Error Log does far more than “explain how to deal with,
or work around, minor issues that can sometimes arise in Horizon for which (often
because of their triviality) system-wide fixes have not been developed and
implemented”. The Known Error Log appears to be a comprehensive record of all the
errors and defects of which Fujitsu have become aware over the life of Horizon. On the
basis of the evidence that has been heard in the Horizon Issues trial, there are many
issues in the Known Error Log that could affect the accuracy of a branch's accounts or
the secure transmission and storage of transaction data, contrary to what is pleaded by
the Post Office. Some KELs led to software fixes too. Indeed, Mr Coyne has used it in
order to discover the existence of, and then investigate, the 22 different bugs from what
was called “the Bug Table” in the 2" Joint Statement, 11 of which Dr Worden agrees
in his written evidence. Dr Worden himself has discovered some of these bugs, from
the sample of the entries in the KELs that he had inspected. It is the KELs that are the
origin of these expert conclusions, due to the text in the entries that they contain.
606.
607
608.
609.
POL00112816
POLO00112816
This discovery of these different bugs by both experts has directly flowed from the
entries in the Known Error Log by Fujitsu, together with the associated PEAKs.
It is only fair to record that after contested disclosure had been considered at the first
hearing before me on 19 October 2017 (dealt with in detail above), the Post Office’s
leading counsel suggested the “pragmatic solution” that was giving Mr Coyne the
opportunity to inspect it. It was because of this that the matter of control was not
resolved at that point, as disclosure was permitted. Mr Coyne inspected the Known
Error Log at Fujitsu’s premises. He found it highly relevant. The claimants received
most of the KELs they had for the trial on 10 May 2018, with further KELs in respect
of particular bugs disclosed on 16 November 2018, one month after the first expert
reports of Mr Coyne and Dr Wurden had been served. Further KELs were provided in
2019, and a very substantial quantity of 5,000, well after the trial in October 2019.
There has been no detailed explanation for that very late post-trial disclosure in any
form of any witness statement from anyone at the Post Office or Fujitsu.
The KELS provided to the experts prior to their reports featured centrally not only in
Mr Coyne’s two reports, but those of Dr Worden too. They were also the subject of a
great deal of cross-examination by both sides. I consider that disclosure of the KELs to
the experts — the pragmatic solution suggested by leading counsel for the Post Office -
has been central in the discovery and investigation of the bugs, errors and defects that
the experts agree were, or are, present in the Horizon system, and also of the bugs,
errors and defects that are not agreed, but upon which both experts opine. Dr Worden
has also analysed them extensively, and devoted a separate appendix to them. The best
description of the KELs is taken from Dr Worden, who stated that they were “a rich
source of evidence”. The notion that they were not relevant, or did not contain relevant
material, is extremely difficult to fathom, and I do not understand why, or how, Fujitsu
would or could sensibly choose to inform its own customer of many years, the Post
Office, to the contrary. If Fujitsu’s description of the contents of the KELs had been
taken at face value and not challenged, then the knowledge now available to the two
experts and to the court about the extent of bugs, errors and defects would not have
been available, with the obvious detrimental impact upon the fair resolution of the
Horizon Issues.
The further developments, after the trial, that led to disclosure of another 5,000 KELs
by the Post Office after the trial in October 2019 are dealt with at [622] below and
following under the heading “Further post trial disclosure”.
PEAKS
610.
Turning to PEAKs, upon inspecting the KELs at Fujitsu’s premises in 2018, Mr Coyne
discovered the existence of a different category of documents called PEAKs, as these
were referred to within different KELs. These documents were requested from the Post
Office in July 2018. The claimants maintain that the Post Office were obstructive over
this, but whether that is correct or not, over 218,000 different PEAKs were disclosed
by the Post Office’s solicitors on 27 September 2018. This was two weeks before the
date for exchange of experts’ reports. A further 3,866 were disclosed on 25 October
2018 — two weeks after the experts’ first reports. OCP documents (which are mentioned
in PEAKs) were disclosed on 24 January 2019, and a number of PEAKs were identified
6ll
612.
613.
614.
POL00112816
POLO00112816
by Mr Coyne in his supplementary report which he had identified but which had not
been disclosed. These were provided (or extracts of them) on 18 February 2019, which
is only three weeks before the Horizon Issues trial started.
A large quantity of further documentation was then disclosed by the Post Office, on
dates that were not only well after the dates ordered for expert evidence to be served,
but on dates that were even after the intended end date of the Horizon Issues trial itself.
2,590 OCRs were disclosed on 23 April 2019, and other PEAKS were disclosed on 31
May 2019, literally one working day before Mr Coyne was called to give evidence on
4 June 2019. This highly unsatisfactory situation led to my seeking an explanation. I
consider an explanation was plainly required given the timescales. This is a large
quantity of documents, dealing with central elements of the Horizon Issues, and the
stage at which they had been provided was very late. PEAKs are important documents.
I was told in submissions that Fujitsu had discovered an old database that had been
copied more than ten years previously, and as soon as Fujitsu had told the Post Office
this, matters had moved speedily and the Post Office had provided the contents of this
old database to the claimants.
However, the day after this explanation was given, and during cross-examination, a
PEAK dated 21 August 2018 was put to Dr Worden and the court was told this was one
of the documents that had been disclosed on 31 May 2019. This plainly could not have
come from “an old database” in excess of ten years of age which Fujitsu had only
recently discovered. A further and more accurate explanation was plainly called for.
I had already ordered another witness statement to be provided by the Post Office’s
solicitors, this one being the 18" witness statement of Mr Parsons, which was served
on 19 June 2019. The content of that witness statement was highly unsatisfactory.
Firstly, it showed that the existence of the forgotten Fujitsu database came about
because of questions raised by the Post Office’s solicitors “during preparation for the
Horizon Issues trial at the beginning of April 2019”. The Horizon Issues trial in fact
had an enforced interlude, from 21 March 2019 when the recusal application was
issued, until that application was dismissed on 9 April 2019. In other words, those
questions were asked very late in the trial process. Further, Mr Parsons’ witness
statement stated, so far as the latter disclosure of PEAKS on 31 May 2019 is concerned,
the following:
“20. In the course of investigations into the bugs referred to by Mr Coyne and/or Dr
Worden, including in the Second Joint Statement, on 29 March 2019 Mr Lenton
identified PCO273234 to Ms Simmonds as being a document relating to the Drop and
Go Bug. For the next couple of months my firm continued to investigate these bugs and
anumber of further documents relating to these bugs came to light. PCO273234, along
with 17 other documents (excluding redundant image files) that had come to my firm's
attention when preparing for the Horizon Issues trial were disclosed in a single batch
on 31 May 2019.”
(emphasis added)
Mr Lenton is a member of Fujitsu personnel. The 2" Joint Statement was signed by the
two experts on 25 February 2019. This passage means that one month after that
document, Mr Lenton identified the specific document relating to the Drop and Go Bug.
This is one week after Mr Godeseth had completed his evidence. I find the rather
POL00112816
POLO00112816
glossed-over reference in the witness statement to “continuing investigations” that went
on “for the next couple of months” extraordinarily opaque, in the context of a trial that
commenced on 11 March 2019, and a document identified on 29 March 2019. I also
find the wholesale lack of any explanation of when the “further documents relating to
these bugs came to light” — and indeed how — puzzling, again in a witness statement
ordered to explain what was already a highly unusual situation. Further, the PEAK
relating to the Drop and Go bug was plainly identified by Mr Lenton, a Fujitsu
employee, on 29 March 2019. It is in my judgment verging on the extraordinary, in the
context of any trial that was actually underway, that it took two months more for this
PEAK to be disclosed to the claimants. This is particularly concerning in the context of
this trial between these parties on these issues. It is also obvious from the witness
statement that the PEAK in question dealing with the Drop and Go bug was not
disclosed immediately, and until “for the next couple of months my firm [ie the Post
Office’s solicitors] continued to investigate these bugs.” There is no reason that
investigations, whether ongoing or not, should delay disclosing a document of this
nature to the other side, particularly in the middle of such a trial as this.
615. The conclusion to be drawn from this is that disclosure was given in a manner that could
only have disrupted and delayed proper investigation of the issues contained in the
documents. In this specific case, that includes the Drop and Go bug. That document
should have been provided to the claimants very rapidly once it came to the attention
of the Post Office’s solicitors, not kept “for the next couple of months”, as Mr Parsons
puts it. In subsequent trials, if relevant documents of this nature are discovered by either
side or their legal advisers, it is a matter of some importance that disclosure is given
very rapidly. This is particularly important whilst the trial is underway. A failure to do
so risks disruption, delay and increased costs.
616. The experts agreed the following about PEAKs and their content. “Peaks record a
timeline of activities to fix a bug or a problem. They sometimes contain information
not found in KELs about specific impact on branches or root causes — what needs to be
fixed. They are written, by people who know Horizon very well. They do not contain
design detail for any change. They are generally about development activities and
timeline rather than about potential impact. Peaks typically stop when development has
done its job, so they are not likely to contain information about follow-on activities,
such as compensating branches for any losses.” It is also agreed, and indeed can be seen
from the actual PEAKs themselves, that some of them record observations of financial
impact. PEAKS do not however always use the symbol “£”, which was a search item
used by Dr Worden to create a set of 259 KELs that contained that symbol. Even where
they referred to a financial discrepancy, PEAKs sometimes just recorded the figure, or
sometimes used “pds”, and did not use the symbol “£”.
Release Notes
617. A spreadsheet listing the Release Notes was produced which shows that there have been
a great number of changes and updates both to Legacy Horizon and Horizon Online
over the years. The total number of Release Notes is 19,842 at final version. Each expert
provided an estimate of how many changes per week there have been to Horizon. Mr
Coyne’s estimate is approximately 19 changes per week. Dr Worden’s estimate is 5
changes per working day. Given the number of working days per week, these figures
are essentially the same. No details of the actual Release Notes were provided. The
spreadsheet was not, in my judgment, complete, nor was it in useful usable form. No
618.
619.
620.
621.
POL00112816
POLO00112816
Release Notes detrimental to the Post Office’s case have been produced. The Post
Office adopted an unhelpful approach to Release Notes, requiring the claimants to
identify from the very difficult spreadsheet which Release Notes they would like to see,
with an offer to provide those. On 14 February 2019 the claimants’ solicitors indicated
which ones they did want to see. On 20 February 2019, the Post Office’s solicitors
stated “we are currently taking instructions in relation to the requests”. Both the request
from the claimants, and the response, are wholly unhelpful and far too late. To be fair
to the Post Office, a late request will undoubtedly lead to a late response. Further, the
spreadsheet itself is almost wholly unusable.
I have commented upon the approach of the Post Office to disclosure in this litigation
before, and not in favourable terms. At the first CMC in October 2017, I said in a short
ruling on disclosure that the Post Office had been “obstructive” in its pre-action
behaviour in this respect, and that its attitude to disclosure up to that point had been
somewhat less than ideal. In Judgment (No.3) I made some further criticisms which it
is not necessary to repeat. It is regrettable that the Post Office’s approach to disclosure,
admittedly this time in conjunction with Fujitsu, has continued in some respects to fall
short of what is required. However, it does seem to be improving and it is to be hoped
that such improvement will continue. Particularly in litigation such as this, with
enormous distrust and suspicion on both sides, it is simply counter-productive to have
that atmosphere added to by an unnecessarily combative stance on disclosure.
The claimants submitted in their opening that the Post Office’s approach to disclosure
had the effect of impeding the claimants from obtaining a full view of the documents
and the totality of the Horizon system. I agree with that submission. However, balanced
against this, it does have to be remembered that the claimants did not issue any
applications for disclosure, as I have already explained.
The Post Office’s case in this litigation will be subject to exactly the same degree of
measured and independent assessment, and weighed in the same objective way, as the
claimants’ case, and as it would be were these failures not present. However, there must
be a change in approach to this group litigation. Disclosing documents late, and in the
middle of trials, in such sizeable numbers, and failing (as clearly identified from the
passage of Mr Parsons’ own witness statement referred to in [613] above) to act with
expedition when clearly relevant documents are unearthed actually during a trial, has
two distinct effects. It is disruptive to the proceedings and leads to an increase in the
costs and/or causes delay. It is the antithesis of co-operation, which the Civil Procedure
Rules expressly require. I consider that these passages within this judgment should
provide sufficient encouragement to a more constructive approach to prompt disclosure
of obviously relevant documents when (or if) they are discovered shortly before, or
even during the course of, forthcoming trials.
A PEAK discovered on 29 March 2019 relating to the Drop and Go bug ought to have
been disclosed to the claimants within a period of about 48 to 72 hours, not two months.
With modern electronic communication, that is a readily achievable timescale.
Further post-trial disclosure
622.
There was then further post-trial disclosure given by the Post Office during September
2019. However, the solicitors for the Post Office notified the court in a letter of 3
October 2019 that further previous information given to the claimants and to the court
623.
POL00112816
POLO00112816
in the Post Office’s e-disclosure questionnaire was wrong. The letter to the court stated
inter alia:
“We write further to the email sent by the Claimants’ solicitors to the Managing
Judge's clerk on 27 September 2019 at 12:29, which brought to the Managing Judge's
attention that 32 documents relating to the Horizon Issues Trial were disclosed by
Post Office on 25 September 2019.
During the course of responding to a number of queries that the Claimants’ solicitors
had raised in relation to the disclosure of these documents (which can be located at
[H/358] of the Horizon Issues Trial Bundle), it has come to Post Office's attention that
a statement within its EDQ is incorrect and that there may be previous versions of the
Known Error Log that have not been disclosed to the Claimants. Please find enclosed
a letter which was sent to the Claimants’ solicitors today explaining the background to
these documents coming to light and the proposals which have been made in respect
of providing any disclosure which may be sought by the Claimants.”
The actual statement that was wrong was further explained in the accompanying letter
to the claimants, which stated:
“In respect of your query regarding the fact that intervening changes to KELs are not
captured on the face of the KELs disclosed, we have made further enquires with Fujitsu
to confirm our understanding that previous versions of the KELs are no longer held.
We regret to say that these enquiries have revealed that our understanding was wrong.
As you will be aware, Post Office's EDQ stated that "[t]he KEL only contains the
current database entries and is constantly updated and so the current version will not
necessarily reflect the version that was in place at the relevant time. The previous entries
/versions of the current entries are no longer available".
This statement was based on info provided by Fujitsu. In response to our recent
enquiries, however, we were informed by Fujitsu on 30 September 2019 that this is
incorrect and that previous versions of KELs are available.
This takes Post Office greatly by surprise. It relied on the information provided by
Fujitsu_at the time of filing its EDQ that such documents were not available. It is
extremely sorry that this information was incorrect.
Regardless of whether these documents may or may not be "adverse", Post Office has
taken immediate steps towards arranging for the previous versions of these KELs to be
extracted and has instructed Fujitsu to begin producing a script to extract the documents
into a readable HTML format. Unless you disagree, once this script has been produced
we will ask Fujitsu to extract all of these versions.
When this has been done, again unless you disagree, we propose to take immediate
steps to disclose previous versions of the KELs which were referred to by either Dr
Worden or Mr Coyne in their expert reports or the joint statements or the bug table, or
were for any other reason included in the trial bundle.
Disclosure of the previous versions of these KELs (where there are previous versions)
will be given as a matter of urgency.
624.
625.
626.
POL00112816
POLO00112816
We invite you to tell us whether the Claimants also wish to be provided with disclosure
of previous versions of the KELs which were not referred to by the experts or included
in the trial bundle.
Given the seriousness of this matter, we propose to notify the Managing Judge of this
development immediately and will send him a copy of this letter under cover of an
explanatory email.”
(emphasis added)
This was correctly characterised by the Post Office’s solicitors as a serious matter.
Further, when these KELs were produced and disclosed, it turned out that there were
5,000 of them. This is an extraordinary number, and far more than were available to the
experts for the trial.
There was no explanation of the way that what was obviously now incorrect
information — that previous versions/entries of KELs were not retained by Fujitsu — had
been communicated to the Post Office’s solicitors, or by whom, and there was no
explanatory witness statement provided on a voluntary basis. I did not order one, nor
did this disclosure interrupt preparation of the draft judgment, or lead to any application
for further evidence or cross-examination.
J include it therefore for completeness. There are two points that arise from this.
1. It would have been a matter for argument, and an exercise in discretion, as to whether
any further submissions or evidence (including cross-examination) would be permitted,
had such an application been made. The Post Office were pragmatic and sensible, and
essentially left this decision to the claimants. The claimants made no such application.
However, had they done so, my preliminary view (without hearing any argument) is
that I would not have permitted the Horizon Issues trial to be resumed and further
evidence adduced. This is for three reasons. Firstly, there has to come a point when the
parties are given a decision on the Horizon Issues. The trial had already been interrupted
already to a substantial degree by the recusal application. This trial was supposed to be
completed in April 2019. Resuming it would have led to that date becoming sometime
in early 2020. The over-riding objective requires consideration of other court users in
other cases. Secondly, subsequent stages of the group litigation would, once again,
become consequentially delayed. It is with a sense of wistfulness that I recall a
statement at [16] in Judgment No.3 that this litigation would be speedily progressed.
Thirdly, the parties have told me that they wish to mediate before Christmas 2019 and
would like to know the outcome of the Horizon Issues trial to assist in that process. This
would not be possible if the trial were resumed.
2. This extensive further disclosure does, however, rather reinforce the criticism which
the claimants level at the Post Office and Fujitsu. The Post Office are the party in the
litigation, and they are the party with the disclosure obligations. They obviously rely
upon Fujitsu in this respect. Quite how Fujitsu, a reputable company with a worldwide
reputation who have been involved in Horizon for nearly 20 years, could be so wrong
on yet another important point is not explained, and will remain unclear. Given the
statements made by Fujitsu, relied upon by Leading Counsel for the Post Office at an
earlier hearing and repeated to the court, that the KELs did not contain any material
relevant to bugs, errors and defects, a pattern has emerged on the part of Fujitsu. This
POL00112816
POLO00112816
is that its statements about KELs have been shown to be clearly wrong on more than
one occasion, and in important respects.
Criticisms of the disclosure of the claimants
627
628.
629.
630.
631.
632.
I turn therefore to the criticism that the Post Office has levelled at the claimants. It
should be borne in mind that the bulk of the disclosure burden fell upon the Post Office,
simply because of the nature of the Horizon Issues. However, that does not mean that
the claimants were not themselves obliged to give relevant disclosure to the Post Office.
This disparity is obvious in the number of documents disclosed by each party, which
the Post Office in its Closing Submissions stated was some 517,965 documents by the
Post Office, and only 1525 by the claimants. Obviously that previous number does not
include the 5,000 KELs disclosed by the Post Office in September and October 2019,
after the submissions. The comparison in the figures demonstrates the different burden
upon the claimants and the Post Office. It is also correct to identify that for many
categories of very important documents, the Post Office is reliant upon Fujitsu. The
claimants are not.
The Post Office submitted that the claimants have making appropriate and narrow
requests for Model C disclosure of documents held by those claimants who were
witnesses in the Horizon Issues Trial, Cs have not engaged in discussions concerning
the scope of Cs disclosure nor provided disclosure of the documents sought by Post
Office, which were made in appropriate and narrow requests.
The Post Office, in the context of its complaints about the claimants’ disclosure,
submitted that “it was therefore not appropriate for Cs to serve claimant-specific
evidence from current or former SPMs” because of the nature of the Horizon Issues and
the terms of the Order that ordered witness statements, and that the Post Office sought
to avoid making disclosure requests nearer to the trial. However, once the claimants did
so, and served their nine witness statements in evidence in September 2018, the Post
Office’s solicitors wrote to the claimants on 22 October 2018, 7 November 2018, 30
November 2018 and 20 December 2018 and did not receive any response. Even though
the Common Issues trial was underway for part of this time, I do not see why such
letters should have gone unanswered. When the response did come, it was to the effect
that there would be no further disclosure as this was not envisaged by existing orders.
The Post Office contrast this with the disclosure it was prepared to give in respect of
specific witnesses, which Given that Cs had flouted that Order, Post Office sought to
understand what searches (if any) had been conducted by Cs so that it could understand
whether further disclosure would be required in light of Cs evidence. A further request
for details by Cs was made by Post Office on 17 January 2019.
This lack of response by the Cs should be viewed in light of the approach adopted by
Post Office. One example of the further disclosure that Post Office had agreed to give
which was outside that ordered by the Court was disclosure made in respect of the
“operation of branches by Angela Burke, Akash Patny, Anup Patny, Jayesh Tank,
Setpal Singh and Adrees Latif...”
The Post Office complains that the claimants “clearly intended disclosure to be a one-
sided exercise”, seeking disclosure of this nature but refusing to provide their own
claimant-specific exercise. Mr Tank did in February 2019 search within an archived
Subpostmasters “Yahoo Groups” forum where he used to post (which means makes
633.
634.
636.
POL00112816
POLO00112816
comments on the site) and after doing so Mr Tank identified a post he made on 13
December 2011 relating to the £195.04 shortfall referred to by Ms Van Den Bogerd,
and also a post he made on 29 September 2014 relating to the £600 shortfall that he
refers to in paragraphs 6-11 of his witness statement.
Mr Tank has also found a letter he received from Post Office’s Agents Accounting
Team in Chesterfield dated 13 October 2014 which related to this shortfall (and which
Mr Tank referred to in his forum post)."
Mr Tank was cross-examined about this, and explained that at the time of his first
witness statement he “‘hadn’t fully researched the whole background” and he did not
realise at that time that the post he had made previously was as important as it had
subsequently become, by which he meant in the trial. He also said he had only become
aware of the £195 loss to which it related once he had read Mrs Van Den Bogerd’s
witness statement. He also explained that he kept a box file and that was where he found
the documents. Complaint is made of the fact that the documents that were disclosed
should have been disclosed on 17 July 2018, the date for disclosure. I accept that, and
even if one makes the assumption in the claimants’ favour that they did not know at
that date that Mr Tank was to be a witness, certainly no later than the date of his first
statement (which was 28 September 2018) disclosure ought to have been given. Mr
Tank’s lack of understanding of relevance and disclosure should not have arisen, and
the existence of his box file should have become known to the claimants’ solicitors far
earlier than it was.
Further disclosure was also necessary in relation to Mr Latif. This came about because
he served an amended witness statement on I March 2019, which came about following
enquires with Mr Latif's former branch manager, who consulted the documents within
the branch to confirm the date on which particular transaction corrections had been
issued.
He explained this in his cross-examination:
"Q. First question about that: by amendment to your statement you now say that this
was in around January 2018 rather than March 2018.
A. Correct.
Q. What caused you to make that correction?
A. Thad a look at the -- we hold the records for the information in the office, so I had
my assistants look at the records, transaction logs and that's when I confirmed that it
was January rather than March. (Inaudible) was logs in March as well. We made calls
effectively every month to Horizon help desk concerning this issue.
. When do you say you asked your assistants to check about the date?
Yes.
. Sorry, when do you say that happened?
. It was after I made the initial statement, I was checking.
. Roughly when, Mr Latif?
. It would have been a few weeks ago, sir.
. So is it right that you didn't check those records from the branch before making
your witness statement?
A. No, I thought I was correct but I double checked and made sure that actually in
fact they were correct, those (inaudible), so I was right but initial incident happened in
DPOAPA PN
637.
638.
639.
640.
POL00112816
POLO00112816
January when TA (inaudible) transaction acknowledgement, the TC, the corrections,
they came in March.
Q. Okay. So you say, do you, that the TCs in relation to these transaction
acknowledgements came in March 2018?
A. Yes
Q. And you say that's something you have checked from your records?
A. Yes."
The Post Office made requests for disclosure during the trial, on 15 March 2019 (which
was the end of the first hearing week) requesting disclosure of various documents
relating to Mr Latif and Mr Tank and their answers in cross-examination. This also
included requests of CCTV recordings which Mr Tank said he had looked at. These
requests were refused on 27 March 2019, and the claimants stated that this refusal was
on the basis that they were either outside their control, no longer existed or were not
relevant to the claimants’ evidence and were not relied upon at trial.
Documents that were held in Mr Latif's former branch were plainly outside of his
control, even though his former branch manager had accessed them previously.
However, that does not mean that the former branch manager could at least have been
asked for them. Neither the claimants did that, nor indeed did the Post Office (so far as
I can tell). I consider the claimants’ attitude in this respect unhelpful, although it should
be noted that for at least the documents in respect of Mr Latif, the Post Office could
have taken its own steps in this regard, and seems to have chosen not to do so.
There are some unsatisfactory aspects to the Post Office’s submissions about the CCTV
footage. Mr Latif clearly said in his first witness statement at paragraph 7 “I also
checked the CCTV to make sure that I had performed the transaction correctly.”
Secondly, the Post Office’s closing submissions refer to requests for “the CCTV
recordings which were consulted by Mr Tank for the purposes of drafting his witness
statement”. He was asked about the CCTV during his evidence and did not say that he
had consulted if for the drafting of his witness statement, he said that he had checked it
at the time. Had the Post Office wished, it could have asked for that footage shortly
after receiving that statement. It is also the case that the event in respect of which he
referred to the CCTV occurred in July 2015. It may be that the CCTV images do not
still survive, so long after the event in question, but this point was not pursued in cross-
examination.
There is, however, one major difference between the claimants and the Post Office in
this respect. The Post Office were entitled to, and did, explore with specific witnesses
certain issues in respect of disclosure by asking the relevant person (in the above
situation, Mr Latif) questions about it. The claimants did not have this opportunity for
the following reason. The majority of complaints made about the Post Office arise as a
result of Fujitsu. Save for rare circumstances, such as Mr Parson’s specific witness
statement about the Royal Mail issue, all that the claimants and the court are told is
“Fujitsu say X” or “Fujitsu told the Post Office” such and such. Specific individuals are
not identified, and they are certainly not called as witnesses to be asked questions about
their approach, or (for example in respect of KELs) how they could have been so
factually wrong in their explanation of the KELs’ contents. I have formed a dim view
of the accuracy of information of this nature coming from Fujitsu, regardless of the
person in that organisation responsible for communicating it.
POL00112816
POLO00112816
641. Notwithstanding this, I wish to make one matter completely clear. If a witness refers to
a particular document in their oral evidence, and the other party in this litigation wishes
to see a copy of that document, then (absent issues of privilege) a request for disclosure
of it ought to result in it being provided. This is by far the most sensible and
proportionate way forward. It is in no party’s interests to engage in squabbling about
relevance of a specific document in those circumstances. It costs far more to argue back
and forth in solicitors’ letters than it does simply to give disclosure, and in the example
which I have provided, the relevance of the document is made clear by the fact that the
witness will have referred to it orally.
642. — It is also correct to say that notwithstanding the different burden upon the different sides
in terms of disclosure in the Horizon Issues trial, which has undoubtedly been far
heavier for the Post Office, this burden will not always be borne in these proportions.
The Post Office also have the benefit of a far higher number of legal advisers. As at the
date of this draft judgment, simply in terms of the total number of Leading Counsel
engaged, the Post Office outnumbers the claimants by 4:1. In terms of the number of
solicitors’ firms, the Post Office outnumbers the claimants by 2:1.
Summary on disclosure
643. The Post Office is, in my judgment, somewhat at risk of downplaying certain
unsatisfactory aspects of its own disclosure that have emerged during the Horizon
Issues trial. For example, the incident concerning the incorrect statements made to the
court about the Royal Mail refusing disclosure without an order of the court was
described in the Post Office’s closing submissions as a “mix up” which was “regrettable
but should not be the subject of criticism of Post Office.” I do not consider telling the
court something in express terms which is factually wrong can properly and correctly
be identified as merely a “mix up”. Two points that were expressly explained were
simply not factually accurate. The Royal Mail was not refusing disclosure, and it did
not require an order of the court before it would provide certain documents. It had not
even been asked for them.
644. Similarly, in its closing submissions the Post Office referred to the interlocutory
discussions about the Known Error Log as a “debate about relevance”. That again
downplays what had happened, including the positive submissions made, that the Log
was specifically not relevant and even if an order were made in respect of documents
showing bugs, errors and defects, the Log would not be disclosed (even if it were in the
Post Office’s control, which it was stoutly maintained it was not). The submissions
made by the Post Office on the adequacy of its own disclosure have been rather
undermined since the trial ended by the production (emanating from Fujitsu, once
again) of over 5,000 KELs, something rightly described by the Post Office’s solicitors
as “a serious matter”.
645. The claimants have not been entirely co-operative on their own disclosure, however. I
consider that the criticisms of the claimants for serving what was described as “claimant
specific” evidence are unjustified. At least one of the experts, Mr Coyne, adopted a
specific approach and investigated specific issues or occurrences, and there is no reason
that both of the experts could not have done so. Dr Worden did not, but that does not
mean that the claimants were wrong to do so. Some of the criticisms made by the Post
Office regarding the claimants are unjustified, therefore, but not all of them are.
Although I do harbour the suspicion that the Post Office may have decided that, on
disclosure at least, attack may be the best form of defence, some of the criticisms made
646.
647.
POL00112816
POLO00112816
in the Post Office’s closing submissions are justified. For example, simply stating that
“However, at present, it is not the Claimants’ intention to make another round of
disclosure in relation to the Horizon Issues trial” as was done in July 2018 is not co-
operative. Further, the reason that documents which Mr Tank had in his box, that were
plainly relevant, were not disclosed was because Mr Tank had not been asked for them.
Had he been asked — which is something that the claimants’ solicitors should have done,
once it was known he would be a witness — these could and would have been disclosed.
They certainly should have been.
The following points are of general application to both parties in this litigation. At some
point a party who seeks to complain to the court about disclosure ought either to shelve
its general disquiet on this subject, or make an application.
It is important to identify, so that the parties in this group litigation are aware, the court’s
approach to disclosure. This is given for the purposes of guidance, and cost-effective
and efficient progress of the group litigation.
1. Care must be taken when explanations or submissions are made to the court. If further
time is needed in order for accuracy to be achieved, that time should be sought. Those
giving instructions to counsel must be astute to ensure that they are giving correct
information.
2. All the substantive issues will be resolved on the evidence, the applicable law, and
taking account of submissions. A party does not obtain a preliminary advantage or
“head start” on the substantive issues because of any criticisms about disclosure.
3. Criticisms about the other party’s disclosure should be proportionate. Depending
upon the circumstances and the nature of the document, the fact that disclosure is given
late might be relevant, but simply because continuing disclosure is given by one party
does not mean that there is anything sinister lurking in the background.
4. The court can always be asked to resolve disputes about documents. Mr Coyne
sought a large number of different types of documents, but no applications were made
by the claimants even though these were not provided. Criticisms of the Post Office’s
disclosure by the claimants has to be seen in the light that the claimants did not consider
it sufficiently important at the time to make any applications to the court. This is not to
encourage applications. Sensible and well-advised parties who co-operate will usually
deal with almost all disclosure issues consensually.
5. The parties are expected to comply with their disclosure obligations. If a document
comes to light that should have been disclosed but has not been, it should be provided
promptly. If this is discovered actually during a trial, it must be disclosed very rapidly
indeed. As happened with the document referred to at [613] above, a document
concerning the Drop and Go bug was discovered actually during the Horizon Issues
trial, then (as Mr Parsons put it in his explanatory witness statement) “for the next
couple of months” kept by the Post Office’s solicitors while “research” was done. It
was then disclosed just before resumption of the trial “in a batch”. This is not working
to the time scale either required, or expected, by the court. This is a laissez-faire
approach to disclosure of an important document during the trial itself, and the
document should have been disclosed very promptly.
648.
649.
650.
651.
POL00112816
POLO00112816
6. Both parties will be held to the same standards. The Post Office has, so far, and given
the issues that have been tried, borne the bulk of the disclosure burden. The claimants
are expected to observe their disclosure obligations in the same way, and will be
expected to do so in the future when further issues and the claimants’ different cases
are tried in 2020.
7. Witness statements will be ordered from a party when the court requires an
explanation. Nothing else should be read into the ordering of a witness statement, other
than it means, precisely that -- an explanation is required.
8. It is obvious that the Post Office has had to rely upon Fujitsu to a large degree.
However, given it was Fujitsu who told the Post Office what the Known Error Log
contained — see further [584] below — Fujitsu has, so far, shown itself not to be entirely
reliable in this respect. Fujitsu are also responsible for the Post Office making a directly
incorrect important statement in its EDQ about retention of KELs, which led to the
disclosure of about 5,000 of these some months after the trial closed.
9. Adoption of the Model C procedure does not mean that parties can keep adverse
documents to themselves and not disclose them until a specific request is made. Such
adverse documents should be disclosed promptly.
10. Finally, disclosure is very expensive. The court will be astute to guard against it
becoming either satellite litigation or a weapon in the interlocutory arsenal.
I have exhorted the parties to co-operation many times before in this group litigation. I
will continue to do so. Eventually, it is to be hoped that they will emerge into the broad.
sunlit uplands of the over-riding objective on disclosure matters.
The Experts’ Agreements
There were four experts’ agreements in this case. Experts’ agreements are very
important. Permission to rely upon IT expert evidence was granted in the First CMC
Order dated 25 October 2017. The two experts met on a number of occasions and agreed.
certain points.
As will be seen from Part H of this judgment, the experts approached their respective
exercises in very different ways. However, they did reach some areas of agreement that
were very helpful. Their joint statements or experts’ agreements did however also set
out a considerable number of areas of disagreement. As the trial approached, I requested
through the parties that the joint statements of the experts focus more on areas of actual
agreement, rather than recitations of position and disagreements.
The 1° Joint Statement was dated 4 September 2018 and set out the following, inter
alia, at the very beginning of the statement:
“Each expert's approach to writing his report, and to this joint memorandum which
foreshadows their reports, could broadly be one of three possible approaches:
a) To focus mainly on negative points found in the disclosed documents about where
Horizon may have fallen short.
652.
653.
654.
655.
POL00112816
POLO00112816
b) To focus mainly on those aspects of Horizon which were intended to achieve
robustness and reliability, and the evidence implying that they succeeded.
c) To provide the court with a clear foundation for understanding the design and
operation of Horizon; then, building on that foundation, to provide a balanced
assessment of the ways in which Horizon succeeded, whilst addressing any disclosed.
issues where Horizon may have fallen short.”
This was the first area of disagreement. Mr Coyne stated in the section immediately
following, entitled Areas of Disagreement:
“In my opinion the technical issues can be answered without reference to the Claimants
high level allegations document. The issues are about how Horizon and its interactive
components operated and the processes employed by Post Office and Fujitsu in
supporting these systems and the data within.
Whilst my report will take a balanced approach, it is the case that many of the issues
require a deep focus on the occurrences of bugs, errors and defects as well as the
potential for modification of transactional data. Whilst context will be provided as to
how Horizon should work in typical circumstances, it is the non-typical operation where
focus will be placed.”
This approach was not agreed by Dr Worden. He stated:
“J intend to take the balanced approach (c).
To provide a joint statement at this stage which will be useful to the court, under each
Horizon issue the experts need first to agree what is worth agreeing having regard to
the case which is being alleged. For instance, it would not help the court for the experts
to list a number of detailed points as agreed, if one or the other expert thinks that those
points are irrelevant or unimportant to the case alleged; or if collectively the points
imply a view which, in the opinion of either expert, is not a balanced view. Therefore,
the areas of agreement may at this stage be limited and more high level.
My approach will also focus on the Horizon system. It will consider Horizon within its
wider business context, but I do not believe that the Horizon issues extend to offering
any opinion on the wider business practices of Post Office.”
I consider that the balanced approach, which both experts stated they would be
adopting, was the correct one. However, I do not find the following statement by Dr
Worden in this 1‘ Joint Statement as helpful:
“under each Horizon issue the experts need first to agree what is worth agreeing having
regard to the case which is being alleged. For instance, it would not help the court for
the experts to list a number of detailed points as agreed, if one or the other expert thinks
that those points are irrelevant or unimportant to the case alleged.”
It is a matter for the court to consider how helpful a particular point of detail, agreed by
the experts, is or is not to the issues to be decided. Obviously the parties can make
656.
657.
POL00112816
POLO00112816
submissions about that, but it is not for an expert to decide that a point of detail, upon
which he may be agreed with his opposite number, should however not be agreed
because that point will not be helpful to the court; or that it should not be included in a
joint statement. It is also not for experts to direct the court in any particular direction.
The Horizon Issues were carefully drafted by the parties, approved by the court, and
specifically included in an Order. They were the issues that the experts were to agree,
and detailed points of an IT nature that arose within those Horizon Issues should, where
possible, have been agreed.
Because Mr Coyne and Dr Worden approached their tasks so differently, the former
inspecting a number of PEAKs and KELs about 10 times greater in number than the
latter, this meant that a great amount of very detailed matters shown within those
documents were not agreed by Dr Worden. He approached his expert exercise from
what he described as “a top down” approach, a methodology that found its most stark
manifestation in his statistical analysis of likelihood in his Section 8.
I will include within this judgment, not the full text of all four of the different Joint
Statements, but some lengthy passages. This is to identify the large degree of agreement
between the experts. I will however omit (with some isolated exceptions) the passages
where the experts set out their separate, not-agreed, views.
The 1" Joint Statement
658.
I also quote at this stage from the 1* Joint Statement, signed on 4 September 2018, and
the points of agreement and disagreement in respect of Horizon Issue 1. This is because
in my judgment it is an important point of agreement, in respect of an important Horizon
Issue, but also it is useful to set out the approach within the Joint Statements.
“2. Issues and Position
2.1 Issue 1
Issue 1 — To what extent was it possible or likely for bugs, errors or defects of the nature
alleged at §§23 and 24 of the GPOC and referred to in §§ 49 to 56 of the Generic
Defence to have the potential to (a) cause apparent or alleged discrepancies or shortfalls
relating to Subpostmasters’ branch accounts or transactions, or (b) undermine the
reliability of Horizon accurately to process and to record transactions as alleged at §24.1
GPOC?
Areas of Agreement:
Evidence exists that bugs/errors/defects have caused actual discrepancies or shortfalls
relating to Subpostmasters’ branch accounts/transactions.
Each time any IT system (including Horizon) is changed there is the potential to
introduce new bugs/errors/defects.
Once bugs/errors/defects are discovered, they take time to resolve and therefore
systems such as Horizon often continue to operate with bugs/errors/defects with or
without workarounds in place.
Theoretically, bugs/errors/defects that existed within Horizon have the potential to
cause apparent or alleged discrepancies or shortfalls in relating to Subpostmasters’
branch accounts/transactions.
POL00112816
POLO00112816
In the event that any specific bug/error/defect had such an effect, the experts have
differing views as to the ‘extent’ of the impact that such bugs/errors/defects may have
had on branch accounts.”
(Emphasis added)
The statement continues:
“Areas of Disagreement:
Jason Coyne: As identified by select Known Error Logs (KELs), many thousands of
bugs/errors and defects occurred and many of these known errors had impacts that
recurred in different circumstances, therefore the reliability of Horizon to accurately
process and record transactions is questionable.
Such facts undermine the reliability of Horizon to accurately process and record
transactions.
Examples:
(i) the Calendar Square/Falkirk bug — which led to Horizon failing to recognise transfers
between different stock units, thereby affecting Branch accounts;
(ii) the Payments Mismatch defect — which affected at least 62 branches and was related
to the process of moving discrepancies into the local suspense account, thereby
affecting Branch accounts. This defect was not capable of being identified by
Subpostmasters, who would have believed from Horizon that their account was
balanced when it was not; and
(iii) the Suspense Account bug — which erroneously replicated suspense account items
for 14 branches from 2010 in the same monthly trading periods in 2011 and 2012,
thereby affecting Branch accounts.
In my opinion there is possibility of bugs/errors/defects existing in Horizon that have
not yet been discovered. Further, most bugs/errors/defects that exist in Horizon have
existed for extended periods of time before they were discovered.
I understand that Horizon had at least 19,842 ‘releases’ of new software, each one of
these could have introduced new bugs/errors/defects.
Dr Worden:
My current preliminary view is that undetected bugs/errors/defects in Horizon were
very unlikely to be the cause of permanent shortfalls or discrepancies in a branch's
accounts.
It is necessary to define measures for the 'extent' of issue 1. At least two measures are
possible: the number of such bugs and errors, and their net expected financial impact
on claimants’ branch accounts, compared to the total shortfall experienced by all
claimants, which is of the order of £18 million. I intend to assess both measures. The
latter measure may be more useful, because if any set of bugs has expected financial
impact much less than £18M, it can do little to account for the claimants’ shortfalls,
either collectively or individually.
I propose to assess these measures from a number of different sources of information:
(a) the defects of Horizon cited by the claimants in para 1 .3 of the outline; (b) the KELs
identified by the claimants in para 1.4 of the outline; (c) other KELs; (d) data on their
shortfalls provided by the claimants in their schedules of information; and (e) data from
other sources (if available).
Tam still scoping the data needed for (e). If it is available, I will provide it to Mr Coyne
as soon as I have it.
659.
660.
661.
POL00112816
POLO00112816
My preliminary analyses of the data on (a) - (d) imply that:
+ For the majority of KELs, (which record known bugs), it can be shown, from an
understanding of the robustness measures built into Horizon or otherwise, that they
would have no permanent impact on branch accounts.
+ The net financial impact on claimants’ branch accounts of any other bugs, including
undiscovered bugs, is very small compared to the claimants’ total shortfalls. I am not
yet able to quantify this, but I intend to do so.”
In my judgment, this presages Dr Worden’s statistical analysis of likelihood, which I
deal with as part of Section I. The following text used by Dr Worden should be
considered:
“<...if any set of bugs has expected financial impact much less than £18M, it can do
little to account for the claimants’ shortfalls, either collectively or individually”
The phrase “much less than £18M” could, potentially, direct expert consideration of
Horizon Issue 1 away from what it is in fact asking, towards a different issue entirely.
The experts were not tasked only with investigating bugs, errors and defects that had a
proven financial impact of (say) £18M, £10M, or any financial figure at all, and
certainly not one in the millions of pounds. This was for the following reasons. It was
never part of the Post Office’s position at the case management stage when the Horizon
Issues were discussed and considered, that there should be any sort of threshold for
financial impact, above which the experts should investigate, and below which they
should not. The wording of Horizon Issue I has already been addressed above, and this
uses the terms “possible or likely”. This notion of imposing a financial threshold must
have come from Dr Worden.
Nor is the litigation intended to be a wide-ranging enquiry into all of the Post Office’s
accounts with all of its SPMs over the 15 or 16 year period in question. The Post
Office’s position is that Horizon is sufficiently robust such that it cannot be the cause
of the different shortfalls complained of by the hundreds of different claimants. The
counter-position to that by the claimants is of that particular discrete sets of bugs had,
individually, an impact of millions of pounds upon their branch accounts. If that were
the claimants’ case, then Dr Worden’s approach as set out in this joint statement might
be understandable. Rather, the claimants’ case is that there are a large number of
different bugs, errors and defects that each affected each of their branch accounts over
time, with the end result being that each of them was (in accounting terms) identified
as owing the Post Office sums in the tens of thousands, or thousands of pounds (or less)
with the consequences which are well known. Some discrepancies were only a few
pounds, but are said to have occurred many times. Some were said to be tens of
thousands of pounds cumulatively. Some occurred many times; others less often, but
with greater individual effect. One bug — of which more later — is said to “double up”,
which would mean an exponential effect as a loss doubles, then quadruples and so on.
Tf one imposes, at the beginning of the expert investigation process, a filter or threshold
that looks for a “set of bugs” causing losses of “millions of pounds” that will have the
effect of restricting the field of investigation. There is no reason for such a restriction,
in my judgment.
On Horizon Issue 2, the experts were agreed, so far as that issue was concerned, that
the extent to which any IT system can automatically alert its users to bugs within the
system itself is necessarily limited, and while Horizon has automated checks which
662.
663.
664.
665.
POL00112816
POLO00112816
would detect certain bugs, there are types of bugs which would not be detected by such
checks. The experts were also agreed, in respect of Horizon Issue 6, that whilst Horizon
contains measures and controls for detecting system integrity concerns, the automatic
mechanisms have failed in the past. The experts did not agree as to the ‘extent’ of
prevention, detection, identification, reporting or risk reduction that the automatic and
manual control measures delivered; however, the agreement that automatic
mechanisms have failed in the past, which was contained in the 1*' Joint Statement, is
in my judgment important. It means that the Post Office cannot rely upon the presence
of automatic mechanisms or processes as being an answer to the consequences of
failures in system integrity.
It was also agreed by the experts that the causes of some types of apparent or alleged
discrepancies and shortfalls may be identified from reports or transaction data available
to the SPMs, but that other causes of apparent or alleged discrepancies and shortfalls
may be more difficult or impossible to identify from reports or transaction data
available to the SPMs in question, because of their limited knowledge of the complex
back-end systems. Identification required the cooperation of PO staff and the SPMs.
Again, I consider this expert agreement to be important. This was, essentially, the
essence of the complaints raised by Mr Bates himself in the very earliest days of Legacy
Horizon, namely that there was or were insufficient information or reports available to
him to enable him to identify the causes of apparent or alleged discrepancies. The
experts are agreed that, for some causes of apparent or alleged discrepancies this is
indeed the case, due to the complexity of the back-end systems (back-end means not at
the branch) and the limited knowledge a SPM would have.
The experts agreed that Horizon had evolved, and its robustness may have varied
throughout its lifetime. They also agreed that the level of robustness may have increased
or decreased as the system was changed. The existence of branch shortfalls was agreed,
but they were not agreed at the stage of the first joint statement that this indicated any
lack of robustness. They were also agreed, as at the date of their 3% Joint Statement,
that Horizon as it is in 2019 is relatively robust. However, given the span of the years
under consideration is 2000 to 2010 for Legacy Horizon, and 2010 to date for Horizon
Online, and given the agreement to which I have just referred that robustness may have
increased or decreased, this only takes one so far.
On Horizon Issue 4, the experts agreed that there are a number of actual reported errors
in data recorded within Horizon arising from (a) data entry, (b) transfer or (c) processing
of data. Therefore, the potential exists. The experts do not agree as to its ‘extent”’. Dr
Worden saw this issue as a subset of Horizon Issue 3. He felt that because of the
measures “built into Horizon”, errors of data entry, transfer and processing were very
unlikely to affect branch accounts. The experts agreed on Horizon Issue 6 that the
automatic measures and controls for detecting system integrity within Horizon had
failed in the past; they could not agree as to the extent that the automatic and manual
control measures delivered. Dr Worden also made the point that bugs, when discovered,
could not be fixed instantly and what he called “trade offs” were required when dealing
with practical release management.
So far as remote access and Horizon Issue 7 was concerned, at the time of the 1°‘ Joint
Statement Dr Worden was unclear about this and the technical document provided to
666.
667.
668.
669.
POL00112816
POLO00112816
him by Mr Coyne had not been received in sufficiently timely manner such that he
could consider it. By the time the Horizon Issues trial had started, the Fujitsu witnesses
such as Mr Parker had corrected their earlier incorrect witness statements and accepted.
that this could be done. However, those statements were not served until November
2018 so would not have been in existence as at the time of the 1* Joint Statement.
On Horizon Issue 9, the experts agreed that the causes of some types of apparent or
alleged discrepancies and shortfalls may be identified from reports or transaction data
available to SPMs. Other causes of apparent or alleged discrepancies and shortfalls may
be more difficult or impossible to identify from reports or transaction data available to
SPMs, because of their limited knowledge of the complex back-end systems.
Identification required the cooperation of PO staff and SPMs.
On Horizon Issue 10, it was agreed that by the very nature of rolling out fixes in any IT
system, including those implemented by Fujitsu, these had the potential to affect
transaction data or data in branch accounts. This could be seen as stating the obvious,
but at least the experts agreed it. The experts were also agreed that the use of tools and
facilities to do this should be auditable, however the maintenance of logs recording this
would be dependent upon retention periods and the size of the logs in question. Again,
this could be seen as stating the obvious. The experts were not agreed on Issue 12, how
many times the ability to do this was used, if at all. Nor was there any agreement on
Issue 13, the extent to which such facilities had the potential to affect the reliability of
branch accounts; Issue 14, a detailed issue with 7 sub-issues concerning functionality;
or Issue 15, which concerned Transaction Corrections or TCs.
The experts agreed, so far as the existence and use of permission controls is concerned,
which is the essence of Horizon Issue 11, that the usage of tools and platforms which
existed specifically for the purpose of accessing and modifying transaction data, should
be auditable. It was agreed however that the maintenance of logs would be dependent
upon retention periods and size. Such tools which were identified initially in the 1*
Joint Statement under Horizon Issue 10 areas of disagreement by Mr Coyne, were at
that stage split into the following:
1. Global Branches which would enable the input of transactions within Horizon as
though it had come from an actual Branch;
2. the Branch Transaction Correctional tool;
3. the Transaction Information Processing repair tool.
It was in respect of the permission control procedures concerning these tools that the
documented authorisation process was identified, using documents called the “Master
Service Change” or MSC documents and “Operational Control Procedures” or OCPs.
These documents were requested by Mr Coyne but no actual disclosure application was
issued.
The 2" Joint Statement
670.
The 2° Joint Statement is dated 25 February 2019 and identified and collated the
different bugs, most of which were given actual names, which had either been identified
671.
672.
673.
POL00112816
POLO00112816
and agreed, or which Mr Coyne maintained were bugs. This statement was in landscape
form and lengthy, and included what became to be referred to as “the bug table”. I adopt
the terms used by the experts for the descriptions of the different bugs, or alleged bugs.
The number of different bugs that were agreed by Dr Worden, by the time the trial
ended, was 11. The description of this in the joint statement itself was
“The structure of the document captures i) A table of bugs/errors/defects containing
evidence of financial impact upon branch accounts that both experts agree (or indicate
if they do not), ii) all expert agreements grouped by Horizon issue, and iii) additional
comments and observations input by the respective expert.”
Prior to the commencement of this litigation, the Post Office had acknowledged the
existence of only two bugs in Horizon. Mr Coyne’s evidence was that there were 29
relevant to Horizon Issues 1 and 4, and he clarified in his cross-examination that the
number that he considered had lasting impact upon branch accounts was 21. I deal with
that cross-examination at [789] to [793] below.
The 2" Joint Statement also said in its Introduction that:
“Because of time pressures and the complexity of the issues, we have not been able to
address all the Horizon issues in this joint statement. We will issue a subsequent joint
statement addressing those issues we have not yet addressed. For those issues we have
addressed in this statement (issues 1, 2, 9, 14 and 15), the layout and references are not
as polished as we would have wished.”
It is extremely useful for the court to have Joint Statements from experts, and I am
grateful to the experts for producing these statements, in particular what was called “the
bug table” in the 2" Joint Statement, in what is a very complex case, and when they
were under considerable time pressure. The bug table was very widely used in both
cross-examination by the parties and indeed in their submissions.
The so-called “Bug Table”
674.
The bug table, and my findings upon the bugs within it, are in my judgment central to
the Horizon Issues. Evidence conceming bugs in the bug table formed a large part of
the expert evidence that was tested in cross-examination. The different bugs, or alleged
bugs, identified in the bug table were as follows. I have added to the following the
summary of the position of the Post Office in Appendix 2 of its Closing Submissions,
which in some cases accepted there was such a bug, and qualified that acceptance in
most instances by references to what was said to be the impact of the bug. I deal with
detailed findings on those bugs that are in issue in the Technical Appendix to this
judgment. For the detailed submissions by the Post Office for each bug, the paragraph
numbering in Appendix 2 started again at I for each bug. This did not cause particular
difficulty, and because the Post Office explained that Appendix 2 had been put together
by different parts having been drafted by different counsel/solicitor teams, this is
understandable. I refer to it here, because otherwise a reader of the passages below
might become confused as to why there seem to be so many paragraphs all (for
example) numbered 2 in Appendix 2. The following summary of the bug table is
intended to be a useful summary for this judgment, and is not restricted solely to the
entries within the 2"¢ Joint Statement.
POL00112816
POLO00112816
1. Receipts and Payments Mis-match bug.
This was agreed in the bug table as an “acknowledged bug” which had an impact on
branch accounts. Dr Worden added in his comments that “Therefore, the extent of this
bug is well established, in the GJ analysis.” GJ is Gareth Jenkins. The identified year
of its effect was 2010. It is accepted by the Post Office in paragraph 7 of Appendix 2
as one of a number of “bugs with lasting impact (although they were resolved)”. In the
summary in Appendix 2, the Post Office stated that “In the event, however, this bug
resulted in transient impact only.”
2. Callendar Square/Falkirk bug.
This too was agreed in the bug table as an “acknowledged bug” which had an impact
on branch accounts. Its identified years of effect were 2000 — 2006. It is accepted as a
bug by the Post Office in paragraph 6 of Appendix 2, but is said to have had “transient
impact”. Dr Worden added different comments, including that “the bug arose from a
fault in the underlying Riposte software, so it is not surprising that it took Fujitsu some
time to understand it, or that they had to rely on the suppliers to fix it. It does not show
poor system design or support by Fujitsu”. That latter sentence is a little surprising, and
seemingly very defensive on Fujitsu’s behalf. This is because the Horizon Issues are all
about the operability and functionality of Horizon, not who is to blame for the presence
of any bugs, errors or defects. Riposte software was part of Legacy Horizon. The fact
that it is a product designed and supplied by Escher, and not Fujitsu, is not relevant to
the dispute between the Post Office and the claimants. In any event, and as accepted
and recited in paragraph 2 of the Summary of Appendix 2 dealing with this bug, “Mr
Coyne asserts that Bug 2: Callendar Square is a bug with lasting financial impact and
in JS2, Dr Worden appears to agree that there is strong evidence of this...”
3. Suspense Account bug.
This was agreed in the bug table as an “acknowledged bug” and its identified years are
2010 to 2013. It is accepted by the Post Office in paragraph 7 of Appendix 2 as one of
a number of “bugs with lasting impact (although they were resolved)”. Dr Worden’s
comments do not include that it had an agreed impact upon branch accounts. This is
because of a concept he introduced (expanded in his reports) of “transient effect”. What
this meant was that if there was an impact on branch accounts by something within
Horizon, but that was then corrected by a TC, he concluded it had a “transient effect”
upon branch accounts and dealt with it differently. I address this concept below. Other
comments of his in relation to this acknowledge bug are:
“Tt was a transient effect arising not from a fault in the software, but from a change in
database archiving policy in 2010. The delay in correcting it arose from a failure of
communication between PO and Fujitsu. Because the bug would only manifest itself
annually for any affected branch, the effects of this delay were not widespread.
Peak PC0223870 shows that Fujitsu were able to identify the branches affected, even
when Subpostmasters did not report it. There is evidence that the branches were
compensated, as I would expect from the normal error correction processes.”
4. Dalmellington bug/Branch Outreach Issue.
This was not acknowledged as a bug in the bug table, but is effectively accepted by the
Post Office as a bug. It is accepted as a bug by the Post Office in paragraph 6 of
POL00112816
POLO00112816
Appendix 2, but is said to have had “transient impact”. Indeed, for some time during
the trial the Post Office would refer to this as “an issue”. I find that it is a bug — that
much does not seem to be controversial, as paragraph 2 of the detailed part of Appendix
2 relating to this bug states the following:
“This is a bug which Mr Coyne states has lasting impact on branch accounts. Post Office
submits that there was no lasting impact on branch accounts.”
The years of its effect were 2010 to 2015. It is named in respect of the branch where it
was “discovered” in 2015, although the investigation undertaken by Fujitsu in that year
identified previous occurrences. There were over 110 occurrences of it over this 5 year
period (the actual number is probably 116, but at least 114) with 88 branches affected.
Some were impacted multiple times. It is what is called a “cash remming error” and
only happened in certain circumstances. It was explained further by Mr Godeseth in his
evidence at [445] onwards above. It would, in my judgment, have an impact upon
branch accounts — it is the extent of that impact that is in issue, whether it was lasting
or transient. Dr Worden relied upon what he described as “a well-tested process of
reconciliation and TCs to detect and correct errors in cash remming (used 20,000 times.
per year)” and said it was “straightforward for Horizon to detect any discrepancy
between a “rem out” and the corresponding “rem in” (a mismatch arising either from a
miscount, or a multiple count of a pouch) and then a TC can be issued.” He also added
that “this process catches and corrects remming errors, whatever their cause - including
if they arise from, or are provoked by, software faults.” He therefore implicitly accepted
that the Dalmellington bug was a software fault, although he did not say so in terms. I
find that it was plainly was. He was also reliant upon the process of TCs to correct it.
Further, the bug was there for 5 years and was not discovered, although its effects
doubtless were. I find that this was a software bug that impacted upon branch accounts
and I will come to the nature of its effect in my findings on the bugs that are not agreed.
5. Remming In bug.
This was not acknowledged as a bug in the bug table. However, Dr Worden accepted it
was a bug or defect in his cross examination and it is accepted as a bug by the Post
Office in paragraph 6 of Appendix 2, but is again said to have had transient impact. The
years it was said to have been present in terms of effect was “March — August 2010 and
recorded as fixed approx. 2011”. Mr Coyne said there were 14 branches affected. Dr
Worden’s entry in the Joint Statement relied upon TCs, and stated “As for the
Dalmellington bug, above — PO had a robust process for detecting and correcting
remming errors, whatever their origin. So, there were no lasting effects on branch
accounts.” In paragraph 2 of the detailed part of Appendix 2 relating to this bug, the
Post Office submitted that “any discrepancy would be transient as instances of this bug
are caught by automatic reporting.”
(emphasis added)
The answer to this is therefore dependent upon my answer to the point concerning lasting
and transient impact.
6. Remming Out bug.
This was split into two in the bug table, 6(i) which was identified in KEL acha508S and
6(ii) which was identified in KEL GMaxwell3853P. The former was identified as
February/April 2007, recorded as fixed approx. 2007, with Mr Coyne identified 57
POL00112816
POLO00112816
branches affected; and the latter was in May 2005 with one branch being affected. They
were both remming out issues, hence the grouping by the experts in the bug table. It
was not acknowledged as a bug in the bug table. However, it is now accepted as a bug
by the Post Office in paragraph 6 of Appendix 2, but is said to have had transient impact.
Again, for both of these iterations, 6(i) and 6(ii), Dr Worden used the same wording as
he had in the entry for bug 5, the Remming In bug, namely “As for the Dalmellington
bug, above — PO had a robust process for detecting and correcting remming errors,
whatever their origin. So, there were no lasting effects on branch accounts” (emphasis
added). In paragraph 2 of the detailed part of Appendix 2 dealing with this bug, the Post
Office stated that this “comprises two separate issues, only one of which was a bug. Any
discrepancy caused by either issue would be transient as instances of both issues were
caught by automatic reporting.” It was also said that Mr Coyne had conflated two unrelated
issues under the heading “Remming Out”. Given this was an entry in the 2™ Joint Statement
agreed by the experts, it is a little unfair to state that Mr Coyne had done this, as the entry
was obviously agreed by Dr Worden. However, it was split into 6(i) and 6(ii) and I will
consider each separately
7. Local Suspense Account issue, not the same as 3. Suspense Account bug
This was reported in 2010 and recorded as fixed in September 2010. It was not
acknowledged in the bug table, but is now accepted as a bug by the Post Office in
paragraph 6 of Appendix 2, although it is one of those bugs said to have had transient
impact. Mr Parker had identified 33 branches affected. Mr Coyne recorded what he said
were four associated KELs, namely acha5259Q (for which there were 6 PEAKs)(this
KEL is mistakenly recorded twice in column 3, with acha5838T only mentioned in the
text in that column); cardc2043L (10 PEAKs); PorterS199P (3 PEAKs); and acha5838T
(which states there are “two different but similar problems” and appears in the text, but
not in the list of KELs at the end of the text by Mr Coyne). Dr Worden’s comments
were that
“The KEL acha5259Q implies that PO and Fujitsu were able to identify all occurrences
of the problem, without being notified by any Subpostmaster. I would therefore expect
them to have corrected any impact on branch accounts as part of normal error correction
processes.
T would not expect evidence of all corrections to accounts to have survived to the
present day. Peaks and KELs are not used to record corrections of financial impact.”
He also relied upon a statement by Mr Parker that there was ‘Temporary financial
impact which would have been cancelled out in the following period by a corresponding
discrepancy’.
This introduces another concept, similar if not identical to Dr Worden’s “transient
impact”, and that is “temporary financial impact”. Dr Worden is correct that PEAKs
and KELs do not record corrections of financial impact; they do however record
financial impact, because when an SPM reaches SSC (which raises the PEAK) they
often start by recording “SPM says he/she has a problem in that...... ” and financial
impact is often then recorded. The Post Office in paragraph 2 of the detailed part of
Appendix 2 dealing with this bug states that “any discrepancy would not be lasting”.
This does therefore accept, albeit implicitly, that it would have an impact upon branch
POL00112816
POLO00112816
accounts. It is also submitted that it was a “teething problem” from the early days of
the Horizon Online pilot scheme.
8. Recovery Issues.
The Post Office does not accept these are bugs at all. These are not agreed as bugs by
the experts, and there are four different types included in the table, with years of effect
from 2010 to 2018. Mr Coyne’s entry stated that “The text within the PEAKS and KELs
suggests that in each case a branch account discrepancy would be evident and would
require correction by the Post Office.” Dr Worden stated that:
“The KELs and Peaks cited by Mr Coyne are not indicative of errors in Horizon. They
provide guidance on how to correct discrepancies caused by human errors or other
errors in transaction recovery (‘recoverable transactions’)
Because there were many such errors, there were many calls to the help desk and many
Peaks and KELs. Normally, correction of errors involved back office reconciliation and
issuing TCs. This was accurate and effective; I have derived an upper limit of £2 per
branch per month on the mean impact of erroneous TCs.
One important KEL acha959T was guidance to the back office MSU, not for
Subpostmasters”
There was a minor typographic error in the submissions in paragraph 4 of the detailed
part of Appendix 2 which referred to bug 9. However, the four different types of
recovery issues are addressed in the subsequent paragraphs and the conclusion
paragraphs deal with bug 8. This was later corrected in a sheet of corrections, which
clarified that paragraph 4 of the detailed part of Appendix 2 should have stated "Mr
Coyne states that Bug 8: Recovery Issues is a bug with lasting financial impact. Post
Office submits that it is not a bug at all". The relevant findings on these issues are dealt
with in the Technical Appendix.
9. Reversals.
This was not acknowledged as a bug in the bug table but is now accepted as a bug by
the Post Office in paragraph 6 of Appendix 2, but is said to have had transient impact.
Indeed, Dr Worden’s entry in the 2™ Joint Statement in relation to this stated
“Transaction reversals are a complex area which, like recoverable transactions, are less
familiar to Subpostmasters and are more prone to human error. They lead to many calls
to the help line and to many KELs and Peaks - not necessarily related to any fault in
Horizon.” This can be seen as yet another example of attributing fault to the SPMs
where possible, but given it is now accepted as a bug it is not necessary to consider that
any further. Mr Coyne’s entry in the 2"¢ Joint Statement stated “In April 2003 due to a
failure in regression testing, Horizon version S30 was released by Fujitsu and this
introduced a bug where the value of transactions reversed by Subpostmasters was
shown twice in the amount of the reversal in branch accounts.”
10. Data Tree Build Failure discrepancies.
This is accepted by the Post Office in paragraph 7 of Appendix 2 as one of a number of
“bugs with lasting impact (although they were resolved)”. The PEAK dealing with this
reads “Data trees have been failing to build fully, and the system has not been detecting
this.” Data trees were part of Legacy Horizon, and is used to build a summary (or
POL00112816
POLO00112816
picture, the word used by the Post Office in its Appendix 2) of the accounts. The
building of data trees is a software function, and given “the system” is Horizon and is
supposed to detect failures of this nature, it is difficult to see how it could ever have
been in issue that this was a bug. Mr Coyne’s entry in the bug table recites that
“Dugannon branch suffered a £43,000 discrepancy but the cause was not immediately
known. £52,814.29 at the Yate Sodbury Branch. £9,368.40 at the Appleby
Westmoreland branch.” These are sizeable sums and arose in the branch accounts. That
this is a bug was de facto accepted by Dr Worden in the bug table due to the text of his
entry which states:
“There was a bug which has potential impact on branch accounts, early in the lifetime
of Horizon. Soon after it arose, the error was trapped and detected by DEP and was then
soon fixed.
The fault was easily noticeable at branches before the error trapping which was soon
introduced and would be even more noticeable after that. Only three branches appear
to have been affected, as described by Mr Coyne.
Because it was so noticeable at the branch, and the Peak is concerned with a software
error rather than any other cause, I would expect any discrepancies in branch accounts
to have been corrected.”
(emphasis added)
This is therefore undoubtedly a bug, but findings in respect of this therefore depend
upon the nature of its impact on branch accounts, which are made in the Technical
Appendix.
11. Girobank discrepancies.
Mr Coyne considered this was a bug. Dr Worden did not and stated that “the first fault
concems reports. A fault in a report is not a discrepancy in branch accounts, and only
causes one if it causes a person to make a mistake.” This is now essentially accepted as
a bug by the Post Office, but it is submitted it had no branch impact. It is included in
paragraph 5 of Appendix 2 under the heading “the following bugs had no branch
impact.” The detailed part of Appendix 2 dealing with this submits that there is no
evidence of any financial impact upon branch accounts, let alone a lasting impact. This
was in the early days of Legacy Horizon. There are said to be six distinct issues arising
under this heading, and further detail on this is included within the Technical Appendix.
I do not know if what are called “giros” are still in use in 2019.
12. Counter-replacement issues.
There were two KELs associated with this dealt within the bug table. It is now accepted
as a bug by the Post Office in paragraph 6 of Appendix 2, but is said to have had
transient impact. Mr Coyne considered that when replacing a counter within a branch,
the process could result in “the total loss of a transaction”. Dr Worden stated that the
cause, recorded in the first KEL (which was created in December 2000 and last updated.
in July 2007) was that Riposte was coming online from the Recovery mode too early,
and causing messages to be overwritten. The nature of the correction was stated as being
“to find the overwritten transactions for reconciliation we need to look at the
Ripostemirror messagestore’ followed by detailed instructions”. Riposte, as has been
POL00112816
POLO00112816
explained, was part of Legacy Horizon and was a product provided by another
company, Escher, but it is plainly part of Horizon. Dr Worden also stated that “the
incident arose from a hardware replacement (probably from a hardware fault) not from
a fault in Horizon. It is a different kind of recovery issue.” In my judgment the hardware
is part of Horizon. Findings in respect of this are made in the Technical Appendix
13. Withdrawn stock discrepancies. The Post Office does not accept these as a bug. Mr
Coyne maintains that it is, and in the bug table he extracted part of a PEAK that stated
these “Can cause confusion and unexpected (though hopefully temporary)
discrepancies at branches by allowing them to declare stock which has already been
withdrawn.” Dr Worden stated that “some impact on branch accounts cannot be ruled
out, although it is small”. The Post Office detailed submissions in Appendix 2 of its
Closing Submissions concentrated on the fact that withdrawn stock — which of course
is part of the way that the Post Office manages its business, adding and removing types
of products from time to time — this removal of products is done by means of an update
to reference data, “and not a change to the core code in the system.” This again is
concentrating on the code, rather than the way the system operates. Findings on this are
made in the Technical Appendix.
14. Bureau discrepancies.
These relate to foreign exchange, hence the name. It must be differentiated from bug
23 (also foreign currency, but entitled Bureau de Change). This one arose in 2017. Mr
Coyne considered it to be a bug, and Dr Worden effectively agreed in that his entry
stated “This appears to be a system error with impact on branch accounts. Although it
is possible that a subsequent discrepancy between branch accounting and POLSAP
would reveal the problem, leading to a correction (e.g. see Peak PC0265443, and Mr
Coyne's para 3.146), I cannot be certain of this.” It is now accepted by the Post Office
in paragraph 7 of Appendix 2 as one of a number of “bugs with lasting impact (although
they were resolved)”. The detailed part of Appendix 2 states that Mr Coyne has drawn
together two distinct issues. Paragraph 4 of the detailed part also states the following:
“Bug 14: Bureau Discrepancies is a bug with the potential for lasting financial impact.
There are two distinct issues which fall under this heading. With regards to the first issue
the branch was made good and a fix was implemented. The second issue was not a bug in
Horizon nor an issue which could have impacted branch accounts; it created what was
essentially a cash flow problem for the branch.”
Findings on this are made in the Technical Appendix, but so far as the first issue, given a
software fix was required, it is undoubtedly a bug, and it is correct to record (as the Post
Office do) that there was the potential for lasting financial impact, as without that, there
would be no need for the “branch to be made good” because that means the Post Office
corrected the branch account discrepancy caused by the bug.
15. Phantom Transactions. There are three different issues grouped under this heading.
The Post Office does not accept these as a bug. Mr Coyne in his cross-examination
stated that this was referable to Horizon Issue 4. He also accepted that some of the
issues were not bugs in Horizon. Dr Worden relied upon the fact that what he called
“the master PEAK” was closed as “no fault in product”; however, the evidence of fact
on this is dealt with above at [209] to [213] when the PEAK was put to Mrs Van Den
Bogerd. Dr Worden stated that “There is no evidence for bugs in Horizon with impact
on branch accounts.” That entry was obviously before the cross-examination, which in
POL00112816
POLO00112816
my judgment provided greater factual information. I do not consider reliance can be
placed upon the Fujitsu conclusion in the PEAK. Findings on this are dealt with in the
Technical Appendix.
16. Reconciliation issues. The Post Office does not accept these as a bug. There are a
number of different issues grouped together in this heading. The issue was that the SPM
was shown a discrepancy on his or her screen. Mr Coyne accepted that the discrepancy
would not be shown in the branch accounts; Dr Worden stated that “as it concerns an
issue in reporting, the software fault (which was fixed after 5 months) had no direct
impact on branch accounts. The only effect of an error in this report would be to mislead
or confuse the Subpostmaster - probably leading him to check his figures more carefully
and costing him some time.” That accepts a software fault; the fault was in fact a
miscounting of the number of files by the system. However, there are six different
issues grouped under this heading and the process of reconciliation, which is effectively
the comparison by Horizon of two different sets of data, is part of its design function.
Mr Coyne in his cross-examination also stated that this was referable to Horizon Issue
4, rather than Horizon Issue 1. The recovery messages were held in the branch
messagestore. Findings on this are dealt with in the Technical Appendix.
17. Branch Customer discrepancies. This was an entry in respect of Horizon Issue 4
and recorded as such in the bug table. The Post Office does not accept these as a bug.
Mr Coyne accepted in his cross-examination that the entry in the PEAK did not suggest
any impact upon branch accounts. The entry in the bug table originated in a single
PEAK, although there were two PEAKs that relates to the same issue, which was a
discrepancy between the financial records held by the Post Office (from Horizon) and
a bank’s records, after a counter crashed whilst a transaction was being processed.
Findings on this are dealt with in the Technical Appendix.
18. Concurrent logins. It is accepted by the Post Office in paragraph 7 of Appendix 2
as one of a number of “bugs with lasting impact (although they were resolved)”. In
paragraph 2 of the detailed part of Appendix 2 dealing with this bug, the following is
stated by the Post Office: “Post Office accepts that Bug 18: Concurrent Logins had a
potentially lasting financial impact. There is no evidence of any discrepancy in the
Peaks referred to by the experts.” The problem was in the early days of Legacy Horizon
when it was possible for users to log in to two terminals at once. Dr Worden in the bug
table made the following useful summary statement:
“discrepancies could occur - manifesting themselves as a receipts/payments mismatch.
This had the potential to affect branch accounts. The mismatch would bring it to the
attention of the Subpostmaster, who would require it to be investigated, except possibly
in the case of small mismatches, which he might pass off as an error in the branch (e.g.
of counting stock).
“*..Fujitsu believed it was a problem with the underlying Riposte software, and passed.
it to Escher. In September 2000, the problem was 'Now formally fixed in Build 223
update 19 which was released overnight.’ However, the new release from Escher did
not, as it was expected to, fix the problem. Escher denied that is was a bug in Riposte,
but Fujitsu believed in July 2001 that 'This is clearly a bug in the Supplier's code’.
POL00112816
POLO00112816
Dr Worden considered that “these faults had to the potential to produce discrepancies
in branch accounts, of small amounts, for a short period of time”. How the bug is
therefore characterised depends upon my findings in relation to lasting and transient
impact, which is dealt with below. There is no doubt, in my judgment, that this was a
bug, and Fujitsu clearly recognised and recorded that in 2001. The fact that the bug was
in the code supplied by Escher, as part of Riposte, and not that of Fujitsu is wholly
irrelevant for the purposes of the Horizon Issues.
19. Post & Go/TA discrepancies in POLSAP. It is accepted as a bug by the Post Office
in paragraph 6 of Appendix 2, but is said to have had transient impact. The entry relates
to Horizon Issue 4 rather than 1. “Post & Go” are self-service terminals, that are no
longer available in branch Post Offices, and are only available in Crown and WH Smith
main branches. They are basically self-service kiosks, designed to avoid queuing, and
a customer can weigh a letter or parcel, the terminal will print the relevant
stamps/labels, and the item is then posted. The Post Office submitted that this issue was
irrelevant to the Horizon Issues trial as ’this issue does not relate to branches that are
the focus of this trial”. Findings on this are dealt with in the Technical Appendix.
20. Recovery Failures. The entry in the bug table identifies this as relating to Horizon
Issue 4. Post Office does not accept these as a bug and states that there is no evidence
of a bug in Horizon. There are three different PEAKs. Mr Coyne accepted in his cross-
examination that this (or these) should be removed from the table that he used as the
originator for his list of bugs (which was Mr Coyne’s Table 1). Dr Worden stated in the
bug table that “there was some implication of hardware faults, with a replacement of a
base unit, but the PEAK has no evidence of software faults in Horizon.” In one of the
PEAKs, he concluded that there was no evidence of any fault in Horizon. Findings on
this are dealt with in the Technical Appendix.
21. Transaction Correction Issues. This is de facto accepted as a bug by the Post Office,
but it is submitted it had no branch impact. Mr Coyne’s entry was that “Transaction
Correction bugs/errors and defects do not cause discrepancies with branch accounts
but” and he then listed various consequences. In his report he referred to “technical
flaws”. Dr Worden, in respect of some PEAKs, accepted they were a bug, such as
PEAK PC0129587 where he stated “In my opinion this bug would result in an
inconvenience to the Subpostmaster (inability to rollover to the next TP) but would not
result in inaccurate processing of any TC, or any impact on branch accounts”. The bug
is included in the list of paragraph 5 of Appendix 2 under the heading “the following
bugs had no branch impact.”
These are submitted by the Post Office as there being “no evidence of any financial
impact upon branch accounts”. Findings on this are dealt with in the Technical
Appendix.
22. Bugs/errors/defects introduced by previously applied PEAK fixes. This is de facto
accepted as a bug by the Post Office, but it is submitted it had no branch impact. It is
included in paragraph 5 of Appendix 2 under the heading “the following bugs had no
branch impact”. It is also submitted by the Post Office that “a number of PEAKs arose
in testing. The PEAKs that arose in the live environment do not indicate evidence of
branch impact.” It is listed in the bug table as concemming Horizon Issue 4. Mr Coyne
stated in the Joint Statement that “Branch accounts would be affected by this bug which
POL00112816
POLO00112816
would cause a discrepancy when handling cheques where the value of the cheque would
be doubled” although he accepted that the SPM in question was processing a cheque in
a different manner to that recommended. Dr Worden used the term “a fault” in his
entries, maintained that there was no impact on branch accounts if the SPM followed
correct procedures, and also in respect of another branch relied upon the fact that the
effect was an error of 2p only. Findings on this are dealt with in the Technical Appendix.
23. Bureau de change. These arose in 2005, 2006 and 2010, and should be differentiated
from bug 14, which arose in 2017, was during Horizon Online and is referred to as
Bureau Discrepancies. The Post Office does not accept this as a bug. The Post Office
also states that Mr Coyne has identified three issues with the same heading, and submits
that they all relate to user error. Dr Worden’s entry in the bug table in relation to one of
them states “Analysing the second KEL (2010) I noted: ‘Impact small until bug fixed -
rounding errors 10° in exchange rates.’ The Post Office submitting that this is not a
bug is a somewhat bold submission, given the KEL to which Dr Worden refers
expressly states that the problem is a different exchange rate appearing on the HNG-X
rateboard and the Horizon rateboard. It states “Problem — there is a bug in the code”
(emphasis added). Findings on this are dealt with in the Technical Appendix.
24. Wrong branch customer change displayed. Mr Coyne concluded this was a bug and
part of his entry in the 2"¢ Joint Statement was “the KEL explains that “the cash amount
entered is multiplied by the Qty and hence the new stack total is wrong”, this Horizon
bug was due to incorrect reference data and led to an incorrect amount of change being
displayed on the branch screen leading to the operator to provide the branch customer
with the wrong amount of money thereby leaving a discrepancy in Branch Accounts. It
is possible that the amount of change shown on screen is more than the actual money
tendered by the customer.” Dr Worden’s entry was “When analysing this KEL I noted
“Sounds like a genuine problem which may have led to giving the customer the wrong
amount - i.e. not recoverable.’ It is now accepted by the Post Office in paragraph 7 of
Appendix 2 as one of a number of “bugs with lasting impact (although they were
resolved)”. It is also said that it is a reference data bug, and that therefore once
discovered could be quickly fixed by changing the relevant reference data. Findings on
this are dealt with in the Technical Appendix.
25. Lyca top up. It is accepted as a bug by the Post Office in paragraph 6 of Appendix
2, but is said to have had transient impact. It is also a reference data bug, and paragraph
2 of the detailed part of the submissions on this in Appendix 2 states “Lyca Top Up is a
bug with the potential for lasting financial impact. This is also [a] reference data bug. As
set out above, the experts have agreed that that while reference data bugs may be a
significant proportion of the bugs with financial impact, once discovered, they could be
quickly fixed (by a change to the reference data) once the bug is correctly identified.” It is
also submitted that it was identified through Fujitsu’s automatic reporting. Findings on this
are dealt with in the Technical Appendix.
26. TPSC 250 Report. It is accepted by the Post Office in paragraph 7 of Appendix 2
as one of a number of “bugs with lasting impact (although they were resolved)”. The
origin of the experts discovering this bug is a KEL raised by Anne Chambers in
February 2005, and last updated in April 2008. Findings on this are dealt with in the
Technical Appendix. I do however find it notable that although Dr Worden says the
amounts involved are what he calls “small”, he accepts it is as a “back end reporting
POL00112816
POLO00112816
problem” — which is nothing to do with the SPM, by definition — and also the KEL
states that “the accounting tree has not handled this properly when calculating the daily
recon figures and it has resulted in a mismatch...” The mismatch in that case was 66p.
The accounting tree is part of Horizon. The KEL also appears to be incomplete because
only the first page is present. The section that follows the heading “Evidence” is entirely
missing. There is also nothing in terms of the text available that demonstrates what
occurred in 2008, even though it is plain for the page that is available that the KEL was
updated then. The Post Office submits that there are five separate issues drawn together
within this bug. Findings on this are dealt with in the Technical Appendix.
27. TPS. It is accepted by the Post Office in paragraph 7 of Appendix 2 as one of a
number of “bugs with lasting impact (although they were resolved)”. One of the Fujitsu
documents referenced in the 2"¢ Joint Statement states that the Transaction Repair Tool
or TRT is being used “to repair 1 harvester exceptions for” a particular branch and
“There is no correction to be performed and hence no call for confirmtiprepair - this is
just an oddity performed by that very flaky mails code.” (emphasis added) There were
40 associated PEAKs, and Mr Coyne observed that both the credit and debit sides of a
transaction were doubled, so the net impact of the bug was zero, although he drew
attention to the entry in two PEAKs that suggested that SSC requested confirmation of
any gain or loss at the counter. Dr Worden believed it was a back-end reporting
problem, although the chances of impact upon branch accounts were small. Findings
on this are dealt with in the Technical Appendix.
28. Drop and Go. It is accepted by the Post Office in paragraph 7 of Appendix 2 as one
of a number of “bugs with lasting impact (although they were resolved)”. This occurred
in July 2017 and related to a duplicate “Drop and Go” transaction for £100. This was
performed twice; the branch was debited with £200, but the customer credited with only
£100 (which had been the amount “topped up” by the customer). Mr Coyne considered.
it a bug with impact upon branch accounts; Dr Worden stated in his entry “My analysis
of this KEL was ‘Possible financial impact. Seems very visible on the counter. Script =
reference data - therefore fixed easily’. Paragraph 2 of the detailed part of Appendix 2
on this bug states “Peak PC0260269 relates to an issue involving a Drop and Go
transaction (a £100 mobile phone top up) that timed out on Horizon” (emphasis added).
This does not appear to be correct, and I do not believe that the transaction relates to a
mobile phone. However, I deal with this further in the findings in the Technical
Appendix.
29. Network Banking Bug. This related to a KEL which was raised in 2004 and updated
in 2005, with 12 associated PEAKs in the range 2004 to 2010. The Post Office does not
accept this as a bug. Mr Coyne stated that “Horizon appears to mis-handle
communications, leading to errors within network banking and in turn causing the
potential for branch account discrepancies.” One of these was pension transactions
being declined yet the customer’s bank account being debited; when the customer
complained, the SPM refunded the £50 in question. Dr Worden considered it was
“mainly about a communication problem from BT, outside Horizon” but also wanted.
to investigate further due to an entry that referred to a “CNIM own goal”. The Post
Office maintained there were two separate issues and the £50 was paid out due to “user
error”. Findings on this are dealt with in the Technical Appendix.
676.
677.
678.
679.
POL00112816
POLO00112816
The names given to the different bugs above came, sometimes, from the names applied
to the bugs by the Fujitsu personnel who knew about these bugs and worked upon them.
over the years. These names would sometimes appear in the PEAKS themselves that
identified the bugs or which represented the views reached by different personnel
within Fujitsu who investigated the background, sometimes identifying that previous
incidents had similar or sometimes identical characteristics to earlier occurrences of the
bug, and who worked to provide fixes or solutions to some of them, or came to
conclusions that there was nothing wrong with Horizon and attributed blame, fault or
responsibility to the SPM. Some of the names are self-explanatory. Some, such as
Callendar Square or Dalmellington, are the names of places where that bug was first
identified by Fujitsu.
Certain terms were used, dependent upon whether a particular bug was agreed as being
a bug (such as the Callender Square bug), or something else, such as “issues” or
“discrepancies.” However, as at the end of the trial, it can be seen from the above entries
that the experts’ agreements, the clarification provided by each during cross-
examination, and the other developments in the trial such as (one assumes) some
reflection by the Post Office on the actual text of PEAKs and KELs, lead to the
following situation. Of the 29 different entries in the bug table, eight were alleged by
the Post Office not to be bugs at all. They are those numbered 8, 13, 15, 16, 17, 20, 23
and 29. This is set out in paragraph 4 on the first page of Appendix 2 to the Post Office’s
Closing Submissions. This careful terminology therefore became a little clearer. The
remaining number could properly therefore be described as bugs, on the Post Office’s
own case, although their effect was challenged.
This means that the remainder of what is, in my judgment, a lengthy list, were accepted
to be bugs. In my judgment, that number of accepted bugs, even on the Post Office’s
own case about their effect, and the non-acceptance of the eight others, is a sizeable
number. It is approximately a ten-fold increase in the number admitted by the Post
Office prior to the Horizon Issues. It is correct that the Post Office did not accept lasting
impact on a number — a point I deal with below under “Lasting and transient impact” —
and maintained that only 9 “had the potential to cause lasting impact, but were resolved.
by the Post Office and Fujitsu” but, in my judgment, that is rather to miss the point. The
wording of Horizon Issue 1, agreed by the parties, and (one assumes) a major issue that
both parties wished to have resolved in order sensibly to progress this group litigation
(and omitting the references to the pleadings) is:
“To what extent was it possible or likely for bugs, errors or defects of the nature
alleged... to have the potential to (a) cause apparent or alleged discrepancies or
shortfalls relating to Subpostmasters’ branch accounts or transactions, or (b) undermine
the reliability of Horizon accurately to process and to record transactions.”
The use of the words “possible” and “potential” in this issue makes it clear, in my
judgment, that the number of “bugs, errors or defects” that fall to be considered is, even
on the Post Office’s submissions, 21 different bugs. I find that to be a significant total.
Part only of Appendix 2 to the Post Office’s Closing Submissions dealing with bug 16,
Reconciliation Issues, was, by administrative error, not submitted by the date for
closing submissions, or until some time after oral submissions were delivered. The Post
Office’s solicitors realised this in September. There were three missing pages. I am
680.
681.
682.
POL00112816
POLO00112816
satisfied that these were drafted prior to the date for service, and I allowed these to be
submitted late, together with a clarification document, and short responsive written
submissions by the claimants (as the claimants had been deprived of the opportunity to
address these three missing pages orally in July). I consider that I am permitted to do
this under the court’s general case management powers, and that to do this was to adopt
a reasonable and proportionate course. The Post Office’s Closing Submissions was a
very substantial document, and it was in my judgment sensible to admit the 3 pages
only that had been omitted from one of the appendices.
Other elements of the 2" Joint Statement were also agreed. These included that Horizon
had produced over 3 million sets of monthly branch accounts (a point relevant to Dr
Worden’s Section 8 statistical exercise), and also that the experts had different views
on “branch impact”. The relevant passages in the Joint Statement were that:
“Mr Coyne refers to any discrepancy that caused a loss (or gain) within branch accounts
that needed corrective action as an “impact to branch accounts”. Dr Worden only
considers an effect or impact on branch accounts where a discrepancy loss (or gain)
was not rectified by a correction such as a Transaction Correction.”
This in particular demonstrates an important difference in the approach of the experts.
Given, as I have explained above, that the Horizon Issues do not refer to Transaction
Corrections, which even the Post Office’s own witnesses explained had not been
addressed because the Post Office’s solicitors said these were outside the scope of the
Horizon Issues trial, I consider Mr Coyne’s approach to “impact on branch accounts”
to be the correct one. Dr Worden’s approach minimises or avoids impact on branch
accounts that have been caused by bugs, errors or defects, even where those that are (or
have been) present in the Horizon system have in fact affected the branch accounts of
SPMs. It is a feature that I take into account when I consider which of the two experts’
approaches and evidence I prefer. I also explain this further in “Lasting and Transient
Impact” below.
The 3“ Joint Statement continued the process of setting out the experts’ agreed position
by working through the remaining Horizon Issues. This statement dealt with Horizon
Issues 3 to 8 and was dated 1 March 2019. The experts agreed the following points in
respect of each. The numbering is mine. One entry below relates to one passage of Dr
Worden’s separate and not agreed view under Horizon Issue 3. It was not the only such
entry — in different places the experts would record their own separate views.
The 3” Joint Statement
Horizon Issue 3
“1. Irrespective of how you define the detail of robustness, in line with most other
large-scale computer systems, Horizon's robustness has generally improved. From our
experience of other computer systems, Horizon is relatively robust. We agree that
‘robust' does not mean infallible and therefore Horizon has and will continue to suffer
faults. Robustness limits the impact of those faults and other adverse events. This
increase in robustness has, in part, developed from Post Office discovering
bugs/errors and defects in live use and then applying fixes and improving monitoring.
POL00112816
POLO00112816
2. Computer systems are considered more robust if access to the back-end databases is
restricted tightly.
3. It was possible for some of the Horizon support staff working at Fujitsu to modify
the Horizon back-end branch database. In 2012, Post Office's auditors observed that
there were inappropriate system privileges assigned to the APPSUP role (which
allowed amendments to the BRDB).
4. Post Office does not consult the full audit data (unfiltered ARQ Data) before
deciding how to handle discrepancies and issuing Transaction Corrections.
5. Peaks show that some defects have lain undetected in Horizon for extended
periods without being diagnosed and fixed.
6. During the life of Horizon there have been 19,842 changes made to it via the
Fujitsu/Post Office release mechanism.
7. It is common modern IT development practice to make frequent incremental builds
and releases of software.
8. Specific release note detail has not been provided. Of the 19,842 changes, we
would expect that many were minor changes. It is likely that others contained changes
to improve the system or to fix bugs and defects.
9. The effectiveness of various countermeasures changed throughout the life of
Horizon.
10. Countermeasures are basic elements of practical IT system design
11, Countermeasures work by limiting the impact of Horizon bugs/error and defects
on branch accounts. Countermeasures do not always eliminate the effects of adverse
events (they are not perfect) but they are often effective in the area where they are
deployed; that is why they have become basic elements of practical IT systems
design.
12. It is difficult to measure the extent of the robustness of Horizon, apart from how it
might limit the extent of impact on branch accounts, as in Issue 1.
13. There are indications that in its first year of operation, and in the first year after
the introduction of Horizon Online, the system suffered from more problems than in
other years. One might expect a higher level of problems in these early periods. The
extent to which these problems were serious, or evaded countermeasures, or caused
discrepancies in branch accounts, is not agreed.”
14. Dr Worden stated the following, which was not agreed, following the entry
immediately above. “The main way in which the experts have assessed the extent of
robustness of Horizon is to ask to what extent failures of its robustness impacted
branch accounts. I have addressed this in Horizon Issue 1. In answering this question.
fluctuations over the years are of less importance than the sum over all years, if the
sum over all years is small. I have found the sum of impacts on branch accounts over
POL00112816
POLO00112816
all years to be very small”. (emphasis added). This approach is considered further by
me when I address Dr Worden’s Section 8 statistical analysis.
15. The agreed passages continued. “The users of any IT system play a role in
assuring its robustness. The designers of a system should not make unrealistic
assumptions about the users of the system. Unrealistic assumptions would lead to
inappropriate design, making the system less usable.
16. As Horizon has changed throughout its lifetime, the existence and effectiveness of
any countermeasures has too. To have considered the time dependence of all
robustness countermeasures over 20 years, would have made the expert reports
impossibly lengthy. There was not the time to do so.
17. Many software bugs can have the same effects as a user error (as illustrated, for
instance, by the Dalmellington bug, which produced a remming error)”.
(emphasis added)
Horizon Issue 4
“1, Bugs, errors and defects identified in relation to Horizon Issue I are often relevant
to Issue 4 in that they are ultimately errors arising from the processing of data in
Horizon.
2. There is evidence within the Peaks and KELs of bugs/errors/defects within Horizon
arising from parts (a), (b) and (c) of this issue that occurred without causing financial
discrepancies as well as some that occurred causing financial discrepancies.
3. Reference data is critical to the operation of Horizon and errors in reference data
have led to discrepancies in branch accounts.
(emphasis added)
4. Of the bugs which in the experts’ opinion had the potential to produce discrepancies
in branch accounts there may be some involvement of reference data in Bureau
Discrepancies, Bureau de Change, Wrong branch customer change displayed, Lyca
top-up, and Drop and Go. (Rows 14, 23, 24, 25, and 28 of the bugs table in the second
expert joint statement). It is notable that these bugs all concerned specific products
(arising from the reference data defining those products).
So, while reference data bugs may be a significant proportion of the bugs with
financial impact, once discovered, they could be quickly fixed (by a change to the
reference data) once the bug is correctly identified.
5. Reconciliation between transactions recorded on Horizon and transactions recorded.
by Post Office's clients is largely automated. Detected discrepancies were subject to
manual corrective fixes and/or the issue of Transaction Corrections/Error Notices to
the Subpostmasters.
6. The adequacy of Post Office back office processes to prevent discrepancies in
branch accounts can be measured by the quality of the TC process. This quality
includes:
+ The processes of consideration of available data
POL00112816
POLO00112816
+ The level of errors observed in the process
* The level of complaints or disputes raised following a TC
+ The level of upheld complaints following a TC
+ The level of financial impact of erroneous TCs
7. Errors in third-party data have led to discrepancies in branch accounts, through
erroneous TCs being issued on Subpostmasters.
8. PO does not control the level of errors made by its third-party client organisations
(which may lead to errors in TCs), or the delays in their processes (which may lead to
delays in TCs).
(emphasis added)
9. PO can and should ensure, by careful investigation of disputed TCs, that only a
small proportion of errors by PO clients lead to losses for Subpostmasters, provided
that the Subpostmasters are in good control of their branches and have the required
information available.”
Horizon Issue 6
“1. It is agreed that there are many measures and controls within Horizon that existed
to prevent, detect, identify report or reduce the risk of varying errors.”
Horizon Issue 7
“1, Fujitsu could access all transaction data recorded by Horizon.
2. Both Post Office and Fujitsu can read data remotely, and FJ needs remote access
for support purposes.”
Horizon Issue 8
683.
“1. Post Office had access to data which would not have been available to
Subpostmasters.
2. The descriptions of facilities for PO in the two expert reports are consistent and can
be taken together as a description of those facilities.
3. Post Office were reliant upon Fujitsu for diagnosis of whether the occurrence of
shortfalls was caused by bugs/errors or defects within the Horizon system.”
(emphasis added)
It can therefore be seen from the passages of agreement reached by the two IT experts
in the 3 Joint Statement, that a great amount of relevant material directly relevant to
the Horizon Issues was not in issue. It was agreed that errors in reference data have led
to discrepancies in branch accounts, and also that errors in third-party data have led to
discrepancies in branch accounts too. Further, the Post Office has no control over the
level of errors made by the third-party clients of the Post Office, and this is agreed. The
point also ought to be made that SPMs have no control over that either. Both errors in
POL00112816
POLO00112816
reference data, and also errors in third party data, being agreed to have led to
discrepancies in branch accounts, are points consistent with the claimants’ case in
respect of the Horizon system generally.
The 4" Joint Statement
684. This dealt with Horizon Issues 10 to 12, and was the last joint statement agreed by the
experts. It was dated 4 March 2019 and was agreed just before the trial commenced.
Again, the numbering is mine, and I also reproduce one passage from each expert that
was not agreed. One entry below in relation to Horizon Issue 11 was agreed with the
word “whenever”. Both experts agreed that this should be worded “usually when” and
this latter agreement on that wording was reached during the expert evidence phase of
the trial.
Horizon Issue 10
1. Dr Worden stated the following, which was not agreed, in relation to privileged
user access, and as one of a number of passages at the beginning of the Joint
Statement that were not agreed concerning remote and privileged access. “The
Privileged User Access logs are not a useful source of evidence about remote access,
including Balancing Transactions.”
The following were agreed:
“2. Within Horizon Online, a Balancing Transaction can be created using the BRDB
Transaction Correction Tool (called BRDBX015).
3. In the one acknowledged Balancing Transaction which has been disclosed, there
were SQL INSERTS into only two tables of the BRDB. These tables were
ops$brdb.brdbb_rx_eposs_transactions
and
ops$brdb.brdb_rx_rep_session_data”
4. Mr Coyne stated the following, as one of a number of passages that were not
agreed. “Outside of the Transaction Correction Tool other methods were available in
Horizon Online.
The APPSUP role used by Fujitsu has elevated privilege and could be used “for un-
envisaged ad-hoc live amendment not covered by BRDBX015” as outlined in the
referenced document.
Fujitsu also has the ability to perform balancing transactions via direct SQL
operations (using a command line interface) to perform corrective transactions in
Horizon Online (including deletions of operational data).
Such modifications may or may not be visible to the Subpostmaster and they could be
done without their consent.”
The following were agreed:
5. “‘Fixes’ implemented by Fujitsu had the potential to affect transaction data for
branch accounts, for transactions which occurred after the fix was deployed.
6. We have not been provided with logs or audits from the Transaction Repair Tool
(TRT).
POL00112816
POLO00112816
7. Certain facilities and procedures used by Fujitsu to repair the more common issues
which arose in Horizon were standardised, and evidence of them persists. However, to
repair less common issues which arose from time to time, standard tools and
procedures might not have been sufficient, and evidence might not persist of what was
done at the time. Even when evidence does persist, it may be extremely difficult for
the experts to interpret it today, because of the scale and complexity of Horizon.
Therefore, it is usually difficult for the experts to make categorical negative
statements of the form: ‘X or Y never happened’”.
Horizon Issue 11
All of the following are agreed entries:
“J, Evidence from several Peaks indicates that usually when [corrected from
whenever] Fujitsu needed to make any change to data which impacted branch
accounts, they were concerned to seek permission from PO to do so, and to ensure
that PO took responsibility for the resulting change.
2. For any large commercial IT system, it is necessary for some technical users to
have privileged access to databases, with wide-ranging capabilities, for system
maintenance and problem-solving purposes rather than application-related purposes.
It is important to keep the number of these users down to a minimum possible and for
each action that they take whilst logged in to be recorded and audited.”
(emphasis added)
“3. The logging of Privileged User Access (in PAU logs) commenced October 2009.
Between 2009 and 2015 these logs only displayed the fact that a Privileged User had
logged on or off but not what actions they had taken whilst the Privileged User was
logged in.
The use of the Transaction Correction Tool cannot be seen in these logs.
4. Atall times, any privileged user access log only shows what tables of BRDB were
accessed for a very small minority of accesses.
There are no privileged user access logs which show access to the two tables of
BRDB used to insert the one acknowledged balancing transaction.
5. The authorisation for the use of the Transaction Correction Tool for the
acknowledged Balancing Transaction can be seen in one OCP within the OCP
disclosure of 25th January 2019.
The record of the uses of the Transaction Correction Tool, including the one
acknowledged Balancing Transaction, can be seen in the .aud files, disclosed 22nd
February 2019.”
Horizon Issue 12
The following are all agreed entries:
“1. Post Office report that there have been 2,297 uses of the Transaction Correction
Tool. For each of these, the usage was logged and the SQL used to alter BRDB has
been disclosed.
685.
686.
687.
POL00112816
POLO00112816
Only one of these uses (POL-0512131) changed transaction data in two transaction
tables, in March 2010.
This Balancing Transaction consists of about 500 lines of SQL, in a pre-defined
template.
None of the other disclosed uses of the Transaction Correction tool contain the
relevant SQL INSERT commands such as: ‘INSERT INTO ops$brdb.brdb
Tx_Tep_session_data ...”
2. Usage of the Transaction Correction Tool is recorded in the
BRDB_TXN_CORR_TOOL_JOURNAL table within the BRDB.
The tools Low Level Design sets out that it has the capability to insert entries into the
following tables:
BRDB_RX_BUREAU_TRANSACTIONS
BRDB_RX_EPOSS_TRANSACTIONS
BRDB_RX_APS_ TRANSACTIONS
BRDB_RX_EPOSS_EVENTS.
BRDB_RX_NWB_TRANSACTIONS
BRDB_RX_REP_SESSION_DATA
BRDB_RX_DCS_TRANSACTIONS
BRDB_RX_REP_EVENT_ DATA
BRDB_RX_CUT_OFF_SUMMARIES"
Of the agreed entries in relation to Horizon Issue 11, I find the following of very great
relevance. The experts are agreed that for any large commercial IT system, which
Horizon obviously is, it is necessary for some technical users to have privileged access
to databases with wide-ranging capabilities. This however, is not for application related
purposes, but is for system maintenance and problem-solving purposes. It is also agreed
by both IT experts that it is important that the number of users with these abilities is
kept down to the minimum possible, AND for each action that they take whilst logged
in to be recorded and audited (my use of block capitals).
I consider that to be entirely conventional, obviously agreed by both the experts, and
also to be technically justified, particularly in a system such as Horizon which deals
with accounting. It is also not requiring an unrealistic level of perfection; it is simply a
matter of having properly controlled access to a system such as this one.
However, when one then considers the subsequent passages of the 4" experts statement,
it can be seen how far from this joint agreed (and technically justified) position the
Horizon system was. The logging of Privileged User Access (in PAU logs) commenced
in October 2009. For the period 2009 to 2015 — obviously a 6 year period - these logs
only displayed the fact that a Privileged User had logged on or off, “but not what
actions they had taken whilst the Privileged User was logged in”. Therefore the actions
they were taking when logged in were being neither recorded nor audited. All that could
be seen is they were logged in. Further, it has already been seen that the number of users
with the relevant privileges was not, in my judgment, restricted to a minimum. Further,
the use of the Transaction Correction Tool cannot be seen in these logs. Yet further, the
experts are agreed that at all times, any privileged user access log only shows what
tables of BRDB were accessed for a very small minority of accesses.
688.
689.
690.
POL00112816
POLO00112816
Yet further, it is agreed by the experts that there are no privileged user access logs which
show access to the two tables of BRDB used to insert the one acknowledged balancing
transaction.
What this amounts to, in my judgment, is a serious deficiency both in the required level
of controls in Horizon, in the recording of what privileged users were actually doing
(other than that they were simply logged on) and also a corresponding absence of
recording and auditing of those activities.
In my judgment, and based upon the experts’ agreement in this respect, this serious
deficiency lasted up to 2015. It also means, because of the extent (or rather the power)
of the APPSUPP privilege, the Transaction Correction Tool is but a small subset of the
way that records in the BRDB can be affected or changed by Fujitsu. It also means that
the tool’s Low Level Design document that was available in the trial bundle, and
considered by the experts, has been overtaken by use of the Transaction Correction
Tool, because the SQL INSERT command is not present in 2,296 acknowledged uses
of the tool. Both the number of uses, and the non-use of the SQL INSERT command,
are in agreed passages by the experts in the 4" Joint Statement. This is also a point that
was put to Mr Godeseth in his cross-examination (and accepted) and is dealt with at
[339] to [344] above.
Lasting and transient impact
691.
692.
Of the total number of bugs accepted by the Post Office, the Post Office submits that
nine of them had, or potentially had, only transient impact. These are the following:
Bug 2: Callendar Square;
Bug 4: Dalmellington;
Bug 5: Remming In;
Bug 6: Remming Out;
Bug 7: Local Suspense (not to be confused with Bug 3. Suspense Account bug)
Bug 9: Reversals;
Bug 12: Counter Replacement;
Bug 19: Post & Go/TA Discrepancies in POLSAP; and
Bug 25: Lyca top up.
Transient impact is a concept introduced by Dr Worden. It does not form any part of
the Horizon Issues as formulated, agreed and ordered. Dr Worden agreed, in his cross
examination, that he had added in the word “lasting” to Horizon Issue 1. He also said:
“, for better or worse I took my role to be concerned with lasting effects on branch
accounts rather than transient ones.”
693.
694
695.
696.
697.
POL00112816
POLO00112816
This is a plain and obvious departure from the wording of Horizon Issue 1, and indeed.
in my judgment represents a narrowing of it. I consider this departure unjustified. It is
not the issue drafted and agreed by the parties, and approved by the court.
Dr Worden also said in the 2"¢ Joint Statement (in one of the not agreed passages) that
“transient inaccuracies in branch accounts, which needed some form of correction, have
arisen so frequently and from so many causes that to list them is not useful; and that
evidence of each correction being carried out is unlikely to persist to this day.”
He stated that “not rectified” means lasting, whereas “rectified” means transient. He
also stated that if a bug caused something that required correcting with a Transaction
Correction or TC, then that would mean (if the TC were issued) that it were a transient
effect; but if the TC was never issued, that would be a lasting effect. The passages were:
“Q. So if the TC process takes a long time, then you might say something was lasting,
even notwithstanding years later it might be corrected?
A. That is not how I understood lasting —
Q. How did you understand lasting?
A. There were delays in the TC process which might be due to client organisations or
might be due to all sorts of things, and they could be at the outside, I believe, several
months. My definition of lasting did not depend on TCs coming in within a certain
timeframe. If TC never came in, that would be lasting, but if TC took several months
to come in that is not what I would call a lasting effect. A lasting effect is permanent;
it is at the end of the day, you know, he has lost money forever.”
The use of the phrase “lost money forever” demonstrates the artificiality, in my
judgment, of this distinction. Whether this was the intended purpose or not, applying
such a concept is to minimise as much as possible the impact on branch accounts of
bugs, errors and defects that otherwise exist. He also stated that “the branch accounts
would look wrong” for the period prior a TC being issued. That is undoubtedly correct;
the accounts would not only “look wrong”, they would be wrong. This makes it even
more clear that Dr Worden’s approach is at odds with the wording of the Horizon Issues,
as well as with the way which the Horizon System was operated.
His approach is, in my judgment, wholly artificial, for at least five reasons.
1. It ignores the express wording of Horizon Issue 1 itself.
2. It ignores the very function of Horizon, which was used as the branch accounting
mechanism between SPMs and the Post Office. A SPM would “roll over” their branch
each trading period, usually a 4 week period (sometimes a 5 week period). At the end
of each trading period, the SPM would produce a Branch Trading Statement (or “BTS”)
using Horizon, which was the accounting statement of the branch for that period. Any
shortfall or discrepancy in the accounts as demonstrated by the BTS would have to be
made good by the SPM (which means the amount of the loss would have to be paid) or
“settled centrally” (which means the SPM would seek time to pay). Therefore, any
occurrence of any bug, error or defect within that trading period would have an impact
698.
699.
700.
POL00112816
POLO00112816
on the branch accounts (by reason of the regular production of the BTS) at the end of
that particular 4 week trading period.
3. The issuing of TCs is done by the Post Office outside of Horizon in any event.
4. It pays no attention to whether a TC is issued within a branch trading period (and
hence before the creation of the Branch Trading Statement for that period) or
afterwards, in the next or any subsequent trading period. The fact that (for example) a
TC might be issued some months later does not mean that the branch accounts have not
been impacted. The branch accounts specifically have been impacted by the occurrence
of the bug, error or defect.
5. The correct characterisation of a bug, error or defect within Horizon, and its impact,
cannot sensibly depend upon an act (the issuing of a TC) that is done outside Horizon.
If a SPM were to find themselves reporting, for the first time, an incident to SSC in
relation to a hypothetical bug, bug H, and upon investigation SSC eventually were to
accept that there was a bug and a software fix was required, then the nature of bug H
should not depend upon how the PO chooses to correct it. If, say, a SPM had retired,
closed their branch, been suspended or in any other way ceased to be a SPM, then a TC
could not sensibly be used to them. One assumes that in this hypothetical scenario, the
PO would refund the SPM (or their estate, if they had died) but that step should not
affect the characterisation of bug H.
The point at (2) above is consistent with the way that branch accounting works. This
was the subject of a considerable amount of evidence at the Common Issues trial which
is dealt with in Judgment (No.3). I do not know if Dr Worden had read this judgment,
and he was not asked about it. However, I do not consider that any aspect of the outline
accounting process operated by the SPM using Horizon at the end of each Branch
Trading Period is controversial.
In my judgment, if any bug, error or defect within Horizon could have an impact upon
branch accounts in any particular trading period, then unless a TC were issued within
the same trading period, it would have had an actual impact upon branch accounts.
Similarly, if any bug, error or defect within Horizon could have an impact upon branch
accounts in any particular trading period, then — regardless of the issuing of any TC in
respect of that occurrence — it would have the potential to impact upon branch accounts.
This can be tested by considering the Callendar Square bug, number 2 in the bug table
list. The evidence I have identified at [401] to [414] above makes it clear that this bug
was discovered in 2005, and Mr Godeseth accepted that it had probably been present in
Horizon since its inception in 2000. It affected at least 30 branches, and was recorded
in one of the contemporaneous documents as “this problem has been around for years
and affects a number of sites most weeks.” Yet on Dr Worden’s somewhat artificial
constraining of impact on branch accounts, it would be categorised as only having a
transient impact. The dictionary meaning of transient is fleeting, passing, brief,
temporary, momentary, transitory, short-lived or ephemeral. In my judgment the use of
this word by Dr Worden and the Post Office is unjustified. The use of it by Dr Worden
was unnecessarily to restrict his analysis.
The Period 21 March 2019 to 4 June 2019
701.
702.
703.
704.
POL00112816
POLO00112816
During this period two matters of note occurred in the group litigation (in addition to
hearing the recusal application and handing down of Judgment (No.4), which it is not
necessary to address in this judgment).
The first was the further disclosure of documentation by the Post Office, which is
explained further at [611] above.
The second was service of a further supplementary expert’s report by Dr Worden
directly upon the court. This occurred on 22 May 2019. It came under cover of an e
mail from Dr Worden, copied only to the Post Office’s solicitors. It is extremely
unusual, if not verging upon unheard of, for an expert witness to communicate directly
with the trial judge, and directly serve material upon him or her. It is also the case that
no party (whether by itself or its solicitors) in ongoing litigation should communicate
with the court without copying in the other parties to the case. The only very narrow
exception to this is ex parte proceedings, in respect of which there are special rules, and
which is not the case here.
In the interests of transparency, I reproduce the whole e mail from Dr Worden here. It
was accompanied by some attachments: “Third Report Open”; and three appendices.
The e mail stated:
“Dear Judge,
As was explained by Mr De Garr Robinson in court on 11 April, I have made some
further analyses of Horizon Issues 1, 12, and 13. This work was done at my own
instigation, and was not prompted by the Post Office or its lawyers. In my opinion this
work has led to a material change to my opinions, and I am therefore obliged to inform
the court of those changes in a report. With this email I am sending you the report and
its Appendices, in completion of the action. I am doing this on the basis that the trial
will re-convene on 4th June, and so the court may need to be aware of it then. The report
is 13 pages long.
As the court is aware from my first expert report, I have interpreted ‘extent’ in Horizon
issue I as requiring a number - the maximum proportion of the claimed losses which is
attributable to bugs in Horizon. My first estimate of that number, in my first report and
revised in the second expert joint statement, is 0.4%. This estimate was adjusted in a
direction to favour the claimants, so that the court may better rely on it.
In my opinion, that estimate is still sound. The enclosed report contains an independent
estimate of the same number, which is 0.6%. Because that estimate is based on a simpler
analysis, and relies on evidence that in my view can be checked over a weekend, I
believe it can be of assistance to the court. That is why I started this analysis, and why
Jam now sending it to the court.
Similarly, for Horizon Issues 12 and 13, the analysis in the attached report is in my
opinion simple, easy to check, and leads to results which I was not able to derive before.
Therefore I believe it can be of assistance to the court.
A draft version of this report was first provided to Mr Coyne on 25th April, and a final
version was sent to him on 16 May. I have not yet been able to have a substantive
discussion of this report with him, other than my explaining points in the report to him.
I understand that when cross examined on the Horizon issues, this report is now a part
of my opinion and I may need to refer to it. It is also my understanding that Post Office
does not intend to make any application to the court in relation to this report.
Yours Sincerely,
706.
POL00112816
POLO00112816
Robert Worden”
(emphasis added)
The reference to what had occurred on 11 April 2019 is as follows. Following dismissal
of the recusal application on 9 April 2019, the trial resumed on 11 April 2019 with the
resumption of the Post Office’s final evidence of fact. This involved calling Mr Parker
to be cross-examined, and explaining that Mr Membury (another of the Fujitsu
witnesses) was unable to be called and the relevant Civil Evidence Act notice being
given under CPR Part 33.2. The position of the claimants on that latter point was
explained as their consenting to the necessary extension of time (rather than a waiver
by them under section 2(3) of the Act, a difference that is immaterial in the
circumstances). Mr Membury’s witness statement was therefore not cross-examined
upon, but Mr Parker was called and was cross-examined.
Upon conclusion of Mr Parker’s evidence, Mr de Garr Robinson, Leading Counsel for
the Post Office, explained that, as he put it, “a matter of some awkwardness” had arisen.
He submitted the following:
“Dr Worden has recently realised that there is a new way of looking at the evidence in
this case which, in his view, could greatly assist your Lordship, assist the court, in
deciding Horizon issues 1, 12 and 13.
This approach involves focusing on those Peaks OCRs, OCPs and MSCs which actually
mention the FAD codes of one or more of the claimant branches. Just to explain, when
Fujitsu did any authorised remote handling of data, to put it neutrally, which might
affect branch accounts, they raised an OCP, OCR or MSC whose text was likely to
include the six digit FAD code of the relevant branch. So it's therefore possible to
search all the OCPs, OCRs and MFCs with a view to finding all of those which mention
the claimant branches during the relevant claimant's period of tenure. This search yields
a limited number of OCPs, OCRs and MSCs, and it's therefore possible to assess expert
issues 12 and 13, which is how often was remote access facility exercised and what
effect did it have. It is possible to assess those questions as they affect the claimants
by examining that much smaller document set. My Lord, that is the first exercise that
he would like to undertake, and indeed he has embarked work on -- I think this week
he has embarked work on that.
Second, if a detected bug affected the accounts of any branch the Peak relating to that
bug was likely to mention that branch's FAD code. Typically, it will also -- it may also
mention a sum of money. It's therefore possible to search all the Peaks in the same way
that I have just outlined, looking for Peaks which mention any claimant's FAD code
during the relevant claimant's period of tenure, and again, this document could shed
some light on Horizon Issue I to which extent is it likely that bugs have affected the
relevant branches.
Now, Dr Worden has specifically asked me to offer his apologies to the parties and to
the court that he didn't think of this before. In fact, frankly, he is kicking himself that
he didn't do so. He believes that he and Mr Coyne would only need a short time to
consider the relevant documents and to consider how it affects their views on those
issues. He wishes to discuss the documents with Mr Coyne with a view to agreeing
what they do or do not show.
My Lord, in the days since the recusal application was issued he started to consider how
the new approach affects his views. He believes on Issue I it allows the parties to make
a much simpler analysis of the point, and he takes a similar view in relation to the
707.
708.
POL00112816
POLO00112816
remote access issues. It makes, in his view, it possible for the experts to form a view
as to how often remote access was exercised and what its likely effect was.
It is Dr Worden's view that it is his duty under CPR Part 35 to inform the court of this
change of view and to allow the court to consider whether or not it wishes to see it
considered. That belief is based, as your Lordship will be aware, on CPR35.3 which
imposes a duty on experts to help the court on the matters within their expertise,
whether or not they are instructed so to do. My Lord, it's also based on CPR35 -- I
should say the practice direction CPR35, paragraph 2.5, which requires experts to
inform the court of any change of views.
I should emphasise this -- none of this comes at the request or instigation of my client.
This has come from Dr Worden. This is his idea. My Lord, he wishes to discuss it with
Mr Coyne in a further meeting between the experts, but of course it's -- it's only right
that your Lordship should be aware of that. I'm not making any application for
permission to put in supplemental expert reports —
Mr Justice Fraser: I don't think you have any supplemental experts’ reports to apply for
permission for, do you?
Mr De Garr Robinson: I'm not making any kind of application, I'm simply sharing with
your Lordship the view that has been expressed to me by Dr Worden.”
The Post Office, in continuing this explanation of what was happening, referred to
another case which had concerned late expert evidence, the judgment in which had been
referred to in the opening submissions for the Horizon Issues trial. That case is Imperial
Chemical Industries Ltd v Merit Merrell Technology Ltd (No.3) [2018] EWHC 1577
(TCC).
The exchange with the court continued:
“MR JUSTICE FRASER: ...In order to apply for permission to adduce extra expert
evidence you would have to have a draft of the report for which you would be seeking
permission, wouldn't you?
MR DE GARR ROBINSON: My Lord not necessarily.
MR JUSTICE FRASER: You don't think so?
MR DE GARR ROBINSON: My Lord, I would submit not. It would depend on the
circumstances. Often one would have such a report. I'm conscious that in the JCI case
your Lordship cited as a reason for not giving the relevant party permission to put in a
report which they had prepared, that the experts hadn't gone through that collaborative
process and I'm quite anxious to ensure that my expert doesn't fall into the same trap, if
T can put it that way.
MR JUSTICE FRASER: Well, depending on whichever point one reaches in terms of
your actually make an application to put in a supplementary expert's report, that
application will be dealt with as and when it's made, so I'm not dealing with that either
positively or negatively at the moment.
MR DE GARR ROBINSON: Thank you.”
709.
710.
711.
712.
POL00112816
POLO00112816
It is not necessary to dwell upon the point, but any party seeking to obtain permission
to rely upon a very late additional expert’s report should ordinarily expect to have at
least a draft of such a report available. It is difficult to see how permission could be
given for evidence that was not available. However, no such application was made. The
trial had commenced on 11 March 2019. Dr Worden’s first expert report is dated 7
December 2018 and his supplementary report is dated 1 February 2019. This
notification to the court on 11 April 2019 of Dr Worden having “recently realised that
there is a new way of looking at the evidence in this case” came over four months after
the date of his first report, over two months after the date of his supplementary report,
and exactly one month after the Horizon Issues trial itself had actually started. I
requested a witness statement from the solicitors for the Post Office setting out the
chronology of this realisation by Dr Worden.
The case to which reference was made on behalf of the Post Office, JCI v Merit Merrell,
concerned a supplementary report by a quantum expert, instructed on behalf of ICI,
which was produced on the first day of trial without warning. This was based on an
analysis of documents the expert had only recently been provided with by ICI, which it
turned out had been in ICI’s possession for over three years, but which the expert had
not been given. The application for permission to adduce that supplementary report in
that case was refused. Points relevant to the application in that different case on those
facts were set out in [158] of the judgment, which stated:
“Two other points relevant to the application should also be identified. Firstly, Mr Kitt
embarked upon this exercise with no notice at all to his opposite number, Mr Linnett.
This is not co-operative behaviour. Mr Linnett should have been told that this was being
done, and the documents could have even been studied together by both experts so that
an agreed position could be produced by them on what the documents did or did not
show, and the use to which they could or could not be put. I conclude that the only
reason for keeping the fact that this exercise was underway, until after it was ready to
be served after the trial had actually started, was to cause maximum disruption to MMT.
This is a much discredited approach to civil litigation. I conclude that this can be the
only explanation for giving Mr Kitt the documents in a more usable (and searchable)
format than Mr Mills, MMT's solicitors.”
Giving permission for late expert evidence is a matter of discretion, and each case will
be different. At [237] in ICI v Merit Merrell had stated the following:
“5. Where late material emerges close to a trial, and if any expert considers that is going
to lead to further analysis, consideration or testing, notice of this should be given to that
expert's opposite number as soon as possible. Save in exceptional circumstances where
it is unavoidable, no expert should produce a further report actually during a trial that
takes the opposing party completely by surprise.”
Here, the following distinct matters fall to be considered. Firstly, just because in the
ICI case that expert performed a new exercise without telling his opposite number he
was doing so, which led to criticism, does not mean that all an expert has to do is to
inform his opposite number during a trial that he has decided to undertake a new
exercise, or produce a new report, and that will be approved. The whole ethos of
exchange of expert evidence well in advance of trial is so that each party (and their
713.
714.
715
716.
POL00112816
POLO00112816
respective experts) have advance notice of the opposing point of view, so that evidence
can be properly addressed and surprise avoided. That is undermined by service of very
late reports containing new analyses. Sometimes this is unavoidable and late material
emerges. It is not possible to be entirely proscriptive, as each case is different. That is
why permission of the court is required. Secondly, the stage of the trial at which this
Third Report was produced by Dr Worden was very late indeed. Thirdly, the extent to
which the court needs to consider all the circumstances depends upon whether the Third
Report was sought to be relied upon by the Post Office. Finally, and of wider
application, is the way in which the Third Report was dealt with, namely by service by
the expert himself directly upon the court.
Dealing with those matters in turn, the first two are effectively bound up with one
another. I do not know the actual date when Dr Worden realised that this new way of
looking at things appeared to him to be useful. However, the court was told he had
started thinking about this on 11 April 2019, and the claimants’ solicitors had already
been told of his intention by the Post Office’s solicitors the previous day on 10 April
2019 (one day after the recusal application was dismissed, and the day before
resumption of the trial). Thereafter, a draft report was provided on a without prejudice
basis to Mr Coyne on 25 April 2019 by Dr Worden, his “current workings” having been
provided on the same basis on 11 April 2019 and the appendices, again on a without
prejudice basis on 25 April 2019. There was then a degree of to-ing and fro-ing between
the solicitors, which led to the actual report itself being sent on an open basis to Mr
Coyne on 16 May 2019 (after another experts’ meeting on 15 May 2019).
Regardless of the exact date when Dr Worden realised there was a different way of
looking at the evidence in the case, this Third Report emerged at an extraordinarily late
stage in the proceedings. As can be seen in [716] below, it was based on material which
Dr Worden accepted he had had for some time.
Turning to the third of the matters identified in [712] above, this can be readily dealt
with. The Post Office made it clear, both on 11 April 2019 in the passages which I have
quoted above, and subsequently, that it did not seek permission to adduce the contents
of Dr Worden’s Third Report in evidence. It issued no application in this respect.
Therefore, it was not necessary for the court to consider the balancing exercise inherent
in resolving any application in this respect (on the basis that such an application was
resisted, which was likely). It is inherent within this decision by the Post Office not to
issue an application to adduce it therefore, that there was no explanation of when or
how Dr Worden had experienced this “new way of looking at the evidence”, other than
what has been identified above. It also meant that this report was not adduced as any
part of his evidence-in-chief, and he was not cross-examined about it. He referred to it
only once, very much in passing, and given the contents of it were not adduced in
evidence, that was as far as it went.
Had I been faced with a contested application by the Post Office to be permitted to rely
upon the contents of the Third Report in evidence, the matter would have been
considered in accordance with the principles in the CPR generally, CPR Part 35, the
Practice Direction that goes with it, as well as the Guidance for the Instruction of
Experts to Give Evidence in Civil Claims 2014 (“Guidance of Experts in Civil Claims”)
which in is the White Book with the Practice Direction. Doubtless such an application,
had it been made, would have been supported by a detailed explanation of all the
717.
718.
719.
720.
POL00112816
POLO00112816
relevant matters, together with argument on both sides. I do not have the benefit of
either a detailed explanation or of any argument. On the material of which I am aware,
but without hearing any submissions on it for obvious reasons, a substantial hurdle to
the success of such an application would have been its lateness. This is because of the
enormous disruption and yet further delay that would have been the inevitable
consequence of permitting reliance upon such a late expert’s report. Severe disruption
and delay had already been caused to the Horizon Issues trial by the time this report
emerged, and by late May, the trial was shortly about to resume after an enforced
interval of over two months. Had this new report been admitted into the evidence, Mr
Coyne would have required time to study it, and then potentially produce a further
expert’s report of his own, and none of that could have been accommodated between
late May (whenever such an application could have been listed) and 4 June 2019.
Calling him would either have been delayed, or he would have had to be recalled later
to deal with it. Further delay would have been undoubtedly caused. Delay is not always
fatal to such applications but the later an application of this kind is made, the stronger
the explanation for that delay has to be.
The claimants had also made the point on 23 May 2019, the day after Dr Worden had
served his Third Report upon the court, that it was based upon an analysis of documents,
namely OCPs, OCRs and MSCs, that Mr Coyne himself had requested in July 2018 and
which the Post Office’s solicitors had then said were irrelevant. In any event, when I
asked him at the end of his cross-examination, Dr Worden confirmed that he had had
the documents upon which he had based this Third Report “for some months” and that
he had had the full set of PEAKS since, as he put it, “way back in 2018”. As the Post
Office’s own counsel explained on 11 April 2019, Dr Worden was “frankly, kicking
himself” that he “didn’t think of this before”. The lateness seems therefore to have been
the consequence of Dr Worden failing to consider this approach until very late in the
day, rather than only very recently being provided with the material. That is not
promising ground for the making of such an application. It is also the case, in my
judgment, that there were no exceptional circumstances which made it unavoidable that
this new report was produced during the actual trial. Indeed, rather to the contrary. This
was entirely avoidable. Finally, the covering e mail from Dr Worden made it clear that
his estimate, included in his first report and revised in the 2" Joint Statement of 0.4%,
was “still sound”. This new exercise simply sought to bolster his conclusions.
I turn therefore to the fourth matter, the issue of direct communication with, and service
upon, the trial judge of an expert’s report by the actual expert rather than through
solicitors.
When the e mail was received to which I have referred at [703] above, my clerk
immediately sent a copy of the e mail and attachments to all the legal advisers for both
parties in the litigation, stating that witnesses should not communicate directly with the
court and also requesting that the claimants should not read any of the attachments until
the matter was addressed in court the next day (when the parties were attending in any
event to argue costs orders and other consequential matters following Judgment
(No.3).
The legal advisers for the Post Office did not seem concerned at the hearing of 12 April
2019 that this direct service upon the court by the expert himself had occurred. An
analogy was drawn with an expert seeking directions under CPR 35.14, and it was dealt
722.
723.
POL00112816
POLO00112816
with by the Post Office as though the direct service upon the court was simply the
actions of an expert seeking to fulfil his duties to the court, and entirely usual. It is,
however, not usual; it is highly unusual. Further, the covering e mail from Dr Worden.
makes it clear that the new report was not a change of opinion; in particular, the
sentence that makes this entirely clear is “in my opinion, that estimate is still sound”
conceming the calculation in his earlier reports. It was a different way of reaching the
same conclusion. Therefore, the provisions of paragraphs 64 to 66 in the Guidance of
Experts in Civil Claims do not apply.
I do not consider that any witness, lay or expert, should communicate directly with the
trial judge about the content of their evidence, or indeed about any subject at all. There
is a substantial risk that the efficient management of proceedings will be compromised
if this is done. Communications with the court should always be done by solicitors if
they are on the record. Further, communications between an expert and those
instructing him, or her, are privileged. If, as here, an expert takes it upon himself to
inform the court of applications that his client may, or may not, be intending to make,
there is a real danger that privileged information might be divulged, both to the court
and also to the other side in the litigation. This is far less likely if communications are
performed by solicitors, who have specific qualifications in legal matters and will be
experienced in litigation. Finally, there is specific provision in the Guidance of Experts
in Civil Claims dealing with the situation where an expert wishes to exercise their right
to ask the court for directions. This is contained in paragraphs 28 and 29. These
proposed directions have to be sent to the party instructing the expert at least seven days
before the request is made, and to the other parties in the litigation at least four days
before. They are to be made by letter in the form prescribed at paragraph 29(a) to (e).
That gives the other parties the ability, firstly to have advance notice of what is
proposed, as well as providing advance notice that the court will be asked for directions
and what those directions are, and why; and also give the expert’s instructing solicitors
the ability to consider the communication in advance so that privileged information is
not included.
In this case, that process was not followed. Mr Parson’s 17" witness statement
explained the outline chronology, and made it clear that Dr Worden’s opposite number
Mr Coyne had a copy of the Third Report, initially on a without prejudice basis, and
then on an open basis, and that Dr Worden told Mr Coyne on 16 May 2019 (when he
sent it on an open basis) that he would be sending the report to the court but not before
22 May 2019.
Finally on this subject, when I asked Dr Worden at the end of his evidence whether he
had ever preformed this type of direct service himself on the court before, he told me
he had not. He also said that he had sent the e mail direct to my clerk as this was what
he had been advised to do by the “Post Office’s lawyers”. He said that he had not given
his instructing solicitors the opportunity to check or approve the covering e mail, but
“had been given some advice as to what it should cover”. I assume that advice did not
include telling Dr Worden that he should inform the court of the Post Office’s potential
intentions on what applications it may, or may not, be minded to make in the litigation.
This is a highly unusual situation. In closing submissions the Post Office invited me to
provide guidance in this judgment as to what should be done, as though the position
was unclear. I do not know why it should be thought necessary to tell witnesses — lay
or expert - that they should not unilaterally communicate with the court. However, if it
POL00112816
POLO00112816
is necessary, no witnesses should communicate with the court directly in this way. Any
communications from legal advisers should always be copied to the other parties in
litigation. No expert should inform the court directly what applications his or her
solicitors might be contemplating, or not contemplating.
724. The points that I draw from this episode are as follows:
1. Experts should not serve their own material, particularly new material during a trial,
and whether or not accompanied by detailed covering e mails, directly upon the court
themselves. Service should always be done by a party’s solicitors if solicitors are on
the record.
2. A party’s solicitors will know that all communications to the court must be copied to
the other parties in the litigation.
3. Experts should not notify the court directly of the intentions of their instructing
solicitors and/or ultimate client in the litigation in which they are instructed. Such
matters are privileged.
4. It is difficult to be critical of Dr Worden in this respect, given he told the court that
he was advised by the Post Office’s legal team to send his report directly to the court.
5. Dr Worden must have considered the documents upon which he relied were, at least,
relevant. These were documents of a type sought by Mr Coyne in July 2018, whose
relevance was disputed at that stage by the Post Office. Mr Coyne’s request ought to
have led to these documents being provided to both the experts in the middle of last
year.
725. I will now turn to each of the expert’s evidence, their reports and their approach to the
Horizon Issues
1 The Expert Evidence
726. The subject matter of this group litigation spans a long period. Mr Bates was a SPM
before Horizon was introduced in 2000, and it is from Horizon’s introduction that year
that, on the claimants’ case, the problems and difficulties arose, and arose as a result of
the functionality of the computing system introduced by the Post Office, which in this
judgment is referred to as Legacy Horizon. As the Post Office pointed out regularly
during these proceedings, there are thousands of branches and millions of transactions
each year in all of those branches, which means that over such a lengthy period, there
is a vast amount of computing activity that had to be considered by the experts.
727. The expert evidence was therefore required to be substantial. The aspects of the
litigation that require expert evidence are not, however, of unheard of proportions, so
far as the High Court is concerned. The High Court often deals with matters that span
a lengthy period; it also often deals with matters of great complexity that require expert
evidence. By comparison with many other cases, one expert discipline alone, with only
one IT expert per party, is not a particularly expert-heavy case. The experts’ reports,
though lengthy, were not of an unusual size. There were two reports from each expert
that were adduced in evidence, and each of them was of a type that would be expected
POL00112816
POLO00112816
in a trial of medium complexity, which I consider the Horizon Issues trial to have been.
Nor was it necessary for the experts to examine every claimant SPM’s separate
experience; the whole purpose of having generic Horizon Issues was to avoid precisely
that. The Post Office would often concentrate on how vast the panorama was before the
court, but this may have been done simply to emphasise how few, in relative terms, the
claimants are; which also emphasised at the same time how well (so far as the Post
Office generally is concerned) the Post Office business operates day by day, week by
week, across so many branches. The experts agreed that Horizon has produced over
three million sets of monthly branch accounts.
728. At times, however, this focus on scale did seem to come close to an assertion that the
IT experts had an impossibly vast task; or that the court faced a significant challenge in
understanding all the expert evidence; or that the parties could not practically deal with
the other side’s expert evidence in the time allotted.
729. As has been set out in Part G of this judgment, the experts reached a great deal of
agreement in their different statements. Notwithstanding that, there remained large
areas of disagreement. Given the modern approach to trying cases generally, it was
neither possible, proportionate nor desirable to give the Post Office as much time as
their advisers maintained was necessary to conduct the Horizon Issues trial. As an
interesting comparison, in the case of The Ikarian Reefer dealt with further below,
which was before the implementation of the CPR and concerned the US$3 million loss
of a vessel, that trial took no fewer than 87 trial days. Such a lengthy trial, post the CPR,
would be highly unusual in a case of the financial scale of this one.
Time limited trials and the approach to detailed evidence
730. Prior to the introduction of the Civil Procedure Rules in 1999, fewer trials were time-
limited and some trials dealing with a large number of complex issues could run for a
great many months, if not far longer. Since 1999, the overriding objective was
introduced, and this is contained in CPR Part 1.1(1). This states “These Rules are a new
procedural code with the overriding objective of enabling the court to deal with cases
justly and at proportionate cost”. That is explained further in Part 1.1(2), which states
that:
“Dealing with a case justly and at proportionate cost includes, so far as is practicable:
(a) ensuring that the parties are on an equal footing;
(b) saving expense;
(c) dealing with the case in ways which are proportionate —
(i) to the amount of money involved;
(ii) to the importance of the case;
(iii) to the complexity of the issues; and
(iv) to the financial position of each party;
(d) ensuring that it is dealt with expeditiously and fairly;
(e) allotting to it an appropriate share of the court’s resources, while taking into account
the need to allot resources to other cases; and
(f) enforcing compliance with rules, practice directions and orders.”
731. Since the Civil Procedure Rules, trials are usually set down for a fixed time estimate
and are expected to be time-tabled to finish within that period. The court system does
732.
733.
734.
735.
POL00112816
POLO00112816
not have the luxury (if luxury it is) of giving endless periods of time to litigants to argue
their cases to the n" degree. This case, as with all litigation, is very important to those
involved in it — to all of the claimants, but also to the Post Office, as an institution
generally, and also to its other directors, employees, SPMs who are not claimants and
its clients too. The issues and different causes of action in both claims and
counterclaims are of considerable importance, and the amount of money at stake is not
small. However, the amount of money is not large in terms of specialist litigation, for
example. It is not measured in the billions, or even many hundreds of millions. Ignoring
claims for general damages, the amount of damages that are specified in liquidated
sums are said to be about £18 million (although that is only an approximate figure
currently).
Judgment (No.5) [2019] EWHC 1373 (QB) made it clear at [78] that the total of the
parties’ reported costs as at the date of that judgment was over £25 million (£12.6
million for the claimants, and £12.8 million for the Post Office). Since then, on 25 June
2019 the Post Office notified the court that its costs had exceeded £13.9 million.
The reason for explaining this in detail is to put into context the following points about
the scope of expert evidence and the number of trial days available for the experts to be
cross-examined. At the Pre-Trial Review, the Post Office submitted that at least four
days was needed for the cross-examination of Mr Coyne. It was realistically accepted
that longer than that was not likely to be permitted, although Leading Counsel for the
Post Office did submit that he would prefer longer. The claimants submitted that they
also wanted four days, but had trimmed that period to three days, for the cross-
examination of Dr Worden. Given there were further experts’ meetings to come, and
given my view of the scope of the expert material and considering the overriding
objective, I set a trial timetable that permitted each side two and a half days to cross-
examine the other side’s expert. That would have given a total number of five days of
cross-examination for all the IT expert evidence. At that time, I considered that would
be sufficient, and also proportionate, taking account of the overriding objective.
However, the subsequent Joint Statements following the PTR did not reach as much
agreement as I had (perhaps naively) anticipated. I therefore adjusted the trial timetable
and set a new one, with the Post Office being given four days to cross-examine Mr
Coyne, and the claimants being given three days to cross-examine Dr Worden. That
trial timetable also allowed an interval of one non-sitting week, to follow the evidence
of fact, for each side properly to prepare their time-limited expert cross-examination in
detail.
In my judgment, the period of time I had allotted at the PTR of 2% days each was
sufficient in itself; but the increased period of time allowed was undoubtedly sufficient
for each party properly to cross-examine and test the expert evidence adduced by the
other side in the litigation. Seven court days for the total cross-examination of one
expert discipline in a case such as this is, given the Horizon Issues in this trial, generous.
Counsel might not have had time to put each and every single individual point to the
expert on the other side, but they had more than enough time to put the main ones (and
a fair number of subsidiary ones too, for that matter). Each side had Leading Counsel,
well-versed in all of the issues, who have been involved in the case from at least the
making of the GLO itself in early 2017, as well as a team of other counsel assisting too.
Considerable pre-reading had been done by the court. As it turned out, the issuing of
736.
737.
738.
739.
740.
POL00112816
POLO00112816
the recusal application on 21 March 2019 led to an even longer period of the trial not
sitting, with only a half day (for Mr Parker) of the trial taking place between 21 March
2019 and 4 June 2019, a period of 75 days, or over ten weeks. The Post Office had a
different counsel team acting on the recusal application than the team acting on the
Horizon Issues trial. That 75 day period was a substantial period of time for each side
to prepare detailed cross-examination on all, and any, points that they considered
important, prior to the resumption of the trial and the commencement of the expert
evidence on 4 June 2019.
The amount of the costs burden explained in [732] above has been incurred on the basis
of the number of trial days made available. Obviously, further time for cross-
examination would have led to even higher cost figures. There was a sufficient and
proportionate amount of time at trial made available for the expert evidence to be
properly challenged and tested, and there was more than sufficient time available for
preparation of the time-limited cross-examination. I have set this out in detail in order
to make it clear that the time limited nature of the Horizon trial, and the number of days
allotted for cross-examination, has in my judgment had no effect on my findings on the
Horizon Issues, or restricted either party’s opportunity to put its own case and test the
case of the other party.
The other feature of time-limited cross-examination, given the depth of experience of
counsel, is that I permitted considerable latitude to the cross-examiners. If counsel
wanted to put the same point to an expert again and again; or spend an hour on one
specific sentence or detail, when the same point could have been accomplished in five
minutes; or devote significant time to high-level issues and avoid descending to
specifics; then that was a judgment call made by that counsel. With more junior counsel,
in another case, the court might discourage excessive time on less important or agreed
points, to encourage progress; but here both sides had highly experienced Leading
Counsel. I therefore intervened rarely. The amount of time made available in the trial
timetable for each side to cross-examine the other’s expert was set down in advance,
and if one side wished to devote (say) half a day to having an expert state, and then
restate, an agreed position on a particular point, then that was very much a matter for
them. Such an approach would lead to that period not being available to allocate to
other subjects, but in my judgment this case in particular benefitted from allowing the
parties to conduct their cross-examination, more or less, in the direction they wished.
I did, from time to time, seek either to have clarified what particular Horizon Issue was
the subject of the cross-examination, or to reference it to particular other subjects.
However, this did not occur very often, and the parties were permitted to use their
allocated time as they wished.
Each side, both in their cross-examination and closing submissions, criticised the other
side’s expert for not being sufficiently independent. This was done in different terms
and in different ways, but the upshot of it was broadly the same. It was that their expert’s
evidence should be preferred to that of the opposing side, for reasons of methodology
and approach, as well as for reasons of detail.
The court, when faced with opposing experts, might dismiss the views of one expert
entirely on the basis that they had not complied with their duties to the court. However,
it is not necessary to do that, in order to prefer the views of one expert over another.
TAL.
743.
POL00112816
POLO00112816
There may, depending upon the subject matter, be room for opposing expert opinion,
and it is simply a case that the court accepts one opinion for a variety of reasons. Also,
in some cases, it might be that one expert’s view is accepted on some points, and the
opposing expert’s views on others.
It is also not necessary to resolve every single point of detail between experts to arrive
at the answers to the Horizon Issues. However, given these criticisms of the experts
have been made, it is necessary to address them. This is because it is only fair both to
the experts, and to the parties, to decide whether each expert has been sufficiently
impartial in the exercise upon which they have been engaged such that the court can
reliably base its answers to the Horizon Issues upon the conclusions reached by the
experts. Impartiality and independence of experts is also of considerable, if not central,
importance to the administration of justice generally.
The case that is most often stated in the context of the importance of independence on
the part of experts is National Justice Compania Naviera SA v Prudential Assurance
Co Ltd [1993] 2 Lloyd’s Rep 68 (Comm Ct), but it is almost always referred to by the
name of the ship involved in the case, The Ikarian Reefer, which ran aground off Sierra
Leone and then caught fire, becoming a total loss. Lord Woolf MR, the author of the
Civil Procedure Rules, made clear in Stevens v Gullis [2000] 1 All ER 527, 533 that
CPR Part 35.1 and its accompanying practice direction “did no more than reflect the
position as it had been well enunciated in the authorities prior to the CPR coming into
force”. The Ikarian Reefer remains central to the way that experts should perform their
role, including the separate duty owed to the court.
In Imperial Chemicals Ltd v Merit Merrell Technology Ltd [2018] EWHC 1763
(TCC), a more recent case in which a number of experts all called for one party were
found to be partisan, at [237] the following was stated:
“The principles that govern expert evidence must be carefully adhered to, both by the
experts themselves, and the legal advisers who instruct them. If experts are unaware
of these principles, they must have them explained to them by their instructing
solicitors. This applies regardless of the amounts at stake in any particular case, and is
a foundation stone of expert evidence. There is a lengthy practice direction to CPR
Part 35, Practice Direction 35. Every expert should read it. In order to emphasise this
point to experts in future cases, the following points ought to be borne in mind. These
do not dilute, or change, the approach in The Ikarian Reefer. They are examples of
the application of those principles in practice.
1. Experts of like discipline should have access to the same material. No party should
provide its own independent expert with material which is not made available to his or
her opposite number.
2. Where there is an issue, or are issues, of fact which are relevant to the opinion of an
independent expert on any particular matter upon which they will be giving their
opinion, it is not the place of an independent expert to identify which version of the
facts they prefer. That is a matter for the court.
3.
4.
5. Where late material emerges close to a trial, and if any expert considers that is
going to lead to further analysis, consideration or testing, notice of this should be
given to that expert's opposite number as soon as possible. Save in exceptional
744,
POL00112816
POLO00112816
circumstances where it is unavoidable, no expert should produce a further report
actually during a trial that takes the opposing party completely by surprise.
6. No expert should allow the necessary adherence to the principles in The Ikarian
Reefer to be loosened.
It is to be hoped that expert evidence such as that called by ICI in this case, and also
in Bank of Ireland v Watts Group plc, does not become part of a worrying trend in
this respect. There are some jurisdictions where partisan expert evidence is the norm.
For the avoidance of any doubt, this jurisdiction is not one of them. Not only experts,
but the legal advisers who instruct them, should take very careful note of the
principles which govern expert evidence.”
The Ikarian Reefer, and the requirements of CPR Part 35, are the touchstones of expert
evidence in this jurisdiction. It is of the utmost importance that they are adhered to both
by experts themselves, and also by those who instruct such experts.
Mr Coyne
745.
746.
TAT.
748.
Mr Coyne is a Partner of IT Group UK Ltd, and has 30 years of experience in IT. He
has designed, programmed and supported Electronic Point of Sale (or EPOS) systems
for use in payment handling, stock control and distribution in addition to full business
accounting systems. He has also assisted public sector councils with revenue and
benefits processing systems, investigated failures within stock market trading systems,
European gambling and gaming networks and dealt with fraud in retail banking, point
of sale, cash in transit and electronic funds transfer systems.
He is a member of the Academy of Experts, the British Computer Society and the
Society of Computers and Law. He has an Expert Witness Diploma from Cardiff
University Law School, achieving this in 2008. He has been instructed as an expert
witness in both civil and criminal cases over the last 19 years, including by both UK
and International firms of solicitors, in connection with technology project fault
analysis, software development and digital forensic investigations. He has appeared
before different tribunals both in the UK as well as overseas. He served two detailed
reports with numerous appendices. He also signed four Joint Statements with Dr
Worden.
Both the experts agreed in the 3" Joint Statement that Horizon as it is now is relatively
robust. The passages in that Joint Statement are that:
“in line with most other large-scale computer systems, Horizon's robustness has
generally improved. From our experience of other computer systems, Horizon is
relatively robust. We agree that 'robust' does not mean infallible and therefore Horizon
has and will continue to suffer faults. Robustness limits the impact of those faults and
other adverse events. This increase in robustness has, in part, developed from Post
Office discovering bugs/errors and defects in live use and then applying fixes and
improving monitoring.”
I consider that this is an important passage of the experts’ agreement, and indeed
robustness is an important aspect of the Horizon Issues. It makes clear that, in part, it is
the discovery by the Post Office of bugs, errors and defects, the application of fixes
(which means changes to the software by means of a software fix) and the improvement
749.
750.
POL00112816
POLO00112816
of monitoring that has led to the increase in robustness that has been achieved in the
system as it is at 2019. This inevitably means that in earlier times, Horizon was less
robust, or not as robust as it is now. This applies both to Legacy Horizon and also to
Horizon Online. There are undoubtedly more modern or recent bugs in Horizon, as can
be seen from the Technical Appendix. Drop and Go, for example, was not even a
product until about 2013, and the relevant bug in the bug table which is bug 28 occurred
in July 2017. That caused an impact to the branch account in question of £100; modest
or tiny in comparison to the total Drop and Go business for that year, or even that month,
but still £100 which appeared as a deficit in the branch accounts. Until it was discovered
the bug could not have been fixed. It would not have been present prior to 2013 (because
Drop and Go did not exist) and any fix cannot have occurred prior to July 2017, if not
a month or two later. The agreement that Horizon is relatively robust as at 2018 and
2019 is something that is doubtless of great importance to the Post Office, as it affects
how the Horizon System as it is at the date of this judgment is both used by the Post
Office, and perceived by the SPMs who use it every day. The Horizon Issues however
deal with Horizon from the year 2000 onwards.
The other aspect of this part of the experts’ agreements that bears consideration is that
the Post Office have to know about bugs, defects and errors in order that they can agree
with Fujitsu that fixes have to be written and applied. This makes it particularly
important, in my judgment, that Fujitsu were open and transparent with the Post Office
about the presence of software bugs, errors and defects. Given Horizon was being used
by the Post Office as the accounting procedure between it and its SPMs, there is every
reason that Fujitsu would (or should) have to make the Post Office aware of bugs, errors
and defects.
A point that was put to Mr Coyne (it was taken from the Post Office’s pleadings, but it
is a theme that runs through both its submissions and also the evidence of Dr Worden)
is that there are both what are called technical controls, and also operational procedures
and practices conducted both by the Post Office and by SPMs, to increase the reliability
of data. The passage of the Defence that was put was as follows:
“Q. So robustness -- we can move on to paragraph 54 on page {C3/3/23}:
" ... in addition to the technical controls referred to above, there are several operational
procedures and practices conducted by Post Office and subpostmasters that serve to
increase the reliability of the data stored in the central data centre as an accurate record
of the transactions entered on branch terminals.
Then there is a list of those. So you will see that what's being said is that the robustness
of the system is said to be affected both by technical measures and by human measures.
Do you see that?
A. Yes, I can see that. That is what is said, yes.”
In my judgment it is correct to describe the second part of these, as the Post Office did
in its pleading, as “human measures”. They rely upon humans actually performing or
being aware of certain occurrences. Two of these rely upon SPMs, namely those at
paragraph 54(3) and 55(4) of the Defence. The second of these is that “the accounting
and record-keeping obligations placed on Subpostmasters reduce the risk of any errors
752.
753.
POL00112816
POLO00112816
going undetected.” In my judgment, for SPMs to be an effective — or indeed any — part
of a “human measure” and to assist or contribute to the process of the Post Office
discovering bugs/errors and defects in live use (so that, where necessary, software fixes
can be initiated, developed and applied) the Post Office ought sensibly to be transparent
with SPMs too.
It does however need to be recognised that human measures are not part of the Horizon
system. They are not included in the definition of the Horizon System that forms part
of the Horizon Issues themselves.
It was also, in substance, put to Mr Coyne that Horizon Issue I should have been
addressed in the way that Dr Worden had approached it, by effectively looking at actual
(rather than potential) impact, although it was not put in exactly those terms.
“Q. Given it was admitted that there have been occasions when these things have
happened, to answer the question: is it possible or likely that these things have happened
by saying "yes" isn't a very helpful answer, is it?
A. Ifthe answer was just simply "yes" and with no further context then I agree that that
would ...
Q. The critical issue that's raised by the four issues that I have read to you is the word
"extent" and the word "likelihood" and the word "risk", isn't it?
A. They certainly are critical elements that would need to be reviewed, yes.
Q. What you were being asked to do by those issues is to give your opinion to the court
on the extent to which it was and is likely that bugs caused shortfalls in any particular
set of accounts?
A. Well, across the accounts of subpostmasters, yes.
Q. The extent to which various measures in place reduce the risk faced by any given
subpostmaster when doing a transaction or when producing a particular set of accounts,
yes?
A. Yes.
Q. That's the critical question on which the court needs your assistance, would you
agree?
A. It is one of the critical questions, yes.
Q. Could you identify a question which you say is more critical than that in the context
of these proceedings?
A. Thaven't taken a view on -- I have taken each of the Horizon Issues that's put to me
with equal priority. I haven't set out to decide a prioritisation for them.”
(emphasis added)
754.
75S.
756.
POL00112816
POLO00112816
This approach by the Post Office was, in my judgment, an attempt effectively to re-
draft Horizon Issue 1 in terms more favourable to its case. I have already identified
above that the Horizon issues were drafted and agreed by the parties and approved by
the court. Horizon Issue 1 plainly states “to what extent was it possible or likely for
bugs, errors or defects of the nature alleged. ...to have the potential to (a) cause apparent
or alleged discrepancies or shortfalls relating to” SPMs’ accounts. The approach taken
by the Post Office (and also Dr Worden) was entirely to ignore the word “potential”.
Talso consider that it is a matter for the court to decide which issues are more important
than others. An expert might decide a particular issue is important in that it represents
a starting point for their opinion, which will then inform or guide their views on other
issues. But it is not for an expert to decide one issue in a list is more important to the
court than others, and concentrate upon that to the potential exclusion of consideration
of some or any of the others. Mr Coyne was therefore correct to treat all of the Horizon
Issues with equal priority.
Mr Coyne had not performed a similar statistical analysis to Dr Worden seeking to
demonstrate either the likelihood, or unlikelihood, of a bug impacting the claimants’
branch accounts. In my judgment such an exercise was not necessary in order properly
to give expert evidence to the court on the Horizon Issues. Dr Worden’s evidence, and
his alternative approach, was properly put to Mr Coyne. Mr Coyne accepted that he
could have approached his evidence in that way, but explained why he had not. He
described the approach adopted by Dr Worden as “‘a fundamentally flawed approach”.
“Q: You could, using your skills as an IT risk analyst, you could have, by reference to
the KELs and the PEAKs that you had identified, formed an assessment as to the likely
financial impact in each of the instances that you had identified?
A. Yes, I could. If I could please answer that by way of illustration to the problem
with that approach.
Q. Please do.
A. We will often see a bug, error or defect with a very wide range of impacts and the
impact is typically whatever the counter was doing at that point in time. So if the counter
was doing a foreign currency transaction for just £50 and something goes wrong, the
discrepancy may be £50.
By knowing that there is a bug, error or defect in the Horizon system that leads to
problems with foreign currencies, you can't then say it is only a £50 defect because that
isn't incorrect. If another person was to be subject to that defect and they were doing a
£10,000 foreign currency transaction, they would likely have that same level of defect.
Now that's an illustration to say you can't value bugs and their potential impact by
looking at what has happened historically. You can't value it in that way.
Q. But what you can do is form an estimate having regard to the totality of the PEAKs
that you have seen, can't you?
757.
758.
POL00112816
POLO00112816
A. But it is a fundamentally flawed approach. If you have seen three branches that
have had an impact because they were doing three foreign currency transactions, a £20,
a £50 and a £100, but then there is a fourth person that believes that they have been
subjected to that but that isn't recorded, you can't simply look at the three where it has
been recorded and say the fourth couldn't possibly have occurred because we know
there is a defect there.
Q. I'm not suggesting that you could arrive at a certain conclusion of an absolute cast
iron number, but I repeat my question. Can't you form an estimate having regard to the
totality of the PEAKs that you have seen?
A. Yes, but your estimate would have to be based on the three people where it has been
recorded to have occurred, so you would say £20, £30 and £50, and the best you could
possibly do is come up with an average of that, and you would say that the value of that
defect is whatever that is.
Q. There is another way that you could do it, isn't there, which is that you could look
at the three bugs, the receipts and payments mismatch, the suspense account bug and
Callendar Square, the ones that have been thoroughly investigated, and you could form
inferences from the scale of those bugs, yes? Would that be reasonable?
A. For those types of bugs, potentially yes.
Q. Those are quite large bugs, aren't they? They are not small bugs in the scheme of
things. That's why they were identified in the letter, because these were major bugs of
which even Post Office was aware?
A. Yes. And carrying on from the one we were not told about until later, the
Dalmellington one, there were some which were only pounds, just a few pounds, and I
think there was at least one that was £25,000.
Q. But just fixing our attention for the moment on those three bugs, would you accept
that the evidence indicates that altogether the total financial impact of those three bugs
is no greater than £100,000. Do you agree with that?
A. No, [haven't done that so I would not know whether that was right or close to right”.
The documentary evidence showed that Dalmellington had over 100 branches with
some being impacted more than once. Using the Post Office’s definition of “lasting
impact”, which means the branch accounts were given TCs to correct the effect that the
bug had upon them, 114 of these impacts were made good. Of the other four, two turned
out not be the Dalmellington bug and the other two had only very small financial
discrepancies.
However, this bug, which had taken 5 years to be discovered, was put to Mr Coyne as
a good example of how countermeasures had to be considered.
“Q. In fact, Dalmellington is quite a good example of how countermeasures need to be
brought into account and how it is important to look at financial impact to make an
assessment of the extent question that's raised in Horizon Issue 1, isn't it? Because it is
759.
760.
761.
762.
POL00112816
POLO00112816
no good saying 118 branches were affected and some of them were affected by large
amounts, when in fact we all know that even before the bug was detected the branches
involved were in fact made whole anyway, yes?
A._ It does suggest that, but it also suggests that there were deficiencies with the
countermeasures because I believe it was five years after the first occurrence of the bug
that the defect was finally discovered.
Q. Because, as you have already agreed, to an outside observer it looks exactly like a
human error because it was a human error. The bug wasn't causing losses, it was
causing errors to be made and those errors were picked up, would you agree with that?
A. No, it wasn't a human error, it was a defect of the system. It made it look like a
human error.”
(emphasis added)
The Dalmellington bug was not accepted as a bug in the bug table at the time of the 3
Joint Statement where it is bug number 4, but is now accepted as a bug but said by the
Post Office to have “transient impact”. I find that it is a bug, and it cannot properly be
described — as the Post Office did during the above cross examination— as human error.
As Mr Coyne pointed out in his answers, it was a defect of the system. It also did take
5 years for it to be discovered. One of the branches affected by it had its branch accounts
affected to the sum of approximately £25,000. The fact that a TC was issued does not
mean that it did not have an impact upon branch accounts. That impact may have been
later corrected by means of the issuing of a TC, but that cannot mask the fact that the
branch account was affected by that amount for the period of time until that correction
was made.
Mr Coyne accepted that Dr Worden’s arithmetic was correct, and also accepted that the
sum Dr Worden had calculated in respect of Dalmellington (£100,000) was “quite small
compared to the £19 million that’s claimed by the claimants in this case. It is less than
1%”. As Mr Coyne accepted, on the “pure numbers”, that was correct.
Mr Coyne did not accept Dr Worden’s basic logic as a starting point, nor did he accept
the logic that there was “no evidence that” the claimants’ branches were “more or less
likely to be hit by bugs than your average Post Office branch”. I deal with that point in
further detail below at [816] and [817] below, but this amounts to an assumption by Dr
Worden that a group of SPMs who specifically allege they have experienced the effects
of bugs are to be treated, in statistical terms, as though they are a random group of SPMs
of the same sample size drawn from the wider population of all SPMs. They plainly are
not a randomly drawn sample of nearly 600 SPMs. They are a very specific group (or
sample) of those who say their branch accounts have been impacted by, or have
experienced, such incidents. In statistical terms, they have a bias — they all allege that
they have experienced the effects of bugs, errors and defects.
An illuminating series of questions and answers highlighted the entirely different
approach of the two experts.
“Q. Are you aware of some special factor applying to the claimant branches which
marks them out as very different from the rest of the branch network making their
763.
764.
POL00112816
POLO00112816
susceptibility to bugs very different from the wide range of branches in the rest of the
network?
A. I'm not aware of it, no, but it would require -- in order to look at that it would require
a detailed understanding of the particular processes of the branches. And I think that's
illustrated by what we know of Dalmellington, that impacted -- I think one branch was
impacted five times, where there was only a handful of branches -- I think, was it 88
that was impacted in total?
Q. Mr Coyne, it is quite important not to get confused between particular instances of
things happening and the law of averages. What I'm asking you about is what's likely
to happen over a large number of cases, do you understand the difference? So of course
if people are all walking down the streets, some of them are going to get hit by lightning.
Everybody won't get hit by lightning in the same way. It will be a very small number
of people who will be hit by lightning and they'll usually be in particular situations
when it happens. But one can make a generalisation about the likelihood of people
being hit by lightning, do you understand, because of the law of averages?
A. I do understand, but in that illustration it would be very unusual for one person to
be hit by lightning five times.”
This then continued with some discussion of instances where some people may have
been hit by lightning more than once.
There are some important points that arise out of this approach. Firstly, the terms of the
contracts between the Post Office and an SPM does not invoke the law of averages, as
it was explained in this cross-examination, or at all. The terms of the contracts, whether
SPMC or NTC, have a very precise accounting process which is run by Horizon.
Secondly, the existence of a bug such as Dalmellington, which was present in the
Legacy Horizon system for 5 years prior to discovery, is of direct relevance to the
Horizon Issues. It does not support the proposition that a SPM with an outreach branch
in the 5 years before it was discovered was more likely (or equally likely) to be hit by
lightning as having their branch accounts impacted by the Dalmellington bug. Thirdly,
even if the “law of averages” approach were the correct one to adopt in terms of
individual SPM complaints that their branch accounts had been impacted by a bug,
error or defect in Horizon (and to be clear, I find that it is not) it is an exercise that could
only sensibly even be attempted by firstly using the total number of bugs present in the
system (which would require at least this judgment, if not more investigation too
following the judgment), a definitive total for the number of instances of each bug
occurring (again, which would require another judgment), a definitive total for the full
amount of losses caused by each incident of each bug (information again not yet
available, if ever) and some sort of probability calculation taking account of all those
matters. That calculation would only, in any event, lead to a figure for the probability
chances of any SPM in the period having their branch accounts impacted by a bug. So
many of the essential ingredients or pieces of data for such a calculation are not known,
that to try to attempt it in the way put by the Post Office, is to start from the end and
work backwards. The “law of averages” approach cannot be used, in my judgment, in
the way contended for by the Post Office.
766.
POL00112816
POLO00112816
So far as Dr Worden’s “scaling factor” is concerned, and his approach to the claimants’
branches, Mr Coyne was entirely reasonable, in my judgment, in how he would
consider hypothetical situations put to him and also explain why they were not valid.
“Q. At this point in the equation what Dr Worden is trying to do is to work out how
likely it is branches are going to be affected?
A. Yes.
Q. He reasons that branches with more transactions, doing more transactions, are
statistically over a large number of occasions more likely to be hit by a bug than a
branch doing fewer transactions and would you agree the principle underlying that
observation?
A. I think that's probably a reasonable principle. If we look at the defects we found
though, there is probably other factors that could be brought into that.
Q. But you are not aware of any, you are not in a position to suggest a single factor
which you have any evidence to think actually applies in relation to the claimants as
compared with the rest of the branch network?
A. I don't know the make up of the claimant, but one example might be, do they have
an outreach branch?
Q. You are aware, aren't you, how small the number of outreach branches there is, how
insignificant that is in the context of the Post Office network, aren't you?
A. Well, it wouldn't be insignificant to the people that are impacted by the defect and
I don't know whether any of the -- because I haven't studied the claimants, I don't know
whether any of the claimants have that or not.
Q. What you are doing is you are speculating that some of the claimants might have
outreach branches?
A. I'm not. I'm illustrating the potential problems with the approach that you are taking
just simply using the unit of number of transactions.”
(emphasis added)
Similar points can be made, in my judgment, to each of the bugs in the bug table, and
not simply Dalmellington and outreach branches. Some affect foreign currency
transactions. Bug 9 in the bug table, Reversals, only affected the value of transactions
that were sought to be reversed by a SPM, and were impacted by the bug actually
introduced into Horizon by version S30 which was released by Fujitsu. That this
occurred at all was due to a failure in regression testing by Fujitsu prior to release of
$30. This would, if the bug hit that branch, show the value of a reversal twice in branch
accounts. The impact on the branch accounts would be the value of the reversal; if it
were a large figure, the impact would be large, and vice versa. Something of that nature
cannot sensibly be dealt with by the law of averages. Concurrent log ins, in the early
days of Legacy Horizon, and numbered bug 18 in the bug table, would only occur if a
767
768.
769.
POL00112816
POLO00112816
user was logged in at the same time to two terminals. Again, that cannot sensibly be
dealt with by use of the law of averages.
As Mr Coyne put it when asked about Dalmellington above and reminded that there are
not many outreach branches in the Post Office network and how insignificant this was,
“well, it wouldn't be insignificant to the people that are impacted by the defect”. This
answer, which in my judgment is an entirely accurate one in the context of this group
litigation, is one that could apply to each of the instances or incidents of each of the
bugs, errors and defects impacting branch accounts that have now come to light in the
Horizon Issues trial. Individually and when considered across the whole of the 3 million
sets of branch accounts in the period from 2000 onwards, the amounts are individually
small amounts (although sometimes they are several thousand pounds). The same point
can be reached by simply taking the sum at stake in the group litigation in terms of
accounting losses for all the claimants (as opposed to other heads of claim not
quantified) which is £18.7 million, and comparing it to the total amount of business
transacted in the Post Office branch network as a whole across the entire UK from the
year 2000 to date. No one has calculated that latter figure so far as I know (and if they
have, they have not brought it to my attention) but it will be an enormous sum. The fact
that £18.7 million is but a small proportion of that enormous figure for total transacted
business over 18 years is of no assistance, and neither is the law of averages approach.
Finally on this, in one of the answers to Mr Coyne’s requests for information (on
another point, discrepancies from external data sources), the solicitors acting for the
Post Office pointed out in the document dated 4 June 2018, the number of transactions
that Horizon deals with each day. This was done in order to give a sense of scale to the
request posed by Mr Coyne. The exact answer provided was “To give this a sense of
scale, it is noted that Horizon processes around 6,000,000 transactions per day.” The
reason for identifying this is that this figure for transactions per day, together with the
agreed figure for the number of monthly branch accounts over the period agreed by the
experts, namely 3 million, gives a sense of scale of the Post Office’s business conducted
by Horizon and the total number of transactions over the period. Given there are fewer
than 600 claimants, the number of impacts on branch accounts complained of by the
claimants is a very small fraction of this (whatever its precise total). The law of averages
approach does not assist the Post Office.
So far as countermeasures are concerned, Mr Coyne stated that “typically if you were
going to design a countermeasure you would want to have it finding the defect
straightaway. If it was only going to find it some time later it would be of limited
value.” Not only do I agree with that in general IT terms, but also in terms of the
Horizon system and its accounting function between the Post Office and each SPM
every branch trading period specifically. A “countermeasure” in this instance would
be something that would correct an impact to the branch accounts. If that is only going
to occur, as Mr Coyne put it “some time later” then it would only be of limited utility,
if any, depending upon the period of time this takes. Further, the Post Office used the
term “countermeasure” as including “human measures and controls”, as made clear to
Mr Coyne during his cross-examination:
“T'm using it as a term to refer to measures and controls in existence, both technical and
human measures and controls, which are designed or have the effect of identifying bugs
770.
771.
POL00112816
POLO00112816
in Horizon or identifying the consequences of bugs in Horizon so that they can be
looked at and fixed.”
This means therefore that the Post Office was relying upon matters outside the
definition of the Horizon System at the beginning of the Horizon Issues, which stated
“the Horizon System” shall for the purposes of this list of issues mean the Horizon
computer system hardware and software, communications equipment in branch and
central data centres where records of transactions made in branch were processed, as
defined in GPOC, at §16 and as admitted by Post Office in its Defence at §37.”
As Mr Coyne explained, countermeasures in the system are addressed at the design
stage:
“Typically countermeasures are a design aspiration. When you sit down to design a
computer system and what controls are in place, then you would consider what controls
are required, where to position them. Should it be the case that the controls fail do you
have a human aspect doing a secondary check? It could go all the way to employing
auditors to re-check figures. You could go on ad infinitum with —
Q. So you accept, then, that these controls or countermeasures, whatever you would
like to call them, some of them have the effect of stopping the bug in its track. Well,
some of them have the effect of stopping a bug arising in the first place?
A. The best way of dealing with defects is to ensure that they don't arise in the first
place ........: so what you would do is throughout your testing regimes you would flush
out all of the defects at that point in time, and you would put in controls to ensure that
if a defect does trigger, there is a way that you are alerted to it and it is fixed as soon as
possible.
Q. Yes. So one of the objectives is to have a testing regime which picks up as many
bugs as possible?
A. Indeed.
Q. But I think you accept that it is impossible to have a testing regime that picks up all
of them?
A. It is impossible to have a testing regime that picks up all of them and that is why
you have a process of categorisation and you would allow a product to go live if -- one
of the terms that's used -- all severity 1 and severity 2 defects have been dealt with.
Severity 3 might be largely cosmetic and you would allow a system to go live with
cosmetic defects.
Q. That's not quite what I'm asking you. You accept there are going to be bugs which
arise in the operation of any IT system which would simply evade any test that any
human being can devise?
A. There's always going to be bugs that are unknown to be within the system at the
point that you go live and they are only discovered sometimes weeks afterwards,
sometimes years afterwards.”
POL00112816
POLO00112816
772. Both defensive programming and impact management are part of, or are descriptive
terms concerning, countermeasures. In my judgment they are both more to be described
as “technical controls” (to use the language of the Post Office’s pleading) rather than
“human measures”. These matters were all effectively part of the 3° Joint Statement.
So far as countermeasures within the Horizon System itself are concerned, these plainly
affect the robustness of the system.
773. The Post Office also has an inconsistent approach to TCs. On the one hand they are
relied upon (through Dr Worden) as being an important countermeasure — even though
they are not part of the Horizon System as defined — yet on the other hand questions
about them were not dealt with as though they were relevant to the Horizon Issues. For
example, one of the requests raised by Mr Coyne in respect of a particular PEAK was
as follows:
“Regarding POL-0032919.pdf, the GoldenGate replication between Oracle 10g and
11g being aborted and resulting in a number of branches reporting cash declaration and
stock reporting discrepancies, were any transaction corrections sent to the 247 affected
branches as a result of the discrepancies and which branches were affected by the
incident?”
774. The answer that was given by the Post Office’s solicitors was as follows:
“The relevance of this request is not understood. Transaction Corrections are issued by
Post Office to correct transient discrepancies in branch accounts in order to restore the
correct position.”
775. This answer to a request by the claimants’ IT expert therefore — correctly in my
judgment — identifies that TCs are not part of the Horizon System. That is entirely at
odds with them being part of a countermeasure within the Horizon System. This is
effectively accepted in the Post Office’s opening submissions which expressly state at
paragraph 114 that the “Post Office’s reconciliation processes are outside the scope of
the Horizon Issues”. TCs are part of the Post Office’s reconciliation processes, and I
find that they are outside the scope of the Horizon Issues. Dr Worden therefore cannot
sensibly take them into account as a Horizon countermeasure. The Post Office cannot
have it both ways.
776. One of the matters that Mr Coyne relied upon was what he described as “the sheer
volume” of KELs and reconciliation reports (a statement, it should be noted, made prior
to discovery of the 5,000 KELs disclosed by the Post Office in October 2019). His
evidence was that this confirmed “the wide-ranging extent of the impact of such
bugs/errors/defects. This evidence demonstrates that such bugs/errors/defects would
undermine the reliability of the Horizon system to accurately process and record
transactions”. He accepted that there was the need for reconciliation reports, and that
this was part of the way that a system would check whether (say) the banking records
at the Post Office transactions matched the corresponding records at (say) the Bank of
Ireland for the same transactions. In my judgment KELs are indeed part of the records
(or contemporaneous records) that can be considered in order to arrive at an overall
conclusion for the existence and potential impact of bugs errors and defects, but the
PEAKs need to be consulted too. This is because the PEAKs are the documents that
777.
778.
POL00112816
POLO00112816
often have greater information within them, added to which one single KEL could relate
to a couple of short PEAKs, or alternatively to many a great many far lengthier PEAKs,
and it is only from the PEAKs themselves that one can gain a better view of the impact
of that particular bug. Indeed, some of the KELs simply have a start and end date, with
very little information on the number of impacts during the intervening period. It is
clear to me that if there is more than one PEAK referring to the same KEL, one would
only realise that there had been a re-occurrence of the same defect if one considered the
PEAKs; the KEL would not show this. Simply looking at the KEL could give the
erroneous impression that such a defect had only occurred once. It is also telling, in my
judgment, that some PEAKs that are clearly related to errors, for example the PEAK
associated with the issue that the subject that “SSC Database users do not have correct
permissions”, namely PC0208119 dated 10 March 2012, expressly states in the 25
October 2011 entry the following “Have relevant KELs been created or updated? No.
None believed to be required”. This clearly demonstrates that Fujitsu would not even
create a KEL for known errors (as SSC users having the incorrect permissions, all
having APPSUPP when they were not supposed to have this) is clearly a known error.
SSC users were (as set out in the PEAK) only supposed to be given “the optional role
'APPSUP' temporarily (by Security Ops authorisation) if required to make emergency
amendments in BRDB Live”. The KELs alone do not constitute a record of all bugs,
errors and defects.
Reconciliation reports are only generated when the information the Post Office has
about a transaction or series of transactions is different from the information held by
the client. These are called exceptions. Mr Coyne was asked why he considered the
volume of these in the history of Horizon was confirmation of his conclusion relating
to the wide ranging extent of bugs, errors and defects having impact on branch accounts.
“Q. But I don't understand, Mr Coyne, why you think that the existence of
reconciliation reports confirms the wide-ranging extent of the impact of bugs in
Horizon which are having an impact on branch accounts?
A. Lhave worked and designed banking systems, stock broking systems. I have never
seen the need for tens of thousands of transactions per week to have a_human
intervention. That suggests that something is going wrong. It is working outside of
process on a larger scale than I would have expected.”
(emphasis added)
The origin of the number of reconciliation reports is as follows. Mr Coyne had asked,
in requests for information prior to the trial, what was the purpose of setting an NB102
exception to F99 by Fujitsu. The Post Office in the answer explained that this
explanation would have to come from Fujitsu, and the explanation was given:
“When there is an incident involving a reconciliation exception in Network Banking
which has been fully processed, then the transaction needs to be set to F99 to indicate
that processing is complete.
Therefore, this is done for any transaction that appears in a reconciliation report, once
the resolution is complete.
779.
780.
781.
782.
POL00112816
POLO00112816
The reconciliation process document is SVM/SDM/PRO/0012 RECONCILIATION
AND INCIDENT MANAGEMENT PROCESS and describes how reconciliation is
carried out.
SVM/SDM/SD/0020 END TO END RECONCILIATION REPORTING is the
specifications of the reports that are produced for the reconciliation service.
The CCD that describes the contracted service:
SVM/SDM/SD/0015 RECONCILIATION SERVICE: SERVICE DESCRIPTION.”
The next request was “How often has [setting an NB102 exception to F99] occurred?”
Initially the Post Office did not answer this, stating inter alia that it was very unlikely
that the information would have been pooled or collated. When Mr Coyne persisted,
and pointed out it would be very different if the answer was “it was an isolated incident
in 2003” or “it was 10,000 transactions each day for the last ten years”, an answer was
provided. I shall reproduce it in full:
“Fujitsu currently "F99" 10,000+ transactions per week across all NB102 associated
reports (DCP and NBS)”.
This was correctly understood by Mr de Garr Robinson for the Post Office who
(understandably) corrected Mr Coyne when he initially said he thought it was hundreds
of thousands per week.
“ asked for the number of reconciliation problems that required manual intervention,
and the answer I was given was that it was hundreds of thousands per week, I think was
the —
Q. 10,000 a week that have to be F99ed, do you remember that?
A. Yes.”
So far as the next request, which was associated with this, is concerned, that paints the
same picture of the Horizon System in my judgment. The request was:
“What is the cause of an ‘Uncleared Transaction Corruptions’ and how often do
[Uncleared Transaction Corruptions] occur?”
The answer was, again from Fujitsu:
“The DCP report NB102 section 3 (Uncleared Corruptions) usually contains 100s if not
1000s of transactions per week (Kiosk transactions only).
This is where there is a discrepancy on banking and payment card transactions. To
determine the exact cause of any individual Uncleared Transaction Corruption, that
particular occurrence would have to be analysed. However, the most usual cause is for
a banking request or authorisation failing to reach its destination, which is something
783
784.
785.
POL00112816
POLO00112816
that can have a variety of causes. An example may be that a bank may authorise a
transaction, but that confirmation is not delivered to the counter so the SPM believes
that the transaction in fact failed due to some issue.”
This is a great number of transactions per week that require manual intervention — over
10,000, on the numbers provided by Fujitsu. I consider that Mr Coyne is correct when
he says “this suggests that something is going wrong”. I do not accept that on a properly
functioning and robust system there should be such a high number as that every week.
T accept Mr Coyne’s evidence in his answer at [777] above.
Mr Coyne also considered, in respect of the control environment, that the Ernst &
Young management letter of 2011 had recommended changes to strengthen the change
management process. E&Y had noted that the Post Office was not usually involved in
testing fixes or maintenance changes to the in-scope applications (which includes
Horizon Online) and they were unable to identify an internal control with the third-
party service provider to authorise fixes and maintenance changes prior to development
for applications. The risks were outlined by E&Y in the following terms:
“There is an increased risk that unauthorised and inappropriate changes are deployed if
they are not adequately authorised, tested and approved prior to migration to the
production environment”
Mr Coyne also drew attention to a PEAK some years earlier than this (in 2003) that
provided evidence that Fujitsu did not even always follow the correct processes that
were in place in any event. In this instance a branch terminal was replaced, which
resulted in a number of bills which branch customers had paid (in terms of making
payment at the branch Post Office) not in fact being paid (to the ultimate payee, the
Post Office clients that had issued the bills). The PEAK stated that “something went
wrong and the marooned transaction did not find their way back to the office”,
something which had occurred because the mirror disc in the branch terminal was
replaced (when it should not have been). The PEAK also stated:
“T believe that we shall have to confess to POCL that we have not followed the correct
procedure and we should advise that POCL make manual payments.....”
POCL means the Post Office, under its earlier name of Post Office Counters Ltd.
Mr Coyne was of the opinion that the specific recommendations in the 2011 E&Y letter
directly impacted upon Dr Worden’s view of countermeasures. He also drew attention
to the fact that the non-acceptance of these recommendations by the Post Office was an
opportunity to impose or improve countermeasures in Horizon which the Post Office
chose not to take. The four areas, and the countermeasures they would impact, were:
“J. Improvements to outsourcing application management (Quality and Change
Control; managing non-functional requirements; testing good practice);
2. Improvement to segregation of duties within the “manage change” process (Quality
and Change Control, Security);
3. Strengthen the change management process (Quality and Change Control);
786.
787.
788.
POL00112816
POLO00112816
4. Strengthen the review of privileged access (Security).”
Mr Coyne also relied upon the fact that although both Legacy Horizon and Horizon
Online did contain a number of measures and controls, these mechanisms had been
shown to have failed, and this was agreed by the experts. The minutes of the Post
Office’s own Risk Compliance Committee meeting of 18 September 2013 showed that
not all of the risks identified in the then-recent E&Y audit had been addressed. All four
pages of these minutes were heavily redacted, but one surviving entry shows that risk
not addressed was “relating to the communication by Fujitsu of changes made to the
Horizon system, was still outstanding. It was identified that it would cost over a £1m to
implement the mitigation being suggested by the audit and that this was not
proportionate to the risk being managed.” Mr Coyne accepted in his cross-examination
that this did not relate to a bug being fixed, and I accept that evidence. In my judgment
it does go to the robustness of Horizon.
There was one area which was concentrated upon towards the end of his evidence, and
this was an entry in the 2" Joint Statement. This was dated 25 February 2019, and under
Horizon Issue 1, under the topic “the extent bugs were possible or likely” the experts
had agreed the following entries relevant to this:
“1.1 We agree that bugs were possible, and the table above displays a number which in
the opinion of either or both experts appeared to have impacted branch accounts.
1.2 Referring to the table of bugs above, the experts agree that the bugs in rows 1, 2, 3,
10, 13, 14, 18, 23, 24, 25, 27 and 28 may have had financial impact on branch accounts.
Other rows, the impact is not agreed between the experts.”
And later on in the 2° Joint Statement:
“1.15 The number of distinct bugs, for which the experts have seen strong evidence of
the bug causing a lasting discrepancy in branch accounts, is between 12 and 29.”
Just before the short adjournment on the final day of his cross-examination, Mr Coyne
was asked about the figures in that range. He was also asked to “forget Horizon Issue
4”. Given that Horizon Issue 4 asked “to what extent has there been potential for errors
in data recorded within Horizon to arise in (a) data entry, (b) transfer or (c) processing
of data in Horizon?”, being asked to ignore it somewhat narrowed down the utility of
the exercise, but it is without doubt the case that Mr Coyne, when asked for a number
for the upper range in the Joint Statement, stated it should not be 29, it should be a
different figure. He initially stated that it should be 13. The court then rose for the short
adjournment. As soon as court resumed one hour later, he immediately sought to correct
the number 13 that he had given. He said the number should be 21. A complication in
this cross-examination exercise was that initially Mr Coyne was using what he called
his bug table in his second report (which had been dated 1 February 2019, and amended.
on 6 March 2019) and the cross-examination was being done by reference to something
also referred to throughout the trial as “the bug table”, which was the one in the 2™
Joint Statement. The former in the report had a column headed “evidence of branch
impact” with the entry “yes” or “no”; the entry in the Joint Statement about which Mr
Coyne was being asked was item 1.15 and is as set out above. The questioning was
789.
790.
791,
POL00112816
POLO00112816
done solely by reference to Horizon Issue 1. Table 1 in Mr Coyne’s 2™ report made it
clear that Phantom Transactions, Reconciliation Issues, Concurrent log-ins, Post &
Go/TA discrepancies in POLSAP, Recovery Failures and Transaction Correction Issues
were all listed under Horizon Issue 4, and not Horizon Issue 1. An associated difficulty
was the distinction that was being made by the Post Office between lasting and transient
effect.
The correct figure being supported by Mr Coyne in his evidence was explored
extensively in cross-examination. Mr Coyne did explain this as follows (in part only):
“Q. Are you suggesting that you have only ever suggested 21 and no one was
suggesting anything above 21?
A. No. There are 29 bugs, errors and defects in the list and in the table starting at
number 1, but some of those go to other Horizon Issues apart from Horizon Issue 1, and
they do say that clearly in the heading.”
There is no doubt that Mr Coyne made matters a little hard for himself during this
passage of questioning, and the Post Office seized upon his change of number (which
was, after all, an agreed number in an agreed statement by both experts) and made much
of it:
“Q. Mr Coyne, everybody on this side of the court, and I mean everybody on this side
of the court, believed that your view was that the evidence showed that there were 29
bugs with lasting financial impact. You are now saying that we made a mistake, are
you?
A. Yes.
Q. Could I ask you to explain where you made it clear what you were actually saying?
A. Well, as set out in my second report, there is a table there that quite clearly says
whether there is evidence of branch impact or not. And there is also a list of which
Horizon Issue it specifically relates to. The only addition to that is when you come to
the very last column, there's other bugs, errors and defects that are re-introduced and
they are only covered in the joint statements.”
Mr Coyne also said:
“A. Throughout the report I have set out next to every single defect whether it is relating
to the test of Horizon Issue 1 or a later Horizon Issue, and I have also set out whether I
believe there was impact on branch accounts all the way through report number 2.
Q. Let's not talk about report number 2 because the statement -- the most up to date
statement of the position, I think you agreed with me at the beginning of today, the
culmination of all your work, including work you did after your second report, is the
bug table in joint statement 2?
A. Yes, but when you put that to me I said that that can't be read in isolation, it must
be read with the second report.”
792.
793.
794.
795.
796.
POL00112816
POLO00112816
So far as the number is concerned, Mr Coyne settled on 22 different bugs, his
explanation of his earlier different numbers was as follows:
“Q. And I would like to ask you why is it you thought there were 21 bugs at 2 o'clock,
whereas you thought -- was it 14 bugs at I o'clock? 13 bugs at I o'clock.
A. Because I was just counting down the table at number I in my report. In the pressure
of the situation I just tallied that column down, I didn't have enough time to consider it
fully.
Q. To go back to a question I asked you earlier, where do you make this clear in the
joint statement?
A. In the heading next to the -- in the bug, error or defect.”
T accept that explanation. Although Mr Coyne was criticised for certain entries in the
2*4 Joint Statement, including the headings, and it was put that the “joint statement was
seriously misleading” I do not consider this a valid criticism of him. It was a statement
the text of which had been agreed by both experts, and it appeared in a joint statement.
There is no doubt that the upper figure of 29 would most likely have come from Mr
Coyne, with the lower figure of 12 having come from Dr Worden (and the claimants’
legal team interpreted the figure of 29 in the same way as the Post Office, as it appeared
in their written opening) but there is a limit to how much can be put on this point by the
Post Office. Joint Statements are (or should be) reached by the parties’ independent
experts between themselves.
It is the experts’ responsibility to agree the content of their Joint Statements, and this is
part of their duty to the court as independent experts. It is the basis on which the court
gives permission for expert evidence. No authority is required for these propositions as
this is wholly conventional, but to the extent that authority may assist, Males J (as he
then was) fully explained this in [3] of Mayr and others v CMS Cameron McKenna
Nabarro Olswang LLP [2018] EWHC 3669 (Comm). The agreed text of the agreed
parts of a joint statement therefore cannot be approached as though it were a unilateral
passage in only one of the expert’s reports. It is an agreed passage. I do not consider
that Mr Coyne’s changes of the number he was asked about demonstrates any
unreliability on his part, or discredits him or his evidence.
I consider Mr Coyne to have been a helpful and constructive witness, and I find the
suggestions made to him that he was biased to the claimants and not independent are
criticisms that are not justified. He, and his small number of assistants, had done a great
amount of investigation into the very numerous PEAKs, and the smaller number of
KELs, and he had embarked upon a careful and sensible exercise necessary for him to
reach conclusions on the Horizon Issues, as drafted and agreed by the parties and as
approved by the court.
There were some fundamental methodological differences between him and Dr
Worden, but the agreements that the two of them reached were detailed notwithstanding
this. I found the contents of all of those agreements of great assistance.
POL00112816
POLO00112816
Dr Worden
797.
798.
Dr Worden was educated at the Université de Grenoble, Cambridge University and the
California Institute of Technology, better known as Cal Tech. He has a Ph.D in
Theoretical Particle Physics. He is currently a consultant with a practice called Charteris
Associates, having started there in 1997, but he has been a consultant since August
2010. Starting with the first entry on his CV in terms of his work experience, between
1975 and 1986 he worked on the development of knowledge-based systems and
developed Logica’s proprietary relational database management system, and between
1986 and 1990 he was the Managing Director of Logical Cambridge. This is an
advanced technology subsidiary of Logica. Between 1991 and 1995 he acted as an
independent reviewer of many of Logica’s largest projects, the key projects that
constituted a large part of Logica’s commercial risks, and also a large part of its
commercial reputation. Between 1995 and 1997 he was a Business and Technical
Consultant in Logica, moving to Charteris in July 1997. He has given evidence in the
High Court before on several occasions, including in what was at that time the biggest
IT dispute tried, namely BSkyB Ltd v HP Enterprise Services UK Ltd (formerly EDS
Ltd) [2010] EWHC 86 (TCC). This case is notable for a range of reasons, not least that
the judgment (excluding quantum) runs to some 2350 paragraphs; and also that the
educational qualifications considered in the judgment included those of one main
factual witness who turned out to have fabricated the establishment from where he
claimed to have been awarded a MBA. This was a point demonstrated by counsel
enrolling his own dog (“Lulu”) in the same establishment, Lulu gaining an MBA with
higher marks than the witness. The hearing ran for 109 hearing days and Dr Worden
was cross-examined in that case for more than 35 hours, which is a very considerable
period of time for anyone to be in the witness box. In his CV he identifies that the
majority of his opinions referred to in the judgment of Ramsey J were accepted by him.
At [227] in that judgment he was described as “more of a generalist in the field of IT
with some experience dealing with CRM systems but with good general knowledge of
the estimating, systems integration and the types of problems which can arise on
projects such as” the project in that case.
He is a highly qualified person, both in IT generally and specifically, and in statistics.
He has earlier experience with relational database management systems when he was
at Logica in the late 1970s and early 1980s, and from the dates it is possible to tell that
he has been involved in the IT field for many decades, and (in comparison with many
IT personnel today) from its infancy. He has also, whilst at Charteris, developed and
deployed software for data integration and migration. He described himself as having
done databases “since the year dot” at one point. He describes himself in his CV as
having strong practical experience of information technology and its business
applications. He has several years’ line management experience, has successfully
managed large IT projects, and acts as a trouble-shooter of major integration projects
and consultant in information strategy. He has particular expertise in XML, in large-
scale information architectures and tools, data integration and data migration, scientific
data processing, and in system performance. He explained that as he is trained as a
scientist and engineer, he has used mathematics all his life and part of that mathematics
is probability theory and statistics. Given the subject of his Ph.D, it will be no surprise
that he uses mathematics extensively. I doubt that there are any theoretical particle
physicists who are not extremely skilled and expert at mathematics at a very high level
indeed. He therefore has great experience in statistics, and at one point in his cross-
POL00112816
POLO00112816
examination was described as a statistician, something with which he did not disagree.
He does not however, as he put it, have any formal qualification in statistics.
799. He is also a highly experienced expert witness, as demonstrated not simply from the
BSkyB case I have referred to above but many others, and he has given expert evidence
in different cases in the Technology and Construction Court, Chancery Division and
Queen’s Bench Division too. He served two detailed reports with numerous appendices
(adding schedules of corrections in the usual way in supplementary evidence in chief),
and he also took part in the joint experts’ exercises and signed the four Joint Statements
with Mr Coyne. He also served a third report directly upon the court, but no application
to adduce this as part of his evidence in chief was made by the Post Office and it did
not form part of his evidence.
800. In the instant case, he was cross-examined for three days. One of his corrections in chief
was in respect of paragraph 761 of his 1* report where he had stated the following in
respect of his summary of what was called a statistical analysis of the likelihood of the
bugs that had been found by the experts being the cause of the claimants’ shortfalls and
discrepancies.
“759. In section 8.5, 'Scaling of Financial Impact of Bugs', I gave what I think is the
simplest analysis of why, in quantitative terms, bugs in Horizon cannot have accounted
for a large part of the Claimants' shortfalls. I summarise it here.
760. Because Post Office has had an average of 13,560 branches over the lifetime of
Horizon, the total number of monthly branch accounts has been about 3 million.
761. Therefore, if a bug like the Suspense Account bug has occurred 16 times in the
lifetime of Horizon, the chance of it having occurred in any given branch in any given
month is about 16 in 3 million. Because the Claimants tended to have smaller branches
than the average, doing fewer monthly transactions (by a factor 0.37), the chances of
the bug occurring in a Claimant's branch would be about 2 in 10 million.
762. I have considered a bug similar to the suspense account bug, which occurred about
10 times, and had a mean financial impact of about £1000 per occurrence. How many
similar bugs would be needed, to give a one in ten chance of one such bug occurring,
with an impact of £1000, on a particular Claimant's branch in a particular month?
763. The answer, given by elementary arithmetic which I describe in section 8.5, is that
there would need to be 50,000 of these distinct bugs. If there were fewer than 50,000
similar bugs, if any Claimant were to assert that in a given month a shortfall of £1000
in his accounts was caused by bugs in Horizon, then the chances of his assertion being
correct are less than one in ten.
764. So the Claimants cannot credibly assert that their shortfalls were caused by bugs
in Horizon, unless there were something of the order of 50,000 such bugs.
765. Only three such bugs have been found. My own search of KELs has found only 8
other possible bugs. Mr Coyne's examination of over 5000 KELs has found no other
bugs which definitely caused shortfalls.
801.
802.
803
POL00112816
POLO00112816
766. Thus the Claimants’ case requires 50,000 bugs in Horizon - but only a handful
have been found by the experts. Neither expert can quantitatively support the Claimants’
case.”
(emphasis added)
His correction was that in respect of paragraph 761 the sentence that reads "the chances
of the bug occurring in a Claimant's branch would be about 2 in 10 million" should be
"the chances of the bug occurring in a Claimant's branch would be about 2 in a million".
He said that this was a typographical error. When he was asked if he had spotted any
other errors, he said:
“A. No, I didn't because there were quite a few statements of this "in a million" nature
in my report. I made careful checks of the main thread that led to my key result in
section 8.7, and that was my focus. And similarly on the transaction corrections I had
made checks to that spreadsheet and, you know, I included the spreadsheets with the
reports so that you can see how the arithmetic is done from the spreadsheets.
Q. And some of it you point out is basic arithmetic?
A. I think all of it is basic arithmetic.”
He said, in relation to a question as to whether Mr Emery, his assistant, had “checked
the maths” that “No, he didn't actually. The maths is there in the spreadsheets for
anybody to check. It is not advanced statistics, it is basic multiplication and division.
and it is laid out in the spreadsheets.”
(emphasis added)
His methodology in terms of his statistical analysis was explained by him in section 8.5
of his 1“ report. This stated:
“617. I shall assess the financial impact of bugs in Horizon over three different scopes:
617.1. (a) Across all Post Office branches, during the lifetime of Horizon.
617.2. (b) Across all Claimant branches, while they held them.
617.3. (c) On a single Claimant branch in a single month.
618. It is possible to relate the financial impacts on these scopes, by numerical scaling
factors. I calculate those scaling factors in this sub-section.
619. Over the period 2000 — 2018, (i.e. 19 years) the Post Office network has consisted
of more than 11,000 branches. The mean number of branches in all years over the period
has been about 13,560. This figure is derived from the spreadsheet referred to at
paragraph 178 of Ms Van Den Bogerd's Second Witness Statement, assuming that the
spreadsheet is accepted. If this evidence is accepted, the number of ‘branch months’ (a
single branch, trading for a single month) has been 13,560 * 12 * 19 = 3,091,680. This
is the number of monthly branch accounts that have been produced.
POL00112816
POLO00112816
620. This means that for a typical Post Office branch, the scaling factor between scope
(a) and scope (c) for the impact of bugs in Horizon is a factor of 3 million.
621. For Claimants' branches, rather than typical branches, the scaling factor of
approximately 3 million may need to be adjusted for two possible effects:
621.1. It might be asserted that Claimants’ branches are more or less likely than other
branches to be hit by bugs in Horizon, because of some special property of Claimants.
621.2. Claimants' branches may, on average, be smaller or larger than typical branches
across Post Office network. If they are smaller, they handle fewer transactions in a
month, and so are less prone to Horizon bugs in those transactions.
622. It seems implausible to me that there is some special factor about Claimants’
branches, which makes them much more prone to bugs in Horizon - bugs which one
would expect to strike any branch at random. Nevertheless, I have considered the
possibility carefully in Appendix F. I have shown there that there is no significant
difference between Claimants' branches and other branches, in proneness to bugs in
Horizon.
623. It appears, from the spreadsheet attached at paragraph 179 of Ms Van Den
Bogerd's Second Witness Statement, that the Claimants’ branches are, in terms of
customer transactions carried out per day, smaller than the average across the whole
Post Office branch network.
624. If this spreadsheet is accepted, it implies the following about Claimants’ branches:
624.1. From summing rows of the spreadsheet, the 561 Claimants' branches carried out
558,000 customer transactions per week in 2007.
624.2. This is 558,000/6 = 93,000 transactions per day, assuming a Post Office branch
is open for 6 days a week.
624.3. Across 561 Claimant branches, this is an average of 93,000/561 = 165 customer
transactions per branch per day.
625. For comparison with this figure, I need to estimate the average size of branches
across Post Office Network. I have done this using two pieces of evidence. Ms Van
Den Bogerd's Second Witness Statement says that, across the whole Post Office
network, there are approximately 48 million customer transactions per week, or 8
million per day in 2017 (assuming again that branches are open for 6 days in the week).
[626. I have omitted the table of EPOSS Transaction Rates, which are then
summarised in 627]
627. The best estimate I can make of the average daily volume in 2003 from this table
is to divide the peak month by 26 working days. This gives an estimate of
approximately 4 million transactions per day in 2003, compared with 8 million
transaction per day in 2017.
POL00112816
POLO00112816
628. I therefore estimate that the average volume of transactions over the period 2000-
2018 has been approximately 6 million transactions per day, mid-way between the 2003
figure and the 2017 figure (assuming Mrs Van Den Bogerd's evidence is accepted. I
note that Mr Coyne quotes the same figure from her).
629. To estimate how much smaller the typical Claimant's branch is than the average
Post Office branch:
629.1. The average transaction rate for all Post Office branches is 6,000,000/13,650 =
439 customer transactions per branch per day.
629.2. So, in terms of customer transactions per day, the typical Claimant branch was
smaller than the average Post Office branch by a factor 165/439 = 0.37.
630. So, Claimants' branches, being generally smaller than Post Office average, have
fewer transactions per month and so are less likely to be hit by a Horizon bug in a given
month. (The calculation of the factor 0.37 contains some minor approximations that can
be improved with further effort. I intend to do so in my supplemental report.) The factor
0.37 increases the scaling factor above, between scopes (a) (see paragraph 617.1) and
(c) (617.3) from about 3 million to about 8 million.
631. I illustrate what the factor of 8 million means using a hypothetical example of a
bug which has occurred 16 times over the lifetime of Horizon, with mean financial
impact on these occasions of £1000. Call this Bug A. The financial impact of Bug A is
similar to that of the Suspense Account bug.
632. If I selected a Claimant's branch and a month at random, then the chances of Bug
A occurring at that branch in that month are only 16 in 8 million, or 2 in a million - an
extremely small probability.
633. Different types of bug occur independently of one another, so their probabilities
of occurring are additive. If there were a second bug similar to the hypothetical bug
above - call it Bug B - then the chances of either Bug A or Bug B happening to one
branch on one month are twice the previous figure - 32 parts in 8 million.
634. If there were 100 similar bugs - called Bug A, Bug B, Bug C.,,... Bug Zz - the
chances of any one of them happening to one branch on one month are still only 100 *
16 parts in 8 million, or one part in 5,000. This is still a very small probability.
635. It then follows that in order for one occurrence of a bug, of similar financial impact
to the Suspense Account bug, to have even a one-in-ten chance of occurring to one
branch on one month, there would need to be 50,000 such distinct bugs - because 50,000
* 16 / 8,000,000 = 800,000/8,000,000 = 1/10. There would have to be a Bug A, Bug B,
Bug C, and so on, ina list with 50,000 distinct bugs.
636. Even if there were some 'super-bug' - with financial impact ten times larger than
the suspense account bug - there would have to be approximately 5,000 such super-
bugs to give a one in ten chance of affecting a Claimant's branch accounts in a given
month. There is no evidence for even one such super-bug - let alone for 5,000 different
ones.”
804
805.
806.
POL00112816
POLO00112816
[637 and 638 then provided a further table and calculation that I omit]
639. The Claimants have never asserted that there are as many as 50,000 distinct bugs
in Horizon, with each bug on the same scale of financial impact as the Suspense
Account bug. Their case rests on two or three known bugs of this scale, and on the
unproven assertion that there may be others.
640. If any Claimant were to assert that, for instance, a deficit of £1000 had occurred
in his branch in a particular month, caused by a bug similar to the Suspense Account
bug, the chances of that assertion being correct are extremely small, because Horizon
bugs strike so rarely - unless Horizon contained of the order of 50,000 distinct bugs of
that kind (and even then, the chances are only one in ten).
641. Mr Coyne has examined more than 5,000 KELs, and not found definite evidence
for even one bug with impact similar to the Suspense Account bug - let alone 50,000 of
them.
642. The implication of this result - the very small probability of any error in one
months’ accounts from a bug in Horizon - is that the accounts for any branch on any
month are overwhelmingly likely to be correct (apart from effects such as delayed TCs,
which are corrected after a variable delay).
643. In my experience, no commercial IT system could ever go live with as many as
50,000 serious bugs - and certainly could not have the good in-service record over 18
years that Horizon has had.”
(emphasis added)
I have reproduced extensive passages of Dr Worden’s explanation of his own
methodology in order to allow his explanation to be available to any reader of this
judgment. I consider this a wholly flawed methodology, and obviously so, and I reject
it.
There is another aspect to this which need addressing at this point. It needs some context
in order properly to be understood. Sums recovered from SPMs for discrepancies in
branch accounts would be put into something called the suspense account. The
claimants pleaded in paragraph 38 of the Generic Particulars of Claim the following:
“38. The Defendant operated one or more suspense accounts in which it held
unattributed surpluses including those generated from branch accounts. After a period
of 3 years, such unattributed surpluses were credited to the Defendant's profits and
reflected in its profit and loss accounts.
39. The Defendant thereby stood to benefit and/or did benefit from apparent shortfalls
wrongly attributed to the Claimants which did not represent real losses to the
Defendant.”
This was pleaded to by the Post Office in the Defence as follows:
“73. As to paragraph 38:
807.
808.
POL00112816
POLO00112816
(1) Post Office does not have a clear understanding of what the Claimants mean by the
term "suspense account" or by the phrase "unattributed surpluses including those
generated from branch accounts".
(2) In relation to its dealing with its clients, Post Office is almost always able to
reconcile its figures to those of its clients. However, as is inevitable given the nature
and scale of those dealings, occasionally Post Office is not able to do so. For example:
(a) A client, such as a bank, makes a payment to Post Office in respect of a particular
transaction undertaken in a branch but the payment exceeds the amount that Post Office
considers to be due.
(b) Post Office seeks but does not reach agreement with the client as to the amount due
and the client does not accept repayment of what Post Office considers to be the
overpayment. In parallel, Post Office also seeks to determine whether the overpayment
should be credited to a branch or to any other relevant part of its business.
(c) Where the proper beneficiary of a credit cannot be determined, Post Office
temporarily records the overpayment in its accounts. If the overpayment is not resolved
within 3 years, the overpayment may be credited to Post Office's profit and loss account.
(3) The existence of processes for recording and resolving credits of the type described
above are an ordinary business practice.”
This was then pleaded to by the claimants in their Reply and Defence to Counterclaim:
“29. The Defendant's professed lack of clear understanding of the term ‘suspense
account’ at paragraph 73(1), is noted at paragraph 2.1 above, and its response to the
Claimant's RFI is similarly evasive.
30. The effect of the Defendant's example pleaded at paragraph 73(2), is that the
Defendant admits the possibility that:
30.1. a sum of money may be received which the Defendant believes to be an
‘overpayment’ from a third party client (e.g. a bank) in respect of a transaction
undertaken in a branch: paragraph 73(a);
30.2. the Defendant would seek to determine whether the overpayment should be
credited to a branch, or to any other relevant part of its business: paragraph 73(b);
30.3 where the Defendant does not identify ‘the proper beneficiary’ of the credit, Post
Office temporarily’ records the overpayment in its accounts and credits that to its
profits, if not resolved within 3 years.
31. This logic would apply where the Defendant’s internal records were wrong due to
a Horizon error or in the case of any failure to identify the relevant branch or
Subpostmaster”.
I find that there is no sensible basis for the professed lack of understanding by the Post
Office in its Defence of the term “suspense account”. That term is widely used by
witnesses for both sides in the evidence, and I will give two simple examples showing
its use. One of the Post Office’s own witnesses, Mr Johnson used it in his evidence, in
809.
810.
811.
812.
813.
POL00112816
POLO00112816
a passage quoted at [276] above. Further, in an internal document which was prepared
after the Callendar Square bug was discovered, it was widely used. An extract from that
document appears at [404] above. Not only is the term used in four separate places in
that document, there is even reference to the Suspense Account Team. The notion that
the Post Office did not understand the term used in the Particulars of Claim is wholly
fanciful. Further, there is an agreed but (number 3 in the Bug Table) which the experts
call the Suspense Account bug.
Putting to one side the fact that the Post Office professed not to know what the term
suspense account meant, the Defence makes it clear that unattributed sums within it,
the correct destination for which the Post Office could not work out, were credited to
its own profit and loss account. I accept the logic explained in the Reply in respect of
the effect of this.
Dr Worden had wholly failed to consider this issue. The point was put to him, not by
reference to the pleadings, but by reference to another document which he admitted he
had read, namely a report by Second Sight. That report had stated what is basically the
same point but in slightly different terms, namely
“2.11 We note that Post Office's control and reconciliation procedures rely on correct
information being supplied by third party clients. It follows that, if incorrect
information is provided by any client company, this can give rise to a loss being charged
to a branch. We also note that, for most of the past five years, substantial credits have
been made to Post Office’s Profit and Loss Account as a result of unreconciled balances
held by Post Office in its Suspense Account.”
The question and answer are as follows, having read out the passage:
“Now I'm not asking you about that as evidence of itself, I'm just asking were you aware
that that was a concern that Second Sight had raised about substantial credits being
made to Post Office's profit and loss account as a result of unreconciled balances held
by Post Office in its suspense accounts?
A. Iread the Second Sight reports and the Post Office's reply.
Q. But did you spot that?
A. I didn't spot that, no.
This point was put in the context of evidence Dr Worden had given, namely the effect
of small bugs or what he called “micro bugs” being unlikely to have an effect.
“Q. Okay. Well, what about -- because elsewhere in your report, I don't want to stray
too much at the moment, but you also mention micro bugs, small bugs, being unlikely
because you would expect to see large sums of money somewhere --
A. Money that disappears somewhere has to pop up --
Q. It's got to come back somewhere else, it's got to pop up somewhere else.”
The above passage was then put to him. Dr Worden’s failure to spot what is, in my
judgment, a point relevant to the likelihood of Horizon bugs, errors and defects having
814.
POL00112816
POLO00112816
a potential impact upon branch accounts is a notable one. Whether the point is taken
from the pleadings, or from the Second Sight report (which Dr Worden accepted he had
read, together with the Post Office’s reply to that document) does not much matter, in
my judgment. I consider that this one omission alone - which I consider to be a serious
omission on his part — is sufficiently fundamental as to undermine his evidence on this
issue its own.
The following points are relevant in respect of Dr Worden’s statistical exercise:
1. There was no permission granted by the court for expert statistical evidence in this
case. The permission that was granted was for expert IT evidence.
2. Had either party sought permission to rely upon evidence from an expert statistician,
I would not have granted it. Statistical analysis does not assist in resolution of the
Horizon Issues, which required (in my judgment) detailed consideration by IT experts
of the detailed occurrences of what could, potentially, be bugs, errors and defects. This
was so that their expert opinion could be brought to bear, firstly, upon the important
issue of whether there were any, and also if there were, their extent and the degree to
which they potentially could impact branch accounts. This both experts did in any
event, and that evidence — based on contemporaneous records and evidence about their
occurrence - is of far greater assistance than this wholly theoretical argument about
why Horizon bugs cannot have affected the claimants’ branch accounts.
3. Dr Worden uses statistics, as he uses mathematics, as part of his professional life.
Any particle physicist would and does use mathematics every day. Dr Worden accepted
that this was not advanced statistics, and he was right to do so. It is a combination of
simple assumption together with basic multiplication and division. The assumptions are
not, in my judgment, valid.
4. None of the Horizon Issues required the experts to perform a mathematical or
statistical analysis of the likelihood of the claimants’ alleged shortfalls and
discrepancies being caused by bugs, errors and defects in Horizon. Horizon Issue 3 —
using as it does the term “extremely unlikely to be the cause of shortfalls in branches”
~ does make this exercise by Dr Worden relevant if that is how he chose to address that
issue. That does not however make it either correct, or of any assistance.
5. The accepted correction to the figure of “2 in 10 million” — which he accepted before
his cross-examination started should be “2 in a million” — shows, in my judgment, how
very broad brush this exercise was. As he put it in his oral evidence, “...there were quite
a few statements of this "in a million" nature in my report”.
6. It is plain that Dr Worden’s own evidence accepts that there are two fundamental
assumptions that are central to this exercise. The first is that any bugs in Horizon strike
branches randomly (which is spelt out in his report). The second is that the claimants’
branches are a random sample. This can be seen, implicitly, from his statement at
paragraph 621.1 that “it might be asserted that Claimants’ branches are more or less
likely than other branches to be hit by bugs in Horizon, because of some special
property of Claimants” and also at paragraph 632 “if I selected a Claimant's branch and
amonth at random....” Neither of these assumptions are valid, and the reasons that they
are not valid are explained in the immediately following paragraphs.
815.
POL00112816
POLO00112816
7. His paragraph 636 makes it clear how fundamentally unsound this exercise it. He
stated “Even if there were some 'super-bug' - with financial impact ten times larger than
the suspense account bug - there would have to be approximately 5,000 such super-
bugs to give a one in ten chance of affecting a Claimant's branch accounts in a given
month.” However, the evidence shows, quite specifically, that even for bugs such as
Dalmellington and Callendar Square, SPMs’ branch accounts were affected, and some
by quite significant amounts. This therefore demonstrates that some SPMs as
individuals could have their branch accounts affected by bugs. If the evidence does not
match a theory (which here it plainly does not) then that must mean that the theory is
flawed.
8. Of his four different ways of addressing financial impact, set out in his paragraph
614, the second assesses the net impact of all bugs referred to in KELs. However, KELs
do not invariably recite known financial impact. In some, known financial impact is
recited, but the conclusion in the KEL is that user error cannot be ruled out and therefore
a conclusion is reached that a bug is not responsible.
9. Dr Worden’s reasoning is entirely circular. By layering assumption upon assumption
(and in places routinely misdescribing these assumptions as estimates) he feels able to
conclude that there must be as many as 50,000 serious bugs if the claimants are correct
(as set out in his paragraph 643) and that “no commercial IT system could ever go live
with as many as 50,000 serious bugs - and certainly could not have the good in-service
record over 18 years that Horizon has had”. The whole rationale of this litigation is to
work out, more or less, exactly what the “in-service record” of Horizon in fact is. I do
not consider reliance on an assertion that it has had “a good in-service record over 18
years” can be used to underpin an exercise, the whole purpose of which is to consider
this very issue.
10. Dr Worden also implicitly assumes that all known bugs have been both reported
and discovered, as he uses this (the total number of occurrences of the Suspense
Account bug) in his table at paragraph 637. It is his label F and the source is “evidence
on the Suspense Account bug”. Mr Coyne estimated that there may be as many as 40,
based on an extrapolation of the number of KELs and PEAKs inspected.
11. Dr Worden also misidentifies the approach of the claimants, or certainly the
approach of Mr Coyne. He states “Their [ie the claimants’] case rests on two or three
known bugs of this scale, and on the unproven assertion that there may be others.” That
is not correct. The approach of Mr Coyne has been to consider PEAKs and KELs; see
what is reported contemporaneously and what the conclusions were (or appeared to be)
within Fujitsu; and consider what that contemporaneous account shows. Mr Coyne then
uses that to draw conclusions.
12. By the end of the Horizon Issues trial, the Post Office admitted the existence of far
more bugs than the “two or three known bugs” that were referred to by Dr Worden in
the preceding sub-paragraph.
Dr Worden also assumed that bugs in Horizon strike branches randomly. This too is not
bome out by the evidence. For example, the Dalmellington bug (bug 4 in the bug table)
appears to have affected only outreach branches. Relatively few of the Post Office’s
816.
817.
818.
POL00112816
POLO00112816
branches are outreach branches. Bug 14 in the bug table, which Mr Coyne said was a
bug but which Dr Worden described as “appears to be a system error with impact upon
branch accounts” is entitled in the bug table “bureau discrepancies” and relates to
foreign exchange transactions. Dr Worden’s exercise takes no account of what is,
effectively, an admitted bug, and one that has an effect only upon foreign exchange
transactions. If a branch conducted very few, if any, of these, it would not be affected
by this bug. The chances of a branch being impacted by the bureau discrepancies bug
depends upon whether it carries out foreign exchange transactions, and if so how many.
That is not a bug striking all branches randomly. Indeed, rather tellingly for Dr
Worden’s exercise, he assumes that any such bug would strike randomly across the
whole branch network and have an equivalent random effect upon the branch accounts
of branches that had performed foreign exchange transactions, and those that had not.
This is plainly invalid.
Dr Worden also assumed that the claimants are a random sample of SPMs. That this is
one of his assumptions can be identified in the following way. His analysis of likelihood
of any bug impacting any SPM’s branch accounts would apply equally to any group of
SPMs chosen at random from those SPMs with smaller branches. The only adjustment
or account taken of the claimant SPMs’ particular characteristics is that they are,
generally, from smaller branches, and for this Dr Worden applies what he calls a scaling
factor. However, the claimants are not a random sample of SPMs, nor are they even a
random sample of SPMs from smaller branches. As a sample, they have already been
filtered or selected in that these particular SPMs already complain of bugs, defects and
errors in Horizon having affected their branch accounts. This means that they are not a
random sample. The way this would be expressed in statistical terms is that the claimant
SPMs do not accurately represent the population of SPMs as a whole (or even the
population of SPMs who had smaller branches). The claimants are essentially self-
selected, from those who believe they have experienced shortfalls and discrepancies in
their accounts from the impact of bugs, errors and defects, and who have been prepared
to join the litigation. They plainly cannot be treated, in statistical terms, as though they
are a random group of 587 SPMs.
The adjustment made by him (or scaling factor, which was refined from that in his 1
report) to take account of the particular characteristics of the claimants does not validate
what is, in my judgment, an entirely invalid base assumption. He attempts to deal with
this by considering whether there is any “special property” of the claimants’ branches
that makes them more susceptible to bugs, and concludes (at paragraph 622 of his
report) that there is not. This is to approach the matter somewhat backwards, in my
judgment. The experts are agreed that there were 3 million sets of branch accounts in
the life of Horizon. The number of sets of branch accounts of which complaint is made
in this litigation is a small (even perhaps tiny) proportion of that 3 million. However,
that basic arithmetical fact does not mean that bugs, errors and defects cannot on the
balance of probabilities be responsible for shortfalls and discrepancies.
The exercise also wholly ignores what branch accounts are, how they are supposed to
be created from the full list of transactions carried out in that particular branch in that
particular period, and how precise they are required to be. The analysis of the “Lasting
and transient impact” approach taken by Dr Worden has already identified that this
ignores the effect on branch accounts of the impact of a bug if no transaction correction
has been issued before the end of the next trading period.
819.
820.
822.
POL00112816
POLO00112816
Finally, there is another fundamental problem with Dr Worden’s exercise, which is that
it entirely ignores the specific purpose of Horizon, at least so far as the contractual
relationship between the Post Office and SPMs is concerned. The Horizon system is,
whether under Legacy Horizon or Horizon Online, the method by which the Branch
Trading Statement or BTS is produced for an individual SPM’s branch Post Office. The
BTS is the accounting mechanism under either the SPMC or NTC (the two different
types of contracts used by the Post Office for SPMs during the period from 2000
onwards, explained in detail in Judgment (No.3) on the Common Issues). It was the
common intention of the parties that the Horizon system would be used for the creation
and recording of accurate information regarding all transactions performed in the
branch which would be used for the production of the Branch Trading Statement for
that branch. This means that at the end of each branch trading period, the BTS will (or
should) provide a statement of the branch activity, which includes whether the accounts
are in balance (which is where the cash and stock in the branch matches the Horizon
system’s records of what should be there) or whether there is a discrepancy. If there is
a discrepancy and it is a shortfall, the contractual terms of the SPMC and the NTC
require the SPM to “make good” the shortfall, which means pay the money to the Post
Office. The other alternative for an SPM is to “settle centrally”, which means seek time
to pay. If the Post Office agrees, the shortfall can be paid over 12 months by deduction
from the SPM’s remuneration. This purpose is entirely ignored in Dr Worden’s
statistical exercise across the whole branch network.
Consider a hypothetical bug, bug X. Also consider that bug X impacts upon branch
accounts in a single branch upon a single occasion leading to a shortfall in the branch
for that branch trading period. That shortfall is treated as a debt by the Post Office and
the SPM either has to pay, seek time to pay or may even end up in a dispute with the
Post Office and/or be suspended and not pay. Analysis and resolution of the correct and
true situation of the branch accounts between the Post Office and the SPM for the period
in question does not depend upon whether, in all the other millions of branch accounts,
there was no such incidence of bug X. The correct analytical approach in my judgment
is to consider the branch activity for that branch for that period; consider the evidence
both for and against (1) the existence of bug X and (2) the likely cause of the
discrepancy, bearing in mind both the burden and standard of proof; make findings on
the cause of the discrepancy; and then apply those findings. Expert IT evidence of most
assistance in that exercise is whether or not bug X exists or existed, and what were its
effects. It is of no assistance to have an exercise that in effect says the statistical
likelihood of any bug having an impact upon the branch accounts of that branch in that
period is very low.
The section 8 analysis is, in my judgment, so riddled with plainly insupportable
assumptions as to make it of no evidential value. It is the mathematical or arithmetic
equivalent of stating that, given there are 3 million sets of branch accounts, and given
there are so many sets of branch accounts of which no complaint is made, the Horizon
system is mostly right, most of the time. It is a little more sophisticated than that, but
not by very much.
The way that this can be further tested against the evidence is as follows. Mr Godeseth
accepted that the Dalmellington bug had probably been present in the system from 2000
until it was discovered in 2005. Dr Worden’s analysis would lead to the conclusion that
823.
824.
825.
POL00112816
POLO00112816
it was overwhelmingly unlikely that it had affected branch accounts during that period.
But all the evidence is that this bug did affect branch accounts. It is correct that the Post
Office relies upon the correction of known and identified discrepancies by means of the
later issuing of TCs — a human measure which in my judgment is outside the definition
of Horizon - but even putting that to one side, during that 5 year period the accounts
were actually affected. The existence of the bug was not known about for 5 years before
it was even discovered. This round of the group litigation is to make findings on the
Horizon system and whether there are any, or many, or no, other such bugs, errors and
defects in the Horizon System similar to this one, or with similar detrimental effects.
The conclusion of Dr Worden’s section 8 exercise is that it was overwhelmingly
unlikely that any of the claimants’ branch accounts were affected by any Horizon bugs.
This must include a conclusion that the Dalmellington bug cannot have affected any of
the claimants’ accounts. This means that he must assume that Fujitsu were 100%
successful in tracking down all incidents of the Dalmellington bug, and that any
claimant SPMs who report symptoms the same as the Dalmellington bug and who had.
an outreach branch are either wrong, or making it up. These assumptions are ones that
cannot sensibly be made, in my judgment.
The whole section 8 exercise effectively is entirely theoretical, and is made in a vacuum
that must ignore real world evidence to the contrary from one of the Post Office’s own
witnesses, Mr Godeseth, as set out at [445] to [450] above.
Dr Worden’s probability calculation so far as two bugs, Bug A and Bug B, striking
independently, was a little puzzling. He stated:
“632. If I selected a Claimant's branch and a month at random, then the chances of Bug
A occurring at that branch in that month are only 16 in 8 million, or 2 in a million - an
extremely small probability.
633. Different types of bug occur independently of one another, so their probabilities
of occurring are additive. If there were a second bug similar to the hypothetical bug
above - call it Bug B - then the chances of either Bug A or Bug B happening to one
branch on one month are twice the previous figure - 32 parts in 8 million.”
(emphasis added)
He described the probability of these two unconnected events both occurring, each with
a probability of 16 in 8 million (or 2/10°), as being additive and he added 16 in 8 million,
to 16 in 8 million, to arrive at 32 in 8 million. In terms of basic probability theory, the
chances of two independent events occurring is usually seen as the product of the two
probabilities, not their addition. The principle of additivity is a fundamental principle
that if the event whose probability is sought can be represented as the union of several
other events that have no outcomes in common, then the probability of the union is the
sum of the probabilities of the individual events making up the union. However,
regardless of this, the point was neither put nor explored in any detail, and the outcome
in any event depends upon how the events, and the union, are defined. The situation is
further complicated by the fact that to impact upon branch accounts, either Bug A or
Bug B (in Dr Worden’s example) would have to occur not necessarily at the same time,
but at any time during the 4 week branch trading period. I refer to this matter for three
reasons. Firstly, to demonstrate that probability theory only takes one so far in this case,
826.
827.
828.
POL00112816
POLO00112816
and that is not very far; secondly, to explain that depending upon how one defines
“event” and the intersection (or joint probability) of both occurrences, one can arrive at
vastly different results; and thirdly, to demonstrate that there were numerous flaws
and/or difficulties with Dr Worden’s approach, not all of which were put, but which
undermine the validity of his exercise. This point about the probability calculation of
two bugs striking a branch is not is however not the main reason (or indeed, any reason)
why I have rejected his analysis. The probability calculation for two bugs occurring in
one month is not a central element of the statistical exercise he performed; only one
bug would be needed in any one branch trading period for it to affect those monthly
branch accounts. I have explained this difficulty in order to illustrate that it depends
upon how one defines the impact of a bug, and chooses to assess its probability, that
one can end up with an answer that rather suits the way the question is initially posed.
Dr Worden accepted so far as his evidence generally was concerned, that he had
assumed that the Fujitsu evidence was true, and in particular that of Mr Jenkins (whom
he had met, although Mr Jenkins was not listed as a source of information in his 1“
expert report) and that “generally, if findings -- if things happen in oral evidence which
go against witness statements then I would need to come back and say what's the
impact.”
This therefore means that his evidence is of limited assistance in any event. I have found
that the evidence of Mr Godeseth was somewhat different, at the end of his cross-
examination, than how it had been presented in his three witness statements. Litigation
cannot go on endlessly. For the Horizon Issues trial, evidence of fact and expert
evidence were both ordered, and findings can only be made by the court in its judgments
that resolve the disputes of fact. Disputes of fact in the evidence for the Horizon Issues
will be resolved in the Horizon Issues judgment, namely this one. This is entirely
conventional. If an expert chooses to prepare his or her evidence on the basis that one
side’s evidence is true — which Dr Worden did in his reports, in respect of the party
instructing him, namely the Post Office — they are not entitled to serve further expert
evidence and wholly amend their opinions if that evidence is not accepted by the court.
Dr Worden must have known this. There was going to be no opportunity for him “to
come back and say what’s the impact”.
This was put to him in his cross-examination.
“Q. The content of this report in this respect, we will come to other examples later,
focuses on what the consequences would be if the defendant's factual evidence is
accepted?
A. Generally, if findings -- if things happen in oral evidence which go against witness
statements then I would need to come back and say what's the impact.
Q. Yes. You understand, don't you, that the moment at which everyone finds out
whether factual evidence is accepted or not is when the judgment is handed down?
A. Absolutely, yes.
Q. So you are proposing to revise your opinions after that, are you?
829.
830.
831.
832.
POL00112816
POLO00112816
A. No. That's a good point. What I was saying there is if it becomes evident, all I can
do is make assumptions. The court will find findings and all I can do is make
assumptions and drive them through my opinions and try and assist the court that way.
So all of this is assumptions I have made based on the evidence I have seen, and the
court may find differently.
Q. Can I pause there. You were assuming there that Mr Godeseth's version was true?
A. Iwas assuming mainly Mr Jenkins' written analysis which Mr Godeseth's evidence
confirmed.”
(emphasis added)
The “evidence I have seen” to which Dr Worden referred was the Post Office’s
evidence, and what Mr Jenkins had said (which was not evidence in any event). He did
not take any account of the claimants’ evidence. This was notwithstanding that in his
expert’s declaration he had expressly stated “I have not assumed that any particular
version of events is true.....” when, indeed, he plainly had. It was also based on Mr
Godeseth’s written evidence, not on his oral evidence which was markedly different.
He also said further on this same subject:
“A. Well, my analysis of the claimants’ evidence is mainly contained in my
supplemental report, and I explained there that individual claimants' evidence,
particularly individual subpostmaster evidence, I did not feel able to make strong use
of that and I gave the reasons in my supplemental report. So my opinions have little
dependence on that. And the core of my opinions, the numerical estimates I make,
those estimates have been designed -- or the process and the method has been designed
so that if -- so that to make my assumptions evident where those assumptions come in,
and if the court finds something different from my assumptions the court can go to the
spreadsheets and re-do the method for itself.”
(emphasis added)
This can therefore clearly be seen as his accepting the Post Office/Fujitsu evidence as
true, particularly that of Mr Jenkins (not even a witness, and in respect of whom no
witness statement was served); and also failing to base his opinions upon, or take
account of, the individual evidence of claimant SPMs served by the claimants.
Dr Worden is also under a misapprehension if he considers that the court’s function is
to “re-do the method for itself” if his assumptions were not made out in the factual
evidence. Firstly, he could have provided alternative expert evidence and opinion, were
the written evidence of Mr Godeseth and the information coming from Mr Jenkins to
be rejected. Secondly, he should have taken the claimants’ evidence into account,
because there is no basis for an assumption that it was not relevant to the Horizon Issues
which his expert evidence was the subject of. It is entirely normal for experts to provide
alternatives of their opinion, depending upon which party’s evidence is accepted.
Thirdly, the “method”, such as it is, demonstrated in the spreadsheets simply cannot be
“re-done” to take account of findings that the assumptions were incorrect. The invalid
and unjustified assumptions lead, in my judgment, to the inescapable conclusion that
833
834.
835.
POL00112816
POLO00112816
the whole methodology is flawed. No amount of “re-doing” of the calculations by the
court can remedy that.
Dr Worden also accepted that although he had missed some days of the trial, he was
“pretty sure” that he had heard all of the Post Office’s evidence; he was then asked the
following:
“Q. Have you had any changes of heart about anything in your report having heard it?
A. Changes of heart?
Q. Anything you want to change about what you said?
A. Well, had there been some fundamental change I would have felt obliged to
communicate it to the court.
Q. Of course, because you take your duty very seriously?
A. Yes, I do. I can't think of any major change of heart, but there may well be things
that you bring me to and I say there's an adjustment here. But to come back to the point
about precision, I am conscious of how precise my numerical results have to be in order
to be of assistance to the court.”
This effectively must mean that Dr Worden did not consider any of the changes, in the
Fujitsu evidence in particular from the cross-examination, would impact upon his
expert opinion. This is notwithstanding that he had assumed that the Post Office’s
evidence would all be accepted, and anyone present for Mr Godeseth’s cross-
examination can only have concluded that the factual position presented by Fujitsu in
the written witness statements was very far from an accurate one. Indeed, the extent to
which the cross-examination did not have any effect — either appreciable or otherwise
— on Dr Worden’s expert opinions shows how very theoretical the whole exercise was.
Finally on this point, due to the disruption caused by the recusal application, and the
unavoidable re-arrangement of the trial timetable, the experts came to give their
evidence in June 2019, and all of the factual witnesses who were cross-examined (with
the sole exception of Mr Parker) had completed their evidence by 21 March 2019, many
weeks earlier. Even Mr Parker had completed his evidence by 11 April 2019. Dr
Worden was called on 11 June 2019, two months after this. It is unusual for there to be
such an interval in a trial, but circumstances in this case led to extra time being
available. There was more than ample time for Dr Worden to have identified how, if at
all, the Post Office’s evidence of fact that emerged earlier in the trial (in particular from
Mr Godeseth, but also from Mr Parker too) impacted upon his opinions. It plainly did
not.
I will provide only one example of Dr Worden’s unsatisfactory approach to the
evidence, although there were many. In terms of whether transactions could be inserted
at the counter in Legacy Horizon, what is called “remote access”, Dr Worden had in his
first report concluded that Mr Roll (a witness of fact) was factually wrong. He expressly
stated that he, Dr Worden, had “established” that Mr Roll was wrong. He accepted that
use of the word was wrong but what he had done in his second report was equally
837.
838.
POL00112816
POLO00112816
telling, in my judgment. This was after those witnesses who were to be called for the
Post Office, still employed at Fujtisu, had corrected (or partially corrected) their
erroneous first witness statements, and accepted that remote access without the
knowledge of the SPM was possible. He said (at paragraph 83) that he required:
“further factual information before I can comment on this evidence. Which ‘specific
person’? Under what circumstances? How frequently? Until I have that information, it
remains possible in my view that any transaction which 'would appear to the
subpostmaster as though it had been carried out through the counter in branch’ might
only be a transaction that he had given his consent for, as the 'specific person’ - and
which had in effect been made on his behalf.”
Dr Worden also then stated:
“84. Therefore, Mr Roll's new evidence does not cause me to alter the opinion expressed
at paragraph 1119 of my first report, when commenting on Mr Roll's first witness
statement, that he could not alter branch accounts without the Subpostmaster knowing.”
This approach by Dr Worden to the evidence for the claimants on the one hand (Mr
Roll) and the Post Office on the other (Mr Parker and Mr Godeseth in their statements)
was explored in cross-examination:
“Q. Yes, the one that we agreed was incorrect in approach and conclusion.
A. We agree now, but at the time I wrote the second report, on the basis of Mr Roll's
evidence and Mr Parker's evidence, I didn't see reason to change that opinion.
Q. I would suggest to you that you were inexplicably reticent to accept something that
was contrary to Post Office's interests.
A. No, I was reticent. Not inexplicable, I was reticent because I had not seen sufficient
evidence to convince me that these things could be done without the knowledge of the
subpostmaster.
Q. But Dr Worden, would you accept that the approach you have taken there contrasts
very, very strikingly in how you approach Mr Roll's evidence with the approach you
took at 1119 in your first report when you accepted, effectively, Mr Godeseth's
evidence?
A. Ihave accepted that my approach at 1119, that use of the word "established" was
wrong and my approach was wrong, and we have established that -- you know we have
done that before the interval —
Q. My question is: do you accept the contrast is very striking?
A. I think the court will have to -- I accept that my attempt to make my position clear
that I'm not trying to find findings of fact, I'm not trying to find one witness or the other,
J accept that on this occasion I fell short of that.
Q. Do you mean only in your first report or also in your second report?
839.
POL00112816
POLO00112816
A. Not in the second report. I believe the second report was -- you know, I believe that
the evidence from Mr Parker and Mr Roll, and if we look again at Mr Roll's paragraph.
20, I believe that the question that I asked in paragraph 83 that I want further evidence.
I believe that was a valid approach.
Q. When you say in paragraph 84 -- because we have dealt with what you accept about
your first report at paragraph 1119 several times, haven't we?
A. Yes.
Q. So in your second report, you say, notwithstanding what you have now accepted
about 1119, in your second report that you were right not to change that view.
A. [hadn't seen sufficient evidence to change that view.
Q. But do you say you were right not to change that view when you revisited the entire
piece in your second ~
A. At the time I believe it was right not to change that view because I hadn't seen
evidence that convinced me that this change could be made without the knowledge of
the subpostmaster. That's where I am.
Q. Dr Worden, do you accept that that betrays a complete failure to appreciate the need
to consider the situation both on the basis of whether the claimants' evidence is right as
well as on the basis that the defendant's evidence is right?
A. No, I don't accept that.”
(emphasis added)
Dr Worden had a further witness statement from Mr Rolls saying effectively that Fujitsu
could perform, and had performed, remote access. Dr Worden wanted “further
evidence” and also referred to needing “sufficient evidence”. He said he “hadn’t seen
evidence that convinced” him. He accepted he was reticent to accept it. All of this
plainly demonstrates that he was taking a partisan view on evidence of fact. I find that
this approach to disputed evidence of fact is simply not the correct approach for any
expert to adopt. By the time of Dr Worden’s 2" report he had a responsive witness
statement from Mr Roll saying that what Dr Worden had “established” could not be
done (a conclusion that was in any event directly contrary to Mr Roll’s earlier witness
statement), very specifically, could be done. Even then Dr Worden would not accept it.
As he stated in his cross-examination, “I hadn’t seen evidence that convinced me” that
this could be done. The evidence was, very plainly, the witness statement saying that it
could be done. He chose to disregard it. This is just a more sophisticated way of stating
the witness statement was not true. He described this as his being “balanced and fair”.
I disagree with that description. He also said that there were many, many occasions
when he had “tried not only to be fair to the claimants but to be biased in my numerical
estimates in favour of the claimants”. I disagree with that too. Firstly, it is not answer
to a criticism that an expert has taken a partisan view of evidence of fact and ignored
one side’s factual account, to say that this has been effectively cancelled out by
840.
841.
842.
POL00112816
POLO00112816
numerical assumptions elsewhere in different parts of the report. Secondly, I do not
accept that Dr Worden was performing numerical calculations assuming in the
claimants’ favour in any event, when one assesses the Section 8 exercise as a whole.
The assumptions he made were very much not in the claimants’ favour at all. Minor
adjustments to numerical figures in an exercise based on assumptions specifically
unfavourable to one group does not either make the result “biased” in favour of that
group, or cancel out the unfavourable assumptions. Further, the court requires an
independent approach from an expert at all stages of their analysis, not a raft of slanted
analyses, either one way and/or the other, with the hope they will eventually cancel
each other out. For the avoidance of doubt, here they do not.
Another area of difference between Dr Worden and Mr Coyne is the issue of what Dr
Worden called “countermeasures”. Mr Coyne described these as “self-defined
countermeasures” and stated that many of them — Dr Worden had identified 18 different
ones — were basic elements of practical system design, and that such design aspirations
did not, in themselves, demonstrate that Horizon was particularly robust.
The 18 different ones are as follows, together with the acronyms adopted by Dr
Worden:
1. Reliable and redundant hardware - RHW;
2. Robust data communication and replication - ROC;
. Transactional Integrity and database recovery ~ TIN;
. Defensive programming — DEP;
Generic, data driven software — DDS;
. Secure kernel hardware and software — SEK;
. Redundant data storage and computing, with cross-checks — RDS;
. Double entry accounting - DEA;
9. Early detection of user errors - DUE;
10. Later correction of user errors - UEC;
11. Manual workarounds — WOR;
12. Testing good practice —- TGP;
13. Manual Inspection of Data —- MID;
14. Bug finding and correction — BFC;
15. Large scale IT architecture - ARC;
16. Quality and change control — QCC;
17. Managing non-functional requirements — NFR;
18. Security — SEC.
OPADARW
These acronyms are, for the most part, not used in the IT industry, and were adopted by
Dr Worden in this case. Some of them are, such as NFR and SEC. I use them in this
judgment for consistency, but their use should not be taken to be a suggestion or
acceptance that the acronyms are industry norms; the majority are not. The actual term
countermeasures suggests that these consist of measures present in the Horizon system
which have the effect of countering something else (bugs, errors or defects, issues that
would have an adverse impact on robustness) and thereby improving the robustness of
the Horizon system as defined. The Horizon system had an agreed definition for the
purpose of the list of Horizon Issues which was as follows:
“the Horizon System” shall for the purposes of this list of issues mean the Horizon
computer system hardware and software, communications equipment in branch and
central data centres where records of transactions made in branch were processed”.
844.
845.
846.
POL00112816
POLO00112816
However, it can be seen simply from the descriptions above, which are Dr Worden’s
own descriptions, that not all of the 18 are even part of the Horizon system. Some are
simply manual checks. Some genuinely are countermeasures, but Dr Worden groups
both types together.
This can be seen from the following. Transactional Integrity and database recovery or
TIN is undoubtedly a countermeasure within the Horizon system. The explanation
given by Dr Worden in his summary table for this is “database management systems
provide many facilities so that numerous kinds of failure cannot leave the data in an
inconsistent, unusable state, or lose any data that have been previously stored.” This
means if there is a failure, which may happen in a variety of different ways, the data
should not be left in an inconsistent or unstable state, or be lost. The database
management system accomplishes this in a variety of ways. Indeed, Dr Worden himself,
with his extensive experience of database management, has particular familiarity with
this. He designed and developed one of the first relational database management
systems; it is an essential requirement in such a system that data should not be capable
of being left in an inconsistent or unstable state. The reliability of any database relies
upon reliability of the data within that database.
Transactional integrity is a fundamental facility built into all database management
software and this is made clear in, amongst others, published material such as An
Introduction to Database Systems 8" edition by CJ Pearson published in 2003 which
was one of Dr Worden’s appendices. As stated at 15.1 of that edition:
“Recovery in a database system means, primarily, recovering the database itself: that
is, restoring the database to a correct state after some failure has rendered the current
state incorrect, or at least suspect. (We will elaborate on what we mean by "a correct
state of the database" in the next section.) And the underlying principle on which such
recovery is based is quite simple, and can be summed up in one word: redundancy.
(Redundancy, that is, at the physical level; we do not want such redundancy to show
through to the logical level, naturally, for reasons discussed in detail elsewhere in this
book.) In other words, the way to make sure the database is recoverable is to make sure
that any piece of information it contains can be reconstructed from some other
information stored, redundantly, somewhere else in the system.
Before going any further, we should make it clear that the ideas of recovery (indeed,
the ideas of transaction processing in general) are largely independent of whether the
underlying system is relational or otherwise—though it is true that most of the
theoretical work on transaction processing has historically been done, and continues to
be done, in a relational context specifically.”
This is a countermeasure. Transactional integrity is intended to give protection against
a wide variety of conditions, such as hardware errors or communication failures,
software errors (which can lead to failures and cancellations) and operations being
cancelled when they have not been completed. Such cancellations can be made by what
Dr Worden calls the operator (the SPM or their assistant) but cancellations can — and
the factual evidence shows this occurred — be caused by outages, power cuts and other
incidents. A lay person might say that a system has “crashed", or one of the terms used
in the factual evidence in this trial was “system outage”.
847.
848.
849.
850.
POL00112816
POLO00112816
Other measures are not strictly speaking countermeasures, they are sensible design
intentions (Mr Coyne used the term design aspirations). An example of this is defensive
programming or DEP. Since programming is at the heart of any software — indeed,
programming creates the software itself — the way that programs are originally written
(what could be called the programming of the system) constitutes the very system itself.
DEP was described by Dr Worden in the following way in his table summary: “software
is divided into small self-contained modules, which do not assume that other modules
are correct, but defend themselves by checking their inputs and raising alerts early.” A.
better description was given elsewhere in his report where he stated that DEP was
“where small parts of a program are written to assume that other parts of the program
may be in error and are written to always check their inputs for the presence of errors.”
In other words, part of a program do not start from the point of assuming that all inputs
are without error; they check for the presence of errors first (in a variety of ways) before
moving to other aspects of the program.
This could be described as a countermeasure, but a more accurate description of it is a
design feature that ensures subsequent steps are taken without repeating or carrying
through errors. It is a function of design.
However, Dr Worden included within his 18 separate countermeasures matters that are
not, in my judgment, countermeasures at all and include remedies. For instance, early
detection of user errors, later correction of user errors, manual workarounds, manual
inspection of data and bug finding and correction are not all countermeasures built into
the system. Just to use that latter item as an example, the many hundreds of PEAKs and
associated KELs show that 3“ level of support in the SSC at Fujitsu would investigate
certain instances when SPMs reported certain matters. Often this would take many
months. Depending upon the outcome of the investigations, matters would sometimes
be reported onwards to Development for a software fix. The investigation by the
personnel at 3 level support (of whom, almost invariably, there would be more than
one) and also the software writer in the Development section, were done by personnel
who are plainly not part of the Horizon system as designed. They are human beings
who are undertaking certain tasks. In fact, a software fix designed to prevent a problem
re-occurring would change the Horizon system when that fix was introduced in live, a
term meaning added to the software within the system. It is, in my judgment, illogical
to treat such a process in the same way as either defensive programming (DEP) or
transactional integrity (TIN).
This approach is at its most extreme in Dr Worden’s view of transaction corrections,
which he considered to be a Horizon countermeasure. The issuing of transaction
corrections are plainly a remedy, or something done by the Post Office outside of
Horizon to correct issues that arise because of Horizon. As Dr Worden accepted, this is
not even done in a consistent way, depending upon the client in question. He stated
“There are many differences of detail in how reconciliation is carried out for different
client organisations, or where it is carried out; sometimes the client organisation does
it from a file sent to it by Post Office, and sometimes Post Office does it. However it is
done, and wherever it is done, the result of reconciliation is always in principle the
same.”
The way that Dr Worden explained the process was as follows:
852.
POL00112816
POLO00112816
“285. When reconciliation finds a transaction for which Post Office's record and the
client record do not match, it is passed to a department in Post Office accounts which
handles reconciliation discrepancies. Each such discrepancy is, until it has been dealt
with, an error in the accounts - and so it must be dealt with. The task of this department
is to determine how each discrepancy arose, which therefore determines how it needs
to be handled.
286. When the appropriate department in Post Office decides that responsibility for a
discrepancy lies with the branch, a request for a TC is issued and is passed from POL
FS to the BRDB in Horizon Online. At this point, there is no impact on branch accounts.
The request is passed on from the BRDB to the branch Horizon system, so that it will
show on the Subpostmaster's screen when he starts the Horizon system the next
morning. At this point, the Subpostmaster may either accept the correction or may
question it and ask for further investigation.”
(emphasis added)
This description makes it clear that there is a substantial link, or series of links, in that
chain that are entirely outside of Horizon. The discrepancy, which Dr Worden correctly
identifies as “an error in the accounts”, is passed to a Post Office department. That
department has to consider it, and determine how it arose. Then that department has to
decide responsibility, potentially issue a request for a TC, and that is then passed to the
BRDB (which is part of Horizon) and the request goes to the branch. A substantial
element of the whole process does not involve Horizon at all. To treat the issuing of
TCs as a countermeasure in the way that Dr Worden does is not logical. Even the Post
Office’s own witnesses of fact did not deal with the process of issuing transaction
corrections, because the Post Office’s solicitor correctly took the view that transaction
corrections were not part of the Horizon Issues and were outside the scope of the
Horizon Issues trial.
Further, the scope of the experts’ agreements in this respect under Horizon Issue 5 in
the 3" Joint Statement is relevant. That issue was “how, if at all, does the Horizon
system itself compare transaction data recorded by Horizon against transaction data
from sources outside of Horizon?” (emphasis added). Agreed entries include the
following:
1. Under sub-topic Reconciliation:
“5.1 Reconciliation between transactions recorded on Horizon and transactions
recorded by Post Office's clients is largely automated.
Detected discrepancies were subject to manual corrective fixes and/or the issue of
Transaction Corrections/Error Notices to the Subpostmasters.”
and:
“5.3 The adequacy of Post Office back office processes to prevent discrepancies in
branch accounts can be measured by the quality of the TC process. This quality
includes:
+ The processes of consideration of available data
POL00112816
POLO00112816
+ The level of errors observed in the process
* The level of complaints or disputes raised following a TC
* The level of upheld complaints following a TC
¢ The level of financial impact of erroneous TCs.”
3. Under sub-topic Third Party Data:
“5 4 Errors in third-party data have led to discrepancies in branch accounts, through
erroneous TCs being issued on Subpostmasters.
5.5a PO does not control the level of errors made by its third-party client organisations
(which may lead to errors in TCs), or the delays in their processes (which may lead to
delays in TCs).
5.5b PO can and should ensure, by careful investigation of disputed TCs, that only a
small proportion of errors by PO clients lead to losses for Subpostmasters, provided
that the Subpostmasters are in good control of their branches and have the required
information available.”
854. This shows that not only do the experts treat TCs as being outside Horizon in the Joint
Statement, also agree that the SPMs must have available the required information, but
also agree that the level of upheld complaints in TCs (and the value of erroneously
issued ones) is necessary in order to assess the quality of the Post Office back office
processes to deal with discrepancies. That data was simply not kept, according to the
Post Office witnesses. Further, Mr Coyne’s analysis showed that 77% of disputed
Santander TCS were upheld. Entry 5.6 in this Joint Statement, not agreed by Mr Coyne,
was an entry by Dr Worden which stated:
“The figure quoted by Mr Coyne (77% of disputed Santander TCs upheld) illustrates
that this process worked well.”
855. This means that Dr Worden’s analysis of “working well” was 77% of TCs issued in
respect of Santander transactions which were challenged by SPMs was upheld. It is
obvious that 77% upheld, means that 23% of TCs were not upheld, in other words were
TCs that should not have been issued. I do not consider that a ratio of approximately 1
in 4 TCs being incorrectly issued can be properly described as the process “working
well”, particularly as each TC would have a direct impact upon branch accounts. In my
judgment, for the process of issuing TCs to be “working well”, the level of accuracy
required would be far higher than 77%, and in excess of 90% at the very least. Nor do
I consider that to be unrealistically or unachievably high, given TCs impact upon branch
accounts, the accuracy of which is required to be very high, and expected by the Post
Office to be such that shortfalls in branch accounts have to be paid by the SPMs.
856. As put to Dr Worden in his cross-examination, if his approach to interventions by
human beings were to be extended to its logical conclusion, the group litigation itself
would be categorised as a Horizon countermeasure. In my judgment, that would be the
logical consequence of his approach. Plainly this litigation cannot properly be seen as
a countermeasure. His approach to countermeasures is not a sensible one; he mixed up
858.
860.
POL00112816
POLO00112816
both genuine countermeasures such as TIN; design functions such as DEP; and manual
corrections as well, and treated them all in the same way. He was wrong to do so.
It was also somewhat unhelpful for some of his countermeasures, such as later
correction of user errors or UEC, to be so widely defined as to include both manual
checks — including by the SPM themselves — and the Post Office and Fujitsu’s both
automated and manual processes. Another criticism of Dr Worden’s approach to
countermeasures relates to one which Dr Worden described in his paragraph 463 as
being “one of the most important countermeasures in Horizon”, and that is manual
inspection of data. I do not see how this can be a countermeasure in Horizon as defined
at all. Dr Worden’s summary states that “Any large business IT system is used by many
people, who view its outputs and check them against each other for consistency, and
against their own knowledge of the business. Subpostmasters, watching their branch
accounts, were a key component of this.”
(emphasis added)
The inherent illogicality in this is, in my judgment, obvious. SPMs, viewing their own
branch accounts, which are created or are part of the Horizon system, cannot in my
judgment be included as a countermeasure in the Horizon system (even if one ignores
entirely the definition of the Horizon system in the Horizon Issues). The fact that a SPM
can or may notice that (say) their accounts are (as the SPM sees it) wrong, or that there
is discrepancy of some thousands of pounds (or more, or less) in relation to a transaction
that they may not recognise, cannot sensibly be described as a countermeasure in the
way that Dr Worden does. As Mr Coyne explained, this “assumes that the person
scrutinising the data is able to identify the correct source of data to be relied upon, in
order to rule out what may or may not be erroneous in the first instance. Additionally,
its utility as a countermeasure is heavily reliant upon the person scrutinising the data
already knowing there is an issue that requires data inspection.” It also obviously relies
upon a human element, which necessarily depends upon the skills or knowledge of the
person. When this is put with the lack of notification to SPMs generally by Fujitsu
and/or the Post Office of existing bugs (and what the effect of those bugs was), which
was a feature of the evidence, it can be seen that there is no basis for including this in
any list of such measures.
The effect of some, but not all, of the bugs might be susceptible to being noticed in this
way, for example if there were a “doubled up” transaction in the same financial figure
at the same time — the SPM might remember that a customer performed a single
transaction in the way that Mr Patny did with MoneyGram as set out at [135] above.
But that would depend both upon the type of bug, the type of transaction, the memory
of the SPM and also crucially the amount. Another bug, Dalmellington, had some
noticeable effects as the evidence in this case shows. Other impacts of other bugs would
not be noticed in the same way. This shows the random nature of what Dr Worden
describes as a countermeasure. In my judgment, Dr Worden was quite wrong to include
this as a countermeasure. I find that something that is a “manual corrective fix” cannot
sensibly be described as a Horizon countermeasure.
One of Dr Worden’s other countermeasures was “testing good practice”, what he
termed TGP. The purpose of testing any system is not to prove that it is correct; rather
it is to prove (or attempt to prove) that it is not, a point explained by Dr Worden.
Examples of this are regression testing, user testing and testing of edge cases. Dr
861.
862.
863.
864.
POL00112816
POLO00112816
Worden relied upon what Mr Membury in particular had given evidence about in his
statement. I have summarised Mr Membury’s evidence at [500] to [507] above,
including his serious omissions regarding E&Y and their concerns. Dr Worden included
what was essentially a caveat in his first report, namely at paragraph 315 where he said
in his section dealing with development and testing of Horizon, specifically in relation
to Mr Membury:
“His witness statement describes several different audit processes, designed to ensure
by independent external review that quality is managed effectively in Horizon (Witness
Statement of Mr William Membery, 28 September 2018). If these audits have taken
place as described in his witness statement, and if the results are broadly as he describes
them, then that would increase my confidence that the quality of Horizon has been
effectively managed.”
He also stated that “a large system such as Horizon can only be robust if sufficient
attention has been paid to issues of quality in its development, testing and support.” I
have already identified that the experts are agreed that Horizon as it is now is relatively
robust. The documents also show that the approach of Fujitsu seems to be an increase
in focus upon such issues since about 2016 onwards. Testing and evaluation is in three
stages (taking this from another of Dr Worden’s appendices):
1. Unit (or component) testing takes place once the product is developed or procured
and ensures that the product conforms to its requirements.
2. Link (or integration) testing verifies that the product interworks with other major
components. This level of testing is generally performed on a complete release.
3. Acceptance testing proves to the Post Office's client that the development meets its
functional requirements and to the Post Office that it may be brought into use without
impacting on the existing solution or other applications.
This is then followed by what is called release management, which may be either major
or incrementally using interdependent components. This approach to testing is entirely
standard in the IT industry. No sensible programmer — or company involved in a system
such as Horizon - would introduce changes without testing their impact. However, there
is a paucity of documentation that demonstrates the degree to which Fujitsu’s testing
regime was successful. Dr Worden, in both his report and appendices, identifies in
general terms what was done and why it was important; there is no document available
to the experts that demonstrates (for example) Fujitsu’s test-pass percentages, or how
specific failures were managed and re-tested.
I do not consider testing to be a countermeasure. I consider testing to be an essential
part of the foundation of any system. I would go further and say that comprehensive
testing is standard in the IT industry.
Finally, so far as security or SEC is concerned, Dr Worden explained that “any system
that could be easily subverted would not be robust. Horizon is secured mainly through
‘separation of duties’, user authentication, access control and audit.” The evidence of
fact made it clear that the degree of access available to the whole of 3“ line support at
SSC was far wider than intended, and was subject to criticism by E&Y. Further, the
extent of the powerful APPSUPP privileges was extraordinarily wide.
865.
866.
867.
868.
869.
870.
POL00112816
POLO00112816
Conclusion on expert evidence
I prefer the expert evidence and approach of Mr Coyne to that of Dr Worden. This is
for the following reasons.
Dr Worden took a partisan view of the evidence of fact, preferring that of the Post
Office’s witnesses (by which I mean in this instance predominantly Fujitsu) to that
adduced by the claimants. This was most obvious in respect of his treatment of Mr Roll,
but also occurred in respect of the claimants’ evidence of individual experiences of
events caused by Horizon. No independent expert should do this. One example of this
was the important issue of whether Fujitsu could inject transactions under Legacy
Horizon without the SPM in question being aware of this. Dr Worden stated that he had
“established” that these would be visible to SPMs. Mr Roll’s evidence of fact was
directly to the contrary. Dr Worden had dealt with this by stating “So, in my opinion,
Mr Roll could not have made these changes to branch accounts “without the
Subpostmaster knowing”. In other words, in this expert’s opinion, the evidence of a
witness of fact (and a very important one at that) was simply not correct in fact.
However, in this case, even the Fujitsu witnesses eventually accepted (but prior to the
trial, in their supplementary witness statements) that this had been possible, and that
Mr Roll had been correct. Dr Worden’ supplementary report dealt with this by stating
that there was a lack of understanding or clarity on this point. He stated that “it seems
to me that I require further factual information before I can comment on this evidence”
and also that “Mr Rolls new evidence does not cause me to alter the opinion expressed
at paragraph 1119 of my first report, when commenting on Mr Roll’s first witness
statement, that he could not alter branch accounts without the Subpostmaster knowing”.
His first opinion in paragraph 1119 was wrong, and in my judgment obviously wrong,
and this should have unequivocally corrected both in his 2 report and also in his cross-
examination. The extract of this in [838] above makes it clear that he did not accept
that.
The sequence is therefore as follows. Mr Roll gave written evidence he had done
something, and Fujitsu gave written evidence this was not possible. Dr Worden
“established” that Mr Roll could not have done what he said he had done, and plainly
preferred the Fujitsu evidence. He therefore adopted a position of which party’s
evidence of fact was to be preferred. More factual evidence was then served and Fujitsu
witnesses accepted this could be done, in contradiction to their earlier evidence. Dr
Worden declined to “comment on this evidence” and remained of the view that Mr Roll
could not have done that which he said he had done — in other words, his factual
evidence was incorrect.
No independent expert giving evidence to the court should do this. It is an obvious
preferring of the evidence of fact of the party instructing him, added, in this case, to a
refusal or failure to accept further evidence of fact to the contrary which subsequently
emerged. This is even though the issue of remote access was accepted by the Fujitsu
witnesses in supplementary statements.
Mr Coyne did not do this. Indeed, he was fair and balanced in identifying where there
were such differences, and he was also clear that he did not prefer one side’s evidence
of fact to the other side. This is the correct approach for an expert to adopt.
871.
872.
873.
874.
POL00112816
POLO00112816
I find Dr Worden’s approach to the factual evidence I have identified conceming Mr
Roll a particularly egregious failure to maintain the necessary standard of impartiality.
Injection of transactions remotely into Legacy Horizon was an important factual point
in this litigation, and indeed in the whole history of the ongoing lengthy dispute between
the Post Office and the claimant SPMs. Further, in his expert declaration, Dr Worden
went wider than the standard wording and specifically referred to an earlier decision in
another case. He stated at paragraph 1194 the following:
“J understand that my duty in providing this report and giving evidence is to help the
Court on matters within my expertise, and this duty overrides any obligation to the party
by whom I am engaged or the person who has paid or is liable to pay me. I confirm that
T have complied and will continue to comply with this duty. I have not assumed that
any particular version of events is true and I have had regard to the case of Imperial
Chemicals Ltd vy Merit Merrell Technology Ltd [2017] EWHC 1763 (TCC) in
producing my report”.
That particular case (of which there are a number of judgments, on liability and
quantum) concerned a liability expert and his preference for the evidence of fact of the
party who had instructed him. I had stated in that judgment at [74] that “An expert's
role is not to decide issues of fact themselves, and choose what facts to believe and
what not to believe”. Specific reference was made by Dr Worden to this judgment in
his expert declaration. It might therefore be thought that the requirement upon him as
an independent expert no/ to take a view on which factual evidence he preferred was,
or should have been, at the forefront of his consideration. In any event, such an approach
by the court to how experts should approach evidence of fact is not particularly ground-
breaking. Independence requires an objective consideration by an expert of both sides
of the factual evidence. It is the court’s role, not that of an expert, to decide which
evidence of fact is accepted and which is not. Notwithstanding this, Dr Worden plainly
accepted the initial factual evidence of Fujitsu and rejected the factual evidence of Mr
Roll. Then when Mr Parker accepted Mr Roll’s account, Dr Worden still would not do
so. He was quite wrong to do this on both occasions.
Further guidance on the approach of an expert, if guidance be needed, is also provided
in [237] ofa later judgment in the same JCI y Merit series, namely [2018] EWHC 1577
(TCC) in which I stated:
“1. Experts of like discipline should have access to the same material. No party should
provide its own independent expert with material which is not made available to his or
her opposite number.
2. Where there is an issue, or are issues, of fact which are relevant to the opinion of an
independent expert on any particular matter upon which they will be giving their
opinion, it is not the place of an independent expert to identify which version of the
facts they prefer.”
Dr Worden went so far as to state, in respect of his consideration of the variations in
robustness of Horizon over time, that “in my opinion, as expressed in section 8, the
Claimants’ shortfalls are not caused by bugs in Horizon, or any lack of robustness in
Horizon.” The question of whether the claimants’ shortfalls was caused by bugs was
876.
POL00112816
POLO00112816
not one of the Horizon Issues as I have explained above, but in any event, Dr Worden’s
conclusions that the specific losses were not caused by bugs/lack of robustness was
only reached by means of his statistical approach in section 8 of his first report. This
was so heavily based on assumptions that were either not valid, or conceptually flawed,
that his conclusions were not valid either. I would go further — the whole process he
adopted so far as his statistical exercise was concerned was both invalid and of no
evidential value.
He also relied — in my judgment heavily - upon information from Mr Jenkins. Mr
Jenkins was not even identified as one of his sources of information in section 1.3 of
his first report, headed “Sources of information”. This means that Dr Worden was given
access to information that was not made available to his opposite number, contrary to
the point at (1) above in [873] above. Although there were some references throughout
the text of the report to Mr Jenkins, Dr Worden did not routinely identify where he had
relied upon Mr Jenkins. He also provided a great deal more information about this
contact with Mr Jenkins in his oral evidence than he did in his written reports. In his
cross-examination, he identified, when asked about a passage, that he had obtained that
information from Mr Jenkins, which plainly took place before the first report was
served as he accepted on his first day in the witness box it was “a year ago”. This was
not clear in the report itself. One example of this was paragraph 654.2, where the report
was dealing with the effect of the Receipts/Payments Mismatch bug, and Dr Worden
stated “Because the operation involved was apparently not a double-entry operation on
the BRDB, the countermeasure of checking the double-entry constraint DEA did not
catch it”. This information came from Mr Jenkins, but until Dr Worden was asked this
in cross-examination, no reader would be able to tell this. The involvement of Mr
Jenkins in this explanation in his report was simply hidden. Nowhere was there a note
or summary of all the information that had been given to Dr Worden by Mr Jenkins. In
this litigation in particular, and given the involvement of and information provided by
Mr Jenkins, who knew so much about the Horizon system, such a note or summary
was, in my judgment, essential. This was particularly important given there was no
witness statement from Mr Jenkins. Dr Worden had been provided with, and had used,
information from Mr Jenkins in addition to the witness statements served by the parties.
Dr Worden also introduced the concept of “lasting” and “transient” effect of bugs or
defects on branch accounts. There was no such distinction in the Horizon Issues
themselves. A further difficulty with this approach is that Dr Worden confirmed that
even an impact on branch accounts that was corrected several months later by a TC
would, so far as he was concerned, be of “transient” impact. A different way of stating
the same concept is that, so far as he was concerned, “transient” could be of quite long
effect. Given TCs are corrections generated outwith the Horizon system and are
effectively corrective actions issued by the Post Office itself, this moved his
consideration away from the generic Horizon Issues into the realm of the efficacy of
the Post Office business systems as a whole, not only including Horizon but their human
processes too. That is not what the Horizon Issues were concerned with, and he frankly
admitted in his cross-examination that had he not done this, his opinion evidence would
have been rather different. Mr Coyne did not use lasting and transient effect, and
correctly addressed the actual Horizon Issues themselves, as drafted and agreed by the
parties and as approved by the court.
877.
878.
879.
880.
POL00112816
POLO00112816
Dr Worden also adopted this different approach to impact upon branch accounts. Dr
Worden only considered an effect or impact on branch accounts where a discrepancy
loss (or gain) was not rectified by a correction such as a Transaction Correction, as set
out in the entry in the 2" Joint Statement about the experts’ different approaches. This
ignored the word “potential” in the Horizon Issues themselves. In my judgment, that
was a wholly unhelpful approach, and also had the effect of minimising or reducing the
number of bugs that have an impact upon branch accounts. This can be demonstrated.
by considering the following hypothetical example.
Consider a situation where there is a software bug, which in this instance I shall call H.
The effects of H are such that they cause the branch accounts of a particular branch Post
Office to suffer an unexplained loss of £999.99. This occurs, and the SPM of that branch
contacts the helpline and goes through to SSC, who create a PEAK and identify this
and after a few weeks of investigation, work out that this is an unexplained software
bug which they have not come across before. A KEL is therefore also created
identifying H. Future incidents of H occurring would (or should) lead to new PEAKs
which will also (or should) link to the same KEL, and to the same bug. Fujitsu report
to the Post Office about the existence of H, which they have now discovered, and
understand is a software bug. The Post Office accept that the sum of £999.99 has arisen
in the branch accounts as a result of H and issue a TC to the branch in that sum, to
correct the unexplained loss in that sum in the branch accounts. If one were to use Dr
Worden’s interpretation of impact upon branch accounts, by reason of the issue of the
TC, H would not be a bug with an impact upon branch accounts. Such an approach is,
in my judgment, wholly artificial and incorrect. It also defies the proper meaning of the
words.
Mr Coyne also considered substantially more PEAKS and KELS than Dr Worden, by
a factor of almost ten. Mr Coyne stated that he and his team had considered a broad
total of about 7,500. Dr Worden had only considered a far smaller sample. Mr Coyne’s
far more vigorous and thorough exercise relied far less on the process of extrapolation
inherent in Dr Worden’s approach, who was inevitably basing his opinion on a far
smaller sample than Mr Coyne. Given the breadth of the resources being deployed by
the Post Office in conducting this case, I find it difficult to understand why Dr Worden
restricted his consideration of PEAKS and KELS in this way. However, I consider that
the conclusions of Mr Coyne, based on the far more substantial consideration of these
important documents, are more reliable than those of Dr Worden.
To be fair to Dr Worden, he entirely accepted that he and Mr Coyne had approached.
their evidence differently. He stated the following when explaining why he had not
supported Mr Coyne’s information requests.
“A. Well, basically I think it became evident, and it is evident now, that two experts
took very different approaches, and my approach was top down, understand the
architecture and work down through things like KEL. So as far as I was concerned, I
had plenty of information to go on. And Mr Coyne's information requests didn't strike
me as things that, yes, I have got to really see that because I was -- had a different
priority. I was trying to do a top down understanding of the architecture and top down
look at robustness, and so on and so forth. And so I had plenty of documents to look at,
basically. So I think it is the different approaches taken by the two experts that led to
lack of overlap.”
881.
882.
883
884.
POL00112816
POLO00112816
This continued:
“Q. Sorry, not having supported the very many requests he had to make?
A. Yes, I felt his interests were different from mine and I really had plenty to do.
Q. And you initially only looked at KELs, didn't you, you didn't actually look at
PEAKs?
A. No, I looked at KELs and I looked at PEAKs where they were relevant. I felt that
KELs were a more distilled form of information. I felt they were sufficient in many
ways, especially when you go and look at the PEAKs.
Q. You looked at one or two PEAKs referred to in the KELs you looked at?
A. Yes. Look at some of them where you feel the KEL doesn't tell you enough.
Q. And Mr Coyne was looking at PEAKs and KELs?
A. I think in the first report he was looking mainly at KELs, like me, and in his second
report he turned on PEAKs.”
The reason that PEAKs are of great assistance is that there is a great amount of further
information contained in them than in the KELs, and they also give a far more complete
picture of the number of different occurrences of a bug. Both types of documents are
however useful. I do not know why Dr Worden should have seen Mr Coyne as having
“different interests” to him.
Dr Worden also entirely seemed either to misunderstand (or restrict his consideration
of) Horizon Issue 1, which concerned bugs, errors and defects having the potential to
cause apparent/alleged discrepancies or shortfalls in branch accounts or transactions, or
undermine the reliability of Horizon accurately to process and to record transactions.
When challenged about his approach to this, including the absence of the word “cause”
and the use of the word “potential” in the issue itself, he stated the following:
“A. What's the distinction between having the potential and actually doing? I mean, if
it doesn't actually do, then in some sense it didn't have the potential.”
This is not the correct, or conventional, use of the word “potential”. It also looks at the
matter in reverse. In the context of the Horizon system, whether bugs, errors or defects
in that system have, or had, the potential to impact branch accounts or transactions,
should focus one’s attention upon the possible effect of those bugs, errors or defects.
If, of course, a particular bug actually has had the effect of in fact impacting branch
accounts, then by definition it must also have had the potential to do so. Actual effect
is a sub-set of potential effect. It means impact upon branch accounts must have actually
happened. However, potential effect does not require that impact to have actually
occurred. The two terms are not synonymous. Dr Worden therefore moved away from
the actual issues before the court, and approached a great deal of his evidence from the
885
886.
887.
888.
889.
POL00112816
POLO00112816
point of rather different issues, which he seemed to have narrowed down himself. I
consider such narrowing of the issues to be unwarranted.
In their opening submissions, the claimants summarised this different approach as
follows:
“We respectfully will invite the court to adopt an approach more consistent with Mr
Coyne's approach which is to look at what actually happened with particular examples
and trace them through to a reasoned and careful conclusion and then from the ground
up, as it were, draw inferences upwards rather than from an overarching hypothesis
downwards.”
The “overarching hypothesis” was that of Dr Worden. Because he had taken such an
overall view, and an overall view on the most recent performance and functionality of
the Horizon System (which is now HNG-A as of February 2017, and is no longer HNG-
X), the degree of assistance his evidence provided to the Horizon Issues across the
whole life of the system, both Legacy Horizon and Horizon Online, was more limited
than that of Mr Coyne.
In this group litigation, when the first individual claims are being tried out, further
analysis will be required of the actual branch accounts of individual claimants. That
stage of the litigation will deal with actual branch losses, if any, that are said by the
claimants to be due to the actual impact of the bugs, errors and defects which exist. It
is at that stage that findings will be made about actual impact upon the branch accounts
of the different claimants. Some of the bugs and defects are agreed by Dr Worden and
Mr Coyne, and those that are not agreed are resolved by me in the Technical Appendix.
It is against the background of those findings that individual claims will be resolved.
The purpose of the Horizon Issues trial was to resolve the generic issues, as the parties
were poles apart about even the existence of such bugs, errors and defects, and whether
they had the potential to act upon branch accounts and transactions in the way alleged
by the claimants. Dr Worden moved away from consideration of those specifically
drafted and agreed Horizon Issues and answered rather different ones.
I also consider that Dr Worden, in his approach to countermeasures, insufficiently
differentiated between measures which were part of the Horizon system and those that
were not. He also included items in his countermeasures that plainly ought not to have
been included as such, for example those the result of human intervention or
involvement, and manual inspection of data by SPMs.
In my judgment, Mr Coyne emerged more or less unscathed from his four days of cross-
examination by Leading Counsel for the Post Office. That is not to criticise the cross-
examination itself, which was very thorough and careful. It is just that what the Post
Office set out to demonstrate, including lack of impartiality on Mr Coyne’s part (if not
positive bias), lack of reliability in his conclusions, and the other detailed criticisms of
him, simply was not achieved. He was described as “evasive” in some of the Post
Office’s Closing Submissions, a term which I find is somewhere wide of the mark in
describing how he gave evidence in the case before me, and how he answered questions.
He was also criticised in the same submissions as adopting “an approach [that] was
simply to look for bugs without giving a proper sense of scale, context or even balance”.
890.
891.
892.
893.
POL00112816
POLO00112816
In the Post Office’s closing submissions, there is a section headed “Judicial criticisms
of Mr Coyne’s approach” where reliance is placed on a judgment in Scotland, called
CGI IT UK Ltd y Agilisys [2018] CSOH 112 by Lord Bannatyne. In that case, Mr
Coyne had appeared as an expert and he was criticised by the judge for an unbalanced
view, amongst other things. The judgment was handed down on 4 December 2018. The
fact that he had been judicially criticised in that case was not specifically put to Mr
Coyne. However, the substantive criticisms of him, that he gave unbalanced and
selective evidence, and the other criticisms, were. The point that he had been previously
criticised in a judgment ought specifically to have been put by reference to that case, so
that he could answer this point. However, in any event I have read (as invited in the
Post Office’s submissions) the entire Scottish judgment, in which the evidence of Dr
Hunt, Mr Coyne’s opposite number, was preferred to his evidence. I have done this
with particular care and I have also considered all the criticisms made of him in that
judgment. The Scottish judgment provides a full explanation of the shortcomings of his
approach in that case. I have also taken this prior judicial criticism into account in
weighing up his expert evidence in this trial.
The Post Office, amongst other things stated that “it is submitted that the examples set
out above [ie in the Lord Bannatyne judgment] show the same tendency to reverse and
switch in response to difficult points.” In other words, it is submitted that Mr Coyne
adopted the same approach in the Horizon Issues trial as he had in the earlier case in
2018. I have considered the criticisms of Mr Coyne with particular care in the light of
this previous criticism, even though I do not have the benefit of Mr Coyne’s explanation
to the specific points arising out of that case. I do not find his evidence suffers from the
same deficiencies in this case, as Lord Bannatyne found in that earlier case. In
particular, one of those criticisms in CGI, that he had failed to consider material
documents and matters, simply does not remotely apply here. He and his team had done
an extraordinarily thorough job of examining many thousands of PEAKs and KELs,
and many more than Dr Worden did. I do not find that he reversed his evidence, or
switched, or that the other criticisms of him are made out. It may be that he has changed
his approach as a result of his experience in that judgment, or there may be other reasons
for it. However, I do not find the criticisms made of him to be made out in this case.
In my judgment, given the Horizon Issues, which expressly include “bugs, errors or
defects”, the approach of an IT expert to start by looking in detail at the large volume
of the Fujitsu PEAKs and KELs for bugs is entirely sensible. I reject the criticism that
Mr Coyne’s evidence was given without having any sense of scale, context or balance.
I conclude that Mr Coyne was impartial in his evidence in the Horizon Issues trial, and
was giving the court evidence that was entirely uninfluenced by the exigencies of
litigation or the interests of the party who called him. I find that he was independent,
and that his conclusions in the Horizon Issues care are reliable.
There were some subjects upon which he became a little flustered in his cross-
examination, for example on the final afternoon when he was taken to task for the entry
in the 2™ Joint Experts’ Statement on an agreed passage where he and Dr Worden had
agreed that “the number of distinct bugs, for which the experts have seen strong
evidence of the bug causing a lasting discrepancy in branch accounts, is between 12
and 29”. That latter number had come from Mr Coyne and he wished to reduce the
number, initially to 13 but then upon reflection to 21. Mr de Garr Robinson made a
great deal of this, as one would expect of a skilled cross-examiner, but in reality it is a
894.
895.
896.
POL00112816
POLO00112816
simple point, as I have explained above. It did show a shift of position because the
number of bugs did shift as I have explained. However, it is easy to overestimate the
degree to which making an error on figures in such circumstances does, or does not,
represent a lack of reliability on the part of an expert who has performed a substantial
body of work. It must be remembered that this was an agreed entry in an agreed Joint
Statement that was 44 pages in length (the experts agreed four separate statements,
running to 100 pages in total including appendix). I deal with the detailed analysis of
each alleged bug in more detail in the Technical Appendix, but the fact that the number
did not remain an immutable “29” in all circumstances is not, in my judgment, a point
against Mr Coyne. His attempts to ensure the final number was correct was not as
smooth as he might have liked, and it gave Mr de Garr Robinson an opportunity to
deploy his forensic skill, but in reality it did not impact upon Mr Coyne’s reliability.
Mr Coyne was widely attacked for a lack of partiality, and indeed, for positive bias in
favour of the claimants. I do not find those complaints or criticisms to be well founded.
They are not based on any real substantive instances that can be shown in any specific
respect to represent Mr Coyne demonstrating any lack of impartiality. They are not
made out. The points made are, in my judgment, insufficient grounds to dent his
credibility as an independent expert whose evidence can safely be relied upon in
resolving the Horizon Issues. He had not been previously instructed to undertake work
for either party, and he had no previous involvement in the history of the wider dispute
between the Post Office and SPMs concerning the Horizon system and its reliability. I
found his evidence before me to be neutral and independent, and not influenced by the
fact that he was instructed for the claimants.
Dr Worden’s decision, very late in the day, to prepare his 3“ Report, appears —by reason
of the dates - to have coincided with the dismissal of the recusal application. This matter
was not explored at all in cross-examination and so it is not necessary to consider the
timing of Dr Worden’s realisation that he could perform a new exercise. Given that his
oral evidence to me was that he was advised to serve that 3“ Report directly upon the
court, and effectively was left to draft the covering email to the court himself, I am
reluctant to criticise him for this unusual approach unduly. This is therefore not
something that I weigh in the scales to any appreciable degree in deciding whether his
opinion evidence and conclusions are more, or less, reliable than those of Mr Coyne.
I do not wish it to be considered that I am being universally critical of Dr Worden in all
respects. He had done a substantial amount of detailed analysis in his two reports, and
the overall knowledge that the court had, at the conclusion of the cross-examination of
both experts, was very much greater than it had been prior to the commencement of the
Horizon Issues trial. Dr Worden discovered some bugs himself, bugs that neither the
Post Office and/or Fujitsu either knew about, or had admitted the existence of, prior to
Dr Worden’s involvement in the litigation. He also took a sensible and considered view
of some elements of the documentation, the importance or relevance of which had not
been admitted by the Post Office earlier in the litigation, such as the KELs. Some of the
matters that were agreed by Dr Worden with Mr Coyne in the 1* to 4" Experts’ Joint
Statements saved a considerable amount of court time. His entries in particular in
respect of the bug table, where he considered the detailed contents of a number of
PEAKs and KELs, were useful as well.
897.
898.
POL00112816
POLO00112816
However, Dr Worden had become closely involved with the Post Office’s case in the
litigation. In my judgment he took a partisan view of the evidence of fact of Mr Roll,
which he should not have done, and he also seemed to be acutely aware of the Fujitsu
position. For example, if there was a problem in Horizon that originated with Riposte,
rather than part of the system written by Fujitsu, he was always enthusiastic to make
clear this differentiation in Fujitsu’s favour. It was not relevant to the Horizon Issues
whether bugs, errors and defects in Horizon were caused by Fujitsu or Riposte
(provided by Escher, a separate company), or any other part of the system. These are
all points that lead to the same conclusion, namely that in my judgment his expert
evidence was not entirely independent of the Post Office’s and/or Fujitsu’s interests,
and his conclusions were substantially less reliable than those of Mr Coyne. I do not
know whether he had found himself subconsciously influenced by his discussion with
Mr Jenkins or not, but regardless of the reason, his conclusions were not reliable and I
reject them in favour of those of Mr Coyne.
In all the major respects therefore, and taking account of all the points made both in
cross-examination and submissions by both parties concerning their own, and the
opposing side’s, IT expert, I prefer the evidence and conclusions of Mr Coyne to those
of Dr Worden.
The number of established bugs, errors and defects in Horizon
899.
K
900.
901.
902.
The
Audit Data
The experts are agreed, in item 4 of the points of agreement referred to above in respect
of Horizon Issue 3 as part of the 3 Joint Statement, that the Post Office does not consult
the audit data. The actual text of that entry is the “Post Office does not consult the full
audit data (unfiltered ARQ Data) before deciding how to handle discrepancies and
issuing Transaction Corrections.” Regardless of this agreement, it is obvious on the
factual evidence that the Post Office does not do so. Indeed, throughout both the
Common Issues trial and the Horizon Issues trial, it has been increasingly obvious that
the Post Office uses sources other than the audit data when it is challenging SPMs about
what they have actually done in their branch Post Offices. Various terms are used for
the audit data. The 3 Joint Statement uses “full audit data” and “unfiltered ARQ Data”.
In places the term “core audit data” and “core audit store” was used, which refers to the
same data and store. I shall refer to it as audit data.
The issue of audit data arises in the following way. Audit data is a complete and
accurate record of everything that has occurred, which in the context of Horizon means
including a full record of keystrokes used by a SPM (or assistant) in the branch. This
accurate record is kept in what is called the audit store. This is a secure place for the
keeping of such data. It is vital to the proper operation of a system such as Horizon that
such accurate audit data is kept.
In his evidence Mr Coyne referred to the concept of “Write Once Read Many”, which
has its own acronym, WORM. Dr Worden accepted that this was “pretty common” in
the industry. It means, effectively, that audit data is only written once — indeed, it must
be only written once. That is the whole point of audit data. However, it is written (or in
other words, recorded) so that it can subsequently be read, and the second part of this
903.
POL00112816
POLO00112816
concept, “Read Many”, means that it can be read many times. The audit data is there,
in my judgment, so that it can be consulted when required. The experts’ agreement
makes it clear that the Post Office does not consult it before deciding how to handle
discrepancies and issue TCs.
The Post Office did not accept that it should access the audit store as suggested by the
claimants, or consult the audit data. The claimants’ case is that the audit data should be
used in circumstances where there was a dispute between the Post Office and the SPMs
about what had occurred in Horizon in a particular branch. Mr Coyne for the claimants
criticised the Post Office in his 24 expert report for not using audit or ARQ data, and
relying instead on what was called management data instead, which was said (by the
Post Office) to be sufficient to demonstrate what had occurred. I will reproduce part of
Mr Coyne’s cross-examination on this.
“A. But did you see any documents of any sort indicating or referring to the stream of
data flowing on a continual basis out of the audit store into Post Office's management
systems?
A. No, but that's not how things would work. If Post Office wanted to get access to
the data in the audit store they would go to a screen or go to an application on their
computer and they would run the request for that data.
Q. Mr Coyne, I would like to suggest to you that it is completely unrealistic to think
that a separate sealed core audit store of the sort we're talking about should be cracked
open hundreds of times a day in preference to using management information systems
which are designed for that precise purpose?
A. I think the word "sealed" is misleading and the concept of cracking something open
to get access to it I think is misleading as well.
Things in an audit store are only -- can be written to and only written to once, and the
term that's often used is write once read many, WORM. So the process is written to
once, but people can read from that store on many occasions.
Q. But just to be absolutely clear, you had not and indeed you have not seen any
documents suggesting that Post Office had the ability to gain access to the audit store
on its own systems, had you? There was no design facility, there was no -- there were
no lines of communication between the audit store and Post Office in any document
you had ever seen, correct?
A. No, it looks as if the majority of the references to audit database access was from
Fujitsu personnel.
Q. And one final thing. Would I be right in thinking that now that you understand how
the audit store actually works and the costs and delays associated with extracting data
ona large basis from the audit store, would you accept that it would be disproportionate
to be using the audit store as a basis for making decisions on transaction corrections in
every single case?
904.
905.
906.
POL00112816
POLO00112816
A. Yes, it would seem that it would be very expensive and very slow to access the audit
store, and effectively for the number of transaction corrections you couldn't do that, and
therefore you accept that you make decisions on the management information systems
rather than the audit store.”
(emphasis added)
The Post Office therefore relied upon the fact that the audit data was effectively kept
by Fujitsu; that accessing it was akin to “cracking open” something that was a “separate
sealed” store; and that it would be disproportionate to use it to make decisions on
transaction corrections in every case.
The first and third of those points are accepted; the second one Mr Coyne described as
misleading. I agree with that response to all three of those points. I also find that the
concept of “cracking open” the “sealed store” is rather off the point of the very purpose
of audit data, and the uses to which it both can and should be put.
The following points are, in my judgment, important on this issue.
1. Audit data is necessary so that there is an accurate record of what has occurred on
the system, including what a SPM has done in branch, in terms of the keystrokes
actually used. The use of the word “audit” makes this clear, and also this is not a
controversial point.
2. The need to consult such an accurate record does not arise in every case where a
transaction correction is considered or issued. The management data is sufficient for a
large number of transaction corrections.
3. However, where there is a dispute between the Post Office and a SPM about branch
accounts, with the arguments on both sides about what or who is to blame, the audit
data should be consulted. That is one of the purposes of having it in the first place. I
can think of no sensible reason not to consult the audit data in such a scenario.
4. Doing so is not equivalent to “cracking open” a “sealed store”. The audit store is
sealed in the sense that data, once in the store, should not be capable of being changed,
or written to again. However, reading that data is not “cracking open” the store. It is
consulting or reading the audit data, one of the main (if not the only) purposes of having
such audit data in the first place.
5. The evidence in both this trial, and the Common Issues trial, where the Post Office
cross-examined a number of SPMs on events in their branch accounts by using a variety
of management information, other than the audit data, makes clear to me just how
important it is to use the audit data, rather than other sources including management
information. The management information is confusing, contradictory, has been shown
to be wrong and requires numerous assumptions or a “take it from me” type of approach
on the part of a questioner. It is rarely agreed what that management data shows. The
audit data, by its very nature, will be far superior and the best evidence available of
what has occurred on Horizon. It should be consulted in circumstances where there is a
dispute between the Post Office and a SPM.
907.
908.
909.
910.
POL00112816
POLO00112816
6. Finally, even the Post Office’s own witnesses were not agreed on what the
management data would, or would not, show. At one point Mrs Van Den Bogerd said
that Credence (which is management data) would record keystrokes in branch. This was
then contradicted by another Post Office witness who said Credence did not do this.
Whether the lack of understanding on the part of Mrs Van Den Bogerd was widespread
or not within the Post Office is unclear to me. It is however clear to me that Credence
data does not in all circumstances record all keystrokes performed in branch by a SPM.
The audit data does.
Dr Worden explained audit data as being a central principle of the whole system in the
following terms:
“Tt is a central principle of Horizon that the Core Audit Database acts as a secure 'gold
standard’ for branch accounts (countermeasure SEK) and that the audit record can only
contain events which originated at the counter - either in customer transactions or
monthly balancing.”
That is an accurate description in my judgment. I can see no sensible or justifiable
reason for the Post Office’s reluctance to consult the audit data in cases of serious
dispute with SPMs, in particular the types of dispute the subject of this case, and without
doubt, any dispute that involves criminal proceedings against SPMs. I also consider the
same point applies in relation to internal Post Office proceedings that lead to the
suspension or termination of SPMs. Acceptance of that point does not mean that that
the audit data has to be consulted for every transaction correction issued by the Post
Office. As can be seen from the passage of cross-examination at [903] above, the Post
Office effectively jumped to the issue of whether audit data should be consulted before
issuing every single TC. Mr Coyne accepted it was not necessary to do that. That is not
the same as accepting it should not be consulted in circumstances where there was a
dispute between the Post Office and SPMs.
Mr Coyne had identified issues with using Credence data. There was a one-hour
difference in the time stamps used between Fujitsu and Credence, which can hardly
have helped sensible investigations when SPMs raised queries, but there is more to this
than that. The E&Y review in March 2011 identified various issues with Credence,
including weak change controls within the back end of the systems which allowed
Logica developers (the third-party provider) to move their own uncontrolled changes
into the production environment, which included both Credence software code and the
data within Credence used for what was called “audit evidence” but which has to be
differentiated from what I am referring to audit records in the audit store. There was a
lack of further documentation to approve fixes and patches applied to Credence outside
of the release process, which meant that linking changes to issue tickets to record the
original request for the bug fix was not possible.
A concern was also noted by E&Y in respect of front end change process weakness.
The passage states:
“During our walkthrough of user administration of the front end of Credence we noted.
several users with administrator rights, including some generic users (this is noted
below as a separate point). These users have the access rights to create and amend.
reports, including those which may be relied upon for audit evidence. These users can
911.
912.
913.
914.
915.
POL00112816
POLO00112816
change report design, and processing without documented request, test or approval.
When users have the rights to change reports that are used by the business for
reconciliation, exception reporting or other processing, there is the risk that the reports
are manipulated either intentionally or accidentally.”
There is also at least one specific occasion, considered in the evidence during the trial,
which shows that the Credence data does not show the correct position. This was put to
Ms Van Den Bogerd. This was an occurrence at Lepton in October 2012. It led to
something that was referred to throughout the trial as the Helen Rose Report or Rose
Report, as the author of the report into it was called Helen Rose. I have dealt with it to
a degree at [227] above. That report records that:
“A transaction took place at Lepton SPSO 191320 on the 04/10/2012 at 10:42 for a
British Telecom bill payment for £76.09; this was paid for by a Lloyds TSB cash
withdrawal for £80.00 and change give for £3.91. At 10:37 on the same day the British
Telecom bill payment was reversed out to cash settlement.
The branch was issued with a Transaction Correction for £76.09, which they duly
settled; however the postmaster denied reversing this transaction and involved a
Forensic Accountant as he believed his reputation was in doubt.”
(“duly settled” means the SPM paid the Post Office that sum).
The Credence data showed that the SPM had reversed the transaction. By consulting
the audit data, Mr Jenkins discovered that he had not. This was expressly confirmed,
both in the Rose Report and also by Ms Van Den Bogerd in her cross-examination.
This shows that the management data is not entirely reliable. Had the SPM at Lepton
simply paid the approximately £80 and not engaged the forensic accountant — in other
words, had he not been prepared to pursue this matter — the Post Office would simply
have received that sum from the SPM to which it was not entitled, and the Rose Report
would not have emerged.
The issue of remote access also further complicates (or rather, weakens) the reliability
of management data. All of these concerns reinforce the view that I have in any event,
that audit data (as in, that stored in the audit store) is what should be used in the
circumstances that I have identified.
One point that occupied the parties a great deal in the Horizon Issues trial, upon which
there was some evidence, and upon which Mr Coyne formed a view based on his
analysis of some contemporaneous documents, is that Fujitsu raised a charge to the Post
Office for audit data requests (also called “ARQ requests”) above a certain number per
year. The numbers that were discussed were in the amount of hundreds of pounds per
request. The parties could not agree on what these charges were, although it was
accepted that there were a number each year that were either free, or the costs of which
were included in the contract sum (depending upon which way one looks at this matter).
The charging structure will, in terms of the amount per request above a particular
number, have changed over the years in any event.
That charges are raised by Fujitsu to the Post Office is not an adequate answer, in my
judgment, to the Post Office’s failure to consult or provide the audit data in cases such
916.
POL00112816
POLO00112816
as those in [907] above. There are some contemporaneous references within Post Office
documents suggesting this may have been a disincentive in some cases to raising ARQ
requests of Fujitsu. Further, there are numerous references within Post Office
documents (at a high level) of the very great cost to the Post Office of the Horizon
system generally, that cost being paid to Fujitsu. Fujitsu were said in one document to
see the contract with the Post Office as a “cash cow”. This entry came in a document
(parts of which were redacted, which I record simply for completeness) dated 17
January 2017 which were the minutes of the Post Office Group Executive Meeting of
that date. Ms Van Den Bogerd was “in attendance” for two items only, numbered 6 and
7, but not in the list of those “present” by which I assume she was not there for the
whole meeting. The Post Office Chief Executive and Group Legal counsel were there,
together with a number of others.
There is only one agenda item which is not redacted, namely item 5, IT Strategy. Parts
of the minutes relevant to the Horizon Issues are as follows:
“RISKS
+ Need to be clear for the Board what operational risks we are facing and how we are
mitigating those risks, are there any choices we can make to accelerate the solution to
get inside the risk appetite.
+ Already making these changes to the areas of risk, but will make this clearer. Need a
strategic discussion at GE on choices in the IT strategy
+ Horizon - cannot accelerate work on the data centre refresh, getting under control. FJ
running this well, risk review & declaration every month. RH will kick again but doing
as much as we can.
* Credence - Support and maintenance has moved to Accenture but on old hardware.
Single platform if goes, goes completely. Driving pressure into Accenture to get more
capability. Accenture picked up difficult pass and the additional costs at the moment.
Starting to see light at the end of the tunnel. RH having difficult conversations, flag up
any issues, assumptions and risks to GE. KPMG engaged to undertake a GAP analysis
to produce an IT control framework.
+ What are the risks of the ‘hop’ to thin client not working? RH - need to prove thin
client works and the ‘hop’ is a safe way to test we could revert to HNGA if the ‘hop’
fails. All nervous as we are dealing with legacy systems. More comfortable that we are
not changing Back & Front office at the same time, but still using Horizon. SSKs have
proven that thin client can work, it is the journey to implement that is vital.
+ Fujitsu - negotiations will be tough as POL do not hold the power. FJ see the contract
as_a cash cow, so need to persuade them that working with POL to migrate to cloud
technology is to their benefit against a 'too good' contract. They may say make us whole
but we cannot give additional work because of the procurement risk. Only choice if
they will not work with us is to start building an alternative. Come back to GE with an
update on the FJ negotiations plan”
(emphasis added)
917.
918.
919.
920.
POL00112816
POLO00112816
This document shows that Fujitsu, at least so far as the Post Office are concemed, are
seen by the Post Office as believing that the contract with the Post Office is “too good”
and a “cash cow”, such that Fujitsu may not even “work with us” [ie the Post Office]
in terms of improvement. Part of that very attractive commercial arrangement
(attractive for Fujitsu) may be charges that Fujitsu is entitled contractually to raise for
ARQ requests. Part of an attempt to reduce the expense of the contract by the Post
Office may be a reluctance to raise such requests. Dr Worden described this document
as “chiming” and said that “I have always had the impression from the governance
structure and the documentation and so on that Fujitsu were not short of budget, really”.
By that he meant Fujitsu were well-remunerated, and this remuneration was paid by the
Post Office.
Such private commercial arrangements between the Post Office and Fujitsu are a matter
for the Post Office and Fujitsu. They do not, in any way, justify any failure to seek the
audit data — the best evidence — in cases where SPMs are being suspended and/or having
their appointments terminated, particularly in circumstances where there are so many
bugs acknowledged as existing, and also at a time (much earlier than this judgment in
2019) when Fujitsu knew there were bugs in Horizon such as Dalmellington and
Callendar Square. There is quite enough in issue in this litigation as it is, without
widening it to include the way the Post Office deals with its commercial arrangements
with Fujitsu.
Finally on this point, I am surprised that the desirability - if not the actual and basic
need - to consult the audit data is a controversial point. In my judgment it is not only
good practice to consult the audit data, given the very purpose of audit data within a
complex IT system such as this one, but it is also obvious common sense. There is little
point in having audit data if it is not consulted in the circumstances that I have identified
above. I appreciate that the audit data itself will not be immediately comprehensible to
some SPMs. There may also be charges raised by Fujitsu in respect of such requests.
However, neither of these are reasons for the failure to consult it. I also make no
findings on whether Fujitsu are, or are not, entitled to raise charges and if so how much.
Those latter two points were not fully argued before me, and Fujitsu were not
represented.
Overall Conclusions
My judgment concerning the two experts, which of their evidence I prefer and why, has
been reached on free-standing grounds. Because of the different way that the experts
went about their tasks, it was not possible to consider in detail and track in Dr Worden’s
reports any analysis and possible explanation of single incidents, and attribute that
either to a specific bug, error or defect, or something else which was none of these
things. The approach he had adopted for his evidence did not permit that, as he was (in
a sense) working from the other end (overall reliability) back towards the beginning
(individual experiences), in contrast to the approach of My Coyne. Dr Worden
described his approach as “top down”. Mr Coyne, on the other hand, using PEAKs and
KELs in particular, identified a great many specific incidents that Fujitsu had recorded,
which plainly demonstrated the existence of numerous bugs. Indeed, some of the terms
used by Fujtisu made this crystal clear that these were known to be bugs. One notable
example is the expression of Anne Chambers — “this bug has been around for years” —
921.
922.
923.
POL00112816
POLO00112816
in February 2006. Dr Worden did engage with the detail of PEAKs both in the joint
expert meetings and also in his oral evidence, and the final result is that by the end of
the trial the summary at the beginning of Appendix 2 of the Post Office closing
submissions was forced to accept the existence of a great many bugs in Horizon.
Those claimants who gave examples of specific incidents specified them as well as they
could, and (usually) Ms Van Den Bogerd attempted to demonstrate that this was not
what had occurred, often by means of an assertion that if the SPMs had followed
procedures properly, what they said happened, simply could not have happened. The
Fujitsu witnesses gave some limited evidence of fact in relation to specific instances
and some specific PEAKs and KELs. The Post Office’s counsel did their best to
challenge the factual accounts that were given. I have provided the relevant findings in
respect of that above. I indicated at the Case Management Conference on 22 February
2018 that the Horizon Issues should be capable of resolution with limited, if any,
evidence of fact. Both parties chose to submit factual witness statements and called
factual evidence. The factual evidence submitted by both the parties was, in my
judgment, limited as expected and required.
The factual evidence of specific instances was of assistance in coming to conclusions
on the Horizon Issues. Indeed, I found some of the factual evidence to be of great
assistance. That of Mr Roll and Mr Godeseth was extremely useful. The latter, one of
the Post Office’s main witnesses and the Chief Architect of Horizon, was sufficiently
damaging to the Post Office’s case on the Horizon Issues that they were, essentially,
forced almost to disavow him, and the Post Office’s closing submissions were highly
critical of the accuracy of his evidence. As a further example only, Mr Latif gave
evidence that he had performed certain basic routine steps to transfer the sum of £2,000
between terminals, and (in accounting terms) that sum had — to use his word —
“disappeared”. The Post Office evidence challenging this was from Mrs Van Den
Bogerd, and essentially amounted to an assertion that if certain steps had been followed
by him, this should not, or could not, have happened. That would be all well and good
in an ideal theoretical world, but it entirely ignores Mr Latif’s primary evidence of fact.
I accept that primary evidence from the claimants’ witnesses in this trial as I have
explained in Part D. Mr Latif had been a trainer trusted by the Post Office to train other
SPMs, as explained at [98] above. Other references in the documents show both Romec
engineers, and even Post Office auditors, are recorded in PEAKs as witnessing events
supporting the type of occurrences that underpin the claimants’ case. These were all
ignored, and conclusions were drawn by Fujitsu that flew in the face of what had
occurred. Mr Latif’s experience is extraordinarily similar to the Dalmellington bug,
which is numbered 4 in the Bug Table. Mr Latif did not operate core and outreach
branches, therefore his experience is slightly different to what happened at
Dalmellington. However, if one is to take an objective and sensible view to the evidence
of a previously trusted SPM who trained other SPMs for some years, who states he did
something correctly and X occurred, then meeting that with an assertion that X simply
cannot have happened is not particularly sensible, nor is it persuasive.
The approach by the Post Office to the evidence of someone such as Mr Latif
demonstrates a simple institutional obstinacy or refusal to consider any possible
alternatives to their view of Horizon, maintained, regardless of the weight of factual
evidence to the contrary. That approach by the Post Office is maintained, even though
924
925.
926.
927.
POL00112816
POLO00112816
now there is also considerable expert evidence to the contrary as well (and much of it
agreed expert evidence on the existence of numerous bugs).
This approach by the Post Office has amounted, in reality, to bare assertions and denials
that ignore what has actually occurred, at least so far as the witnesses called before me
in the Horizon Issues trial are concerned. It amounts to the 21 century equivalent of
maintaining that the earth is flat.
When real world examples such as Mr Latif’s are put together with the expert evidence
that I have accepted — or even with Dr Worden’s lower figure for accepted bugs of 11
different ones — it can be seen that this institutional obstinacy by the Post Office
amounts to little more than repeated assertions that the Horizon system (both Legacy
and Online) cannot be to blame for the claimants’ experiences, coupled with (for some)
challenges to the claimants’ witnesses because the Post Office simply cannot accept
their factual accounts. The findings that I have made, on the evidence in the Horizon
Issues trial, show that the reality is rather different, and the existence of the bugs, errors
and defects that I have found do have the effect explained by Mr Coyne.
The approach that I have adopted to the evidence can be considered as follows. Firstly,
to consider the factual evidence, the relevant contemporaneous documents, and
applying the burden and standard of proof, reach evidential findings. I have considered
the challenges to that evidence, and if the specific instances of factual witnesses
explaining what they experienced could be explained (as the Post Office sought to do
with some of the SPMs) by them simply not being reliable, remembering incorrectly,
having been careless or even (for some) being criminal. I have also considered all of
the Post Office factual evidence which was admitted in this trial, the explanations of
the Post Office’s witnesses, and reached my factual conclusions. I have considered the
expert evidence on its merits, and then tested my conclusions against the evidence of
fact that was adduced by both parties, and considered whether my judgment on the
evidence of fact matched and supported, or contradicted, or in any way affected or
influenced my conclusions on the expert evidence. In a sense this was to consider the
factual evidence, and the expert evidence that I accepted and preferred, separately, and
see if they both reached the same conclusion. Secondly, the evidence can also be
approached in a different sequence. This is by considering the specific factual evidence
firstly, and then considering the expert evidence secondly and only in the light of the
conclusions on the factual evidence. Alternatively, one could simply decide the Horizon
Issues on the expert evidence alone, although that would be far less preferable and
would ignore entirely the important factual evidence of the Fujitsu witnesses, in
particular Mr Godeseth. However, even if that third way were to be adopted, the same
answers to the Horizon Issues would, in my judgment, be reached.
Whichever way is adopted, it must be remembered that any claimant can only ever have
personal experience of one small element of the Horizon system on a number of
occasions. The overview and historical performance of Horizon, both Legacy Horizon
and Horizon Online, has to come from Fujitsu witnesses and the two experts. Dr
Worden and Mr Coyne had far more of an overview of the life of the system, as did Mr
Godeseth and Mr Parker. Mr Roll had experience of it in its early years, but his evidence
was what led to the correct picture on remote access finally emerging.
928.
929.
930.
931.
POL00112816
POLO00112816
Thave done all of these exercises. Regardless of whichever approach is adopted, the
same conclusions are reached. In this respect, all roads lead to Rome. In my judgment,
the evidence of the factual witnesses accepted by me entirely supports and corroborates
the conclusions reached by Mr Coyne. Indeed, I would go further than this, and state
that Mr Godeseth’s evidence alone is enough to support and corroborate Mr Coyne’s
conclusions. When that is put together with the evidence of Mr Roll, and the
concessions that were obtained from the Fujitsu witnesses (in the circumstances of their
performance as witnesses, to which I have already referred) it is clear to me that the
correct conclusions to be drawn on the Horizon Issues are those drawn by Mr Coyne,
save and to the extent that I have modified them in any specific respect.
I consider that the evidence of the Fujitsu witnesses in particular, both former and
current, has been of considerable assistance to me in resolving the Horizon Issues. Mr
Roll, Mr Godeseth, and even in his own way Mr Parker (though not in the way that Mr
Parker himself would have intended) have all provided clear evidence of the problems
with the Horizon system, the bugs, errors and defects within both Legacy Horizon and
Horizon Online in its HNG-X form, the way that these problems were (or were not)
dealt with, and the way that Fujitsu had powers which, until shortly before the trial
started, Fujitsu sought to keep from the court, and may not even have fully disclosed to
the Post Office. Because the extent of these powers was kept secret in this way, the Post
Office finds itself now having made misleading public statements previously. If one
looks back to an earlier case management hearing and assesses how Fujitsu, through
the Post Office, sought to portray the contents and lack of importance and relevance of
PEAKs and KELs, then it can be seen that there has been a pattern of considerable
defensiveness over the Horizon System. There has certainly been a lack of
transparency.
I consider, as explained in the Technical Appendix, that Legacy Horizon was not robust,
and that although Horizon Online in its HNG-X form was better than Legacy Horizon
(not least, I consider, because Riposte was no longer part of Horizon) its robustness was
questionable, and did not justify the confidence placed in it by the Post Office in terms
of its accuracy. HNG-A is a different matter, and the experts are agreed that it is far
more robust than Horizon in earlier times. On the face of the relevant KEL, it is not
possible to say whether the Drop and Go bug, number 28 in the Bug Table, which
occurred in June 2017, was HNG-X or HNG-A.
I turn therefore to the points identified by Males LJ in Simetra Global Assets Ltd and
another v Ikon Finance Ltd and another [2019] EWCA Civ 1413 to which I have
referred at [74] above. I have already dealt with the first point, succinctness (obviously
lacking in a judgment of this length). The second point is a requirement to consider all
the evidence. I have done this, both for and against each of the cases advanced by both
the claimants and the Post Office. I have not recited every single disputed fact as to do
so would lead to an even longer judgment, and substantially delay production of this
judgment well into 2020, and which would be contrary to the overriding objective. The
third point is to identify the issues, consider the evidence in respect of the issues, and
give reasons. For this trial, the Horizon Issues were agreed by the parties and approved.
by the court; the review of the evidence has been longer than is ideal, but I have
considered it all, whether recited in this judgment, or the Technical Appendix, or not.
The fourth point is somewhat salutary, namely that fairness requires that a judge should
932
933.
934.
POL00112816
POLO00112816
deal with apparently compelling evidence, where it exists, which is contrary to the
conclusion which he proposes to reach.
In my judgment, there is no such evidence advanced by the Post Office. It is pointless
to speculate on how such compelling evidence might have been presented, if there had
been any. The evidence that has been advanced for the individual occurrences, for
example by Mrs Van Den Bogerd, can be seen once tested in cross-examination to
amount to unfounded assertion and, sometimes, an expressed desire in 2019 to go away
and do more research. This is not apparently compelling evidence to the contrary to my
conclusions. Additionally, the time for further research by the Post Office is long past.
The Post Office’s approach to evidence, even despite their considerable resources
which are being liberally deployed at considerable cost, amounts to attack and
disparagement of the claimants individually and collectively, together with the wholly
unsatisfactory evidence of Fujitsu personnel such as Mr Parker. The Post Office
evidence also includes a very high level overview of Horizon by its expert which
amounts to a claim that it has worked quite or very well, most of the time. This is
effectively what Dr Worden’s section 8 exercise consists of. Counsel for the Post Office
at one point put to Mr Coyne that there was more likelihood of an SPM being hit by
lightning than of having a bug, error or defect in Horizon cause a branch account loss
in their branch. Mr Coyne did not agree with that, and neither do I. Indeed, such a point
wholly ignores the enormous number of PEAKs that show that branch accounts were
potentially impacted on a great many occasions.
Also relevant are the other points which Males LJ went on to consider, namely the
importance of contemporaneous documents. He stated the following on this subject at
[48] and [49] of Simetra:
“T48] In this regard I would say something about the importance of contemporary
documents as a means of getting at the truth, not only of what was going on, but also
as to the motivation and state of mind of those concerned. That applies to documents
passing between the parties, but with even greater force to a party's internal documents
including emails and instant messaging. Those tend to be the documents where a
witness's guard is down and their true thoughts are plain to see. Indeed, it has become
a_commonplace of judgments in commercial cases where there is often_extensive
disclosure to emphasise the importance of the contemporary documents. Although this
cannot be regarded as a rule of law, those documents are generally regarded as far more
reliable than the oral evidence of witnesses, still less their demeanour while giving
evidence...
[49] It is therefore particularly important that, in a case where there are contemporary
documents which appear on their face to provide cogent evidence contrary to the
conclusion which the judge proposes to reach, he should explain why they are not to be
taken _at face value or are outweighed by other compelling considerations. It is,
however, striking that the judgment in this case contains virtually no analysis of the
contemporary documents many of which appear to shed considerable light on the nature
and purpose of the critical confirmations and the way in which they were understood.”
(emphasis added)
T have already explained that the subject matter of the Simetra case is very different to
this one, and I emphasise here that my analysis of the contemporaneous documents is
935.
936.
937.
POL00112816
POLO00112816
in respect of the Horizon Issues in the context of this case, not the Simetra case which
concemed very different allegations. Here, the categories of documents that are most
illuminating in terms of specific incidents with Horizon over the years are the very
numerous PEAKs and KELs. These emanate from, and are created within, Fujitsu. They
are, in my judgment, a very good means of getting at the truth in this case. They show
what was going on and the type of unexplained problems that numerous SPMs were
experiencing in practice over the years, as they were reported to the SSC. They contain
statements made when Fujitsu personnel’s “guard is down and their true thoughts are
plain to see”. Some of them also record that Romec engineers, or the Post Office’s own
auditors, have seen what has occurred and ruled out user error. Notwithstanding this,
Fujitsu attribute user error to what has occurred.
They also contain some very telling expressions, not in the context of a Fujitsu
employee giving evidence as a Post Office witness in this trial, but at the time, with
such unguarded comments as Anne Chambers in February 2006 stating that “this
problem has been around for years and affects a number of sites most weeks” and “this
appears to be a genuine loss” on another occasion (as examples only). These entries are
at odds to the publicly stated position by Fujitsu both then and indeed later. In my
judgment, the PEAKS and KELs deployed in the Horizon Issues trial, and there were
very many, are consistent with my conclusions on the evidence and the answers that I
have arrived at to the Horizon Issues. Obviously no account was taken in any of the
evidence, and no account is taken by me in this judgment, of the 5,000 KELs that Fujitsu
disclosed in September 2019, months after the trial had ended.
The Post Office’s internal Horizon Online Induction Training slides and accompanying
notes dated 7 December 2009 form a useful snapshot of how Legacy Horizon was
viewed. The purpose of HOL was “to deliver a significant reduction in the total annual
cost of ownership of Horizon, whilst ensuring the system remains fit for purpose in the
21st Century.” These materials make it clear that the Programme had been identified in
2005 and was "to deliver a significant reduction in the total annual cost of ownership
of Horizon". Legacy Horizon was seen within the Post Office as being very expensive,
although its overall cost is not relevant to the Horizon Issues. The notes to the materials
state:
“However, we need to continue to manage firmly any over-expectations of the frontline
that Horizon Online will deliver improved functionality - they may see this as a missed
opportunity so will not cure all the issues and problems that users have with Horizon
although where practical, and at no extra cost, we have smoothed away a number of
"rough edges".
(emphasis added)
Another type of document that is illuminating in terms of the Horizon Issues are both
internal Fujitsu briefing documents, and to a lesser extent some internal Post Office
emails, that show the same approach to Horizon within those organisations as drawn by
my conclusions. One example relates to the so-called Ping fix. After it was brought in,
the number and value of TCs fell dramatically so far as Camelot/the Lottery was
concemed. This appears at [427] to [428] of my judgment on the Common Issues,
Judgment (No.3). In 2007 there had been the following TCs in respect of the Lottery:
approximately (to 1 decimal point, which represents £100 thousand) £22.8 million;
2008 were £12.5 million; 2009 were £12.0 million; in 2010 £11.3 million; and in 2011
938
939.
940.
POL00112816
POLO00112816
£4.5 million. In 2012, the year the Ping fix occurred, these fell below £1 million for that
year.
The Introduction to a Post Office document of 2011 entitled “Transaction
Acknowledgements — End to End Incident Management and Data Flow” explains the
following purpose to the Ping fix:
“Introduction
The PING Project was part of the Back Office Efficiency Programme within Post Office
Ltd.
The objective of this project was to automate a process that converts Client transactional
data files (e.g. Paystation), into a data feed to the branch, ie. a Transaction
Acknowledgement file.
The branch will then process the data electronically on Horizon Online and thus avoid
non compliance issues and the resulting cost to the business to process queries as a
result.
Please note - The term PING is an internal Post Office Ltd term and branches and
helplines will know this new functionality as 'Transaction Acknowledgements’.
(emphasis added)
The compliance issues were caused by the fact that the Lottery terminals were on one
system, namely that operated by Camelot, whereas Horizon was the system operated.
by the branch that was selling the Camelot products, and there was no electronic transfer
of data in to Horizon in respect of Camelot sales. The “objective” in the Post Office
document makes it clear that the Ping fix is a required improvement to the functionality,
to overcome issues caused by having two different systems which meant that there was
(prior to the Ping fix) no automated process to convert the client transactional data files
into the data feed necessary for Horizon for the branch accounts. This has nothing to do
with correcting excessive carelessness or fault on the part of SPMs. It is, in my
judgment, about remedying a deficiency in the functionality of Horizon. That document
also made it clear, because there are entries to this effect, that outages might mean that
the system would not deal with the matters sufficiently or accurately. A template
message for SPMs is even included in the document for when this occurs, although that
template message does not say that outages would lead to inaccuracies. It is plain that
outages would indeed lead to this, as this is mentioned in many places in the document,
and acknowledged by its authors. This information was not however, based on the
templates, to be given to SPMs.
A theme contained within some of the internal documents are an extreme sensitivity
(seeming to verge, on occasion, to institutional paranoia) concerning any information
that may throw doubt on the reputation of Horizon or expose it to further scrutiny. One
entry in a document that makes it clear that the Post Office itself has already recognised
this is contained in a document authored by Mrs Van Den Bogerd, entitled “Extracts
from Lessons Learned Log” document of 11 November 2015. One entry under “issues
identified” was as follows in respect of the Post Office’s behaviour up to that date:
941
942.
943.
944.
945.
POL00112816
POLO00112816
"Failure to be open and honest when issues arise eg roll out of Horizon, HNGx
migration issues/issues affecting few branches not seemingly publicised."
(emphasis added)
The Post Office’s guard may have been down when that document was written; it
predates issue of the first claim form by about 6 months. I consider that it is the type of
document that Males LJ had in mind when he spoke of a witness’ guard being down,
when their true thoughts are plain to see. It plainly records Mrs Van Den Bogerd’s view
of the Post Office’s approach to Horizon when it was written in November 2015.
In July 2016, following the disclosure of the Dalmellington bug, the following e mails
went to and from very senior people in the Post Office. On 1 July 2016 the Chief
Executive, Paula Vennells stated to Alisdair Cameron and Rob Houghton:
“Subject: The Dalmellington Error in Horizon / problemswithpol
Dear both,
This needs looking into please.
https://problemswithpol.wordpress.com/2015/11/10/the-error-in-
horizon/?iframe=true&theme_preview=true [This is a reference to an internet post
about a conviction of an SPM]
Can I have a report that takes the points in order and explains them.
Tim McCormack is campaigning against PO and Horizon. I had another note from him
this am which Tom will forward, so you are both in the loop.
We must take him seriously and professionally.
This particular blog is independent of Sparrow but clearly related in that it appears to
present similar challenges that were raised in the course of the scheme.
I'm most concerned that we/our suppliers appear to be very lax at handling £24k. And
want to know we've rectified all the issues raised, if they happened as Tim explains.
Thanks.
Paula”
Mr Houghton of the Post Office sent it on to someone else on the same day, copied to
Mrs Van Den Bogerd. This stated:
“T need an urgent review and mini <taskforce> on this one. It probably needs to link up
heavily with Angela’s work as FSC are mentioned extensively - Angela cfi. I don't
know how we respond to this but can we section a few inside people to get all over it
and give me/AI/ Paula evidence and understanding.”
However, the same recipients also received the following message from Mr Houghton
on the very same day:
“Can you stand down on this please? [A redacted section then follows]
Any specific actions and I will revert.
My apologies.”
The e mail chain is heavily redacted and therefore the reasoning behind this volte face
within the Post Office is not shown, nor was it explained by any Post Office witness.
946.
947.
948.
949.
POL00112816
POLO00112816
Thus the entirely understandable initial reaction of the Chief Executive of the Post
Office, that a point by point investigation was required in respect of specific criticisms
of Horizon in respect of one particular SPM who blamed Horizon for shortfalls and
discrepancies, appears to have been very swiftly shut down and not pursued due to a
decision at the highest levels in the Post Office. I can think of no justifiable reason
why the Post Office, institutionally, would not want to address the Chief Executive’s
points and investigate as she initially intended, and find out for itself the true situation
of what had occurred.
Both the recipients of the Chief Executive’s e mail of 1 July 2016 above are
specifically named in the Technology Strategy Update Decision Paper of 30 January
2017 in which the following is stated specifically. Mr Houghton is the Author of the
paper, and Mr Al Cameron is “the Sponsor”. Part of that document records:
“This document forms an update to the IT Strategy approved in July 2016 by the PO
Board. In July we outlined that IT was not fit for purpose, expensive and difficult to
change.”
The same document also states that "We need to quickly rationalise and resolve
misaligned contracts enacted to support legacy IT, obsolescence, a lack of PO
technical competence, particular focus on Fujitsu and Accenture”.
(emphasis added)
This therefore means that the decision taken no/ to investigate further in July 2016
and to “stand down”, in itself a surprising decision for a reputable institution to take
given all the circumstances, was taken at broadly the same time as a conclusion was
reached that the IT “was not fit for purpose”. The reference to the “lack of PO
technical competence” is a concern, given [75] of the Technical Appendix which
makes it clear that currently the Post Office can introduce new clients into Horizon
without requiring Fujitsu input. However, that level of competence may have changed
since 2016. The IT Strategy, which is referred to in this Update Decision Paper, and
outlined that “the IT was not fit for purpose” and was approved in July 2016 by the
Post Office Board is, in my judgment, entirely consistent with my conclusions on the
evidence. It is not, however, consistent with the Post Office’s response to the
claimants’ pre-action letter. The Post Office’s response is a lengthy letter dated 28
July 2016 which states that “the investigations to date have consistently pointed
towards human error or dishonest conduct in branches as the most likely cause of
shortfalls.”
Nor is it consistent with the way the Post Office explained, in its oral opening
submissions, that the Post Office had been investigating incidents earlier in terms of
bugs. So far as Dalmellington is concerned, the oral submissions were as follows:
“’...even the documents that your Lordship saw this morning with the Dalmellington
bug, your Lordship will see the rigour that's applied. There's concern that postmasters
aren't given advice that might be incorrect. The rigour associated with that process and
the determination of Fujitsu and the other people involved, the other stakeholders, to
get to the bottom of what happened is quite striking in my submission.”
950.
951.
952.
953.
POL00112816
POLO00112816
Dalmellington was investigated in 2015, and the outcome of that investigation is
available to this court. The outcome of that investigation is far from favourable to the
Post Office’s position on bugs and robustness of Horizon. The oral submission that
the Dalmellington bug was investigated rigorously (which it plainly was) does not
easily sit with the documents showing that one year later the Post Office specifically
decided not to investigate other incidents at all, and to “stand down” those tasked with
investigating it. This is notwithstanding that the Chief Executive’s initial (and
understandable) reaction was that the allegations needed to be looked at “seriously
and professionally”, that an investigation was needed, and that an urgent review and
mini task force was required.
Further, a conclusion in the terms that the IT is “not fit for purpose” is not something
that would have been reached lightly in an IT strategy, or approved lightly by the
Board. The Post Office Board is a serious level within the organisation. The Board are
not likely to be involved in, nor to have matters brought to their attention, that are
anything other than serious, considered and fully researched. This conclusion in this
contemporaneous document has not driven my conclusions on the evidence — rather, it
is the other way around. I have reached my conclusions on the evidence independent
of contemporaneous references such as this one, but then tested my evidential
conclusions against these to see if there is a contradiction. However, there is none,
and they are entirely consistent.
There are only very isolated Post Office documents inconsistent with my conclusions.
I shall deal with only one. On 17 August 2015 the BBC Panorama programme
broadcast a programme which contained allegations about Horizon. The
contemporaneous response to that by the Post Office in its statement to SPMs started
with the following passages, which I shall reproduce. The bold emphasis was present
in the original:
“The Post Office wholly rejects extremely serious allegations repeated in BBC’s
Panorama programme of 17 August 2015. The allegations are based on partial, selective
and misleading information.
e The Post Office does not prosecute people for making innocent mistakes
and never has
e There is no evidence that faults with the computer system caused money
to go missing at these Post Office branches
e There is evidence that user actions, including dishonest conduct, were
responsible for missing money”
The first and third of those bullets points are for another day, and are not for this court
in any event for the reasons explained at [60] to [66]. This judgment deals with the
detailed background behind the second bullet point. The confidence expressed in 2015,
in the second bullet point, in the computer system and its accuracy is not consistent
with the evidence I have accepted, the expert evidence on the number of bugs in the
Horizon System (both Legacy Horizon and HNG-X) and my findings. The document
was however drafted for public consumption, and this explains the inconsistency. To
the extent that the second bullet point conflicts with my findings in this judgment, that
public statement in 2015 by the Post Office is factually incorrect. This is not to make
any findings on any individual claim, or on the cause of money said to be missing in
the accounts of any particular branch Post Office.
954.
955.
956.
957
958.
POL00112816
POLO00112816
In my judgment, a number of both the Post Office’s own internal documents, and
Fujitsu ones too, namely those that were not drafted for public consumption, plainly
support my conclusions on the evidence. Further, certain matters that have emerged in
the Horizon Issues trial — such as discussions within Fujitsu itself as to whether the Post
Office should be told certain detrimental information about the Horizon system, and
the Post Office’s own decision at the highest level not to investigate certain matters as
recently as in 2016 — are of great concern. The Post Office has gone to great lengths
over the years, and spent a great deal of time and a huge amount of money, in defending
the performance of Horizon. It is also the case that the Post Office must have been
reliant on Fujitsu to a certain degree in terms of being provided with accurate
information. That is not only obvious from the evidence, but has also been agreed by
the experts in the 3“ Joint Statement. That accuracy from Fujitsu has not always been
available, as demonstrated by this judgment.
However, regardless of that — and this judgment does not deal with who, if anyone, at
the Post Office knew precisely what about Horizon, and when, as these are not part of
the Horizon Issues — in my judgment a number of the specific internal contemporaneous
documents, rather than being contrary to my conclusions on the evidence, are entirely
consistent with those conclusions. I do not know if the Post Office Board, who were
told in July 2016 that its own IT was not fit for purpose, will be particularly surprised
at the findings on the expert evidence in the bug table, and the number of admitted bugs.
It does not much matter whether they are or not.
The experts’ agreements in particular have been of great assistance, but everything has
been considered. This judgment, together with the Technical Appendix, is of substantial
length, and to recite everything that has been deployed by both sides, in terms of
evidence, submission and reference to all contemporaneous documents, is neither
necessary nor desirable.
It must also be remembered that Horizon as it is today, or at least in the last couple of
years, since it became HNG-A, is a very different system to earlier times. Legacy
Horizon ceased to be used and was discontinued in 2010. Modern Horizon Online as it
is today is not the same as the system that was introduced in 2010; it is different and
more robust, on the agreed expert evidence, to the system as it was when the litigation
started with the issue of the first claim form on 11 April 2016. It is now HNG-A rather
than HNG-X, and runs on a completely different Windows platform. The dates of the
bugs in the bug table are (between them collectively) from 2000 to 2018, although there
are far fewer in recent years. Having said that, some — such as Bug 8, Recovery Issues,
and Bug 28, Drop and Go have been experienced up to 2018 and 2017 respectively.
The latter is a reference data bug and hence easily fixed. The Technical Appendix at
[421] provides a summary of the different bugs from Legacy Horizon and Horizon
Online. It can be seen that there were more in Legacy Horizon than HNG-X in any
event, and only very few in HNG-A.
The Post Office has been very concerned as to the outcome of this litigation on its wider
business. It described the group litigation as long ago as its Opening Submissions in
November 2018 for the Common Issues trial as representing “an existential threat” to
its business. I do not consider that these answers to the Horizon Issues represent a
significant threat to the Post Office’s entire business. Findings in this judgment as to
the performance and robustness of Legacy Horizon from 2000 to 2010, and then of
POL00112816
POLO00112816
Horizon Online (in both its forms, HNG-X and HNG-A) from 2010 to 2018 are not
findings on the Horizon system as it exists at the date of distribution of the draft of this
judgment, in November 2019. These findings cannot be routinely applied to the way
that HNG-A operates as at December 2019, which is when the judgment will be
formally handed down. It is agreed by the experts that the Horizon System in its HNG-
A form is now relatively robust. This judgment is a historical analysis of the Horizon
System as it relates to the period in question in the group litigation, not a judgment upon
Horizon HNG-A as it is today.