WBON0000134
WBON0000134
McSherry, Nicola
Pheasant, Andrew
Sent: 05 July 2013 16:23
To: Rodric Williams
Subject: FW: Second Sight Report
Attachments: _DOC_27096022(1)_POL Interim Report v24 + BD comments.DOC
Hi Rodric,
If you need anything over the weekend do not hesitate to give me a call on my mobile. I am around and happy to
help.
Many thanks
Andy
Andrew Pheasant
Associate
for and on behalf of Bond Dickinson LLP
Follow Bond Dickinson:
Blin)
www.bonddickinson.com
From: Parsons, Andrew
Sent: 05 July 2013 12:59
To: Rodric Williams (~
Cc: Matthews, Gavin; Gaisford, Rosie
Subject: Second Sight Report
Rodric
My comments attached. On the whole, I'm deeply unimpressed with the report.
Headline thoughts:
1. Despite the rhetoric, SS have not managed to identify a problem in Horizon or a gap in our SR responses.
2. In general, SS has provided little if any analysis. Predominantly, they have just quoted the SPMRs and POL's
views without adding their own assessment. This was not the scope of work agreed with SS who were
supposed to neutrally evaluate the position — I see little if any actual evaluation.
3. The few views expressed by SS generally lack justification and explanation. They draw conclusions without
providing supporting reasons. This is in breach of the Investigation agreement.
WBD_000004.000001
WBONO000134
WBON0000134
SR1—the most problematic. SS's conclusion is that the Horizon system does not provide timely responses in the
event of a power/comms failure. The facts simply do not support this conclusion. Horizon recovered in just 4 minutes
and the SPMR knew immediately that there was a problem as Horizon did not print a customer receipt. ®
SRs, 5, 21 and 22 are dealt with more fairly, but mainly because SS haven't reached any conclusions yet.
Kind regards
Andy
Andrew Parsons
Senior Associate
for and on behalf of Bond Dickinson LLP
Direct:
wovie: GRO I
www.bonddickinson.com
WBD_000004.000002
Confidential
24 - Interim Report into alleged problems with the Horizon system
1. Introduction and Scope
1.1. Following discussions with Post Office Limited ('POL') Senior Management in June and
July 2012 and with the Rt Hon James Arbuthnot MP and Alan Bates and Kay Linnell
representing the Justice for Subpostmasters Alliance (‘the JFSA’), Second Sight was
appointed to carry out a Review into alleged problems with POL's 'Horizon' System.
1.2. The remit of the Investigation/Inquiry was later defined as:
"to consider and to advise on whether there are any systemic issues and/or concerns
with the "Horizon" system, including training and support processes, giving evidence
and reasons for the conclusions reached".
1.3. It was also agreed that Second Sight's report would:
“report on the remit and if necessary will contain recommendations and/or
alternative recommendations to Post Office Limited relating to the issues and
concerns investigated during the Inquiry. The report and recommendations are to be
the expert and reasoned opinion of Second Sight in the light of the evidence seen
during the Inquiry.”
1.4. It became necessary to ensure that references to “the Horizon System" were understood
and agreed by all stakeholders. Was Second Sight to look only for defects in the software
code of Horizon? Or, was it to take a broader view and also examine:
a) the surrounding Operational Processes, both at branch level and in POL's central
processing centres;
b)
the interfaces between the Horizon system and other systems that are the
responsibility of organisations other than POL such as Camelot, the Bank of Ireland,
the Co-Op, various Energy Companies and the 'LINK' system for processing Credit
and Debit Card payments and withdrawals;
¢)
the power supply and telecommunications equipment that connects every Horizon
terminal to POL's centralised data centres;
qd
the training available to Sub-Postmasters (‘SPMRs’) and their staff and whether it
was commensurate with the demands of the day-to-day job at the counter;
e
the actions need to ‘balance’ at the end of each Trading Period (‘TP’) and the
investigation work needed in dealing with errors and Transaction Corrections ('TCs');
f) the level of support available to SPMRs and their staff from POL's Helpdesk system;
Page 1
WBONO0000134
WBON0000134
WBD_000004.000003
WBONO0000134
WBON0000134
8) the effectiveness of POL's audit and investigative processes, both in assisting SPMRs
who called for help in determining the underlying root cause of shortfalls and in
providing evidence for other action by POL such as in Civil and Criminal Proceedings.
1.5. In answering the question as to whether Second Sight was to only examine the narrowly-
defined Horizon software, or the far more broadly-defined Horizon system, POL's own
definition of ‘Horizon’ provided much of the answer.
1.6. In May 2011, POL's Information Manager defined "Horizon" as follows:
“I can advise that the name Horizon relates to the entire application. This
encompasses the software, both bespoke and software packages, the computer
hardware and communications equipment installed in branch and the central data
centres. It includes the software used to control and monitor the systems. In
addition, I can advise you that testing and training systems are also referred to as
Horizon".
1
N
This POL definition does not include the above-mentioned "audit and investigative
processes”, but it quickly became clear that POL's audit and investigation methods have
had a profound impact on the SPMRs involved in almost all of the cases examined.
1.8. Second Sight's Investigation has consequently addressed matters well beyond the
narrow definition of the core software component of Horizon in order to ensure that we
have adequately addressed the totality of the concerns raised by SPMRs.
1.9. Before describing the approach adopted in this Investigation, it is necessary to put the
scale of the Investigation in context.
1.10. Second Sight has been asked to investigate 47 cases submitted to either the JFSA or
to the office of the Rt Hon James Arbuthnot MP. All of these submissions are highly
critical of POL's Horizon system and in many cases, the way that POL has dealt with the
matter reported.
1.11. [The entire Horizon system involves approximately 68,000 users and processes over 6
million transactions every day. The entire population of over 11,800 branches was
notified about the proposed investigation by Second Sight and this resulted in only a
small number of additional cases being reported. Whilst in no way minimizing the
potential importance of the cases reported, this level of response suggests that the vast
majority of SPMRs and branches are at least reasonably happy with the Horizon system. ‘Comment [AP61]: Critical point that
needs greater repetition. After making this
point atthe start, it is then forgotten in the
remainder of the report which means the
report loses this context.
2. Approach adopted
2.1. Second Sight has examined cases submitted from two sources. The first selection of
cases were those submitted by SPMRs, with the endorsement of their constituency MP,
through the office of the Rt Hon James Arbuthnot MP. There were 29 such cases.
2.2. The second source of cases was through the JFSA. These cases were submitted in
accordance with an Agreement dated December 2012 between POL, Second Sight and
Page 2
WBD_000004.000004
23.
2.4.
25.
26.
27.
WBONO0000134
WBON0000134
the JFSA (see Appendix 1). That Agreement set a cut-off date of 28th February 2013 for
the submission of suitable cases to the JFSA or directly to Second Sight.
In the event, over 60 SPMRs contacted the JFSA and 18 cases were considered by the
JFSA to be suitable for submission to Second Sight. These 18 JFSA-sourced cases were
generally simpler, more recent and better documented than the cases submitted via
MPs.
In dealing with each case, Second Sight first requested copies of all the documents in
POL's Case File. The initial plan was to interview each SPMR after all the POL-sourced
documentation had been examined. This has proved to be much more difficult than was
expected. Delays in producing case documentation to Second Sight have added
materially to the cost of the investigation and to the time taken to complete it. The main
problem here seems to be that POL does not maintain one central file for each case.
Rather, documents have to be gathered from multiple internal sources.
Where MP sponsored cases have been subject to either Civil Recovery or Criminal
Prosecution, POL's centralised Legal Department was able to supply many documents.
However, we found that a significant number of cases had not progressed this far and
that documentation was held in many locations within POL, including the National
Business Support Centre (‘the NBSC’), the Helpdesk, the Branch Support Team, the
Security Team, the Former Agent Accounting Department, and Legal Services.
In several instances, POL's seven-year Document Retention Policy has meant that little or
no documentation was available for Second Sight to examine. The same retention policy
applies to the underlying Horizon computer data. In a number of instances we were
provided with POL created documents by SPMRs, where POL had been unable to supply
the same document, even though it was within the 7 year retention period.
After examining all of the available documents and in some cases the Horizon computer
data relating to each case, Second Sight has been making contact with each SPMR in
order to obtain, through telephone calls and face-to-face interviews, the SPMR’s version
of events. Second Sight then summarised the SPMR's assertions into one or more ‘Spot
Reviews’. To date, 29 Spot Reviews have been created by Second Sight and other Spot
Reviews are planned. (Nine Spot Reviews have been sent to POL and a formal response ‘Comment [AP62]: There are 10 SRs.
received. Twenty Spot Reviews are currently ‘work in progress’.
3. The concept of a ‘Spot Review’
EB
3.2.
It became clear at an early stage in the investigation that it would not be efficient or cost
effective for Second Sight to examine all of the issues raised by SPMRs or covered in
POL's Case Files.
Accordingly, and with the consent and approval of both the JFSA and individual SPMRs,
Second Sight conducted a ‘fast track’ review of the available information in each case and
identified the key issues that were relevant to the remit of the Investigation. Each key
Page 3
WBD_000004.000005
WBONO000134
WBON0000134
issue was then dealt with as a ‘Spot Review’. A case with multiple issues would give rise
to multiple Spot Reviews, each of which would be dealt with on an individual basis.
3.3. It was agreed by POL, Second Sight, the JFSA and the Rt Hon James Arbuthnot MP that
any report issued by Second Sight would maintain anonymity with regard to the identity
of individual SPMR cases. Accordingly, this Report does not reveal the identity of any of
the cases being considered. in all instances where POL was asked to respond to specific
issues, the SPMR's identity was revealed to POL, but only after the SPMR's permission
had been obtained.
3.4. [This approach to Spot Reviews was intended to be a self-contained, easy to understand
procedure, free from unexplained acronyms and backed up by SPMR supplied evidential
materialI Each Spot Review was then submitted to POL for a formal response. The POL ‘Comment [AP63]: Wrong. On the
response was then discussed with both the SPMR and the JFSA and an attempt made to Fed csr pee eee
reach agreement and closure between POL and the SPMR, as to the issues dealt with in clay points. Key info was often missing
ite name of affected product; date of
each Spot Review. tei!
3.5. Regrettably, in all instances so far, no such agreement and closure has been achieved. In ‘Comment [AP64]: Only ad feedback
the face of assertions, by both the SPMR and by POL, supported in many cases by only Beil cesta aicte ee
partial or conflicting evidence, Second Sight has attempted to find out what really picture on this
happened. In most of the Spot Reviews investigated so far, we have been able to find
additional information that has been of assistance in understanding what actually
happened.
3.6. This Interim Report covers 4 Spot Reviews where we have been able to reach a
preliminary conclusion or at least make substantial progress on the matters being
reviewed.
3.7. As Spot Reviews were prepared, discussed and responded to by POL, Second Sight was
able to see a number of ‘thematic issues’ that were of concern to many SPMRs. These
issues, some of which are described in Section 7 of this Interim Report, and which will be
addressed in more detail in the Final Report, are those that have featured in many of the
47 cases reported to us.
4. Involvement of the JFSA:
4.1, At the request of the MPs representing their SPMR constituents and with agreement
from POL, Second Sight has worked closely with Mr Alan Bates of the JFSA and with the
JFSA's appointed Forensic Accountant Kay Linnell. This developed into a sound working
relationship and Second Sight wishes to put on record its thanks to both Mr Bates and Ms
Linnell for their help and professional conduct throughout the investigation.
5. Spot Reviews and Responses from POL:
5.1. This Interim Report deals with just 4 of the 29 Spot Reviews so far prepared by Second
Sight. These 4 Spot Reviews deal with events that are typical of the matters reported to Comment [
3 I context. Only affected a few SPMRS
Second Sight by many SPMRs, They also relate to matters that appeared, both at the icouparedI teeta eenter ofa
Page 4
WBD_000004.000006
time they were issued to POL and when the selection was made for inclusion in this
Interim Report, to be particularly relevant to the remit of the Investigation.
5.2.
Second Sight has asked POL to deliver Spot Review responses that would prove as easy to
understand as the Spot Reviews themselves; that addressed the spirit, as well as the
letter, of the SPMRs' complaints; and that were backed up by evidence.
Si
I. Whilst the Spot Review responses received from POL can be seen to be thorough, they
fare long and highly technical documents that are likely to prove hard for most SPMRs to
understand. Also, in some cases, they present counter-assertions, based on Standard
Operating Procedures and Controls, rather than tangible evidence of what actually
happened. Accordingly, it has been necessary to summarise and simplify the responses
received.
5.4. Our experience shows that even apparently robust controls sometimes fail to work, or
can be circumvented by a determined and skilful person. Second Sight is therefore
seeking further evidence in support of POL's responses to some of the issues covered by
the Spot Reviews dealt with in this Interim Report.
5.
w
Aconsequence of this is, that whereas we had expected that some form of closure would
be reached between POL and the SPMR associated with each Spot Review, this has so far
not been achieved.
5.
a
Itis of course hard for POL to prove the negative (i.e. that controls have not been
circumvented) but it is only fair to say that POL now finds itself in the same situation that
has faced all of the SPMRs who have submitted cases. They too, were unable to prove
that the shortages or transactions that they reported to POL, and in respect of which they
sought POL's help, were not the result of their own (or their employees’) errors or
criminal activity. In every case we have looked at, only limited support has been provided
to SPMRs by POL.
Sa
N
\Unfortunately, so far, POL has only acknowledged minor failings in the implementation of
its procedures and processes, or in other relevant areas. It has agreed in principle to a
number of process improvements relating to the matters under investigation by Second
Sight, and some of these have been implemented already. Unfortunately, many of the
SPMRs we have dealt with remain aggrieved and dissatisfied with what they see as POL's
overly technical and apparently unsympathetic response.
6. Did defects in Horizon cause some of the losses for which SPMRs or their
staff were blamed?
6.1. There is still much work to be done on the cases Second Sight has been asked to
investigate. We have concluded in one of the four Spot Reviews covered by this Interim
Report (Spot Review SRO1) that although the Horizon system operated as designed, the
Jack of timely, accurate and complete information presented to a SPMR was a significant
factor in the SPMR failing to follow the correct procedure.
Page 5
WBONO000134
WBONO0000134
‘Comment [AP66]: $s's requirement is
impossible to achieve. POL cannot be both
detailed around what has happened in
‘each case whilst also being brief and
accessible.
‘Comment [APG7]: Very odd comment.
‘No IT system is perfect. Every IT system
‘works by trusting its users. Of course if an
'SPMR is determined to commitment fraud
then this is possible.
{ Comment [AP68]: What controls are
I they considering here? None of the Spot
Reviews showed any suggestion that
"controls" had been circumvented,
{ Comment [AP69]; Because there 2)
no malo flings :
‘Comment [AP610}: Ss need to offer a
vview on whether the SPMRs position is
justified, SS were not engaged to simply
re-state the complaints of SPMRS.
Comment [AP611]: see detailed
‘comments further below.
WBD_000004.000007
6.2.
63.
6.4.
6.
6.
6.7.
6.8.
6.9.
6.10.
w
a
In that incident, shortcomings in the branch's primary and fall-back telecommunications
equipment exposed a system weakness that led to a poor counter-level experience both
for the SPMR and his customer.
We also note, in Spot Review SR22, that POL made a change, just a few days after the
SPMR was suspended, to the underlying technical processes relating to the events
reported. It is possible, that if this change had been implemented earlier, many of the
problems would not have occurred. 2 ‘Comment [AP612]: ve
Als change was unconnected to SPM
In the course of our extensive discussions with POL over the last 12 months, POL has
disclosed to Second Sight that, in 2011 and 2012, it had discovered "defects" in Horizon
online that had impacted 77 branches.
The first defect, referred to as the "Receipts and Payments Mismatch Problem", impacted
63 branches. It was discovered in September 2010 as a result of Fujitsu's monitoring of
system events (although there were subsequent calls from branches). The aggregate of
the discrepancies arising from this system defect was £9,029, the largest shortfall being
£777 and the largest surplus £7,044. It is not yet clear whether the SPMRs with the
surpluses profited from them, or whether those showing shortages had to make them
good.
The second defect, referred to as the "Local Suspense Account Problem", affected 14
branches, and generated discrepancies totalling £4,486, including a shortfall of £9,800 at
one branch and a surplus of £3,200 at another (the remaining 12 branches were all
impacted by amounts of less than £161).
POL was unaware of this second defect until when, a year after its first occurrence in
2011, it re-occurred and an unexplained shortfall was reported by an SPMR.
POL's initial investigations in 2012 failed to reveal the system defect and, because the
cause could not be identified, the amount was written off. Fujitsu looked into the matter
early in 2013 and discovered, and then corrected, the defect.
It seems however, that the shortfalls (and surpluses) that occurred at the first occurrence
(in 2011) resulted in the SPMRs being asked to make good incorrect amounts.
POL has informed us that it has disclosed, in Witness Statements to English Courts,
information about subsequently-corrected defects or “bugs” in the Horizon software.
7. Thematic or Systemic Issues
7A,
7.2.
It has become clear that whereas the Horizon system appears to achieve its intended
purpose almost all of the time and operates smoothly for most SPMRs and their staff,
some combinations of events can trigger situations where problems occur.
The following issues have been reported to us by multiple SPMRs as being of particular
concern about the Horizon system:
Page 6
WBONO000134
WBON0000134
e”*
but so what?
WBD_000004.000008
WBONO000134
WBON0000134
a) IA multi-product system that is far more complex and demanding than, for example,
that found in a typical high street bank;
b) Multiple transactional interfaces (‘hand-offs') to systems outside of Horizon such as
Lottery Scratch Card and Bank of Ireland ATMs that cause repeated and possibly
large shortfalls that take undue amounts of time to investigate and resolve;
¢)
Unreliable hardware leading to printer failures, screen misalignment (pressing one
icon sometimes results in the system selecting an incorrect icon) and failed
‘communications links;
d
The complexity of end of Trading Period ('TP') processes and the lack of a ‘suspense
account’ option which would allow disputed transactions to be dealt with ina
neutral manner;
e
inexperienced trainers and gaps in training coverage;
The lack of some form of on-site Supervision and Quality Control similar to that
made available to staff employed in POL Crown Offices;
The receipt of centrally input, overnight ‘corrections’ and other changes allegedly
not input by SPMR’s or their staff;
hy
Inadequate Helpdesk support, with responses that are ‘script-based’ and sometimes
cause further or greater problems;
POL investigation and audit teams that have an asset-recovery or prosecution bias
and fail to seek the ‘root cause’ of reported problems;
j) Acontract between SPMRs and POL that transfers almost all of the commercial risk
to the SPMRs, but with decreasing support being provided. In its risk reward
decision making, POL benefits from any savings, while SPMRs may suffer increased
risk! ‘Comment [AP613]: Again, ss have
provided no commentary on these issue
[ust restated SPMR's positions.
7.3. We have read all of the examples of problems reported to us by the SPMRs we have
contacted. We can’t help concluding that had POL investigated more of the "mysterious
shortages" and problems reported to it, with the thoroughness that it has investigated
those reported to it by Second Sight, POL would have been in a much better position to
resolve the matters raised, and would also have benefited from process improvements, ‘Comment [AP614]; $5 have failed to
consider the viability / cost of
implementing this change ~ again poor
analysis on their part.
7.4, It may be that a significant limitation in the way that POL responds to matters reported to
it are the terms of reference for the POL Investigations Division. The standard contract
between POL and SPMRs states:
“The Investigation Division does NOT enquire into matters where crime is not
suspected.”
Page 7
WBD_000004.000009
7.5. This appears to suggest that POL does not provide any investigation support to SPMRs,
except where criminality is suspected. The cases we have examined show that POL does
sometimes provide limited investigative support to SPMRs reporting problems, but
clearly, POL's ability to do this is constrained.
rh
a
\it is also unfortunate, in our view, that when POL does investigate cases, there is often a
focus on ‘asset recovery solutions’ without first establishing the underlying root cause of
the problem. This is also an example of a missed opportunity to be in a much better
position to resolve problems and to benefit from process improvements,
7.7. Another issue raised, by some of the SPMRs that we have had contact with, is the
allegation that the only time they were provided a copy of the full contract between POL
and SPMRs, was when POL commenced litigation or recovery actions. This is contrary to
POL's policy and procedures and enquiries are underway to find out what has happened
in the cases where this allegation has been made.
7.
Py
The following 4 sections now cover the 4 Spot Reviews where we have been able to reach
preliminary conclusions or at least make substantial progress in investigating the matters
raised.
8. Spot Review SRO1
8.1. The SPMR reports that there were intermittent internet connectivity problems on 4
October 2012. Online payments and withdrawal transactions were sometimes
successful, but also failed on occasions. It is likely that Horizon was partially operating
through its back-up (mobile phone) connection. Some card payments had to be
attempted two or three times before being accepted. At approximately 10:32 hrs, a
customer tried to pay his £76.09 phone bill with his bank debit card, but was not
successful. The customer then withdrew £80.00 cash and used this to pay the phone bill.
8.2. The SPMR stamped the customer's phone bill as evidence of receipt of payment,
returning change of £3.91. Several weeks later, the customer returned from holiday to
find his phone had been cut off due to non-payment of the phone bill. The SPMR’s
examination of the Transaction Log showed that all components of the transaction had
been reversed by POL. The SPMR states that he did not initiate those reversals, nor did
he receive any reversal notifications.
8.3. The SPMR raised this as an issue with POL but was told that due to cost issues the
Horizon transaction data, necessary to fully investigate the matter, could not be
requested. It was implied that the SPMR had stolen the money and he was told to make
good the shortage. This meant that 2 people had paid the phone bill: the customer who
handed cash to the SPMR, and also the SPMR on instructions from POL to make good the
shortage, after POL centrally had paid the bill.
8.4, The SPMR was subsequently informed that he should have had a surplus of £76.09 due to
the reversal of the transactions.
Page 8
WBONO0000134
WBON0000134
{ Comment [AP616]: This is quoted
I from the SPMR but iti not clear that this is
the SPMR’s view and not SS's analysis.
WBD_000004.000010
WBONO000134
WBON0000134
8.5. POL's 10-page response to Second Sight asserts that the Spot Review does not
demonstrate any failing in Horizon and that the root cause of the difficulties suffered by
the SPMR was his failure to follow the on-screen and printed instructions given by
Horizon. POL states that the SPMR should have realised that some transactions had been
automatically reversed because:
a) _ when the transactions in question first failed to be processed (because Horizon
could not get a response from the Data Centre), Horizon asked the SPMR whether
he wished to cancel or retry the transactions in response to which the SPMR opted
to retry the transactions;
b
when the transactions failed again, the SPMR opted to cancel the transactions;
c) Horizon then automatically disconnected and printed a "disconnect" receipt that
showed the transactions that had been automatically reversed;
d) astandard customer receipt was not produced and this should have told the SPMR
that the full transaction had not proceeded;
e) following the disconnect, the SPMR was required to log back on and, as part of the
standard recovery process, Horizon printed a "recovery" receipt which again showed
the transactions that had been reversed and those that had been recovered.
8.
a
POL's response states that there were 4 attempts (at roughly 45 second intervals) to send
the completed basket of transactions to the Horizon Data Centre. All attempts used a
mobile phone (back-up) connection. The SPMR’s records all show these connection
attempts to have failed. However, from the Data Centre's perspective, one of the
attempts did result in all of the data in the Horizon transaction ‘basket’ being successfully
transmitted to the Data Centre but, due to the connectivity issues, the branch did not
receive a confirmation of this at the time from the Data Centre.
N
The cash withdrawal transaction for £80 could not be cancelled as this had already been
processed by the Bank.
8.
cs
The net effect of all of this was that, whilst his telephone bill was not paid, the £80 debit
to the customer's bank account was correctly processed, even though this was not
reported to the SPMR at the time this transaction was processed by Horizon. The success
of this part of the transaction was only notified to the SPMR after the customer had left
the Branch, fas it took some time to complete the recovery and reconnection processes. ‘Comment [AP617]: WRONG the
terminal never printed the transaction
receipt so SPMR knew immediately that
the transaction had failed.
8.
io
. Procedurally, the SPMR was at fault here because he was not meant to allow the
customer to leave the counter until Horizon had finished its Recovery Processing.
8.10. The SPMR had stamped the customer's Telephone Bill, as proof that it had been paid
at 10:32 hrs, but this should not have been given to the customer until the Horizon
system had printed out all of the Session Receipts. This did not occur until 10:36 hrs,
which was after the customer had left with his stamped Telephone Bill. It was therefore
Page 9
WBD_000004.000011
impossible for the SPMR to return the customer's £76.09 or to retrieve the receipt-
stamped Telephone bill.
8.11. Second Sight is more sympathetic to the SPMR’s position than POL appears to have
been. POL's view is that the Horizon system operated as designed. In our view, timely, ‘Comment [AP618]: What's the
8.12.
8.13.
8.14.
8.15.
9. Spot Review SROS
9.
9.
9.3.
a5
2.
accurate and complete information was not presented to the SPMR at the time the
transaction occurred and that lack of timely, accurate and complete information was a
significant factor in the SPMR failing to follow the correct procedure.
At the time this problem occurred, there were multiple telecommunications failures
in the branch's main (ADSI/telephone) links and Horizon was using a mobile phone link to
communicate transaction data over a poor quality signal.
When operating, in that degraded mode, with a complex multi-part transaction
(involving communications to the banking system as well as to Horizon), the Horizon
system did operate in accordance with its design.
But, not being able to reverse the customer's banking transaction (the £80.00 bank
Debit Card withdrawal), Horizon relied on the SPMR being able to give the customer all of
his money back and either turning him away with his Telephone Bill unpaid or starting the
whole process again.
Even if the customer had still been present when the recovery processes were
completed (five minutes after being handed his stamped Telephone Bill) and even if the
SPMR had been able to immediately work out what had happened and what remedial
actions were necessary, this would not amount to an acceptable SPMR/Customer
justification behind ths vie
recovered in just 4 minutes.
WBONO000134
WBON0000134
Horizon
experience, It also raises questions about the suitability of the mobile phone backup ‘Comment [AP619]: On what basis do
connection and whether a more resilient service should be provided.
they reach this conclusion. Again, a loss of
Context. This type of fallure occurs very
rarely so customer experience if only
impacted infrequently.
This SPMR states that on Tuesday 19th August, 2008 he observed an individual in the
basement (or a boiler room type area with lots of pipe work) of the Fujitsu office in
Bracknell who demonstrated an ability to pass transactions directly into the Horizon
system and in so doing to alter, in real time or overnight, the recorded holdings of
Foreign Currency in POL Branch Offices. The SPMR also stated that this person, after
altering a branch's cash balance, then “made light of it” saying “I'd better reverse that
entry now or he'll have a shortage tonight.”
The SPMR further states that the person did this by generating an outgoing remittance
for a branch (known as a ‘Rem Out). The SPMR explained that what he observed was
contrary to POL's repeated reassurances that any form of ‘remote access’ to Horizon
transactions at branch level was possible.
Of potential significance is the alleged comment that “he'll have a shortage tonight.” This
could mean that the alleged transactions were not directly input to Horizon but to some
Page 10
WBD_000004.000012
WBONO0000134
WBONO000134
other system that was linked to Horizon by way of overnight batch processing, or in some
other way.
9.4. To put this allegation in context, over two years later, in a 7th December 2010 letter to
Alan Bates (Chairman of the JFSA), signed by Mr. Edward Davey, MP (the then Minister
for Employment Relations, Consumer and Postal Affairs), Mr. Davey gave the following
assurance:
"I recognise that the core of the JFSA's concerns relates to the Horizon system to
which you attribute the financial discrepancies and shortages which have led to a
number of subpostmasters having their contracts terminated and subsequent court
action. However POL continues to express full confidence in the integrity and
robustness of the Horizon system and also categorically states that there is no
remote access to the system or to any individual branch terminals which would allow
the accounting records to be manipulated in any way."
9.5. POL’s response states that:
a) In August 2008, the basement of Fujitsu's building did contain a Horizon test
environment with access to four test versions of Horizon;
b) It is this test environment that is believed to have been witnessed by the
SPMR;
c) This test environment was not physically connected to the live Horizon
system so it was not physically possible for the alleged transactions to have
occurred. It is possible that someone showed the SPMR some form of
adjustment to the test environment that was misunderstood.
9.6. Simply stated, POL has rejected this allegation, stating that none of its staff who were
present at the alleged 19" August 2008 meeting, had any access to live data.
9.7. POL has also subsequently suggested that the SPMR's assertion that the employee may
indeed have used the phrase "this is the live system" because, in addition to the test
version of the then un-released new version of Horizon (‘HNG-X’), being accessible from
there, so was a test version of the then-current and live (old) Horizon system.
9.8. It is unfortunate that, due to the length of time that has elapsed since the alleged visit,
neither POL nor Fujitsu have been able to identify any individual who met with the SPMR
on the date of his alleged visit to Bracknell.
Comment [AP620]: We have now
identified this person - Matin Rolfe. We
9.9. However, on 1 July 2013, the SPMR managed to find an email proving that the meeting pete ictopad eet
took place and identifying one of the POL employees involved. Further urgent enquiries
about this matter are continuing.
9.10. In order to compensate for the lack of information from either POL or Fujitsu which
could resolve this conflict of evidence, Second Sight requested and was provided with the
Page 11
WBD_000004.000013
WBONO0000134
WBON0000134
email records of 7 POL employees believed to have been working in the Fujitsu office at
Bracknell at the relevant time.
9.11. Unfortunately, due to a change in email systems, emails from 2008 have not yet
been provided to us, but we have reviewed the relevant email records for 2011. This
review has shown:
a) Anumber of different teams of POL employees were working in the Fujitsu
office in Bracknell in 2011 and possibly earlier. These teams were located on
the Ground Floor and the 2" and 4" Floors of the Fujitsu office
b) An email sent to a number of POL employees in April 2011, including a
member of the Testing team in Bracknell, included the following comment:
“Although it is rarely done it is possible to journal from branch cash
accounts. There are possible P&BA concerns about how this would
be perceived and how disputes would be resolved.”
9.12. “P&BA” refers to ‘Product and Branch Accounting’, which is a team within POL that
is responsible for the back-office accounting system.
9.13. POL has told Second Sight that the comment noted above, describes a method of
altering cash balances in the back-office accounting system, not Horizon. We note
however that any changes to Branch Cash Account balances in this way would be
subsequently processed in Horizon using the Transaction Correction (‘TC’) process. This
would be notified to SPMRs and requires their consent in order for the TC to be
processed. The TC process typically runs on an overnight basis and is necessary to ensure
that the back-office accounting system remains synchronised with the Horizon system.
9.14. Second Sight notes that this method of ultimately adjusting branch cash accounts in
Horizon is similar, but not identical to, what was described by the SPMR, albeit in an
indirect rather than a direct way. We have subsequently been told that none of the POL
employees working in Bracknell in 2008 had access to the back-office accounting system.
9.15. Weare left with a conflict of evidence on this issue and our enquiries are continuing,
particularly in the light of the new information confirming that the meeting on 19 August
2008 did in fact occur! ‘Comment [AP621]: What conflict -the
system in Bracknell had not live access. Of
10.Spot Review SR21 course the transactions looked real, it was
a test environment mirroring Horizon.
10.1. This SPMR reports a situation where, on 4 Nov 2009, the Horizon system appears to
have generated a series of transactions, reversing four Positive Stock Adjustments
(‘SAPs') that she had entered at 09:04 hrs that morning. The SAPs related to 15,576
stamps left over from the previous Christmas.
10.2. The aggregate value of the four SAPs input by the SPMR was £5,577.93.
Subsequently, 9 separate Negative Stock Adjustments (transaction reference: 'SAN')
appear to have been generated automatically by Horizon. Those nine entries total
Page 12
WBD_000004.000014
£6,892.23 which equate to 16,834 stamps. All nine entries were timed at 12:22 hrs and
show the SPMR's Identification Code (i.e. as though she had entered them).
10.3. The SPMR, however, denies executing any of these SAN adjustments. She states
that she was unaware of their existence until long after the Audit of her Branch. She has
no idea whether they had any impact on the shortfall attributed to her.
10.4. We have found no evidence that POL investigated this combined set of transactions
or, if they were investigated, that the findings were ever discussed with the SPMR.
10.5. A POL Auditor on 6th January 2010, after becoming aware of the large quantity of
excess stamps held by this Branch, asked the SPMR:
"Why didn't you declare your stamps?"
10.6. The SPMR states that she told the POL Auditor that she did declare the stamps using
the SAP procedure. It is not clear whether the eventual £9,616.66 shortfall, for which
POL held the SPMR accountable, included the impact of those stamps.
10.7. The SPMR is adamant that she raised this issue with the POL Auditor but states that
she was never provided with any answers. Neither the problem with the stamps, nor the
SPMR's assertions about intermittent problems with the PIN Pad, raised both at the time
of the Audit and in subsequent interviews, seem to have been adequately addressed by
POL's investigators.
10.8. _POL's 3-page response to this Spot Review states that:
a) Horizon does not generate automatic stock adjustments. The function
simply does not exist within Horizon;
b) The stock adjustments questioned in this Spot Review were all recorded
against the SPMR's user ID which demonstrates that those transactions
were manually conducted in the branch;
c) _ Evenif there were erroneous stock adjustments, these adjustments could
not cause the SPMR to suffer a shortfall due to the "double entry”
balancing process inherent in Horizon
10.9. POL's response does suggest a possible explanation as to what happened here,
stating:
"The appearance of positive and negative stock adjustments for stamps made by
SPMRs on the same day reflects a common non-conformance issue in the manner by
which SPMRs inputted data. It led to significant branch conformance instructions in
2009 to encourage branches to record their rationale for why they were using the
stock adjustment function."
and continuing:
Page 13
WBON0000134
WBON0000134
WBD_000004.000015
WBONO0000134
WBON0000134
"Adjustments of the type shown at this branch are indicative of a situation where
branches prefer to sell all varieties of 1st class stamps via the same icon (i.e. whether
the stamps are standard 1st class or special issue commemorative 1st class). Post
Office requires sales via the correct icons to properly drive sales, remuneration and
billing data. However, branches found it easier to serve customers by adjusting stock
out of "Specials" into "Standard" categories and then making sales from those
Standard icons. It is however impossible for Post Office and Fujitsu to say for certain
why the SPMR made stock adjustments in this particular branch.”
10.10. Once again, we are dealing with a conflict of evidence where the SPMR states that
she did not enter the stock adjustments and POL states that the Horizon system could not
have entered them either. POL has, at Second Sight's request, produced the underlying
Horizon detailed transaction data and it will be examined to try to establish what really
did happen.
10.11. In any event, POL did not arrive at agreement with the SPMR as to what had
happened. This failure to arrive at closure has left this SPMR with the powerful and
lasting conviction that her "mysterious £9,616.66 shortfall" was wholly or partially
accounted for by those transactions that she says she did not enter, even though the
system says, on the basis of her User ID, that she did.
10.12. Further contact with this SPMR indicates that she remains confused as to what really
happened so it is possible that the £9,616.66 shortfall was the result of mistakes made by
her or by her staff. Further investigative work is therefore needed and, as yet, Second
Sight cannot reach a firm conclusion on this case.
11.Spot Review SR22
11.1. This SPMR reports a situation where the Camelot and Horizon records for 'Remitted
In’ (or ‘Remmed-In’) Camelot Scratch Cards ('Instants') were out of synchronisation and
were incorrectly shown in Horizon. The SPMR claims that the material differences
between the two systems resulted in substantial losses being incurred and that POL failed
to fully investigate and/or to communicate its findings in respect of those differences.
11.2. As an example of this, the SPMR reports, that on 17th February 2010, the Horizon
print-out of 'Remmed-in' cards shows £1,280 worth of cards (8 full packs) whereas a POL-
produced Excel spreadsheet shows that, on that date, £2,080 worth of cards (13 full
packs) were Remmed in. The difference here is £800, which was a shortfall that the
SPMR had to make good.
11.3. It is clear that this SPMR experienced numerous problems with Scratch Cards and a
review of TCs issued to the branch shows that, between 3rd November 2009 and 29th
September 2010 (the period during which unexplained losses were occurring at the
branch) 36 of the 47 TCs issued to this branch related to Scratch Cards. Also, 13 of those
36 TCs were for amounts exactly divisible by £160 (i.e. the value of a full pack of Scratch
Cards).
Page 14
WBD_000004.000016
WBONO000134
WBONO000134
11.4. Those 13 TCs comprised 4 Debit TCs totalling £2,560 and 9 Credit TCs (which serve to
reduce the branch's stock value) totalling £7,840.
11.5. Together therefore, the 13 TCs produced a net deficiency of £5,280. In pure
monetary terms this was approximately 36% of the total shortfall of £14,842 that POL
claimed, in the ensuing criminal prosecution, had been stolen by the SPMR.
11.6. POL seems to have been aware, well before February 2010, of errors made by many
‘SPMRs in dealing with Scratch Cards. For example, an article in the 17-23 January 2008
Issue of 'Branch Focus' had warned SPMRs that:
“In the last three months there have been over 1,100 Transaction Correction
notices issued to branches to a value of £744,000".
11.7. {In spite of that Corporate level awareness, POL does not seem to have detected the
pattern of repeated errors relating to Scratch Cards occurring at this Branch. -{ Comment [AP622]: WRONG ~ POI did
detect this. It carried outa vist on 11 Feb
2010. This information was in the SR
11.8. _ Having not detected the problem, POL failed to provide the necessary guidance or
re-training needed by the staff at this branch. We have established that during the I Cama Apeas Te RoRES
relevant period, all packs of Scratch Cards should have been activated on the Camelot guidance on remitting in scratchcards was
: - sent with every TC. Ful details ofthis were
terminal before being Remmed-in to Horizon. The SPMR asserts that she was instructed Seserenided brie siraatonsel
not to do that by POL.
11.9. {it also transpires that, ina process change that took place a week after this
particular SPMR was suspended, SPMRs were no longer required to remit packs of
Scratch Cards into Horizon.
24]: Process change
just coincidental
‘Comment [AI
and this SPMR.
11.10. _ It follows, that it is now impossible to have packs of Scratch Cards recorded in
Horizon whilst awaiting activation. It Is also clear that a balance should be struck before
start of trading on a Thursday morning, rather than at 17:30 hrs on a Wednesday evening
as had been the standard practice of this SPMR. POL also appears to have been unaware
of this breach of procedure.
11.11. _ Inits response to this Spot Review, POL says that it cannot find any evidence that
there is a problem with the Horizon system with regard to Remmed-in Scratch Cards.
11.12. POL also states that, during the period being examined in this Spot Review, if SPMRs
had correctly Remmed-in Scratch Cards to the Horizon system, the final figures recorded
in the Horizon system at the end of each day, would match the final figure in the Camelot
system for the activation of Scratch Cards.
11.13. POL's investigation has established that, on 17th February 2010, there were 2
remittance sessions relating to Scratch Cards at this branch. It follows, says POL, that two
receipts would have been automatically produced by the Horizon system. The
discrepancy in the figures on that day resulted from the SPMR presenting only one of the
two receipts. The SPMR, however, disputes POL's assertion, stating that not only did she
Page 15
WBD_000004.000017
WBONO0000134
WBON0000134
not make that second entry in Horizon but that she can't recollect ever Remming-in two
Scratch Cards entries within a 5 minute period.
11.14, POL has also told us that:
"Further to the discovery of large Scratch Card losses at Post Office branches
(for example £147,000 in aggregate losses were discovered following the
audit of 20 branches in and around May 2009), a process change was rolled
out during January and February 2012. This process change was designed to
significantly reduce loss/waste associated with Scratch Cards".
11.15. The SPMR was charged with Theft and False Accounting but the Theft charge was
dropped on the basis that the SPMR pleaded guilty to False Accounting. The SPMR was
convicted on the False Accounting charge and an order made to repay the £14,842, plus
costs of £1,000 and 120 hrs of Community Service. The total of £15,842 was repaid
before the court-assigned deadline.
11.16. _ The key issue here, that seems to have been the root cause of this branch's frequent
Camelot/Horizon differences, was the difference between the opening hours of the shop
and its Post Office Counter. The shop was open from 06:30 hrs until 21:30 hrs from
Monday to Saturday and from 08:00 hrs until 21:30 hrs on Sundays, whereas its Post
Office counter was only open from 09:00 to 17:30 on Monday to Friday and from 09:00
to 12:30 on Saturdays.
11.17. The difference in opening times, particularly on Wednesday’s when balancing
(incorrectly) took place, and at the end of each Trading Period, meant that the shop was
selling Scratch Cards both before, and then long after, its Post Office counter (and
therefore the Horizon system) was able to record them.
11.18. It was perhaps inevitable, in ‘open-all-hours' outlets like this one, that the Horizon
and Camelot systems would be ‘out of sync’ a great deal of the time and [POL should
perhaps have recognised that its standard operating procedure was presenting a real
challenge to this type of retail outlet.
Comment [AP625]: I presume itis the
SPMIRs choice as to when they open and
close the retails
11.19. Second Sight notes that the February 2012 system change eliminated the possibility
of synchronisation errors between the 2 systems, but this software change came a long
time after the problems had become serious.
11.20. The fact that the synchronisation process between the 2 systems is now far better
than it was in 2010 seems to give some support to the SPMR's assertion that the then-
existing process was deficient and that her consequent errors were a material factor in ‘Comment [AP626]: Again, SS have
the confusion that ultimately led to her conviction for False Accounting. teed na ew ons
11.21. Further investigative work is needed to get to the bottom of this complex matter.
Page 16
WBD_000004.000018
WBON0000134
WBONO000134
12.Preliminary Conclusions
12.1. This is an Interim Report and there is much work still to be done. Any conclusions
reached at this point will need to be updated in the light of new information that arises
as the Investigation continues.
12.2. Our preliminary conclusions are:
al
We have so far found no evidence of system wide (systemic) problems with the
Horizon software;
b) We are aware of 2 incidents where defects or ‘bugs’ in the Horizon software gave
rise to 77 branches being affected by incorrect balances or transactions which took
some time to identify and correct;
¢)
(Occasionally, an unusual combination of events such as a power or communications
failure during the processing of a transaction, can give rise a situation where timely,
accurate and complete information about the status of a transaction is not
immediately available to a SPMR;
d)
When individual SPMR’s experience or report problems, POL’s response can appear
to be unhelpful, unsympathetic or simply fail to solve the underlying problem. The
lack of a ‘user forum’ or similar facility means that SPMR’s have little opportunity to
raise issues of concern at an appropriate level within POL;
e)
The lack of an effective ‘outreach’ investigations function within POL, results in POL
failing to identify the root cause of problems and missed opportunities for process
improvements;
f)
The end of Trading Period processes can be problematic for individual SPMRs,
particularly if they are dealing with unresolved Transaction Corrections (‘TCs’). The
lack of a ‘suspense account’ option means that it is difficult for disputed TCs to be
dealt with in a neutral manner.
lan R Henderson CCE, CISA, FCA XX July 2013
Ron Warmington CFE, FCA
Second Sight Support Services Limited
Page 17
a
WBD_000004.000019