POL00000692
POLo0000692
Filed on behalf of the: Defendant
Witness: Steven Paul Parker
Statement No.: First
Exhibits: SPP1
Date Made: 16 November 2018
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
IN THE HIGH COURT OF JUSTICE
QUEEN'S BENCH DIVISION
ROYAL COURTS OF JUSTICE
BETWEEN:
ALAN BATES & OTHERS
Claimant
AND
POST OFFICE LIMITED
Defendant
WITNESS STATEMENT OF STEPHEN PAUL PARKER
I, STEPHEN PAUL PARKER of Lovelace Road, Bracknell, Berkshire RG12 8SN WILL
SAY as follows:
1. I am employed by Fujitsu Services Limited (Fujitsu) as Head of Post Office
Application Support.
2. I am authorised to make this statement on behalf of Post Office Limited (Post
Office), the Defendant in these proceedings, in relation to the Horizon Issues trial
listed for March 2019.
3. The facts set out in this statement are within my own knowledge, or if they are
outside my knowledge, I have explained the source of my information or belief.
4. In this statement I refer to copy documents attached and marked Exhibit SPP1.
BACKGROUND
5. I started working on the Royal Mail Group Account, later called the Post Office
Account, in July 1997, before the introduction of Horizon. I have continued to
provide support to the Post Office Account in my various roles at Fujitsu
throughout the whole of Horizons life.
6. Prior to my work on the Post Office account, I held a number of roles within the IT
industry, including an in-house operations and support role for critical online
systems, support consultancy services and design and development role in an
AC_152871086_1 1
E2/11/1
POL00000692
POLo0000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
End User Technology group in relation to Microsoft tools for internal and external
customers.
7. I began working in July 1997 as a support consultant and deputy manager within
the Software Support Centre (SSC) for the Royal Mail Group Account, providing
third line application support for the Horizon application. Within this role I was the
lead designer and part of development team for the internal website providing
support knowledge (KEL) database, technical documentation management and
operational change control. I also assisted the SSC manager in the provision of
the support service and operational management. Although I did not have the
formal title, I acted as the deputy manager to the SSC as a whole
71 Between December 2009 and March 2010 I was a full time Problem Manager /
Operational Manager of the SSC, responsible for the management of incidents
through the whole support process.
7.2 In March 2010 I became the Manager of the SSC and was responsible for the
provision of third line application support to the Post Office Account, including the
management of the staff working on the account. The SSC subsequently
expanded into a shared service, providing support services to a number of Fujitsu
customers, the largest of which is still the Post Office Horizon system. As head
of this unit I am currently responsible for strategic support, managing 25 — 40
staff.
RICHARD ROLL'S STATEMENT
8. I have been asked to comment on the witness statement of Mr Roll dated 11 July
2016 put forward by the Claimants in relation to the Horizon Issues trial in March
2019. I worked with Mr Roll while he was employed by Fujitsu in the SSC (I
understand from Fujitsu's HR department that this was from 5 March 2001 until 17
September 2004). As explained above, although I did not have the formal title, I
acted as deputy manager while Mr Roll was there.
9. The Horizon system was rolled out across the Post Office network between 1999.
and 2000. In 2010 there was a migration from the system commonly referred to
as “Legacy Horizon" to an online version ("HNG-X" or “Horizon Online").
Therefore, when Mr Roll refers to the Horizon system he is referring to Legacy
Horizon and where I respond to an assertion made by Mr Roll I am also referring
to Legacy Horizon unless otherwise stated. Much of my statement also applies to
Horizon Online, however.
10. I found Mr Roll to be a conscientious worker and provided him with a personal
reference for a position that he applied for after leaving Fujitsu (it was a personal
AC_152871086_1 2
E2/11/2
11.
12.
13.
14,
15.
16.
17.
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
reference because Fujitsu does not allow staff to provide references on behalf of
the company).
In his statement Mr Roll suggests that there were frequent instances of software
problems in Horizon that had an impact on branch transaction data and which
Fujitsu resolved "remotely" (i.e. not in a branch), not merely by changing the
software but also by frequently changing branch transaction data (by injecting
new transaction data and by editing or deleting existing transaction data), without
informing branches that such actions were being taken. As I explain below, those
suggestions are incorrect and Mr Roll's account of Fujitsu's actions and powers is
inaccurate and misleading.
In order to explain why, it is necessary to have a proper understanding of what
Fujitsu was able to do, what the effects of its actions were and the extent to which
such actions had any effect on transaction data in branches. This is described
further below, but I would like to make some general responses immediately.
It is correct that the SSC had (and has) the ability to view data in branches and
other sources such as data centres remotely (in read-only mode): that is to be
expected given their support role which is described in paragraph 26 below.
Post Office did (and does) not have the same ability — what it could (and can) do
is view copies of transaction data as replicated from Horizon to a variety of Post
Office systems. In Horizon Online, facilities have been added which allow Post
Office to remotely examine branch data held in the Branch Database (BRDB) in
read-only mode for a variety of business reasons, such as monitoring levels of
cash that are in branches based on the data entered onto Horizon by branch staff.
It is also correct that issues sometimes arose which necessitated changes to the
Horizon software. However, this was not something that Mr Roll played any
significant part in, as I describe in paragraphs 34 and 35 below.
It was (and is) theoretically possible for there to be a software problem which
could cause a financial impact in branch accounts, but this was (and is) extremely
rare and Horizon's countermeasures were (and are) very likely to pick such
matters up. In my experience, these problems have always represented a very
small proportion of the issues which led to software changes and a very small
proportion of the overall issues dealt with by the SSC.
On the very rare occasion that a software problem which could cause a financial
impact in branch accounts arose, it would be investigated and resolved and
Fujitsu would determine its impact on the Horizon estate and inform Post Office of
any financial impact on branches so that they could be resolved.
AC_152871086_1 3
POL00000692
POLo0000692
E2/11/3
18.
19.
20.
20.1
21.
214
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
In Legacy Horizon it was possible for the data in a particular counter in a branch
to become inconsistent with replicated copies, and Mr Roll appears to be
describing this in paragraph 17 of his statement. In that situation there could be
remote management by Fujitsu to correct the problem, but branch transaction
data was not changed in any way. As explained in paragraph 55 below, the work-
round involved replicating the correct data from another counter in the affected
branch or from the data centre copy.
The suggestion that Fujitsu edited or deleted transaction data is not correct. In
Legacy Horizon it was not possible to delete or edit messages that had been
committed to the message store. See the explanation of my colleague Torstein
Godeseth in paragraph 37 of his witness statement dated 27 September 2018
which reflects my understanding and experience of Horizon's functionality.
In para 18 of his statement, Mr Roll also suggests that in Legacy Horizon he and
others in the SSC could "insert transactions and transfer money without the sub-
postmaster knowing". However:
No Fujitsu personnel have ever had the ability to "transfer money’ out of Horizon
into, for example, an individual's account.
Some members of the SSC were (and some remain) able to insert transaction
data. SSC access privilege gave the ability to inject transactions, but appropriate
change controls were in place and no such insertion would have happened
without complying with those controls.
I should make it clear that, in this witness statement, I am concentrating on what
the support process is designed and able to do and not any malicious misuse of
these facilities. Malicious misuse makes most things possible, as with any other
IT system, however, Horizon had a number of checks and security settings in
place that made it very difficult to carry out malicious misuse. In any event such
misuse would have been discovered by consistency checks or colleagues (all
access was controlled and audited) and would have resulted in instant dismissal.
But even a malicious user would not have been able to “transfer money”.
As there is no way in which money could be taken from a branch and moved
anywhere else (for example into the employee’s bank account), it follows that
there was no motive for anyone to do this. It is not clear to me why anyone at
Fujitsu would insert transaction data into a branch's accounts without there being
a legitimate reason for doing so. Furthermore:
Any transaction that was inserted would immediately cause a discrepancy to
arise in the branch's accounts. For example, if a transaction were to be inserted
AC_152871086_1 4
POL00000692
POLo0000692
E2/11/4
POL00000692
POLo0000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
which stated that £1,000 of stamps had been bought by a customer who paid
cash, that would immediately cause a reduction in stock levels of stamps in that
branch and the branch would have £1,000 less in cash than Horizon expected it
to have.
21.2 In other words, although a transaction could be inserted, it would immediately
become apparent that this had been done and ultimately it would not benefit any
member of staff to behave in this way.
22. It is correct that the "remote access" described above could have been carried out
without the permission of a Subpostmaster. However, any additional transactions
inserted remotely would be identifiable as such from the transaction logs that are
available to Subpostmasters from Horizon.
23. I will now describe the structure of the Horizon support teams during the period
when Mr Roll was employed by Fujitsu before responding to the specific
suggestions that Mr Roll makes.
The Horizon Support Teams.
24. There were four lines of support for Horizon while Mr Roll was employed by
Fujitsu and they are described in paragraph 26 below. There are still four lines of
support for Horizon today, albeit that some names have changed and some
responsibilities have moved around teams.
25. It is common within the industry to have a multi-level support model. Generally,
as you move up through the levels of support the cost of the staff providing the
service increases because they are more qualified. Having said that, there is
often overlap of skills between adjacent lines of support and while a team may be
responsible for a particular level of support, staff within that team can have skills
which allow them to perform a role that is more usually performed by the next
level of support.
26. The four lines of support for Horizon while Mr Roll was employed were as
follows:-
26.1 1" line: The 1" line involved several different elements:
26.1.1 the Horizon Service Desk (HSD)' was a helpdesk operated by Fujitsu
that branches could contact with issues relating to the Horizon
application or the hardware provided in branch by Fujitsu to run the
‘ The HSD has also been known as the Horizon System Helpdesk and the Horizon
Incident Team.
AC_152871086_1 5
E2/11/5
POL00000692
POLo0000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
Horizon application. I estimate that there were around 80 members of
the HSD while Mr Roll was employed, who:-
(a) dealt with straightforward enquiries such as password issues and
scheduling engineers to attend branches to investigate reports of
hardware issues;
(b) monitored the live estate and took corrective actions defined in
knowledge documents (this role was fulfilled by two units: the
System Management Centre (SMC) for data centre events; and
the Counter Eventing Team for Post Office counter events. The
corrective actions did not involve any changes to any branch
transaction data); and
(c) referred other issues to the 2™ line support function.
26.1.2 there was also a 1“ line Communications Management Team operated
by Fujitsu which specifically focused on communication incidents; and
26.1.3 Post Office also operated a 1“ line helpdesk for operational issues
called the National Business Support Centre (NBSC).
If a branch required assistance to attempt to determine the cause of a
discrepancy they would contact NBSC in the first instance. Discrepancies are not
unusual in a retail system. They indicate a difference between the operator's
declaration of cash and stock on hand and the systems calculation and as such
are a business operation issue. However, it was not always possible for NBSC to
identify the cause of a discrepancy. For example, a user may enter a deposit of
£100 into a customer's bank account on Horizon but rather than taking £100 from
the customer, they may make a mistake and give the customer £100 as if it had
been a withdrawal. In that scenario, NBSC would not have been be able to
identify the cause of a discrepancy. Clearly, NBSC is also unable to assist when
losses have been caused by theft.
If NBSC were unable to identify the cause of a discrepancy they would often fall
back on a default statement along the lines of “this looks like a software issue" so
that the SSC would investigate it. However, Mr Roll's statement that "[iJf an error
was referred to us then it was extremely unlikely to be due to a mistake made by
a postmaster’ is not correct. The vast majority of discrepancies investigated by
the SSC as pseudo “software issues” were (and are) not caused by software
issues.
AC_152871086_1 6
E2/11/6
26.2
26.3
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
2" line: 2™ line support was provided by senior members of the HSD and SMC
and junior members of the SSC (as explained below, the SSC also fulfilled a 3
line support function). The 2™ line support function mainly involved staff
searching knowledge articles based on the descriptions of issues reported by
branches, gathering evidence and applying simple, well-defined work-arounds
(often on the phone). An example of this would be resetting passwords.
3" line: the SSC also provided 3" line support. The staff that provided 3” line
support had a detailed knowledge of the Horizon application based on
documentation and some inspection of source code. They:-
26.3.1 designed, tested and documented work rounds for the 4“ and 2™ lines
of support;
26.3.2 applied analytical skills to the symptoms and evidence gathered by the
1*' and 2" line functions and undertook in-depth investigation into
incidents (incidents are the basic unit of work for the support team and
come from helpdesk calls and other Horizon support teams);
26.3.3. undertook complex configuration (configuration items can be used to
alter the behaviour of the application) and data fixes which might have
required the generation of special tooling;
26.3.4 designed, wrote and documented new support tools;
26.3.5 undertook source code examination, complex diagnosis and
documentation (including methods to recreate faults) of new application
problems before sending them to the 4"" line support group for root
cause software fix; and
26.3.6 provided technical support to other internal Fujitsu teams working on
Horizon.”
The 3” line support function used a system called Peak (until 2003 it was called
PINICL) to log and manage incidents passed to them which were suspected to be
faults. It also maintained a Known Error Log (KEL) which describes the
symptoms of problems with some analysis of causes, (potential) solutions to the
problems and workarounds that might be needed before a permanent solution
2 An example of this which applies to Horizon Online is support to the Management
Support Unit (MSU) who are responsible for the Reconciliation Process documented in
SVM/SDM/PRO/0020 (Reconciliation and Incident Management Joint Working
Document). The reconciliation process also applied to Legacy Horizon.
POL00000692
POLo0000692
[SVM/SDM/PRO/0012
F051}
AC_152871086_1 7
E2/11/7
POL00000692
POLo0000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
can be implemented. The Peak and KEL systems are still in use today and are
described further in paragraph 62 onwards below.
26.4 4" fine: 4" line support staff had an intimate knowledge of narrow areas of the
system and were (and are) ultimately responsible for the production of permanent
fixes to repair the root cause of an incident or problem in the live application.
They had knowledge of computer languages which they used to amend source
code to fix problem in the live application code. There was often overlap between
4" line and developers, who added new features into the application.
27. The structure of the support for Horizon is broadly the same today. One of the
main changes is that the HSD function is no longer operated by Fujitsu (it has
been operated by Atos since June 2014).
Further context — call volumes and transfers
28. Between 1 January 2001 and 31 December 2004 (the years Mr Roll worked for
Fujitsu), the SSC received a total of 27,005 calls, meaning that on average 563
calls per month were dealt with over this 4 year period. This is shown by a
spreadsheet prepared by a team in the SSC which appears at Exhibit SPP1 (Tab —_[iF/1839}
“RRP_Live_Peaks_Into_SSC"). Where (as here) I analyse data in this statement,
the analysis is mine.
29. Transferred calls (i.e. those not resolved by the SSC) are of interest. A very small
proportion of calls transferred to 4" Jine support would have concerned software
errors requiring resolution, so it would be interesting to know the number of calls
transferred to them. However, while the SSC have records of the volume of
transferred calls, we do not retain records of where they are transferred to and it
is not the case that all of these would have been transferred to 4" line support.
For example, incidents would often arrive at SSC from internal teams for routing
back to helpdesks.
30. As evidenced by the data in Tab "RRP Live Peaks Out of SSC" of Exhibit SPP1, {F/1839}
an average of 78 calls per month (14%) were transferred to teams outside SSC,
for example, to 2™ and 4" line support. This indicates that only a small volume of
calls received and escalated to the SSC related to software errors requiring
resolution.
31. In terms of calls to the HSD, Fujitsu only holds records of the volume of calls
made to HSD from December 2008 onwards. I understand from my colleague
Sandie Bothick (Fujitsu’s Service Delivery Manager) that the HSD received
13,225 calls in December 2008 and 13,005 in January 2009. The witness
statement of Angele Van Den Bogerd provides that Post Office's NBSC call
AC_152871086_1 8
E2/11/8
32.
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
volume data shows an average of 1,096 calls per day were received in the period
2015 to 2018. While these figures relate to different periods, they indicate that
only a tiny proportion of issues reported to 1% line support helplines were
escalated to SSC.
From the SSC, only a tiny proportion of incidents were escalated to the 4" line
support team. It follows that only a tiny fraction of incidents raised actually needed
to be looked at by the only team who might potentially effect changes in software.
The SSC and Mr Roll's role within it
33.
34.
35.
36.
I agree with Mr Roll's recollection that there were around 30 individuals working
on the 6" floor in Bracknell (i.e. in the SSC) at any one time while he was
employed.
As noted above, the SSC team provided both 2" and 3° line support. As with
any mix of people, there are (and were) various levels of talent within SSC. Mr
Roll was primarily used in Operational Business Change (OBC), which involved
supporting the engineers who were opening and closing branches and increasing
and decreasing the number of counters in branches. Mr Roll would also have
been regularly correcting the application environment after engineers had
replaced failed counter hardware and clearing temporary files to increase disk
space. This could fairly be described as 2" line work and it was done by the SSC
because it required a higher level of access to the system than other support
teams had.
Some members of the 3” line support group identified the need for software fixes
via source code examination and would pass this on to the 4" line team for a
code fix to be written. Mr Roll did not play any significant part in this and was not
involved in any extensive source code examination. An application code fix would
not be written by anyone in the 3" line team and he was not involved in the
provision of 4" line support
I disagree with Mr Roll's suggestion that much of the work being carried out by the
SSC while he was employed could be described “as "fire fighting" coding
problems in the Horizon system." There were times when the SSC was very
busy, for example, networking problems causing application issues across the
whole estate and data centre outages. But there were only rare circumstances
where a coding issue had an estate wide impact and, in those instances, Mr Roll
would have been involved executing avoidance actions to mitigate impact to the
estate (i.e. following established work-arounds) rather than working on the root
cause.
AC_152871086_1 9
POL00000692
POLo0000692
E2/11/9
37.
38.
39.
40.
40.2
40.3
40.4
40.5
40.6
41.
44.4
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
The SSC had (and has) access to view, but not amend, source code. Senior
members of the team would have looked at it from time to time to confirm exactly
how the Horizon application would process a given input and what the outputs
would be when investigating specific issues or general education on how the
system works. However, the access was rarely used. Moreover, Mr Roll was not
working at a level where he would be required to review code. His role in the
SSC was predominantly following work around processes designed by other
people and making configuration changes.
In support of the statements above relating to SSC workload I have undertaken
some analysis of the work carried out by SSC between 1 January 2001 and 31
December 2004 (as stated above Mr Roll was employed from 5 March 2001 until
17 September 2004). My colleague John Simpkins provided the summary data
from the Peak system, which I analysed.
When an incident is resolved, the SSC team member (or technician as they are
sometimes called) types a summary of the incident (known as a Final Response)
and allocates a response code to the incident in order to classify it. While
guidance is provided on when to use each response code (see paragraph 64.5
below), allocation is the subjective view of the technician closing the incident and
there is no re-examination of the response codes later to ensure consistency.
With that in mind, the final response codes that were allocated to incidents (i.e.
Peaks) reported to SSC between 1 January 2010 and 31 December 2004 were
as follows:~
known issue / work around - 35.3%;
admin - 27.5%
reconciliation - 15.7%;
potential user error - 10.9%;
potential software error - 8.3%; and
hardware error - 1.2%.
It should be noted that:-
a major part of 1° line's raison d’étre is to deal with user error and therefore the
percentage of issues attributable to user error would be much higher at 1" line;
AC_152871086_1 10
POL00000692
POLo0000692
2/11/10
41.2
41.3
41.4
42.
42.1
POL00000692
POLo0000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
very few hardware incidents reached the SSC because they were the preserve of
the HSD (i.e. they were relatively easy to spot and therefore filtered out by 4“ ine
support);
8.3% of calls to the SSC (2252) are attributed to potential software errors over
these four years. This includes duplicates and does not provide any clarity on the
significance of the error corrected. Many software errors, particularly in a new
product, were insignificant, such as correcting capitalisation in printed output. I
cannot be more precise without examining each Peak and even then it might not
be possible to determine how a Peak should have been properly classified
(Peaks are essentially notes made by Fujitsu personnel to chart the progress
made in resolving an issue and these notes can vary in fullness and clarity); and
Classifying an incident as a “potential software error does not necessarily mean
that there was a software error and, even if there was, it does not mean that the
error was one that could have caused a financial impact in a branch's accounts (a
large proportion of these would be errors in numerous data centre resident
systems that the Subpostmaster never sees — errors were often as trivial as the
use of “Kg” instead of “kg” on receipts). As stated in paragraph 16 above, such
errors were extremely rare. They were all resolved (resolutions include a source
code fix, a configuration fix or a workaround).
Mr Roll's suggestion in paragraph 10 that software issues in Horizon "routinely"
caused discrepancies in branch accounts is misleading. In the vast majority of
cases such an occurrence would cause a receipts and payments (R&P) mismatch
that would be flagged by the branch system as part of the balancing process (the
Horizon system carries out self-consistency checks which generate alerts in the
event of a receipts and payments mismatch that are picked up by SMC and
incidents raised for the SSC) and appear on MSU reporting. These would then
be investigated and resolved by the SSC.
Since the introduction of Horizon in 1999 there have been 735 live incidents
which refer to “Payments and Receipt mismatch” (i.e. incidents recorded against
components of the system providing Horizon service to Post office rather than
incidents raised against test systems). This figure has been obtained using a
textual search across all incidents where the title or one of the incident updates
contains all of the words “receipts”, “payments”, “mismatch”. It should be noted
that this is not 735 unique incidents; there will be a lot of duplicates with the same
root cause. The only way to determine how many unique incidents there were
would be to manually review all of the incidents. All of them were resolved.
AC_152871086_1 "1
2/11/11
42.2
42.3
43.
44.
45.
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
These incidents are reported as a result of self-consistency checks carried out by
Horizon. It should be noted that a R&P mismatch is not only caused by a
software error. It can also be caused by incorrect product reference
(configuration) data
Receipts and payments mismatches happened more often during the early life of
Horizon (see Tab “All RnP by RCode and Date” of Exhibit SPP1). My analysis of
that data shows that there were around 8.6 such incidents per month on average
between 1 January 2001 and 31 December 2004 (417 out of a total of 27,005
incidents into SSC or 1.5% of SSC incidents during that period).
Mr Roll refers to a "perception...that the Service Level Agreements between Post
Office Ltd and Fujitsu involved financial penalties payable by Fujitsu to Post
Office" (paragraph 12). I am aware that there were Service Level Agreements for
issues such as stuck transactions (Fujitsu had 10 days to retrieve transactions
that had not replicated from a counter). It is quite normal for contracts such as
the one between Fujitsu and Post Office relating to Horizon to have such
agreements. The same level of diligence was (and is) applied to all incidents,
whether an SLA was relevant of not. The possibility of financial penalties was
never a factor for the SSC.
I do not understand what Mr Roll means when he says that “any discrepancy in
the post office accounts had to be resolved speedily" (paragraph 12). There was
(and is) a process run by the Management Support Unit (MSU) which involves
examination of various system reporting and may result in Business Incident
Management Service (BIMS) entries going to Post Office. An incident may also
be raised by MSU with the SSC to provide support to the MSU in resolution of the
BIMS. These are subject to Service Level Agreements and Mr Roll may be
referring to this process. However, if Mr Roll is suggesting that Fujitsu routinely
rushed out fixes or workarounds due to SLA time pressure, that is not the case.
Fixes would be expedited based on service impact. It would be quite wrong to
suggest that they were not done properly because of any SLAs.
It is correct that there are a limited number of opportunities to release software
updates and that these releases could take 6 weeks or more to be released to the
live service (Mr Roll's statement, paragraph 13). These longer timescales would
be employed for non-urgent updates wrapped up into a consistent set for
deployment. On those rare occasions when a problem has an impact on the
financial integrity of the system a "hot fix" would be deployed which involved
much shorter timescales. I would expect a timescale measured in days not weeks.
to deploy a hot-fix. Mr Roll also states that a bug could reappear several weeks
AC_152871086_1 12
POL00000692
POLo0000692
{F/1839}
2/11/12
46.
47.
47.2
47.3
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
after a coding fix had been released due to software issues. I am aware of only
one or two cases where a fix regressed in my time at Fujitsu.
In paragraph 14 Mr Roll states, "/ would reiterate that the main recurring issues
were software issues." It is a symptom of working within a software support team
that the majority of issues that come in have been attributed to a software issue
by, for example, a lower line of support. This can lead to a mind set of “look at all
these Horizon errors”, but what this indicates to me is that the previous levels of
support are functioning correctly, removing the majority of other causes (user /
hardware problems). It does not indicate that the majority of Horizon errors could
be attributed to software.
Mr Roll states (paragraph 7) that "[sJoftware programs were written by us to strip-
out irrelevant data, to enable us to more easily locate the error." The support
tools are used to filter information and present information to technicians in ways
that make the support process easier. I am aware of two support tools (also
known as software programs) that were written while Mr Roll was employed by
Fujitsu:-
the Smiley support tool written by my colleague John Simpkins, which
amalgamates information from various sources (e.g. databases) into a single
view pertinent to a particular support task and provides a unified interface to run
various tools to achieve a single support outcome; and
another tool (I cannot remember its name) written by my colleague Richard
Coleman whose function was to retrieve messages (i.e. data) from the
correspondence server to local text files for examination and which was
eventually subsumed into the Smiley support tool.
Neither of these tools changed the underlying data in a branch's accounts.
The work carried out by Mr Roll
48.
48.1
48.2
Mr Roll states that he would investigate financial discrepancies that had arisen in
branches by “workfing] sequentially through all transactions over the relevant
period, and also work{ing] through thousands of lines of computer coding"
(paragraph 7).
It is not the role of the SSC to routinely investigate discrepancies.
In very rare circumstances a discrepancy could be caused by a software issue
and in those circumstances it might be necessary for the 4" line support function
to work through thousands of lines of computer coding to determine the root
AC_152871086_1 13
POL00000692
POLo0000692
2/11/13
49.
50.
51.
2.
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
cause. However, Mr Roll would not have worked through thousands of lines of
computer coding to investigate a discrepancy in a branch: he did not work in the
4" line.
Mr Roll further states that if SSC was “unable to find the cause of the discrepancy
then this was reported up the chain and it was assumed that the postmaster was
to blame" (paragraph 10). That is not my experience: it is a simple truth of support
that the majority of issues reported in a system are attributable to user action or
user misunderstanding of system functionality. Hence, someone working in a
support environment analysing a new issue would examine the possibilities of
user error as a first hypothesis but any final conclusion is only generated based
on the evidence. Where the evidence does not support a conclusion that there is
a problem with Horizon, the SSC feeds the existent factual data back to Post
Office and might say something along the lines of "ail indications are that the
branch has made a mistake", but Fujitsu neither attributes "blame" or agrees the
final conclusion with the Postmaster
When an incident is resolved, the SSC team member (or technician as they are
sometimes called) types a summary of the incident (known as a Final Response)
and allocates a response code to the incident in order to classify it. While
guidance provided on when to use each response code, allocation is the
subjective view of the technician closing the incident and there is no re-
examination of the response codes later to ensure consistency.
The Peaks that Mr Roll works on while employed by Fujitsu indicate that he dealt
with 915 incidents (see Tab "Final Responses" of Exhibit SPP1). To give some
clarity on these incidents I have analysed their final response codes that were
allocated to them by Mr Roll. He classified them as follows:-
Response code % allocated
by Mr Roll
Known issue / work around 61.9
Reconciliation 14.5
Admin 9.3
Potential software error 3.2
This supports my recollection that Mr Roll mainly followed work-arounds devised
by other people (61.9%) and that he was rarely involved in the detailed
examination of potential software errors (3.2%). As explained in paragraph 41.4
above ‘potential’ software errors do not necessarily mean software errors, let
alone software errors resulting in discrepancies to branch accounts.
AC_152871086_1 14
POLO0000692
POLo0000692
(F/1839}
2/11/14
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
Remote access
53. I understand from Post Office's solicitors that the term "remote access" has been
used to describe Fujitsu and Post Office carrying out the following actions when
not physically present in a branch:-
53.1. accessing branch data in read-only mode;
53.2 _ inserting new transaction data; and
53.3. editing or deleting existing transaction data.
54. As noted above, it is correct that Fujitsu had (and has) the ability to view data in
branches remotely given their support role.
55. Mr Roll claims that "/djuring the course of resolving the software issues, we would
frequently access a Post Office counter IT system remotely" (paragraph 15). Mr
Roll gives the example of ("when a binary bit would "flip", thus a "1" became a
"0m):=
55.1. This probably relates to a condition known as a CRC (Cyclic Redundancy Check)
error which would happen when a hard drive became faulty at a branch counter.
Although this is a hardware problem, remedial action was needed by the SSC to
resolve. To clarify, this process did not involve changing any transaction data.
55.2 As explained by Mr Godeseth in paragraph 35 of his statement, in Legacy
Horizon:-
55.2.1 all counter data was held in a bespoke message store in each branch*
and the data was replicated to all counter positions in the branch and
from the branch to the data centres where it was held in
correspondence server message stores;
55.2.2 any data inserted into the message store at the data centre (for
example reference data or authorisations for banking transactions)
would be replicated back to the branch counters; and
55.2.3 selected data was extracted from the correspondence servers to
update Post Office's back end systems.
55.3 If one of the sets of data on a branch counter became corrupted it would
generate an event that would be picked up by the SMC and/or reported to HSD
3A message means data and transaction data is a subset of the data in the message
store.
AC_152871086_1 15
POL00000692
POLo0000692
2/11/15
55.4
56.
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
by the branch (an incident reporting a “CRC Error”). There were a total of 629
CRC errors over the life of Legacy Horizon (see Tab "All CRC by Date" of Exhibit
SPP1).
The issue would be reported to the SSC, who would delete the entire set of data
on that counter and replace it with a copy of the data from one of the other
sources that had not been corrupted. While this process involves deleting and
replacing a set of data, no new data is produced; all that happens is that the
replicated data is used to replace the data that has become corrupted from
another counter in the branch. It would have been necessary for the SSC to
inform a branch before carrying out this task because it is likely that any attempt
to use that counter would fail while the process was being carried out. In Mr Roll’s
capacity as an OBC specialist, he would have been involved in this type of
activity.
Alternatively, Mr Roll's reference to a binary code flipping may relate to a
configuration item (an item that can change the behaviour of the application)
which can become locked in the wrong binary setting (1, 0). One example of
which would be a stock unit lock which, in the wrong state, would prevent
updates to stock units within a branch. This issue was corrected by a member of
the SSC accessing Horizon remotely, but it did not involve accessing or editing
transaction data in any way or re-creating databases.
I cannot think of any other examples of incidents that Mr Roll may be referring to
in paragraph 15.
Mr Roll claims that "some errors were corrected remotely without the sub-
postmaster being aware" (paragraph 16).
It may be that Mr Roll is referring to issues relating to the end of day concept in
Legacy Horizon. Essentially there was a cut-off point for transactions every day
at 7:00pm and each counter had to write an end of day message to the branch's
master counter to enable the master counter to write a branch end of day
message, which would then trigger the data centre to harvest messages
(including details of transactions) to Post Office's back end systems.
Occasionally a counter in a branch would fail to write an end of day message and
there was a process for correcting this. The issue would be reported to SSC by
way of an incident (either as a result of a call to HSD or sometimes Fujitsu could
spot issues via system events).
AC_152871086_1 16
POLO0000692
POLo0000692
F/1839}
2/11/16
56.2
POL00000692
POLo0000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
In lay terms, SSC would force the counter to generate a report based on the data
already in the counter. A message injected in this way would go into the audit
trail. This would not alter the branch's transaction data.
57. Mr Roll also states that, there were some errors where it was necessary to
“download and correct the data and prepare it for uploading back on to the post
office computer, then call the postmaster to inform him that there was problem
and that we needed two or three minutes to correct it" (paragraph 17).
57.1 It is not clear what errors Mr Roll is referring to or how he says they
were corrected. The issue referred to could be another instance
where the work round of re-creating transaction data from a
replicated copy was required as described in paragraphs 55.3 and
55.4 above.
SUMMARY
58. In summary, the suggestion that Fujitsu would manipulate a branch's transaction
data in a way which was detrimental to a particular Postmaster and undetectable
Is wrong.
58.1 All support action taken by Fujitsu is directed to ensuring that legitimate
transactions entered by Subpostmasters are correctly processed by the Horizon
application and correctly passed to other POL systems as appropriate.
58.2. Any financial corrections required are communicated by Fujitsu to Post Office for
execution or approval.
58.3. Any support intervention that requires the insertion of a transaction is identifiable
in the audit trail.
58.4 There is no financial incentive for a support technician to circumvent the controls
within the system.
58.5 There are strong controls in place to prevent intervention by support staff with
malicious intent or misguided attempts at financial gain.
59. The statement that issues with coding in the Horizon system were extensive and
impacted branch finances is incorrect for the reasons stated above.
KELS AND PEAKS
60. I have been asked by Post Office's solicitors to provide some more information on
the two toolsets used to support the Horizon system, the KEL knowledge base
AC_152871086_1 17
2/11/17
61.
61.1
61.2
61.3
61.4
61.5
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
(AKA KEL) and the incident management system (PINICL / Peak). This
information is applicable to both Legacy Horizon and Horizon Online except
where explicitly stated otherwise.
Fujitsu use a custom solution, developed and administrated by the SSC, which
allows us to record support knowledge into a structure known as a KEL (Known
Error Log). KELs record support knowledge which is intended to assist staff in
the support and understanding of the Horizon system. KELs do not contain the
history of an incident (see my analysis of the Peak system below). KELs are
generated for a number of reasons, for example: to explain system behaviour or
messages that originate from central and counter systems; to record symptoms
and outcomes from incidents referred by help desks or identified as a result of
Fujitsu's reconciliation processes; and to record information on issues seen in test
environments (resolved before the feature is passed on to users).
The acronym KEL is a misnomer inherited from a previous system. KEL entries
are support knowledge entries and do not have a one to one relationship with
errors on the system. There are a lot more general supporting knowledge KELs
than KELs relating to specific errors.
Guidelines for the generation and use of KELs are documented in
SVM/SDM/PRO/0875 (End to End Support Strategy) section 11 Knowledge Base
Maintenance. Although some aspects of this document need revision due to
changes in the support structure for Horizon, Section 11 is still fundamentally
correct in relation to Horizon.
KELs reflect community sourced knowledge to assist staff involved in the support
of the Horizon solution. There are no mandated rules for when a KEL should be
created other than a desire to make resolving a problem easier for all concerned.
Guidelines exist in SVM/SDM/PRO/0875. Any KELs created or updated are
referred to the SSC for approval to provide a basic check that the information at
the time of the change is valid.
KELs can be created by the senior people in HSD (2 line) to supplement their
own knowledge base (rather than taking an active role in the KEL process) and
the SMC monitoring team. The 1“' line helpdesk function do not create KELs
(HSD 1" line used their own knowledge base).
AKEL is a living document that reflects support knowledge at a given point in
time. KELs are not designed to provide a history of a particular symptom or
support process and a particular KEL cannot be considered the definitive source
containing all possible information regarding the problem it addresses. It is up to
AC_152871086_1 18
POL00000692
POLo0000692
2/11/18
61.6
61.7
61.8
61.9
61.10
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
the technician to check other potentially relevant KELs and information sources
(e.g. support guides) and use their analytical skills to determine the right course
of action to take in a given situation.
There have been historic requests to remove large numbers of KELs based on
date updated to reduce the number of search results that are returned to help
desks when naive search terms were used. The dates given in the KELs, while
an indicator of potentially irrelevant support information, are not precise. This led
to the concept of “deactivated” KELs (deactivated KELs are removed from the
default searches support people use although the user interface allows the user
to explicitly request a search to include deactivated KELs). At the time of writing,
there are 113 deactivated Legacy Horizon KELs and 1024 deactivated Horizon
Online KELs.
For most of its lifetime, there has been no fixed routine for the review of KELs. If
a technician recognises an inaccurate KEL as they analyse information they are
expected to update in order to improve the knowledge base. Approximately 2
years ago the KEL review forum was introduced. This forum meets weekly to
review KELs and update or deactivate as appropriate.
Before creating a KEL there is an expectation that the creator will search the
existing information to ensure that it is not duplicated. Because people express
information using different words, it is not possible for the system to perform such
a check. Human fallibility and unique expression mean that duplicated
information is present in the KEL system.
Archiving: There is no requirement to keep historic support information. Once an
item is no longer relevant to current systems it can be removed without any
implications to the support of the system. KELs are deleted when they have no
value to the support of the current systems. This can happen at any time and is
carried out by the SSC who are the final arbiters of what information is currently
relevant. That does not mean that all current KELs are still relevant; it may be
that irrelevant KELs have not been deleted yet. At the time of writing there are
1,491 deleted KELs.
KEL entries have a single field to record an incident (Peak) reference. This is not
a record of all incidents the KEL is relevant to. Generally, it is used to record the
Peak that was being investigated when the KEL was created or updated. This is
a manual entry and is not checked by the system because a KEL may not have
an associated incident. There is no requirement to update the KEL when
information in it is re-used to provide guidance on a different incident.
AC_152871086_1 19
POL00000692
POLo0000692
2/11/19
62.
62.1
62.2
62.3
62.4
62.5
POL00000692
POLo0000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
Peak (not an acronym) is browser-based software incident and problem
management system used by Post Office Account for all development, test and
support teams except the 1“ line help desk. It enables details of the incident and
diagnostic progress to be captured in a searchable format and allows the tracking
of problems from detection through to resolution. Peak was developed in-house
by the SSC from the PinICL system it replaced in 2003. The system has been
customised and enhanced over its 15 years of operation and still continues to be
developed to Post Office Account requirements today.
When Peak was implemented, data from PINICL was migrated to Peak.
One source of Peak incidents are the 1° line support teams (including the HSD
the NBSC helplines). Peak is also used to process incidents generated by other
support units, monitoring and testing teams.
The structure of a Peak enforces a workflow (it gives a process structure leading
to a defined outcome). As a result, the Peak system has also been used to
record and progress other items loosely associated with incident management.
For example, Release management process, reference data delivery process,
Post Office Data Gateway route definition process. So the Peak system contains
incidents which do not directly impact the Horizon service provided to the Post
Office counters (AKA “Live service”). These additional types of “incident” are
differentiated by the Peak type: L = Live service, R = Release management etc
For most of its life cycle a Peak is assigned to a particular support team and a
person within that team who is responsible for the next action on the incident. As
the incident is progressed by various members of the support community, they
add textual comments and supporting evidence to the Peak to document the
progress of the incident. These updates are date / time stamped and form a
record of the diagnostic and resolution process. Progress updates cannot be
deleted / amended by users once committed to Peak. A Peak may also be
transferred between teams and people as it progresses to final resolution.
A final response code (numeric) is applied to an incident when it has reached its
conclusion along with text that supports the response code. Response codes are
the subjective opinion of the person closing the incident and are not subject to
review but they remain the best way to classify the final outcome of an incident.
Response codes and their expected usage are documented in
SVM/SDM/PRO/0875 (End to End Support Strategy) in Section 9 Incident closure
categories. Although some aspects of this document need revision due to
changes in the support structure for Horizon, Section 9 is still fundamentally
correct and is relevant to Horizon (Legacy and Online).
AC_152871086_1 20
2/11/20
62.6
62.7
62.8
62.9
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
The Peak system has no archiving policy. Effective incident management
requires that you can track the current issues and those from the recent past.
Data retention on Peak has been impacted by a lack of disk space, the primary
cause being large evidence files attached to Peaks. I am told by my colleague
who maintains the Peak system that:-
62.6.1 encrypted evidence files are removed one month after the incident is
closed; and
62.6.2 due to disk space issues in the past it has been necessary to remove
evidence attached to older Peaks.
KEL references can be added to a Peak entry. There is no system check to
ensure a KEL has been added since KELs are not relevant to all incidents being
processed.
Incident priorities and appropriate usage are described in SVM/SDM/PRO/0875
(End to End Support Strategy) in Section 7 and SVM/SDM/PRO/0018 (Incident
Management Process). Incidents with a financial impact on branches are treated
as high priority.
Target times to resolve software incidents are described in SVM/SDM/PRO/0875,
(End to End Support Strategy) in Section 8.
KEL ANALYSIS
MR COYNE'S KELS
63.
64.
65.
I have been shown paragraph 5.114 of Mr Coyne’s report, in which he says that
he has analysed 5,114 KELs to determine the scope and impact of potential
Peaks. Mr Coyne explains that out of these 5,114 KELs, he believes he has found
163 that could be of "significant interest" and that he has referred to 76 of these in
his report.
Post Office's solicitors have reviewed Mr Coyne's report and have provided me
with a list of KELs that they have identified as being referred to in the report
(Coyne KELs). I do not know why there appears to be a difference between this
number and the number of 76 quoted by Mr Coyne.
It is not clear what Mr Coyne means by “significant interest’. \t may be that he
means that a KEL presents evidence of a bug, error or defect in Horizon that has
AC_152871086_1 21
POL00000692
POLo0000692
2/11/21
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
caused a financial discrepancy in branch accounts. I refer to this as “financial
impact" as shorthand in this statement.
66. KELs are written by and for members of Fujitsu’s support groups (i.e. all of the
teams who support the Horizon solution for Fujitsu) who have a deep knowledge
of the design and operation of Horizon, and they are often expressed in a
shorthand way. I believe that it would be helpful to explain the significance and
implications of the Coyne KELs. Annexed to this statement is a table which
contains the initial explanations that have been produced by a team from SSC at
my request in the time available.
DR WORDEN'S KELS
67. Dr Worden has selected a sample of 48 KELs. A list of these KELs was passed
to Fujitsu by Post Office's solicitors (the Worden KELs).
68. A table which explains the Worden KELs is annexed to this statement. As with
the Coyne KELs, given the limited time available to prepare this statement, the
initial explanations have been produced by the team referred to in paragraph 66
above.
STATEMENT OF TRUTH
I believe that the facts stated(in Ibis witness. statement are true.
stones GRO
Date: LA
AC_152871086_1 22
POL00000692
POL00000692
2/11/22
ez/bza
Appendix 4
POL00000692
POL00000692
Filed on behalf of the: Defendant
Witness: Steven Paul Parker
Statement No.: First
Exhibits: SPP1
Date Made: 16 November 2018
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
T I wrightm Payments 56-511 I This issue is reporled as causing I This issue, referred to as the ‘Payment Mismatch’ by Mr Coyne, is dealt I The issue caused a receipts and
33145J Mismatch discrepancies showing at the I with substantively in the second witness statement of Torstein Olav I payments mismatch in the accounts
(F71450) Horizon counter which disappeared I Godeseth of affected branches, which was
when branches followed certain detected by the monitoring of system
process steps. While a workaround
was established by KEL wright
331454, it is not clear how many
corrections were required to fix all
instances of this or event that all
instances were fixed.
events by Fujitsu. Post Office were
informed and corrected the relevant
branch accounts.
AC_152871086_1
ve/Liza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
2. I achai233J I” Unexplained 5.21-5.23 I Discrepancies between the branch I The cash planning system calculates the branch's cash levels figures from I No impact.
Discrepancies
(cash
declarations)
cash declarations and the amount
by SAP (the cash
management system). The KEL
states that this is not a user error or
received
anything that can be corrected at
This is
consistent with the problem being
branch level. therefore
due to the existence of a software
bug
one day to the next. This was a timing issue due to figures from a previous
day being used in association with other figures from the current day.
This issue affected the figures being used by the back end cash planning
system and did not affect any branch accounts. The KEL exists to deal
with further complaints
In the event that there is a discrepancy between the amount received by
the Post Office cash centre in a cash pouch returned by a branch and the
amount the branch entered on Horizon as having been put in, this would
be identified by Post Office and a Transaction Correction for the
appropriate amount would be issued.
AC_152871086_1
S@/LWea
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL
Mr Coyne's Report
Fujitsu's Comments
ID
Short-name
(if applicable)
Paragraph
No.
Mr Coyne's Summary
(paraphrased)
Response to Mr Coyne
Financial impact on branch
accounts
o
achat7i7T
{F/1303}
Unexplained
Discrepancies
(cash
declarations)
5.22
‘There is an acknowledgment that
cash declaration discrepancies
could be due to an "Unknown
System Problem’
By way of context, this is a generic KEL relating to the handling of calls
concerning discrepancies. There are a number of possible causes for
unexplained discrepancies between the cash declaration and figures on
SAP, including:
+ Subpostmaster has made an incorrect declaration
* Transactions as recorded on the system do not match what actually
happened at the branch
* Outstanding recovery
* Withdrawn products
+ Known system problem (these should be monitored for or be easy to
spot from events ete.)
* Unknown system problem
This term "unknown system problem’ is a term that the KEL creator would
have used to indicate that the issue may have been caused by a new
(previously undetected) defect, that would follow the normal diagnosis
process and will then become a known system problem once fixed.
The relevant Peak (PC0202239) relates to an investigation surrounding
the possible explanation of a £240 discrepancy. The investigation
concluded that the discrepancy was likely to have been caused by human
error and it appears that the Postmaster accepted this conclusion.
None.
AC_152871086_1
9t/LZa
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
a acha62iP Unexplained 523 Evidence of cash declaration This issue, also referred to as the Dalmellington’ issue is dealt with This issue caused a discrepancy in
Discrepancies
(duplicate Rem
In)
discrepancies arising from clerks
duplicating Rem In transactions as
a result of wrong messages being
presented on the Horizon counter
screen.
‘substantively in the second witness statement of Torstein Olav Godeseth.
Fujitsu identified the branches affected by this issue and gave the
information to Post Office to resolve the issue with affected branches.
the Subpostmaster's outreach branch
which was easy to identify from the
transaction logs available through
Horizon and the fact that separate
receipts were printed for each
transaction. Post Office issued
Transaction Corrections or advised
Subpostmasters how to take
corrective action to remove the
discrepancies.
AC_152871086_1
Leeda
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
5. I LKiang3014S_I~ Unexplained 52a Issue following multiple cash This KEL relates to the next KEL (MScardifield22198) as follows: This issue may have had a
Discrepancies declarations and trial balance report temporary financial impact, but it
(multiple cash being inaccurate. On this occasion * LKiang3014S describes the Symptoms of the issue; and would have been resolved when the
declarations) the support department was unable + MScardifield describes the cause of the issue branch next declared the cash held in
to identify the root cause of the
discrepancy although it was
reported that a correction could be
made at the Post Office counter
level by redoing the cash
declaration using the same amount
already declared. Within KEL
MScardifield 2219S Fujitsu identify
the underlying software bug as
being caused by ‘cached data’ not
being updated via Riposte
This was a software/ environmental issue involving the Horizon system
struggling to find cash declarations, which would tend to happen if a
Subpostmaster was undertaking multiple declarations in a stock unit
This particular instance involved Riposte failing to find one of the Cash
Declarations and thus generating a temporary discrepancy. If left
unresolved this would result in a loss to the Postmaster for this period
However, when the Subpostmaster subsequently declared the cash held
in branch accurately, this would generate an equal and opposite
discrepancy that would cancel out the earlier discrepancy.
This issue was not resolved, but further diagnostics were added to enable
further investigation should the problem happen again.
the branch accurately.
AC_152871086_1
82/L za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
6. I MScardifield2 I~ Cached Data 52a Delay in ‘cached data’ being This KEL relates to the previous KEL (LKiang30148) as follows: As above.
2198 Delay updated via Riposte. This resulted
in incorrect data being presented in
any discrepancy, variance and
balance reports.
Mr Coyne comments that this is
likely to be confusing for the
Postmaster and could lead them to.
making unnecessary modifications if
they are unaware that the problem
should clear itself overnight.
+ LKiang3014S describes the symptoms of the issue; and
* MScardifield describes the cause of the issue.
This is not a'Live' issue, but something found on a Test rig. Experiments
did find that this could occur around once in 100,000 balance calls when
the rig was heavily loaded
13 Live Peaks reference this KEL, each Peak was treated separately and
the issue was discussed with the Subpostmasters. The work around of
subsequently correctly declaring cash declarations was identified and
communicated to Subpostmasters.
AC_152871086_1
62/LL/24a
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
7. I DSeddon542_I~ Cash Pouch 5.25 Rfailure in pouch delivery resulted I The cause of this was that the system was in “Price shopping mode” This had a financial impact, but when
6p Delivery in acash gain when the branch which can be used to buy, for example, £15 of ‘st class stamps. Price the branch investigated the
(F/356} declaration was carried out. shopping is not supported for products like cash, so when the branch discrepancy the failed remittance
attempted to remit the cash deliver in while in Price shopping mode the
cash remittance failed.
In this particular KEL, the remittance was repeated later and was
successful. A fix to ignore “Price shopping mode” for Rems was applied to
Live in April 2007.
would have been identifiable in the
logs that are available to users of
Horizon.
In addition:-
* — acritical event was written
that will have been picked
up by Fujitsu's support
teams and Fujitsu will have
advised Post Office to
contact the branch; and
. Post Office's own
reconciliation procedures
would have identified that
the remittance in was not
completed successfully
and contacted the branch
and resolved the matter
with a Transaction
Correction
AC_152871086_1
Og/L za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
8. I achatoaL Katomated 5.26 Problem affecting around 15% of I Automated cash declarations from self service kiosks failed at random. No impact on branch accounts.
Cash kiosk branches that prevented these I Analysis shows that this was due to rounding errors on the arithmetic
Declarations branches from being able to cartied out due to use of incorrect data types. The business impact of the error is
automatically make cash ‘on cash management and the
declarations. The lack of cash declarations have no impact on Branch accounts, but will I delivery of cash meaning the cash
result in cash planning potentially having incorrect or missing figures and I needed at a branch may be
thus failing to calculate accurate amounts of cash to send to the branch. _I incorrectly estimated, leading to late
or insufficient cash deliveries.
This was a coding error and was fixed in December 2015.
The vast majority of self service
kiosks are located in Crown branches
and the others are located in large
mains branches
9.I DSeddon3i4 I Reference Data 5.30 Examples have been found in KEL’ I The particular KEL related to the incorrect maximum values being set up I No impact - it was not possible to
Q Peak records that indicate in Reference Data and the counter not handling this error correctly. This __I rem in the coin and therefore it could
Reference Data has an impact upon I was caused by a human error by Post Office in the Reference Data. not be sold by branches.
daily counter activities Specifically, it was not possible to rem in a particular commemorative coin
because it did not have the requisite Reference Data.
The Reference Data was fixed on 14/3/06 and a code fix to handle the
scenario better was applied in June 2006. This did affect a number of
branches during the day that the problem was live, but a message was
sent out to all branches advising them of the issue and how to correct it.
AC_152871086_1 8
Le/bWza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
70! johnbascoGS 531 ‘An Automated Payment transaction I For some unknown reason, the counter was associating an Automated No impact.
222H was reported as having failed due to I Payment transaction with an invalid client code. There was no fault in the
“Unknown Agent Code 3046”, Client I Reference Data. The fault was impossible to reproduce, and so not
account code 3046 was found to not I understood,
exist in Reference Data and the
fault was not reproducible when the I Failure to complete a transaction would not produce an error in accounts -
problem was analysed and tested. It I a double entry transaction would either all succeed or all fail
was acknowledged that, due to the
business impact, a fix would be
provided to check and validate the
client account code exists in
Reference Data before the
transaction is committed
Ti] acha0L__I Reference Data 5.32 KEL documents how branches were I Incorrect Reference Data was issued, which had the effect that payments I No impact.
Errors unable to accept cards for rent and I of Council Tax to Vale of Glamorgan Council were rejected. The
council tax payments due to Reference Data was fixed overnight and it all worked the next day. Failure
incorrect Reference Data to complete a transaction would not produce an error in accounts - a
double entry transaction would either all succeed or all fal.
AC_152871086_1
ceiLiza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
72) MWrighti458 I Withdrawn 5.33 Tssue involving withdrawn products I When products are withdrawn, there is a withdrawal process which Temporary financial impact caused
Q Products impacting a branch accounting requires users to have Remmed Out the relevant products (which involved I by the Subpostmaster’s failure to
{F/419} position because the Postmaster _I the stock being returned to the stock centres) or the stock would need to I follow the correct procedure in
will have products that cannot be _I be adjusted. The grace period for the Subpostmaster either Remming Out I relation to the Remming Out of
accounted for as there is no or adjusting the stock was 6 weeks. After the Reference Data was obsolete stock
remaining Reference Data for them I withdrawn, any unused stock would result in a discrepancy (a change was
to later declare that stock item in the I introduced by Impact in 205 which meant that this could no longer occur). I Fujitsu was able to resolve the issue
accounts. The assumption was that the branch had actually sold the missing stock I by adding a ~—ccompensating
and failed to record it on Horizon so the stock was removed from the transaction
system and a cash discrepancy would be generated for its value.
In this particular case, the branch failed to Rem Out its unused stock of
obsolete products (stamps) within the grace period despite being asked to
do so on multiple occasions. This left the Subpostmaster with a problem
she could not solve herself, but needed Post Office support to do so. To
resolve the issue, corrective transactions were added to the Message
store in consultation with Post Office at the data centre to reflect the value
of the obsolete stock which enabled the branch to roll over correctly.
AC_152871086_1 10
ee/Lza
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
73] wbraS363J__I Reference Data 5.4 ‘The customer was charged twice for I This was a fault involving a self-service Kiosk, which resulted in a I No impact.
Errors the same transaction which was customer being debited three times, after which the session was
{F/1208} reported to be a side effect of errors I cancelled.
within Reference Data.
This was an issue with how the NGR kiosk interfaced to Horizon, which
attempted multiple Debit Card transactions with the same ID. NCR
diagnosed this as being due to some invalid Reference Data being sent to
the kiosk
The fault appears to result from two causes:
i. faulty Reference Data (the result of human error), which was
easily corrected
ii. a fault in the kiosk software, which came from an external
supplier
AC_152871086_1
velLiza
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
74 GMaxwell365 Duplicate 5.38 Failures or interruptions in service I This was an issue where approximately 835 transactions were sent to I No impact.
1K Payment with the harvesting process can Streamline (Post Office's previous payment services provider) on 2
Transactions cause duplicate payment successive days due to an operational issue in Horizon back end
transactions to be processed processing. These were identified by Streamline as duplicates and not
processed so customers were not charged twice. This also resulted in
errors picked up by Fujitsu's Reconciliation service so it was trapped by
both ends of the interface. The DRS (Data Reconciliation service) takes
inputs from multiple sources (counters, TPS database and Fl reports) and
compares them and produces reports detailing transactions where they
disagree
75] surs357P 5.38 Failures or interruptions in service I Same type of incident as previous KEL at row 14 - same analysis. No impact.
with the harvesting process can
cause duplicate payment This KEL additionally states: as no customer accounts have been debited
transactions to be processed twice no further reconciliation is needed.
By way of further explanation, the reason for there being 2 KELs is that
this issue occurred in both December 2004 and again in March 2009.
However, in both cases both Streamline and Fujitsu picked up the
problem
AC_152871086_1
Se/LWza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
76) jharr8325 Recoverable Bat ‘Acknowledgment that the recovery I This KEL relates to a case where the system was behaving as designed, I There would be no impact if the user
Transactions process (in relation to recoverable I but the user in branch failed to answer the questions correctly and did not I followed the recovery process
transactions) is a complex area follow the recovery process properly presented by Horizon. If the user
failed to do this there could be an
impact due to the user's error.
T7I__carde464Q Failed Baz Particular difficulty in processing a__I This KEL does not indicate any software error. The specific scenario ‘As Wir Coyne acknowledges, in this
Recoveries recoverable transactions, In this described by this KEL would always appear as a failed transaction at a case there was no impact on the
instance the settlement of the Branch, so it is highly unlikely that any cash was exchanged and the branch account.
transaction had not been written branch or customer was out of pocket.
into the Branch database so the
recovery failure would have had no I All failed recoveries are monitored by Fujitsu and result in the exact
impact on branch or customer circumstances being checked out and a transaction correction issued in
accounts. the event of a discrepancy,
Allfailed recoveries are automatically identified in a daily report that the
security operations team receives (formerly MSU). They review this and
raise Peaks for any that require further investigation by the SSC team
AC_152871086_1 13
9e/L Za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
18 seng2037LTiF/1040) 343 ‘These KELs describe how various I These are both generic KELs describing how support should process There would be no impact if the user
and transaction states may also indicate I specific scenarios found on reconciliation reports. These reports are followed the recovery process
acha959T a failed recovery. generated daily for banking/debit card/e top up transactions where the presented by Horizon. If the user
F/1700)
counters, agent and financial information (Fl) feeds differ in some way (for
example, the counter timed out a transaction but the F! authorised it and
ring fenced the funds). They are sent daily to the security operations,
team. Each transaction is checked and sometimes further investigation is
required from the SSC team via a Peak.
These KELs describe legitimate states that can occur during reconciliation
following a failed transaction which is usually recovered from correctly.
There are 23 legitimate (normal process) states and 39 error states that a
transaction can enter, Each state has a specific meaning depending upon
the responses from the Counter, TPS and Fi. A legitimate state would be
that the TPS and Counter both know about a transaction but the Fl may
never send information about it (state 6); this is a legitimate state but it
would warrant investigation.
failed to do this there could be an
impact due to the user's error.
AC_152871086_1
Le/-ea
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
79] dsed4733R 34a Example where transaction This KEL refers to a failed recovery report (report of failed recoveries) and I This was a zero value transaction so
(no 20) recovery has failed due to a wrongly I in particular some unexpected items in it. there was no impact on branch
named recovery script. Jason accounts.
Coyne refers to a Horizon system —_I The existence of the failed recovery report is evidence of routine
error arising because of incorrect I robustness countermeasures in place to deal with failed recoveries. In If the transactions had a financial
Reference Data. this case, the unexpected behaviour seems to have arisen from faulty value then the issue would have had
Reference Data, which was corrected within a couple of days. an effect on the branch. However, it
would have been raised on Fujitsu's
This KEL refers to problems specifically with an ADCScript-HPBB_REC1 I failed recovery reports, investigated
recovery script. HPBB stands for Home Phone Broadband and it appears I and reported to the POL via a BIMs
that the transaction would have been a customer application for a Post report.
Office Home Phone Broadband service where information is collected
from the customer but no payment is made.
AC_152871086_1 15
8e/L za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
20] PSteed2847N Incorrect 551 Software issues resulting in Horizon I A user had Remmed In some cash to the wrong stock unit. When they Temporary financial impact which
Mathematical applying the wrong mathematical _I realised this they carried out a reversal of the Rem In but, due to a was obvious to the Postmaster (who
Sign sign when reversing transactions __I software error the value of the Rem In was doubled instead. The reversal I reported the issue) and corrected by
(ie. a plus (+) instead of a minus (-
»
receipt would have informed the user that something had gone wrong and
in this case they stated that their Rem In amount had doubled when
reporting the issue. Presumably they looked at a remittance report or
balance report to see this.
A Rem In reversal is not a particularly common transaction and it was
prohibited as part of Impact changes in 2004. The user has attempted it
for a specific reason so if after performing the reversal it hasn't had the
desired effect you would expect the user to clearly notice this and raise a
call with the helpdesk to query it. Upon confirming the error the NBSC
could then issue an ‘error notice’ to correct any anomaly.
It took just 13 days from reporting to active a fast track fix
Post Office issuing an error notice.
AC_152871086_1
6e/L Za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
21] carde5756N I Pouch Rem 552 This is an example where the This concerned the reversal of a Rem Out of a pouch, a rarely used Temporary financial impact which
Out Reversal system failed to reverse all items in I process, by which when a banch has prepared a pouch returning cash to I would be picked up by Post Office's
a multi-line pouch, meaning only the I the cash centre and then realised that they either made a mistake (as. cash centre reconciliation process
first item was reversed. happened in this case) or need the money after all. It was not possible to I and corrected by a transaction
rule out the possibility that this was caused by a software issue, butit was I correction.
not possible to replicate it so this could not be investigated further.
The Rem Out reversal appears to have gone wrong in this case and only
part of the pouch contents was reversed, thus leaving some of the value
still in suspense
22] GCSimpsont Foreign 5.54 All currencies in branch doubled up I The discrepancy (between the Horizon record and physical cash) was I Temporary financial impact which
04st. Currency following successful balancing eight I picked up in branch and a call raised. It appears that this particular I was resolved when the branch
Discrepancies
days previously.
incident was resolved by the branch upon monthly balancing.
Due to the delay in providing the information to the development team, the
lack of any record of this incident having happened previously and there
being no further reports of similar problems, we are unable to confirm the
root cause
carried out the next monthly balance
(when it declared the actual value of
currencies held in branch).
AC_152871086_1
Ov/LL/Za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
23] MHarvey3527 I Insufficient 555 Evidences that in certain The KEL says: No impact.
1 Diagnostic Data
investigations there was insufficient
diagnostic data to be able to fully
diagnose an issue.
‘As this is, at the moment a one off event and clearly no further progress
can be made at this stage, I have therefore closed PC113202 as
“insufficient evidence". However, any further occurrences should be sent
to APS Counter Dev for investigation.”
This is an issue in the transferring of copies of transactional data from
Horizon to Post Office's back-end systems. The specific data mentioned
here is not financial. The underlying issue was that there was
unnecessary validation of copies of Reference data being passed back to
Post Office and this validation was removed as part of Impact (a joint
working body to introduce improvements to the system and processes) in
2008.
24) CObengii23 I Stock Gains 556 Unexplained discrepancies (gains) I This was a complicated memory loss issue in branch. Extensive searches I Not known due to the age of the
a for different stock unit types (Cash I were made for memory loss issues in test at the time and only one was_—_I matter.
and Stamps) was reported. The found and explained (which did not relate to this issue). It therefore
incident remained unexplained and I appears that this was a one off incident. Due to the passing of time, we're
no record of advice having been unable to identify the cause, but if the counter software was running short
provided to Postmasters of memory we would expect the counter to display a warning to the user
which would have been seen.
25] DRowei625K 387 This KEL is not available
AC_152871086_1
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
biLWza
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
26I dsed5250 PIN pad 5.68 There was a failure in the This was a faully PIN pad. It prevented the Postmaster from carrying out I No impact.
Failures Postmasters being able to transact I some transactions but this would not affect branch accounts.
various types of transactions
including payment transactions
using a PIN pad. An error message
and code were generated, and a
new PIN pad was the
recommended solution.
27] surs3941P PIN pad 5.68 There was a failure in the This appears to have been an issue with a corrupt customer card. Itwas I No impact.
Failures Postmasters being able to transact I set up with no CVM (CVM is something on a card that indicates whether a
various types of transactions PIN or a Signature is to be used to authorise a transaction). SSC suggest
including payment transactions that this was an attempt to do a Balance Enquiry on a Credit Card (which
using a PIN pad. An error message I isn't allowed). Reference Data should have prevented that being
and code were generated, and a attempted and it's not clear why it didn't. It could have been a corrupt card
new PIN pad was the that the customer had.
recommended solution.
AC_152871086_1
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
Cp/L Za
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
28] BrailsfordS22 PIN pad 5.68 There was a failure in the This was a faully PIN pad. It prevented the Postmaster from carrying out I No impact.
39K Failures Postmasters being able to transact I some transactions but this would not affect branch accounts.
various types of transactions
including payment transactions
using a PIN pad. An error message
and code were generated, and a
new PIN pad was the
recommended solution.
29] carde2i9R PIN pad 5.69, 5.135 I This KEL indicated that PIN pad The KEL relates to a specific (one-off) hardware issue with the old No impact.
Failures related issues would usually result I Hypercom PIN pad where the transaction was authorised but not
in the recommendation of a new confirmed due to the hardware fault. The process is request, authorisation
PIN pad, regardless of the error. InI and confirmation, however, the final stage of confirmation did not take
this case a transaction had been place. As per the KEL, this was identified as part of the reconciliation
declined by the PIN pad but did not_I process (i.e. the same DRS reconciliation process referred to above) and
get reversed passed back to Post Office and a Transaction Correction issued
automatically. There was no impact on branch accounts, however, the
customer was charged twice and they should have been reimbursed by
Post Office's back end processes.
30] dsed4733R 5.92 Identifies multiple failed recoveries I See analysis at line 19 above. See analysis atline 19 above.
ier106) occurring because of a wrongly
named recovery script
AC_152871086_1 20
ev/Liza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
37] obenged933K 5.93 There was a loss of This is further evidence of the failed recovery report doing its job and This may have caused a temporary
communications following network _ I alerting Fujitsu to failed recoveries to enable them to investigate them and_I financial discrepancy
(F787) banking transactions and the make any necessary corrections to accounts by sending a BIMs to Post
printing of the customer receipts Office. If this had not happened, a transaction correction would have
resulting in a message to the data_I been issued as a result of Post Office's own reconciliation processes.
centre timing out. Consequently, the
Postmaster was asked to follow The incident was caused by a complex ‘grey! communications failure (ie.
recovery but the transaction was the network kept switching between good and bad; not solid good or solid
only able to recover partially. bad), which the development team could not reproduce.
The KEL gives no reason to suppose that, even if this condition had
persisted, the backstop of reconciliation and Transaction Corrections
would not have corrected any resulting errors in accounts.
As per KEL, the failed recovery will be centrally reported and investigated
via the DRS reconciliation process.
32 wrightm33145 I Payments 5.176, 5.118 I Non-zero trading position. This is the same issue as line 1 above. See analysis at item 1 above. See analysis at item 1 above.
J Mismatch
AC_152871086_1
21
pela
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
33{ AmoldA2753_ I Withdrawn 5.117 This generates the same Receipts I This issue was found during internal Horizon Online testing and fixed There may have been an issue if the
P Stock and Payments Mismatch error before Horizon Online went live branch had continued to rollover
message (to KELs in lines 1 and 31) before the products were reinstated,
but in fact relates to a mismatch In 2016, some products were withdrawn during a trading period when they I however, in the 2 cases recorded,
during the balancing of a stock unit I were stil being traded. This is contrary to Reference Data procedures the branches were advised to delay
that contains withdrawn product. and caused an issue in branches that had traded those products. the rollover until the products were
re-instated so no impact on branch
A fix was put in place involving the products being re-instated and the I accounts.
branches affected rolled over successfully.
34] ballanij 17590 Payment 5118 Deiails 3 conditions that may cause I This is a generic KEL ensuring that the event monitoring team raise a call _I N/A (this is a generic KEL).
Mismatch a receipts/ payments mismatch that I every time a Receipts and Payment mismatch event is seen. It references
{F/798} may impact on branch accounts. other KELs that are known issues for specific cases. This is an example
of how our monitoring is made to work effectively. All of these calls get
investigated and may need manual correction to avoid errors in branch
accounts. From a Horizon perspective, none of the calls raised should
therefore be left without investigation / resolution.
AC_152871086_1 22
Sv/LLiza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
36] achat3570 Incorrect 5.120 Ttis possible for discrepancies to The issue related to declarations from the same trading period a year ago I There may have been some financial
Declarations have been accepted by the becoming visible again and thus causing confusion. It only affected impact, but there were reports
{F/1655}
Postmaster based upon incorrect
declarations. The problem could
arise due to old stock declarations
not being automatically removed
from the system. These
could only be removed by making
zero-value declarations or deleting
the stock unit then waiting overnight
before balancing.
branches that had done a Stock Declaration the previous year but
normally didn't do them. The fix was to change the archiving strategy so
that all declarations that had not been updated for 6 months were
removed. A check was made at the time for any branches that had old
discrepancies that might become current again in the next 2 months (to
allow for the archiving fix to be made) and these were removed manually
by MSC. A
The initial call was raised on 11th Feb 2011. A central workaround to
avoid further issues was implemented under MSC and the official fix was
released in June 2011
available in branch that could have
been identity
declared amounts.
used to incorrect
Further, a corresponding gain / loss
would occur in a subsequent trading
period so that would resolve the
issue
AC_152871086_1
23
Ov/L LIZA
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
36I achad1450 5.121 Provision of a full support solution I This KEL is cited in the previous KEL dealt with at line 35 and relates toa I There may have been some financial
for incorrect stock declarations and
discrepancies
stock balancing problem caused by the user doing some uncommon
sequence ~ i.e. not caused by withdrawn products.
impact is is branch process at this
stage, it is effectively as if the SPM
had wrongly declared the cash or
stock and the system will warm them
that this match the
calculated value. In this case it was
does not
an old declaration that got included.
Redeclaring the cash/stock will fix the
issue. If they do not they will roll with
a loss this BP/TP but have a gain
next TP/BP.
37] allend1645p Horizon
Interface
Controls
5.129
Provides an example of Horizon's
weak interface controls and lack of
data entry validation. In a single
sales transaction the user was able
to select and enter different
methods of payment (Debit Card
and Fast Cash). Horizon allowed
the transaction to be settled via Fast
Cash when the Debit Card payment
method had already been selected.
Horizon allowed the clerk to select ‘Debit card’ as a method of payment
and later switch to ‘fast cash’ at the end of the customer session. This was
a subsequent user error which involved the user at the branch failing to
take payment of £500 for the 540 euros
This is a case of the Postmaster
being responsible for errors made by
staff
discrepancy caused by user error in
not taking the payment that was due
This would have shown as a
from the customer and the
Postmaster would be liable for this.
AC_152871086_1
24
LviL za
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
36[acha621P 5.130 The correct screen to successfully I This issue, also referred to as the ‘Dalmellington’ issue is dealt with ‘Temporary financial impact which
process a cash pouch did not
appear resulting in the clerk in an
outreach branch inadvertently
doubling up the amount of cash
recorded. The issue appears to
have been caused because of an
earlier system logout or inactivity
which in turn resulted in incomplete
checks being conducted by Horizon
post logon
substantively in the second witness statement of Torstein Olav Godeseth.
was rectified by a Transaction
Correction being issued
AC_152871086_1
25
8v/L Za
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
39] EJohnson393 Horizon 5.132 This enabled Postmasters to carry I Whilst Remming In currency it is possible to create a transaction with a No impact.
7R Interface out “Rem In’ transactions without a I positive quantity and a zero value
Controls value being entered
This issue was caused by the Reference Data test team (not following the
correct process). The functionality was changed the following year so that
amounts did not need to be entered.
Auto Rems (introduced around 2004) meant that the content of cash and
currency pouches was sent to the branch electronically so when a pouch
was delivered, the system automatically told the Subpostmaster what the
content was and used that value for the Rem in rather than asking the
‘Subpostmaster to key it in
AC_152871086_1
26
6p/L LIZA
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
40; PSteedi45J_I Phantom Sales 5.133 Involved ‘phantom’ sales appearing I These “phantom” sales were caused by hardware problems and fixed by the transaction related
‘on the Horizon counter screen but
which had not been selected by the
user.
replacing hardware
No impact
to stock, when the branch declared
their stock and cash the discrepancy
would cancel out (eg. a sale of
stamps would reduce the stock of
stamps and increase the cash figure
by a corresponding amount; when
balancing
stamps should be declared and this
will cancel out the effect of the
the correct number of
phantom sale of stamps).
41{ pearroll1235R.
Screen Freezes
5.133
Instructions on how to deal with
environmental issues and hardware
are contained in this KEL. Jason
Coyne says "it is not known how
widely these were distributed to
SPMs', implying that they should
have been
These instructions (which relate to issues that do not concer bugs or
errors in Horizon) were distributed to those who called for help via this.
KEL
No impact on branch accounts
AC_152871086_1
27
Os/LL/Za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
42) jharrT323L 5.136 Example of a successfully recorded I This relates to a fishing rod licence request not sent to the Environment I No impact.
(F/1542}
transaction initiated in a Post Office
branch (where a customer receipt
was generated) which failed to
appear in the Post Office Data
Gateway.
‘Agency and was only detected at a later date. In this particular instance,
the transaction had been reversed by a user; this is not a software issue.
Once a transaction is reversed, all relevant data is discarded and not sent
to the AP Client as its effectively as though the transaction never took
place (Fujitsu only keep Post Office Data Gateway records readily
available for 30 days) but it is committed to the audit store and therefore
any additional investigation would have needed to be undertaken by
audit),
Itis not known why the reversal took place. This could be due to
fraudulent activity or could be that the customer sought a refund and the
money was refunded
AC_152871086_1
28
tS/LWz@a
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
Bl MAris34331 5.137 The ability of Horizon to erroneously I This was a software bug which allowed a transaction to be recorded twice
(no 46)
{F168}
record the same transaction twice
after a session transfer to a different
counter. This happened with both
NS&l (National Savings &
Investments) and Network Banking
(NWB) transactions. The KEL was
passed to a development team to
provide a bug fix as part of the S60
rollout but it is unknown if this was
ever resolved.
after a session transfer and was a fairly rare circumstance. This issue was
picked up during internal testing and despite numerous attempts
development were unable to recreate the scenario. There are no Peaks
which refer to this KEL.
No impact (fixed in testing)
AC_152871086_1
29
es/bViza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
44) CharitonJ275 I Previous Key 5.139 Counter level corrections made via__I A software errorin a PDL file meant that when a user used the “Previous” I No impact.
ar
Software Error
the “Previous” key led to both the
old value and amended value being
stored and used in error in the
transaction. A fix was released in
the Live environment eight days
after the issue was first raised
key for a transaction that used an ADC Script, old and amended values
were stored and used. This resulted in an incorrect transaction for the sum
amount of both the old and amended values being added to the sales
basket.
As the user has used the "Previous" key to go back and amend a value, it
should have been obvious to the user if then a completely different value
item is added to the sales basket. If, for whatever reason, this was not
noticed then the the customer will end up being overcharged as the
system will ask the user to take a larger payment. Assuming the user does
what the system says there will be no impact to the branch accounts.
This issue affected 3 products and only occurred when "Previous" key
was used to correct the amount entered. It was fixed 8 days after it was
spotted. A search was made for all branches that had used those
products twice within a session and the results were sent to Post Office.
AC_152871086_1
30
es/b za
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
46 SSurs43P 5.141 Example of a declined network An error in network banking caused the customer's account to be debited I No impact.
banking although the transaction failed at the branch and the Postmaster was told
transaction that resulted in money _I at the counter that the transaction failed. Banking transactions with a
being taken from the customer's response code of 26 (Fl Unavailable, Try again later) would be recorded
account. as zero value transactions at the branch, a DECLINED receipt would have
been produced and so no money should have been handed over to the
customer. Therefore no impact on branch accounts. On rare occasions
the financial institution may have debited or credited the customer bank
account despite us not receiving the authorisation. In these instances, if
the automatic reversals fail to resolve matters then the issue would be
picked up as part of the DRS reconciliation process meaning Fujitsu would
inform Post Office of what has happened at the counter so that they can
liaise with the financial institution to ensure their systems match so
customer is not out of pocket. The root cause was between the NBE and
the Fl, which is outside of Horizon.
AC_152871086_1 31
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
pS/Lza
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
46) LKiang326R 5.142 Examples of E-Pay transactions This was caused by two authorisation agents being active at once, when I No impact.
i260) crediting the customer account only one should have been, resulting in the phone being credited with £10
twice although only one payment —_I twice and ePay charging Post Office or the phone provider (depending on
has been taken, the arrangement between the two) being charged twice, meaning the
customer would have got two sets of top up for the price of one
This is a back end system problem which would be picked up by counter
measures, causing a BIMs to be raised.
47, SSurS310P 5.142 Examples of E-Pay transactions Similar to the KEL above at line 46 No impact.
crediting the customer account This is a back end system problem which would have had no impact on
twice although only one payment —_I the branch accounts. Despite a phone being topped up twice at the
has been taken. branch only a single top up would have been recorded in branch along
with the required payment for it.
Again, this is a back end system problem and would need to be resolved
by Post Office centrally and would not impact branch accounts.
AC_152871086_1
32
Ss/LWz@a
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
POL00000692
POLoo000692
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
48) pothapragada 5.165 Jason Coyne says “itis The issue was the ability to declare stock that could not then be This may have had a financial impact
c4359R
{F/836)
acknowledged that simple fixes
ought and were implemented to
either fix bugs or provide additional
data validation checks".
transacted (due to Reference Data rules)
The only impact on a branch's account were if the branch were to actually
declare that it held an item of such stock. This is unlikely as the item had
been withdrawn and should be returned to the stock centre. It should also
be noted that most branches do not undertake stock declarations as stock
is normally manged using stock adjustments (which didn't have this
issue). Should the stock be declared by mistake by user-error, then a
further declaration of the correct (i.e. zero) holdings would resolve the
issue
but if so it would be due to human
error (ie. declaring that it held an
item of stock that it couldn't transact).
This discrepancy would be removed
if the branch accurately declared that
it had no such stock.
49] Marris4123N_
F165)
5.165
‘Jason Coyne says “it is
acknowledged that simple fixes
ought and were implemented to
either fix bugs or provide additional
data validation checks"
This was a problem observed during test in Disaster Recovery for DVLA
transactions - a very rare circumstance, which should be handled
correctly, but would have no impact on branch accounts in the unlikely
case that a branch encountered the issue
When recovering a failed counter, the user is asked to input data from a
receipt. To handle poor typing there is a check sum which should ensure
that it has not been altered. This bug relates to the fact that not all the
data entered is controlled by the check sum. Therefore, when a tester
deliberately input incorrect data, the system did not detect it. NB this did
not include financial data.
No impact
AC_152871086_1
33
9s/Lza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
50] acha2230K Additional 5.186 Jason Coyne highlights a problem I This issue, also referred to as the Local Suspense Account issue is dealt I This issue had a financial impact
Checks with additional checks which were —_I with substantively in the second witness statement of Torstein Olav which was resolved by Post Office
{F/1150} i” .
implemented to identify system Godeseth writing off discrepancies
errors! inconsistencies when
balancing
during branch and references a
note within the KEL as follows
“This should never happen -
something has gone horribly wrong.
Or possibly the checks haven't been
implemented as intended.”
AC_152871086_1 34
ZS/ 2a
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
51] dsed2049S 5.187 Jason Coyne suggests that this KEL I This KEL relates to withdrawn products that were converted to cash on No impact if the correct process is
highlights the lack of system rollover but the losses are carried forward into the next period instead of _I followed for returning the withdrawn
‘communication and/or support being dealt with there and then products to Post Office. In the event
‘communication in respect of certain that the process is not properly
system features which could Withdrawn products should be sent back by the Postmaster to Post Office I followed, a Transaction Correction
subsequently result in errors. so the branch is not holding stock that cannot be sold can be issued to correct any impact
on branch accounts.
If this process is not followed, the branch will be left with a loss at the next
trading period and could be corrected by a Transaction Correction. The
stock will also be converted to cash if the Postmaster has purchased it
personally, for example. The fix was made to make this clear to the
Postmasters.
Jason Coyne implies this was a bug which took 6 months to fix. There
was a minor bug in that the cash value of the withdrawn stock was not
included in the current rollover, but delayed until the next rollover.
However there was no specific loss to the branch (other than the value of
the withdrawn stock which they were responsible for as described)
35
AC_152871086_1
8s/LL/Za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
52] acha3250R 5.189 The reconciliation process used by _I This was an issue with Post Office's back end reports, caused by timing I No impact.
Post Office to assist with identifying I issues (APS transactions arriving a day late) which were outside of Post
any accounting differences isnot I Office or Fujitsu's control
able to easily identify genuine
differences and/or differences This caused discrepancies in certain back-end reports, which could
resulting from external APS however be understood by cross checking other reports. This is a “false
transactions from old trading dates. I error” being reported by reconciliation relating to transactions occurring
around the end of day, being 7pm.
There is nothing wrong in the actual transactions — just an error in the way
that reconciliation totals are calculated.
53] achai947L 76 During the recovery process (when I The KEL says: As there is no reconciliation needed,
some transactions recover but there is no impact on branch
others fail to recover) itis only the I "This is not really a problem, it is just confusing when investigating a state I accounts
recovered transactions printed on _I 4 call. The Disconnected Session receipts will show all the transactions in
the receipt. The disconnected the session. The successfully recovered transaction needs no
session receipt should also identify I reconciliation."
those transactions not recovered.
These are printed for the This shows that any incomplete information went not to the branch, but to.
Postmaster to retain. someone in Post Office or Fujitsu investigating a state 4 call
AC_152871086_1 36
6S/LL/Z4
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
B4{sursi147Q Failed 77 This relates to the solution toa While the advice to the user in the KEL is somewhat counter-intuitive and I No impact.
Recovery failed recovery requesting the user _I indicates that nothing should be done and the user should wait for the
to log onto the relevant counter and I counter to time out, leaving the system error, there is nothing to suggest
start the recovery process, but the recovery was not resolved. The root cause of this issue was an error
leave the counter displaying the in Post Office's script relating to the Dangerous Goods products, that
system error message. resulted in recovery failing. We're unable to tell whether or not the script
was corrected.
This would have no direct impact on branch accounts, but clearly would
be very inconvenient as the counter is out of action.
56] wrightm33145 Payment 79,741 I Cited by Coyne This issue, also referred to as the Payment Mismatch’ issue is dealt with I See item 1 above.
j Mismatch KF/1450} substantively in the second witness statement of Torstein Olav Godeseth
56I boismaisons? I Disc Space oT This KEL describes the running This is an issue with hard disks. Disc space sizes have nothing to do with I No impact.
328M Sizes ‘commands on counters to assess __I branch transaction data which in any case at the time (2012) was not
(F052) disk space sizes. stored in the branch.
AC_152871086_1 37
Og/LL/Za
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
KEL Mr Coyne's Report Fujitsu's Comments
ID Short-name I Paragraph Mr Coyne's Summary Response to Mr Coyne Financial impact on branch
(if applicable) No. (paraphrased) accounts
57] SeemungalG I~ Transaction 949 Records an instance where This was an issue when posting transactions from old Horizon to BRDB I No impact.
5199 Amendments transaction prior to a branch migrating from old horizon to Horizon Online. In some
amendments carried out are cases, such transactions fell foul of validation in TPS and needed to be
causing exceptions. amended before being sent to Post Office's backend systems. Such
amended transactions were also posted to BRDB. Any amendments were
related to the trading period they related to and not any financial values.
They were only in BRDB to be used if the branch migrated during the
current trading period from Old Horizon to Horizon Online.
Tip Repair is a back end process to make sure all required transactions
get sent to the relevant external systems. There is no effect on the branch
accounts.
56] MHarvey2255 I Corrective 9.50 Records the manual addition of This is a back-end balancing issue caused by missing or invalid No impact.
P (no 64) Balancing corrective Reference Data. The insertion of correction records is done into the TPS
Transactions balancing transactions inserted by I database to allow the branch data to be forwarded to POLSAP. There is
{F/345) SSC affecting the TPS system. no effect on the Branch Database therefore no effect on the Branch
Accounts. It is not understood why this is marked as remote access as the
work is done within the Data Centre.
AC_152871086_1
38
L9/LLZa
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
APPENDIX 2
KEL
Fujitsu's analysis
Nickname
Summary of the KEL
Analysis
Impact on branch accounts
7 achadZ3aK Cash Button I The Cash button on the seftlement This is not a bug, rather itis a feature of how the system No impact unless the system is misused by
screen can be used for either a receipt in I operates. branch staff.
{F/379} ‘or a payment out of cash. Horizon
decides on the context depending on
whether the stack total button says TAKE
or PAY,
ZI achadsss Tange This involved both transactions being I Although the clerk took the money from the customer, the I No impact.
Transactions I completed in a single session. session wasn't settled because £1m is too large for FastCash.
The settlement should instead be entered as two £500,000 cash
payments. This is an issue with hitting system limits with very
large transactions over £1M. This would be very noticeable and
there is an avoidance action to take two smaller payments.
Looking at the Peak, the Transactions were actually recovered
automatically by the system (AP Txns were recoverable in old
Horizon) so no impact on branch accounts once recovery was
carried out.
3. acha508S
Postmaster reports
remming out, in particular differences
between the two receipts which are
problems with
This was a bug in the handling of multipie bags of coins when
It was fixed by a code fix issued in April
2007 and fully rolled out by June 2007.
being Remmed Out
Temporary impact that would have been
identifiable from the reports available to
Subpostmasters. The KEL description states
AC_152871086_1
39
Z9/LL/Za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
printed after the pouch barcode is that “differences between the two receipts
scanned. The original problem was found on 12" Feb 2007 and was I which are printed after the pouch barcode is
presumably due to a software update rolled out at that time. I scanned... The second receipt (Office Copy)
Investigations were carried out and a list of affected branches I only shows one bag of each’.
was generated (this is no longer available) and provided to Post
Office. NBSC was informed about a workaround to the issue. _I It would have been rectified by a transaction
correction
a) achas22T Cash This involved two separate cash I This is not a bug, rather the incident appears to be caused by I Impact caused by human error.
Withdrawal I transactions most likely by two separate I human error of the user not reading the screen carefully when
{F/697} customers as follows: doing a withdraw limit CAPO transaction after failing to settle an
earlier customer basket.
= customer 1: given £200 in cash
= customer 2: given £320.90 in cash
(being the total value of the stack)
The loss was caused by the user not
setting stack in between banking
transactions.
5.I acha2i40S I Eim Cheques I ifthe branch holds cheques to a value of I This will be rare and if encountered it will be highly visible by the I No impact,
more than £1M, then the value of I Postmaster.
F374) cheques cannot be adjusted using the
normal Stock Adjustment mechanisms. I The best thing to do would be to rem out the Cheque for £1M
and then any further adjustments can be handled as normal
6 I acha3347Q Ifa stock unit carried forward from This was an issue for a branch following migration from old I No impact.
Horizon is deleted before the first trading I Horizon to Horizon Online and only affected a branch if it had
period rollover on Horizon Online, the deleted a Stock Unit after migration and before the first rollover.
AC_152871086_1
40
eg/LLiza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
check for Tast stock unit’ may not be
applied properly, and all the stock units
can roll over without Local Suspense
being cleared.
Local Suspense for a particular trading
period has to be zero before the branch
can be rolled over. Once a stock unit is in
the new trading period, it can put
gains/losses into Local Suspense and
clear them, but this has no effect on
Local Suspense for the old trading period
- think of them as two separate
containers.
The issue would be clearly visible as the Branch would be
unable to rollover without following a complex work around,
~
acha36i0P
{F/679)
Advantageous
Exchange
Rates
There are advantageous exchange rates
for transactions over certain limits
(shown on Foreign Currency report as
DDE for Euros, DDU for US dollars). In
this case the limit was £500.
The user pressed 'Buy Euros’ and
entered 600. The exchange rate shown
was the standard exchange rate, not the
rate for transactions over the 500 limit,
the reason being the £500 limit had not
been reached
This appears to be a misunderstanding of what exchange rates
to use. The “over 500” rate applies to when the sterling value is
cover £500 (not the Euro value).
No impact.
I
acha4221Q
The clerk went into the ‘Delivery’ menu
and scanned two pouches (one of
This was a bug in the early days of Horizon Online following an
unusual (but valid) sequence of events. It was fixed on 19"
Financial impact would have been clear to the
branch because a duplicate receipt was
AC_152871086_1
41
v9/LLza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
Guirency and one of coins). The second
pouch was recorded twice on the
system, resulting in a loss of £80
Two Remittance In slips relating to the
second pouch were output, both
identical, as well as one for the first
barcode.
April 2070
printed. It would have also been identifiable
from the reports available from Horizon.
It was resolved by a transaction correction
being issued.
o
achad353P
\(F/828}
The counter froze while cashing a Postal
Order and now recovery won't complete
so the counter could not be used The
counter was rebooted, but when they
logged back in they got a Postal Order
encashment transaction —_— recovery
message for a negative amount, followed
by the Invalid Value message again.
This was due to an invalid Postal Order with a negative value
being set up by an external system which resulted in a counter
being frozen.
A fix was applied to check the value of Postal Orders coming
from external systems as being positive and this was applied on
15/8/2011. A system fix was made to allow recovery to be
bypassed on this counter to make it useable again.
No impact
FS)
acha5226J
(F/734}
When a branch puts through a bureau
transaction in excess of £5,000 a
message should appear on the screen to
remind the user to ask the customer to
take two forms of ID from the customer
to conform with Anti Money Laundering
regulations. This reminder prompt is not
appearing on the Horizon Online
counters.
This issue was fixed in October 2010
No impact
acha5259Q
{F/637}
The Postmaster wrote a discrepancy of
£167.17 to Local Suspense, then this
was cleared from Local Suspense as
it appeared to only affect branches balancing in April 2070 and
33 branches were identified as being impacted. Details of these
branches were passed to Post Office.
Temporary financial impact which would have
been cancelled out in the following period by a
corresponding discrepancy.
AC_152871086_1
42
So/LLZa
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
normal and the Postmaster selected to
make good the losses. At this point the
system printed out a final balance report
for the trading period with the cash
figure amended and nil discrepancy.
Normally the system would then come up
with a message to confirm rollover but,
instead went back to the screen asking
how the discrepancy was going to be
made good.
This was an issue found in the early days of Horizon Online and
was resolved in July 2010.
FS)
AChambers 183.
3
{F/186)
Report Issues
‘A transaction for £6.67 was done at
17:23 and was settled at 17:25. The daily
APS transaction report was done on a
different counter at 17:25 and did not
include the APS transaction.
A timing issue to do with printing reports at a counter. There
was no impact on the branch accounts ~ just confusion due to a
transaction being missed from a report and the report not being
re-printed when it appeared
No impact.
43) AChambers225
2R
{F/316)
Postmaster sold some foreign currency
(1000 euros, sale value: £750). The
Postmaster realised the transaction had
been settled to the wrong product in this,
case being cash instead of debit card.
Existing Reversal was used to reverse
the transaction, and then re-run correctly.
When it came to balance at the end of
the trading period, the currency stock
holding on the system was too high by
4000 Euros. When corrected, this gave a
gain on currency of £720 and a cash loss
This was not a bug, rather an issue in how a Currency
transaction was incorrectly reversed on old Horizon
Foreign currency transactions consist of two parts: the currency
itself, which has a value based on the exchange rate, and
margin, which is added to cover the cost of the transaction.
When the transaction was reversed, the Postmaster entered the
transaction for the cash settlement part of the transaction. While
the Postmaster believed the whole transaction had been
reversed, it had not as the margin had not been reversed. When
the stock unit was balanced, the wrong number of Euros
became apparent. The stock holding was corrected by the
impact, but guidance on how to correctly
perform an existing reversal was all that was
needed to rectify it.
AC_152871086_1
43
99/23
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
‘Gf E750, being a net loss to the branch of I declaration of the actual number held. Again, this did not correct
£30 the margin, which is generated as part of the currency sale and
is not directly linked to the stock
The way Reversals are handled on Horizon Online means that
this sort of issue can no longer happen.
74! AChambers473 Multiple quantity for stamps/postage I When a quantity greater than 1 is entered for a Smartpost I This may have had an impact but a user should
aR label affects cash settlement or I transaction, the Quantity is not reset to 1 when the user moves I have been able to spot it and the sums involved
GEREN subsequent transactions on to the settlement screen are likely to be small due to the issue affecting
mails products.
If the transaction is settled to Fast Cash / Fast Cheque or Debit
Card, this doesn't matter, but some users habitually use the
Cash (F2) button to enter the cash presented by the customer,
then give the customer change as indicated by the new stack
total. If this is done, the cash amount entered is multiplied by
the Quantity and hence the new stack total is wrong,
This was fixed in December 2005 (Reference Data fixed)
75} AChambers447 I Receipts and I This appears to have been an issue with I This was picked up by Reconciliation Reports(looking for I Temporary financial impact which would have
3a Payments I doing a Transfer Out which was not I receipts and payments mismatches) and investigated. been cancelled out in the following period by a
(F266) Mismatch I picked up correctly when balancing, corresponding discrepancy
76I AChambers571 Postmaster balancing on counter 7, then I This occurred due to a counter failure around the time EOD I No impact
4K completing on counter 2 due to counter 1 I activities were happening and thus resulted in a mismatch in
KF /188) timing out, causing a discrepancy I Reconciliation reports.
between reports.
77) Agnihotriv24a5L Horizon I The last digit of the exchange rate is I No Financial impact. It was fixed in September 2070. No impact.
{F667} Online occasionally being displayed incorrectly
AC_152871086_1
44
9/24
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
Exchange
Rate
‘on the rates board
The system does use the correct values
to calculate the rates -the issue here is
with the display only. However, because
the display is on the Customer facing
rates board there is potential for
annoying Customers as they may get a
slightly different rate to that advertised on
the board.
18 ~AOConnor158!
{F/81}
Transaction
Limits
Failed to Harvest a quantity of 11743997
pennies as being too large
This is another problem with limits. The branch tried to declare a
cash holding of over £100,000 in ‘p and this hit a limit in
Reference Data. The fix was to police the system limit on
declarations.
No impact.
19} AOConnor52571
{F/80)
The Postmaster remmed in a cheque for
£3,200, however, it did not show up on
his balance snapshot or adjust stock, but
it is showing in his Suspense Account.
This is a user error in how Post Office cheques were handled.
Following advice and Guidance the problem was resolved
‘Temporary impact caused by user error,
20) arnolda229R_
{F/530}
‘An Open Value Encashment for £5.00
was performed, the transaction was
authorised and added to the Basket but
the counter crashed before the Basket
could be settled. On Login Recovery was
invoked and a "Recovery Failure’ receipt
was printed for £0.00.
‘There was a typographical error in the script causing the issue.
This issue was found during testing of Horizon Online and fixed
before the first counter went live.
No impact (resolved before Horizon Online
went live).
21) AmoldA2341L
Currency
All currency codes, in terms of "IMoney"
This was an issue identified by developers regarding the way
No impact.
{F/533)
AC_152871086_1
45
89/L Za
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
Code
Validation
“Objects should be [SO-compliant, or
verified by £-sign However, we do not
validate this at all anymore, in order to
support future currency codes. This
means that junk codes are accepted
currency was handed within the counter code. It had no impact
in the real world it was closed.
221 ballantj020J
{F/698)
Postmaster states that she has sold a
Lyca top up for £10, the message appear
unable to connect to the data centre then
logs Postmasters out.
The transaction request has been
authorised and the reversal may not be
effective which will cause an'E21"
reconciliation error.
There were 2 issues here:
1. There was an issue with how Reference Data was generated
which resulted in some counter scripts failing. This was fixed on
20/08/2010 (2 days after this problem occurred);
2. The Ref data issue caused an e-pay transaction to fail and
the Postmaster didn't handle the recovery correctly
The failure to handle recovery correctly may have resulted in a
loss of £10.
This may have resulted in a small impact due to
a failure to follow the correct procedure.
23I — ballantj2547K
{F/633)
The Transaction Processing System
Total and Counter total values for the
Number and Absolute Quantity columns
are the same but the Absolute Value for
Counter Total is greater than the
corresponding Transaction Processing
System Total by £14.80
If the session nets to zero (add up all the
SaleValues for the same Sessionld) no
reconciliation is needed. If it doesn't, a
correction must be made to send the
data to POLFS (see <a
This is @ problem with Smart Post which seemed to write slightly
corrupt transactions in that there were missing attributes
required by back end systems and reconciliation systems, but
are complete as far as the branch accounting is concerned.
This therefore has no impact on branch accounts, but does
result in reconciliation errors which are fixed by amending the
transaction copies in the backend systems. The fixes are
always to dates and not to values.
This KEL refers to amount mismatches, but the amounts used
by the reconciliation are different from those used by
accounting. They should be identical, but in this case are not.
No impact.
AC_152871086_1
46
69/LL/Z4
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
POL00000692
POLoo000692
ref=kel_view_kel.jsp?KELRef=Maxwell
G460L>MaxwellG460L</a>) and the PM
may need to be told about a possible
receipts and payments mismatch, or at
least watch out in case one is raised
‘The accounting values are the correct amounts
24) ballantj3342L
{(F 1600}
Reconciliation picked up a scenario
associated with a failed banking
transaction. Specifically, the request
from the counter never reached the
Branch Access Layer.
The application event log shows
AdminCfg receiving data and causing a
VPNKeyChange
In this case the request never reached
the Authorising Agent and therefore no
money was requested, the CO did reach
the Authorising Agent but was
unexpected and has caused this
reconciliation incident.
Instruct MSU that no reconciliation
required.
As the transaction failed there was no impact on the branch
accounts.
This did identify some issues in the way that failed transactions
were handled and why they resulted in reconciliation errors and
these were fixed in April 2010
No impact.
25) bambers3553L
symbol
{F488}
Currency
Specification
A MoneyGram Send transaction was
initiated. There are 3 options to define
the amount being sent
i. including fee
ii. £ not including fee
This was an issue found during testing of Horizon Online and
relates to a “£” sign being displayed when the user is being
asked to input an amount in another currency to a MoneyGram
transaction
It was agreed not to fix this until a later time (not clear if it ever
No impact
AC_152871086_1
47
OL/L Za
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
POL00000692
POLoo000692
ii, Receive Amount excluding
fee.
The 3rd option allows a customer to
specify an amount to be sent in local
currency, such as $300, in which case
the receiver will get $300 and the send
amount is calculated in sterling
However, at the input of the receive
amount, the prompt appears (correctly)
as ‘Amount in USD" but the input box
(which is a currency datatype) has a '£’
symbol present. This leads the user to
think that they have selected the wrong
option and would lead to incorrect
amounts being entered here. Please see
the attached screen shot for evidence
was fixed, however, the MoneyGram product was re-engineered
in 2015 so it would have behaved differently after that time
anyway).
This would have no impact on branch accounts, but may have
caused some mild confusion to the Subpost master.
26) bambers4236K
F/524)
Electronic Top
Ups ("ETU")
ETU E-voucher for £10.00 is erroneously
declined as a New Reversal.
The basic problem is that, to support
ETU Reversals, we rely on the
Authorising Agent remembering details
of the original ETU transaction. Ina CTO
we only have an Agent Simulator and it
is not configured to handle ETU
Reversals. Given the simplicity of the
Simulator it would be very difficult to
support ETU Reversals.
This is an issue in the Counter Training service in that it doesn't
support reversals of E Top ups
It was discovered during testing and was agreed that this would
be a restriction on the functionality supported for Counter
Training,
As this only impacted counter training then there is no impact on
any Live Branch accounts.
No impact
AC_152871086_1
48
LLibW2a
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
This as a restriction for CTOs but is not
documented in DES/GEN/REP/0006 nor
REQ/CUS/STG/0004
27I BluerP6546R I Upside Down I For PING transactions (Transaction I This was a cosmetic issue when processing Transaction I No impact.
Pound Sign I Acknowledgments) £ signs will appear I Acknowledgments on a Training Counter in that the “£” sign was
{F/700} as upside down question marks in the I displayed incorrectly. This has no Financial Impact as it was
training counters, only affecting training counters.
This issue was spotted in internal testing and we had no reports
of this issue from Live Counter Training Offices. A fix went live
in January 2014
28{ cardo235Q DropandGo IThe user inifiated a Drop and Go I This was a problem in handling errors correctly in a Drop and I This would have caused a loss in the branch
transaction for £100 which failed due to I Go script provided by Post Office. accounts, although the issue was identified by
{F160} timeouts. Following the failure, a success the Subpostmaster and it would have been
message was displayed. The user I This was passed to Post Office to fix the scripts. resolved by a transaction correction.
settled the transaction and the customer
handed over £100. The customer
checked the balance and stated that the
top up had not gone through so the clerk
performed another Drop and Go
transaction which was successful. The
customer has paid in £100 but the
branch account has been debited by
£200. Accenture verified that only the
second Drop and Go top up was
successful
28] carde339P Receipt I A Transfer Out of £577 cash was done I The system is behaving correctly and there would be no I No impact - the Transfer Reports and the
{F377}
AC_152871086_1
49
CLILLIZA
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
Printing
on counter 14, session id 14-2966714-1
A corresponding Transfer In was done
immediately afterwards on counter 2,
session id 2-2046304-1
Transfer In had completed, a receipt with
the wrong session id (14-2966714-1)
was printed. After the Transfer In had
completed, the correct receipt was
printed.
Before the
The extra receipt, which is almost
identical to the actual Transfer In receipt,
is quite confusing. It may lead the
Postmaster to suspect that the transfer
has been carried out twice, when in fact
it has not. Advise the Postmaster to use
the ‘Preview’ rather than the Print button
if they wish to view the individual
products being transferred.
financial impact on the branch account, but the Postmaster may
be confused as to what exactly has happened.
Transaction log will show exactly what has
actually happened.
30! carde427T
(F/1362}
The transaction appeared as a state E26
on the NB102 Section 2 Link report.
This issue has also resulted in a
transaction being reported as both a
State 4 and State 6 - see PCPC0244934
PC0197368 was fixed and released as
part of release
This was an issue with failed banking transactions and then
recovery from a subsequent counter failure using the same
identifiers. Some of these appeared in reconciliation reports
when they shouldn't be and so causing additional work to
‘Support teams.
However as these are all failed transactions they are all for zero
amounts and so have no impact on branch accounts.
No impact.
transaction the next banking transaction may
reuse the same unique reference. This will
cause that transaction to fail also as the
Following a failed banking
authorisation software knows the transaction
previously failed and will not pass it on to the
Fi."
AC_152871086_1
50
eLibiza
POL00000692
POLoo000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
CTROT_22_07_00_RELEASE Gune
2010).
Unfortunately the change was regressed
as part of a BAL change in February
2015 - see peaks PCPC0243030 and
PCPC0241771. Another fix is in
progress
3 cardc2326R
End-of-session
sales prompts -
usability issues
{F739}
The user will usually count out the cash
to be paid before pressing Fast Cash,
because once they have pressed Fast
Cash then the amount to be paid out is
reset to zero and the stack disappears.
However, if the transaction results in a
sales prompt then the stack is not
cleared and the amount payable remains
on the screen. If the user selects the
sales prompt and transacts another
product, this is added to the stack and
the Total Due To Customer’ is updated
by the relevant amount.
If the user is distracted or busy then they
potentially pay out the new amount in
addition to the original amount. The
system is working as designed and
Postmasters should be referred to the
The system is operating as designed and no change was
requested by Post Office.
There may be confusion in relation to the way Sales Prompts
are handled at the end of a session.
if the correct procedures were followed there
would be no impact.
AC_152871086_1
51
pLiLiza
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
NBSC.
32] carde3335R
Vodafone Text
Pack Vouchers
being declined
(F/516}
A call was raised with e-pay to determine
why requests were being declined. This
is e-pay's response: The reason for the
Vodafone £10, £15 and £20 Text Pack
vouchers being declined by e-pay is
because they were deactivated on
request by Vodafone, We sent out a
Product Configuration document in May
detailing this change. This document was
sent to Dave Cooke at Fujitsu as well as
Clare Tetley and lain Gilbert at Royal
Mail. The deactivation was rolled out on
June ‘st.
Not a bug. Certain E Top Up products had been withdrawn but
the Reference Data had not been updated to remove them from
the counter. This meant that e-pay declined the requests.
Following the investigation, then the Reference Data was
updated to remove the products from the counter the following
weekend.
No impact
33{ cardc3415N
{F666}
When a branch migrated from Horizon to
Horizon Online, differences were
reported between the ‘Pre Quantity
Move’ and ‘Post Quantity Move' figures
This was a problem in the migration of a branch from old
Horizon to Horizon Online.
The report produced pre-migration on the old Horizon didn’t
take into account a Transaction Correction carried out in the last
trading period so adjust stock levels. The report produced post
migration was correct.
No impact - the figures post migration were
correct and the issue down to an inaccurate
pre-migration report.
34 cardc5946P_
{F/808)
Halifax / Bank of Scotland bank cards
declined with response 05 - ‘do not
honour’.
Branches with an “&” in their name were resulting in Banking
transactions being declined by Halifax / Bank of Scotland.
No information was given by Halifax / Bank of Scotland
regarding why they are declining the cards. This was fixed by
May 2014
No financial impact on branch accounts.
AC_152871086_1
52
SL/LWza
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
POL00000692
POLoo000692
3] CCardi2230 II [F7340} Counter hangs when attempting to clear I It would appear that an Invalid option was presented on the I No impact.
local suspense during stock unit rollover. I menu of options available when settling local suspense. This
seems to have been fixed shortly afterwards
36I CCarad658N I Tera 36) The stock unit balance report only I Buttons were introduced to record when cash was added and I No impact.
includes figures for Add or Remove Cash I removed due to vatiances being spotted. However the
transactions done in the current I behaviour of these was not carried forward from one balancing
balancing period. It should however I period to the next and so caused confusion
show the cumulative total since the start
of the trading period. When the problem was identified, the buttons were “padlocked”
and a fix was issued in March 2006.
37] ChartonJ2zaL Log on event timestamp can be after the I The log on event appears to be recorded at the time the log on I No impact.
KF/1188) I I Jog off and other associated events when I is processed by the HBS server, but the other rep events are
looking at rep events for the HBS Kiosks I recorded against the "DateTime' field in the incoming message.
38] CharltonJ2752T [fa 060} See item 24 of Appendix 1 See item 24 of Appendix 1 See item 24 of Appendix 1
36] CObeng? 1230 [prr1@5y I_I See tem 44 of Appendix 1 ‘See item 44 of Appendix 7. ‘See item 44 of Appendix 7.
40 -achad3agK Reconciliation reports relating to declined I This has no impact on the branch but caused unnecessary work I Zero value transactions have no financial
e-pay transactions are not clearing down I for the Fujitsu and Post Office's reconciliation teams. Issue was_I impact on branch accounts.
correctly. The affected transactions are I fixed on 11/10/2010
zero value, and have been declined by e-
pay
ai acha4745R This was an issue relating to back end I KEL suggests issue was with reconciliation reports not being Not known.
reconciliation where there was a £20
difference between 2 totals relating to
millions of pound worth of LINK
transactions.
(manually) processed correctly. Peak PC0219762 applies. This
is due to a recovery performed at the branch on the following
day which reversed the transaction. IT caused confusion in the
reports.
This was raised by POL's reporting systems
AC_152871086_1
53
9L/L 2a
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
POL00000692
POLoo000692
AQ] achas650L
I(F/1025)
‘A user logged into a counter where there
was an unsettled banking transaction
requiring recovery, which had been done
in stock unit AA.
Stock unit AA was still in TP 12 BP 4.
Stock unit BB was already in TP1 BP 4
Recovery completed successfully,
correctly writing the transaction and its
settlement into stock unit AA, but for TP
1BP1
The stock unit was short by the amount
of the banking transaction in TP 12 BP 4,
but then had a matching gain in the
following period.
If the TP/BP is incorrect, but the stock
unit will roll into that period in the future,
then this problem will cause a
discrepancy in the current period, but it
will be balanced by an equal and
opposite discrepancy in the future.
Advise the PM of this
Bug in the recovery process that could post transactions to the
wrong TP / BP. This would result in 2 discrepancies in 2
separate periods which would cancel out, so no long term
impact on Branches.
Fix issued in June 2010, but in theory could impact counter that
migrated and hit this problem on the day of migration to Horizon
Online until September 2010 (when migration completed)
No impact.
43I acha633R
The Settle Gain/Loss Centrally products
have a minimum transaction value of
£150 and should not be available if the
This is an issue identified by Post Office rather than a branch in
that branches were being allowed to Settle Centrally small
losses (limit should be £150 or more). This was identified on
No impact.
AC_152871086_1
54
LL @4
POL00000692
POL00000692
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
discrepancy to be cleared from Local
Suspense is less than £150. Horizon
Online does not appear to check the
minimum value when building the pick
list and so branches can choose settle
Centrally for any value.
48" May 2070 and the fix rolled out to all branches by 5" July
2010
44) AChambers253
L
This was an issue picked up by the
reconciliation checks due to smart Post
not correctly checking that the pre-paid
amount for postage was less than the
actual amount and then attempting to
generate a postage label for a negative
amount
Unclear what branches would have done at the time, however
the impact is likely to have been very small (pence rather than
pounds). This was fixed in 200.
‘Any impact would have been very minimal,
pence rather than pounds.
45I AllenD2519J
POLSAP report that a particular TC or
TAis missing from the expected BLE file
The TC / TA entry is actually present in
the BLE file, but it lacks the TC / TA
Reference value which allows POLSAP
to identify the item
Back end problem, involving data sent to POL from a PO client.
This is an issue caused by incorrect Reference Data with
passing data to POL’s Back end systems (POL SAP) and may
delay the processing of a TA/ TC.
No impact.
46} AllenD429U
One of the central systems failed during
the evening and when it restarted it
picked up the wrong time, which meant
that when it was trying to decide whether
a Txn happened before or after 7pm (the
EOD cut off) if got the answer wrong and
this meant that some data was
associated with the wrong day in back
end systems.
Looks like a back end problem, with no impact on branches
This is a problem in the data Centre which delays passing data
to Post Office's back end systems. However, it has no impact
on Branch Accounts.
No impact.
AC_152871086_1
55
SL/L L/Za
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
POL00000692
POLoo000692
47! cardc4027Q
(F/1149}
This was a problem in incorrectly
handling a transaction which had been
rejected by the PIN PAD resulting in a
spurious reconciliation error on a report.
The issue first occurred in 2011 and another similar issue
occurred in 2013. It appears to be related to the old Hypercom
PIN Pads which were replaced after the second occurrence of
this issue so no further action was taken
As the transaction had been clearly declined, then there would
be no financial impact at the Branch,
No impact
48] cardc5444K
F740)
The Postmaster received a Planned
Order as follows:
Based on the last declared Cash on
Hand figure of
£97,875,00 notes + £9, 156.29 coins
on 13.07.2010 you will need to remit to
the Cash Centre
£85,000.00 in notes on your next
scheduled collection day.
The PM had declared cash for all his
stock units on that day, however his
actual Cash Declaration value was much
lower than the Planned Order cash
declaration value. The Planned Order
asked him to remit too much cash
The PM normally used Till Id 1 for his
shared stock unit, but had accidentally
done a declaration with Till Id 16 during
the previous week. He was unaware that
the Till 16 declaration was being added
In a Shared SU, then it is possible to make multiple Cash
Declarations from different tills which are added together.
In this case a Cash Declaration had been made accidently for
Till 16 (instead of Till 1) which resulted in the value of cash
being doubled
This should have been spotted when the branch balanced.
However, before then the Cash Planning identified that the
Branch had too much cash and asked for some to be returned
This is a case of branch user error and there was no actual
impact on the accounts, once the spurious Cash Declaration
was identified and removed.
‘Temporary impact due to human error.
AC_152871086_1
56
6L/L 24
Claim No: HQ16X01238, HQ17X02637 & HQ17X04248
POL00000692
POLoo000692
to every overnight cash declaration sent
to the cash centre.
AC_152871086_1
57