Deloitte PRIVATE AND CONFIDENTIAL ~ SUBJECT TO
eo LEGAL PRIVILEGE
‘Bramble’ — Draft Report
Draft for discussion
THIS IS A DRAFT DISCUSSION DOCUMENT AND REPRESENTS A WORK IN PROGRESS AND MAY
CONTAIN PRELIMINARY RESULTS OR CONCLUSIONS, INCOMPLETE INFORMATION OR INFORMATION
WHICH IS SUBJECT TO CHANGE
19 January 2018
“@
‘This report and the work connected therewith are subject to the Terms and Conditions of the engagement letter dated 09 April
2014 (amended 11 March 2016) between Post Office Limited (Post Office) and Deloitte LLP. The report is produced for the
General Counsel of Post Office Limited (Post Office), solely for the use of Post Office Limited (Post Office) for the purpose of,
assessing assurance sources and the design of certain controls relating to the Horizon system. Its contents should not be quoted
or referred to in whole or in part without our prior written consent, except as required by law. Deloitte LLP will accept no
responsibility to any third party, as the report has not been prepared, and is not intended for any other purpose.
© Deloitte 20137 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
Contents
1 Executive Summary
2 Management Summary
3 Background
3 Scope and Approach
4 Work Performed
5 General Assumptions and Limitations
Appendix 1
Appendix 2
Appendix 3
Appendix 3a
Appendix 4
Appendix 4a
Appendix 5
Appendix 6
Appendix 6a
Appendix 7
Appendix 8
Statement of Responsibility
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
29
32
38
79
80
83
85
88
90
91
122
124
128
136
152
POL00139537
POL00139537
1. Executive Summary
1.1 Horizon Online Core Summary
144
142
113
1.14
In assessing the Horizon Online system, our work has focused on a broad suite of controls which, in
collaboration, work to assure that the integrity of transactional data is maintained from branch to
Audit Store. The controls respond to the fundamental risks of data integrity which are:
1.1.1.1 Completeness — All data is transmitted from source to destination in its entirety.
1.1.1.2 Accuracy — Data is accurately transmitted from source to destination without change.
1.1.1.3 Validity — The data is valid and has not been doctored or changed such that it is no longer
representative of the information the original data was recorded to capture, or has been
created spuriously and not linked to a real life data generating event.
The system controls across the areas of the Horizon Online system we have examined are robust at
the point our work was conducted with minimal exceptions noted from our testing. They are
appropriate to a system the size and scale of Horizon, and the distributed electronic point of sale
(EPOS) function it performs. The system controls have been designed to meet a high standard of
control and have been assessed similarly in the reports of other independent assurance
‘organisations such as Emst and Young (Service Auditor Report) - although not specifically in the
context of responding to these allegations.
(Our-workOur work has focused on the core data flow within Horizon Online, from the Counter in
branch to the Audit Store. We focussed on this particular core data flow because itis this data flow
which leads to the initial capture of transaction data and its subsequent long term storage and, in the
‘event of an issue or challenge, data is downloaded from the Audit Store to enable Post Office to
‘carry out an investigation. This data flow is subject to industry standard cryptographic controls which
are automated, inherent system controls and they are applied by the system to each and every
transaction processed by the Counter. They represent the most reliable control type possible over
data integrity — they are hardcoded into the system and no manual intervention is required for them
to operate. As a consequence of being inherent to the technology they have been in operation
throughout the life of Horizon Online
Working together, the Digital Signature (paragraph 1.3.3.1 (d)) and JSN (paragraph 1.3.3.1 (c))
controls respond to the fundamental data integrity risks of Completeness, Accuracy and Validity and
make it extremely unlikely that the record of transactions contained within the Audit Store is not
representative of the transactions input by staff in branch. As with all large scale computer systems
whilst itis theoretically possible that glitches and coding errors in the system could have resulted in
errors in the recording of transactions to occur, the likelihood of such errors occurring in a manner
which has adversely affected only certain branches materially whilst not affecting other branches at
all / minimally is in our view remote given the controls in place. The testing we have performed over
these controls was designed and executed to assess their operation in responding to these
fundamental risks. Noting the assumptions and limitations detailed in section 1.5, this testing has not
resulted in any matters being identified that would call into question the integrity of the core data flow
within Horizon Online from the Counter in branch to the Audit Store.
While we have identified an exception in the cryptographic controls (paragraph 1.4.2.10 and
1.4.2.1) which would theoretically allow a malicious actor to undermine them and potentially change
data, it is limited to a third party (Fujitsu) and would be technically very challenging to achieve. It
would require significant motivation for one of the limited set of Fujitsu staff members to exploit this
vulnerability given the technical challenges and risks of tripping monitoring controls and, although we
have not performed procedures in this area, it would almost certainly require collusion with Post
Office staff or Post masters. Although our investigations have not been exhaustive, they have been
‘extensive and we have seen no such evidence of malicious misuse of the system.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 2
POL00139537
POL00139537
1.2 The Allegations
1.2.1 Post Office-andOiice ~theand the legal firm acting on behalf of Post Office (Womble Bond
Dickinson) have informed us that the Claimants in the Group Litigation have asserted that Post
Office / Fujitsu has the ability to add / delete / change transactions recorded by branches without the
‘consent / knowledge of a Postmaster and that this may have been the cause of discrepancies in
‘some of the Claimants’ branch accounts. We understand that the allegation has been formulated in
several different ways:-
1.2.1.1 Post Office / Fujitsu have the ability to log on remotely to a Horizon terminal in a branch so
to conduct transactions.
1.2.1.2 Post Office / Fujitsu have the ability to conduct transactions (either remotely or locally)
under another user's 1D.
1.2.1.3. Post Office / Fujitsu have the ability to insert transactions into a branch's accounts without
either a Postmaster's (a) knowledge or (b) consent.
1.2.1.4 Post Office / Fujitsu have the ability to amend or delete transactions entered by branch
staff on Horizon (and can do so in a way that is hidden from Postmasters).
1.2.2 More generally, we are informed that the Claimants also allege that Horizon makes errors in
accurately recording the transactions input by Postmasters.
1.2.3 We have also been told that the Claimants have made a variety of other allegations, principally
relating to Horizon providing a poor user experience or having insufficient safeguards against user
error. Testing the user experience and safeguards in place to protect against user error was—notwas
hot within the scope of our work.
1.3. Overview of Horizon and Our Approach
1.3.1 Horizon is the core operational and Electronic Point of Sales platform for the Post Office network.
Although the system has been in use for over 15 years, it is important to note that in 2010 there was
a migration from the system commonly referred to as ‘Legacy Horizon’ to the online variant that is
operated today {"HING-X" or "Horizon Online”). We have been informed by Fujitsu that the key
difference between the two variants is the way in which data is stored; local versus central. Below is
an overview of Horizon Online.
1.3.2. Adiagram showing the high level flow of data from transaction origination through to the Audit Store
is set out in section 3 (Background). In summary:-
1.3.2.1 Transactions conducted on Horizon terminals in branches are bundled into virtual baskets
(ie. one basket of transactions per customer) and securely transferred over the internet to
the Branch Database (BRDB). The BRDB is hosted on a central server farm operated by
Fujitsu (there is more than one BRDB server for resilience, and a set of gateway servers
collectively termed the Branch Access Layer (BAL) are also used).
1.3.2.2 Camelot, Paystation, and Post & Go transactions are conducted on their own separate
terminals (hence they are often referred to as Non-Counter transactions) and accepted into
the BRDB by way of Transaction Acknowledgments (TAs) on a daily basis.
132.3. The BRDB hokds tho five version ofthe transaction data used in day to day operations,
u also hosts other centralised data services to support reporting activities which are
Sawn fom summarised dats on BROB
1.3.2.4 From the BRDB, transaction data is fed into various other Post Office systems that then
connect to various third party systems.
1.3.2.5 The transaction records in the BRDB are also transferred to the Audit Store via the Audit
Server. The Audit Store is not involved in the live operation of a branch or Post Office's
business. Its the long term repository of transaction data. In the event of a challenge to
the integrity of any transaction data, the Audit Store is considered to be the master record.
In usual circumstances, it holds that data for 7 years but this has been extended to 10
years owing to the Group Litigation, and this is now reviewed on an annual basis.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 3
POL00139537
POL00139537
(Formatted: Font Bld
(Formatted: Font Bld
(Formatteds Fort: Bad
1.3.3. There are a number controls in place to protect the integrity of transaction data within Horizon (i.e.
from the Counter to the Audit Store):-
1.3.3.1. Counter transactions:-
(a) must balance to zero (e.g. the value of payment taken or given by the branch equals the
value of goods and services provided);
(b) are atomically written (i.e. entirely or not at all) to the BRDB so that there can be no
partial transactions; and
(©)are each given a unique Journal Sequence Number (JSN) of 1 greater than the
previous transaction so that the completeness (density) of the flow of transactions from
a particular branch can be checked when data is extracted from the Audit Store.
(@)are signed by a digital signature, which in accordance with commonly adopted
cryptography techniques, is used to secure the integrity of transactional data once it has
been initiated at the counter and allows all transactions to be checked for subsequent
interference once they have left the counter.
1.3.3.2 Non-Counter transactions:-
(a) must be accepted into the BRDB by branch staff by way of a TA in order to affect the
branch accounts. Branch staff can obtain reports from the Camelot, Paystation and
Post & Go terminals and compare those reports to the TAs that they are asked to
accept; and
(b) are digitally signed and subject to JSN fingerprinting by the Counter after being
accepted and those digitally signed files can be compared against raw data files that are
interfaced into the Audit Store in order to verify completeness when data is extracted
from the Audit Store (i.e. once they have been accepted by the Postmaster non-Counter
transactions are subject to the same data integrity controls as counter transactions).
Due to the nature of the allegations and the investigations that they have necessitated, we have
‘carried out work in phases and, within each phase, scope areas. The main body of this report
contains a summary of the work that we have been asked to do, and then designed, in each phase
‘and scope area and that shows how the overall project has expanded and developed. However, the
purpose of this Executive Summary is to outline our overall findings and apply them to the
allegations set out in section 1.2 above. We also provide a summary of the procedures performed
and the findings of each stage of the work.
1.3.5 In broad terms, we have performed five methods of investigation:-
1.3.5.1 _ reviewing Horizon technical documentation provided by Fujitsu;
1.3.5.2 asking questions of key Fujitsu staff;
1.3.5.3 _ reviewing transaction and event data generated by Horizon;
1.3.5.4 testing controls, such as walking through some of the Horizon processes on screen with
Fujitsu; and
1.3.5.5 _ analysis of Horizon's source code.
1.4 Our Findings
1.4.1 As the way data was handled by Horizon changed materially in 2010 with the introduction of Horizon
Online, legacy Horizon and Horizon Online need to be addressed separately.
1.4.2 Horizon Online
1.4.2.1 When branch staff lookup transaction records, the terminal in branch contacts the BRDB to
retrieve the necessary information. Given that branch accounts (as seen and operated in
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 4
POL00139537
POL00139537
POL00139537
POL00139537
branch by Postmasters) draw on data from the BRDB, additions, edits or deletions in the
BRDB could impact upon branch accounts.
1.4.2.2 The integrity of transaction data (as recorded in branch and then communicated via the
BRDB to the Audit Store) is protected in the following ways-
(a) Counter transactions (i.e. those transactions that originate in branch due to the actions
of branch staff) are given a unique number of 1 greater than the previous transaction so
that the data can be checked for missing or duplicate transactions;
(b) Counter transactions are also digitally signed (i.e. a unique “hash’ is applied to each
message) so that the accuracy and validity of the transaction data can also be checked;
(c) Non-Counter transactions (i.e. those generated by TAs that originate from Post Office)
must be accepted by branch staff before they enter into the BRDB. Once accepted, the
TAis digitally signed by the Counter and sent to the BRDB and then on to the Audit
Store;
(d) The original non-Counter raw data file in respect of non-Counter transactions is also
‘sent direct to the Audit Store and the digitally signed file from the BRDB can be
‘compared to the raw date file to check its integrity (this has been represented by Fujitsu
to be the case but not tested during the course of our work);
(e) When data is sent from the BRDB to the Audit Store via the Audit Server it is sealed
(while in the Audit Server) and a database of sealed files is maintained so that when.
data is subsequently retrieved from the Audit Store, its integrity can be checked. The
mechanism to do this (MD5 hashing algorithm) was previously a well adopted industry
standard mechanism. Although this is now a technically obsolete (in that for encryption
purposes it can be cracked relatively easily) significant technical expertise would still be
Fequired to exploit this vulneral
s
Indirectly the integrity of the transaction data is protected by the interface of BRDB data
‘ona regular basis to downstream systems (some of these feeds for a number of
products are in real time) and therefore interception and adjustment of data would have
to concurrently update BRDB and downstream data sources to remain in alignment and
prevent being subsequently spotted.
1.4.2.3. Setting aside the "remote access" issues discussed below, in our view:
(a) controls are in place to support the integrity of the processing of data from the Counter
into BRDB and the Audit Store. The level and design of the controls is proportionate to a
system of the size, scale, usage and data sensitivity, of Horizon;
(b) itis very unlikely that the data input by branch staff and as recorded in the BRDB and
the Audit Store would be incomplete or inaccurate.
(0)in the event that data was incomplete or inaccurately recorded, the controls in Horizon
provide tools that can be used to effectively identify such issues; and
(d) therefore a suitably skilled and qualified person could review the raw data from the Audit
‘Store to determine whether any data was incomplete or inaccurate (as a result of one of
the actions set out at 1.2), and when data is extracted from the Audit Store a number of
checks are performed to validate the completeness and accuracy of the data (none of
which have been flagged to us as failing by Post Office or Fujitsu in the extraction of
data relevant to this case).
1.4.2.4 In response to the allegations referred to in section 1.2 above, we would highlight that our
testing supports the following assertions (subject to the limitations in section 1.5 below) (ease! TAL]: Whats a fujtsu user? Do we simply ]
mean employee?
(a) Neither Post Office nor Fujitsu have the ability to log on remotely to a Horizon terminal {Commented [A2]: Yes, updated 5
ina branch so to conduct transactions. /_{ Commented [AS]: See above ]
(b) Neither Post Office nor Fujitsu employee’s nor administrators have the ability to conduct _/ {Commented [a4]: As above i}
transactions (either remotely or locally) under another employee 's ID (unless that _/ {Commented [AS]: See above )
‘employee shares their password but this would be a breach of operational procedure). {Commented [A6] As above )
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 5
14.25
14.2.6
14.27
14.28
14.2.9
1.4.2.10
14.2.1
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 6
‘As with the majority of computer systems there are some generic accounts (service
accounts, or other accounts named so not specifically assigned to a specific employee).
(©) Neither Post Office nor Fujitsu have the ability to inject additional transactions into a
branch's accounts, through normal system functionality, without either a Postmaster's
(a) knowledge or (b) consent (however see 1.4.2.5 below)
This is with one exception for a small group of Fujitsu users (30 users) who may do so
via Balancing Transactions (BTs):
(i) BTs do not require formal acceptance through the Horizon terminal by branch staff
(unlike transaction corrections and transaction acknowledgements) and so can be
pushed into the branch accounts by Fujitsu.
(ii) We reviewed a population of BTs since the inception of Horizon HNG-X extracted
from the Audit Store by [Fujitsu] and this highlighted that there had only been one
usage of BTs for general data correction in this period (at a branch not associated
with the allegations). BTs are used more routinely (although still infrequently ~ 1,643
instances during the period of approximately 6 years) to update a flag which can
become locked in the wrong binary setting (1, 0), preventing updates to stock units
within a branch.
Fujitsu (but not Post Office) has the ability to add, amend or delete transactions entered by
branch staff on Horizon via Privileged Users outside of specific functionality of the Horizon
application.
limited number of authorised Fujitsu personnel (19 at the Operating System layer and 26
at the database layer at the time of testing - May 2016) have sufficient privileges to
theoretically add / delete / change data in the BRDB (‘Privileged Users”). However, see
Paragraph 1.4.2.10 below regarding the segregation of access conditions. These users
may also have access to other systems, such as the Audit Store, however in relation to the
allegations, access to the BRDB is the most important as it is the BRDB that generates the
branch accounts and is the source of the data ultimately used by Post Office to investigate
shortalls.
Post Office personnel do not have this Privileged User access and Fujitsu and Post Office
have asserted that they have never had such access, however there is no historic record of
all the Privileged Users that there have ever been, so this cannot be verified,
Changes to a branch's transaction data in the BRDB by Privileged Users would be visible
to branch staff. The amended transaction would show up in transaction reports produced
in branch but would not be flagged as a change by a Privileged User and to the best of our
knowledge would appear like a normal transaction generated in branch (although see
paragraph 1.4.2.14 below around database logging of the actions of Privileged Users).
We would expect a system such as Horizon to have this type of Privileged User access as
it will be used to undertake maintenance on the system or to implement updates. Such
access comes with a risk of it being misused, either by accident or maliciously. It is
impossible to eliminate this risk entirely (within Horizon or any other IT system) and so
systems generally have controls over the use of privileged access so as to reduce the risk
of misuse or to make it detectable.
‘A.key control in Horizon is the segregation of access permissions between Privileged
Users who an access the BRDB and those users who may access the Key Management
Server (KMS). The KMS holds the digital keys that underpin the controls listed in
paragraph 1.4.2.2. Segregation of Privileged Users from KMS users means that a
Privileged User cannot get around the controls in paragraph 1.4.2.2 and therefore cannot
‘cover up any changes they make in the BRDB (Controls 1.4.2.2 b, ¢, d and e are of
particular importance). If a proper segregation of duties is in place, any changes by a
single Privileged User to the BRDB would be detectable in line with paragraphs 1.4.2.2 b,
¢,d and e above. This does not eliminate the risk of misuse entirely as there could be a
conspiracy between a Privileged user and a KMS user.
‘Through our enquiries, we have identified that 25 current Privileged Users have access to
the KMS such that they could theoretically cover up any changes made to the BRDB data.
POL00139537
POL00139537
_—{Commented [ATI See above }
~{ Commented [AB]: As above ]
‘(Commented [A9]: Please confirm of amend as appropriate)
‘(Commented [A10]: This was exracied by Fu
POL00139537
POL00139537
This is a failure by Fujitsu to implement its own segregation of duties policy. We are
unable to determine how long this vulnerability has existed as records of historic users are
not kept and as far as we are aware it has not been fixed by Fujitsu
1.4.2.12 Despite this vulnerability there are a number of other factors to consider in determining the
likelihood that actions by a Privileged User would be the cause of shortfalls in a branch,
1.4.2.13 First, Horizon has functionality (in the form of transaction corrections and balancing
transactions) to resolve the significant majority of imaginable operational errors in branch
or technical errors in Horizon. There is therefore little need to use privileged access to
manipulate transaction data to resolve an error — such use would be a last resort and
outside of mandated process (Balancing Transactions in particular are a deliberately
engineered process to support the exceptional corrective processing that less controlled
privileged access would typically be used for).
1.4.2.14 Fujitsu have advised us that there is a key split in dates around the audit trail of Privileged
User usage in July 2015:
(a) Pre July 2015 - only Privileged User log on and log offs were logged. However these
are expected to be of a low volume and would always (if valid ‘accesses’) be approved
by a documented access request form.
(b) Post July 2015 —fuller transaction audit logging was enabled by Fujitsu. A reviewer
could always see the last action by a Privileged User. If a Privileged User deleted their
actions, it would always leave a footprint of the deletion of logs. They could theoretically
remove what they have done, but they cannot remove that they have done something.
1.4.2.15 Fujitsu have advised us during the course of testing that turning off audit logs completely
would ‘break’ the application, causing it to crash and become un-functional, We have not
performed any testing to validate this assertion. A ‘Delete’ record on the audit trail is likely
to be highly unusual and easy to spot, and should facilitate further testing should it be
required.
1.4.2.16 Second, subject to the circumstances described in paragraph 1.4.2.17 immediately below,
any change to a branch's transactions in the BRDB by a Privileged User would be visible to
Post Office, Fujitsu and branch staff through the reports available from the system. Further,
Fujitsu has the data and, they have informed us, the expertise to identify that the root
cause of the change was the actions of a Privileged User via interrogation of the audit trails
maintained on such activity and a likely lack of coherency between amended data and the
records of that data within other systems. This means that if a Privileged User had
materially changed a branch's transaction data via unauthorised mechanisms, itis likely
that it would be spotted and resolved; any unauthorised Privileged User change, regardless
of materiality, would be identified by [describe the control] (1.4.2.1 73(d)) if transactions __- [Commented [A141]: Please add }
were checked. ‘(Commented [A12]: Check/update reference i}
1.4.2.17 Third, there is a theoretical risk of a Privileged User maliciously changing a branch's data
and successfully covering up that fact that those changes were made by the Privileged
User (due to Fujitsu's failure to segregate duties) but in our view the risk is unlikely to
crystallise because, in summary (a more detailed outline of the steps that a Privileged User
would have to take is at paragraph {add referencel}):in section 4.7.2 within the table) {Commented [A13]: Please add I
({a)__There are a limited number of Privileged Users (25 at the time of testing — June + (Formatted: Level 5 Number }
2076) who could theoretically (due to the segregation of duties breach between
database administration and the key management server) amend the Message Log for
‘one or more Counters in one or more branches and make the transaction/s amended,
look legitimate when itis retrieved from the Audit Store (through spoofing of the digital
signature);
(9(>)__However , this would require an existing systems administrator with a large
‘amount of technical expertise and systems knowledge, it would almost certainly require
‘a program to be installed onto the Horizon online system and a release process would
have to be bypassed in order for this to be installed maliciously (and avoid file integrity
checking controls operated by Fujitsu);
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
(i)(c)__There is also a time restriction of under 24 hours where the amendment of the
message log would have to complete by (likely to be a much smaller time window of
‘opportunity for the majority of transaction types where there is real-time or near real-
time processing -for example a number of postal transactions generate a ‘Track and
Trace’ message as they are carried out in the branch and these messages are sent to
Royal Mail/Parcetforce in near real time). This is because such real-time or near real-
time processing would copy data to other data sources and as a result expose data
which had been edited post such transfer, through lack of coherency between differing
data sources;
()(d)__ Further since July 2015 there will always be a record of a Privileged User
‘amending transactions in this way due to audit logging always logging DELETE actions
(even if a Privileged User deleted their actions this action would be logged). Pre-July
2015 there will be logs of Privileged User log and log offs to the Horizon system
databases, Privileged User log on / offs should be inherently rare, and would (if
legitimate accesses) always be accompanied by a request for access and appropriate
approvals; and
(\(e)__Although we have not performed procedures in this area, it would almost certainly
Fequire collusion with Post Office staff or Postmasters and the impacts would be to a
branch's accounts or Post Office's P&L, rather than benefitting them persona
0)1.4.2.18 In light of the above, it seems unlikely that a Privileged User would have the * (Formatted: Level 4 Number )
motivation or the ability to do this.
1.4.3 Legacy Horizon
1.4.3.1. The old Horizon system was named ‘Riposte’. This was a third party supplied product
which provided a similar functionality and service as the current Horizon system, but with a
number of important distinctions, in the context of the allegations detailed in Section 1.2 of
this Executive Summary above:
(a) On the Riposte system the data was held locally on a Branch Server and then replicated
toa cluster of central servers overnight. On HNG-X a key principle is branch data is only
held centrally (on the BRDB).
(b) A Cyclic Redundancy Check (CRC) function provided functionality similar to the digital
signature in that it provided a checksum capability. A checksum applies a
predetermined algorithm to a set of data to produce an output. When transmitted or held
with the data the checksum algorithm can be reapplied to validate whether the dataset
is complete and accurate. Unlike the digital signature applied by Horizon to Counter
transactions, this is not oryptographically secure and technically competent people can
generate CRCs simply (i.e. itis at greater risk of tampering).
(©) Once data was received from a branch server by the centralised server farm it would be
duplicated to all nodes in the farm (they were four clusters), meaning that challenges in
terms of altering the data would likely still exist (as data would now be different between
different nodes), causing a data integrity issue which would be likely to generate system
errors and be noticed if only one location was updated.
(d) Riposte was a third party provided system, meaning that neither Fujitsu nor Post Office
would have been likely to have ready access to the source code of the application.
Access to source code is an imperative requirement to being able to change the
underlying code or functionality of the application and the fact that Post Office or Fujitsu
would have probably had to approach the vendor in order to do so, would inherently
lower the risk of malicious changes to the application from privileged users (although
not necessarily the amendment of transactions data within the system).
(e) The Audit Store technology was largely identical to the instance supporting Horizon
HNG-X at the point of adoption (including the audit server)..
(f) Fujitsu have stated that due to (b), (c) and (d) above an application would need to be
inserted onto a branch server to manipulate transactions prior to them being distributed
to the datacentre and that whilst there is a theoretical possibility this could be achieved,
itis considered by Fujitsu to be a ‘remote possibility’
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 8
POL00139537
POL00139537
1.4.3.2 Our assessment of Riposte was largely dependent on interviews with Fujitsu SMEs and a
review of the available technical documentation. Given the time that has elapsed and
because there is no system to perform direct controls testing or substantive procedures on,
our ability to test this system was very limited,
1.4.3.3 We did perform some analysis of the baskets generated by Riposte which I identified (Commented [A14]: Reworded and added further detail.)
0.0015% of transactions had ‘errors’ and did not balance. It was subsequently
demonstrated these were false positives as set out in section 2.2.1 (ie. they were not
actually errors). As explained in section 4.5.2 below, further sample testing work was 4 (ee TA1S}; This sentence needs to be re-postioned
undertaken on the 48 items and it was established that none of the items originally noted A the erfors were notin fact erors, what testing dd we do
were issues when looking at fresh extracts of the system data — the original extracts . Recent cette esr Pree
provided were out of balance as they had not been extracted correctly. (Commented [A16]: Re-worded and added further detail.
= TAL7]: As the errors were afuncton of our
approach I'd prefer not to need to refer to this at all. The fact is
they worked is not?
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 9
POL00139537
POL00139537
1.5 List of Assumptions and Limitations
1.5.1 Where we have reviewed technical documentation we are reliant on the accuracy of the information
contained within the reports reviewed.
1.5.2 Where we have interviewed with Fujitsu and Post Office SMEs, we are reliant on the accuracy of the
information provided by these individuals.
1.5.3 We have only performed procedures over the areas of system functionality that we believe to be
relevant to the allegations set out in section 1.2, based upon the review of technical documentation
set out in Appendix 1 and interviews with Fujitsu and Post Office SMEs.
1.5.4 Our work has utilised sample testing to support the operation of controls and substantive procedures.
‘Whilst sample testing is a useful tool for assessing an overall population, there is a possibilty that
‘our samples and the conclusions we have derived from them are not representative of the overall
population. Samples selected to support the controls testing were aligned to Deloitte's standard
methodology for sample selection, based upon the frequency of operation of the control.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 10
2. Management Summary
2.1 Background and Scope
Post Office continues to respond to allegations that the Horizon IT system used to record transactions in Post
POL00139537
POL00139537
Office branches is defective and the processes associated with it are inadequate (the ;“Allegations”). The (Formatted: Font: Bold
Allegations span a period of over 15 years, some pre-date 2000 and others relate to the present day. In response
to the commencement of litigation proceedings, Deloitte was originally instructed to plan and execute procedures
and respond to three scope areas supporting Post Office's ability to understand how Horizon (HNG-X) has been
operated to prevent incorrect system operation that could have resulted in Postmaster detriment,
After the completion of the initial procedures (Phase 0 and Phase 1) over the three scope areas, it was identified
that further investigations would be required following the identification of exceptions in key controls tested by
Deloitte and the identification of key areas of risk. As such, Deloitte was instructed to provide responses to specific
questions to aid Post Office's ability to understand a number of areas within Horizon (HNG-X), namely:
1, The usage of Privileged Users and the configuration of audit logs (specifically over the actions of Privileged
Users, including audit logs over Riposte); and
2. The control environment over Non-Counter Transactions.
All procedures performed throughout the various phases of work have been designed to assess the level of
relevant risks surrounding financial loss to Postmasters or levels of reliance that can be placed on data used by
case handlers.
It should be noted that this report is to be considered a ‘living’ document, and in its current format represents the
final format following the completion of Phases 0 — 4. Future updates may be required if additional work is scoped
in at a future date.
2.1.1 Phase 0 and Phase 1
The scope areas over which Deloitte was originally instructed to perform procedures are as follows:
1. Scope Area 1 - To carry out an analysis of the relevant transaction logs for branches within the Complaint
Review & Mediation Scheme (the ;"Scheme”) to confirm, insofar as is possible, whether any bugs in the (Formatted: Font: Bold
Horizon system are revealed by the dataset which caused discrepancies in the accounting position for any
of those branches (see 2.2.1).
2. Scope Area 2- To carry out a full review of the use of Balancing Transactions throughout the lifetime of the
Horizon system and, insofar as is possible, to independently confirm from Horizon system records the
number and circumstance of their use (see 2.2.2).
3. Scope Area 3- To carry out a full review of the controls over the use and capability of authorised Fujitsu
personnel to create, amend or delete baskets within a sealed Audit Store throughout the lifetime of the
Horizon system, insofar as is possible (see 2.2.3).
Against each of these three scope areas the main body of this report will outline further:
Background and context in relation to this engagement;
2. The approach Deloitte have taken to planning the procedures;
3. The testing procedures Post Office have, upon Deloitte's recommendation and with agreement of Post
Office and their advisors, requested Deloitte undertake in response to the planning activities; and
4. Results of these testing procedures.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
2.1.2 Phase 2
This additional phase of work constituted ‘Phase 2, the ‘Further Investigations Phase’, whereby Deloitte performed
procedures it recommended and agreed with Post Office and their advisors in response to certain findings or
outcomes of ‘Phase 0’ and ‘Phase 1’ against the three scope areas examined during that phase.
The three additional scope areas were:
1. Additional Scope Area 1 - To perform an investigation of Privileged User Audit Logs from Branch
Database, the controls over them, and corresponding data extract and interrogation options (see 2.2.4)
2. Additional Scope Area 2~ To perform an investigation of analytics test results 1: ‘Identify Gaps in Audit
Logs Sequencing’, and 6: ‘Identify branches which are out of balance based on transactional data
available’ (see 2.2.5).
3. Additional Scope Area 3 To perform an investigation of controls over the integrity of non-Counter initiated
transactions, e.g. Paystation (see 2.2.6).
2.1.3 Phase 3
This additional phase of work constituted ‘Phase 3’, the ‘Non-Counter Transactions Phase’ whereby Deloitte
performed procedures it recommended and agreed with Post Office and their advisors in relation to Non-Counter
transactions to provide an assessment against the following questions:
1. Are there any gaps in the controls around Non-Counter transactions that could call into question the
Integrity of the data generated in relation to these transactions? (see 2.2.7)
2. Ifthere are gaps (see 2.2.8):
a. Could they be the cause of discrepancies in branch accounts (or could they mean that errors in
Horizon would not be revealed and those errors could then be the cause of discrepancies in
branch accounts); and
b. What is the risk of those gaps (or resulting discrepancies) materialising?
2.1.4 Phase 4
This additional phase of work constituted ‘Phase 4’, whereby Deloitte performed procedures it recommended and
agreed with Post Office and their advisors in relation to the Fujitsu Report ‘Database Security in Horizon Online’,
specifically:
1. Deloitte review of Fujitsu Report in conjunction with initial comments raised (see 2.2.9).
2. Workshop with appropriate Fujitsu resource (see 2.2.10) to:
©. Answer any outstanding comments / questions on the report.
d. Produce a detailed commentary on what steps would need to be taken to replace the message log,
as per Section 2 of the Fujitsu report (reproduced in Appendix 9)
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 12
POL00139537
POL00139537
2.2. Summary of Results
Asummary of the key controls tested and the results of that testing are set out below for all Phases (1-4 — Note
Phase 0 was a planning phase to determine the procedures needed for Phase 1 and so did not produce testing
results in its own right, hence it is not referenced below). A full set of agreed procedures tested and associated
results has been included in Section 4 of this report. These should be reviewed in tandem with the assumptions
and limitations that have been included in Section 5 and at the end of this Management Summary.
2.2.1 Phase 1 - Scope Area 1
Scope Area 1: To carry out an analysis of the relevant transaction logs for branches within the Scheme to confirm, insofar
4s is possible, whether any bugs in the Horizon system are revealed by the dataset which caused discrepancies in the
‘accounting position for any of those branches.
We have performed testing of key inherent system controls, together with a review of some of the source code
which supports the correct operation of the system in relation to ‘bugs’ (an error, flaw, failure or fault in a system
that causes it to produce an incorrect or unexpected result, or to behave in unintended ways) which, it is alleged,
could have caused shortfalls in branch accounts. These are controls which in our scoping discussion with Post
Office and Fujitsu have been determined to be fundamental to protecting the integrity of transaction data within the
system.
The key controls identified were:
1. Allltransactions on the Horizon Counter balance to zero — No Relevant Exceptions Noted.
2. Transactions are atomically (either in entirety, or not at all) written to the Branch Database — No Relevant
Exceptions Noted.
3. Digital signature controls are applied to the Message Journal during initiation of transfer to Branch
Database, ensuring the integrity of data. - No Relevant Exceptions Noted.
4. Access to mechanisms for managing the digital signatures are segregated from database administration
responsibilities (via system access rights restrictions), meaning that even if such access rights be abused
the digital signature that is included with every Counter and Kiosk transaction could not be spoofed. —
Relevant Exceptions Noted.
The exception noted was (at the point our testing was conducted — June 2016)
- 25 /Tusers (i.e. non-Branch staff) have access to mechanisms for managing the digital signatures (i.e.
access to the key management server and related technologies) and have database administration
responsibilities and access. This raises the theoretical risk of a user ‘spoofing’ the digital signature. It is
understood that for this risk to be realised, due to time limitations and volume of work required in order
to successfully ‘spoof the signature, @ program would have to be written (see section 2.2.10)"
5. Transaction Acceptance (in relation to interface file receipt for non-Counter originated interface files) is
required by Postmaster’s in order to be accepted into branch accounting records. — No Relevant
Exceptions Noted.
6. Recovery processes are in place for transactions in the event of connectivity failure. — Relevant Exceptions
Noted.
The exceptions noted were (at the point our testing was conducted):
- For one of the transaction recovery scenarios tested (whereby @ user session is automatically logged
out after a period of inactivity — 59 minutes after the session screen being locked), it was noted that Post
Office business rules (as would be enforced by the system) are in place for Horizon to automatically
commit unprocessed transactions to the Branch Database tables. This would have the effect of
committing any unprocessed transactions within a basket to the Branch Database. However when next
authenticating into Horizon, after being automatically logged out, the user is immediately presented with
a til receipt confirming that the transactions had been committed to the Branch Database.”
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 13
POL00139537
POL00139537
The first exception could lead to an increased risk that Postmasters are unaware of transactions being
posted in a power failure, although they are notified by receipt that this has occurred.
The above controls were tested at a recent point in time, as they are system controls. Given this limitation the
following procedures were undertaken over change control, as changes to the system are subject to the change
control process in place over the Horizon system:
4
‘A review of sources of assurance around change control was performed and it was noted that three
ISAE3402 reports were performed covering the period April-December in 2012, 2013 and 2014 by
professional services firm Ernst & Young LLP. The scope of the report was seen to include ‘Fujitsu's
system of IT Infrastructure Services supporting Post Office's POLSAP and HNG-X applications’. Within
each reports’ scope was a control objective relating to change management, and in each report reviewed
no deviations were noted against this objective, or any related controls.
Further it was identified through change documentation review, and discussion with Fujitsu SMEs that
various controls tested had specifically changed, either since inception of HNG-X (replacing Riposte) in
2010, or changed during the lifespan of Riposte. Please see Appendix 5 for a full list of controls tested and
‘a view on whether the controls have been consistent.
In summary, the major change affecting the operation of controls in relation to this scope area is the creation of the
Branch Database (BRDB) to replace individual Branch Databases (2010). This change fundamentally altered the
operation of many controls tested. Whilst Fujitsu have attempted to give a view on controls in operation in the
Riposte system, much of the knowledge of this system has left the business.
Whilst not causing an exception to one of the controls covered by the scope of our work, the following exception
relating to General IT Controls over H.
on was noted:
(One Fujitsu user has access to both development and live environments of HNG-X, contravening typically
expected segregation between environments in a change control process (note: this is a separate point to.
the segregation of duties issue).
As a result they would have the technical knowledge and access rights to make changes to Horizon HNG-
X which were not sanctioned by management — this ability could theoretically be used to subvert the
controls over Horizon, or system functionality, for personal gain or other malicious outcomes.
Fujitsu stated that:
“Whilst we appreciate that there is lack of segregation of duties here for the <specified user> between Live
‘and Development, it is felt that there is a strong business need for this access for <specified user>. He
provides 4th line/final line support for the audit service and is in regular weekly contact with the Security
‘audit team to assist them in resolving queries with the audit service. He is the lead designer/developer and
system owner.
‘Additionally there are compensating controls in place such as CCTV, and the auditing (performed by
Fujitsu) we have in place (and the technical controls around not being able to change audit items for 7
years) acts as @ safeguard against anyone with access trying to change anything in an unauthorised way.”
In addition to the system controls noted above, the following analytics procedures were performed to support this
scope area
1
Review of the case data available (relevant to allegations) for transactions indicating items of risk from a
system functionality perspective. The analytical procedures outlined in Appendix 6 were undertaken, and a
number of items of interest were originally noted (see Appendix 6a for details and summary of initial
findings). One such initial finding of note was that ‘there were 48 (0.0015%) session ids from a total of
3,124,140 which were out of balance based on the transactional data received. Those 48 session ids out of
balance related to 18 distinct branches from 118 in total. The session ids out of balance were all pre
system migration to HNG-X in 2010. As explained in section 4.5.2 below, further sample testing work was
Undertaken on the 48 items and it was established that none of the items originally noted were issues when
looking at fresh extracts of the system data — the original extracts provided were out of balance as they had
not been extracted correctly.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 4
POL00139537
POL00139537
2.2.2 Phase 1 - Scope Area 2
Scope Area 2: To carry out a full review of the use of Balancing Transactions throughout the lifetime of the Horizon
system, insofar as is possible, to independently confirm from Horizon system records the number and circumstance of their
In performing our procedures against this scope area, we worked with Post Office and Fujitsu to identify other
methods of posting transactions which could impact a branch accounts, without the knowledge of the Postmaster
which, in the context of the allegations, could present similar risks to that of Balancing Transactions. This
highlighted other areas of potential risk, such as:
1. ‘Global Users’ — being central users who can access branches remotely for support purposes (by remotely
this means accessing a branch terminal without being physically present in the branch). Critically we have
been informed by Fujitsu and have verified as far as possible, that such users are not able to post
transactions remotely (i.e. view only access), and they can only make edits when physically in the branch.
2. Database and Operating System Users with sufficient privileges to post transactions directly to the
database from outside of Horizon, thereby bypassing the system controls to manage activity.
‘These areas were brought into scope.
In summary across each of these areas, including Balancing Transactions, controls were noted to be operating
effectively. In particular, based on the procedures we have performed:
1. Logical Access rights to these sensitive functions had been appropriately restricted. - No Relevant
Exceptions Noted.
2. Any writes by the Shared Service Centre (SSC) to the BRDB must be audited. The mechanism for inserting
a correction record must ensure that the auditing of that action performed must be atomic. - No Relevant
Exceptions Noted.
3. Access to these mechanisms is segregated from key management respon: s (via system access
rights restrictions), meaning that should such access rights be abused the digital signature that is included
with every Counter and Kiosk transaction could not be spoofed. — Relevant Exceptions Noted.
The exception noted (at the point our testing was conducted — June 2016) was:
= '25 users have access to mechanisms for managing the digital signatures and have database
administration responsibilities and access. This raises the theoretical risk of a user ‘spoofing’ the digital
signature. It is understood that for this risk to be realised, due to time limitations and volume of work
required in order to successfully ‘spoof the signature, a program would have to be written.”
4. Itwas also noted via a control walkthrough that any Transaction Corrections created by Post Office
Finance must be accepted by a Postmaster at branch prior to affecting branch accounts. — No Relevant
Exceptions Noted.
5. Inherent system controls around Global Users were tested, notably that Global users with a Role of ADMIN
‘cannot log onto any Branch other than Global (including Remote access controls to branch infrastructure
(e.g. Counter). = No Relevant Exceptions Noted.
6. SSC will have privileges of only inserting balancing / correcting transactions to relevant tables in the
database. SSC will not have any privileges to update or delete records in the database. — Relevant
exception noted.
The exception noted was:
- ‘The control wording is not accurate. 26' users (at the time of testing - May 2016) are granted extended
privileges which enable them to update / delete records. However the contro! is operating in line with
* This number, 26, is different to the previous figure of 25 quoted around privileged users with access to both BRDB
and the Key Management Server (thus contravening expected segregation of duties), as it relates purely to SSC
users with privileged access rights, contravening the above control wording,
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 15
POL00139537
POL00139537
management's expectations. Access to the privileged role is restricted to users explicitly authorised for
this access. User actions are audit logged, but not proactively reviewed. ’ (Note: This has been raised as
an exception as it is stating that 26 users within SSC also have backend database access allowing them
to directly insert transactions).
The above controls were tested at a recent point in time, as they are system controls. Given the limitations around
this, the following procedures were undertaken over change control, as changes to the system are subject to the
change control process in place over the Horizon system:
1. Areview of sources of assurance around change control was performed and it was noted that three
ISAE3402 reports were performed covering the period April-December in 2012, 2013 and 2014 by
professional services firm Ernst & Young. The scope of the report was seen to include ‘Fujitsu's system of
IT Infrastructure Services supporting Post Office's POLSAP and HNG-X applications’. Within each reports
‘scope was a control objective relating to change management, and in each report reviewed no deviations
were noted against this objective, or any related controls.
2. Further, it was identified through change documentation review and discussion with Fujitsu SMEs that
various controls tested had specifically changed, either since inception of HNG-X (replacing Riposte) in
2010, or changed during the lifespan of Riposte. Please see Appendix 5 for a full list of controls tested and
a view on whether the controls have been consistent.
In summary, the major change affecting the operation of the controls tested is the creation of the BRDB to replace
individual Branch Databases (2010). This change fundamentally altered the operation of many of the controls
tested. Itis not known whether balancing transactions existed in Riposte, as much of the knowledge of this system
has left the business.
An exception was noted relating to a core General IT Control exception around Segregation of Duties; please see
section 2.2.1 above where this issue is described in detail
In addition to the system controls noted above, the following analytics procedures were performed to support this
scope area:
1. All available audit data over the use of Balancing Transactions was inspected (12/03/2010 — 28/05/2016)
and it was noted that only 1 ‘true’ Balancing Transaction was inserted, it did not relate to a branch involved
in the allegations, and the branch was made aware of the transaction prior to insertion. The insertion was
four new lines of transactions each for a value of £4000. Other uses of the tool used to insert Balancing
Transactions were noted, however they do not affect transactional data and relate to the update of a
specific flag (SU) to enable continued processing.
2. Additional context around the usage of this tool was obtained from ticket review:
a. The original TFS helpdesk ticket was obtained which is the legacy system used by Post Office
where branch incidents are recorded. The TFS ticket 2091569 was reviewed and it was noted that
this had been raised by Anthony Vasse (Service Desk) on 02/03/2010 and transferred to Cheryl
Card (SSC Product Specialist). Within the incident ticket it was noted that the after investigation the
clerk had incorrectly doubled a transfer of stock of £4000.00 to £8000.00; therefore creating an
incorrect loss to the branch of £400.00. It was noted this issue was required to be fixed by
17103/2010 as the branch was due to roll into the next transaction period; therefore meaning that
the branch required a fix to ensure the accounts were correctly recorded. An update was provided
by Cheryl Card on 11/03/2010 confirming that the issue had been resolved using the transaction
tool to insert transactions into the BRDB_RX_REP_SESSION and
BRDB_RX_EPOSS_TRANSACTIONS tables to reverse the incorrect £4000.00 charge. The ticket
confirmed that the Post Master had been advised to print a balance snapshot of the accounts
before and after the transaction correction took place to ensure the transaction had been reversed
correctly. A subsequent update was provided confirming that the balances had been correctly
fixed. The ticket was subsequently closed on 04/04/2010.
b. Evidence was obtained of the Peak Incident ticket raised in relation to this balancing transaction
performed. Incident ticket 'PC0195561' was raised by Lorraine Elliot (Service Desk) on 15/04/2010
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 6
POL00139537
POL00139537
(Formatted: Fort 10p
POL00139537
POL00139537
in relation to a Post Master attempting to transfer £4000.00 when the system crashed resulting in
the post master being issued with 2 x £400.00 receipts.
. An OCP ticket was also raised which is the solution management system used by Fujitsu which
tracks issues and resolutions. From this OCP reference 25882 it could be seen that the branch had
performed a transfer out of stock for the value of £4000.00 but due to a system error this had
incorrectly doubled in value creating an imbalance of £4000. Therefore, a balancing transaction of
£4000.00 was required to correct the loss using the transaction correction tool. This was approved
by Emma Langfield (Post Office) on 10/03/2010 at 15:33. From this OCP ticket, it could be seen
that this incident was raised by Cheryl Card (SSC Product Specialist) on 10/03/2010 who
subsequently performed the work and inserted the balancing transaction.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 7
2.2.3 Phase 1 - Scope Area 3
Scope Area 3: To carry out a full review of the controls over the user and capability of authorised Fujitsu personnel to
create, amend or delete baskets within a sealed Audit Store throughout the lifetime of the Horizon system, insofar as is
possible.
In performing our procedures against this scope area, we worked with Post Office and Fujitsu to identify how
baskets of transactions flow from creation at the counter, through the sealed Audit Store (See Section 3,
Background, for a high level overview).
Further, we tested controls over the accuracy, completeness and validity of the flow of data into the Audit Store,
which is used as the master data source for audit purposes (and hence the primary source of data in relation to
these cases). We identified the following key controls during scoping as being fundamental to ensuring the
accuracy, completeness and validity of this data flow:
1. The flow of data from counter to Audit Store was mapped at a detailed level (See Section 1 for high level
overview). Security controls over data at rest (when held in an intermediate location), and completeness
and accuracy controls over data in transit (transfer of data from one holding location to another) including
exception monitoring were tested. — No Relevant Exceptions Noted,
2. Security controls over access to the audit servers, and Audit Store were tested, specifically that there are
separate roles and a clear segregation between audit server administration staff, who administer the
architecture, and Fujitsu service audit staff, who have access to retrieve data from the Audit Store via an
audit workstation. — No Relevant Exceptions Noted.
3. Access to mechanisms for managing the digital signatures are segregated from database administration
responsibilities (via system access rights restrictions), meaning that even if such access rights be abused
the digital signature that is included with every Counter and Kiosk transaction could not be spoofed. —
Relevant Exceptions Noted.
The exception noted (at the time of testing - June 2016) was:
- '25 IT users have access to mechanisms for managing the digital signatures and have database
administration responsibilities and access. This raises the theoretical risk of a user ‘spoofing’ the digital
signature. It is understood that for this risk to be realised, due to time limitations and volume of work
required in order to successfully ‘spoof’ the signature, a program would have to be written.’
4. The ATS (Audit Track Scheduler) collects files for sealing and records a log of its activities to the ATD
(Audit Track Database). In sealing a file the seal is generated using a MD5 hash algorithm. Once a file has
had a seal calculated the file is written to Centera and details are stored in the Audit Track Seal Database.
— No Relevant Exceptions Noted.
5. Audit tracks and seals are copied to the equivalent import area on the remote audit server as part of Audit
server overnight schedule. On arrival, the sealer on the remote audit server recalculates the seal value of
the imported audit track and compares it with the original value in the imported seal file. Assuming they
match, the file is then written to the remote Audit archive. If the seals do not match, the Audit track and seal
file are moved to a holding area and an event is raised. Manual investigation is necessary to investigate the
‘cause of the discrepancy (Which could be indicative of tampering with the data in between the two Audit
servers). - No Relevant Exceptions Noted.
6. Audit tracks that are gathered at one data centre are replicated to the Audit server at the remote data
centre. — No Relevant Exceptions Noted.
7. As Audit tracks are retrieved from the archive, their seals are checked (by re-application of the MDS
message digest function) to ensure that the source data has not been tampered with while it was stored in
the archive. The digital signature check is also applied at this point to ensure data integrity. — No Relevant
Exceptions Noted (reader should be mindful of the more general limitations of the MDS algorithm as
highlighted within the executive summary).
8. The remote directories from which the Audit Server gathers Audit Tracks is configured so that only the
‘Audit Server (or an administrator who has been explicitly given permission) is able to delete files in the
directory. — No Relevant Exceptions Noted.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 8
POL00139537
POL00139537
{Commented [A18}: Please capitalise throughout the report)
~{ Commented [A19]: Done throughout
9. All users (including administrators) of the Audit Workstation and Audit Server log onto systems using two
factor authentication in conjunction with the HNG-X Active Directory system. Each user is uniquely
identifiable. ~ No Relevant Exceptions Noted.
10. The following operating system level events on the Audit Server are audited via the System Management
event monitoring facilities:
a. Log on/Log off (including unsuccessful log on attempts)
b. File Creation, Deletion and Modification (on selected files)
©. Modifications to system configuration (incl. software configuration and account details)
d. System start up and shut down
fe. Change of user rights.
Relevant Exceptions Noted:
- ‘Review of the audit settings for the Audit Server noted that the audit policy change which relates to
change of user rights was set to log success events only, with failure not enabled.’ (i.e. a failed attempt
to update audit policy would not be reported on)
‘The above controls were tested at a recent point in time, as they are system controls. Given the limitations around
this the following procedures were undertaken over change control, as changes to the system are subject to the
change control process in place over the Horizon system:
1. Areview of sources of assurance around change control was performed and it was noted that three
ISAE3402 reports were performed covering the period April-December in 2012, 2013 and 2014 by
professional services firm Ernst & Young. The scope of the report was seen to include ‘Fujitsu's system of
IT Infrastructure Services supporting Post Office's POLSAP and HNG-X applications’. Within each reports
‘scope was a control objective relating to change management, and in each report reviewed no deviations
were noted against this objective, or any related controls.
2. Further, it was identified through change documentation review and discussion with Fujitsu SMEs that
various controls tested had specifically changed, either since inception of HNG-X (replacing Riposte) in
2010, or changed during the lifespan of Riposte. Please see Appendix 5 for a full ist of controls tested and
a view on whether the controls have been consistent.
3. Insummary, it is understood controls relating to the audit server and store have been relatively consistent
throughout the lifetime of Riposte and Horizon. It should be noted that whilst Fujitsu have attempted to give
a view on controls in operation in the Riposte system, much of the knowledge of this system has left the
business.
‘An exception was noted relating to a core General IT Control exception around Segregation of Duties, please see
Section 2.2.1 above where this issue is described in detail,
In addition to the system controls noted above, the following procedures were performed to support this scope
area’
1. The process of Joumial-Sequence-Numbering (each transaction is given a unique ID of 1 greater than the
previous transaction), whereby completeness checks are performed over these JSNs, is an optional setting
within the system (which assures the completeness of messages from the counter in the Audit Store).
Testing supported that this control has been enabled since 2010 and not turned off since inception in 2010,
This was determined via interrogation of the supporting log file settings.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 19
POL00139537
POL00139537
2.2.4 Phase 2 - Scope Area 1
Scope Area 1: Investigation of Privileged User Audit Logs from Branch Database, the controls over them, and
corresponding data extract and interrogation options.
In performing our procedures against this scope area, we held a workshop with Post Office and Fujitsu in which the
approach was decided for future phases, and centred on a report produced by Fujitsu on how Privileged User
access is controlled by Fujitsu.
The key facts determined at this stage of the investigation (to be illuminated further by the production of the Fujitsu
report) were that:
1. Regardless of access rights to amend and delete audit logs, the digital signature controls should still allow
for the detection of data which had been modified, deleted or inserted subsequent to receipt from the
Counter (subject to the next point).
2. There are a limited number of users (25 at the time of testing — June 2016) who could theoretically, due to
a segregation of duties breach between database administration and the key management server, amend
the Message Log for one or more Counters in one or more branches and make the transaction/s amended,
look legitimate when it is retrieved from the Audit Store (through spoofing of the digital signature)
3. However to do this would require an existing systems administrator with a large amount of technical
expertise and systems knowledge, it would require a program to be installed onto the Horizon Online
‘system, and a release process would have to be bypassed in order for this to be installed maliciously (and
avoid file integrity checking controls operated by Fujitsu).
Given the above findings, Fujitsu were requested to prepare a paper outlining the steps a Privileged User would
need to take to subvert the digital signature controls, and this has been documented within our write up of Phase 4
(see below).
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 20
POL00139537
POL00139537
(Commented [AZO]: Captain
(Commented [A23]: Done
2.2.5 Phase 2 - Scope Area 2
Scope Area
nvestigation of analytics test results 1: ‘Identify Gaps in Audit Logs Sequencing’, and 6: ‘Identify branches
which are out of balance based on transactional data available’.
We performed further investigations over certain analytics test results from Phase 1. Analytic 1 — ‘Identify gaps in
audit log sequencing’ and Analytic 6 ‘Identify branches which are out of balance based on transactional data
available (should not be possible based on inherent system controls)’. These further procedures highlighted in each
case that there was a reason for each of the results and they were not therefore indicative of a problem with the
operation of the Horizon system,
The original challenges highlighted were:
1. Analytic 1 — In order to identify gaps in audit log sequencing, the transactions data was sorted into
ascending order on session id and txn id, and any gaps in the sequence at both the session and txn level
were identified. There were 212,372 (1.60%) gaps in audit log sequencing from a total of 13,666,238
transactions.
Analytic 6 - In order to identify branches which were out of balance based on transactional data available
(which should not be possible based on inherent system controls), the transactions data was summarised
by branch (Group) and session id and those session ids that do not sum to zero were identified, and are
ordered by balance descending. The data used was filtered for transaction mode ‘SC’ only. There were 48
(0.0015%) session ids from a total of 3,124,140 which were out of balance based on the transactional data
received. Those 48 session ids out of balance related to 18 distinct branches from 118 in total. The session
ids out of balance were all pre system migration to HNG-X in 2010.
The results after responding to the challenges in the original analytic were:
1. Analytic 1—The analytic logic was revised following discussion with Fujitsu and following this revision there
were no gaps in audit log sequencing
Analytic 6— There was a logic error in the production of the extracts originally provided by Fujitsu. A
‘sample of 15 items which were errored in the original data was investigated to confirm they were fixed
when looking at the revised data provided by Fujitsu and confirmed the root cause was issues with the data
extraction rather than the underlying data within the system.
2
Given the original discrepancies in these analytics were resolved, no further work against this area was
recommended or required.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
POL00139537
POL00139537
2.26 Phase 2- Scope Area 3
[Scope Area 3: Investigation of controls over the integrity of non-Counter initiated transactions, e.g. Paystation. ]
I. (Formatted: Fort: (Defaut) +Body (val) )
Following our work for Phase 1, we understood that non-Counter transactions were not subject to the same level of
controls as counter transactions for the following three non-Counter sources:
1. Camelot (Current) (Formatted: Font: (Default) +Body (Arial) )
2. Paystation (Current)
3. Post and Go (Historic)
Akkey area of focus in the operation of these controls is the ability of the Postmaster to validate that the data being
received from these extemal data sources is correct, and this was incorporated within the procedures that we
suggested for inclusion and testing in Phase 3. In addition, a diagram highlighting the understanding gained of the
data flows, and the related controls understood from technical documentation has been included within Appendix 8.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 2
POL00139537
POL00139537
2.2.7 Phase 3 - Question 1
‘Question 1: Are there any gaps in the controls around non-Counter transactions that could call into question the integrity
cof the data generated in relation to these transactions?
This piece of work investigated any risk of data relating to non-Counter transactions not being complete / accurate ___ f ‘Commented [A22]: Change to non-Counter transactions
and being at risk of interference. It can be split into two parts:- throughout the report
, I a ~{ Commented [423]: Done
1. the controls applicable to data in respect of non-Counter transactions before it is accepted into Horizon by
branch staff; and (Formatted: Font: (Default) Arial
2. the controls applicable to data in respect of non-Counter transactions after it is accepted into Horizon by . (Formatted: Font: (Default) Arial
branch staff. »\[ Formatted: Font: (Default) Arial
Our procedures centred on:- »( Formatted: Font: (Default) Arial
1. understanding data flows and controls over the current reconciliation process and how Transaction
‘Acknowledgements are utilised;
testing key reconciliation controls between key data sources within the data flow;
performing detailed walkthroughs of the Transaction Acceptance process to confirm the granularity of the
information Postmasters are provided with; and
4. performing an analytics pilot to assess the feasibility of performing a reconciliation between raw data files
received by PODG (Post Office Data Gateway — the interface hub utilised by Post Office for receipt of
external data files from Third Parties).
2.
3.
The first key area of interest from a controls perspective in relation to the completeness and accuracy of the flow of
data is around the sending, processing by, and subsequent receipt of data from third parties. The primary control in
relation to this is the requirement for Postmasters to accept ‘Transaction Acknowledgements’ in relation to such
data before it is accepted into their accounts, but the formalisation of the processes and controls ensuring
Postmasters reconcile their financial positions before accepting transactions has not been enforced. We have
reviewed supporting documentation, primarily from the Horizon Online Help, which Post Office has informed us is.
available to branch staff which describe a number of reports which are available to assist Postmasters to reconcile
data received from third parties. We have not validated this.
Originally it was theotised there was a second key area of interest, being that no digital signature is applied to non-
Counter transactions, potentially opening up this category of transactions to greater risk of interference subsequent { Formatted: Font: (Default) Arial
to processing into the BRDB (as the digital signature is the primary control preventing downstream editing of
transactions once they have been processed by the Counter). However, further discussion with Fujitsu established
that when the BRDB receives non-Counter transaction data, it pushes it down to the counter for acceptance by the (Formatted: Font: (Default) Arial
Postmaster, at which point the Counter digitally signs the acknowledgement of the transaction and therefore in
theory a reconciliation between these digitally signed TAs and the raw data files received from the third parties
(which are interfaced into the Audit Store) should also be possible mitigating this risk. Note however that this (Formatted: Font: (Default) Arial
means the data is digitally signed only from the point it is accepted by the Postmasters and not prior to that point,
making visibility and reconciliation of the data back to source by the Postmaster at the point of acceptance even
more important.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
2.2.8 Phase 3 - Question 2
Question 2: I/there are gaps:
a) Could they be the cause of discrepancies in branch accounts (or could they mean that errors in Horizon would not
be revealed and those errors could then be the cause of discrepancies in branch accounts); and
b) What is the risk of those gaps (or resulting discrepancies) materialising?
In theory, if a third party incorrectly reflected the data they had received from a non-Counter system, and this
incorrect total was then downloaded into the Branch accounts, then in the absence of formal controls to reconcile
data transmitted to the third party, back to data received, this could cause discrepancies in the branch accounts.
However, Postmasters are able to challenge any transactions that they do not recognise / agree with through the
aforementioned Transaction Acknowledgement process.
Without a full investigation of the controls at the third parties, and any other mitigating controls which may exist, itis
difficult to quantify the risk exposure. However, the control which Post Office relies on to mitigate this risk is
Transaction Acknowledgements and, as noted above, Post Office has informed us that branches have access to.
reports which assist with the reconciliation of data received from third parties (see 2.2.7).
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 24
POL00139537
POL00139537
POL00139537
POL00139537
2.2.9 Phase 4- Question 1
[Question
’o perform a review of Fujitsu Report in conjunction with initial comments raised. ]
For this specific scope area our procedures centred on performing a review of the report produced by Fujitsu and (Formatted: Font: (Default) Arial
the initial comments raised by Womble, Bond Dickinson. __{ Formatted: Font: (Defaut) Aral
In the context of the allegations, this was to:~
1. provide Post Office with an independent view of the Fujitsu report;
2. provide the expertise required to challenge it; and
3. identify the residual risks
Following the review we provided an email which set out our views in line with the above. This was then
supplemented by the workshop and challenge described in the next section.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
2.2.10 Phase 4 - Question 2
Question 2: Hold a workshop with appropriate Figitsu resource to:
@) Answer any outstanding comments / questions on the report; and
5) Produce a detailed commentary on what steps would need 10 be taken to replace the message log, as per section
2.2 of the Fujitsu report
For this specific scope area our procedures centred on holding a workshop with appropriate Fujitsu personnel in
order to answer specific questions on the Fujitsu report and address any outstanding comments. Further, we
provided a detailed commentary on next steps required to replace the message log as per section 2 of the Fujitsu
report (See Appendix 9).
Our review of the Fujitsu deliverable highlighted that Fujitsu have acknowledged (in drawing the conclusions below,
we have taken what Fujitsu have said on its merit) that;
1. There is a theoretical risk of Privileged Users making edits to the Branch Database without leaving an audit
trail, as Privileged Users can, in theory, delete the audit trail of any edits they may have made to the
Branch Database. Fujitsu have however also represented that audit trails cannot be ‘switched off; in their
entirety as this would 'break’ the Horizon application i.e. any such attempt would be immediately
identifiable.
2. The audit trails have been limited to logon/logoff events prior to 2015. This limits the value of the audit trail
in trying to determine any misuse (or indeed legitimate use, of Privileged User accounts prior to this date).
3. Based upon the above findings, the value of further work over Privileged Users is di ed due to the
lack of granularity of audit trail pre-2015, and the capability of users to only leave a trace audit trail (their
final delete action — as described in 1. above).
4, The Fujitsu report further highlighted that to take advantage of these deficiencies would require the use of a
program. See section 2.2.4 above for further details.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 26
POL00139537
POL00139537
(Formatted: Font (Default) Ail )
-{Formatted: Font: (Default) Arial )
2.3 Fundamental Limitations and Assumptions
Any procedures performed during our work against each scope area are subject to a number of assumptions and
limitations,
2.3.1 Phase 0 and Phase 1
1.
4.
It should be noted that controls tested/to be tested for Phase 1 relating to the system were tested on the
system (HNG-X) operating at the time of our review, It must be noted that at the time of some allegations
the Legacy Horizon system was stil in use. In performing our testing we have commented on the evidence
that supports the view that the control was operating in the relevant period where we were able to do so.
Further, all analytical procedures for Phase 1 were subject to the availabilty of data / evidence. A full
transactional audit log is available since October 2007. We have not interrogated this entire data set. Our
procedures have utilised data provided by Fujitsu extracted from this broader population. Also any controls
testing is subject to the availability of evidence.
Finally, our work performed for Phase 0 and Phase 1 are specifically limited to the three scope areas
outlined in the scope section above. Our work is focused on identifying, and performing procedures to
validate, the facts in relation to the Horizon system with regard to the three scope areas as above.
Please see Section 5 for a full list of assumptions and limitations.
2.3.2 Phase 2
1
5.
It should be noted that procedures performed for Phase 2 relating to the system were tested on the system
(HING-X) operating at the time of our review. It must be noted that at the time of some allegations the
Legacy Horizon system was stil in use. In performing our testing we have commented on the evidence that
‘supports the view that the control was operating in the relevant period where we were able to do so.
Non counter transactions work was dependent on technical documentation and our understanding was
based on these documents. Subsequent conversations with Fujitsu highlighted that in a number of cases
this documentation was out of date. Certain controls were originally scoped in for testing and then de-
scoped as a result of these discrepancies within the available technical documentation (see section 4.6.2
for details)
Further, all analytical procedures for Phase 2 were subject to the availabilty of data / evidence, and
reliance was placed on Fujitsu around the successful extraction of data. Procedures were performed to
support the completeness of the population of data provided by Fujitsu.
Finally, our work performed for Phase 2 was specifically limited to the scope areas outlined in the scope
section above,
Please see Section 5 for a full list of assumptions and limitations.
2.3.3 Phase 3
1
It should be noted that procedures performed for Phase 3 relating to the system were tested on the system
(HING-X) operating at the time of our review. It must be noted that at the time of some allegations the
Legacy Horizon system was stil in use. In performing our testing we have commented on the evidence that
supports the view that the control was operating in the relevant period where we were able to do so.
Further, all analytical procedures for Phase 3 were subject to the availability of data / evidence. A full
transactional audit log is available since October 2007. We have not interrogated this entire data set. Our
procedures have utilised data provided by Fujitsu extracted from this broader population. Also, any controls
testing is subject to the availability of evidence.
‘Our identification of non-Counter Transaction flows has been dependent on the availability of technical
documentation, and the accuracy of the facts and figures communicated within this technical
documentation.
‘Our testing of reporting available to Postmasters in branches was based upon testing at the Model Office
facility within Finsbury Dials, and we are therefore reliant on this being representative of the live
environment.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 2
POL00139537
POL00139537
POL00139537
POL00139537
5. We have not been asked to validate or test controls at third parties such as Wincor, Ingenico and Camelot,
which would be a key component in managing the risks associated with completeness and accuracy of the
data flows associated with non-Counter transactions.
6. Finally, our work performed for Phase 3 is specifically limited to the scope areas outlined in the scope
section above.
2.3.4 Phase 4
1. Our work for this phase was based on a report produced by Fujitsu. We reviewed that report and raised
questions on it which were answered, but we did not conduct any further testing on it. = 1A261: im not sure thes enely accurate, ]
2. Further, our work performed for Phase 4 is specifically limited to the scope areas outlined in the scope ‘ ore create ates, Dey wore: GRE TOR
fscfioni above, (Commented [A25]: Agreed, updated }
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 28
3. Background
The Horizon system was developed by Fujitsu and is the core operational and Electronic Point of Sales (EPOS)
platform for the Post Office network. Whilst formal benchmarking data is not available, it is considered by
interviewed stakeholders to be one of the largest computer systems in existence in terms of the number of
transactions it processes on a daily basis, and it sits at the core of a complex systems estate with multiple
interfaces with other Post Office systems as well as third party systems.
The system has been in use for over 15 years and is audited by multiple parties for statutory audit, service auditor
reporting, and accreditation purposes. Given its size and scale, and the considerable intellectual property that
Fujitsu has built within the system, in relation to this piece of work, there is a significant quantity of documentation
articulating how the various modules and features comprising the system operate. Much of this documentation has
formed the focus of our review during Phase 0 and Phase 1 of the work.
In understanding Horizon it has been important to distinguish between features which are of relevance today, and
the time period to which that relevance applies. In particular we would highlight the migration between the system
commonly referred to as Legacy Horizon, and the online variant operated today, referred to as Horizon HNG-X.
The key difference between these two iterations of the platform is the way data is stored. In the Legacy version
data was replicated between the data centre and the branches (this system was called Riposte), whilst over the
course of 2010 a migration event occurred whereby the Riposte system was replaced by the Branch Database
model, the Branch Database being a data centre only database storing the transactional and accounting data for
the branches, with a Counter application held locally within the branch which connects to the Branch Database as
necessary. This change may have influenced the relevance of some of the controls in existence at the present time
and care must be taken to consider this when prioritising procedures.
The Branch Database (BRDB) is also key to understanding the flows of data to the Audit Store given that it acts as
a hub for all branch transactional and accounting records. The diagram below provides clarity on the high level flow
of data from transaction origination through to the Audit Store:
Indicative Data Flow Overview
om
=
ssa
oe
\
a =e CEA ac oceeSeand toe
ee] ‘be ‘Transaction Accepted’
‘dented toy ar plod ito re At Server or to,
Ie} ee) BRDB, Dala is heldin the Audit Server for approximately
fore
‘Aut Store Ato approtmately 5 day
Server tho Aut Sire
poranenty (a
‘sequencing
audit Upon rogues! rom Post Office, Fujteu at staff can uso
Desktop the Audi Desktop fo query the Audit Store to extract
‘specified data. Upon extraction fem Audit Store — Audit
Doshi, the integrity of ho data ls confimod via tho iia
‘signature and seal check
co ‘ACD produced wih te requested Aud Data.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT Ey
POL00139537
POL00139537
This diagram shows most but not all of the data feeds associated with the Branch Database, but does show all of
the direct transactional feeds to the Branch Database. It demonstrates the convergence of the data flows at the
Branch database and the chain of subsequent data movements.
In considering these diverse data feeds a key concept is those which use a public key infrastructure (Counter) for
completeness and accuracy of the message journals to the Branch Database, versus those which use a
combination of interface controls (header and footer records) for completeness, combined with manual
interventions from Branch staff around the completeness of the associated data (being the data feeds external to
the Horizon infrastructure e.g. Paystation).
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
3 Scope and Approach
3.1. Scope of Work
3.1.1 Phase 1
We structured our work around the three scope areas Post Office asked us to review, as shown in the table below:
‘Scope Area #
4
Post Office Instruction
Post Office consider instructing a suitably qualified
party to carry out an analysis of the relevant
transaetion logs for branches within the Scheme to
confirm, insofar as is possible, whether any bugs in
the Horizon system are revealed by the dataset which
caused discrepancies in the accounting position for
any of those branches.
Post Office instruct a suitably qualified party to carry
‘out a full review of the use of Balancing Transactions
throughout the lifetime of the Horizon system, insofar
as is possible, to independently confirm from Horizon
system records the number and circumstance of their
use,
Post Office instruct a suitably qualified party to carry
out a full review of the controls over the user and
capability of authorised Fujitsu personnel to create,
amend or delete baskets within a sealed Audit Store
throughout the lifetime of the Horizon system, insofar
as is possible.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
Proposal
Post Office will instruct Deloitte to
determine whether such an
analysis/review is feasible, and if it
is, to provide an indication of the
cost, time and process that would
be incurred.
Post Office will instruct Deloitte to
determine whether such an
analysis/review is feasible, and if it
is, to provide an indication of the
cost, time and process that would
be incurred.
Post Office will instruct Deloitte to
undertake this review, throughout
the lifetime of the Horizon system,
insofar as is possible.
POL00139537
POL00139537
3.1.2 Phase 2
The three additional scope areas agreed with Post Office were:
Scope Area# Description Proposal
1 Investigation of Privileged User Audit Logs from Hold a series of workshops and
Branch Database, the controls over them, and discussion meetings with Fujitsu
‘corresponding data extract and interrogation options. personnel in order to discuss the
relevant controls and audit trail
configurations.
2 Investigation of analytics test results 1: ‘Identify Gaps Pick a sample of 15 items from
in Audit Logs Sequencing’, and 6: ‘Identify branches __ each analytic population for further
which are out of balance based on transactional data _ investigation in conjunction with
available’ Post Office investigators and
Fujitsu.
3 Investigation of controls over the integrity of non- Hold a series of workshops and
Counter initiated transactions e.g. Paystation. discussion meetings with Fujitsu
personnel in order to discuss the
relevant controls and audit trail
configuration.
The approach to ‘Phase 2' was to hold workshops with relevant stakeholders from Post Office to support the
delivery of the analysis described
3.1.3 Phase 3
This additional phase of work constituted ‘Phase 3', the ‘Non-Counter Transactions Phase’, whereby Deloitte
performed agreed procedures in relation to non-Counter transactions to provide an assessment, as fully as
possible within the time allocated to this exercise, on the factors to consider, controls, and risks in answering the
following questions:
1, Are there any gaps in the controls around Non-Counter Transactions that could call into question the
integrity of the data generated in relation to these transactions?
2. Ifthere are gaps:
a. Could they be the cause of discrepancies in branch accounts (or could they mean that errors in Horizon
would not be revealed and those errors could then be the cause of discrepancies in branch accounts); and
b. What is the risk of those gaps (or resulting discrepancies) materialising?
We performed the following procedures:
1. Provisional workshop to corroborate understanding of data flows and validate the existence and
‘completeness of controls over the current reconciliation process, and how Transaction Acknowledgements,
are utilised.
2, Review and test key reconciliation controls between key data sources within the data flow as highlighted
within separate table (Appendix 8).
3. Perform detail walkthrough of the Transaction Acceptance (TA) process to confirm the granularity of
information the Postmaster is provided with. Perform procedures to corroborate a TA is required for all
Non-Counter Transactions.
4. Analytics pilot to assess feasibility and then perform reconciliation between raw data files received by
PODG and the interpretation of these Non-Counter Transactions into the BRDB transaction files.
The approach to ‘Phase 3’ was to hold workshops and meetings with relevant stakeholders from Post Office and
Fujitsu to support the delivery of the analysis described above.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 32
POL00139537
POL00139537
3.1.4 Phase 4
Deloitte performed procedures in relation to the Fujitsu Report ‘Database Security in Horizon Online’, specifically:
1. Deloitte review of Fujitsu Report in conjunction with initial comments raised.
2. Workshop with appropriate Fujitsu resource to:
a. Answer any outstanding comments / questions on the report.
b. Produce a detailed commentary on what steps would need to be taken to replace the message log, as
per section 2 of the Fujitsu report (reproduced in Appendix 9).
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
3.2. Summary of Approach and Work Performed
The work was performed in multiple phases. Phase 0 was ‘Discovery’ and Phase 1 was ‘Testing’ of the original
scope. Additional phases of work were commissioned and specific agreed procedures were performed to provide
larity over the initial findings from Phase 1, against the three scope areas performed during that phase (Phases 2-
4).
3.2.1 Phase 0 - Discovery
This phase of work constituted ‘the ‘Discovery Phase’, whereby Deloitte performed initial enquiries and
investigations across the three scope areas to identify procedures which Post Office could undertake for each
scope area.
In performing work for Phase 0, Deloitte conducted the following procedures:
1. Reviewed relevant technical documentation as requested and provided by Fujitsu/Post Office during the
course of this engagement.
2. Held workshops with Post Office Finance staff in Chesterfield on 14th and 23rd March, and 18th April 2016.
3. Held workshop with Fujitsu staff in Bracknell on 14th April 2016.
4, Held workshop with Case Handlers in Chesterfield on 8th April 2016
The aim of these procedures was’
1. To enhance Deloitte's previous understanding of the key concepts, processes, risks and controls
associated with the Horizon system, relevant to the three scope areas highlighted above (see 3.1.1)
2. To identify the fundamental limitations and assumptions which will need to be made and considered by
management when deciding which procedures they wish to conduct during Phase 1 (see 1.5, 2.3 and
Section 5).
3. Asa result of “1” and °2” above, the identification of possible procedures which could be adopted by
management in order to provide assurance over the potential risks posed in relation to the three scope
areas highlighted above (see 3.1.1). We identified three core procedure types which were then utilised
during Phase 1
a. Analytics — Procedures using data tools to analyse large volumes of data for particular
characteristics of interest or the absence thereof. For example verification for a given set of case
data that the JSN sequence is complete.
b. Controls review and testing — Verification through walkthrough, enquiry, and subsequent evidence
gathering that the controls relating to the Horizon system operate as expected or otherwise, to
support in mitigation of the associated theoretical risks. For example testing that the population of
Fujitsu users who can administer the Oracle DB estate underpinning Horizon directly is
appropriate.
©. Substantive procedures — Direct inspection of selected samples or information for confirmation of
its qualities or characteristics of note (Analytics is an example of ‘ull population’ substantive
procedures). In this instance the main substantive procedures expected will be inspection of
source code to verify that the system functions as expected.
3.2.2 Phase 1 - Testing
Deloitte conducted the following procedures:
1. Performed on-site review and visit to Fujitsu and tested controls between May 2016 and September 2016.
2. Reviewed case data provided by Post Office case handlers and tested for characteristics which could
illustrate the Horizon system has not operated as expected
3. Reviewed relevant technical documentation as requested and provided by Fujitsu/Post Office during the
course of this engagement.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 34
POL00139537
POL00139537
3.2.3 Phase 2~ Further Investigations Phase
The objective of the further investigations phase was to obtain sufficient information and background on the specific
areas in response to findings in cer
scope areas looked at in Phase 1, and report on the associated findings
from these procedures.
In performing work for Phase 2, Deloitte conducted the following procedures:
1
Held workshops with Fujitsu personnel to investigate the controls over Privileged User Audit Logs from
Branch Database
2. Tested a sample of items from each analytic population, 1: ‘Identify Gaps in Audit Logs Sequencing’, and 6:
3.
‘Identify branches which are out of balance based on transactional data available’
Held workshops with Fujitsu personnel to investigate controls over the integrity of non-Counter
transactions, e.g. Paystation
The aim of these procedures was to answer the following questions, posed by Post Office:
6.
What exact information is logged by the Privileged User Audit Logs?
Would this logged information definitively reveal that:
a. APrivileged User had done something that could change a branch's accounts in the real-world; and
b. What that Privileged User had done (e.g. does it show the change in such a way that it could be
identified and either isolated or reversed out)?
If the Privileged User Audit Logs would not reveal all actions by Privileged Users that could affect branch
accounts, please describe (in detail) the types of ways that a Privileged User could amend a branch's
accounts in a way that would not leave behind a footprint of their activity?
What is the root cause of the gaps identified in analytics 1 and 6?
a. Are these root causes indicative of problems in Horizon / evidence of flaws in Horizon’s controls,
around the core audit process?
b. Would these issues cause discrepancies in the branch accounts?
‘Are there any gaps in the controls around Non-Counter Transactions that could call into question the
integrity of the data generated in relation to these transactions?
If there are gaps:
a. Could they be the cause of discrepancies in branch accounts (or could they mean that errors in
Horizon would not be revealed and those errors could then be the cause of discrepancies in branch
accounts); and
b. Whats the risk of those gaps (or resulting discrepancies) materialising?
3.2.4 Phase 3—Non-Counter Transactions
This additional phase of work constituted ‘Phase 3', the ‘Non-Counter Transactions Phase’ whereby Deloitte
performed procedures agreed with Post Office in relation to Non-Counter Transactions to provide an assessment
as fully as possible in the time allotted by the exercise, on the factors to consider, controls and risks, in answering
the following questions:
1
‘Are there any gaps in the controls around non-Counter transactions that could call into question the
Integrity of the data generated in relation to these transactions?
If there are gaps:
a. Could they be the cause of discrepancies in branch accounts (or could they mean that errors in
Horizon would not be revealed and those errors could then be the cause of discrepancies in branch
accounts); and
b. Whats the risk of those gaps (or resulting discrepancies) materiali
ing?
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
‘The procedures performed were as follows’
1.
Held initial workshop to corroborate understanding of data flows and validate the existence and
‘completeness of controls over the current reconciliation process and how Transaction Acknowledgements
are utilised.
Reviewed and tested key reconciliation controls between key data sources within the data flow as
ighlighted within separate table.
Performed detailed walkthrough of the Transaction Acceptance (TA) process to confirm the granularity of
the information the Postmaster is provided with. Performed procedures to corroborate a TA is required for
all Non-Counter Transactions.
Performed analytics pilot to assess feasibility and then performed reconciliation between raw data files
received by PODG and the interpretation of these non-Counter transactions into the BRDB transaction
files.
3.2.5 Phase 4~— Privileged User Access
This additional phase of work constituted ‘Phase 4’, whereby Deloitte performed procedures agreed with Post
Office in relation to the Fujitsu Report ‘Database Sec
in Horizon Online’, including a review of the report and a
subsequent workshop to clarify understanding on certain areas,
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
4. Work Performed
4.1 Summary of Work Performed
For each scope area for Phase 0 and 1 we have laid out our work performed as follows:
1. Setting the Scene — We have described in a narrative format the work we have performed, and our
understanding of the relevant subject matter.
2. A tabular format of the procedures performed in Phase 0, and the key learnings relevant to our planning.
3. The procedures which have been performed in Phase 1 and the findings obtained from the performance of
those procedures.
For each scope area for Phases 2 to 4 we have laid out our work performed as follows:
1. Setting the Scene — We have described in a narrative format the work we have performed, and our
understanding of the relevant subject matter.
2. The procedures which have been performed in this phase and the findings obtained from the performance
of those procedures.
4.2: Phase 0 and 1 - Scope Area 1
Scope Area 1: To carry out an analysis of the relevant transaction logs for branches within the Scheme to confirm, insofar
4s is possible, whether any bugs in the Horizon system are revealed by the dataset which caused discrepancies in the
‘accounting position for any of those branches.
4.2.1 Work Performed, and Analysis Results
Our procedures centred on the workshops and documentation reviews highlighted in Section 3.2.1 and 3.2.2. In
addition, specific to this scope area we reviewed the case data which had been provided to us, and assessed the
feasibility of performing analytics over the available case data in order to ascertain whether evidence of the system
not operating in accordance with expectations could be identified
(ur work highlighted a number of fundamental system controls designed to ensure the integrity of processing, and
correct functionality. Key principles/items identified include:
1. Ata holistic level, IT change control processes and procedures operate over the Horizon system, and the
related controls around testing, approval, and the overall software development lifecycle should provide
assurance over the correct operation of the system. The operational effectiveness of this control framework
has, since 2012 been assessed on a regular basis, via Service Auditor Reports (ISAE3402 produced by
EY). Further sources of assurance is provided by regular IS027001 certification and ongoing audit and
attestation regime, and ongoing IT focused Internal Audit and External Audit activity. Errors in the system
would be more likely in an environment with inadequate change control procedures, and the level of
‘comfort that can be gained over such controls provides a view on the inherent risk of such errors.
2. There are some fundamental inherent system controls, specifically designed to support correct processing
within the system. These include:
‘a. Joumal Sequence Numbers (JSNs) are applied to each Counter transaction within the Horizon
system. These JSNs are generated using Public Key Encryption and are used by each piece of
Counter Hardware to ‘digitally sign’ a transaction. The digital signature is passed to all latter stages
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT a7
POL00139537
POL00139537
POL00139537
POL00139537
of the infrastructure including the Audit Store (and beyond). This si
critical control points over the data captured:
ining process provides two
i. The completeness (‘density’) of the flow of transactions for a particular Branch, meaning that
completeness of the audit trail behind transactions can be ascertained.
ii, The validity and accuracy of the transactions as any changes to a transaction after the
application of the digital signature would invalidate the signature. The Audit Store extraction
routines check for this at the point of extraction.
b. Transaction Acknowledgements (TAs) — Whilst JSNs are a powerful inherent system control over
the correct origination and completeness of the Message Journals from the Counter, other feeds to
the Branch Database are not subject to this control. However as an alternative control mechanism
the interface files, which issue data to the Branch Database contain Header and Footer records.
Which allows Horizon to automatically check the completeness of data. In addition, Branch staff
accept these interface files into their Branch accounts via Transaction Acknowledgements,
meaning these staff are directly responsible for verification that the data being received into the
Branch Database via sources outside the Counter are valid and accurate.
©. Recovery Procedures ~ In acknowledging that the Horizon system is dependent upon connectivity
between a data centre, a branch, and various third parties, seven recovery processes have been
designed to combat instances when a loss of connection causes an error in the completion of
transaction processes. The recovery process used depends on the nature of the connectivity issue.
Recovery scripts designed by Post Office are an integral part of this process.
d. The commit of transactions to the Branch Database is all performed as one Oracle DB write action,
ie. itis atomic in nature,
e. Alltransactions from the Counter are checked by Horizon to ensure they balance to zero (double
entry principle). If the Counter attempts to write a transaction which does not balance to zero, this,
should be rejected via the Counter.
{External file feeds (i.e. for data feeds not from the Counter or Kiosks) are received by the Branch
Database and into the database by Horizon before being sent to the Audit Store. Alongside this,
data flow, the raw interface files are also processed directly to the Audit Store.
3. Alongside the inherent system controls available for our review, we identified two tranches of data analytics
work that we performed to assess the potential risk of system failure or ‘bugs’:
b. Using the case data provided (for all the branches with links to Postmasters for which this case
relates via Post Office's file sharing platform ‘Huddle’) we performed specific profiling tests which
support the operation of these inherent controls or rule out the occurrence of particular risky events
(e.g.{non balancing session IDs] from within the relevant data set
©. The BRSS (Branch Support Database) is a copy of the main Branch Database used by Fujitsu staff
for support purposes. This database contains the most recent six months’ worth of transactional
data (the Branch database itself contains only 5 days’ worth of unsummarised transactions). Using
tools already available via Fujitsu we were able to profile this data to look for characteristics of
potential risk (such as recovery situations, Balancing Transactions, transactions posted by staff not
related to a Branch etc),
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 38
4.2.2. Summary Table of Phase 0 Procedures and Conclusions
Post Office Instruction
Procedures Performed
What we have discovered
Carry out an analysis of the relevant
transaction logs for branches within the
‘Scheme to confirm, insofar as possible,
whether any bugs in the Horizon system
are revealed by the dataset which
caused discrepancies in the accounting
position for any of those branches.
Identified relevant business processes and
areas of interest.
Review of existing technical documentation and
identification of key inherent system controls.
Workshops with Case Handlers (Post Office) in
order to understand how to interpret the case
data.
Workshops with Systems Architects (Fujitsu) in
order to understand how to interpret the case
data and technical documentation.
Awalkthrough on-screen as to how the system
works.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
There are a set of inherent system controls within Horizon targeting the
completeness, accuracy and validity of the flow of data from Counter and
other in-branch data sources, onwards to Branch Database, and ultimately
the Audit Store.
Central to these controls is the digital signature applied to each message
journal of branch transactional data sent from Counter to Branch Database
and beyond.
Connectivity issues are managed via Recovery processes, and so issues
with loss of connectivity have been built into the design of the system from
the outset, in recognition this could be an area of potential data corruption
or loss.
A strategy for our analytic procedures was to profile the available case
data for characteristics of interest in relation to the correct operation of the
system.
POL00139537
POL00139537
POL00139537
POL00139537
4.2.3 Phase 1 Procedures
Performed Procedures
Procedures Findings
Controls Controls
1. Validate inherent system controls around: ‘1a. No issues noted
a) All transactions on Counter system balancing to zero
b) Atomic write and commit controls of transactions to the Branch Database.
¢) Digital Signature controls applied to Message Journal during initiation of transfer
to Branch Database.
Transaction Acceptance in relation to interface file receipt for non-Counter ‘1c, Issue noted. ‘25 IT users (at the time of test) have access to mechanisms
originated interface files. for managing the digital signatures and have database administration
) Recovery of transactions in the event of connectivity failure. responsibilities and access. This raises the theoretical risk of a user ‘spoofing’
2. Review of existing sources of assurance around Change Control and confirmation of 1 digital signature. itis understood that for this risk to be realised, due to time
relevant coverage — plus targeted testing to attempt to identify changes relevant to _I/mitations and volume of work required in order to successfully ‘spoof’ the
the key controls on Horizon ‘signature, a program would have to be written.’
‘Ib. No issues noted
d)
Data 1d. No issues noted
3. Review case data for transactions indicating items of risk from a system functionality 1e. issue noted. ‘For one of the transaction recovery scenarios tested as part of
perspective (e.g. recovery transactions are present in the case data). See Appendix ecovery scenario 6, whereby a user session is automatically logged out after a
2and6. \period of inactivity, it was confirmed that Post Office business rules are in place
4. Review of population of Balancing Transactions (to validate population of Balancing {COMfgured within Horizon) for Horizon to automatically commit unprocessed
Transactions relative to total transaction volumes). transactions to the Branch Database tables. As part of the walkthrough testing
performed, it was observed that Horizon is configured to automatically lock a
Substantive user account after 15 minutes of inactivity, at which point the user is required to
ye-enter their user credentials. After a further period of 59 minutes of inactivity,
5. Review source code on screen at Fujitsu headquarters which supports the key Horizon is configured to automatically log the user out, ending a user session
inherent control operation around: and committing any unprocessed transactions within a basket to the Branch
a) All transactions on counter balancing to zero
b) Atomic write and commit controls of transactions to the Branch Database.
¢) Digital Signature controls applied to Message Journal during initiation of transfer
Database. When next authenticating into Horizon, after being automatically
jogged out, the user is immediately presented with a till receipt confirming that
to Branch Database. the transactions had been committed to the Branch Database. From review of
d) Transaction Acceptance in relation to interface file receipt for non-Counter the printed receipt, an enhancement point was noted in that there is scope for
originated interface files the till receipt to include further detail to the user, highlighting that an
e) Recovery of transactions in the event of connectivity failure.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
Procedures
Findings I
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
unattended transaction had automatically been committed by Horizon to provide
greater visibility to Post Masters that a recovery session had been initiated.’
2. Issue noted. See Appendix 5 for details of which controls have been subject
to change. ‘It was noted one user has access to both development and live
environments of HNG-X.’
Fujitsu stated that;
“Whilst we appreciate that there is lack of segregation of duties here for
<specified user> between Live and Development, itis felt that there is a strong
business need for this access for <specified user>. He provides 4th line/final
line support for the audit service and is in regular weekly contact with the
‘Security audit team to assist them in resolving queries with the audit service. He
is the lead designer/developer and system owner.
Additionally there are compensating controls in place such as CCTV, and the
auditing we have in place (and the technical controls around not being able to
‘change audit items for 7 years) acts as a safeguard against anyone with access
trying to change anything in an unauthorised way."
Data
3. Review of the case data available (relevant to allegations) for transactions
indicating items of theoretical risk from a system functionality perspective.
The analytical procedures outlined in Appendix 6 were undertaken, and a
number of items of interest were noted, see Appendix 6a for details and
summary of findings. One finding of note is that there were 48 (0.0015%)
session ids from a total of 3,124,140 which were out of balance based on
the transactional data received. Those 48 session ids out of balance related
to 18 distinct branches from 118 in total. The session ids out of balance
were all pre system migration to HNG-X in 2010
The results after responding to the challenges in the original analytic were:
POL00139537
POL00139537
Procedures
Findings
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
a) Analytic 1 The analytic logic was revised following discussion with
Fujitsu and following this revision there were no gaps in audit log
sequencing,
Analytic 6 - There was a logic error in the production of the extracts
originally provided by Fujitsu. A sample of 15 items which were errored
in the original data was investigated to confirm they were fixed when
looking at the revised data provided by Fujitsu and confirmed the root
cause was issues with the data extraction rather than the underlying
data within the system.
b)
Given the original discrepancies in these analytics have been explained away,
no further work against this area is recommended or required.
4. No issues noted. 1 Balancing Transaction identified (in the period where
data was available for review 12/03/2010 — 28/05/2016) which did not relate
toa branch involved in the allegations and was appropriately approved and
governed.
‘Substantive
5a. No issues noted
5b. No issues noted
5c. No issues noted
5d. No issues noted
5e We have observed a theoretical risk in relation to this control
Post Office have the ability to create their own APADC transactions. So they
an create a product, and a transaction and then also specify the recovery
script which would be initiated when any of the recovery scenarios kick in.
POL00139537
POL00139537
POL00139537
POL00139537
Procedures I Findings ]
his could, theoretically cause an issue where a new product is created, and
1e recovery script is then coded to do nothing. So if the cashier sold that
oduct for the customer, and then in the event of the connection going down
nd the recovery process kicking in - no rollbacks or roll-forwards would happen
in this case.
ur testing has shown no evidence which would suggest this has happened,
Ithough we have not specifically performed procedures to verify this.
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 43
4.3, Phase 0 and 1 - Scope Area 2
Scope Area 2: Carry out a full review of the use of Balancing Transactions throughout the lifetime of the Horizon system,
insofar as is possible, to independently confirm from Horizon system records the number and circumstance of their use.
4.3.1 Work Performed, and Analysis Results
Our procedures centred on the workshops and documentation reviews highlighted in Section 3.2.1 and 3.2.2
above,
Balancing Transactions are exceptional processes used by Fujitsu support staff to correct exceptional errors in
system processing/fix issues or bugs in the recording of data. The inherent controls around the integrity of data
recording are designed to ensure that such issues manifest themselves in the data on an exceptionally rare basis,
and therefore volumes of Balancing Transactions should be inherently low (substantive procedures performed
support management representation there has been only 1 true Balancing Transaction since 2010).
Balancing Transactions should not be confused with Transaction Corrections which is a more routine process,
used to centrally correct issues by Post Office Finance staff, which are then subject to Transaction
Acknowledgement by Postmasters prior to being accepted into a branches accounts.
Fujitsu have advised that whilst there have been several hundred instances (circa 1,650) of Balancing Transactions
used throughout the known lifecycle of the HNG-X system, only one has been a complex usage of the functionality,
to correct a bug around double writing of a transaction, immediately subsequent to the migration to Horizon HNG-
X. The remainder relate to switching a flag on Stock Units (SU are a Counter concept to allocate transactions to a
particular ‘sub-branch’ area to enable users to process transactions on that stock unit (following communications
failure Stock Units occasionally become locked to editing).
Our work has highlighted a number of fundamental controls which are designed within the system to control the
use of Balancing Transactions and to ensure that the use of Balancing Transactions is recorded. Key
principles/items identified include
1
Balancing Transactions are the only transactions that do not either originate at Branch, or have to be
acknowledged / accepted by branch. As such the use of Balancing Transactions is very rare.
‘Any writes by Fujitsu Support to BRDB must be audited (record created and stored in Audit Store). The
mechanism for inserting a correction record must ensure that the auditing of that action is atomic with the
insert of the record.
Fujitsu Support with access to post Balancing Transactions cannot amend the related audit files.
Fujitsu Support will have privileges of only inserting balancing / correcting transactions to relevant tables in
the database. They will not have any privileges to update or delete records in the database.
There are various inherent system controls around Balancing Transactions, notably that each Balancing
Transaction must only contain 1 transaction (single SQL statement) and the balancing transaction module
can only be run by limited appropriate personnel.
In assessing the risk posed by Balancing Transactions we have also enquired as to additional ‘privileged account’
transactions which could also be used to post transactions centrally without the knowledge of Branch staff. These
enquiries have highlighted two additional areas of consideration against this risk:
4
Global Users of the Horizon System — These are users that can log on at any HNG-X Branch, and are used
for a number of purposes including global user administration.
Other ‘Privileged Users’ — At various layers of the Horizon infrastructure there exist accounts with privileged
access rights which could be used to modify or insert data relevant to transactions at branches should they
not be adequately controlled. For example a Privileged User account on the Oracle DB forming the nucleus
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 44
POL00139537
POL00139537
of the Branch Database could insert transactions directly onto the backend (effectively Balancing
Transactions are a specialised ‘legitimised’ way of using such Oracle access).
A number of key controls were noted to operate on Horizon to mitigate these broader ‘Privileged User’ risks:
1. Global Users are subject to two fundamental controls reducing their risks. The first is that they cannot post
transactions in a branch unless they are physically present at that branch. The second is that the Global
‘Admins can only create users and there is therefore a Segregation of Duties between users who can
create users, and users who can post transactions.
2. Privileged User activity is monitored via log files which are transferred to the Audit Store following
aggregation by the Event Management System which collects log files from across the Horizon estate,
Regardless of this control, for transactions related to the Counter and Kiosks any attempt to insert
transactions into the database by an individual with the privileged access rights to do so, would be
identifiable due to the Digital Signature process applied to Message Journals from the Counter. To
circumvent this a ‘Privileged User’ would require the relevant access rights to the key management
infrastructure which controls the Digital Signature processes, and therefore the segregation of duties
between such infrastructure and the remaining Branch infrastructure is a key control
Alongside the inherent system controls around balancing transactions, and the completeness and accuracy of the
audit log of Balancing Transactions available for our review, there are various data analytics procedures which can
be performed:
1. As discussed above Fujitsu highlighted that while the Balancing Transaction module has been used
several hundred times in the past 7.5 years, only 1 of these uses has been a ‘complex’ Balancing
Transaction. As described in our procedures below, analytical procedures were performed to validate the
number and nature of Balancing Transactions which have been performed in:
a. The Case Data available
b. The BRSS most recent 6 months data available
c. The full period of data available — (7.5 years)
‘Sample (or full population) testing could then be performed to validate that for all Balancing Transaction records
(except the 1 known Balancing Transaction, for which the branch was aware of) no transactional postings were
made using Balancing Transactions.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 4
POL00139537
POL00139537
4.3.2. Summary Table of Phase 0 Procedures and Conclusions
Post Office Instruction Procedures Performed
What we have discovered
to carry out a full review of the use of interest.
Balancing Transactions throughout the
circumstance of their use.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
Post Office instruct a suitably qualified party Identified relevant business processes and areas of
lifetime of the Horizon system, insofar asis I Review of existing technical documentation and
possible, to independently confirm from identification of key inherent system controls, and
Horizon system records the number and support in interpreting the transactional data
Workshops with Systems Architects (Fujitsu) in order
to understand how to interpret the technical
documentation and the availability of Audit Store data.
Awalkthrough on-screen as to how the system works.
There are a sequence of inherent system controls within
Horizon which ensure Balancing Transactions have certain
standard characteristics, use of them is controlled, and usage
is recorded in the Audit Store.
Other privileged access rights which would lead to similar
potential risks of central posting of transactions with
Postmaster knowledge, such as Global Users, and ‘Privileged
User accounts on the Horizon infrastructure, are also subject
to key controls, most notably the segregation of duties between
the key infrastructure for digital signatures and the
infrastructure supporting the processing of Branch transactions.
These controls have been tested at a point in time.
The strategy adopted across our analytical procedures was to
Investigate a sample / full population of all Balancing
Transaction records found to validate the branch was aware of
their usage / no transactional postings were made in the
balancing transaction
POL00139537
POL00139537
4.3.3 Phase 1 Procedures
Performed Procedures
Procedures
Findings
Controls
1
2,
z
Validate inherent system controls around Balancing Transactions (See Appendix 3
for detail of controls A — C).
Validate any writes by Fujitsu support staff to BRDB must be audited. The
mechanism for inserting a correction record must ensure that the auditing of that
action performed is atomic.
Validate Fujitsu support staff cannot amend audit files for Balancing Transactions.
Validate Fujitsu support staff only have privileges for only inserting balancing /
correcting transactions to relevant tables in the database. Confirm SSC do not have
any privileges to update or delete records in the database
Validate broader population of Balancing Transaction controls identified. (See
Appendix 3a for detail of controls A — N)
Validate there is a Segregation of Duties between BRDB Administration and Key
Management Software Administration,
Validate inherent system controls around Global Users, notably that Global users
with a Role of ADMIN cannot log onto to any Branch other than Global (Including
Remote access controls to branch infrastructure (e.g. Counter))
Data
8.
9.
Review case data for Balancing Transactions to validate population of Balancing
Transactions relative to total transaction volumes (Balancing transactions should be
inherently rare, and only deployed in response to actual loss/bugs in code.)
Review full population (already extracted by Fujitsu - 7.5 years) of balancing
transactions (sample vs full population depending on feasibility) to validate the
branch was aware of their usage / no transactional postings were made in the
balancing transaction.
Substantive
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
Controls
1. No issues noted
No issues noted
No issues noted
RON
‘Through discussion with Fujitsu management it was noted that the
contro! wording is not accurate. A small number of users are granted
extended privileges which enable them to update / delete records.
However in mitigation this access is appropriately restricted to
authorised users. Users do not have the ability to bypass this role
restriction by running SUDO command. User actions are audit logged
but not proactively reviewed, and all instances of users being granted
the APPSUPP role are also captured in audit logs.’
5. Issues noted for control 2A and 2C.
2a finding noted - ‘Through discussion with Fujitsu management it was
noted that the contro! wording is not accurate. A smaill number of users are
granted extended privileges which enable them to update / delete records.
However in mitigation this access is appropriately restricted to authorised
users. Users do not have the ability to bypass this role restriction by running
SUDO command. User actions are audit logged but not proactively reviewed,
and all instances of users being granted the APPSUPP role are also
captured in audit logs."
2c finding noted — ‘The technical document <DESAPPLLD0142> is
inaccurate. The user OPS$SUPPORTTOOLUSER does require update
access to the table BRDB_BRANCH_INFO, however the document does not
reflect this.’ This is a documentation finding only.
6. Issue noted: ‘25 users (at the point of test) have access to mechanisms
for managing the digital signatures and have database administration
responsibilities and access. This raises the theoretical risk of a user
POL00139537
POL00139537
Procedures
Findings ]
10. Review source code on screen at Fujitsu headquarters which supports the key
inherent control operation around Balancing Transactions.
Review of Transaction Correction source code on screen at Fujitsu headquarters to
validate that Transaction Corrections must be accepted by branches, in order to
validate Balancing Transactions are the only transactions branches would not have
to accept.
12. Review the 9 Balancing Transaction Templates to validate balancing transactions
would, if the template was followed, logically perform as expected
3
Walkthrough a Transaction Correction being raised by SCC, and the notification /
acceptance of it by a branch
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
Data
‘spoofing’ the digital signature. It is understood that for this risk to be
realised, due to time limitations and volume of work required in order to
successfully ‘spoof the signature, a program would have to be written.’
No issues noted
A direct observation of Fujitsu extracting Balancing Transactions was
performed and supported that the population of Balancing transactions
Fujitsu had originally extracted was correct. This direct interrogation of
the data supported that there had been only one ‘true’ usage of
Balancing Transactions for the available data period of HNG-X data
(12/03/2010 through to 28/05/2016 when the testing was carried out).
There were 1,644 Balancing Transactions utilised in total throughout that
period, but the others were all used to update the Recovery
Transactions flag in response to a known bug with the system where it
frequently gets set to the wrong value.
No issues noted. 1 Balancing Transaction identified (in the period where
data was available for review 12/03/2010 — 28/05/2016) which did not
relate to a branch involved in the allegations and was appropriately
approved and governed
Additional context around the usage of this tool was obtained from
ticket review:
POL00139537
POL00139537
a. The original TFS helpdesk ticket was obtained which is the (Formatted: Font: 10 pt
legacy system used by Post office where branch incidents are
recorded. The TFS ticket 2091569 was reviewed and it was
noted that this had been raised by Anthony Vasse (Service
Desk) on 02/03/2010 and transferred to Cheryl Card (SSC
Product Specialist). Within the incident ticket it was noted that,
the after investigation the clerk had incorrectly doubled a
transfer of stock of £4000.00 to £8000.00; therefore creating an
incorrect loss to the branch of £4000.00. It was noted this issue
48
Procedures
Findings
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
was required to be fixed by 17/03/2010 as the branch was due
to roll into the next transaction period; therefore meaning that
the branch required a fix to ensure the accounts were correctly
recorded. An update was provided by Cheryl Card on
11/03/2010 confirming that the issue had been resolved using
the transaction tool to insert transactions into the
BRDB_RX_REP_SESSION and
BRDB_RX_EPOSS_TRANSACTIONS tables to reverse the
incorrect £400.00 charge. The ticket confirmed that the Post
Master had been advised to print a balance snapshot of the
accounts before and after the transaction correction took place
to ensure the transaction had been reversed correctly. A
‘subsequent update was provided confirming that the balances
had been correctly fixed. The ticket was subsequently closed on
04/04/2010.
Evidence was obtained of the Peak Incident ticket raised in
relation to this balancing transaction performed. Incident ticket
‘PC0195561" was raised by Lorraine Elliot (Service Desk) on
15/04/2010 in relation to a Post Master attempting to transfer
£4000.00 when the system crashed resulting in the post master
being issued with 2 x £4000.00 receipts.
‘An OCP ticket was also raised which is the solution
management system used by Fujitsu which tracks issues and
resolutions. From this OCP reference 25882 it could be seen
that the branch had performed a transfer out of stock for the
value of £4000.00 but due to a system error this had incorrectly
doubled in value creating an imbalance of £4000. Therefore, a
balancing transaction of £400.00 was required to correct the
loss using the transaction correction tool. This was approved by
Emma Langfield (Post Office) on 10/03/2010 at 15:33. From this
OCP ticket, it could be seen that this incident was raised by
Cheryl Card (SSC Product Specialist) on 10/03/2010 who
POL00139537
POL00139537
POL00139537
POL00139537
subsequently performed the work and inserted the balancing
transaction.
jubstantive
10. No issues noted
11. No issues noted
12. No issues noted
13. No issues noted
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 50
4.4 Phase 0 and 1- Scope Area 3
‘Scope Area 3: Carry out a full review of the controls over the user and capability of authorised Fujitsu personnel to create,
amend or delete baskets within a sealed Audit Store throughout the lifetime of the Horizon system, insofar as is possible.
4.4.1 Work Performed, and Analysis Results
Our procedures centred on the workshops and documentation reviews highlighted in Section 3.2.1 and 3.2.2
above,
For this specific scope area our procedures focussed on understanding the specific controls and processes around
protecting the integrity of data from inception to Branch Database, and subsequently to the Audit Store. Our work
highlighted a number of core concepts relevant to understanding the related risks and controls during this data
flow:
In essence the data journey can be divided into a number of distinct phases:
4
A
4.
‘Transaction initiation within either the Counter, Kiosk, or ‘third party interface source’, and subsequent
interface to the Branch Database.
Archival from the Branch Database to the Audit Server.
‘Sealing of Audit Tracks via MDS Message Digest and Archive to the Audit Store itself (Now based on
Eternis technology).
‘Subsequent Retrieval of Tracks, validation via the ARQ (Audit Track Retrieval) process, and Investigator
validation on the received data,
Non-Branch Transaction Data Records of Relevance
Transaction Initiation within either the Counter, Kiosk or third party interface source’
For Counter and SSK (Kiosk) initiated transaction data, the JSN remains a core element of control for the
Audit Store process as it validates the origination and completeness of data for a particular Counter and is
independent of the MDS message digest elements.
Given the wealth of ‘data at rest’ (stored in a directory/database awaiting onward processing) and ‘data in
transit’, security controls over access to ‘data at rest’ and interface controls over monitoring completeness.
and accuracy of ‘data in transit’ are both pertinent. However the JSN concept provides assurance
regardless given interruptions in the sequence, or mismatch between signature value and message
content, would highlight downstream risks of data corruption.
The other interfaces pertinent to our understanding have been represented by Fujitsu systems architects to
be
a. Logistic Feeder Service
b. Post and Go (discontinued in 2015, but relevant prior to that date)
c. Near Real Time (NRT) feeds
d. Paystation
e. Camelot
For non-Counter and Kiosk interfaces to the Branch Database completeness i
file header and footer record, with accuracy and validity provided by manual
themselves via the Transaction Acknowledgements process.
provided by the interface
\spection by Branch staff
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 5
POL00139537
POL00139537
POL00139537
POL00139537
5. For many of these interfaces the Post Office Data Gateway (PODG) provides the point of entry to Post
Office infrastructure.
B. Archival from the Branch Database to the Audit Server
1. Archival from the Branch Database of data take place to the Audit Server (which is the gateway to the Audit
Store infrastructure) in accordance to an automated routine which is central to the operation of the Horizon
system. If archival did not take place then very quickly the system would run out of available capacity. Two
intermediate directories are used to hold records prior to transfer to the Audit Server.
2. Asreferenced above both ‘data at rest’ and ‘data in transit’ controls are therefore relevant to this stage of
the process.
C. Sealing of Audit Tracks via MDS Message Digest and Archive to the Audit Store itself
1, The Audit Track Gatherer (ATG) is a routine which is permanently scanning for new Audit files on the
upstream infrastructure (including the Branch Database) which are then copied to the Audit Server, sealed
by the Audit Track Sealer (ATS), using the MDS message digest algorithm, copied to the Audit Store
Eternis architecture itself, and then purged from the Audit Server when copied across.
2. The Audit Server maintains a database of sealed files and their seal values, for later interrogation when
locating files, and validating their integrity has not been violated.
3. Therefore once again both ‘data at rest’ and ‘data in transit’ controls are relevant to this stage of the
process.
4. Once on the Eternis hardware which has now replaced the EMC Centera hardware solution, the data is
subject to a number of controls around access, deletion and amendment, all of which are designed to
maintain the integrity of the audit trail during storage. Both EMC Centera (historical solution) and Eternis
(current solution) are specialised hardware solutions for the storage of audit trail data intended to be used
forensically.
5. Previously there was a seven year limit to the retention of data in the Audit Store, after which it was purged
by the system in line with Retention schedules. Given recent history this policy has recently been changed
to indefinite retention of all Audit Store data. As a result all transactions should be available for as long as.
the Audit Store continues to exist from 04/10/2007, and therefore a complete audit trail of all transactions
ever posted on Horizon HNG-X should exist (given the migration date).
D. Subsequent Retrieval of Tracks, validation via the Audit Track Retrieval (ARQ) process, and Investigator
validation on the received data itself
1. Extraction of the data from the Audit Store is via a defined process known as the ARQ process. A
specialised Audit Desktop estate is utilised to interrogate the Audit Server database, retrieve relevant
sealed files, process the data, and burn to CD (or email as a data file), whereby it is made available to Post
Office investigative staff.
2. There are a number of logical access controls operating over this process, including role based access
mechanisms, a strict ‘segregation of duties’ from Post Office staff and audit logs over the process.
3. Upon receipt of the data files Post Office investigators carry out a number of additional checks themselves
in order to validate the data integrity.
E. Non-Branch Transaction Data Records of Relevance
1. Alongside the Branch Database data flowing into the Audit Store there are a number of other relevant data
sources:
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 52
a. Interface files received from third party systems which are then processed into the Branch
database, are also sent directly to the Audit Store as raw files, allowing potential future
jon between the two data sources.
reconeil
b. The Event Management System captures System Audit Logs from across the Horizon estate, and
processes these to the Audit Store.
Given the above understanding of the process gained from our work to date, our approach to assurance against
this scope area is largely based upon controls assurance, in combination with some limited analytics procedures to
support completeness, security and integrity of the data throughout the relevant data flows.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
4.4.2. Summary Table of Phase 0 Procedures and Conclusions
I Post Office Instruction
Carry out a full review of the controls over the _ Identified relevant business processes and areas of
user and capability of authorised Fujitsu interest
personnel to create, amend or delete baskets
within a sealed Audit Store throughout the Review of existing technical documentation and
lifetime of the Horizon system, insofar as is _identification of key inherent system controls, and
possible. support in interpreting the transactional data
Workshops with Systems Architects (Fujitsu) in order
Procedures Performed
to understand technical documentation.
‘walkthrough on-screen as to how the system works.
Walkthrough of Audit Store specific controls in order to
determine relevance and accuracy for inclusion within
the scope of our work,
What we have discovered
The Branch Database is a key point in the data journey at
which all Branch relevant data whether generated by the
Counter or by a third party data source external to Horizon will
interface to
There are a number of intermediate points at which data is at
rest during the flow of data to the Audit Store, and
understanding the Security controls over such data will support
the integrity of data flowing into the Audit Store,
Regardless of the opportunity or otherwise for interception and
tampering of data pre its arrival in the Audit Store, for key data
originating from the Counter and the Kiosks, the digital
signatures should highlight any tampering with data prior to its
usage within the Cases.
The Case data provided can be reviewed with a view to re-
performing the key integrity checks performed by investigators,
over the completeness and accuracy of the data.
The Audit Store controls should have remained relatively
constant over the period of allegations when considering those
relating to infrastructure downstream of the Branch Database
This is due to the HNG-X project which has influenced a
number of other key control areas, leaving the Audit Store
architecture relatively untouched
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
4.4.3 Phase 1 Procedures
Performed Procedures
Procedures
Findings
Controls
1. Validate Audit Store controls identified (See Appendix 4 for detail of controls 1A~
10),
2. Digital Signature controls applied to Message Journal during initiation of transfer to
Branch Database.
3. Additional Audit Store Controls identified (See Appendix 4a for detail of controls 3A
~3F),
4. Identification of Audit Store Data Flows at a Detailed Level, including security
controls over data at rest, and completeness, accuracy and validity controls over
data in transit.
Data
NIA
Substantive
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
Controls
1. No issues noted
2. Issue noted: '25 IT users (at the point of test) have access to mechanisms
for managing the digital signatures and have database administration
responsibilities and access. This raises the theoretical risk of a user
‘spoofing’ the digital signature. It is understood that for this risk to be
realised, due to time limitations and volume of work required in order to
successfully ‘spoof’ the signature, a program would have to be written.’
8. No Issues Noted except for control 3A.
3A finding - ‘Review of the audit settings for the Audit Server noted that the
audit policy change which relates to change of user rights was set to log
success events only, with failure not enabled.’
4. No issues noted
Data
NIA
‘Substantive
5. No issues noted
POL00139537
POL00139537
POL00139537
POL00139537
Procedures I Findings ]
5. Review source code on screen at Fujitsu headquarters which supports the key 6. See Appendix 5 for details of which controls have been subject to change.
inherent control operation around digitally signing transactions posted from the
Counter to the Branch Database.
6. Identification of changes relevant to the Audit Store from review of historical
documentation, and validation that the Audit Store has remained broadly consistent
cover time from a controls perspective for the period relevant to the allegations.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
45 Phase2
4.5.1 Work Performed, and Analysis Results
(Our procedures centred on the workshops and documentation reviews highlighted in Section 3.1 above,
In particular the following procedures were central in each case to our understanding:
‘Scope Area 1 — Audit Logs for Privileged Users
1. Awork shop was conducted with Fujitsu in order to discuss Privileged Users, audit logs, and the controls
thereon,
‘Scope Area 2— Analytic 1 and 6 Follow Up
1. Workshops were conducted with Fujitsu in order to determine the relevant root cause in each case.
2. Where necessary additional data was requested.
3. Analytics were re-run with revised logic and the issues found in the original analytic were found to have
been rectified by the changes made in each case.
Scope Area 3- Non-Counter Initiated Transactions
1. Technical documentation was reviewed in order to determine the nature of non-Counter transaction
process flows, the related risks, and the responding controls for the three non-Counter transaction sources
(Camelot, Paystation, Post and Go),
2. Aworkshop was held with Fujitsu in order to validate this understanding
3. Amemo was produced highlighting the proposed recommended procedures, which was then translated
into Phase 3b scope and approach.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 37
4.5.2 Phase 2 Procedures
Performed Procedures
Procedures I Findings
Scope Area 1 ‘Scope Area 1
1. Perform workshop with Fujitsu in order to ask further questions around Privileged 1. The workshop was held and an approach adopted whereby Fujitsu
Users, and determine scope for future meetings. produced a report on Privileged Users for future review (See Phase 4).
Some basic findings were determined around the audit logs operating over
Privileged Users in order to support with these determinations:
a. Regardless of access rights to amend and delete audit logs, the
digital signature controls should still allow for the detection of
data which had been modified, deleted or inserted subsequent
to receipt from the Counter.
b. There are a limited number of users who could theoretically,
due to a segregation of duties breach between database
administration and the key management server, amend the
Message Log for one or more Counters in one or more
branches and make the transaction/s amended, look legitimate
when it is retrieved from the Audit Store (through spoofing of
the digital signature).
c. However to do this would require an existing systems
administrator with a large amount of technical expertise and
systems knowledge, it would almost certainly require a program
to be installed onto the Horizon online system, and a release
process would have to be bypassed in order for this to be
installed maliciously (and avoid file integrity checking controls,
operated by Fujitsu),
Scope Area 2
Scope Area 2
Analytic 1
Analytic 1
2. Workshops were performed with Fujitsu in order to determine the root cause in the
gaps in sequencing highlighted by the original analytic. 2. There was an error in the original analytic logic which was supposed to
remove duplicated transactions from the dataset but was in actuality
removing both the duplicates and the original transactions from the data
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 88
POL00139537
POL00139537
Procedures
Findings
3. The analytic was re-run with revised logic to determine if the correct root cause for
the gaps had been determined for these 25 data items.
Analytic 6
4, The original data for the 48 session IDs which were noted to be out of balance were
investigated. To do this a sample of 15 out of balance session IDs were selected for
further investigation with Fujitsu support.
5. Root causes for the original data appearing to show a branch as being out of
balance were determined.
6. Aworkshop was performed with Fujitsu and the data provided to support for all 15
items the established root cause was responsible.
Scope Area 3
7. Avariety of Fujitsu technical documents pertaining to the Horizon system were
reviewed in order to understand the data flows for non-Counter transactions, and
identify the relevant risks and areas of control
8. An approach memo was produced highlighting the relevant approach details and
used as the basis for Phase 3.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
3. When the analytic was corrected for this it was noted that there were no
gaps in JSN sequencing were identified based on the data provided
Analytic 6
4. The root cause for the 48 transactions appearing not to balance was
determined as:
a. Some of the audit log sequences were missing a start time and
hence were not extracted properly.
b. Some of the audit log sequences were missing a SC (Serve
Customer) record and hence were not extracted properly.
5. These issues were shown to have been overcome by looking at the raw
audit log sequence data (as it was the extraction logic performed by Fujitsu
which was causing records to be dropped).
6. It was confirmed through the walkthrough with Fujitsu and through checking
the 15 sampled files independently that there were no session ids out of
balance based on the new transaction data provided and it was concluded
that the out of balance session ids identified on the initial run through were
out of balance due to the 2 errors identified above in extracting the data
from the raw audit log sequence
Scope Area 3
7. The technical documents were reviewed, analysed and used to highlight the
controls and risks as documented in Appendix 8.
8. An approach memo was produced and utilised in formulating the scope for
Phase 3.
9. The review performed highlighted that the key area of risk was in ensuring
Postmasters had adequate visibility of the data being received from
systems external to Horizon and were in a position where they could
reconcile the Transactions Acknowledgements they received back to the
data captured on Camelot, Paystation and Post and Go devices at source.
POL00139537
POL00139537
46 Phase3
“ounter initiated transactions that could call
‘Scope Area 1: Are there any haps in the controls around non
into question the integrity of the data generated in relation to these transactions?
4.6.1 Work Performed, and Analysis Results
In commissioning this work Post Office asked for a Deloitte viewpoint on the below questions which we have
provided:
1. Are there any gaps in the controls around non-Counter transactions that could call into question the
Integrity of the data generated in relation to these transactions?
The first potential area of interest from a controls perspective in relation to the completeness and accuracy
of the flow of data, is around the sending, processing by, and subsequent receipt of data from third parties.
The primary control in relation to this is the requirement for Postmasters to ‘Transaction Acknowledge’ such
data before it is accepted into their accounts, but the formalisation of the processes and controls ensuring
Postmasters reconcile their financial position accepting transactions has not been enforced. Reviews of the
supporting documentation primarily from the Horizon Online Help references a number of reports which are
available to facilitate this and Post Office has represented that Postmasters can reconcile data received
from third parties.
Originally it was theorised there was a second key area of interest, being that no digital signature is applied
to non-Counter transactions, potentially opening up this category of transactions to greater risk of
interference subsequent to processing into the BRDB (as the digital signature control is the primary control
preventing this). However, further discussion with Fujitsu established that when the BRDB receives non-
Counter transaction data, it pushes it down to the counter for acceptance by the Postmaster, at which point
the Counter digitally signs the acknowledgement of the transaction and therefore in theory a reconciliation
between these digitally signed TAs and the raw data files received from the third parties (which are
interfaced into the Audit Store) should also be possible mitigating this risk.
2. Ifthere are gaps:
a. Could they be the cause of discrepancies in branch accounts (or could they mean that errors in
Horizon would not be revealed and those errors could then be the cause of discrepancies in
branch accounts); and
Theoretically they could ~ if a third party incorrectly reflected the data they had received from a
non-Counter system, and this incorrect total was then downloaded into the Branch accounts, then
in the absence of formal controls to reconcile data transmitted to the third party, back to data
received, the branch could, theoretically, cause discrepancies in the branch accounts. The control
which Post Office relies on to mitigate this is the Transaction Acknowledgements,
b. What is the risk of those gaps (or resulting discrepancies) materialising?
Without a full investigation of the controls at the third parties, and any other mitigating controls,
which may exist, itis difficult to quantity the risk of any exposure.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 60
POL00139537
POL00139537
(Formatted: Fort color
‘Auto
(Formatted Fort coer
A
to
(Formatted: Fort color
‘Auto
(Formatted: Fort obr
‘Auto
(Formatted: Fort cobr
‘Auto
BT rormatindl Fort car:
‘Auto
I rormatin Fort cor:
‘Auto
Formatteds Fort ctr
‘Auto
POL00139537
POL00139537
4.6.2 Phase 3 Procedures
I Procedures Findings
Hold an initial workshop to corroborate understanding of data flows and validate the _This workshop was performed with Fujitsu on the 9th May 2017.
existence and completeness of controls over the current reconciliation process, and how
Transaction Acknowledgements are utilised /As a result of the workshop the understanding that Deloitte had originally
tained on the operation of the interfaces between the systems was validated
ith a couple of amendments. The attached diagram (Appendix 8) displays the
inalised viewpoint in relation to the data flows.
/As part of this review the decision to exclude ATMs from scope as non-Counter
ransactions was examined and it was highlighted by Fujitsu that all interactions
between ATMs and the Counter/BRDB are by rekeying of the data — i.e. this is
not a system driven process. Therefore the original decision to exclude ATMs
rom scope was adhered to.
Review and test key reconciliation controls between key data sources within the data Fujitsu discussion highlighted that one of the controls identified for potential
flow testing was only operated temporarily during the switch from Riposte to the
Branch Database, and as a result no control exists to test in the present day.
The remaining two controls are legitimate controls to test, as they are currently
ded, and one requires a wording tweak in order to test.
The below table captures the controls in scope, and the required updates to the
original control wording where required:
#_] Summary Control Wording
1 I External transactions sent via PODG Not an existing control. TPS —
such that the Extemal Transaction BRDBis a rec, not Credence ~
files that are currently sent from BRDB.
Ingenico (PAYSTATION) and Wincor
Nixdorf (POST&GO) are routed to
the Branch Database as well as
sending the data to the Credence Update final sentence of control
system. There is a reconciliation Wording to ‘There is a
between Credence and BRDB.
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 61
Procedures
Findings
Perform detailed walkthrough of the Transaction Acceptance (TA) process to confirm the
granularity of the information the Postmaster is provided with. Perform procedures to
corroborate a TA is required for all Non-Counter Transactions.
‘conciliation between TPS and
BRDB’.
For each Transaction
Acknowledgement generated, a new
transaction pair is created for
POLSAP. The transaction delivered
to POLSAP will have a Reference
number that matches the reference
number used in the Transaction
‘Acknowledgement record
generation. This allows POLSAP to
match with the Transaction
Acknowledgement once the TA has
been accepted by the Postmaster.
Control exists.
30
AP Client File Reconciliation
APSS2222.ksh will reconcile the
data in the files that it delivered to a
Client with the data in the files that
Credence delivered to a Client.
jo longer an existing control — no
further testing to be performed.
3T
TPS to AP Reconciliation
TPSC227 writes APS transaction
data to a formatted file that will later
be used by the APS host program
APSC2051 to reconcile data from
TPS with that from APS.
Kontrol exists.
steps:
Detailed analysis of the TAs process was conducted through the following
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
Procedures
Findings
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
1. Review of Horizon Online functionality within the Model Office at
Finsbury Dials on 29/03/2017 with assistance from Mark Underwood
and Phil Jeary.
2. Confirmation via review of the system screens that the Horizon system
included TA functionality relating to all of the non-Counter transaction
areas under review, including:
a. Post and Go;
b. Paystation; and
c. Camelot.
No evidence was witnessed during this review, that there were other transaction
ypes for which TAs would apply, although this should not be construed by the
eader to categorically mean other non-Counter transactions for which
Transaction Acknowledgements would be processed do not exist. To provide
juller assurance over the completeness of the transaction population for which
TTAs are produced and relevant a detailed review of product types, and the
elated population of transaction types, would be required, and this was beyond
the scope of this piece of work.
1. Walkthrough of the receipt and processing of Transaction
Acknowledgements on the Model Office test system. This walkthrough
highlighted the following key points:
a. On Receipt of a TA the Postmaster is able to review both ata
header and line level of granularity.
b. On Receipt of a TA the Postmaster must complete the
processing of it, before trading can continue.
c. If the Postmaster disputes the TA, then the TA ID should be
noted to dispute with the helpline after the TA is processed (this
could then trigger a further Transaction Correction).
2. Review of the Model Office counter for each of these transaction types,
in particular the Horizon Online Help Guide pages (which are available
within the system to all Post masters), confirmed that various reports on
the balances are available to allow reconciliation between the terminals
involved and the TAs received and values within the Branch Database,
POL00139537
POL00139537
Procedures
Findings
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
as well as guidance on the usage of TA functionality. Below is a
summary of the findings against each of the three transaction types
which have been represented by Fujitsu and Post Office to formulate
the population of non-Counter transaction types for this work
Paystation TAs
The following sections of the Horizon Online Help Guide were reviewed:
‘Paystation Transaction Acknowledgements’
This is a ten page document which upon review provides guidance on:
1. What TAs are, (Page 1)
2. Accounting for TAs (page 2)
a. Including having to reconcile / check against all Paystation
transactions.
3. Non Receipt of TAs (Page 3)
4, Receipt & Processing TAs (page 6)
5. Including guidance on checking/reconciling the TAs against Paystation
transactions
6. Office Daily Reports (Page 9)
a. Including details of a ‘Outstanding & Processed TAs’ report that
is available
b. This report gives detailed information on all TAs that have beenI
received over the last 40 days and their existing status.
c. “There are no audit requirements for you to print and retain this,
report. However you may find it useful if you need to verify
information contained within the TAs against any terminal
reports”
POL00139537
POL00139537
Procedures
Findings
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
‘Accounting and Balancing Instructions for Paystation’
This is a four page document, which upon review provides guidance on
1. What a TAis (page 1)
2. Reconciling transactions from Paystation against the TAs
Post and Go TAs
The following section of the Horizon Online Help Guide was reviewed:
‘Transaction Acknowledgements for Post & Go’
This is an eight page document which upon review provides guidance on:
1. Whata TAis in relation to Pay & Go (Page 1)
2. Daily processing of a trading report at close of business & prior to
business the next day to compare against TAs received. (Page 2 & 3)
3. Non Receipt of TAs
4, Receipt and Processing of TAs (Page 6)
a. Including recommending all Post & Go transactions are
checked/reconciled against the TAs received
5. Office Daily Reports (Page 7)
a. Including details of a ‘Outstanding & Processed TAs’ report that
is available:
b. This report gives detailed information on all TAs that have been!
received over the last 40 days and their existing status.
c. “There are no audit requirements for you to print and retain this
report, However you may find it useful if you need to verify
information contained within the TAs against any terminal
reports”
POL00139537
POL00139537
Procedures
Findings
6. TA Accounting Arrangements (Page 8)
a. Including recommendation to check and reconcile the cash
against the TAs received the following working day.
(Camelot TAs
The following section of the Horizon Online Help Guide was reviewed:
Transaction Acknowledgements for Camelot’
This is a three page document which upon review provides guidance on:
1. What a TAis. (Page 1)
2. Accounting instructions for TAs
a. Including check and reconcile the cash against the TAs
received the following day (Page 2)
3. Non Receipt of TAs (Page 2)
4. TA report (page 3)
Additional Sections of Horizon Online Guide Identified as of Relevance
\n addition to the above it was confirmed that there is a help page within
Horizon Online Help providing contact details which Postmasters can use
should they have issues with Transaction Acknowledgements for Paystation
This page was entitled Contact Names, Addresses and Telephone Numbers’
land was two pages long.
1. To supplement these procedures further a review of additional sources
of process narrative and guidance were obtained and reviewed from
Post Office staff. The documents reviewed as part of this further
exercise were:
a. ‘Self-Serve Kiosk User Guide V4.1"
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
Procedures
Findings
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
b. ‘HNG Branch Trading Reports 310317"
c. ‘HNG BT Balancing and despatch of docs 310317'
d. ‘HNG Camelot Lottery On-line games 030417"
e. ‘HNG Camelot Scratchcard games 030417"
f. ‘HNG Cash and Secure Stock Rem Services 310317’
g. ‘HNG Equipment and Admin Pages 310317’
Review of these documents, highlighted a number of areas which provided
additional context/assurance:
Guide NG BT Balancing and despatch of docs 310317"
This document makes reference to an ‘Office Snapshot Report’ and details how’
fo create the report, but does not explicitly say this can be used to reconcile
against TA's:
1. ‘Producing the Office Snapshot report to list stock and cash on hand
and all the transactions carried out during the current Branch Trading
Period up to the time the report was requested, for all stock units in
your branch.’ (Page 109)
(Guide HNG Camelot Scratchcard games 030417’
This document has a section that details account of scratchcards. This section
highlights that National Lottery transactions are accounted for via Transaction
‘Acknowledgements and that a Camelot terminal creates a report which shows:
1. The total daily scratchcards sales
The daily prize payments
Any retuns
RON
Commissions (this figure will always be zero)
POL00139537
POL00139537
POL00139537
POL00139537
However the guide does not explicitly say that this report that shows all non-
‘ounter transactions for National lottery should be reconciled against the TA
ich accounts for National Lottery transactions.
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 68
4.7 Phase4
[Scope Area 1: Deloitte review of Fujitsu Report in conjunction with initial comments raised.
4.7.1. Work Performed, and Analysis Results
For this specific scope area our procedures centred on reviewing the Fujitsu report in conjunction with the
comments raised, and providing commentary on residual question areas or concerns back to Post Office.
‘Subsequent to these procedures a workshop was held with Fujitsu staff, whereby residual questions and concerns
were dealt with.
These procedures confirmed that a Privileged User would, theoretically, be able to amend data in a manner where
it looked legitimate, and delete the audit trail of them carrying out such activity with minimal footprint. The technical
hurdles that would need to be overcome would be significant, and the user in question would likely require access
toa programme to do so. The Privileged User would then be required to locate the programme on the correct
hardware, and Fujitsu have pointed to the state monitoring software which should detect if unauthorised
programmes have been added to the relevant hardware, whilst recognising this is not a formal control.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 6
POL00139537
POL00139537
POL00139537
POL00139537
4.7.2. Phase 4 Procedures
Procedures Findings ]
Deloitte review of Fujitsu Report in conjunction [This review has been performed with an email provided as per the agreed deliverable in the Statement of Work. (Formatted: Font color: Auto )
with initial comments raised.
Workshop with appropriate Fujitsu resource to: A workshop was held on 11/05/2017 with attendees from: I (Formatted: Font color: Auto J
1. Answer any outstanding comments / - Deloitte (Mark Westbrook, Lewis Keating)
questions on the report = Fujitsu (Torstein O'Godeseth, Gareth Jenkins)
2. Produce a detailed commentaryon what - Bond Dickinson (Jonathan Gribben, by telephone)
steps would need to be taken to replace
ray ronnie fila 2.20f 2) The following agenda items were discussed, with Deloitte asking the numbered questions, and Fujitsu providing ____[ Fermatted: Font color: Auto )
peer esponses (in italics)),. I (Formatted: Font color: Auto }
We have also produced section (c), which Se (Formatted: Font color: Auto )
includes, as requested recommendations on the Horizon Online (Formatted: Font: Not Italic, Font color: Auto }
further work to be performed in relation to the
Fujitsu report. 1. Is the segregation of duties breach between database administration and the key management server, the only
way in which a weakness could be exploited to overwrite transactional information in a way where it cannot be
traced and looks legitimate to the system?
is the only way known by Fujitsu staff. Fujitsu do however stress that there are numerous levels of security which I ‘[Formatted: Font: Italic, Font color: Auto )
ould make any way to break through very difficult, I___> (Formatted: Font: Italic )
2. Is 1am the following day stipulated as the date and time by which overwrite would need to be achieved by due
solely to the Audit Store, and if so are there other more timely data feeds which would highlight a discrepancy
between actual ‘transactional reality’ and what is recorded in the Audit Store or the BRDB?
Yes, 1am is when ‘harvesting’ of data from BRDB to the Audit Store, happens (the job is scheduled to run at 1am, so I (Formatted: Font: Italic, Font color: Auto )
factual harvesting is likely to happen in the minutes after this time). Therefore the maximum time slot for manipulation (Formatted: Font: Italic, Font color: Auto J
would end at tam.
In reality there are other interfaces which occur on a more frequent basis, (which would leave a footprint on another
system / part of Horizon) however only certain products are involved in these interfaces (mainly transactions which
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 70
Procedures
Findings ]
ssettle with clients, and are recorded in somebody else's system). Therefore any manipulation would have to avoid these
‘specific products. This adds another layer of complexity, theoretically however these transactions in the session could
be replaced ‘correctly’ (like for like), and not leave a footprint if done before 1am,
3. For step 6 of the replacement routine, can you remind us the technical reasons for requiring access to the BAL
Private Key?
‘The BAL private key signs messages which come from the counter. If you are going to create a fake counter key, you
need the correct BAL private key to make the digital signature look legitimate,
4. On step 9 on the Privileged User audit log — how long can this log be edited by the Privileged User? Same 1am
window before transmission to the Audit Store? Also a reminder that it is the hardware protection rather than
the digital seal which is important on the Audit Store due to the usage of the cracked MDS algorithm for
sealing?
is a daily pull occurring at around 1am, therefore the window is as previously described.
5. On the point on editing the log, if I'm reading correctly it would always be possible to see the last action by the
Privileged User, even if they deleted all else?
‘Can we be provided with further detail on how this would work — In order to make the changes to the Message Log
described in section 2.2, the Privileged User would need Read access to the Key Store database which runs on the
PS and Read / Write access to the BRDB. Note that should the rogue application run on the BAL, then this isn’t
necessary as the BAL’s have access to the Key store based on the IP address
You can always see the last action by a Privileged User, if a Privileged User deleted their actions, it would always leave
a footprint of the deletion of logs. They could theoretically remove what they have done, but they cannot remove that
they have done something.
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 7”
POL00139537
POL00139537
(Formatted: Font: Italie )
(Formatted: Font: Italic, Font color: Auto }
(Formatted: Font: Talc }
(Formatted: Font: Italic, Font color: Auto )
{ Formatted: Font: Italic, Font color: Auto )
Procedures
Findings
Fujitsu note that turning off audit logs completely will break’ the application.
A ‘Delete’ record on the audit trail is likely to be highly unusual & easy to spot.
Could a Privileged User (theoretically) cover their tracks completely by removing log on / log off activity *
from the audit log without leaving a trace? If not how feasibly is a comparison between all log on/ log off
activities of Super-Users’ and MSCs in order to detect un-authorised access?
(As above in answer to question 5, they would always leave a trace.
is noted that log on /log offs by Privileged Users on BRDB / BAL are likely to be very rare (limited to system
upgrades) and should always be approved by an MSC (record of the reason Super-User access is required and
approval for this access).
6.7.Although the Database Audit tables are not regularly examined they were recently checked as part of an
external Audit of Horizon Online. — Could you provide further context on this audit? What was checked and
why?
Fujitsu have confirmed that NCC Group conducted an audit of HNG-X security in 2014 and PCI DSS audits are I
conducted annually. Further detail on the coverage of these audits of the database audit tables has not been provided, I __
7.8.How often would the individuals who contravene access SoD between the NPS and BRDB tend to logon to the
NPS? Also does the point raised on not needing to logon with access to the BAL broaden this concern?
Fujitsu advised they would not expect Privileged Users to log onto the NPS on a regular basis (limited to upgrades /
changes etc),
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 72
POL00139537
POL00139537
Formatted: List Paragraph, Numbered + Level: 1 +
Numbering Style: 1, 2, 3, .. + Start at: 1 + Alignment: Left +
Aligned at: 0.63 cm + Indent at: 1.27 on
(Formatted: Font: Tale, Font or: Auto }
{ Formatted: Font: Italic, Font color: Auto }
(Formatted: Font: Italic )
Formatted: Font: Italic, Font color: Auto )
(Formatted: Font: Italic )
POL00139537
POL00139537
Procedures Findings
&.9.For step 2, how big is the average message log associated with any log on session. (i. is a log on session
generally all day and therefore the message log will hold thousands of transactions?)
lormally 2-3 hours if not all day. (Some quiet post offices it would be notably less) (Formatted: Font: Italic, Font color: Auto )
AA log on session likely to be hundreds / thousands of lines. (1 audit line for every customer been server plus few extras
for printing reports etc.)
However even if a session was a small number of lines, a program would still be required in order to effectively amend
transactions without leaving a footprint due to the complexity of re-creating the digital signature for 1 transaction /
locating a suitable transaction., __- (Formatted: Font: Italic )
9.10. For step 4, are there any barriers to uploading this application onto Fujitsu systems (if this would be
required). Presumably this would be required due to the volume of work required?
There is a detailed release process which all releases should follow. However there are no preventative logical access I__—_( Formatted: Font: Itc, Font color: Auto }
controls preventing a user from releasing programmes outside of this process.
However if someone tried to not follow this process then File Integrity monitoring is in place on BRDB & BAL. This
‘checks if files appear on a platform and flags things which have changed, the security ops team then investigate.
40.11 On step 8, is there a formal control operated by Fujitsu which can be referenced which would provide
evidence for ‘any instance of slow running on the system would be investigated by the support teams’. If not
can we articulate how obvious this would be to evidence it would be picked up in BAU activity?
Fujitsu noted that if the system behaves poorly this would be very obvious to Fujitsu employees who monitor system I [Formatted Font: Italic, Font color: Auto )
performance on an ongoing basis,, (Formatted: Font: Italic )
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 73
POL00139537
POL00139537
Procedures Findings
Riposte
1. The Riposte product managed the Message Store and it did not allow any message to be updated or deleted, —
Is there any further information available on this control?
Each message also had an associated CRC, this was basically a checksum that was included to ensure that the
message had not become accidently corrupted. Note that this was not a cryptographically secure seal and it would be
possible for a sufficiently technically skilled person to alter a message and recalculate the CRC if they had access to
the message outside the message store. — i.e, the level of protection on Riposte was lower?
‘The message store was a specialised database designed so that all you could do was add messages on, not amend I __— (Formatted: Font: Itai, Font color: Auto }
messages.
There is no known loophole by Fujitsu to amend transactions due to the nature of database.
As soon as messages arrived centrally, they were copied into audit trail immediately.
2. The Digital Seal for the Riposte Audit Store remained the same as for Horizon Online — i.e. MDS? And the
hardware protection was applied the same as well?
Yes, (Formatted: Font: Italic, Font color: Auto )
I (Formatted: Font: Italic J
3. Due to the size of the Post Office Network, branches were split into 4 separate Clusters. Each Cluster included
4 Correspondence Servers (2 in each Data Centre), thus ensuring that there were normally 4 copies of the data
held in the Data Centres. — Does this mean you would need to duplicate corrupted data across 4 servers?
{To inject rogue transactions theoretically a user would inject artificial messages into Riposte, as they could not amend I (Formatted: Font: Itai, Font color: Auto }
messages due to this replication., I (Formatted: Font: Italic )
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 74
POL00139537
POL00139537
Procedures Findings
4. In Detecting Changes to the Audit Trail the following is stated, however, if such data were injected at the
Correspondence Server, it would be clear that this had occurred since the Node Id associated with the
message would be that of the Correspondence Server at which the message had been injected and not a
normal Counter Node Id. This would be clearly visible in any audit extract. Could this not be spoofed?
(Would need to run application on the counter remotely to inject transactions from the counter. (Formatted: Font: Italic, Font color: Auto }
“Very difficult but not impossible’.
(This was not a Fujitsu owned system, source code owned and managed by another third party. As Fujitsu did not own
‘or manage the source code, changes to the source code of the system would have to be applied by the third party, this
adds another step of complexity in running a rogue application)., (Formatted: Font: Italic )
{b) Detailed commentary on what steps would need to be taken to replace the message log, as per section 2.2
of the Fujitsu report
in theory, a Privileged User, could amend the Message Log for one or more Counters in one or more branches. The
following describes what would be required to replace the Message Log for a single counter in a single branch. This
process could be repeated for multiple counters / branches if required,
1. To exploit this, the work would need to be completed before 1am the following day (since the Message Log is
extracted from BRDB at some point after 1am each night and the data is then sealed and held in the Audit
Server). As such there is a limited window of opportunity. A log on session can last up to all day for a counter in
a branch, and is essentially how long the counter machine is ‘logged into’ in any one sitting.
Ifa branch is still logged into a session, and performing transactions in that session whilst someone was
attempting to amend the transactions in the BRDB there are likely to be additional complications around
maintaining JSN continuity and order, and ensuring the digital signature for all transactions in the session are
valid and ‘match’,
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
Procedures
Findings
The entire Message Log associated with a Log On Session that is to be corrupted would need to be replaced,
as a new Counter private key would need to be generated, and as such all messages would need to be signed
by this key.
This is because there is no known way to obtain the counter's Private Key and so a new one would
need to be generated as described below.
The records being replaced would have to correspond on a one-to-one basis to the original records otherwise
there would be gaps or duplicates in the sequence of JSNs which would then be detected as part of the Audit
Retrieval process. There is an estimated 1 to 1 ratio between ‘records’ and transactions, as such there can be
hundreds of transactions in any one session; all of which would need to be re-signed. Amending or replacing
certain records relating to transactions which are involved in more regular interfaces from BRDB such as third
party systems would have to be specifically avoided (or replaced on a like for like basis — this is replacing the
transaction with a transaction which matches it exactly) otherwise they would trigger errors in other
reconciliations; this adds an additional layer of complexity to this process.
An application / programme would need to be run by a Privileged User in order to correctly construct the
revised Audit Records due to the high level of complexity involved in generating new private keys / digital
signatures, and the volumes of transactions these would be required for within the time limitations noted in
point #4
There is a release process which would have to be bypassed in order to get an application /
programme onto the relevant systems. It is expected file integrity monitoring / checks would identify if a
user attempted to introduce a rogue application / programme onto the relevant systems.
This application would need to generate a Private / Public key pair similar to the one originally generated by the
counter. Called an “Attack Counter key’ in the rest of the document.
The application would need to have access to the BAL's Private Key. Since this is stored in the Key Store
which is an Oracle Database running on the NPS, then it is assumed that a Privileged User would be able to
read this value and make it available to the application. This would then enable the application to generate a
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 76
POL00139537
POL00139537
Procedures
Findings
(c)
Log On Message Log message containing the fake Counter Public Key and to sign it using the genuine BAL
Private Key.
All subsequent messages for the session would then need to be amended as required and then re-signed using
the Attack Counter Private Key generated at step 5. An application would be needed to do this due to the high
complexity.
Having constructed all these false Message Log messages, the Privileged User would then need to delete all
the genuine messages from the Message Log in BRDB and replace them with the false messages on a one for
one basis.
Note that as stated earlier, corrupting the Message Log in this way has no impact whatsoever on the Branch
Accounts, since these never refer to the Message Log. The Branch Accounts are based on copies of some of
the data held in the Message Log being stored in “working tables” within the BRDB. Clearly any application that
is capable of corrupting the Message Log in BRDB would also be capable of updating (ie. corrupting) the data
used to calculate the Branch Accounts. Therefore the above steps, if followed, could theoretically amend the
Audit Store record without leaving a trace, however there would be no impact on branch accounts unless a
programme was also configured to make the same amendments to data used to calculate Branch Accounts in
order to impact on branch accounting. This adds another layer of complexity to this hypothetical and unlikely
scenario
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 7
POL00139537
POL00139537
POL00139537
POL00139537
5. General Assumptions and Limitations
5.1 General Assumptions and Limitations
Our work has been subject to the following exclusions: (Formatted: Not Highight )
1 We,have only, verified and tested information and or assertions provided directly by Post Office, or (Formatted: Not Highlight }
directly or indirectly by third parties, where it is stated in the report; ANG (Formatted: Not Highlight )
. ; (Formatted: Not Highlight 3}
2. For scope areas across all Phases, only matters relating to Horizon Features? and Audit Store within \\
the Horizon processing environment have been considered during our workshops and discussions; \\\ [Formatted: Not Highlight ]
5 Weh ded aloo her oni , . ; (Formatted: Not Highlight
. We have not provided a legal or any other opinion as to the completeness and accuracy of processing Genres }
of Horizon at any point throughout the work; =
(Formatted: Not Highight }
4. We have not had direct contact with any third parties other than named contacts that you have
provided to us (Appendix 1). These third parties have been limited to Fujitsu and Accenture;
5. We have not reviewed any contractual provisions in place between you and third parties;
6. Our review of technical documentation is limited to that provided to us by Post Office and Fujitsu and
as set out in Appendix 1. Our work was limited by gaps existing in the technical documentation
available, relating to both the granularity of information and the existence of the Horizon Features over
the entire timeline of operation of Horizon process documentation. The effect of which is that there are
gaps within what we are able to comment upon over this timeline;
7. We have not validated or commented on the quality of the Assurance Work? supplied to us.
(Our work was also based on the assumption that the documents provided and assertions made are a complete and
accurate representation of the Horizon design, and Audit Store process. We therefore cannot comment as to
whether other processes would need consideration in the context of the Matters.
We have performed work on control in place and operating at the time of the review, and not those operating at the
time of the allegations. Other evidence has been obtained, where available, to provide a view as to whether the
control was likely to have operated at the time of the allegations.
2 "Horizon Features” is a term we have introduced to represent those features of the Horizon processing environment, including IT management
‘and business use controls, which provide that:
‘+ Movements in Branch ledgers have the full ownership and visibility of Postmasters; and
‘© Audit trails kept by the system are complete and accurate
3 since its implementation in branches, Post Office has commissioned or has received a number of pieces of work relating to the Horizon
processing environment, to provide comfort over its integity. This work, referred to in our report as the “Assurance Work’, provides documented
assertions relating to aspects of the design and operation of the Horizon processing environment. The Assurance Work includes IT project
documents; operational policies and procedures; internal and external investigations and reviews; independent audits; and emails confirming
otherwise verbal assertions.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 78
POL00139537
POL00139537
Appendix 1
Documents Reviewed
Document Ref Document Title
DES/APP/HLD/0047 HING-X Counter Application High Level Design _—- (Formatted: Not Highlight }
DES/APP/HLD/0020 Branch Database High Level Design __-{ Formatted: Not Highlight )
DES/APP/HLD/0030 ‘Audit Data Collection and Storage High Level Design __-[ Formatted: Not Highight }
[DES/APP/HLD/0029 Audit Data Retrieval High Level Design (Formatted: Not Highlight )
RCISOLIARC/0006 HNG-X Architecture - Global Users __ (Formatted: Not Highlight )
DEV/APP/LLD/0065 BRDBC002 — BRDB Message Journal Auditing LLD _—fFen Not Highight i}
(DEV/APP/LLD/O014 Host Branch Database Audit Archive Purge Low Level Design __-{ Formatted: Not Highlight }
DEV/APP/LLD/0142 Host BRDB Transaction Correction Tool Low Level Design _-{ Formatted: Not Highlight )
DES/APP/SPG/0001 Host Branch Database support guide -{ Formatted: Not Highlight }
(DEV/APP/LLD/0199 ‘Schema definition for Branch Database, standby Branch Database and branch I __/ Formatted: Not Highight }
support system
DES/APPIHLD/0035. Exceptions and logging frameworks high level design. (Formatted: Not Highlight }
DES/APP/IFS/0002 HNG-X:RDDS to Branch Database - Counters and HBS Reference Data and - (Formatted: Not Highlight J
Memo Submission Interface Specification
DES/APP/IFS/0012 BAL Service Interface Specification (Formatted: Not Highlight }
DES/APP/HLD/0083 HNG-X Counter Subsystem : Recovery Management -{ Formatted: Not Highlight I
.DES/APP/HLD/0021 Branch Database Scheduling High Level Design _-( Formatted: Not Highlight )
DES/APP/IFS/0007 Branch Database to Legacy Host Interface Specification (Formatted: Not Highlight }
ES/APP/IFS/0001 HNG-X: RDMC / RODS to Branch Database Application Interface Specification I __—{ Formatted: Not Highlight }
DES/APP/HLD/0049 HING-X Generic Reports Data Extract HLD (Formatted: Not Highlight }
DES/APP/HLD/0057 HNG-X Counter infrastructure: Service and Process Control High Level Design (Formatted: Not Highlight ]
ARCISOLIARC/0001 HING-X Solution Architecture Outline __{ Formatted: Not Highight )
DEV/APP/LLD/O071 ‘Audit Data Retrieval Low Level Design
POLSAP/DES/APP/STG/0001_I POLSAP Archiving Strategy
DEV/INFAON/0001 ‘Archive Server Configuration
DES/SEC/HLD/0003 HNG-X KEY MANAGEMENT HIGH LEVEL DESIGN
DES/APP/HLD0041 HNG-X Counter Applications: Business Logic Subsystem High Level Design
DES/APP/IFS/0018 XML Message Audit between Counter or HBS and BALIOSR
DES/APP/HLD/0012 DVLA Internal Web Service High Level Design
ARCISEC/IARC/0003 HNG-X Technical Security Architecture
DEV/APP/LLD/0204 Host BRDB Update Outstanding Recovery Transaction Tool Low Level Design
DES/APP/HLD/0070 Host Applications Monitoring High Level Design
DEV/APP/LLD/O151 HNGX BRDB HOST: BRANCH SUPPORT DATABASE LOW LEVEL DESIGN
DES/APP/DPR/O006 Design Proposal for Transaction Acknowledgments
EAVIFS/006 Application Interface Specification
‘SVM/SDM/SD/0020 End to End Reconeiliation Reporting
REQ/APP/AIS/0004 Transaction Acknowledgements Application interface Specification
NA Post Office Pay Station Manual
NA 1- Self Serve Kiosk Guide
NA HIG Branch Trading Reports 310317
NA HIG BT Balancing and despatch of docs 310317
NA HNG Camelot Lottery On-Line games 030417
NA HNG Camelot Scratchcard games 030417
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 73
NA
HNG Cash and Secure Stock Rem Service 310317
NA
HNG Equipment and Admin pages 310317
© Deloitte 20182 Private and Confidential - Subject to Legal Privilege - ORAFT
80
POL00139537
POL00139537
Individuals Interviewed
Job Title
Patrick Bourke
Post Office — Corporate Affairs Director
Mark Underwood
Rodric Williams
Post Office ~ Head of Portfolio: Legal, Risk & Governance
Post Office — Post Office Legal
Rod Ismay
Post Office - Head of Finance Service Centre
Lorraine Garvey
Post Office - Enquiries Manager
‘Sarah Haywood
Post Office - Finance Team Leader
Tracy Middleton
Post Office - Finance Team Leader
Paul Smith Post Office - Operations Support Manager
Loma Evans Post Office - Central Data Manager
John Willacy Post Office — Financial Control Framework Manager
Neil Page Post Office ~ Client Settlement Team
Gillian Hoyland Post Office — Operational Support Manager
Joy Lennon Post Office - Master Data Manager
Andy R Pearson
Post Office - Finance
Debbie Gratton
Post Office — Finance
Shirley Hailstones
‘Stuart Nesbit Post Office — Finance Director
Phillip Jeary Post Office - Finance
von Hulme Post Office - Domain Architect
Post Office — Support Services Resolution Team Manager
Katherine Alexander
Post Office — Support Services Resolution Team Manager
John Simpkins SSC Team Leader
Paul Stewart Fujitsu — Database Administrator
Ken Westfield Fujitsu - Change Manager
Michael Greene
Fujitsu — Support Technician
Michael Harvey
Pete Newsome
Fujitsu - Head of Commercial
Fujitsu - Business Change Manager
Torstein O'Godeseth
Fujitsu - Chief Architect
Steve Bansal
Fujitsu - Senior Service Delivery Manager
Alan Holmes
Fujitsu - Customer Solution Architect,
Gerald Barnes
Fujitsu - Senior Software and Solutions Designer
Gareth Seemungal
Fujitsu - Senior Software and Solutions Designer
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
POL00139537
POL00139537
Appendix 2
Scope area 1 -Analytics Procedures
Ref I Analytics Procedure
‘A__I Completeness Test - Identify gaps in audit log sequencing
Completeness Test - Identify gaps in transaction times during working hours
C__I Completeness Test - Identify two user logon events in sequence without the expected logoff event in
between, an indicator of a connectivity issue
D__I Completeness Test - Identify recovery transactions
E _I Accuracy Test - Identify zero valued transactions
F I Accuracy Test - Identify branches which are out of balance based on transactional data available (should
not be possible based on inherent system controls).
G__I Integrity Test - Identify transactions posted by non-branch users without subsequent branch
acknowledgement.
H_I Integrity Test - Identify balancing transactions.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 82
Appendix 3
Scope area 2 - Balancing Transactions Controls
Ref _ Control Description
A__I SSC will have privileges of only inserting balancing / correcting transactions to relevant tables in the
database. SSC will not have any privileges to update or delete records in the database.
B__ I Ifthe process fails (e.g. transaction file is found to be invalid), then the transaction file will not be moved
and an error message will be written to standard output
C I Any writes by the SSC to BRDB must be audited. The mechanism for inserting a correction record must
ensure that the auditing of that action performed must be atomic.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537
Appendix 3a
Scope area 2 - Balancing Transactions Controls (Broader population)
Ref
Control Description
All inserts will be audited in the table BRDB_TXN_CORR_TOOL_JOURNAL.
‘The PLISQL package PKG_8RDB_TXN_CORRECTION will be owned by Oracle user
“OPS$SUPPORTTOOLUSER’,
‘The PLISQL package PKG_BRDB_TXN_CORRECTION will execute with the permissions of the
OPS$SUPPORTTOOLUSER account and can only insert rows into the transaction tables as controlled by
an entry in BRDB_SYSTEM_PARAMETERS. The account will not have update or delete privileges,
Each of the transaction tables that are allowed to have balancing transactions inserted on them has an
associated template file. Each file contains a template of an INSERT statement for that table, in the
required format, and listing all of the columns on the table. Users should create their own transaction file
based upon the relevant template file, substituting the values they require into the SQL. Note that some of
the column values specified in the template should not be changed ~ these are annotated with comments
as appropriate.
When execution is complete the file is then moved to directory ‘/app/brdb/trans/support/brdbx01S/output’
and the log file is created in directory ‘/app/brdb/trans/support/brdbx01 Silog’. Log file will be named using
the following convention:
<transaction_file_name>_<CCYYMMDDHHMISS> log
Access to these 2 directories is appropriately restricted.
Itis expected that only a small number of skilled staff will run this tool and that they will have detailed
guidance as to when and how to use the tool (For example by restriction of staff to
“OPSSSUPPORTTOOLUSER’)
From the Unix command prompt, execute the following
JBRDBX015.sh MyTransactionFile.sql 2001
where the first parameter is the transaction file name and the second parameter is the branch code where
the balancing transaction is going to be applied. Note that the branch code must exist in the database, and
must not be for a closed branch. If this is not the case, then an error message will be shown and the run
aborted
‘The correction tool places a number of constraints on the contents of the transaction file. These are
necessary in order to provide a defined baseline upon which it can base its operation. If any of the
constraints are violated then validation will detect it and abort the run with a meaningful error message.
The constraints are as follows:
‘+ The transaction file must be less than 32K in size
‘+The transaction file must only contain Unix-style end of line markers (EOL), not DOS format end of line
markers (CRIEOL)
‘+ The transaction file can only contain a single SQL statement. If more than one balancing transaction is
required then more than one transaction file must be created, each of which is executed with a separate run
of the tool
‘+ Ifthe transaction file contains an introductory comment, then it must be a ‘* ...... “7 style comment, not
a‘ ......" style comment
‘+The closing “of the introductory comment must have a trailing space (ie. '.....*/")
‘+The run symbol at the end of the SQL must be a“, not /’, and must have a trailing space (i.e. ‘....\')
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT a4
POL00139537
POL00139537
POL00139537
POL00139537
Ref Control Description
‘+ The SQL must be a valid SQL statement according to the normal Oracle SQL parsing rules (e.g. valid
syntax, objects accessible etc)
‘+ The SQL must begin with ‘INSERT INTO OPS$BRDB.’ and be of the form ‘INSERT INTO
FROM dual, (SELECT ..... FROM .... WHERE .....).
+ The table name must ~be one of the_—tables ~=s named =—in_—the
BRDB_TXN_CORRECTION_ALLOWED_TABLES1 or
BRDB_TXN_CORRECTION_ALLOWED_TABLES2 configuration parameters
‘+ All of the columns that exist on the table in question must be explicitly named. It is not necessary for
every listed column to be on a separate line, but this is advisable for readability.
‘+ The values to be inserted must be provided by the ‘SELECT ... FROM dual ...’. Each value must be on
a separate line. Trailing comments are allowed, but must be " style comment. Any such comment
‘must not include any commas. All columns must have values provided for them (even if that value is NULL).
‘+ Certain columns are common between a subset of the transaction tables. In some cases, these columns
should be set to the same value no matter what table is in use. With the exception of the bind variables
listed earlier, the value that the SQL will try to insert is under the control of the user (ie. itis determined by
the value specified in the SQL). However, the tool can be configured to validate that the value specified in
the SQL matches that expected. In. order_ «sto. «I do—sthis,-~sset_—the
BRDB_TXN_CORRECTION_ENFORCED_VALUES configuration parameter to include the field and the
required value.
‘The parameter is populated as a comma-delimited list of name/value pairs, where the name is the name of
the column name, and the value is the value to be enforced. As released, this configuration parameter is
set to:
NODE_ID=99,APP_SERVER_NODE_NAME=999,BRANCH_USER=:bind_SSC_user,BRDB_INSTANCE
_NAME=:bind_instance_name
which, for example. ensures that if a ‘node_id’ column exists on the transaction table, it's value is specified
‘as 99. If there is no ‘node_id’ on the transaction table, then no value is enforced for that field. Note that if
the parameter does not exist, then no values are enforced in the SQL.
SELECT
1 I The SAL statement being executed will be logged in the table BRDB_TXN_CORR_JOURNAL. The format
of the data to be written to the column JOURNAL_XML is:
“<?xml version="1.0" encoding="UTF-8"2>
<Support_insert>
<Unix_User>Unix User Name</Unix_User>
<Oracle_User>Oracle User Name</Oracle_User>
<Sql>SAL Statement</Sql>
</Support_Insert>”
where :
‘© Unix User Name is the Unix user name under which the user logged in
+ Oracle User Name is Oracle user that is carrying out the actual insert ie. SUPPORTTOOLUSER
‘+ SQL Statement is the final (i.e. after substituting actual values for bind variables) SQL that is executed
to insert the balancing transaction
J I Astecords are being written to the audit files, the process must optionally be able to monitor if the set of
Journal-Sequence-Numbers for a node in a Branch is dense. The check should only be performed when
the value of mandatory System-Parameter JOURNAL_SEQ_DENSE_SET_CHECK_ENABLED’ is
“TRUE”. When a missing journal entry is encountered, a message should be written on standard output
along the lines of *...reords between sequence numbers M and N are missing...”. Once the list of
auditable messages for a node is completed, an Operational exception should be raised to indicate the
count of missing sequence numbers. Duplicate records are not possible due to the primary key on this
table.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 85
Ref
Control Description
Unix shell script BRDBXO15.sh whi the /app/brdb/trans/support/brdbx015 directory. It is deliberately
kept separate from the standard $8RDB_SH directory so that access to the script and the associated
components can be restricted to authorised users. The shell script calls the PL/SQL package
PKG_BRDB_TXN_CORRECTION.
PUSQL package PKG_BRDB_TXN_CORRECTION, which resides within the Branch Database and is
owned by Oracle user OPS$SUPPORTTOOLUSER. The PLISQL package is the component that validates,
creates and audits the balancing transaction.
fan Oracle node/instance failure occurs, the utility will fail with an error code of 99. For all other failures, it
will fail with an error code of 1 and log an operational exception in BRDB_OPERATIONAL_EXCEPTIONS.
‘The SQL in the transaction file is validated as follows. Any validation failures are displayed to standard
output and logged to the log file.
‘+ Check that the file does not contain any carriage retums, indicating DOS format EOL markers
‘+ Check that the SQL in the transaction file parses according to the standard Oracle rules (e.g. syntax,
privileges etc). This is done using the standard Oracle DBMS_SQL.PARSE procedure.
‘+ Check that there is only a single SQL statement in the transaction file. Note that in most cases, this will
be detected by the previous parsing step. However, the fact that the parsing does this is not described in
the Oracle documentation, so it may be changed in future releases of Oracle. Therefore, this validation
provides security if the behaviour of the Oracle procedure is changed at a later date.
‘* Check that the SQL begins with ‘INSERT INTO OPS$BRDB.’
+ Check that the table named in the SQL is one of the tables listed in the two
BRDB_TXN_CORRECTION_ALLOWED_TABLES«<n> configuration parameters. Note that as long as the
privileges are set up correctly (i.e. OPSSSUPPORTTOOLUSER only has insert privileges on the allowed
tables), any attempt to insert a balancing transaction on a non-allowed table will cause the previous parsing
step to fail (because the user would not have the necessary privileges). Therefore, this validation provides
security in case the privileges are not correctly set up.
‘+ Check that all the columns named in the SQL exist on the table, and that all the columns on the table
are named in the SQL
‘+ Check that the values to be inserted are provided by a SELECT ... FROM dual, (SELECT ... FROM
WHERE) i.e. not a VALUES
* Check that if any of the name/value pairs that «are listed inthe
BRDB_TXN_CORRECTION_ENFORCED_VALUES configuration parameter are present on the table, they
are set to the listed value.
Balancing transaction audit files (SRDBCO33), unlike the files produced by BRDBC002, are not
compressed, but are still encrypted.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 86
POL00139537
POL00139537
Appendix 4
Scope area 3 - Audit Store Controls Listing
Ref _ Control Description
A
‘Audit tracks that are gathered at one data centre are replicated to the Audit server at the remote data
centre. This replication process is managed by the Audit Track Sealer. As Audit tracks are secured to the
‘Audit archive, they are moved to an export area awaiting transfer to the remote campus. A second file,
containing the calculated seal value for the audit track is also stored in the export area.
‘Audit tracks and seals are copied, using robocopy, to the equivalent import area on the remote audit server
as part of Audit server overnight schedule. On arrival, the sealer on the remote audit server recalculates the
seal value of the imported audit track and compares it with the original value in the imported seal file.
‘Assuming they match, the file is then written to the remote Audit archive. If the seals do not match, the Audit
track and seal file are moved to a holding area and an event is raised. Manual investigation is necessary to
investigate the cause of the discrepancy
There will be a single instance of the ATS that concurrently accepts files for sealing/seal checking from ATG
and ATR and notifies sealed files to the ATD and into the Sealer Database for subsequent use by the Audit
Track Extractor.
The ATS shall collect files for sealing via /ATS-4 and shall write a log of its activities to the ATD via -ATS-
2. In sealing a file the seal shall be generated using a secure hash algorithm, the MDS algorithm has been
selected.
‘Once a file has had a seal calculated the file will be written to Centera and details will be stored in the Audit
Track Seal Database via -ATS-5.
‘Access to the Audit Track files for gathering shall be via Samba (for Unix systems) or NTFS (for Windows
systems). Access to the sub directory shall be limited to the application generating the Audit Track and the
I Audit Track Gatherer. Audit track files should be written in write-append mode.
All users (including administrators) of the Audit Workstation and Audit Server shall log onto systems using
two factor authentication in conjunction with the HNG-X Active Directory system. Each user shall be
uniquely identifiable.
The remote directories from which the Audit Server gathers Audit Tracks will be configured so that only the
Audit Server (or an administrator who has been explicitly given permission) is able to delete files in the
directory.
Al Audit Server and Audit Workstation and Centera hardware shalll be held in physically secure areas
where physical access to the systems is controlled.
There shall be separate roles for:
. ‘Audit Server (inc. Audit Workstation) Administration
. Fujitsu Services Audit Staff
‘The roles shall be mutually exclusive, i.e. no one individual shall be given access rights of more than one
role.
‘The Fujitsu Services Audit Staff role shall not have any write, modify or delete access to the Audit Archive.
The following integrity checks will be applied to the data
. Completeness of data - contiguous message sequence numbers
. Integrity of individual messages
For Riposte data the message CRC should be checked
° For HNG-X data the message signature will be verified
Separate Riposte and HNG-X summaries of the results of the integrity checks are generated. They should
detail
. ‘Summary of the message sequence runs broken down by counter Id. This should include start and
end date/times and start and end message sequence numbers. Any gaps in the message sequence runs
must be highlighted.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 87
POL00139537
POL00139537
POL00139537
POL00139537
Ref I Control Description
. ‘Summary of messages that have failed individual message integrity checks
‘Any failure of the data integrity checks will not prevent subsequent execution of the query. The audit
workstation user will be warned of the failure via the server process status notification mechanism.
K I As Audit tracks are retrieved from the archive, they are seal checked (by re-application of the MD5 message
digest function) to ensure that the source data has not been tampered with while it was stored in the
archive.
L__I Only authorised users may access the Audit workstation applications. Authorised users are required to log
on to the workstation using two factor authentication and the HNG-X Identity Management system. An
Active Directory group named AUDIT_USER will be created with the rights required to utilise the
_ workstation applications. Authorised users will be added to this group.
M__Allretrievals of audit data are performed using the Audit Extractor Client, and all such user actions are
themselves audited. Itis not possible for users to access the archive by any other means.
N__I Audit workstations and Atalla NSPs are located in secure areas. Only authorised users are given physical
access to these areas.
O All auditable messages logged during a calendar day will be made available to the audit system in
uncompressed form as a part of Branch Database batch overnight processing.
The message journal is implemented in the form of a single Oracle table named
BRDB_RX_MESSAGE_JOURNAL. Uniqueness is controlled at the level of a Branch counter using a dense
sequence known as the Journal-Sequence-Number
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 88
POL00139537
POL00139537
Appendix 4a
Scope area 3 - Audit Store Controls Listing (broader population)
Ref _ Control Description
‘A__I The following operating system level events on the Audit Server will be audited via the System Management
event monitoring facilities
* Log oniLog off (including unsuccessful log on attempts)
* File Creation, Deletion and Modification (on selected files)
* Modifications to system configuration (inc software configuration and account details)
+ System start up and shut down
+ Recovery actions
+ Exception conditions
+ Change of user rights
B I The Audit Server Administrator role shall have full access to manage all of the Audit Server and Audit
Workstation file stores and shall be granted the necessary Windows privileges.
C I Post Office staff will not be given direct access to the Audit Workstation to safeguard other parts of the
HNG-X system. Instead nominated Fujitsu Services personnel will supply audit information as requested by
Post Office
D I User Log/On events are included in the Windows event log. Users are allocated to a specific role which
enables them to access the Audit databases.
E I Baskets are stored for a defined period of time. The configuration of this parameter and the audit trail
‘around changes to it need to inspected in order to provide assurance over the maintenance time period for
I audit purposes.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT a9
POL00139537
POL00139537
Appendix 5
Change Control - list of controls and their change dates.
Whilst it has not been
, corroborated by review of
1 ta ae INe * “ technical documentation /
. testing it is expected this
control applied in Riposte.
In Riposte this control is of less
, importance given each Branch
1 tb aeiaindco en No 7 . 3 No There is no visibility of
connate reconciliation controls in place
7 between local and central
databases in Riposte.
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 20
POL00139537
POL00139537
Digital Signature did not exist
in Riposte, However a CRC
check was applied, which
whilst Fujitsu assert that this is
less complex than the digital
copied Mensane courel signature check, and it is noted
1 te eat intent soul I No . ‘ 2 that this check has not been
ty Beech Database tested in detail, if operating
y correctly the check would
notify Fujitsu on retrieval of
audit data from the Audit Store
if any amendments to data had
been made.
The changes
introduced are
assumed to be ‘Win in
Mails’. As part of this
Release I initiative an extra file
votamed I iS feceived from
Pos Paystation and used
Any non-Counter originated reviewed. al)
interface files (POLSAP or R13and__ I Seento Royal Mail) irems on I N/A-see
1 1d third party sources) must be I Yes Ri30g, I SOCuTe To eee stad I Shonge'to len I N/A=see change to left I} {Formatted Table }
Transaction Accepted by a various reflecting postal lems ve
the Eranen: managemen I delivered to and from
trove! I the branch but there
BL testing I nO financial impact
tape on the branch from
s this.
The transactions
impacting the
financial state of the
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 1
POL00139537
POL00139537
branch are received
in the same file as
previously - i.e. via
Transaction
Acceptance.
In the event of connectivity ‘As each branch operated its
{ te failure there is a transaction I \), . . . Vea ‘own database, transaction
recovery process which is recovery processes were of
initiated. less importance in Riposte.
Review case data for
transactions indicating
items of potential risk from a
N/A Data I N/AData_ I N/A Data NIA Data
1 3 ‘system functionality N/A Data Procedure N/A Data Procedure
parspective (e.g tecovery I Procedure I Procedure I Procedure Procedure
transactions are present in
the case data).
Source code was reviewed at
a point in time. The Digital
Signature did not exist in
; Riposte. However a CRC
coer atu ion check was applied, which
feciiqoarters Which whilst Fujitsu assert that this is
pong less complex than the digital
1 5 earl ‘operation peeks No 6 . a Yes signature check, and it is noted
digitally signing transactions eta WFeneniiee.
posted from the Counter to coreetl the check would
the Brenen Database. notify Fujitsu on retrieval of
audit data from the Audit Store
if any amendments to data had
been made.
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 92
POL00139537
POL00139537
Review of existing sources
of assurance around
Change Control and
1 2 confirmation of relevant N/A (this I N/A (this I N/A (this
coverage — plus targeted —_I procedure) I procedure) I procedure)
testing to attempt to identify
NIA (this
NIA (this procedure) I re edure)
NIA (this procedure)
changes relevant to the key
controls on Horizon.
Review of population of
balancing transactions (to
Fi 4 validate population of N/AData I N/AData I N/A Data
Balancing Transactions Procedure I Procedure I Procedure
relative to total transaction
volumes)
Review source code on
soreen at Fujitsu
1 : headquarters which
supports the key inherent
control operation around:
Source code was reviewed at
i - Refer te control 11 No - - a point in time. Please refer to
1 5b Refer to control 1.2 1.1-1.5.
N/A Data
N/A Data Procedure I procedure
N/A Data Procedure
1 Se Refer to control 1.3,
1 5d Refer to control 1.4
1 Se Refer to control 1.5
Any writes by Fujitsu
support staff to BRDB must
2 2 be audited. The mechanism I No - - - NA
for inserting a correction
record must ensure that the
Its not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 93
auditing of that action
performed must be atomic.
Fujitsu support staff cannot
amend audit files for
Balancing Transactions.
No
NIA
Its not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
Fujitsu support staff will
have privileges of only
inserting balancing /
correcting transactions to
relevant tables in the
database. SSC will not have
any privileges to update or
delete records in the
database.
No
NIA
Its not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
Review case data for
Balancing Transactions to
validate population of
Balancing Transactions
relative to total transaction
volumes (Balancing
transactions should be
inherently rare, and only
deployed in response to
actual loss/bugs in code.)
N/A Data
Procedure
N/A Data
Procedure
N/A Data
Procedure
N/A Data Procedure
N/A Data
Procedure
N/A Data Procedure
Review source code on
screen at Fujitsu
headquarters which
supports the key inherent
No
NIA
Its not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
POL00139537
POL00139537
control operation around
Balancing Transactions,
The Digital Signature did not
exist in Riposte. However a
CRC check was applied, which
whilst Fujitsu assert that this is
pineal less complex than the digital
btioen BRDB signature check, and it is noted
2 6 Aainietatorrand Key —_I NO . z é No that this check has not been
Management Software tested in detail if operating
Administration. correctly the check would
nolify Fujitsu on retrieval of
‘audit data from the Audit Store
if any amendments to data had
been made.
Validate inherent system
control around Global
Users, that Global users
with a'Role of ADMIN Fujitsu represented that no
; ; cannot log onto to any - ‘ . . _ such equivalent role or ability
Branch other than Global to remote access onto
(Including Remote access counters existed in Riposte,
controls to branch
infrastructure (e.g.
Counter))..
Review a sample of the full
population (already
extracted by Fujitsu-7.5 I N/AData_ I N/AData__ I N/A Data NIA Data
2 9 years) of balancing Procedure I Procedure I Procedure I W/ADataProcedure I procedure N/A Data Procedure
transactions to validate the
branch was aware of their
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 95
usage /no transactional
postings were made in the
balancing transaction.
Review of Transaction
Correction source code on
screen at Fujitsu
headquarters to validate
that Transaction ;
2 1 Corrections must be No . - 5 NIA ae
accepted by branches, in Pp .
order to validate Balancing
Transactions are the only
transactions branches
would not have to accept.
Review the 9 Balancing
yadets bela It is not known whether
2 12 transactions would, ifthe I No - : > NIA Balancing Tensei (or ;
template was followed, equivalent) and associated tool
logically perform as existed in Riposte.
expected,
Release
notes
obtained F
Walkthrough of a and Penn Tae for
Transaction Correction Release I feviewed. I Fone ie eloase
2 13, being raised by SCC, and I Yes a Seento I Changed at Release I see Left See Left
the nemicalion I secoplancs) . document I introducing Client File
of it by a branch various introducing
managemen By:
treviews /
approvals
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
96
POL00139537
POL00139537
and testing
steps.
‘SSC will have privileges of
only inserting balancing /
correcting transactions to
relevant tables in the
It is not known whether
Balancing Transactions (or
R account and can only
insert rows into the
2 fa database. SSC will not have I N° - NIA equivalent) and associated tool
any privileges to update or existed in Riposte,
delete records in the
database,
All inserts will be audited in Its not known whether
3 Ie the table No . WA Balancing Transactions (or
BRDB_TXN_CORR_TOOL equivalent) and associated tool
JOURNAL. existed in Riposte,
The PLISQL package
PKG_BRDB_TXN_CORRE Itis not known whether
9 sp CTION willbe ownedby I x, . WA Balancing Transactions (or
Oracle user equivalent) and associated tool
“OPS$SUPPORTTOOLUS existed in Riposte,
ER’.
The PLISQL package
PE RDB ON CORRE It is not known whether
will execute with the ye no memn veel
2 5e permissions of the No 5 NIA alancing Transactions (or
equivalent) and associated tool
OPSSSUPPORTTOOLUSE
existed in Riposte.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
97
POL00139537
POL00139537
POL00139537
POL00139537
transaction tables as
controlled by an entry in
BRDB_SYSTEM_PARAME
TERS. The account will not
have update or delete
privileges
Each of the transaction
tables that are allowed to
have balancing transactions
inserted on them has an
associated template file,
Each file contains a
template of an INSERT
statement for that table, in
the required format, and
listing all of the columns on eet ee ee or
2 5d the table. Users should I No 7 : NIA ancing :
the ibis. Voers should equivalent) and associated tool
e existed in Riposte,
le based upon the relevant
template file, substituting
the values they require into
the SQL. Note that some of
the column values specified
in the template should not
be changed ~ these are
annotated with comments
as appropriate
When execution is complete
the file is then moved to It is not known whether
2 5e directory No . . . NA Balancing Transactions (or
‘Tapplbrdb/trans/supporvbrd equivalent) and associated tool
bx015/output’ and the log existed in Riposte.
file is created in directory
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 98
‘Tapplbrdb/trans/supporvbrd
bx015/log’. Log file will be
named using the following
convention:
<transaction_file_name>_<
CCYYMMDDHHMISS>.log
‘Access to these 2
directories is appropriately
restricted.
If the process fails (e.g.
transaction file is found to
be invalid), then the
transaction file will not be
moved and an error
message will be written to
standard output.
NIA
Its not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
Itis expected that only a
small number of skilled staff
will run this tool and that
they will have detailed
guidance as to when and
how to use the tool.
NIA
It is not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
From the Unix command
prompt, execute the
following
5g
JBRDBX015.sh
MyTransactionFile.sql 2001
No
NIA
Its not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
99
POL00139537
POL00139537
POL00139537
POL00139537
where the first parameter is.
the transaction file name
and the second parameter
is the branch code where
the balancing transaction is
going to be applied. Note
that the branch code must
exist in the database, and
must not be for a closed
branch. If this is not the
case, then an error
message will be shown and
the run aborted
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 100
The SQL statement being
executed will be logged in
the table
BRDB_TXN_CORR_JOUR
NAL. The format of the data
to be written to the column
JOURNAL_XML is:
“<?xml version="1.0"
encoding="UTF-8"2>
<Support_Insert>
<Unix_User>Unix User
Name</Unix_User>
<Oracle_User>Oracle User
Name</Oracle_User>
<SqI>SQL Statement</Sql>
</Support_Insert>”
where :
+ Unix User Name is the
Unix user name under
which the user logged in
+ Oracle User Name is
Oracle user that is carrying
out the actual insert i.e
SUPPORTTOOLUSER
+ SQL Statement is the final
(ie. after substituting actual
values for bind variables)
SQL that is executed to
insert the balancing
transaction
No
NIA
itis not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
POL00139537
POL00139537
Any writes by the SSC to
BRDB must be audited. The
mechanism for inserting a
correction record must
ensure that the auditing of 7 No. own database, BRDB did not
‘As each branch operated its
that action performed must
be atomic. There also
needs a level of obfuscation
to ensure that the audit
mechanism is robust.
exist in Riposte.
As records are being written
to the audit files, the
process must optionally be
able to monitor if the set of
Journal-Sequence-Numbers
for a node in a Branch is
dense. The check should
only be performed when the
value of mandatory System-
Parameter JSN check in its current format
“JOURNAL_SEQ_DENSE_ did not exist in Riposte.
2 5) SET_CHECK_ENABLED’ is I No - : - No However Fujitsu assert that a
“TRUE”. When a missing data density check was
journal entry is applied.
encountered, a message
should be written on
standard output along the
lines of *...records between
sequence numbers M and
Nate missing...”. Once the
list of auditable messages
for a node is completed, an
Operational exception
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 102
POL00139537
POL00139537
should be raised to indicate
the count of missing
sequence numbers.
Duplicate records are not
possible due to the primary
key on this table.
Unix shell script
BRDBXO15.sh which is in
the
Jappibrdb/trans/support/brd
bx015 directory. It is
deliberately kept separate
from the standard It is not known whether
2 5k SBRDB_SH directory so I 4, . . . WA Balancing Transactions (or
that access to the script and equivalent) and associated tool
the associated components. existed in Riposte.
can be restricted to
authorised users. The shell
script calls the PLSQL
package
PKG_BRDB_TXN_CORRE
CTION.
PL/SQL package
PKG_BRDB_TXN_CORRE
CTION, which resides
within the Branch Database It is not known whether
2 si and is owned by Oracle ne . . . WA Balancing Transactions (or
user equivalent) and associated tool
OPSS$SUPPORTTOOLUSE existed in Riposte.
R. The PLISQL package is
the component that
validates, creates and
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 103
audits the balancing
transaction.
If an Oracle node/instance
failure occurs, the utility will
fail with an error code of 99.
For all other failures, it will
fail with an error code of 1
and log an operational
exception in
BRDB_OPERATIONAL_EX
CEPTIONS.
No
NIA
It is not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
5n
The SQL in the transaction
file is validated as follows.
Any validation failures are
displayed to standard
output and logged to the log
file.
+ Check that the file does
not contain any carriage
returns, indicating DOS
format EOL markers
+ Check that the SQL in the
transaction file parses
according to the standard
Oracle rules (e.g. syntax,
privileges etc.). This is done
Using the standard Oracle
DBMS_SQL.PARSE
procedure.
No
NIA
Its not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
POL00139537
POL00139537
+ Check that there is only a
single SQL statement in the
transaction file. Note that in
most cases, this will be
detected by the previous
parsing step. However, the
fact that the parsing does
this is not described in the
Oracle documentation, so it
may be changed in future
releases of Oracle.
Therefore, this validation
provides security if the
behaviour of the Oracle
procedure is changed at a
later date.
+ Check that the SQL
begins with ‘INSERT INTO.
OPS$BRDB.’
+ Check that the table
named in the SQL is one of
the tables listed in the two
BRDB_TXN_CORRECTIO
N_ALLOWED_TABLES<n>
configuration parameters.
Note that as long as the
privileges are set up
correctly (i.e.
OPS$SUPPORTTOOLUSE
R only has insert privileges
on the allowed tables), any
attempt to insert a
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 105
POL00139537
POL00139537
balancing transaction on a
non-allowed table will cause
the previous parsing step to
fail (because the user would
not have the necessary
privileges). Therefore, this
validation provides security
in case the privileges are
not correctly set up.
+ Check that all the columns
named in the SQL exist on
the table, and that all the
columns on the table are
named in the SQL
+ Check that the values to
be inserted are provided by
a SELECT ... FROM dual,
(SELECT ... FROM...
WHERE) ie. not a VALUES
+ Check that if any of the
name/value pairs that are
listed in the
BRDB_TXN_CORRECTIO
N_ENFORCED_VALUES
configuration parameter are
present on the table, they
are set to the listed value.
Balancing transaction audit Itis not known whether
; - files (BRDBCO33), unlike I . . . WA Balancing Transactions (or
the files produced by equivalent) and associated tool
BRDBC002, are not existed in Riposte.
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 106
compressed, but are still
encrypted.
The correction tool places a
number of constraints on
the contents of the
transaction file. These are
necessary in order to
provide a defined baseline
pon which it can base its
operation. If any of the
constraints are violated then
validation will detect it and
abort the run with a
meaningful error message.
The constraints are as
follows:
+ The transaction file must
be less than 32K in size
+ The transaction file must
only contain Unix-style end
of line markers (EOL), not
DOS format end of line
markers (CR/EOL)
+ The transaction file can
only contain a single SQL
statement. If more than one
balancing transaction is
required then more than
‘one transaction file must be
created, each of which is
executed with a separate
run of the tool
No
NIA
Its not known whether
Balancing Transactions (or
equivalent) and associated tool
existed in Riposte.
POL00139537
POL00139537
I {Formatted Table
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
107
POL00139537
POL00139537
+ If the transaction file
contains an introductory
comment, then it must be a
Pr ssa. *f style comment,
nota ‘~......"style
comment
+ The closing “/ of the
introductory comment must
have a trailing space (i
+ The run symbol at t
of the SQL must be a
not ‘7, and must have a
trailing space (i.e.
+The SQL must be a valid
‘SQL statement according to
the normal Oracle SQL
parsing rules (e.g. valid
syntax, objects accessible
etc.)
+ The SQL must begin with
‘INSERT INTO
OPSSBRDB.’ and be of the
form ‘INSERT INTO
‘one of the tables named in
the
BRDB_TXN_CORRECTIO
N_ALLOWED_TABLES1 or
BRDB_TXN_CORRECTIO
N_ALLOWED_TABLES2
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 108
POL00139537
POL00139537
configuration parameters
+ All of the columns that
exist on the table in
question must be explicitly
named. It is not necessary
for every listed column to be
on a separate line, but this
is advisable for readability.
+ The values to be inserted
must be provided by the
"SELECT ... FROM dual
...', Each value must be on
a separate line. Trailing
comments are allowed, but
must be a ‘-.....’ style
comment. Any such
comment must not include
any commas. All columns
must have values provided
for them (even if that value
is NULL).
+ Certain columns are
common between a subset
of the transaction tables. In
some cases, these columns
should be set to the same
value no matter what table
is in use. With the exception
of the bind variables listed
earlier, the value that the
SQL will try to insert is
under the control of the user
(ie. itis determined by the
value specified in the SQL).
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 109
POL00139537
POL00139537
However, the tool can be
configured to validate that
the value specified in the
SQL matches that
expected. In order to do
this, set the
BRDB_TXN_CORRECTIO
N_ENFORCED_VALUES
configuration parameter to
include the field and the
required value.
The parameter is populated
as a comma-delimited list of
name/value pairs, where
the name is the name of the
column name, and the
value is the value to be
enforced. As released, this
configuration parameter is
set to:
NODE_ID=99,APP_SERVE
R_NODE_NAME=999,BRA
NCH_USER=:bind_SSC_us
er, BRDB_INSTANCE_NAM
ind_instance_name
which, for example. ensures
that if a ‘node_id’ column
exists on the transaction
table, it's value is specified
as 99. If there is no
‘node_id' on the transaction
table, then no value is
enforced for that field. Note
that if the parameter does:
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 110
not exist, then no values are
enforced in the SQL.
Validate inherent system
controls around Global
Users, notably that Global
users with a Role of ADMIN
cannot log onto to any
Fujitsu represented that no
such equivalent role or ability
r Branch oiher than Global I N° ves to remote access onto
(Including Remote access counters existed in Riposte,
controls fo branch
infrastructure (e.g.
Counter)),
Audit tracks that are
gathered at one data centre
are replicated to the Audit
server at the remote data
process is managed by the techni yirevie\
Audit Track Sealer. As Audit nical documentation /
tacks ero soared the testing it is expected this
ta beep heeg No No control applied pre HNG-X.
Pole Apt Fujitsu attested that controls
moved to an export area i!
awaiting transfer to the ‘surrounding the Audit Store
ince eaiaia A pscond have remained largely
file, containing the unohanged.
calculated seal value for the
audit track is also stored in
the export area.
Digital Signature controls Digital Signature did not exist
2 applied to Message Journal I .,, "Vos in Riposte, However a CRC
during initiation of transfer check was applied, which
to Branch Database. whilst Fujitsu assert that this is
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
1
POL00139537
POL00139537
less complex than the digital
signature check, and it is noted
that this check has not been
tested in detail, if operating
correctly the check would
notify Fujitsu on retrieval of
audit data from the Audit Store
if any amendments to data had
been made.
Identification of Audit Store
Data Flows at a Detailed
Level, including security
Whilst it has not been
corroborated by review of
technical documentation /
testing itis expected this,
controls over data atrest, I No No control applied pre HNG-X.
and completeness, Fujitsu attested that controls
accuracy and validity surrounding the Audit Store
controls over data in transit. have remained largely
unchanged.
Review source code on
screen at Fujitsu
headquarters which Source code reviewed at a
supports the key inherent I 4,4 ‘Yes point in time. Digital signature
control operation around
digitally signing transactions
posted from the Counter to
the Branch Database.
check in its current form
originated in HNG-X
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
12
POL00139537
POL00139537
‘Agree that the system
changed to the extent
that it is now
copied, using robocopy, to
the equivalent import area
on the remote audit server
as part of Audit server
overnight schedule. On
arrival, the sealer on the
3 tb remote audit server No
recalculates the seal value
of the imported audit track
and compares it with the
original value in the
imported seal file. Assuming
they match, the file is then
written to the remote Audit
No
Hasna implemented on
Identification of changes Petained I diferent hardware. A
relevant to the Audit Store oe crucial point is that
from review of historical R10.20 I ag I the audit data was not
documentation, and (Refresh of Seen to - changed and the
validation that the Audit Eternis digital signatures NIA - see
3 6 Store has remained broadly I Y°S Storage fasrulai created in the change to let I N/A- See change to left
consistent over time from a infrastructu I YA'OUS I branches at the time
controls perspective for the re) Trovee) I that transactions were
period relevant to the eevee I oattied out were
allegations. aeeecting I persisted and
Sepe ?I demonstrate that the
ps. data in the audit trail
has not been
tampered with.
Audit tracks and seals are
Whilst it has not been
corroborated by review of
technical documentation /
testing itis expected this,
control applied pre HNG-X.
Fujitsu attested that controls
surrounding the Audit Store
have remained largely
unchanged.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
113
POL00139537
POL00139537
POL00139537
POL00139537
archive. If the seals do not
match, the Audit track and
seal file are moved to a
holding area and an event
is raised. Manual
investigation is necessary to
investigate the cause of the
discrepancy.
There will be a single
instance of the ATS that
concurrently accepts files
for sealing/seal checking
from ATG and ATR and
notifies sealed files to the
ATD and into the Sealer
Database for subsequent
lise byjttie Alli Track Whilst it has not been
Pamacior: corroborated by review of
The ATS shall collect files technical documentation /
for sealing via I-ATS-4 and testing it is expected this,
tc shall write a log of its No F - - No control applied pre HNG-X.
activities to the ATD via I- Fujitsu attested that controls
3 ATS-2, In sealing a file the surrounding the Audit Store
seal shall be generated have remained largely
using a secure hash unchanged.
algorithm, the MD5
algorithm has been
selected.
Once a file has had a seal
calculated the file will be
written to Centera and
details will be stored in the
‘© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT 114
Audit Track Seal Database
Via LATS-5.
Access to the Audit Track
files for gathering shall be
via Samba (for Unix Whilst it has not been
systems) or NTFS (for corroborated by review of
Windows systems). Access technical documentation /
3 td to the sub directory shall be I 1, testing it is expected this
limited to the application ° No control applied pre HNG-X.
generating the Audit Track Fujitsu attested that controls
and the Audit Track surrounding the Audit Store
Gatherer. Audit track files have remained largely
should be written in write- unchanged.
append mode.
All users (including
administrators) of the Audit Whilst it has not been
Workstation and Audit corroborated by review of
Server shall log onto technical documentation /
3 7 systems using two factor I 4, testing itis expected this,
authentication in lo No control applied pre HNG-X.
conjunction with the HNG-X Fujitsu attested that controls
Active Directory system. surrounding the Audit Store
Each user shall be uniquely have remained largely
identifiable, unchanged.
Jystom evel events onthe corrboraod by rove
t corroborated by review of
3 a Audit Server will be audited I N technical documentation /
via the System ii testing it is expected this
Management event li
eto re rncilice: control applied pre HNG-X.
jonitoring facilities: Fujitsu attested that controls
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
115
POL00139537
POL00139537
+ Log on/Log off (including
unsuccessful log on
attempts)
+ File Creation, Deletion and
Modification (on selected
files)
+ Modifications to system
configuration (Inc. software
configuration and account
details)
+ System start up and shut
down
+ Recovery actions
+ Exception conditions
+ Change of user rights
surrounding the Audit Store
have remained largely
unchanged.
The remote directories from
which the Audit Server
gathers Audit Tracks will be
configured so that only the
Whilst it has not been
corroborated by review of
technical documentation /
testing it is expected this
3 it Audit Server (or an No 7 . z No control applied pre HNG-X.
administrator who has been Fujitsu attested that controls
explicitly given permission) surrounding the Audit Store
is able to delete files in the have remained largely
directory. unchanged.
Release Whilst it has not been
R10.10 ‘Agree that the system °
All Audit Server and Audit and poles og I changed to the extent Pipareed by Taicy ot
Workstation and Centera R10.20 I Se that it is now ‘testing it ls copeoied thas,
3 1 hardware shall be held in (Refresh of I 2n¢ implemented on gti expert
9g ? Yes : reviewed. I No control applied pre HNG-X.
physically secure areas Etemis I Sento different hardware. Poitaatested thateoniols
where physical access to Storage I SScument I Operational curoun ding the Audit Store
the systems is controlled. fot woe prooessas were not fave tomabed lergely
managemen ged. unchanged.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
116
POL00139537
POL00139537
treviews /
approvals
and testing
steps.
There shall be separate
roles for:
+ Audit Server (Inc. Audit
Workstation) Administration
+ Fujitsu Services Audit
Staff
The roles shall be mutually
Whilst it has not been
corroborated by review of
technical documentation /
testing itis expected this
th exclusive, ie. no one No : No control applied pre HNG-X.
individual shall be given Fujitsu attested that controls
access rights of more than surrounding the Audit Store
one role. have remained largely
The Fujitsu Services Audit unchanged.
Staff role shall not have any
write, modify or delete
access to the Audit Archive.
Whilst it has not been
corroborated by review of
technical documentation /
testing itis expected this,
1i No - No control applied pre HNG-X.
Fujitsu attested that controls
surrounding the Audit Store
have remained largely
unchanged.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
417
POL00139537
POL00139537
Whilst it has not been
Te Reh seve corroborated by review of
iministrator role shall polenta coca
have full access to manage nical COCUMeNERON,
all of the Audit Server and testing itis expected this
3 3b Audit Workstation file stores I N° control applied pre HNG-X.
Fujitsu attested that controls
and shall be granted the cise mares
necessary Windows Surrollning mpsiore)
ivileges. have remained largely
privileges unchanged.
Post Office staff will not be Whilst it has not been
given direct access to the corroborated by review of
Audit Workstation to technical documentation /
safeguard other pars of the testing itis expected this
3 I3c HNG-X system. Instead I No No control applied pre HNG-X.
nominated Fujitsu Services Fujitsu attested that controls
personnel will supply audit surrounding the Audit Store
information as requested by have remained largely
Post Office. unchanged.
The following integrity
3 checks will be applied to the a :
data:
Whilst it has not been
corroborated by review of
4j No technical documentation /
+ Completeness of data — testing itis expected this,
3 contiguous message No control applied pre HNG-X.
sequence numbers Fujitsu attested that controls
‘surrounding the Audit Store
have remained largely
unchanged.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
18
POL00139537
POL00139537
4 + Integrity of individual
messages
© For Riposte data the
3 message CRC should be
checked
© For HNG-X data the
3 message signature will be
verified
Separate Riposte and HNG-
X summaries of the results
3 of the integrity checks are
generated. They should
detail:
Whilst it has not been
corroborated by review of
technical documentation /
testing itis expected this,
control applied pre HNG-X.
Fujitsu attested that controls
surrounding the Audit Store
have remained largely
unchanged.
No
Whilst it has not been
corroborated by review of
technical documentation /
testing itis expected this
control applied pre HNG-X.
Fujitsu attested that controls
surrounding the Audit Store
have remained largely
unchanged.
Yes
For Riposte CRC control
above was in place.
No
Whilst it has not been
corroborated by review of
technical documentation /
testing it is expected this
control applied pre HNG-X.
Fujitsu attested that controls
‘surrounding the Audit Store
have remained largely
unchanged.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
119
POL00139537
POL00139537
+ Summary of the message
sequence runs broken
down by counter Id. This
should include start and end
date/times and start and
end message sequence
numbers. Any gaps in the
message sequence runs
must be highlighted.
Whilst it has not been
corroborated by review of
technical documentation /
testing it is expected this
control applied pre HNG-X.
Fujitsu attested that controls
surrounding the Audit Store
have remained largely
unchanged.
Whilst it has not been
corroborated by review of
technical documentation /
+ Summary of messages testing itis expected this
3 that have failed individual No control applied pre HNG-x.
message integrity checks Fujitsu attested that controls
surrounding the Audit Store
have remained largely
unchanged.
Whilst it has not been
Any tals of the dala, corroborated by review of
Integrity chiseks will ot technical documentation /
prevent subsequent testing it is expected this
execution of the query, The :
3 4 J No control applied pre HNG-X.
audit workstation user will pw
per wll jitsu attested that controls
be warned of the failure via t t
the server proooss status surouning the Aut Sore
notification mechanism, unchanged.
As Audit tracks are Whilst it has not been
retrieved from the archive, corroborated by review of
3 1k they are seal checked (by I No No technical documentation /
re-application of the MDS testing it is expected this,
message digest function) to control applied pre HNG-X.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
120
POL00139537
POL00139537
ensure that the source data
has not been tampered with
while it was stored in the
archive.
Fujitsu attested that controls
surrounding the Audit Store
have remained largely
unchanged.
Only authorised users may
access the Audit
workstation applications.
Authorised users are
required to log on to the
workstation using two factor
authentication and the
HNG-X Identity
Whilst it has not been
corroborated by review of
technical documentation /
testing itis expected this
allocated to a specific role
which enables them to
access the Audit databases.
‘ " Management system. An I N° No Genito! appli pre HNGX
Active Directory group jitsu attested that control
named AUDIT_USER will surrounding the Audit Store
be created with the rights have remained largely
required to utilise the ged.
workstation applications.
Authorised users will be
added to this group.
Whilst it has not been
corroborated by review of
included in the Windows technical documentation /
event log. Users are testing it is expected this.
3 [3d ? No No control applied pre HNG-X.
Fujitsu attested that controls
‘surrounding the Audit Store
have remained largely
unchanged.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
121
POL00139537
POL00139537
All retrievals of audit data eoaborad by ar of
: testing itis ex is
3 1m all such user actions are oy I No control applied pre HNG-X.
jemselves audited. It is not Fujitsu affected that controls
ossibfe for users to sooess surrounding the Audit Store
archive by any other have remained largely
means. unchanged.
Whilst it has not been
; ; corroborated by review of
pve shitet aaa otal oe in technical documentation /
aocite areas, Only. testing itis expected this
3 tn authorised users are given I NO No control applied pre HNG-X.
physical acoass to'these Fujitsu attested that controls
pee surrounding the Audit Store
i have remained largely
unchanged.
All auditable messages
logged during a calendar
day will be made available
3 to the audit system in Whilst it has not been
uncompressed form as a corroborated by review of
part of Branch Database technical documentation /
batch overnight processing. testing itis expected this,
to No No control applied pre HNG-X.
The message journal is Fujitsu attested that controls
implemented in the form of surrounding the Audit Store
a single Oracle table named have remained largely
3 BRDB_RX_MESSAGE_JO unchanged.
URNAL. Uniqueness is
controlled at the level of a
Branch counter using a
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
122
POL00139537
POL00139537
dense sequence known as
the Journal-Sequence-
Number
Baskets are stored for a
defined period of time. The
configuration of this
parameter and the audit trail
3 3e I around changes to itneed I No
to inspected in order to
provide assurance over the
maintenance time period for
audit purposes.
No
Whilst it has not been
corroborated by review of
technical documentation /
testing itis expected this,
control applied pre HNG-X.
Fujitsu attested that controls
surrounding the Audit Store
have remained largely
unchanged.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
123
POL00139537
POL00139537
Appendix 6
Case Data Analytics Overview
The below analytical procedures were performed on ‘Case Data’. ‘Case data’ refers to transactional data provided by Post Office, which had been extracted by Fujitsu from
the Audit Store, and relates specifically to the branches involved in the ‘allegations’. The data extracted is in 1 month periods relating specifically to the period of the
allegations for each specific branch.
Relevant Analytics
Scope Area Post Office Instruction Proposal pai Analytic
1 Post Office consider instructing a suitably qualified party to I Post Office will instruct Deloitte to I Review case data for 12,3,4, 4a,
carry out an analysis of the relevant transaction logs for __I determine whether such an transactions indicating 5,6, 6a, 7
branches within the Scheme to confirm, insofar as is analysis/review is items of risk from a system
possible, whether any bugs in the Horizon system are feasible, and if tis, to provide an functionality perspective
revealed by the dataset which caused discrepancies in the I indication of the cost, time and (e.g. recovery transactions
accounting position for any of those branches process that would be incurred. are present in the case
data).
Tab Index Description
Analytic 4 Identify gaps in audit log sequencing
Analytic 2 Identify gaps in transaction times during working hours
Analytic 3 Identify two user logon events in sequence without the expected logoff event in between; an indicator of a connectivity issue
Analytic 4 Identify recovery transactions
Analytic 4a Identify recovery transactions that indicate a connectivity issue
Analytic 5 Count of zero valued transactions summarised by product
Identify branches which are out of balance based on transactional data available (should not be possible based on inherent system
Analytic 6 controls).
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
Analytic 6 Group and Session _ Identify branches which are out of balance based on transactional data available (should not be possible based on inherent system
id controls).
Analytic 7 Identify transactions posted by non-branch users without subsequent branch acknowledgement.
‘© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT
125
POL00139537
POL00139537
Appendix 6a
Case Data Summary Findings
Procedure
Comments
Summary
Impact
Analytic 1: Identify gaps in audit
log sequencing
In order to identify gaps in audit log
sequencing, the transactions data was
sorted into ascending order on session id
and txn id, and any gaps in the sequence at
both the session and txn level were
identified
There were 212,372 (1.60%) gaps in audit
log sequencing from a total of 13,666,238
transactions
There was an error in the original analytic
logic which was supposed to remove
duplicated transactions from the dataset but
was in actuality removing both the duplicates
and the original transactions from the data
When the analytic was corrected for this it
was noted that there were no gaps in JSN
sequencing were identified based on the data
provided.
None — following further work
performed
Analytic 2: Identify gaps in
transaction times during working
hours
In order to identify gaps in transaction times
during working hours, the transaction data
was ordered by branch, date and time.
Gaps that were significantly higher than the
average gaps in transaction times were
identified, only transactions with the same
date were compared. Transactions with a
stock unit of ATM, LOT, OOH or BUR were
excluded
There were 49,320 (0.36%) gaps in
transaction times that were more than 20
times higher than the average transaction
gap of alll stores with the same number of
positions from a total of 13,666,238
transactions
In less busy branches these could
legitimate gaps.
Extensive further manual analysis
would be required to positively
be
conclude these findings are indicative
of issues which would seem
disproportionate.
Analytic 3 : Identify two user logon
events in sequence without the
expected logoff event in between,
an indicator of a connectivity issue
In order to identify two user logon events in
sequence without the expected logoff event
in between, an indicator of a connectivity
issue the events data was ordered by date
and time and logon events (event code 12
or “EPOSSTransaction.Ti of Logon
There were a total of 1,064 (0.93%) logon
events in sequence without the expected
logoff between; from a total of 114,491 log
on/off events.
This is a low volume and could be
indicative of power / communications
fluctuation / failure. Extensive further
manual analysis would be required to
positively conclude these findings
are
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
Procedure
Comments
Summary
Impact
Completed’) not followed directly by a log
off event (event code 13, 27 and 102 or
“EPOSSTransaction. Ti of Logoff
Completed’) were identified.
indicative of issues which would
seem disproportionate.
Analytic 4: Identify recovery
transactions
In order to identify recovery transactions the
eventDetailMsg column of the Events data
was searched for words like ‘successfully
recovered’ but not like 'No recovery
required.”
There were 30 (0.00057%) recovery
transactions identified from a total of
5,289,369 transactions in the events data
This is a low volume and likely to be
indicative of expected system
functionality. Specific controls have
been tested over recovery
transactions, during our production of
this report.
Where legal counsel is aware that
part of the case may focus upon hard
reset of branch counter equipment
(eg. by physical removal of network
connectivity), these transaction types
may support that this activity was
ocourring
Analytic 4a: Identify recovery
transactions that indicate a
connectivity issue
In order to identify connectivity issues of
none recovery transactions the
eventDetailMsg column of the Events data
was searched for words like ‘could not
recover’ and 'No recovery required.”
There were 258 ‘no recovery’ transactions
that indicate a connectivity issue from a total
of 5,289,369 transactions in the events data
This is a low volume and likely to be
indicative of expected system
functionality. Specific controls have
been tested over recovery
transactions.
Analytic 5: Identify zero valued
transactions
In order to Identify zero valued transactions,
all transactions with a sale value of 0, a
quantity not equal to zero and a mode of
either 1 or SC for Serve Customer’ were
identified and a summary per item is
produced.
There were a total 1,344,773 (9.84%) zero
valued transactions with a quantity not equal
to zero from a total of 13,366,238. These
transactions were against a total of 432
products
The impact of a zero value
transaction is not likely to affect
branch accounts, unless a value
should have been present. Extensive
further manual analysis would be
required to positively conclude these
findings are indicative of issues
which would seem disproportionate.
Analytic 6: Identify branches which
are out of balance based on
transactional data available
(should not be possible based on
inherent system controls).
In order to identify branches which were out
of balance based on transactional data
available (which should not be possible
based on inherent system controls), the
transactions data was summarised by
branch (Group) and session id and those
session ids that do not sum to zero were
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
There were 48 (0.0015%) session ids from a
total of 3,124,140 which were out of balance
based on the transactional data received.
Those 48 session ids out of balance related
to 18 distinct branches from 118 in total. The
session ids out of balance were all pre
system migration to HNG-x in 2010
None — following further work
performed
POL00139537
POL00139537
Procedure
Comments
Summary
Impact
identified, and are ordered by balance
descending. The data used was filtered for
transaction mode ‘SC’ only.
The root cause for the 48 transactions
appearing not to balance was determined as:
a. Some of the audit log
sequences were missing a
start time and hence were
not extracted properly.
b. Some of the audit log
sequences were missing a
SC (Serve Customer) record
and hence were not
extracted properly.
40. These issues were shown to have
been overcome by looking at the raw
audit log sequence data (as it was the
extraction logic performed by Fujitsu
which was causing records to be
dropped).
44 Itwas confirmed through the
walkthrough with Fujitsu and through
checking the 15 sampled files
independently that there were no session
ids out of balance based on the new
transaction data provided and it was
concluded that the out of balance session
ids identified on the initial run through
were out of balance due to the 2 errors.
identified above in extracting the data
from the raw audit log sequence.
Analytic 7: Identify transactions
posted by non-branch users
In order to identify transactions posted by
non-branch users without subsequent
branch acknowledgement, any users whose
There were 19 (3.31%) users from a total of,
574 users classified as non-branch users
who posted transactions
The specific transactions are listed
below in ‘Analytic 7 detail.’ Extensive
further manual analysis on the
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
(Formatted: Indent: Let: 0.65 an, No bulets or numbering )
Procedure Comments Summary Impact
without subsequent branch id did not take the usual format (6 digits - 1** population of transactions identified
acknowledgement. letter of forename followed by 1 and 2"¢ would be required to draw
letters of surname and numeric 001) were
identified. A user id of *PS98 are Paystation
transactions and were ignored here, a user
id beginning with a * are identified as global
users
meaningful conclusions, as well as a
further understanding of the owners
of these 19 accounts. This would
seem disproportionate.
Analytic 7 detail.
BranchNo User _I Debit Value No of rows
394329 *BMAO01 233089.08 170
198424 I *JHOOS 214684.08 30
394329 I *GDRO1 204135.62 184
197941 I *NSTO1 95703.47 130
207320 I “DWAOt 91762.85 12
158644 I *JBAO3 83825.54 3
219420 I *RLYO1 74781.24 16
363642 I *DsO03 63600.32 66
260604 *TAKO1 51489.96 62
229555 I *DCU02 45022.32 7
243205 I *PJOO7 39660 12
202604 I *STUO3 29267.14 4
6458 I *DSI02 25425.82 5
266418 I “MWEO1 24724.77 6
363642 *LSHO1 23798.63 15
362217 I *JCA01 13485.55 2
282422 I *TAKO1 8382 2
225329 I *BMAO1 7500.18 4
238420 *RCRO1 5923.36 4
198424 I *TAKOt 1080 é
243205 I “GMUO1 1040 10
197941 I *PJOO2 15.07 10
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
Appendix 7
Clarification questions
The below clarification questions and associated answers attempt to provide clarity on queries arising from the
content of this report.
Key questions
1. From the perspective of the Group Action, we are trying to understand:
a.
Whether Fujitsu can edit or delete transactions recorded by branches in a way that could impact on
the branch's overall accounting position?
Yes - Transactions can be deleted at database layer (BRDB) by DBA’s.
Before Audit Store access locked down, transactions could be deleted at Audit Store level (and
still can be once a transaction has been in the Audit Store for 7 years), but this would not affect a
branch's overall accounting position unless there was a query that resulted in the extraction of
data, If data was extracted from the Audit Store and records had been tampered with or removed,
this would be flagged upon extraction by the process to report on data integrity, so it would be
transparent that the data has been edited. It should be noted the warning that the data integrity
check failed can be ignored by the operator.
How difficult it would be to do (a)?
Firstly, access to do (a) is restricted to appropriate personnel by Fujitsu. However, for users who
have DBA access on the BRDB, this could be done.
However the window of opportunity to do (a) in the BRDB is finite, if the edit/delete of the
transaction was not done before the data had been ‘collected’ by the Audit Server (typically every
15 minutes for some products with a maximum exposure in the order of 24 hours for others), then
this would not affect the record of data in the Audit Store. The Audit Store is the location where
data is retrieved from in the event of a dispute.
Any amendment to transactions after the BRDB, whilst potentially impacting the Audit Store
record, would not impact branch accounting, only the master record in the Audit Store. Further, if
the edit/delete of the transaction was performed prior to the data being ‘collected’ by the Audit
Server, whilst it would be reflected in the Audit Store date, upon retrieval of branch data from the
Audit Store, if a transaction had been removed, the ‘data density’ check would highlight a missing
transaction. If upon retrieval of branch data from the Audit Store a transaction had been
amended, the digital signature check would highlight an issue with the integrity of the data.
Whether (a) is possible without leaving a “footprint” that is visible to either (i) Postmaster or (ji) Post
Office / FJ.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 30
POL00139537
POL00139537
POL00139537
POL00139537
3) Amendment / deletion of transactions would not be overtly notified to the Postmaster, however if
the amendment / deletion happened at the BRDB, this would affect the declarations made by
Postmasters (encouraged to do so on @ daily basis) and also declarations are required to be done
in order to rollover into the next accounting period (typically 4-5 weeks). The monthly Branch
Trading Statement which @ Postmaster must sign off on in order to roll into the next accounting
period would also be impacted by a change of this nature which would capture summarised totals
of transactional data, which could be reconciled by branch back to the granular transaction log
reports. All of the mentioned reports are mechanisms by which the Postmaster would be made
aware of any such changes.
‘Amendment / deletion of data in the audit server / store has no effect on branch accounting and
would only impact a branch (Postmaster be made aware) if data was retrieved from the Audit
‘Store. Further if upon retrieval of branch date from the Audit Store @ transaction had been
removed, the ‘data density’ check would highlight a missing transaction. If upon retrieval of
branch data from the Audit Store a transaction had been amended, the digital signature check
would highlight an issue with the integrity of the data.
ii) Branch Database privileged Oracle user operations are audited by Oracle to the SYS.AUDS
table. This table is extracted into audit files every night by a batch job into a directory from which
the audit archiving system extracts the data. The audit data is currently stored for 10 years. This
table can be extracted from the Audit Store by Fujitsu.
Any amendment / deletion of data in the Audit Store would be visible to Fujitsu only when data is
retrieved. Upon retrieval of branch data from the Audit Store a transaction had been removed, the
‘data density’ check would highlight a missing transaction. If upon retrieval of branch data from
the Audit Store a transaction had been amended, the digital signature check would highlight an
issue with the integrity of the data.
As per the exception noted previously, there is @ small theoretical risk of a user ‘spoofing’ the
digital signature, arising from a failure in SOD controls relating to the digital signature, thus there
is the theoretical risk transactions could be amended with no footprint left. However to do (a)
without leaving a footprint in the system would be @ complex procedure, new ‘keys’ would need to
be generated for all messages in the session, which is a time consuming process, as such it is
likely a ‘programme’ would have to be written and performed in order to perform this.
d. Whether (a) has ever actually happened?
Audit logs of Privileged User access in the BRDB exist. Fujitsu have confirmed where amendment /
deletion of live database tables would be identifiable from this log.
Our work has not included obtaining logs for the relevant time period and performing analytics over
them to identify any instances where this has happened, and investigate if so. Such procedures
should be theoretically possible however.
2. The key points we need to understand are whether (i) Balancing Transactions and (ii) changes by
Privileged Users can effect branch accounts from the perspective of the Postmaster, in particular:
a, Are these changes visible to the Postmaster?
There is no system setting which would flag to the Postmaster when a change had been made by
@ Privileged User.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 3
POL00139537
POL00139537
The Transaction Log report gives the Postmaster a way of identifying Balancing Transactions, as
transactions that have been inserted can be identified as the associated user would be displayed
as “SUPPORTTOOLUSERS99" (i.e. not a member of staff at the Branch)
b. Can these generate a shortfall in the branch accounts?
If used in a certain way, BTS or a Privileged User change could theoretically cause a shortfall in
branch accounts.
c. How would this impact on the making of daily cash declarations?
Daily cash declarations are a real time report generated by a branch (counter) which queries the
BRDB live database; therefore any balancing transaction inserted into the BRDB or change of
transactional BRDB deta by a Privileged User, would automatically impact the daily cash rec
report (impact dependent on nature of BT / change).
d. How would this impact on "monthly" branch trading balances?
The monthly Branch Trading Statement, which a Postmaster must sign off on in order to roll into
the next accounting period would also be impacted by a change of this nature.
The monthly branch trading statement, reports on data live from the BRDB, and aggregated data
from the BRDB, therefore any balancing transaction inserted into the BRDB or change of
transactional BROB data by @ Super User, would automatically impact the daily cash rec report
(impact dependent on nature of BT / change).
© Asthis report been through multiple instances, the following queries were raised by POL and their -.__{ Formatted: Font: Aral, 10 pt, Bold, Font color: Auto
advisors, they were, deemed to be, useful to the integrity / value of the report so have been left in. {> (Formate No bles or berg
(Formatted: Font: Avia, 10 pt, Bold, Fart color: Auto
(Formatted: Font: Arial, 10 pt, Bol, Font color: Auto
4, Wate flow ovecviow: »( Formatted: Font: Arial 10 pt, Bold, Font color: Ako
(Formatted: Font: Arial, 10 p, Bold, Font color: Auto
"(Commented [A26]: updated
a. Transfer of data from BAL to BRDB - Does this happen daily? If so when during the day? Is it
overnight?
BAL is a compilation of servers used for the transfer of data from Counter to BRDB, this
processing is done in a near real time manner. As such transfer of data from BAL to BRDB is
instantaneous once a basket is complete.
i. Given the daily polling of data from which source does the Counter pull data when the
Postmaster conducts an end of day cash declaration? (The above suggests the data must
be pulled from BAL as all other sources would not be up to date in real time?)
BRDB. A request from counter is raised (via the BAL) to BRDB using pre-defined SQL
scripts at the BRDB layer to generate this cash declaration report/process. When a cash
declaration is raised by a branch a message transfer is sent via the BAL which
‘communicates with the BRDB to query the live transaction tables using a pre-defined
SQL script
b. Transaction corrections generated by Post Office: Where does a Transaction Correction fit on this
diagram?
Transaction Corrections are inserted directly into BRDB by a defined data transfer process.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 32
POL00139537
POL00139537
c. How long is data held in the Audit Server?
Most date is held in BRDB for approximately 5 days, (depending on specific type of data). Certain
values are also aggregated and the aggregated data held for up to 60 days to allow for real time
reports, and the monthly branch trading statement, ran by the counter to include this date if
required.
Most date is held on the Audit Server for approximately 5 days, (depending on specific type of
data).
2. Transaction Correction: ‘(Commented [A27]: done )
a. Can Post Office staff "amend" a transaction? If so, how? {Commented [A28]: done J
This refers to @ Transaction Correction (TC). A TC could, depending on the detail of the TC, have
the effect of ‘amending’ an existing transaction. A TC must be accepted at the counter before
impacting branch accounting.
3. BRDB and Balancing Transaction: __- (Commented [A29]: ? i}
‘(Commented [A30]: done i}
a. What is meant by the phrase: "Any writes by Fujitsu Support to BRDB must be audited"?
Branch Database privileged Oracle user operations (Fujitsu Support) are audited by Oracle to the
SYS.AUDS table.
b. What is the difference between “Correcting” and “updating"? We did not think FJ could “correct”, ‘(Commented [A31]: done )
only “insert” transactions? ‘(Commented [A32]: done )
ABT could, depending on the detail of the BT, have the effect of ‘amending’ an existing
transaction. A BT can only insert, and not update or delete existing records. The possibility of a
Privileged User amending existing transactions does exist. ‘(Commented [A33]: done J
4. BTs in relation to the Stock Unit issue:
‘a. Please can you explain the situation with using Balancing Transactions to solve the SU problem?
The usage of the BT too! for this purpose is not a ‘true’ BT as no data (transactions) is/are
injected into the database. However the same tool which allows a BT to be posted, is used to
perform this procedure.
The procedure is performed to update the transaction recovery table of a Stock Unit (SU) in the
rare instance when the recovery flag for a transaction gets into an inconsistent state, and needs
to be manually updated, to show that the transaction has been recovered by the branch.
This procedure is managed by an MSC (change request) process prior to the updates taking
place.
b. Other than the one use of a BT to solve a bug, are you sure that all other uses of BTS relate to the
SU issue?
For the period data was available for and therefore reviewed (12/03/2010 — 28/05/2016).
All other uses of the tool in this period updated the specific table
BRDB_RX_RECOVERY_TRANSACTIONS' (SU issue) and did not contain INSERT statements.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 33
POL00139537
POL00139537
¢. Will the branch be aware of the SU issue?
The Branch would not be notified of the too! being used for this purpose, however this process is
generally initiated by the branch when the branch is struggling to perform this task manually using
the counter.
d. Can the SU issue ever cause a discrepancy in the branch accounts?
The usage of the tool to update the transaction recovery table of an SU does not insert / remove /
amend transactions. So no.
5. BT audit files:
a. What do the “audit files” in relation to BTs track and show?
All usages of the tool used for inserting BTs. The logs show the actual SQL commands used to
insert the BT, and contain all fields updated and their respective values (quantities and product
ids). There are also user timestamps which identify the user who inserted the BT.
b. How far back do the audit files go?
The audit files commence at 12/03/2010
6. FJ access to conduct a BT
a. How many staff at FJ have permission to inject a BT?
31 (of these 31, 26 also have direct DBA access to the live BRDB database and therefore could
theoretically make changes to transaction tables as described in (10b) below.)
ia
What is the process followed by FJ for using a BT?
The process followed by Fi is:
An error is recognised by the branch and they raise @ request/call to SSC.
A TFS/Peak Incident service desk tool is then used to record incidents raised by Post masters
(TFS has subsequently been retired and incidents all 1st and 2nd line branch incidents are now
recorded in Peak Incident Management)
This issue will then be investigated by SSC. if @ BT is required then this is passed to Fujitsu for
further work and solution management.
Ifa BT is required this is recorded on the Peak Incident ticket.
Approvals are then sought by senior members of Post Office before this is executed which is
captured within the ticket request.
c. What operational controls are there around the use of BTs at FJ?
A branch would initiate the process described in (b) above for a BT to be executed. (Formatted: Font: (Defaut) +Body (Arial), 10 pt
Senior approvals are required by Post Office before this process can be completed.
Use of BT tool is audited and any transactions inserted would be recognised by branch through
transactional log reports.
The BT tool is restricted to a limited number of Fujitsu personnel who are independent to the Peak
incident process.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 34
POL00139537
POL00139537
{Commented [A34]: Removed as agreed )
7. BT visibility
‘a. Would a BT show in the branch accounts from a Postmaster's perspective?
i, What report would a Postmaster need to run?
A Postmaster is not notified if @ Balancing Transaction is inserted into the live transaction
tables.
There are various real time reports @ Postmaster can run which would be affected by
something of this nature (notably the Transaction Log report, which is able to display
transactions that have been posted over the last 60 days.). Transactions in this report
would be identifiable by the user code “SUPPORTTOOLUSER99" (i.e. not a member of
staff at the Branch).
Further any Balancing Transaction impacting @ branch's transactional data would impact
declarations made by Postmasters (encouraged to do so on a daily basis) and also
declarations are required to be done in order to rollover into the next accounting period
(typically 4-5 weeks). The monthly Branch Trading Statement which a Postmaster must
sign off on in order to roll into the next accounting period would also be impacted by a
change of this nature which would capture summarised totals of transactional date, which
could be reconciled by branch back to the granular transaction log reports. All of the
‘mentioned reports are mechanisms by which the Postmaster would be made aware of @
Balancing Transaction. The reporting functionality of counters was described by Fujitsu
and this understanding was corroborated by review of technical documentation, no
walkthroughs were performed of this process.
ii. How would it be identifiable from other transactions?
Transactions in the Transaction Log report would be identifiable by the user code
"SUPPORTTOOLUSERS9" (i.e. not a member of staff at the Branch).
b. Cana BT by back-dated (i.e. injected into the branch accounts at an historic date)?
Whether the Balancing Transaction would be successful or not is not known by Fujitsu as it has
never been attempted.
Fujitsu have stated the answer has to be yes in the sense that ifthe fix involves inserting @ record
with an associated date then the date would be chosen as part of the design to fix the problem.
The choice of date would have to be made carefully as transactions will only be harvested from
the Branch Database for processing by back-end systems if it meets the correct selection criteria
hence the need to test any proposed fix. . The issue is simply that we would have to invent a
scenario from scratch and then check that out. I don't see that such an exercise would add value
given that we have already carried out a walkthrough of the tool.”
©. Were BTs (or something similar) possible in Old Horizon?
Fujitsu have advised they have attempted to make contact to retired staff on the matter but are
unable to provide a definitive answer on processes in place pre HNG-X relating to Balancing
Transactions, only that the transaction correction tool used to inject BTs that has been used since
HING-X implementation in 2010, was not used.
i, What controls were there around these?
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 35
Due to the response on the previous question from Fujitsu we cannot comment on these
controls.
Were they logged?
Due to the response on the previous question from Fujitsu we cannot comment on these
controls.
8. Privileged Users.
a. Can Privileged Users only access the BRDB or can they access other servers (i.e. audit server,
Audit Store)?
Privileged Users could theoretically access data at any other point in the flow of data from
Counter — Audit Store. This flow of data has been mapped by Deloitte and access rights at each
point tested.
i, In Deloitte's Board Briefing Paper dated 4 June 2014, on page 2, it notes: "it is possible for
Fujitsu staff with suitably authorised privileged access to delete data from the Audit Store.”
Has this issues been addressed / will it be addressed?
Yes, once data is in the Audit Store it cannot be amended / deleted for 7 years, as
described in (1a) above.
Would deleting data from the Audit Store have any effect on branch accounting?
No, unless data was retrieved from the Audit Store which would only happen in the case
of @ query being raised / investigation. It would only impact usage of this historical data
for any purposes when subsequently extracted from the Audit Store.
All Postmaster reporting functionality is generated from the live BRDB transactional
tables (and tables which aggregate this data and store it for up to 60 days). Any
amendment / deletion of data in the Audit Store therefore has no effect on branch
accounting and would only impact a branch if data was retrieved from the Audit Store.
Further if upon retrieval of branch data from the Audit Store a transaction had been
removed, the ‘data density’ check would highlight a missing transaction. If upon retrieval
of branch data from the Audit Store a transaction had been amended, the digital
signature check would highlight an issue with the integrity of the data. As per the
exception noted on page 3, there is @ small theoretical risk of a user ‘spoofing’ the digital
signature, arising from a failure in SOD controls relating to the digital signature.
b. Ifa Privileged User edits data in the BRDB, how might this affect the branch accounts from the
perspective of the Postmaster?
i. Where does the edited data flow to?
The edited data would remain in the BRDB transactional tables assuming that it was
entered in the correct logic.
The data in this table would then follow the normal data flow processes (i.e. BRDB >
audit server > Audit Store, BRDB > POLSAP, BRDB > Counter reporting etc.) if this
transaction had not already been picked up by the mechanisms which transfer
transactional tables downstream (e.g. Audit track gatherer which runs every 15 minutes.)
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 36
POL00139537
POL00139537
POL00139537
POL00139537
ii, Could the edited data cause a loss in a branch's accounts?
Yes, from a branch reporting perspective any change to data in the BRDB would affect
the real time reports ran on the counter, which are used for branch accounting,
specifically the monthly Branch Trading Statement which a Postmaster must sign off on in
order to roll into the next accounting period.
However if @ branches data was retrieved from the Audit Store, any amendment to
transactional data would cause the ‘digital signature’ integrity check to fail, and Fujitsu
would be notified of this failure upon retrieval of the audit data, As per the exception
noted on page 3, there is @ small theoretical risk of a user ‘spoofing’ the digital signature,
arising from a failure in SOD controls relating to the digital signature.
iii, Will the edited data be visible to the Postmaster?
A Postmaster is not specifically notified if a change had been made by a Privileged User.
Any changes to transactional data would impact declarations made by Postmasters
(encouraged to do so on a daily basis) and also declarations are required to be done in
order to rollover into the next accounting period (typically 4-5 weeks). The monthly
Branch Trading Statement which a Postmaster must sign off on in order to roll into the
next accounting period would also be impacted by a change of this nature which would
capture summarised totals of transactional data, which could be reconciled by branch
back to the granular transaction log reports. All of the mentioned reports are mechanisms
by which the Postmaster would be made aware of any such changes.
iv. Would the edited data be visible to Post Office / FJ?
Yes, as the data amendments would impact transactional records in the BRDB, and
subsequently this data would flow through to the Audit Store. Post Office / FJ would be
able to identify this through review of audit logs as described in 1C above.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT a7
POL00139537
POL00139537
Appendix 8
Non-Counter Initiated Transactions —- Understanding of Data Flow and Related Risks and Control
i
© Deloitte 20187 Private and Confidential - Subject to Legal Privilege - DRAFT 138
Reconciliation Controls
Note: Errors sources are Completeness (C), Accuracy (A) and Validity (V).
# Error Sources I Summarise Control Wording
Addressed
1 C&ARV External transactions sent via PODG such that the External Transaction files that are currently sent from Ingenico
(PAYSTATION) and Wincor Nixdorf (POST&GO) are routed to the Branch Database as well as sending the data to the Credence
system. There is a reconciliation between Credence & BRDB.
2 A&V For each Transaction Acknowledgement generated, a new transaction pair is created for POLSAP. The transaction delivered to
POLSAP will have a Reference number that matches the reference number used in the Transaction Acknowledgement record
generation. This allows POLSAP to match with the Transaction Acknowledgement once the TA has been accepted by the
Postmaster.
30 C&ARV AP Client File Reconciliation
APSS2222.ksh will reconcile the data in the files that it delivered to a Client with the data in the files that Credence delivered to a
Client.
34 C&ARV TPS to AP Reconciliation
TPSC227 writes APS transaction data to a formatted file that will later be used by the APS host program APSC2051 to reconcile
data from TPS with that from APS.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
Interface Controls
# Error Sources _ I Summarise Control Wording
Addressed
3 AaV If any one transaction fails validation / load, then the whole sub file (all rows for the same branch / trading date) will be rejected
4 C&ARV Processing of the files will commence when the last file is received. The last file is identified by ‘Y’ in the Last File Indicator field
in the File Trailer Record
5 C&ARV Generic file receipt process (BRDBCO038) will handle receipt of the different files that arrive at the external interface and will
perform registry of the files in the file audit trail and will move the files to the input directory and the audit directory.
6 C&AaV Any transactions that would have been incorporated in the Transaction Acknowledgement feed that are delivered in the
Paystation / Post&Go files will be automatically included in the Branch Accounts without being presented to the Postmaster for
acceptance. Transaction Acknowledgements for this transaction detail will be created at the same time for later acceptance by
the branches.
It also takes transactions that have previously been held aside due to the lack of Transaction Acknowledgement / Stock Unit
mapping or due to the SU being locked at the time of original posting and retries posting of these transactions.
7 c An automated Daemon process operates that starts to look for the arrival of the Extemal Transaction files at hh:mm O'clock but
gives-up and alerts if not arrived by nnn minutes later. (This allows Horizon transactions to get processed if External Transaction
files are late). This process performs the necessary copy / rename and creates links to audit directory. Hh:mm will initially be
18:00 and nnn minutes will be 120 minutes.
8 C&A FILE PROCESSOR
+ Ifthe file pre-processor returned with an error in the range of 102-105, then the table BRDB_FILE_ERRORS will have a row
added to it with an error value equal to the return value of the file pre-processor and the associated row in
BRDB_FILE_AUDIT_TRAILS will be updated to status ‘x’. No other error values are expected and, if they occur, the process will
abend and alert the Operations staff.
+ Ifthe file pre-processor was successful, then the file validation and database upload process will be called and exit status
checked.
9 C&A FILE PRE-PROCESSOR
The pre-processor performs a number of operations including splitting the files according to parameters. In addition it validates:
+ The first record is a header record
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
Error Sources
Addressed
C&A
‘Summarise Control Wording
+ The last record is a trailer record
+ The number of sub-files in the file equals the count in the trailer record
+ The total value of sub-fies (in the trailer) equals zero
If any of these validations fails, then the whole file will be rejected, a row will be inserted into BRDB_FILE_ERRORS and no
further processing is performed on the current file
Page 60 HAS TABLE OF THESE 8!
DATA LOADING & VALIDATION
This function is initiated by the File Processor. The 8 files generated in the previous process will be attached to Oracle as
external tables and the data therein will be validated and loaded into staging tables. It will validate data items such as product,
mode, branch etc. A log will be held for each file processed and each sub-file processed that will indicate the filename, status
(valid/not-valid), and history of the file processing. A separate error table will record each error type and error code encountered.
C&A
Ensure that the count and value of transactions equals the number recorded in the sub-file trailer and that the value of
transactions nets to zero otherwise record in BRDB_FILE_ERRORS with record type = STZ, Error Code = 108, Description =
“Sub-File Trailer totals incorrect”
C&A
Load the Transaction Data and Validate
At this point, the file structure has been validated and we now need to copy the data from the external files into the Branch
Database in preparation for Transaction Posting later-on in the schedule. During the copy process the data will be enriched with
missing attributes and validated against reference data held in the Branch Database.
During processing of each record, transaction-level validations will be performed and any errors found will be written to
BRDB_FILE_ERRORS with record type = OXZ and FAD Code and Business date = Sub-File details. The error code depicts the
type of error found
15b
C&A
If there is an entry in the error file with error_code = 101, then the file is a duplicate. The previous file that was delivered of the
same name might have had errors recorded against it and, so as not to confuse matters, only the 101 error is returned in the
error file.
Completeness Check
A process will check the table BRDB_SUB_FILE_AUDIT to test whether data has been received from all extemal sources for the
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
# Error Sources I Summarise Control Wording,
Addressed
current date, If it has not, then an alert will be raised that lists all External Transaction sources that have not provided data so
that relevant stakeholders can be notified
17 c External transaction processing
Immediately following the cessation of the Transaction Loading Daemon, the transaction posting process will be invoked using
TWS Schedule BRDB_TXN_POST.
18 c The final stage of External Transaction Posting is to copy the transactions for the current sub-file from the Staging/Holding tables
into the Branch Database Receipt tables ready for onward delivery to the TPS and the APS subsystems.
19 AaV A validation process will be followed that validates the content and format of data and records errors against bad rows.
20 CBA Transfer of data to TPS & APS...
Reconciliation totals are generated to ensure that the data that is sent to TPS and APS matches with the totals of data within
BRDB.
24 AaV Rejected and Held-up Transactions Report
A report is produced which highlights any transactions that have been loaded into BRDB but withheld from processing due to
lack of Transaction Acknowledgement mapping or due to the associated stock units being locked. The report will also list those
Sub-Files that have been rejected and have not yet been re-delivered error-free. This report will execute in the BRDB_EXT_REP
schedule.
24 C&ARV External data imported into Branch Database is copied across into BRDB_REP_SESSION_DATA. This ensures that they are
picked up for any Branch reports and Branch accounting,
25 Aav In order to post the transactions to the branch accounts, two criteria need to be met:
+ A mapping of External System and Terminal Id for all transactions must exist in the Transaction Acknowledgement/SU mapping
table
+ The stock unit for the branch must not be locked
26 c A report will be produced that lists any sub-files that have been held-back from processing for more than one day.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
142
POL00139537
POL00139537
# Error Sources _I Summarise Control Wording
Addressed
27 A&V Camelot ONLY:
Retailer data is required to validate that the Retailer Number is a valid, Validation includes a check that the Retailer Number
maps to the correct valid FAD Code.
28 AaV POLSAP Load process: The Post Office SAP load process in XI has some explicit checks (introduced to prevent files being
accidentally loaded more than once) that there will not be multiple sub-files with the same Branch / Trading Date combination.
29 A&V Validation should be performed such that when loading the data from external files it is checked that the Product can be
transacted on that particular type of external system.
32 TPS Processing monitoring
A monitor job tests for successful completion of the TPSTIPL schedule at 03:00 and alert operations if not.
34 v PODG will be used to transfer data between the Fujitsu data centre and External Transaction Suppliers. For External Transaction
interface files, there needs to be an inbound route to the Branch Database and also there needs to be an outbound route from
the Branch Database to Suppliers for the return of Error/confirmation files. Logical access rights to these holding directories are
appropriately secured.
35 v PODG to APS Interface
Old process:
APS already has links to EDG1 and EDG2 for the delivery of AP Client Files. Access to these directories is appropriately
secured.
New process:
APS configuration has been updated to deliver client files to revised directories that will be shared with PODG. Access to these
directories is appropriately secured.
36 A&V Post & Go: Post Office ETL will validate incoming files in terms of shape, structure and check totals.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
# Error Sources I Summarise Control Wording,
Addressed
37 Post & Go:
The Transaction Detail record will always contain a core of mandatory fields, and the records will be rejected if these fields are
not populated.
An alert will be raised within Wincor Nixdorf in the event that the file transfer fails. The Post Office Live Service (Team) will be
AaV informed and procedures invoked to rectify the problem
38 Post & Go:
If the file and sub-files contain no errors, Post Office ETL will rename both the copy file held on Post Office ETL and create an
error file with records type OKZ, to be sent back to Wincor Nixdorf to indicate the file is good.
When Wincor Nixdorf have investigated and corrected the records in error a new / corrected file it sends with the same name as
the error file, as Post Office ETL will know it has sent the errorfile and will expect the corrected error file to be replaced.
NB: If Post Office ETL receives a duplicate transmission file and / or sub-file(s), Post Office ETL will report this error to Wincor
A&V Nixdorf, and will also send these back to Wincor Nixdorf.
39 Post & Go: Validation criteria for received Post and Go Files are as follows:
+ Post Office ETL to reject a file should any error be found within the file, sub-file, or records within the sub-file that Post Office
ETL cannot accept. In such a case, Post Office ETL will create an error file specifying the errors found
+ Post Office ETL will return the error file to EDG to be picked up by Wincor Nixdorf, specifying any rejected files that need to be
corrected and resubmitted
+ Wincor Nixdorf will return repaired error records in a new file (and sub-file) for repaired records
+ Post Office ETL must inform Wincor Nixdorf of an error within 24 hours. Wincor Nixdorf must keep the source files for 7
A&V calendar days in case Post Office ETL require a file to be re-sent.
42 A&V Paystation:
The Transaction Detail record will always contain a core of mandatory fields, and the records will be rejected if these fields are
not populated.
43 ABV Paystation:
When Post Office ETL has processed the file it will rename the file as shown in Table 2 indicating whether:
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
144
POL00139537
POL00139537
# Error Sources ‘Summarise Control Wording
Addressed
a. The incoming file from Ingenico has been received OK (suffix .TPB)
b. Any errors have been detected in the file (suffix .TPX) together with an error file (suffix .TPZ)
44 A&V Paystation: Any files which are re-sent are to be given the same File Name and File Header information, with the ‘Transmission
Status’ set to RES. RES is to be used for whole file rejections only.
47 A&V Paystation: For reversal transactions, the original Transaction Mode is shown in the transaction details that are sent to Post
Office ETL. Post Office ETL will know if a reversal has taken place by referring to the reversal indicator within the transaction
line.
© Deloitte 20187 Private and Confidential ~ Subject to Legal Privilege - DRAFT
POL00139537
POL00139537
POL00139537
POL00139537
Appendix 9
Note: This content has been produced by Fujitsu and reproduced faithfully here.
Fujitsu Report on Privileged Users
Database Security in Horizon Online
Ref: 170406__DOC_3545053Database Securityv1.0.docx
Author: Gareth I Jenkins
Updates: Pete Newsome, Torstein Godeseth
Date: 06 April 2017
$:07:0048/04/2048 48:07.0017/04/2018 48:06.0016/04/2018.08:18:0002101/2018 (Formatted: Fort 11 pt, Do not check speling or grammar
(09:38:0020/42/2047-44-20:0028/42/2047 -15:26:0027/42/2047-18:17:0027/42/2047-15:00:
={ Formesteds Fort 1 ot, Do not check paling or grammar
\\ (Formatted: Font: 11 pt, Do not check spelling or grammar
\\\ (Fermatted: Font: 11 pt, Do not check speling or grammar
I), \ (Formatted: Font: 11 pt, Do not check spelling or grammar
{=m [A35]: Need to update the original date as itis
Version: 1.0
Introduction
f . . toupdating, Replace the formula wit a hardcoded actual
‘This note has been prepared by the Author to provide some clarity as to the veracity of some of the pe as ca
concems raised by Deloittes in its report to the Post Office regarding the Horizon system and I Formatted Fonts 11 pt Do ot check epling or grammar
specifically Super User access. The purpose of this note is to explore the extent to which it is Formatted: Font pt, Do not check speling or grammar
technically possible for:
(Formatted: Font 11 pt, Donat check speling or grammar
1. Post Office / Fujitsu to haye the ability to log on remotely to a Horizon terminal in a branch
so to conduct transactions.
. Post Office / Fujitsu to have the ability to conduct transactions (either remotely or locally)
under another user 's ID,
vy
3. Post Office / Fujitsu to have the ability to push transactions into a branch's accounts without
either a postmaster’s (a) knowledge or (b) consent.
4. Post Office / Fujitsu to have the ability to amend or delete transactions entered by branch
staff on Horizon (and can do so in a way that is hidden from postmasters).
In the previous reviews by Deloittes they have concluded there was a theoretical possibility that a
limited number of ‘Super Users’ had a level of access that would allow some of the actions 1 to 4
above, or their equivalents, to take place.
‘This paper shows the steps that a “Super User’ would have to make in order to alter the Horizon
Online Audit Trails. It also goes on to discuss, if such modifications were made, how they would
be detected. This is covered in Section 3.
Section 4 then considers the equivalent mechanisms in the old, Riposte based, Horizon system.
Finally, section 5 considers things from the Postmaster’s viewpoint.
‘This document assumes that readers have some familiarity with the technical documentation that
has been supplied to Deloitte to support their reviews.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 146
POL00139537
POL00139537
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
Executive Summary
Fujitsu’s view and conclusion remains that whilst such unauthorised amendments by Super Users
to the Horizon Online Audit Trails are theoretically possible (as in all IT systems) they would be
very difficult and even if they were made, would almost certainly be detected as a result of the
discrepancies in the relevant logs and audit trails. In addition, it should not be forgotten that the
various Horizon systems provide records of transactions as opposed to access to the funds
themselves and as such, even if one were to satisfy oneself that this theoretical risk were real then
it is difficult to imagine how someone exploiting this approach would be able to benefit financially
without detection. It is our reasoned estimation that to carry out the complex and detailed technical
steps necessary to over-ride the systems check and balances and then to make artificially generated
funds disappear is not credible.
Horizon Online
Section 1 describes how the Horizon Audit Trail is generated and secured. Section 2 then explores
how, in theory, a portion of the Audit Trail could be deliberately replaced by a Super User. Finally,
section 3 discusses how such changes could be detected.
1. How the Horizon Online Message Log is Generated and Secured
‘The Horizon Online Message Log is primarily a log of all auditable messages sent from the Horizon
Counter to the Horizon Data Centre where they are processed by the Branch Access Layer (BAL)
which in turn updates the Branch Database (BRDB).
Note, that not all messages sent from the Counter are audited. However, any that could impact on the Branch
accounts should be audited. Each message sent from the Counter to the Data Centre indicates whether or not
it is to be audited (‘ei.e. the decision is part of the counter application and not the BAL).
The BAL configuration also decides which messages pass through the Audit Filter (which does the auditing).
Itis part of the system's design to ensure that the counter and BAL configurations are consistent in this respect.
Specifically, messages have jsns if'and only if they are auditable and so the checks on jsn sequences described
below ensure the completeness of the audit trail.
Each Auditable message sent from the counter includes a “digital signature” generated using the
counter’s Private Key. This key is generated by the counter as part of the Log On process, ‘The
corresponding Public Key is included in the Log On message sent from the counter to the BAL
allowing the BAL to confirm subsequent messages in the session come from the same counter.
Unlike other messages in the Message Log, the BAL adds a wrapper to this Log On message which
includes a further digital signature (of the entire message including the counter’s digital signature
and the counter’s Public Key) generated by the BAL, using the BAL Private Key which is obtained
from the NPS Key Store by the BAL at start-up.
Alll auditable messages are written to a single table within the BRDB known as the “Message Log”.
Each day (at some point after 1am) the previous day’s Message Log is written to a number of files
which are then passed to the Audit system which then “seals” each file and stores them until they
are retrieved (if they ever are) or deleted. Note that each file will include records from a number of
different Branches and there may be multiple files for a single day containing the records for a
specific Branch.
Deletion of Audit records is currently suspended. They should be deleted after 7 years, but deletion was
‘switched off sometime in 2014 (I think). Therefore, all audit records since Horizon Online went Live in 2010
should be available.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 148
POL00139537
POL00139537
POL00139537
POL00139537
This seal is cryptographically generated and is based on the entire contents of the Audit File. Any
subsequent change to the contents would then invalidate the seal. The seal is held in a seals database
separate from the Audit Data. A feature of the Audit System is that data cannot be amended or
deleted until the pre-defined “Purge Date”, Super Users do not have access to the Audit data.
Alll updates to the BRDB will be based on the information held in the auditable message and the
accounts (both as seen in the Branch and also as passed to Post Office Ltd’s back end accounting
systems) are based on this information (and not on the actual auditable message). ‘This means that
in order to corrupt the Branch accounts, it is necessary to corrupt a number of different records
within the BRDB and not necessarily the Message Log. However, any evidence provided by Post
Office Ltd is based on the Message Log — hence the need to corrupt the Message Log as well.
It is asserted that by going back to the audited data (iei.e. the Message Log) sent from the Counter
to the BRDB, then all Transactions that that counter carried out and their implications on the Branch
Accounts can be re-calculated and compared with the reports produced by Horizon based on the
other data held in the BRDB and Post Office Ltd’s back end systems. This would enable any
corruption of the data used to create the Branch reports to be detected from examination of the
Message Log
When audit data from the Message Log is retrieved for whatever reason, a number of checks are
carried out to ensure the completeness and integrity of that data. ‘These checks are:
* Each entire Audit File is checked to ensure that the digital seal stored at the time the Audit
was produced (iei.e. the day after the transactions took place) is valid.
Normally a Data retrieval will be for a number of days and so a number of Audit files will need to be retrieved.
The data for the Branch in question is then filtered out from these audit files and checks are
then carried out on a counter by counter basis as described below for the period of the extract:
© No part of the Message Log is missing or duplicated. Each auditable message sent
from the counter to the BAL incudes a unique sequence number (the Journal
Sequence Number or jsn). The audit records for any counter over a period of time
should have no missing or duplicate jsns. The standard Audit Extracts into Excel
include a report indicating that this check has been successfully carried out. This is
a sheet labelled ‘Summary’ in the standard ARQ report provided to Post Office Ltd.
© The message audited as part of the Log On process, is checked and the Digital
Signature generated by the BAL is checked by using the BAL’s Public Key (which
is known to the Audit System). ‘This shows that this message was signed by an
application which had access to the BAL’s Private Key. This then provides access
to the Counter’s Public Key for that Log On Session (as this is included by the
message audited by the BAL and was signed by the BAL’s private key).
All subsequent messages sent from the counter to the BAL during that Log On
Session are then checked to ensure that their Digital Signatures are correct (using the
Counter’s Public Key obtained from the Log On message)
2. Replacing the Message Log
In theory, a Super User, could amend the Message Log for one or more Counters in one or more
Branches. The following describes what would be required to replace the Message Log for a single
counter in a single branch. This process could be repeated for multiple counters / branches if
required.
1. The work would need to be completed before 1am the following day (since the Message
Log is extracted from BRDB at some point after 1am each night and the data is then sealed
and held in the Audit Server
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 149
2. The entire Message Log
need to be replaced
ssociated with a Log On Session that is to be corrupted would
This is because it is not possible to obtain the counter’s Private Key and so a new one would need to be
generated as described below.
3. The records being replaced would have to be in one-to-one correspondence to the original
records otherwise there would be gaps or duplicates in the sequence of jsns which would
then be detected as part of the Audit Retrieval process.
4. An application would need to be run by the Super Users in order to correctly construct the
revised Audit Records
5. This application would need to generate a Private / Public key pair similar to the one
originally generated by the counter. Called an “Attack Counter key” in the rest of the
document
6. The application would need to have access to the BAL’s Private Key. Since this is stored
in the Key Store which is an Oracle Database running on the NPS, then it is assumed that a
Super User would be able to read this value and make it available to the application. This
would then enable the application to generate a Log On Message Log message containing
the fake Counter Public Key and to sign it using the genuine BAL Private Key.
7. Alll subsequent messages for the session would then need to be amended as required and
then re-signed using the Attack Counter Private Key generated at step 5.
8, Having constructed all these false Message Log messages, then the Super User would need
to delete all the genuine messages from the Message Log in BRDB and replace them with
the false messages on a one for one basis.
Or this could be done just by updating the rows with the new data.
Note that the table in the database is set to allow ‘append access only and has been designed to be appended
10 at all times. The Super User would need to amend the access rights to the table before records could be
amended or deleted, and this would change the performance characteristics of Oracle; this alone may be
sufficient to make such an attack detectable as any instance of slow running on the system would be
investigated by the support teams.
9. Note that as stated earlier, corrupting the Message Log in this way has no impact whatsoever
on the Branch Accounts, since these never refer to the Message Log. The Branch Accounts
are based on copies of some of the data held in the Message Log being stored in “working
tables” within the BRDB. Clearly any application that is capable of corrupting the Message
Log in BRDB would also be capable of updating (‘ei.e. corrupting) the data used to calculate
the Branch accounts.
Note that the relationship between data held in the Message Log and the Branch accounts is ¢ complex.
Therefore, a significant amount of knowledge and skill would be required to attempt this.
It should be noted that since R12 (July 2015) all access and actions carried out by Super Users to
any database is strictly audited to an Oracle Audit table. The records in the Audit Table records the
following information:
© User Id of the Super User
© Action (ege.2. Log On, Execute a SQL command, Log Off ete.)
© Date and Time of the action
© Actual SQL statement executed (where applicable)
‘This Audit Table is again extracted from BRDB soon after 1am and the data picked up and sealed
before being copied to the Audit Server.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 150
POL00139537
POL00139537
POL00139537
POL00139537
Note that itis possible for the Super User to manipulate the Audit table (including removing entries
from the table). However, should the table be removed entirely, then the database would stop
working. If only old entries are removed, then the removal of entries will be recorded, thus making
it clear that the table has been manipulated though the details of the changes would not be fully
visible.
Prior to R12, when the BRDB was upgraded to run on a later version of Oracle, (iei.e. from 2010 to
July 2015), only Log On and Log Off activities by Super Users were audited. For all planned access,
then an MSC (Managed Service Change document) would have been signed off and the Logs of the
Super User activities would have been attached to the MSC. Therefore, a correlation of Log On /
Log Off activities of Super Users against MSCs should detect any rogue activities.
3. Detecting Changes to the Audit Trail
In order to make the changes to the Message Log described in section 0, the Super User would need
Read access to the Key Store database which runs on the NPS and Read / Write access to the BRDB.
Note that should the rogue application run on the BAL, then this isn’t necessary as the BAL’s have
access to the Key store based on the IP address.
Note that the BAL Private Key only needs to be accessed once as the same key i
used for a year.
However the BRDB would need to be accessed each day that the Message Log is to be corrupted.
Alll such access by a Super User would be audited in the Audit Table; since R12 such logs would
show the activities carried out whilst logged on, and prior to R12 the logon (and logoff) events
would be recorded.
‘Therefore, should there be any allegations that any data has been corrupted, an examination of the
Database Audit tables should ensure that this has not occurred. Although the Database Audit tables
are not regularly examined they were recently checked as part of an external Audit of Horizon
Online and no issues were reported.
Old Horizon
On the old Horizon system, the mechanisms were very different from those used by Horizon Online.
Section 4 provides an overview of how the Riposte Message store operates, then section 5 describes
the Riposte Audit Trail, Finally section 6 shows how injections of messages by a super users would
be detected in the audit trail.
4. Overview of Riposte
All Counter data was held in a bespoke Message Store which was part of the Riposte product
supplied by Escher Inc, This data was replicated within each Branch to all counter positions and
from each Branch to the Data Centres where it was held in the Correspondence Server Message
Stores. Similarly, any data inserted into the Message Store at the Data Centre (ege.. Reference
Data or authorisations for Banking Transactions) would be replicated back to the Branch Counters.
Selected data was then extracted from the Correspondence Servers to update Post Office Ltd’s Back
End systems.
All accounting at the counter was carried out based on the data held in the Message Store. The
Riposte product managed the Message Store and it did not allow any message to be updated or
deleted. Therefore all that could be done to corrupt the data in the Message Store was to inject
additional messages which could then influence the Branch accounts, Such injections were possible
at the Correspondence Server for users with sufficient access permissions.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 151
Each message included 3 key bits of information which together provided a unique identification
for each message:
© Group ID: this was the 6 digit FAD Code of the Branch with which the message was
associated
* Node ID: This indicated the Counter Position at which the message was originally written
for messages generated at the Counter or the Correspondence Server identifier for messages
generated at the Data Centre. Counter Node Ids where between 1 and 31, and
Correspondence Server Node Ids where between 32 and 63.
Message ID: A unique number for each Group ID / Node ID, This number starts at 1 for
the first message written at that Node, and increase by 1 for each subsequent message. This
allows checks to be made that no messages are missing as that would result in gaps in the
sequence of Message IDs
The concept of jsns used in Horizon Online was based on this.
Messages also have an associated “Expiry Date”, This indicates the number of days after the
message is first written before it can be deleted. An archive process ran on each counter and
Correspondence Server at around 3am which deleted all messages that were past their Expiry Date,
thus ensuring that the Message Store did not continue to grow indefinitely.
Some special messages which are referred to as “Persistent Objects” did not expire in this way but could be
removed after they were replaced.
However again they were all held for a minimum of 34 days and in general were not relevant to generating
the Branch Accounts.
In particular, Riposte was configured such that no messages where allowed to expire until they were
at least 34 days old. This was to allow for counters that were offline for a significant period.
Each message also had an associated CRC, this was basically a checksum that was included to
ensure that the message had not become accidently corrupted. Note that this was not a
cryptographically secure seal and it would be possible for a sufficiently technically skilled person
to alter a message and recalculate the CRC if they had access to the message outside the message
store.
Due to the size of the Post Office Network, Branches were split into 4 separate Clusters. Each Cluster included 4
Correspondence Servers (2 in each Data Centre), thus ensuring that there were normally 4 copies of the data held
in the Data Centres
5. The Riposte Audit Trail
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
‘An Audit Application was run on the Correspondence Servers to take an audit copy of all data
visible to that Correspondence Server.
The Audit Application was run on one Correspondence Server on each Cluster in each Data Centre. This
‘means that there were two independent Audit Trails for each Branch. However when retrieving the data only
one Audit Trail was used.
is application read every record that was visible to that Correspondence Server (iei.c. all data in
that Cluster) and wrote a text copy of that data to a text file. Each Audit application wrote data to
10 text files (based on one of the digits in the FAD Code) and when the text file got to a certain size
it was closed and a new file created for that text stream. ‘The file included a hash value of the file
contents to ensure that should it be accidentally corrupted, then this would be detected. Also around
lam each day the file was swapped thus ensuring that data associated with a given day was in
discrete files.
Once these files had been written they became visible to the Audit server which would pick the files
up and Seal them and store them until they are retrieved or deleted.
POL00139537
POL00139537
POL00139537
POL00139537
This process was not changed for Horizon Online.
Deletion of Audit records is currently suspended. They should be deleted after 7 years, but deletion was
‘switched off sometime in 2014 (I think). Therefore all audit records since sometime in 2007 should be
available. Those from before that time are no longer available.
f the audit trail is retrieved, then similar checks to those carried out on Horizon Online were made,
namely:
Each entire Audit File is checked to ensure that the digital seal stored at the time the Audit
was produced (iei.c. the day after the transactions took place) is valid.
‘Normally a Data retrieval will be for a number of days and so a number of Audit files will need to be retrieved.
There would also normally be a number of audit files for each day:
© The data for the Branch in question is then filtered out from these audit files and checks are
then carried out on a counter by counter basis as described below for the period of the extract:
© No part of the Audit Data is missing or duplicated. ‘This is done by ensuring that
there are no missing or duplicate Message Ids for each counter / CS. The standard
Audit Extracts into Excel include a report indicating that this check has been
successfully carried out.
© The CRC is recalculated and confirmed as correct for the message.
6. Detecting Changes to the Audit Trail
Ifa malicious Super User wished to interfere with the Branch’s Data, then that would need to be
done by injecting messages into the Correspondence Server or Counter Message stores using the
Riposte APIs. Support Staff did have the capability (and occasional need) to do this at the
Correspondence Server. Processes where in place to ensure that any such messages included
information as to who had done this. Such information would not be visible in the standard audit
extracts (but would be visible in a detailed examination of the raw audit data), Clearly any malicious
corruption would be done without such trace information. However, if such data were injected at
the Correspondence Server, it would be clear that this had occurred since the Node Id associated
with the message would be that of the Correspondence Server at which the message had been
injected and not a normal Counter Node Id. This would be clearly visible in any audit extract.
Postmaster’s View
‘The Horizon system records all transactions and uses its records to generate a view of how much
cash and other items of value should exist in a branch at any time. Subpostmasters are required to
carry out daily cash balances where they should check that the cash they hold in their tills
corresponds to what the system says they should have. If the system has been manipulated to present
a false view of the cash they hold in the branch this should be immediately obvious when they carry
out these daily processes.
In the theoretical scenario that a Super User has manipulated the transactions for a Branch they thus
have to address the problem of making the physical cash or some other representation of value,
appear in, or disappear from, the Branch without triggering any investigation.
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 153
POL00139537
POL00139537
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT 54
Statement of Responsibility
We take responsibility for this report which is prepared on the basis of the limitations set out in the report.
‘The matters raised in this report are only those which came to our attention during the course of our work and are
not necessarily a comprehensive statement of all the weaknesses that exist or all improvements that might be made.
Deloitte LLP
London
November 2017
Other than as stated below, this document is confidential and prepared solely for your information and that of other
beneficiaries of our advice listed in our engagement letter. Therefore you should not, refer to or use our name or this
document for any other purpose, disclose them or refer to them in any prospectus or other document, or make them
available or communicate them to any other party. If this document contains details of an arrangement that could
result in a tax or National Insurance saving, no such conditions of confidentiality apply to the details of that
arrangement (for example, for the purpose of discussion with tax authorities). In any event, no other party is entitled
to rely on our document for any purpose whatsoever and thus we accept no liability to any other party who is shown
or gains access to this document.
© 2017 Deloitte LLP. All rights reserved.
Deloitte LLP is a limited liability partnership registered in England and Wales with registered number OC303675
and its registered office at 2 New Street Square, London EC4A 3BZ, United Kingdom.
Deloitte LLP is the United Kingdom member firm of Deloitte Touche Tohmatsu Limited (‘DTTL"), a UK private
company limited by guarantee, whose member firms are legally separate and independent entities. Please see
www.deloitte.co.uk/about for a detailed description of the legal structure of DTTL and its member firms.
Member of Deloitte Touche Tohmatsu Limited
© Deloitte 20132 Private and Confidential - Subject to Legal Privilege - ORAFT
POL00139537
POL00139537