POL00059474
POL00059474
000022
Post Office ltd
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9; Magistrates Court Act 1900, sub section. 5A(3) (a) and 50; Criminal
Procedure Rules 2005, Rule 27.1
Statement Gareth Idris JENKINS
of:
Age if under Over 18 (if over 18 insert Occupation: Business
18: ‘over 18’) Architect
This statement (consisting of eleven pages each signed by me) is true to the
best of my knowledge and belief and I make it knowing that, if it is tendered
in evidence, I shall be liable to prosecution if I have wilfully stated in it
anything which I know to be false or do not believe true.
Dated the 15th day of January 2013
Signature
Introduction
I am Gareth Idris Jenkins. I am employed by Fujitsu Services Ltd who
have been contracted by Post Office Ltd to provide the Horizon systems
operating in Post Offices around the country. However I understand that
my role is to assist the court rather than represent the views of my
employers or Post Office Ltd.
I graduated from Cambridge University with a degree in Mathematics in
1973 and was awarded an MA by Cambridge University in 1997. I was
employed by ICL in September 1973 and have worked for that company ever
since (though its name was changed to Fujitsu Services about 10 years
ago). During my time with ICL / Fujitsu I have held a number of roles
in customer support, development, design and architecture. During the
early 1990s I was involved with representing ICL in developing Systems
Management Standards and in 1992 I was the head of the UK delegation on
Systems Management at the International Standards Organisation
conference in Ottawa, Canada. In the late 1990s I become a
Distinguished Engineer within ICL. Distinguished Engineers, were about
100 or so of the senior technical staff within the company (out of about
6000 to 7000 technical staff).
I am a member of the British Computer Society (MBCS), a Chartered
ety GRO.
witnessed by \ i
POL00059474
POL00059474
000023
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sud section, 9A(3)(a) and SE; Criminal
Procedure Rules 2005, Rulc 27.1
Page 2 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
Engineer (CEng) and a Chartered IT Professional (CITP).
Since 1996 I have been working on the Horizon project in association
with Post Office Ltd. My initial role was in the integration of the
Riposte messaging system which is responsible for storing all data in
the Post Office branches and replicating it to the Data Centres. I was
also responsible for the design of the interface between Horizon and
Streamline which processes all Credit and Debit Card payments for Post
Office Ltd. More recently I’ve been involved in projects associated
with interfacing data from Horizon to Post Office’s back end accounting
systems.
I have been asked to provide a statement in the case of Jishaan Patel.
I understand that the integrity of the system has been questioned and
this report provides some general information regarding the Integrity of
Horizon.
The purpose of this xeport is to provide some further background
information.
1.1 The Document Structure
Section 2 of the document describes the Horizon system at a high level,
giving a time-line for its development, the Business scope and
Architecture diagrams for both the original Horizon System and the
current Horizon Online system.
Section 3 then summarises my views on the overall integrity of the
Horizon system.
2. The Horizon System
2.1
Fujitsu were originally awarded a contract in 1996 to provide a Horizon
Timeline
System to Post Office Ltd. The following provides some key dates and
Signature Signature
witnessed by
POL00059474
POL00059474
000024
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. SA(3)(a) and 5B; Criminal
Procedure Rules 2005, Rule 27.1
Page 3 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
functional changes:
e Horizon Pilot 1996
e Horizon Rollout 1999 - 2002
e Network Banking 2003
e EMV 2004
e Cash Account removed 2005
e Data Centre Migration 2009
e@ HNG-X Rollout 2010
Horizon Online (or HNG-X) was a major re-implementation of Horizon. It
was a complete re-implementation of the business functionality at the
counter and utilised a central Database to hold details of all
transactions rather than the MessageStore used by the original Horizon
system.
All Post Office Branches migrated from the original Horizon to Horizon
Online between January and September 2010. Historical transactions were
made visible in the new system as part of the migration process.
2.2 Business Scope
The Business scope of Horizon is:
¢ Point Of Sale Application
e Transaction Recording
© All such transactions are Audited
e Posting Summary Transactions to POL SAP (Post Office Ltd’s back
end accounting system)
e Posting Detailed Transactions to Credence (Post Office Ltd’s back
Signature
witnessed by
POL00059474
POL00059474
000025
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. 5A(3) (a) and 58; Criminal
Procedure Rules 2005, Rule 27.1
Page 4 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
end Management Information system)
© Posting Remuneration Data to HR-SAP (Royal Mail Group’s back end
Payroll system)
© Delivering Client Data to Post Office Ltd’s Clients (ie 3™ parties
that Post Office Ltd acts as an agent for such as Local
Authorities and Utility companies etc)
2.3. Architecture Diagrams
Audit Agent
Audit
Extract
Harvester
Agents
Data
Extract ce
Message
store
Audit IStore
Audit System
Replication
Audit I Retrieval
Message
store
Journal
Counter.
Extract
1
Figure 1 - Horizon Data Flows
Signature... , Signature
I GRO I witnessed by
‘PUREE
POL00059474
POL00059474
000026
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. 5A(3) (a) and 58; Criminal
Procedure Rules 2005, Rule 27.1
Page 5 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
The Horizon system was designed to store all data locally on the
counter’s hard disk in what is referred to as the messagestore. Once
the data has been successfully stored there it is then replicated
(copied) to the hard disks of any other counters in the branch (and in
the case of a single counter branch to the additional external storage
on the single counter). Data is also passed on from the gateway counter
to the Horizon data centre using similar mechanisms where it is stored
in the CS Messagestore.
The replication process is designed such that should the data fail to be
copied immediately (for example due to a failure on the local IT network
within the branch or another counter being switched off or the branch
being disconnected from the data centre), then further attempts are made
to replicate the data at regular intervals until it is finally copied
successfully. Once the data reaches the Data Centre a further copy is
taken by the Audit Agent which writes it to an Audit File which is added
into the audit trail where it is available for retrieval for up to 7
years, Data in the audit trail is “sealed” with a secure checksum that
is held separately to ensure that it has not been tampered with or
corrupted.
Other systems can also access the data from the CS Messagestore via
Harvester Agents. However such systems are outside the scope of the
integrity of the Audit trail.
Every record that is written to the transaction log has a unique
incrementing sequence number. This means it is possible to detect if
any transitions records ‘have been lost.
While a customer session is in progress, details of the transactions for
that customer session are normally held in the computer’s memory until
Signature
witnessed by
POL00059474
POL00059474
000027
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. 5A(3)(a) and 5B; Criminal
Procedure Rulee 2005, Rule 27.1
Page 6 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
the customer session (often known as the “stack”) is settled. At that
point all details of the transactions (including any methods of payment
used) are written to the local hard disk and replicated (as described
above). It should be noted that double entry bookkeeping is used when
recording all financial transactions, ie for every sale of goods or
services, there is a corresponding entry to cover the method of payment
that has been used. When a “stack” is secured it is written in such a
way that either all the data is written to the local hard disk or none
of it is written. This concept of “atomic writes” is also taken into
account when data is replicated to other systems (ie other counters,
external storage or the data centre).
The data for a stack will have been successfully secured to the local
hard disk before the screen is updated indicating that a new customer
session can be started. Note that although an attempt will have been
made to replicate the data to an external system at this time, there is
no guarantee at this point that such replication will have been
successful. For example if there is a Network Failure followed by a
Terminal failure there is a slight risk that transactions in the
intervening period could be lost.
All data that is written includes a “checksum” value (known as a CRC)
which is checked whenever the data is read to ensure that it has not
been corrupted. Any such corruptions detected on reading will result in
failures being recorded in the event logs which are held on the local
hard disk for a few days for immediate diagnosis and also immediately
sent through to the data centre where they are held for 7 years.
Any failures to write to a hard disk (after appropriate retries) will
result in the counter failing and needing to be restarted and so will be
immediately visible to the user.
Signature
witnessed by
POL00059474
POL00059474
000028
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. 5A(3)(a) and 5B; Criminal
Procedure Rules 2005, Rule 27.1
Page 7 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
Whenever data is retrieved for audit enquiries a number of checks are
carried out:
1. The audit files have not been tampered with (ie the Seals on the
audit files are correct)
2. The individual transactions have their CRCs checked to ensure that
they have not been corrupted.
3.A check is made that no records are missing. Hach record
generated by a counter has an incremental sequence number and a
check is made that there are no gaps in the sequencing.
Other
Systems:
Audit I Store
BAL IMessage Audit System
Audit I Retrieval
Audit
Extract
Counter
Figure 2 - Horizon Online Data Flows
Signature
GRO 7} witnessed by
Q@
A
fo}
POL00059474
POL00059474
000029
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. 5A(3) (a) and $B; Criminal
Procedure Rules 2005, Rule 27.1
Page 8 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
Horizon Online is designed to store all data in an online database known
as the Branch Database (BRDB). In particular no data concerning
Business Transactions is retained at the counter other than in the
memory of the Counter Business Application.!
Transactions are carried out locally on the Horizon Online counters and
a Basket is built up during a Customer Session. Each transaction will
result in a Basket Entry consisting of one or more Accounting Lines. At
the end of a Customer Session when the Basket has been completed and all
Settlement items (or Tender lines) have been processed and added into
the Basket as further Accounting Lines, such that the total value of the
Basket is zero, the entire Basket is sent to the Data Centre as a BAL
Message where the Branch Access Layer (BAL) processes the message and
all the Accounting Lines are recorded and committed to the BRDB as part
of a single Oracle Commit. This means that either all the transactions
within a Basket are successfully written or none of them are. Once the
Accounting Lines have been successfully committed a response is returned
to the counter indicating this success and this then allows any receipts
to be printed. The Basket is deemed to be fully completed once all
relevant receipts have been successfully printed. Note that if there
are no receipts to be printed, then the screen is updated to show the
top level menu indicating successful completion of the previous Basket.
The Oracle Commit also includes an Audit of the data originally
transmitted from the counter to the BRDB. This data is digitally signed
at the counter using a key generated as part of the Log On process. It
is this audit record that is used to provide the extract of transactions
‘In order to support recovery, the identifier of the last successfully completed Basket is recorded on the
Hard disk at the counter. However this is not classed as Business Data.
Signature
witnessed by
POLOIL
POL00059474
POL00059474
O0004N
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. 5A(3) (a) and $8; Criminal
Procedure Rules 2005, Rule 27.1
Page 9 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
used for Litigation support.
Any auditable message from the counter is stored, together with its
Digital Signature and other key attributes in an “Audit table” (known as
the Message Journal) in BRDB. Each night after midnight, the contents
of this table for the previous day are copied from the BRDB to a number
of serial files.
A number of files are generated due to the volume of data
processed each day. All data from a given Branch will be
concentrated into a small number ot these files for ease of
retrieval.
At this point a check is made that indeed there are no missing or
duplicate jsns for any counter and should any be found an alert is
raised.
Note that this could only happen as a result of a bug in the
code or by somebody tampering with the data in BRDB and this
check is included specifically to check for any such bugs /
tampering.
These files are then copied to the Audit system where they are sealed
with digital seals. They are held there for a period of 7 years during
which time they may be retrieved and filtered to produce the relevant
audit data for a particular Branch.
The audit record may also include application events that have been
accumulated at the counter since the last auditable message was sent to
the Data Centre. All major activities that affect the Branch also have
an audit of the data sent from the counter to the Data Centre included
-
Signature
witnessed by
GRO
POL00059474
POL00059474
000031
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. 5A(3)(a) and 5B; Criminal
Procedure Rules 2005, Rule 27.1
Page 10 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
in the audit log.
Each Audit record includes the following identification:
e Branch identifier (i.e. FAD Code)
e Counter identifier
e Sequence Number (known as a Journal Sequence Number or jsn)
* Counter timestamp
Within any counter (i.e. for a given Branch Id / Counter ld
combination), the jsn will always increase by exactly one for each
successive audit record. This enables a check to be made that there are
no records missing from the audit trail when they are retrieved.
The transactions in a basket are constructed using the principle of
double-entry bookkeeping. This means that in addition to the Accounting
Lines that relate to the actual business transactions, separate
Accounting Lines are also generated for the tender items (such as Cash,
Cheques or Credit / Debit Cards), resulting in the total value of all
Accounting Lines in a Basket adding up to zero. When the contents of a
Basket are written to BRDB a check is made that the net value of all the
accounting lines is indeed zero and should it not be, then an alert is
raised and the basket is discarded and an error response returned to the
counter.
Note that this could only happen as a result of a bug in the
code and this check is included specifically to check for any
such bugs.
Baskets are also built up during Back Office Sessions and such Back
Office baskets are handled in a similar way to Customer Baskets.
Signature
witnessed by
‘eoLsiT
POL00059474
POL00059474
CONFIDENTIAL
Witness Statement
Criminal Justice Act 1967, Section 9, Magistrates Court Act 1980, sub section. 5A(3)(a) and 58; Criminal
Procedure Rules 200, Rule 27.1
Page 11 Of 11 Pages
Continuation Gareth Idris JENKINS
Statement of
Horizon Integrity
This is described in the separate integrity documents
ARCGENREP0004.HorizonDataIntegrity.doc which I now produce as exhibit
GIJ/1 and HorizonOnlineDataIntegrity POL.doc which I now produce as
exhibit GIJ/2.
I have been involved personally in a number of challenges to the
integrity of the original Horizon system and produced Witness Statements
for a number of cases where the Integrity has been challenged. I am not
aware of any cases where the Integrity of Horizon Online has yet been
successfully challenged in court.
The main challenges in the cases in which I have been involved were
presented as “Hypothetical issues” and my previous Witness Statements
went through each of these hypotheses and showed that there was no
specific evidence for any of them in the data presented.
In summary I would conclude by saying that I fully believe that Horizon
will accurately record all data that is submitted to it and correctly
account for it. However it cannot compensate for any data that is
incorrectly input into it as a result of human error, lack of training
or fraud (and nor can any other system).
Signature
I GRO I witnessed by