WITNO03680100
WITNO3680100
Witness Name: Simon Geoffrey Oldnall
Statement No: WITN 0368_1
Exhibit: WITN 0368_1/1 — WITN 0368_1/2
Dated: 20th July 2022
THE POST OFFICE HORIZON IT INQUIRY
Witness Statement of Simon Geoffrey Oldnall in the Post Office Horizon IT
Inquiry
I, SIMON GEOFFREY OLDNALL of Finsbury Dials, 20 Finsbury St, London EC2Y
9AQ, SAY AS FOLLOWS:
1. Myname is Simon Geoffrey Oldnall. I have been employed by Post Office Limited
("POL") since September 2020 as the IT Director for Horizon.
2. Except where I indicate to the contrary, the facts and matters contained in this
witness statement are within my own knowledge. Where any information is not
within my personal knowledge, I have identified the source of my information or
the basis for my belief. The facts in this witness statement are true to the best of
my knowledge and belief.
3. In this statement I use the term "Postmaster" to refer to those individuals,
companies and partnerships who/which are responsible for running branches in
the Post Office network pursuant to contracts with POL, but excluding those
individuals who are directly employed by POL to work in branches that are
directly managed by POL. Depending on the context, references to "the
Page 1 of 16
11/75599520_1
WITNO03680100
WITNO3680100
Postmaster" should be read as referring to or including, as appropriate, the
Postmaster's staff and/or representatives.
4. Where I refer to specific documents in this statement, copies of those documents
are exhibited to this statement in a series of exhibits marked "WITN 0368_1/[X]'
and identified by the Inquiry's unique reference number for that document.
My role at POL and in relation to the Teach-In sessions
5. In my role as IT Director for Horizon, I am responsible for managing the POL
Horizon Leadership team and working with third party suppliers involved in the
Horizon IT system ("Horizon"), including testing of changes and service
management of the current Horizon platform. In addition, I am responsible for
designing and implementing changes following the GLO litigation to ensure that
POL IT takes steps to resolve issues identified in the Horizon Issues Judgement
("HJ").
6. I am responsible for supporting the POL Group CIO and I lead the POL Horizon
Leadership team which is made up of various sub-teams which focus on different
aspects of the Horizon IT System (including Postmaster Experience, Commercial
Management, Quality Assurance, Horizon Live Services, Remediation and
Change as well as Security, Risk & Investigations). The POL IT team was
supplemented at the time by external consultancy support (KPMG) focussed on
supporting POL with the remediation activities identified following the HlJ (please
see paragraph 14 below for further details on KPMG's work).
7. I led the three POL IT Teach-In sessions which took place on 12 May 2021, 25
May 2021 and 9 June 2021 (the "IT Teach-In sessions").
8. This witness statement is provided in response to the request made by the
Horizon IT Inquiry (the "Inquiry") pursuant to Rule 9 of the Inquiry Rules 2006,
Page 2 of 16
11/75599520_1
WITNO03680100
WITNO3680100
dated 17 December 2021, for POL to provide the same or similar information as
POL previously provided to the Inquiry at a number of "Teach-In" sessions
between April and June 2021 (the "Teach-In Rule 9 Request"), including the IT
Teach-In sessions. The purpose of the IT Teach-In sessions was to assist Inquiry
personnel’s understanding in respect of certain key areas relating to:
a. Product Management & Usability;
b. Supplier relationships;
c. Future plans;
d. Horizon Improvement Programme;
e. Management of bugs, errors and defects;
f. Testing;
g. Architecture;
h. Automatic Payment and Advanced Data Capture scripting ("APADC")';
i. The POL IT Team;
j. Governance Processes; and
k. Architecture overview.
9. For the purpose of the IT Teach-In sessions, I prepared and shared with the
Inquiry a PowerPoint presentation (which I will refer to as “the presentation’).
For reference, I exhibit a copy of the presentation at "Teach-In 3 Powerpoint
Presentation" (WITN 0368_1/1). This presentation formed a single reference
document, collated from various sources by my team, to support the dialogue
with the Inquiry team, provide visual aids and as a means of structuring
discussions during the Teach-Ins.
1 APADC is, broadly, the mechanism by which a transaction is executed on the counter in branch. It is
a script that runs in the background that translates and order and places it into the basket so that the
branch staff can take payment.
Page 3 of 16
11/75599520_1
WITNO03680100
WITNO3680100
10. Inthis witness statement, I address questions 42 to 53 in the Annex to the Teach-
In Rule 9 Request (relating to the IT Teach-In sessions) and I will provide the
same or similar information as was provided during the sessions that I led. The
statement will therefore represent the position as it was on the dates of those
sessions. Where this statement makes any reference to policies, processes or
systems that have been adopted since those sessions, this statement reflects
the position as of 8 February 2022 when this statement was first provided to the
Inquiry in draft.
POL’s IT software architecture system and its key data flows
11. Slide 43 of the presentation (WITN 0368_1/1) contains a detailed diagram which
includes the key systems and data flows, grouped and coloured by function. It is
not exhaustive, but does provide context against which the detail of the system
overviews that I subsequently provide can be understood.
Horizon's infrastructure
12. Slide 21 of the presentation (WITN 0368_1/1) details the suppliers (namely
Fujitsu, Computacentre, Verizon, Ingenico and NCR) involved in the provision of
the Horizon Infrastructure. The slide further provides the term and end dates of
the relevant contracts. In summary:
a. Computacenter provides the Point of Sale Terminal on which the Horizon
software operates.
b. Verizon provides the network over which Horizon transactions are sent
to both Fujitsu and POL.
Page 4 of 16
11/75599520_1
WITNO03680100
WITNO3680100
c. Ingenico Paystation provides the paystation* service (these transactions
are recorded and settled in Horizon).
d. Ingenico Pin Entry Devices ("PED") are used for customer card payment
transactions which are recorded and settled in Horizon.
e. NCR provides the Self-Serve Kiosks for customers to process a discrete
number of Horizon services.
13. Most branches are connected to POL via the Verizon network through an ADSL
(broadband) connection. POL does not supply Wi-Fi to branches although some
do have Wi-Fi in branch.
Past Management of Horizon
14. Slides 7 to 19 of the presentation (WITN 0368_1/1) outline the approach taken
to POL's management of Horizon as a product. During the IT Teach-In
sessions, I provided the Inquiry with an overview of the current product
management approach, which was revised following an assessment conducted
by our external advisors, KPMG. This involved an assessment both of
remediation activities required to address the issues identified in the HlJ and of
POL's current systems, processes and capabilities to manage Horizon. The
assessment was conducted between October 2020 and January 2021. The
approach is based on three "pillars" of product and service management, as set
out on slide 7 of the presentation (WITN 0368_ 1/1), namely:
a. Strategy / Demand — this involves POL's long term strategy in respect of
technology;
2 A paystation is a device in some branches that processes transactions such as bill payments.
Page 5 of 16
11/75599520_1
WITNO03680100
WITNO3680100
b. Architecture and Service — this includes ensuring that the system is
robust and is not compromised; and,
c. Delivery —this relates to POL's ability to deliver and maintain the service.
It includes ensuring that POL's strategy and plans are practical and
realistic.
15. These are the three "pillars" that POL uses in respect of the technology; each
"pillar" has a governance forum including Horizon Design Authority and Horizon
Live Service Review which each report into Horizon SteerCo (this is set out at
slide 40 of the presentation (WITN 0368_1/1)). The key for POL is to ensure that
users are at the centre of the discussion. This has involved something of a
cultural pivot in comparison to past practice, when POL's focus had been more
commercial rather than "what does this mean for Postmasters?".
16. By way of example, POL has implemented a change in respect of the ability for
those in branch to investigate discrepancies. At the end of a trading period,
Postmasters declare a set of accounts. If the accounts do not balance, namely if
it is positive or negative compared to what appears on the Horizon system, this
is referred to as a discrepancy. Postmasters are able to "roll over" discrepancies
to the next trading period. There is now a dispute button on Horizon which
enables the functionality to allow Postmasters and branch staff to raise a dispute
with POL and receive support to resolve discrepancies
Impact of the Horizon Issues Judgment
17. Slides 32 and 33 (WITN 0368_1/1) outline the immediate areas that POL and the
Horizon Architecture Team addressed following KPMG's audit, referred to at
paragraph 14 above.
Page 6 of 16
11/75599520_1
WITNO03680100
WITNO3680100
18. Remediation activities include "Fast Fix" activities which were driven by the need
to address issues (some of which were identified as a result of the HIJ) such as:
a. Greater visibility of branch data to Postmasters and improved dispute
mechanisms;
b. Postmaster involvement in user experience improvements;
c. 45 of 62 identified Known Error Logs ("KELs") have been resolved;?
d. Providing a more considered and co-ordinated approach to change and
release management; and,
e. Securing the platform from interference by restricting access and
providing a clear audit trail when access has been used.
The phrase "Fast Fix", included in slide 32 (WITN 0368_1/1), describes a series
of activities, both technical and process-based, to rapidly address high priority
areas. High priority areas relate either to (i) issues which risked causing
discrepancies (as discussed at paragraph 16 above) and therefore potential
detriment to Postmasters; or (ii) items which were comparatively straightforward
to improve.
19. The immediate improvements were delivered between January and April 2021
(following KPMG's assessment) and were largely complete at the time of the IT
Teach-In sessions.
20. There were approximately 450 potential improvements to the functionality of the
current Horizon platform that POL's Horizon Architecture Team identified from a
range of sources. These sources included retailer feedback, training feedback,
meetings with the National Federation of SubPostmasters, product teams, and
3 Since the Teach-In sessions, all 62 of the KELs have now been resolved.
Page 7 of 16
11/75599520_1
WITNO03680100
WITNO3680100
supplier feedback (for example, from Computacenter) (shown at slide 14 of the
presentation (WITN 0368_1/1)), together with the focus areas for the team at the
time of the IT Teach-In sessions (shown at slide 15 of the presentation (WITN
0368_1/1)), including, for example, Moneygram cancellations (i.e. when a
customer transfers funds via Moneygram and the transaction is cancelled) and a
review of the Horizon screens.
21. The processes that were followed to change Horizon depended on the change.
The two primary routes for making changes to Horizon functionality have been
in place since 2015. These are: (i) enhancement / alteration of the Counter
Business Application ("CBA"), requiring development by Fujitsu, and (ii) changes
to the APADC scripts and reference data.
22. Reference data is the means by which the majority of the Horizon user
functionality is configured and created. This functionality consists of the
navigations between screens, the user menus, the help functionality and the
names of buttons and labels which are provided to branch staff in the execution
of their duties. Reference data provides user interface functionality. APADC
scripts are a means of coding sequences of transactions executed by branch
staff.
23. These mechanisms were created in 2008 and from 2015 to 2021 Atos
Information Services were responsible for the change, configuration and
deployment of user functionality coded in reference data to the Horizon counter
application used by branch staff with minimal input from Fujitsu. In April 2021
Post Office ended this arrangement with Atos and its Information Services team
is now directly employed by POL.
Page 8 of 16
11/75599520_1
WITNO03680100
WITNO3680100
24. The internal POL process for making data changes to Horizon is shown at slides
8 and 9 (WITN 0368_1/1) of the presentation. As can be seen from the slides,
the Request and Quotation ("RAQ") process has multiple stages. RAQ is a fairly
standard industry workflow model. It allows different teams to effect the changes
being made to Horizon as the system is developed. It is rigorous and requires
additional approvals specifically with the user in mind. Prior to Horizon product
management, a lack of end-to-end ownership and accountability meant: (i)
product decisions were made without sufficient guidance and oversight; (ii)
Postmaster requests in respect of Horizon were passed from team to team
resulting in longer issue resolution times or unresolved issues; (iii) more internal
interactions and effort was needed to address Horizon support, changes and
fixes; and (iv) there was limited product improvement planning or innovation due
to a lack of end-to-end ownership of all of the components that make up Horizon.
25. However, through Horizon product management the aim is to provide clearer
accountability. POL places a greater emphasis on customer-centric thinking and
on having a stronger Postmaster voice in IT and POL's business. This results in
a better understanding of Postmasters' needs and overall an improved response
to Postmasters.
26. Changes to the CBA follow a commercially driven process for scoping and
costing the changes and then a standard Software Development Lifecycle
("SDLC") as detailed on slide 17 of the presentation (WITN 0368_1/1). Here, the
diagram shows the input of Fujitsu (the third party referred to) and POL and lists
the types of changes which were implemented (including, for instance,
functionality changes, product changes and infrastructure changes).
Page 9 of 16
11/75599520_1
WITNO03680100
WITNO3680100
27. Following the HIJ, POL undertook preliminary diagnostics to identify where it
needed to make improvements to the management of the Horizon system and
allocated £11.7 million of funding to start to make those improvements. This
budget was used to identify underlying issues and their causes and to develop
then implement a programme of activity to remediate those. These covered
improvements in the management of changes, testing, governance, security and
controls related to the Horizon platform.
28. Slides 39 to 41 of the presentation (WITN 0368_1/1) outline the revised
governance process for the management of Horizon that POL was in the process
of implementing at the time of the IT Teach-In sessions. As noted at paragraph
15 above, these included the new Horizon Design Authority and Horizon
SteerCo, looking across all aspects of managing the Horizon platform.
Relationship between POL and Fujitsu as at 29 April 2021
29. In terms of the relationship between POL and Fujitsu, with respect to the carrying
out of updates and changes (including the means of communication about
these), as at 29 April 2021, amendments to the CBA are managed by Fujitsu (as
outlined above at paragraph 26). These changes follow a standard SDLC and at
the time of the IT Teach-In sessions, POL was in the process of reviewing this
SDLC with a particular focus on improving testing of changes to Horizon.
30. For reference, an overview of the POL and Fujitsu testing delivery model was
provided at slide 17 of the presentation (WITN 0368_1/1) and a high-level view
on the test improvements being implemented was included at slide 16 of the
presentation (WITN 0368_1/1). POL, from the time of the Teach-Ins sessions
onwards, typically adopts a risk-based approach to testing. For example,
Page 10 of 16
11/75599520_1
WITNO03680100
WITNO3680100
anything particularly high-risk, for example with a cash transaction attached to it
which could cause issues in Horizon, would have a longer testing period than a
smaller, low-risk change.
31. In response to question 47 in the Annex to the Teach-In Rule 9 Request, Fujitsu
is involved in the change process but it is not part of the "communication"
process. Instead, POL initiates the change and commissions Fujitsu, as the
provider of Horizon, to design, build, test and deploy the changes to Horizon.
32. Separate to the formal governance processes in place as referred to at
paragraph 28 above, the POL Horizon Leadership team have a standing weekly
meeting with the Fujitsu account team to discuss escalations, resolutions and
alignment of priorities.
33. Changes to the way in which POL engages with users were demonstrated in
slides 12 and 13 of the presentation (IWUIXXX]_1/1): [URN] which focus on the
role of the Postmaster when implementing changes to Horizon. This includes, for
example, the Postmaster IT Forum in which user feedback is sought in order to
improve Horizon and, ultimately, move away from the Horizon IT system (as
discussed at paragraph 34 below). User feedback is now regularly sought in
order to help POL prioritise improvements and change.
‘Pivot to Cloud’ and anticipated change to the POL IT landscape
34. The overall direction of travel for POL's IT is set out on slide 26 of the
presentation (WITN 0368_1/1). This slide highlights the approach being taken by
POL not only to pivot to a cloud-based service, but also to provide a new data
platform and to upgrade our current network infrastructure. Migrating to a cloud-
based service provides POL with greater flexibility in how it manages its systems;
Page 11 of 16
11/75599520_1
WITNO03680100
WITNO3680100
the migration should be fully complete by March 2023. It is just one part of POL's
IT transformation strategy, alongside POL's Strategic Platform Modernisation
programme ("SPMP").
35. The scope of the SPMP and the high-level timeline for this project is set out on
slides 29 and 30 of the presentation (WITN 0368_1/1). As set out on those slides,
the new system will be POL's core IT system in branch for a number of years
and branch migration is planned for 2023 to 2025. POL aims to migrate all users
to the new platform by March 2024 which will allow Horizon to be retired.
36. The SPMP delivered the first new technology for use in POL's new format
branches in 2021. Slide 31 (WITN 0368_1/1) sets out the new format branch
types; the type of branch determines what services are provided and what
devices will be used there. The devices include, for example, payzone
technology, parcel drop-off and pick-up, open counters, fortress counters and
self-service devices.
The Horizon contract in place as at 25 May 2021
37. I exhibit a copy of the Horizon contract which was in place on 25 May 2021
"Horizon Contract" (WITN 0368_1/2) (the "Horizon Agreement").
38. This agreement originally came into effect on 28 July 1999 between Post Office
Counters Limited (which subsequently changed its name to Post Office Limited
(i.e. “POL”)) and ICL Pathway Limited (which subsequently changed its name to
Fujitsu Services (Pathway) Limited) and was later novated to Fujitsu Services
Limited.
39. The contract shows that an extension of the Horizon Agreement took place in
2006. There have been subsequent change control notices agreed between the
parties during the term of the Horizon Agreement. By way of example, POL
Page 12 of 16
11/75599520_1
WITNO03680100
WITNO3680100
entered into two contract change notice (“CCN 1400” and “CCN 1500”) for
transitional support services, with the latter extending the agreement to 31 March
2018, for some but not all services.
Prioritisation of HIJ changes
40. As detailed at paragraph 40 above, POL identified a number of "Fast Fix" items
(detailed at slide 32 of the presentation (WITN 0368_1/1)) following diagnostic
activities carried out between October 2020 and January 2021. The prioritisation
of these activities was based on recommendations from our external advisors,
KPMG.
41. As explained at paragraph 18 above, issues with the Horizon system which
carried a risk of discrepancy (and therefore potential detriment to Postmasters)
were identified as being the highest priority.
42. Similarly, there is now a feedback loop in place (from the time of the Teach-in
sessions) which allows POL to prioritise change according to user feedback, with
Postmasters being directly engaged through a number of channels (for example,
the Postmaster IT Forum). This was a new forum at the time of the Teach-in
sessions which has subsequently expanded.
43. POL's communication with Postmasters has therefore improved.
The future of the Horizon contract as at 9 June 2021
44. On 6 April 2021, POL published a VEAT Notice [https://www.find-
tender.service.gov.uk/Notice/006992-2021] on the TED Europa portal notifying
the market of its intention to further modify the Horizon Agreement, including
amongst other things, a 12-month extension with an option to extend for a further
12 months.
Page 13 of 16
11/75599520_1
WITNO03680100
WITNO3680100
45. This modification provides for an extension on a reduced scope of services,
following POL data migration to 3rd party cloud hosting infrastructure. The
contract extension will also provide for termination assistance services and early
access to intellectual property which will assist POL in maintaining the Horizon
IT system before ultimately replacing it.
Replacement of Horizon
46. Asnoted at paragraphs 34 to 36 above, slides 28 to 30 (WITN 0368_1/1) outline
POL’s approach to the replacement of the current Horizon platform.
47. In particular, slide 29 (WITN 0368_1/1) sets out how POL will, over time, replace
the Horizon functionality. This was the position as of the time of the IT Teach-In
sessions and the SPMP for this is ongoing.
48. The SPMP has adopted the same engagement paths that POL uses for Horizon
improvements. I am aware that the SPMP team have also completed branch
visits in order to help identify what works well and what does not using the same
model and mechanisms set up for obtaining feedback on Horizon.
Development of Horizon replacement systems
49. As outlined at slide 13 (WITN 0368_1/1), POL has created a number of routes to
gather user feedback and their inputs into the design of the changes to Horizon
and also design of the future platform. These include meeting Postmasters
regularly, creating a pool of Postmasters for IT (the Postmaster IT Forum as
discussed at paragraph 42 above), Postmaster Co Creation workshops and a
culture of open discussions.
50. When POL has discussed the status of the current Horizon system with key
stakeholders, the POL Horizon Leadership team (as set out on slide 36) have
Page 14 of 16
11/75599520_1
WITNO03680100
WITNO3680100
also looked at the development and architecture of the new system which will
ultimately replace Horizon as well.
Statement of truth
I believe the content of this statement to be true.
GRO
Page 15 of 16
11/75599520_1
WITNO03680100
WITNO3680100
Witness Name: Simon Geoffrey Oldnall
Statement No: WITN 0368_1
Exhibit: WITN 0368_1/1 — WITN 0368_1/2
Dated: 20 July 2022
THE POST OFFICE
HORIZON IT INQUIRY
WITNESS STATEMENT OF SIMON
GEOFFREY OLDNALL
Herbert Smith Freehills LLP
Exchange House
Primrose Street
London
EC2A 2EG
Ref: 2066/9100/31048180
Page 16 of 16
11/75599520_1
WITN03680100
WITNO3680100
THE POST OFFICE HORIZON IT INQUIRY
Index to the exhibits to the Witness Statement of Simon Geoffrey Oldnall
No Exhibit Number Document Control URN
Description Number
1 WITN 0368_1/1 Teach-In 3 POL-0023190 I POL00028013
Powerpoint
presentation
2 WITN 0368_1/2 Horizon
- POL-0023096
Contract POL00027919
WITN 0368_1/2 Horizon 7
Contract POL-0023097 POL00027920
WITN 0368_1/2 Horizon 7
Contract POL-0023101 POL00027924
WITN 0368_1/2 Horizon 7
Contract POL-0023102 POL00027925
WITN 0368_1/2 Horizon 7
Contract POL-0023103 POL00027926
WITN 0368_1/2 Horizon 7
Contract POL-0023104 POL00027927
WITN 0368_1/2 Horizon 7
Contract POL-0023105 POL00027928
WITN 0368_1/2 Horizon 7
Contract POL-0023106 POL00027929
WITN 0368_1/2 Horizon 7
Contract POL-0023107 POL00027930
WITN 0368_1/2 Horizon POL-0023108
Contract POL00027931
11/75439354_1
WITN03680100
WITNO3680100
WITN 0368_1/2 conizon POL-0023109 POL00027932
WITN 0368_1/2 Horizon POL-0023110
Contract POL00027933
WITN 0368_1/2 conizon POL-0023111 POL00027934
WITN 0368_1/2 corizon POL-0023112 POL00027935
WITN 0368_1/2 conor POL-0023113 POL00027936
WITN 0368_1/2 conizon POL-0023114 POL00027937
WITN 0368_1/2 porizon POL-0023115 I 6) y9907938
WITN 0368_1/2 conizon POL-0023116 POL00027939
WITN 0368_1/2 porizon POL-0023117 I 6) ggq27940
WITN 0368_1/2 Contract POL-0023118 POL00027941
WITN 0368_1/2 conor POL-0023119 POL00027942
WITN 0368_1/2 conizon POL-0023120 POL00027943
WITN 0368_1/2 conzon POL-0023121 POL00027944
WITN 0368_1/2 qonzon POL-0023122 POL00027945
WITN 0368_1/2 Conon POL-0023123 POL00027946
WITN 0368_1/2 conan POL-0023124 POL00027947
11/75439354_1
WITN03680100
WITNO3680100
WITN 0368_1/2 Horizon 7
Contract POL-0023125 POL00027948
WITN 0368_1/2 Horizon POL-0023126
Contract POL00027949
WITN 0368_1/2 Horizon 7
Contract POL-0023127 POL00027950
WITN 0368_1/2 Horizon 7
Contract POL-0023126 POL00027950
WITN 0368_1/2 Horizon 7
Contract POL-0023129 POL00027952
WITN 0368_1/2 Horizon 7
Contract POL-0023130 POL00027953
WITN 0368_1/2 Horizon POL-0023131
Contract POL00027954
WITN 0368_1/2 Horizon 7
Contract POL-0023132 POL00027955
WITN 0368_1/2 Horizon POL-0023133
Contract POL00027956
WITN 0368_1/2 Horizon 7
Contract POL-0023134 POL00027957
WITN 0368_1/2 Horizon POL-0023135
Contract POL00027958
WITN 0368_1/2 Horizon 7
Contract POL-0023136 POL00027959
WITN 0368_1/2 Horizon 7
Contract POL-0023137 POL00027960
WITN 0368_1/2 Horizon 7
Contract POL-0023138 POL00027961
WITN 0368_ 1/2 Horizon 7
Contract POL-0023139 POL00027962
WITN 0368_1/2 Horizon 7
Contract POL-0023140 POL00027963
11/75439354_1
WITN03680100
11/75439354_1
WITNO3680100
WITN 0368_1/2 Horizon 7
Contract POL-0023141 POL00027964
WITN 0368_1/2 Horizon POL-0023142
Contract POL00027965