POL00278014
POL00278014
From: Ben Cooke[/o=ExchangeLabs/ou=Exchange Administrative Group
(FYDIBOHF23SPDLT )/cn=Recipients/cn=e1 1¢8d293fe6498392449fee5452b72f-Ben Cooke]
Sent: Thur 04/07/2019 7:38:22 AM (UTC)
To: Alisdair CameronI
Houghton[E-
Martin Kearsley[f_
lartine Munbyff_
7}; Angela Van-
Ce: Paula Jennerf,
Subject: RE: Branch Impacting Incidents Summary
Dear all,
Update on the open item.
Post Master Coin/Note receipt:
e — The fix was deployed to live by FJ at 6pm last night successfully
e — FJ will continue to send reporting on possible failures for some time until we are convinced it has worked.
e I’ve already seen some comms proposals ready to go to the network, but sensibly they are waiting to see how
this morning goes in live operation.
e Resolving the issues:
o Discussions have taken place with teams on handling the current and historical cases with finance,
supply chain and the Branch support center owners all clear.
o Historical cases have now been checked also, pre-April we had:
= Sterling - 173 cases, of which 167 are cleared, but 6 remain open
= Bureau — 33 cases, of which 31 remain open. Bureau issues were not being picked up by the
running process, but are now being addressed.
o Sterling have mostly been resolved
From: Ben Cooke
Sent: 28 June 2019 08:1
To: Alisdair Cameron
Martine Munby <r
lisdair Cameron <
; Debbie.K Smith <
Mark R Davies
Mohinder Kang
3 -; Julie Thomas
Angela Van-Den-Bogerd <ai3;
POL00278014
POL00278014
3 Somita Yogi
; Sree Balachandran
; Jessica Lloyd-Davies
“k>; Kim Abbotts
Subject: RE: Branch Impacting Incidents Summary
Dear all,
Update this morning:
Bureau forecasting: The inventory team last night confirmed that the bureau database they use for forecasting orders
had been rebuilt accurately. Tech teams from Arrow (DCoE) and Accenture stood by this morning and all was received
and working. The inventory team are now checking Wednesdays sent orders, against what they would have shipped
had they used accurate data, any branches that are short will have cash sent directly to make up the difference.
Post Master Coin/Note receipt: Post Masters are receiving packages of coins/notes but when they scan the pouch a
far smaller amount appears in Horizon as in-bound cash, leaving them surplus.
Update:
e Technical teams were engaged yesterday and working through the issue to try and isolate root cause.
e Analysis has begun of the data over the past months, in May/June:
© 180 emergency barcodes were used in total — these are issued by Supply Chain Accounts Team in the
case a discrepancy is found (& in other cases, hence these are not all instances of this problem)
o There are 12 open items on our GL for the period that have a mis-match between CWC shipped value
and Horizon received value. A check is taking place with the Supply Chain Accounts Team to confirm
these have all been rectified.
o Allinstances to date leave the Post Master with a surplus requiring correction.
e =Acallis occurring this morning with those who interact with branches, supply chain and those managing the
incident technically to agree the right way forward.
Regards
Ben
From: Alisdair Cameron
Sent: 27 June 2019 15:05
Ben Cooke <i
Rob Houghton 4 GRO ~~ 1; Debbie.K Smith
‘Hancock
>; Rina Patel
Julie Thomas
Martin Kearsley
Martine Munby ¢.
>
ncidents Summary
Thanks. We need to communicate, clearly, simply and openly with Postmasters so they know we are on it
and will feel reassured. Secondly, it is again not OK that the 5-10 were not escalated. Russell, in this instance
this is yours.
Rina, we need to decide if this is one comms or 2-3 given the other issues.
Thanks Al
POL00278014
POL00278014
Al Cameron
Interim Chief Executive
20 Finsbury Street
London
EC2Y 9AQ
From: Ben Cooke <&
Sent: 27 June 2019 14:46
To: Alisdair Cam
Debbie.K Smith
“>; Jeff Smyth
Martin Godbold
Jessica Lloyd-Davies 4
Cc: Paula Jenner Kim Abbotts < GRO
Subject: Branch Impacting Incidents Summary
Dear all,
We have two incidents running today that impact Post Masters/customers. Quick synopsis below:
Bureau forecasting: Inaccurate data feeds used by the Bristol inventory team may lead to some Post Offices cashing
out of EUR/S.
Problem — CFS data on branch positions is correct, but an incident yesterday morning on the CFS database caused a
delay in the batch schedule which led to a corruption of the bureau file. Hence the information used this morning to
create deliveries for branches was wrong. The technical teams are working to resolve the issue today, the inventory
team will need to evaluate which of these orders not have been correct, so that Royal Mail Direct Deliveries can be
made to branches under supplied.
Post Master Coin/Note receipt: Post Masters are receiving packages of coins/notes but when they scan the pouch a
far smaller amount appears in Horizon as in-bound cash, leaving them surplus.
Understanding to date — An incident has been on the Back Office program log of a single instance of this issue for
some time, however it was resolved with a work around provided and not flagged as a repeating issue. It’s now clear
that the Supply Chain Accounts team have been dealing with ~ 5-10 incidents a day. The priority has been upgraded
and teams are being engaged to investigate urgently the magnitude of the issue and the technical fix. A review is
taking place with Supply Chain, the project team and Kim Abbotts to ensure that there are no other incidences being
under reported that need more urgent prioritisation.
\’'m pleased to report that the fix deployed during the incident with duplicated Transaction Acknowledgments on
Monday has worked, with negative acknowledgements cancelling out all errors.
I’m sure you have more questions — I will brief Rob with the root causes and more information for Monday’s GE and
continue to update as we progress and close.
POL00278014
POL00278014
Ben