POL00448075
POL00448075
Compliance
Policy Monitoring Report
Postmaster Accounting Dispute Resolution Policy
February 2023
CONFIDENTIAL
POL00448075
POL00448075
Contents
1. Scoring Table
2. Overall Rating / Residual Risk Score
3. Objective of the review
4, Background
5. Methodology
6. Source of information/References
7. Findings
8. Recommendations
9, Policy Owner Response
10. Agreed actions to be taken
11. Review Dates/Sign Off
1. Scoring Table
Below table sets out the Residual Risk Score and Rating that will apply upon review of the
Postmaster Accounting Dispute Resolution Policy, to determine how effective the policy is, any
control weaknesses or gaps and whether the policy needs enhancements/improvements.
CONFIDENTIAL
POL00448075
POL00448075
The framework of governance, risk management and control is adequate and effective.
Some improvements are required to enhance the adequacy and effectiveness of the
framework of governance, risk management and control.
There are significant weaknesses in the framework of governance, risk management and
control such that it could be or could become inadequate and ineffective.
There are fundamental weaknesses in the framework of governance, risk management
and control such that it is inadequate and ineffective or is likely to fail.
2. Overall Rating / Residual Risk Score of The Review
The overall rating and residual risk score applied to this review is:
There are fundamental weaknesses in the framework of
governance, risk management and control such that it is
inadequate and ineffective or is likely to fail
This is predominantly driven by:
¢ = The failure to track all appropriate metrics, as set out in the table at section 7
Discrepancies in the evidence provided which suggest that standards are not being applied
across the board. In particular, call monitoring logs are incomplete or missing.
@ The minimum controls described within the policy are also not adequately articulated as
controls.
@ The need to assess the E2E processes and their monitoring to ensure ‘dispute resolutions’
outcomes are appropriate and as per policy expectations.
What is working well:
¢ Investigations are undertaken quickly
¢ Postmasters are provided with information at each relevant stage
3. Objective of the review
To assess the validity of the policy within the universe of risk framework. To review the lead and lag
indicators of the policy and to sample check some of the key minimum control standards in the
Postmaster Accounting Dispute Resolution Policy. Finally, to assess whether the effectiveness of
the policy is being implemented across the group.
4. Background
The Postmaster Accounting Dispute Resolution Policy is sponsored by the Central Operations Director,
who has overall accountability to the Board of Directors for the design and presentation of controls
for managing accounting disputes with postmasters.
CONFIDENTIAL
POL00448075
POL00448075
The policy sits within a suite of postmaster support policies that have been established with the inten-
tion to set the minimum operating standards relating to the management of our postmaster contract
risks throughout the business. The suite of policies purport to be in-line with Post Office’s risk appetite,
which is primarily averse.
The Postmaster Accounting Dispute Resolution Policy was drafted, reviewed and approved by the
RCC & ARC in March 2022. This review forms part of the policy assurance review programme.
The Postmaster Accounting Dispute Resolution Policy has been established to capture the
minimum operational standards required relating to: (i) POL’s obligation to investigate accounting
disputes; and (ii) ensuring ongoing transparency between POL and postmasters.
5. Methodology
The assurance review will consist of the following:
- Is risk adequately identified?
- Is the risk appetite correctly identified?
- Are the key personnel correctly identified?
- Are reported minimum controls actually controls?
- What are the key controls?
- What are the key metrics?
- _ Is the process/procedure correctly articulated?
- Does the evidence show the policy is working?
- Given the above, can we be sure the policy is fit for purpose?
6. Source of Information
The review is based on examining the policy, assessing the control metrics against the desired
outcomes and reviewing supporting materials supplied by the Policy Owner.
The source of the information came from:
- Postmaster Accounting Dispute Resolution Policy
- Policy Owner-supplied evidence
Initial requests for evidence from the business to show compliance with the policy resulted in only a
single example for each control being produced. This was insufficient to reach any considered
findings and so access was sought to Dynamics, the system that houses case files and investigations.
Despite support being provided by the business area, access was not available at the time of drafting
the first iteration of this report. Preliminary findings were shared with the Policy Owner and a
workaround was found to access additional evidence. The findings in this report have been updated
to reflect the provision of the additional evidence. Gaps in the evidence still remain and these are
set out in the tables in section 7 below, along with the relevant explanation for their absence.
7. Findings
Issue Finding Rating
CONFIDENTIAL
Is the policy capturing the
correct risks?
The key elements of the inherent risk
have been identified.
Cause — By failing to carry out a timely,
fair and transparent investigation;
Event — there may be loss of postmaster
confidence in POL, financial losses and
legal challenges, as well as a lack of
accountability;
Impact — leading to damage to the POL
brand, financial loss and regulatory,
reputational and financial risks.
Is the risk appetite correctly
identified?
There is an adverse risk appetite in
connection with dispute resolution. The
process identified within the policy is
consistent with this appetite level.
Are the key personnel
correctly identified?
The policy is missing several key players
within the accountability framework.
Consider whether there should be
reference to the Group Investigation Unit,
Assurance, and GE.
Are reported minimum
controls actually controls?
Ten out of the 14 minimum controls that
are identified within the policy (identified
below by *) are actually process points
rather than controls. The policy should be
reworked to assess controls.
What are the key controls?
Key controls are not readily identifiable
within the policy as whilst the relevant
areas are identified, they have been
formulated as processes and procedures.
Are the KPIs adequately
identified and measured?
The examples of the weekly report
provided, indicate that most of the
appropriate KPIs are being tracked. There
is a clear record of how many
investigations are underway, how old
they are, how they are being monitored
and who by. It is not clear whether there
is a tracker in place identifying what’s
happening on each case, eg if it is written
off etc. In the absence of any clear
capturing of end-to-end process
monitoring, this is likely to result in
issues/problems being missed.
CONFIDENTIAL
POL00448075
POL00448075
POL00448075
POL00448075
Is the process/procedure When the controls are viewed as a
correctly articulated? process plan, this seems like a suitable
process/procedure to achieve the end
result.
Does the evidence show the There are weaknesses identified in
policy is working? records management, especially around
training. There are discrepancies in the
records or reviews of call logs and the
coaching log tracker. There is a risk that
this will result in inadequate oversight as
an inaccurate picture is being provided on
escalation. Given the inadequacies in the
supporting evidence, it is not possible to
be confident the policy is working exactly
as it should. The evidence required to
show the policy is working would be:
Reduction in complaints from
Postmasters in respect of
inadequate/unfair investigations;
reduction in reported losses to
Postmasters and Post Office; reduction in
legal/regulatory challenges; increased
Postmaster confidence in this area.
Given the above, canwebe _I The policy appears to be fit for purpose,
sure the policy is fit for however there is insufficient evidence to
purpose? show that the policy is being adhered to.
It is tracking the appropriate metrics and
there is some evidence that the policy is
being adhered to. Additional evidence is
required to fully assess the policy and
there are discrepancies in some of the
documents provided.
CONFIDENTIAL
Conducting an
incomplete or
unfair
investigation
If Post Office fails to
investigate
accounting discrepancies
thoroughly, fairly = and
transparently,
Postmasters may lose
confidence
in our ability to investigate,
and in
some cases may lead to
postmasters incurring
financial loss
If Post Office fails to
investigate
accounting discrepancies
thoroughly, fairly and
transparently,
Post Office may be in breach
of
their contractual or
regulatory
obligations, which may lead
to legal
challenges.
If Post Office fails to
investigate
* Post Office will
undertake an
investigation into
accounting
discrepancies in a fair,
transparent, and
impartial manner in
order to — confirm
whether they are
genuine — Established
Losses or Established
Gains.
* When required, a
Case Review is held by
managers to review
escalated cases and
the open position *
* Weekly Case Review
is held by managers to
make a decision on
escalated cases and,
where necessary,
prepare cases for
escalation to the
Monthly Committee *
Head of
Network
Support and
Resolution
Head of
Network
Support and
Resolution
Head of
Network
Support and
Resolution
As required
As required
Weekly
Screenshot
examples of
Tier 2
investigations
were
provided by
the Network
Support &
Resolution
Team.
Links — were
provided to
an Excel
spreadsheet
of a weekly
review anda
slide deck of
monthly
committee
escalated
cases.
POL00448075
POL00448075
Evidence was not
provided of any
direct evidence of a
Tier 1 investigation
taking place,
although some
indirect evidence
was available in the
form of examples of
audited Tier 2 calls.
The Tier 2
investigation reports
provided by the team
meet the criteria
required under the
policy.
The Tier 2 review
examples also
provide some
evidence of
escalation as
required. Similarly,
the weekly and
monthly escalated
cases provide
evidence of
compliance.
CONFIDENTIAL
If Post Office does not set out
the
investigation process to the
Postmaster at the beginning
of the
investigation,
will not
understand their options at
each
stage of the process.
Postmasters
either resolve the case
or explain the next
steps.
* When cases are
resolved by Tier 2 or
Tier 3, the postmaster
is contacted. The
report is discussed on
the telephone and a
copy shared with the
postmaster on
request. If the
postmaster disagrees
with the finding they,
out to a
Postmaster
following
resolution of a
dispute.
accounting discrepancies I * Monthly Review
thoroughly, fairly and I Committee is chaired Monthly
transparently, by the Head of
Postmasters may lose I network Support and
confidence Resolution, includes
in our ability to investigate. representatives from
legal, contracts,
Network Monitoring
and Branch Support
Centre. *
Insufficient If Post Office fails to keep * Recorded calls are I Team Quarterly Call Full access to the call
communicatio I postmasters informed of I monitored, reviewed I Manager monitoring monitoring logs was
nwith progress, and feedback given. * Daily logs and I provided by the
Postmasters postmasters may lose Team coaching log I team. Monitoring
confidence in « All cases are I Manager tracker. began in June 2022
the transparency of the acknowledged by the and all _—_—s records
investigation process. Triage team, without I Operations I Daily An example of I between June and
undue delay. They will I Manager a letter sent I September in the
sharepoint folder
were reviewed.
Monitoring consists
of one call per team
member per financial
period/month being
reviewed against set
criteria. If the call
falls below an
identified standard,
an improvement plan
should be identified.
The call monitoring
template _ indicates
that 4 calls should be
CONFIDENTIAL
POL00448075
POL00448075
or their
representative, can
escalate the case to
Tier 3 or the Review
Committee. *
reviewed each
month, however an
excel — spreadsheet
entitled “call
monitoring control”
provided indicated
only 1 call should be
reviewed.
Discussions with the
team revealed that
the figure had been
reduced from 4 to 1
in light of the size of
the team and the
consequential
workload that would
ensue.
Looking at the review
logs, one call per
team member listed
in the sharepoint
drive does appear to
have been conducted
each month since
June, or a reason
such as sick absence
for the period has
been provided.
However, when cross
reviewed with the
coaching log there
are additional names
CONFIDENTIAL
POL00448075
POL00448075
present each month
which do not appear
in the sharepoint
folder and do not
have a saved record
of a call review.
There is also no
evidence that
improvement _ plans
are being — given.
Whilst there is a tab
present for a
coaching plan on
almost all of the
records, none are
completed. One
team member was
identified as
“developing/2” in the
June reviewed call
and no plan was
identified. In the July
call, the same
employee’s average
score had dropped to
1.5 and still no plan
was identified.
Scores are routinely
rounded up from the
average. Someone
with an average
score of 1.5 will be
CONFIDENTIAL
POL00448075
POL00448075
rounded up to 2
giving them a
developing grade
instead of minimum
standards, despite
meeting only the
minimum standard in
half of the recorded
targets. This carries
an inherent risk of
overconfidence in
standards, meaning
the opportunity to
improve — standards
through training may
be missed.
The Coaching Log
Tracker noted that
some people marked
as “developing” were
in need of monitoring
and others were
recorded as needing
a coaching session. It
was not possible to
link the findings in
this log back to the
call monitoring
records as some
records were absent
from the call log
reviews and no
CONFIDENTIAL
POL00448075
POL00448075
coaching plan tabs
were completed.
More confusingly,
most people who
were graded as
“competent” were
marked as needing
no further action,
however some were
marked as in need of
monitoring. Again,
no clear explanation
for this distinction
was apparent from
the log and it was not
possible to cross
reference the log to
the call ~— review
records.
Furthermore, there
were entries on the
log where staff were
marked as Advanced
in the review, but the
actual log recorded
them as only
competent and vice
versa.
One example of a
letter to a
Postmaster was
provided which met
CONFIDENTIAL
POL00448075
POL00448075
the criteria set out in
the policy. This letter
included telephone
numbers to call if the
Postmaster was
unhappy with the
offer set out therein.
Failing to If Post Office do not offer I « If support I Team Daily Examples of a I The examples
signpost support requirements are I Manager completed provided indicate
support when it is identified, the I identified, the case Training that training was
requirements same issue handler will escalate to Quarterly Intervention being appropriately
could reoccur the relevant I Operations form, as well I identified and
department, which I Manager as an email I escalated.
could include Training, I and escalating a
Network Monitoring I Head of case to I Additionally, some
(SPEAR visit support), I Network Network staff had not
Area manager support. I Support and Monitoring completed the
Lai Resolution training at the point
Copy of Root I of the evidence being
* Root cause analysis is case slides I provided (although it
performed to identify from was agreed that this
the common causes of committee would be remedied
discrepancies and that week).
feedback provided to
relevant areas. *
Failure to If the Service Level is not I ¢ A report is run from I Team Daily Access to the I Access to the weekly
investigate met, MS Dynamics to show I Manager weekly case I report of cases was
within agreed Postmasters may lose I all cases that have report provided. A
timescales confidence breached the — set Daily/Weekly I sharepoint significant number of
in our ability to investigate in I Service Level. * Team folder the cases on the list
a Manager were marked as red
CONFIDENTIAL
POL00448075
POL00448075
timely manner.
* Daily and weekly
case reviews are
A screenshot
of a Kanban
and
older
substantially
than the 21
carried out to review board and I days set out in the
the case position, associated policy.
including age and management
progress. * Information
which is
stated as
being used
daily and
feeds into
weekly review
meetings
Insufficient If investigation I « The Case I Team Daily There is insufficient
documentation I documentation is Investigation Report is I Manager documentation on
not uploaded to Dynamics, I uploaded to the file. The coaching log
no audit Dynamics record and lists call records
trail will remain on the I includes all of the key gradings and
decision-making process and I evidence —_ associated coaching plans that
data used to with the decision are not replicated in
inform it. making. * the records.
Policy Non adherence to the policy I * The Accounting I Head of I Once Whilst there is clear
adherence could Dispute Resolution I Network approved evidence of the
result in financial loss, legal I Teams and the I Support and I and policy being followed
and Postmaster Account I Resolution annually on some occasions,
regulatory risk, detriment to I Support Team will be Head of I thereafter there is evidence of
postmasters and I provided with training I Network (or incomplete or
reputational on this policy, with Support and I sooner in the inaccurate records
damage to Post Office. regular refresher I Resolution event of being kept.
sessions. material
changes to
the
CONFIDENTIAL
POL00448075
POL00448075
* The Policy should be policy).
reviewed, and if
necessary updated. As required
(but
reviewed at
least
annually)
Upon review of the Postmaster Accounting Dispute Resolution Policy and the above-mentioned Minimum Control Standards which are stated within the
policy, it is evident that some of these areas and controls have been adhered to and are fit for purpose and work in pursuit of day-to-day business.
However, some gaps have been identified and it has not been possible to view the necessary evidence for some areas in order to be able to draw a
complete conclusion. As a result, the policy assurance has been recorded as ‘needs significant improvement’.
8.
Recommendations
Following the assurance review of the Postmaster Accounting Dispute Resolution Policy, the following recommendations are made:
The “minimum control standards” section of the policy is heavily focused on procedural steps rather than actual controls — the policy needs
updating to include key controls and metrics.
The policy is updated/amended to reflect the missing key personnel.
Reviewed call logs need to be monitored and all completed records should be uploaded. Training plans should be completed as needed and the
tracking log should match the records held in the logs.
Review policy processes and procedures with an EDI lens.
The policy is further reviewed after access to the main systems is obtained, so that additional evidence can be assessed. Evidence should be
provided within four weeks of receiving the preliminary findings and the further review completed within three months.
CONFIDENTIAL
POL00448075
POL00448075
9. Policy Owner Response
10. Agreed actions to be taken
Action
Owner
Date to be completed by
13. Review Date/Sign Off
Policy Review Date
Next Policy Review
Date
Review Conducted
Br
Review Signed Off
Br
fi ‘Senior
Legal Counsel, DP & FO!)
CONFIDENTIAL
POL00448075
POL00448075