POL00028365
POL00028365
John Dicks
Director, Customer Enquiries
ICL Pathway Limited
Forest Road
Feltham
Middlesex TW13 7EJ 10 August 1999
Dear John
Re: Review of Acceptance Incident 218 - Training
An analysis of the evaluation against the business impacts identified in the
Acceptance Incident is attached.
Although many of the criteria have been met, it is regarded as significant that the
training and go-live process relies on the deployment of POCL HFSO resource. On
the basis of this evaluation, we are not prepared to reduce the severity rating from
‘high’.
POCLs view is that without this resource there would have to be a complete revision
of the training approach in order to ensure helpdesks were not rendered ineffective
by the high level of calls following the first and, to some extent, subsequent balances.
POCLs view is that HFSO resource was not deployed as an extension of training.
The cost impact and diversion of resource which this requires must be addressed by
ICL Pathway.
It is also POCLs view that the related adequacy of HSH support must be integrated
with this Acceptance Incident and removed as an additional source of concern.
The training improvements identified as part of the qualitative research by Post
Office Business Consultancy must also be addressed as part of a rectification plan.
Yours sincerely
Bruce McNiven
Director
Horizon Programme
cc. Mike Coombs, Chris French, Ruth Holleran, John Meagher
Horizon Incident Number 218 - Evaluation
1.
Criterion : 534/1
POL00028365
____ POL00028365
“Pathway’s Training solution shall take account of users experience in terms of automated products and platforms (ECCO+,
APT, ALPs) and their differing abilities to learn”.
2. BUSINESS IMPACT
SUMMARY OF SUCCESS CRITERIA.
MEASURE
EVALUATION
1. The Office Managers ability to
undertake daily balancing and
produce a cash account is adversely
impacted resulting in a failure to
support accurate POCL accounting.
This is a high severity impact on
POCL's ability to perform its normal
business functions.
™ Reduction in the number of offices unable to
complete the cash account balance process
and produce a cash account balance (relative
to the sample).
™ Continuing or better level of success in the
pass rate of the Performance Standard
Assessment (PSA) test.
™ Performance Standard Assessment (PSA) to
reflect live operation and standard practices;
Horizon users complete PSA again on day 10.
™ Data from BSM telephone survey for balance
related to the 4» August contained the
following; 22 offices produced an account, 1
office had a two week cash account. This
criteria is therefore met.
™ Criteria met.
™ Criteria met.
2. The consequential effect is that the
amount of time taken to produce the
cash accounts is excessive in relation
to the time taken on the previous
(manual) system and significantly in
excess of POCLs expectations for the
service,
™ Reduction in time taken to produce a stock
unit balance , the office balance and finally
produce a cash account (relative to the
sample.
™ Balancing statistics for the 4 August Cash
Account indicate an overall reduction in time
taken to complete balances in both sub offices
and ECCO offices. Even at the reduced level,
concerns remain about overall balancing times.
POL00028365
POL00028365
o
BUSINESS IMPACT
SUMMARY OF SUCCESS CRITERIA
MEASURE
EVALUATION
3. The consequences are also that the
number of cash account related
incidents reported to POCL NBSC is
considerably greater than expected.
(About a third of the calls coming to
NBSC Help Desk indicate a lack of
understanding of the cash accounting
and balancing process). HSH are
responsible for resolving these service
incidents but are unable to cope with
the content and volume of calls which
are therefore having to be dealt with
by NBSC. As the Manger’s training
course is deficient, NBSC and
presumably HSH staff who receive
this training course, are also
inadequately trained.
™ Reduction in demand on support - Measured
through a reduction in the number of calls (at
the peak time on Wednesday evening and
Thursday morning) for advice and guidance
to support stock unit balancing, office
balancing and production of the cash account
received at the HSH and/or at the NBSC.
™ Reduction in the length of calls from the
additional 25 offices.
™@ The overall number of calls in weeks 1, 2 and 3
by the LT2 offices showed a reduction on the
LT1 mirror offices for the equivalent three
weeks.
The average number of calls made by offices
during the non-peak days also showed a
reduction.
However, it should be noted there is a significant
increase in the 2.4 week cash account for both
LT1 and LT2 offices when there is no support at
these outlets, suggesting that some of the outlet
managers still do not have the confidence or
ability to complete the process unsupported.
™ The evidence to analyse this criteria is limited
and was regarded as indicative only. The broad
conclusion is that the evidence is insufficient to
make a substantive judgement regarding first
cash accounts but there is overall evidence to
suggest a reduction in call times for 2.4 cash
accounts. However, it again has to be noted that
the length of calls for both LT1 and LT2 offices
was significantly higher on 2"4 cash accounts
than the 1s cash account suggesting the critical
requirement for training to be
supported/delivered by HSH. It also
underlines the necessity of the HFSO support to
POL00028365
POL00028365
balancing in week 1.
BUSINESS IMPACT
SUMMARY OF SUCCESS CRITERIA
MEASURE
EVALUATION
4. The practical effect of the incident is
also causing the HFSO’s to devote a
disproportionate amount of time to
support the outlets on cash accounts.
The number of HFSOs that would be
required to support National Roll-Out
would be significantly greater than
currently envisaged (initial indications
are that two to three times as many
HFSOs as planned would be required.
This compounds the major impact on
POCLs resources.
™ No specific success criteria was identified to
address this business impact. Overall, POCL
would wish to reduce the cost of extended
training support at outlets through HFSOs.
POCL are now planning for 100% support of
first cash accounts and recognise that significant
additional support may be required for second
and subsequent balances at some offices. This is
acost and resource drain on POCL. It is also a
change to the original HFSO role which was to
support the KPI delivery for POCL and to
accelerate the learning curve at outlets. POCL
concerns on this impact remain.
5. There is also an impact on TP who are
having to process a significant
increase in errors on Class and Pivot
(up to 3 times as many weekly errors).
This is having a significant impact on
resources in TP during the live trial.
These errors will also raise liability
issues between the POCL and
subpostmasters, and POCL and client
organisations.
™@ Reduction in both the number of incidents
where Receipts do not equal Payments and
Incidents where balance B/F does not equal
balance due to PO on previous Cash Account.
™ Reduction in the number of errors reported
by TP - both CLASS and PIVOT errors
(relative to the sample).
Overall, the incidents of receipts not equal to
payments have reduced and the residual causes
are under investigation or have been resolved.
Criteria met. ;
The level of CLASS errors between 26'* May and
21st July has reduced. Without full information,
the indications are that PIVOT errors have also
reduced.
3.1
Qualitative Measures
Although the small sample size of 18 responses limits the validity of the findings, some significant improvements were found in
comparison to Live Trial 1 (a sample of 102). Overall, attitudes towards Horizon are better at the LT2 offices compared to the
LT1 experience. The key outstanding issues to emerge from research were as follows:
™ The course is still considered to be too short and intensive. ICL have proposed a pre-training course but details are
awaited.
@ The need to further stream the training groups. This issue has not been addressed by Pathway beyond the streaming
required by POCL for ECCO+ staff. Pathway’s response is to do this wherever possible. There are impacts on the
number of training places.
™@ Variation in trainer quality. Discussions taking place between POCL and ICL Pathway to look at how there can bea
greater quality assurance for trainer ability and consistency of delivering the course specification.
™ There are significant problems with technical and software faults in the training sessions. POCL regard these are
significant issues which will require rectification.
POL00028365
POL00028365