ICL
Pothwoy
Audit of Customer Service
Ref:
Version:
Date:
FUJ00080683
FUJ00080683
TA/REP/025
1.0
17/04/00
Document Title:
Document Type:
Abstract:
Status:
Distribution:
Author:
Comments to:
Comments by:
Pathway Customer Service Audit
Report
This report documents the outcome of an internal audit of
Customer Service conducted during April 3" and 4" -
2000.
APPROVED
M. Stares
S. Muchow
A. Nicholson
R. Brunskill
D. Wilcox
B. Parker
Library
Brian Parker - Group Q&BE
Jan Holmes
M. Coombs
M. Bennett
P. Curley
J. Wright
P. Lywood
D.Groom
© 2000 ICL Pathway Ltd
COMMERCIAL IN CONFIDENCE
Page 1 of 15
ICL
Pathwoy
Audit of Customer Service Ref:
Version:
Date:
FUJ00080683
FUJ00080683
TA/REP/025
1.0
17/04/00
O Dotwment control
O.L Document history
Version Date Reason
on 10/04/00 First internal draft for comments
Lo 17/04/00 _I Raised to Issue status following presentation to CS Management Team by B.
Parker 14/04/00.
0.2 Approvel authoritiey
Name Position Signature Date
M. Bennett Director Quality & Risk
0.3 Associated documenty
Reference Vers I Date Title Source
fi] AUD/3/4/13 1.0 (02/02/00 _ I Audit Terms of Reference PWAY
COMMERCIAL IN CONFIDENCE
Page 2 of 15
ICL
FUJ00080683
FUJ00080683
Audit of Customer Service Ref; IA/REP/o25
Version: 1.0
Pathway Date: 17/04/00
0.4 Table of content
UMtrOMUCTION .....esceescesseeseeeseessessesseesseeseessecseesssssessecsseessesueesessessseesessseesessnsasecseeeeeses
Scope & CONMUCE oo. cceccssesceecsesesseseseeesseseseessesessessseeaesseseseeeeseseseeseeneee
3Management Summary
3.1Incident & Problem Management
3.2Business Incident Management (CS/MAN/o05) .
3.3Change Management - Release Management .
3.4Change Management - Reference Data (CS/MAN/003 - 5.2).
CPN DU US
3.5Operational Test Team .........ccssessecssessecseesessesseessesseesseeseessessecsssnssseseeseeses
3
4Detailed Observations .......cccssessseesessessesessessesssseeessesssneseeseenssnesesseeeeeeeassneeneneenees
)
4.rCurrent Cocumentation occ ssseeeeseseeeneseeeeneneseeesnenseseeeneneeateneee
3
4.2PrOCeSS IMPFOVEMENL .......cseceesesesesesteesesesessesesesceseseseeeesesessseecsnesssssesesesetecaes
4.3Training and skills records .........sceccecsesssecseeseeseessessecseessecseesseesessesssecseesseess LO
4.4Quality Management System.......c.cccsssecesseessseseessessessseeseeseessesssssessesssessesy 10
5Annex A Terms of Reference.....ccccsssssesessessessesestessesssnesssseeesseatessseesestsssseseeseenees LO.
6Annex B Proposed Process Model .........:+ssessessseessseessecsseessseesseessseessessseesseeesseess IO.
COMMERCIAL IN CONFIDENCE Page 3 of 15
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Vent IA/REP/025
ion: 1.0
Pathway Date: 17/04/00
1 Introduction
The purpose of this audit was to review the activities and operation of ICL
Pathway Customer Service, with particular regard to their key management
processes. The plan was agreed to include all aspects of the following activities
and their key interfaces with other operations:
> Incident Management & Problem Management
Vv
Business Incident Management
v
Change Management - Release Management
Vv
Change Management - Reference Data
Vv
Operational Test
Within each of those processes to take a closer look at key management aspects
that are present :
Escalation
Vv
Reporting
vv
Root Cause Analysis (where appropriate)
Vv
Service Improvement
2 Scope & Conduct
Formal Terms of Reference, reference AUD/3/4/18, were provided at a pre-
assessment planning meeting, together with a Customer Service Organisation
Chart and copies of the following related procedures:
CS/MAN/0o02 - version 3.0 - Support Services OM
CS/MAN/003 - version 3.0 - Operational Services OM
CS/MAN/o0s5 - version 3.0 - Infrastructure Services OM
CS/MAN/006 - version 3.0 - General Information OM
PA/POL/o02 - version 1.2 - Quality Management System
Note: The procedures seen often repeat sections of text and flowcharts. For
example, CS/MAN/oo2 Appendix A.3 - Support & Release Management Process is
the same as CS/MAN/o03 Appendix A but this is a later version. It will become a
difficult, unnecessary and labour intensive task to try to keep all these documents
aligned, particularly as they have different owners.
COMMERCIAL IN CONFIDENCE Page 4 of 15
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Vent IA/REP/025
ion: 1.0
Pathway Date: 17/04/00
The requirements expressed in PA/POL/oo2 Version 1.2 Dated: 28/02/00, which
conforms to the requirements of BS EN ISOg001:1994, were used as a basis for
the work.
The audit was carried out by Brian Parker, Group HQ Q&BE - Programme
Manager - Process Management who also produced this flowchart to aid his
understanding of the end-to-end process. A modified version, correcting any
misconceptions, would form a sound basis for an electronic QMS in line with
corporate strategy. This is shown at Annex B,
The audit consisted of interviews with:
> Paul Curley, Service Mgr responsible for Incident and Problem
Management;
> Richard Brunskill, responsible for Business Incident Management;
> John Wright, Service Manager - Change Management, for Release
Management;
> Dave Wilcox, Reference Data Management;
> Pat Lywood & Denise Morris - Operational Test Management.
All staff seen were most co-operative and they have my thanks for their support
and hospitality.
The relevant procedures were tested during the implementation of the plan and
the results are included in the management summary.
3B Management Summary
There is a rota for the role of Duty Manager whose responsibilities include the
appointment of a Problem Manager where the situation warrants one. The
Duty Manager function was not included in the scope of this assessment but
has a significant impact on the management of a problem as described later.
3.1 Incident & Problem Management
The role assessed was that of Service Manager whose task is to ensure that high
priority incidents and problems are resolved within the required time and
within SLA, whose key resources are the virtual team of Problem managers.
This is a relatively new role and the majority of effort is currently dedicated to
‘A’ &'B' priority calls.
An Incident was defined as "An occurrence during normal operations that has
been reported to the ICL Helpdesk".
A Problem was defined as "A significant problem or trend” where significant
can mean an incident of 'A’ priority; money related; adverse effect on service
COMMERCIAL IN CONFIDENCE Page 5 of 15
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Vent IA/REP/025
ion: 1.0
Pathway Date: 17/04/00
levels; business continuity; penalty clause may be invoked. (Measured as
throughput, ICL to counter/client.)
There is a continuous monitoring of volume and progress via the PinICL and
Problem Management databases.
A key measure is achievement of Action Date and any slippage is progressed on
a one-to-one basis. These one-to-one interviews are not recorded so trends and
individual performance cannot be assessed.
There is a formal weekly review at which trends and hot spots are discussed.
These are minuted and actions progressed.
The appointed Problem Managers also have a variety of day jobs and associated
skills and tend to operate to the style most familiar to them, achieving the desired
outcome in a variety of ways.
There is an opportunity to identify who has the most effective working practices
and adopt them as common practice.
The Service Manager is developing a common check list to include, reporting,
resolution and escalation routes. An early draft was seen during this
assessment.
Trend analysis.
There is a pack of standard reports, produced monthly by MIS, that show the
analysis of closed calls by outlet, long outage etc as part of the contractual
agreement with POCL.
There is no evidence of preventative action resulting from the information
provided in the standard reports.
E.g. Belfast outlet shows a call rate that is above the threshold and the reason is
still being sort.
The stated intent to design reports specific to problem management, that will
include a ‘traffic light’ system of exception reporting allowing available time to be
focused where most needed, should be pursued with urgency.
[ The Service Management process is not described in any of the procedures seen.
COMMERCIAL IN CONFIDENCE Page 6 of 15
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Vent IA/REP/025
ion: 1.0
Pathway Date: 17/04/00
3.2 Business Incident Management (CS/MAN/OOS)
The Management Support Unit are tasked with the resolution of Business
Incident.
A Business Incident was defined as any sot transaction failure due to a
mismatch between information held in different parts of the Pathway system
and cause financial irregularities.
The process followed by the MPU is broadly described in CS/MAN/005.
The trigger for the start of this process is the daily report from the EPOSS GAPS
systems, delivered electronically to Pathway Customer Service via a secure
system. The Pathway SSC extracts these reports and Email to the MSU. This
dependency is not included in CS/MAN/oo5 - 4.3 EPOSS & APS reconciliation or
CS/MAN/o02 - 4.0 System Support Centre procedure.
Detailed procedures for Business Incident Management were not available but
are planned for completion by mid-April.
Note: Printing an extract from CS/MAN/oos5 resulted in the paragraphs being
renumbered.
Examples were seen of reconciliation errors caused by reference data errors,
where the data available at the outlet does not match the POCL master
reference system. Since POCL provide the reference data in the first place, and
having examined the rigorous pre-release test criteria for updates to reference
data that includes confirmation by POCL, it was difficult to see how this could
happen.
The root cause analysis should include a supplier assessment of the POCL
reference data validation process.
ICL also appears to be responsible for identifying the cause of a large number of
incidents with no control over the cause.
POCL provide corrections direct to the end user. How the POCL, ICL
Reference Systems and the Outlets all stay in sync was not determined.
The success of this task is dependent on the skill and tenacity of the team. The
records of training provided and skill levels achieved were not available as
required by ICL's Performance Plus process.
COMMERCIAL IN CONFIDENCE Page 7 of 15
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Vent IA/REP/025
ion: 1.0
Pathway Date: 17/04/00
3.3 Change Management - Release Management
Deirdre Conniss was unfortunately not available on the day and John Wright
kindly deputised.
The software aspect of Release Management process is included in
CS/MAN/003 - 5.1. Change requests are considered weekly by the Release
Management forum, a formal review group. These sessions are minuted and
actions emerging are progressed. Decisions to proceed with a software change
result in an authorisation for Development to design a fix. Where there is a
greater urgency that cannot be held until the next weekly review then there is
an escalation route to fast track the problem
Reasons for not proceeding are also identified and agreed with POCL
Fixes received from Development result in an Authorisation to Test which is
then routed to the OTT.
Reference Data release is included in CS/MAM/003 - 5.2 and CS/PRO/o81
Control of release is via the RDMC workstation. The type of data change
required determines priority. Change requests in the pipeline are identified as
ready for release by the change of status. When final validation is confirmed by
POCL, status is changed to '8' which authorises others to distribute.
Overall, this is a well known and secure process although not all operatives are
familiar with the supporting documentation.
3.4 Change Management - Reference Data (CS/MAN/OO3 -
5.2)
Reference Data was said to include system changes such as new screen buttons
as well as data related to products and prices.
The bulk of data is received electronically from the Ref. Data Mgt Centre -
Bootle (RDMC) but can also be by Email or telephone.
This operation is supported by detailed procedures such as Cs/PRD/o30 Vi and
CS/PRD/058 V3 - Lead times.
The process for evaluating achievement of POCL targets is under development.
The emphasis is on getting date released to the outlets when it is required
rather than to the published measures of turn-round times.
A number of reports of achievement were seen. An RDORF report for February
2000 shows an average detected error rate of 29% of data received from POCL
COMMERCIAL IN CONFIDENCE Page 8 of 15
ICL
FUJ00080683
FUJ00080683
Audit of Customer Service Ref; IA/REP/o25
Version: 1.0
Pathway Date: 17/04/00
All errors are returned to POCL for correction. The corrected data then follows
the same validation route. There is no indication of what percentage of the
reworked data fails these second or subsequent checks. It is not possible at this
stage to determine if all product codes being released are included in POCL's
own systems used for reconciliation purposes. (see Business Incident
Management).
Although the Reference Data Failure/rejection process is well understood by the
Reference Data team, it is not included in any flowcharts or documentation seen.
There are a number of instances where the expected turn-round times have not
been met (approx 2%). A preliminary analysis of the cause of late deliveries
forms part of the standard report.
The final confirmation that the new data can be released is by Email form
POCL. These confirmations should be kept for a defined period in case there
should be a dispute.
3.5 Operational Test Team
Operational Test receives an "Authorisation to Test" from Release Management
database which is monitored daily by the OTT Manager.
Work packages, and their urgency, are identified and allocated to a daily work
schedule. The history file is used to recreate the original problem on the test
rig.
A Test Script is produced based on the original problem and the nature of the
solution. This script is later filed so that the exact test sequence can be
repeated if necessary.
OSD are then authorised to apply the fix to the test rig and all tests are carried
out again to confirm that the fix has been successful and that there are no other
adverse effects.
When all tests are successful the Release Note is returned to Release
Management.
There were no work instructions for configuring the test rig, pre-testing or for
rejecting a work package should the tests fail other than the overview in
CS/MAN/o02 - 5.0.
COMMERCIAL IN CONFIDENCE Page 9 of 15
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Ref: IA/REP/o25
Version: 1.0
Pathway Date: 17/04/00
Operational Test are developing a set of short but succinct set of work
instructions that describe how each operation is to be performed. It would be
easier to demonstrate their completeness if a simple flowchart were used to
structure the material. It would then be a simple task to present this
electronically with hyperlinks to the appropriate work instructions.
eg.
Relum
Yes -eIRelease — f-eeI OTTH0D4
Receive Reproduce , Note Le
Release Original rely tix & ay
Note Problem NY Reject
1 1 + No—eelthe [oe] OTTIDOS
package L—
oTT001 oTT02 oOTTMO3
Delivered Quality - OTT are tasked with carrying out random samples of
hardware as an ongoing check of delivered quality. This activity, and what
happens with results, is not recorded in procedures seen
4 Detoiled Observations
4.1 Current documentation
The QMS procedures seen were the top level documentation appropriate to the
operations being audited and are part of a centrally controlled library (PVCS?).
There is no indication on the hardcopies seen for how they would be updated
with more recent issues and they therefore appear to be uncontrolled once
printed. If there is a distribution control it was not evident, if there is not then
all hardcopies should be clearly marked:
Any hardcopy of this document is NOT UNDER CHANGE CONTROL
unless otherwise stated. (see Group/Q&BE/03 on Café Vik)
There are instances where the procedures describe the core flow of a process
but not what happens when a decision or test fails.
4.2 Procesy improvement
The following opportunities for improvement were identified and discussed
during this audit.
COMMERCIAL IN CONFIDENCE Page 10 of 15
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Vent IA/REP/025
ion: 1.0
Pathway Date: 17/04/00
> Identify which Problem Managers have the most effective working practices
and adopt them as common practice.
> Structure management reports to provide exception reporting, providing
focus on the failing areas most in need of attention/improvement.
> Consider agreeing a supplier assessment of POCL to find out why, after all
the rigorous pre-release testing, Reference Data team find 30% errors and
reconciliation failures occur on validated data.
4.3 Training and skilly recordy
No training or skills records could be shown in areas where the success of the
operation is dependent on the skill and tenacity of the team. The records of
training provided and skill levels achieved are a requirement of ICL's
Performance Plus process, PA/POL/oo2 - section 3.18 and BS EN ISOgoo1
section 4.18.
4.4 Quality Management System
The sections of the QMS seen were loose hardcopy procedures that could not
always be easily located when required. Staff were aware of the central library
but seemed reluctant to use it.
There is an opportunity to structure the QMS around a top level flowchart with
hyperlinks to the appropriate procedures, all being available on line. The
source document could still be held centrally if required. Staff interviewed said
they were more likely to use this type of system than a paper version.
The recommended style for such a system is held on Café Vik/Quality &
Business Excellence/QMS at:
. http://www.cafevik.icl.co.uk/Howto/pmm/pmm6.htm
COMMERCIAL IN CONFIDENCE Page n1 of 15
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Ref: IA/REP/o25
Version: 1.0
Pathway Date: 17/04/00
S Annex A Termy of Reference
ICL PATHWAY : Internal Audit: Terms of Reference
AUDITTITLE : Audit of Customer Services management Processes
File Reference : AUD/3/4/18
Date : 14!" March 2000
Aim
1999 saw a major audit into Customer Services and its readiness for the increased workload and
responsibilities placed upon it by National Roll Out. A report (IA/REP/on) was issued in March
1999 and all Corrective Actions identified in the CAP (IA/CAP/oog) closed down during
Qu/2000.
The Internal Audit Plan for 2000 has identified a need to conduct an audit of the management
processes in place in Customer Services paying particular regard to escalation, reporting, root
cause analysis and service improvement. Pathway’s own ISO Certification programme also
requires that CS undergoes a more general audit of its processes. It was decided to combine the
two audits into a single piece of work.
The quality requirements expressed in ISOg000 : 1994 will be used as a basis for the work.
Objectives
1 To review the activities and operation of ICL Pathway Customer Services with particular regard
to their key management processes :
* Change Management
« Problem Management
Incident Management
© Operational Test
2. Within each of those processes to take a closer look at key management aspects :
¢ Escalation
© Reporting
« Root Cause Analysis
* Service Improvement
Dates
The audit will commence in March with completion and draft report production and circulation
targeted for mid to late April. A final report will be issued by Friday 21" April.
Audit Resources
The audit will be conducted by Jan Holmes, Pathway Audit Manager on behalf of the Internal
Audit Committee, ICL Pathway.
Reporting
The report reference is IA/REP/o25.
COMMERCIAL IN CONFIDENCE Page 12 of 15,
FUJ00080683
FUJ00080683
ICL Audit of Customer Service Ref: IA/REP/o25
Version: 1.0
Pathway Date: 17/04/00
At the conclusion of the audit a draft report will be produced and discussed with the auditees. A
final report will be produced and distributed to the Director and Senior Managers of Customer
Services, as well as the Managing and Deputy Managing Directors.
Further distribution will be at the discretion of the Customer Services Director.
Based on the report content a series of Corrective Actions will be agreed and documented in a
Corrective Action Plan. This will be issued and the agreed actions monitored on a regular basis.
TOR Distribution
Mike Stares : Managing Director
Mike Coombs : Deputy Managing Director
Martyn Bennett : Director of Quality and Risk Management
Stephen Muchow : Customer Services Director
Alec Nicholson : ISO Co-ordinator (for further distribution within CS)
COMMERCIAL IN CONFIDENCE Page 13 of 15,
FUJ00080683
FUJ00080683
ICL Audlit of Customer Service Ref TA/REP/025
Version: 1.0
Pathway Date: 17/04/00
6 Annex B Proposed Procesy Model
COMMERCIAL IN CONFIDENCE Page 14 of 15
ICL
Pathway
Audit of Customer Service
Ref: IA/REP/o25
Version: 1.0
Date: 17/04/00
Process Name: Pathuay Customer Service
Process Ouner: Steve Michou
-
ae i = =e C=) esa
Henzon Help ry Receive fornew Ss
Desk system ‘escalated call development!
——<Kiaes>> onan pian
Resolved? ion Spd progress: aabare
SN Record on Los ee
aan! cee en aay 1
Close oa! Fax] Avery Saanase inedert Camry oun treed
‘KEL a? Process Release analysis of closed
~ ee Update nich. Document calls
a mn a Ree 1
Yes—I wtanagement ee ac Resolved cance haxyspets
‘Centre - Ne line support e.g.
<> Poses? Esoher eto eee we
Produce reference]
data update
t
Passio
Reference data
manager
I rece
I rateren
[Mat for Charge
Request
wm
I Route to Release I
I Managerert for baw,
dsmbion I
is
(Cary cut delivered
‘quality ceoks on
harcore
Bustess Incidere Management
avoncled
mismatch
Rsceive Business
report™I incident Ca
FUJ00080683
FUJ00080683
COMMERCIAL IN CONFIDENCE
Page 15 of 15