Fe)
FUJITSU
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
Document Title:
Document Type:
Release:
Abstract:
Document Status:
Author & Dept:
Internal Distribution:
External Distribution:
Security Risk
Assessment Confirmed
Approval Authorities:
RMGA Customer Service Major Incident Process
Process (PRO)
Not Applicable
This document describes the Customer Service Major Incident
Management Process.
APPROVAL,
Mike Woolgar — RMGA CS Service Delivery Manager
Peter Thompson, Adam Parker, Mike Woolgar, lan Venables,
Wendy Warham, Andy Gibson, Mik Peach, Dave Jackson, John
Flannigan, Joep Niens, Rosemary Burgess, Barry Flemming, Mike
Stewart, Nick Crow, Denise Miller, Dave Wilcox, lan Mills, Kirsty
Gallacher, lan Cooley, Brian Pinder, Michael Jacklin, Sarah Hill,
Leighton Machin, Karen Harrod, Sandie Bothick, Susan Appleby-
Robbins, Adrienne Thompson
Dave Hulbert (POL), Mark Weaver (POL), Gary Blackburn (POL)
Alan Simpson( POL)
YES
Name Role Date
Wendy Warham Director Customer Services
Note: See Royal Mail Group Account HNG-X Reviewers/Approvers Role Matrix (PGM/DCM/ION/0001) for
guidance.
©Copyright Fujitsu Services Ltd 2009
COMMERCIAL IN CONFIDENCE Ref. SVM/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 1 of 39
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process
FUJITSU COMMERCIAL IN CONFIDENCE
©Copyright Fujitsu Services Ltd 2009 ‘COMMERCIAL IN CONFIDENCE Ref: SVMISDMIPRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 2 0f 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
Fe)
FUJITSU COMMERCIAL IN CONFIDENCE
0 Document Control
0.1 Table of Contents
0 DOCUMENT CONTROL.
0.1 Table of Contents.
0.2 Document History.
0.3 Review Details..
0.4 Acceptance by Document Re’
0.5 Associated Documents (Internal & External
0.6 Abbreviations.
0.7 Glossary...
BN
0.8 Changes Expecte
0.9
0.10
1 INTRODUCTION..
1.1. Process Owner.
1.2 Process Obje
1.3. Process Rationale.
2 MANDATORY GUIDELINES.....
3 DEFINITION OF A MAJOR INCIDENT.
3.1 Incident Classification...
3.2. Influencing Factors in calling Major Incident..
3.3. Major Incident Triggers.
3.3.1 Network Triggers...
3.3.2 Infrastructure Components Trigger:
3.3.3 Data Centre Triggers....
3.3.4 On-Line Services Triggers.
3.3.5 Security Triggers...
4 CALLING THE MAJOR INCIDENT......
5 PROCESS FLOW.....
5.1. Process Description (Any reference below made to T, = Time of incident occurring. Hence
T+3 = Time Incident Occurred plus 3 minutes). eae
6 CONFERENCE CALLS / WAR ROOM
6.1. Major Incident Conference Cal!
6.2 Technical Conference Call
6.3. War Room.....
7 FORMAL INCIDENT CLOSURE & MAJOR INCIDENT REVIEW....
7.1 Calculating Liquidated Damages payable on Major Incidents......
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMI/SDM/PRO/0001
Version: v3.0
Date: 07-July-2009
Page No: 3 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
FUJITSU COMMERCIAL IN CONFIDENCE
8 FUJITSU SERVICES ROLES AND RESPONSIBILITIES.
8.1 Horizon Service Desk...
8.1.1 Role of the HSD in a Major Incident.
8.1.2 HSD Templates...
8.2 Service Delivery Manager / Ownel
8.2.1 Role of the Service Delivery Manager / Owner in Major Incident.
8.2.2 Service Delivery Templates...
8.2.3 Service Delivery Activities.
8.3. Service Support Manager — Recovery Manager.
8.3.1 Role of the Service Support Manager in Major Incident.
8.3.2 Service Support Templates.
8.3.3 Service Support Activitie:
8.4 SDUs: Technical Teams / Third Parties.
8.4.1 Role of the SDUs: Technical Teams / Third Parties during a Major Inciden
8.4.2 Technical Teams Templates. sseeeeceeeeeeeeeeseseaeenenecaeneeeeeee
9 POST OFFICE / FUJITSU SERVICES INTERFACES
10 APPENDICES.
10.1 List of Templates...
10.2 Service Delivery Managers, Areas of Responsi
10.3 Escalation Communication Protocol
10.4 Major Business Continuity Incidents (
10.5 Security Major Incidents..
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMI/SDM/PRO/0001
Version: v3.0
Date: 07-July-2009
Page No: 4 of 39
Fe)
FUJITSU
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
0.2 Document History
Version No. Date
Summary of Changes and Reason for Issue
Associated Change -
CP/PEAK/PPRR
Reference
0.4 03-Oct-06
First draft — to detail the Major Incident Escalation process. Draft
taken from Horizon Document CS/PRD/122, V1.0.
1.0 11-Oct-06
Revision following comments from Reviewers
2.0 02-Sep-08
content changes.
Changes for Acceptance by Document Review: insertion of
Section (0.4) containing table of cross references for Acceptance
by Document Review and addition of note to front page. No other
24 24-Feb-2009
Changes made for Acceptance by Document Review by Fiona
‘Woolfenden including the removal of references to CS/PRD/O74
which has been Withdrawn and replaced by SVM/SD/PRO/0018
and other tidying up changes.
Other changes to update Contact details.
22 14-Apr-2009
Some Personnel Name changes and POA to RMGA +
Abbreviations. Security Updates to sections 5.1, 6.3, 8.2.1, 9.0,
23 3-June-2009
May 2009
Some Personnel Changes and minor changes following review in
3.0 T-July-2009
Following security audit 2 minor changes. Alan Simpson now on
distribution and 6.3, an extra bullet entry added at end of list
0.3 Review Details
Review Comments by 30" Apr 2009
Review Comments to
Mike Woolgar & PostOfficeAccountDocumentManagement
Mandatory Review
Role
Name
Director Customer Services
Wendy Warham
Service Support Manager
Role
Optional Review
Kirsty Gallacher
Name
FUJITSU (UK & CS Business
Continuity Manager
IRELAND)
Adam Parker
FUJITSU (UK & IRELAND) CS Service Delivery
Manager Engineering
Leighton Machin
Centre Manager
FUJITSU (UK & IRELAND) CS Service Delivery _I Mike Stewart
Manager BankOnLine
FUJITSU (UK & IRELAND) CS System Support I Mik Peach
FUJITSU (UK & IRELAND) HSD Operations
Manager
Michael Jacklin
FUJITSU (UK & IRELAND) SMC Manager
Karen Harrod
©Copyright Fujitsu Services Ltd 2009
‘COMMERCIAL IN CONFIDENCE
Ref SVM/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 5 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
Fe)
FUJITSU COMMERCIAL IN CONFIDENCE
FUJITSU (UK & IRELAND) Acceptance Manager I David Cooke
FUJITSU ( UK & Ireland) RMGA Quality Manager I Nigel Hatcher
Issued for Information Please restrict this
distribution list to a minimum.
Position/Role Name
FUJITSU (UK & IRELAND) CS Service Delivery I lan Mills
Manager: Networks
FUJITSU (UK & IRELAND) CS Security Manager Peter Sewell
Unix Team Leader Andy Gibson
NT Team Leader Adrienne Thompson
Network Manager Dave Jackson
Operations Manager John Flanagan
Acceptance Manager David Cooke
(* ) = Reviewers that returned comments
0.4 Acceptance by Document Review
The sections in this document that have been identified to POL as comprising evidence to support
Acceptance by Document review (DR) are listed below for the relevant Requirements:
r NFR DR _ Internal FS POL Document
Acceptance Ref NFR Reference Section Number
Document Section Heading
SER-2200 SER-2178 Whole Document
SER-2202 SER-2179 Whole Document
SEC-3095 SEC-3266 3.3.5 Security Triggers
SEC-3095 SEC-3266 10.5 Security Major Incidents
0.5 Associated Documents (Internal & External)
eference Version Date Ti ource:
PGM/DCM/TEM/0001 Fujitsu Services Royal Mail Group Account I Dimensions
(D0 NOT REMOVE) HNG-X Document Template
CS/IFS/008 RMGA/POL Interface Agreement for the PVCS
Problem Management Interface
CS/PRD/021 RMGA Problem Management Process PvCS
CS/PRO/110 RMGA Problem Management Database PVCS
Procedures
PA/PRO/001 Change Control Process PVCS
CS/QMS/001 Customer Service Policy Manual PVCS
©Copyright Fujitsu Services Ltd 2009 ‘COMMERCIAL IN CONFIDENCE Ref. SVMISDM/PROI0001
Version V3.0
Date: 07-July-2009
Page No: 6 of 39
fee)
FUJITSU
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
FUJ00080071
FUJ00080071
SVM/SDM/SD/0001 Horizon Service Desk — Service Description Dimensions
SVM/SDPRO/0018 RMGA Customer Service Incident Dimensions
Management Process
CS/FSP/002 Horizon System Helpdesk Call Enquiry PVCS
Matrix and Incident Prioritisation
TBA SMS Messaging User Guide PvcS
CS/PRD/121 SMS Major Communication Framework PvcsS
Process
CS/PLA/079 Horizon Services Business Continuity Plan PvCS
CS/PLA/080 Horizon Support Services Business PVCS
Continuity Plan
CS/PLA/O15 Horizon Systems Helpdesk and Business PVCS
Continuity Plan
SVM/SDM/PLA/0001 HNG-X Support Services Business Dimensions
Continuity Plan
SVM/SDM/PLA/0002 HNG-X Services Business Continuity Plan Dimensions
SVM/SDM/PLA/0030 HNG-X Engineering Service Business Dimensions
Continuity Plan
SVM/SDM/PLA/0031 HNG-X Security Business Continuity Plan Dimensions
SVM/SDM/SD/0011 Branch Network Services Service Dimensions
Description
Unless a specific version is referred to above, reference should be made to the current approved
versions of the documents.
0.6 Abbreviations
Abbreviation Definition
AtG Advice & Guidance
BCP Business Continuity Plan
HSD Horizon Service Desk
Iso International Standards Organisation
ITIL Information Technology Infrastructure Library
KEDB Known Error Database
KEL Known Error Log
MBCI Major Business Continuity Incident
MSU Management Support Unit
ocP Operational Change Proposal
PO Post Office
RFC Request For Change
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 7 of 39
Fe)
FUJITSU
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
RMGA Royal Mail Group Account
POL Post Office Limited
SCT Service Continuity Team
SDM(s) Service Delivery Manager(s)
(NB: Throughout this document SDM refers to a person responsible for the Service,
and the SDM could work in, but not limited to, the Service Delivery, Service
Support, Release Management or Security teams.
SDU Service Delivery Unit
SLT Service Level Targets
SMC Systems Management Centre
SRRC Service Resilience & Recovery Catalogue
SSC System Support Centre
VIP. VIP Post Office, High Profile Outlet
0.7 Glossary
T
Term Definition
Time of incident occurring
T+3,
Time Incident Occurred plus 3 minutes
0.8 Changes Expected
ae
0.9 Accuracy
Fujitsu Services endeavours to ensure that the information contained in this document is correct but, whilst every
effort is made to ensure the accuracy of such information, it accepts no liability for any loss (however caused)
sustained as a result of any error or omission in the same.
©Copyright Fujitsu Services Ltd 2009
‘COMMERCIAL IN CONFIDENCE Ref: SVMI/SDM/PRO/0001
Version: v3.0
Date: 07-July-2009
Page No: 8 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
fee)
FUJITSU COMMERCIAL IN CONFIDENCE
0.10 Security Risk Assessment
Security risks have been assessed and it is considered that there are no security risks relating specifically to this
document.
1 Introduction
1.1 Process Owner
The owner of this process is the RMGA Service Support Team Manager.
1.2 Process Objective
The key objective of the process is to ensure effective and efficient management of Major Incidents,
through:
e Improvements of communication channels.
« Clarify the need to communicate awareness of potential incidents
e Improved accuracy of reporting against status of incident
e Allowing technical teams the right amount of time to diagnose and impact an incident
e Avoid unnecessary alerting of the customer
« Demonstrate to the Post Office a more professional approach
e Provision of clear defined roles and responsibilities
e Defined reporting/update timelines through duration of a major incident.
e Improved governance
e Assessing which incidents are major and which are ‘Business as Usual’
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMI/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 9 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
fee)
FUJITSU COMMERCIAL IN CONFIDENCE
1.3 Process Rationale
This document outlines the communication and management process and guidelines to be followed in
relation to Major Incidents impacting the live estate.
The methodology defined within this document augments the existing SMS framework process presently
deployed within the live estate.
The aim of the document is to provide a pre-defined process on which future major incident
communication and management will follow and that any parties involved in that process provide
updates /receive updates at defined intervals from inception to closure of any major service impacts.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMI/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 10 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
Fe)
FUJITSU COMMERCIAL IN CONFIDENCE
2
Mandatory Guidelines
Whilst it is important to maintain a balance between:
a) Allowing the technical teams the right amount of time to diagnose and impact an incident
b) Avoid unnecessary alerting of the customer
c) Assessing which incidents are major and which are ‘Business as Usual’
The following guidelines should be adhered to.
During HSD Core Hours (Monday — Friday 08:00 - 18:00 and Saturday 08:00 — 14:00) Post
Office Horizon Service Desk should be the first point of contact for operational contact between
Fujitsu and the end user. Outside these hours SMC acts as the first point of contact.
Any activity detailed in this document which is assigned to the HSD is handed over to SMC
outside the HSD Core Hours.
The relevant technical teams who are monitoring and aware of a potential major incident must
page/call the appropriate Fujitsu Service Delivery Manager / Owner (Duty Manager out of hours)
as soon as possible, rather than wait. The Fujitsu Service Delivery Managers are detailed in
Appendix 10.2. This is not limited to major incidents alone, but must be delivered wherever a
state other than Business as Usual has been detected. The Fujitsu Service Delivery Manager /
Owner must in turn communicate the potential incident, to POL SCT for awareness and
monitoring in POL. However this could be done via the HSD IMT.
The Fujitsu Service Delivery Manager / Owner (or Duty Manager out of hours) is responsible for
communicating both up the Fujitsu Organisation and across (see appendix 10.3) to their
counterpart in POL. Where this is impractical (i.e. leave, out of hours, unavailable), the initiative
should be taken to jump up the organisation. The important fact is that the customer is informed
in a timely manner and at the correct touch point. This communication should be by voice or
direct SMS. The communication should include the date, time, name, nature of problem,
severity, if service affecting, likely impact, and the owner for contact.
The Fujitsu Service Delivery Manager / Owner (Duty Manager) should also initiate
communication using SMS via HSD, 08.00 to 18.00 Monday to Friday, and Saturday 08.00 to
14.00. Outside of these hours, SMS should be sent via SMC.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMI/SDM/PRO/0001
Version: v3.0
Date: 07-July-2009
Page No: 11 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
Fe)
FUJITSU COMMERCIAL IN CONFIDENCE
3 Definition of a Major Incident
3.1 Incident Classification
As a general rule a Major Incident will always be an incident rated as severity level A (critical) in the
RMGA Customer Service Incident Management Process Details document (SVM/SDM/PRO/0018)
version 2.0, or a series of connected lower severity rated Incidents which combine to have a significant
business impact. However not all incidents rated at severity level A qualify. This is because the severity
levels do not necessarily translate to the global business impact on POL’s business. For example a
single counter post office which is unable to transact, regardless of its business volumes is rated as a
Severity A.
For simplicity, Incidents are classified into three impact levels: High, Medium and Low.
High — An Incident that has occurred with a significant and potentially prolonged adverse impact on POL
business. Typically these Incidents will initially require a significant amount of reactive management
before they can be controlled and resolved.
Medium — An Incident that has the potential to cause significant impact to POL business but can be
controlled and mitigated against through effective management.
Low — An Incident that requires business attention but if managed effectively will not have significant
impact on POL business.
3.2 Influencing Factors in calling Major Incident
It is important that a major incident is defined as such, because of its business impact on the day when it
occurs, rather than simply being defined as a major incident because it appears on a list. The following
parameters will also feed into the consideration of whether a major incident exists, as follows:
e Duration i.e. how long has the vulnerability to service already existed
e Impact across the estate, including consideration of whether a service is merely degraded or
actually stopped
e Time at which the event occurs in relation to the 24 hour business day
e Time of year - eg Christmas / Easter / End of month/quarter DVLA
e Anticipated time before service can be resumed
e Impact to POL Branches, customers, clients or brand image
e Business initiatives e.g. product launches
3.3 Major Incident Triggers
The criteria below could trigger a major incident, however as detailed in 3.2, the influencing factors must
be considered. As such the list below can never be exhaustive, whilst if an incident occurs which is not
detailed, this should not be precluded from being a major incident.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMI/SDM/PRO/0001
Version: v3.0
Date: 07-July-2009
Page No: 12 0f 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
fee)
FUJITSU COMMERCIAL IN CONFIDENCE
3.3.1. Network Triggers
Network Major Incident triggers are as follows:
« Complete or significant outage of Central network
« Complete or significant outage of BT network
« Complete or significant outage of VSAT sites
3.3.2 Infrastructure Components Triggers
Infrastructure component Major Incident Triggers are as follows:
e Total loss of environments providing individual on-line service capability
« Breach of access to data centres
e Breach of security
« Virus outbreak
3.3.3. Data Centre Triggers
Data Centre Major Incident triggers are as follows:
e Network/LAN outage
« Loss of data centre
e Breach of security
3.3.4 On-Line Services Triggers
On-Line services Major Incident Triggers are as follows:
e On-line service unavailable within Data centre (not counter level)
e Number of Branches not able to provide on-line services — as defined by POL
* 3% party provided service failure — Link, Fujitsu Group
3.3.5 Security Triggers
Security major incident triggers are as follows:
e Actual or suspected attacks on the Fujitsu Services RMGA Network or Information System.
e Theft of IT equipment / property, including software
In the event of a Security Major Incident (which may also include PCI Incidents), the security incident
management process as detailed in
SVM/SDM/PRO/0018 Appendix A must also be followed by the SDM owning the Service.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMI/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 13 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
fee)
FUJITSU COMMERCIAL IN CONFIDENCE
SVM/SDM/PLA/0031 HNG-X Security Business Continuity Plan defines the actions to be taken if
security violations are identified. This must also be followed by the SDM owning the service.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVMI/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 14 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
fee)
FUJITSU COMMERCIAL IN CONFIDENCE
4 Calling the Major Incident
During business hours the Service Delivery Manager / Owner responsible for the Primary Service
affected declares Major Incident and is designated the Incident Manager for the duration of the Incident
(with handovers to the Duty Manager where applicable.)
Where the impact of the incident is not immediately obvious, and it is not clear if Major Incident should
be called, escalation and discussion with the Service Delivery / Service Support Team Manager, Head of
Service Management or CS Director should occur, and a collective decision made. If Major Incident is
not called, the incident should be monitored until closure, to ensure that the impact does not increase to
Major Incident.
In the event that multiple Services are impacted, an Incident Manager will be appointed by Service
Delivery / Service Support Team Manager, Head of Service Management or CS Director, who will
remain in this role until incident closure.
Out of hours the Duty Manager is responsible for declaring Major Incident.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMI/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 15 of 39
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
o
FUJITSU COMMERCIAL IN CONFIDENCE
5 Process Flow
©Copyright Fujitsu Services Ltd 2009. COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 07-July-2009
Page No: 16 of 39.
Fe)
FUJITSU
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
FUJ00080071
FUJ00080071
5.1 Process Description (Any reference below made to T, = Time of incident occurring. Hence T+3 = Time Incident Occurred plus 3 minutes).
I Box Title
“I Description ~
Major incident identi ied?
Incident identified, the definition of an Incident is “Any event which is is not part of the standard ‘operat n of a service and
which causes, or may cause, an interruption to, or a reduction in, the quality of that service.” (SVM/SDM/PRO/0018). An
Incident may be reported from within POL domain, a supplier domain or other route
Major Incident Triggers Met?
An initial impact assessment of the incident is undertaken by
members of the RMGA Service Team taking into account
impact on:
Live Service, Financial Integrity, Business Image
The incident is profiled as a Major Incident as outlined within
this document, including consideration of all influencing
factors, time, geographical coverage, business impact,
security, public perception, duration and relevant business
initiatives coinciding at POL and decision taken to call Major
Incident
POL SCT will be informed by the Service Delivery Manager /
Owner managing the Major Incident of the incident & the
incident will also be escalated to Service Delivery / Service
Support team managers, if this has not already occurred.
With agreement from RMGA Service Delivery Manager, or
Duty Manager out of hours, SMS will be sent to RMGA and
POL Management alerting to the potential existence of a Major
Incident.
T+3
All timescales
quoted within
this document
as viewed as
maximum, to be
improved upon
wherever
possible
T+5
RMGA Service Delivery Manager /
Owner(Duty Manager), RMGA Service
Delivery / Support Manager, Head of
Service Mgmt, CS Director
Service Delivery Manager / Service
Owner
HSD (in hours)
SMC (out of hours)
BAU Incident Process
If Major Incident is not declared then the BAU Incident process is followed — POL SCT will be informed that there is no MI &
a closure SMS sent. The SDM for the service should ensure that the Incident is re-impacted during its lifecycle to ensure
that_the impact_has not increased. If, subsequently the incident_is declared Major Incident, move to box “Incident
©Copyright Fujitsu Services Ltd 2009
COMMERCIAL IN CONFIDENCE
Ref: ‘SVM/SDM/PRO/0001
Version: v3.0
Date: 07-July-09
Page No: 17 0f 38
Fe)
FUJITSU
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
FUJ00080071
FUJ00080071
Investigation and Diagnosis started”.
Incident Investigation & Diagnosis. Relevant internal SDUs / Third Parties contacted to initiate I T+ 5 Service Support Manager
investigation & diagnosis.
Major Incident Report opened Service Delivery Manager
Major Incident Conference call scheduled & agenda
distributed.
Major Incident Bridge 1 The Major Incident Bridge is chaired by the Service Delivery T+15 Service Delivery / Service Support
Manager / Owner who has been nominated as the Major
Incident Manager. The Major Incident Bridge is SERVICE
FOCUSSED.
The Major Incident Bridge aims :
e To discuss & agree the recovery investigation & resolution
of Major Incidents
e To provide a forum for up-to-date progress reports
« To aid communication, and the creation of Technical
Bridge Minutes which are distributed to all involved parties
and RMGA & POL Management. This ensures that Major
Incident progress is known by all.
* To collate information for inclusion on the Service Portal.
Attendees at the Major Incident Bridge include, but are not
limited to, RMGA Service Management, SDU, Third Parties,
POL., CS Security & POL Security Managers
The Major Incident Bridge follows a set agenda which covers:
Manager
©Copyright Fujitsu Services Ltd 2009
COMMERCIAL IN CONFIDENCE
Ref: SVM/SDM/PRO/0001
Version: v3.0
Date: 07-July-09
Page No: 18 of 38
Fe)
FUJITSU
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
FUJ00080071
FUJ00080071
Roll call, Summary of Incident, Incident Overview,
Current Impact, Current Investigation / Recovery Action,
Remedial Actions, Actions to carry forward to Major
Incident Review
The agenda template is stored on
\\ATCFS7\POcust_serv\01Public\Major Incident Management
\Templates
Following the Major Incident Bridge, a further SMS will be sent,
providing an update on the Incident
If the outcome of the Technical Conference Call is that the
Incident is determined Business As Usual (low) then an SMS
communication will be sent stating that the Incident is not a
Major Incident.
From this point forward, SMS communication, timing and
delivery requests, becomes the responsibility of the RMGA
Service Delivery Manager / Owner acting as Major Incident
Manager. 30-minute updates should be the norm
Service Delivery Manager / Owner will also distribute minutes
following the conference call
MIB + 15
Clear recovery path identified
If during the conference call a clear recovery path is identified, this should be discussed and agreed on the call. Following
agreement the recovery should be implemented.
If there is no clear recovery path, further investigation will be undertaken.
Recovery / Investigation The RMGA Service Support Team Manager, acting as I T+x RMGA_ Service Support Team
recovery manager will liaise with the SDUs and /or third parties Manager
during the investigation / recovery.
Where appropriate a Technical Bridge will be called for a
technical discussion of the Major Incident.
©Copyright Fujitsu Services Ltd 2009 ‘COMMERCIAL IN CONFIDENCE Ref. SVM/SDM/PRO/0001
Version: v3.0
Date: 07-July-09
PageNo: 19 of 38.
Fe)
FUJITSU
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
FUJ00080071
FUJ00080071
Major Incident Bridge 1+ At the time agreed at the first Major Incident Bridge, the I T+x RMGA Service Delivery Manager
subsequent Major Incident Bridges are held. The same
agenda is followed, and progress on actions / recovery is
provided.
If no clear recovery path is identified, the decision is then
taken on whether to escalate for War Room direction
Escalate for War Room direction The nature of the incident determines which RMGA Service I Timescale RMGA_ Service Support Team
Team members and POL Managers are involved in the War I dependant on I Manager
Room but it would include all or some of the following: impact and POL Service Manager
* POL nature of
incident.
e RMGA Service Support Manager (War Room Chairman)
« RMGACS Director
e 3rd party Executives
e Appointed working group representatives as appropriate
The purpose of the War Room is to:
« Provide appropriate direction on Incident resolution
e Provide added impetus to restoration of service ASAP
«Involve 3" party Executives
e Define communication intervals to Key Stakeholders
e Provide focused Incident Management in line with the
impact and severity of the Incident.
‘©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: ‘SVM/SDM/PRO/0001
Version: v3.0
Date: 07-July-09
Page No: 20 of 38
Fe)
FUJITSU
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
FUJ00080071
FUJ00080071
War Room drive working group actions War Room provide the appropriate direction on the incident I Timescale RMGA_ Service Support Team
resolution priorities. dependant on I Manager.
impact and
nature cof I War Room.
incident.
Incident Resolution
Once the incident is deemed to be resolved, final Major
Incident conference call is held to agree & confirm resolution
of incident. Major Incident Review date to be set at the final
Major Incident Conference call.
SMS communication sent confirming resolution of incident.
Draft Major Incident Report distributed within 24hrs of
resolution of Major Incident.
RMGA_ Service Support Team
Manager
Major Incident Review & formal Incident
Closure
Formal Closure of the Major Incident & a review of the Incident
including consideration of:
« Lessons learnt
e Incident definition
« What went well
* Timeline
« Changes required to infrastructure
e =A review of the Major Incident Communication Process
e Root Cause Analysis * if known at this point
e Business impact
e Action plan
e Service Improvement Plan update
RMGA Service Delivery Manager
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: v3.0
Date: 07-July-09
Page No: 21 of 38
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
Fe)
FUJITSU COMMERCIAL IN CONFIDENCE
6 Conference Calls / War Room
6.1 Major Incident Conference Call
This is a Service Focussed call for Service Management, Technical experts and POL to discuss the
service impact of the Major Incident and to receive updates on progress of the Incident.
Invitations to the Major Incident Conference Call will be via SMS, email or voice. The dial in details will
be provided at the same time as the meeting invitation.
The Major Incident Conference Call will be incepted at T + 15, and at regular intervals during the Major
Incident, the exact scheduling will be discussed and agreed at each preceding Major Incident Call.
Each Major Incident Conference Call follows a set agenda which will be distributed with the meeting
invitation where possible. The conference call is chaired by the Service Delivery Manager / Owner
acting as Major Incident Manager.
Following each Major Incident Conference Call, a set of minutes will be published, which will
subsequently form part of the Major Incident Report.
6.2 Technical Conference Call
This is a technical conference for experts to discuss and analyse the incident enabling an appropriate
action plan to be formulated to restore the service to POL without delay. The Technical Conference Call
will baseline the anticipated response, covering resolution, time and resources required.
The Technical Conference Call will be incepted as required by the Service Support Manager.
6.3 War Room
The purpose of the War Room is to provide a focused area from which strategic decisions can be made
regarding a Major Incident confirmed a MBCI.
Attendance will be mandatory from the following or their designated representative:
« RMGA Customer Services Director
« RMGA Service Management Team Manager
e RMGA Business Continuity Manager
« RMGA Service Delivery Manager / Owner (Business line specific)
« POL Head of Technical Services
e POL Service Continuity Manager
e POL Service Delivery Manager
« POL Business Continuity Manager
* 3 Party Account/Service Delivery Manager
e RMGA Security Manager (If MI is a PCI or Major Security Incident)
e POL Security Incident Manager ( If MI is a PCI or Major Security incident)
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 22 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process &
FUJITSU COMMERCIAL IN CONFIDENCE
Actions within the War Room include:
e Agreement of Containment Plan
« Documentation of all agreed actions with owners, and timescales
e Consistent management of the major incident across all involved locations
« Management of potential / MBCIs within POL & RMGA
e Co-ordinate meeting times and locations
In the event of a major incident requiring a War Room to be incepted, it is envisaged that this will be in
place at T+60. Participants required in the War Room will be contacted via SMS as appropriate.
Telephone number for the War Room working group it
prior to the attendance of other callers and enter a designated PI
callers.
_j, the Chairman will enter the call
lowing direct entry for subsequent
The Major Incident Review is chaired by the Major Incident Manager and follows a set agenda, which
should be distributed with the Major Incident Review meeting invite, along with the draft copy of the
Major Incident Report.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 23 of 38
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
Fe)
FUJITSU COMMERCIAL IN CONFIDENCE
7 Formal Incident Closure & Major Incident Review
The purpose of a Major Incident Review is:
1. To understand the Incident that prevented a Service or Services from being delivered.
2. To confirm impact to the business, during and after the Incident and agree number of branches
impacted and duration of Major Incident.
3. To confirm the end-to-end recovery process & timeline and identify that all documented
processes were followed.
4. To analyse the management of the Incident & the effectiveness of the governance process.
5. To identify corrective actions to:
i. prevent recurrence of Incident
ii. minimise future business impact
iii, improve management of Incidents
6. To formally close the Major Incident
Output: To confirm details provided in the draft MIR provided to POL, update with corrective actions and
redistribute. The agreed impact of Major Incident must be sent to Branch Network Service Delivery
Manager for inclusion in the Counter Availability SLT Figures.
If this review highlights areas where improvements can be made, an agreed Service Improvement Plan
will be produced with appropriate actions, owners and timescales. Service Management will track all
actions to resolution. Third party actions will be reviewed at Service Review meetings.
The Agenda for the Major Incident Review is stored on: \\ATCFS7\POcust_serv\01Public\Major Incident
Management \Templates
It is critical that the number of branches impacted and the duration of the Major Incident is agreed at the
Maior Incident Review and this agreement must come from POL. If for some reason POL is not present
at the Major Incident Review meeting, a separate conversation must take place. This information is
fequired because Major Incidents affect Counter Availability and Liquidated Damages (LDs) are paid on
Counter Availability as detailed in 7.1 below.
7.1 Calculating Liquidated Damages payable on Major
Incidents.
Liquidated damages are payable on Major Incidents which qualify as Failure Events as detailed in the
Branch Network Service Description (SVM/SDM/SD0011). Failure Event is defined in this document as
an event or series of connected events which causes one or more Counter Positions to be deemed to be
Unavailable due to a Network Wide Failure or a Local Failure. Ongoing failures will be deemed to be part
of such Failure Event until the Failure Event is closed in accordance with the Incident closure and Major
Incident Review process as detailed in section 7.0.
For a Failure Event the Incident Closure & Major Incident Review Process will require Post Office and
Fujitsu Services to agree the number of Branches and Counter Positions affected and the duration of the
outage;
The duration of the Incident must be agreed with Post Office & rounded to the nearest 30 minutes as
detailed in the Network Wide Rounding Table below
Network Wide Rounding Table
Duration of Incident Deemed duration for the
purposes of LD
calculations
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVMISDM/PRO/O001
Version; V3.0
Date: 7-July-2009
Page No: 24 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process &
FUJITSU COMMERCIAL IN CONFIDENCE
30 minutes or less 30 minutes
More than 30 minutes I 1 hour
but less than 1 hour
1 hour or more but less I 1 hour
than 1 hour 30 minutes
1 hour 30 minutes or I 2 hours
more but less than 2
hours
N hours or more but I N hours
less than N hours 30
minutes
N hours 30 minutes or I (N+1) hours
more but less than
(N+1) hours
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 25 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process &
FUJITSU COMMERCIAL IN CONFIDENCE
8 Fujitsu Services Roles and Responsibilities
This section defines the roles and responsibilities individuals and teams have with regard to the Major
Incident Escalation Process.
8.1 Horizon Service Desk
8.1.1. Role of the HSD in a Major Incident
The role of the Horizon Service Desk (HSD) in the event of a Major Incident is two-fold
* Receive & log calls from the Post Masters, and communicate the progress of investigations to any PMs
who call into the desk.
* The HSD should also complete the “HSD Impact” template and forward to the RMGA CS Service
Delivery Manager, who is managing the Incident. This template should be completed every 15mins from
the point of declaring Major Incident, until the RMGA SDM asks for this to cease. The Service Delivery
Manager / Owner will provide the Summary of Incident to be included in the template.
8.1.2 HSD Templates
NAME ‘OF I LOCATION DESCRIPTION / NOTES: ; 7] DISTRIBUTION
TEMPLATE a 5
Impact on HSD YATCFS7\POcust_sen I This template is used to communicate the I RMGA Service Delivery
01Public\Major Incident I impact of a Major Incident to RMGA Service I Manager
Management \Templates I Management (Service Delivery.) The
template can be copied to email
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 26 of 38
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
Fe)
FUJITSU COMMERCIAL IN CONFIDENCE
8.2 Service Delivery Manager / Owner
Throughout this document the term Service Delivery Manager refers to the RMGA owner of the Service
impacted, and is not limited to a manager working as part of the Service Delivery Team.
8.2.1 Role of the Service Delivery Manager / Owner in Major Incident
The primary role of the Service Delivery Manager / Owner in a Major Incident is to facilitate the
management of the Incident, through investigation and diagnosis to resolution, with the aim of making
the process as efficient & effective as possible. The Incident Manager acts as the central point for
communication and non-technical information flow, allowing the Recovery Manager to focus on the
technical & the resolution of the Incident. The Service Delivery Manager / Owner is also responsible for
creating and maintaining all the associated documentation.
The Service Delivery Manager:
* Has responsibility for creating the Major Incident documentation: Major Incident Report
* Manages the communication, internally within Royal Mail Group Account (RMGA).
* Communicates progress of Incident with POL SCT
* Identifies Business & Service impact, though discussions with the users, POL SCT & HSD — providing
this input into the Major Incident Conference Call.
* Calls & chairs the MI conference call
* Produces & distributes the MI conference call minutes.
* Liaises with SSC to update the Service Portal / updates the Service Portal
Following resolution of the Incident the Service Delivery Manager / Owner schedules and chairs the
Major Incident Review and creates the Major Incident Report Document. Any corrective actions arising
from the Major Incident Review will be added to the Corrective Actions log and tracked through to
completion. The updates will be distributed to POL as required, and in the case of a Security Major
Incident associated with PCI Failures, then the POL Security team will receive a copy of the report.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 27 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process _
FUJITSU COMMERCIAL IN CONFIDENCE
8.2.2 Service Delivery Templates
NAME OF I LOCATION DESCRIPTION /NOTES DISTRIBUTION
TEMPLATE
Notification of I \ATCFS7\POcust_serv\_ I Used to communicate the key information I POL
Major incident 01Public\Major incident I as requested by POL.
, Management Templates 3 RMGA Service Support &
THIS MUST BE EMAILED WITHIN 15 _ I Service Delivery
MINS OF MAJOR INCIDENT
INVOCATION. RMGA Head of Service
Management
RMGA Head of CS
Major Incident I WATCFS7\POcust_ser_ I The Major Incident Report contains all the I POL
Report Template I 01Public\Major Incident I information about a Major Incident. This
Management \Templates I document is distributed to POL. RMGA Service Support &
Service Delivery
RMGA Head of Service
Management
RMGA Head of CS
Mi Conference Call I WATCFS7\POcust_ser I Used to provide structure to the Ml I Attendees of Ml
Agenda 01Public\Major incident I Conference Call and ensures that all I CONFCALL
Management \Templates I relevant information is collected,
RMGA Service Support &
Service Delivery
RMGA Head of Service
Management
RMGA Head of CS
Mi Conference Call I WATCFS7\POcust_seri I Ensures that minutes from the Mil I Attendees of MI CONCALL
Minutes O1Public\Major Incident I Conference Call are produced in a
Management \Templates I consistent format, and that all attendees are I RMGA Service Support &
aware of actions & status of recovery. Service Delivery
RMGA Head of Service
Management
Aid icati
ids communication RMGA Head of CS
Major Incident I WATCFS7\POcust_serv\_I Standard format for Major Incident Review I Attendees of MIR
Review Agenda O1Public\Major Incident I & ensures that all areas are addressed.
Management \Templates RMGA Service Support &
Service Delivery
RMGA Head of Service
Management
RMGA Head of CS
Major Incident I WATCFS7\POcust_servi_I Standard format for Major Incident Review I Attendees of MIR
Review Minutes I 01Public\Major Incident I minutes. Ensures that information is ,
Management \Templates I collected in correct format for inclusion in I RMGA Service Support &
other reports / to be taken to Service I Service Delivery
Review. RMGA Head of Service
Management
RMGA Head of CS
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref SVM/SDM/PRO/0001
Version:
Date: july-2009
Page No: 28 of 38
FUJ00080071
FUJ00080071
oO POA Customer Service Major Incident Process “
FUJITSU COMMERCIAL IN CONFIDENCE
8.2.3. Service Delivery Activities
This template can be printed out and used as a checklist during a Major Incident.
Incident Management Timeline I Service Delivery Manager Activities
Incident + 0 I Incidentidentified I C Invoke Major Incident
mins
Documentation
Open Major incident Report (Powerhelp number, start timeline)
Save Major Incident Report on central server.
Open Service Portal Incident
C_Prepare Notification of Major Incident template
Communication
(Ensure HSD / SMC is aware of Incident & request HSD Impact template to be
completed. Provide Incident Summary details for HSD / SMC to add to the
template
© Inform POL
O Escalate to RMGA management
© Send alert SMS
Incident + 15 Determine Business impact (impact on transactions, & branches.)
mins © Calls Mi Conference Call
Documentation
O_HSD Impact template received.
C2 Information added to POL Major incident invocation template.
Email completed POL Major incident invocation template to POL
Update Major incident Report
1 Update Service Portal
Communication
O_Send update SMS (including time of MI Conference Call - if applicable)
Incident + x I MI Conference I 1 Chairs MI Conference Call
mins Call
Information Required
1 HSD impact including Business impact (impact on branches / counters &
transactions)
Incident + x I Post Mi I Gl Creates & distributes the MI Conference Call Minutes
mins Conference Call_I (1 Save copy of Mi Conference Call Minutes on central server
MI Documentation
CONCALL + Update Major incident Report (timeline & impact)
15 mins C1 Update Service Portal
Communication
O_Send update SMS
To be repeated following every MI Conference Call
Incident + x I Post recovery Documentation
mins (Update Major Incident Report & save on server.
Email draft Major Incident report to POL
Update Service Portal Incident
Within 1 Schedule follow up Mi Conference Call to ensure that recovery has been effective
24hrs) & that Incident is resolved.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMISDM/PRO/O001
Version:
Date: july-2009
Page No: 29 of 38
fee)
FUJITSU
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process _
COMMERCIAL IN CONFIDENCE
Communication
Ensure HSD is aware of Incident resolved / downgraded to BAU.
Cease completion of HSD Impact template.
Inform POL / RMGA Management of Incident Status.
Incident
24hrs
+
‘Schedule MI Review and distribute agenda
Chair MI Review
a
a
a
Oi Send closure SMS
a
a
Oi Track Corrective Actions
Documentation
Update Major Incident Report & save on server.
Email Major incident report to POL
Incident
48hrs
>
Major Incident
Review
o
Chair Major Incident Review
(Agree timescales and number of branches impacted as required for HNG-X LD
payments
Incident
48hrs
>
Post Major
Incident Review
O Create Major Incident Review Minutes and save to central server.
Distribute Major Incident Review Minutes.
O_Add corrective actions to the “corrective actions spreadsheet”
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version:
Date: july-2009
Page No: 30 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process &
FUJITSU COMMERCIAL IN CONFIDENCE
8.3 Service Support Manager — Recovery Manager
8.3.1 Role of the Service Support Manager in Major Incident
The primary functions of the Service Support Manager is to act as Recovery Manager and to co-ordinate
and manage the restoration of Service, focusing on the technical teams and acting as the
communication point for the technical teams and 3 parties.
The Service Support Manager:
* Manages the technical recovery of the Incident — liaising with SDUs & Third Parties.
* Provides updates on the recovery, when technicians / representatives of technical teams are unable to
attend the MI Conference Call.
* Is the only person to liaise directly with the technical teams, including technical third parties.
* Ensures that technical investigative actions are tracked and completed.
* Tracks technical corrective actions through Service Review meetings
In the event of a central infrastructure Major Incident with no impact on the business, the SDM will not be
involved and the SSM will be responsible for chairing any meetings / creation of documentation.
Service Support may well feel the need to schedule a Technical Conference Call, which will have a
technical focus.
8.3.2 Service Support Templates
NAME OF I LOCATION DESCRIPTION / NOTES DISTRIBUTION
TEMPLATE
Technical Update I \ATCFS7\POcust_ser I Contains technical information which should I None
for Ml Conference I 01Public\Major Incident I be brought to the MI Conference Calls
Call Management \Templates
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref SVMISDM/PRO/O001
Version: V3.0
Date: 7-July-2009
Page No: 31 of 38
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process _
fee)
FUJITSU COMMERCIAL IN CONFIDENCE
8.3.3. Service Support Activities
This template can be printed out and used as a checklist during a Major Incident.
In the event that there is no business impact, the Service Support Manager will assume the
documentation responsibilities.
incident Management I Service Delivery Manager Activities
Timeline me :
Incident + I Incident identified I C) Invoke Major Incident
O mins Ensure that Data Centre Ops are running ping scripts (every 15 mins from each DC)
© Contact Technical Teams for update
_ Agree next update point with Technical Team (if not Ml Conference Call)
Incident +x I MI Conference I C Provide technical input into the MI Conference Call if technical teams are unavailable.
mins Call
incident +x I Post Mi I © Complete actions assigned to Service Support
mins Conference Call I C1 Ensure that SDUs complete actions.
MI
CONCALL
+15 mins
To be repeated for every Ml Conference Call
Incident + x I Post recovery O Ensure that all Technical teams are aware that the Incident has been resolved /
mins downgraded. Thank teams for their effort.
Provide input into update of Major Incident Report
(Within
24hrs)
Incident + I Major Incident I C1 Attend Major Incident Review
48hrs Review
incident + I Post Major I C1 Add corrective actions to the “corrective actions spreadsheet” to be tracked through
48hrs Incident Review Service Review meetings.
Ensure SDUs complete MI Corrective Actions
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref SVMISDM/PRO/O001
Version:
Date: july-2009
Page No: 32 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process &
FUJITSU COMMERCIAL IN CONFIDENCE
8.4 SDUs: Technical Teams / Third Parties
8.1.1 Role of the SDUs: Technical Teams / Third Parties during a
Major Incident
The role of the Technical Teams / Third Parties during a Major Incident is to investigate the Incident, and
in the event of no pre-determined recovery options, suggest and evaluation of potential recovery options
to resolve the Incident.
The technical teams should not be contacted by any party other than the Service Support Manager
(Recovery Manager.)
The Technical Teams / Third Parties should send an attendee to the MI Conference Call and following
resolution to the Major Incident Review meeting. Where attendance on the MI Conference Call is not
possible, a full update MUST be provided to the Service Support Manager to ensure that that the Bridge
can be updated.
8.1.2 Technical Teams Templates
This template has been designed to assist the technical teams / Third Parties in compiling the
information required for a MI Conference Call.
NAME ‘OF I LOCATION DESCRIPTION / NOTES DISTRIBUTION
TEMPLATE
Technical Update I \ATCFS7\POcust_ser I Used to provide structure to the MI I Not for distribution
for MI Conference I 01Public\Major Incident I Conference Call and ensures that all
Call Management \Templates I relevant information is collected.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 33 of 38
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process _
fee)
FUJITSU COMMERCIAL IN CONFIDENCE
9 Post Office / Fujitsu Services Interfaces
The Post Office / Fujitsu Services interfaces are all detailed in the Process Description, section 5.1,
however for ease the interfaces have been extracted into the table below.
Major Incident Triggers I POL SCT will be informed by the Service Delivery RMGA SDM / Owner
Met? Manager / Owner managing the Major Incident of the (Duty Manager)
incident.
SMS will be sent to RMGA Management alerting to the I T*S HSD / SMC (out of
potential existence of a Major Incident hours)
BAU Incident POL SCT will be informed that there is no Major Incident I As RMGA SDM 7 Owner
appropriate I (Duty Manager)
HSD / SMC (out of
SMS sent hours) (out ol
incident Investigation & I POL invited to Major Incident Bridge TH RMGA SDM 7 Owner
diagnosis (Duty Manager)
Major Incident Bridge I POL attendance on MI Bridge THE NA
Mi Minutes distributed I POL recipient of the Ml Bridge Minutes MI Bridge I RMGA SDM 7 Owner
+15 (Duty Manager)
As determined on MI I Regular SMS / phone updates fo POL SCT As RMGA SDM 7 Owner
Bridge / in person with determined I (Duty Manager)
POL on MI Bridge
/ in person
with POL
War Room POL attendance at War Room Timescale I RMGA SDM 7 Owner
dependant I (Duty Manager)
on impact
and nature of
incident &
progress of
resolution
incident Resolution POL attendance on conference call RMGA SDM 7 Owner
(Duty Manager)
HSD / SMC (out of
SMS sent hours) (out
Major Incident Draft I POL sent Mi draft report including Security for PCi Major I MI+ 24hrs I RMGA SDM 7 Owner
Report Failures (Duty Manager)
Major Incident Review I POL attendance at Major Incident Review Mi+46hrs__I RMGA SDM 7 Owner
& — formal Incident (Duty Manager)
Closure
As detailed under Appendix 10.3, escalation of the Major Incident will also occur; this activity running
concurrently to the interfaces detailed above.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 34 of 38
fee)
FUJITSU
POA Customer Service Major Incident Process
COMMERCIAL IN CONFIDENCE
FUJ00080071
FUJ00080071
10 Appendices
10.1 List of Templates
All templates are stored on the central share.
Management \Templates
\\ATCFS7\POcust_serv\01Public\Major Incident
NAME ‘OF I DESCRIPTION/ NOTES: ] DISTRIBUTION
TEMPLATE : :
impact on HSD I This template is used to communicate the impact of a Major I RMGA Service Delivery Manager
Incident to RMGA Service Management (Service Delivery.)
Notification of I Used to communicate the key information as requested by I POL
Major Incident POL.
ator incicen RMGA Service Support & Service Delivery
THIS MUST BE EMAILED WITHIN 15 MINS OF MAJOR ;
INCIDENT INVOCATION. RMGA Head of Service Management
RMGA Head of CS
Major Incident I The Major Incident Report contains all the information about I POL
Report Template I a Mejor Incident. This document is distributed to POL.
RMGA Service Support & Service Delivery
RMGA Head of Service Management
RMGA Head of CS
Mi Conference Call
Agenda
Used to provide structure to the MI Conference Call and
ensures that all relevant information is collected.
‘Attendees of MI CONCALL
RMGA Service Support & Service Delivery
RMGA Head of Service Management
RMGA Head of CS
MI Conference Call
Minutes
Ensures that minutes from the MI Conference Call are
produced in a consistent format, and that all attendees are
aware of actions & status of recovery.
Aids communication
Attendees of MI CONCALL
RMGA Service Support & Service Delivery
RMGA Head of Service Management
RMGA Head of CS.
Major Incident I Standard format for Major Incident Review & ensures that I Attendees of MIR
Review Agenda all areas are addressed, RMGA Service Support & Senice Delvery
RMGA Head of Service Management
RMGA Head of CS
Major Incident I Standard format for Major incident Review minutes. I Attendees of MIR
Review Minutes
Ensures that information is collected in correct format for
inclusion in other reports / to be taken to Service Review.
RMGA Service Support & Service Delivery
RMGA Head of Service Management
RMGA Head of CS
Technical Team
Update to MI
CONCALL
The technical teams / third parties are responsible for
Updating the Ml Conference Call with
Input into the Incident Timeline
O Service & Infrastructure impact
C2 Risk & Impact assessment of recovery actions
O Progress against recovery actions / investigation
1 Provide update from third parties from which the
technical teams / third parties are responsible for.
Not distributed — to assist in preparation for the
MI CONCALL.
©Copyright Fujitsu Services Ltd 2009
COMMERCIAL IN CONFIDENCE.
Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 35 of 38
FUJ00080071
FUJ00080071
POA Customer Service Major Incident Process
fee)
FUJITSU COMMERCIAL IN CONFIDENCE
10.2 Service Delivery Managers, Areas of Responsibility &
Contact Details.
All alerts to be raised to Service Manager within 3 minutes
17.30 - 09.00 Monday to Thursday
17.00 - 09.00 Friday
24 hour
. all day Saturday/Sunday
Outside of these times, please use the contacts list below
AREA OF RESPONSIBILITY IEXAMPLE INCIDENTS ey CONTACT DETAILS BACK UP
Infrastructure issues. Network Incidents lan Mills Kirsty Gallacher
Data centre issues
NT/ UNIX / Data Centres Storage Incidents
All network issues NT JUNIX Incidents
Banking and Online Services I Online Service outages I Mike Stewart Pager Kirsty Gallacher
(Ine: DVLA, Debit Card, EPAY)
1st point of 3° Point of Contact
contact _
Banking and Online Services _I Online Service outages I Mike Woolgar Pager Kirsty Gallacher
(Inc: DVLA, Debit Card, EPAY)
3° Point of Contact
2nd point of r
contact tom
Reference data escalations New product not Dave Wilcox Kevin McKeown
functioning
APS/TPSISAPIPOLFS, LFS _ IFiles Not delivered, Data] Mike Stewart Pager Kirsty Gallacher
Transfer
1 Point of 3° Point of Contact
Contact
Mobile
APSITPS/SAPIPOLFS, LFS _ Files Not delivered, Data] Mike Woolgar Kirsty Gallacher
Transfer
2nd point of 3° Point of Contact
contact
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: ‘SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 36 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process _
FUJITSU COMMERCIAL IN CONFIDENCE
Mobile
Business Continuity Invoking business I Adam Parker Kirsty Gallacher
continuity incident
Engineering Major issue with Leighton Machin] Sarah Bull
engineering service
1* Point of contac 3 Point of Contact
Engineering Major issue with I Susie Appleby-I [ Sarah Bull
engineering service Robbins
3° Point of Contact
2°° Point of
Contact
Branch services Potential environmental I Denise Miller Nick Crow
issues at individual
branches
OBC Escalations OBC job going wrong/ I lanVenables I I GRO I Chris Bourne
press involved /unhappyI . }
postmaster
Major software problem —_I Major software problem I Mik Peach rGRO I John Simpkins
HSD/SMC/PostShops Major Powerhelp call I Sandie Bothick Sarah Bull
logging issues. Major
PostShop outage
Security Security Issues Peter Sewell
Virus Alerts
Release Management / Service I _ Issues caused by Sarah Bull Z Steve Denham
Introduction Releases -
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref. SVMISDM/PROIO001
Version:
Date: july-2009
Page No: 37 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process &
FUJITSU COMMERCIAL IN CONFIDENCE
10.3 Escalation Communication Protocol
The primary principle:
Up”
and
“Across
Escalation protocol:
Fujitsu; Service Owners SCT / Problem Managers
Service Delivery Team Manager Service Improvement Manager
Head of Service Management Supplier and Service Performance Manager
Customer Service Director Head of Network Support / IT Director
10.4 Major Business Continuity Incidents (MBCI)
For Horizon the MBCI triggers are listed in:
e Horizon Services Business Continuity Plan (CS/PLA/079)
« Horizon Support Services Business Continuity Plan (CS/PLA/080)
e Horizon Service Desk Business Continuity Plan (CS/PLA/015).
For HNG-X the MBCI triggers are listed in:
« HNG-X Support Services Business Continuity Plan (SVM/SDM/PLA/0001)
« HNG-X Services Business Continuity Plan (SVM/SDM/PLA/0002)
e HNG-X Security Business Continuity Plan (SVM/SDM/PLA/0031)
* _HNG-X Engineering Service Business Continuity Plan (SVM/SDM/PLA/0030).
These documents should be referred to as appropriate in the event of Major Incident to determine if
Business Continuity needs to be invoked.
10.5 Security Major Incidents
In the event of a security major incident, the incident processes as detailed in the RMGA Customer
Services Incident Management Process (SVM/SDM/PRO/0018 Appendix A) must also be followed.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 38 of 38
FUJ00080071
FUJ00080071
(oe) POA Customer Service Major Incident Process &
FUJITSU COMMERCIAL IN CONFIDENCE
SVM/SDM/PLA/0031 HNG-X Security Business Continuity Plan defines the actions to be taken if security
violations are identified.
©Copyright Fujitsu Services Ltd 2009 COMMERCIAL IN CONFIDENCE Ref: SVM/SDM/PRO/0001
Version: V3.0
Date: 7-July-2009
Page No: 39 of 38