MOT & E2E Test Governance
Background and Requirement
Model Office and End to End Testing will be performed in parallel by teams based at the POCL
offices at Borough and ICL Pathway, Feltham respectively. The co-ordination, management and
reporting of each stage is clearly both essential and challenging. Earlier experiences with MOR3
and E2E2 showed that improvements to the process were necessary. As part of the recent pre-
proving exercise new management procedures were implemented and have demonstrably improved
the governance of these phases. This paper takes into account lessons gained from pre-proving and
suggests a formal approach to governance for MOT and E2E leading to Live Trials.
Key lessons learnt show that daily checkpoints, POCL test co-ordination, daily status monitoring
of all parties and carly identification and escalation of any issues found ensured a common
understanding of progress and permitted a clear basis for progress on a daily basis.
Planning and Control
Underpinning the pre-proving exercise was an agreed daily schedule of events against which
progress for all involved parties could be confirmed. A draft schedule will be available by
09/02/99.
The daily schedule complements the High Level test Grid and Low Level test plans produced by
the MO and E2E teams. The plan is to run day-for-day and accordingly any suspension of testing
will be agreed in multiples of one day i.e. day-for-day.
Progress Monitoring
Based on the experience of pre-proving, daily status meetings between the Horizon test teams and
Pathway proved extremely effective and it is proposed that this is extended and continued for MO
and E2E trials.
Each day of MOT and E2E testing will start with short Readiness to Proceed meetings at Borough
and Feltham. If necessary, a follow on meeting will be called to escalate problems and take
appropriate action. Towards the end of each day a progress meeting will be held at each location to
review progress against the plan and manage any issues. A daily management report will be
produced for each test stream.
During the trials there will also be a weekly progress/status meeting, scheduled to start at 1 lam,
each Thursday. This meeting will consider progress against plan, issues - including RAB
acceptance issues, and any management actions carried forward from previous meetings. This
meeting will be held at Feltham, Newcastle and Chesterfield to the following schedule:
Meeting Date [Location
[February 18 [Feltham
Page I
FUJ00078540
FUJ00078540
FUJ00078540
FUJ00078540
MOT & E2E Test Governance
February 25 [Benefits Agency, Newcastle
IMarch 4 Feltham
[March 8 (Chesterfield
Progress meeting arrangements are likely to be adjusted following MOT to cater for any targeted
testing, but are likely to follow the same principles.
The proposed daily meeting schedule is as follows:
(MOT - Borough IE2E — Feltham
19:30 Readiness Meeting (09:30 Readiness Meeting
10:00 Escalation Meeting (if required) 10:00 Escalation Meeting (if required)
15:00 _Progress/Status Meeting
16:00 _Progress/Status Meeting
All meetings will be attended by a core representation from Pathway, CAPS and POCL (Horizon),
and Co-ordination testing. For the morning Readiness Meeting, the Horizon representative will
normally confirm status with remote system representatives by telephone before the meeting.
For the Progress/Status meetings it is proposed that all POCL back end systems are represented by
the Co-ordination or TIP representatives, except where issues require personal representation from
HAPS, Apachi or CBDB. The representation of POCL Reference Data is to be determined.
CAPS and TIP may attend the meetings via a conference call, but will be on-site in Feltham at
least one day a week.
It is expected that CAPS will represent all other Benefit Agency systems e.g. OBCS.
Daily Readiness Meeting
This un-moderated meeting will be held by the MOT and E2E teams at Borough and Feltham with
the objective of confirming that the day’s test schedule can start. Core attendees will include at
least one representative from each system area listed on the daily schedule as appropriate although
it is permissible for representations to be made by conference call or by prior discussion. Again,
back-end systems may be represented by the Co-ordination or TIP representatives.
The meeting will be restricted to an absolute maximum of 30 minutes and is expected to take less
than 10 minutes under normal circumstances. If the meeting is unable to agree actions to resolve
outstanding issues or has a major concern about system integrity the Escalation meeting will be
called. In practice the need for this is likely to be known before the start of the Readiness Meeting
and interested parties contacted in readiness.
The first MOT Readiness Meeting will take place Wednesday February 17 at 09:30. This is the
first day of operation of the MO following installation and data set-up.
Page 2
MOT & E2E Test Governance
Readiness Meeting — Attendance
MOT ___IRole/Representing
ICo-ordination Testing Mark Burley
[Horizon ames Brett
[TIP/Central Systems [TBC - phone
[Pathway Pat Lywood
ICAPS [Ron Watson
[E2E __IRole/Representing
ICo-ordination Testing Marc Reardon
[Horizon [Keith Hall
[TIP/Central Systems [TBC - phone
[Pathway Simon Palladino
ICAPS [Hadley Baldwin
Escalation Meeting - Attendance
[Role/Representing
IChair [Richard Gaze/Chris Young
[Horizon (Chris Young/appropriate T/L
[Pathway Pete Jeram/Chris Wannell
ICAPS [Carol Jepmond / Peter Beuster (Remote)
ICo-ordination Testing Mare Reardon/Peter Jones
Progress/Status Meetings
The daily Progress Meeting will be held in Borough and Feltham for MOT and E2E respectively,
however the start times will be staggered to permit key staff to have an input to both meetings.
Meetings will be moderated by the Horizon Test Manager or by a delegated team member as
appropriate.
This meeting will review progress on a daily basis and any issues arising during the day or
significant issues from previous days’ testing. At this meeting key management decisions will be
agreed, for example in extreme circumstances to suspend testing or to regress to a previous day.
The daily progress meeting will review the following metrics:
Progress against daily schedule (all areas)
Test progress against test plan (conditions tested or other appropriate measure)
Fault clearance against fault clearance plan
Hot spots emerging
Issues raised
Particular note will be taken of the progress against the daily schedule and any slippage occurring
in any area. The tolerance for slippage in back-end processing is one day unless this is agreed at
this meeting. The default position will be to halt processing until all areas have caught up with the
slippage.
Page 3
FUJ00078540
FUJ00078540
MOT & E2E Test Governance
Issues which require input from, or resolution by Horizon Product Management or Pathway
Design will be followed up outside the meeting. Where necessary attendance by Product
Management or Pathway Design will be requested as appropriate.
A report will be produced following the meeting with an agreed daily management summary and
key metrics. The report will also list issues raised that day and progress with issues raised
previously.
Additionally, progress reports will be made to the Horizon Management Team [HMT] and the
Horizon Checkpoint Meeting [bi-weekly]. It remains to be confirmed whether additional Senior
Management reviews will be necessary and how the information and reports from these meeting
will be applied to the pre-RAB meetings during the same period.
The Thursday Progress Meeting is a face-to-face meeting intended to drive out management
concerns and to review progress against all outstanding issues. The meeting will be rotated
between Feltham, Chesterfield and Newcastle.
Progress/Status Meeting - Attendance
(MOT __IRole/Representing
IChair (Chris Young
[Horizon ames Brett
[TIP/Central Systems [Sue Rhodes
[Pathway Pete Jeram/Pat Lywood
ICAPS Ron Watson/Carol Jepmond
ICo-ordination Testing Mark Burley
[Reference Data {TBC
[E2E [Role/Representing
IChair [Richard Gaze
[Horizon [Keith Hall
[TIP/Central Systems Martin Box
[Pathway Nikki O’Sullivan/Simon Palladino
ICAPS [Hedley Baldwin/Carol Jepmond
[Co-ordination Testing
[Mare Reardon
[Reference Data
[TBC
Page 4
FUJ00078540
FUJ00078540
MOT & E2E Test Governance
Thursday Progress/Status Meeting — Attendance
[Role/Representing
[Chair [Richard Gaze
[Horizon (Chris Young
[TIP/Central Systems [Martin Box /Sue Rhodes
[Pathway Pete Jeram or representative
(CAPS (Carol Jepmond
(Co-ordination Testing Peter Jones
[Reference Data [Ted Baldwin
[Product Management [TBC
Escalation
Problem notification and escalation on a day to day basis will be via the daily Progress/Status
meeting with the following key individuals empowered to make decisions on behalf of their
respective organisations:
Pete Jeram [Pathway]
Richard Gaze [Horizon]
Dave Parnell [POCL]
Carol Jepmond [CAPS]
As appropriate, issues that cannot be resolved or managed using this process will be raised
immediately to the respective Horizon/CAPS and Pathway Programme Boards for direction.
Other Principles
= All parties will need to agree detailed synchronisation of checkpoints for restart including
database backups (see Readiness for Testing notes).
Page 5
FUJ00078540
FUJ00078540