POL00028436
POL00028436
Horizon Release Management
Memorandum
To: ’ Vince Gaskell From: Andrew Simpkins
. Horizon Release Management
3rd Floor, Terminal House
“+ $2 Grosvenor Gardens
LONDON SW1W 0AB
Telephone:
Fax:
Date: 23 September 1998 Copy: Dave Miller
Simon Rilot
Meeting on 24th September 1999
In addition to the discussion on testing progress we would like to take the opportunity to -
cover some other points with you at the meeting tomorrow. In part these are a follow-up to
the meeting we had with Parry Jenkins at the end of August which provided ourselves and
Pathway with more information on the CAPS activities within the Replan. I attach our notes
of this meeting. The points we would like to cover are:
© weare still awaiting written confirmation of the contents of the CAPS releases 3.5, 4.0
and 4.5. These are needed for Pathway to confirm their delivery plan in support of multi-
benefit and for NR2+ . We at least need to agree assumptions if confirmation is not yet
possible. .
in particular we have received a note from Colin Galloway indicating that On-line
Enquiries will now be in CAPS 3.5 not 4.0 as Parry then believed. We have not received
any communication from CAPS however as to whether they require Pathway to bring
forward their corresponding facility from NR2+. Is there a proposed change to the
required date ? Is this based on clearer estimates of the projected rate of Help Desk calls
that would otherwise be required ?
* Pathway have prepared a CCN offering more free calls to the Help Desk if this would be
an acceptable interim solution to the sponsors which would provide more flexibility over
when they would need to deploy On-line Enquiries. Is this CCN still relevant ?
© we need to confirm with BA/CAPS the specific functional exclusions for multi-benefit
processing that will not be included in the Release Authorisation for NR2 Child Benefit
only. * m
© CAPS Seals of Approval are understood to be required for the release authorisation of
NR2 with Child Benefit only, although this is a non-CAPS release and will only cover
deployment to the POCL live trial offices. Do CAPS have the necessary actions in hand ?
It would be helpful if we could agree with you at least the next steps on these points
tomorrow.
Andrew Simpkins
Horizon Release Management
23/09/98 Page I of 1
POL00028436
POL00028436
PLAN OF ACTIVITIES FOR TESTING REVIEW
. Walkthrough of proposed agenda and contents
DM/MC+teams —_1/10(A)
ON 6" OCTOBER 1998
1, Prepare draft ‘terms of reference’ for the review AS 23/9
2. Discuss with Vince Gaskell DM. 24/9
3. Discuss with M.Coombs DM 25/9
4, Agree data to be supplied by Pathway DM/MC/SR. 25/9
5. Develop detailed agenda/approach DM/SR/AS 28/9
6. Agree participants with all parties AS _ 29/9
7. Agree detailed meeting agenda and presenters © DM/MC/VG ~ 29/9
8.
9.
. Review final preparations
10. Testing Review Meeting
Notes
A. Meeting 1.30 to 3.30 in Feltham
B. Meeting starts at 1.30 - open ended
Checkpoint Meeting 6/10 am
Horizon/BA/Pathway 6/10 (B)
POL00028436
POL00028436
Joint Testing Review Meeting
6" October 1998
Proposed Framework
Purpose: To review testing progress on the POCL Single Benefit Model Office stage
and to agree outstanding issues and further actions required.
Note: This meeting is not a forum in which decisions can be made to change dates in
the agreed Replan.
Areas to be covered:
1. Progress against plan since June 1998 - Horizon and CAPS
2. Current plans to complete Model Office - Horizon and CAPS
3. Model Office test coverage to date - number of scripts run, passed, failed
4, Pathway System, BIT and Regression test coverage to date
5. Releases into MO cycles - release notices and exclusions
6. Closure reports for MOR] and E2E cycle 1
7. Fault Status - items raised, cleared and outstanding by cycle (Horizon, and CAPS
and Pathway Systems Test/BIT)
8. Fault Status - trend analysis
9. Fault Status - itemisation of current ‘severe’ faults and impact assessment
10.Fault Status --analysis by functional area .
11.Objectives for temaining cycles
12.Issues/risks to be addressed
POL00028436
POL00028436
IS 18™ DECEMBER A REALISTIC DATE ?
RESPONSE TO CONCERNS
1, Pathway’s Track Record is not Credible
Pathway clearly have had a history of failure on the Programme. Since April 1998 the
position has changed. Although BA/CAPS have cast doubts in advance of every
significant Horizon testing milestone since this date they have all been consistently
achieved i.e
© start of DIT2 on schedule (Direct Interface Testing)
¢ end of DIT2 on schedule
© environment available to support JSA Model Office on 8 June
© Horizon Model Office started 3 August
« MORI finished ahead of schedule on 27 August when BA claimed a multi-benefit
MORI would take 4 (four) times as long
¢ E2E finished on schedule including multi-benefit testing when BA claimed it
would take twice as long
© MOR2 has started on time with CAPS
BA must recognise the recent good progress and not only sed to the difficulties
experienced under the PDA.
2. What do the ‘Experts’ Advise ?
DSS/BA have many years experience of large systems implementation, and believe
the 18 December is risky. But they do not have full visibility of testing activity and
they lack the ‘inside information’ available to Pathway as the supplier.
From Horizon’s formal and informal contacts with the senior technical managers in
Pathway, who have the best informed view of progress, we know that they believe the
18" December date is achievable. This does not mean it is guaranteed. But so far there
is no evidence of any compelling nature which would cause them to change this view.
We believe the evidence arising from our close contact with Pathway is of more
substance than the more remote assessments of BA senior management.
3. Is the Plan Realistic ?
Pathway re-baselined their plan for the Model Office phase on 11 September. The
plan therefore now covers a period only slightly more than 3 months, and takes into ,
account more than 6 months of testing experience to date. The margin for error is now
significantly less than it was when the 18" December date was first set in May. If all
the Pathway activities could not be synchronised around the 18" December it would
have come out in this new baseline.
POL00028436
POL00028436
The overall plan provides a contingency period of up to 4 weeks more testing in
January. This is a high level of contingency (30%) given the remaining planned test
period,
4, Analysis of Testing Errors (‘pinnicles’)
BA have placed much eniphasis on this area and it is clearly a critical measure. A
comprehensive analysis of experience to date will be presented at the meeting on 6"
October. It will also be necessary however to take into account the full picture of
testing progress which should include:
* analysis of the percentage of test scripts run in Model Office and the success rate
(which were on target in the first cycle)
¢ asimilar analysis for Pathway’s internal system test environment
¢ ability.to run the remaining test cycles within the planned periods (for which good
evidence has now been obtained in the completion of the first cycles on schedule)
the completeness of functionality in the delivered system (no hangouts are
expected to remain by 12" October)
© assessment of Pathway’s ‘fast track’ capability to correct and retest any errors
found late in the testing programme.
We intend that the meeting on 6" October will form a balanced judgement on all these
aspects.
5. Other Factors
Two other key factors have to be taken into account in managing this test phase:
¢ BA and POCL do not have the contractual right to tell Pathway to extend the
testing period which would be interpreted as causing a delay. Their sanction is to
decline the quality of the system submitted for release authorisation into live
Tunning. :
¢ The goal of this testing phase is to achieve release authorisation for the live trial in
300 post offices for the period January to June 1999. While the system has to be
adequate for live use the overall ‘quality hurdle’ for the live trial does not have to
be quite as high as for the national rollout in July 1999. There are a further 6
months during the trial period when any outstanding non-critical technical and
functional problems can be addressed as well as the new problems that can be
expected to emerge in the trial. The judgement to proceed with release into live
trial on 18" December must reflect some commercial realism of the risks involved
and not be based on an idealistic standard of ‘no errors’.
Horizon Release Management
23" September 1998