POL00028427
POL00028427
%
Vy
To: Peter Jones Copy: Catherine Fraser
: Gail Morley
Sue Harding
Ruth Holleran
David Parnell
Janet Topham
Simon Rilot
From: Chris Young Date: 8 October 1998
Subject: MO & E2E Testing Issues
Peter
Thank you for copying me in to your memo to Simon Rilot, I will attempt to respond to
each of your areas of concern.
Migration
Ihave read your previous letter regarding this area, and can confirm that all but one of the
elements of reconciliation that you highlighted have been exercised during MOR2, with
varying degrees of success.
We have been unable to test end of cash account week where ITIP has only had part of
the weeks transactions, because we were unable to produce a balanced cash account
during MOR2. However, the test has been planned and is due to be run in MOR3. As you
mention, there were some problems encountered during migration that affected cash
account production, and they fell into the following 3 areas:
i) During Miman and MiECCO some stock values were not migrated
correctly, resulting in those products being omitted from the subsequent
stock unit balance/cash account.
ii) During MiECCO some stock quantities were not migrated correctly, i.e.
fixed value items were incorrectly migrated with a quantity of 1, witha
value not equal to default selling price.
iii) I During MiMam and MiECCO some reference data mapping issues were
identified i.e. ECCO products report to different Horizon products.
All occurrences of the above were reported to Pathway as incidents and raised as PinICLs.
These problems, combined with known difficulties in Cash Account production, resulted
in ineffectual validation of the migration process.
POL00028427
POL00028427
7
&
Ican confirm that each days planned transactions were provided to ITIP from test day 6,
although admittedly, clean day for day running was not possible until test day 13.
A large number of script errors were encountered during MOR2 as a result of the need to
control expected results. Originally the test scripts had been designed to accomplish
procedural testing only and did not contain the functional detail required for rigid expected
results. As you are aware, the decision to focus data integrity onto MOR2 was taken very
late and did not allow time for any necessary amendment to the scripts. There were also a
number of occasions when unscripted activities (including transactions) were required in
order to put the stock units into a position where a cash account could be produced.
It is hoped that much of this re-work will be in place for MOR3.
It has also proved difficult to control operator error during MO, as the operators consist
of real counter clerks rather than experienced testers. Although the operators are
supervised, it is impossible to monitor their every action, and indeed would be seen as
negative to the objectives of procedural testing. I see this as a problem that will
potentially remain during MOR3, therefore I would like to forward the following proposal
for your consideration in order to tighten up this area.
At present we are attempting to carry out a strict control of expected results across 7 test
offices incorporating approximately 50 stock units, which is proving difficult for the
Horizon test team to manage, and time consuming for ITIP staff to reconcile.
I would like to propose that we concentrate future MO data integrity testing on 2 specific
Offices (e.g. 1 x 1c and 1 x NR2) and run the other 5 as originally intended i.e. looser
expected results, This will allow full monitoring of operators, possibly using experienced
testers, and lighten the burden in terms of numbers of transactions to be reconciled.
I would also like to remind you that E2E cycle 2 will be carrying out full data integrity
checking, albeit without data centre migration, using test scripts designed with rigid
expected results.
I would appreciate your thoughts on the above.
Rejected/Missing Files
T understand that file transfers were being somewhat ‘hand cranked’ during MOR2,
therefore it is doubtful that Pathway were exercising procedures to handle rejected files.
Pathway have confirmed that they will be running FTMS during MOR3 which should
overcome these problems. I will ask Simon Palladino to contact you regarding files
outstanding from E2E and MOR2, and seek his confirmation that Pathway system
processes and procedures will be tested during future test phases.
POL00028427
POL00028427
Model Office/Operational Running
Again, I shall speak to Simon Palladino regarding resetting to ‘live’ schedules.
Cash Accounts
A total of 9 incidents (displayed below) were raised against the cash account during
MOR2, however they were only observed and checked where obvious, e.g. incorrect
totals etc. It is assumed that a full mapping exercise including checking back to original
transaction detail has been carried out prior to MO and E2E during a functional test phase.
Issue Resolution Incident Status Comment
‘Method Of Object’ Printer driver PC0014379 Pending I Solution with
problem prevents configuration problem I E-9810010075 Pathway for re-test
production of Cash
Account reports.
System allows roll Fixed for MOR3 build I PC0016604 Pending I Solution with
over after production E-9810010127 Pathway, awaiting re-
of cash account snap test in MOR3.
shot
Table 2(a) - Duc to new C/A PC0016143 Pending I Awaiting retest by -
Discrepancies not structure implemented I E-9809220009 Pathway and in
populating for MOR2 MOR3
Table 12 - Parcel Duplicate of PC0016399 Closed I Problem could be duc
traffic not populated I PC0010552. Unsure I E-9809270011 to Cash Account
of resolution mappings.
Table 10 (g) - Non- Fixed for MOR3 build I PC0016398 Closed I Solution with
Accounting data not E-9809270010 Pathway, awaiting re-
populated test in MOR3
Requirement for 2 C/A production rules I PC0016397 Closed I Solution with
copics of Cash implemented for E-9809270009 Pathway, awaiting re-
Account to be MOR3 test in MOR3
produced
Payments Table - Duplicate of PC0016242 Closed I Awaiting retest in
CHEC figure PC0013984. Unsure I E-9809220143 MOR3
displayed as negative _I of resolution
Reccipts Table - Duplicate of PC0016183 Closed I Awaiting retest in
Instant Win figure PC0013985. Unsure I E-9809220142 MOR3
displayed as negative _I of resolution
Payments Table - Info, added to PC0016192 Closed I Awaiting retest in
‘Benefit’ Line spelt PC0015315. Unsure I E-9809220140 MOR3
incorrectly of resolution
It is expected that these PinICLs, together with a number of cash account PinICLs from
MORI will be resolved and included in the MOR3 baseline, allowing a balanced cash
account to be produced.
POL00028427
POL00028427
“
Reference Data
Reference data change drops were not applied as per script during MOR2, but they were
applied, which is an improvement on MOR1. It is assumed that they will be applied on
expected days during MOR3 and E2E cycle 2.
PinICLs
An objective of Model Office Testing is to exercise the live Helpdesk procedures including
incident reporting and monitoring. This includes implementation and testing of Pathway
Helpdesk Service Level Agreements, as part of the testing of the SLA the Helpdesk are
obliged to provide regular updates on all outstanding open calls. The frequency of the
update depends upon the category of the call, but can range from every 4 hours for high
category incidents to once a day for lower categories. This could potentially involve the
Borough site receiving over 150 update calls a day, which is completely unmanageable. It
was therefore agreed with Pathway that if an incident is resolved, or a fix has been
identified and tested in another test phase, the Helpdesk call will be closed but identified as
awaiting retest in MOR3, allowing the test team to concentrate on incidents undergoing
analysis. Any PinICL resulting from the Helpdesk call may be closed or remain open.
All PinICLs awaiting retest in MOR3 will be included in the MOR3 Release Notes, and
the Horizon Test Team have recorded all open incidents, including those that Pathway
have closed, to ensure that they are retested.
I hope that this addresses all of your issues, I am happy to discuss any further concerns.
Regards
Chris Young
Horizon Model Office Test Manager