FUJITSU
FUJITSU SERVICES
Test Report for Functional and Non-Functional
Integration Testing of BI3 S80
Company-in-Confidence
FUJ00086360
FUJ00086360
Ref: WI/TRP/077
Version: 0.1
Date: 03/08/2005
Document Title:
Document Type:
Release:
Abstract:
Document Status:
Originator & Dept:
Test Report for Functional and Non-Functional Integration
Testing of BI3 S80
Test Report
S80
This is the test report for the functional and non-functional
integration testing, of BI3 S80, performed by the SV&I team.
DRAFT
Neil Gormley - SV&I
Internal Distribution: Reviewers
External Distribution: Reviewers
Approval Authorities:
(See PA/PRO/010 for Approval roles)
Name
Role
Signature
Date
Pete Drewatt
ITU Test Project Manager
Sheila Bamber
ITU SV&I Manager
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 1 of 51
FUJITSU
FUJITSU SERVICES
Test Report for Functional and Non-Functional
Integration Testing of BI3 S80
Company-in-Confidence
FUJ00086360
FUJ00086360
Ref: WI/TRP/077
Version: 0.1
Date: 03/08/2005
0.0 Document Control
0.1 Document History
Version No. Date Reason for Issue Associated
CP/PEAK/PPRR
Reference
0.1 03/08/2005 _I Initial version for review
0.2 Review Details
Review Comments by :
12/08/2005
Review Comments to :
Originator & Document Management
Mandatory Review
ITU SV&I Manager
Sheila Bamber
Test Design Peter Robinson
Optional Review
RASD TDA Gareth Jenkins
ITU V&I Test Team Leader Hermia Figueiredo
ITU V&I Test Team Leader Michael Welch
Release Programme Manager
Gill Jackson
ITU Manager
Pete Drewatt
Development Manager
Mark Scardifield
Development Manager
David Harrison
(*) = Reviewers that returned comments
©Copyright Fujitsu Services Ltd 2005
Compan, ‘Confidence
Page: 2 of 51
FUJITSU
FUJITSU SERVICES
Test Report for Functional and Non-Functional
Integration Testing of BI3 S80
Company-in-Confidence
Date:
FUJ00086360
FUJ00086360
Ref: VI/TRP/077
Version: 0,1
03/08/2005
0.3 Associated Documents
Reference
Version
Date
Title
Source
[1] PA/TEM/001
Fujitsu Services Document
Template
PVCS
[2] VUSTR/062
13/12/2004
Fujitsu Services _ Testing
Strategy for Horizon System
PVCS
[3] VUSTR/064
15/08/2003
Testing Approach For The
Horizon System
PVCS
[4] VISTR/080
21/01/2005
Fujitsu Services Test Strategy
for S80.
PVCS
[5] VITSC./472
26/05/2005
SV&I High Level Test Plan for
S80
PVCS
[6] VITSC/474
22/03/2005
SV&I Non-Functional
Level Test Plan for S80
High
PVCS
[7] VITSCAA75
20/04/2005
High Level Test Plan for
Direct Interface Testing of
Horizon to POL FS Interface
at S80
PVCS
[8] VI/TSC/476
1.0
20/04/2005
High Level Test Plan for
Direct Interface Testing of
Horizon to POL Client
Transmission Summaries
Interface at S80
PVCS
[9] VITSCA77
1.0
20/04/2005
High Level Test Plan for
Direct Interface Testing of
Horizon to HR SAP Interface
at S80
PVCS
[10] VITSC/478
20/04/2005
High Level Test Plan for
Direct Interface Testing of
Horizon to POL MIS Interface
at S80
PVCS
[11] VITSC/479
0
20/04/2005
High Level Test Plan for
Direct Interface Testing of
Horizon to EDG Track and
Trace Interface at S80
PVCS
[12] V/TSC/480
20/04/2005
High Level Test Plan for
Direct Interface Testing of
Horizon to FRTS Interface at
S80
PVCS
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 3 of 51
FUJ00086360
FUJ00086360
foo) Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 S80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
[13] V/TSC/481 I 1.0 20/04/2005 High Level Test Plan for I PVCS
Direct Interface Testing of
RDS to Horizon Interface at
S80
Unless a specific version is referred to above, reference should be made to the
current approved versions of the documents.
N.B. Printed versions of this document are not under change control.
0.4 Abbreviations/Definitions
Abbreviation Definition
AIS Application Interface Specification; standard document type required for each
external interface to the Horizon system
APS ‘Automated Payment System
DCS. Debit Card System
DeLT Development Link Testing
DIT Direct Interface Test
DRS Data Reconciliation Service
DWH Data Warehouse
E2E End to End
EDG External Data Gateway
EPOSS Electronic Point of Sale Service; Horizon service that supports Retail
functions in Branches
FTMS File Transfer Management Service; Horizon process that provides
configurable file transfer services between Horizon and Post Office Ltd.
Clients. Services available include data compression and encryption
GP General Purpose
HLTP High Level Test Plan
Horizon Horizon is the end-to-end solution that provides Post Office with a range of
counter services
HR SAP The SAP System used by Royal Mail Group’s Human Resources to pay
sub-postmasters
ITU Integration and Test Unit
LFS Logistic Feeder Service
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 4 of 51
FUJ00086360
FUJ00086360
foo) Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 S80 Version:
ersion: 0,1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
MIS Management Information System; Horizon application set that provides
access to and analysis of the information held in the Data Warehouse
NWB Network Banking
POA Post Office Account
POL Post Office Limited
POL FS Post Office Limited Financial System, name given to the new SAP service
POL MIS Post Office Limited Management Information System; from S80 Point 40 this
is fed transaction data previously sent to TIP.
RDDS Reference Data Distribution System.
RDMC Reference Data Management Centre.
RDS. Reference Data System; Post Office Ltd. system that provides a Reference
Data feed to Horizon and other systems
SAP ADS SAP Automated Distribution Service, for managing Cash Distribution
sco Single Counter Outlet
SV&I System Validation and Integration. A unit within ITU
TIP Post Office Ltd. Transaction Information Processing system
TIS Technical Interface Specification
TPS Transaction Processing Service; Horizon service that formats data for
transmission to a number of external systems.
V&l Volume and Integrity
0.5 Changes in this Version
Version Changes
0.1 Initial Version for internal review
0.6 Changes Expected
Changes
Review comments
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 5 of 51
FUJ00086360
FUJ00086360
foo) Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 $80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
0.7 Table of Contents
1.0 INTRODUCTION.
2.0 SCOPE AND OBJECTIVE!
3.0 MANAGEMENT SUMMARY.
4.0 SYSTEM UNDER TEST.
4.1 ENVIRONMENT
4.2 SOFTWARE...
4.3. TEST TOOLS.
5.0 TEST CONDUCT.
§.2_ TEST RESULTS.....
5.2.1
5.2.2
6.0 TEST STATISTICS.
Overview by are:
DIT results.
6.1 SCRIPTED TESTS. ....cccccccsssssssssssesssssssussssnsssseeceseessnunsnnnssneceesesttnuunnsnnsnsseesesesiasnnnnsssseeeeeeeeee 20
6.2. TEST COVERAGE. 21
6.3 PASS RATE 22
7.0 PEAKS....
7.1 PEAKS BY CYCLE
7.2. DEFERRED PEAKS
8.0 CONCLUSION.
8.1 SUMMARY.
8.2 ISSUES. .
8.2.1 Changing requirements/design.
8.2.2 Some areas not tested before SV.
8.2.3 Steep learning curve for many of the test team
8.2.4 Complex low-level changes.
8.2.5 Reference Data.
8.2.6 Stock Unit Rollover testing.
8.3. TEST PROCEDURE IMPROVEMENTS FOR FUTURE RELEASES.....
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 6 of 51
FUJ00086360
FUJ00086360
foo) Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 $80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
1.0 Introduction
This is the final test report for the S80 integration testing performed by SV&L
The testing comprised:
- Direct Interface testing of each of the interfaces introduced or modified as
part of the S80 release;
- three 3-week cycles of functional and non-functional testing;
- an extended Peak clearance cycle.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 7 of 51
FUJITSU
FUJITSU SERVICES
Test Report for Functional and Non-Functional Ref:
Integration Testing of BI3 S80 Version:
ersion:
Company-in-Confidence Date:
FUJ00086360
FUJ00086360
VI/TRP/077
0.1
03/08/2005
2.0 Scope and Objectives
The objective of this document is to summarise the results of the testing.
The scope of the testing is defined in the functional [5], non-functional [6] and
DIT [7-13] High Level Test Plans.
For ease of reference the change proposals delivered and tested at S80 follow:
cP Description
3716 I[MPACT: Rel 3 - Solution Build, Test and Implementation Stages
SAP: Hosting Impact Programme Release 3 - Solution
3717 Specification and Solution Build, Test and Implementation
Plus One Prompts’ project — Solution Build, Test and
3731 [Implementation
3739 IAKB - Solution & Test and Implementation
[Remove Support for Short Term Closure OBC Operations on
3747 IOCMS Client and SYSMAN
3770 [Smart Post: Track and Trace (T&T) Integration
3778 Sales Prompt Support for Money Laundering regulations
3781 IVPN server Key expiry alert
3787 Split The Suspense Account Report
3842 I[IMPACT: Rel 3 - Counter Changes
3843 IIMPACT: Rel 3 - Host Changes
3844 IIMPACT: Rel 3 - Further Host Changes
3888 [Impact Release 3: Week numbers printed on forms and reports
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 8 of 51
FUJ00086360
FUJ00086360
foo) Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 S80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
3951 IChanges to the A&L Banking receipt
3980 Impact Release 3: Remove the Variance report
3998 [Impact Release 3: CAP/TP Ambiguity
4002 [Impact release 3: Change the layout of Branch Trading Statement
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence
Page: 9 of 51
FUJITSU
FUJITSU SERVICES
Test Report for Functional and Non-Functional
Integration Testing of BI3 S80
Company-in-Confidence
FUJ00086360
FUJ00086360
Ref: WI/TRP/077
Version: 0.1
Date: 03/08/2005
3.0 Management Summary
Three 3-week cycles of Integration testing have been completed followed by an
extended period of Peak re-tests
The results of the tests can be summarised as follows:
Cycle 1 Cycle 2 Cycle 3
Test Coverage 88% 89% 99.8%
Pass Rate 88% 95% 97.8%
A total of 1267 S80 peaks were raised by SV&I. This includes 223 Peaks
relating to incidents raised during E2E up to and including E2E Cycle 3. It also
includes 45 Peaks which are to be fixed at a future release.
Priority Raised Closed Open Future
A 13 3 0 0
B 824 809 5 0
c 375 329 7 38
D 35 8 0 7
Total I 1267 I 1199 [ 22 I 45
All Direct Interface Tests (DITs) have been completed successfully and, with
the exception of the POL FS DIT, all have been 'signed off. Sign-off of the
POL FS DIT is expected shortly.
Recommendations from S80 are detailed in section 8.3.
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 10 of 51
oO
FUJITSU Integration Testing of BI3 $80
FUJ00086360
FUJ00086360
Test Report for Functional and Non-Functional Ref: VI/TRP/077
Version: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
4.0 System Under Test
4.1 Environment
The majority of functional testing was performed on the main integration rig
BTCI. Security, Performance and System Management testing was performed
on the BTC3 rig - as was some counter end-of-year testing. The DITs were
performed on the E2E test rig, BTC7.
The main integration environment is made up of the following platforms:
Solaris Host running DWH, DRS, APS, TPS, OBCS, RDMC, RDDS, LFS, NPS
& TES (with EMC Disk array)
16 Correspondence Servers supporting Riposte messaging software in a dual
locale, four cluster configuration.
Web Servers - PAF & DVLA
ACDB Client. (residing on the SSC Workstation)
ACDB Server.
8 Generic Agent Servers.
1 NBX Routing Agent/Reversal Agent
4 Network Banking Agents.
DCSM Server.
DCSA Server.
Post Offices, which include varying configurations such as Single counter (fixed
SCO and mobile SCO), Double counter, Triple, and five counter position outlet.
English, Welsh, Scottish and Northern Ireland offices are included.
All Post Offices supporting Riposte messaging software.
FTMS gateways supporting file transfers to/from TIP, POCL RDMC, Horizon
Help Desk, sub-set of AP Clients, NS&I, DVLA & EDG.
Workstations to support the reference data and message broadcast applications.
APS workstation.
DRS Workstation
EPAY Emulator
DVLA Emulator
KMS Server
©Copyright Fujitsu Services Ltd 2005 Compan
Confidence Page: 11 of 51
oO
FUJITSU Integration Testing of BI3 $80
FUJ00086360
FUJ00086360
6
Test Report for Functional and Non-Functional R
Version: 0.1
FUJITSU SERVICES
f: VI/TRP/077
Company-in-Confidence Date: 03/08/2005
4.2
4.3
* KMS Workstation
* VPN Servers.
* Certification Authority Workstations (Shared across test rigs)
« OCMS Servers
* ~Centera box
« EMC
¢ Full Maestro
¢ Bootloader Server
« Range of SYSMAN boxes.
¢ Audit Server and Audit Workstation.
© QA server to allow use of automated scripts for Performance testing purposes.
¢ Test Director Workstations to allow the test scripts to be executed.
Software
The software under test is defined in the system baseline documents lodged in
PVCS.
As is standard practice, each test cycle was begun with the software available
at that point and the test rig was regularly 'topped-up' with updates/fixes as
they became available.
Test Tools
Two new test tools were introduced at S80:
A track and trace emulator which emulated the Track and Trace Web Server
on the Post Office Enterprise Data Gateway (EDG). This allowed the T&T
XML messages to be parsed and automatically checked for conformance to the
AIS.
A counter transaction bulking and loading tool which was developed to support
the testing in relation to requirement BT001 (SU rollover timing). This tool
made it possible to load the five weeks of transactions needed to record the
time taken to rollover a stock unit at the end of a five week Trading Period.
©Copyright Fujitsu Services Ltd 2005 Compan
Confidence Page: 12 of 51
FUJITSU
FUJITSU SERVICES
Company-in-Confidence
Test Report for Functional and Non-Functional
Integration Testing of BI3 S80
FUJ00086360
FUJ00086360
Ref: WI/TRP/077
Version: 0.1
Date: 03/08/2005
5.0 Test Conduct
5.1 Tests Performed
In the three execution cycles all of the key areas of S80 functionality were
tested.
The following table shows the individual areas and their inclusion in the S80
test cycles:
Area
Cycle 1
Cycle 2
Cycle 3
Impact - Migration
v
v
v
Impact - Balancing/Rollover
Impact - Other Reports
Impact - Counter Transactions
Impact - Variances
Impact - Transaction Corrections
Impact - POL MIS/FTRS/CTS
Impact - POL FS
Impact - HR SAP
Impact - DRS
Impact - DWH
RDMC-RDDS
Sales Prompts
Track and Trace
Security
New/Modified FTMS Interfaces
AKB
VPN server key expiry alert
Estate Management
Counter Performance
NIN INDICE E EVER GEE GEGENE GEGEN
NIN NIRS SNIRRERIESRTSISN NERERESISISIS
CaN IN ININENIDNINE ENR UE EEE ENERGIES
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 13 of 51
FUJ00086360
FUJ00086360
foo) Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 $80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
5.2
5.2.1
5.2.1.1
5.2.1.2
After the end of cycle 3 the BTC] test rig was reset and a Peak clearance cycle
was begun. This had a number of objectives:
- to test fixes for all peaks outstanding at the end of cycle 3;
- to test fixes for peaks raised during end-to-end (E2E) testing;
- to test CP's delivered after the end of cycle 3 (3951, 3980, 3998 and 4002);
- to perform comprehensive regression testing of all areas for which peak
fixes had been taken or changes made as a result of the CP's;
- to perform the SU rollover timing test in support of Impact requirement
BTOOI.
The peak clearance cycle continued up until the cut-off for counter fixes at
S80.
Test Results
Overview by area
Impact - Migration
Given the complexity of the counter migration (with functional changes
triggered at three points: 20, 30 and 50) Migration testing went well from the
very first cycle and, by the third cycle, there were very few new problems found
in this area. The ability to migrate successfully - specifically to Branch Trading
at Point 50 - was a fundamental pre-requisite to the counter testing which
could only take place after migration had occurred (e.g. Transaction
Corrections, Branch Trading Statement, the handling of volume stock).
Host migration - specifically points 25, 30 and 40 - also was largely problem
free from an early point in our testing.
Impact - Balancing/Rollover
This was the area that caused most concern from the first cycle. In that first
cycle in particular, many problems were identified, e.g. cash and stock
disappearing, volume stock treated as value stock, incorrect figures on the
balance reports after stock adjustment and declaration, problems with local
suspense and the Trading Statement. The situation improved in Cycle 2 but it
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 14 of 51
FUJ00086360
FUJ00086360
foe] Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 $80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
was only in Cycle 3 that the code began to stabilise and testers were
consistently able to balance and rollover without error.
5.2.1.3. Impact - Other Reports/Counter Transactions
There were Impact related changes to a number of counter reports and, of
course, these were specifically tested. In addition, however, all counter reports
were tested in the course of a widespread and in depth regression test of the
counter as a whole. This included the testing of all transaction types
(NB/DCS/ETU/APS/Bureau/EPOSS) in all modes. There were very few
problems found in respect of either reports or transactions.
5.2.1.4 Impact - Variances
There were a number of problems with the Variance Report in the first cycle
but these were superseded by the removal of that report in response to CP
3990. Otherwise, the changes in this area - Add/Remove Cash, Variance check
and changes to the Cash declaration process - were largely problem free.
5.2.1.5 Impact - Transaction Corrections
This was a major new area of functionality at S80, with host and counter
components. There were few problems with the host (file loading and
processing) component but the counter component (the selection, processing
and reporting of Transaction Corrections) threw up a number of problems.
Amongst these were issues in relation to the rules for the use of Transaction
Corrections, specifically for currency and travellers cheques. By Cycle 3, the
issues had been resolved and the counter functionality stabilised.
5.2.1.6 Impact - POL MIS/CTS/FRTS
At S80 the feed of transactions to TIP is replaced by a feed to POL MIS with
additional data provided for bureau and on-line transactions. In the course of
testing a number of problems, with that additional data, were found. But the
problems were not serious (at least in terms of their impact on testing) and
were quickly fixed. The changes at S80 to the FRTS and CTS feeds were error-
free.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 15 of 51
FUJ00086360
FUJ00086360
foo) Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 $80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
5.2.1.7 Impact - POL FS
POL FS testing was performed using reference data (product to article
mappings) prepared by the test team. There were some problems with this data
in early testing but these were quickly resolved. The testing itself went well -
there were no 'show-stoppers' but a substantial number of Peaks were raised in
the early cycles in particular.
5.2.1.8 Impact - HR SAP
There were very few problems with HR SAP.
.9 Impact - DRS / DW
The changes in these areas were relatively minor and the few problems found
were resolved by the start of Cycle 3.
5.2.1.10 RDMC/RDDS
RDMC/RDDS testing was initially delayed by resourcing issues: time had to be
spent on reference data preparation and management at the expense of specific
testing. The testing was completed successfully but only after a small number
of Peaks had been raised, fixed and retested.
5.2.1.11 Sales Prompts
The introduction of Sales Prompts was one of the major changes at S80 (along
with Impact and Track and Trace). In terms of the testing, however, it was the
least problematic, with only a small number of Peaks raised.
5.2.1.12 Track and Trace
There were a steady stream of Peaks raised during all three cycles of Track and
Trace testing. In the first two cycles of testing, Mails reference data prepared
by Development (in Crewe) was used. In the third Cycle (and subsequent Peak
clearance) Mails reference data provided by PO Ltd was used and this led to
the identification of problems not previously seen.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 16 of 51
FUJ00086360
FUJ00086360
foo) Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 $80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
5.2.1.13 Non-functional - Performance
The Performance testing performed on the 20-counter office went well, with no
major problems reported. The SU rollover performance testing performed
subsequently, in relation to Impact requirement BT001, was also completed
successfully - it being demonstrated that the time taken to rollover a stock unit
post-Point 50 (i.e. with five weeks of data) was no more than five times the
time taken to roll over a stock unit at S75 (i.e. with one weeks data).
5.2.1.14 Non-functional - Other
The AKB, VPN and Estate Management changes were all virtually problem-
free. A number of problems, none of them very serious, were found during
Security testing. Similarly the testing of the new and modified FTMS interfaces
generated a relatively small number of Peaks.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 17 of 51
FUJ00086360
FUJ00086360
foe] Test Report for Functional and Non-Functional Ref: VI/TRP/077
FUJITSU Integration Testing of BI3 $80 Version:
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
5.2.2 DIT results
The following DITs have all been signed off:
Horizon-CTS
Horizon-FRTS
Horizon-POL MIS
Horizon-HR SAP
Horizon-EDG Track and Trace
RDS-Horizon
The Horizon-POL FS DIT has now been completed - the last remaining test
(Error file handling) failed on the first two attempts but was finally completed
successfully in E2E Cycle 4. Sign-off is expected shortly.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 18 of 51
FUJITSU System Integration Test Report for S80
FUJITSU SERVICES
Company-in-Confidence
FUJ00086360
FUJ00086360
Ref: WI/TRP/077
Version: 0.1
Date: 03/08/2005
6.0 Test Statistics
6.1 Scripted Tests
Scripted Tests
15000
2
a
2 14000
o
a
2 13000 :
© 42000 14147 14125
3 13056
E 11000
2
10000
Cycle 1 Cycle 2 Cycle 3
Test Cycle
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence
Page: 19 of 51
FUJ00086360
FUJ00086360
i Ref: VI/TRP/077
FUJITSU System Integration Test Report for S80 Version
ersion: 0,1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
6.2 Test Coverage
Test Coverage Achieved %
ce 100
@
2
o
2 90
x)
S
£ 80
8
Ey
* 70
Cycle 1 Cycle 2 Cycle 3
Test Cycle
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 20 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU SERVICES Version: 0.1
Company-in-Confidence Date: 03/08/2005
6.3 Pass rate
Cycle Pass Rates
Oftests Executed mOftests Scripted
2 100 88. 2 __g9 es
e
so:
3 8
Da
ga
o
$
S
a
Cycle 1 Cycle 2 Cycle 3
Test Cycle
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 21 of 51
FUJITSU
FUJITSU SERVICES
Com
System Integration Test Report for S80
pany-in-Confidence
FUJ00086360
FUJ00086360
Ref: WI/TRP/077
Version: 0.1
Date: 03/08/2005
7.0 Peaks
7.1 Peaks by cycle
There follows a breakdown of Peaks raised by SV&I during S80 testing for resolution
at S80. Note the figures include Peaks raised as a result of incidents identified during
E2E testing up to and including E2E Cycle 3.
$80 Cycle 01
Priority Raised Closed Current
Open
A 2 2 oO
B 237 237 ()
Cc 164 163 1
D 27 27 0
Total 430 429 1
$80 Cycle 02
Priority Raised Closed Current
Open
A 3 3 O
B 187 187 0
c 66 66 oO
D 3 3 Oo
Total (259 259 [')
$80 Cycle 03
Priority Raised Closed Current
Open
A 2 2 )
B 154 153, 1
c 38 7 7
D 6 @ 0
Total 220 218 2
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 22 of 51
foe) .
FUJITSU System Integration Test Report for S80
FUJITSU SERVICES
Company-in-Confidence
FUJ00086360
FUJ00086360
Ref: WI/TRP/077
Version: 0.1
Date: 03/08/2005
Peak Clearance Cycle
Priority Raised Closed Surert
pen
A 6 6 0
B 246 236 10
Cc 49 45 4
P) 12 12 0
Total 313 299 14
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 23 of 51
FUJ00086360
FUJ00086360
FUNITSU System Integration Test Report for S80 \ Ref VI/TRP/077
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
7.2 Deferred Peaks
This is the list of peaks raised by SV&I during $80 testing (cither as a result of SV&I functional testing or E2E testing) which are to be fixed at a
future release.
Peak Description Analysis and Proposal Severity Proposed
Target Release
PC0113449 S80A CTR : Layout of SU Trial balance This is considered a low priority fix as the required information is I Low Future Release
Cc Stock Unit Trial Balance Report differs from the I “sPlayed correctly.
layout specified in the UI (EA/IFS/011) in that it
is missing a blank line before the line "VALUE
STOCK & MOP",
PCO113835 S80A - further logon checks after TC logon The preferred option is for Transaction Corrections to be the last check, I Low Future Release
c prompt however it is not essential, as the Corrections are not being made to APS
. . transactions. If we were to go ahead with the changes at this stage it
The Declaration/Correction and Revaluation UI I would involve changes to three components which could result in further
(EA/IFS/012) states that the Transaction problems, amounting to instability.
Correction Check should be the LAST logon
check to be performed. On the first logon on of
the day at a counter it is NOT the last - the APS
Disaster Recovery/Fallback checks are the last.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 24 of 51
Fe)
FUJITSU
System Integration Test Report for S80
Ref: VI/TRP/077
Version: 0.1
FUJ00086360
FUJ00086360
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
PCO113851 S80A - Format of Branch TP displayed This is considered a low priority fix and although introduces an Low Future Release
c Having rolled an Office out of TPO7 into TPOS, I Mconsistency, the impact is negligible,
selection of the 'Select Branch TP' icon displayed.
the message "The branch is currently in Trading
Period 8." This should be '08' for consistency
with other screens/reports
PC0113979 S80A CTR : Reval warning marks SU as dirty This is a design issue rather than a coding bug. The system could and Low Future Release
c Prior to $80, a product price change causes each should allow stock units to be rolled over as inactive after a price change
ra pnedn . but it is just operating as it did prior to S80 (i.e. counter staff wont have
stock unit to marked as ‘dirty’, preventing the ‘ -
. . \ was to do anything more than they currently have to do).
stock unit from being rolled over as 'Inactive' (in
case it holds any of the stock which would need
to be revalued).
From Point 50 this is no longer necessary (since
stock can no longer be revalued) but stock units
are still marked as ‘dirty’ on receipt of a price
change and therefore still cannot be rolled over
as inactive.
PCO114197 S80A MIG : Screens displayed at logon This is a screen flicker and would require the introduction of an Low Future Release
Cc . . intermediate screen. As there is no opportunity for any action to be
The desktop menu is displayed before the APS taken on the screen, i.e. button presses the impact is considered low and
recovery prompts. At $70/75 this was not the a fix not recommended.
case, all prompts were displayed before the first
display of the desktop menu
PC0114419 I S80A-CTN-No space in pop-up screen of B- No impact, cosmetic change. Low Future Release
'D Snapshot
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 25 of 51
Fe)
FUJITSU
FUJITSU SERVICES
System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0.1
Company-in-Confidence Date: 03/08/2005
FUJ00086360
FUJ00086360
The screen displayed while the Balance snapshot
is printing has the text 'Printing Report Balance
Snapshot(By volume)’. There is no space afier
the word 'Snapshot"
PCO115028
Cc
S80B-MIG-In MIGRO1 user CAP still shows as
Ol
When logging in as MIGRO1 user and
performing a migration. The System displayed
CAP 30 automatically and migration was
completed successfully. However, the
information on the bottom right hand corner of
the desktop appeared as below.
Stock: DEF
User: MIGROL
(Share)
CAP 01/BP 01
This was corrected once the MIGRO1 user is
logged out and logged back in.
This has not changed at $80, it has always been like this when you log
in to a new counter in the DEF SU. This is therefore recommended for
closure
Low
For closure.
PCO115319
Cc
S80A-CTR-Stock Adjust should report adjust
failure
This problem only occurs if volume stock
This is not considered to be an essential fix because it will only occur if
the data is not specified correctly.
To protect against this possibility a fix should be considered in a Future
Low
Future Release
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence Page: 26 of 51
FUJ00086360
FUJ00086360
FUNITSU System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU SERVICES Version: 0.1
Company-in-Confidence Date: 03/08/2005
products have an adjustment price and that Release.
adjustment price is outside of the product's
min/max values, i.e. it will only occur where the
reference data is invalid.
In such a case, if an attempt is made to adjust the
volume of the product, the system will not make
the adjustment (as expected) but it will also not
display an error message. The Peak relates to the
fact that no error message is displayed.
PCO115852 S80 - CTR - SAN gives -ve Cash volume This is a code error but the problem has been in the system since before
C adjustment $80 and doesn’t appear to be causing any significant confusion.
Stock adjustment transactions (positive and
negative) were successfully completed and a
Transaction Log was then printed. This showed
the cash settlement transaction for the Stock
Adjust Negative transaction as having a negative
volume and a positive value. The volume should
have been shown as positive.
Note that adjustment transaction was correctly
reflected in the stock unit and office balance
reports (i.e. the impact of this problem is limited
to the Transaction Log).
Low
Future Release
PCO116211 S80 B - ETU icon missing after session tfr Historically the menus formed a disconnected hierarchy in that each
submenu has a single parent menu, and thus the branches of the menu
tree always move outwards from the root and never join back onto other
branches. To avoid large-scale reference data duplication, at S60 release
Cc When transferring a session from one counter,
which is on the Mobile Phone E Products
Services screen, to another counter for the first
Medium
Future Release
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 27 of 51
FUNITSU System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU SERVICES
Version: 0.1
Company-in-Confidence Date: 03/08/2005
FUJ00086360
FUJ00086360
time the Vodafone icon disappears.
If the user prevs out of this screen and goes back
into the Mobile Phone E Products Services
screen, the icon reappears.
some submenus that are shared by 2 of more branches have been
introduced. For example, the 'Mobile E prods' menu is specified just
once in reference data but it appears on both the following menus :-
Transactions/Serve Customer/Other Products
Transactions/Serve Customer/Other Products/Travel
This sharing of menu data appears to be confusing the Desktop when it
restores the customer session on a destination counter during a session
transfer. The Desktop fails to display the icon on the first (top left)
button on the shared menu, if that icon has never been displayed on that
counter since the Desktop was last cycled. It is only menus shared in this
that experience the problem. The list of menus affected in this way
is given below.
Transactions/Serve Customer/Other Products/Mobile E prods
Transactions/Serve Customer/Other Products/Travel/Mobile E prods
Transactions/Serve Customer/Functions/smartpost Admin
Transactions/Non Acc Data/Functions/smartpost Admin
Transactions/Non Acc Data/Other Non Acc/Functions/smartpost Admin
Transaction/Remittances/Out ADC
Transaction/Remittances/Out ADC/Stock/Out ADC
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 28 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
On the basis that this will be a rare occurrence and the fact that
Reference Data to deliver approximately 100 additional buttons would
be required, we do not suggest this is fixed at S80. Recommend for
Future Release.
PCO116293 S80B CTR : SU imbalance warning clears This is not a new problem at S80 and the problem of SU imbalance is Low Future Release
C automatically not a regular occurrence. Also, the message tablet is displayed for the
If there is an imbalance in a SU then a warning duration of the time it takes to print the report.
is displayed at rollover. This warning has a
"Continue" button and should not be cleared
until the clerk presses the button, but it actually
clears by itself. So if you're not looking at the
screen you may miss seeing this message.
PCO116581 S80B CTR Trans Corrections OnLine Err A new report definition would be required for this printed slip. The Low Future Release
cC message suggestion is as it is only visible to the PM and since it is only used for
. . . when he is going to call NBSC as long as the text is regarded as
Ifa Transaction Correction fails to process at the I irrelevant this would not cause an issue.
counter the correct message appears on screen
and on the print out, however the top of the
receipt print out also includes the text ‘On-line
services unavailable’. This is obviously
misleading and would probably be reported to
NBSC by the Post Master when he called with
the reference number for the failed TC.
PCO116972 S80C - Session transfer with mails items on Development have analysed the problem and concluded the following: Low Future Release
'c stack i) All session transfers are thought to be slower within 2 migration
In the course of testing it has been found that
session transfer with Track and Trace items on
windows:
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 29 of 51
Fe)
FUJITSU
FUJITSU SERVICES
System Integration Test Report
Company-in-Confidence
for S80 Ref: VI/TRP/077
Version: 0.1
Date: 03/08/2005
FUJ00086360
FUJ00086360
the stack is taking longer than does session
transfer with non-Track and Trace items on the
stack. The length of time it takes has varied - on
some occasions it has been almost as quick (a
few seconds) as a session transfer with non-track
and trace items but on other occasions it has
taken as much as 3 and a half minutes.
(1) From Counter Roll-out (Point 20) until the end of the CAP when the
CBDB is switched off
(2) The last CAP until the first TP inclusive
(ii) However transfers with mails transactions appear to be taking
significantly longer at S80 compared with S75.
As T&T is only in pilot until post $90, it is likely that this problem will
not be an issue for the bulk of the estate.
Part (i) we can’t do anything about - it is believed to be only a problem
during S80 CAP to TP migration. So the recommendation is that we
publish a KEL.
Part(ii) is under investigation with Escher. Component showing
problem is Escher desktop, so a fix at $80 is unrealistic. If an Escher fix
is identified and can be supplied within the necessary timescales this
could potentially be included at S90.
PCO117023
'c
$80 E2E : Pouch dispatch msg wrong for nil
pouch
When an ACC card is scanned when there are no
pouches to collect, the system displays a message
"Pouch Despatch Successful’, however this is
immediately followed by another message 'No
Pouches to Despatch’.
This problem has existed since before S80 and does not appear to be
causing any problem in Live.
A KEL will be raised and the call closed.
Low
Future Release
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 30 of 51
Fe)
FUJITSU
FUJITSU SERVICES
Company-in-Confidence
System Integration Test Report for S80
Ref: VI/TRP/077
Version: 0.1
Date: 03/08/2005
FUJ00086360
FUJ00086360
PCO117096
Cc
S80C-T&T- Error on speed bulk receipt.
There is an inconsistency between the
Mails/Bulk Mails receipts and the Speed Bulk
receipt.
The Mails/Bulk Mails receipt states:- Posted
after Last Collection?
The Speed Bulk receipt states:- Posted After Last
Collection?
The 'A' in "Afier" is in uppercase on the Speed
Bulk receipt and in lower case on the
Mails/Bulk Mails receipts.
For consistency the 'a’ should be in lower case on
all receipts.
Low risk cosmetic change.
Low
Future Release
PCO117173
'c Format of new-style SU snapshot report does not
conform to the design - it should not contain
lines for Discrepancy Shortages Transferred or
Discrepancy Excess Transferred
S80C CTR : Format of SU Snapshot
The problem here is consistency as the SU Snapshot should be identical
to the SU Final Balance Report. The missing lines will always be zero.
This would be a medium change in the code and is therefore not
recommended at this stage.
Low
Future Release
PCO117310 S80 E2E : First logon displays malformed screen
c On the first logon of the day, the DESKTOP.
screen is displayed for a few seconds with a
number of empty ‘white boxes’.
Raised as an incident during end-to-end testing but subsequently agreed
to be of a low priority and deferrable to a future release.
Low
Future Release
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 31 of 51
FUJ00086360
FUJ00086360
FUNITSU System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
PCO117852 S80C-TCO-Print error results in Option list The only reasons a failure would occur at the counter is if the TC was I Low Future Release
C again for a Non Core product not available at that office or if the end date of
If an error occurs while processing a Transaction the product or mode has passed (afier the TC products and modes were
C . . A aa validated by TPS). These situations are unlikely to arise in live and this
‘orrection (TC) and the Print option is selected, “ak hha . ss
: peak has therefore been moved to Future Unspecified.
after the Error has printed the user will not be
returned immediately to the TC list screen. They
will briefly see either the TC settlement options
or the Housekeeping Menu before being returned
to the list of TCs. The user will briefly see one of
the following screens:
(a) The settlement options list
again i.e. ‘Make Good: Cash’, ‘Settle
Centrally’, etc, if the TC just attempted
had ‘Accept Now’ then ‘Make Good’ or
“Settle Centrally’ selected before the
error.
(b) The Housekeeping Menu if the TC just
attempted had 'Assign to HO’, 'Send to
P&L' or 'Stock WO' selected before the
error.
PCO118048 S80D-CTR-Isolated node warning * 2 on some Coding error results in two different components issuing warning I Low Future Release
c reports message (so a fix would require changes to both components).
When a report is produced at an ‘isolated node’ As the problem does not affect functionality a KEL has been raised,
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 32 of 51
FUJ00086360
FUJ00086360
FUNITSU System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
(i.e. node which is not connected to at least one Mscardifield1738R and a fix proposed for a Future Release
other) the system displays an ‘isolated node’
warning message, For a small number - 4 - of
reports that warning is being displayed twice.
The reports affected are:
Reprint Stock Unit Balance report
Processed Transaction Corrections report
Reprint Branch Trading Statement
Sales Report.
PCO119894 S80D-TCO-text appears bottom right - The current mode is being flashed up at the bottom right-hand of the I Low Future Release
c Detail/Options screen. The (Escher) Desktop component is doing this automatically
whenever the mode is changed. This happens elsewhere in the existing
product but is particularly noticeable in the new Transaction Correction
functions
A Transaction Correction was selected for
processing. 'Accept Now' was selected and then
‘Make Good' from the settlement options screen.
'OK' was then selected on the same screen. At KEL Mscardifield4232P has been raised, suggested that call is closed.
this point there is a brief flicker of the text
‘Accept Now’ at the bottom right-hand corner of
the screen, just below the 'OK' button.
Similar behaviour occurs for the other
Transaction Correction processing options, e.g.
when 'Settle Centrally’ is selected there is a brief
flicker of the text ' Settle Centrally’.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 33 of 51
Fe)
FUJITSU
FUJITSU SERVICES
System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0.1
Company-in-Confidence Date: 03/08/2005
FUJ00086360
FUJ00086360
PCO119913 S80D - CTR - SU Trial balance layout See also PC0113449. Low Future Release
'c The example layout for the (post Point 50) Trial I This is an error in the coding of the report but does not affect
Balance layout shown in EA/IFS/011 (Impact __ I functionality.
Release 3 - Report production) differs from the Propose fixing at a Future release
actual report in the following respects: i gata Put a
- The example has a blank line at line 56 (just
after 'Discrepancy Short Resolved’. There is no
blank line on the current report.
- The example has just one blank line before the
‘STOCK VOLUMES ' line. There are currently 3
blank lines.
PCO119991 S80D Dev : MiMan into TP mode - report The scenario where an office is MiManed, will only impact a small I Low Future Release
Cc problem number of offices. The report can still be produced and its content is
An office is Mimaned into TP Mode and a report correct (it is only the selection picklist which is wrong) and the problem
ce . disappears after the first Trading Statement is produced.
produced which includes selection of a TP (e.g.
Transaction Log). The TP selection picklist for
the report shows the current TP as 'CAP' - e.g.
instead of showing TP7 it shows CAP7.
PC0120060 S80D CTR : Reprint error message overlaps. Cosmetic fix, considered to be of no Business Impact. Low Future Release
tablet
D>
Select Reprints/SU Balance and select a stock
unit. Select Continue without selecting a
CAP/TP
At S75 the message displayed is split across 2
lines No CAP/BP Criteria specified At S80 the
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence Page: 34 of 51
Fe)
FUJITSU
FUJITSU SERVICES
System Integration Test Report for S80
Company-in-Confidence
Ref: VI/TRP/077
Version: 0.1
Date: 03/08/2005
FUJ00086360
FUJ00086360
message is displayed on a single line No TP/BP
Criteria specified. This makes the message too
long to fit into the error tab The final latter (d)
overlaps the RH edge of the tab
PC0120245
Cc
S80D DCS : Can settle cheque by debit card
The Settlement menu was selected with nothing
on the stack (i.e. via the Functions menu).
‘Cheque’ was the selected and a value of £30
entered. This correctly showed on the stack as
‘Pay £30.00'. A Debit Card was inserted in the
Pin Pad and Chip and Pin selected. Since there
was no money owed by the customer the system
should not have allowed a debit card payment to
be made. Instead the amount display showed
session total -30.00, max amount 30.00, default
amount 0.00 and a DC transaction for £30.00
was completed. The stack now showed that the
customer was to be paid £60.
This is an unlikely scenario and not a new problem at $80. It is only a
problem if the Settlement menu is on display with nothing on the stack —
under normal circumstances where a cheque is used in part payment
against, say, a TV licence, then the max and default amounts for the
Debit card payment are calculated correctly.
Medium
Future Release
PC0120467
'D
S80D-CRE-Printing report APS trans list (S80)
When the APS transaction report is printing, at
an S80 counter, the Printing report tablet has the
message: 'Printing Report APS Transaction List
(S80)'. The text '(S80)' is superfluous.
Only this report is affected and, while the text '(S80)' is unnecessary, it
has no impact on the production or content of the report.
Low
Future Release
PC0120468
'C
S80D-CRE-Misc Trans report shown as 0/s
The 'Summaries' button was selected from the
Balancing menu and the picklist of outstanding
This is not a new problem at S80 and is highly unlikely to occur in Live.
For the problem to occur it is necessary NOT to have performed any of
the transactions normally included on this report, e.g. Parcel Traffic and
Low
Future Release
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 35 of 51
Fe)
FUJITSU
FUJITSU SERVICES
Company-in-Confidence Date:
System Integration Test Report for S80
Ref: VI/TRP/077
Version: 0.1
03/08/2005
FUJ00086360
FUJ00086360
reports displayed. That picklist included the
Miscellaneous Transactions report. The report
was selected and printed. It was empty.
BdC, and to have performed non DVLA APS transactions (which are
not included on the report). The part of the system which determines
what appears on the outstanding summaries pick list does not recognise
that all other APS transactions are excluded from the report.
PC0120471 S80D-CBL-adj stock: no error on invalid po fee I The problem only arises if adjustment price is set on PO fee products I Low Future Release
I” . - Fi . (don't believe this will be the case in Live) AND the reference data is
c In Adjust stock, if an attempt is made to adjust a I i correctly specified (e.g. the min/max values are wrong). The system
Product and that Product has an adjustment 7 will behave as it should - no adjustment is made - but the lack of
Plue then the adjustment a ne diagnostic information will make it a bit more difficult to diagnose the
error event is written to the event log, problem. No Business Impact - proposed for fixing at a future release.
Ifan attempt is made to adjust a Postal Order
product and the associated PO fee product has an
adjustment price which fails the min/max value
validation then, again, the system does not make
the adjustment but this time it DOES NOT write
an error to the event log.
PC0120509 S80D-CBL-Discrepancies after attempted inact This problem dates from before S80 - i.e. is present in the S75 counter. I Medium Future Release
c To The scenario is an unusual one and one which does not appear to have
In this test the office had two stock units, $1 and
II. 11 is inactive but holds a product, which has
been re-valued and as such, the SU cannot be
rolled over as inactive. In S1 all declarations
were made without discrepancy. An attempt was
then made to roll Il over as inactive. The
appropriate error was displayed. Still in S1, the
'Discrepancies' button was then selected and
ever happened in Live. A fix at this stage would be considered high risk
and it is therefore proposed to raise a KEL and propose a fix for a Future
Release.
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 36 of 51
Fe)
FUJITSU
FUJITSU SERVICES
System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0,1
Company-in-Confidence Date: 03/08/2005
FUJ00086360
FUJ00086360
discrepancies were displayed for cash and
stamps. The value of the discrepancy in each
case was the value calculated for 11.
PC0120545 S80D-CRE-bal reports: blank lines after tfr While this DOES comply with the layout given in the IFS (EA/IFS/011), I Low Future Release
'D in/out it is inconsistent. Cosmetic fix, with no Business Impact, suggests the
On Balance reports there are two blank lines document is amended to reflect.
after the 'Transfers In' line (and before the first
Rem in line) and only one blank line after the
‘Transfers Out' line (and before the first Rem Out
line).
PC0120670 S80 Frequency of CMService Msgs if ADSL This problem is not new at S80 and has no Business Impact, the impact I Low Future Release
'D Disconnec is to the support teams in diagnosis of the problem.
Tf, at an ADSL office, the ADSL connection is
lost the CMService constantly outputs a warning
message (ID 125) approx every 60 seconds and
an error message (ID 132) after every five ID 125
messages.
PCO120783 S80D-CTN-Mob E-prods button not locked when I Though this is inconsistent behaviour of the buttons and both should be I Low Future Release
c offline displayed as locked in this scenario, any button driven ETU transaction
The 'Mobile E prods' button (Transactions/Serve
Customer/Other Products/Mobile E prods) is
disabled when the off-line indicator is on display.
However, this button also occurs on the Other
Products/Travel submenu and that second
instance of the 'Mobile E prods’ button does not
get disabled when the off-line indicator is on.
attempted could not be completed if there is no on-line connection.
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 37 of 51
Fe)
FUJITSU
FUJITSU SERVICES
System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0.1
Company-in-Confidence Date: 03/08/2005
FUJ00086360
FUJ00086360
PCO120812
'D
S80D-CTR-Incorrect message header & No
space
The message displayed when a stock unit is
rolled over with a discrepancy (post Point 50)
has the header 'Discrepancy Transferred’. The
screen shot for this message in the IFS
(EA/IFS/013, section 4.2.10) has the header as
"Discrepancy Transfer’.
Also, in the text of the message there is no space
between the sign of the discrepancy figure and
text ‘(Loss)’ or ‘(Gain)’ (e.g. £50 loss appears as
'£50.00-(Loss)’.
No Business impact, cosmetic fix only.
Low
Future Release
PCO120873
'C
S80D-MIG-Event log has CAP in heading afier
PSO
The Event Log reports continue to show 'CAP' in
the column headings after Point 50:
SU USER CAP BP NODE
DATE AND TIME — EVENT TITLE
EVENT DESCRIPTION
We believe this to be a low Business Impact and as such recommend
fixing in a Future Release.
Low
Future Release
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence Page: 38 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
PCO121001 S80 E2E 1b : Parcel Traffic pick list re-displayed I Raised as an incident during end-to-end testing but subsequently agreed I Low Future Release
c When £0.00 Value, 0 quantity Parcel Traffic to be of a low priority and deferrable to a future release.
entries are made for cach ‘product’ in the PT pick
list, a message tablet is output alerting the user to
the zero entries. When the message is confirmed
that the zero entries are correct, the pick list
screen is re-displayed for a second or two before
the receipt is printed.
The pick list is not re-displayed when quantities
other than zero are used, nor is it re-displayed for
this scenario on an $75 counter.
PC0121136 S80D-CTN-Rem/Tfr reversal receipts show vol This has been looked at by Phil Hemmingway (Design). He notes that I Low Documentation
cC 0.00 the content of reversal receipts is not specified in the Report Production update
On Rem and transfer in and out receipts, volume
stock items are shown with value blank. Ifa Tfr
out is reversed or a Rem In/Out volume stock
transaction is reversed then the receipt produced
is a ‘normal’ session receipt which has unit
price/value zero ('0.00') for the volume stock
products.
User Interface document (EA/IFS/011). However he believes that 'stating
an item has been reversed at zero item value, giving a total reversal
value of zero is correct and representative of what has happened.’
He therefore believes there is 'no fault’ but suggests that the IFS is
updated to record the fact that volume stock products appear on reversal
receipts with value 0.00.
PO Ltd have requested that this be fixed at S80, The peak has been
further discussed by design and development and both are strongly of the
opinion that to make such a change at this point would carry
considerable risk. Currently, the reversal receipt is a generic session
receipt (i.e. it is already different from the Rem and Transfer receipts).
This is because 'reversal' is also a generic function (used for all modes).
To accommodate the change, reversal functions and receipts which are
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 39 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
specific to Rems and Transfers would have to be developed. Ifa change
is required then the details need to be agreed between FS and POL.
PCO121249 S80D-CBL-Office snapshot has Reval up/down We believe this to be a low Business Impact and as such recommend I Low Future Release
'c Post Point 50, the office snapshot still contains fixing in a Future Release.
‘Reval Up' and 'Reval Down' lines. These should
not be present (and are not present on SU
Balance snapshot and Trial and Final balance
reports).
PCO121314 S80 E2E 1b : CD Giro slip Despatch Date in yr 1-I This has already been accepted by PO Ltd as deferrable and the related I Low Future Release
'c 9 incident updated accordingly.
The peak was raised during E2E testing
(Incident 2659).
Girobank pre-printed stationery reports were
produced at Office D (506 880), cycle 1b, day 10.
The space to the right of the datestamp box has a
space for printing of the Stock Unit, Week
Number and Balance Period, shown in the
format of AA:09:01. This is being printed as
AA: 9:01,"
PC0121925 S80 E2E 2 : Shared SU Cash dec Gain after mig I Development believe that this incident (and a second similar incident in I Medium Future Release
c >TP Cycle 3) was caused by a problem in Riposte and that that problem is an
E2E Cycle 2 Incident 2783
A cash declaration was made in "Stock
Balancing" for the amount displayed on the
Snapshot. When the Cash Variance was checked
existing and very intermittent ‘live’ problem. The concern has been that
the E2E incidents suggested that the problem - a Riposte procedure does
not return details of all transactions when called to build the 'balance
tree’ - is occurring more frequently at S80.
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 40 of 51
Fe)
FUJITSU
FUJITSU SERVICES
System Integration Test Report for S80 Ref: VI/TRP/077
Version: 0,1
Company-in-Confidence Date: 03/08/2005
FUJ00086360
FUJ00086360
afierwards a Gain of £45.05 was displayed.
The discrepancy was the cash value of the
transactions performed on the stock unit after
rolling into Branch Trading.”
However, in automated testing in which the test scenario was repeated
more than 10,000 times, the problem did not occur once. It is suspected
that the rig set up on E2E has contributed to the problem occurring and
there is no reason to believe that live will see any more instances once at
S80.
As stated above the underlying cause would require an Esher delivery.
This would not be available for S80 or S80R. Development will
continue to investigate to identify whether a 'work round’ change could
be made in our code.
PCO121959 S80D EMV : Retail card with invalid LUHN 560 (S75) This change in behaviour is thought to have occurred between I Low Future Release
c When a card fails the LUHN check, the customer the 557 ($70) and the versions of the PINPad.
is still asked to enter a PIN, before the error is It is an unlikely occurrence (corrupt card or hardware problem) and the
reported to the clerk . This is different behaviour I error is handled cleanly if it ever should occur.
10 pre-S80. The PEAK will be treated as a low priority Hypercom problem for
further investigation at S90. We recommend that this PEAK should be
deferred until a later release.
PC0122000 S80 E2E 2 : 'Date produced for B, Trading This is felt to be of low Business Impact. Low Future Release
'D reprint
E2E Incident 1937
For a reprint of a Branch Trading statement all
screens now show correct captions but "Date
Produced' line is still missing from the Produce
Report screen.
PC0122003 S80 E2E 2 : Vol stock sort order Office/SU snaps I On investigation Development have found that this problem, which I Medium Future Release
affects all products which do not have a Level 1 Primary Mapping, has
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 41 of 51
FUJ00086360
FUJ00086360
o System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU 9 Version
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
Cc E2E Incident 2795 existed in Live since 2001. On that basis, Po Ltd have already signalled
. here willi ferral until a fi I
Stock Unit and Office Snapshots were produced I "efe willingness to accept deferral until a future release
at Office B (501 680) today (Cycle 2, Day 8). On
referral to the "Stock Volumes" section, the £1
Littlewoods Lotto Scratchcards appeared before
the £2 ones on the Office Snapshot, but the other
way around on the Stock Unit Snapshot.
PCO122856 S80 E2E 3 : Reprint for blank OW Redeem Stp This problem is felt to have a low business impact and a change for S80 I Low
C report would, at this late stage, be risky. It is therefore proposed that it should
E2E Cycle 3 incident 3032 be fixed at a Future Release.
When re-printing a "Redeemed Savings Stamps"
report, there is an option to print a report that is
blank. This particular report seems to be
produced by the implied cut-off being set at the
roll over of the Trading Period. The report was
printed at Office E (004 038) and had the date
range of 04/07/2005 - 04/07/2005. There was a
Trading Statement produced on this date at this
office.”
Future Release
PC0122911 I S80 E2E3 : T&T Mails Dsptch Report Page 2 I The problem only occurs where EPSON printers are used and has I Low
‘Cc header minimal business impact. PO Ltd have already agreed that the
E2E Cycle 3 Incident 3019 associated incident should be deferred.
Mails Despatch" report for "All Carriers" was
produced at Office E (004038) which included
transactions for both Royal Mail and
Future Release
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 42 of 51
Fe)
FUJITSU
FUJITSU SERVICES
System Integration Test Report for S80
Ref: VI/TRP/077
Version: 0.1
Company-in-Confidence Date: 03/08/2005
FUJ00086360
FUJ00086360
ParcelForce. The header from page 2 was printed
on the bottom of page lof the report. This office
uses the "Epson" back office printer. When the
same scenario was completed at Office F (505
038) the header was printed correctly. Office F
uses the "OKI" back office printer."
PC0122916 S80D : T&T Mails Dsptch Report shows details I The problem is not a serious one - the report includes all of the data that I Low Future Release
al wa is available - and only arises for transactions which are more than 42
Ifa Mails Despatch Report is reprinted for a days old.
period for which the original transactions have
been archived then, as per design, the report
includes for each archived transaction the
message "T & T details not available. The data
has been archived. NaN' followed by the bar-
code. The text 'NaN' should not be included.
PC0122950 S80 E2E 3 : Txn Corr report for A&L shows Various Products (e.g. 108, 111, 135, 166, 173, 177-179, 181-186) I Low Future Release
c A&&L contain a double ampersand in one or more of the Short, Medium and
E2E Cycle 3 Incident 3019
"4 Transactions Corrections were received at
Office E (004 038) for items 6790, 6791, 6792
and 6793. The names for these products are
"A&L Deposit TC", "A&L Withdrawal TC",
"NS&I Deposit TC" and "NS&I Withdrawal
TC". The names are displayed correctly on the
counter screen, although when the outstanding
transaction corrections report is printed, they are
Long name attributes. This is necessary to allow the single ampersand to
be displayed on the screen. Existing reports which include these
products suppress the second ampersand. The Transaction Correction
reports (new at $80) do not.
Given that only a relatively small number of TCs are likely to be affected
and the problem is cosmetic with low business impact, it is proposed
that it is fixed at a future release.
©Copyright Fujitsu Services Ltd 2005
Company-in-Confidence
Page: 43 of 51
FUJ00086360
FUJ00086360
FUNITSU System Integration Test Report for S80 \ Ref VI/TRP/077
ersion: 0.1
FUJITSU SERVICES
Company-in-Confidence Date: 03/08/2005
shown as "A&&L Deposit TC", "A&&L
Withdrawal TC", "NS&&I Deposit TC" and
"NS&&I Withdrawal TC" respectively."
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 44 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU SERVICES Version: 0.1
Company-in-Confidence Date: 03/08/2005
8.0 Conclusion
8.1 Summary
S80 has been one of the most complex Horizon releases to date. The changes at the counter -
for Track and Trace and Sales Prompts but most particularly for Impact - have been
fundamental and wide reaching. The Data Centre also has seen significant change, specifically
to the TPS host system (again, Impact related). The complexity of the release has been further
increased by the fact that the S80 changes are triggered at various migration points.
It has not been an easy release to test. But the SV&I team have worked extremely well and
helped deliver a working, robust system which meets the requirements.
We have needed and received considerable support from the Design and Development teams
and they deserve much credit and our thanks for that. Development consistently provided us
with a quick turnaround on Peak fixes and the PIT and SPTS teams were kept busy processing
and applying those fixes. They typically did this without any delay and they, also, deserve
credit and thanks for their support.
Finally, we must also thank our colleagues in the Post Office End-to-End test team. They
always make a very valuable contribution and this has been particularly true of this release.
8.2 Issues
8.2.1 Changing requirements/design
As happens with all releases, requirements changed/were added during the testing. For S80,
however, the level of change certainly appeared that much greater than in previous releases -
and occurred later (the last S80 CP - 4002 - was not authorised until after the end of cycle 3).
The design also evolved and changed during the course of testing, partly as a result of changed
requirements and partly in response to problems identified during testing.
This was particularly true of critical areas like the handling of local suspense, the derivation of
fields on the Branch Trading statement and the reporting of Travellers Cheques.
8.2.2 Some areas not tested before SV&I.
For some areas of change - notably Transaction Corrections and those introduced by the late
CP's - there was no DeLT testing in advance of SV&I testing. Not surprisingly, there were
instances in which the untested code did not perform even its most basic function(s) when
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 45 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU SERVICES Version: 0.1
Company-in-Confidence Date: 03/08/2005
applied to the SV&I environment. In those instances, the time spent by PIT/SPTS in
packaging and applying the code was effectively wasted.
8.2.3 Steep learning curve for many of the test team.
Most of the S80 test team only joined the team after the HLTP had been produced (because
they were working on S75). They missed out on the main S80 analysis and, as a result, had a
steep learning curve. They were then required to write the low level test script from a high
level test plan (functions and conditions) prepared by another tester. This was necessary in the
circumstances and did not compromise the testing but the low level scripts took longer to
prepare and were not as detailed as would typically have been the case had the same person
been responsible for both.
8.2.4 I Complex low-level changes.
Typically, in order to test a system, we need to know what it is supposed to do, not how it
does it. However, the nature and complexity of some of the changes at S80 (for example those
in relation to Data Server and soft launch) were such that they could only be fully tested if the
different processing paths and the conditions under which they were followed, were
understood. While it made testing more difficult, there was a positive aspect in that we
probably worked closer with Design on this release than ever before.
8.2.5 Reference Data
At S80 there was a very considerable amount of test reference data to be prepared, including:
- an update to every stock item
- POL FS mappings
- HR SAP mappings and schedules.
- Sales Prompts
- Mails reference data
Technically it should not matter who prepares the reference data: providing the data is defined
in accordance with the AIS (in all its permutations) then the system should support it and
behave accordingly.
But it is desirable if we can use the test reference data prepared by the Post Office (providing
that data reflects the conditions which our analysis has shown need to be tested). The
advantages are obvious:
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 46 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU SERVICES Version: 0.1
Company-in-Confidence Date: 03/08/2005
= Test reference data preparation can be time-consuming and requires considerable
experience - limited in our team - and attention to detail.
= The data prepared by PO Ltd to support their E2E testing can be expected to be more ‘live-
like' than data we prepare ourselves - important since there is always a risk that we miss a
permutation which, when used, results in the system behaving invalidly.
= There is an advantage for the Post Office in that if we use their data before they do then
errors in the data can be identified and fixed before E2E testing begins.
= It is easier to manage test rigs if, in the main, only one set of test reference data is used
across all of the rigs.
The problem, of course, is that typically our testing begins before the PO Ltd data is ready.
This was certainly the case at S80 - with the exception of Mails reference data, PO Ltd test
reference data was not available until after Cycle 3 had started. Also, if we use PO Ltd
reference data we don't have control of it and risk being affected by problems/delays in
preparing the data. Finally, there are always some specific reference data variations which we
need which will not be reflected in PO Ltd data.
So, the SV&I team had a lot of test reference data to prepare and manage at S80 and,
although the team now has two people experienced in reference data preparation, the
workload was such that they were stretched. One consequence was that specific
RDMC/RDDS testing planned for S80 (and performed by one of the two) was delayed and
was not fully completed until the Peak Clearance cycle.
I would note that although we used test reference data prepared by SV&I in cycles 1 to 3, we
used at least some of the data - the item updates for Impact - prepared by PO Ltd for the Peak
Clearance cycle. This brought its own problems: there were errors in the data (although these
were quickly cleared) and it was necessary to separate the PO Ltd data we were using from
the data we were not (i.e. the POL FS and HR SAP mappings).
Mails reference data deserves special mention. Mails reference data is not prepared by SV&I
(we do not have the know-how/tools). Nor do we define tests which cover all permutations.
We expect the Mails reference data to be defined to support the stated requirements/design
and write our tests accordingly. If the data is not available from PO Ltd then it must be
prepared by the Mails Development team in Crewe. This was the case at S80. For cycles I and
2 we used Mails reference data prepared by Crewe and in Cycle 3 we used Mails reference
data prepared by PO Ltd. The Cycle I and 2 testing was still valid (the Crewe reference data
supported most of the tests) but it would have been better to have had the PO Ltd reference
data from the start.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 47 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU SERVICES Version: 0.1
Company-in-Confidence Date: 03/08/2005
8.2.6 Stock Unit Rollover testing
The testing in relation to Impact requirement BTO01 was only begun towards the end of the
Peak clearance cycle. BT001 states:
Production of a balance report for a stock unit must be possible to be produced within 5
times the current production time for a stock unit with a busy transaction profile, long
trading statement period
There were various reasons for the delay in starting this testing, not least the fact that a new
bulk loading tool had to be developed to inject five weeks worth of transactions into a counter
message store. It must be accepted however that the issue did not receive attention as early in
S80 testing as it should have done. As it happens, the testing was completed successfully and
the requirement was met.
8.3. Test procedure improvements for future releases.
The desired improvements are self-evident from the issues listed above and they go beyond
improvements in test procedures:
= Clear unambiguous requirements, covering all aspects of the changes and available as early
in the testing process as possible. Change is inevitable as the design evolves but the test
team need to be better kept informed of the planned changes (rather than having to await
the reissue of Conceptual Designs / Design Proposals).
= As has already been noted, there was particularly good working relationship between the
test team and design at S80. This was important because it has not been - will not be -
possible for the test team to do their job without understanding the design at a low level.
The relationship between SV&I and Design needs to be maintained and formalised, with
the test team actively involved in design reviews and, again, kept informed of planned
changes to design.
= The decision to have some low-level S80 scripts written by someone other than the analyst
who wrote the HLTP was made from practical necessity - it is not SV&I standard practice
so no change to procedure is required to address the issue. At S90 those responsible for
test analysis are also responsible for the production of the low-level tests scripts, thus
avoiding the 'steep learning curve' issue at S80.
= It is recommended that no major change is accepted by SV&I until that change has passed
basic, at least, link testing (Delt).
= With the exception of Mails reference data, SV&I testing must not be dependent on the
availability of Post Office test reference data. The SV&I test reference data requirements
should be identified at the earliest possible moment in the test analysis phase and
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 48 of 51
FUJ00086360
FUJ00086360
FUJITSU System Integration Test Report for S80 Ref: VI/TRP/077
FUJITSU SERVICES Version: 0.1
Company-in-Confidence Date: 03/08/2005
preparation (and test loading) of that data begun as far in advance as possible of its first
usage. The extent of the work involved in reference data preparation/management must be
better understood and properly resourced. One additional person should be trained in, and
subsequently involved in, reference data preparation. If PO Ltd test reference data is
available and can be used seamlessly then it should be used. It is most desirable that Mails
reference data, in particular, is made available by the PO Ltd before Mails related testing is
begun. However, SV&I team members responsible for Mails related testing must develop
an understanding of Mails reference data (for which they will probably require some
training) to the extent that they can, at least, specify data for preparation by Crewe.
= S80 demonstrated our weakness in respect of performance testing (especially after our
main performance tester left the company). The required performance testing was
completed but we clearly need someone to take responsibility for future performance
related testing and that person (or persons) needs the appropriate guidance, from Test
Design, on the requirements (what performance is being measured against what
benchmark) and needs the technical skills to run the tests.
©Copyright Fujitsu Services Ltd 2005 Company-in-Confidence Page: 49 of 51