Fujitsu Services
FUJ00232541
FUJ00232541
Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
Document Title:
Document Type:
Release:
Abstract:
Document Status:
Originator & Dept:
Contributors:
Internal Distribution:
External Distribution:
Approval Authorities:
Transaction Benchmark Service: Service Description
Service Description
BI3 onwards
In compliance with Schedule 19, this document describes the
Transaction Benchmark Service in terms of the type and level of
service provision, the method of delivery and the targets against
which the Horizon counter system is measured.
Approved
Victor Gough
Post Office Account, Customer Services
Richard Brunskill
Hilary Forrest
Martin Riddell
Richard Brunskill
Jan Ambrose
Document Management
Bernadette O’Donnell, Post Office Ltd, Network Support
David Sugden, Post Office Ltd, Network Support
Name Position Signature Date
Bernadette O’Donnell _I Post Office Ltd - Supplier
and Service Performance
Manager
Richard Brunskill Fujitsu Services — Post Office
Account, Business Service
Manager
© 2004 Fujitsu Services Commercial in Confidence Page: 1 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
0.0 Document Control
0.1 Document History
Version No. I Date Reason for Issue Associated
CP/PinICL
0.1 26/11/02 Initial Draft
Ola 29/11/02 Re-draft following informal discussion with Post
Office
0.2 18/12/02 Changes made following agreement with Post Office
0.3 20/12/02 Fujitsu Services amendments
1.0 20/12/02 Agreed version of 0.3 by Post Office with no
changes
Ll 26/03/04 Addition of two sections covering a description of
method of service delivery and the counter system
performance targets.
0.2 Review Details
Review Comments by: I 12" July 04
Review Comments to: I Vic Gough
Mandatory Review Authority Name
Fujitsu Services, Business Services Manager Richard Brunskill
Fujitsu Services, Technical Design Authority I Allan Hodgkinson
(Performance)
Fujitsu Services, Commercial Hilary Forrest
Post Office Ltd, Supplier and Service I Bernadette O’Donnell
Performance Manager
Post Office Ltd, Network Support David Sugden
Optional Review / Issued for Information
Fujitsu Services, Information Analyst Jan Ambrose
Fujitsu Services, Customer Service Director Martin Riddell
(* ) = Reviewers that returned comments
0.3 Associated Documents
Reference Ver. I Date Title
Source
© 2004 Fujitsu Services Commercial in Confidence
CONTRACT CONTROLLED
Page: 2 of 19
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
BP/CON/319 I 1.0 31/12/02 I Schedule 19: Operational Services PVCS
CS/PER/046 I 3.0 I 03/12/03 I Counter Transaction Performance - PVCS
Measurement and Results
CS/TSC/003_ I 1.0 06/01/03 I Benchmarking Scripts. PVCS
BP/SPE/035 I 1.0 I 23/12/02 I NBS definition PVCS
CS/PRP/O001 I 3.0 30/10/02 I Counter Transaction Performance PVCS
Measurement and Benchmarking. (NB:
To be withdrawn upon approval of this
document)
VIU/STRI064 I 1.0 I 15/08/03 I Testing Approach for the Horizon PVCS
System
Unless a specific version is referred to above, reference should be made to the current
approved versions of the documents.
0.4 Abbreviations/Definitions
Abbreviation Definition
Agreement The Agreement between Post Office Ltd and Fujitsu Services Ltd, dated
28" July 1999 as amended, in particular by CCN1100.
APS Automated Payments Service
ccD Contract Controlled Document
EPOSS Electronic Point of Sale Service
Major Release A release of software (as defined in Schedule I of the agreement) that the
Service Management Forum decides is likely, upon its implementation, to
have an effect on the system component times.
If the Service Management Forum cannot agree whether a release is a
Major Release, the matter will be escalated as a dispute in accordance with
Annex 2 to Schedule 4 of the Agreement, commencing with the
“Negotiators Review”.
NBS Network Banking Service
OBCS Order Book Control Service
Target Time(s) Contracted counter system component time(s) for the Horizon applications
EPOSS, OBCS and APS, or contracted counter system component times at
the aggregate level for the nominated NBS transaction types.
0.5 Changes in this Version
Version Changes
2.0 Detail added regarding method of service delivery (allowing withdrawal of
CS/PRP/001 on approval of this document) and regarding the counter
system Target Times.
© 2004 Fujitsu Services Commercial in Confidence Page: 3 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
0.6 Changes Expected
Changes
Document to be updated as and when system-based performance measurement tools are made available
and there use agreed by Post Office Ltd., or as and when system components are discontinued or
introduced as agreed between Fujitsu Services and Post Office Ltd.
© 2004 Fujitsu Services Commercial in Confidence Page: 4 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
0.7
1.0
Ll
1.2
13
14
15
1.6
2.0
2.1
2.2
2.3
2.4
3.0
3.1
3.2
4.0
Table of Contents
SERVICE PROVISION.
SUMMARY -
SERVICE PRINCIPLE:
SERVICE DEFINITION
SERVICE AVAILABILITY..
SERVICE TARGETS & LIMITS
SERVICE DEPENDENCIES & POST OFFICE LTD. RESPONSIBILI
BENCHMARKING METHOD.
INTRODUCTION
SCOPE..
REQUIREMENT
MEASUREMENT METHOD.........000:.0++
SYSTEM PERFORMANCE TARGETS.
EPOSS, OBCS AND APS TARGETS.
NBS TARGETS. ceseeeeeeeee
FUTURE DEVELOPMENTS.
APPENDIX A: ILLUSTRATION OF BENCHMARK CALCULATION
PARAMETERS
© 2004 Fujitsu Services Commercial in Confidence Page: 5 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
1.0 Service Provision
1.1. Summary
1.1.1 General
LAL The Transaction Benchmark Service provides a measure of Horizon system
performance for an agreed set of transactions (Section 2.0).
1.1.1.2 Measurement (or benchmark evaluation) is limited to the system components of
the nominated transactions, no account being taken of any manual or
human/computer interface components.
1.1.1.3. The service is provided for every Major Release.
1.2 Service Principles
1.2.1 General
1.2.1.1 Benchmark evaluation shall be carried out in accordance with the method
described in Section 2.0.
1.2.1.2 Benchmark evaluations shall be auditable at all stages of the process. Documents
to be retained include:
e The video recordings of the benchmark test transactions
« A spreadsheet comprising the processed benchmark results, the data captured
from the video recordings and the intermediate processing, including any
revision made thereto following review by Post Office Ltd.
e The test scripts
Such documents shall be retained for a period of seven years.
1.2.1.3. Benchmark evaluations shall be supported by this document and the performance
statement arising out of each evaluation, CS/PER/046, “COUNTER
TRANSACTION PERFORMANCE - MEASUREMENT AND RESULTS”.
1.2.1.4 I The contracted Target Times against which the benchmark evaluation shall be
measured are stated in Section 3.0.
1.2.1.5 CS/PER/046, “Counter Transaction Performance - Measurement and Results”,
shall be up-issued following every benchmark evaluation and shall include:
© 2004 Fujitsu Services Commercial in Confidence Page: 6 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
e Amendment(s) to the Target Time(s) arising out of agreed system changes
resulting from the implementation of a Change Request or Customer Work
Package including reasons for any such adjustment
e The new benchmark system component and aggregate times relating to the
release under examination
e The variance between the new benchmark times and the Target Times
e The variance between the new benchmark times and the previous benchmark
times
1.3 Service Definition
1.3.1
1.3.2
1.3.3
1.3.4
1.3.5
The data recording part of the benchmark evaluation is carried out prior to the
implementation of a Major Release. This is not to say that the software
implementation is dependant in any way upon the benchmarking exercise or the
results arising therefrom. It is the successful completion of end-to-end testing,
which is conducted under the control and management of Post Office Ltd, that
leads to the implementation of the software into the live environment (Ref.
VI/STR/064 “Testing Approach for the Horizon System” para. 4.4.3.6).
The service comprises:
¢ Benchmark evaluation of individual system component times within:
e Electronic Point of Sale Service (EPOSS)
e Automated Payments Service (APS)
e Order Book Control Service (OBCS)
e Benchmark evaluation of the transaction level aggregate of the counter system
component times for the Network Banking Service (NBS) transaction types.
The system component times are generated from the execution of a representative
set of transactions (Section 2.0 Table 1) on a test rig representing a real Post
Office branch. Normal counter operation is assumed
The outcomes of the benchmark evaluation are compared against the counter
system component contractual times (Section 3.0) and the results of the previous
benchmark evaluation. Any adverse variances are analysed and explained as far as
is practicable.
The initial benchmark evaluation results shall be presented to Post Office Ltd for
review within six weeks of the commencement of the implementation of the release
in question.
© 2004 Fujitsu Services Commercial in Confidence Page: 7 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
1.3.6
1.3.7
The final results are formally presented to Post Office Ltd. via CS/PER/046,
“Counter Transaction Performance — Measurement and Results”.
Regardless of whether the results of the benchmarking exercise have been
delivered, in the event that a Major Release is implemented and there is an obvious
degradation in performance in terms of transaction times and such degradation is
visible to the branch and reported to the Horizon System Helpdesk, the incident
identified by Fujitsu Services as causing the degradation will be resolved via the
established incident and problem management process. To the extent that any such
incident is caused directly by Fujitsu Services’ actions, the resolution of that
incident will be at Fujitsu Services’ cost.
1.4. Service Availability
1.4.1
1.4.2
1.4.3
The Transaction Benchmark Service shall be provided for every agreed Major
Release.
The rig exercise shall be scheduled in the release plan so that the data recording
part of the exercise is completed before implementation of the software release.
The Transaction Benchmark Service shall be provided upon request by Post Office
Ltd, (in a timescale to be agreed with Fujitsu Services) following the
implementation of an agreed change resulting from a Change Request or Customer
Work Package. Under such circumstances, the Transaction Benchmark Service
shall be chargeable.
Service Targets & Limits
There are no Service Level Targets or limits relating to the Transaction Benchmark
Service. For the avoidance of doubt, there are no Service Level Targets associated
with counter system component times.
There are system performance targets (Target Times) and these are stated in
Section 3.0.
The Target Times are subject to a 5% uplift with each Major Release to
accommodate the likely increase in system contention arising from increased
functionality and complexity inherent in the release.
The Target Times shall be changed whenever an agreed system change affects one
or more of the measured system components.
The Target Times may also be changed as the result of an agreement to change the
benchmarking method. Under such circumstances, credits gained under the
previous benchmarking method shall be carried forward in an equitable manner.
© 2004 Fujitsu Services Commercial in Confidence Page: 8 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
1.5.6 When, on completion of a benchmark evaluation, the counter system component
timings prove to be greater than the Target Times, this shall be deemed a
“Benchmark Time Discrepancy”.
1.5.7 In the event of Benchmark Time Discrepancy the following provisions shall apply:
1.5.7.1 The Post Office and Fujitsu Services shall agree (such agreement not to be
unreasonably withheld) a method of rectification, including a timetable, (a “
Rectification Plan”) for each Benchmark Time Discrepancy (which plan may
involve the agreement of an adjustment to the Target Times). If the
Rectification Plan requires changes to be made to the provisions of the
Agreement (including any CCD), such changes shall be made pursuant to the
Hard Change Control Procedure. Any changes to the Horizon Service
Infrastructure required to rectify a Benchmark Time Discrepancy shall be made
at no charge to the Post Office.
1.5.7.2 Each Benchmark Time Discrepancy shall not be a Default unless within 90 days
after the identification of that Benchmark Time Discrepancy the Parties have
failed to reach agreement upon a Rectification Plan and the Benchmark Time
Discrepancy has not been rectified by Fujitsu Services. In such a circumstance
that Benchmark Time Discrepancy shall become a Default upon expiry of that
90-day period.
1.5.7.3. A failure by Fujitsu Services to comply with a Rectification Plan shall be a Default.
1.5.7.4 This paragraph 1.5.7 shall be without prejudice to Post Office’s rights with respect
to release authorisation.
1.6 Service Dependencies & Post Office Ltd. Responsibilities
1.6.1 Fujitsu Service responsibilities
1.6.1.1 Fujitsu Services Customer Service shall ensure that the benchmarking rig activity is
inserted in the software release plan.
1.6.1.2 Fujitsu Services shall provide a test-rig configuration that will meet the scope of
the benchmark activities any Rectification Plan.
1.6.2 Post Office Ltd. responsibilities
1.6.2.1 Post Office Ltd. shall review and approve the Transaction Benchmark Service
results (reported in CS/PER/046, “Counter Transaction Performance -
Measurement and Results”), such approval not being unreasonably withheld.
1.6.2.2. Post Office Ltd. shall review and approve any revision to the contracted targets or
the latest benchmark, such approval not being unreasonably withheld
© 2004 Fujitsu Services Commercial in Confidence Page: 9 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
2.0 Benchmarking Method
2.1 Introduction
This section describes the method by which Fujitsu Services undertake benchmark
evaluation.
The Agreement, (ref. BP/CON/319, “Schedule 19: Operational Services”, effective as
of 1“ January 2003) introduces a new benchmarking philosophy whereby Fujitsu
Services are no longer responsible for measuring and reporting performance in terms
of end-to-end transactions.
Instead, performance is measured in terms of the system components only.
Then, for EPOSS, OBCS and APS, results are reported at individual system
component level and, for NBS, the system component times are aggregated and
reported by transaction type.
To reiterate, there is now no measurement or estimate made by Fujitsu Services of any
manual activity present within any of the transactions under scrutiny.
2.2 Scope
Counter performance is measured across the four counter services:
= APS
= OBCS
= EPOSS
= NBS
using a representative set of transaction types (see Table 1), executed on a
representative test rig.
Normal counter operation using best practice is assumed. In particular, fallback
scenarios are not considered.
With reference to Table 1, Post Office Ltd has accepted that the (system parts of the)
EPOSS process for selling a 2™ class stamp is the same as that for selling a 1“ class so
only one of the transaction types is scripted and executed during the benchmarking
data recording.
The same applies to: -
e POSS: Girobank Business Deposit - less than and more than £60
e EPOSS: National Savings - Deposit and Withdrawal
e APS: Magnetic Card Payments - with and without token
© 2004 Fujitsu Services Commercial in Confidence Page: 10 of 19
CONTRACT CONTROLLED
Fujitsu Services
Transaction Benchmark Service: Service Ref:
Description
Version:
Commercial in Confidence Date:
FUJ00232541
FUJ00232541
CS/SER/010
2.0
2™' July 2004
SERVICE
TRANSACTION TYPE
EPOSS:
1“ Class Stamp
2™' Class Stamp
Postage Stamps (non-book)
Stamp Book
Girobank Business Deposit (less than £60)
Girobank Business Deposit (more than £60)
Transcash Payment
Personal Giro Withdrawal
DSS Green Giro
Postal Order
Colour TV Licence
Motor Vehicle Licence (MVL)
National Savings Deposit (Ordinary account)
National Savings Withdrawal (Ordinary account)
OBCS:
Book Issue (Local)
Book Issue (Foreign)
Encashment (Local)
Encashment (Foreign)
APS:
Magnetic Card (Cash, No Token)
Magnetic Card (Cash, Token)
SMART Card (Cash)
NBS:
Cash Deposit
Cash Withdrawal (signature verification)
Cash Withdrawal (PIN verification)
Cash Withdrawal + Balance (signature verification)
Cash Withdrawal + Balance (PIN verification)
Cash Withdrawal to Limit (PIN verification)
Balance Enquiry (signature verification)
Balance Enquiry (PIN verification)
Change PIN
Table 1: Representative Transaction Types for Benchmarking Purposes
© 2004 Fujitsu Services
Commercial in Confidence Page: 11 of 19
CONTRACT CONTROLLED
Fujitsu Services
Transaction Benchmark Service: Service
Commercial in Confidence
Description
FUJ00232541
FUJ00232541
Ref: CS/SER/010
Version: 2.0
Date: 2™' July 2004
2.3 Requirement
Any transaction can be considered as comprising:
= manual components e.g. checking document validity, cash handling
= system components e.g. collection of data from input peripherals
(keyboard, scanner, PIN pad, magnetic card reader, SMART card reader,
touch screen), data processing, data communication, output of information
via screen and/or printer.
The system components that feature and are reported against in the representative
transaction types for EPOSS, OBCS and APS are summarised in Table 2, Table 3 and
Table 4.
System Component
Prod Prod Fee Amo Prod Settle I Mai
ucts ucts Seree I unt uct ment n
Othe Item ay Displ ay Displ Disp
r ay ay lay
I Class Stamp Yes Yes
Postage Stamps (Non-Books) Yes Yes Yes
Stamp Books (Non-Vended) Yes Yes Yes Yes
Girobank Business Deposit Yes Yes Yes Yes Yes Yes
Transcash Payment Yes Yes Yes Yes
Giro Withdrawal/Outpayment Yes Yes Yes Yes Yes Yes
BT Bill Payment Yes Yes Yes
DSS Green Giro Yes Yes Yes
Postal Orders (All Fee Bands) Yes Yes Yes Yes
Colour TV Licence Yes Yes
MVL Yes Yes Yes Yes Yes
NS Deposit (Ordinary Account) Yes Yes Yes Yes Yes Yes
Table 2: System Components arising from EPOSS transaction types
© 2004 Fujitsu Services
Commercial in Confidence
CONTRACT CONTROLLED
Page: 12 of 19
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
System Component
c
us
to
Pr m
oO od I Pr er
B uc I od Se I M
c ts uc w I ai
s Di I ts Iss e n
se I Mi I Q sp! I Di I we I M I se I M
an I Ik I ue ay I spl I s on I re I en
Ba I To I ry I se I - ay I In I et en I ou
r ke I st I an I of I - str I ar I Re I Di
Co I ns- I op I Fo I he I tte I uc I y/ I fre I spl
Transaction Type de I 1 s is Ir m t FI I sh I ay
OBCS Book Issues (Local) Yes Yes I Yes I Yes Yes
OBCS Book Issues (Foreign) I Y° Yes I Yes I Yes Yes
OBCS Encashment (Local) Yes I Yes Yes Yes I Yes
OBCS Encashment (Foreign) I YS I Yes I Yes I Yes Yes I Yes
Table 3: System Components arising from OBCS transaction types
System Component
sw I Ins I In Se I Pr I Te I Pr I Te I Cu
ipe ert pu ect int ar int ar sto
M s t ca I of I of I cu I of I me
ag M M sh fic I f sto I f r
net I A on Pa I e Pr I me I Pr I Se
ic R eta I ym I Re I int I r int I rv
Ca I T ry ent I cei I Re I Re I Re I e
rd Ca I A pt po I cei I po I Se
rd mo -1 rt pt rt ree
un -1 I -2 I -2 I o
t Re
fre
Transaction Type sh
Cash, No Tokens, Magnetic Yes Yes Yes I Yes I Yes I Yes I Yes
Card
Cash, SMART Cards Yes I Yes I Yes I Yes I Yes I Yes I Yes I Yes
Table 4: System Components ari
g from APS transaction types
For EPOSS, OBCS and APS, Fujitsu Services are required to measure the time taken
by the system components and report performance achieved for the system
components averaged across all transaction types that use that system component.
© 2004 Fujitsu Services
Commercial in Confidence
CONTRACT CONTROLLED
Page: 13 of 19
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
2.4
2.4.1
2.4.2
2.4.2.1
For NBS, only the aggregate of the system components for each transaction type is
reported.
The results are then compared against:
= the Target Time
= the results achieved in the previous benchmarking exercise.
Measurement Method
Data Capture
Each transaction type is decomposed into a series of clearly identifiable system events,
generally defining the start and end points of the system components of the
transaction. These events could be, for example, the receipt of data from a peripheral,
a screen change or output to the printer.
The sample transaction types for each service are executed on a test rig configured to
represent three types of counter; a single counter branch, the slave of a two-counter
branch and the gateway of a two-counter branch'. Each transaction is executed four
times on each counter configuration.
The process is recorded on videotape. The videotapes are then processed to expand
the time frame by 25, so that each frame lasting 0.04 seconds in real time lasts 1
second when replayed, and the time and frame count superimposed.
The event-to-event times are then read from the expanded video. The maximum
precision to which the event times can be read is 0.04 seconds.
NBS transactions are slightly different in that the authorisation part of the transaction,
though technically a system component, is outside of Fujitsu Service’s control, hence
responsibility. Further there is no visible evidence of exactly when the start and end
events (the Request and the Authorisation) occur. Instead the duration of this
component (the [R]-to-[A] time) is recorded as a system message that can be
downloaded after the data recording is completed.
Data Processing
The data recording exercise provides twelve values per system component per
transaction type. To reduce the risk of skewing the results, the best and worst values
for each system component are disregarded leaving ten values.
EPOSS, OBCS and APS
For EPOSS, OBCS and APS transaction types, the total sample size for each system
component is given by:
10*Nrr
where Nrris the number of Transaction Types using that system component.
" CS/TSC/003, “Benchmarking Scripts”, specifies the steps to be followed for each transaction and states the system events that form the start
and end of each system component.
© 2004 Fujitsu Services Commercial in Confidence Page: 14 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
2.4.1.2
3.0
3.1
3.1.1
The system component benchmark time is then calculated as the average of all those
samples. Appendix A: illustrates this point using a sample of data from the CSR+
results for OBCS.
NBS
For the NBS transaction types, the benchmark time is calculated as the sum of
averaged system components comprising each transaction type.
System Performance Targets
The change from transaction level reporting to system component level reporting
necessitated the generation of a new set of contractual targets.
For EPOSS, OBCS and APS services, the targets were calculated by extracting the
system component elements of the transaction target times used when benchmarking
CSR+ (amended, where necessary, to reflect product changes implemented since
CSR+).
For NBS, being a new type of service, completely new targets were generated using
data from a variety of sources. NBS targets were originally stated (as “initial” targets)
in BP/SPE/035, “NBS Definition”, and later ratified (as contractual targets) in
CS/PER/044, “NBS Benchmark Counter Transaction Times for Release BI3”.
All the targets are restated here for completeness.
EPOSS, OBCS and APS Targets
The calculation of the new, system component, Target Times comprised three stages:
= Carrying forward of any credit/debit achieved at CSR+ (at transaction type level)
= Addition of the credit/debit to the individual system component measurements
(within each transaction type) and averaging of the results at the system component
level within each product.
= Addition of a 5% uplift, to allow for increased system contention arising from the
increased functionality and complexity of the next software release’.
Carrying Forward Credit
In order to generate the new Target Times at System Component level, it was
necessary to carry forward any accumulated credits and debits gained under the
previous contract. This was achieved by dividing the credit/debit for the transaction
equitably among the system components according to how much the particular system
component contributed to the transaction time, relative to the other contributing
system components i.e.
Let Cr = Amount of credit gained at transaction level
? This is in accordance with a principle established in calculating target times for network banking
performance
© 2004 Fujitsu Services Commercial in Confidence Page: 15 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
Let Be = System component benchmark
Let Ba = Aggregate of system component times for the transaction in question
Then Tc = Target time at system component level
= Bc + (Bc/Ba) * Cr
3.1.2 Averaging
The calculation in section 3.1.1 was carried out for each of the system component
measurements selected under section 2.3. The results for each system component
were then averaged across all contributing transaction types.
3.1.3 Target Uplift
The results produced under section 3.1.2 were then uplifted by 5% to provide the new
system component level Target Times.
3.1.4 Target and Benchmark Values
The resulting contractual targets and the corresponding CSR+ benchmark are stated in
Table 5, Table 6 and Table 7. All times are quoted in seconds.
SYSTEM COMPONENT ELEMENTS I Target Time CSR+
Benchmark
Product Menu Display - Other 0.45 0.26
Product Menu Display - Item 0.46 0.20
Fee Screen Display - Item 0.92 0.33
[Amount Screen Display - Item 0.54 0.19
Product Sale Display - Item 0.85 0.26
Settlement Menu Display - Item 0.42 0.20
Main Menu Display 1.04 0.37
Table 5: EPOSS Target Times and CSR+ Benchmark
SYSTEM COMPONENT ELEMENTS I Target Time CSR+
Benchmark
Scan Bar Code 5.36 2.47
Milk Tokens 0.75 0.53
Query Stops 3.77 3.30
Scan Foils NIA N/A
Product Menu Display - Other 0.62 0.21
© 2004 Fujitsu Services Commercial in Confidence Page: 16 of 19
CONTRACT CONTROLLED
Fujitsu Services
FUJ00232541
FUJ00232541
Transaction Benchmark Service: Service Ref: CS/SER/010
Description Version: 2.0
Commercial in Confidence Date: 2"! July 2004
Product Menu Display - Item 0.36 0.12
Issues Instruct 0.28 0.09
Monetary / F1 0.24 0.17
Customer Serve Screen Refresh 0.66 0.48
Main Menu Display 0.86 0.29
Table 6: OBCS Target Times and CSR+ Benchmark
SYSTEM COMPONENT ELEMENTS I Target Time CSR+
Benchmark
Swipe Magnetic Card 1.25 1.06
Insert SMART Card 5.22 4.95
Input Monetary Amount 0.13 0.12
Cash Payment Select 4.44 4.21
Print Office Receipt -1 5.18 4.59
Tear Off Print Report - 1 5.26 4.65
Print Customer Receipt - 2 4.55 4.00
Tear Off Print Report - 2 0.16 0.14
Customer Serve Screen Refresh 0.58 0.52
Table 7: APS Target Times and CSR+ Benchmark
3.2 NBS Targets
The contractual targets presented in Table 8 are reproduced from CS/PER/044. The
units are seconds.
Transaction Type
Cash Deposit
Cash Withdrawal, Signature verification 15.16
Cash Withdrawal, PIN verification 15.61
Cash Withdrawal, with Balance, Signature verification 15.77
Cash Withdrawal, with Balance, PIN verification 16.22
Cash Withdrawal, with Limit, PIN verification 15.45
Balance Enquiry, Signature verification 14.59
Balance Enquiry, PIN verification 15.17
© 2004 Fujitsu Services Commercial in Confidence Page: 17 of 19
CONTRACT CONTRO!
ILLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
Change of PIN 15.25
Table 8: NBS Contractual Targets
© 2004 Fujitsu Services Commercial in Confidence Page: 18 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Transaction Benchmark Service: Service Ref: CS/SER/010
Description
Version: 2.0
Commercial in Confidence Date: 2™ July 2004
4.0 Future Developments
Fujitsu Services and Post Office Ltd. shall keep the benchmarking activity under
review with a view to:
reducing the time required for each benchmarking activity
allowing a greater scope and/or increased focus of analysis
reducing costs
reducing the risk of errors occurring in the measurements
enabling easier integration of benchmarking into the release programme
making benchmarking business specific rather than sampled product specific
providing simplification in both processes and application
facilitating variable remuneration opportunities for the Post Office Ltd
integrating benchmarking with performance and sizing process
Such review may cover, for example, an investigation into the appropriateness and
usefulness of system diagnostic and other software-based testing and measurement
tools.
© 2004 Fujitsu Services Commercial in Confidence Page: 19 of 19
CONTRACT CONTROLLED
FUJ00232541
FUJ00232541
Fujitsu Services Document Template (TITLE) Ref: CS/PER/046
Version: 1.0
SECURITY CLASSIFICATION Date: 16-MAY-2003
Appendix A: Illustration of Benchmark Calculation Parameters
Single Counter - Two Counter - Two Counter - Slave
Gateway PC Gateway PC PC
Transaction Type ISystem Component 1 2 3 4 5 6 7 8 9 10 11 12
issues (Local) Other products screen (Products Display -Other) 0.16 I 0.20 I 0.20 I 0.20 I 0.20] 0.20) 0.24) 0.20 I 0.20 10.24) 0.20 I 0.24
Display P&A menu (Products Display - Item) 0.16 0.16 I0.42) 0.20) 0.16 I 0.12) 0.12) 0.12 I 0.12 }0.12 0.12 I 0.12
Issues screen (Issues Instruct) 042)I 0.12 I 0.08 0.12 0.12]0.08) 0.08 I 0.08 I 0.12 I0.12 0.08 0.08
‘Scan Bar Code (OBCS Scan Bar Code) 3.24 1.28 0.76 I 0.56) 1.40 I 3.60 I 0.84 I 3.72 I 2.06 }4.28) 0.76 I 4.16
Main Menu Display 0.32I 0.28 0.28 0.28 0.32 I 0.28 I 0.28 I 0.28 I 0.27 10.28 0.32 0.28
issues (Foreign) IOther products screen (Products Display -Other) 0.20 I 0.20 I 0.20 I 0.247 0.20 10.20] 0.24) 0.20 I 0.20 0.20) 0.24 I 0.24
Display P&A menu (Products Display - Item) 0.12/0.08 0.08 0.12 0.12I0.08I 0.12 0.12 I 0.08 I0.08I 0.08 0.12
Issues screen (Issues Instruct) 1.88 0.08 0.12 0.08 I 0.08 I 0.08 I 0.08 I 0.08 I 0.08 I0.08 0.08 0.08
‘Scan Bar Code (OBCS Scan Bar Code) 1.88 2.60 3.68 2.80 5.52 I'8108/ 5.40 I 5.52) 5.32 I5.32 4.64 4.48
Main Menu Display 0.28 I 0.28 0.28 I 0:32 I 0.32 I 0.28 I 0.28 I 0.32 I 0.28 I0.28 0.28 I 0.24
lEncash (Local) ‘Scan Bar Code (OBCS Scan Bar Code) 5.72 0.72 1.24) 0.88 5.24]1.48I848)I 1.04 1.64 I0.88 0.92 1.48
Scan Foils 0.24 I 0.08 0.08 I 0.56 I 0.12] 0.16 I 0.12) 0.12 I 0.24 I0.12 0.16 I 0.12
500 enter / F1 (Monetary / F1) 0.20 0.20 0.16 0.16 0.16]0.20/0.12) 0.16 I 0.16 }0.16/ 0.24 0.16
Enter '1' milk token (Milk Tokens - 1) 0.52 0.52 0.44 I 0.48 1.08) 0.52I040I 1.20 0.88 I0.48 0.52) 0.56) <
Customer serve screen refresh (CS Screen Refresh) I 0.64 0.44 I 0.32) 0.40 0.44] 0.52 0.44 I 0.48 I 0.56 I0.60 0.32 0.44
lEncash (Foreign) IScan Bar Code (OBCS Scan Bar Code) 4.56 I 1.20 1.40 I 1.60 I 1.40] 0.88 [0/767 0.84 I 0.76 I 1.44) 0.76 I 1.00
Query stops screen (Query Stops) 3.20 I 0.96 2.48 4.00 3.08] 3.24 I 3.36I592) 3.48 I3.40 3.40 3.32
Scan Foils 012 0.16 0.12 0.16 0.12] 0.16 I 0.16 0.16 I 0.92 I0.12/ 0.16 0.12
500 enter / F1 (Monetary / F1) 0.20 0.20 I 0.42) 0.20 0.16] 0.16 I 0.16 I 0.12 0.16 I0.16 0.20 I 0.12
Enter '1' milk token (Milk Tokens - 1) 0.48 I 0.44 0.44 0.44 I0.72I 0.44 I 0.44I 0.36] 0.52 I0.48 0.44 0.40I<
Customer serve screen refresh (CS Screen Refresh) I 0.40 I 0.64 0.36 I 0.44 0.48 I 0.52 I 0.56 I 0.44 I 0.56 I0.64) 0128! 0.48
In calculating the system component benchmark time, the highest and lowest values in each set of twelve (shaded) are disregarded.
Then for each system component the average is taken from the remainder. For example, for the “Milk Tokens — 1” system component, the average of the
un-shaded values in the two indicated (<4) rows is calculated giving a value of 0.53 (to 2 decimal places).
© 2003 Fujitsu Services SECURITY CLASSIFICATION Page: 20 of 19
CONTRACT CONTROLLED