oO
FUJITSU
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
FUJ00097962
FUJ00097962
Document Title:
Document Reference:
Release:
Abstract:
Document Status:
Author & Dept:
External Distribution:
Security Risk
Assessment Confirmed
Approval Authorities:
Geoff Butts
lame Role
Programme Manager
TST/SOT/REP/0866
HNGX Release 1
APPROVED
Michael Welch
None
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
Date
See Dimensions for record
This is the Volume and Performance Test Report for Phase 2 of
testing conducted on the VOL rig for HNG-X Release 1.
YES security risk has been considered, see section 0.9 for details
Debbie Richardson
Testing Manager
See Dimensions for record
Lee Farman (POL, JTT)
POL Test Manager
See Dimensions for record
Dave Chapman
Capacity Management
See Dimensions for record
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Ref: TST/SOT/REP/0866
Version: 1.0
Date: 17-Sep-2010
Page No: 1 of 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
0 Document Control
0.1 Table of Contents
o DOCUMENT CONTROL
0.1 Table of Contents
0.2 Document History
0.3 Review Details ..
0.4 Associated Documents (Internal & External)
0.5 Abbrevi ns and Terms
0.6 Glossary.
0.7 Changes Expected .
0.8 Accuracy....
0.9 Security Risk Assessment
4 INTRODUCTION...
2
3 MANAGEMENT SUMMARY...
3.1 Test Approach..
Peak Clearance Cycle
Test Case Coverage
4 PRE-REQUISITES
4
1 Environment
4.2 Application Under Test
5 TEST CASE DETAILS...
5.4 Test Group XCAL —- HNG-X Calibration Test
5.4.4 HNG-X Calibration - Simple EPOSS.
5.4.2 HNG-X Calibration - Simple EPOSS for MVP
5.1.3 HNG-X Calibration — Basket Mix
5.2 Test Group C01 — HNG-X Counter Performance... 27
5.3 Test Group H11 — APOP...... 27
5.3.1 APOP Contracted Peak Hour / Peak 5 Minutes 27
5.4 Test Group H12 — DVLA... 28
5.4.1 DVLA Contracted / Design Limit Peak Hour / Peak 5 Minutes .
5.4.2 DVLA Stress Test. =
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSUSOTREPISES
Limited 2010 Version: = f.0.
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS PageNo: 2 0f 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 af
FUJITSU COMMERCIAL IN CONFIDENCE
5.5 Test Group H13 — PAI
5.1 PAF Stress Test
5.6 Test Group R01 — Branch Router 31
5.7 Test Group X01 - HNG-X Volume .. 31
5.7.1 HNG-X Volume (Monday; 31
in
.8 Test Group X05 — Debit Card .
~ 8.8.1 Debit Card Contracted / Design Limit Peak Hour / Peak 5 Minutes 33
Debit Card & ETU Stress Test 34
5.9 Test Group X06 — HNG-X CAPO. 36
5.9.1 HNG-X CAPO Contracted / Design Limit Peak Hour / Peak 5 Minute:
5.9.2 I HNG-X CAPO Stress Test...
0
5.10 Test Group X07 — LINK..
5.10.1 HNG-X LINK Stress Test
bi) Test Group X08 — HNG-X A&L
5.11.1 HNG-X A&L Stress Test ....
5.12 Test Group X09 - HNG-X ETU
5.12.1 HNG-X ETU Contracted / Desig
5.13 Test Group X10 — HNG-X T&T...
5.13.1 HNG-X T&T Contracted / Design Limit Peak Hour / Peak 5 Minutes ..
5.13. HNG-X T&T Stress Test ....
5.14 Test Group X11 - BAL & BRDB.
5.14.1 BAL & BRDB - Start of Day Profi
5.14.2 BAL & BRDB — Mid-Day Profile
5.14.3 BAL & BRDB — End of Day Profile
5.15 Test Group X12 - BAL & BRDB Stress Test
5.15.1 BAL & BRDB Stress Test...
5.16 Test Group X13 — Logon / Logoff..
5.16.1 Logon / Logoff Contracted / Design Limit Peak 5 Minutes
5.16.2 Logon / Logoff Stress Test.....
5.16.3 Maximum Concurrent Sessions
5.A7 Test Group X15 — Reports ..
5.17.1 Reports Contracted / Design Limit Peak 5 Minutes .
5.17.2 Reports Stress Test....
5.18 Test Group XCB — HNG-X Combined Banking
5.18.1 HNG-X Combined Banking Contracted / Design Limit Peak Hour / Peak 5 Minutes .
5.18.2 HNG-X Combined Banking Stress Test.
on
6 ADDITIONAL TESTING +95
Zz LESSONS LEARNED 95
8 RECOMMENDATIONS 96
9 CONCLUSIONS.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet TSUSOTREPIDEES
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 3 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 af
FUJITSU COMMERCIAL IN CONFIDENCE
0.2 Document History
Version No. I Date Summary of Changes and Reason for Issue Associated Change -
CP/PEAK/PPRR
Reference
04 24-AUG-2010 _I First draft for review
1.0 17-SEP-2010 _I Final version for Approval following review. Changes made:
3.1.1 —minor change to 3 paragraph
3.2 — updated table for Test Group X11 details
3.4 — updated to reflect current Peak status
4.1 —IP addresses and platform names removed from table
5.14.2.1.1 — corrected graphs legend
5.14.3 — updated result details following re-test after fix for
PC0201529 applied
5.18.2 —clarify wording of CAPO results
8. — updated wording for 2 bullet concerning ORM settings
0.3 Review Details
Review Comments by :
Review Comments to :
Mandatory Review
Role Name
Programme Manager Geoff Butts *
Testing Manager Debbie Richardson
POL Test Manager Lee Farman (POL, JTT) *
Capacity Management Dave Chapman
Role Name
Solution Design / Development ‘Andy Beardmore
Solution Design / Development ‘Andrew Aylward
Solution Design / Development ‘Andy Thomas
Solution Design / Development Martin Tonge
Solution Design / Development Peter Jobson
Solution Design / Development Graham Allen
Capacity Management Mitul Unalkat
Via TE Manager Mark Ascott
VOL Test Team Raj Kumar
VOL Test Team Richard Locke
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 UNCONTROLLED WHEN PRINTED OR bates t7-sep-2010
STORED OUTSIDE DIMENSIONS Page No: 4 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 af
FUJITSU COMMERCIAL IN CONFIDENCE
(*) = Reviewers that retumed comments
Issued for Information — Please restrict this
distribution list to a minimum
Position/Role Name
LST Manager Sheila Bamber
SV&l Manager Chris Maving
0.4 Associated Documents (Internal & External)
References to associated documents should normally be made to the latest approved version in
Dimensions; only refer to a specific version if necessary.
Reference Version Date Title Source
(1) RMG BU Generic Document Dimensions
PGM/DCM/TEM/0001 Template (This document)
(DO NOT REMOVE)
[2] HNG-X Volume and Performance Dimensions
TST/SOT/HTP/0003 High Level Test Plan
13] Horizon Capacity Management and I PVCS
PA/PER/033 Business Volumes
(4) ‘System Qualities Architecture Dimensions
ARC/PER/ARC/0001
(5) Volume and Performance Test Report I Dimensions
TST/SOT/REP/0008 ~ Phase 1
[6] HNGX Counter Performance High Dimensions
TST/SOT/HTP/0008 Level Test Platt
0.5 Abbreviations and Terms
AaL Alliance & Leicester
AG4 Acceptance Gateway 4
APOP Automated Payments-Out Processing
BAL Branch Access Layer
BDB / BRDB Branch Database
BRS Branch Support Database
CAPO Card Account at Post Office
DC Data Centre
bcs Debit Card Service
DRS Data Reconciliation Service
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet STS OUIRERIDERS
Limited 2010 UNCONTROLLED WHEN PRINTED OR bate tsep20t0
STORED OUTSIDE DIMENSIONS PageNo: 5 of 98
(ee)
FUJITSU
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
FUJ00097962
FUJ00097962
Abbreviation
Definiti
DVLA Driver Vehicle Licensing Authority
EPOSS Electronic Point Of Sale Service
ETU Electronic Top-Up
HLTP. High Level Test Plan
HYP High Volume Pilot
MvP Medium Volume Pilot
NPS Network banking Persistent Store
ORM Oracle Resource Manager
OSR Online Service Routing
PAF Postal Address File
Pcl The Payment Card Industry Data Security Standard
SVP Small Volume Pilot
T&T Track & Trace
TPS Transaction Processing System
tps transactions per second
TWS Tivoli Workload Scheduler
V&l Volume & Integrity
VOL Volume Test Rig Environment
0.6 Glossary
Term
Performance Testing or
Load Testing
Definition
Aseries of tests to measure the performance characteristics and behaviour
of specific components of the solution under load to demonstrate their ability
to achieve defined performance requirements. For example, to test that a
particular web service is capable of supporting the relevant contractual peak
rate of transactions per second (the defined load).
Volume Testing
A series of tests to demonstrate that the whole solution is able to support a
typical daily workload. For example, to test that a peak weekday transaction
profile can be successfully harvested and that all scheduled batch
processing can be completed within expected timescales.
0.7 Changes Expected
None
0.8 Accuracy
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
TST/SOT/REP/0866
1.0
17-Sep-2010
6 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
Fujitsu Services endeavours to ensure that the information contained in this document is correct but, whilst every
effort is made to ensure the accuracy of such information, it accepts no liability for any loss (however caused)
sustained as a result of any error or omission in the same.
0.9 Security Risk Assessment
Security risks have been assessed and it is considered that there are no security risks relating specifically to this
document.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 7 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
1. Introduction
The Volume and Performance High Level Test Plan [2] outlined a 2-phase approach using different test
environments:
>» Phase 1 on the VI (Live rig) in IRE11 prior to pilot.
> Phase 2 on the VOL rig in IRE19 post pilot
This is the Test Report for Volume and Performance testing only, as executed during Phase 2.
Phase 2 Test Cycles were periods of testing interspersed with rig build activity to introduce new Data
Centre and Counter releases.
This report provides a factual statement on the test status metrics at completion of Phase 2. The status of
particular Test Cases and any associated defects can be ascertained from the Quality Centre system.
2 Scope
This report is solely concerned with the Phase 2 Volume and Performance test activities and is not
reporting on the rig build activities that occurred before and during the period,.
The main objectives of Phase 2 Volume and Performance testing focussed on those test areas that must
be completed prior to full rollout. In essence, this concentrated on demonstrating performance
characteristics for a fully migrated branch estate at HNG-X with no Horizon elements. In addition, prior to
commencing the formal performance tests, HNG-X ‘calibration’ testing was performed so as to provide
comparison with the benchmark performance characteristics gathered in Phase 1 on the V&l rig. This
was to demonstrate that the VOL rig had live-like characteristics to support the validity of all testing
conducted during Phase 2.
The test outputs of Phase 2 testing supported:
o SVP (Small Volume Pilot)
o MVP (Medium Volume Pilot)
o —HVP (High Volume Pilot)
o AG4 (Acceptance Gateway 4)
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 8 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 af
FUJITSU COMMERCIAL IN CONFIDENCE
3 Management Summary
For completeness, the following table shows all tests originally identified in the HLTP.
Green = tests conducted during Phase 1 as detailed in the Phase 1 Test Report [5].
Yellow = tests conducted during Phase 2 on the VOL rig, which are the subject of this report
Pink = Not tested (as agreed on completion of Phase 1). These were either subsumed in other tests or
considered to be low risk.
Horizon volume - Monda
Horizon T&T to contractual limits
PAF to contractual & de
Horizon combined banking to contractual limits - non-PCI
Horizon PCI combined banking & Debit Card to contractual limits
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Lal TSUSOTREPIDEES
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 9 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 af
FUJITSU COMMERCIAL IN CONFIDENCE
HNG-X calibration test
3.1. Test Approach
There were 3 planned Cycles of test iteration for Phase 2, run out of IRE19 and BRAO1 within a
dedicated VOL test environment. SSC Workstations in Bracknell were used to connect to the Data Centre
and BRA01 platforms.
3.1.1 Cycle 1
Cycle 1 started on the Sth February 2010 and formally finished on 26th February 2010.
The original aim of Test Cycle 1 was to provide supporting evidence that from a performance perspective,
the HNGX Release 1 solution was capable of supporting Medium Volume Pilot (MVP) and High Volume
Pilot (HVP). Delays in starting Cycle 1 meant that the test outputs derived would only have real value in
informing Programme Management decision making concern HVP.
Test coverage during Cycle 1 was limited and a number of key issues were identified during the cycle.
Specific details of coverage and issues identified are documented in the separate Cycle 1 Test Report
(NB: This was an informal document that only reached draft status and is not registered in Dimensions. A
copy is available on request, though any important information of relevance is included in this report.).
Following review by the Test Management team, it was decided to curtail Cycle 1 in order to upgrade the
rig to Release 01.08.03.10A. The main reasons for curtailing the cycle was due the high number of
patches and fixes that had been applied to the Live estate and the difficulties of identifying performance
related hot fixes that had been applied to LST. The value being obtained from the test runs was
diminishing. In order to re-establish a clean and known position, the C1/C2 rig top up was brought
forward.
As a result of the solution issues identified by the key defects/peaks identified during cycle 1, the
conclusions drawn were:
. MVP would be able to proceed, albeit at reduced Branch numbers. This was because the
transaction level of the MVP branches would not exceed a transaction rate of 500 tps.
. HVP would be able to start as the solution would support up to 1500 Branches. However it
could not complete until fix resolutions to the key defects/peaks identified above had been
proven on VOL and LST and were available for release into the Live estate.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE tat TSUSOTREPIDEES
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 10 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
3.1.2 Cycle 2
Cycle 2 started on the 1t March 2010 and formally finished on 25" March 2010.
Following a live incident resulting in Red Alert status being raised, the attention of VOL testing was
switched to supporting the investigation and resolution of issues emanating from the Red Alert. In effect,
therefore, the planned VOL Cycle 2 testing was abandoned. No separate Cycle 2 Test Report was
produced.
3.1.3 Cycle3
Following conclusion of Red Alert testing, VOL Cycle 3 commenced on 18" June and formally finished on
28"" July.
In light of the abandonment of Cycle 2, the stress tests targeted for Cycle 3 were modified to demonstrate
conformance of contractual and design limit volumes as part of those tests, and a number of tests
previously completed successfully during Cycle 1 were also repeated in consideration of the significant
number of solution updates introduced since completion of Cycle 1.
A test review identified and prioritised the specific planned tests required to be completed to support
AG4. These were completed on 27" June 2010 to enable AG4 decision to start full live roll-out on 30!
June 2010.
Additional tests over and above those specified in the HLTP were also included in Cycle 3 following the
test review. These included
. 9 day soak test (running at Peak load for an extended number of days)
. test against Egenera recommended changes
. test against a single C-blade
. test with constrained DB Caching
. test with node failure and reintroduction of failed node during on-line day
. tests for various ORM configuration settings
Monitoring and measurement of Streams behaviour was also captured throughout Cycle 3 and Peak
clearance.
3.1.4 Peak Clearance Cycle
This Cycle focussed on specific testing of further Peak fix deliveries, and other additional tests requested
by Development and Capacity Management to help confirm understanding of system behaviour and
identify optimal performance settings. This Cycle also included regression testing of BRDB performance
following the introduction of new blade (v249) hardware.
3.2 Test Case Coverage
The following table provides an overview of the final test status at the end of Phase 2 for each Test Case
within those test groups highlighted in yellow in the table in section 3 above..
Test 7 Date DC Build
Group I Test Description Status I astrun I T°? I cTR Build
cot I Co1.MO1_HNG-X CAPO Balance Enquiry & Withdrawal vy Pass I 28/04/10 oroen
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/0866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS PageNo: 11 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 .
FUJITSU COMMERCIAL IN CONFIDENCE
Feet, I Test Description Stats I igstron I Test I creeuia
C01.M02_HNG-X A&L Business Deposit vy PASS 28/04/10 peed
C01.M03_HNG-X LINK Cash Withdrawal v Pass I 28/04/10 peed
CO1.M04_HNG-X Purchase E-Top Up y PASS I 28/04/10 ones 1
C01.MO5_HNG-X Purchase Phone Card vy PASS I 20/06/10 cr6 1
CO1.MO6_HNG-X Tax a vehicle for 12 months no VAT v Pass I 28/04/10 giz
CO1.MO7_HNG-X SORN with vehicle address changes v PASS I 28/04/10 20
C01.M08_HNG-X Purchase postal order v PASS 20/06/10 Hi os re
C01.MO9_HNG-X Encash postal order ¥ PASS I 21/06/10 at 2 6
CO1.M10_HNG-X Payment by Debit-Credit card vy PASS I 21/06/10 Berry
CO1.M11_HNG-X Address Lookup vy PASS I 28/04/10 user
CO1.M12_HNG-X Purchase Saving Stamp v PASS I 06/04/10 aust
CO1.M13_HNG-X Purchase 1st Class Stamps v PASS I 21/06/10 err)
C01.M14_HNG-X Purchase Pure Air Sure Europe Stationary v PASS 06/04/10 pe
C01.M15_HNG-X Purchase Scratchcard ¥ PASS I 06/04/10 eres0
CO1.M16_HNG-X Purchase Electricity Tokens v Pass I 28/06/10 Mer
CO1.M17_HNG-X Purchase Premium Bonds vy Pass I 19/02/10 eran
CO1.M18_HNG-X Purchase 1 Day Fishing License v Pass I 19/02/10 rine
CO1.M19_HNG-X Post Special Delivery Small Letter v PASS I 28/04/10 panned
C01.M20_HNG-X Post Large Letter v PASS 28/04/10 Cae
CO1.M21_HNG-X Post Letter USA Weighting vPass I 19/02/10 Cs
C01.M22_HNG-X Packet y PASS =I 28/04/10 ore 1
CO1.M23_HNG-X Bulk Post v PASS I 28/06/10 Herr
CO1.M24_HNG-X Pay Bil v PASS I 21/06/10 0: os o
CO1.M25_HNG-X Freedom Passes vPAss I 17/06/10 01.22.00
C01.M26_HNG-X Moneygram v PASS 06/04/10 et
C01.M27_HNG-X Travel Insurance vPass I 23/06/10 Henry
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet TSTISOTIREP/0866
Limited 2010 UNCONTROLLED WHEN PRINTED OR bates t7-sep-2010
STORED OUTSIDE DIMENSIONS Page No: 12 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 .
FUJITSU COMMERCIAL IN CONFIDENCE
Test oh Date DC Build
Group I Test Description status I cae I Testi I Po buie
C01.M28_HNG-X Purchase Foreign Currency vy PASS 06/04/10 pe
CO1.M29_HNG-X Purchase Foreign Currency in Travellers Cheques I v PASS I 21/06/10 at oo i
C01.M30_HNG-X Purchase Pre-Order Foreign Currency yvPass I 17/06/10 ps ae 00
C01.M31_HNG-X Foreign Currency Buy Back vpass I 19/02/10 pa ae
C01.M32_HNG-X Remittance Acceptance vPass I 15/06/10 01.22.00
01.22.00
01.22.00
C01.M33_HNG-X Remittance Rejection v Pass I 28/06/10 ery
C01.M34_HNG-X Transaction Correction for a Remittance Shortage I Redundant I _N/A : :
C01.M35_HNG-X Spoiled Postal Orders v PASS 22/06/10 ot ie te]
P 01.22.00
CO1.M36_HNG-X Stock Unit Reports, v Pass I 22/06/10 ser
01.22.00
CO1.M37_HNG-X Office Reports v Pass I 15/06/10 Heres
01.20.00
C01.M38_HNG-X Logon vpass I 28/04/10 aoe
01.20.00
C01.M39_HNG-X Logoff y¥ PASS 28/04/10 CTR25.11
01.22.00
CO1.M40_HNG-X Help v Pass I 22/06/10 Hcy
APOP Contracted Peak H
Ht —— vpass I 140310] os2 I 21.08.03
APOP Contracted Peak 5 Minutes CTR25.07
DVLA Contracted Peak Hour
ig [DVLA Gontracted Peak 5 Minutes veass I 131310 I 079 I 01:08:03
DVLA Design Limit Peak Hour CTR25.07
DVLA Design Limit Peak § Minutes
01.22.00
H12b DVLA stress test vy PASS 02/07/10 205 01.22.00
H13b_ I PAF stress test (covered by X10b) vpass I 29/06/10] 201 a o °
01.08.03
R01,M01 Branch Router Performance 1 Counter vPAss I 16/03/10 Petey
07.08.03
RO1.M02 Branch Router Performance 5 Counter ¥ PASS 18/03/10 CTR25.07
Rot 01.08.03
RO1.M03 Branch Router Performance 10 Counter vy PASS 18/03/10 CTR25.07
01.08.03
R01,M04 Branch Router Performance 20 Counter v Pass I 23/03/10 een
01.08.03
x01 HNG-X Volume Monday Schedule vy PASS 19/03/10 097 CTR25.07
HNG-X Debit Card Contracted Peak Hour
x05 LHNGX Debit Gard Contracted Peak 5 Minutes veass I 250210 I oss I 01.07.04
HNG-X Debit Card Design Limit Peak Hour CTR24.05
HING-X Debit Card Design Limit Peak 5 Minutes
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: = f.0.
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 13 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 .
FUJITSU COMMERCIAL IN CONFIDENCE
ae Test Description status I Dae I testi I Do Build
X05b HNG-X Debit Card stress test (combined with X09b) vy PASS 01/07/10 204 a os ty
HNG-X CAPO Contracted Peak Hour
os LHNG-X CAPO Contracted Peak 5 Minutes veass I 220210 I os: I 01:07:04
HNG-X CAPO Design Limit Peak Hour CTR24.05
HNG-X CAPO Design Limit Peak 5 Minutes
X06b I HNG-X CAPO stress test (covered by XCBb)
X07b I HNG-X Link stress test (covered by XCBb) vpass I s0/06/10 I 202 I 01-2200
X085_I HNG-X AaL stress test (covered by XCBb)
HNG-X ETU Contracted Peak Hour (covered by X09b)
x09 LHNG-XETU Contracted Peak 5 Minutes (covered by X0%b) ot2200
HNG-X ETU Design Limit Peak Hour (covered by X09b) vpass I o1o7/10 I 204 I 612200
HNG-X ETU Design Limit Peak 5 Minutes (covered by XO9b)
X096_I HNG-X ETU stress test (combined with X05b)
HNG-X T&T Contracted Peak Hour (covered by X10b)
xio _ LHNG-XT8T Contracted Peak 5 Minutes (covered by X10b) ot2200
HNG-X T&T Design Limit Peak Hour (covered by X10b) vpass I 200610 I 201 I 12500
HNG-X T&T Design Limit Peak 5 Minutes (covered by X10b)
X10b HNG-X T&T stress test
BAL and BRDB Contracted Peak Hour - Start of Day mix
BAL and BRDB Contracted Peak 5 Minutes - Start of Day mix 01.22.00
- y¥ PASS 13/08/10 247
BAL and BRDB Design Limit Peak Hour - Start of Day mix 01.22.00
BAL and BRDB Design Limit Peak 5 Minutes - Start of Day mix
BAL and BRDB Contracted Peak Hour - Mid-Day mix
x11 [BAL and BROB Contracted Peak 5 Minutes - Mid-Day mix veass I tama I o4q I 01:22:00
BAL and BRDB Design Limit Peak Hour - Mid-Day mix 01.22.00
BAL and BRDB Design Limit Peak 5 Minutes - Mid-Day mix
BAL and BRDB Contracted Peak Hour - End of Day mix
BAL and BRDB Contracted Peak 5 Minutes - End of Day mix vpass I 2410 I 254 I 91:22.00
BAL and BRDB Design Limit Peak Hour - End of Day mix 01.22.00
BAL and BRDB Design Limit Peak 5 Minutes - End of Day mix
X12 I BAL and BROB stress test vpass I 17/08/10 I 251 I 01-2200
x1g I Losen /logoff Contracted Peak 5 Minutes veass I toma} ore I 01.0803
Logon / logoff Design Limit Peak 5 Minutes CTR25.07
X13b Logon / logoff stress test v PASS 01/07/10 203 ot e rs
X13¢ Maximum concurrent session v PASS 24/03/10 145 oo
HNG-X Reports Contracted Peak 5 Minutes (covered by X15)
x15 198- I 01.22.00
HNG-X Reports Design Limit Peak 5 Minutes (covered by X15b) vpass I 27et0I “Ry I 012500
X15b_I HNG-X Reports stress test
HNG-X Combined Banking Contracted Peak Hour (see XCBb)
HNG-X Combined Banking Contracted Peak 5 Minutes (see XCBb) 01.22.00
XcB - vpass I 30/6/10 I 202
HNG-X Combined Banking Design Limit Peak Hour (see XCBb) 01.22.00
HNG-X Combined Banking Design Limit Peak 5 Minutes (see XCBb)
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
i Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 14 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
ae Test Description status I Dae I testi I Po buid
XCBb I HNG-X Combined Banking stress test
XCAL I HNG-X Calibration tests v PASS 07/02/10 on pa
{ Note: I BAL / OSR release was updated independently to the rest of the Data Centre Build stated above as follows:
19/05/10 BAL/OSR version 01.22.01.01
16/06/10 BAL/OSR version 01.23.01.01
23/08/10 BAL/OSR version 01.23.02.01
Full results for each of the above tests are detailed in Section 5 of this report.
Additional testing over and above those specified in the HLTP was also conducted in support of specific
activities or enhancements. See section 6 for details.
3.3 Defects
3.3.1 Defects Raised
A total of 135 Volume and Performance related defects were raised up to the conclusion of Phase 2
testing. The following table shows the priority and status breakdown for these defects as at the date of
this report.
Current Priority I Open I Awaiting I Informally I Closed Totals
PEAK (QC) Test Tested
A (4— Very High) 1 47 48
B (3 - High) 1 4 65 70
C (2—Medium 3 4 9 16
D (1 -Low) 1 1
Totals 4 9 122 135
Open Calls are with a development or support team and fix not yet available
Awaiting Test Calls have been returned to V&I awaiting retest or some other action
Informally Tested Calls have been successfully tested with an informal fix applied to VOL,
but formal delivery not yet available to allow call closure.
Closed Calls are closed, no further actions required
3.3.2 Open Defects
The following table shows the priority and call summary for defects that currently remain Open, including
those for which an informal fix has been successfully tested (highlighted in green).
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 15 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
Defect
Peak Pty I Summary er Assigned Team Updated
13463 I Pco196s44 I c NOL penped up Logons - Clone of 20100326 I Crypto-Dev 20100809
VOL: Counter Performance Stats log doesn't
log logins and logoffs 20100326
VOL C2 - Enable formal delivery of Oracle
13481 I PC0196695 I B I Listener Timeout Value changed in Live and 20100329 I RelMngmntForum 20100709
VOL on 28/03/10
13464 I PCO196549 I C xCtr_SUC_GDC 20100804
43487 I pcotgeste I p I VOL-PFEMIS Payment file creation reports I 9199339
> TSC-Dev 20100903
errors suggesting negative amounts.
VOL - BRDB nodes being evicted and
13858 I PCO19725 I B I rebooting overnight during schedule runs.
20100414 I Infra Networks 20100708
VOL - Inconsistent counter processing
13629 I PC0198381 I C I (according to counter stats) across many 20100430 I xCTR_NT_GDC I 20100804
transaction types
VOL - When a parse error occurs with a
13879 I PC0200282 I C I message in the OSR the log file fails log the I 20100616 I QC Interface 20100915,
invalid message
13881 I PC0200284 I C I VOL - Primary Key violations invoked 20100616 I xOSR_GDC 20100809
13913 I PC0200531 I B_I Live - Stats gathering improvements 20100622 I RelMngmntForum I 20100908
VOL - High CPU usage results when running
14016 I Pco201529 I A I Group Reports 20100714 I QC Interface 20100915,
VOL - OSR issues response before DB
commit is complete
VOL - APSC2085 abends with column
overflow message
14101 I PC0202763 I C 20100810 I xOSR_GDC 20100909
14111 I PC0202857 I C 20100812 I RelMngmntForum 20100812
Live - OSR Minimum and Maximum
14112 I Pco202881 I B I Connection Settings once BDB established I 20100812 I xOSR_GDC 20100901
on v249 pBlades
The above open defects have the following target release as at the date of this report.
Release
No Target 02.10.03 02.11.00 02.12.00 02.13.00 03.00.00
PC0196544 Pc0200282 I PC0202857 I PC0200284
PC0196549 PC0201529
PC0196695,
Pco196818
PC0197725
PC0198381
PC0200531
PC0202763
fix unavailable for VOL testing
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 16 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
3.4 Conclusion
A significant number of performance and capacity related issues have been identified and resolved
throughout VOL testing. Additional testing over and above that specified in the HLTP has also been
conducted to support further ‘tuning’ of performance characteristics. As at completion of Cycle 3, testing
has demonstrated ability of the solution to support full contractual and design limit workloads for each of
the service types under test. Stress testing has also identified the headroom capacity to support future
growth in business volumes.
Some Peaks remain open which would deliver further performance improvements. Whilst desirable, they
are not deemed critical to be able to support current live volumes.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 17 of 98
oO
FUJITSU
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 af
COMMERCIAL IN CONFIDENCE
4 Pre-Requisites
4.1. Environment
The platform instances comprising the VOL rig and their location are shown in the following table.
Platform Type
No. Instances
Comments
IRE11
IRE19 I BRAO1
ACD
C)
ARC
AUW
AWS
BAL
BDB
2} oI of of oI
a] ©} ny} of ny]
I Oo] of sf oJ o
Node 1 upgraded to v249 on 30/07/10
Nods 2, 3, 4 upgraded to v249 on 12/08/10
BDS
I
4 pBlades required to build all nodes of the cluster, 3
instances are then turned off and only 1 pBlade required for
default operation of BDS.
BMX
Instance 2 Offline, no pBlade resource allocated
BPL
Instance 2 Offline, no pBlade resource allocated.
BRS
BSL
BSM
BSS
BSW
BWS
Instance 2 Offline, no pBlade resource allocated
cDG
cDS
DAT
DCM
DEA
Instance 2 Offline, no pBlade resource allocated
DNP
DNS
BOX
OWS:
Instance 2 Offline, no pBlade resource allocated.
DXxC
EAS
EDS
EES
EFS
EMD
EMM
EMS
EPM
ERP
EST
EUI
I 2} 2} 2} oI oJ of Oo} Oo} oJ of of oO} Oo} of of Of Of I Of Of Of 4 4) Of =I oO} I
a] a} a} a} a} a) a) no} a} a] a) sp nd el a) a} of a} a] of a} sf of of =] of a} nu]
I 2} 2} of of Of Of Of of of Of Of 0} Oo} I Of Of Of OJ I Of Of Of DJ DJ Of OJ OJ
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
Ref: TST/SOT/REP/0866
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 18 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 af
FUJITSU COMMERCIAL IN CONFIDENCE
FLG
FRG
HWS:
KMN
KSN
LLX
LRC
LRG
MSH
MWS
NAA
NAC
NAL
NDM
NPS:
NRS
OES
Ows
PLG
PRG
Instance 2 Offline, no pBlade resource allocated
Instance 2 Offline, no pBlade resource allocated
Instance 2 Offline, no pBlade resource allocated.
Instance 2 Offline, no pBlade resource allocated
RDM
‘SPN
SRG
SSC
SSN
ssw
SYS
TRE
VDX
VPX
VSD
I 2} 2} 2} of of of of OJ of of Of Of OJ Of Of 0} Oo} I OJ Of 0} OD I of Of oO} I J Of Of DJ Oo
=I cof =} a] a) 3) of 3} a] of 3} of ma} of =} of 3} a} no) =] af sf a] no) So} lf of ol of =} ns] of =
I o} o} a} of of af of o} I of 5] 0} 4) of of 0} 0} I of of 0} oI I of of 4} NJ I Of of a] 0
‘vsd001 hosts Irvpeds007
vsd002 hosts Irvpauw001
vsd003 hosts vdx, vpx and domain controllers
vsd004 hosts Irvpfrg001, Irvpprg001, Irvpsrg001
VSH 0) 41 C) hosts tinvma101
XCS: C) 0 5 xcs001 - DVLASIM
xcs002 - MASim
xcs003 - TNTSim
xcs004 - ETUSim + tools
xes005 - LINK Multiplexors
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet TSUSOTREPIDEES
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 19 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
4.2 Application Under Test
Component Description
TWS Schedule Tivoli Workload Scheduler for the complete Batch
Schedule pertaining to all platforms contained on VOL
HNGX Counters / Branch Application used to complete business transactions with
Router Data Centre systems
ACEBlade, BAL/OSRs Solution Load Balancer and Branch Access layer
systems
BDB, BRS, Streams Branch Database server and Branch Support Database
server and replication application
NPS, APOP. Network Persistent Storage and APOP Database
CAPO, A&L, Link Main Banking Transaction Applications
DCS, ETU, PAF, DVLA, T&T Key Web Service Applications
The relevant build release for the test environment was upgraded in accordance with the following
timescales.
VOL rig Build Status
Main OSR build
Date Data Centre I Counter Build _I (where different)
02-Feb-2010 01.07.04 CTR24.03
16-Feb-2010 CTR24.05
02-Mar-2010 I __01.08.03 CTR25.07
27-Mar-2010 CTR25.10
20-Apr-2010 I __01.20.00 CTR25.11
19-May-2010 I 04.22.00 01.22.00 04.22.01.04
18-Jun-2010 01.23.01.00
23-Aug-2010 01.23.02.00
Critical Peak fixes were also intercepted and applied to the relevant platforms throughout the duration of
testing, whether through formal baseline delivery or by informal change where appropriate.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 20 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
5 Test Case Details
This section provides a more detailed view of each Test Case executed for each of the Test Groups
during Phase 2. For many of the tests, several iterations were run during the different test cycles
following application of fixes for defects raised, or other upgrades to system components (software and
hardware). The results shown in this section are for the latest iteration of each test case concerned.
However, for the main BRDB load tests, results are shown for execution of these tests against both old
and new blade hardware for comparison.
The results detailed are the ‘front-end’ view in terms of transaction rates, response times and error rates.
Performance metrics of the key system components were also gathered from analysis of Athene data for
each test. The detailed information and graphical view of Athene data is not shown in this report due to
the sheer volume of information. Instead, a simple summary statement of any key observations is made
where appropriate. Full Athene analysis is available for any given test on request.
5.1 Test Group XCAL — HNG-X Calibration Test
The objective of the ‘calibration’ test was to run a reduced basket mix of HNG-X transactions, but at
reasonable volumes, in order to capture performance metrics of the V&l rig using Metron Athene Data
gathered for the period of the test run. The same tests would then be performed on the VOL rig to help
understand any environment differences and how this could affect interpretation of performance results
The following sections detail the results of the calibration tests as performed on the VOL rig.
An initial test was performed for EPOSS transactions only, before later attempting a more representative
basket mix.
5.1.1. HNG-X Calibration - Simple EPOSS
This test comprised of simple EPOSS transactions (selling a stamp) only and the conditions under which
the test was run are the same as during Phase 1 on V&l as follows:
There were 16 OSR instances in total, with 2 OSRs running on each of servers BALOO1 —
BALOO08.
There were 4 BRDB nodes available.
The bandwidth was throttled to 256Kbps from the load generator to the ACE Blade.
All messages were compressed.
The 400 branches used in the test were spread across the fad hashes to generate an even load
across the 4 BRDB Nodes.
The incremental rate of transaction volumes during the run was as follows:
150 tps (across 150 branches) for 15 minutes, then increased to
200 tps (across 200 branches) for further 15 minutes, then increased to
300 tps (across 300 branches) for further 15 minutes, then increased to
400 tps (across 400 branches) until the end of the test.
Throughout this test run, transaction response times were good and similar to those witnessed during
Phase 1, with just 29 transaction failures out of a total of 1.6 million
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 21 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
Average Settlement Response Time - Settlements per Second
g
£
5
. 5
8
&
f
«
e
H
& 0010 © 00:20» 00:30 0040 OSD ON:00Ot10t:20
Elapsed scenario time hh:mm
[IM Settlement response tine D] Settlements per second J
Transaction Summary
1,800,000
3,800,000 4
2,200,000
(E00 000
“00,000
2 600,000
2 400,000
° § € 8 @ = ®
i z F e = g
2 a, ey 3 o 2
5, a Es ES 5
° § o eo e
- 8
ScaleMeasurement
1 Pass
1 Fail
Color
Utilisation data gathered by Athene was as expected in comparison to the same test run during Phase 1
Test Status: Passed
Date of Final Run: 2" February 2010 16:18 to 18:04
Build Release: 01.07.04 / CTR24.03
TestID: 002
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 22 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
5.1.2 HNG-X Calibration - Simple EPOSS for MVP
An additional Simple EPOSS test was also run at the request of the Performance Architect. This was not
previously run during Phase 1 on V&l, but was considered useful for gaining an indication of likely
performance during live medium pilot.
The environment conditions under which the test was conducted was the same as in section 4.1.1 with
incremental rate of transaction volumes as follows:
100 tps (across 275 branches) for 30 minutes, then increased to
300 tps (across 550 branches) for further 30 minutes, then increased to
650 tps (across 1000 branches) until the end of the test.
Again, throughout this test run, transaction response times were good apart from a brief ‘spike’ 1 hour 15
minutes into the run during which 165 transaction timeouts occurred out of a total of 1.9 million. This
demonstrates acceptable performance at a rate of 300tps expected during pilot, but a potential issue at
higher rates which will be further understood during the formal performance testing at peak rates.
Average Settlement Response Time - Settlement per Second
Settlements per second
Settlement Response Time (seconds)
00:05 00:10 00:15 00:20 00:25 00:30 00:35 00.40 00:45 00:80 00:55 01:00 01:05 01:10 O1:15 01:20 01:25 01:30,
Elapsed scenario time hh:rnm.
[Mi Settiements response time [] Settlements per second J
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
i Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 23 of 98
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
Fe)
FUJITSU
FUJ00097962
FUJ00097962
Transaction Summary
2,200,000
2,000,000
#,600,000
“,800,000
$400,000
2,200,000
*$,000,000
3.200.000
600,000
= 400,000 (m0) (i000) (a0) (a0)
200,000
0 A a a
(i000)
P
‘egon
‘gon
I_Logott
C_O10_FirstLe
C_030_GetMemos
€_040_CountSusy
C0501:
‘C_020_Secondl
0020_010_STMPSettle
Color ScaleMeasurement
me:
Test Status: Passed
Date of Final Run: 2" February 2010 18:30 to 20:03
Build Release: 01.07.04 / CTR24.03
TestID: 003
5.1.3 HNG-X Calibration - Basket Mix
The following calibration tests were run under similar conditions as during Phase 1 as follows:
There were 16 OSR instances in total, with 2 OSRs running on each of servers BALOO1 —
BALOO8.
There were 4 BRDB nodes available.
The bandwidth was set to maximum between the load generator and the ACE Blade.
All messages were compressed.
All branches used in the test were spread across the fad hashes to generate an even load across
the 4 BRDB Nodes.
All banking transactions were submitted for agent hash value 1, meaning that only the capo_b,
al_b and link_b services were handling transaction load.
An outline of each iteration is shown in the following sections. In each table, the value shown for Txn/sec
is the total number of transaction records per second for each service type, including settlement
transaction (i.e. the number of transactions addressed to the BAL layer). The values shown for Auth/sec
is the number of actual requests that would be forwarded to each relevant Authorisation service. The end
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref
Limited 2010 Version
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
24 of 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
[oe]
FUJITSU COMMERCIAL IN CONFIDENCE
column shows the median number of transactions per second successfully authorised for banking, debit
card and ETU services.
During each of the following tests, there was a gradual decline in successful authorisations for debit card,
with dropped connections between the OSR services and the DCS agent being observed (Peak Ref
PC0194218). Similar failures were also observed with connections to other banking services. Further
analysis of the DCS agent showed an issue with saturated CPU, with a single thread (NX_NQ_DCS/47)
running at 100% CPU. This is the same behaviour as was observed during similar tests for Phase 1. In
effect, therefore, this verified that the VOL rig was comparable to the V&l rig as a starting position and a
suitable environment against which to diagnose the issues observed and to enable Phase 2 testing.
5.1.3.1 Mix of EPOSS/Banking/Debit/E-TopUps/DVLA/Login/Logoffs @ 100tps
Function Users Txn/sec Authisec I Pass/sec
0700_CAPO_CAL 37 37 13 12.33
0800_AL_CAL 3 3 1 1
0900_LINK_CAL 7 8 3 24
1000_EPOSS_CAL 23 24 24
1100_DEBIT_CAL 7 6 3 0.6
7200_LoginLogoff_CAL 8 4 1
7300_DVLA_CAL 7 ir 6
1400_ETU_CAL 8 8 3 2.667
100 101 54
Test Status: Completed
Date of Final Run: 5'" February 2010 07:55 to 09:04
Build Release: 01.07.04 / CTR24.03
TestID: 004
5.1.3.2 I Mix of EPOSS/Banking/Debit/E-TopUps/DVLA/Login/Logoffs @ 200tps
Function Users Txn/sec I Auth/sec I Pass/sec
0700_CAPO_CAL 75 75 25 25.016
0800_AL_CAL 7 7 2.33 2.336
0900_LINK_CAL 13 14 4.68 3.953
1000_EPOSS_CAL 47 49 49
1100_DEBIT_CAL 13 13 4.17 2.258
7200_LoginLogoff_CAL 17 9 174
7300_DVLA_CAL 73 20 70.14
1400_ETU_CAL 15 15 5 5.008
200 202 103
Test Status: Completed
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 UNCONTROLLED WHEN PRINTED OR bates ' 7-sep-201 0
STORED OUTSIDE DIMENSIONS Page No: 25 of 98
oO
FUJITSU
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
COMMERCIAL IN CONFIDENCE
Date of Final Run: 5" February 2010 11:19 to 12:26
Build Release: 01.07.04 / CTR24.03
TestID: 008
5.1.3.3 Mix of EPOSS/Banking/Debit/E-TopUps/DVLA/Login/Logoffs @ 300tps
Function Users Txn/sec I Auth/sec I Pass/sec
0700_CAPO_CAL 112 112 37.33 34
0800_AL_CAL 1 1 3.66 2.933
0900_LINK_CAL 19 21 6.84 5.4
1000_EPOSS_CAL 71 75 75
1100_DEBIT_CAL 19 19 6.16 0.667
1200_LoginLogoff_CAL 26 13 4.45
1300_DVLA_CAL 19 30 9.88
1400_ETU_CAL 23 23 7.66 6.933
300 303 151
Test Status: Completed
Date of Final Run: 7" February 2010 14:02 to 15:09
Build Release: 01.07.04 / CTR24.03
TestID: 014
5.1.3.4 Mix of EPOSS/Banking/Debit/E-TopUps/DVLA/Login/Logoffs @ 400tps
Function Users Txn/sec I Auth/sec I Pass/sec
0700_CAPO_CAL 150 150 50 31.4
0800_AL_CAL 15 15 5 3.133
0900_LINK_CAL 25 27 9 4.867
1000_EPOSS_CAL 95 100 100
1100_DEBIT_CAL 25 22 8 0.013
1200_LoginLogoff_CAL 35 30 6
1300_DVLA_CAL 25 26 13
1400_ETU_CAL 30 30 10 10
400 400 201
Test Status: Completed
Date of Final Run: 7" February 2010 15:36 to 16:39
Build Release: 01.07.04./ CTR24.03
TestID: 015
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 UNCONTROLLED WHEN PRINTED OR bates {7-sep-201 0
STORED OUTSIDE DIMENSIONS
Page No: 26 of 98
Fe)
FUJITSU
FU,
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
COMMERCIAL IN CONFIDENCE
FUJ00097962
IJ00097962
5.2 Test Group C01 — HNG-X Counter Performance
Specific Counter (C01) and Branch Router (RO1) performance tests are the subject of a separate HLTP
[6]. Accordingly, whilst the test result status is shown in section 3.2 of this document for completeness,
detailed test results are to be documented in a separate test report aligned to the Counter Performance
HLTP.
5.3 Test Group H11 — APOP
5.3.1 APOP Contracted Peak Hour / Peak 5 Minutes
Target Transaction Rates:
Contractual Peak Hour 33tps
Contractual Peak 5 Minutes 39tps
APOP transactions were ramped up to a rate of 33tps for 1 hour and then further increased to 39tps for
an additional 5 minutes. Average response times were good with no transaction failures.
Average Transaction Response Time - Transactions per Second
Title: Average Transaction Response Time - Th ser Second Graph date in Excel format [%%
Greph Type: The 4
Graph ty Average Transaction Response Time
Graph 2+ as ond par Second
Gramularity: condy
Average Transaction Response Time . Transactions per Second
F
H
2
g
= 5
z =
5
&
z
H
e
i
Fi 127000 «123000 = 124000 #30000
Time hh:hhime
fe 7.0800 _010_APCPAGA D500 Ba0 AFCA scar “0800 D3 APOPO Sele
T0500 _o10_aPcemuth Pass 500 090_APOP Satie Pass I
Test Status: Passed
Date of Final Run: 14" March 2010 12:15 to 13:35
Build Release: 01.08.03 / CTR25.07
Specific Configuration Details:
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet STS OUIRERIDERS
i Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 27 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
4 x BRDB nodes
Swappiness=0 on BRDB
160SRS (BAL002 — BAL009) min connection value set to 5 and max connection value set to 20
SGA @ 16G
PGA @ 4G
TestID: 082
5.4 Test Group H12 -DVLA
5.4.1. DVLA Contracted / Design Limit Peak Hour / Peak 5 Minutes
Target Transaction Rates:
Contractual Peak Hour 29tps
Contractual Peak 5 Minutes 30tps
Design Limit Peak Hour 34tps
Design Limit Peak 5 Minutes 36tps
DVLA transactions were ramped up to a rate of 29tps for 1 hour, then further increased to 30tps for an
additional 5 minutes, then further increased to 34tps for another hour and finally further increased to
36tps for an additional 5 minutes. This test was performed alongside an additional load of APOP
transactions at a rate of 28tps for 2 hours and then increased to 34tps.
Average response times were good with no transaction failures
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 28 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 e
FUJITSU COMMERCIAL IN CONFIDENCE
DVLA Transaction Response Time - Transactions per Second
Ties DVLA Transaction Rerponte Tine - Transadions per Second Graph data in Exeal formar [SS
‘Graph Type: Te a
Graph ts Average Transaction Response Time
Graph 2: Trans jong per Second
Gremdeetiy: 43 Seconds
uae DVLA Transaction Response Time - Transactions per Second
eeesig
Number of Teanaecteona
dwerage Response Time (seconds)
Y7on00 174500 179200 174500 sRODGD 18:15:00 INRA Transaction Responee Time Transactions par Ser:
Absolute Time hivhh:mm,
TEVT0050 O10 DVLAAN —-MNET_0050_020_DVLASettio MN T_0050_010_DVLAAuh Dass
100501000 _DVLASeme: Pass
Test Status: Passed
Date of Final Run: 13" March 2010 16:50 to 19:08
Build Release: 01.08.03 / CTR25.07
Specific Configuration Details:
4 x BRDB nodes
Swappiness=0 on BRDB
16OSRS (BAL002 — BALO09) min connection value set to 5 and max connection value set to 20
SGA @ 16G
PGA @4G
TestID: 079
5.4.2. DVLA Stress Test
The objective of this test was to take the DVLA service beyond Design Limits up to a rate at which either
a) the service is no longer able to support the transaction load and thus identifying the maximum
headroom capacity limit, or
b) anotional transaction rate (as agreed with Capacity Management) such that, if achieved, it would
demonstrate sufficient headroom capacity for future growth without needing to find the absolute
breaking point
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet TSUSOTREPIDEES
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 29 of 98
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
FUJ00097962
FUJ00097962
The target transaction rate for this test was 72tps and this was achieved with capacity to spare..
Historical Peak Design Limit
5
Service Hour I 5mins I Hour I 5mins I Hour I mins I Per sec peak I Sustained HIR %
not
DVLA spec'd 34, 29 30 34 36 36 72 I 100.00%
HR% above is the sustained tps rate % increase over and above the Design Limit Peak 5 min rate.
Transaction response times were good and there were no failures
DVLA Average Transaction Response Time - Transactions per Second
Bou
2 12
g 08
g o6
B os
B 02
:
Absolute Time hh:hh:mm.
0
13:00:00 13:05:00 13:10:00 13:15:00 13.20.00 13:25:00 13:30:00 13:35:00 13,402.00 13:45:00 13:50:00 13:55:00 14:00:00
TT _0050_010_DVLAAuh
Gl T_0050_020_DVLASettle-Pass
Ti T_0050_020_DVLASettie
(IT _0050_010_ DVLA Pass
Number of Transactions
The following table gives a summary of the response times (min/max/avg/90th percentile) and pass / fail
rates for each transaction type.
Std. 90
Transaction Name Minimum I Average I Maximum I Deviation I percent I P@SS Fail
C_010_FirstLogon 0.125 0.134 0.459 0.026 0.149 1,000 0
C_020_SecondLogon 0.13 0.161 0.652 0.031 0.189 1,000 0
C_030_GetMemos 0.07 0.073 0.122 0.003 0.08 1,000 0
C_040_CountSusp 0.072 0.075 0.525 0.02 0.079 1,000 0
C_050_Logoff 0.086 0.088 0.52 0.015 0.09 1,000 0
T_0050_010_DVLAAuth 0.378 0.865 4.621 0.257 1.248 _[ 187,309 0
T_0050_020_DviASettle I 0.108 0.115 2.863 0.057 0.13 [I 187,309 0
Test Status: Passed
Date of Final Run: 2° July 2010 12:58 to 14:03
Build Release: 01.22.00 / OSRs at 01.23.01.00
TestID: 205
5.5 Test Group H13 — PAF
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: = f.0.
" UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 30 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
5.5.1 PAF Stress Test
This test was covered by Test Group X10 (Track and Trace Stress test) as each transaction request for
that test required a PAF lookup to be performed. The test was performed against a single PAF Web
Server (as opposed to 2 servers which would share the load in live).
The objective of this test was to take the PAF service beyond Design Limits up to a rate at which either
a) the service is no longer able to support the transaction load and thus identifying the maximum.
headroom capacity limit, or
b) anotional transaction rate (as agreed with Capacity Management) such that, if achieved, it would
demonstrate sufficient headroom capacity for future growth without needing to find the absolute
breaking point
The target transaction rate for this test was 100tps and this was achieved with capacity to spare. The
PAF Web Server CPU peaked at 15% utilisation during the test.
Historical
Peak Design Limit
5 5 Per sec
Service I Hour min I Hour I 5 min I Hour I min peak I Sustained I H/IR%
PAF 18 18 45 46 54 55 I not specified 100 I 81.82%
HR% above is the sustained tps rate % increase over and above the Design Limit Peak 5 min rate.
Transaction response times were good and there were no failures
See the test results for X10 T&T Stress Test for a detailed breakdown of transaction response times.
Test Status: Passed
Date of Final Run: 29" June 2010 15:22 to 17:22
Build Release: 01.22.00 / OSRs at 01.23.01.00
TestID: 201
5.6 Test Group R01 —- Branch Router
Specific Counter (C01) and Branch Router (RO1) performance tests are the subject of a separate HLTP
[6]. Accordingly, whilst the test result status is shown in section 3.2 of this document for completeness,
detailed test results are to be documented in a separate test report aligned to the Counter Performance
HLTP.
5.7 Test Group X01 — HNG-X Volume
5.7.1. HNG-X Volume (Monday)
Volumes:
The volumes for transactions processed during this test in comparison to contractual and design limits is
as follows:
Contractual I Design Limit Txn Countintestrun I _ % of contractual % of Design
EPOSS 8602518 10323021 13938369 162.03% 135.02%
APS 3031573 3637887 1717674 56.66% 47.22%
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 31 of 98
FUJ00097962
FUJ00097962
(-e) VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
Contractual Design Limit Txn Count in test run % of contractual % of Design
NBS. 2553600 I 3064320 2319810 90.84% 75.70%
A&L 135983
Link 367816
CAPO 1816011
DCS 288425 346110 272111 94.34% 78.62%
ETU 121200 145440 95997 79.21% 66.00%
DVLA 737774 885329 533698 72.34% 60.28%
PAF 1100788 1320945, 255832 23.24% 19.37%
Settlement 9565842 11479010 7565296 79.09% 65.91%
T&T 213000 255600 (255832 120.11% 100.09%
Bureau 76775 92130 95999 125.04% 104.20%
Some of the business transactions are routed to additional online services. This applies to NBS, DCS,
ETU, DVLA, PAF and APOP,; it does not apply to EPOSS, APS and Settlement which just update the
Branch Database.
These online business services are subject to additional overall contracted volumes and design limits for
the combined total over any contractual period (this is less than the sum of all the services). These are
shown below
Contractual I Design Limit I Txn Count in test run % of contractual I % of Design
3556145 4267374 3477448 97.79% 81.49%
Online
Services
The total Online Business Services generated during the test was 97.79% of the Contractual Peak Day
volume.
TOTAL of ALL transactions for TRADING_DATE 19th March: 18,439,960
Batch Processing:
There were a number of issues encountered during the test run which caused the overall batch suite to
overrun. However, investigations confirmed that these issues were rig specific. Allowing for the
constraints of the test environment causing certain schedules to take a long time to complete (due to
compressing large files to a ‘dummy’ NAS share set-up on VOL), it was determined that the overnight
batch would complete in good time in the live environment before start of the next trading day. All of the
actual data processing schedules completed in good time.,
Asummary of processing times for some of the critical overnight batch processes is as follows:
Aggregation Schedules:
co BRDB_NCU_TXN_AGG
Started at 01:15 and completed in 10 minutes
o BRDB_CU_TXN_AGG
Started at 01:23 and completed in 11 minutes
Data Feed Schedules:
o BRDB_EPOS_TO_TPS
Started at 19:10 and completed in 13 minutes
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 32 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
o BRDB_DCS_TO_TPS
Started at 19:20 and completed in 9 minutes
o BRDB_DCS_TO_DRS
Started at 20:30 and completed in 10 minutes
o BRDB_NWB_TO_TPS
Started at 19:28 and completed in 12 minutes
o BRDB_NWB_TO_DRS
Started at 19:46 and completed in 61 minutes. Of this total time, the BRDBX003 jobs performing
the actual data feed completed in 25 minutes (rest of schedule is made up of check jobs and
waits)
o DRS_NWB_C12_PARS (DRSC301 jobs) and TES_CO_HARV (TESC310 jobs)
From the data fed to DRS by BRDB_NWB_TO_DRS schedule, the parsers began processing the
data at 19:46 and completed by 21:23.
From the data processed by the DRSC301 parsers, the TES_CO_HARV jobs began processing
the data at 19:48 and completed by 23:24
Test Status: Passed
Date of Final Run: 19!" March 2010
Build Release: 01.08.03 / CTR25.07
TestID: 097
5.8 Test Group X05 — Debit Card
5.8.1 Debit Card Contracted / Design Limit Peak Hour / Peak 5
Minutes
Target Transaction Rates:
Contractual Peak Hour 29tps
Contractual Peak 5 Minutes 29tps
Design Limit Peak Hour 57tps
Design Limit Peak 5 Minutes. 57tps
Target rates for DCS as specified in ARC/PER/ARC/0001 have since been increased. The target rates
above are as specified by Capacity Management Team.
DCS transactions were ramped up to a rate of 29tps for 1 hour, then further increased to 57tps for an
additional 5 minutes.
Average response times were good with no transaction failures
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 33 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
Average Transaction Response Time - Transactions per Second
3
5
5
5
2
12:30.00
12:50,00 13.00.00
Absolute Time hh:hh:mm
Average Response Time (seconds)
12.4000 13:10:00 13:20,00
13:30:00
T_0300_010_DBTAuth TB T_0300_020_DBTRecov
Gi 1_0300_020_DBTRecov:Pass [i] T_0300_030_DBTSettle-Pass
TB T_0300_030_DaTSettle
TT _0300_010_DBTAUthPass:
Test Status: Passed
Date of Final Run: 25'" February 2010 12:20 to 13:34
Build Release: 01.07.04 / CTR24.05
TestID: 055
5.8.2 Debit Card & ETU Stress Test
The Debit Card Stress Test and ETU Stress Test were combined as a single run.
The objective of this test was to take the Debit Card and ETU service beyond Design Limits up to a rate
at which either
a)
headroom capacity limit, or
b)
the service is no longer able to support the transaction load and thus identifying the maximum
a notional transaction rate (as agreed with Capacity Management) such that, if achieved, it would
demonstrate sufficient headroom capacity for future growth without needing to find the absolute
breaking point
The target transaction rates for this test was 100tps for Debit Card and 20tps for ETU and this was
achieved with capacity to spare. The transaction rates were run at the contractual and design limit rates
on the way to achieving the end target rate.
Historical Peak Design Limit
Service Hour I 5min I Hour I 5min I Hour I 5min I Per sec peak I Sustained I _HIR%
Debit Card _I not spec'd 22 29 20I s7I 57 57 100 I 75.44%
E-Topup not spec'd 5 5 5I 6 6 8 20 I 233.33%
Total not spec'd 27 34 34 63 63 65 120 90.48%
HR% above is the sustained tps rate % increase over and above the Design Limit Peak 5 min rate.
Transaction response times were good and there were no failures
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref
Limited 2010 Version:
UNCONTROLLED WHEN PRINTED OR Date:
STORED OUTSIDE DIMENSIONS Page No:
1.0
17-Sep-2010
34 of 98
TST/SOT/REP/0866
Fe)
FUJITSU
COMMERCIAL IN CONFIDENCE
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJ00097962
FUJ00097962
DCS Average Transaction Response Time - Transactions per Second
)
Average Response Time (second:
Absolute Time hh:hh:mm
14:25:00 14:30:00 14:35:00 14:40:00 14:45:00 14:50.00 14:55:00 15:00:00 15:05:00 15:10:00 15:15:00 15.20.00 15:25:00
Number of Transactions
(T_0300_010_DeTAun
(IT _0300_020_DBTRecov
1_0300_020_D8TRecov Pass Gi T_0300_{
_DBTSettle Pass
TB T_0300_030_DeTSettie
(BT _0300_010_DeTAUnPass
ETS Average Transaction Response Time - Transactions per Second
Average Response Time (seconds)
Absolute Time hh:hh:mm
oO
14:25:00 14:30:00 14:35:00 14:40:00 14:45:00 14:50:00 14°55:00 15:00:00 15:05:00 15:10:00 15:15:00 15:20:00 15:25:00
160
140
420
100
80
80
40
20
Number of Transactions
[BT _0180_010_ETAAuth
T_0180_020_ETARecov
MM T_0160_020 ETARecow-Pass [ll T_0180_030_ETASettle Pass
TT _0160_030_ETASettle
BT _0180_010_ETAAuthPass
The following table gives a summary of the response times (min/max/avg/90th percentile) and pass / fail
rates for each transaction type.
90
Transaction Name Minimum I Average Maximum Std. Deviation I Percent Pass Fail
C_010_FirstLogon 0.125 0.137 0.251 0.012 0.15 1000 0
C_020_SecondLogon 0.137 0.175 0.331 0.023 0.21 1000 0
C_030_GetMemos 0.07 0.075 0.2 0.006 0.089 1000 0
C_040_CountSusp 0.072 0.075 0.127 0.004 0.09 1000 0
C_050_Logoff 0.083 0.088 0.336 0.01 0.099 1000 0
T_0180_010_ETAAuth 0.132 0.266 1.114 0.048 0.32 48806 0
T_0180_020_ETARecov I 0.099 0.104 1,007 0.014 0.12 48806 0
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
i Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 35 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
90
Transaction Name Minimum I Average I Maximum Std. Deviation I Percent Pass Fail
T_0180_030_ETASettle 0.11 0.114 1.531 0.017 0.13 48806 0
T_0300_010_DBTAuth 0.166 0.31 1.468 0.06 0.391 260102 i)
T_0300_020_DBTRecov 0.104 0.108 1.556 0.014 0.12 260102 0
T_0300_030_DBTSettle 0.124 0.128 1.311 0.015 0.14 260102 0
Test Status: Passed
Date of Final Run: 1% July 2010 14:23 to 15:26
Build Release: 01.22.00 / OSRs at 01.23.01.00
TestID: 204
5.9 Test Group X06 — HNG-X CAPO
5.9.1 HNG-X CAPO Contracted / Design Limit Peak Hour / Peak 5
Minutes
Target Transaction Rates:
Contractual Peak Hour
Contractual Peak 5 Minutes
Design Limit Peak Hour
Design Limit Peak 5 Minutes — 250tps.
134tps
160ps
161tps
CAPO transactions were ramped up to a rate of 134tps for 1 hour, then further increased to 160tps for an
additional 5 minutes, then further increased to 161tps for another hour and finally further increased to
250tps for an additional 3 minutes (the test was terminated slightly early).
Average response times were good with no transaction failures
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Ref:
Version:
Date:
Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
36 of 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
[oe]
FUJITSU COMMERCIAL IN CONFIDENCE
Avorage Transaction Response Time . Transactions per Second
08
or
2 8
& os
04
i 03
2 02
B91
$ 0
a 16:60:00 7:00:00 17:10:0017:20:00 17:90:01 7:40.00 17:50:0018 00:00 18:10:0018:20:00 18:20:0018 40.0018: 60.00
Absolute Time hhehhimnm
(2 T_0010_010_CaPOAMth (@ T_0010_020_CAPORecoy [ T_0010_090_CAPOSettie
T0010, 010. CAPOAUN Pass Ml T_0010.020.CAPORecov Pass El T_0010.030 CAPOSettie: Pans
Test Status: Passed
Date of Final Run: 22™ February 2010 16:41 to 18:55
Build Release: 01.07.04 / CTR24.05
TestID: 051
5.9.2 HNG-X CAPO Stress Test
This test is covered by the Combined Banking Stress Test. See section 5.18.2 for detailed test results.
5.10 Test Group X07 — LINK
5.10.1 HNG-X LINK Stress Test
This test is covered by the Combined Banking Stress Test. See section 5.18.2 for detailed test results.
5.11 Test Group X08 — HNG-X A&L
5.11.1 HNG-X A&L Stress Test
This test is covered by the Combined Banking Stress Test. See section 5.18.2 for detailed test results.
Number of Trareactens
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet TSUSOTREPIDEES
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 37 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
5.12 Test Group X09 — HNG-X ETU
5.12.1 HNG-X ETU Contracted / Design Limit Peak Hour / Peak 5
Minutes / Stress Test
These tests are covered by the combined Debit Card and ETU stress test. See section 5.8.2 for detailed
test results.
5.13 Test Group X10 — HNG-X T&T
5.13.1 HNG-X T&T Contracted / Design Limit Peak Hour / Peak 5
Minutes
These tests are covered by the T&T stress test. See section 5.13.2 below for detailed test results.
5.13.2 HNG-X T&T Stress Test
The objective of this test was to take the T&T service beyond Design Limits up to a rate at which either
a) the service is no longer able to support the transaction load and thus identifying the maximum
headroom capacity limit, or
b) anotional transaction rate (as agreed with Capacity Management) such that, if achieved, it would
demonstrate sufficient headroom capacity for future growth without needing to find the absolute
breaking point
Target rates for T&T are not specified in ARC/PER/ARC/0001 and the rates detailed in PA/PER/033 have
since been increased. The target rates below are as specified by Capacity Management Team.
Historical Peak Design Limit
5
Service Hour 5Smin I Hour I 5min I Hour I min I Persec peak I Sustained I H/R%
not not
TaT* spec'd spec'd 43 43 52 52 52 100 I 92.31%
HR% above is the sustained tps rate % increase over and above the Design Limit Peak 5 min rate
The transaction rates went through the contractual (43tps) and design limits (52tps) up to a maximum of
100 Track and Trace transactions per second. As part of the script, PAF lookups and settlements were
carried out as well; so taking into account all parts of the different transactions (XML) taking place, an
overall hit rate of 400tps was achieved which equates to 100 T&T transactions per second..
The target rates were achieved with excellent response times and no failures. The graph below shows
the overall transactions rates achieved and the response times.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet TSUSOTREPIDEES
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 38 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
Average Transaction Response Time - Total Transactions per Second
§
2
2
2
Average Response Time (seconds)
15:30:00 15:40:00 1:50:00 18:00:00 18:10:00 18:20:00 18:30:00 16:40:00 16:50:00 17:00:00 17:10:00 17:20.00
Absolute Time hh:hh:mm
TET _0080_010_LasTPOInd Ml T_0080_020_ Pat TIT_0080_030_PSBarCode TE] T_0080_040_POSRVSettle
Pass
The table below is a summary of the test; showing the response time (min/avg/max/90th percentile) for
each transaction type and also the number of transactions which ‘passed & ‘failed’.
Transaction Name Minimum I Average I Maximum Std. 90 Pass I Fail
Deviation I Percent
C_010_FirstLogon 0.126 0.138 0.443 0.025 0.149 I 1000 0
C_020_SecondLogon 0.137 0.174 0.688 0.037 0.21 1000 0
C_030_GetMemos 0.07 0.073 0.152 0.004 0.08 1000 oO
C_040_CountSusp 0.072 0.075 0.526 0.02 0.078 I 1000 0°
C_050_Logoff 0.084 0.087 0.311 0.01 0.09 1000 oO
'T_0080_010_LASTPOInd 0.072 0.136 3.005 0.021 0.14 [457419I 0
T_0080_020_Paf 0.152 0.164 1.93 0.018 0.18 [4574i9[ 0
/T_0080_030_PSBarCode 0.07 0.074 1.931 0.012 0.08 [457419] 0
{T_0080_040_POSRVSettle 0.128 0.132 2.987 0.032 0.14 [457419] 0
The BDB to NPS feed kept up to date with the transaction load. The last batch of transactions was
passed to NPS at approximately 17:25. The non-online part of processing, by the TTEDGInt agents
running on the FLG platform, continued clearance of the backlog to complete update status by 19:17
The table below gives a view of transaction status in the NPS database at various points. It includes a
few snapshots over the last 20 minutes of the test to demonstrate the total transaction rate from the BDB
feed keeping up, and then continued snapshots until all processing by the FLG agents is completed.
feed of
ID ACTIONED 16:59 17:04 17:09 17:20 pre 17:30 17:55 18:25 18:55 19:10 19:17
5 N 29052 34114-35606 «45284 NPS. 40957 28314 13033 0 0 0
5 Y 52952 56138 58772 64616 finished =» 70581 83224 «98505 111538 111538 111538
6 N 37690 44269 46520 59308 PDK 55413 42768 «27488 «11805 «3793 t)
6 Y 50908 5408956729 62586 totaltxns 68590 81235 96515 112198 120210 124003,
7_N 3352439722 41797 53889 _—MOlonger 50028 37365-22014 ~—6336 0) ()
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREP (D866
Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
TORED OUTSIDE DIMENSIONS Page No: 39 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
7 Y 55580 58767 61405 67260 I inefeasing 73241 5904 101255 116933 123269 123269
8 oN 30906 36659-38518 + 49717 45661 33025 177962112 0 t)
ay 54513 57694 60332-66170 72124 84760 99989 115673 117785 117785
TOTAL TXNS 345125 381452 399679 468830 476595 476595 476595 476595 476595 476595
ID MAX(INSERT_TSMP) MAX(UPDATE_TSMP)
5 29/06/2010 17:25 29/06/2010 18:50
6 29/06/2010 17:25 29/06/2010 19:17
7 29/06/2010 17:25 29/06/2010 19:07
8 29/06/2010 17:25 29/06/2010 18:59
Test Status: Passed
Date of Final Run: 29"" June 2010 15:22 to 17:22
Build Release: 01.22.00 / OSRs at 01.23.01.00
TestID: 201
5.14 Test Group X11 —- BAL & BRDB
Earlier iterations of BAL/BRDB testing at design limit transaction load identified a number of concerns
with regard to system utilisation and memory usage. Whilst the target transaction rates were achievable,
response times were sometimes erratic and performance characteristics of the BRDB were at
undesirable levels, which in turn had a knock-on effect at the BAL layer. A number or patches and
performance improvements were implemented, but upgrade of the pblades hosting the BRDB nodes to
v249 blades demonstrated a considerable improvement such that performance concerns have been
eliminated.
The transaction rates and response times shown in the following sections are those for the iteration
performed against all 4 BRDB nodes operating on the new v249 blades. However, to demonstrate the
significant improvement in system performance, graphs have been included to illustrate system metrics
gathered by Athene for comparative tests run against the old blades and against just node 1 on a new
v249 blade.
5.14.1. BAL & BRDB - Start of Day Profile
This test had a high transaction rate with ‘counter’ logins/logoffs occurring throughout the run. The
transaction types included EPOSS, Banking, Debit Card, E-TopUps, DVLA, PAF, Postal Orders, Bureau
and Postal Services. The scenario had a target rate of 1074 transactions per second on average which
equates to 498 settlements per second.
Basket/Script Hits/sec I Settle/sec
0010_BKT01_CPO_WDL(2) 240 80
0011_BKTO1_AL_WDL(2) 15 5
0012_BKTO1_LNK_WDL(2) 45 15
0020_BKT02_STP_SLE(3) 150 150
0050_BKTO5_DVL_TDC(3) 30 15
0060_BKT06_STP_SLE(3) 50 50
0070_BKTO7_PTL_SRV(5) 90 30
0080_BKTO8_PTL_SRV(8) 20 5
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
i Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 40 of 98
Fe)
FUJITSU
FU,
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
COMMERCIAL IN CONFIDENCE
FUJ00097962
IJ00097962
Basket/Script Hits/sec I Settle/sec
0140_BKT14_EPO_CCQ(3) 80 80
0170_BKT17_BIL_CSH(3) 25 25
0180_BKT18_ETA_CSH(2) 9 3
0190_BKT19_BDC_CSH(3) 25 25
0300_BKT30_EPO_DBT(3) 30 10
0400_BKT40_LIN_LOF 250 50
0500_BKT50_POS_ORD(4) 15 5
1074 498
The test iteration run against all 4 BRDB nodes on new v249 blades produced excellent results, both in
terms of throughput and response times. The graph below shows the total transaction rates for the test.
Throughput was steady with 3 minor dips.
Total Transactions per Second
Tite:
Current Results:
Falters:
Group By:
Granularity:
‘Total Transoctioné pir booed Graph data in Excel format P'S}
2\Rerults\20100823_c3_x22_batbdb_start_day_247
AAHNGHAPA 4
\20100813_.
10 Seconds
Total Transactions per Second
Showing - Average
Total Number of Transactions
o8B8888R 88 BS
14:95:00 11:40:00 11:45:00 11:50:00 11:55:00 120200 12.0500 121000 1215:00 122000 122500 1290.00 12:35:00 12.40.00
Absolute Time hh:bh:mm
The response times were steady with a few spikes coinciding with dip in throughput as shown in the
merged graph below.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 41 of 98
oO
FUJITSU
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
FUJ00097962
FUJ00097962
Total Transactions per Second - Average Transaction Response Time
We: Total Transactions per Second - Average Transaction Response Time
Graph Type: wl
Graph 1 Total Transactions per Second
Graph 21 Average Transaction Response Time
Granularity: 10 Seconds
Total Transactions per Second . Average Transaction Response Time
11:30:00 11:40:00
Color Graph
MRI F024 Tirrerctinns net Second
The table below is a summary of the test; showing the response time (min/avg/max/90th percentile) for
ScaleMeasurement
1 Pass.
11:80:00 120000
1210.00
Absolute Time hh:hh:mm
Graphs Mi
BB
S67 784
122000
11346.
123000 12:40:00
19778
295, 798,
Average Response Time (seconds)
irimumGraph's Average Graphs Mavimum Graph's MedianGraph's Std. Deviation
each transaction and also the number of transactions which ‘passed & ‘failed’. The response times were
good and the number of failures were low; 40 logoffs failures which was due to a data issue with the user
details for those particular branches, and not a genuine issue. There were no other failures.
Transaction Name Minimum Average Maximum Sue 90 Percent I Pass Fail
C_010_FirstLogon 0.124 ] 0.149 3.298 0.058 0.17 193426 0)
C_020_SecondLogon 0.117 0.191 35 0.066 024 I 193426 I 0
C_030_GetMemos 0.07 0.08 3.414 0.016 0.09 I 193426 I 0
C_040_CountSusp 0.07 0.08 3.023 0.014 0.09 I 193426 I 0
C_050_Logoff 0.084 0.098 3.112 0.041 0.11 193386 I 40
T_0010_010_CAPOAuth 0.167 0.323 3.548 0.075 0.421 I 308111 I 0
T_0010_020_CAPORecov 0.106 0.119 3.213 0.053 013 [308111 I 0
T_0010_030_CAPOSettle 0.423 0.134 3.186 0.033 014 [308117 I 0
T_0011_010_ALAuth 0.184 0.304 3.246 0.075 0.364 19254 0
T_0011_020_ALRecov 0.404 0.418 3.06 0.066 0.43 19254 0
T_0011_030_ALSettle 0.422 0.435 3.161 0.032 0.15 19254 0
T_0012_010_LINKAuth 0.174 0.323 3.433 0.078 0.421 37748 0
T_0012_020_LINKRecov 0.405 0.118 3.093 0.058 0.13 57748 0
T_0012_030_LINKSettle 0.122 0.134 3.201 0.031 0.14 57748 0
T_0020_010_STMPSettle 0.093 0.165 3.412 0.053 018 I 574830I 0
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
i Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS
Page No: 42 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
Transaction Name Minimum Average I Maximum cua m 90 Percent I Pass I Fail
T_0050_010_DVLAAuth 0.411 1.427 4.197 0.243 1.42 57752 0
T_0050_020_DVLASettle 0.108 0.122 2.728 0.054 0.13 57752 0
T_0060_010_STMPSettle 0.095 0.167 3.255 0.049 0.18 192405 )
T_0070_010_LASTPOInd 0.072 0.142 3.378 0.031 0.15 115703 CY)
T_0070_020_LABELEVTSrv 0.088 0.101 3.074 0.052 0.11 115703 LY)
T_0070_030_POSRVSettle 0.104 0.116 3.104 0.035 0.13 115703 0
T_0080_010_LASTPOInd 0.072 0.143 2.037 0.028 0.15 19265 0
T_0080_020_Paf 0.153 0.177 2.074 0.026 0.2 19265 0
T_0080_030_PSBarCode 0.071 0.082 1.573 0.021 0.09 19265 0
T_0080_040_POSRVSettle 0.128 0.143 2.527 0.054 0.15 19265 )
T_0140_010_EPOSSettle 0.095 0.167 3.373 0.054 0.18 307788 )
T_0170_010_BCSHSettle 0.104 0.176 3.276 0.058 0.19 96223 ()
T_0180_010_ETAAuth 0.185 0.289 1.984 0.061 0.36 11518 0
T_0180_020_ETARecov 0.099 0.116 2.651 0.067 0.14 11518 0
T_0180_030_ETASettle 0.11 0.129 3.151 0.046 0.16 11518 LY)
T_0190_010_BDCSettle 0.101 0.173 3.327 0.052 0.18 96164 0
T_0300_010_DBTAuth 0.182 0.313 3.256 0.066 0.371 38575 0
T_0300_020_DBTRecov 0.104 0.116 3.118 0.058 0.12 38575 )
T_0300_030_DBTSettle 0.124 0.136 2.519 0.034 0.15 38575 )
T_0500_010_APOPAuth 0.204 0.276 2.237 0.044 0.33 19253 0
T_0500_020_APOPDRecov 0.105 0.119 3.104 0.058 0.13 19253 0
T_0500_030_APOPDSettle 0.123 0.138 2.864 0.041 0.16 19253 LY)
Test Status: Passed
Date of Final Run: 13'" August 2010 at 11:30 to 12:42
Build Release: 01.22.00 / OSRs at 01.23.01.00 (all 4 BRDB nodes on new v249 blades)
TestID 247
5.14.1.1_I BRDB Performance Characteristics
The following graphs taken from Athene data captured, show a comparison of selected system
performance characteristics for the following comparative test runs:
TestID 200 — with all 4 BRDB nodes on the original pblades (NB: ORM was OFF for this run so it is not
an entirely fair comparison, but does still serve to highlight the main differences)
TestID 230 - with only BRDB Node 1 on new v249 pblade, others on the original pblades
TestID 247 — the test reported above with all 4 BRDB nodes on new v249 pblades
In regards to test 247, there were no capacity constraints anywhere in the system and the results were as
expected. In comparing this test with TestID 200 and 230, the significant difference was the BDB memory
profile and CPU Utilisation. There is a lot more free memory available post-upgrade, thus alleviating
swapping on the nodes, and CPU utilisation is lower..
5.14.1.1.1BRDB Packets In
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 43 of 98
Fe)
FUJITSU
COMMERCIAL IN CONFIDENCE
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJ00097962
FUJ00097962
Bl irvpbab001 .etho++
© Irvpbdbo02,etno++ [MM Irvpbdboo3.etho++ [i Irnvpbc
4,000,000
3,500,000
3,000,000
2,500,000
2,000,000
Packets In
1,500,000
1.000.000
500,000
0
Test ID: 200
28/06/2010
BB irvpbaboo1
ethO++
i Inpbaboo2.etho++ ff Irvpbdb0o3 eth
5,000,000
4.500.000
4,000,000
3,500,000
3,000,000
2,500,000
2,000,000
1,500,000
1,000,000
500,000
0
Packets In
Test ID: 230
12-400
02/08/2010
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Ref:
Version:
Date:
Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
44 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
BB irvpbab001 etno++ Invpbdb002.etno++ J Irvpbdb003 etn
6,000,000
5,000,000
4,000,000
3,000,000
Packets In
2,000,000
1,000,000
11amoo
Wee
12.2000
12:30:00
12.5500
13/08/
3
2010
Test ID: 247
5.14.1.1.2BRDB CPU Utilisation
There was a bit of a spike in Test 247 for node 3 at the start of the test. Other than that, utilisation was
down as expected.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS, Page No: 45 of 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
oO
FUJITSU
— Irvpbdb002 = — Irvpbdb0o3 Irvpbaboo4
— Irvpbdb001
oner st
ou-or SL
OD-SE'SL
OD-0E'SL
oa-c2'SL
O0-02:SL
OOF) St
O0-E0: 4b
00-00: Sh
OD-SSSb
28/06/2010
100
a
O0-0F Ft
OU-EE: pL
O0:0E FL
OD-8é: FL
8 8 €¢ 8 °
% pevodey [elo] UCHEZIIN NAD
Test ID: 200
— lIrvpbdb002 — Irvpbdb003 — Irvpbdb004
— Irvpbdb001
N
OO=E2 ZL
0:02 21
OO-CL: 21
OO-O1 2b
OO-CO: ZL
O0-00' 2b
00:85: II
OO-0S: II
OOS I I
0-0 LL
OU-SE Ib}
O0-0F: LL
00-87 I}
90-0211
OSL b+
0-01 ‘11
OUESO LL
00:00:11
02/08/2010
°
3
oo
ooco0g00000
SOaroentand
% payodey [e]O] UCHeEAIIN Ndo
Test ID: 230
TST/SOT/REP/0866
1.0
Ref:
COMMERCIAL IN CONFIDENCE:
© Copyright Fujitsu Services
Limited 2010
17-Sep-2010
46 of 98
Version:
Date:
Page No:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdb001 + — Irvpbdboo2 — Invpbdboo3 +— Irvpbdboo4
100
# 90
3
2 60
8 79
tc
= 60
& 50
=
£ 40
s 30
5
2 20 —-
Qo 10
i}
ele lels lo Je je Ie Je Ig
eee eEeeERE
3 le lig Is ic mle he 16 5
AN 4% Co nN oo TF ra
BEES E Bae ae B
1308/2010
Test ID: 247
5.14.1.1.3BRDB Run Queue
The CPU queue length was right down on tests 230 and 247, slightly more so on test 247.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet STS OUIRERIDERS
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 47 of 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
oO
FUJITSU
{© Irvpbaboo2 fl Irvpbdb003 fill Irvpbaboo4
I ivpbabo01
00-00-81
ODES Sh
O0F0S'S1
On-OR SL
ODES
O0-0FS/
0-7-9
OU-07 St
OOS Ss
OU-00'S1
OO-ES FL
OOFSr FL
0-0 PL
OUEE PL
O0-0F FL
O0-8é FI
28/06/2010
a
°
f=}
o 2 9
o 8 F
anany umy Jo y}6Us7
Test ID: 200
irvpbabo02 )« Irvpbabo03 fi) Irvpbaboo4
I Irvpbaboo1
ons 2b
O00 ZL
OO-E1: 21
00:6: II
0-05: bb
OOS 7 Lb
OO-OF LL
OO-OL sh)
ONFEO: LL
0-00: 11
02/08/2010
°
a
°
i=}
o2 2 9
o Go F
@nenp uny Jo yybusT
Test ID: 230
TST/SOT/REP/0866
1.0
Ref:
COMMERCIAL IN CONFIDENCE:
© Copyright Fujitsu Services
Limited 2010
17-Sep-2010
48 of 98
Version:
Date:
Page No:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
HM Ivpbaboot =f Irvpbaboo2 I Irvpbaboos fill Irvpbaboo4
120
100
80
60
40
Length of Run Queve
20
11:25:00
1145200
11:55:00.
12:40:00
12:50:00
a a
13/08/2010
Test ID: 247
5.14.1.1.4BRDB Used Memory
Memory utilisation on the new pblades was significantly lower than that of the old pblades (as expected).
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 49 of 98
oO
FUJITSU
COMMERCIAL IN CONFIDENCE
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJ00097962
FUJ00097962
— Inpbdb001 — Irvpbdb002 =— Irvpbdb003 Irvpbdb004
90
80 + or
70
& 60
=
5 50
= 40
B 30
20
10
ty)
le ls ele le lg lo
I ba Sele Ea
a IS 3 le le 8 5
a 2 a's 'g sig
a cd “22 ie)
28/06/2010
Test ID: 200
— Irvpbdboot ~— Irvpbdboo2 ~— Irvpbdboos Irvpbdboo4
100
90
80 =
& 70
a 60
3 50
B 40
EU Enemas sd
3 30
20
10
0
seeeeeeeeeeeeekeek
Slo ls bs IS ks lé he lS he 16 he 1S le IS lie IS Ie
SS ER SS 8 Bese BR B'S SPINS
2108/2010
Test ID: 230
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS
Page No:
50 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 :
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdb001 — Inpbdb002 — Irvpbdb003 Irvpbdb004
100
90
80
& 70
= 60
2
§ 50
g 40
3 30
20
10
*RERERE e sEEEER ig
= I8 I2 I6 ad ad Se (8 2 Ie 5
lp le IS le IS le Io & 15 16 le Ie he 5
a 9 9 +t tT HH GS ZT UN Ne ray
EESeSEsees San 9 x
13/08/2010
Test ID: 247
5.14.2 BAL & BRDB - Mid-Day Profile
This test subjected the BAL/BRDB layers to a very high load for over an hour. The transaction types
included EPOSS, Banking, Debit Card, E-TopUps, DVLA, PAF, Postal Orders, Bureau, Bill Payments
and Postal Services. The scenario had a target rate of 1393 transactions per second (average) which
equates to 829 settlements per second
Basket/Script Hits/sec I Settle/sec
0010_BKT01_CPO_WDL(2) 390 130
0011_BKTO1_AL_WDL(2) 36 12
0012_BKTO1_LNK_WDL(2) 120 40
0020_BKT02_STP_SLE(3) 300 300
(0030_BKT03_BIL_PYT(2) 30 30
0050_BKTO5_DVL_TDC(3) 48 24
0060_BKT06_STP_SLE(3) 30 30
0070_BKT07_PTL_SRV(5) 90 30
0080_BKT08_PTL_SRV(4) 40 10
0140_BKT14_EPO_CCQ(3) 120 120
(0150_BKT15_PTL_SRV(2) 40 20
0160_BKT16_BIL_DBT(2) 30 10
0170_BKT17_BIL_CSH(3) 25 25
0180_BKT18_ETA_CSH(2) 24 8
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE ie nel IREP/OGES
i ersion:
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 51 of 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
COMMERCIAL IN CONFIDENCE
Fe)
FUJITSU
Basket/Script Hits/sec I Settle/sec
0190_BKT19_BDC_CSH(3) 25 25
0500_BKT50_POS_ORD(4) 45 15
1393 829
The test iteration run against all 4 BRDB nodes on new v249 blades produced excellent results, both in
terms of throughput and response times. The graph below shows the total transaction rates for the test.
The throughput achieved was consistently good (~1390+tps) with a few dips.
Total Transactions per Second
ter Total Trans:
econd Graph data in Excel format (4S)
currant Resuite: — OH\HNGINE #\20100029_<2_x11_balbdb_249
Npozooea 323. sibabazae.ler
Fitters: None
Granularitys 10 Seconds
Total Transacti
Showing - Average
72000 120500 13:10:00 13:15:00 19.2000 13:25:00 12.9000 13.95.00 124200 12.45.00 12:50:00 13:55:00 14.0000 14.05.00
Absolute Time hh:hh:rnm
[Rann FSA A ATAPI ERA Mn FRAN MAHAN A RNS
ParvemnaMaManeannni
The response times achieved had a steady profile with a few spikes coinciding with the dips in throughput
as shown in the merged graph below.
© Copyright Fujitsu Services
COMMERCIAL IN CONFIDENCE fet TSTISOTIRERI0886
Limited 2010 fersion
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 52 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 af
FUJITSU COMMERCIAL IN CONFIDENCE
Total Transactions per Second - Average Transaction Response Time
Titles Total Transactions per Second - Average Transaction Response Time Graph dat
Excel format [8S]
Graph Type: ”
Graph i: Total Transactions per Second
Graph 2: Average Transaction Response Time
Gramutarity: 10 Seconds
Total Transactions per Second . Average Transaction Response Time
14
12
08
os
o4
02
°
Average Response Tine (seconds)
2 100
bs mary tr
1.200 T 7
+000
a)
a)
§ «0
2 2
- °
130000 431000132000 aas000 "494000 426000 140000
Absolute Time hh:bhrnm
Cole Gtaoh ScaleMeasuement Graphs MnimuGraohs Average Giaph's Manimum Glaphs MedianGaphie Std Devistion
MT 1__ Pass. a 1277 979, 1497 1A797. 38 87.
The table below is a summary of the test; showing the response time (min/avg/max/90th percentile) for
each transaction and also the number of transactions which ‘passed & ‘failed’. The response times were
good and the number of failures were low; 25 logoffs failures which was due to a data issue with the user
details for those particular branches, and not a genuine issue. There were no other failures.
Transaction Name Minimum Average I Maximum een 90 Percent I Pass Fail
C_010_FirstLogon os 0.13 0.233 0.011 0.14 1000 0
C_020_SecondLogon 0.127 0.199 0.395 0.035 0.24 1000 0
C_030_GetMemos 0.07 0.074 0.452 0.006 0.08 4000 0
C_040_CountSusp 0.063 0.081 0.652 0.021 0.09 4000 0
C_050_Logoff 0.078 0.096 0.592 0.029 O41 975 25
T_0010_010_CAPOAuth 0.162 0.369 5.966 0.182 0.471 491046 I 0
T_0010_020_CAPORecov 0.094 0.138 5.561 0.162 0.15 491046 I 0
T_0010_030_CAPOSettle 0.075 0.154 8.865 0.171 0.17 491046 I 0
T_0011_010_ALAuth 0.466 0.334 9.485 0.188 0.401 45437 0
T_0011_020_ALRecov 0.404 0.438 3.48 0.471 0.46 45437 0
T_0011_030_ALSettle 0.121 0.154 3.447 0.151 0.18 45437 0
T_0012_010_LINKAuth 0.164 0.422 9.235 0.191 0.551 150909 I 0
T_0012_020_LINKRecov 0.105 0.135 5.529 0.165 0.15 150909 I 0
T_0012_030_LINKSettle 0.103 0.151 9.115 0.157 0.17 150909 I 0
T_0020_010_STMPSettle 0.082 0.192 9.042 0.176 0.21 1117247 I 0
T_0030_010_BiLLSettle 04 0.205 5.754 0.189 0.23 112907 I 0
T_0050_010_DVLAAuth 0.405 0.97 4,809 0.321 14 90811 0
T_0050_020_DVLASettle 0.102 0.141 5.565 0.187 0.15 90811 0
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
i Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 53 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
Transaction Name Minimum Average Maximum Se 90 Percent I Pass Fail
T_0060_010_STMPSettle 0.095 0.199 9.015 0.187 0.23 112953 i)
T_0070_010_LASTPOInd 0.072 0.163 3.783 0.149 0.18 112704 0
T_0070_020_LABELEVTSrv 0.069 0.122 5.536 0.176 0.14 112704 i)
T_0070_030_POSRVSettle 0.086 0.135 8.929 0.156 0.15 112704 i)
T_0080_010_LASTPOInd 0.072 0.171 3.821 0.172 0.2 37592 i)
T_0080_020_Paf 0.153 0.206 3.322 0.168 0.24 37592 i)
T_0080_030_PSBarCode 0.071 0.104 3.262 0.152 0.13 37592 i)
T_0080_040_POSRVSettle 0.126 0.168 8.89 0.192 0.19 37592 i)
T_0140_010_EPOSSettle 0.073 0.194 9.074 0.178 0.22 452452 0
T_0150_010_LASTPOInd 0.072 0.172 3.923 0.172 0.2 74760 i}
T_0150_020_POSettle 0.091 0.144 4.261 0.169 0.17 74760 i}
T_0160_010_DBTAuth 0.187 0.351 3.868 0.194 0.421 37706 i}
T_0160_020_DBTRecov 0.104 0.139 3.271 0.168 0.16 37706 i)
T_0160_030_DBTSettle 0.129 0.162 3.465 0.147 0.19 37706 i)
T_0170_010_BCSHSettle 0.103 0.21 5.805 0.189 0.24 93646 0
T_0180_010_ETAAuth 0.138 0.316 4.175 0.179 0.381 30264 i)
T_0180_020_ETARecov 0.097 0.139 3.439 0.177 0.16 30264 i}
T_0180_030_ETASettle 0.097 0.147 8.902 0.155 0.18 30264 i}
T_0190_010_BDCSettle 0.101 0.208 6.069 0.194 0.24 93631 O
T_0500_010_APOPAuth 0.183 03 3.865 0.159 0.341 56579 i)
T_0500_020_APOPDRecov 0.104 0.14 3.425 0.177 0.16 56579 0
T_0500_030_APOPDSettle 0.111 0.156 3.614 0.158 0.18 56579 i)
Test Status: Passed
Date of Final Run: 13'" August 2010 at 12:57 to 14:08
Build Release: 01.22.00 / OSRs at 01.23.01.00 (all 4 BRDB nodes on new v249 blades)
TestID: 248
5.14.2.1_I BRDB Performance Characteristics
The following graphs taken from Athene data captured, show a comparison of selected system
performance characteristics for the following comparative test runs:
TestID 183 - with all 4 BRDB nodes on the original pblades
TestID 231 — with only BRDB Node 1 on new v249 pblade, others on the original pblades
TestID 248 — the test reported above with all 4 BRDB nodes on new v249 pblades
In regards to test 248, there were no capacity constraints anywhere in the system and the results were as
expected. In comparing this test with TestID 183 and 231, the significant difference was the BDB memory
profile and CPU Utilisation. There is a lot more free memory available post-upgrade, thus alleviating
swapping on the nodes, and CPU utilisation is lower.
5.14.2.1.1BRDB Packets In
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 54 of 98
FUJ00097962
FUJ00097962
co VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
BB irvpbdb001 etho++ fl) Invpbdb002.etho++ [ Irvpbdb003.eth
BBB !pbaboo4 etho++
7,000,000
6,000,000
5,000,000
@ 4,000,000
g
& 3.000.000
2,000,000
1,000,000
0
S s Is Ie = lo lo se Io
EE £18 I gf S Ig iS
g B12 le le Ig Ie Ie ie lg iB
= zag a8 a8 BS]
18/06/2010
Test ID: 183
BB irvpbdb001 etho++ fi) Invpbdb0o2.etho++ [ Irvpbdb003 eth
By pbb 004 etho++
7,000,000
6,000,000
5,000,000
4,000,000
2£
® 3,000,000
2,000,000
1,000,000
0
S s le Io Io [= lo Ie
€ sis iS 8 Ig Sg
5 & IS 5 S Ie iS Ie
NOR a 8 a IS
2 #2 2 2 = z=
02/08/2010
Test ID: 231
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
me UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS, Page No: 55 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
BB irvpbab001.etho++ i) Irvppdboo2.etho++ fl Ivpbdboo3.eth
HB inpbabo04 etho++
7,000,000 7
6,000,000
5,000,000
» 4,000,000
2
®B 3,000,000
2,000,000
1,000,000
)
lo co lz Io lo
3 Sz 8 iS
5 te IE ls In
BH alse fa
13/08/2010
Test ID: 248
5.14.2.1.2BRDB CPU Utilisation
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
ime UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 56 of 98,
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
oO
FUJITSU
— Irvpbab004
— Irvpbdb003
— Irvpbdboo2
— Irvpbdbo01
PIN
.
OUESS: ZL
Q0=0S' 21
O0-Sb 21
O0-0F Zt
00-21.
OG-OL: ZL
O0-f0 eb
I o-00:21
I Q0-E¢' II
00-09 11
OUSh I}
oor LL
OSE LL
O0-0E:LE
18/06/2010
°
i]
es20000999090
oor onu TAA
% peyodey [PIO1 UCHETIIN Ndo
Test ID: 183
— Irvpbdb002 + — Irvpbdb003 — Irvpbdb004
— Irvpbdb001
OnOr PL
OUT PL
I 0008: bL
ou-ez' ph
00:02: FL
O0-£0: FL
00: FI.
OU-ES:EL
ros B}
Ooer el
LO0-0F E
O0-se Ee)
Ob-08
Oo:S2e)
O00 EL
OD-C1 EL
02/08/2010
°
Ss
esccogg0gggo9o
Oor on Tana eH
% pepodey Ie}, ucLeETIIN Ndo
=
Test ID: 231
TST/SOT/REP/0866
1.0
Ref:
COMMERCIAL IN CONFIDENCE:
© Copyright Fujitsu Services
Limited 2010
17-Sep-2010
57 of 98
Version:
Date:
Page No:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdb001 + — Irvpbdb002 ~— Irvpbdb003 +— Irvpbdboo4
100
# 90
3
2 80
2 70
a
= 60
e 50
=
£ 40
i
& 30
5
2 20
G 10
0
e Ig le Ig iS ls Jo fe fe 2
3 le 1s Ww 63 1S he Ié lig is 5
on o br uo tT PH YS i.
oN 8 2 peagogs z
13/08/2010
Test ID. 248
5.14.2.1.3BRDB Run Queue
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 58 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdb0o1 +— Invpbdb0o2 — Invpbdboo3 ~— Invpbaboo4
100
90
80
70
60
50
40
30
20
10
i}
Length of Run Queve
00
HO
00
‘00
010
00
12:30:
a
5:
2
C=}
a
12:25:00
12:3:
12.40:
18/06/201
11:30:
11:38
1140:
x]
I
0
Test ID: 183
— Irvpbdb001 — Irvpbdb002 — Irvpbdb003 — Irvpbdb004
100
90
80
70
60
50
40
30
20
0) eS Se ——
0
Length of Run Queve
00
00
13-20
D0
14.40)
5:
nO
ef
13:25:00
13:30)
13-1
4
14:95:00
02/08/2010
Test ID: 231
Ref: TST/SOT/REP/0866
Version:
1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 59 of 98
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE:
Limited 2010
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJI I SU COMMERCIAL IN CONFIDENCE
— Irvpbdb0o1 +— Irvpbdb002 — Irvpbdb0o3 + — Irvpbdboo4
100
90
@ 8
3 70
2 60
5
& 50
3 40
H 30
20
10 EF z =
t)
elgle 2 2 Je je 2
8 lin 16 315 lig ls 5
on o = YS i.
aa 2 geagag s z
13/08/2010
Test ID: 248
5.14.2.1.4BRDB Used Memory
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 60 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 .
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdbo01 — Irvpbaboo2 — Irvpbabo03 Ivpbdboo4
“ ne
_ iN
#
E60
=
2
% 40
8
a
20
()
2
8
&
3
Test ID: 183
— Irvpbdb001 — Irvpbaboo2 — Irvpbdbo03 Invpbdb004
100
80 :
2
5 60
=
2
2
% 40
8
a
20
0
clelglele ele ellieelitelee
BS lS 1S le lS le lS lie 1S Ie 1S he IS
= oN ce 7 tt OH ne oa7T UN MY
2 2 See tents eserete es
0208/2010
Test ID: 231
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 61 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 ad
FUJITSU COMMERCIAL IN CONFIDENCE
— Inpbdbo01 — Irvpbdboo2 — Irvpbdb003 Invpbdb004
100
80
e
60
2
$
Be 40
i
2
20
0
alg A ig E lg lg
a 8 SaEaes #
Test ID: 248
5.14.3 BAL & BRDB - End of Day Profile
Since version 0.1 of this report was issued, this test has been re-run following delivery of an informal fix
for PC0201529 (High CPU usage results when running Group Reports). To illustrate the significant
performance improvement delivered by this fix, the results shown below provide a comparison of TestID
254 (after the fix was applied) and TestID 249 (before the fix was applied).
This test was run after the data loading tasks were fully completed. The test includes the Group Reports.
with other single reports and with a background transaction load.
The reports component had a selection of ‘single’ reports and ‘group’ reports (see table below). It should
be noted that the ‘Reports’ in red below are not strictly reports but other end-of day activity.
Reports Description
7800_CDCHL_RPT I Counter Daily Cheque Listing
2100_BALSS_RPT I Balance Snapshot Report
2400_OFFSS_RPT I Office Snapshot
2700_CWMTX_RPT I Weekly Group Report (3001)
'2800_CSH_DEC Cash Declaration
2900_STK_DEC ‘Stock Declaration
3000_STP_DEC ‘Stamp Declaration
3100_CDCUT_RPT I Daily Group Report (3000)
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 62 of 98
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
FUJ00097962
FUJ00097962
The following multiplication factors apply to the group reports:
1 x 2700_CWMTX_RPT is equivalent to 8 single reports, and
1 x 3100_CDCUT_RPT is equivalent to 10 single reports
The test produced excellent results with good response times and throughput. Comparing with Testld 249
there was a significant improvement in every respect for both the reports and declarations as well as the
background transaction load.
The graphs below show the Total Transaction Rates for the test. Spikes of up to 1100/sec was achieved
for Testld 254 compared with 750/sec in Testld 249.
TestID 254 (after fix for PC0201529)
Total Transactions per Second
Tite: Total Transactions per Second Graph data in Excel format [3
dAHNGx\Phase2\Recults\20100824_balbdb_endofday_02
Current Results: \20100824_balbdb_endofday_Od.lrr
Filters: None
Group By:
Granularity: 10 Seconds
Total Transactions per Second
Showing - Average
1,100
4,000
ransactions
S600
3 so
€
5 40
= oo
- 200
100
of
184000 16:50:00 47:00:00 471000 1720.00
Absolute Time hh:hh:rmm
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 63 of 98,
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
Ps)
FUJITSU COMMERCIAL IN CONFIDENCE
FUJ00097962
FUJ00097962
TestID 249 (before fix for PC0201529)
Total Transactions per Second
Title: Total Transactions per Second
d:\HNGX\Phaze2\Results\20100814_balbdb_endofday_o2,
Current Results: (20100814 balbdb endofday_01.Irr
Filters: None
Group By:
Granularity: 10 Seconds
Total Transactions per Second
Showing - Average
Graph data in Excel format [SS
Total Number of Transactions
16:20:00 163000 16:40:00 16:50:00 17:00:00
Absolute Time hh:hh:mm.
17:10:00
17:20:00 1730.00
The graphs below show the transaction rates for the Reports and Declarations (cash/stamp/stock) and
demonstrate up to 160/sec at points in the test. Taking into account the multiplication factors of the Group
Reports then the overall rate is equivalent to up to 345tps for reports, and around 65tps for the
declarations. This compares with an equialent rate of 148-158tps for reports and 35-37tps for
declarations in TestID 249
TestID 254 (after fix for PC0201529)
Copy (1) of Transactions per Second
Title: Copy (1) of Transactions per Second
di\HNG2\Phase2\Results\20100824_balbdb_endofday_o2.
Current Results: (35 beaze balbdb endatiayO1.in
Graph date in Excel format [3
ruteare: Gndude Think Time)
Copy (1) of Transactions per Second
Showing - Average
100
160
2 10
= 120
E10
5 0
z=
2 «
20
od
462000 16:9000 164000 16.80.00 17.00.00 17:10:00 17.20.00 17:30.0
‘Absolute Time hh:hh:mm
[a 12700_10_CamaTxRenserv:Pass
1 173000_010_GetDec:Pass
Getbec Pass
I_CDGDPRepSery: Pass
lim 1_2900_010_GetecPass
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE
Limited 2010
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS.
Ref:
Version:
Date:
Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
64 of 98
VOLUME AND PERFORMANCE TEST REPORT - PHASE
COMMERCIAL IN CONFIDENCE
Fe)
FUJITSU
FUJ00097962
FUJ00097962
2
TestID 249 (before fix for PC0201529)
Copy (1) of Transactions per Second
Tue: Copy (4) of Transactions per Second
d:\HNGx\Phase2\Results\20100814_balbdb_endofday_o1
{20200814 balbdo endatday_O4. lr
(andude Think Time)
Current Reeulte:
Filters:
Group By:
Granularity:
Transaction End Status
20 Seconds
Reports & Declarations of Transactions per Second
Showing - Average
Graph dats in Excel format
Ss
Number of Transactions
16:50:00 17-0000
Absolute Time hh:hh:mm.
TET 1800_10_COCHLRepServ Pace El T_2100_10_BALSSRepServ Pass (ll T_2400_10_OFFSSRepServPass
72800 018_GetDec:Pass
The following graph shows the average response times for the Reports and Declarations. They are fairly
stable for both tests, but a lot better for Testld 254
TestID 254 (after fix for PC0201529)
Copy of Average Transaction Response Time
Title: Copy of Average Tranzaction Response Time
d\HNGX\Phase2\Results\20100824_balbdb_endofday_o1
(20100824 balbdbendofday 01. Irv
‘Transaction End Status = (Pass), (Include Think Time)
10 Seconds
Copy of Average Transaction Response Time
Graph data in Excel format
Zz wie
S 9
& se
é 7
a
g 4 lH
3 34 dee * - oF eet ost
© 1 *ebe :
=< 1
Py z
16.2000 16:30:00 16:40:00 416:50200 17200200 4740200 17:20:00 47:3010¢
Absolute Time hh:hh:mm
“> T_1800_10_COCHLRepSery -*- T_1600_20_CDCHLCMON > T_3100_10_BALSSRepServ © T_2400_10_OFFSSRepServ
“> 112700"10_CHMTxRepSery © T2800_010_GetDec “> 172900_020_Recbee “> 172800030 _UpaierCode
+ 1/2900 010 _Getbec fecDec “ T-2800_030_UUpaBercode — -#- T_3000.1 Dec
“21 3000_020"Recbec jpdBarCode 2 _T_3100_10_CDGOPRepSery_—e- T73100_20_CDUpdateculort
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: = 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 65 of 98
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
oO
FUJITSU
FUJ00097962
FUJ00097962
TestID 249 (before fix for PC0201529)
Copy of Average Transaction Response Time
Title: Copy of Average Transaction Response Time
d:\HNGX\Phase2\Results\20100814_balbdb_endofday_01.
\20100014_balbdb_endofday_o4.ler
Transaction End Status = (Pass), (Include Think Time)
Current Results:
Filters:
Group By:
Granularity: 10 Seconds
Copy of Average Transaction Response Time
Graph data in Excel format [fS
45
40
35
30
25
Average Response Time (seconds)
46:20.00 163000 164000 1650.00 47:00:00
Absolute Time hh:hh:mnm
4710.00 1
7:20.00 4730.00
“e T1800_10_CDCHLRenServ © T_1800_20_COCHLOMOf! —* T_2100_10_BALSSRepServ © T_2400,
© 7_2800_020 Recbec © T2800,
-% 72900 _030_UpdBarCode — -¢- T_3000,
© T_3100_10_CDGDPRepSery _-e- T3100,
‘© 7/3000_020 RecDec
+ T_3000_030_UpdBarCode
OFFSSRepServ
UpdBarCode
GetDec
1_CDUpdateCutort
The table below provides a summary of the test; showing the response time (min//max/90th percentile)
for each ‘step’ of the reports and baskets run and, also, the number of steps which ‘ps
assed’. The
response times on the 90th percentile were excellent in Testld 254 and significantly better than Testld
249. The improved response times resulted in a much higher throughput of successful transactions in
TestID 254
Minimum Maximum 90 Percentile Pass
Transaction Name 254 249 254 I 249 254 249 254 I 249
C_010_FirstLogon 0.124 0.125 I 6999 I 21.169 I 0.24 I 0.895 I 63627 I 32253
€_020_SecondLogon 0.128 0.136 I 7.635 I 22037 I 0371 I 1443 I 63627 I 32253
€_030_GetNMemos 0.063 oo7 I 3457 I 216 I 044 I 0724 I 63607 I 32253
_040_CountSusp 0.063 0.071 I 5701 I 20978 I 014 I 0.705 I 63627 I 32086
€_050_Logoff 0.078 0083 I 7.172 I 23215 I 018 I o877 I 63627 I 32085
T_0010_010_CAPOAuth 0.164 0.162 I 6324 I 22849 I 0461 I 1.167 I 407910 I 301680
T_0010_020_CAPORecov 0.106 0.106 I 7.355 I 22013 I 016 I 0.724 I 407910 I 301680
T_0010_030_CAPOSettle 0.097 0.123 I 7.299 I 44337 I 018 I 0.844 I 407910 I 301679
7_0011_010_ALAuth 0.172 o2t9 I 368 I 12422 I 0431 I 1.228 I 40271 I 17179
T_0011_020_ALRecov 0.104 0.105 I 6907 I 20.732 I o17 I 0.744 I 40271 I 17178
T_0011_030_ALSettle 0.122 0.122 I 6697 I 10.921 I 0.21 0.886 I 40271 I 17179
T_0012_010_LINKAuth 0.187 0.177 I 3685 I 22921 I 0461 I 1.217 I 40777 I 29934
7_0012_020_LINKRecov 0.105 0105 I 6501 I 21.315 I o17 I 0761 I 40777 I 29934
7_0012_030_LINKSettie 0.118 0123 I 7.096 I 21.651 I 02 0.856 I 4077 I 29934
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
i Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 66 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 =
FUJITSU COMMERCIAL IN CONFIDENCE
Minimum, Maximum 90 Percentile Pass
Transaction Name 254 249 254 249 254 249 254 249
T_0020_010_STMPSettle 0.093, 0.093 7.04 21.878 0.23 0.816 488611 I 422712
T_0050_010_DVLAAuth 0.484 0.59 4.671 22.96 1.389 2.145 40119 22899
T_0050_020_DVLASettle 0.108 0.109 7.108 21.329 0.18 0.836 40119 22899
T_0180_010_ETAAuth 0.163 0.195 3.786 20.849 0.401 1.206 20404 15246
T_0180_020_ETARecov 0.099 0.099 6.939 21.469 0.18 0.745 20404 15246
T_0180_030_ETASettle 0.088 0.11 7.159 23.339 0.21 0.888 20404 15246
T_0300_010_DBTAuth 0.198 0.224 3.675 13.825 0.441 1.217 20480 18244
T_0300_020_DBTRecov 0.103 0.104 6.91 21.521 0.17 0.776 20480 18243
T_0300_030_DBTSettle 0.124 0.124 7.01 21.021 0.2 0.882 20480 18244
T_0500_010_APOPAuth 0.199 0.239 3.619 21.227 0.381 1.247 8211 8052
T_0500_020_APOPDRecov 0.106 0.106 6.931 10.069 0.18 0.745 8211 8052
T_0500_030_APOPDSettle 0.12 0.124 7.027 20.866 0.22 0.907 8211 8052
T_1800_10_CDCHLRepServ 0.084 0.095 5.61 24.587 0.22 1.982 91400 47520
T_1800_20_CDCHLCutOff 0.089 0.089 7.066 23.039 0.16 1.839 91400 47520
T_2100_10_BALSSRepServ 0.348 0.544 6.505 24.587 0.793 4.898 83540 40290
T_2400_10_OFFSSRepServ 04 0.669 6.653 25.929 0.823 4.959 83120 39900
T_2700_10_CWMTXRepServ 0.924 1.777 129.935 I 108.108 2419 9.188 58693 24922
T_2800_010_GetDec 0.067 0.075 5.417 20.706 0.14 0.724 89380 55640
T_2800_020_RecDec 0.089 0.091 7.366 21.98 0.21 1.047 89380 55640
T_2800_030_UpdBarCode 0.072 0.084 7.012 23.292 0.15 0.755 89380 55640
T_2900_010_GetDec 0.041 0.075 7.382 21.436 0.15 0.753 77350 36830
T_2900_020_RecDec 0.267 0.269 8.399 30.091 1.035 4.332 77350 36830
T_2900_030_UpdBarCode 0.084 0.084 7.193 21.383 0.16 0.782 77350 36830
T_3000_010_GetDec 0.032 0.075 5.544 44.445 0.15 0.724 87900 52740
T_3000_020_RecDec 0.098 0.098 7.097 22.364 0.31 1.391 87900 52740
T_3000_030_UpdBarCode 0.046 0.084 7.272 21.035 0.15 0.775 87900 52740
T_3100_10_CDGDPRepServ 0.97 2.593 115.567 I 113.571 1.989 11.968 59880 19117
T_3100_20_CDUpdateCutOff 0.078 0.073 7.009 17.481 0.18 1.93 59880 19141
The Improvement at the BAL layer has resulted in increased throughput and lower CPU Utilisation which
is excellent - a two-fold improvement. The knock-on effect of removing the bottleneck at the BAL layer is
that there is now additional load at the BDB layer. CPU Utilisation has gone up as a result of increased
packets/in and where we were previously seeing ~10% utilisation in TestID 249, we are now seeing just
under 50% in TestID 254. This is illustrated in the following section 5.14.3.1.
Test Status: Passed
Date of Final Run: 24" August 2010 at 16:10 to 17:31
Build Release: 01.22.00 / OSRs at 01.23.02.00 (all 4 BRDB nodes on new v249 blades)
TestID: 254
5.14.3.1 BRDB Performance Characteristics
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Version:
Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
67 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
The following graphs taken from Athene data captured, show a comparison of selected system
performance characteristics for the following comparative test runs:
TestID 212 - with all 4 BRDB nodes on the original pblades
TestID 232 — with only BRDB Node 1 on new v249 pblade, others on the original pblades
TestID 249 — the test reported above with all 4 BRDB nodes on new v249 pblades
In regards to test 249, there were no capacity constraints in relation to the BRDB and the results were as
expected. In comparing this test with TestID 212 and 232, the significant difference was the BDB memory
profile and CPU Utilisation. There is a lot more free memory available post-upgrade, thus alleviating
swapping on the nodes, and CPU utilisation is lower. During this test run, there was an outstanding issue
with high CPU Utilisation on the BAL servers for processing Group Reports (PC0201529)
TestID 254 — a repeat of TestID 249 after applying an informal fix for PCO0201529 to address the High
CPU utilisation at the BAL servers.
In regards to test 254, there were no capacity constraints in relation to the BRDB and the results were as.
expected. In comparing this test with TestID 249, the significant difference was the increase in
Packets/In, thus indicating that the bottleneck had previously been at the BAL layer. CPU utilisation also
increased in line with Packets/In
5.14.3.1.1BRDB Packets In
Hl Ivpbab001.etho++ J Irvpbaboo2.etho++ [ Irvpbdb003.etho++ I Ivpbab
6.000.000
5,000,000
_ 4,000,000
2 3,000,000
&
a
2,000,000
1,000,000
0
s Ig la FS = Ie Ie Ie io
€ I3 I2 Ed = (8 Ie Ie fe
8 Ig le 2 @ ig Ip Ip i
4 2 8 fy
1207/2010
Test ID: 212
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 68 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
Hl ivpbdb001 etno++ == Invpbdboo2,etno++ [J Irvpbdb003.etno++ [ Irvpbab
6,000,000
5,000,000
_ 4,000,000
e
& 3,000,000
&
a
2,000,000
1,000,000
()
PaEIE: ellssiels 8
w ra 1 wis la is un
8 lel 13's Ie le le fa
ese Seeeerres =
03/08/2010
Test D: 232
The Packets/In at the BDB layer have increased massively in TestID 254 (indicating that the bottleneck
has been the BAL layer).
HB irvpbdb001.etno++ § Irvpbaoo2.etno++ fl Ivpbdboo3.etno++ [ Irvpbdbo04. etno+
10,000,000
9,000,000
8,000,000
7,000,000
= 6,000,000
5,000,000
4,000,000
3,000,000
2,000,000
1,000,000
0
14/08/2010
Test ID: 249 vs. 254
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS, Page No: 69 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
5.14.3.1.2BRDB CPU Utilisation
— Irvpbdboo1 — Irvpbaboo2 — Irvpbdbo03 — Irvpbaboo4
100
# 90
3
A 80
70
© 60 L\ fe
3
e 50 \ f
A V
£ 40
5 20 [ ee \
s* 77 Xo
2° 7 S
ot
Fy 2 Je lo FE S ig g FE S 2 cs
5 IS I8 lo bs [6 lie 1S be IS Ie IS le fy
a Pir Bie Sige sve 's g
2 Seerrrrartae =
12/07/2010
Test D: 212
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 70 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbaboot — Irvpbdbo02 — Irvpbdboos — Irvpbaboo4
100
90
80
70
e ES,
50 fr
40 [
30 ii
20 yi
10
i)
CPU Utilization Total Reported %
200
5:
16.90:00
16:0:
03/08/2010
Test ID: 232
— Inpbdb001 — Irvpbdb002 — Irvpbdb003 — lInpbdb004
100
90
80
70
60
50
40
30
20 4
10 FA
0
CPU Utilization Total Reported %
00
00
16-20-00
00
08/2010
S
ee
5-00
3700
+15:
1640-00
16:50:00
16:55:00
17:13:00
17:20:06
16: 10:
16:4
16:3)
164
17:10:
is
Test ID: 249
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Bet TSUSOTREPIDEES
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 71 of 98
FUJ00097962
FUJ00097962
fe) VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 =
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdb0o1 — Irvpbdboo2 ~— Irvpbdb0o3 ~— Irvpbdboo4
100
90
80
70
60
5)
40
50
SS
A
Le]
30
20 ff
Lt
CPU UtilZation Total Reported %
10 iy
"4
)
Fy eee eegeesee
3 IS lig 1S 3 IS Ng IS he 16 ho 1S bs IS Mig
io oO Os SH We eT TENN
2 Se OS SG SRRARRA RRA
24/08/2010
Test ID: 254
CPU Utilisation on the BDB layer has increased (inline with packets/in). ORM did NOT kick-in during the
run.
5.14.3.1.3BRDB Run Queue
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Ref:
Version:
Date:
Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
72 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 .
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdb0o1 ~— Irvpbdb0o2 ~— Invpbaboo3 Invpbdb004
70
60
2
3 50
3
5 40
ina
3 30
=
3
§ 20
be Se
ty)
e lele lo 2 ]elele
3 IS lip 5 5 \s IS Ip
ad t 313 '3'3
ee 2 - Stee
Test ID: 212
— Invpbdb001 — Irvpbdb0o2 — Irvpbdb003 Irvpbdb004
70
60
2 I
é. iit
§ 40 - ~
: I \y Af VA
3 30 A
=
& y IW AKA
3° PS NCAREN
10 TS SZ
0
gels a
8 ls IS & lb
Rig SB aia
Se 2 Se &
03/08/2010
Test ID: 232
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 73 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 7
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdb0o1 + — Irvpbdb002 — Irvpbdb003 Irvpbdboo4
70
60
Fy
% 50
g
g
s 40
or
3 30
s
>
3 20
10
0
ls e je le Iele fe Ie le le e le Je le
is 6/8 I/§ jg I8 \s [8 [8 8 8 \8 8
iS ERB I Beek le I 8 Ie
lod Na @ HO 3 TH HS a 2 0 St
ie Seeeerrere BEE
14/08/2010
Test ID: 249
— Irvpbdb001 — Irvpbdb002 — Irvpbdb003 Irvpbdb004
70
60
2
3% 50
3
5 40
«
&
5 30
s
Poo
4
van
10
0
le Ey a Ie Ig Ie
BI S Slip 1S hs
fo B SR URS
= s SEEE
24/08/2010
Test ID: 254
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 74 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
As expected, there was an increased level of queuing in TestID 254, but not at levels to cause concern.
5.14.3.1.4BRDB Used Memory
— Irvpbdb0o1 +— Invpbdb002 — Invpbdb003 Irvpbdb 004
100
80 %
= 60
é
5
=
RB 40
3
-
20
0
Ie B
5 iS
3S S
er 2
Test ID: 212
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref: TST/SOT/REP/0866
Limited 2010 Version: == 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 75 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
— Inpbdb001 — Inpbdb002 — Inpbdb003 Invpbdb004
100
80
: 60
RB 40
3
2
20
()
e elglelele ele
3 Ws 1S lin le he IS IS lig
i=] oO +H 4S it 0
s = eeekri BE
03/08/2010
Test ID: 232
— Ivpbdb001 ~— Irnvpbdboo2 — Invpbdb003 — Inpbdboo4
100
80
= 60
EB
RB 40
g
J
20
0
seeeeeee aeeee
Slo Is le Io 15 16 Ib 6 lis IS ha [6
esses PEERS
44/08/2010
Test ID: 249
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 76 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
— Irvpbdb001 — Inpbdb002 — Inpbdb003 Invpbdb004
100
90
80
70
60
50
40
30
20
10
0
Used Memon %
RI
16:05:00
17:30:00
17:35
24/08/2010
Test ID: 254
5.15 Test Group X12 — BAL & BRDB Stress Test
5.15.1 BAL & BRDB Stress Test
The objective of this test was to take the workload of the BAL and BRDB layers beyond Design Limits up
to a rate at which either
a) the service is no longer able to support the transaction load and thus identifying the maximum.
headroom capacity limit, or
b) anotional transaction rate (as agreed with Capacity Management) such that, if achieved, it would
demonstrate sufficient headroom capacity for future growth without needing to find the absolute
breaking point
This test gradually increased the load until all Loadrunner VUsers were running and was maintained for
30 minutes before ramping down. The ‘mid-day’ transaction mix was used for this test, which was run
with all 4 BRDB nodes on the new v249 blades.
The limiting factor in this test was the number of LoadRunner VUsers available. A higher throughput
could have been sustained had it been possible to generate a greater load on the system. However, the
throughput actually achieved is well above design limit and demonstrates significant headroom capacity
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREP (D866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 77 of 98
Fe)
FUJITSU
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
COMMERCIAL IN CONFIDENCE
FUJ00097962
FUJ00097962
Historical
Peak Contractual Limit Design Limit
5
Layer I Hour I 5 min Hour 5 min Hour I min I Per sec peak I Sustained HIR%
BAL na nla 932 940 1119 I 1128 1466 1800 59.57%
not not not
BRDB I n/a na spec'd I spec'd spec'd I 1068 1388 1800 68.54%
HR% above is the sustained tps rate % increase over and above the Design Limit Peak 5 min rate.
The total rate (at 5 second granularity) shows spikes over 1800 tps
1,800
1,600
1,200
1,000
Total Number of Transactions
Total Transactions per Second
Showing - Average
10:25:00 10:30:00 10:35:00 1040200 10:45:00 10:50:00 10:55:00 11:0000 11:05:00 11:1000 11:15:00 11:2000 11:25:00
Absolute Time hh:hh:mm
The response times across the transactions can be seen to increase and plateau in line with the
increasing transaction load.
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Ref:
Version:
Date:
Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
78 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 "4
FUJITSU COMMERCIAL IN CONFIDENCE
Total Transactions per Second - Average Transaction Response Time i
$
ry
E
=
i
P
i :
: ;
§ s
g
§
z r r rT rT u u ul r r r u rT
3 10:25:0010:30:00 0:35:001 0:40:00 10:45:0010:50200 10:55:00 1:00:00 11:05:0011:1020041:15:004 1:20:00 11:25:
be Absolute Time hh:hh:mm
+ Pass -@-T_0010_10_CAPOAuth © T_0010_020_CAPORecov © T_0010_030_CAPOSettle
© T0011 _010_ALAuth
> T_0012_020_LINKRecov
© T_0050_010_DVLAAuth
© T_0070_020_LABELEVTStY
© T_0080_030_PSBarCode
- T_0150_020_POSettle
-@- T_0170_010_BCSHSettle
© T_0190_010_BOCSettle
T_0011_020_ALRecov
© T_0012_030_LINKSettle
-@ T_0050_020_DVLASettle
4 T_0070_030_POSRSettle
-- T_0080_040_POSRVSettle
> T_0160_010_DBTAuth
© T_0180_010_ETAAuth
+ T_0500_010_APOPAuth
© T_0011_030_ALSettle
-@- T_0020_010_STMPSettle
- T_0060_010_STWPSettle
> T_0080_010_LASTPOInd
+ T_0140_010_FPOSSettie
T_0160_020_DBTRecov
> T_0180_020_ETARecov
© T_0500_020_APOPORecov _-@ T_0500_030_APOPDSettle
© T_0012_010_LINKAuth
+ T_0030_010_BLLSete
-©-T_0070_010_LASTPOId
+ T_0060_020 Pat
© T_0150_010_LASTPOnd
+ T_0160_090_DBTSetle
+ T_0180_030_ETASette
The table below is a summary of the test; showing the response time (min/avg/max/90th percentile) for
each transaction and also the number of transactions which ‘passed & ‘failed’. The response times were
good and the number of failures were low; 25 logoffs failures which was due to a data issue with the user
details for those particular branches, and not a genuine issue. There were no other failures.
Transaction Name Minimum I Average I Maximum I BY; I 80 oa I Pass I Fail
C_010_FirstLogon 0125 7] 02 3.237 0.193 0.328 I 1000 I 0
C_020_SecondLogon 0.124 0.27 6.292 0.303 0.439 1000 i)
C_030_GetMemos 0.07 0145 I 3.302 0.316 023 I 1000 I 0
C_040_CountSusp 0.072 0.13 3.1441 0.26 0.168 I 1000 I 0
C_050_Logotf 0.084 0.14 3.249 0.215 0199 I 975 I 25
T_0010_010_CAPOAuth 0.168 0621 I 9.508 0.487 0.692 I 428505 I 0
T_0010_020_CAPORecov 0.106 0.251 I 10.874 0.441 0.341 I 428505 I 0
T_0010_030_CAPOSetle 0.124 0.274 I 9.403 0.455 0.361 I 428505 I 0
T_0011_010_ALAuth 0.176 0476 I 9.207 0.485 0.602 I 55433 I 0
T_0011_020_ALRecov 0.104 0.244 I 9.041 0.425 0341 I 55433 I 0
T_0011_030_ALSettie 0.122 27m I 9.096 0.447 0361 I 55433 I 0
T_0012_010_LINKAuth 0.165 0.597 9.71 0.504 0.813 131971 0
T_0012_020_LINKRecov 0.105 0.249 8.99 0.439 0.341 131971 0
T_0012_030_LINKSettle 0.122 0.273 9.133 0.453 0.371 131971 0
T_0020_010_STMPSettle 0.093 0.328 I 10.666 0.478 0.431 I 817029] 0
T_0030_010_BiLLSettle 4 0349 I 9.224 0.486 0451 I 63509 I 0
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
i Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 79 of 98
FUJ00097962
FUJ00097962
Fe) VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
Transaction Name Minimum I Average I Maximum I St. i Pass I Fail
ion _I Percent
T_0050_010_DVLAAuth 0.407 1.242 9.764 0.527 1.552 74608 0
T_0050_020_DVLASettle 0.108 0.26 9.147 0.441 0.351 74608 i)
7_0060_010_STMPSettle 0.096 034 I 20.658 0.49 0441 I 80691 I 0
T_0070_010_LASTPOInd 0.072 0298 I 8.974 0.46 oat I 56641 I 0
T_0070_020 LABELEVTSr I 0.088 0247 I 9.003 0.455 034 I 56641 I 0
T_0070_030_POSRVSettle 0.105 0.267 8.977 0.466 0.361 56641 0
T_0080_010_LASTPOInd 0.072 0.259 9.065 0.407 0.361 61886 i)
T_0080_020_Paf 0.153 0.304 9.106 0.419 0.411 61886 i)
T_0080_030_PSBarCode 0.07 0.195 8.986 0.404 0.29 61886 1°)
T_0080_040_POSRVSettie I 0.129 025 I 9.182 0.397 0361 I 61886 I 0
T_0140_010_EPOSSettle 0.095 0.334 I 9.303 0.482 0431 __I 256781 I 0
T_0150_010_LASTPOInd 0.072 0301 I 9.143 0.466 0401 I 56025 I 0
T_0150_020_POSettle 0.105 0266 I 9.034 0.452 o3e1 I 56025 I 0
T_0160_010_DBTAuth 0.211 0.449 8.124 0.434 0.582 50396 i)
T_0160_020_DBTRecov 0.104 0215 I 9.087 0.382 oat I 50396 I 0
T_0160_030_DBTSettle 0.129 0249 I 9.019 0.406 0341 I 50396 I 0
T_0170_010_BCSHSettle 0.104 0.353 I 9.696 0.491 oat I s2i21 I 0
T_0180_010_ETAAuth 0.186 44 I 9.252 0.45 0561 I 42336 I 0
T_0180_020_ETAResov 0.099 0239 I 9.179 0.432 033 I 42336 I 0
T_0180_030_ETASettle 0.11 0.259 6.051 0.445 0.351 42336 1°)
7_0190_010_BDCSettie 0.401 035 I 9.628 0.482 o4et I 62556 I 0
7_0500_010_APOPAuth 0.189 0439 I 9.144 0.468 0551 I 55049 I 0
T_0500_020_ APOPDRecov I _0.105 0247 I 9.12 0.429 0341 I 55949 I 0
T_0500_030_APOPDSettle I 0.123 0279 I 9.019 0.46 oa7i__I 55949 I 0
Test Status: Passed
Date of Final Run: 17'" August 2010 at 10:23 to 11:28
Build Release: 01.22.00 / OSRs at 01.23.01.00 (all 4 BRDB nodes on new v249 blades)
TestID: 251
5.16 Test Group X13 — Logon / Logoff
5.16.1 Logon / Logoff Contracted / Design Limit Peak 5 Minutes
The test target was to achieve 100 logons & Logoffs per second (i.e. giving 200 business operations per
second), which is the design limit target. Transaction rates were ramped up as follows:
50 login/logoff per second for 5-minutes.
60 login/logoff per second for 5-minutes
100 login/logoff per second until test stopped.
Each logon comprises 3 x XML requests and each logoff comprises 2 x XML requests so the total
hits/sec when running at full load was 500.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREP (D866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 80 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
Due to limitations of the test tooling available, it was not possible to design a test that would do all logons
or all logoffs to test each individually. The above scenario was therefore considered to be a reasonable
compromise.
The test was run against only 3 active BRDB nodes and for longer than it needed to be, so as to give
added confidence that the rates could be sustained. The response times were acceptable and there were
only a very small number of failures..
Average Transaction Response Time - Transactions per Second
g 5
: z
é
©
g ieono) = 1a0sq0 = 4gio00 19400 ~«—«1a2n00 I taason —tgana0 I ae as00
Absolute Time hhrmm:ss
"010 FrstLogon "1820_Secondlogen 030_Gelttemos T_D40_CouréSusp
--050_Logott "010 FirstLogon:Pass [M}C_020_SecondLogon:Pass [il] C_030_GetMemos:Pass
TC _040_CountSusp:Pass _ EIC_050_Logott-Pass
Transaction Summary
[194302] [194250] {1e4231] [184283]
5 160,000
at
fH
e
£
Pe
s
5 20,000
2
£
5
Zz
5 5 2 e =
i i : i i
§ ‘ 5 ¢ §
% = 4 d
£ 5 3 Es g
rc 8 o 3 g
a Fi ek Pa o
5 pt 8 Ef
e I é é
o
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 81 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
Test Status: Passed
Date of Final Run: 10" March 2010 at 18:56 to 19:57
Build Release: 01.08.03 / CTR25.07
TestID: 072
5.16.2 Logon / Logoff Stress Test
The objective of this test was to take the user logon/logoff service beyond Design Limits up to a rate at
which either
a) the service is no longer able to support the transaction load and thus identifying the maximum.
headroom capacity limit, or
b) a notional transaction rate (as agreed with Capacity Management) such that, if achieved, it would
demonstrate sufficient headroom capacity for future growth without needing to find the absolute
breaking point
The ceiling target rate for the test was to attempt to reach 300 logons plus 300 logoffs per second for a
combined total 600tps. The actual rate sustainable before the response times increased and the
throughput struggled was 170 logons plus 170 logoffs per second.
Historical Peak I Contractual Limit Design Limit
5 5 i
Service I Hour I_min Hour I min Hour I min I Persecpeak I Sustained I __HIR%
Logon 170
Logoft 170
ToTAL I not spec'd I 77 I notspecd I 100 I notspectd I 120 I 200 340 I 183.3%
Each logon comprises of 3 x XML requests and each logoff comprises of 2 x XML requests. The load was
increased steadily throughout the test but began to struggle at just below 900 hits/sec, The equivalent
combined 340tps sustained is well in excess of design limits. There were no failures.
Total Transactions per Second
Showing - Average
1,100
ges
i of Transactions
{oSs00 e000 110500 111000 119800 112000 11.2500 113000 113800 114900 114500 11s000 115500 120000
Absolute Time hh:hh:mm
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref STS OUIRERIDERS
Limited 2010 UNCONTROLLED WHEN PRINTED OR bate tsep 2010
STORED OUTSIDE DIMENSIONS, Page No: 82 of 98
Fe)
FUJITSU
COMMERCIAL IN CONFIDENCE
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJ00097962
FUJ00097962
The response times remained steady up until the load reached approximately 170 logon/sec plus 170
logoffs/sec (i.e. 240 operations/sec) with the response times then increasing as the load further
increased.
Average Transaction Response Time - Transactions per Second
5
3
8
g
g 6
2 $
= 6
5 3
fo
z 11,0000 11;10.00 11:20:00 11:90.00 11:40.00 1:50.00
Absolute Time hh: hh:nm
, 020 _SecondLogon WB C_030_Gethemos '-040_CourtSusp
010 FirstLogon Pass [EIC_020_SecondLogonPass "030 _GetMemos:Pass
-050_Logott Pass
The following gives the summary of the response times (min/max/avg/90th percentile) for all the
transactions including total passes and failures.
Transaction Name Minimum Average I Maximum Stuy oo Pass Fail
Deviation Percent
C_010_FirstLogon 0.124 I 0.525 13.471 0.651 1.46 I 483420 I 0
C_020_SecondLogon 0.126 0.961 10.196 1.547 3.686 I 483420 I 0
C_030_GetMemos 0.07 0.151 7.583 0.226 0.451 I 483420 I 0
C_040_CountSusp 0.07 0.152 8.165 0.227 0.451 I 483420 I 0
C_050_Logoff 0.086 0.512 13.033 0.736 1.531__I 483420 I 0
Test Status: Passed
Date of Final Run: 1% July 2010 at 10:54 to 12:02
Build Release: 01.22.00 / OSRs at 01.23.01.00
TestID: 203
5.16.3 Maximum Concurrent Sessions
The objective of this test is to have 30,000 counter positions logged onto the Data Centre.
Over a period of time, logon of 30,000 counter positions was attempted and there were 241 failures.
However, this test is somewhat irrelevant because having 30,000 counter positions logged in does not
mean there are 30,000 open sessions from the BALs to the BRDB. The maximum number of sessions
possible at any one time is 20 (the max osr connections parameter) multiplied by 16 (number of active
OSRs during the test) multiplied by 4 (BRDB nodes) which equals 1280 concurrent connections. This has
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Ref: TST/SOT/REP/0866
Version: 1.0
Date: 17-Sep-2010
Page No: 83 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
already been achieved during many other tests. The fact this test is counter positions logging in is neither
here nor there because it is treated as just another online transaction.
However, the results of the test are reported below for completeness.
The graph below shows the transaction rates for the duration of the test. Logins are characterised by 3
XML submissions; FirstLogon, SecondLogon and GetMemos.
Transactions per Second
Number of Transactions
10:20:00 40:30:00 10:40:00 10:50:00 44:00:00 14:10:00 14:20:00 41:30:00
Absolute Time hh:hh:mm
C010 FirstLogon:Pass — - C_020 SecondLogonPass -* C_030_GetMemos:Pass J
The following graph shows the response times during the test.
Average Transaction Response Time
Average Response Time (seconds)
10:20:00 10:30:00 10:40:00 10:50:00 11:00:00 11:10:00 14:20:00 11:30:00
Absolute Time hh:hh:mm.
“= © 010 FretLogon > 6.020 Secoralagon +6 060 GelMenos —]
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 84 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
The table below is a summary of the test; showing the response time (min/avg/max/90th percentile) for
each transaction and also the number of transactions which ‘passed & ‘failed’. 29,759 successful logins
were completed.
Transaction Name Minimum I Average I Maximum I BY. I 80 1. I Pass I Fail
C_010_FirstLogon 0.124 0.132 0.64 0.017 0.14 29999 1
€_020_SecondLogon 0.135 1.077 17.404 0.524 1.339 I 29759 I 240
€_030_GetMemos 0.069 0.072 2.961 0.018 0.08 I 29759 I 240
The 241 failed transactions related to branches that mapped to Fad_Hash 0. This was a known issue (fix
still pending) as at the date and build against which the test was run.
Test Status: Passed
Date of Final Run: 23 April 2010 at 10:19 to 11:38
Build Release: 01.20.00 / CTR25.11
TestID: 145
5.17 Test Group X15 — Reports
5.17.1 Reports Contracted / Design Limit Peak 5 Minutes
These tests are covered by the Reports stress test. See section 5.17.2 below for detailed test results.
5.17.2 Reports Stress Test
This test was run twice; firstly with ORM ON (TestID 198) and then repeated with ORM OFF (TestID
199).
The following report types were included in the test. These comprise of 2 or 3 XML requests.
Report Type Description
7800_CDCHL_RPT Counter Daily Cheque Listing
2100_BALSS_RPT Balance snapshot
2400_OFFSS_RPT Office snapshot
2800_CSH_DEC Cash Declaration
2900_STK_DEC Stock Declaration
3000_STP_DEC Stamp Declaration
For each report type, branches were logged on, the report repeated 10 times and branches then logged
off.
The transaction rate sustainable versus the contractual and design limit was as shown below. The peak
reporting period is relatively short each day, therefore values are only specified for the peak 5 Minutes
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 85 of 98
oO
FUJITSU
COMMERCIAL IN CONFIDENCE
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJ00097962
FUJ00097962
transaction rates. It should be noted that the actual per second variance is much higher than for
transactional online services.
Historical Test 198 Test 199 HIR
Peak Design Limit I Sustained** I Sustained** I %
Per By By
Report XML Type SminI 5minI ©] sec} ypYI RepI yey I Rep
peak Type Type
T_1800_10_CDCHLRepServ uly uly
T_1800_20_CDCHLCutoff 4 34
T_2100_10_BALSSRepServ 434 [3s
T_2400_10_OFFSSRepServ ul 34 4[ 3s
T_2800_010_GetDec 34 34
T_2800_020_RecDec 4] 34 4) 4
T_2800_030_UpdBarCode 4 34
T_2900_010_GetDec 4 32
T_2900_020_RecDec 4] 34 32] 32
T_2900_030_UpdBarCode 4 32
T_3000_010_GetDec 34 33
T_3000_020_RecDec 34 34 33 33
T_3000_030_UpdBarCode 4 33
TOTAL XML Requests /sec 444 433
TOTAL Reports / sec too I 125 I 150] 300° 204 201 I 36% I
HR% above is the sustained tps rate % increase over and above the Design Limit Peak 5 min rate
* The Branch Access Layer should allow for an actual per second peak of 300 reports a second whilst
other transaction levels are low, otherwise 250.
** This test was constrained by the available number of Loadrunner VUsers, which limited the transaction
rate achievable rather than system resources being the constraint.
The graphs below show the combined rates of all the XML steps for each Report.
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Ref:
Version:
Date:
Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
86 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
TestID 198 (ORM ON)
Transactions per Second
Fiters: None
croup By Transaction End status
Cranalarity: 410 Seconds
‘Transactions per Second
‘Showing - Average
“0
»
5
3 so
Eom
Sx
3
é
5 10
2
feaoontaasoo 145000145600 TS0000 150500 ts1000 181600 182000152500
Absolute Time hh:mm:ss
TET 1000.70_ GOO RepSen Pace GADT1000_20_CDOHLOGOTT Pass BEIT 7100-10_BALSSRepSan Poss
00Li0LoProsRepseryPass I] Y_2600_010_Celbec Pore 500020 Recdes Pace
Upitercote ace LY
a UpiBercode ass YO
I 2000.090.eoarcode Paes
TestID 199 (ORM OFF)
ES or Gut topes Toth Wino
FSRUSRo/V x oo/ CRESS Ie Pi we
Stevan Rept I Ara Trea Rese Tm SSSR] el orate pr acd I Wart Seman I Wasa Rg. Ta Gc) [Yates «
a
“
=
2
HE
ES
aS
1e
‘°
°
Teepe ae 1
Barat] (DB —_ FO WNGRDeaoaaset_10 [Eve LoadRumer Analy _B) Arete Summary Roper. I Beko
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: = f.0.
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 87 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 "4
FUJITSU COMMERCIAL IN CONFIDENCE
The graphs below show the response times for the report steps.
TestID 198 (ORM ON)
Average Transaction Response Time
Te: ‘Average Transaction Response Time Graph data in excel format (83)
urventRemites — (abiooea ads tnglrepons 196i aspen a?e
Filters: “Traneaion End Status = (44), (Include Think Tis)
rou By:
Granadariy 40 seconds
Average Transaction Response Time
3
2
veston 145500 ‘soo tS0500~=—=sts100 = tS1500 «= 1S@MOD 152509
Absolute Tire hh:mm:ss
TestID 199 (ORM OFF)
Average Transaction Response Time
Tite: Average Tr: ction Response Time Graph data in excel format [3
Current menue: (ahrnoeaT ads seslerepera.t98,e mere a
Fitares “rantadtion End statue = (Paes), (Indude Think Tine)
Group By
Cranudatty: 10 seconds
—_ ‘Average Transaction Response Time
a as a
Ess
3 =
Eas
:
os
. 15:55:00 16.00:00 18:05:00 ‘t610000 16:15:00 162000 16:25:00
‘Absolute Time hh:mm:ss
THO BALSGRepSew Tan 10_GFFSSRepSene
cbec 9 T-pO00-030-Upetwende 6. 172000,00.eDee
arose + T= 3IC0_D10_OaDe0 Recdee
“1am -130- perce
The tables below provide a summary of each test; showing the response time (min/avg/max/90th
percentile) for each ‘step’ of the reports run and, also, the number of steps which ‘passed & ‘failed’.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 88 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 .
FUJITSU COMMERCIAL IN CONFIDENCE
TestID 198 (ORM ON)
Transaction Name Minimum I Average I Maximum I 5c &) Pass I Fail I %Fail
feviation I Percent
C_010_FirstLogon oes I 0199 I 7253 I 0279 I 032 I 1281] 5 I 04
C_020_SecondLogon 0.144 3.914 20.02 3.996 10.232 1,116 I 115 I 9.34
C_030_GetMemos 0.07 0.079 0.521 0.033 0.08 1,104 12 1.08
C_040_CountSusp 0.072 0.308 9.829 0.899 0.35 849 O 0
C_050_Logoff 0.085, 0.546 10.687 1.065 0.903 849 O i)
T_1800_10_CDCHLRepSevv I 0.087 I 0.197 I 7612 I 0123 I 035 I73661I 1 I 0
T_1800_20_cDcHLCuto I 0089 I 0.125 I 10573 I 0126 I 021 I 7358] 4 I 001
T_2100_10_BALSSRepSev I 0.11 I 0.227 I 10301 I 0141 I 0371 I 73580I 1 I 0
T_2400_10_OFFSSRepServ 0.115, 0.232 7.96 0.125 0.381 73,410 3 i)
T_2800_010_GetDec 076 I 0155 I 6932 I 0089 I 019 I 72677I 0 I 0
T_2800_020_RecDec 0.001 I 0215 I 9108 I 0258 I 0592 I 72676I 1 I 0
T_2800_030_UpdBarCode I 0.084 I 0.115 I 10.621 on ota I 72676I 1 I 0
T_2900_010_GetDec 0076 I 0152 I 108s I 009 I o18 Ie9asiI 1 I o
T_2900_020_RecDec o3at I 1492 I 13439 I 2165 I 5305 I 69a50I 2 I 0
T_2900_030_UpdBarCode 0.067 0.108 5.608 0.075 0.17 69,851 1 0
T_3000_010_GetDec 076 I 0155 I 9413 I 0.091 02 I 7724] 2 I 0
T_3000_020_RecDec 0.098 I 0372 I 1057 I 0487 I 1.166 I 71,726I 0 I 0
T_3000_030_UpdBarCode I 0.084 I 0.115 I 7.385 I 0.092 02 I 71,726] 0 I 0
TestID 199 (ORM OFF)
Transaction Name Minimum I Average I Maximum I _ Std. 90 I Pass I Fail I %Fail
Deviation I Percent
C_010_FirstLogon 0.097 0.183 73 0.293 0.23 1,209 13 1.06
C_020_SecondLogon 0.141 3.601 20.007 3.919 9.13 41,115 94 7.78
C_030_GetMemos 0.07 0.082 6.961 0.208 0.08 1,106 9 0.81
C_040_CountSusp oo72 I o2i7 I 7152 I 0553 I 0247 I 49 I 0 I 0
C_050_Logoft oss I 033 I 7763 I 0635 I 047 I e49 I 0 I 0
T_1800_10_CDCHLRepSevv I 0.086 I 0.153 I 3582 I 0033 I 016 I72,269I 1 I 0
T_1800_20_CDCHLCutOff 0.089 0.096 5.196 0.039 0.14 72,266 4 0.01
T_2100_10_BALSSRepServ 0.14 0.186 8.358 0.058 0.2 72,195 4 oO
T_2400_10_OFFSSRepServ 0.116 0.189 4.992 0.049 0.21 72,300 2 oO
T_2800_010_GetDec 0.077 0.142 3.747 0.041 0.15 71,723 4 oO
T_2800_020_RecDec 0.091 I o105 I 5703 I 0049 I 013 I 71,722] 2 I 0
T_2800_030_UpdBarCode I 0.084 I 0.091 I 26.015 I 0.106 ot I 71723I 1 I 0
T_2900_010_GetDec oo76 I 0143 I 7.281 I 0057 I 015 I 72399] 0 I 0
T_2900_020_RecDec 0328 I 0524 I 6126 I 014 I 0652 I 72309] 0 I 0
T_2900_030_UpdBarCode I 0.083 I 0.091 I 2819 I 0.028 ot I 7a309I 0 I o
T_3000_010_GetDec 0076 I o142 I 3.805 I 0033 I 015 I 70557I 0 I 0
T_3000_020_RecDec 0.098 I 0144 I 7.257 I 0063 I o19 I 70557] 0 I 0
T_3000_030_UpdBarCode. 0.084 0.091 2.421 0.026 0.4 70,555 2 oO
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
i Version: 1.0
Limited 2010 UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 89 of 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
oO
FUJITSU
COMMERCIAL IN CONFIDENCE
The graphs below show the total errors per second during the run. They were mostly concerned with the
logons at the beginning and logoffs at the end of the run. Issues with logon / logoff failures have been
resolved since this test was run.
TestID 198 (ORM ON)
Total Errors per Second
Tie: Total Erors par Second
D:\HNGx\Phare2\Results\20100627_x45_tinglereports 198
Current Results: 29200627. 15_smglereports 196.1
Graph data in Excl format (3
Filters tone
crow
rome 10 seconde
Total Errors per Second
Showing -Aweraye
°
55
5
“s
Was
eo
€ 2s.
2 2
15
1
os
Te4o00 144500 1eeno 14500 tsano0 S050 isto Ta7s00 tsan00 182500
Absolute Time hems
fay)
TestID 199 (ORM OFF)
a
s
=
5
%
oY
op
(ea)
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TSTSOTIREPI086
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS
Page No:
90 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
Test Status: Passed
Date of Final Run: 27" June 2010
Build Release: 01.22.00 / OSRs at 01.23.01.00
TestID: 198 (ORM ON) and 199 (ORM OFF)
5.18 Test Group XCB — HNG-X Combined Banking
5.18.1 HNG-X Combined Banking Contracted / Design Limit Peak Hour
/ Peak 5 Minutes
These tests are covered by the Combined Banking stress test. See section 5.18.2 below for detailed test
results/:
5.18.2 HNG-X Combined Banking Stress Test
The objective of this test was to take the Banking service beyond Design Limits up to a rate at which
either
a) the service is no longer able to support the transaction load and thus identifying the maximum
headroom capacity limit, or
b) anotional transaction rate (as agreed with Capacity Management) such that, if achieved, it would
demonstrate sufficient headroom capacity for future growth without needing to find the absolute
breaking point
The ceiling target rates for the test were as follows.
Basket Hits/sec I Auth/sec
0010_BKT01_CPO_WDL — CAPO 900 300
0012_BKTO1_LNK_WDL - LINK 300 100
0011_BKTO1_AL_WDL — A&L. 90 30
1290 430
Transaction rates for CAPO, Link and A&L were taken through their respective contractual and design
limits before the following rates were sustained.
Historical Peak Co Design Limit
Service
Type Hour 5 min Hour 5min I Hour I 5 min Per sec peak I Sustained HIR %
CAPO 409 124 134 160 161 192 250 225 17.19%
LINK 14 17 27 32 33 39 51 70 79.49%
AL 6 7 10 42I 12 14 19 30 I 114.29%
TOTAL * 110 176 174 204 I 206 245 not specified 325 32.65%
:* The target for combined total banking is not the same as the sum of individual parts
NB: design limit values for LINK are incorrectly stated in ARC/PER/ARC/0001. The numbers here are as
advised by Capacity Management and are derived from those specified in PA/PER/033
For CAPO, whilst a rate of 225tps was sustained (exceeding the Design Limit for Peak 5 minutes of
192tps), an actual per second peak of 260tps was achieved (exceeding the Design Limit for actual per
second peak of 250tps)..
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 91 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
The following graph shows the overall transaction rate for the run. The ceiling target limit was NOT
reached.
Total Transactions per Second
Showing - Average
Total Number of Transactions
14:30.00 11:3500 18:40:00 11:45:00 14:60.00 11:8500 120000 120500 121000 121800 122000
Absolute Time hh:mm:ss
The following graph shows the transaction rates for CAPO and the response times. Between 12:00 and
12:15 CAPO should have been running at the target rate of 250 Auths/sec which equates to 750 hits/sec
across the Authorisation/recovery/settlement XML. This was NOT achieved; at its maximum 225
Auths/sec was sustained.
After 12:15 when additional load to CAPO was applied the response times increased and the actual
throughput reduced.
Average Transaction Response Time - Transactions per Second
A
i
s
g
- §
2
#3
& 2
$ 45
iH os
§
g 0
@ 11:90:00 11:35:00 11:40:00 11:45:00 11:5000 11:5500 120000 120500 121000 121500 122000
Absolute Time hh:merss
[T0010 _010_cAPOAth T_0010_020_CAPORecov T_0010_090_CAPOSetie
GB T_0010_010_CAPOAuthPass [BI T_0010_020_CAPORecov Pass lll T_0010_030_CAPOSette:
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
fi Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 92 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2
FUJITSU COMMERCIAL IN CONFIDENCE
The following graph shows the transaction rates for A&L and the response times. It can be seen from the
graph that the target rate of 30 Auth/sec equating to 90 hits/sec across the
Authorisation/recovery/settlement XMLs was achieved
ABL
Average Transaction Response Time - Transactions per Second
160
140 i
420
10 8
o 3
60
: j
A . *
i 0
3 4s
é
i 1
2 os
é
go
2 11:90:00 11:9800 11:40.00 11:4800 11:8000 11:5500 120000 120500 121000 121500 122000
Absolute Time hh:mm:ss
GT_0011_010_ALAuh ()T_0011_020_ALRecov Gy T_0011_030_AL Sette BT _0011_010_ALAuth Pass
"0011-020 _ALRecov Pass [J 1_0011_030_ALSettiePass
The following graph shows the transaction rates for LINK and the response times. It can be seen from the
graph that the target rate of 100 Auth/sec equating to 300tps across the
Authorisation/recovery/settlement XMLs was NOT achieved because the Authorisation response times.
increased dramatically after 12:14. However a rate of around 70Auths/sec was achieved which equates
to 210 hits/sec across the Authorisation/recovery/settlement XMLs
LINK
Average Transaction Response Time - Transactions per Second
= I
10
m 5
g
$ 2s
& 2
i 15
g 4
© os
H o
I BR eer sopro caper aspen npr cag rnongaanagraunpannngenss
Absolute Time hhime:ss
DT_0012_010_ neath TT 0012020 LnwRecov MB T_0012_030_LAMSette
1 LOIOLLINKAUnPass HD J20_LINKRecov Pass [i T_0012_030_LIKSettie: Pass
The following gives the summary of the response times (min/max/avg/90th percentile) for all the
transactions including total passes and failures. All failures occurred with CAPO.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref STS OUIRERIDERS
Limited 2010 Version: = 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 93 of 98
FUJ00097962
FUJ00097962
oO VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
Transaction Name Minimum I Average I Maximum I St. a Pass I Fail I %Fail
eviation I Percent
€_010_FirstLogon 0.125 I 0.148 0.28 0.022 0.18 I 1000 I 0 0
€_020_SecondLogon 0.13 0.198 I 0.682 0.054 I 0.269 I 1000 I 0 0
C_030_GetMemos 0.07 0.079 I 0.545 0.03 0.09 I 1000 I 0 0
€_040_CountSusp 0.072 I 0.079 I 0.548 0.026 0.09 I 1000 I 0 0
€_050_Logoft 0.083 I 0.103 I 0.402 0.037 I 0.149 I 1000 I 0 0
T_0010_010_CAPOAuth 0.165 I 0.803 I 20.334 I 1.485 I 1.339 I 535857 I 717 I 0.13363
T_0010_020_CAPORecov I 0.09 0.12 4.453 0.079 0.13 [536574 I 0 0
T_0010_030_cAPOSettie I 0.121 I 0.134 I 3.365 0.04 0.i4_[ 536573 I 1 I 0.00019
T_0011_010_ALAuth 0.17 0.317 I 3.322 0.096 I 0.401 I 5i4i2 I 0 0
T_0011_020_ALRecov 0.104 I 0.115 I 2.996 0.069 0.13 I 51412 I 0 0
T_0011_030_AlSettle 0.122 I 0.133 I 3.336 0.044 oa I 51412 I 0 0
T_0012_010_LINKAuth 0.181 I 0.864 I 10.335 1.567 I 0.884 I 132538 I 0 0
T_0012_020_LINKRecov I 0.105 I 0.115 I 4.005 0.07 0.12 I 132538 I 0 0
T_0012_030_LINKSettle I 0.122 I 0.131 I 3.247 0.04 0.14 I 132538 I 0 0
Analysis of Athene Data showed no bottleneck in terms of the system performance characteristics of any
of the system components. Instead, the limitation was confined to the performance capability of the agent
code. However, each banking service type individually and combined exceeded the respective design
limits.
Test Status: Passed
Date of Final Run: 30" June 2010 at 11:28 to 12:23
Build Release: 01.22.00 / OSRs at 01.23.01.00
TestID: 202
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE:
UNCONTROLLED WHEN PRINTED OR
STORED OUTSIDE DIMENSIONS
Ref:
Version:
Date:
Page No:
TST/SOT/REP/0866
1.0
17-Sep-2010
94 of 98
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
[oe]
FUJITSU COMMERCIAL IN CONFIDENCE
6
Additional Testing
Over and above the planned tests specified in the HLTP, additional testing was also conducted in support
of the following:
°
°
°
°
°
°
Red Alert Peak list
Various Oracle Patches
Streams Fixes
ORM configuration settings
BAL/OSR Fixes
pblade v249 upgrade
Specific test details for any of the above are available on request.
7
Lessons Learned
The usefulness of the VOL testing should not be underestimated. For example,. many examples
of “timing” issues were found during VOL testing that could not otherwise have been identified.
Original project plans did not make sufficient allowance for volume and performance testing. The
test cycles were ‘squeezed’ only for reality to bite later.
Expectations of performance testing was sometimes unrealistic; The team were often requested /
expected to run tests for other teams at ‘the drop of a hat’ without prior warning or planning. On
occasions the requests were not supportable as appropriate test data could not be prepared in
the time available.
During the early stages of testing in particular, development of counter changes invalidated the
prepared Loadrunner test scripts and rendered them unusable. This resulted in considerable re-
work having to re-capture the XML that forms the basis of the scripts. Later developments tended
to be backwards-compatible.
Whilst testing achieved the aim of understanding the performance profile of the solution, it is not
possible to truly replicate behaviour of a live estate due to the considerable variances in day to
day activity, the complexities of the solution, and the limitations of test tooling. Expectations in
this respect sometimes needed to be managed.
In some cases, the limitation of available Loadrunner VUser licenses constrained the amount of
load that system components could be subjected to.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 95 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
8 Recommendations
o Consider upgrade of the pblade for BRS to v249:
The performance improvements demonstrated on the upgraded BRDB nodes suggests that an
upgrade of the BRSS would further improve Streams performance.
o Review ORM configuration settings for min / max servers:
Anumber of ORM configuration settings were exercised during the Red Alert testing phase with
settings of Low=7 and High=16 being selected for deployment into the Live estate. Once the
new hardware v249 blades were available, further ORM configuration testing was conducted
including with ORM switched off. The Technical Review group concluded that the optimal
configuration settings were Low=16 High=32. TestID:240-246 record the ORM test results.
During the tests above, ORM did not activate at all. The Technical Review group were unable to
identify a test which would “force ORM to invoke”. It is felt that there is a degree of risk in terms.
of not being able to observe the solution behaviour of ORM activating on the new hardware. It is
recommended that further reviews of the need for ORM are conducted with a decision to
(a) switch off ORM, or
(b) remain with the current settings, or
(c) change the settings to the optimal determined during TestID:240-246.
NB: Further to the above, the settings have now been changed in ‘live’ to settings of
low=16 and high =32. This was applied by MSC043J0277083 on 15'" September 2010.
o Review OSR min /max connection settings:
Tests have been conducted with various min / max connection settings up to 30 / 30. The optimal
settings should be considered in light of the test results observed.
NB: Further to the above, the min connection setting has now been changed in ‘live’ toa
value = 20. This was applied by MSC043J0277111 on 17" September 2010.
o Update Operations Support Guide for re-introducing a previously failed BRDB node during
the on-line day:
If a previously failed node is to be re-introduced during the on-line business day (the current
recommendation in the Operations Support Guide is to only reintroduce a node overnight), then
careful consideration needs to be made as to the potential impact on service. The amount of time
it will take for live service to recover from either a node failure or reintroduction of a node, will be
higher or lower depending on the transaction load at the time. Also, in terms of reintroducing the
node, recovery time will depend on how long the node has been down. This is because the
longer an oracle instance has been unavailable, the more time it will take to go through a re-
mastering exercise with the other nodes. This has an impact on the node performance and will in
turn impact the transaction load it can process. A FadHashRefresh on the BAL servers to pick up
the reintroduced node should not be performed until re-mastering activity on the BRDB nodes
has completed. The Operational Support Guide should be updated to note all relevant
considerations when re-introducing a node.
o Implement Egenera recommended configuration settings:
The Egenera recommended configuration settings aimed at preventing BDB node evictions (as
proven by Peak testing against PC0197725) should be applied to the Live estate at an
appropriate time during Release 2 timescales. The possible exception being the changes to CPU
affinity.
o Consider upgrade of all BAL servers to v216
The current BAL server estate is a mixture of v216 and v411 servers. Testing has demonstrated
that the v216 servers have double the processing capacity of the v411 servers. At peak
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
i Version: 1.0
Limited 2010
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 96 of 98
Fe)
FUJITSU
FUJ00097962
FUJ00097962
VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
COMMERCIAL IN CONFIDENCE
transaction volumes, as the 411's reach saturation, the amount of packets they take in stops
increasing. The 216's having double the power, can take in double the amount. This leads to an
in-balance of workload across the estate at peak volumes
Consider life without VOL
Future performance testing or modelling capability should be carefully considered, not only for
future releases but also in support of ‘live’ as full rollout continues to maturity.
© Copyright Fujitsu Services
Limited 2010
COMMERCIAL IN CONFIDENCE Ref TSTSOTIREP (D866
Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 97 of 98
FUJ00097962
FUJ00097962
o VOLUME AND PERFORMANCE TEST REPORT - PHASE 2 “i
FUJITSU COMMERCIAL IN CONFIDENCE
9 Conclusions
A significant number of performance related issues have been identified and resolved throughout VOL
testing. Additional testing over and above that specified in the HLTP has also been conducted to support
further ‘tuning’ of performance characteristics. As at completion of Cycle 3, testing has demonstrated
ability of the solution to support full contractual and design limit workloads for each of the service types
under test. Stress testing has also identified the headroom capacity to support future growth in business
volumes.
Some Peaks remain open which would deliver further performance improvements. Whilst desirable, they
are not deemed critical to be able to support current live volumes.
© Copyright Fujitsu Services COMMERCIAL IN CONFIDENCE Ref TST/SOT/REP/O866
Limited 2010 Version: 1.0
UNCONTROLLED WHEN PRINTED OR Date: 17-Sep-2010
STORED OUTSIDE DIMENSIONS Page No: 98 of 98