Exception plan for delivering KMS within current CSR+
timescales
1. Background
This paper follows the paper entitled “Requirements for KMS at CSR+’. It supposes that
Pathway accepts the KMS as critical for CSR+ acceptance, and summarises the various options
that was open to Pathway to de-risk the delivery of KMS within Programme timescales. These
timescales are themselves constrained by a number of outside influences:
> Itis critical that ICL achieves go live of CSR+ on 17" April 2000
» This is required to bolster ICL’s financial position with regard to flotation and the
consequential benefits related to that
Currently KMS Development, including Module Test but not Link Test, is on track to the plan
baselined in April. However, continuing increase in workload, coupled to a lack of contingency
and budget allocation to cover all planned work have resulted in an increasing risk that the
KMS will not be delivered within current Programme timescales.
Additionally, it has been accepted that there is no budget provision that could be allocated to
‘shore up’ the current development plan, and that the current aggressive downsizing of the team
after July will cause “Phase 2’ deliveries to be slipped should any issues be found in the testing
of the first Phase of delivery.
Therefore, this paper was commissioned to consider the options available to Pathway to effect
the successful delivery of KMS for CSR+.
2. The way forward
2.1. Options
This section gives an overview of the options open to Pathway with regard to the delivery of KMS.
Although it does not expand on any given option in any depth, it affords the reader an insight to the
considerations undertaken before a decision was made with regard to the exception plan.
2.1.1 Maintain Current plan
Achievement against the current plan, as previously discussed, was displaying increasing
workloads to meet milestone dates. It was becoming increasingly unlikely that the team could
continue to maintain the increased effort in order to meet the scheduled delivery dates for code
completion.
Additional to this the scope for Development Link Testing (DeLT) had been completed which
resulted in a longer than anticipated DeLT phase. Pressure from the opposite end (BTC), along
with emerging migration test requirements had caused the current plan to be non achievable
without additional resource. This was further compounded by the extensive ‘ramp down’ of the
team afier code completion of the first Phase of delivery, adding significant risk to the support
capability of the team whilst having to achieve equally aggressive delivery dates for the planned
Phase 2 work.
Overall, even with additional resource, there was significant risk that the KMS would not have
undergone sufficient testing in time to meet the current Programme Timescales. The table
below demonstrates the timescales involved:
FUJ00077864
FUJ00077864
Timescale Jun_I Jul I Aug I Sep I Oct I Nov I Dec I Jan_I Feb I Mar
KMS Devt Phi Ph2
KMS DeLT Phase 1 Prtl Pri2
EXCEP2.doc Page I of 10 v0.2
FUJ00077864
FUJ00077864
KMS DeLT Phase 2 Prt! I Prt2_I Prt3
TI Pll I P12 I P2.2 I P2.3
System Test PL. I P12 I P2.2 I P2.3
B&TC P12 I P2. I P2.
2 3
Even if Pathway were to accept the above timetable, considerable additional resource would be
required to give effective support whilst completing the Phase 2 coding. This in itself may not
be sufficient should any significant problem be found during the latter stages of testing.
2.1.2 De-scope: e.g. Go live with “phase 1”
Phase I of the KMS was never put together as a coherent set. Although it does provide
functionality which can be tested, it does not represent functionality that could make up a
release.
The advantages of this approach s that there will be less modules to both develop and test.
Also, completion of the deliveries from Development to the test phases will be significantly
sooner than the two Phase Strategy. However, to achieve this, some modules will need to be
brought forward from the Phase 2 development work. There will be a further requirement for
additional manual processes to be in place until such time as the Phase 2 functionality can be
introduced.
It should also be noted that there will still be a requirement for additional resource to maintain
the current quality processes.
2.1.3 Invoke the Contingency Plan
Early on in the KMS Project, it was established that it fell on the critical path for CSR+
delivery. As such, effort was expended to look at options for contingency should any major
problems be encountered in the development cycle. This is detailed in TSC/CRY/006.
Although on the face of it this may be considered a viable option, it should be noted that the
contingency plan was established to mitigate against “disaster? as opposed to being a cost
effective and viable long term solution. As such it has the following drawbacks:
lL. Extra cost & resources required to achieve this.
2. These really need to start immediately as elements of the plan are sketchy.
3. Guarantees that we take our eye off achieving the current KMS plan; it is not possible
to do both.
4. Contingency plan still subject to all the risks of the migration issues; indeed it is KMS
that provides a migration mechanism (but still subject to risks), and without KMS, we
have to achieve the migration manually.
2.1.4 Negotiate out contracted commitments
To de-scope the requirement for KMS, Pathway would need to negotiate ‘lets’ on the
Cryptography requirement, i.e. KMS only supports the crypto solution. To make significant
inroads into KMS complexity the following five requirements, specified in either the contract
or contractually controlled documents, need to be negotiated out of CSR+:
. AP Signing
° VPN (with individually managed Keys)
. Crypto protection for the Quantum product
. Keys being generated using hardware random number generation (including
FEKs)
. Public Key Certificates
History shows that the customer has been intransigent with regard to relaxing the ‘contracted’
security requirement. Although the removal of BPS would give credence to a review of the
EXCEP2.doc Page 2 of 10 v0.2
security solution, in particular the use of cryptography, the timescales in which CSR+ is
operating make reliance on such activity too high a risk.
2.1.5 Spend contingency plan / other money on current plan
The contingency plan is not cheap and would need to be effected in addition to current work on
the KMA (holding the KMA team on longer to complete this in the long run). The extra costs of
doing this could be spent on bolstering the current plan and de-risking the Development link
Test and KMS System Test. However, this would require the employment of additional
resource, who would require a working environment. This would be difficult to achieve in the
current climate at Bracknell.
2.2. The way ahead
Steer from the meeting with Terry Austin held 9" June
At the meeting held on 9" June [att. Terry Austin, Pete Jeram, Alan D’Alvarez, Chris
Humphries, Gill Jackson] the above options were considered with the following results:
2.1 Maintain current plan - considered too high a risk in the light of increasing work
over that planned and the resulting pressure on the teams ability to meet planned dates
2.2 De-scope : Go live with “Phase 1’ — Considered a viable way forward with
minimal overall impact on cost
2.3 Invoke the contingency plan — Considered to have a fairly high risk factor which
would incur additional nugatory cost
2.4 Negotiate out contracted commitments — Considered too high risk due to past
customer intransigence
2.5 Spend contingency plan / other money on current plan — No budget available to
fund this option
The clear steer from the meeting was to pursue option 2.2 and to scope whether this was a
viable option within the current budgeted head count. Completion of Phase 2 would need to be
scheduled for introduction in a future release supported by a budget CP.
3. Exception plan for de-scoping the content of KMS at CSR+
Accepting the above steer, this exception plan documents the deliverables required to effect a
functional KMS for CSR+. It also considers the components being de-scoped from this release,
along with the impacts and associated risks.
3.1 Phase 1+ KMS
The Drivers for cryptography at CSR+ are:
¢ Crypto is required to support the business requirements of APS & L&G
© Crypto is required to support the infrastructure requirements of VPN & FEKs
¢ = Certification is required
The events when the KMS functionality are required:
Data Centre Migration to CSR+
Roll-out of CSR+ Counters
Counter Migration to CSR+
Counter Swap out
Key Compromise
Uwe
EXCEP2.doc Page 3 of 10 v0.2
FUJ00077864
FUJ00077864
FUJ00077864
FUJ00077864
6. Key Change
The phase I KMS devt to KMS DeLT drop targeted for July delivers the majority of code for
this functionality. However, there are a number of deliveries, currently scheduled for Phase 2,
which contain functionality required to manage the above events.
The phase 2 KMS drop contains the following modules:
¢ Manual New Key
¢ Key Importer
¢ Capu checker
e Garbage collector
¢ Integrity Checker
¢ Revoker
The above functionality can be achieved if we add to phase I the following three components
currently scheduled as part of the Phase 2 delivery:
¢ Manual New Key Required to support manual channel for Sl and KMA
e Key Importer Required to initialise KMA and for migration of SI
The following table annotates the events will be satisfied; firstly within the original scheduled
release, and then the proposed additionalmodules to be brought forward from Phase 2:
Event Phase1 Phase2 Phase1+ Phase 1+ Component
1 Data Centre Migration to CSR+ XxX Y Y Man New Key/Key Importer
2 Roll-out of CSR+ Counters xX Y Y Man New Key/Key Importer
3 Counter Migration to CSR+ Y - Y
4 Counter Swap out Y - Y
5 Key Compromise Xx Y Y Revoker
6 Key Change Y - Y
So how would we live in a phase 1+ world without:
1. Capu checker Work round this with Tivoli type scripts; developed after July
2. Garbage collector Defer to later release (sufficient EMC disc space for 2 years)
3. Integrity Checker Either drop (and accept the risk involved) or slot in later
3.2 Additional actions to reduce KMS content for CSR+
Further actions to reduce KMS content in phase 1+:
The requirement for early VPN (i.e. prior to introduction of KMS) means that an off line
process is required to generate the initial set of VPN keys for the campus NVPN (Private VPN.
Keys for Campus VPN Servers) and FVPN/EVPN (Global keys for Post Offices).
This will be performed using the KMS CA Workstation (i.e. a Utimaco Cryptware server) using
the standard Utimaco GUI interfaces. By providing secure access for an additional CAW role,
the management of these keys can be kept manual for KMS phase 1+. Live support for these
platforms by KMS would be restricted to CRL distribution. Design effort is required (post
July) to document the process for management of these including key compromise situations.
The latter would require testing, but there is minimal KMS specific code.
The removal of BPS means that 3 protection domains can be removed, and consequently
implementation of any remaining incomplete Phase 2 KMA deliveries can be reduced, for
example:
3. CAPS No support for CAPS keys in Key Importer and Manual New Key
EXCEP2.doc Page 4 of 10 v0.2
4. PA No support for migration of PA keys in Key Importer
5. CMS No special case code (i.e. data driven) but no need to test this feature.
[Note that there is a potential requirement for PA key to sign AP acknowledgements, but this
proposal is not within the scope of Phase 1+. Any such move to introduce this for CSR+ would
need to be covered by a CP]
Pathway confirmed that KMS at CSR+ should only interface with counter facing Keys; which
will significantly reduce the test requirements for the KMS at CSR+ on the following
platforms:
1. FTMS links for POCL TIP
FTMS links for Archive Server
FTMS links for HAPS disaster standby system
FTMS links for POCL TIP disaster standby system
FTMS links for AP Clients
6. Rambuttan reminder mechanism for Key Manager
This means that for all except the last item, these would (continue to) be supported by an NR2
style offline Manual Key Service operation. There is a residual issue concerning the
requirement for Public Key Certificates for these platforms which is being investigated by Dave
Johns.
weEYWN
3.3 Implications on Testing phases:
1. Review of risks, identify an immediate release to other Pathway DU test streams of the
CSR+ Crypto product on the supported platforms (ie. FTMS platforms would stay with
NR2 Crypto)
2. Single drop strategy into DeLT means that KMS DeLT can proceed without use of testing
stubs, effectively being a pre-PIT systems test but without fully representative platform
builds
3. By going for a single drop of KMS, we can eliminate some aspects of the work — in
particular re-work building different environments as new code arrives
4. Some code may be released at an earlier stage in quality cycle (e.g. pre code inspection /
module test), providing main paths are unit tested, further updates could be incorporated in
bug fix handovers as a calculated risk
5. Testing time is only really reduced by removing functionality from product. Phase 1+ goes
some way on this, but need to re-review in case further de-scoping is possible to reduce risk
3.4 Follow on KMS release:
This de-scoped release of KMS provides a viable system for managing Crypto keys for the
initial period, but an update to KMS will be required within 2 years to address the shortfall
1. The FTMS software will need validating on the new style crypto to address shortfall such
as Certification
2. To provide a manageable way of supporting the target AP Client population (300?)
3. To reduce the support load on the Crypto team by removing support for NR2 code
4. To reduce the ongoing Managed Key Service workload and skills necessary to do it.
5. To automate the CAPU checking mechanism and reduce manual load
6. To provide garbage collection for KMA database
7. To manage all VPN keys, and remove manual (potentially error prone) mechanisms around
Key changing and revocation.
8. To rectify any shortfall and manual work-arounds around Key compromise and revocation.
9. To support additional Protection Domains as they emerge (e.g. signed AP
Acknowledgements).
Appendix A shows the revised KMS Development project plan to support the Exception Plan.
EXCEP2.doc Page 5 of 10 v0.2
FUJ00077864
FUJ00077864
4 SDU Test Strategy to Support Delivery of Phase 1+
Appendix B shows the high level SDU test project plan supporting the Phase 1+ strategy.
Contained within this are three delivery streams into Technical Integration, after which formal
System testing can take place. During each delivery stream, there will be an element of joint
Link Test/System Test activity, much of which will be done prior to the product being handed
over to PVCS. This will allow the teams to stabilise the release prior to the application being
entered into a formal baseline which would bring with it the extensive overheads associated
with introducing fixes on. At this stage, the team will utilise an access database to track and
record progress with regard to any bugs, fixes and issues.
Each ‘Baseline’ will have a content description, the target environments for that baseline and
availability dates from Technical Integration. There will also be a dependency list which, if
missed, will slip dates. Risks are catalogued at the back of this paper.
4.1 Baselines
4.1.1 Baseline 1
This baseline will consist of the redelivery of Crypto code which will be supported by KMS.
This element is seen as invasive to the business functionality and therefore has been targeted
for introduction to the test arena as soon a practicable. Once business applications have
confirmed that they are ale to sign and verify messages with the revised Crypto code, there will
be minimal requirement for regression testing with regard to the business applications on future
baselines.
Content:
> Crypto functionality, AP and SI - module tested
Stubs for the supply of Crypto Keys.
Siemens Metering protection (to be confirmed)
Will not support migration (will require handcrafting for CSR - CSR+ building)
Will not support VPN (VPN will be tested utilising Global Key at this stage)
POLO will be retained at CSR (not CSR+ product)
will not support CSR+ autoconfig.
VVVVVVY
Target Environments:
» Security Delivery Unit System Test - Crypto and VPN (Global Key)
> POCL Products Delivery Unit
> B&TC
Availability from Technical Integration:
> System Test - 13" August
> B&TC - 15" October
Dependencies/Assumptions:
» System Test requires environment (both space and test rig) to be built and operational by
30" July
» System Test will require KMS rig, VPN rig and Secure Builds rig (to be defined)
» Technical Integration will take 5 days to assimilate handover into a baseline
4.1.2 Baseline 2
This will consist of the first delivery of KMS functionality to support the revised Crypto code.
The prime purpose of this baseline is to pipe clean the formal build process and to start the
SDU System Testing of KMS. Hence, there is no target audience for this baseline outside of
the SDU or Technical Integration. There will be a link test phase along with a joint system test
pre validation exercise prior to hand over to PVCS.
EXCEP2.doc Page 6 of 10 v0.2
FUJ00077864
FUJ00077864
Content:
> Phase I KMS (with the two additional modules)
Target Environments:
» Security Delivery Unit System Test
> Technical Integration
Availability:
» Into Technical Integration - 24 August
» Into SDU System Test 7 September
Dependencies/Assumptions:
Link Test requires delivery of equipment by 30" June
Will require Technical Integration resource assigned to the Crypto Team early in July
Technical Integration will take 10 days to assimilate handover into a baseline
Will require CM/Technical Integration to maintain a dual baseline
VVVWV
4.1.3 Baseline 3
This will consist of the first delivery of KMS functionality to all units in Pathway.
Content:
> Phase I KMS (with the two additional modules)
> Test Keys
> Supports migration
Target Environments:
Security Delivery Unit System Test
Internal Infrastructure Delivery Unit System Test
Technical Infrastructure System Test
B&TC
Live
VVVVYV
Availability:
» From Technical Integration for System Test - 15 October
> For B&TC - 26 November
Dependencies/Assumptions:
» Migration dependencies for Data Feeds, Autoconfig, Counter Processes and System
Management are met
The KMS specific platforms will be required to utilise this baseline
Will require other Delivery Units to either have a KMA to test Key interfaces, or to
utilise/develop stubs
» May require CM/Technical Integration to maintain multiple baselines
>
>
5 Risks
Going live with Phase I of the KMS delivery, along with selected other modules, presents a
number of risks. These are catalogued in more detail in the risk paper. The Crypto team have
been working to the production of KMS to an established baseline and it must be recognised
that a change in focus and direction in itself presents risk.
EXCEP2.doc Page 7 of 10 v0.2
FUJ00077864
FUJ00077864
FUJ00077864
FUJ00077864
The above strategy for a way forward, although sound in concept, requires technical validation.
Bringing the various modules forward to create a manageable cohesive release will necessitate
the cutting of quality processes for these modules to achieve the revised timescales.
Additionally, the missing functionality will necessitate the adoption of manual workarounds,
which in turn will require resource to develop and test.
It is certain that issues related to migration, and other factors such as CPs and design creep
found to be necessary (vs creep that can be rejected) threaten the achievement of successful
delivery.
The following table catalogues those risks from the SDU risk database pertinent to this
exception plan:
RD I Description of Risk Proba- I Impact Risk Mitigation Migtn
Ref bility Factor Impact
R02 I Migration Complexity causes P 10 TBA I 1) Focus Resource on Pathway P) P
conflicts between products migration issues 1) -2
causing additional work/rework 2) Employ Full-time Migration TDA
impacting delivery date and
costs
R37 I Inadequate budget, will 8 10 80 1) Assign additional resource for P) -4
compromise delivery and initial support at early stage of lL -3
support strength with potential testing.
for delays when encountering 2) Manage risk of potential issues
technical problems downstream at the latter part of testing
through constant review of
resource requirements
R71 I Replan, condensed timescales 8 9 72 1) Allfurther change / P) -3
& budget constraints absorb all resource/impacts must be D-3
contingency plus ability to resisted
handle further CPs, change or 2) Budget for contingency
resource hits
R58 I Reduced support capability for 8 9 72 7) Retain additional support P) -5
KMS in System Test will delay resource ) -5
PinlCL fixes
R41 I Build availability - lack of, will 8 E) 72 1) Obtain resourced TI plan to link P) -4
slow down DeLT, SDU Sys Test with SDU plan 1) -2
& B&TC
R59 I MKS to address KMA 1+, 8 8 64 1) Additional design effort required I P) -6
shortfall unknown, leading to to scope work -2
delays in production of testkeys,
and manual processes
R60 I Reduced support capability for 8 7 56 I See R37
KMS in B&TC will delay PinICL
resolution
R44 I CM/PIT capability to support 7 8 56 7) Document and agree process I P) - 5
multiple baselines is critical to for managing multiple baselines Il) -0
downstream testing. Risk of with CM/PIT
large delays whilst test streams
recover from wrong baselines
R53 I Disappearing Skill Base: with 8 8 64 I 1) Create time (hence cost) for Pp) -5
potential for delays caused skills transfer into core support D-5
when encountering technical team
problems downstream
R61 I Reducing scope of KMA MTSs 9 8 72 7)Run KMA pre-DeLT integration I P) -4
causes extra bugs in DeLT, and 2)Additional development support Ip) - 3
could delay completion of DeLT for DeLT
/ reduce quality of output 3)Run remaining MTS tests during
DeLT
R36 I Inadequate or late DeLT kit 6 8 48 Progress DeLT KIT orders P) -2
threatens DeLT ability to immediately 1D -0
complete and hence could give
tise to delays downstream
R62 I Reduced support capability for 6 7 42 Monitor support requirement. P) -5
KMS in Live Service, could through test phase. Assess and 1) -4
delay PinICL fixes procure appropriate resource
EXCEP2.doc
Page 8 of 10
v0.2
FUJ00077864
FUJ00077864
RD I Description of Risk Proba- I Impact I Risk I Mitigation Migin
Ref bility Factor Impact
R48 I DeLT & System Test time- 6 7 42 I Budget required for parallel P) -5
scales condensed & overlapped resources D -6
R32 I Lack of space for Dev. 70 4 40 7)issue escalated through Pathway
Infrastructure and DeLT staff, Management route - Alan D'Alvarez
threatens DeLT ability to
complete and hence could give
rise to delays downstream
R47 I Long PIT time-scales, could 8 8 64 1. PIT staff to participate in KMS P) -5
threaten delivery dates. (Plans build process ) -5
assume builds available within 5 2. Production of resourced PIT
days) plan to dovetail into SDU plan
R21 I Requirements Document [REQ] 4 8 32 I 1. Baseline as rapidly as possible. I py -3
still not baselined; risk of further 2. Allow additional capacity in plan I 1) - 4
change for this
R73 I Inadequate security in L&G test 4 8 32 1. Review proposed test plans for
envt & processes requires L&G
additional cost
R56 I External interfaces with other 7 8 56 I 1. Separate documents covering I P) -3
Pathway infrastructure areas not agreements with other DUs D-1
committed threatening stability 2. Design team to monitor other
of design; possible rework and units for design specs and delivery
delay dates.
R64 I Bringing forward Ph2 8 7 56 1. Run KMA pre-DeLT integration I P) -4
components into Ph1 reduces 2. Additional development support I 1) - 6
quality of other components with for DeLT
potential delays in subsequent
stages eg DeLT, & SDU System
Test
R68 I Cutting KMA code inspections 8 7 56 1. Run KMA pre-DeLT integration P) -4
causes extra bugs in DeLT; with 2. Additional development support I 1) - 6
potential risk of delays for DeLT
downstream
R40 I Design creep, could threaten 10 5 50 Freeze design P) -10
delivery dates Tl) +3
R51 I External dependencies on KMS. 6 5 30 Additional resource to cover risk P) -0
not declared yet (eg stubs, test D-4
keys etc), and resourcing such
requests will delay deliveries
and support
R72 I Other risks in risk register 5 5 25
underestimated causing
additional hits if occurring
R1_ I System test environment not yet 8 10 80 Allocate secure test cell to SDU P) -7
(ST) I allocated and equip with additional test 1) -9
equipment for KMS by 30 June
Note - Where the probability is assessed as ‘P’, this dictates that the risk is outside of SDU control and
would need to be assessed at a higher level within Pathway with due cognisance to all available
information.
6. Costs
6.1 Cost of exception plan
The main driver behind the exception plan was to deliver KMS within the current Programme
timescales and within the current KMS Budget. Costs are measured as headcount resourcing
levels only.
The following table details the KMS Development and System test team headcount profile to
support the exception plan. The previous headcount profile is illustrated to show how resource
is being re-allocated to achieve the Phase 1+ baseline and to mitigate against some risks.
EXCEP2.doc Page 9 of 10 v0.2
FUJ00077864
FUJ00077864
Team Apr I May I Ju I Jul
199 n
Aug
Sep
Oct
Nov
Dec
Jan
00
Feb
Mar
Total
KMS Dev 50 50 50 I 50
Budget
30
26
25
23
20
20
20
20
384
KMS Dev 45 47 49 I 48
Forecast
29
24
21
12
12
384
Under/OverS I -5 3 -1 [2
end
8
8
System Test
Budget
System Test
Forecast
Under/Over
Spend
[Note : the System Test headcount profile will also be responsible for validating VPN and the
Secure Builds]
6.2 Cost of Risk Mitigation
The table below assesses the cost (in additional headcount profiled across the timeline) required to
mitigate the risks identified in section 5. Unless budget is released to invoke mitigating actions, these will
not be factored into the exception plan.
Risk I Apr I May I JunI Jul I Aug ISep I Oct I Nov I Dec I Jan I Feb I Mar I Total
£99 ‘00
R2 1 1 1 1 1 2 2 2 ul
R71 2 3 2 2 1 1 1 1 13
R58 4 4 4 12
R41 2 2 2 8
R59 1 1 1 1 1 1 1 1 8
R6O 4 4 4 4 16
R48 4 4 4 4 4 20
R73 1 1 1 3
R56 1 1 1 1 4
R64 1 1 1 3
R68 1 1 1 3
RSI 1 1 1 1 1 1 1 1 8
7 Summary
On approval of the exception plan a supporting Project Plan will be lodged in the Programme
Office to replace the existing KMS plan. Pathway will assess the success of delivery against
the exception plan and a CP will be raised for the introduction of de-scoped functionality at a
later release.
EXCEP2.doc
Page 10 of 10
v0.2