POL00090473
__ POL00090473
PWKP4-41 RESTRICTED CONTRACTS
Bringing Technology to Post Offices and Benefit Payments
PROGRAMME VALUE FACTOR ASSESSMENT - 25 APRIL 1996
Author: Derek Selwood Version: 1.0
Authority: Derek Selwood 27 April 1996
Reference: PWKP4-41
‘To update Contents: Edit, Select All, and press FS
Contents Page
1. PURPOSE..
2. BACKGROUND oc.
3. PROGRAMME SUMMARY......
4, JUSTIFICATION
4.1, Introduction ...
4.2, Value Factor 1 - Customer Acceptabil
4.3. Value Factor 2 - Staff/Agent Acceptabil
4,4. Value Factor 3 - Fraud Pree Method of Payer
4.8. Value Factor 3a - Fraud Free Systems for POCL
4.6, Value Pactor 4 - Credibility of delivery in Steady State.
4.7, Value Factor 5 - Start Up .
4.8. Value Factor 6 - Innovatio!
4.9. Value Factor 7 - Flexibilit
4.11. Value Factor 9 : Reliabili
4.12. Value Factor 10: Stabi
1. PURPOSE
LL. This paper is an addendum to PWKP4-22, version 2.3, and PWKP4-33, version 2.0,
lodged with the Programme lawyers prior to receipt of tenders and retenders
respectively. It provides a summary of the Programme Value Factor assessment of
25 April 1996. This followed assessments completed by the Contracts and
Demonstrator strands to take account of evidence gathered from the start of further
negotiations with Service Providers on 29 March to examination of the retenders
received on 22 April.
27 April 1996 Page I of 11 Lo
POL00090473
_ POL00090473
PWKP4-41 RESTRICTED CONTRACTS
2. BACKGROUND
21 Following receipt of retenders on 22 April, pre-allocated teams carried out assurance
activities from a Contracts, Demonstrator and Partnership standpoint on the retender
material. The activities consisted of reviewing the retenders against the pre-[TR
position, considering whether any changes had been made and, if so, the impact of
those changes. The full results of the assurance activities are described elsewhere,
2.2, In the course of the assurance activities, technical and contracts reviewers concluded
that, notwithstanding their exposure to the pricing content of tenders and retenders
since the last review, they should have the opportunity to revisit the existing Value
Factor assessments in view of the more recent relevant experience and the changes in
Service Providers’ positions in some areas evidenced by the content of the retenders.
The teams held formal reviews on 24 & 25 April to bring together reviewers’
experience.
A Programme Review took place on 25 April to reconsider the Programme scores
and validity markings in the light of the three stream reviews referred to in para 2.2.
A summary of the results is given in section 3 and the detailed changes and rationale
are set out in section 4,
24. The scores for factors 1 - 10 (Le. excluding 3A, Fraud free systems for POCL) were
fed into the evaluation in accordance with the procedures agreed in PWKP4-5, Issue
3.0.
27 April 1996 Page 2 of 11 Lo
POL00090473
PWKP4-41 RESTRICTED CONTRACTS
3. PROGRAMME SUMMARY
BAL. The following table shows the Programme scores for the Value Factors. Where there
have been changes in scores or validity markings the previous score/marking is
shown in brackets. .
RAyts
a Watashen>
Es fee
1 Customer Acceptability a
2 Staff/Agent Acceptability B a
t
[3 /Fraud Free Payment Method A
3A Fraud Free Systems for POCL I 5.7 B 5.9 B SA B
4 Credibility of Delivery 57 B 3.5 B 4 B
5S Start-Up $4 B- $4 B- 44 B- ae
6 Innovation $3 B 3.8 B 3.4 B ae
7 Flexibility 47 B 42 B 44 B
I 8 Management Capability 47 >c+# I 45 I B 3.7 I C+
G5) G3)
9 Reliability and Support 3.3 Ce 3.2 C+ 435 C+
y
I 10 Stability and Coherence 62, B I 6o}I B I 44) B
GB.
* This mark refers to the non compliant bid against the ITT.
3.2. The need for Value Factor 3A was identified after the issue of details of the ten
factors to Service Providers, The results of the assessment are presented here, since
they offer a degree of comfort in respect of all three Service Providers, but they are
not proposed as an element of the evaluation.
27 April 1996
Page 3 of 11
LB
POL00090473
POL00090473
PWKP4-41 RESTRICTED CONTRACTS
4. JUSTIFICATION
4.4. Introduction
4.1.1. The outcome of the reviews by the Contracts and Demo Streams can be summarised
4.2,
42.
as follows:
(a) the Contracts Stream revised their marks for factors 3, 4, 8, and 10. The
results and a summary of the associated rationale are given later in this paper;
{b) the Deme Stream only considered a change necessary to factor 3; in some
other factors their view on some of the sub-factors changed to a degree, but not
enough to alter the factor score.
The revised scores (and rationale for the factors where scores changed) are given in
the paras below.
Value Facter 1 - Customer Acceptability
The service provider needs to ensure that the services are viewed favourably by
customers at post offices, and that the benefit payment services are acceptable to all
benefit customers choosing to be paid at post offices.
Partnership . - ~
Contracts - E E
Demonstrator 3.8 Cc
Programme 5.8 B C
No change from previous review.
27 April 1996 Page 4 of 11 10
POL00090473
POL00090473
PWKP4-4] RESTRICTED CONTRACTS,
4.3. Value Factor 2 - Staff/Agent Acceptability
The service provider’s services need to be Post Office and BA local office staff
friendly; for example, by being easy to use, responsive and supportive of their job
functions.
Contracts Doe E - E - E
Demonstrator 6.0 Be
Programme 6.0 B+
43.1. No change from previous review.
4.4, Value Factor 3 - Fraud Free Method of Payment
‘The measures proposed to make the s for benefit payment fraud free and to
maintain that fraud free level.
Partnership - - -
Contracts 2S Bt 2.0 Be 3.6 Be
(4.0) (2.5) (* 3,0)
Demonstrator 6.2 A 63 A 43 A
(7.8) GA) (4.4)
Programme $5 A A
(7.0) (N4.D
*Refers to their non-compliant bid against the ITT
44.b. position the”
rt, PWKP4-40,
442. Contracts Stream considered that as a result of the deterioration in the Cardlink
position in their retender, the Pathway bid now represented the best deal in terms of
transfer of fraud risk. Their position in the negotiations and retender was unchanged
from their non-compliant bid against the ITT, so their score should remain at 3.0,
27 April 1996 Page 5 of 11 10
POL00090473
POL00090473
PWKP4-41 RESTRICTED CONTRACTS.
4.43. Cardlink and IBM were equal in their non-acceptance of cardholder verification
fraud, but inferior to Pathway. On other fraud, however, the cover from Cardlink
was better than from IBM. On this basis the Cardlink score should be 2.5 and the
IBM score 2.0,
444, The Demo Stream had divided this factor into two sub-factors:
(a) the physical measures to prevent fraud
(b) the procedural measures (Fraud & Risk Management’) to back up the physical
measures.
44.5, Although positions on physical measures (and thus scores) were unchanged, the
Demo Stream noted that all three Service Providers had shifted their position in
varying degrees 3
won
44.6, The extent of this downscaling was not documented in the responses (Cardlink had
commented that the Security Proposal was “significantly impacted” and required
updating to be consistent with their new bid). area
d, as this would require additional information from and
e Providers. However from the information held it was
possible to form the view that the scores of all three would be reduced.
4.4.7, The Demo Stream agreed that, in view of the lack of detailed information, supplier
scores for the procedural measures sub-factor should be either the score appropriate
to the security review definition ‘there are major reservations which will require
significant work post award of contract’ or the existing score, whichever was the
lower.
44.8, In deriving the Programme scores, it was agreed that the 20/80 balance between
Contracts and Demo should be used again.
44.9. In passing, the Programme review noted the Contracts view that, whilst their scores
were fair assessments against the original expectations, experience of the
negotiations with the three shortlisted suppliers had shown that these were set at a
level which the market was not prepared to mee! I
4.4.10, The results of such scores at Programme level, using the 20/80 formula, would be:
Cardlink 5.8
IBM 5.6
Pathway 44
27 April 1996 Page 6 of 11 1.0
POL00090473
POL00090473
PWKP4-41 RESTRICTED CONTRACTS,
45. Value Factor 3a - Fraud Free Systems for POCL
The measures proposed to make the service for POCL fraud free and to maintain that
fraud free level.
Partnership
Contracts - E - E - E
Demonstrator 57 B $9 B SA B
Programme 5.7 B 5.9 B S.A B
4.51, No change from previous review.
4.6, Value Factor 4 - Credibility of delivery in Steady State
The service provider's designs, procedures, tools, methods, resources and
organisation need to ensure that the steady-state services will be delivered to time and
quality, showing understanding of and empathy with the BA and POCL requirements
and objectives.
Partnership
Understanding. of POCL Good Very Good Good
Capability and Experience Good Good OK
Contracts 3.5 D 3.5 D 3.5 D
GO)
Demonstrator $7 B 3.5 B- 41 I Bt
Programme 5.7 B 5.5 B 4A I B
4.6.1. Contracts Stream reduced Cardlink’s score to reflect their refusal to accept implied
requirements in the contract.
4.6.2. The Programme noted the above change, but concluded that the previous agreement
on the relative importance of the scores should stand, i.e. the Demo scores should
prevail in view of the low validity marks on the Contracts evidence.
27 April 1996 Page 7 of 11 10
POL00090473
POL00090473
PWKP4-41 RESTRICTED CONTRACTS
4.7. Value Factor 5 - Start Up
The service provider's design, development, acceptance and initial implementation
services need to be credible, showing appropriate controls, management interfaces
and capability for managing, controlling and delivering the start-up of the services.
Partnership
Contracts - E - E - E
Demonstrator 3.4 B- 3.1 B- 44 B-
Programme 5.4 Be OBA B- 44 B-
4.7.1, No change from previous review.
4.8. Value Factor 6 - Innovation
The s provider needs to be pro-active, change-orientated and demonstrate a
genuine “can do” attitude.
He should generate creative ideas and understand how to apply technology profitably
building upon his initial services, complementing rather than competing with POCL’s
core competencies.
He should seek to do existing things better and better new things.
Partnership
Analysis of Opportunities Good Very Good OK
Contracts 4.0 D 40 Dd 49 D
Demonstrator 3.3 B 5.8 B 5.4 B
Programme 53 B 5.8 B 54 B
4.8.1. No change from previous review.
2? April 1996 Page 8 of 11 10
POL00090473
POL00090473
PWKP4-41 RESTRICTED CONTRACTS
Ad, Value Factor 7 - Flexibility
The service provider and his services need the ability to react to external change and
to meet a diverse range of existing and potential needs, thereby maximising value for
money and faster delivery of new products and services.
Partnership > - .
Contracts 4.0 Cr 3.0 ct 3.5 C+
Demonstrator 54 B $3 B 3.2 B
Programme 47 B 4.2 B 44 B
4.9.1. No change from the previous review.
4.10. Value Facter 8: Management Capability
The service provider management needs to be competent, customer focused,
accountable, with evidence of well defined internal controls enabling a proper
external focus. Suitable quality processes must be in place. and key resources must
have skills and experience appropriate to their roles.
Partnership . .
Management Relationship Good Good -
Ovals OK Very Good -
Contracts 3.0 B+ 2.5 Bt
4.7) I 4)
Demonstrator 6.3 Cc 64 C+
Programme 47 c+ 45 B
(3.5)
4.101. The Demo score remained unchanged.
4.10.2. The Contracts Stream considered that since the previous review there had been
considerable changes in performance by two suppliers under this Value Factor, as
evidenced by behaviour during the negotiations and in the submission of the
retender,
27 April 1996 Page 9 of 11 10
POL00090473
POL00090473
PWKP4-41 RESTRICTED CONTRACTS
4.10.3. Pathway had come over as clearly the most focused and competent team since ITT.
During the negotiation and retender process they had shown a willingness to meet all
the Authorities’ information requirements and had provided all that was required
comfortably within each deadline. There was a noticeable improvement in their
performance compared with previously, and they were marked up accordingly.
4.10.4, Cardlink’s performance had fallen considerably below their previous standards,
culminating in failure to complete their retender properly - evidenced by obvious
gaps in the material submitted on 22 April and subsequent submission of a number of
unsolicited letters amplifying what had been said. Their mark was reduced
accordingly.
4.10.5. In establishing the Programme score, it was agreed that the previously adopted
principle of averaging and rounding towards Demo should be maintained.
4.41. Value Factor 9 : Reliability and Suppert
Denner sees
piacenin
The service provider needs to anticipate and prevent problems, with robust fallback
procedures for benefit payments and other customer services in the event of system
failure. He should regard his first priority as maintaining continuity of service rather
than referring to the contract in the event of unforeseen problems.
Partnership i - - -
Contracts 5.0 B 5.0 B 4.0 B
Demonstrator I 55 c 54 c 4.9 c
Programme I 53 C+ 5.2 C+ CH
4.11.4, No change from the previous review.
27 April 1996 Page 10 of 11 10
PWKPS-45
RESTRICTED CONTRACTS
POL00090473
POL00090473
4.42. Value Factor 16 : Stability/Coherence
The prime service provider and his associated consortium members and/or main
subcontractors need to have a stable relationship from which to provide the services.
The member organisations should balance each others’ skills and resources to match
those needed to deliver the services. The prime service provider should show his
competence to successfully lead such a group of organisations to deliver similar
services,
Partnership
Leadership and Synergy Very Good Very Good OK
Contracts 6.0 B 6.0 B $.0
4.0)
Demonstrator 6.5 c 6.0 Cc
Programme 6.2 B 6.0 B
8.7)
4.12.1. The Contracts stream had now had a further period to experience the difference in
performance generated by the change in the Pathway company structure made
immediately prior to issue of the ITT. Their score was increased accordingly.
412.2. In deriving the Programme scores, it was agreed that the previously adopted bias
70/30 bias in favour of the Contract Stream scores should be maintained.
27 April 1996
Page llof iL