POL00120499
POL00120499
To: Bruce McNiven Copy Eric Logan
Andy Radka
Mike Granville
Douglas Craik a
Peter Restarick
Trevor Rollason
Clare Dryhurst
From: Kathryn Cook 16 June 1999
HORIZON CAPABILITY RESEARCH
As you know Research Services has been continuing to collect data about user
capability before, during and after Horizon training to allow us to identify
gaps in capability over time and across groups (and to identify how we might
fill these gaps). We now have the latest results from this work which I’m sure
will be of interest. (NB: This note just gives the edited highlights: if you want
any more detail then let me know).
Before attending Horizon Training :
Our research in April indicated that those going into Horizon training had an
overall pre course competency of 79%. Our latest research replicates this
exactly over an even larger sample (i.e. the existence of a large current
competency gap has -again- been confirmed)
Attitudes to Horizon continue to be positive:
© with 81% of people looking forward to using it and
© 91% believing it to be essential for their office to succeed.
However, significant numbers of people are also concerned:
e 34% said that they were worried that they would find it difficult to
learn how to use Horizon;
e only 38% believed that it would be easy to change from current
methods;
e only around half believing that Horizon would be easy to use.
Two thirds were familiar with using a computer key board; almost half used a
computer in their office.
There was a clear relationship between age , office type, previous use of
computers and confidence about using Horizon prior to training (e.g. older
users were less comfortable using a computer, less likely to use one now and
less confident that the Horizon system would be easy to use)
Horizon training:
As I'm sure you know from other sources, the live trial participants had an
overall competency test failure of only c1% but it becoming less clear that the
POL00120499
POL00120499
competency test used will actually measure competency in live use.
Qualitative feedback collected thus far indicates a general level of concern
about the adequacy of the training provided e.g. the length of training course
and amount (and quality of) of support provided after training; amount of
time devoted to areas such as balancing and inadequate time to practice
during the training time; that the course didn’t adequate prepare users for the
live office experience.
Performance after training (i.e. in live usage) :
Much of this data is still being collected and whilst we can’t yet assess the
nature and length of the ‘learning curve’ and its impact on service, it is worth
pointing out that there are already indicators of concern e.g. expressions of
unhappiness with the time taken to balance and about the length of
transaction times; concerns about Helplines etc. It is also worth pointing out
that these offices were provided with a level of (HFSO) support that is not
currently proposed for national rollout (see below).
Taking things forward:
There are two things we can conclude from the research so far:
1. There are capability gaps that can and should be filled by the
business to improve both confidence and performance (although we
won't know what all of these are until the final research is available);
2. That we can, to some extent, predict the types of people (and the
kinds of offices that they work in) who will have difficulty and these
people should be targeted for additional help.
This suggests several actions (a number of which are , I know , already under
consideration). They can be summarised as:
Prior to Horizon training:
¢ improve user knowledge and understanding of issues that are fundamental
(and which feedback has indicated are causing difficulties) e.g. the Stock
Unit concept;
e provide basic best practice guides e.g. re balancing to help improve
standards before change takes place
Training event:
¢ in amending the training event to take account of the changes re Benefits
Agency transactions, use the time freed up to devote to areas of known
capability gap e.g. balancing, and what to do if things go wrong anda
POL00120499
POL00120499
realistic simulation exercise; change the balance of training so that it better
address the ‘whys’: feedback suggests that users would benefit from
understanding why things needed to be done in certain ways (this should
support conformance as well as reducing errors/helpline calls)
¢ provide additional documentation to provide ‘what if things go wrong’
guidance;
e rework the competency test to cover areas of known concern e.g. balancing.
(This will need to be looked at any way once the post-training competency
data is available);
After training:
© Support first balances (I understand that , with some rejigging, between.
80% and 86% of first balances can be supported using current resource.
This support should be targeted towards those area s of known concern
¢ implement software changes to support balancing (I believe that this is
already in hand);
¢ documentation (particularly manuals) are perceived as being too complex
to refer to during busy times: t the opportunity to extend the use of user
guides should be explored
e mechanisms to share best practice should be developed (particularly in
larger offices e.g. ’top tips’ put on the White board / note book of problems
resolved: perhaps we could provide something to help to do this?
Perhaps we could speak about this soon? (I’m conscious that our lead times
for national roll out are short and that many of the above recommendations
therefore need resolving soon so that we would be in a position to have them
in place for August).
Kathryn Cook