15 November 2022 – Robert Booth and John Meagher
Hide video Show video
(10.00 am)
Ms Hodge: Good morning, sir.
Sir Wyn Williams: Good morning.
Ms Hodge: Can you see and hear me clearly?
Sir Wyn Williams: Yes, thank you.
Ms Hodge: Thank you, sir. May I call Mr Booth, please.
Robert Booth
ROBERT BOOTH (affirmed).
Questioned by Ms Hodge
Ms Hodge: Good morning, Mr Booth. Please give your full name.
Robert Booth: Robert Jonathan Booth.
Ms Hodge: Thank you. May I ask you to raise your voice slightly to ensure you can be heard by all in the hearing room and by the Chair.
Robert Booth: Certainly.
Ms Hodge: Thank you. You should have in front of you a witness statement dated 16 September of this year. Do you have that open before you?
Robert Booth: I do.
Ms Hodge: That statement runs to 80 pages, I believe?
Robert Booth: Correct.
Ms Hodge: Could I ask you please to turn to page 63 of your statement. Do you see your signature there at the end of your statement?
Robert Booth: I do, yes.
Ms Hodge: Is the content of the statement true to the best of your knowledge and belief?
Robert Booth: It is.
Ms Hodge: Thank you. I’m going to begin, Mr Booth, by just asking you a few questions about your professional background, if I may. In terms of your education, you obtained a degree in mechanical engineering from the University of Surrey; is that correct?
Robert Booth: That’s correct.
Ms Hodge: And upon graduating you worked for approximately six years as a programmer for a company called Computer Analysts and Programmers Limited; is that right?
Robert Booth: That’s correct.
Ms Hodge: Rising over the time of your work there to the role of a team leader?
Robert Booth: That’s correct.
Ms Hodge: I believe you joined Post Office Counters Limited in October 1990, and were assigned to work on the development of an electronic point of sales system know as ECCO+?
Robert Booth: That’s correct, yes.
Ms Hodge: Could you please describe for us what ECCO+ involved.
Robert Booth: ECCO+ was a computerised system which would assist the directly managed branches within Post Office to do their accounts, to do basic transaction capture, and at the time it ran on a PC per counter and was probably the first proper automation at Post Office branch.
There had been previous attempts with a thing called Thames Valley, which was before my time there, and at the time PCs were relatively new, and subpostmasters were buying other systems to assist them with their manual balancing, but it was still very much a paper based system and ECCO+ was starting to move into an automated world.
Ms Hodge: In your view, was ECCO+ a successful automation project for the Post Office?
Robert Booth: Yes, it was.
Ms Hodge: What do you believe accounted for the success of the ECCO+ program?
Robert Booth: I think because the IT was developed effectively in-house and with people who were ex-counter clerks, and it was very close collaboration with the actual Post Office Limited as it was Post Office Counters. There was fully open – we had a common goal to deliver something. We understood what was needed and we had direct access to people who’d either used the counters in the past or were setting an election from the information system services within Post Office Counters Limited about what they were wanting to achieve. So it had all the right components there.
Ms Hodge: Do you recall what, if any, consideration was given at the time to extending ECCO+ to the wider Post Office network?
Robert Booth: I wouldn’t have been privileged to that. It was very much aimed at the directly managed branches. Post Office, as you probably know, works on the basis of franchisees – subpostmasters run their own business – and then about probably 750/800 directly managed branches which has reduced over the years, and those are owned and managed by Post Office. It was very much looking at how those branches, the bigger branches, could be automated, and I think the success of that kind of paved the way that automation was seen as a viable for the bigger estate. At the time computerisation, it was on PCs, but PCs were a bit of a misnomer. They weren’t personal computers, not everybody had one as we do now, and it was cutting edge technology for the time.
Ms Hodge: Thank you, Mr Booth. I’m very grateful for the detail of your answers. I wonder if you could slow down ever so slightly in your response, because we have someone transcribing what you are saying, and I anticipate that the pace may be presenting some challenges. But it’s no criticism.
You later joined the Post Office Counters private finance team during the procurement of what became known as the Horizon System; is that right?
Robert Booth: That’s correct.
Ms Hodge: From that point, I believe you worked on Horizon continuously until June 2003; is that correct?
Robert Booth: Yes.
Ms Hodge: When you transferred to an external companies named, is it Xansa?
Robert Booth: Xansa, yes.
Ms Hodge: Whilst employed by Xansa, you worked on a variety of Post Office products including Horizon; is that right?
Robert Booth: Less so on Horizon. I finished the project I was involved with and then moved on to a project called Paystation, and then one Automated Enrolment and Identity, which was biometrics program both again for the Post Office.
Ms Hodge: I believe you left Xansa in October 2006 to rejoin the Post Office; is that correct?
Robert Booth: Yes, I’d been TUPE’d into Xansa and I resigned from Xansa and joined Post Office as an employee.
Ms Hodge: I think you worked at that stage on a separate project but which was related which interfaced with Horizon; is that right?
Robert Booth: Yes.
Ms Hodge: You later came to work on Horizon again in 2014 following your transfer to Atos; is that right?
Robert Booth: Yes.
Ms Hodge: I think you were mostly focused at that stage on counter hardware; is that fair?
Robert Booth: That’s correct, yes.
Ms Hodge: Finally you remain employed by Post Office Limited as a solutions architect today; is that right?
Robert Booth: Yes.
Ms Hodge: And your most recent project, I think, was in the area of Post Office compliance with payment card industry standards; is that right?
Robert Booth: That’s correct.
Ms Hodge: It follows, I think, from that brief summary that you first worked on Horizon during the procurement phase?
Robert Booth: That’s right.
Ms Hodge: Do you recall the stage at which the procurement had reached at the point at which you joining the programme?
Robert Booth: I believe it was when the stage was down to three tenders, three potential suppliers, and it was – I was brought in to help the technical evaluation of those three suppliers.
Ms Hodge: You’ve described joining Post Office Counters Limited private finance team. Can you describe the nature and composition of that team as you recall.
Robert Booth: In general terms, my understanding was the private finance initiative, PFI, was a shared risk reward contract. The eventual service provider would have been providing a service for both Benefits Agency and Post Office Counters, automating the Post Office Counters and the Benefit Agency as a major partner in providing electronic payment for their services.
Ms Hodge: Like me, Mr Booth, I think you are quite softly spoken. Just for the benefit of the chair, I wonder if you would mind doing your very best to speak a bit more loudly; thank you.
There is reference in some of the documents we’ve obtained to joining the programme several weeks into the demonstration phase. Does that sound familiar to you in terms of timings?
Robert Booth: It does. I was brought in to bolster the technical expertise within the team.
Ms Hodge: That’s as a junior member of the technical evaluation team as you described.
Robert Booth: Yes.
Ms Hodge: Do you recall to whom you reported at that time?
Robert Booth: I believe I reported into Jeremy Folkes.
Ms Hodge: We know from the documents obtained by the Inquiry that you participated in what was known as the Post Office Counter Limited infrastructure demonstration strand; is that correct?
Robert Booth: Yes.
Ms Hodge: Can you describe in your own words what the purpose of that strand was, please.
Robert Booth: It was a two-way exchange, I think spread over about six to eight weeks, where we would engage with the service providers on a one-to-one basis. It was, some of the documents you recently provided, based on a Tuesday, Wednesday and Thursday, with ICL Pathway being the Wednesday service provider meeting. It was there for us to gain an understanding of their proposed solution and to try and shape anything that we thought would give them problems. Where they had a misunderstanding, they could ask for clarification questions, and we could also give them guidance.
We’d invite them, or suggest that they may wish to look at certain areas. At the time the Post Office head office was up in Old Street, and some of the service providers had seen the Old Street Post Office which was a multi-position, what we call a fortress. So it had glass partitions in front. But we did point out that the Post Office was a very diverse, and still is a very diverse retailer with multiple different types of branch, and we encouraged them to visit their local branch and, if they are out for a drive on a Saturday, “If you see a post office sign, stop, go in and look at it, get a feel for it”, because a Spar, a corner shop, and a Trafalgar Square multi-counter branch are very, very different environments.
Ms Hodge: What distinguished the Post Office counter structure strand from the other strands, the other demonstrator strands that were operating at that stage?
Robert Booth: The infrastructure strand was less defined from a business perspective. It’s like you turned a light switch and just expect the lights to come on. You don’t necessarily think about how the light has to come on. You don’t think about the wiring, the power station the distribution network. The infrastructure was about getting all those pieces in place so the things the Post Office were concentrating on, such as the EPOS system, such as the things that the users would touch and feel, they would all be supported by the POCL infrastructure.
So we would make sure and try and assure ourselves that the communications to the branches were sufficient, that the data centres were resilient, that the service provider understood that in an estate the size of the Post Office, if you think it can go wrong, it would go wrong, and you needed to be defensive and you need to be resilient because, kindly, anyone can program something that’s going to work in a perfect world but we’re not in a perfect world, and it was trying to make sure the service providers understood the reality of the environment they were working in.
Ms Hodge: As part of your work on the demonstrator stream you attended a series of meetings with the three service providers –
Robert Booth: Yes.
Ms Hodge: – that you have explained. One of those was Pathway. Do you recall any of those meetings that you attended with Pathway, the consortium?
Robert Booth: Not in specifics, apart from a couple of sort of key moments, but in general there was a different flavour between the three first service providers.
Ms Hodge: Can you give a flavour of what your overall impression of Pathway was from your participation in those demonstrator meetings?
Robert Booth: I think they were more defensive of their solution and didn’t want to listen to the complexity. They had more of a fixed world view about: this is the solution we’re proposing; it will work.
The other providers were more structured and more amenable to listening and to understanding why we were trying to say, “Have you thought about”, because ultimately the service provider solution was their solution. We were not designing it, we were not going to be responsible for it; it was the service provider’s call about how they did things. All we could do is try and highlight was that walking over glass in bare feet wouldn’t be a good idea; “Maybe you should get some flip flops. Have you thought about it”, and trying to highlight those kind of things to them.
We did have one meeting where the owner chief – I think probably the owner of Escher who were providing a key component came in. He was much more technically aware of the system, it being was his underlying software that was being used. His view was much more that technology could solve it if you threw more technology at it. The ICL Pathway people were more aware of the business implications, building a solution to a price.
Ms Hodge: You have referred just now to a meeting attended by a representative of Escher. We have a number of written reports which were produced into the various meetings which you attended with representatives of the consortium. Before we go to that specific report, do you recall whether you personally contributed to the information and findings contained in the written reports which have recently been provided to you?
Robert Booth: Yes. So the format of the meeting would be on the day we would have – we would notify the service provider the preceding week about the topics for the next week, thus giving them time to prepare material at everything else they needed.
We would attend the meeting, and that evening the notes I’d taken, notes that Jeremy who I was working with had taken, we’d collate. That would be over a dial-up modem exchange of emails. It wasn’t connected as it is now; we didn’t have Teams or things. Then on Friday those would be finalised into the form that was provided to me last week.
Ms Hodge: Thank you. I wonder if we could just pull up one of those reports for now. It’s WITN05970107. This is a written report of the infrastructure meeting held on 8 November 1995. We can see under the heading Attendees, as you say, this one was attended by you and Jeremy Folkes on behalf of the Benefits Agency Post Office Counter programme, and in the right-hand column we see a number of representatives of the supplier identified, some I believe from Pathway, such as Martin Johnston; is that right?
Robert Booth: Yes.
Ms Hodge: As well as An Post and finally Mike Murphy from Escher?
Robert Booth: That’s correct.
Ms Hodge: This is a record of the meeting to which you’ve just referred a short time ago; is that right?
Robert Booth: Yes.
Ms Hodge: The purpose of this meeting we can see under the heading there was to focus on the technical aspects of Riposte; do you recall that?
Robert Booth: Yes.
Ms Hodge: There’s one specific entry against your name which I hope you might be able to assist us in clarifying. If we can go to the bottom, please, of page 1, the final bullet point, there’s a heading Compatibility with TCDs. What are TCDs?
Robert Booth: Fortunately that’s one of the things I can remember. TCD is a telecash dispenser. These were post offices moving away from the fortress, so didn’t have glass, but have an open-plan desk like the one I’m sitting at today with a computer and, because benefits could be several hundred pounds, it would not be safe to have hundreds of pounds sitting in an open drawer beside you. So a telecash dispenser effectively is a steel safe box that was connected to the computer, and you could get it to dispense cash and it could also accept cash, so effectively a secure drop box and ATM-ish to dispense notes so that the money was held secure in an open form.
Ms Hodge: We can see that entry read:
“No requirement at present.”
Does that reflect, in effect, the absence of a detailed requirement from the sponsors for –
Robert Booth: That was because the use of TCDs was where Post Office were moving to open plan. That was very much an experiment to see how the format would work. It would need more space. It had other constraints about accepting mails because, if you give me a lot of parcels and I’m sitting here, where do I put all the mail? If I have to get up and put it in a secure place, which I need to because obviously we don’t want another customer taking your package, that means that I need to have secure storage around me.
For that reason the open-plan concept was tried, it had some problems, and it did not at that time look like it was going to go forward. Therefore, there was no requirement for it.
If we needed to move to an open-plan format, we would have introduced it under exchange control.
Ms Hodge: We can see the final sentence reads:
“Bob to brief JM prior to next week on requirement implications such as cash holding replenishment and ownership within the office.”
Who was JM?
Robert Booth: John Meagher.
Ms Hodge: Thank you. What did you understand the nature of his involvement to be at this stage in the process?
Robert Booth: At that stage, I believe he was the programme or project – I don’t know quite the name given but, effectively, he was one up from Jeremy. So I’ve been fortunate to have umbrella managers that have allowed me to get on with the job and I haven’t been involved too much with the politics infrastructure set up above me. So I’m a bit hazy on that but …
Ms Hodge: So far as you are aware, the sort of reporting line was you to Jeremy Folkes and Jeremy Folkes to John Meagher; is that correct?
Robert Booth: That’s correct.
Ms Hodge: We can see from the report that Mike Murphy joined the meeting late to discuss the role of the EPOS – sorry, the Riposte product, and that’s at page 2, please. If we can scroll down, please.
Point number 3, it appears from the report of this meeting Mr Murphy expressed some fairly critical views about Pathway’s understanding of their own proposed solution. Is that a fair characterisation?
Robert Booth: My recollection of Mike was that he was a fairly assured person, as you expect an owner of a company to be, very wed to his product to Riposte and how it should be used, with firm views about how it should be being used, and I don’t think that at the meeting we got the impression that the ICL Pathway and Escher way were necessarily aligned.
Ms Hodge: We can see that, please, if we scroll down to page 5. There’s a second bullet point it reads:
“When queried about sizing model Pathway had agreed to produce paper. When this was mentioned later in day with Escher present, the Mike Murphy response to Pathway co-members was ‘How can you do that, Pathway haven’t got a clue how it works.’”
We then see the comment:
“Lack of demonstrable thought processes.”
Did that reflect your views?
Robert Booth: It did. That would be reflecting the disjoint of philosophies between the two companies that we saw.
Ms Hodge: We see at the fourth bullet point a comment to the effect that there was lack of cohesion between the people at the meeting, which I understand to mean between the various representatives of the consortium; so Riposte and Pathway:
As a result of which:
“[there] must best doubt over ability to manage project if this interface to their customer is so weak.”
Were you concerned at this stage about Pathway’s ability to manage the project?
Robert Booth: I think this reflects a concern and a benefit. The concern was that you should have – if you’re presenting to the customer, you would have a pre-meeting, you would agree the company line and you’d toe it. You’d make a decision whether you agreed with it or not, that is the decision, that is what you are going to tell the customer – in a bid phase especially.
The benefit was seeing some of these divisions actually exposed more about the solution than we’d otherwise be able to obtain, because it wasn’t so polished. We actually – “What do you mean by that” kind of question, and get a little bit under the skin, which we had been denied at the time.
Ms Hodge: A further issue identified there at the 5th bullet point was a general problem in relation to documentation; do you recall that?
Robert Booth: It was very difficult to get documents out of ICL Pathway. It was also one of the – if I look back on the other strand, I think that their production of documents was better. I don’t have notes from their meetings to correlate – corroborate that, but I do believe that the level of engagement was more professional at a technical level with Cardlink and IBM than it was with Pathway. It was much more ad hoc and not always convinced that they knew what they were saying.
Ms Hodge: To what extent were the concerns articulated in this meeting about Pathway’s ability to deliver the solution resolved effectively by the time the contract was awarded?
Robert Booth: This was very much about a discovery. This wasn’t trying to ascertain their delivery as such. This was very much how they are going to deliver the solution; what is the solution they are going to deliver. The disjointed nature was the concern, but we weren’t focused, or I at least was not focused on the actual delivery of it, and whether they had the correct governance and procedures in place. Does that answer the question?
Ms Hodge: I think up to a point. It may be that we will come to the assessment phase a little bit later. But, so far as these demonstrator meetings were concerned, you attended a the further meeting, I believe on 22 November, so several weeks later. We have a report of that meeting at WITN05970141.
So the same attendees on behalf of Benefits Agency and Post Office Counters. We have some new names under the supplier heading. Martin Bennett, do you recall his role?
Robert Booth: Martin, I think, was a member of the management team within Fujitsu. As it says here, he was the risk manager, and at the time risks were one of the very few leads we had to get information out of Pathway.
Ms Hodge: We can see that the purpose of this meeting was to focus on system management, Riposte papers and TMS sizing and scalability. Dealing with the first of those, what was meant by system management, please?
Robert Booth: Both within a data centre and a distributed estate, you have computers as we have in this room. System management is about: how do you update those computers; how do you keep them current? So, if we need to put a new version of software or update the operating system, update the anti-virus, how do we do that?
So systems management is about how you manage your system, how you push out updates, and how you discover what is on a computer so that you know what your estate consists of. It also applies to your data centre elements as well.
Ms Hodge: Would that extend, for example, to dealing with faults and defects that might be detected during the course of live operation, for example?
Robert Booth: Answer to that in two parts: yes and no. It wouldn’t actually be the resolution and necessarily the identification of such faults, but it would be the means by which you could gather evidence to investigate a fault, and it would be the means by which you distributed a fix to remedy a fault.
Ms Hodge: Thank you. The reference to TMS sizing and scalability, can you explain that, please.
Robert Booth: TMS, I believe, stands for transaction management system. It was a central component within the ICL Pathway solution and, where you have at the time around 40,000 counters talking into a data centre, even today that’s a large number. Back then it was a very large number. The Riposte product we had seen had been used in An Post, which is a much smaller Irish Post Office than the UK Post Office, and therefore we were saying, “Well, how will it work? Have you got enough horsepower to drive the system? Will it be performant? Will you come across problems where a day’s job takes 25 hours”, and, you know, you just can behind and never catch up.
So this was looking at how big TMS would be and, if it had problems, how would you scale it up, how would you make it bigger so that it coped with additional load, because one of the other things you can look at in a system is you can start a system small and, as you add counters, you can grow it out. So we were trying to understand what ICL Pathway’s approach was to making sure their central system was up to the job.
Ms Hodge: Thank you. We can see, if we please scroll down to page 5 – could we just scroll a little bit further down, please. Sorry a little bit further, please.
Thank you. So the final bullet point there above section 4, it reads:
“Updated document. Pulling teeth on performance and scalability issues that are going on behind the scenes.”
Does this reflect your ongoing concerns at this stage about the success of your efforts to extract information from Pathway about its solution?
Robert Booth: There was an impression that they were aware of the need to make sure that it was scalable, but they were not forthcoming in any evidence that they were actually – what they were doing to address the concerns, and that’s why we then start using the risk lever to elicit, “What are you actually doing about things”, rather than having a verbal statement in a meeting.
Ms Hodge: Returning to the question of the assessment phase, we know that the demonstrator stream of the procurement process which we’ve just discussed was followed by what’s known as an evaluation or an assessment phase – I think the terminology’s used interchangeably – in which the three shortlisted service providers were scored according to a number of factors, some of which related to financial and contractual aspects of their offering, and others relating to the quality of the technical solutions. Is that your recollection?
Robert Booth: My understanding was there were several streams – I don’t know how many – of which technical was just one.
Ms Hodge: What was the nature of your involvement in the assessment phase of the procurement process?
Robert Booth: I participated in the scoring of the suppliers. So the way that it was done for my strand at least was that the ITT2 – I can’t remember – SRR, that’s statement of – there was an ITT2. I can’t remember the proper name for it.
Ms Hodge: The Invitation to Tender?
Robert Booth: Yes, but there was a follow-up to that. There was a revised one from the dialogue that had been had with the service providers, which was one they actually were contracting against. So the initial one effectively had been refined and then, as part of this second one – this was going through the evaluation phase – there was set criteria saying: how are they going to demonstrate X, Y and Z, with measures against them such it was a qualifiable and repeatable process. So it wasn’t based on feel, it was something which if questioned we could justify and say, “We gave that a 2 because over here it says they are doing this and that. We gave that one a 5 because they are doing this and that, and they are doing this and this.” So we could have relative scoring of the providers.
Ms Hodge: Do you recall how Pathway performed during the assessment stage relative to the other two service providers?
Robert Booth: I’m sorry, I don’t know.
Ms Hodge: Moving on to another topic, if I can, please, you remained working on the BA Post Office Counter Limited programme upon award of the contract to ICL Pathway; is that correct?
Robert Booth: Yes, it is.
Ms Hodge: Who was responsible for your line management during that phase of the programme?
Robert Booth: I can’t recall if John Meagher was around then. Probably he still was, but at that time it may have been a guy called Colin Standring. There were several project managers and also, over the course of that period, several consultancies that came in with various sort of the management structures. I’m afraid I can’t recall on that timeline.
Ms Hodge: In your statement you have identified two principal aspects to your role during the design and development phase of the programme. The first you’ve described as refining the sponsor’s requirements; is that right?
Robert Booth: Yes, it is.
Ms Hodge: We know that a process of defining those requirements took place during the procurement phase. What did this process of refinement entail?
Robert Booth: It was one of clarification and almost compromise. If you have a requirement to do something, you may have phrased it in a way that leads to a certain solution, but there may be an alternate solution that has the same outcome and, if that is easier for the service provider to provide and the outcome is the same, you can adjust your requirement wording such that the outcome is the same even though it’s arrived at by a slightly different method.
So it was a win-win for us. We would have a service provider that had something that fitted their solution better, but we would still have the outcome that we required fulfilled.
Ms Hodge: Earlier in your evidence, when we were discussing your involvement in the ECCO+ automation project, you described that that project involved a great deal of input from those working at the counter, as I understand it. How did that contrast with the approach adopted here in relation to the Horizon IT System?
Robert Booth: The Post Office people that were interfacing with ICL Pathway, several of those were ex-counter. So the experience, the real-world experience was there. They were generally from the directly managed branches, but ultimately the Post Office had the same accounting requirements across all its branches.
So the expertise was available to ICL Pathway. How they used it changed. There was a rapid-application approach at one stage where colleagues in my office would effectively disappear for the day and sit down with ICL Pathway to try and come up with mutually beneficial outcomes, which met both the business requirements and fitted in with the ICL Pathway solution.
Ms Hodge: In terms of the rapid-application technique to which you have just referred, to which component of the Horizon product did that initiative relate?
Robert Booth: I believe it was primarily EPOS but that was, as I say, though we shared an office, it was very much a separate strand within the office.
Ms Hodge: That is to say, the development of EPOS was a separate strand?
Robert Booth: Yes.
Ms Hodge: What was the focus? What was the particular focus of your strand, please?
Robert Booth: It was still on the infrastructure primarily, then moving on to how it would interface with the Benefits Agency, who were developing their own back-end systems, so it was how system to system would work. The counter behaviour was effectively a given. You have a given token, it has to go off and be verified, and you pay money. That’s sort of relatively straightforward and constrained.
It’s getting those transactions into the data centre and then interfacing off to the Benefits Agency where they were, I think, trying to consolidate 17 systems into one, to give a single-payment view, and it was looking at how that interface would work with the Benefits Agency.
Ms Hodge: I described requirements refinement as one aspect of the role you have identified in your witness statement. The second aspect of your role that you’ve described is as implementation to deploy – these are your words – deploy the first generation of Horizon. What do you mean by the term “implementation”?
Robert Booth: It was how – we had Fujitsu – ICL Pathway at the time responsible for the in-branch installation, the training, and we had other teams that were looking after that, and it was looking at how the roll-out and the POCL infrastructure would support that roll-out, how we’d get computers into the branch, and making sure that, though a separate team was looking at the training, was looking at the branch fit-outs, et cetera, that the technology sitting behind that would function.
Ms Hodge: During the design and development phase, you have explained you worked both on the front end and the back end of the system; is that correct?
Robert Booth: Yes.
Ms Hodge: I wonder if we could, please, just briefly bring up your statement as we can see quite a helpful description of that at paragraph 28.
So in terms of front-end design, you explain that you were mostly involved with the magnetic card acceptance and client interface for the Post Office Card Account. I think we also know that as the benefits payment card; is that correct?
Robert Booth: Yes, it is.
Ms Hodge: A little further down in relation to back end interfaces you say:
“I also worked on back-end interfaces to the clients to deliver transactions to them and get authorisation verdicts from them, and fed into the reconciliation of the counter view and the client view. The back office was responsible for invoicing … settlement and … and remmuneration …”
So far as the back-end interface of Post Office Counters was concerned, this was known as transaction information processing; is that correct?
Robert Booth: Yes.
Ms Hodge: Please can you describe in a little further detail the work that you carried out in relation to TIP, please.
Robert Booth: When a transaction is undertaken at a branch counter, it’s captured and fed back to the data centre. There was then a harvesting process that would collect that day’s transactions and send them off to the TIP process for distribution on to the automated payment client at that time. That may have been actually using HAPS, the host automated payment system.
I’m not quite sure of the split because now things have kind of merged, so I’m a bit hazy on that, I’m afraid.
But the work was basically to try and make sure that what came in went out. So, if I did ten transactions at the front end, we deliver ten transactions at the back end.
Ms Hodge: So essentially ensuring that what was carried out at the front end could be reconciled with the back end?
Robert Booth: Yes.
Ms Hodge: You have explained in your statement that, in order to carry out the functions you have described, that is to say requirements, refinement and overseeing implementation, you worked from the Fujitsu site in Feltham; is that correct?
Robert Booth: That’s correct, yes.
Ms Hodge: Were any conditions or limitations placed on your right of access to or your ability to share information which you obtained during your time working there?
Robert Booth: Feltham, I think the best way to describe it, was divided into sort of four areas. There was the common area which was the canteen, et cetera, which facilities you could access. There were – there was then, as you came in, on the right-hand side, there was the BA POCL office to which we had access, but Fujitsu staff did not, and the rest of the building was Fujitsu staff only. So we did not have access to the rest of the building and, if we wanted to go and see somebody, we’d have to phone them and they’d meet us at the door and buzz us through and escort us out.
There were different arrangements, I believe, set up for the testers, that they had what was known as Chinese walls set up, such that we had Post Office staff doing testing, but they did it on the basis that what they found out there was not necessarily shared. Now that’s my recollection which may be incorrect.
Ms Hodge: What did you understand the rationale for that to be?
Robert Booth: It was so that Pathway had access – because testers again had Post Office Counters experience so that Pathway would have access to that experience. They would be able to get input about what was wrong with the system quicker and in a non-contractual manner, such that they wouldn’t be afraid of hiding – of exposing shortcomings. So it was a much more open way of working.
Ms Hodge: But your understanding is that any information which they gleaned about the system during the testing phase they were not entitled to share with their employer?
Robert Booth: That’s my recollection. As I say, it may be an error but that’s how I remember it.
Ms Hodge: You have described in your statement attending regular face-to-face meetings with staff employed by ICL Pathway, I think primarily to discuss and review technical documentation; is that correct?
Robert Booth: Not just technical documents because, being at the site, I was also used as what was called a document champion. So my name would appear on lots of documents on which I’m not subject-matter expert. But having a physical presence would mean that I would be effectively be the face of the Post Office for that document and engage with ICL Pathway on it. So I would review the document, not necessarily from a specialist point of view, but then seek specialist input.
Ms Hodge: How would you characterise the behaviour and conduct of ICL Pathway staff during these meetings?
Robert Booth: I think professional would be the best term, not necessarily overly collaborative. I would suggest that throughout the process they were more commercially aware. But again you must realise they weren’t a charity, they weren’t going to do this for nothing. But everything was costed and considered. It wasn’t like some of the other suppliers I have worked with, which is more collaborative and more interested in getting the right solution.
Ms Hodge: Were you concerned that ICL Pathway were not particularly interested in getting the right solution?
Robert Booth: Their world view was that their solution was the right one, and it was difficult to move them from that world view.
Ms Hodge: One of the challenges you have identified in your statement relates to ICL Pathway’s refusal to grant access to its low-level designs and code; is that right?
Robert Booth: That is correct.
Ms Hodge: What effect did this have on your ability to assure the quality of the product that was being produced and the effective implementation of Post Office Counters’ requirements?
Robert Booth: It hindered it greatly. We could get – we could only assure high-level documents which were shared. They were not always initially of the best quality. That did improve. I think that depending on where you look in the continuum of time, at the early stage it was very difficult. The threat of the risk registered for the evaluation pre-tender did elicit documents from them and things, as I understood why, were less hampered about sharing information. It did get better. So we did get high level but we never actually got under the cover.
Ms Hodge: What, if any, efforts did you make to obtain documents or to glean information informally through your working on site and your access to ICL Pathway staff?
Robert Booth: There were corridor conversations. There was, at a working level, a better relationship than when “management was present”, whether that be physically present or with a management hat on. So the people generally were there trying to help and trying to assist. They were constrained by the ways of working that I believe was probably imposed from top down.
Ms Hodge: What were you able to glean in your more informal interactions with ICL Pathway staff about issues that they were experiencing in development?
Robert Booth: I think we were aware they had issues but the depth of the issue was never revealed. I mean, there was nothing that said that there was anything fundamentally wrong, and it was always being looked upon as, in my understanding at the time, probably still is, that in any development project you’re going to have issues, you’re not going to get it right first time, you’ll be correcting it and improving it continuously. I don’t think we understood how far behind where we were expecting them to be they were.
Ms Hodge: When you say you had an awareness of issues, are you able to be more specific as to what they related?
Robert Booth: It was just in passing; there was nothing in particular that I can recall, I’m afraid.
Ms Hodge: I wonder if we could please turn to page 39 of your statement, paragraph 134. I’d just like to clarify, if I may, a comment there. This paragraph reads – do you have that before you, paragraph 134:
“It is worth noting that ICL Pathway did draw boundaries that were not to be crossed in terms of access to lower level designs and correlation of unit/system tests to design to confirm all paths had been considered and tested.”
We’ve addressed the first part already, so the question of your access to low-level designs. Can you explain, please, what you mean in the second part of that sentence to the correlation of tests to designs, please.
Robert Booth: In a software life-cycle, you will generally have designers setting out the shape and direction of where they want the coding to go. You’ll then have the programmers taking that design and working down to a lower level and actually implementing it, and then doing unit testing to say what they’ve implemented meets the design and, generally, previously on previous projects, you get how many tests were run, how many tests passed first time, and from that you can get a judge of not just the quality of the code but how close to the design it is, and you then expect a quality-assurance function to come in and to check-step it, to make sure that Bob’s view of – that he’s coded what’s been asked and it’s 95 per cent of the test passed first time is correct, and you get that quality assurance, and you would then be able to say, “Oh, Bob isn’t as good at coding as Fred is, so we need to do some mentoring”, and be able to look at those areas, or it may actually highlight that designs are not clear enough and the programmers need more guidance.
It may highlight a lack of testing environment, or a lack of supporting test material to adequately test the units and, as things from units move up and get integrated with other units, it may well be that you find that all the wheels are perfectly round, but they are the wrong size for the car, and that then highlights a different problem. We didn’t get any of these metrics, so we couldn’t see how well the software life-cycle was being adhered to and how good or bad it was.
Ms Hodge: What did you consider to be the reason for why you couldn’t obtain that level of insight?
Robert Booth: I think there were probably two factors. The first factor was, “It’s contractual. Go away. This is the boundary, you shall not cross it.” Actually it’s probably – the next words were, “Why are you interested? We’re the professionals here. Leave us and let us get on with it. Every time you ask us a question, you are diverting us from actually doing the work. So, if we keep answering your questions, we’ll never get anything done”, and that was played out quite a lot.
I think that, from some of the things that I’ve gleaned more recently, they probably didn’t have that material to hand.
Ms Hodge: A further problem which you have identified in your statement relates to Post Office Counters’ visibility of the nature and extent of defects that were being identified in the system during the testing process; is that right?
Robert Booth: Yes.
Ms Hodge: If we could just scroll up, please, to paragraph 132 you said there:
“It should also be noted that ICL Pathway development was behind closed doors, and only outstanding defects that were present would be declared to Post Office Counters Limited when its User Testing began.”
Can you explain that in a little more detail, please.
Robert Booth: It kind of follows on from what I was saying that, if it took them ten goes to get a bit of code right to add up two numbers, we wouldn’t see that. We’d only see that the numbers were adding up correctly now.
When user testing began, that was our testers in Post Office uniforms, so where they could share the information they found, actually testing the system and testing it from the user perspective.
I think I mention elsewhere that that means that we couldn’t test some of the technical break points, and failure conditions in particular, which require programmer intervention. So by that, if I send a message and I don’t get a response, I need to do something. From a user perspective, I can pull that wire to stop that message, but I don’t know if I’ve pulled out that wire before it’s been sent, so has it reached the end and I haven’t got the reply, or did the request never get there, because, if the request never got there, the person I’m talking to doesn’t know about it, so they’ve done nothing.
If I’ve asked – let’s say I’ve asked you for £100 and you have got that request and you set back “yes”, if I don’t get back your “yes”, well, I won’t pay that £100, but you will have affected the account by £100. So those kind of technical tests and failure conditions would need Fujitsu intervention to force them, because the user couldn’t actually repeatedly do these kind of tests, they couldn’t force the failure conditions in a repeatable manner, and they couldn’t force all of the failure conditions you would expect a unit test to do, because a system talking back to us may have 50 or 60 return codes that it can return which have different behaviours at the counter depending on what they say.
We would need a system, an emulator, to give those return codes back, and we would be dependent upon Pathway to write that emulator, provide that emulation so the behaviour at the front end could be confirmed as correct.
Ms Hodge: If I understood you correctly, the technical tests and failure conditions you have described, these are processes that would be implemented by ICL Pathway prior to the user acceptance testing; is that correct?
Robert Booth: That’s correct, yes.
Ms Hodge: What exactly was the principal purpose of the user acceptance testing, please?
Robert Booth: The user acceptance testing was effectively what an end user would typically experience. So can they sell a stamp? Does it appear with the right price? Does it – do all the stamps in the basket add up? Do they go to the right place in the cash account? When I’m doing an online transaction, when I’m presented with a magnetic card, does it recognise that, yes, we support that card, it is a Herts County Council card, that should have Herts County Council on the receipt. It should allow you to pay up to £1,000. Are all those elements correct and working?
So really it’s very much the end-user experience, with some failures, and where emulators and simulators are available, with not always a proved verdict coming back from a request – I’m asking you for £100 but I don’t have it, you will send back a decline. So certain behaviours, you can emulate, but it wouldn’t give all of the failure conditions. It wouldn’t allow us to check that the defensive programming that needs to be in place in any system was in place.
Ms Hodge: It’s right, is it not, user acceptance testing didn’t represent the end of the testing process so far as Horizon was concerned? There was further testing of the system that we know as model office and end-to-end testing.
Robert Booth: So model office testing is actually – the model office is actually several offices. They are real offices, they are live offices, they’re not test offices. They are there to prove that tested software has been promoted into the live environment correctly and it works correctly when it’s in that live environment.
If we look at things as we did later with banking we can test against the simulator and we can test that a £50 withdrawal will work or won’t work. But, when we can then connect it up to the real bank and we get back a balance enquiry, that is making sure that the systems work end to end, but the model office is a real branch, it does real transactions, it affects real accounts. It is not there to test. So before it goes into model office we have high confidence that it will work properly.
Ms Hodge: Did you have that confidence in the system prior to its entry into model office and end to end?
Robert Booth: We’re between a rock and a hard place, I think. We had exhausted the testing evidence we could get from Pathway. We had done user-acceptance testing which proved that generally it would work correctly. We had no highs. So for acceptance we would have three categories of fault.
We have a high fault. That’s catastrophic, you can’t go live with it. Fundamental flaw, no way it’s going out the door.
You would have medium-severity faults. These are ones that are significant but would have an acceptable work around. It may mean you have to do something in addition, do something differently, follow a procedure to make sure that things work, but generally the work-around would be acceptable. There would not be more than, I think, ten of these because ultimately a work-around is not natural behaviour, and we couldn’t rely on people doing work-arounds. We couldn’t rely on system work-around because they weren’t always human ones.
Then we would have low ones where we may not have receipts lined up correctly. So the text may be a bit wobbly – cosmetic effectively. Again you wouldn’t want too many of those because, at the end of the day, it’s a professional system. It’s customer and facing real people and, if you have got typing errors and spelling errors in a product, you would start to wonder if what was underneath it was right.
So those would be the categorisations, and model office would be: it has no highs, it has – and it may have some mediums, and they would then try and execute the scenario to cause that failure and then implement the work-around, confirm that it works in the real world because, again, we’ve done it in a testing environment, we need to make sure that it translates to the real world, to the live world. From that the end-to-endness would happen because that model office real-life branch has gone from its counter, through the live data centre to the live account, live client, and we’ve got an end-to-end balance, and we can check that the invoice we give to that client at the end of the day matches what we’ve done at the counter, so it gives us our end to end.
Ms Hodge: What you described just now as an acceptance process, I don’t understand you to be referring to acceptance of the system as a whole. Is it right that what you’re referring to here is the acceptance of a particular release or a component?
Robert Booth: Yes.
Ms Hodge: Prior to its entry into model office and end-to-end testing; is that correct?
Robert Booth: Yes.
Ms Hodge: Now, when you say at paragraph 132 that ICL Pathway’s development was behind closed doors, I don’t understand you to mean that you had no awareness of technical issues; is that right?
Robert Booth: We only knew what we were told. We did not see, as I, say, the unit tests. We didn’t see how many attempts they’d taken to get through a test. The development was effectively done elsewhere, and we didn’t see that development. We saw the outcome of that development.
Ms Hodge: You did see some quite serious issues arising during model office and end-to-end testing; is that right?
Robert Booth: I can’t recall whether they were seen in model office or whether we had – I think you are now referring to acceptance instance?
Ms Hodge: No, forgive me. I’m referring to the model office and end-to-end testing of in particular the EPOS component in late 1998. So we’re looking at the sort of period of November to December 1998.
Robert Booth: I can’t recall any specifics, I’m afraid, sorry.
Ms Hodge: I wonder please if we could bring up the document WITN05970122, please. I believe you’ve been shown a copy of this report; is that correct?
Robert Booth: I’ve been shown many documents. That may well have been one of them, yes.
Ms Hodge: This report is entitled Contingency Options for the Non-availability of EPOS feed to TIP. We established a little earlier that you had some involvement in TIP, the transaction information processing, in the design and development phase, and you are recorded here as one of the authors of this report. It’s dated 12 January 1999, version 1.0. Do you have any recollection of this report?
Robert Booth: Not at the time of writing, no.
Ms Hodge: If we could scroll down, please, to the second page, under the main heading of Background, this is paragraph 2, if we go a little further, paragraph 2.1.2 it reads:
“During the Model Office and end-to-end test phases with the ICL Pathway Horizon solution, significant problems have been experienced in using the data stream provided by ICL Pathway. It is believed that ICL Pathway may not be able to fix these problems within the necessary timescales to enable the full data feed to TIP to be proven without causing a severe impact on the New Release 2 timescales.”
Does that assist your recollection at all?
Robert Booth: I’m afraid it doesn’t, no.
Ms Hodge: As the author of this report, this must have been an issue of which you were aware at the time; would you accept that?
Robert Booth: Yes.
Ms Hodge: If we go a little further, please, to paragraph 2.2.1, which may be over the page. Yes, thank you. This provides a summary of the errors that were being experienced. They included an inability to make files available to TIP, there being no balanced cash accounts in the first phases of testing; the fact that files contained wrongly dated transactions; incorrect versions of reference data; spurious outlets not known to RDP or TIP.
Can you explain what RDP means?
Robert Booth: I believe that would have been the personal reference data system. So that would be providing information saying, “We’ve got these branches and this is the address to put on the receipt”, et cetera.
Ms Hodge: And also:
“Files containing transactions that cannot be performed according to the submitted reference data and business rules causing TIP to abnormally end (abend) processing.”
What’s meant by “abend”, please?
Robert Booth: Abend is just the shorthand for abnormal end.
Ms Hodge: “Final file resubmission taking several days and then being rejected as the handcrafted fixes invalidate the file integrity totals.”
So a number of fairly serious concerns; would you agree –
Robert Booth: Yes.
Ms Hodge: – about the accuracy of the accounting data being harvested and transferred to TIP?
Robert Booth: Yes.
Ms Hodge: At paragraph 2.2.2 we can see reference to the problems being experienced appearing to occur across the board, and not being confined to one or two specific areas. It’s noted that this means that it’s not just a single area where there’s a deficiency that’s in need of attention but several. In terms of the source of the problems, you appear to have concluded at paragraph 2.2.3 that the problems were wholly within the ICL Pathway domain at that stage. Again do you have in recollection at all –
Robert Booth: No, I’m effectively reading this for the first time, if you know what I mean. I have no recollection of actual events at the time.
Ms Hodge: Presumably what you’re saying in effect is you simply don’t recall any of these matters?
Robert Booth: No, I don’t, I’m afraid.
Ms Hodge: Does that explain why they don’t feature at all in your 63-page witness statement?
Robert Booth: Yes.
Ms Hodge: Sir, that might be a convenient time for us to take a short break this morning?
Sir Wyn Williams: Yes, that’s fine. What time shall we recommence.
Ms Hodge: Shall we resume at 11.15?
Sir Wyn Williams: Yes, thank you.
(11.05 am)
(A short break)
(11.15 am)
Ms Hodge: Good morning, sir, can you see and hear me?
Sir Wyn Williams: Yes, I can thank you.
Ms Hodge: Thank you very much. Mr Booth, I would like to ask you some questions now about your knowledge of ICL Pathway’s access controls and privileges. It appears that one aspect of the project in which you became involved prior to its roll-out related to issues of fraud and security; is that correct?
Robert Booth: It is.
Ms Hodge: We see that you feature on the distribution list of a number of ICL Pathway documents relating to access control and security management. I think you are described as a member of the Horizon Fraud and Security Group?
Robert Booth: I don’t recall being part of that group.
Ms Hodge: Can you describe what the function of that group was, please.
Robert Booth: I don’t actually know what that group was, I’m afraid.
Ms Hodge: If we could just bring up FUJ00001493, please. This of course is an ICL Pathway document, so it may be that some sort of the misunderstanding has arisen, but it relates to what’s described as the NR2 access control and user administration processes and procedures, and we can see it’s dated 30 June 1999. This is version 2.0. If we scroll down, please, to Distribution, your name features under the heading Horizon Fraud and Security Group.
Is that a group of which you were aware?
Robert Booth: Not necessarily that title. There were people dealing with fraud and security, in particular. I was not part of the inner circle of that, if you like. Again, given my location at Feltham and general knowledge, I’ve been a participant in some of those documents, but I wasn’t – I wouldn’t have been the lead subject matter expert on that.
Ms Hodge: When you say there were people involved in those issues, are you referring to people sort of the back in Post Office Counters, or are you referring specifically to employees that Feltham site that you had?
Robert Booth: No, it wouldn’t have been at the Feltham site. It would have been both Post Office and – at the time would this have also been when BA were still around?
Ms Hodge: I think this is June 1999. So by that time they would have withdrawn from the project.
Robert Booth: It would probably have still had their input from the start, because the board risk management system was very much a BA-driven item because that was primarily looking at reducing fraud of the benefit books, and that was a BA initiative with Post Office participation, and I assume that this would then carry over from that group.
Ms Hodge: We see you referenced again, please, in POL00043742. This is a document entitled Security Management Procedures, co again an ICL Pathway document dated 17 September 99, version 1.2, and again under Distribution we see the Horizon library is identified but specifically for your attention.
Is it likely that you were invited to review these documents as part of your role as the single point of contact or document review, the document champion that you have described.
Robert Booth: Yes. Basically I had my names on lots of documents I reviewed, as I said earlier, not necessarily as the subject matter expert, but as the conduit both to and from them.
Ms Hodge: In doing that, would I be right to understand that you would have liaised with those in Post Office Counters who were responsible for fraud and security issues?
Robert Booth: Yes.
Ms Hodge: And who were the experts in that field?
Robert Booth: Yes.
Ms Hodge: Do you happen to recall who they were at this time?
Robert Booth: I knew you were going to ask that one. I’m afraid I don’t.
Ms Hodge: Presumably, if you had the task of reviewing these documents relating to access controls and security management, you had some understanding of the purpose of these controls; would that be fair?
Robert Booth: I will have had from two perspectives. One would have been my understanding of the rest of the system and, therefore, was it at odds or was it trying to either expose or gloss over something that I was aware of elsewhere. As part of my mindset, if you like, reviewing the document and searching for the inconsistencies within a document is what I would also have been doing from that perspective, but as an educated layman about these subjects rather than as the expert on these subjects.
Ms Hodge: What did you understand about the controls placed on the ability of ICL Pathway staff to access or to amend transaction data recorded on Horizon?
Robert Booth: I was unclear about how they would amend but, having been around IT systems, ultimately the systems administrator can always go in and, no matter what controls you’ve got, if they have a high enough level of privilege, you can do whatever you like. So – not whatever you like, but you can always get into a system.
It’s then around the integrity of the data and the tooling, to make sure that any changes are audited. I can’t remember if this document – or it was common understanding was that changes would be four eyed i.e. you would have two pairs of eyes on it, so that one person was doing the change, another person would look at the change to make sure they only change what they said they were going to change and there was a record of that.
Ms Hodge: From what you have just said, it seems to be that your experience from working in programming and other automation systems was that this type of remote access to transaction data was quite common – is that right – it was a necessary component of a system of this type?
Robert Booth: There would be two types of access. There would be the read-only for investigation, which would be more common, because that would enable you to find any underlying issues, and it would also allow you to do enquiries which were not – you could run a report that wasn’t otherwise there for you. So, if you wanted to know how many branches had an odd number of counters, there wouldn’t be a report for that but, if you had access to the data, you could go in and find it.
Where amendments are made to data, generally that would be done via an auditable tool, rather than just going and over-typing data, which I think may have occurred later, from some of the – especially just before the break, some of the handcrafted fixes and files we saw, generally I’d have expected tooling to have done that rather than a manual process.
Ms Hodge: The mere fact that ICL Pathway could do this was something that you were aware of at the time, was it?
Robert Booth: It’s something that’s endemic in any system.
Ms Hodge: You consider that knowledge of that was something that was widely held within Post Office Counters at the time?
Robert Booth: Yes. Well, there may have been a naiveté amongst the non-technical people, but anyone who has a technical background will know that, if you have sufficient privilege, you can go in and look at the data, and you can amend the data.
The integrity of the data and the integrity of the database that you need to put around it to protect that, and controls you need to put on such that you know that this user with those privileges has logged on at that time, you’d know where they were in the system and, having an audit of what they’ve done, that was the controls that you’d put on top of it.
Ms Hodge: What did you understand about the extent of these privileges that ICL Pathway staff had in relation to their ability to access and amend the data? Did you have much knowledge at all of the extent?
Robert Booth: No. I mean, the expectation would be that it would be very limited in terms of the number of people that could do that, and they would, as I say, be overseen whenever they did do any amendment.
Ms Hodge: During your time working in Feltham, did you ever discuss with ICL Pathway staff the extent to which these privileged rights were exercised?
Robert Booth: Not to – I think that the setup at that time – Pathway would have had staff in Ireland doing such work. So it would have had – they had data centres in Belfast, and their support service centre, I think, at the time was based in Belfast. I may be wrong on that.
Ms Hodge: Thank you. I’d like to move on now, please, to a new topic, this being the acceptance of the Horizon IT System that took place in the summer and autumn of 1999. What do you recall about your involvement in the acceptance of Horizon?
Robert Booth: With the documentation and things, I was involved in acceptance incidents that were brought to our attention and managing out of the system. So a defect would be raised, it would be qualified as: is it a defect or as Pathway would like to say, is it a feature? So is it a true defect that needs remedy, or is it just misinterpretation? Then what is the material impact of that?
I go back to my high, medium, low type criteria. Is it a service stopper, is it service impacting, or is it just an annoyance?
From that they would then be tracked down with the intent of clearing as many as possible, obviously before going live because you don’t want a system with known defects in it.
Ms Hodge: One of the incidents which you raise was known as Acceptance Incident number 372. This related to systems management. What do you recall, if anything, about that incident?
Robert Booth: Again from some of the documents, I believe that that was around concerns that, at the volume, Pathway were not necessarily able to distribute software in a reliable manner or a timely manner anyway.
Ms Hodge: One of the high severity AIs which you were involved in – forgive me, acceptance incidents we abbreviate to AI – was Acceptance Incident number 298 relating to system stability. Do you recall that?
Robert Booth: Yes, I have some recollection of that from material especially some of the other material that’s arisen.
Ms Hodge: What were the nature of your concerns in relation to that incident?
Robert Booth: There was some sympathy that the system would lock up and freeze. PCs of the day were less reliable than they are now, and there was at the time – “blue screen of death” was an industry phrase because of Microsoft’s frequency of locking up and freezing.
From some of the material, it’s obvious that we were suffering similar kind of behaviours, and occasionally we’d have an hour glass come up where the system was busy and effectively would hang, meaning that the users couldn’t use the system. This went on for several months.
Pathway were in denial at some stage of the extent and the impact of the problem. The issue you have is, if your machine is effectively hung, the only way you can unhang it is to turn it off and turn it on again and, when you do that, you run the risk of corruption.
You also then have an unknown account rate because Pathway would not put in something that would say, “This machine has just started up”, and collect those metrics automatically, which would have been something we would have asked for and would have been a reasonable thing to say how often is this happening in the estate. They would much rather turn to Post Office and say, “Can you phone round your branches and ask them how often they are rebooting.” So there was, in my view, a simpler technical measure that could have given us a very accurate picture, but Pathway would prefer us to do a manual, more – less accurate, more onerous phone-round to find out what was happening.
The eventual solution – in the same way that Microsoft even today suggest you don’t leave your machine turned on and hibernating, but you turn it off regularly – was to instigate nightly reboots such that the machine would reboot itself each night on the belief that the problem was due to running out of resources. So that I keep on – I don’t know what the root cause was but, say I keep on opening a file, so I have a file open, and I have consumed one opening. If I don’t release that opening, if I open it again, I’ve consumed 2, I’ve consumed 3, I’ve consumed 4. I can only consume so many before I run out of them, and then I would hang.
If you restart your machine each day, you always start at 1 and, by the end of the 23 hours and 59 minutes, hopefully, you haven’t got to the big number where you would have hung. So by restarting it regularly, it means – you still have the problem, but you don’t have the consequence of that problem.
Ms Hodge: Was that a solution implemented by ICL Pathway, that is to say, were the systems automatically rebooted?
Robert Booth: Yes.
Ms Hodge: It didn’t require the subpostmaster or the clerk to do that?
Robert Booth: No.
Ms Hodge: One of the medium severity acceptance incidents of which you had oversight was AI211. This related to a receipts and payments mismatch. Do you have any recollection of that?
Robert Booth: Not familiar with that one as much.
Ms Hodge: I think records on that are relatively limited, but if we could bring up, please, POL00028360. This is what appears to be effectively an action log prepared recording the output of a meeting on 11 August 1999, where we see the various acceptance incidents recorded there, severities as defined by Pathway and Post Office Counters, and then the final column some remarks. Please could we scroll down to page 9.
So we see here AI211 receipts not equalling payments, and there’s a reference to a memo from you. I don’t think we’ve been able to obtain a copy of that, but it would tend to suggest that you had some involvement in this AI. Would you agree with that?
Robert Booth: I would agree with that, yes.
Ms Hodge: There’s reference there to the implementation of a rectification plan, and the results having been shown to work, but there being some remaining incidents due to reference data. I’m looking here at the third column, as I’m sure you are aware, and one incident being reported due to kit swap-out reintroducing pre-LT2 which would be a reference to live trial 2, would it? SW –
Robert Booth: Software.
Ms Hodge: There’s a point to say that action is with Pathway to investigate, and there be a possibility that a combination of builds have not been tested properly. Does any of that assist your recollection of this particular AI?
Robert Booth: Not – I can interpret what it says now, but I wouldn’t say at the time.
Ms Hodge: Please could you do so to the best of your ability.
Robert Booth: I would suggest that the medium to low was contested, and I had reason to say that it shouldn’t be moved down to a low, that it should be maintained at a medium severity incident. But without that memo, I’m afraid can’t substantiate that.
On terms of the reasoning, it would suggest that the software that was put on the counter was not the correct build because it was a swap-out and therefore a bug that had thought to have been cleared was reintroduced because it was on old software. That’s my reading of what’s in front of me.
Ms Hodge: Was that a problem of which you were aware, namely the risk of reintroducing bugs that had previously been fixed?
Robert Booth: If you’re bringing kit off of the engineer’s van that is – it will not be current necessarily. So, if today I fix this computer and I fix this one, fine. I now put that on my van and I have an update, this one’s turned on and it will get updated, the one on the van won’t. When it gets installed, it needs to be brought up to the current build before being used. Looking at this, it would appear that that didn’t happen and, therefore, an older version that the engineer installed was actually used in live with these consequences. That’s my reading of what’s in front of me.
Ms Hodge: Thank you.
Finally before we move on to another topic, although you were not directly responsible for the incident, you were aware, were you not, that an incident had been raised relating to the accounting integrity of the Horizon System under AI376?
Robert Booth: Yes, I would have been aware of – because the Acceptance Incident form went through all of the incidents, I would have been party to that, yes.
Ms Hodge: Bearing in mind your earlier involvement with transaction information processing and the problems that had been identified in January of 1999, was that an Acceptance Incident in which you had any involvement or took any particular interest?
Robert Booth: Sorry, I can’t comment on that.
Ms Hodge: Did you have any concerns at the time about that particular AI and its impact upon Post Office Counters and its accounting integrity?
Robert Booth: All AIs were of concern. As we lovingly called it, the route to green. So to get them to be a green light you could go ahead with was to rectify them. So having an Acceptance Incident was bad, but it did mean that the fault had been found and, if you find something that’s wrong, you can fix it.
So the volume of AIs was disappointing, but what was important was, before going live, the volume that were left outstanding and the type that was left outstanding before going live –
Ms Hodge: Sorry. Thank you. One of the final topics I would like to deal with, please, relates to what you knew at the time about the role of investigations and prosecutions in Post Office Counters. Now, one of the low severity AIs for which you were responsible was AI370. Do you recall that AI?
Robert Booth: I don’t. Do you have a …
Ms Hodge: Yes, by all means. Please could we show POL00028508. This is an email from Min Burdett. Did you have any direct dealings with Min?
Robert Booth: Min was one of the managers around in Post Office, probably based in the London office. Yes, I had some dealings but not day-to-day.
Ms Hodge: This email is addressed to, we can see, Keith Baines and Chris French. You are not on the face of it copied into this email, but it contains an attachment which relates to what is described as “the new acceptance process”. So this is in late January 2000 after the system has been formally accepted by Post Office Counters and the roll-out has started and indeed recommenced.
If we can scroll down, please, to page 6, if we perhaps go back to the second page just so you can see the title of the document, sorry, this is a document called Concluding Pathway Acceptance version 0.1. It contains a table with a number of acceptance incidents that were outstanding as at 28 January 2000, it would appear. We can see that, please, at page 6.
This is a list of the medium severity acceptance incidents. We can see that you’re still named in relation to AI372, the system management, an incident to which we referred earlier. There was another outstanding medium severity 314 relating to the provision of documentation and, at page 8, please, about halfway down the page, can you see an entry AI370 the title of which is Witness Statement?
Robert Booth: I can, yes.
Ms Hodge: Under the heading Post Office Counters Limited ATM?
Robert Booth: Yes.
Ms Hodge: Do you know what ATM stands for?
Robert Booth: I assume Acceptance Test Manager.
Ms Hodge: You are named there.
Robert Booth: Yes.
Ms Hodge: Now, before we move on from this document, if we may please just go back to page 5, this relates to the procedure for closing outstanding acceptance incidents, and provides:
“The ATM’s role in closure of AIs will continue as now. When the ATM is satisfied that the closure criteria have been met, he or she should email his or her line manager explaining the reasons for and recommending closure. The ATM should consult with appropriate business representatives who typically [would] have been involved during the lifetime of the AI.”
If we go on, please, just a little further down under the heading Handover of Closure Baton Recipient, at 3.9, it provides:
“For a number of the AIs, there will be ongoing monitoring to ensure that the AI solution does not regress. The ATM [the Acceptance Test Manager] will be responsible properly briefing the baton recipient …”
Then there’s a note in parenthesis:
“More guidance on this [is needed] to be included here when the issue has been discussed with BSM” – would that be business service management?
Robert Booth: Yes.
Ms Hodge: “Typically”, it says, “as shown in appendix 1 the baton will be passed to BSM. In the past this has not always been done, and a review of all currently closed AIs will take place short to identify any missing batons.”
So just pausing there briefly before we return to AI370, does that broadly reflect your understanding of how these incidents were closed? That is to say the process by which responsibility for an incident was transferred?
Robert Booth: I can’t recall this detail, but it’s an obvious transition into business as usual, by saying that the program looks to be winding down at this stage and we’re handing things over to the normal business service managers for monitoring, and anything that is not closed, for them to then manage to closure.
Ms Hodge: Thank you. We can see some reference to AI370 in a document that bears the reference WITN05970134, please. This document is entitled Review of Horizon Cash Account System Stage 2. I don’t believe you were the author of this review or rather the report into this review.
Robert Booth: No.
Ms Hodge: But it does helpfully provide some information about AI 370. If we could go, please, to page 3, we see reference to An outstanding acceptance incident 370 graded as low. Can you see – forgive – me in the box?
Robert Booth: Yes.
Ms Hodge: This report, I believe, is dated July 1999. So it’s at or around the time where these acceptance incidents were first raised and being addressed. It says:
“[This incident] exists against the POCL requirement, on the assertion by Post Office Counters Limited that Pathway should” – it says “product” but presumably it should have read “produce”?
Robert Booth: Yes.
Ms Hodge: – “a witness statement to support prosecution. This AI revolves around the interpretation of ‘ensure that all relevant information is evidentially admissible’. POCL’s view is that to be admissible it will need to be supported by witness statements, et cetera; Pathway have stated that they will ‘provide PACE statements as necessary to support a fraud prosecution’, but that ‘the work required to produce draft witness statements’ is not within the scope of the requirement and will be done once POCL raise a Change Request.”
So does that assist you at all in relation to this particular AI?
Robert Booth: I can read that, and my interpretation is that basically Fujitsu was saying, yes, they will do it, but they want paying for it each time it’s requested. So it is not part of the service that is included, but it would be done under change control, i.e. the Post Office would say, “We require a witness statement from you”, and they’d say, “Cost you £5,000, work order, change request, please raise.” Post Office would then raise that and Pathway would then produce the witness statement. That is my interpretation of what’s there.
Ms Hodge: So this effectively was a request being raised of Pathway in the summer of 1999; that’s correct, isn’t it? That is to say, the provision of a witness statement of this type?
Robert Booth: No, I’m reading that they’re saying that the provision of witness statements does not fall within the paid-for service. Each witness statement that’s required would require separate payment.
Ms Hodge: Forgive me, for this to arise as an Acceptance Incident presumably –
Robert Booth: That would have been the disagreement, that Post Office would have thought it was within the costed service, and Pathway were saying, “No, it’s an add-on”, and I think that, from reading this, would have been the element of discussion about why it’s an Acceptance Incident, because Post Office were not getting from the service what they thought was included.
Ms Hodge: Indeed, but presumably having had oversight of this Acceptance Incident, it was on your radar at the time that Post Office Counters were expecting to obtain evidence of this nature from Pathway in order to support the prosecution or – well, the investigation into and potentially eventual prosecution of subpostmasters or managers or clerks suspected of fraud. Is that a fair inference to draw?
Robert Booth: I had very little to no involvement in witness statements per se. So it could easily have been providing a witness statement to say that the person was not guilty of a fraud –
Ms Hodge: Forgive me, I’m not suggesting you would have had a detailed understanding of what a statement contained or indeed whether –
Robert Booth: No, understanding that they provision for something that’s evidentially admissible, yes.
Ms Hodge: You were aware of that at the time?
Robert Booth: Yes, in terms of that context, but probably only to this level.
Ms Hodge: And that certainly part of the Post Office’s investigation and audit function was to enable these prosecutions to be brought?
Robert Booth: Yes.
Ms Hodge: Now, it appears from this document that the part of the Post Office with responsibility for this aspect was known as the Security and Investigations Executive; is that correct, do you know?
Robert Booth: The only acronym I knew of them was POID, Post Office Investigation Department. So the acronym, I’m not familiar, but it makes sense.
Ms Hodge: What, if any, liaison did you have with them?
Robert Booth: I don’t think I had any. It would have gone through the security forum.
Ms Hodge: Sorry, forgive me?
Robert Booth: The security forum that we mentioned earlier, the fraud risk management people were working, if you like, on the human factors, the fraud risks, et cetera, and it would have been that forum that would have taken such considerations forward.
Ms Hodge: Obviously, as the Acceptance Test Manager, you had personal responsibility for the resolution of this Acceptance Incident; that’s correct, isn’t it?
Robert Booth: It is.
Ms Hodge: In that role would you have liaised – would you have been liaising directly with either –
Robert Booth: I can’t say who I was liaising with. But again, looking at this, it wasn’t a technical defect. This is about a commercial arrangement. So, providing Post Office were willing to pay, the service they were expecting that would be included would be fulfilled, and that’s why it would be a low Acceptance Incident. There was no – from what I’m reading here, there was no contention about production of witness statement. It was purely about whether there was going to be an additional charge from Pathway for that.
Ms Hodge: I think, Mr Booth, what I’m trying to get to the bottom of, if I perhaps make this a bit clearer, you were obviously aware at the time of an Acceptance Incident relating to data integrity society number 376, and you were aware at the time of an Acceptance Incident relating to the provision of witness evidence which was number 370.
To your knowledge, were those in the fraud and security group kept abreast of the variety of acceptance incidents that were being dealt with, so those relating to data integrity as well as, for example, those relating to witness statements or those aspects more directly concerned?
Robert Booth: I don’t know what other people were aware of, I’m afraid, sorry. I wouldn’t know how these matters were propagated and who the audience and recipients were.
Ms Hodge: Did you yourself communicate any of the concerns which you previously articulated about accounting integrity to those with whom you were liaising on AI 370, for example?
Robert Booth: Yes, anyone that – if we are talking about a specific AI, the audience for that would be fully aware of it. So I’m afraid I don’t quite understand the question.
Ms Hodge: Sorry, forgive me. We established you were aware of AI 376 and the issues relating to data integrity. They reflected concerns that you had previously articulated in January 1999 in the report to which we referred.
Robert Booth: Yes.
Ms Hodge: We can see you also having oversight of an Acceptance Incident relating to the provision of witness evidence to support a prosecution. What I’d like to know is whether you recall taking any steps to bring to the attention of those who were dealing with prosecutions the issues of which you were aware relating to data integrity.
Robert Booth: I would not have done that, no.
Ms Hodge: You wouldn’t have done that?
Robert Booth: I don’t think I would have, no.
Ms Hodge: Why is that?
Robert Booth: Because the Acceptance Incidents, the only involvement in terms of the witness statement was the a commercial one. It wasn’t the provision or the underlying data and, therefore, that wouldn’t have been relevant to the actual Acceptance Incident.
The security forum or, if you like, the people who were X from the fraud risk management, would have had other ways into the service, I believe, and other liaison points. The focus was very much on each individual Acceptance Incident and not widening it out. It was trying to close it down rather than trying to cascade it out and make it bigger. Does that make sense?
Ms Hodge: Yes, I think so. In relation to closure of AI 370, we saw in the earlier plan for closure that one of the avenues for closure was into the business service management. Is it likely that AI 370 would be one of the Acceptance Incidents that were resolved in that way?
Robert Booth: It would have – I think, in this instance I would have guessed the natural home would have been more under the commercial forum, because against commercial question marks and technical deficiency. So it would have found its way – a baton passed to someone who I don’t know.
Ms Hodge: Thank you. Finally, Mr Booth, I would like to ask you about a document that’s been provided to the Inquiry by Mr Folkes who was your manager at the time. It’s entitled the Horizon Brain Dump. I think you’ve been provided with a copy of that document; is that correct?
Robert Booth: I have, yes.
Ms Hodge: That bears WITN05970123, please.
Have you had an opportunity to read this document?
Robert Booth: I have.
Ms Hodge: Do you recall whether or not you were shown a copy of the document at the time it was produced?
Robert Booth: I don’t, but I suspect very much that I was because of the relationship I had with Mr Folkes.
Ms Hodge: There’s a reference to you, Mr Booth, at page 23 of that document. I wonder if we could bring that up, please. Thank you. Under the heading C6.7 System Management, it reads:
“ICL Pathway’s ability to detect and manage certain failures in the system is as yet somewhat unproven; although we have assurances from Pathway on a number of issues, evidence that failures would indeed be detected and responded to (rather than just logged and ignored). Will only come from live running.
“There are a number of scenario discovered through the technical assurance work which gives examples of possibly failures”, which we can see listed there.
Then in brackets that bottom:
“Bob Booth is our expert in this area and has access to the previous technical assurance work.”
Does it follow from this that you were expected in effect to carry forward the system management aspect of –
Robert Booth: I think the intent here was the document was giving pointers to the reader about, if you want more information or if you have a concern where to go for this, because the background of an involvement I had previously, I would have been a primary contact. So the document highlights that, if you want to know some of the background, then drag out an incident at the time, then I’d have been a good point of contact to explain the background to it, what we’d done to resolve it, and where we felt it was going at the time.
Ms Hodge: Notwithstanding the formal resolution of AI 372 which related to system management, did you share the concern that Pathway’s ability to detect and manage certain failures remained unproven?
Robert Booth: I think so, yes. It was more to action what they found. The information sometimes was there but they were not proactively looking for it. So, if you knew what you were looking for, afterwards you could go and find it, but they were not necessarily looking for defects that were apparent.
Ms Hodge: Why do you say that they weren’t proactively looking for it?
Robert Booth: Because the majority of the defects, I believe, came from the result of Post Office saying, “Something’s not working”, rather than Pathway saying, “We’ve got a problem here.”
Ms Hodge: Are you referring here to problems identified during the live operation and trial and the acceptance process, or are you referring more generally?
Robert Booth: More generally. I mean, that was the start obviously, when things would have been more under a microscope. But as more and more users – because you need to realise that, if you test a system with a team of sort of the 5/10 testers, you get 5 or 10 man-days. You put it out to 300 branches as a live trial and there’s three people, you are getting a thousand man-days a day. So the scale of exposure a system gets when you start putting it out into the real world is far, far greater than you get in a test environment.
We would get feedback from users, such as the hanging, which were not really experienced because we probably didn’t do enough duration and leaving the machine on long enough before a patch would occur to a machine such that it would reboot and clear things. But, when it was being used in the field, these were being experienced. So again slight difference between live and test.
Ms Hodge: If I’ve understood you correctly, you remained concerned even after acceptance and roll-out that ICL Pathway were not doing enough to identify and detect faults in the stem. Is that a fair summary –
Robert Booth: I think as the end customer, you always want your IT supplier to be doing more. So yes, it is. Whether it was really reasonable or unreasonable, I think that’s a hard call. I don’t think that Pathway – I think Pathway could definitely have done more. How much more is questionable before it becomes an onerous task with very little return.
Ms Hodge: Did you articulate your ongoing concerns to others within Post Office Counters Limited?
Robert Booth: I think that everyone was aware that the amount of information we were getting out and the amount of benefit of raising those things was a lost cause. It was very much the contractual boundaries were set, and that’s the rules that ICL Pathway were playing to.
Ms Hodge: Thank you, Mr Booth. I have no further questions for you but there will be some questions, I believe, from the representatives of other core participants.
Sir Wyn Williams: All right. Who is first up?
Mr Jacobs: Sir, I think it’s me, Mr Jacobs.
Questioned by Mr Jacobs
Mr Jacobs: Mr Booth, I ask questions on behalf of 153 subpostmasters who were dismissed, prosecuted, sued and lost their reputations as a result of the actions of Post Office.
In your evidence today you have made it clear that as a result of the contractual boundaries between Pathway and Post Office and as a result of Pathway’s attitude – and you have said that Pathway said, “Why are you asking questions? This is taking up too much time. We’re the professionals here”, and you said that the Post Office were hindered by a lack of access to technical details from Pathway. Is that a correct summary of your –
Robert Booth: That’s correct, yes.
Mr Jacobs: – evidence? What my clients want me to ask on their behalf is: do you accept, looking back and from what you knew at the time, that the Post Office didn’t have anywhere near enough visibility and knowledge of the system to dismiss, pursue and prosecute my clients for these alleged shortfalls that arose?
Robert Booth: I don’t know any of the circumstances of the prosecution. So I’m afraid I can’t comment on that.
Mr Jacobs: But your evidence is, I think it’s right to say, that Post Office didn’t have sufficient, from your perspective, knowledge and visibility of the system that Pathway had developed.
Robert Booth: The degree of technical exposure was not enough to give the level of assurance before putting the system live that I would have liked. The method by which we gained confidence was by putting it live, and we would rely upon ICL Pathway for technical statements and for technical information. We would not be – or would not be able to speak authoritatively ourselves about it. Even though we would have a good shape at a high level, we would not be able to know at a low level the detail, and that would be within the ICL Pathway domain only.
Mr Jacobs: Your evidence is you didn’t know the low-level details in relation to the system.
Robert Booth: Correct.
Mr Jacobs: Do you consider then with the benefit of the hindsight that the postmasters ought to have been given the benefit of the data by Post Office before being pursued?
Robert Booth: Again I don’t know any of the details of individual cases, and can only say that anything that should have – anybody who’s being prosecuted should have the same access as the prosecutor, that any evidence that was available should have been made available, be that from the Post Office or be that from the IT supplier who does have access to it.
Mr Jacobs: In light of what you know and what you told the Inquiry, how do you feel about the way the subpostmasters were treated?
Robert Booth: From a personal view appoint even just having to give a witness statement, it’s traumatic. So on a human level, it’s traumatic. I have seen some of the statements. It is not good news. It’s just not good news.
Mr Jacobs: Thank you. I don’t have anything else.
Sir Wyn Williams: Anyone else?
Questioned by Ms Page
Ms Page: Ms Page, on behalf again of a group of the subpostmasters, and I’ve got two areas that I’d like to ask you about, please, Mr Booth. The first relates to your witness statement and paragraph 30. I don’t think we need to call it up. I will read it out and go from there.
You said, when the Benefits Agency withdrew from Horizon, there was no substantive change in your role of being a point of contact for Fujitsu and reviewing the Fujitsu documents. But then you said:
“The main difference was that I felt that my views were given more weight as any concerns I raised would be moderated by Post Office as opposed to Benefits Agency where the single Benefits Agency product outweighed the needs of all the POL products.”
So, in other words, what you were saying was that, before the Benefits Agency dropped out, their single need this is need to have benefits paid through the Horizon System, was absolutely pre-eminent.
Robert Booth: It had undue weight, in my opinion. The Post Office had 180/190 products. Benefits Agency was one of them. The Benefits Agency made it very clear in their day-to-day workings beside us as colleagues that they viewed us, the Post Office, as the junior partner and that it was only because the Benefits Agency was there that the automation was occurring.
Where effort was to be expended, the pressures from the two contracting authorities, one who had one product, one who had 180, it wasn’t 180 to 1, it was more sort of 1 to 1.
Ms Page: The other way round?
Robert Booth: The Benefits Agency was one of the prime contractors; therefore, their interest had to be served.
Ms Page: So when they dropped out and Pathway still had to develop a system that would manage your 180 products or thereabouts, they were suddenly doing so on a highly contracted timescale, weren’t they?
Robert Booth: I can’t recall but probably, yes.
Ms Page: May ‘99 was when Benefits Agency dropped out and you were rolling out that year going into the following year.
Robert Booth: They would have been working on it before that. It was just not necessarily concentrating on it exclusively. They would have been having resources working on the Benefits Agency product as well as the Post Office products.
Ms Page: Can I just ask, please, for Mr Cipione’s report to be pulled up. This is the second area that I want to look at, please, and it’s EXPG0000001, and if we go to page 118, please.
If we scroll down to 7.1.5, I’ll just read a few paragraphs out, please, and it starts:
“In November 1999, at least one full month and possibly two full months after acceptance was granted, ICL Pathway reported that ‘POCL have come round to the understanding that dead with residual AI 376 concerns in the short to medium term will rely on processes and tools but no new software features as such.”
“In January 2000 ICL Pathway states, ‘If pressed POCL would agree that AIs 342, 372, 376, 378, 218, 391 are closed/incapable of further update. Their acceptance manager is leaving the project at the end of February.’ Further, in the same report it states, ‘The outturn on AI 376 was 0.06 cash account discrepancies, exactly an order of magnitude better than the target. Under this activity, John P made significant contributions to the third supplemental agreement, specified the committed CS repair facility, aligned the operating agreement on reconciliation to support the contract, sorted out the necessary PinICLs to clear.
“In February 2000, ICL Pathway declared that the POCL acceptance manager had left the project and transferred the residual actions to ‘business as usual’.”
Then this is Mr Cipione’s comment on that:
“It is unclear to me what exactly took place to close AI 376. The reading of these entries leaves much room for interpretation.”
So those paragraphs you wouldn’t have seen, of course, the ICL Pathway reports but that’s their take on what’s going on to close AI 376. Mr Cipione says, in conclusion, it’s entirely unclear to him what actually happened to close AI 376 and, indeed, when he looks at those management reports from Pathway, what we see is terminology that is rather concerning, is it not? If pressed, POCL would agree that certain IA’s are closed. POCL have come round to the understanding that they are not going to get any new software features.
Was POCL really under the cosh here from Pathway? Were you just accepting their desire to get things rolled out?
Robert Booth: No, no, no, I don’t have first-hand knowledge of this but I disagree with the tone. I’d say it was very much internal ICL positioning rather than necessarily the facts on the ground.
Ms Page: What about this sort of suggestion that it’s not at all clear how AI 376 was concluded? Can you tell us how it was concluded?
Robert Booth: I can’t comment on that. I don’t know.
Ms Page: Thank you.
Ms Hodge: Thank you, sir. Unless you have any questions for the witness, that concludes the evidence of Mr Booth.
Sir Wyn Williams: No, I don’t have any questions.
Thank you, Mr Booth, for making a detailed witness statement and for coming to the Inquiry to give evidence. I’m grateful.
Robert Booth: Thank you, sir.
Sir Wyn Williams: Where do we go from here Ms Hodge?
Ms Hodge: Our next witness is Mr Meagher, I believe – forgive me, “Mar” is the correct pronunciation. I understand he arrived only a short time ago and may wish to have a brief consultation with Counsel to the Inquiry.
Sir Wyn Williams: I mean, I think the only issue is whether we take our lunch break now.
Ms Hodge: If you would be content to, we certainly would be happy to rise early and return early. We’re in your hands.
Sir Wyn Williams: All right. So it’s now ten past, or thereabouts at least, 12. If we started again at, say, 1.20, would that give time for everybody to have lunch and for Mr Meagher to have a chat with whomsoever he’s going to speak?
Ms Hodge: I’m sure that will be sufficient. Thank you.
Sir Wyn Williams: All right. We will start again at 1.20.
(12.10 pm)
(Luncheon Adjournment)
(1.20 pm)
Sir Wyn Williams: I can’t hear you, if that’s what you asked me, Mr Stevens.
Mr Stevens: Mr Stevens. It is, yes.
Sir Wyn Williams: I can hear you now.
New Speaker: Mr Stevens. If I may call Mr Meagher.
John Meagher
JOHN DOMINIC MEAGHER (affirmed).
Questioned by Mr Stevens
Mr Stevens: As you know, my name is Sam Stevens and I ask questions on behalf of the Inquiry. Please could I ask you to state your full name.
John Meagher: John Dominic Meagher.
Mr Stevens: Thank you for giving evidence to the Inquiry today. There should have been a bundle of documents in front of you. The start of which should be a witness statement dated 13 September 2022.
John Meagher: Yes.
Mr Stevens: Firstly, can I just ask you please to turn to page 18.
John Meagher: Yes.
Mr Stevens: You should see your signature there. Is that your signature?
John Meagher: That’s mine, yes.
Mr Stevens: Now, I understand if you now turn – sorry, if I could ask you to turn to page 5, paragraph 10 –
John Meagher: Yes.
Mr Stevens: – you in your statement speak about the Logistics Feeder Service, and I understand you want to clarify what you say in that paragraph.
John Meagher: Indeed. When I was first asked a question, I had more or less forgotten about the logistics feeder system which came in very much at the end, and I went – mentioned some things about the reference data system. So there is confusion in that statement based on that. So the logistics feeder system, I was asked about late stage changes to that system, and the answer is that I don’t know. I can’t remember what the reason for those changes were, but the logistics feeder system had been put together in a much more formalised way than reference data and was in no way compatible. It was a relatively straightforward system that had been well documented and, from my recollection, worked fairly well.
Mr Stevens: So are you distinguishing two systems, on the one hand reference data system and, on the other hand, the logistics feeder system?
John Meagher: Correct.
Mr Stevens: And you have no criticism of the logistics feeder system?
John Meagher: Correct.
Mr Stevens: Thank you. With that clarification, are the contents of your statement true to the best of your knowledge and belief?
John Meagher: They are.
Mr Stevens: That statement now stands as evidence in the Inquiry. I am going to ask you some questions about it but not all aspects of it. I’d like to start with your background.
You joined the Post Office in 1990 following a career in the oil and gas industry?
John Meagher: Correct.
Mr Stevens: Did you have experience of managing large IT projects before you moved to the Post Office?
John Meagher: Yes, I did. I’ve been involved in a number of – when I was in the oil industry, we’d been moving into the use of computer systems both in terms of planning, project management, and also the support services offshore. I’d also worked on the development of the FirstDirect Bank and on the privatisation of the central electricity generating board, which essentially was an IT project, because all of their previous systems had to be split up for the individual component companies that were created during that de-merger.
Mr Stevens: You applied to join the Horizon project in 1994 –
John Meagher: Roughly.
Mr Stevens: – around then, and you assisted with the procurement exercise that led to ICL Pathway being selected?
John Meagher: Yes, I first of all worked on the evaluation of the five potential suppliers, and then into the demonstrator phase where we finally chose ICL Pathway.
Mr Stevens: I want to move ahead a bit in the timeline, actually, and speak about you being the head of Horizon product assurance.
John Meagher: Yes.
Mr Stevens: In your witness statement you say that you started that role not long after the contract with ICL Pathway was signed.
John Meagher: Correct.
Mr Stevens: In your own words, what did you see your role to be as head of, sorry, Horizon product assurance?
John Meagher: I saw the role as one whereby we would ensure that the emerging solution was compliant with the requirements and we would assist ICL Pathway to that end.
Mr Stevens: When you say the requirements, do you mean the requirements that were drafted as part of the PFI tendering process?
John Meagher: Yes, the contracted requirements.
Mr Stevens: You weren’t involved in drafting those requirements?
John Meagher: I didn’t draft any requirements. I didn’t have specific business knowledge. I’d worked on a couple of projects in the Post Office. So I was aware of the roughly what they did but I was not someone who could have drafted a requirement.
Mr Stevens: Once they had been drafted, though, did you read those requirements?
John Meagher: Oh, yes, yes, I read them all.
Mr Stevens: So you could carry out your role, yes. You refer in your statement to a technical assurance team led by Jeremy Folkes?
John Meagher: Yes.
Mr Stevens: Did Mr Folkes report to you?
John Meagher: Well, at various stages we had sort of different reporting lines. They were never massively structured. I know that during the demonstrator phase he was responsible for the technical, the POCL technical team, and I was responsible for the POCL applications team. Subsequently, I remember it more as working together rather than there being a strict reporting line but it’s possible.
Mr Stevens: Just so we’re clear, how would you distinguish your roles? You say him on the technical side and you with the application side.
John Meagher: Well, on the application side we focused on the applications, EPOS automated payments, OVSC, et cetera, whereas the technical people looked that nuts and bolts, the network, the boxes and the plumbing.
Mr Stevens: Who did you report up to in your role?
John Meagher: Well, I remember – I mean, initially I think I reported to Andrew Stott who was in charge of the PDA up until the award. He was from the Benefits Agency. Then Peter Crahan headed up the PDA he was also from the Benefits Agency and then at some stage I reported to Dave Miller who was from the Post Office.
Mr Stevens: So you reported you say to Dave Miller. Did you ever have a direct report to Stuart Sweetman?
John Meagher: No.
Mr Stevens: In paragraph 12 of your statement you discussed the Initial Go Live pilot in 1996.
John Meagher: Yes.
Mr Stevens: You say that this:
“… distracted from the primary objective and wasted valuable time and resources.”
Do you consider this to have had a long-term impact on the programme, this diversion of resources?
John Meagher: Well, there was a lot of resource allocated to this, and to my recollection very, very little of what was put in place there was subsequently used as part of the operational system. It had to my mind it had a PR objective. It could very well have been successful in that area, but I thought it took away attention and resources from the thrust of developing the main solution. You know, whether it had a detrimental effect overall in the timescales, I couldn’t say. It definitely didn’t have a positive one.
Mr Stevens: You refer in your statement as well at paragraph 13 to a PinICL assessment team?
John Meagher: Yes.
Mr Stevens: Which you say:
“The purpose of the PinICL Impact Assessment Team was to judge if it was possible (i.e. was the business impact acceptable?) for fixes to known faults to be either deferred or temporarily managed through a ‘work-around’.”
John Meagher: Yes.
Mr Stevens: For how long did that team exist?
John Meagher: Well, the process existed right through. Exactly who managed it on the ongoing basis, I don’t know. But, I mean, there’s always such a process involved. I mean, it’s beneficial for the developer if they don’t have to fix every little problem regardless of its business impact. So that was put in place. I can’t remember when but I know – you’ll see references to it right through to the start of roll-out.
Mr Stevens: Were you involved with that team? The material you were assessing, were you actually seeing PinICLs themselves being handed to you?
John Meagher: Yes, yes, I didn’t personally, but people who worked for me and other people reviewed PinICLs, and that was the assessment they made based on the PinICL, whether the impact was such that needed to have a software fix, or whether it could be handled in a work around.
Mr Stevens: Were the PinICLs provided to Post Office or did anyone from Post Office have any access to the actual PinICL system itself?
John Meagher: I’m honestly not sure. We were – whether we were presented with printed PinICLs or whether we had access to the system I’m not sure. I don’t remember having direct access to the PinICL system. I think we were consulted on PinICLs as they emerged.
Mr Stevens: You mentioned it in your oral evidence and in your statement, this managing through workarounds. Was there a wider ethos in the programme to try to manage faults through workarounds rather than directly through fixes?
John Meagher: No. No, I mean, it’s a common practice. I mean, you know, if there’s – I don’t know – a misspelling on a script or something, the developer will say, “Look, can we defer that to later, because we need to get on with more important things.”
Mr Stevens: I’d like to now actually bring up a paragraph from your statement. The reference is WITN04150100 and it’s page 10, paragraph 19.
I think four lines up it starts saying:
“Early in the project the assurance team attempted to gain access to Pathway’s application design documents to (a) provide POCL confidence that the development was proceeding in accordance with the requirements and (b) to assist Pathway with early identification of any misunderstanding or errors they were making in their interpretations. Pathway did not co-operate with this request and due to the PFI nature of the contract we were unable to insist.”
How did you feel this lack of visibility of design documentation impacted on the assurance process?
John Meagher: It undermined it.
Mr Stevens: In what way?
John Meagher: It meant that we had less evidence upon which to perform our job.
Mr Stevens: At the time, what did you think was the reason for the lack of transparency?
John Meagher: Well, we had understood that – and I’ve got no evidence necessarily for this but this was our understanding – we’d been invited by Terry Austin to participate in what he called joint working, because of pressure of timescales.
We had a large team down in Feltham, about a dozen people, some of which were involved in the Benefits Agency application, some of which were involved in the POCL applications, providing clarifications to Pathway. We had a number of workshops with Pathway on the clarification of the applications, but we had nothing upon which to judge their interpretation of what was being said at these workshops.
What was your question again?
Mr Stevens: The question was, at the time, what did you think was the reason for the lack of transparency?
John Meagher: Well, I believe they didn’t have it. They didn’t have – we suspected that there was no written design.
Mr Stevens: We’re very much in the context of the PFI contract at this time. Do you think that the Post Office did all it could to obtain such documentation?
John Meagher: Well, I don’t know. I mean, I escalated this repeatedly within the PDA. You’ll see references to it through ‘98 and ‘99. In the end, or toward the back end, there’s an action on me to draft a letter for Dave Miller to give to Stuart Sweetman, but of course by that time it’s too late. You know, I don’t know to what extent the Post Office pressed Pathway hard. I witnessed more pressure from the Benefits Agency on Pathway than I did from the Post Office.
Mr Stevens: The Inquiry has heard evidence in the last few weeks that ICL Pathway created something called an EPOS task-force.
John Meagher: Mmm.
Mr Stevens: Have you been – have you heard about that in the hearings so far?
John Meagher: Yes.
Mr Stevens: That task-force was implemented to resolve, identify defects in the application the EPOS application between August and September 1998 which –
John Meagher: ‘98?
Mr Stevens: Yes ‘98 – which were being raised faster than they could be cleared. The task-force was unable to fix all those defects at that point, and it reported significant concerns about the quality of the EPOS application code.
John Meagher: Yes.
Mr Stevens: Were you aware that ICL Pathway had created such a group in ‘98?
John Meagher: No.
Mr Stevens: When was the first time you became aware of that?
John Meagher: When I heard it on – from this forum from one of the witnesses; I can’t remember which one.
Mr Stevens: The Inquiry has also heard evidence that employees within ICL Pathway made recommendations to rewrite or redesign the EPOS application in ‘98 and ‘99. Were you aware of those internal recommendations?
John Meagher: No.
Mr Stevens: If you had been aware either of the task-force or of the recommendations to rewrite the EPOS application, would that have changed your approach either to assurance or acceptance?
John Meagher: Well, I probably would have raised a high-level Acceptance Incident and it would have, I believe, given us more evidence upon which to press the further for a change of approach within Pathway to EPOS.
Mr Stevens: Could I ask for a document to be brought up, please. The reference is POL00028370. Could we turn to page 9. Thank you. Please could we focus on – sorry, one page up, I think. Sorry, could we have page 9 please. There we are. Thank you.
So this is a Programme Risk Status report. You see from the top right it’s dated 8 March 1999 and the first risk there says owner and your name. The description says:
“Due to lack of adequate visibility of the ICL Pathway design, and the lack of support from the contract to leverage the visibility, we have been unable to gain a high level of assurance in the adequacy or suitability of the service to support the POCL business. POCL therefore risks the implementation of a service in Live Trial and beyond, which will have negative operational impacts, resulting either in a level of service degradation or delay to the start of National Roll-Out.”
These are the concerns we’ve just been discussing. It’s apparent in your view that continued in March 1999?
John Meagher: Yes.
Mr Stevens: You state in your witness statement – and we’ve discussed it – that the lack of transparency was blamed on the PFI nature of the contracts. Did anything change once the contractual relationship shifted from PFI to a more traditional basis?
John Meagher: Nothing in this respect.
Mr Stevens: Do you know if – you may have already sought to answer this earlier but just so I’m clear I’ve put the question to you – do you know if the Post Office took steps to address the lack of transparency when changing from the PFI basis to the codified agreement in the summer of 1999?
John Meagher: I’ve no understanding of anything having happened.
Mr Stevens: Please could we turn the page on this document. I want to now look at the 0069 row. That says:
“Type B procedures in place for live trial, in particular Cash Account, are insufficiently robust to support consistency between RDP, Pathway and TIP.
“Risk that cash accounts will not balance, rejections wills occur on the Pathway and TIP interfaces and problems will occur on POCL back office systems.”
Do you recall what type B procedures were?
John Meagher: I’m afraid I don’t.
Mr Stevens: In summary, does this show that the Post Office were aware of a real risk that Horizon would fail to balance satisfactorily in February 1999?
John Meagher: All the indications were that that was going to happen.
Mr Stevens: How well known was that amongst the people working on the project at the time?
John Meagher: Everyone knew that.
Mr Stevens: Can you assist us in interpreting the Action in Mitigation column. Firstly it says:
“Ensure procedures are tested ‘end to end’ after completion of the E2E and MO …”
That’s end to end and model office testing?
John Meagher: Yes.
Mr Stevens: What was that first mitigation plan; can you recall?
John Meagher: Well, frequently the path into some of these test phases, particularly model office, is often through the procedures, because the model office test phase was designed not – it wasn’t simply a technical test, it was designed to test the system, but also the interaction between the system and the user and the procedures that the user had, okay?
So – which is what I suspect the type B may be, an earlier stage of approval of those procedures. I think that’s what’s being referred to here. Does that make sense?
Mr Stevens: It does. Thank you very much, yes.
That document can be taken down, thank you.
Could we bring up back up your witness statement it’s WITN04150100, page 16, please. Paragraph 29 four lines down you say – I should say for context you are referring to a question about external pressure, whether there was any external pressure to cancel, not to cancel the contract, and you say:
“Indeed, I recall a meeting chaired by Dave Miller some time in 1999, probably in the early part of the Dave’s tenure, when he canvassed views at his team meeting on whether we should persist with Horizon or cancel, and I voted to cancel as did my other staff member colleagues: Jeremy Folkes and Jan Topham. I recall there were others but their names now escape me.”
At that stage what were your reasons for wanting to cancel the project?
John Meagher: Well, I thought it was what is called a sunk-cost fallacy which is that you continue to pour effort into something that’s inevitably failing and, instead of being able to cut your losses, you end up losing everything. I felt we were on a death march, and I didn’t see it ending well. I had seen the Benefits Agency exit. I thought there were better ways to do this and definitely not the way we were doing it.
Mr Stevens: I think you used the words you were under a death march. Did you have any confidence in ICL Pathway’s ability to improve the situation to a level which would be satisfactory for acceptance at this stage?
John Meagher: Well, it was going to be difficult. I mean, one assumed that they’d get over the line in some way at some stage. The question was when and at what cost.
Mr Stevens: I want to turn to look at where we get to, because we know that the system is eventually accepted and there is a process of acceptance from this stage. So if I could firstly bring up document POL00028367 and page 2, please. This is a record of the fourth meeting of the acceptance board and you are described there as the Horizon acceptance manager chairing it. Could you just describe your role as the Horizon acceptance manager.
John Meagher: Okay. Well, the role of the Horizon acceptance manager was to first of all negotiate the Horizon process. So to obtain agreement from all of the different parties, to obtain agreement on the acceptance specifications, and then to ensure that that process was operating.
So it was a process responsibility rather than being responsible for the content of each of the acceptance specs.
Mr Stevens: Thank you. Could I turn to page 5, please, of this document which should be the Acceptance Board terms of reference. The second bullet point under Purpose, it says:
“To submit appropriate Acceptance recommendations to the managing director of POCL for endorsement.”
Was that your role, to report directly to Mr Sweetman at this stage, or was that someone else’s?
John Meagher: No, that would have been through Dave Miller.
Mr Stevens: Thank you. That document can be brought down now.
We’ve heard evidence that ICL Pathway and the Post Office subsequently entered the codified agreement on 28 July 1999. Were you involved in the negotiation of any of which contractual terms?
John Meagher: No.
Mr Stevens: In your statement you refer to disagreements between Pathway and the Post Office over how to classify the severity of acceptance issues, and the inquiry has heard evidence of Acceptance Incidents being classified as high, medium and low. Does that reflect your recollection?
John Meagher: Yes, indeed.
Mr Stevens: At the time what was your view of what a high severity incident was?
John Meagher: Well, I think we had a definition for it, and –
Mr Stevens: If it assists, the definition was defined as a failure to meet essential acceptance criteria.
John Meagher: Yes.
Mr Stevens: In sort lay terms of how you would interpret that, what would that be?
John Meagher: Well, I think there was a danger with acceptance that – number 1, that acceptance could be interpreted as being that the system was, that everything was rosy. You know, you could – in fact, I think – I’d have to defer to someone on the contract side, but I think, for instance, if we didn’t raise Acceptance Incidents during the allocated period, then acceptance was by default. So it was incumbent on us to raise Acceptance Incidents, and the severity was something that was frequently discussed.
A high Acceptance Incident would be something that would have an across-the-board impact on customer service, on the integrity of the data, those types of issues. So if the – in fact, I mean, I can give you no better definition really than, if you refer to the three high Acceptance Incidents that remained to the end, one focused on the ability of the users to understand the system, one focused on the stability of the technical solution, and the other one focused on the integrity of the data.
So those three Acceptance Incidents, which in a sense were portmanteau Acceptance Incidents because there were multiple reasons and causes for them, are a good way of defining or understanding what a high-level Acceptance Incident would be.
Mr Stevens: We will come to those in due course.
Could I please bring up your witness statement again. It’s WITN04150100 and page 13, paragraph 23. One thing you say is:
“Pathway would invite Post Office to accept that elements of an acceptance spec had passed or had exclusions, i.e. issues that were understood and did not present a significant business impact and could therefore be parked for later attention.”
When you refer to exclusions here, are you referring to the classification of the severity of the Acceptance Incident?
John Meagher: Yes.
Mr Stevens: And you go on to say that:
“There was always disagreement on this impact assessment because Post Office always had correctly reviewed the business and operation impact …”
Just pausing there, could you just explain the process by which Post Office would arrive at a classification for an Acceptance Incident.
John Meagher: Well, these incidents would frequently have been pre-allocated a severity by Pathway, a pathway, and then the Post Office would read those, understand them, discuss them, and come up with its own view.
Mr Stevens: Would you be involved in that process?
John Meagher: Yes.
Mr Stevens: Please could we bring up POL00043699. This is an email from that you sent to Keith Baines on 5 August, and the first paragraph says:
“Initial results from the exercise of identifying additional acceptance criteria to our existing acceptance incidents indicates that there is an opportunity for additional criteria to be added and therefore potentially additional acceptance incidents to be generated.”
Do you recall what the exercise you’re referring to here is?
John Meagher: I don’t exactly, but I think what I was probably thinking is that there was a risk that we wouldn’t see the wood for the trees; in other words, that the acceptance criteria by definition were broken down to a low level of detail and if, you know, one could pass or one could fail. If one failed, it may not be clear how that incident was connected to other features of the system.
We were trying to get a better understanding how these incidents were impacting in a broader sense.
Mr Stevens: If we carry on it says:
“Taking acceptance incident 211 – one where we expected additional criteria to be available – we have now matched the incident to five criteria. Whilst in theory this could therefore result in five incidents, the risk we run if we were to split the incident in this way is that we then reduce the severity of the incidents from a Medium to a Low.”
John Meagher: Yes, that’s what I was just discussing.
Mr Stevens: So following this exercise, was there any change of strategy as to how the Post Office would raise or categorise Acceptance Incidents?
John Meagher: Well, the – we initially did not have access, for instance, to the incidents that were being raised by Pathway from the help desk. We sought and did achieve access to those incidents. So, in reviewing incidents say from the help desk, you can look for patterns. In fact 298 is a good example of that, where you can – you know, whereas one incident in and of itself may seem inconsequential, if you can see patterns of similar incidents, you may be able to identify a more systemic problem.
Mr Stevens: If you could turn this is page, please, there should be a list of Acceptance Incidents, and it records both Pathway and the Post Office’s position before and after what’s described as a tranche 2 workshop. There are a number – the first two is an example – where Post Office’s initial severity rating before the workshop was high/medium, and it becomes a medium after the workshop.
Can you recall why it seems Post Office consistently after its workshop ranked the Acceptance Incident in the lowest or the medium severity?
John Meagher: Well, it’s possible that we understood it better, and also that Pathway will have brought forward rectification plans or better explanations that the problem could be contained.
Mr Stevens: And did you find these initial tranche workshops to be satisfactory in the sense of did you have a good working relationship with ICL Pathway during them?
John Meagher: Well, the working relationship was robust amongst most people on the PDA with Pathway. We’ve been on this a long time. But the workshops were constructive, you know – full and frank, as the phrase goes, exchange of views.
Pathway did treat acceptance very seriously for commercial reasons, and other reasons – I don’t want to – I don’t want to cast aspersions on their motives.
They treated acceptance very seriously. There was a lot of effort and immediate effort on Pathway’s part to bring forward rectification plans. John Dicks, I think, was very active. We’d known him from the early stage of the project, and he brought forward a lot of rectification plans to show that they understood it, could deal with it.
Mr Stevens: We will come to look at some of those in relation to the three Acceptance Incidents you have described. Before we do that, could I please ask for POL00028357 to be brought up and page 16, please. This is an Acceptance Incident form, this one for 298, one of the high-severity incidents we discussed.
Do you recall using these forms, the pro forma form?
John Meagher: I don’t recall it but I recognise it.
Mr Stevens: Do you recall who had access to the information within these forms?
John Meagher: I don’t remember anything around the distribution. I mean, I think they were fairly commonly available.
Mr Stevens: This one concerns the screen lock-ups and screen freezes?
John Meagher: Yes.
Mr Stevens: The dates it says first observed is 1 July 1999. Would you have been made aware of this around that time?
John Meagher: Yes.
Mr Stevens: Who would you have reported that issue to?
John Meagher: Dave Miller.
Mr Stevens: Please can we turn to page 57 of the document. This is for 376, which I know the Inquiry has heard evidence on, and I believe you referred to it earlier. It is issue of data integrity.
John Meagher: Yes.
Mr Stevens: You recall this Acceptance Incidents?
John Meagher: Oh, yes.
Mr Stevens: At the top again it says the date this was observed was 19 July 1999. Again would you have been aware of this around that point?
John Meagher: I would have, yes.
Mr Stevens: Again, would you have reported that to David Miller?
John Meagher: Yes. I mean, these types of incidents would have been well known by most people on the programme.
Mr Stevens: When this one arose, 376, did you think – was it immediately obvious that this was a serious concern?
John Meagher: Absolutely.
Mr Stevens: If you could bring that document down, please, and if you could bring up POL00028332, page 2, please. This, you see, is a Horizon programme management resolution meeting. You are listed to be in attendance. Do you have any recollection of it? It’s on 12 August 1999.
John Meagher: No.
Mr Stevens: Please could we just pull the screen down a little bit. There should be a list of – there we go, thank you. It shows a list of Acceptance Incidents was discussed, including the ones you referred to, 376, 218 for training, and 298. Please could we turn the page. Sorry could we go to the next page under 376. My apologies.
No, sorry, back a page. Sorry, my apologies. And can we go back a page again, sorry. There we are thank you very much. That’s great, thank you.
So under 376, JD is that John Dicks?
John Meagher: I’m not sure. I can’t remember who the attendees were.
Mr Stevens: Can we go back to the attendee list, please, on the second page.
John Meagher: Yes, that will be John Dicks, yes.
Mr Stevens: Then back on to the following page, please, at the bottom:
“Pathway recognise that not all transactions have been harvested and sent to TIP. A provisional fix went in on 2 August and this has worked satisfactorily so far with the effect that all records had been sent. A root-cause analysis has been developed, identifying 8 contributory problems, and all but one has been diagnosed and tested in Pathway to date. Pathway cannot guarantee, however, that all problems have been trapped. They will need to see evidence from the fix of the 8 known problems, and will continue to monitor the problem for three months to be confident of its resolution.
“The provisional fix and the control procedures developed allow Pathway to identify any errors, to patch the file, and to notify TIP in advance. Since implementation there have been no errors to report and hence Pathway contend that the action taken to date and the result they have observed justify the downgrading of this incident.”
If we could go over the page, please – that’s fine, thank you – the second paragraph down says:
“Pathway believe they have made sufficient progress to rate as low. POCL believe they must have the opportunity to verify the fix.
“Pathway pointed to the fact that no new errors have occurred since 2 August. POCL believe they cannot downrate until their own observation can be performed given the risk to the accounts. They recognise the corrective action that has been taken, but the potential size of errors that can occur is too serious to risk proceeding without proof, which will not be available until 30 August. Advice from the POCL external auditors was that without clear verification of the success of the solution there would be implications on the qualification of Post Office Limited accounts.”
So is it a fair summary at this point that Pathway have said, “We’ve got a resolution or rectification plan and wanted a downgrade”, but Post Office were saying, “Well, let’s see that proved”?
John Meagher: Yes.
Mr Stevens: At this stage, what, if any, discussion was there internally in Post Office about what would constitute a successful fix of this? What level of errors would represent a successful fix?
John Meagher: Well, there was agreement, I think, between the two acceptance meetings – so, on the first acceptance meeting, Pathway did not achieve acceptance. I think that was in August, and then they had a cure period or whatever it was called, and then there was a further meeting in September.
Mr Stevens: Yes, and we’ll come to that. Sorry, just to be clear, my question is at this stage in August –
John Meagher: I don’t think there was a metric suggested. I think it was developed subsequently between the two phases.
Mr Stevens: Could we turn to page 5, please, of this document. This concerns a system stability issue. At the bottom – sorry, if we can move down, thank you – it says:
“BMc [will be I imagine Bruce McNiven] and RH [Ruth Holleran] emphasised the business impact on staff time and cost, and on customers, of loss of service. This was particularly severe in one or two position offices. Fuller impact details would be ready for Monday. The frequency is significantly greater than on the current POCL legacy systems.
“JM [which I think will be you] said that the incident was wider than reboots – it was the overall loss of service to the customer, and this had to be taken into account in the severity.”
So in broad terms what was the business impact of the screen freezes?
John Meagher: Well, the outlet, the office, was unable to conduct business, to transact business, number 1, and, secondly, it was by no means clear what impact this was having on the integrity of the data.
Mr Stevens: Why were you concerned as to the integrity of the data because of the screen freezes?
John Meagher: Well, because we weren’t sure what was happening. It was unclear why it was freezing. It could lead to reboots by the users in the middle of a process, and we weren’t sure what the implications would be on the data from that. But the main impact was the fact that the outlet was – you know, it probably took – I can’t remember how long it took to reboot a system, but I don’t think it’s like rebooting your phone now. It took quite a while. So in the meantime customers are queueing up.
Mr Stevens: We know, as you referred to already, that – that document can be taken down, thank you – acceptance wasn’t achieved in August, and this led to what has been referred to as the first supplemental agreement between ICL Pathway and Post Office and that proposed a series of acceptance workshops.
John Meagher: Yes.
Mr Stevens: I want to turn to look at those now. Please could I bring up POL00028342. This is an email which you sent to a number of people at Post Office involved in the acceptance workshops, dated 26 August 1999. If I could turn the page, please, it’s the note that’s attached. This describes acceptance workshop roles. There were three roles: the champion; the business expert; and the technical expert. That’s right?
John Meagher: Yes.
Mr Stevens: The champion is referred to in the third paragraph, and the last sentence says:
“Whilst it is Pathway’s responsibility to propose a rectification plan, the champion will be required to have defined in advance the success criteria for the incident.”
Now, would the champion be solely responsible for that, or would they do it in conjunction with others?
John Meagher: Oh, no, they’d do it in conjunction with others.
Mr Stevens: Who would be involved in that joint decision-making process?
John Meagher: I think what they would call the business assurance people, people like Ruth Holleran, people who were not part of the PDA per se but were closely aligned to us within what Post Office called the business.
Mr Stevens: It may help actually could we go to the bottom of this page, please. So, for example, at 376 the business expert is Graeme Seedall, the technical expert is Calum Craig, and the champion is Ruth Holleran.
John Meagher: Yes.
Mr Stevens: Ruth Holleran as champion would be, it said, the responsibility to define in advance success criteria.
John Meagher: Yes, yes.
Mr Stevens: In doing that, would she consult with the business expert and technical expert within, say, assigned to 376, or would it be wider to everyone involved in the acceptance process?
John Meagher: Well, I don’t know, to be honest. Ruth would definitely not have come up with something on her own. She would have perhaps proposed some measures and then circulated those and discussed them. She definitely would have discussed them with Graeme Seedall and probably with other people back in Chesterfield, or Dave Smith as well.
Mr Stevens: To what extent would David Miller have been involved in this process?
John Meagher: I mean, Dave would have been aware of it. I don’t think he would have been down into the detail of it.
Mr Stevens: We can take that document down, please. We’ll just deal with this last document, sir, if I may before a break.
You were at some acceptance workshops on 25 and 26 August. Can I bring up POL00028476, which is a note of an internal acceptance update meeting or action points from it and, if we could go to point 11, please. I should say this is 27 August. Point 11 says:
“Speak to Ruth Holleran about producing a clearance plan for 376 and ensuring fast-track monitoring/reporting process.”
The Action By is listed, JM, which I assume will be you. Do you recall what this action point was or what it involved?
John Meagher: Well, I think Ruth for some reason wasn’t at that meeting – I’m not sure why – and I think we were looking to try and help Pathway to understand what level of acceptance might be possible, or what mitigations might be possible for these incidents, in general. You know, what would be acceptable, and I think the output from this actually was that 0.6 metric.
Mr Stevens: Which we’ll come to in a moment. Because of this, were you involved in the discussions? Were you directly involved in the discussions on or the decision-making which led to 0.6 per cent being –
John Meagher: No, I wasn’t involved in the development of that number. That would have been more of an internal POCL discussion.
Mr Stevens: When you say internal POCL discussion, who do you mean specifically?
John Meagher: I mean Ruth, Dave Smith, possibly others.
Mr Stevens: Why wouldn’t you have been. involved with that?
John Meagher: Well, because I wasn’t representing the business in that sense. I was representing the project.
Mr Stevens: Sir, thank you. I think that would be a good time to take a break.
Sir Wyn Williams: What time shall we start again?
Mr Stevens: 25 past, sir?
Sir Wyn Williams: Yes, that’s fine.
(2.15 pm)
Sir Wyn Williams: (A short break).
(2.25 pm)
Mr Stevens: Sir, can you see and hear me?
Sir Wyn Williams: Yes, I can.
Mr Stevens: Can I start by bringing up POL00028471, page 2, please. Thank you. This is an email from Ruth Holleran, known as Ruth Read, to Keith Baines dated 27 August 1999. She says:
“Sooner or later (probably sooner having got 218 [training Acceptance Incident] out of the way early next week) Pathway will be seeking reclassification of 376 and 298 to medium on the strength of a rectification plan. I suggest we need to be thinking … about the tactics on this before Pathway (certainly) raise this at the next workshops.
“I am assuming with both 298 and 376, the required rectification to downscale to medium will not be proven, but we will be under extreme pressure to reclassify. Bluntly speaking we will be breaking our proof positive rule if/when do this but I’m not sure we will have any alternative – or do we?”
Let’s just take that in stages. Firstly, what does Ruth Holleran mean here when saying “breaking our proof positive rule”?
John Meagher: Well, I don’t know, but I think she possibly means that we would be reclassifying on the basis of a plan as opposed to evidence.
Mr Stevens: And she refers to, “We will be under extreme pressure to reclassify.” At the time, where was the pressure coming from, if any, to reclassify Acceptance Incidents?
John Meagher: Pathway. I don’t think there was pressure coming from within POCL.
Mr Stevens: When she says as well:
“I’m not sure we will have any alternative – or do we”, was the alternative simply not to accept the product?
John Meagher: Well, I think what Ruth is seeking is some way through this. I saw another document – I don’t know, maybe in this pack here, I mean, attached here – from Dave Smith who’s recommending that we not down-grade in advance of the decision on the basis of plans alone, but that we retain the classification and hold our options open. If the business viewed it as wanting to proceed but retain the classification, then it would do so.
So my suspicion is that Ruth is looking for some level of additional flexibility here.
Mr Stevens: I didn’t want to interrupt your answer. I think, if we turn to page 1 – the bottom of page 1 of this document, maybe what you were referring to, is it this, from Keith Baines to Ruth Holleran, second paragraph:
“I would be against any artificial reclassification of Incidents from High to Medium”?
John Meagher: Yes.
Mr Stevens: So in short, I think, summarising what you said before, your evidence is that there was no internal pressure to artificially reclassify?
John Meagher: None.
Mr Stevens: If we can go to the top of page 1, please, there’s an email there from David Smith to Keith Baines on 1 September, and it says:
“I would hope that the terms included financials – Pathway indemnifying us for any unexplained balances, would present them with a considerable incentive to get things fixed and/or withholding part of the £68m due on Acceptance.”
Do you know if at this stage there was any consideration being given to requiring ICL Pathway to indemnify Post Office for errors in balancing caused by the system?
John Meagher: I believe there was discussion; I wasn’t party to them. I don’t know what the outcome was, but I believe there was discussion on that topic.
Mr Stevens: Now, I think I’m not going to take you to the document. That document can be taken down, thank you, because you already referred to 0.6 per cent error rate in respect of cash account discrepancies, and your evidence before the break was that you weren’t involved in that decision-making process.
John Meagher: Correct.
Mr Stevens: Whilst you may not have been involved in the decision-making process, do you have any knowledge of the reasons that were considered when arriving at that 0.6 per cent figure?
John Meagher: I believe it was a measure of what the Post Office could manage in terms of corrective activity.
Mr Stevens: And that’s – so I’m not sure if you heard the evidence of Stuart Sweetman earlier in the Inquiry.
John Meagher: I did hear it. I don’t recall.
Mr Stevens: One of the things he said was this idea of what is a material error rate.
John Meagher: Oh, yes.
Mr Stevens: Material for the Post Office was different to what would be material for an individual subpostmaster affected by a discrepancy. Would you agree with that?
John Meagher: Well, I’d agree with that in broad terms, yes, but my understanding always was that when it came to financial errors in transactions that had to be subsequently reconciled with third parties, then there was really no acceptable level of error. So, for instance, in the bill payment business, the APS application, people paid their bills through post offices, and a lot of the people who paid their bills through post offices were people who couldn’t afford to pay their bills through direct debit.
So these people go in and pay their electricity bill – they were mainly utility bills but also local housing bills as well. Those transactions, we would be looking for zero errors on those types of transactions.
Mr Stevens: Why do you think there’s a difference in approach between the transactions you just referred to and the cash account discrepancies in respect of the error rate?
John Meagher: Because I think the cash account discrepancy would be corrected – it’s between two people who have an existing relationship – whereas, if someone on housing benefit goes in and pays one of their bills and that transaction’s lost, then that person risks having that service turned off. I think there’s a greater degree of jeopardy involved with the people who are involved.
Mr Stevens: During this acceptance process, was anyone on the acceptance team within the Post Office considering or representing the interests of individual subpostmasters in respect of balancing?
John Meagher: Well, I’m not sure balancing was considered a function that was solely for subpostmasters. All post offices had to balance. The whole balancing process was being looked at by many people in the Post Office.
Mr Stevens: Although you weren’t involved in the decision-making, what was your view – well, with hindsight, what’s your view, sorry, on the 0.6 per cent error rates? Do you think it’s satisfactory or should it have been lower?
John Meagher: Well, I wouldn’t want to offer a view, really, because I don’t really understand the implications of those errors. I don’t understand the materiality of – you know, someone has come up with a figure. They will have come up with that figure with more knowledge than I have, or even had then. So I really wouldn’t want to comment on it.
I mean, I stick by what I said with regard to I think it depends on the types of transactions. I think that is significant.
Mr Stevens: The Inquiry has heard evidence that one of the Post Office’s requirements was that Pathway could produce data from Horizon that would be used to support prosecutions. Were you aware of that as a contractual requirement at the time?
John Meagher: I wasn’t conscious of it at the time. I’ve since had a document released to me that quotes that requirement.
Mr Stevens: You said earlier in your evidence that you would have read the requirements –
John Meagher: Yes.
Mr Stevens: – in the contract?
John Meagher: Absolutely, yes.
Mr Stevens: At the time of acceptance, was anyone in the acceptance team considering the need for Horizon to produce data that could be used in prosecutions when dealing with these Acceptance Incidents?
John Meagher: I don’t know what happened to that requirement. That requirement would have found its way into an acceptance spec, and that spec would have had criteria against it. How that was subsequently satisfied, I don’t know.
Mr Stevens: Could we please bring up POL00028465 and if we could go to the second page, please. Thank you. This is a letter from David Miller to Stuart Sweetman on 8 September.
John Meagher: Yes.
Mr Stevens: If we could go down a bit further in the page, please, to show the heading System – that’s it, thank you – System Lock-ups and Screen Freezes Requiring Reboots. So this is an update on Acceptance Incidents incident 298, and the second line of that paragraph says:
“The problem is being contained but it isn’t yet being improved and John Meagher thinks it will take months to get right.”
Does that fairly reflect your view at the time?
John Meagher: Yes, it does.
Mr Stevens: Were you confident at that point that the issue would be fixed?
John Meagher: I was confident that it would be reduced. This is one of these situations where there are multiple reasons, not all of which were understood by Pathway. So I was confident it would be reduced as they understood what was happening, but wouldn’t say I was confident it would be removed.
Mr Stevens: What gave you confidence that it would even be reduced?
John Meagher: Well, because they had identified a couple of reasons for it and were trying to engineer those out of the system. Some of them were behavioural, some of them were to do with the system, some of them were classic Microsoft problems that you could identify, blue screens, et cetera. So, you know, the more evidence that Pathway were able to gather, you know, the greater the opportunity they’d have to make corrective action.
Mr Stevens: Please can we turn the page and have the summary section in view. Thank you. It says:
“Of our six key players (Keith Baines, Ruth Holleran, John Meagher, Bruce McNiven, David Smith, Andy Radka) the first four would opt (somewhat reluctantly) for conditional acceptance toward the end of September. Andy Radka and David Smith would not accept and seek to use the full period until 15 November to force improved performance from ICL Pathway.”
Do you agree with that? At that stage were you siding towards conditional acceptance?
John Meagher: I was, yes.
Mr Stevens: We don’t need to turn it up but, in your witness statement at paragraph 26, you say this about acceptance:
“As far as I recall, and there was a contractual agreement, Pathway would be awarded acceptance once it had completed the agreed tests and a threshold regarding the number of Acceptance Incidents within severity classifications had not been breached. This was achieved albeit with many acceptance issues remaining which Pathway undertook to resolve.”
At this stage, did you consider that Horizon had satisfied the criteria for acceptance?
John Meagher: No.
Mr Stevens: Why?
John Meagher: Well, it hadn’t – there were Acceptance Incidents above the threshold.
Mr Stevens: And the rectification plans in place you weren’t sufficiently confident with?
John Meagher: Well, we needed to see – some of those rectification plans we didn’t fully understand. The process for the data integrity check, we hadn’t seen the design, et cetera. I mean, it’s … I forget where you are going on this.
Mr Stevens: If you didn’t think that it satisfied the criteria for acceptance, why did you think that conditional acceptance was appropriate?
John Meagher: Because I thought it was inevitable. I mean, I think the Post Office had already thrown in the towel by increasing the threshold in one of the previous contract negotiations, and that it was going to happen.
Mr Stevens: So you –
John Meagher: I didn’t think acceptance would stop Horizon going live.
Mr Stevens: One of the reasons you have given is the lowering of the thresholds for acceptance.
John Meagher: Yes.
Mr Stevens: Was there anything else, any other communications within Post Office, which made you think that Horizon was going to go ahead even if it didn’t meet the accepted criteria?
John Meagher: No, no, there was nothing. I wasn’t involved really in discussions within the – in levels above Dave Miller.
Mr Stevens: What did you consider of the robustness of the system at this point?
John Meagher: I thought it was delicate.
Mr Stevens: Could you expand on “delicate”. Well, did you consider it to be robust?
John Meagher: No, but I wouldn’t have considered anything robust at that stage anyway because I think, you know, robustness is something that would have to be proven over a longer period of time, and it was a small number of outlets that were out there, et cetera. I thought robustness is something that would have to be built up. I didn’t think it was robust but I wouldn’t have – I wouldn’t have – it would be premature on anything at that stage.
Mr Stevens: When we’re speaking here of conditional acceptance, at that point was there an understanding of what those conditions would be?
John Meagher: Well, there would be the conditions around the rectification plans, the ability to – plus the ability to stall roll-out. They are the two main features.
Mr Stevens: What was your view in the sense of, if conditional acceptance went ahead, did you think it would be more difficult later on to stop the roll-out or to pause it or slow it down?
John Meagher: No. I didn’t, no, no, because I thought that the evidence on the ground, if necessary, would stall the roll-out. If we got to a position of where it wasn’t manageable either in terms of the volume of errors or the problems that the users were having, then we would just have to stop regardless of what’s in the contract.
Mr Stevens: So we heard earlier your decision to vote to cancel the project earlier in 1999, and now we’re at a point of conditional acceptance. You said the reason for conditional acceptance was you thought it was going to happen anyway.
John Meagher: Mmm.
Mr Stevens: Was there any other reason to explain your change in position?
John Meagher: Well, I think I need to distinguish in my mind between acceptance and the contractual event of acceptance and the progression of the project, of the roll-out. So I felt that we needed to continue to stress the system, and I thought that the only way that was clear that that was going to happen was some level of conditional acceptance.
Mr Stevens: Please can we move down to 3.4 on this letter. Thank you.
This is David Miller’s tactical advice to Stuart Sweetman, and it says:
“It is likely that ICL Pathway may play Peter Copping in for a decision on the 3 high level incidents as of today. If so, there is nothing we can do, and I expect he will give his view within 48 hours anyway. This will give us a clear steer on where he is coming from so we can stop the second guessing his position.”
Was there any reluctance in the Post Office to refer this incident to Mr Copping at this stage?
John Meagher: I never remember Mr Copping. I believe he attended some of the acceptance workshops. He kept his own counsel. I’ve no recollection of him.
Mr Stevens: I want to move on to the topic of reference data, and please could I bring up POL00028564. This is an email from you to a number of people at the Post Office. If we could go further down, please, in the paragraph above the second set of bullet points, it refers to a meeting you had with Steve Muchow, Mike Coombs and John Dicks, and it refers to various issues relating to reference data.
Before we consider those, can I ask who at Post Office was responsible for managing reference data in relation to Horizon.
John Meagher: Okay, there was a project, a live project. The individual whose name I remember is Geoff Derby.
Mr Stevens: Geoff Derby. When did that project start, the live project?
John Meagher: Well, there was always a reference data maintained in the Post Office for some of the earlier projects, et cetera. I mean, reference data – you know what reference data is by now presumably.
Mr Stevens: Yes, but if you would like to give your explanation of it.
John Meagher: So there was lots of different types of reference data. I mean, there’s reference data that identified the outlets themselves, that was called a FAD code. Each outlet had a code, where – its location, the manager, et cetera. So there was data around that. Then there was product data in terms of pricing and how that product data mapped to the cash account.
Then there was local reference data, and I’m afraid I can’t explain how that was managed, but there was – different regions would sometimes have separate customers in the Post Office, local bus companies and things like that. So there was a degree of regional reference data as well.
Now, reference data was continually mis – not given an adequate level of seriousness by Pathway through the whole project, and we were scampering around at the end to try and make it work. But really the whole integration of reference data and EPOS within Pathway should have been the subject of a wholesale business process analysis and it wasn’t.
I’m not saying – I know a lot of people have been critical of reference data, a lot of Pathway people have been critical of reference data, and they were right to be. There were errors that came from that data into Pathway as well.
Mr Stevens: Do you recall why it was being raised at this stage which was October 1999? You said you thought it should have been raised before. Why was it being raised at this point?
John Meagher: Well, it was a serious concern. It was contributing to problems, and also Pathway appeared to have been coming to a realisation that they may not be able to manage the volume or the volume of change that they were receiving from Post Office.
Mr Stevens: So the concerns that they’ve listed there, is first is:
“Pathway are concerned with POCL not maintaining the agreed lead times between receipt of data, (notification of change) and activation of data (implementation of change).”
What was broadly that concern about and how did it affect Horizon?
John Meagher: I’m not 100 per cent sure, but I suspect that what Pathway is saying that we had an agreement whereby we would pass reference data changes to them for activation at a time in the future, and we may have been – the Post Office may have been passing data with a shorter period for activation than had been agreed in the interface spec.
Mr Stevens: From the way you answered that question, do you not independently recall that –
John Meagher: I don’t.
Mr Stevens: The second one is:
“Pathway are waiting for a Reference Data business rules document which is in development by Geoff Derby and would welcome the earliest delivery of the document.”
Do you have any recollection of that issue?
John Meagher: I don’t, I’m sorry.
Mr Stevens: Then thirdly:
“However, the most significant issue for Pathway is that the current design (agreed by all) for the provision of data changes from Post Office to Pathway results in the delivery of large volumes of data which contain no actual change for Pathway.”
John Meagher: Yes.
Mr Stevens: You referred to that issue earlier in your answer, about they didn’t have the system in place to deal with the volumes of data.
John Meagher: Well, I think there’s two things there. One is that it would appear that Post Office is passing data to Pathway and that Pathway – which there is no need for Pathway to act upon, and I think there’s also an issue where the volume of change, the volume of data is greater than what Pathway anticipated. This should have been agreed, you know, way back.
Mr Stevens: Why wasn’t it?
John Meagher: I don’t know why it wasn’t, but it wasn’t.
Mr Stevens: Who was responsible –
John Meagher: Both sides. I mean, Geoff Derby on the reference data project, and whoever his opposite number was in Pathway.
Mr Stevens: Even beforehand, when say ‘98 early parts of ‘99, when this is being developed, it should have been Geoff Derby and his counterpart at ICL?
John Meagher: Yes. This should never be arising at this stage.
Mr Stevens: When this was raised with you, do you recall what your sort of action plan was from thereon?
John Meagher: We had – I don’t remember, but I know there was an extraordinary amount of activity ongoing. There were – I think there were joint working teams, joint teams working together to try and iron out how we could best sort this out.
Mr Stevens: On that point then, please, could we turn to POL00028552. This is a document entitled Terms of Reference at the top, but the title below says Horizon Reference Data Review, and the author is you. The authority is then Andy Radka who is at Post Office; is that right?
John Meagher: Yes.
Mr Stevens: And Steve Muchow. So it’s a joint review.
John Meagher: Yes Steve Muchow is from Pathway.
Mr Stevens: Do you recall the process of drafting these terms of reference?
John Meagher: I don’t recall it.
Mr Stevens: Do you remember what your role was in relation to this joint review?
John Meagher: I was trying to get it set up.
Mr Stevens: Did you have any involvement in the technical aspects of how the reference data was changed or –
John Meagher: No, no.
Mr Stevens: Please can we turn to page 4 of this document. The second paragraph says:
“Previous discussions between both companies had been hampered by reluctance to provide full visibility of all aspects of their respective technical and operational design. It is essential, therefore, for the success of the study that sufficient detailed information is provided to ensure the maximum level of knowledge and inform the development of appropriate solutions. To this end, it is proposed that workshops are supported by the attendance of appropriate members of both the PONU …”
Just stopping there, what does that stand for?
John Meagher: Something in the Post Office.
Mr Stevens: “… and the Pathway Reference Data and Business Process Design teams and that appropriate documentation, if required, is made available.”
Would that have been reported to you? In drafting these terms of reference, would that have been reported to you as a problem that had happened, or is that something you had first-hand knowledge of?
John Meagher: That’s something I was aware of. I mean, this goes back to the point mentioned before about the EPOS design. So EPOS was intrinsically connected to reference data. All of the mapping of the cash account, between the cash account and the transactions, was in reference data. So, although we knew what data that we were providing to Pathway, we did not know how it was being used within their system, and that is what we’re trying to – that’s what I’m trying to have made available here, that that would be exposed.
Mr Stevens: Do you think this issue would have been identified earlier or addressed earlier, if there had been better sharing of design documentation at an earlier stage?
John Meagher: Absolutely. I mean, there’s another document somewhere in the pack where I had approved another attempt at reference data, at a joint reference data review back in ‘98, and I can’t remember what happened to it, but it was drafted by Andy Scott and it was copied to the Horizon management team, to the PDA management team, and copied to Terry Austin. I don’t know what happened to that initiative but I believe it didn’t happen. We made attempts to try and get this sorted out before.
Mr Stevens: I wanted to go to – I appreciate you say you weren’t involved in the technical aspects of it, but you may be able to assist in interpreting one document. Could I ask, please, to turn to POL00028544. This is the second meeting of a reference data review which clearly relates to this joint review, of which you were the chair.
In chairing the meeting, was your role to facilitate?
John Meagher: Yes.
Mr Stevens: Could we please turn to page 6. If we go down slightly to sub-paragraph 4, there’s a note in handwriting on the right there. Do you recognise that handwriting?
John Meagher: No, I don’t. It’s not mine.
Mr Stevens: Do you have any recollection of this meeting at all?
John Meagher: I’m afraid I don’t.
Mr Stevens: That document can be brought down, please.
So you were involved in the review. You left the Post Office in January 2000. Do you have any knowledge of how the review finished or the outcome of the review?
John Meagher: My recollection is that, following the second acceptance agreement, the Post Office pursued those rectification plans and any other problems that were arising. So I think the role that I’d had reverted into what we would call business as usual or a project team within POCL. I think Min Burdett possibly took over what I had been doing.
Mr Stevens: You weren’t there to see the end of the reference data review?
John Meagher: No, no. I mean, I gave in my notice between Christmas ‘99 and New Year ‘00 and I probably – I don’t think I spent a lot of time – I had outstanding leave, et cetera, so I would have been just involved in handing over.
Mr Stevens: Please could I bring up POL00028553. This is an email to you from Min Burdett who I think you just referred to earlier saying you would be handing over to; is that right?
John Meagher: Yes, I think so. I mean, I’ve seen documents subsequently that she – similar areas to where I would have been involved in.
Mr Stevens: Please can we turn to page 3, which is one of the attachments. This is a roll-out decision demand position paper.
John Meagher: Yes.
Mr Stevens: Do you recall this document at all or what its purpose was?
John Meagher: I think I remember it from the pack; I don’t remember it at the time.
Mr Stevens: It sets at the top:
“In anticipation of the criteria due to be met by 24 November not having been met by that date, and given POCL’s associated concerns, POCL is proposing that the existing monitoring activities should continue and be strengthened, and is also proposing the introduction of some new activities. POCL proposes the following.”
I want to look under Data Integrity. It says “Objectives”. The first one is:
“To ensure that the integrity control is capable of detecting all relevant incidents.”
Do you know to what that’s referring?
John Meagher: Yes. So Pathway had proposed initially and developed and introduced a piece of software that would perform a comparison – I can’t remember exactly between what two points – but it would perform a comparison to provide an early identification of data differences between the cash account and what was being received at TIP.
Mr Stevens: This was to identify the discrepancies basically?
John Meagher: Yes.
Mr Stevens: Under Activities, the first bullet point says:
“The EPOS TIP reconciliation feature must be proven to be capable of detecting all data errors that would be detected by the TIP cash account compare process. For the avoidance of doubt, this includes an explicit check that receipts are equal to payments …”
It goes on to say how that’s to be achieved.
So in essence is Post Office’s position at this point that the reconciliation tool needed to pick up all data errors or all discrepancies?
John Meagher: That would always be the target.
Mr Stevens: Presumably that was an important point for Post Office.
John Meagher: Absolutely.
Mr Stevens: The Inquiry has heard evidence about the third supplemental agreement, and that the reconciliation tool was not guaranteed to pick up all cash account discrepancies. Are you aware why that position was changed from this position being the reconciliation feature needed to pick up everything?
John Meagher: Well, I think that’s the target. I think this is the target you are reading here.
Mr Stevens: But are you aware as to why that couldn’t be achieved?
John Meagher: I – I don’t quite understand your question.
Mr Stevens: Well, the Post Office’s target is there to be a reconciliation feature that is capable of identifying all discrepancies.
John Meagher: Yes.
Mr Stevens: That wasn’t what was agreed in due course. It was accepted that the reconciliation feature could not identify all of the data errors.
John Meagher: Right.
Mr Stevens: Are you aware as to why the Post Office accepted that position rather than push for its target?
John Meagher: Well, I wasn’t party to the agreement to the 0.6. I can only assume that it was a level of – a level that they thought they could manage. I don’t think the Post Office would ever accept a situation whereby it expected to get errors. It would always be striving to drive those out, to drive those down. But the reality of the situation is that that was going to happen, and the reality that we’re coming from is that there was a considerable number of errors. So we must never – there were probably created by multiple causes. So it was never going to be a situation that we would go from whatever level to zero in one bound.
Mr Stevens: Finally, can I please turn to your witness statement again. It’s WITN04150100. Can we bring up page 18, please. Thank you.
You say here that:
“Acceptance was a prescribed contractual event; it served its purpose, but it did not mean that the solution was ‘reliable and robust’. From my experience care and serious attention would need to continue to have been applied to the system as it moved through roll-out and beyond. Years later, when I learned that Post Office had prosecuted multiple subpostmasters based on assurances from Fujitsu that no explanation other than fraud was possible, I was shocked. Shocked because among other things Post Office would appear to have completely changed its view of Horizon since the time up to 2000 while I worked on the project.”
That point, that Post Office would appear to have completely changed its view since you worked on the project, please could you expand on what you considered the general Post Office view of Horizon to be at the point you left the project.
John Meagher: Well, I thought it was viewed as delicate. That’s how I viewed it. No-one could have continued in that period after I left and not been involved in multiple, multiple rectification activity. It did not present itself as a stable system. You know, there may have been three high-level Acceptance Incidents but there were multiple, multiple causes for those.
Mr Stevens: So, if someone had said to you in 2000 that the Post Office planned to investigate subpostmasters for fraud and false accounting, based on data produced from Horizon, what would you have said?
John Meagher: Well, I don’t even believe they did base it on data from Horizon. They based it on assurances from Fujitsu staff that no other explanation was possible, which would have shocked me even more.
Mr Stevens: Just finally, when you said Post Office would appear to have completely changed its view, are you referring to specific persons or just generally across the organisation?
John Meagher: I’m talking about everyone I worked with from Dave Miller down to Ruth Holleran, et cetera, on the Chesterfield site, everyone on the programme, everyone who was involved at a working level.
Mr Stevens: I have no further questions. Sir, I believe there are some questions from recognised legal representatives.
Sir Wyn Williams: Before they ask them, can I just pursue what Mr Stevens has been asking you about. My understanding is that your evidence is that everyone with whom you were engaged on a day-to-day basis working in Horizon may not have used the word “delicate” but essentially had the same view of Horizon as you did at the time you left. Have I understood that correctly?
John Meagher: That’s exactly what I said.
Sir Wyn Williams: And that’s from Mr Dave Miller down, as I understand it.
John Meagher: Yes.
Sir Wyn Williams: Do you have any information you can give me about any views expressed from Mr Dave Miller upwards?
John Meagher: I can’t. We never really – I mean, no.
Sir Wyn Williams: What I have in mind is that sometimes – I’m not saying it happened; I’m just asking for your help if you can give it – Mr Miller and you, or two people like you, might say to each other, “Well, we agree what this system is really like, but I’m afraid I’m having trouble from Mr X or Ms Y above me”. Those sort of discussions didn’t take place, did they?
John Meagher: No. I mean, Dave Miller, I believe reported to Stuart Sweetman. I never met Stuart Sweetman. I did meet Paul Rich and Dave Smith and, as you’ve seen, Dave Smith would have been involved, but I don’t know –
Sir Wyn Williams: Sorry to interrupt but, from one of the documents we looked at, I think Dave Smith was one of the people who would have voted against going ahead with it at a point in time.
John Meagher: Indeed.
Sir Wyn Williams: Okay. So –
John Meagher: I think that was a negotiating position he was expressing there.
Sir Wyn Williams: Yes, yes, sure. All right. Anyway I think, so that I don’t take the floor too much, you can speak for Mr Miller down, but you can’t speak for Mr Miller up?
John Meagher: Correct.
Sir Wyn Williams: Fine, okay. Thank you. Who’s first?
Mr Stein: I believe it’s me on behalf of Howe+Co clients.
Questioned by Mr Stein
Mr Stein: I represent a large number of people who were ex-subpostmasters, mistresses and managers, all them who have suffered at the experience of dealing with Horizon and Post Office.
I am going to start just with a bit of procedural side of matters, just so that we can understand your evidence that relates to reducing requirements into acceptance spec. It was a matter you mentioned earlier.
To do this, I’m going to look, please, at a codified agreement which is FUJ00000071 and, if we could have page 1, please, Frankie. Hopefully on the screen you will see codified agreement first page.
When we’re looking at this document, we can see from this point of view, from this very first page, that this is the basic agreement between the parties, and I’m going to take you to two separate parts. First of all, requirements at page 249-250, relativity pagination.
John Meagher: What date is this?
Mr Stein: This is 1999. Can you enlarge the page, please. If it’s possible to enlarge the page so that we can see the bottom requirement, 829. Page 249 again and the bottom part of page 249. You should see there that you have got requirement 829, General Security Prosecution Support.
I’m just going to read through this with you, and we’ll go over the page as it then goes straight into page 250:
“The contractor shall ensure that all relevant information produced by the POCL service infrastructure [then over the page, please and then next part] at the request of POCL shall be evidentially admissible and capable of certification in accordance with the Police and Criminal Evidence Act.”
Then it refers to other legislations in the United Kingdom and equivalent legislation covering Scotland.
Then next paragraph:
“At the direction of POCL, audit trail and other information necessary to support live investigations and prosecutions shall be retained for the duration of the investigation and prosecution irrespective of the normal retention period of that information.”
So, when Mr Stevens asked you questions, he asked you about your awareness of the requirement in relation to providing evidence for support in relation to investigations and prosecutions. So this is the received part of the codified agreement.
Now, I take you to the next part which will then deal with solutions, as we’re working our way through what are the procedural ways these matters are dealt with. That we will find, please, at page 392, just at the bottom of page 392, and then we see the reference number 829 at the very bottom, Pathway Response, and Frankie, if we go over the page to 393, it makes it clear that it refers to the same requirement. So:
“Pathway confirms that it will ensure that all relevant information produced at the request of POCL shall be evidentially admissible …” and so on.
Then the second paragraph:
“Pathway confirms that at the direction of POCL and subject to reasonable regard to the data volumes and storage [solutions] incurred, [then the] audit trail … will be [maintained] …”
If we go to the bottom of that page, please, Frankie, just to see what we’re referring to here, right at the very bottom of the page, we see the bottom left-hand corner, at the page that we’re looking at, we see this is a reference to solutions, okay?
So we’ve got a requirement built into the codified agreement. We’ve then got a solution set out within that codified agreement, and the entire document is 914 pages long. So this is a considerable document that’s required a fair bit of work, one assumes.
Your evidence was that, in relation to acceptances requirements, there would be an acceptance spec, and then there would be criteria drafted against it. That’s what you said roughly to my learned friend a little earlier today.
So we’ve got so far within this document to solutions. What’s next? So how do you take it from requirement and solution and onwards –
John Meagher: Yes, so the requirement tended to say, “You must do this”, and the solution didn’t normally go further than saying, “We will do this”, okay? So there are a series of subsequent documents that were produced that expanded the solution out, and the first one would have been what was called the SADD, the service architecture design document. That would have had a bit more detail. Then there would have been a series of subordinate technical documents, and in fact there was a security functional specification.
Now I can’t remember how this is addressed in the security functional specification, but that would be where that would be developed. Then subsequently the acceptance specification – sorry, yes, the acceptance specification for security would have a series of criteria, and against each criterion there would be an explanation about how that would be satisfied.
So some of it would be satisfied by review of document and some of it would be satisfied by witnessing.
Mr Stein: Right. Now you have referred to the document you used, the acronym for which is SADD, then you have these technical documents. Do they have an acronym?
John Meagher: There was a security functional specification.
Mr Stein: The fact that you are using the words “security functional specification”, does that mean it was dealt with by the security department?
John Meagher: By the, pardon?
Mr Stein: By the Post Office Internal Security Department the security bit that you are referring to, why do you call that “Security”?
John Meagher: Yes, okay. It was wider, obviously, than this.
Mr Stein: Yes.
John Meagher: Obviously when the project first started all the Benefits Agency transactions were there, so one of the main objectives of the Benefits Agency was to reduce encashment fraud. So there was a huge amount of attention given to that and to the security of the data, okay.
We had people who represented security a function on the PDA, and they would have been in touch with security experts, both in the Benefits Agency or the DSS wider area and the Post Office security people.
Mr Stein: So the Benefits Agency’s difficulty was one that they had with fraud in which they were losing a lot of money at that time in relation to fraud?
John Meagher: Yes.
Mr Stein: And the particular type of fraud they were concerned about was identity fraud, as we understand it.
John Meagher: Well, they had two types of fraud they were concerned with. One was misrepresentation of circumstances, which was not addressed by this project.
Mr Stein: No.
John Meagher: And they had what was called encashment fraud, which was – the intention here was to reduce encashment fraud.
Mr Stein: The encashment side of it is identity fraud, someone representing themselves as another in order to get money from benefits, yes?
John Meagher: Mmm.
Mr Stein: Now, you mentioned security a number of times there. Now, the security department that you are referring to within the Post Office, who was that headed by?
John Meagher: I don’t know.
Mr Stein: Right. The person that would be drafting the sort of reports that you are talking about, the technical documents in relation to security and this particular requirement, who would that have been?
John Meagher: Well, okay. So we had a representative of POCL audit on the PDA – his name was John Bruce – and we had people who attended to technical security – that would have been Jeremy Folkes.
Mr Stein: So it’s Mr Bruce and Mr Folkes. I’m going to move on and deal with one other aspect of things before coming back to particular documents. Can we go please to POL00043682.
This is a fairly short document, a three-page document. You should have had reference to this in the documents that you have been supplied. It’s an electronic memo. You can see the date top right-hand corner, indeed set into the middle of the document, it’s in August of 1999 and it’s to you, Mr Meagher, and then others referred to in the document. This from Min Burdett, and we can see all the names that the people have all got Post Office email references.
If we then go down to the second page, please – thank you very much – this refers to AI408, HSS Service Level Failure. If we just scroll our way down to Severity Rating to start off with, please, help us a little bit with this. Under Severity Rating we’ve got Pathway’s severity rating is low and then POCL’s severity rating which is medium, okay?
John Meagher: Am I looking at the – okay, right. I see it.
Mr Stein: Point 3.
John Meagher: Yes.
Mr Stein: So Pathway’s severity rating is low, POCL’s severity rating is medium, and then let’s just read through this couple of paragraphs:
“The agreed definition for High severity is: ‘consistent failure to meet minimum acceptable thresholds (MATs)’. The definition for Medium severity is: ‘occasion failure to meet MATs but MATs met on average’. POCL’s view is that the consideration should be whether this is a High or Medium incident.”
Then it goes on to say:
“Consistent failure has been shown. However, as there has been only one opportunity to measure service levels during the Core Observation Period (COP) it was originally agreed between POCL and Pathway that the Severity of this incident should be Medium as POCL could not show a consistent failure during COP.”
So far this seems to say that the view of the Post Office was that the severity level was such that it met the original criteria of high severity; agree?
John Meagher: Yes.
Mr Stein: But then it seems it was agreed between POCL and Pathway that the severity of this incident should be medium as POCL could not show a consistent failure during the period of observation.
John Meagher: Yes.
Mr Stein: From the point of view of the people that we represent, how could Pathway be getting away with it?
John Meagher: Well, I think what is being said here on the Post Office side is that the sample size is too small to be able to sustain an argument that it’s a high severity classification.
Mr Stein: Yes, I think we can all read this. How could Pathway be getting away with this? POCL’s view is clear: consistent failure has been shown. This is the contracting party that is buying in the Horizon System thinking oh, sorry, but the consistent failure is shown; this is high severity. How is Pathway allowed to get away with it?
John Meagher: I don’t know how I can answer your question.
Mr Stein: All right. Let’s read on:
“Pathway have changed their view to Low …”
Then it refers to workshop 10/8:
“… on the basis that the original figures were misreported (but Pathway have not shared these figures with POCL). POCL consider Low unacceptable as the severity rating should be based on Pathway’s originally reported service levels.”
Now, this appears to show within these few paragraphs that the POCL view this as high severity yet Pathway is consistently arguing it’s low severity.
John Meagher: In general Pathway, in practically every case, would ascribe a lower severity than the Post Office did.
Mr Stein: This is a fairly substantial difference, isn’t it? This is POCL believing high severity in relation to help desk requirements and –
John Meagher: Yes, yes, I don’t disagree with you.
Mr Stein: And again the rationale appears to be that Pathway have not shared the figures, the figures that it’s using to try and say that it’s in fact low.
John Meagher: Yes. I think what is being said there is that the evidence has not been shared and is being questioned.
Mr Stein: Yes. This then is leading us – in relation to the date of this, this is in the August period of 1999.
John Meagher: Yes.
Mr Stein: We’re getting very close to roll-out, yes? We’re looking at a system that you’re saying is delicate. Is this in any way acceptable, what’s on the content of this document so close to roll-out? Mr Meagher?
John Meagher: Sorry?
Mr Stein: Is it in any way acceptable, so close to roll-out, that there are such huge differences between POCL’s view about consistency and severity of incidents versus Pathway’s at this time?
John Meagher: Well, there’s always – I think there was always a difference of view.
Mr Stein: Is it acceptable, Mr Meagher, so close to roll-out?
John Meagher: Under what rules? I mean, we’re dealing with a contract here, okay? So I don’t quite understand what acceptability in your question means.
Mr Stein: So close to roll-out, in relation to a big system that’s going to have a huge impact on the Post Office, do you find yourself, professionally, this level of disagreement –
John Meagher: No, I mean –
Mr Stein: – let me finish – is acceptable, Mr Meagher?
John Meagher: I would distinguish between the difference of opinion and the reality of the system.
Mr Stein: You have answered questions from Mr Stevens regarding page 18 of your statement.
John Meagher: Yes.
Mr Stein: The reference in relation to your statement is at WITN04150100, page 18, please. If you could highlight, please, Frankie – thank you very much.
This is the reference that my learned friend finished on in relation to years later you learned that:
“… Post Office had prosecuted multiple subpostmasters based on assurances from Fujitsu that no explanation other than fraud was possible, I was shocked.”
You have also added to that part of your statement that you regarded the system itself as being delicate.
John Meagher: Mmm.
Mr Stein: Yes? Pamela Lock is someone I represent, and she ran the Post Office branch in Swansea from September 1974 until July 2000. The Horizon System was installed in January of 2000, Mr Meagher. She was experiencing problems with shortfalls in January of 2000 shortly after installation. She was then investigated by auditors from the Post Office in July of 2000. You left, I think, your employment at the Post Office in January of 2000.
John Meagher: Yes.
Mr Stein: At the time when you left the Post Office in January of 2000, do you believe that anybody should have been investigated and thereafter prosecuted in relation to shortfalls that were being found in a system that was as delicate as you describe?
John Meagher: I do not.
Mr Stein: Mr Meagher, when were you first aware of the people that were being prosecuted and civil actions being taken against them by the Post Office, essentially based upon the system, and I know that you have a caveat to that, which is that you say it’s based upon what is being said by people from Fujitsu. So whichever way round you look at it, when you were first aware of what was going on by way of prosecutions and civil actions?
John Meagher: I think when we first started to see it in Computer Weekly.
Mr Stein: So that takes us to – well, it’s 2010 onwards, and then you’re aware of what’s going on with Second Sight, were you? Did you hear about that?
John Meagher: I remember reading – no, I don’t.
Mr Stein: You were aware then at a much later stage, the time we’re reaching, 2019, of the civil action at the High Court?
John Meagher: Yes.
Mr Stein: After that the Court of Appeal cases where individuals had to take their case to the Court of Appeal and get themselves cleared, as did Ms Lock, the lady I just referred you to, yes. Given your evidence is that the system was in a state of delicacy by the time you left, why didn’t you come forward and say something and say, “Look, something’s wrong here. Nobody should have been prosecuted based upon this system”? Why didn’t you come forward to the litigants or the people being prosecuted that you knew about at that stage?
John Meagher: Well, I wasn’t aware that anyone was being called.
Mr Stein: Why didn’t you come forward and say you could help, Mr Meagher?
John Meagher: Well, I wasn’t aware that I could help.
Mr Stein: Why didn’t you come forward and say, “I knew the system had problems”?
John Meagher: It was 20 years previously. I mean, the system had been changed on a number of occasions. I didn’t think I had anything to offer.
Mr Stein: Thank you, sir.
Ms Page: Nothing from me. Thank you.
Mr Stevens: Sir, I think that’s all the questions from the room.
Sir Wyn Williams: Well, thank you very much, Mr Meagher, for making your witness statement and for spending a number of hours this afternoon being questioned about these matters. I’m grateful to you.
John Meagher: Thank you.
Sir Wyn Williams: So that concludes today’s business and we’ll start again at 10.00 tomorrow morning. Is that correct, Mr Stevens?
Mr Stevens: That’s correct, sir. Thank you.
Sir Wyn Williams: All right. Thank you very much.
(3.31 pm)
(Adjourned until 10.00 am the following day)