16 May 2023 – Michael Peach
Hide video Show video
(10.00 am)
Mr Beer: Good morning, sir, can you see and hear me?
Sir Wyn Williams: Yes.
Mr Beer: May I call Michael Peach, please.
Michael Peach
MICHAEL EDWARD PRYOR PEACH (affirmed).
Questioned by Mr Beer
Mr Beer: Good morning, Mr Peach. My name is Jason Beer and I ask questions on behalf of the Inquiry. Can you give us your full name, please.
Michael Peach: Michael Edward Pryor Peach.
Mr Beer: I think in the documents you are certainly known as Mik Peach?
Michael Peach: Yes.
Mr Beer: With Mik being spelt M-I-K; is that right?
Michael Peach: Yes.
Mr Beer: Thank you very much for coming to the Inquiry to assist it in its work, for previously providing a witness statement to the Inquiry. We’re very grateful to you. You should have in front of you a hard copy of the witness statement. It’s dated 3 March 2023 and, if you turn to the last page of it, which is page 52, there should be a signature?
Michael Peach: Yes.
Mr Beer: Is that your signature?
Michael Peach: Yes, it is.
Mr Beer: Before I ask you whether the contents of it are true, can you turn up paragraphs 1 and 7, please. In paragraph 1, second sentence, you say:
“I left Fujitsu in September 2009.”
In paragraph 7, second sentence, you say:
“I held this role until I left Fujitsu on 30 September 2009.”
Is there a correction that you would like to make to those two sentences?
Michael Peach: Yes, there is. I now, having reviewed documents from last week, believe that I did not actually leave until December 2009.
Mr Beer: Is that because we have shown you, amongst other things, some emails with your name on it that post-date September 2009?
Michael Peach: That’s correct.
Mr Beer: I think you heard the evidence of Mr Parker the other day?
Michael Peach: I did.
Mr Beer: Thank you.
I think there’s another issue of emphasis that you’ve raised later in the witness statement – I won’t get you to make the correction now – we’ll deal with that in due course.
Michael Peach: Understood.
Mr Beer: But, subject to those points, are the contents of this witness statement true to the best of your knowledge?
Michael Peach: Yes, they are.
Mr Beer: Thank you very much. For the purpose of the transcript the URN is WITN04510100.
Can I start, please, with your professional experience, career and qualifications. Do you have any professional qualifications that are relevant to the issues that we’re to discuss today in your evidence?
Michael Peach: No.
Mr Beer: I think you joined ICL, as it was then known, in 1980; is that right?
Michael Peach: That’s correct.
Mr Beer: Having worked there for 17 years, you joined the Pathway project or Pathway, as it was then known, in 1997; is that right?
Michael Peach: Yes, I did.
Mr Beer: You joined as manager of the SSC.
Michael Peach: Yes.
Mr Beer: A role which you occupied for 12 or so years until you left in December 2009?
Michael Peach: Yes.
Mr Beer: When you were manager of the SSC, did the person to whom you reported remain the same over that 12-year period?
Michael Peach: No.
Mr Beer: Was the identity of their role the same, ie their job function –
Michael Peach: No.
Mr Beer: – or did that change?
Michael Peach: That changed as well.
Mr Beer: Can you tell us first by job function and then by name, if you can remember it, who the relevant report was?
Michael Peach: Initially I reported to Stephen Muchow, who was Customer Service Director. At later times, I reported to the Support Services Manager, who – they reported to the Customer Service Director.
Mr Beer: Who was the Support Services Manager?
Michael Peach: There were a number: Peter Burden, Carl Marx, Andy Hall, Naomi Elliot, at which point I’ve run out. There were more.
Mr Beer: Okay. Can you remember roughly in the 12-year period when the change over occurred, from when you were reporting straight into a director and then when there was somebody who was between you and a director?
Michael Peach: The first change occurred under Stephen Muchow, so it would have been round about 1999. There were times at later dates when the structure changed and I reported to the CS Director, other times when I reported to the Support Services Manager. So it wasn’t a consistent move down the organisation: it was move down, move up, move down, move up.
Mr Beer: I see. I understand. How many staff did you manage in the SSC?
Michael Peach: Initially six, later moving to between 25 and 30.
Mr Beer: We’ve heard that there was a flat reporting structure with everyone reporting to you; is that accurate?
Michael Peach: That’s correct.
Mr Beer: I think, however, Mr Parker, Steve Parker, was nominally your deputy and, in particular, he deputised for you when you were away; is that also correct?
Michael Peach: That’s correct.
Mr Beer: Did he perform any other roles as deputy manager?
Michael Peach: Not that I can think of.
Mr Beer: Can we look at the role of the SSC, please. We’ve heard a lot of evidence about this already, so I’m going to take things relatively briefly. Can we do so through a document, FUJ00119994.
You should have in front of you a document called “End to End Support Process, Operational Level Agreement”, dated 10 October 1999 as version 1 and, if we just scroll down a little bit, please, we can see that the author of it is you.
Michael Peach: That’s correct.
Mr Beer: This version is marked as a draft. If we go over the page, please, to “Document control”, you can see the provenance of it, when it was first drafted, moving to version 1. Do you know why it would still be marked on the front page as a draft when it seems to have achieved the status of version 1 in document control?
Michael Peach: The version number that I have at the top of the page is 1.
Mr Beer: Yes. If we just go back to the first page, you can see under “Status” it says “Draft”?
Michael Peach: An oversight – probably mine.
Mr Beer: Okay. Can we go to page 7, please. This section of the document sets out the responsibilities, I think, of the first and second line support up to the third line support; is that right?
Michael Peach: That’s correct.
Mr Beer: So HSH and SMC obligations up to SSC, third line obligations?
Michael Peach: Yes.
Mr Beer: If we just go down, please, to (d), the responsibility is said to be, for those two lines of support:
“To ‘filter’ all calls for which the problem is already known to the support community and for which a resolution is already known or has been generated. In this case context the term ‘resolution’ can take a number of forms, including:
“The statement that the problem is resolved in release xxx of the Horizon solution.
“There is a documented workaround for the problem.
“The documentation relating to that part of the system is under review of being changed.”
Then in bold and italics:
“No calls passed to the SSC which are subsequently resolved as known errors, except in cases where the symptoms reported by the customer did not match the symptoms recorded in the known error documentation, and which therefore the HSH/SMC could not reasonably have been expected to find.”
Could you explain what this direction is to the first and second lines of support, please?
Michael Peach: The structure of the SSC and its function meant that we were supposed to receive from second line only the first instance of a new software problem. The targets throughout this document were aimed at the HSH and SMC to ensure that they did not overload the SSC with calls that they could have filtered themselves.
Mr Beer: So was this a direction given to them from the very start, to reduce calls related to so-called unknown errors from being diverted and escalated to the SSC and then through to fourth line support?
Michael Peach: That was the intention of the document. I don’t think I would use the term “from the very start” because HSH and SMC existed before I joined, but the exact relationship and targets placed on them were not there until I wrote this document.
Mr Beer: I see. So why did you introduce this?
Michael Peach: I think it was – the first job that Stephen Muchow gave me to do when I arrived was “You need to sort out the relationship between the four lines of support”. I based this document on previous experience of supporting VME systems in order to make sure that the SSC weren’t overloaded.
Mr Beer: Before we proceed can I just check, sir, that your camera is working? You appear to have disappeared from our screen – and your microphone.
Sir Wyn Williams: I think I probably mute myself generally and inadvertently stopped the video. Sorry about that.
Mr Beer: Yes, I think that’s what happened, sir. Thank you.
Mr Peach, what was the reason, as you understood it, that Mr Muchow said you needed to sort out the relationship between the four tiers of support?
Michael Peach: Because although the four tiers of support were there, the relationship between them had not been adequately documented.
Mr Beer: Was it working adequately?
Michael Peach: There was no live system at that time and my impression was clearly not.
Mr Beer: Why was it your impression that it wasn’t working adequately?
Michael Peach: Because there was no document such as this that defined the relationship between the lines of support.
Mr Beer: Was this issue – the passing of calls inappropriately from lines 1 and 2 to line 3 – an issue that remained over the duration of your time as manager of the SSC?
Michael Peach: No, it improved greatly as first and second line became better trained, properly staffed and as the SSC got more experience with the system.
Mr Beer: In that period of 12 or so years, were you aware of any inappropriate pressure being placed on first and second line support not to pass calls on to third line support?
Michael Peach: No, I was not aware of such pressure.
Mr Beer: Can we move on a little bit, please, to FUJ00120446. You will see this is dated 29 January 2001. It’s described as the “Customer Support Services Operations Manual”. The owner of it is Peter Burden. The author is “Richard Burton, A&TC”. Can you recall what that stands for?
Michael Peach: No, sorry.
Mr Beer: Then it says “Technical Authors” and then Peter Burden. At this time, what function do you think Peter Burden would have been performing?
Michael Peach: I think he was the Support Services Manager.
Mr Beer: So somebody to whom you reported?
Michael Peach: Correct.
Mr Beer: The distribution at the bottom of the page, second from the bottom “SSC Manager”. That’s you?
Michael Peach: That’s correct.
Mr Beer: Now, the role of the SSC is set out in this policy document. Can we turn to, please, page 8 and look at paragraph 4.1:
“The principles by which the SSC operates are documented in “End-to-End Support Process Operational Level Agreement …”
I think that’s the document we just looked at?
Michael Peach: Yes.
Mr Beer: The reference CS/FSP/006 is the document we looked at:
“… which defines the responsibilities of the four levels of support towards each other. This document is effectively a service level agreement between the support units, outlining specific tasks and measures of success.
“The aim of the SSC is to provide a support capability to Pathway that resolves technical problems in the minimum time and with the minimum amount of disruption to the service. The SSC aims to provide a centre of technical expertise for Customer Service, providing technical advice, guidance and expertise relating to all parts of the Pathway system.
“… specifically the SSC has responsibilities to:
“First and second …
“Fourth line support.”
Then 4.1.1:
“SSC responsibilities to first and second line support.”
If we can expand that to shown all 13 obligations, ie look at the next page as well, if possible. Thank you very much. You can see that there are 13 or so obligations set out imposed on the SSC down to first and second line support. What did you understand the idea of this document as opposed to your document was in setting these out in this way?
Michael Peach: My recollection is that this document was a services manual for the whole of Customer Service and that the CS/FSP/006, so the previous document that we looked at, formed the basis of the SSC part of the CS operations manual.
Mr Beer: So this is looking at all four lines of support?
Michael Peach: This document is the services manual for Customer Service. It’s not just the SSC, it’s the other units within Customer Service who had their own obligations to other people. So this is basically collating all of those into one document.
Mr Beer: You see obligation number 5 is to:
“Ensure that the incident is resolved within the total time allowed by the contract between the customer and Pathway.”
Michael Peach: Yes.
Mr Beer: Were there written service level agreements regulating the work of the SSC setting out times, volumes and other metrics?
Michael Peach: No. As far as I’m aware, there were no SLAs or SLTs in the contract that related to the resolution of software problems. Most of the SLAs and SLTs related to hardware issues and network.
Mr Beer: So what does this obligation mean then?
Michael Peach: For me, it meant try and keep the SSC on track with the obligations which were stated in the previous document. But, in terms of obligations to the customer in the contract, it has no meaning.
Mr Beer: Do you know why it’s there, if it has no meaning?
Michael Peach: Only because I believe it was extracted from the previous document as one of the SSC’s obligations.
Mr Beer: The previous document being the one we looked at –
Michael Peach: Yes.
Mr Beer: – the one that you drafted?
Michael Peach: Yes.
Mr Beer: Why did you, therefore, include something in a document that had no meaning?
Michael Peach: I didn’t write this document. I will have reviewed it but I didn’t write it. I think it’s probably a common clause that would have included the other units that did have SLAs and SLTs attached to them.
Mr Beer: This is taking the reader and, therefore, taking the SSC back to the contract as a measure of progress, performance or success, isn’t it?
Michael Peach: It is.
Mr Beer: But you’re saying, in fact, to your knowledge, the contract didn’t contain such a measure?
Michael Peach: Not for the resolution of software calls, no.
Mr Beer: Were there SLAs in respect of the responsiveness of the HSH or the SMC?
Michael Peach: Yes, there were.
Mr Beer: Do you know why there wasn’t an equivalent for the SSC?
Michael Peach: No, for certain, no. I believe, however, that most of the SLAs and SLTs related to hardware so there were specific times for engineers to visit post offices to replace counters, et cetera, but I think it was always accepted that, when it came to software problems, any code fix would require extensive testing before it was released to the live estate and would generally be included in either a maintenance or a major release.
Mr Beer: Can we look at obligation 7, please:
“[To] Create and maintain a register of known deficiencies within the Pathway system and the solution to these problems, where known.”
8:
“Allow the HSH and SMC access to this register so that they can fulfil their function of filtering out known errors.”
Does this essentially describe the KEL system?
Michael Peach: Yes, it does.
Mr Beer: Was the design and creation of KEL the response to obligation 7, essentially –
Michael Peach: Yes.
Mr Beer: – or the manifestation of obligation 7?
Michael Peach: Yes.
Mr Beer: Did both first and second line support have access to the KEL system?
Michael Peach: Yes, they did.
Mr Beer: Can we look, please, at 4.1.2 further down the page – thank you – which sets out SSC responsibilities to fourth line support. Again, at item 2 there is recorded an obligation to:
“Filter out all calls for which the problem is already known to the support community and for which a solution is already known or has been generated [including] problems for which the SSC knows a resolution but has not yet incorporated the resolution into the Known Error Log.”
Is this a common feature of support services; namely, the filtering out at every stage of calls before passage to the next stage?
Michael Peach: Yes, I would describe it as that.
Mr Beer: Was the SSC, to your knowledge, ever under any pressure to avoid passing problems up to the fourth line of support?
Michael Peach: No.
Mr Beer: If an issue was resolved under existing KEL guidance, or an existing KEL, or if a problem was referred to the SSC with insufficient evidence, would that be sent to fourth line support for investigation?
Michael Peach: Sorry, could you repeat the two conditions there?
Mr Beer: Yes. If an issue was thought to be resolved under existing KEL guidance – it’s been referred in to SSC –
Michael Peach: Right.
Mr Beer: – or if the issue had insufficient evidence of a system problem, would that be referred to fourth line support for further investigation?
Michael Peach: It certainly could be if the KEL was believed to not actually fix the problem. There was no restrictions placed on any call that we could send to fourth line. I mean, if we believed there was still an underlying issue and it was a code problem, then we would send it to fourth line, regardless of what documentation was there.
Mr Beer: That document can come down now.
You said that “if we believed that it was a code problem”.
Michael Peach: Yes.
Mr Beer: If there was no evidence of a code problem, what would happen then?
Michael Peach: The SSC person who was handling the call would make a judgement about where they thought the problem existed in the system. If it was possible that it was a code problem, then it would still go to fourth line. If it was likely to have been a hardware problem, it would have gone back to the SMC, and so on.
Mr Beer: Would it be dismissed as a user error or possible user error?
Michael Peach: It would not be dismissed as a user error but it’s certainly possible that the SSC staff member could have said, “I believe on the balance of probability that this is most likely to be a user error”. Actually, the term we tended to use was “possible user error” not “this is a user error”.
Mr Beer: In what you have just said there you’ve used “on the balance of probabilities it is a user error” –
Michael Peach: Yes.
Mr Beer: – and “it is possible that it is a user error”.
Michael Peach: Yes.
Mr Beer: Do you recognise that there’s a difference between those?
Michael Peach: Yes, I do.
Mr Beer: To what level of satisfaction did an SSC diagnostician need to be satisfied in order to attribute the code “user error” to a problem?
Michael Peach: I could not quote you a percentage on that. I mean, they would need to be fairly certain before passing it back as a user error, as they would need to be fairly certain that something was a code error to pass it through to fourth line. If they were uncertain, they would gather more evidence and diagnose it properly.
Mr Beer: If there was no evidence that it was a code problem –
Michael Peach: Right.
Mr Beer: – would that cause them to say “Possible user error, refer back to the subpostmaster for more information”?
Michael Peach: Yes, that’s certainly possible.
Mr Beer: What further information – I realise we’re talking at a theoretical level at the moment, without a practical example – what kind of further information would you expect a subpostmaster to provide?
Michael Peach: Recollection of what it was that they had done prior to reporting the error. It’s a very difficult area because there was not sufficient diagnostic capability on the counters to examine exactly what the postmaster had done. So, whilst the SSC could take all of the evidence and put it through code or utilities that SSC staff had produced, in order to check the code, what we could never do was find out precisely what the postmaster had done on the counter.
Mr Beer: You said that there wasn’t a sufficient diagnostic facility at the counter level.
Michael Peach: That’s correct.
Mr Beer: Can you just explain what you mean by that?
Michael Peach: A log of keystrokes performed on the counter would have been useful in a number of cases.
Mr Beer: Can you explain to us what you mean, because we’ve heard different descriptions of what a keystroke log means, what you mean by a keystroke log?
Michael Peach: A log of every key depression or screen touch that had taken place on the counter.
Mr Beer: Was it your understanding that that did not exist at all?
Michael Peach: During the time that I was SSC manager, I don’t believe it existed.
Mr Beer: What was the greatest level of scrutiny you could give to what had occurred at a counter level?
Michael Peach: The Riposte message store and there were three or four log files that were kept on the counter. Their exact contents I couldn’t tell you.
Mr Beer: I described those, in the past, as recording when a transaction occurred or when you committed something to a stack.
Michael Peach: Yes.
Mr Beer: That may be imprecise language. Using your language, what would you say those message stores and files recorded?
Michael Peach: The message stores recorded all of the transactions done by the Riposte software and there may well have been a number of other things that I probably never knew. There were also the NT event logs, which is when an application or, indeed, the Microsoft software writes to a log. There were, I believe, at least one, possibly two others, PS Standard Log rings a bell but the contents I couldn’t tell you.
Mr Beer: Do you remember something called there POC log?
Michael Peach: Only because I heard it mentioned when Anne Chambers gave evidence to this Inquiry.
Mr Beer: Have you got no greater recollection than that?
Michael Peach: No, and it’s way too detailed technically for my knowledge.
Mr Beer: So referencing an issue back to – or referring an issue back to a subpostmaster for the provision of more information and evidence, that was difficult for the subpostmaster – would that be right – because they couldn’t look at their system and themselves say, “The system shows that I did X, Y and Z”?
Michael Peach: That is correct.
Mr Beer: If the subpostmaster couldn’t produce any more evidence or information as to what had occurred, would the matter then be – would the PEAK be closed?
Michael Peach: The PEAK would have been closed at the moment that the call went back to SMC and HSH to ask for the further evidence. It would then, if they managed to get the further evidence, would be reopened.
Mr Beer: I see. So the closure of the call occurred upon reference down. If the subpostmaster didn’t come back to first or second line support, was there any obligation on the SSC to follow the call up?
Michael Peach: On the SSC, no.
Mr Beer: Was there any obligation on first or second line support to follow the call up?
Michael Peach: I don’t know HSH and SMC’s processes, so I couldn’t comment on that.
Mr Beer: Can I examine, please, moving on a year still further into June 2002 now, and look at POL00000877. This is an internal assessment prepared by Fujitsu on 11 June 2002 and I think we can see from the second page there’s a list of those who were involved in the internal assessment conducted over two days, I think, at Feltham and Bracknell – sorry, three days at Feltham and Bracknell and we see your name in the list in Customer Services.
Michael Peach: Correct.
Mr Beer: Going back to the first page, please, and just scrolling down to assessment summary, can you recall what this was, this three-day assessment, at Feltham and Bracknell?
Michael Peach: No, sorry.
Mr Beer: Just looking at the document now, can you recall what its purpose or function was?
Michael Peach: The format appears to be similar to a BSI audit. I can only assume it was an audit done for compliance with ISO 9001, done internally not through BSI.
Mr Beer: So an internal audit?
Michael Peach: Oh, yes.
Mr Beer: Can we look, please, back to the second page and look at the summary. Just scroll down, please. Thank you.
The last bullet point on that summary says:
“… the main findings, and recommendations … were as follows …
“There is considerable challenge to the Pathway to continue to operate profitably in the context of a demanding customer facing considerable change and costs-reduction in their own business.”
Did you understand the customer, ie the Post Office, to be a demanding customer?
Michael Peach: Yes.
Mr Beer: In what way was POL, the Post Office, a demanding customer?
Michael Peach: I would draw that conclusion purely from the number of SLTs in the contract.
Mr Beer: So it was demanding from the start, as a matter of contract –
Michael Peach: Oh, yes.
Mr Beer: – rather than in the way that it behaved in the course of the extract; is that right?
Michael Peach: The SLTs in the contract were monitored and reported on frequently and, if Fujitsu failed them, then there were financial penalties. From the SSC point of view, that didn’t impact us at all. All of that was done by the MSU, the Management Support Unit. For a while, the Management Support Unit and the SSC both reported to the Support Services Manager, so I was aware that the reviews were taking place because there were monthly management meetings.
Mr Beer: Within the SSC, were the considerable challenges?
Michael Peach: With regard to a challenging customer?
Mr Beer: Yes.
Michael Peach: No, there were technical challenges associated with each call as it came in.
Mr Beer: This records “a considerable challenge to Pathway to continue to operate profitably”, so that’s the Horizon System within Fujitsu –
Michael Peach: Yes.
Mr Beer: – being a challenge for it to continue to operate profitably? Did you feel that challenge within the SSC?
Michael Peach: No.
Mr Beer: What did you understand this to refer to?
Michael Peach: I would have taken this to refer to performance against the SLTs in the contract.
Mr Beer: Were there any SLTs in the contract that impinged on the work of the SSC?
Michael Peach: The only ones that I can recall were the obligation to pass all transactions through to Post Office within a certain time period. As I said, there were no SLTs that I was aware of relating to the fixing of software calls.
Mr Beer: Thank you. That can come down.
You were in the SSC from 1997 onwards –
Michael Peach: Correct.
Mr Beer: – and, therefore, provided third line support whilst the product was being tested and rolled out?
Michael Peach: Yes.
Mr Beer: Had you been involved in the testing or the provision of line support when other projects had been tested and rolled out?
Michael Peach: Are you referring to my previous time in ICL?
Mr Beer: Yes.
Michael Peach: Only releases of VME and, when I was managing a Rapid Application Development unit, then we were obviously releasing applications. But those tended to be very small, ten or more users – certainly not 37,000 users.
Mr Beer: So nothing of this scale?
Michael Peach: No.
Mr Beer: So had you got a reference point against which to compare how easy or problematic the provision of a support service was when you were engaged in the provision of such services whilst Horizon was tested and rolled out?
Michael Peach: Only experiences in relation to releases of the VME operating system.
Mr Beer: What was your experience, speaking in general terms, of the provision of third line support when Horizon was being tested and rolled out, so speaking between ‘97 and mid-2000?
Michael Peach: Sorry, can you explain what you meant by that?
Mr Beer: Yes. Looking at it in general terms –
Michael Peach: Right.
Mr Beer: – what was your feeling, your impression, your judgement, on how easy or difficult it was to provide third line support between, say, 1997 and mid-2000?
Michael Peach: Initially hard. Lots of inexperience in first and second line of support and, indeed, with postmasters using a completely new system, becoming progressively easier as the different lines of support became more experienced and the KEL system was populated.
Mr Beer: By mid-2000, was everything running smoothly?
Michael Peach: I think, mid-2000, the rollout had not been completed. I’m not certain when the rollout was completed.
Mr Beer: Take it by reference to the end of the rollout period then.
Michael Peach: At the end of the rollout period, it was already beginning to become easier.
Mr Beer: Did you have a view as to the robustness and reliability of the Horizon System by the end of rollout?
Michael Peach: That’s very difficult for someone in support to answer. Nobody ever phones you to tell you the system is working properly.
Mr Beer: Sorry, can you say that sentence again, please?
Michael Peach: Nobody ever phones you to tell you the system were working properly; you are constantly phoned when it’s not. So you obviously get a fairly jaundiced view.
Having said that, I would have described it as generally working the way I would have expected it to work. That sounds very vague, I know. I’m sorry. I don’t think I can be more precise.
Mr Beer: Do you remember Richard Roll?
Michael Peach: Yes, I do.
Mr Beer: He worked in the SSC between January 2001 and August 2004.
Michael Peach: Yes.
Mr Beer: So a period of about three and a half years and you would have been his manager for the entirety of that period?
Michael Peach: I would.
Mr Beer: With Mr Parker acting as your deputy?
Michael Peach: Correct.
Mr Beer: Mr Roll told the chair that:
“It was widely accepted that the underlying or root cause [that was with problems with the system] were that the system was crap, it needed rewriting but that that was never going to happen because the money was not available, the resources were not available to do so.”
In that period, would you say that was a common view: that the system was “crap”?
Michael Peach: No.
Mr Beer: Was it widely accepted within the SSC that the system was “crap” and “needed rewriting”?
Michael Peach: No. I think what Richard failed to understand was that, by the time that the code gets to the live estate, it has already been through extensive testing and acceptance formally by Post Office. So, essentially, from the point of view of the support teams, that’s the code. There is no point in saying “I want this completely rewritten”, because it’s already been through a testing and acceptance process.
Mr Beer: What about if it’s gone through the testing and acceptance process and things have been papered over and a decision has been made to proceed with a system that is riddled with faults?
Michael Peach: I would not use the term “riddled with faults”. The acceptance criteria, as specified between development testing teams and the customer, would indicate that you do not take the product to live based on a number of criteria and those criteria would be things like no more than “N” A priority calls outstanding, “X” B priority calls, et cetera. So it would not, in my opinion, be “crap” when it went out to the live estate.
Mr Beer: Are you saying that because a system has been accepted it cannot have faults?
Michael Peach: No, I’m saying that, because it has been accepted, the number of acceptable faults, as agreed between Fujitsu and the customer, would have been defined. Obviously when it goes out to the live estate it has faults. Every software has faults.
Mr Beer: What if the customer had decided to rewrite the acceptance criteria a number of times to progressively allow more and more faults to be present in the system because there was pressure on the customer to move to acceptance?
Michael Peach: I have no knowledge of that taking place.
Mr Beer: Were you aware of variations to the contract between Fujitsu and the Post Office –
Michael Peach: No.
Mr Beer: – in 1999 and 2000 –
Michael Peach: No.
Mr Beer: – where exactly that occurred?
Michael Peach: No.
Mr Beer: Can we look please at WITN04600104. Thank you.
This is a document that you weren’t copied in on or were not an author nor a reviewer. It’s dated 10 May 2000 and you’ll see from the “Abstract” it presents the observations and recommendations resulting from an internal audit, along with agreed corrective action, the action owner and the date by which the action is to be complete.
If we go to page 9, please, you’ll see in the top left-hand box, against the reference 4.2.1, it is recorded that:
“The audit identified that the EPOSS continues to be unstable. PinICL evidence illustrated the numbers of PinICLs raised since the 1998 Task Force and the rate of their being raised.
“The EPOSS solutions report made specific recommendations to consider the redesign and rewrite of EPOSS, in part or in whole, to address the then known shortcomings. In light of the continued evidence of poor product quality these recommendations should be reconsidered.”
Did you know that there had been something called the EPOSS Task Force?
Michael Peach: No.
Mr Beer: Did you known that a report had been produced recommending the consideration of the total or partial rewrite and redesign of EPOSS?
Michael Peach: No.
Mr Beer: Did you know that in May 2000 there had been a recommendation by this internal audit that those selfsame recommendations in the light of continued poor product quality should be reconsidered?
Michael Peach: No.
Mr Beer: Are those facts and matters of which you ought to have known?
Michael Peach: I don’t think so. From what I can see from this document, it’s an internal discussion between Development and Testing as to the state of the product before it goes to the live estate. I would not have been involved in any decisions that were taken on this nor would I have expected to be.
Mr Beer: After this time, May 2000, you find within third line support a preponderance of problems with EPOSS?
Michael Peach: I can’t recall and don’t have the figures to tell you how much was counter-based problems and how much was central systems problems.
Mr Beer: Thank you. That can come down.
Going back to what Mr Roll told the Inquiry, he said that, rather than a redesign and rewrite, which was never going to happen on cost grounds, the SSC was left to seek to patch up with the Development team the system on an ad hoc basis. Is that accurate?
Michael Peach: I don’t agree with Richard’s comments. I don’t agree with his initial premise. He didn’t know – I mean, I didn’t know the head count or the development budget. I’m quite certain he didn’t. So saying that it’s all due to lack of money or lack of resources, as far as I’m concerned, is supposition on his behalf. When it comes to the statement “The SSC were patching up things”, examining the cause of problems and fixing them is what the support team did.
Mr Beer: He told the Inquiry:
“The software issues we were routinely encountering could and did cause financial discrepancies at branch level, including shortfalls being incorrectly shown on the Horizon System.”
Does that accord with your recollection?
Michael Peach: There were certainly bugs in the system that could cause those symptoms, yes.
Mr Beer: He told us that the Horizon cash accounts were, in his words, “pretty ropey” and that he told you that, adding, “Surely, these should be rewritten”, and you agreed with him and said:
“Yes, but it’s never going to happen.”
Is that accurate?
Michael Peach: I don’t recall that conversation at all.
Mr Beer: Are you saying, through the passage of time, it might have occurred but you now do not recall or that, given your view of the quality of the Horizon System, it is something that is unlikely to have happened?
Michael Peach: I am saying that an individual member of the SSC may have expressed reservations of the code but I don’t recall the conversation, so I can’t give you a reason why I may have said what he believes I said.
Mr Beer: Mr Roll told us that:
“If we in the third line support were unable to find the cause of a problem, this was reported up the chain to fourth line but it was assumed that the postmaster was to blame.”
Was that a practice of which you were aware?
Michael Peach: Absolutely not.
Mr Beer: Was it a common theme throughout the time that you were the head of the SSC, that if positive evidence of a software fault could not be found, it was assumed that the subpostmaster was to blame and that’s how it was written up?
Michael Peach: No, on two grounds. Firstly, whenever any call came in I expected people to look at all the evidence and diagnose it properly and that means you have no fixed starting position. You don’t assume from the beginning that it’s a user error, you don’t assume it’s a software bug.
Secondly, we, to my knowledge, never used blame. Even when calls were being returned as possible user error, that could mean any number of things. It could mean that documentation at the Post Office wasn’t accurate, hadn’t been followed – it’s not a question of blame.
Mr Beer: Mr Roll told the Inquiry that:
“Sometimes we were instructed not to let the subpostmaster know that we had altered his system whilst he was logged on. To my recollection, sometimes the Post Office requested this, sometimes Fujitsu and sometimes only our department knew of it.”
Did you ever give any instructions not to inform subpostmasters to tell them that their system had been altered whilst they had been logged on?
Michael Peach: No, I didn’t give instructions of that sort.
Mr Beer: Were you aware of that practice?
Michael Peach: I have become aware through a couple of documents that I was sent to review by this Inquiry that somebody in Post Office management had said “Don’t tell the subpostmaster about this”. But, as far as I can see from the documents that I’ve been supplied, there appear to be two instances of it which were sent to my staff and, without knowing the reason behind, I wouldn’t like to comment on that.
Mr Beer: Mr Roll told the Inquiry, and I’m afraid this is a long quote:
“I recall one particular case where branch data was not being replicated from a mobile Post Office correctly and it appeared that the subpostmistress was turning off the power mid-transaction. As we couldn’t fix the problem over the phone with the subpostmistress she sent her laptop to Fujitsu for examination. Using Post Office tests rigs on the 6th floor and comparing the results with the laptop that had been returned to Fujitsu, I discovered that the button which should have put the laptop into standby mode was actually switching off the power resulting in the disk crashing. I disassembled the laptop to confirm this. Thus when the posts mistress thought she was switching her counter to standby mode, which would have initiated a controlled shut down and allowed the data store to replicate the servers, she was actually switching about power off, which is what we were seeing in the SSC.
“When I raised this with my manager, Mik Peach, who subsequently talked to the hardware team, I found this was a known problem. One of the engineers had made a mistake with a batch of laptops which had been sent out to branches before the error was detected. No-one outside the team responsible for building the laptops had been informed of this. This meant I spent several days investigating the problem. Whereas the subpostmistress in this case was provided with a replacement laptop, knowledge of this problem was kept within the departments concerned and the batch of faulty laptops was not recalled. It’s my belief that Fujitsu senior management and the Post Office was not informed.”
Do you remember that incident?
Michael Peach: From the time that it happened, no. From the Group Litigation, yes, because I was called during that trial – not to go to the trial but I was telephoned and asked if I remembered a specific hardware call from that period. So “no” was the answer that I gave at that time. I am aware of it now because I’ve read Richard Roll’s testimony in court and his appearance at this Inquiry.
Mr Beer: What do you now recall then about the incident?
Michael Peach: I’ve read through the original call and it’s clear from that – I believe it says on it something like “This is happening six minutes before POLO”, which is Post Office Log On”. Since it’s happening before the postmaster has logged on, then no financial transactions can have been impacted.
Secondly, he made comments that I had talked to the hardware manager, which is certainly possible.
Mr Beer: He said “My manager, Mik Peach, knew. His friend who ran the build team knew”. Is that what you are referring to?
Michael Peach: That’s what he said. To be clear, the lady that was running or was our contact for hardware was based in Stevenage. I was based in Bracknell. I don’t think – I don’t think we ever met face-to-face and we certainly didn’t meet socially until about five years after I’d left Fujitsu. So to say I was doing her a favour as a friend is his interpretation and, in my opinion, nonsense.
Mr Beer: He said that it never got up the chain beyond the pair of you, that he was told to hush it up. I asked him “Who told you to hush it up” and he said you. Is that accurate?
Michael Peach: No. To be specific, if I had phoned the hardware manager and was doing her the favour of hushing it up, then the first person I would not have told would be Richard Roll. I mean, if I would have wanted to hush it up, I wouldn’t have informed him of what had happened and, in any case, as I’ve said in evidence to this Inquiry, I told senior managers about that issue in my monthly report that month.
Mr Beer: Why would he be the last person you’d tell? Was he problematic?
Michael Peach: No, just if I was going to hush it up, I just would not have told him what had happened.
Mr Beer: Can we turn, please, to FUJ00087994. Can you see this is a “Group Definitions” document for the secure NT build release 2, dated 22 December 1998, yes?
Michael Peach: Yes.
Mr Beer: If we just read the “Abstract”:
“The ACP requires that access to Pathway systems be controlled by the use of pre-defined roles to which users can be assigned. Such roles will allow users to access only those parts of the system, with associated objects, they need in order to complete the tasks associated with that particular role. This document summarises this requirement and defines the roles, with associated objects, domains and access requirements.”
We can see that if we scroll down a little bit you’re amongst the distributees?
Michael Peach: Right.
Mr Beer: Looking at this document, can you summarise, even having read the abstract, what its purpose is? I think I understand but can you help us to translate the delightful language used?
Michael Peach: Can you give me a moment to read that summary?
Mr Beer: Yes.
Michael Peach: Okay, as I understand it, it’s a way of setting up Windows NT systems with defined roles each of which will have defined access to the system and how the setup of those roles should be achieved.
Mr Beer: So it’s a means of writing into the system limitations on the access rights of users?
Michael Peach: Correct.
Mr Beer: Permissions, one might call it?
Michael Peach: Indeed.
Mr Beer: This kind of document and this kind of approach is natural in a system of this kind?
Michael Peach: Yes.
Mr Beer: One might say essential?
Michael Peach: I would say essential, yes.
Mr Beer: Why would you say essential?
Michael Peach: Because you have to be clearly able to decide who is accessing what and why.
Mr Beer: Why do you have to be able to identify who is accessing what and why?
Michael Peach: Partly because there will be contractual requirements, partly because you have a need to establish an audit trail for support people and what they’re doing.
Mr Beer: Why would you need to establish an audit trail to see what people are doing?
Michael Peach: I would just regard that as being an essential part of any system. Why – I could not explain why. Just all the systems I’ve ever worked on behave that way. It’s just natural.
Mr Beer: Just think about it a little more. Why in a system that concerns financial data, for example –
Michael Peach: Yes.
Mr Beer: – might you need a system of access limitations, permissions and auditability after the event?
Michael Peach: You would need them there in order to write an audit trail. If you needed an audit trail, then I would assume that it would be because of some form of possible litigation after the event.
Mr Beer: What would you have in mind there, some litigation after the event?
Michael Peach: I really can’t answer that.
Mr Beer: Speaking generally at your first couple of years, maybe even further, maybe into 2000, 2001, 2002, were you aware that the financial data produced by Horizon was used as the basis for bringing civil and criminal proceedings against subpostmasters?
Michael Peach: No, I was not.
Mr Beer: When did you first become aware that the Horizon data was used as the foundation for criminal proceedings or civil proceedings?
Michael Peach: When Anne Chambers went to court in what I subsequently found was the Lee Castleton case.
Mr Beer: So about 2006?
Michael Peach: Yes.
Mr Beer: Had anyone before then explained to you that one of the reasons why audit or auditability of the system might be essential was for that reason?
Michael Peach: No.
Mr Beer: Can we look, please, at FUJ00088082. Can you see this is a document dated 2003?
Michael Peach: I can.
Mr Beer: So we’ve previously looked at Mr D’Alvarez’s document of December ‘98 saying this is what we need to do, these are the access rights and permissions that need to be written in, and these are the reasons why they need to be written in.
Michael Peach: Yes.
Mr Beer: Looking at again the abstract of this document, it describes the support and use of OpenSSH. Can you now recall what OpenSSH was?
Michael Peach: It was a piece of software that provided secure access to the system for the support teams which was both secure and auditable.
Mr Beer: When was it introduced?
Michael Peach: I’m not certain in terms of dates. I know it was introduced with the Network Banking release of the Horizon software because this product required software on all of the counters as well as in the central systems.
Mr Beer: Can we look please at page 15 and paragraph 7.1. This is under “Permissions Problems”:
“When attempting to diagnose problems with OpenSSH … it should be noted that permissions displayed by OpenSSH don’t necessarily reflect the full set of permissions applied by Windows. This is because the rich set of permissions supported by Windows with access specified individually for multiple users and groups cannot generally be mapped to the simple user group other model offered by POSIX. Hence OpenSSH will generally only display an approximation of the permissions in POSIX form but will usually apply the full set of Windows permissions. The permissions displayed and applied are also affected by the setting of the CYGWIN environment variable. As a result, you should not rely on the permissions information displayed in CYGWIN commands such as …” and then an example is given.
Can you translate what that means, please?
Michael Peach: No. Most of those terms mean nothing to me at all. That’s way too technical for me.
Mr Beer: Does that reflect that you were a manager and, therefore, managed people rather than carried out any technical work yourself?
Michael Peach: I didn’t carry out technical work on the live Horizon System at all and this sort of document, had I received it for review, I would have passed to one of my technical staff.
Mr Beer: We saw on the front page that it was distributed to you.
Michael Peach: Right.
Mr Beer: And you were a mandatory review authority?
Michael Peach: Yes.
Mr Beer: We can see that from the second page against your name. Perhaps we should just look at page 2, please. “Mandatory Review Authority, “Mik Peach”, and then it’s got “by proxy”. Does that reflect what you have just said, that you would have got somebody else to do it?
Michael Peach: Yes.
Mr Beer: Who amongst your team would you habitually pass these things down to?
Michael Peach: One of the five senior people.
Mr Beer: Who were they?
Michael Peach: Steve Parker, Anne Chambers, Pat Carroll, Mark Wright, John Simpkins.
Mr Beer: So if they did reply here – and it looks like they did because there’s an asterisk against your name –
Michael Peach: Yes.
Mr Beer: – it would have been one of those five?
Michael Peach: It would have been one of those five.
Mr Beer: Sir, that’s an appropriate moment. I’m about to move to a new topic. I wonder whether we could come back at 11.25, please?
Sir Wyn Williams: Yes, of course. That’s fine.
Mr Beer: Thank you very much, sir.
(11.10 am)
(A short break)
(11.25 am)
Mr Beer: Sir, good morning. Can you see and hear me?
Sir Wyn Williams: Yes, I can thank you.
Mr Beer: Mr Peach, can we turn, please, to a passage in your witness statement. It’s paragraph 123, which is on page 41. If we just blow up paragraph 123 – thank you – you say:
“If a postmaster made a mistake, a transaction could be ‘reversed’ (by inserting a ‘reversal’ or ‘corrective’ transaction) but it could not be deleted. There were processes by which SSC staff could, under instruction or approval from POL and with assistance from the postmaster, insert corrective transactions and I recall that there were processes in place to control this rare occurrence, involving dual-person sign-off on the PEAK and approved OCP requests for the SSC to do the work, which I believe had been to be approved by POL as well as Customer Service. An example of this process is OCP 21918 … dated 2 March”, and you give the reference:
“my recollection is that the process was technically complex and could only be done in agreement with the postmaster and was extremely rare.”
So you are saying that it was very an extremely rare occurrence which could only be done with the agreement of the subpostmaster, with the knowledge and approval of POL itself and Customer Services?
Michael Peach: Correct.
Mr Beer: Can we just look then at the example that you give or the reference you give there, which is FUJ00084131. We can see the OCP number that you previously gave, 21918.
Michael Peach: Yes.
Mr Beer: The title of the OCP “Insert corrective transactions at branch 382137”, and, if we can just read through this, we haven’t seen many of these before so I want to use you to look at this.
Michael Peach: Okay.
Mr Beer: “A set of unbalanced SC currency transactions were written in error at branch [then the number is given] on 18 February. A set of equal but opposite transactions is to be inserted to undo the effects.
“Justification: Enables the branch to balance correctly, and data in POL FS will also be correct.”
The date when it is going to be done by is set out.
“Extra detail: Tested within SSC and proved to generate a further [receipts and payments] mismatch which negates the first, and also a gain to negate the loss of just under £1,000 caused by the problem and currently outstanding at the branch. The gain may not be precisely the same as the original loss because of variations in the exchange rates.
“POL (Julie Edgley) have already agreed to the change, in an email attached to …” and then the PEAK number is given, yes?
Michael Peach: Yes.
Mr Beer: “Regression: This change cannot be regressed.”
Then further down, the email is in the comments section at the bottom, I think:
“Anne,
“As discussed, POL are happy for you to make the necessary system adjustments.
“From speaking to Wendy, the manager in the branch, first thing on Tuesday morning (between 9 am and 10 am) is the quietest for them.
“I have advised Wendy that you will call her as you are about to start and as you finish.
“Thank you.”
So there is a record – I mean, if we just look at the second page of the document, POL approve this change. Then scroll down to the foot of the page.
So there is a record in there that Julie – that’s Julie Edgley, who was a live service assistant in POL Service Delivery – had spoken to the subpostmistress?
Michael Peach: Yes.
Mr Beer: There is a record, therefore, that POL had agreed to the change?
Michael Peach: Correct.
Mr Beer: Was that always the case?
Michael Peach: I can’t recall any occurrences where it was not.
Mr Beer: Was it always supposed to be the case?
Michael Peach: Absolutely.
Mr Beer: Who would write this document, the OCP?
Michael Peach: This OCP – I’m not sure what Gaby Reynolds’ exact position was at that time but she would be the liaison between Post Office and the SSC. So she would be acting, effectively, as a problem manager for this instance.
Mr Beer: She was a Fujitsu employee?
Michael Peach: Yes, she was. I’m not – could we go back up?
Mr Beer: Yes, to page 1, please.
Michael Peach: Yes.
Mr Beer: Look at the bottom half of the page.
Michael Peach: Okay, I’m not aware of who actually raised the OCP itself. Certainly at times, an OCR was used, rather than an OCP. Has the Inquiry been told the difference?
Mr Beer: Yes, it has.
Michael Peach: If it was to be an OCR to insert a transaction at a branch, it would have been written by the SSC person who was going to do the work because they would have received the PEAK which highlighted the error. So they would raise the OCR and it would then go to POL for their approval and subsequently to me for sign off before the work was done.
Mr Beer: How would you sign it off?
Michael Peach: My recollection is there was – electronically on the form, just by sitting at my PC and putting my name in.
Mr Beer: Just go to the foot of page 2, please. You see “Approval status” there. There appear to be some what might, on a screen, be tiles to click on.
Michael Peach: Yes.
Mr Beer: Am I right in thinking that they might be a printed version of a tile to click on?
Michael Peach: Yes. I think with this particular one it’s an OCP and the reason that we used OCRs more frequently was that there are mandatory approvals on an OCP, which were not relevant for an OCR: POA, Core Services SMC, for example would not be required to approve a change of this sort.
Mr Beer: So this appears to be evidence of in support of what you were saying in paragraph 123 of your witness statement; namely, POL sign off and branch knowledge and agreement?
Michael Peach: Yes.
Mr Beer: Could we look at FUJ00087194, please. Different OCP 17510:
“Write corrective bureau message for …” and then a branch code is given:
“A single SC message 183227 [et cetera] was written in error on 26 November … selling 1,000 US dollars, with no corresponding settlement line. To remove the effects of this message at both the branch and on POLFS, we will insert a new message to negate the effects of the original message.
“Justification: If the change is not made in the counter messagestore (before the stock unit is balanced on Wednesday), the branch will have an unexpected gain of £484 (or thereabouts – depends on exchange rate), and a receipts and payments mismatch. This gain would have to be resolved at the branch. There would also be an inconsistency between the branch and POLFS to be resolved. By correcting the problem locally, the branch may not be aware of the problem, and there will be no inconsistency between the branch and POLFS.”
Then when it’s planned for, some extra detail is given. Then scroll down, please:
“The message will include a comment to show it has been inserted to resolve this problem (this will not be visible to the branch).”
Them there’s some more detail. This appears to suggest that a correction was to be made and made deliberately, in a way that ensured that the branch was not aware of the problem.
Michael Peach: I’m not convinced that the wording of that means that the branch were not aware there was a problem. Certainly POL, as is stated there, were aware of the problem.
Mr Beer: Yes, I’m focusing on the branch.
Michael Peach: Okay. Okay, I don’t think it’s clear from the wording whether the problem was not visible to the branch or whether the comment that would be inserted into the message would not be visible to the branch.
Mr Beer: Let’s take it in stages. Do you agree that there’s no record on this document of the branch being informed of the nature of the error –
Michael Peach: Yes, I agree –
Mr Beer: – the cause of the error and the way in which it’s going to be corrected?
Michael Peach: I agree that there’s no evidence in this document of that.
Mr Beer: From what you said, there should be, shouldn’t there?
Michael Peach: I would have expected there to be, yes.
Mr Beer: So there should be?
Michael Peach: Yes.
Mr Beer: Because, in your witness statement at paragraph 123, you told us this was the system and you showed us exhibited an OCP, which was evidence that the system was working?
Michael Peach: Correct.
Mr Beer: So this is evidence of something different than that, isn’t it?
Michael Peach: It’s evidence that – there is no evidence on this document which suggests that the process was followed fully.
Mr Beer: And, indeed, there’s some evidence to suggest that, if we just scroll up to “Justification”, the last line of “Justification”:
“By correcting the problem locally, the branch may not be aware of the problem …”
Michael Peach: I agree.
Mr Beer: Under “Extra detail”, second paragraph:
“The message will include a comment to show it has been inserted to resolve this problem (this will not be visible to the branch).”
Can you think of reasons for recording the fact that the branch will not be aware of the problem and the message to correct the problem will not be visible to the branch? Why would it be important to record those?
Michael Peach: I don’t have an explanation for that.
Mr Beer: Might it seem that they were positives: it’s a good thing that the branch won’t be told and can’t see?
Michael Peach: In my opinion, whether or not the branch would know that they had a problem is not a reason for not telling them. Did that makes sense? Was that …
Mr Beer: Yes, I understood it. But that’s an answer to a different question. I’m asking why somebody would record in two places on this OCP?
Michael Peach: I don’t have an explanation for that. In the second part where it’s under the “Extra detail”, the comment, “The message will include a comment to show it has been inserted”, was part of the standard procedure from the SSC. When inserting a message into a counter message store there would be an addition made to the Riposte message which said something along the lines of “inserted by SSC to resolve PEAK thing”.
So that second comment saying, “will include a comment to show it has been inserted to resolve this problem (this will not be visible to the branch)”, I would take to mean that that message inserted into the Riposte message would not be visible to the branch.
Mr Beer: Yes. Can we turn back to paragraph 141 of your witness statement, please, which is on page 45. It’s the bottom half of the page, thank you. You say:
“The purpose of the System Outline Design [this is a document I took you to earlier] seems to be to specify a tool set for different support units to enable them to continue to support the systems, and to be fully auditable. The System Outline Design resulted in the use of SSH software, which was fully auditable – I believe via the audit servers, which were not accessible by the SSC.”
In that last sentence there, you say that the SSH software was fully auditable?
Michael Peach: Yes.
Mr Beer: In what respect or respects was it fully auditable?
Michael Peach: The SSC would log in to the secure access servers in the data centres and that was the sole route down to counters. On the secure access servers, every keystroke that was typed on the SSC work station was recorded in a file and then I believe that file was sent down to the audit servers. So, effectively, every keystroke on every SSC work station was recorded.
Mr Beer: So there was a full keystroke record when members of the SSC used the SSH software?
Michael Peach: Yes, absolutely.
Mr Beer: You said that you believe it was sent down to audit; is that right?
Michael Peach: Yes.
Mr Beer: Automatically sent down to audit –
Michael Peach: Yes.
Mr Beer: – or periodically?
Michael Peach: I don’t know the exact mechanism. I just remember seeing a design document that said the files are held on the SAS servers and then transferred to audit.
Mr Beer: Was that audit trail ever examined, to your knowledge, ie to look at the keystrokes made by SSC staff?
Michael Peach: I know that SSC did not support the audit server and did not have direct access to it, so it would never have been viewed by SSC staff. Whether or not it was viewed by other staff, I have no knowledge.
Mr Beer: When you became aware that there were prosecutions and civil proceedings based on Horizon data –
Michael Peach: Yes.
Mr Beer: – were you aware of the SSH audit files ever being accessed for those purposes?
Michael Peach: My recollection is that I only ever knew of one case and that was the one which involved Anne Chambers, and I was not aware that the audit data was being used for prosecutions at all. Does that answer the question or is that …
Mr Beer: Well, if you were only ever aware in your 12 years of one case –
Michael Peach: One prosecution, yes.
Mr Beer: One civil proceeding?
Michael Peach: Yes.
Mr Beer: Were you aware of an individual called Andrew Dunks?
Michael Peach: Yes.
Mr Beer: Andy Dunks?
Michael Peach: Yes.
Mr Beer: What’s your recollection of where he worked?
Michael Peach: My recollection is that he worked in the Security team inside Customer Service.
Mr Beer: So the Customer Services POA Security Team?
Michael Peach: Yes.
Mr Beer: CSPOA Security. What did you understand his job function to be?
Michael Peach: I don’t recall knowing what his job function was.
Mr Beer: We understand that he was said to be the cryptographic key manager. Does that ring any bells?
Michael Peach: Yes. In addition to not having access to the audit server, SSC did not have access to a key management server, both of which, my understanding is, was controlled by the Security team. So Andy would have controlled the work of the key management applications on that server.
Mr Beer: How frequent was your contact with Mr Dunks?
Michael Peach: Difficult to say. My recollection says perhaps once a month.
Mr Beer: Were you aware that Mr Dunks had contacts with members of the SSC?
Michael Peach: Yes.
Mr Beer: What was the nature and content of such contact, the purpose of it?
Michael Peach: I don’t remember.
Mr Beer: How frequently would Mr Dunks be in contact with members of your team?
Michael Peach: That would be, to my recollection, once/twice a month.
Mr Beer: You didn’t know what they were talking about or exchanging emails or other communications about?
Michael Peach: Not that I recall. I recall that the key management server was kept in a locked room inside the secure area in the SSC and, therefore, whenever Andy had to do some work on that server somebody would have to let him into the secure area.
Mr Beer: We’ve heard from Mr Dunks that he produced written witness statements, so evidence –
Michael Peach: Right –
Mr Beer: – in written witness statements and exhibits to those witness statements, for the purposes of taking criminal proceedings against subpostmasters. Do you understand?
Michael Peach: I understand.
Mr Beer: Did you know, in your decade or so of working in the SSC, that that was part of his job?
Michael Peach: I knew that there was a function inside the Security team which was litigation support. I don’t recall ever associating that function with Andy Dunks.
Mr Beer: What did you understand litigation support did?
Michael Peach: My understanding of that, which is very limited, was that they were there to, in my mind, protect Customer Service from possible litigation from outside. I was not aware that they were acting in prosecutions of postmasters.
Mr Beer: So you didn’t know they were supporting litigation, rather than defending against litigation?
Michael Peach: I don’t think I ever thought of it in those terms.
Mr Beer: In any event, we’ve heard from Mr Dunks that he produced witness evidence and exhibits for the purposes of criminal proceedings against many subpostmasters and mistresses. You didn’t know that that was his job or part of his job?
Michael Peach: Not that I can recall.
Mr Beer: I think it follows that you wouldn’t know why he, the Crypto Key Manager, had been selected to be the witness that produced evidence against subpostmasters?
Michael Peach: Him specifically, no, but he would have been one of the few people that had access to the audit servers, so, as a function of the Security team, I can understand it but I would not have associated it with one individual.
Mr Beer: You referred to one of the few people that would have had access to the audit servers.
Michael Peach: Yes.
Mr Beer: What are you referring to as the audit servers there?
Michael Peach: The audit servers were holding data from the system which, I believe, included data from Riposte and from all the SSC workstations. I didn’t really get involved with what the function of that server was because SSC were not allowed to touch it and we didn’t support it.
Mr Beer: I think it follows that you didn’t know that Mr Dunks was providing witness statements for the purposes of prosecutions that made assertions, the witness statements, that were, in part, based on conversations that he was having with members of your team.
Michael Peach: No, I don’t recall anything of that sort.
Mr Beer: He told the Inquiry that when he received a request for evidence, he would speak to a member of your team:
“… to get them to give a clear understanding so I could make my judgement on that particular call.”
So he was making a judgement on whether the content of a call made by a subpostmaster or a mistress could or could not explain the shortfall for which the subpostmaster was being prosecuted. Do you understand?
Michael Peach: I understand.
Mr Beer: He called this his due diligence exercise, that he was speaking to members of your team to help to get their help in explaining what calls meant and whether or not the content of the call could explain away the shortfall on which the subpostmaster was being prosecuted. Understand?
Michael Peach: I understand. I understand why Andy would have come to members of the SSC for technical advice on a call and what the Riposte messages meant. I don’t recall ever being aware that that was going to be used in any form of litigation.
Mr Beer: Why would you known that he would be coming to members of your team to ask for an explanation of what the content of calls meant?
Michael Peach: Because they were the technical expert on the contents of the calls.
Mr Beer: Do you know why they weren’t being asked to provide evidence on the basis of the technical expertise that they had of what had happened, rather than Mr Dunks who performed a different function, Crypto Key Manager, being asked to provide witness statements on the basis of unrecorded and undocumented conversations with members of your staff?
Michael Peach: No. As I said earlier on, during my time as SSC manager I was only aware of the one case.
Mr Beer: So this was going on below the surface without you ever knowing about it?
Michael Peach: I cannot recall ever knowing about it and I’m not certain that the SSC staff members would have been aware of why they were being asked about the calls. We were completely open with anybody about what is the impact of this PEAK, what’s happening with it. So I’m – no, I’m in the dark as to much of this process.
Mr Beer: By that last answer, are you suggesting that Mr Dunks may not have disclosed to members of your staff the purpose of his call or the use to which the information that he may be given might be put?
Michael Peach: I am not certain that, whatever was being – involved in the discussion between Andy and SSC staff, that I was ever aware of the use. I don’t wish to ascribe responsibility to that to Andy Dunks not telling SSC staff or SSC staff not telling me. I just don’t think the subject came up.
Mr Beer: If you had been aware that Mr Dunks was conducting what he described as a due diligence exercise, in deciding whether or not the call or calls and the content of the call or calls to the SSC could possibly explain away the shortfall for which a subpostmaster was being prosecuted, presumably you would have looked askance at that?
Michael Peach: I don’t know is the honest answer to that. That’s me trying to predict emotions from a long time ago.
Mr Beer: Would you have been happy with your staff providing evidence informally in this way that was being used to prosecute subpostmasters?
Michael Peach: I don’t think that I would have been happy about it but I can’t be certain.
Mr Beer: Why do you think you probably would have been unhappy?
Michael Peach: Because my understanding, limited as it was, of any form of litigation process was that all of the data had to come from the audit servers and that is specifically why the SSC were never to touch the audit servers, so that it was completely untouched by those people who had write access to the parts of the system.
If I would have known that evidence was being gathered from elsewhere, then I think in my mind that would have put in question the origin of the data being used in a case.
Mr Beer: Thank you. That document can come down from the screen now.
You’ve mentioned the Lee Castleton case being your sole experience of data from the Horizon System being used in legal proceedings involving a subpostmaster.
Michael Peach: Yes.
Mr Beer: Can we turn to paragraph 47 of your witness statement, please – sorry, page 47, and look at paragraph 147. In paragraph 147, under “Conduct of Prosecutions”, you say:
“I was not involved in the case of POL v Lee Castleton, and I did not know of this case before receiving the Request.”
Can I just understand what you meant by that sentence there, because the “Request”, capital “R” – I am not going to take you right back to it but right at the beginning of the statement you define “Request” as meaning the Rule 9 request that we sent you in January this year.
Michael Peach: That was –
Mr Beer: That can’t be right, can it?
Michael Peach: When I wrote my witness statement, I was asked a specific question: was I involved in the case of POL v Lee Castleton?
Mr Beer: Yes.
Michael Peach: And I said no because, at the time, I did not know that that was the case in which Anne Chambers was involved. It’s a question of terminology. I didn’t know that that was the name of the case. All I knew was that Anne Chambers had had to go to court for a prosecution. Does that …
Mr Beer: So, essentially, what you mean by paragraph 147 is two things, “I was not involved in the case which Anne Chambers was involved in, which I now know to be called Post Office v Lee Castleton”, full stop?
Michael Peach: That’s correct.
Mr Beer: Secondly, “I did not know that the case in where Anne Chambers was involved was called POL v Lee Castleton”?
Michael Peach: Correct.
Mr Beer: Understood.
If we look over the page, please, you set out from paragraph 153 down to 156 your involvement in the case that you now know to be the Castleton case, yes?
Michael Peach: Yes.
Mr Beer: In paragraph 153, you say:
“In this particular case, the person at Fujitsu who was originally responsible/going to give evidence at court declined to go. I cannot recall who this person was or why they declined. My recollection is that Brian Pinder was the Customer Service manager of the security team at the time, and I believe it would have been his responsibility to perform this task within his team.”
So the way you’re describing it there was that there was originally a person within Fujitsu who was going to give evidence at court and they declined.
Michael Peach: That was my impression at the time, yes.
Mr Beer: Can you recall why they, that person, were originally selected to give evidence?
Michael Peach: No.
Mr Beer: Can you help us with why they declined to give evidence?
Michael Peach: No. I don’t know that I was ever told. As far as my recollection, Anne was, to my belief, pressured to go to court. I believed that that was a function of the Security team.
Mr Beer: Sorry, just stopping there, the function of pressurising her was the function of the Security team or the function that she stood in for was their function?
Michael Peach: The function that she stood in for. I believed that she was being pressured to go to court because the person in the security team was not going to go.
Mr Beer: Can you help us with – I’m going to press you on this – why that person declined to go to court?
Michael Peach: No, I don’t know and I’m not sure that I was ever told.
Mr Beer: Who told you that they had declined to go to court?
Michael Peach: I think that that came out in an argument and I was having the argument with one of three people but I don’t recall which one. Specifically, I think Dave Baldwin was the CS director at the time, Naomi Elliot, I believe to have been the Support Services Manager, and Brian Pinder was the head of the Security team.
Mr Beer: So you had an argument with one of those three people?
Michael Peach: Yes.
Mr Beer: Where was the argument?
Michael Peach: Probably in a corridor.
Mr Beer: Did they, the Security team, work in the same building as you?
Michael Peach: They did.
Mr Beer: On the same floor?
Michael Peach: No, they were, I think, 5th floor. SSC were 6th floor.
Mr Beer: And it was in the course of that argument that you learnt that the person who was originally responsible had declined to go to court?
Michael Peach: That was certainly the impression I got. I don’t know if it was specified in those terms. I can’t – I obviously can’t remember which one of the three people I was having an argument with, so I certainly can’t remember the exact form of words that were spoken.
Mr Beer: In the third sentence there – so, second sentence you say you can’t recall who this person was or why they declined. That’s to go to court?
Michael Peach: Correct.
Mr Beer: The third sentence, you say:
“My recollection is that Brian Pinder … it would have been his responsibility to perform this task within his team.”
By that, are you saying that it ordinarily would be Brian Pinder’s job to go to court to perform this task?
Michael Peach: No, I am saying that Brian Pinder managed the team within which I believed this task should have been done.
Mr Beer: You say there “I believe it would have been his responsibility to perform this task”.
Michael Peach: Yes.
Mr Beer: You’re only talking about going to court in that paragraph.
Michael Peach: Yes.
Mr Beer: Is that section of the statement incorrect then? That gives the impression, does it not, that your belief was that it was Brian Pinder’s responsibility ordinarily to perform the task of going to court?
Michael Peach: No. Perhaps it would be clearer if you read the last part as being “I believe it would have been his responsibility to perform this task from within his team”.
Mr Beer: Or “I believe it would have been the responsibility of a person within his team”?
Michael Peach: Correct.
Mr Beer: Rather than it would have been his responsibility?
Michael Peach: Yes.
Mr Beer: You weren’t intending to say, by this paragraph, that it was Brian Pinder’s job to go to court and he had declined to do so?
Michael Peach: No, I wasn’t intending to say that.
Mr Beer: What was the argument about then?
Michael Peach: The principle of sending an SSC person to court or producing a witness statement.
Mr Beer: Why was that a principle that you were fighting for or against, the idea that somebody should go to court?
Michael Peach: A number of reasons. Firstly, nobody in the SSC was trained to do presentations, certainly not trained in court etiquette or court procedures.
Secondly, it’s an open-ended commitment for somebody to go to court, which means that I was going to lose one of my most skilled diagnosticians for an unspecified period of time.
Thirdly, on a purely personal level, she was clearly being very stressed by it. I wanted to make sure that that did not happen to any of my staff in the future.
Mr Beer: You say in paragraph 154:
“I was instructed by the Director of Customer Services … whose name I cannot recall, to detail someone from the SSC to go to court to explain the workings of the message store. I strongly objected that nobody in the SSC had any experience of courts, or was legally trained. I was overruled.”
I think you just named the Director of Customer Services at that time as Dave Baldwin; is that right?
Michael Peach: My recollection – and my timescales may be off – at one time Dave Baldwin was Director of Customer Service and Naomi Elliot reported to him. At a different time, Naomi was herself the Director of Customer Services. I can’t be precise about the timescales because I don’t remember.
Mr Beer: You don’t know whether that Director of Customer Services was at the time Mr Baldwin or Ms Elliot?
Michael Peach: That’s correct.
Mr Beer: You tell us in paragraph 155, if we just scroll down, that, essentially, it was up to you to choose somebody from the SSC to give evidence in the case against Mr Castleton. You had a free choice?
Michael Peach: That is my recollection.
Mr Beer: Was the choice not informed or dictated by the fact that Anne Chambers was the one who had dealt with the relevant PEAK arising from Mr Castleton’s calls?
Michael Peach: I almost certainly considered that, yes.
Mr Beer: Ie you picked the person who knew about the call that was going to be relevant in evidence?
Michael Peach: I’m fairly certain that that would have been one of the criteria that I used to pick her, yes.
Mr Beer: In this paragraph, you say you picked her because she was the most experienced and technically best in the area of counter code. You had confidence in her honesty and integrity –
Michael Peach: Yes.
Mr Beer: – and she wouldn’t be rattled?
Michael Peach: Yes.
Mr Beer: Rather than “I picked her because she was the one that knew about the call”?
Michael Peach: I think that I had forgotten that she was involved in the original call until reading some of the more recent documents that the Inquiry have sent to me.
Mr Beer: In your discussion with the Director of Customer Services, was there any discussion about whether the witness would be giving evidence as an expert witness or as a witness of fact of what had gone on in the call?
Michael Peach: I don’t recall the conversation, sorry.
Mr Beer: Do you understand the distinction that I’ve just made?
Michael Peach: I believe so.
Mr Beer: What do you understand the distinction to be?
Michael Peach: Sorry, can you go through the terms again?
Mr Beer: Yes. I asked whether there was a discussion over whether the person giving evidence would give evidence as an expert witness or a witness of fact of what had gone on in the call.
Michael Peach: In that case, the correct answer is, no, I don’t understand the difference between those.
Mr Beer: Was there any discussion between you and Anne Chambers, therefore, over the basis on which she was going to give evidence, what she was going to give evidence about, the limitations of it?
Michael Peach: My understanding was that she was going to give evidence on the factual basis of the Riposte message store.
Mr Beer: Were you told, as part of the Director of Customer Services’ persuasion or overruling you, that the Post Office was treating the Castleton case as something of a test case and was going to use it, if it won, to try and discourage other postmasters from either bringing cases against the Post Office or defending them?
Michael Peach: I don’t recall that being in any way part of the discussion and I don’t think that I knew or became aware of those implications until I received documents from this Inquiry.
Mr Beer: So you weren’t aware that, for the Post Office, it was judged that to be the case that a lot was riding on this?
Michael Peach: No.
Mr Beer: Can we look at some documents, please. Firstly, POL00099397. Thank you.
This is an email exchange in 2013, so many, many years later, after you had left Fujitsu, and it’s an exchange in which you are not involved, therefore, but there’s something in it that I want to ask you about.
Can we look at the bottom of page 1 and the top of page 2, please. You can see an email from Mr Parker to Mr Winn, yes?
Michael Peach: Yes.
Mr Beer: Then if we scroll down, please, in the third paragraph, it’s the second paragraph on this page, Mr Parker says:
“The litigation bit [that’s referring to an earlier exchange in a chain that I’m not going to take you to] is all to do with chain of evidence for prosecutions and delivery in court. I’m sensitive about it because in the distant past one of my team was ‘persuaded’ (by our side not yours) [that means by Fujitsu, not the Post Office, in context] to write an evidence statement without fully understanding the implications. As you know, our ‘professional witness’ for these types of cases is Gareth Jenkins but in this case, because process was not followed, Gareth couldn’t do it and preparation for court became very difficult.”
Firstly, do you understand what the process to which Mr Parker is referring there ought to have been where he says “process was not followed”?
Michael Peach: Not really. If process wasn’t followed – since the Castleton case was the first one that I had come across, I’m not sure I would have known what the process being followed by the Security team was.
Mr Beer: And, therefore, you wouldn’t know in what respect it hadn’t been followed?
Michael Peach: Correct, except clearly one of my staff was going to end up in court when I did not believe that was appropriate.
Mr Beer: It says, Mr Parker’s email, that, because the process was not followed, Mr Jenkins couldn’t give evidence. Was that said to you back in 2006?
Michael Peach: Not that I can recall.
Mr Beer: Can you think why a process not being followed meant that Mr Jenkins could not give the evidence?
Michael Peach: No, I don’t think that I was even aware at that time that Gareth was the nominated person in 2006 to give evidence.
Mr Beer: What was your knowledge of Mr Parker’s involvement in these events?
Michael Peach: Steve Parker would have been involved as the SSC manager –
Mr Beer: Back in 2006 I’m talking about. I’ve asked Mr Parker about this already and he said “You’d better ask Mik about it”?
Michael Peach: I think Steve was involved because in December 2006 I was on honeymoon, so he was in charge of the SSC.
Mr Beer: In your statement, you’ve told us that it was you that had the conversation with the Customer Services Director –
Michael Peach: Yes.
Mr Beer: – and it was you that persuaded Ms Chambers to give evidence?
Michael Peach: Yes.
Mr Beer: So those things didn’t happen whilst you were on honeymoon?
Michael Peach: No.
Mr Beer: So what did Mr Parker do, then, outside of the conversation that you had with Customer Services and outside of the conversation persuading Anne Chambers to give evidence when he was deputising for you?
Michael Peach: I’m sorry, perhaps I’m being dense. I’m not understanding the question.
Mr Beer: I’m trying to work out what Mr Parker’s involvement was, what his footprint was on this issue back in 2006. Did he have any involvement in it at all, to your knowledge?
Michael Peach: Only if I handed over what had been going own at the time that I was going to be away from the office.
Mr Beer: Can you recall whether now you had handed over this issue to him?
Michael Peach: No, I can’t recall if it was an extant issue when I going on – I would have briefed him along with all the other things that were going on in the SSC before I went on leave, but I cannot recall this specifically being mentioned.
Mr Beer: Can we look, please, at FUJ00152300. I’m about to show you a couple of documents now that we very recently received from Fujitsu, over the weekend, I think. If we look at the foot of the page, please, an email exchange between you, Brian Pinder and Naomi Elliot, copied to Anne Chambers of 29 January 2007.
Michael Peach: Right.
Mr Beer: So just to locate that in time, this is after Anne Chambers had given evidence –
Michael Peach: Yes.
Mr Beer: – but I think before judgment. You say:
“Brian,
“I understand from Anne that you do not intend to have an internal review on the Castleton case.”
Why would there need to be an internal review on the Castleton case?
Michael Peach: Because Anne was concerned about the process.
Mr Beer: So it wasn’t a review of the case as a whole, to your mind; it was a process by which Mrs Chambers had come to give evidence?
Michael Peach: And her concerns with the evidence that she’d given.
Mr Beer: “Nevertheless, we are concerned that POA made some errors during the course of this case which could prove critical in any future litigation.”
The reference to POA there, is that a reference to the Post Office Account within Fujitsu, not a reference to the Post Office?
Michael Peach: I would read it as the Post Office Account within Fujitsu.
Mr Beer: So that should read, essentially, “We are concerned that part of Fujitsu made some errors during the course of this case”?
Michael Peach: That’s how I would read that, yes.
Mr Beer: What errors did you think that part of Fujitsu had made in the course of the Castleton case?
Michael Peach: I don’t recall. I only saw this document just before we came in this morning. I believe that there is also another document in which Anne makes her concerns clear.
Mr Beer: You refer to that in your next paragraph. You say:
“… Anne has written up her thoughts and comments [which are attached], and I would welcome your comments.”
The subject line of this being a “‘Mop up’ on the Castleton case”.
Michael Peach: Yes.
Mr Beer: Let’s look at the paper that was attached to your email then.
Michael Peach: Okay.
Mr Beer: FUJ00152299. Thank you very much.
You will see that this is the paper that was attached to that email.
Michael Peach: Yes.
Mr Beer: It was written by Mrs Chambers, if you just scan, please, both pages of the document at the same time or put both up at the same time, you can see its length, date and authorship.
So you can see it’s got four headings. It’s signed off by Anne Chambers on 29 January 2007. That was the date of your email sent at 11.34 that morning.
Michael Peach: Yes.
Mr Beer: It’s headed “Afterthoughts on the Castleton case”. If we can just go through this newly disclosed document, please, starting on paragraph 1 or heading 1 at the top, “Approach to SSC staff”. Maybe if that can be blown up for those that are looking online. She says:
“In the summer of 2006 I was asked directly by the Security Manager whether I would be prepared to speak to a solicitor about a call I had dealt with in February 2004. My initial response was that this was not the normal process … he reassured me that it was more or less a formality so somewhat reluctantly I agreed.”
You will see there that Anne Chambers has it down more contemporaneously with events that it was she that was asked directly by the Security Manager, whereas the way you’ve described it is that it was you that asked her somewhat reluctantly, following a request from Security and a row.
Michael Peach: I agree that’s way it reads.
Mr Beer: Does reading this more contemporaneous document help you to remember how matters, in fact, unfolded?
Michael Peach: My memory is probably coloured by the fact that I regarded it as my responsibility because she was one of my staff; so I may well have remembered that I persuaded her when she had, in fact, been contacted by the Security Manager. Her memory is actually considerably better than mine so I would defer to her.
Mr Beer: She says that she was asked directly by the security manager. Who would that be?
Michael Peach: I suspect that, at this time, it would have been Brian Pinder.
Mr Beer: So she records being asked directly by the somebody who you think would likely be Brian Pinder and the request was to speak to a solicitor about a call that she dealt with back in 2004. Mr Pinder “reassured me it was more or less a formality”, so she reluctantly agreed, and then she said:
“Subsequently, before the meeting with the solicitor, he asked me what my availability was in the autumn for the court case. This was the first time there was any mention of the possibility of having to go to court. Repeated assurances that this would all be settled before getting to court proved to be unfounded.
“I appreciate that there may be circumstances where witnesses are summoned and have no option but to comply, but I was not at all happy about how this was handled.”
Does this jog a memory in you in one of the elements of unhappiness, the reassurance you’re not going to be required but, in the event, having to go to court?
Michael Peach: I cannot recall precisely but I would suspect that this reaction from Anne formed the basis of the argument which I had with one of the three people that I had the argument with.
Mr Beer: Paragraph 2 or section 2, please, “Review of technical evidence”:
“When I took the initial call in February 2004, I spent only a few hours on it before deciding that I could not see any sign of a system problem. I only looked at a couple of weeks’ information.
“While in this case I am now sure that I did not miss anything and my initial analysis was correct, I am concerned that there was no technical review of the Horizon [System] between the original call and the case going to court. It is probable that any system problem affecting the accounts would have shown up to Post Office staff who did check the all the figures very carefully, but since the subpostmaster was blaming the system for the losses I think it would have been sensible to have double checked this within Fujitsu before it got as far as court. I was certainly concerned, in the early stages, that there might be something I had missed.”
Just dealing with that paragraph first, were you aware that between a call coming in about a shortfall or a discrepancy and any action being taken against the subpostmaster, there was no what she described as technical review of Horizon evidence conducted by SSC staff or by anyone?
Michael Peach: That’s difficult to say because this was the only case that I knew of.
Mr Beer: What did you think when you read this?
Michael Peach: I cannot recall reading this until I was presented with it this morning.
Mr Beer: Would you, reading it now, recognise this as being a serious issue, serious because one is moving from an examination of a fault or error in the context of a Helpdesk call, essentially –
Michael Peach: Yes.
Mr Beer: – and then jumping into a prosecution or civil proceedings without any intervening review?
Michael Peach: I would agree. Yes, I would be concerned.
Mr Beer: Albeit Mrs Chambers saying, “In the event, I don’t think in this case it was a problem because my view is that I didn’t miss anything”?
Michael Peach: I understand. I agree with her position in both respects. If she said that she’s reviewed it since and she didn’t miss anything, then I have absolute confidence that that is the case. Her concerns about the process I can only agree with.
Mr Beer: It’s one thing answering a Helpdesk call amongst a stack of tickets.
Michael Peach: Yes.
Mr Beer: It’s quite another thing being used as a witness to speak, as she does in her next paragraph, being treated as an expert witness in answering a wide variety of calls about the system?
Michael Peach: Yes, I agree.
Mr Beer: You see that she says:
“… I found myself being treated as an expert witness and answering a wide variety of questions about the system, although nominally I was a witness of fact and my witness statement covered just the investigation done in 2004. Fortunately I do have extensive knowledge of the system and was able to fulfil the wider role – but what would have happened if the initial call had been handled by a less experienced SSC person?
“If there is a similar case in the future, where the system is being blamed, would it not be sensible to have a technical review of all the evidence, at the first indication that a case may be going to court? Someone involved in that review would then be well placed to give evidence in court.”
Just dealing with those questions that she asked – in a future case where somebody is blaming the system, wouldn’t it be sensible to have a technical review of all of the evidence – what was done as a result of that question being asked?
Michael Peach: I don’t know.
Mr Beer: We’re 2006 here, end of 2006/beginning of 2007 and there are many prosecutions that follow this. Can you help us what happened to the, on the face of it, not unreasonable question that Mrs Chambers is raising?
Michael Peach: I agree that the question is entirely understandable. I know that, at some point, Gareth Jenkins took over the responsibility for doing technical presentations. My recollection is that, even between 2006 and the time that I left, I was not aware of any other prosecutions.
Mr Beer: Just going back to the email that you got in response to this – we’ll come back to the document in a moment – that’s FUJ00152300, look at the response from Mr Pinder at the top, back to the same distribution list, ie you, Mrs Chambers and Ms Elliot:
“Mik Anne
“Thanks Mik, there was no intention to have a wash up on this particular case as such but I must stress that from the outset this was ‘new ground’ and a particularly unusual case (1st of its kind in 10 years) for all concerned. It involved many … variables which, at any point in time could have culminated in a totally different outcome.
“This enquiry took well over a year to conclude and routine procedures which have served us well for 10 years were suddenly being stretched to new limits, but it does highlight how [the Post Office Account] can be called to account and I totally agree we must learn from this.
“Anne (many thanks for your comments) you have highlighted some interesting areas of procedure which we need to recognise, and I will discuss these with Naomi and will keep you both informed.”
On one view, that reads as sort of a pat on the head where nothing much is going to happen or am I being unfair?
Michael Peach: No, I would agree with you.
Mr Beer: Did anything happen?
Michael Peach: I don’t recall this email –
Mr Beer: What about the substance of the issue then? Do you remember anything happening? He says he’s going to keep you both informed.
Michael Peach: I don’t have any recollection that I was kept informed of any progress on those issues.
Mr Beer: Can we go back then to Mrs Chambers’ document FUJ00152299 and look at the foot of the page “Disclosure of evidence”:
“Fujitsu made a major legal blunder by not disclosing all the relevant evidence that was in existence. I found myself in the invidious position of being aware that some information (Tivoli event logs) existed, but not sure whether they had been disclosed or not, since I had not been party to any of the requests for disclosure. It became evident in court that they had not been disclosed.
“Quoting from an email received from POL’s solicitor after my revelation …”
Then there’s a quote from an email. For the Core Participants who are aficionados in this area, if they want to look at that email it’s POL00070104. Anyway, it reads:
“‘In any litigation, the parties involved have a continuing obligation pursuant to the Court rules to disclose all documents that may help or hinder their case or the other side’s case. In this context, a “document” means anything in which information of any description is recorded, so it includes, just for example, a computer database. Previously, I had asked Fujitsu to let me have all the info it had and had been helpfully given HSH call logs, transaction logs and event logs. I was also recently told that there was a message store which had everything else on it and we invited Mr Castleton to look at this, but he didn’t take up the opportunity.’”
She continues:
“This suggests that disclosure of the message store itself was an afterthought, though it is fundamental to the system. I know that for fraud cases the ‘transaction log’ and ‘event log’ are extracted from the … message store and submitted, but surely the full message store has to be disclosed in all cases?
“Many other files are also archived to the audit servers as a matter of course and could hold relevant information, although the Security team are not necessarily aware of their existence or potential relevance. I’d like to suggest that a list of these files is compiled so that similar mistakes are not made in the future.
“And what about calls on PEAK, which may have evidence attached? And any evidence which might have been kept within SSC? I was not asked whether I had anything that might have been relevant (as it happens, in this case I did not).
“Of course there may be subtleties to this that I am unaware of, whereby data may exist but there is no obligation to disclose it. If this is the case, could any future witnesses be briefed appropriately? The response ‘no-one has ever asked for that before’ does not seem to be a good reason for non-disclosure.”
Would you agree that Mrs Chambers is raising there a series of reasonable, focused and pertinent questions –
Michael Peach: Yes.
Mr Beer: – against the context of her saying that Fujitsu, the company that you work for, had made a major legal blunder by non-disclosure of evidence?
Michael Peach: I’m not sure that I would have categorised it in that way but the implications of what she is saying I would certainly agree with.
Mr Beer: Why wouldn’t you categorise the non-disclosure as a major legal blunder?
Michael Peach: Because I’m not sure what constitutes a major legal blunder. I agree that if these documents were not presented at litigation, then that was an error.
Mr Beer: The series of questions that she asks (for example, surely the message store has got to be disclosed on all cases), did that happen thereafter?
Michael Peach: This was the only court case that I knew about, so I have no idea.
Mr Beer: What did you – she refers there in the second paragraph down:
“I know for fraud cases the ‘transaction log’ and ‘event log’ are extracted from the message stores but surely the full message store has to be disclosed.”
That’s telling you that there’s another species of case, isn’t it?
Michael Peach: It is but I don’t recall seeing this document until this morning and I don’t recall being aware that there were any other cases.
Mr Beer: So you weren’t aware that members of your team were speaking to Mr Dunks to help him compile witness statements to prosecute people?
Michael Peach: That’s correct.
Mr Beer: You weren’t aware then of your team’s indirect involvement in the prosecution of subpostmasters?
Michael Peach: Not that I can recall.
Mr Beer: You got this document. It was sent to you by Anne Chambers and indeed you sent it on to Customer Services, Customer Support, to Mr Pinder –
Michael Peach: Yes.
Mr Beer: – and Ms Elliot?
Michael Peach: Yes.
Mr Beer: Can you recall what happened as a result of these reasonable, focused and pertinent questions being asked?
Michael Peach: No, I cannot.
Mr Beer: As a manager, you would want to grip a situation like this, wouldn’t you?
Michael Peach: I agree.
Mr Beer: Can you help us what you did do to grip it then?
Michael Peach: No. I’m sorry, I’ve no recollection of this.
Mr Beer: Can we move down to paragraph 4, please:
“This case highlighted a common problem, both in 2004 and now. The postmaster raised many calls about his continuing losses, both with Horizon and with the NBSC. These kept being bounced [back] and it took weeks before a call was passed to SSC.”
Is that accurately describing a common problem; namely a subpostmaster continually raising calls about continuing losses which were bounced back or were being bounced at the lower levels of Customer Support?
Michael Peach: It’s not something that I recognise. As we discussed earlier, the SSC was supposed to receive the first instance of a new incident. Subsequent incidents would become a problem, rather than an incident, and would be handled through the problem management process. So if there were multiple calls on a single problem, then the SSC would receive the first. Subsequent ones would be referred to the problem managers by HSH and SMC.
Whether or not calls were being bounced around between Horizon and the NBSC, I wouldn’t know.
Mr Beer: This is telling you that it is a problem and it’s a common problem.
Michael Peach: I agree, in which case Anne must have actually researched that to find that.
Mr Beer: Again, that needs something to be done about it, doesn’t it?
Michael Peach: I was not in the – when I was in the SSC, I was not involved with any relationship between the Horizon Helpdesk and the NBSC, so I can’t comment on that.
Mr Beer: But your staff were.
Michael Peach: In the normal course of events, no, they weren’t.
Mr Beer: How was Mrs Chambers able to write this then? She’s got some knowledge.
Michael Peach: She researched – presumably researched it having recognised that there was an issue.
Mr Beer: And that it’s a common one?
Michael Peach: She may have deduced that from the evidence that she’d got having researched it.
Mr Beer: That’s what she’s saying: it’s a common problem?
Michael Peach: I agree that’s what she’s saying.
Mr Beer: So what would you want to do, receiving this?
Michael Peach: I would have wanted to know how big a problem this was.
Mr Beer: It’s a common one.
Michael Peach: That’s not very specific. How I would deal with it now, I don’t know. How I dealt with it at the time that Anne brought this, I don’t remember.
Mr Beer: Mrs Chambers continues:
“Strictly speaking, problems with discrepancies do need to be investigated by NBSC in the first instance, but where there are continuing unresolved problems it should be possible to get the issue investigated properly, and one of the helpdesks should be prepared to take responsibility for the incident. Personally I think the fact that the Horizon Helpdesk is penalised for passing ‘Advice and Guidance’ type calls on to third line leads to too many calls being closed without proper investigation or resolution. This is very frustrating for postmasters, though possibly not an issue of concern to POL.”
She’s raising a systemic issue there, isn’t she?
Michael Peach: She’s raising what she believes to be an issue in process, which I believe was handled by the problem management process, not the incident management process.
Mr Beer: So she was somebody who was one of your most experienced and trusted diagnosticians, wasn’t she?
Michael Peach: Agreed.
Mr Beer: You had absolute faith in her competence and integrity?
Michael Peach: Yes.
Mr Beer: So what did you do with this problem that had been raised?
Michael Peach: I probably discussed it with her and went through the problem management process.
Mr Beer: What does “went through problem management processes” mean?
Michael Peach: Pointed out to her that the problem management process is the place where repeat issues should have been handled.
Mr Beer: Is that another form of a pat on the head?
Michael Peach: No, it’s me disagreeing with what she said.
Mr Beer: Sir, I wonder whether we could take the lunch break earlier today at 12.45 and come back at 1.45, please.
Sir Wyn Williams: Yes, by all means.
Mr Beer: Thank you, sir.
Sir Wyn Williams: There is just one question, I think, while it is in my mind, Mr Peach, and it relates to questions you were asked about 15 minutes ago now when you were looking at the events which led to Mrs Chambers actually giving evidence in the Castleton case, and then shortly after you answered questions about that, Mr Beer took you to an email much later, 2013, when Mr Parker had suggested, or at least it could be thought that Mr Parker was suggesting, that Gareth Jenkins was due to give evidence but there had been a process failure which meant that that couldn’t happen.
Now, the question I want to ask you is simply this: from your memory when you were speaking either to – well, when you were speaking to anyone about whether or not Mrs Chambers should give evidence, was the name “Gareth Jenkins” mentioned to you as someone who normally would give such evidence but couldn’t on this occasion?
Michael Peach: No, his – to my recollection, no, his name was not mentioned and the impression that I’ve got from the email from Brian Pinder was that this was the first occurrence of this sort of event and, therefore, I would not have expected there to have been a process which involved Gareth Jenkins.
Sir Wyn Williams: And I ask the question because, as Mr Beer pointed out to you, Mr Parker couldn’t really explain how the name “Gareth Jenkins” came to be written by him in this context in 2013 but suggested you might know the answer. But it looks as if we have the unsatisfactory position that neither of you can explain that reference to Mr Jenkins. Is that really where we are?
Michael Peach: That’s correct.
Sir Wyn Williams: All right. Does that mean we start at 1.50 instead of 1.45, Mr Beer?
Mr Beer: Yes, thank you, sir.
Sir Wyn Williams: All right, fine.
(12.45 pm)
(Luncheon Adjournment)
(1.50 pm)
Mr Beer: Good afternoon, sir. Can you see and hear me?
Sir Wyn Williams: Yes, I can, thank you.
Mr Beer: Thank you very much. I said I was going to move to a new topic. I realised that I omitted to ask a few questions on our last one that I ought to ask now. Can we go back, please, to FUJ00152299. This was the Anne Chambers document of 29 January 2007, the two-page document; do you remember?
Michael Peach: Yes.
Mr Beer: I’ve taken you through individual parts of it and asked you questions about it. Can I take a step back and ask you to take a step back and look at the document as a whole.
Michael Peach: Yes.
Mr Beer: Would you agree that the document raises a series of fundamental issues about the way that evidence is presented to a court by Fujitsu in proceedings against subpostmasters?
Michael Peach: I don’t think I can draw that inference from one case. I agree that what’s in this document means that, in this case, I would agree with Anne there were things that were missed, but I couldn’t extrapolate that into other cases.
Mr Beer: Did you ask, “What do we do in any other cases”?
Michael Peach: No, I did not.
Mr Beer: So this is your total sample size, yes?
Michael Peach: I agree.
Mr Beer: Why wouldn’t you ask, “Is this an outlier, do we do this in all cases? Do we do other cases at all?”
Michael Peach: I don’t recall this document and it was only presented to me this morning, so I haven’t had the opportunity to research whether or not perhaps I made comments in my monthly report. I just don’t know.
Mr Beer: So we should look, should we, to your monthly – we got this document on 12 May from Fujitsu –
Michael Peach: Right.
Mr Beer: – which is why you got it recently.
Michael Peach: Yes.
Mr Beer: If we are to see what action you took as a result of this, where should we look?
Michael Peach: I can infer, but not be certain, that having seen this document at the time that could have been when Gareth Jenkins started taking on the responsibility for litigation. Until this morning, I had always thought of Gareth as being a technical expert on one part of the system. I had seen him in the office and I knew that people deferred to him for expert advice. Whether or not his taking over on the litigation side was a result of actions that I took from seeing this from Anne, I just cannot say. I don’t remember.
Mr Beer: Just going back to my question then: where should we look for evidence of action that you took?
Michael Peach: Management meetings, discussions between the CS Director and Brian Pinder. That’s all I can suggest.
Mr Beer: Would you agree that this document, in general terms, standing back, calls for action to be taken?
Michael Peach: Yes, I would agree.
Mr Beer: Because it’s sending a warning, isn’t it? It’s raising red flags on a number of fronts to Fujitsu about the way that evidence is presented?
Michael Peach: Yes, it is.
Mr Beer: And I think you would probably agree that, in the light of what we now know, this was a harbinger for many of the events that we were subsequently to see, confusion over whether a witness was giving evidence as a witness of fact or an expert witness, non-disclosure of documents?
Michael Peach: I agree to that. As I said before, as far as I was aware, this was the only case that I knew of.
Mr Beer: Why was Anne Chambers raising it with you if it was the only case? Surely the memo itself contemplates that there are more.
Michael Peach: I agree to that too. It appears that the action that I took was certainly to forward her concerns to the Security Manager. What happened after that I have no memory of.
Mr Beer: Thank you. That document can come down.
Do you remember a man called Alvin Finch working in the SSC?
Michael Peach: I remember interviewing and subsequently offering a job to Alvin. I confess I had not remembered his surname until I saw him on the witness list.
Mr Beer: Do you remember how long he worked in the SSC for?
Michael Peach: I remember it being a very short time and that his leaving was not happy.
Mr Beer: Why was his leaving not happy?
Michael Peach: I went on leave. When I came back from leave, I was informed that he’d thrown his pass across the floor and announced to everybody that he was being bullied.
Mr Beer: He’s informed the Inquiry in his witness statement that he felt uncomfortable working at Fujitsu because he felt that there was a culture of fear, that he was bullied by a member of the staff and, therefore, walked out, and that in the SSC there was a culture, which he says came from the top, of prioritising short-term fixes over long-term solutions.
Taking those allegations in turn, did you know that he felt uncomfortable working within the SSC?
Michael Peach: No, I don’t recall any occasion when he raised that with me.
Mr Beer: That he felt that there was a culture of fear within the SSC?
Michael Peach: I would dispute that.
Mr Beer: Did you know that he said he was being bullied by a fellow member of staff?
Michael Peach: I knew that because it was reported to me when I got back from leave.
Mr Beer: And that there was, he says, a culture which came from the top of prioritising short-term fixes over long-term solutions?
Michael Peach: Within the SSC, our role was to generate workarounds for any incident if we could and then to pass the call to development for a code fix. The code fix prioritisation was done at the release management forum of which I was a member. I can see that it may be considered that from within the SSC prioritising a workaround in order to keep the system up and running was important but the generation of the code fix was equally important, but probably not visible or as visible to someone of Alvin’s level.
Mr Beer: Can I turn to the KEL system then, please. Mrs Chambers has told the Inquiry that an issue or concern with the KEL system was that service tickets would be passed to the SSC with the wrong KEL quoted on them. Were you aware of that problem?
Michael Peach: Yes, particularly in the early days and I had meetings with the SMC manager when it happened, initially, I recall, monthly, because the SMC’s filtration rate was low, later by exception, saying something along the lines of “SMC missed this or got the wrong KEL, why? What was wrong with it? Do we need to amend it? Do we need to amend your procedure?”
Mr Beer: What was the result of that?
Michael Peach: Frequently an update to the KEL. A lot depended on where the KEL had been written and by whom because technicians at third or indeed fourth line would specify a problem in technical terms, whereas what the postmaster was reporting was clearly something that was happening on a counter, and there were certainly occasions where it was not clear from the description of the problem from the postmaster that a particular KEL applied.
Mr Beer: Did the issue have an adverse impact on Fujitsu’s ability and POL’s ability to respond timeously and appropriately to complaints made by subpostmasters about Horizon?
Michael Peach: It would have had some impact purely because the call was being passed to the SSC with an incorrect KEL. It was then our responsibility to update the KEL so that that didn’t happen again. Did it make any difference to the way in which a specific incident was diagnosed? No, I don’t think so.
Mr Beer: Mrs Chambers also told the Inquiry that the SSC and fourth line support development did not always know how many branches had reported a particular problem because tickets never made their way up to the SSC or to the fourth line, and also because the breadth of a problem in unreported instances of the problem never made their way to the SSC because subpostmasters didn’t report them.
Were you aware of those two problems?
Michael Peach: When there were multiple instances of a particular incident, then the process for handling that and tracking the duplicates was part of the problem management process and, therefore, may not have been visible immediately to the SSC. However, there was frequent contact between the problem managers who were also the Service Delivery Managers responsible for each area. So if an incident was happening multiple times, then the problem manager would be dealing with it as a problem and he would talk to me and say “N” post offices have this problem.
Mr Beer: How would he find out what “N” equalled?
Michael Peach: From the HSH and SMC. They had a direct line through to the Service Delivery Managers. So duplicates and the trend analysis were part of the problem management process between the HSH and the problem manager. The SSC was tasked specifically with dealing with incidents, that being the first event of a new problem.
Mr Beer: How would a problem manager know or understand the breadth of a problem by reference to access to HSH databases?
Michael Peach: They would be the ones that knew how many post offices had called in with that problem.
Mr Beer: What if the post office didn’t know what the problem was?
Michael Peach: Sorry, I don’t –
Mr Beer: They say, for example, “I’ve just balanced and there’s a discrepancy. That’s all I know”.
Michael Peach: Okay. So the initial version of that would come to the SSC for us to do the diagnosis and, if necessary, organise a fix with development. Multiple post offices phoning in saying they had the same problem would be treated as a problem.
Mr Beer: So were they attributed a code by the HSH?
Michael Peach: I have no knowledge of exactly the way that HSH and SMC communicated that through to the problem managers. I know that the problem managers kept a database.
Mr Beer: What was that database called?
Michael Peach: I think it was just called the Problem Management Database.
Mr Beer: What was it populated with?
Michael Peach: I don’t know. It was populated by the problem managers.
Mr Beer: So if you had to describe top to bottom – or bottom to top, bottom-up, the system that Fujitsu employed to ensure that the nature, extent and breadth of a problem was established, how would you describe it?
Michael Peach: I would describe it as solely within the remit of the Service Delivery Managers who would interface to their counterparts in POL.
Mr Beer: How would the Service Delivery Managers know about these pockets of instances around the country?
Michael Peach: Because HSH would tell them.
Mr Beer: What would HSH tell them?
Michael Peach: That, I don’t know.
Mr Beer: Can we look, please, at FUJ00079939. You’ll see this is a 2005 document entitled “[Post Office Account] Customer Service Incident Management Process Details”.
Michael Peach: Yes.
Mr Beer: The “Abstract” says that the document describes the customer service incident management process and I think we’ll see that you’re one of the people to whom it was distributed.
Michael Peach: Yes.
Mr Beer: Can we look at page 9, please. The document says:
“The inputs to this process are:
“All Incidents reported by Contact with the [Post Office Account] Horizon Service Desk. Contact is defined as voice or Tivoli alert as the methods of communication with the Horizon Service Desk and fall into the following categories:
“Business process error.
“Hardware or software error.
“Request for information, eg progress of a previously reported Incident.
“Network Error.
“Severity and SLT information.
“Evidence of an Error.
“System Alerts received automatically from OMDB.”
So is this recording the fact that problems might be identified through a number of routes via a Tivoli alert, via a system alert or via contact with the Helpdesk itself?
Michael Peach: I would agree with everything in that statement except of “problem” instead of “incident”.
Mr Beer: So incident issues?
Michael Peach: Incidents, yes.
Mr Beer: You don’t like the word “issue”?
Michael Peach: I got used to using the ITIL nomenclature for identifying the difference between incidents and problems, which was a struggle for me at times. So you have to make a distinction on an incident between a problem – you have to make a distinction between an issue and an incident and a problem.
Mr Beer: So the problem managers weren’t really there to manage problems, they were only there to manage incidents; is that right?
Michael Peach: No, it’s exactly the reverse. The problem managers were there to manage problems. They were not there to manage incidents.
Mr Beer: What’s an incident?
Michael Peach: A single occurrence of an anomalous event somewhere in the system.
Mr Beer: Anomalous in what sense?
Michael Peach: Just something going wrong. Tivoli – of all those things in there. So Tivoli error messages being reported by the SMC, incidents being reported by postmasters, errors occurring on the central systems, they would all be an incident. If there were multiple instances of an incident, then HSH would forward that to the problem managers.
Mr Beer: What was the remit of the problem manager then?
Michael Peach: From personal experience, I don’t know. There is a document very similar to this one called the “Problem Management Process” which would detail that.
Mr Beer: Where did the problem managers sit? Where were they physically located?
Michael Peach: Physically on the 5th floor in Bracknell.
Mr Beer: So is this identifying incidents that may arise either because of automatic detection by a Fujitsu system or those which are reliant on a subpostmaster calling them in?
Michael Peach: Both of those, yes.
Mr Beer: If we look further down the page under “Dependencies”:
“The process is dependent on:
“Effective incident handling by the Service Desk.
“The Known Error Database being available and kept up to date with all errors as the root cause becomes known to Problem Management.”
Just stopping there, what was the difference between the Known Error Database and the Service Desk Knowledge Database?
Michael Peach: The Service Desk meaning HSH, in this case?
Mr Beer: Yes.
Michael Peach: HSH used the KEL system as did the SMC. The HSH, because, speaking from memory, 90 per cent of their calls were hardware-related, they kept an additional database which acted as their known error database for hardware problems.
Mr Beer: It was restricted, was it, to hardware problems?
Michael Peach: As far as I’m aware but the SSC never used it, so I can’t be certain.
Mr Beer: If we just go over the page, you’ll see that it’s described, in the next bullet point:
“Service Desk knowledge database (HSH ONE) …”
Is that what it was known as?
Michael Peach: Yes.
Mr Beer: It says it’s kept up to date with POL business and services knowledge. You’re saying, to your knowledge, that was only in relation to hardware issues?
Michael Peach: The only times that I ever came across it, it was relating to hardware problems but the SSC didn’t keep it, access it or maintain it, so its exact structure I couldn’t tell you.
Mr Beer: Whose responsibility was it to ensure that the content of the KEL database was consistent with the HSH one?
Michael Peach: I don’t recall.
Mr Beer: Was it anyone within the SSC’s responsibility?
Michael Peach: No. Because the SSC did not have access to HSH ONE.
Mr Beer: So would you think that it was the HSH’s responsibility then?
Michael Peach: I really couldn’t comment. I have no knowledge of the contents of HSH ONE.
Mr Beer: Whose role was it to ensure that KELs were communicated effectively to HSH members?
Michael Peach: SSC.
Mr Beer: How was that done?
Michael Peach: We made sure the database was there, we made sure that HSH and SMC, which in this context we would treat as one unit, made sure that they understood it, had meetings with their manager to ensure that they were able to use it. SSC made sure that the server that contained it was up and running all the time.
Mr Beer: So maintaining the content and continuity of operation of KEL?
Michael Peach: Yes.
Mr Beer: Was there a system in place to see how consistently HSH were using the KEL database?
Michael Peach: Initially there were monthly reviews between myself and the SMC manager, who was my interface. After the filtration rates of the SMC rose sufficiently, then that was done on an ad hoc basis.
Mr Beer: What do you mean by that?
Michael Peach: I mean, if SSC staff said, “SMC have raised this KEL and we don’t agree with it”, then I would go back to the SMC manager and say, “Why is this, what’s happening? Do we need to update the KEL?”
Mr Beer: Can we go to page 15, please and look that foot of the page at paragraph 2.5. The document records that:
“If the incident is not routine, the Service Desk agent checks for Known Errors listed in HSH ONE and the SSC KEL against records relating to the incident symptoms. If a match is found, the agent informs the caller of the workaround or resolution available and links the call to the master incident record.”
Would you agree that, quite aside from the importance of the KEL, the content of the HSH ONE knowledge database is key to how and whether a problem was escalated or resolved properly?
Michael Peach: Yes, and I would agree with that statement.
Mr Beer: If we go on over the page, please, to paragraph 2.7, the document records that:
“If no match is made against the Problem Database …”
The “Problem Database”, is that referring to something other than the KEL and the HSH ONE?
Michael Peach: That’s how I would read it.
Mr Beer: Is that the thing that you were describing earlier?
Michael Peach: Yes, I think so.
Mr Beer: “If no match is made against the Problem Database, the Service Desk continues with first line resolution of the Incident assisted by the Product Support Engineers (PSEs)”.
So that’s if there’s no match on HSH ONE, no match on KEL and no match on problem database, the incidents refer back down to first line support; is that right?
Michael Peach: I’m not sure who Product Support Engineers are in this term.
Mr Beer: Are they people that are concerned with hardware?
Michael Peach: Reading the next paragraph I would read it as both, since the SDUs are Service Delivery Units that would include the hardware and the SSC.
Mr Beer: You have to explain that answer in a bit more detail. Just sticking with 2.7 for the moment –
Michael Peach: Okay.
Mr Beer: – if there’s no match of the incident in HSH ONE or KEL and no match on the problem database, the incident gets sent back to first line resolution, yes?
Michael Peach: The way I was reading this was that we were currently in the first line when they were checking HSH ONE and the SSC KEL. So the Product Support Engineers I would read, as being part of the HSH.
Mr Beer: Okay. You think they are people within the HSH?
Michael Peach: I would read that that way, yes.
Mr Beer: I have suggested that they are hardware specialists and then you referred to paragraph 2.8. What in paragraph 2.8 causes you to think that Product Support Engineers may also be involved with software?
Michael Peach: Because it refers to SDU which stands for Service Delivery Unit and SSC is listed in many documents as one of the Service Delivery Units.
Mr Beer: So if the other people within first line support can’t resolve the incident, it goes back to SSC; is that right?
Michael Peach: I don’t think so. Can you reword that or …
Mr Beer: Yes. You said that SDU, Service Delivery Unit, is defined in a number of documents as meaning the SSC?
Michael Peach: No, the SSC is one of the Service Delivery Units. Hardware engineers would also be a Service Delivery Unit, as would the Network team.
Mr Beer: Okay. So it’s an umbrella term, SDU, referring to a group of different units?
Michael Peach: Yes.
Mr Beer: So this paragraph 2.8 means: if HSH doesn’t contain the answer, if KEL doesn’t contain the answer, if the Product Support Engineers can’t find the answer, then you go back to one of the units within SDU using the support matrix in HSH ONE?
Michael Peach: You don’t go back to, you go – in my terms, you go up to.
Mr Beer: Up to.
Michael Peach: If first line can’t resolve the incident and there are no known errors on any of the systems, then they will pass it through to, effectively, the third line support units.
Mr Beer: Why do you say, “effectively, the third line support units”? What about the second line?
Michael Peach: Well, because for – as I can recall the process, HSH would send calls which were clearly hardware to the engineering team. They would send calls which were clearly or possibly software issues to the SMC and, whether or not the interface to the Network team went through HSH direct or through the SMC, I can’t recall.
Mr Beer: Can we move forwards please to page 18 and paragraph 4.4. The document records:
“Where this incident has a number of calls referenced to it, or where there is a probability that proactive action is required to prevent further occurrences of this Incident the IMT [I think that’s Incident Management Team] initiate a Problem record to be authorised by the Service Management Team and passed to Problem Management.”
What do you understand that process to involve?
Michael Peach: I understand that to be the process by which HSH, having received a number of calls, would attach those calls to the incident and then report to the problem managers for handling as a problem.
Mr Beer: How many calls would be sufficient to justify doing so?
Michael Peach: I believe that the problem management process says on receipt of the second call. Whether or not that was modified to say “Only do it with ‘N’ calls”, I don’t know.
Mr Beer: Alternatively it says:
“… where there is a probability that proactive action is required …”
Who would decide whether it was probable that proactive action would be required?
Michael Peach: Reading that, I would assume HSH because they’re the ones that are having the conversation with the problem managers.
Mr Beer: How would they decide the probability that proactive action is required?
Michael Peach: I don’t recall if I ever knew the details of the interaction between HSH and the problem managers.
Mr Beer: Thank you. Can we move on to the PEAK system, please. At paragraph 42 of your witness statement, which is on page 17, you say:
“If the SSC recognised that a particular problem could have implications for multiple branches, this was added to the PEAK and the KEL. It is important to note that problems which occurred in overnight processing sometimes had the potential to affect all Post Office branches, but not every potentially affected branch would be listed on the PEAK.”
How would SSC recognise that a particular problem could have implications for multiple branches?
Michael Peach: In two ways. If HSH has received multiple calls, then they had the ability to attach the call references to the master call which was presumably by this time in the SSC. If an SSC person, when analysing a problem, realised that it might have wider implications, then they would actually trawl the system to find out if other post offices are being affected by it.
Mr Beer: How would they trawl the system?
Michael Peach: I think that would vary widely depending on the nature of the problem. If I can think of an example … if we take as an example, and it’s hypothetical, an issue in a certain type of reference data, then having received the initial call, the SSC would go through the correspondence server message stores in the central systems and identify which other post offices could be affected by this particular issue.
Mr Beer: And what would they do then?
Michael Peach: Note the details of those post offices on the PEAK, probably on the KEL as well and, if it was felt to be a problem rather than a single incident, then we would talk to the problem managers in that area.
Mr Beer: Whose responsibility was it to get in touch with the Post Office to tell them that a problem had –
Michael Peach: Problem managers – Service Delivery Managers who became de facto problem managers for their areas had interfaces through to Post Office management.
Mr Beer: Now, I think earlier in the process, before calls got up to the SSC, you were aware, obviously, that, from what we’ve discussed so far, that the HSH had access to their own knowledge database and KELs.
Michael Peach: Yes.
Mr Beer: Were you aware that they were provided with scripts?
Michael Peach: I have become aware that they were provided with scripts. I must have been aware at the time because it was clear from some of the PEAKs coming to us the responses from postmasters to questions were being repeated. So I was aware they were running on scripts. The exact origin of those scripts, I cannot remember.
Mr Beer: Did you ever see the scripts?
Michael Peach: Not that I can recall as scripts. I can remember seeing on PEAKs a series of questions and answers which were clearly the same between some PEAKs. So a series of questions, a series of different answers. So, yes, I was aware there were scripts there but the full nature of the script I don’t recall ever seeing.
Mr Beer: So you saw extracts from what appeared to be scripts cut into a range of PEAKs –
Michael Peach: Yes.
Mr Beer: – and saw them in that way?
Michael Peach: Yes.
Mr Beer: Was it part of the SSC’s function to either write or verify the accuracy of the content of scripts?
Michael Peach: I don’t recall it ever being a responsibility of the SSC to produce scripts or verify the accuracy. I do recall on occasions Service Delivery Managers saying, “What questions do we need to ask in order to find out the root cause of this particular issue?”
Mr Beer: So contributing to the content in that way?
Michael Peach: Yes.
Mr Beer: But were you aware of any quality assurance process of the scripts?
Michael Peach: No.
Mr Beer: We’ve heard evidence from some subpostmasters that, when they contacted HSH, they were aware that the person they were speaking to was reading from a script and, amongst other things, they were told they should take no action, that a shortfall would probably resolve itself. Did you see any scripts of that nature?
Michael Peach: Not that I can recall. As I said, I don’t recall seeing any complete scripts at all.
Mr Beer: And that others have suggested that they were told that, under the subpostmaster contract, they were liable to repay to Post Office any and all shortfalls and, therefore, they should do so?
Michael Peach: I was not aware of the terms of the contract between Post Office and subpostmasters probably until March this year when it became obvious from the documents that the Inquiry had sent to me. If I had been made aware of that, then I don’t recall when it was done and it obviously didn’t register.
Mr Beer: Can we turn to paragraph 43 of your witness statement which is just underneath. Thank you.
You speak about the Service Management Portal –
Michael Peach: Yes.
Mr Beer: – between this paragraph and paragraph 49. You say it was:
“… an initiative proposed by the then customer service director (Dave Baldwin), to be written by the SSC as something completely separate from the live estate and the formal development of Horizon … created in about 2006, and it was a prototype to demonstrate the ways in which Fujitsu could display data of interest to POL.”
Then over the page, please, you summarise in paragraph 44 – I’m not going to read it all now – the incidents which the SMP would deal with or the matters that the SMP would deal with, including a number of system functions, including Alliance & Leicester Network Banking, overall failure account on Network Banking transactions, ePay connections for E-Top Ups, et cetera?
Michael Peach: Yes.
Mr Beer: What was your involvement in the creation of the Service Management Portal?
Michael Peach: I wrote large portions of it and included it in several tools that had been developed within the SSC.
Mr Beer: Can we look, please, at FUJ00142216. If we look on the second page, we can see your name as an “Originator”. Does that mean author?
Michael Peach: Yes.
Mr Beer: Then, if we look at page 4 of the document, please, there should be an index, if we just zoom out, please. You’ll see that we get an idea of the size of the document and its coverage, yes?
Michael Peach: Yes.
Mr Beer: So, after the introductory section there are five main sections: Section 4, “The Management Monitors”; Section 5, “Major Incident Management”; Section 6, “Reports and SLTs”; then, over the page, please, Section 7, “Operational Business Change”; and section 8, “Operational Change”.
Overall, standing back from the document and without going through all 70-odd pages of it, was the idea of the portal to provide a high level overview of the status of the component parts of the Horizon System?
Michael Peach: Both high level overview in real-time monitoring and more detailed description of performance against SLTs.
Mr Beer: It was also a record of business and operational changes that it was proposed would be made?
Michael Peach: Yes.
Mr Beer: Its purpose was not to record or to summarise particular bugs or the effect of particular bugs on subpostmasters?
Michael Peach: No.
Mr Beer: Neither the document nor the Portal?
Michael Peach: No.
Mr Beer: In practice, is this right: the Service Management Portal was not used to record or to summarise particular bugs or the effect of bugs on subpostmasters?
Michael Peach: No, I don’t recall that ever being my intent when I wrote it.
Mr Beer: And, in practice, it wasn’t used for that purpose?
Michael Peach: No.
Mr Beer: Thank you.
If we just go back to paragraph 44 of your witness statement, please, which is on page 18, you say in that paragraph that the Post Office (POL) could also check on the current state of an individual Post Office?
Michael Peach: Yes.
Mr Beer: The data which was used to provide the information was detailed on the Service Management Portal.
Michael Peach: Yes.
Mr Beer: Do you know who within POL was provided with access to the system in this way?
Michael Peach: At the time, I knew in terms of names because I had to set up individual users with a username and password. Their roles within Post Office, no, I never knew.
Mr Beer: How many were there?
Michael Peach: No, I’m sorry, I can’t remember that.
Mr Beer: Are we talking 5, 50, 500?
Michael Peach: We’re talking tens but not hundreds.
Mr Beer: What sort of information was held on this system concerning individual branches?
Michael Peach: As I recall, there was a map of the UK with areas that would go red if a post office was not operating, details of the nearest working post office and, for any post office, if you put in the FAD code, you would get back details of address and, I think, number of counters.
Mr Beer: And that was the extent of the data?
Michael Peach: In that particular section, as I recall, yes. The user guide would detail exactly what was available in there.
Mr Beer: Yes, and we’ve got it. Thank you.
Michael Peach: Okay.
Mr Beer: Lastly from me, can we discuss another way of sharing information with the Post Office and look at paragraph 107 of your witness statement, please, which is on page 35. I should start with 105 to give some context. Under “Sharing information” you say:
“There were no procedures or work instructions of which [you were] aware that restricted the flow of information about any workaround or potential bug anywhere in Horizon.”
Michael Peach: Correct.
Mr Beer: Then, 107, you say:
“When incidents were closed, this would be communicated to the postmaster who raised the incident by the HSH/SMC. In cases where the SSC was communicating with a postmaster about an incident, SSC staff would sometimes agree closure of the incident with the postmaster. If a workaround was being applied, POL would sometimes liaise with the postmaster as to when the workaround was to take place – for example, if messages needed to be inserted into the counter message store. As I recall, these types of workaround required liaison between Fujitsu, the postmaster and POL because (a) the postmaster would have to have the Post Office branch open (otherwise transactions would appear after end-of-day processing and cause failures), (b) Post Office branch staff would have to log out of their counters (otherwise the Riposte sequence number would mismatch and cause error), and (c) POL would also be asked to confirm that the inserted transaction had been successful. However, workarounds that were applied to data centre systems were not always agreed, or discussed with POL.”
Can you explain why workarounds that applied to data centre systems were not always agreed or discussed with POL?
Michael Peach: If I may, I would like to correct some of the statements that come before that.
Mr Beer: Yes, go ahead.
Michael Peach: I think I was asked specifically would it be possible to put messages into a message store out-of-hours. I can’t remember who asked me that, to be honest.
So what is written there was in response to that particular question and why it would not work. It’s not to be taken as detail of the process. Was that not clear?
Mr Beer: No, it’s not clear. What’s the correction you wish to make?
Michael Peach: Yes, I recall that it required liaising between Fujitsu, the postmaster and POL but the reason behind that, in the working day, was because the counter needed to be left switched on, logged in and left alone. That was against POL procedures, which is why we sought POL’s authorisation for any such addition to a message store and why it required us to talk to the postmaster to arrange a specific time.
Mr Beer: Can we go back to the last sentence:
“[The] workarounds that were applied to data centre systems were not always agreed, or discussed with POL.”
Why was it that workarounds that applied to data centre systems were not always agreed or discussed with POL?
Michael Peach: As I recall, there were occasions where the error had arisen in the calculations inside Fujitsu code in the data centre. Again, a hypothetical example: three post offices sell three stamps and the system then turns around and says, “Yes, ten stamps sold, receipts for 9”. Clearly, that’s a software bug. Equally clearly, there is a correction required to the data but it does not affect any of the individual post office’s branch accounts and, therefore, there is no point in contracting them.
Mr Beer: Was it the case that no contact was made principally on the basis that, for logistical or technical reasons, which you list in this paragraph, it was necessary to make contact in order to allow the workaround to be carried into effect?
Michael Peach: I’m sorry, I’m not sure I understood that question.
Mr Beer: You have described in this paragraph occasions on which liaison was required between Fujitsu, the subpostmaster and POL.
Michael Peach: Yes.
Mr Beer: You told us that some workarounds were effected, you have just given an example of one, without such contact with either POL or the subpostmaster.
Michael Peach: Yes.
Mr Beer: Was the reason that no contact was made with them because it wasn’t necessary to make contact with them to carry out the fix? You didn’t need to ask them to leave their machine on, for example?
Michael Peach: No. The whole process about leaving the machine on refers solely to the very rare occasions where messages needed to be inserted into the branch message store. Where it was not necessary was where corrections were being made to the central systems and where there was no implications for the individual branch’s accounts.
Mr Beer: Wouldn’t POL wish to know that there was a bug in their system?
Michael Peach: Yes, and they would know because the daily reports on receipts and payments would not match.
Mr Beer: I thought you told us that it would have no impact.
Michael Peach: It would not have an impact on individual branches. It would have an impact on the overall concatenation of the data. Where it would have an impact on an individual branch, then we would know which branches and the branches would have been told by SSC.
Mr Beer: So is it your evidence that, for each and every bug that was discovered by Fujitsu, either the postmaster was told directly or POL were told indirectly through the provision of the data that you’ve just mentioned?
Michael Peach: No, that’s not true for every bug because there were bugs in the central systems which would not have implications of that sort.
Mr Beer: What do you mean implications of that sort?
Michael Peach: Any sort of financial implications on either branch or POL business.
Mr Beer: Were they the only ones that were kept from the Post Office then, ones where Fujitsu assess that there was no financial impact on anyone?
Michael Peach: No, I think that’s reading something into what I said that isn’t there. There was, in my mind, no case in which POL could not be told of any bug in the system.
Mr Beer: And I’m asking you, of all of the bugs of which you were aware, were they told?
Michael Peach: No, because I am aware that there were incidents in the central systems which would not be communicated unless they became problems.
Mr Beer: Is the existence of a bug not a problem about which POL should be told?
Michael Peach: No. The existence of a bug can cause an incident. Only if it is deemed important or there are multiple sorts will it be a problem and, as far as I am aware, problems were always communicated to POL. It’s very difficult to be specific about this without looking at an individual problem that may have occurred in a central system overnight.
Mr Beer: Thank you very much, Mr Peach. Those are all the questions I ask.
Mr Stein: Sir, I have a few questions for Mr Peach. May I go ahead now?
Questioned by Mr Stein
Mr Stein: Mr Peach, my name is Sam Stein. I represent 157 subpostmasters and mistresses. I’m instructed by the solicitors Howe+Co.
Can you just help us a little bit more to understand what people within the SSC were doing by way of the more general nature of work. Is it right that not everyone was engaged in fixing bugs and problems but other work was going on; is that correct?
Michael Peach: Yes, it is.
Mr Stein: Now, the other work that was going on, and just help us a little bit more on this, is that there was pressure to complete testing; is that correct?
Michael Peach: No, SSC were not directly involved in testing except for workarounds that were produced in the SSC.
Mr Stein: Right. So regarding workarounds that had to be looked at by different members of the team; is that correct?
Michael Peach: No a workaround for any specific incident would need to be tested before going out to the live estate.
Mr Stein: Right. So that work was ongoing? What about new features for the Horizon System? Was that part of the work that was ongoing?
Michael Peach: Only in as much as we would review the technical documents that came from development and architecture prior to them going live and review support guides for each product that was going into the estate that was new.
Mr Stein: Later on, did that SSC also play a role in relation to Horizon Online? So as we move from the early days of what we call Legacy Horizon into Horizon Online, did it play a role there as well?
Michael Peach: Yes. We specified a large number of requirements to Development in order to help us support the system and one of my staff was put on more or less permanent secondment to the Development teams to try and ensure those requirements were met before the system went live.
Mr Stein: Okay. So we’ve looked at, first of all, with Mr Beer, King’s Counsel, in the questions that he has asked you today, about the different roles of the different tiers of problem solving, the taking the calls, the more that developed, the more expert individuals that engaged in solving problems, the workaround, the testing within that that had to go on and then also the work that was ongoing as regards development of Horizon Online.
Michael Peach: Yes.
Mr Stein: Have I missed anything?
Michael Peach: Yes.
Mr Stein: Okay. Because it seems as though there’s a lot going on here.
Michael Peach: Yes. SSC developed both the KEL system and PEAK, so we were effectively fourth line support for those particular applications. We did a large number of ad hoc requests for data which came usually from POL and usually through the Service Delivery Managers. So POL would need certain information, they’d talk to the Service Delivery Managers, the Service Delivery Managers would come and ask us whether it was feasible, how long, how much.
There were other things going on but I’m struggling to remember.
Mr Stein: Okay. All right, so we’ve got a very good idea in outline terms about the number of different things that were happening.
Now, presumably, this was all part and parcel of the delivery by Fujitsu of its contractual obligations to the Post Office, yes?
Michael Peach: Yes, and in the case of the ad hoc data requests beyond the contract.
Mr Stein: Just to clear up one matter, was the SSC or any of its staff members also engaged on other projects, so working on maybe development of other systems for other companies?
Michael Peach: Not while I was manager.
Mr Stein: So we know that the SSC, therefore – you used the term workarounds. One of the big drivers here is to make sure that the system remains up and running, is servicing and need of the branches and the Post Office; is that right?
Michael Peach: Yes.
Mr Stein: So a number of things all going on. There’s a contractual obligation to try to do what it can for the Post Office, whole thing needs to be kept going, yes?
Michael Peach: Yes.
Mr Stein: It sounds as though it’s a fairly, how can I put it, pressured or busy place?
Michael Peach: Busy, yes. Pressure is subjective.
Mr Stein: All right. Now, help us little bit more in relation to the way that things worked. We know from your evidence that what you are saying is that you weren’t aware until around, what was it, 2006/2007 that there was the use of the Horizon System to support court cases.
Michael Peach: And, to my memory, only in 2006 and I didn’t – I wasn’t aware that it was being used for any other cases.
Mr Stein: Right. Help us a bit more with your line management, okay. So you’re in charge of the SSC. Who’s in charge of you? Who’s your boss?
Michael Peach: On occasion it was the Customer Service Director. More usually –
Mr Stein: Name – can you name the individuals as we go, please?
Michael Peach: Customer Service Director, Stephen Muchow, Martin Riddell, Dave Baldwin, Naomi Elliot, Wendy Warham, possibly two or three more I’ve forgotten.
Mr Stein: The different names you are giving, are they all people who held a line managerial job relation to you over a different period of time or were they all together managing you?
Michael Peach: No, over different periods of time. So those were the Customer Services Directors. I did not report direct to all of those. A lot of the time I reported to the Support Services Manager who, in turn, reported to the Customer Service Director.
Mr Stein: Okay. So, in terms of your knowledge about the use of the data from Horizon System, it going to support or going to be used as part of court cases. Now, you probably know by now the sort of court cases we’re talking about are the prosecutions of individual subpostmasters and mistresses?
Michael Peach: Yes.
Mr Stein: You know by now that that also included some people being prosecuted, a failure of disclosure and people going to prison.
Michael Peach: Yes.
Mr Stein: You must have read all about that?
Michael Peach: Yes.
Mr Stein: You also must be aware by now that we’re not just talking about the use of the criminal system but also data from the Horizon system was used in relation to civil prosecutions, civil cases before the county courts, yes?
Michael Peach: Yes.
Mr Stein: A serious business.
Michael Peach: Yes.
Mr Stein: Now, help us understand a little bit more. The people that you have mentioned, the people in your line management, who out of those people do you think should have told you that, “Look, Mik, we’ve got this system, it’s going to be used by the Post Office, they’re going to prosecute people with it, just so you know, you know, make sure you design the thing, make sure you get your people working to that level”.
Michael Peach: Yes.
Mr Stein: Who should have told you that?
Michael Peach: That’s extremely difficult to answer because it’s one of those cases where you don’t know what it is that you don’t know. Given that I was not aware that data was being used in that way, how could I say who should have told me?
Mr Stein: Let’s try it another way round.
Michael Peach: I would have expected it to come through my line management.
Mr Stein: Right. Was that Mr Muchow?
Michael Peach: I was not aware of any cases when Steve Muchow was my manager.
Mr Stein: All right. Let’s try this another way. Do you think it’s unimportant, do you think it doesn’t matter at all, that the SSC was not put into the state of knowledge that this stuff is going to be used for core purposes, think it doesn’t matter?
Michael Peach: No, I certainly do not think that –
Mr Stein: No, well, quite. Let’s go back then to my question: who should have told you? If it’s something that was important and you weren’t told, who should have told you, Mr Peach?
Michael Peach: Again, I would have expected that to have come through my line management.
Mr Stein: Right. Well, over that period of time can you name the individuals that should have told you this important bit of the business?
Michael Peach: Over which period of time?
Mr Stein: Well, why don’t you start from the first manager that you can think of that should have told you that and name them, please.
Michael Peach: The first case that I knew of was 2006.
Mr Stein: No, Mr Peach, let’s try this another way round. We’ve got a situation whereby you agree it’s important that you should have been told that the information from the Horizon System was going to be used in court proceedings, right. You’re saying you weren’t told that, okay?
Michael Peach: Right.
Mr Stein: Right. Who out of your line managers should have told you that?
Michael Peach: If they knew, the entire list of my line managers.
Mr Stein: Excuse me one moment. (Pause)
Thank you, sir.
Sir Wyn Williams: Any other questions? It looks as if that’s it, Mr Peach.
Mr Beer: No, it’s not there’s two lots of questions to come.
Sir Wyn Williams: I see.
Mr Beer: I think Ms Page is going to go first and then –
Sir Wyn Williams: I’m conscious that we have been going for longer than an hour and I don’t want things to be too difficult for Mr Peach. Do we need a short afternoon break?
Mr Beer: I will just check, if they can hold up their hands, for how many minutes they are going to be. Five for one and …
Ms Page: It may be more like 15.
Mr Beer: I think that sounds like an afternoon break then, sir.
Sir Wyn Williams: Let’s have our break now then, sir.
Mr Beer: Can we come back at 3.15, please?
Sir Wyn Williams: Certainly.
Mr Beer: Thank you.
(3.00 pm)
(A short break)
(3.15 pm)
Ms Patrick: Sir, can you see and hear me.
Sir Wyn Williams: Yes, I can thank you.
Questioned by Ms Patrick
Ms Patrick: Thank you.
Mr Peach, my name is Angela Patrick and I ask questions on behalf of a number of subpostmasters who were wrongly convicted and then acquitted and I am instructed by Hudgell Solicitors led by Mr Tim Moloney. We only have a couple of questions about two documents.
One you will be quite familiar with, and I’m not going to spend too long on, is Mrs Chambers’ January 2007 afterthought document, which Mr Beer has taken you to at some length. It’s FUJ00152299 and you will be happy to hear I only want to look at the last two paragraphs of the document, which are on page 2.
I don’t think we need to read it again. These are the two paragraphs that deal with what she, Mrs Chambers, calls the common problem of SPMs, subpostmasters, being bounced from one Helpdesk to another and she says about some calls being closed without proper investigation or resolution when they are bounced back, and those calls are problems with discrepancies needing to be investigated but personally they are being penalised for passing some back.
Is that fair? She’s basically saying there might be a problem about calls, about discrepancies, being bounced back inappropriately; is that right?
Michael Peach: I agree that, from what she says, calls were being bounced between Horizon and the NBSC but she does make the point in there that that resulted in it taking a long time before the call got to the SSC.
Ms Patrick: Okay.
Michael Peach: I can’t answer for the processes in NBSC and HSH, sorry – only for the SSC.
Ms Patrick: Okay. Do you recall now whether at the time you took any specific steps to follow up on that concern she was raising?
Michael Peach: It sounds as though I’m being evasive and I’m really trying not to be. The first time I saw this document was at about 9.45 this morning, other than when it happened and I didn’t recall. I would think that Anne would have made sure I took some action but what that action was, I couldn’t say because I don’t recall the document.
Ms Patrick: Just to be absolutely clear, and I know that it’s some time ago, having had a bit of time to reflect, can you recall whether it was raised with the Post Office?
Michael Peach: I can’t recall but, since NBSC were part of Post Office, I would have expected them to have known from NBSC. If calls being bounced between Horizon and NBSC had been raised within Fujitsu, then it would have been raised with the Service Delivery Manager, ie problem manager, responsible for that area and I would have expected it to have been notified to POL through that route.
Ms Patrick: Thank you. Taking that on board, can we look at another document and it’s FUJ00080096.
If we can look at the top of the page to start with, can you see that, Mr Peach?
Michael Peach: Yes.
Ms Patrick: I think this is a joint working document. We’ve seen these before. It’s got both logos on the top; is that right?
Michael Peach: Sorry – yes.
Ms Patrick: You’d recognise that as a joint working document, would you?
Michael Peach: Yes.
Ms Patrick: We see the title there. It’s a document on the operation of the Horizon Service Desk and it’s identified expressly as a joint working document. If we look at the very bottom of the page we can maybe put a date on it. Thank you very much.
Can you see at the bottom right-hand side there it’s dated 4 September 2008. So that would have been about a year, or more than a year, after Mrs Chambers’ feedback in her afterthought note, which was January 2007?
Michael Peach: Yes.
Ms Patrick: If we see there above in the approval authorities, you weren’t an approval authority for this document, would you, Mr Peach?
Michael Peach: No.
Ms Patrick: If we look at page 27 though, near the bottom of the page, we can see you’re named there four boxes up from the bottom as a key contact for major software problems?
Michael Peach: Yes.
Ms Patrick: Can you recall whether at the time you may have been consulted on this document, which was, as we see, a joint working document for the operation of the Service Desk?
Michael Peach: I don’t recall the document. I would have expected to have been consulted on procedures relating to software problems, which were going to go through the SSC.
Ms Patrick: Okay. Now, if this includes instructions for the Service Desk, including when something would be directed towards the SSC or elsewhere, as the manager of the SSC at this time in 2008, would you expect to have been shown it?
Michael Peach: This document, yes, I would have expected to have been shown it.
Ms Patrick: Can we look at page 11, please. About part of the way down this page, you can see at the bottom there’s a header 3.4 “Inappropriate Calls”. This is the part that I’d like us to look at together. It reads:
“Inappropriate calls are defined as below; all inappropriate calls are logged on the Incident Management System. If the incident has been investigated by an Agent and the call is deemed to be an inappropriate call, the desk Agent will give the postmaster an Incident number and either transfer the postmaster to the NBSC or cold transfer the postmaster.”
We see a list there of what calls are considered to be inappropriate. Can you see the fourth bullet point there, Mr Peach:
“Caller requests advice on cash account/discrepancy issues.”
Is this document advising the HSH, the Helpdesk, to treat calls from subpostmasters about cash account and discrepancy issues as inappropriate calls?
Michael Peach: The way I would read that would be to say, before any call of that sort was dealt with by HSH, it should have gone to NBSC first.
Ms Patrick: But the direction there is “inappropriate calls are defined as below” –
Michael Peach: Yes.
Ms Patrick: – to include calls about cash account and discrepancy.
Michael Peach: Yes.
Ms Patrick: And the direction is to transfer to NBSC or cold transfer. What was a cold transfer?
Michael Peach: I have no idea.
Ms Patrick: But, essentially, they are being told to bounce them back to the NBSC?
Michael Peach: I would not disagree with your interpretation of that.
Ms Patrick: Was that exactly the problem that Mrs Chambers was raising concerns about in January 2007?
Michael Peach: That’s the way I would read her statement, yes.
Ms Patrick: If Mrs Chambers’ feedback had been taken up with Fujitsu management and with POL management, as you said you would have expected it to have been, it wasn’t reflected in this new joint guidance to the HSH, who were providing front line support to subpostmasters, was it?
Michael Peach: It does not appear to have been but, as I said before, I do not know the processes that were operating in either NBSC or HSH in this area, so I don’t know what’s behind that.
Ms Patrick: Thank you. I don’t have any more questions.
Sir Wyn Williams: Thank you.
Ms Page next then.
Questioned by Ms Page
Ms Page: Thank you, sir.
I also act for a group of subpostmasters, including Mr Lee Castleton.
Michael Peach: Right.
Ms Page: Can I start, please, with the document which you’ve seen some of but I’d like to take you to a different bit of, which deals with the quality of the EPOSS code and the question of whether it was going to be rewritten. If we could look at WITN04600104. You were taken initially to page 9 and you told us that, when it came to a debate about whether the code should be rewritten, you were not surprised that that wasn’t communicated to you because that was a matter for the Development team, yes?
Michael Peach: Yes.
Ms Page: If we look at page 10, we can see what happened with respect to the debate about whether it ought to be rewritten and, in that column headed “Agreed Action Commentary”, we can see that, ultimately, on 10 May – and this is in the year 2000 – that large block at bottom:
“Following response received from MJBC: as discussed, this should be closed. Effectively, as a management team we’ve accepted the ongoing cost of maintenance rather than the cost of a rewrite. Rewrites of the product will only be considered if we need to reopen the code to introduce significant changes in functionality. We will continue to monitor the code quality (based on product defects) as we progress through the final passes of testing and the introduction of the modified CI4 codeset into live usage in the network. PJ, can we make sure this is specifically covered in our reviews of the B&TC test cycles. Closed.”
What I’d like to ask you about is the suggestion there that there needed to be a monitoring of the code quality based on product defects as the modified codeset is put into live usage. As the front-line team dealing with complaints about the quality of the code, you were the team to do the monitoring, were you not?
Michael Peach: No. As I’ve explained, if there were multiple instances of an incident or if a particular issue was considered important, then problem management process would deal with it. SSC would deal with finding the technical solution to an individual incident. That having been said, if the problems encountered were clearly issues with code, then we would know the number of calls that we had passed to Development requesting code fixes.
Ms Page: So it would make sense, wouldn’t it, for your team to have been told about this so that you could do that job of counting up how many calls you had had about EPOSS code and doing that sort of collecting together exercise?
Michael Peach: That’s – with any release of any software, at some time the decision is made “We’re not going to keep rewriting this, we will fix forward”, and that is the point at which the code is delivered to the system and the SSC becomes responsible for supporting it.
Would I have liked to have known about this decision? Yes, I think I probably would. Was it my decision to make? No, and the people who made the decision were the people empowered to make that decision.
Ms Page: It’s not about blaming you, Mr Peach. I’m asking really because the question is: would it have been sensible for your team, as the front-line team dealing with the first initial software code problems, to have been able to know about this, so that you could have fed your team’s findings in to that ongoing decision about whether there ought to be a rewrite? They were keeping it under review, that question?
Michael Peach: Yes, and, as I said, I probably would have liked to have known. Would it have made any material difference to the way we handled incidents, no.
Ms Page: No, but it’s about how they could have handled the question of whether they should have rewritten the code.
Michael Peach: At the point that this document is written, I don’t believe that this code was in live. It’s talking about a specific release and it’s a discussion between development and testing and a management decision at that point. I would have liked to have known that there were those issues in testing but I don’t think I would have added significantly to the discussion.
Ms Page: All right. Well, I’ll leave it in that case because I’m actually trying to focus on not on whether you would have added to the discussion. But whether it would have been sensible for your team’s feedback to go into that discussion.
Now, I mean, if you don’t want to take that any further, I’ll leave it there.
Michael Peach: My team would not have had any feedback to this code because this release had not been – had not gone to live.
Ms Page: No but once it had gone into live?
Michael Peach: Once it had gone into live, knowing that there was specific focus and there was supposed to be more monitoring in place, yes, I would have appreciated knowing that. From this document, I can’t demonstrate that we were not asked to specifically monitor. I don’t remember that we were but I can’t demonstrate that we were not.
Ms Page: You have no recollection of being asked to monitor. That’s as far as we can take that line, I suppose.
Thank you. Let’s just turn then to another issue, which you’ve already touched upon and which I’d like to just explore a little further with you. The document FUJ00088082 is one that you’ve seen a little of. If we could turn to page 15, please, if we scroll down a little from the bit that you’ve looked at and just a little further – sorry, I may have got the wrong page. Could we just scroll on a bit further.
Sorry, I had the wrong page. So if we go down to 7.4. Oh, dear, I seem to have made a bit of a mess.
I don’t know if this is a document that you’ve already had sight of but there’s a passage where it talks about the SSC having a form of access on a needs must basis. Is that something that you recall seeing?
Michael Peach: It is but I don’t think it’s this document. I think it was …
Ms Page: You may well be right. In which case, it may be FUJ00087994. But it probably doesn’t matter actually. I won’t ask for that to come up. If you’ve seen the document, we’ve all seen it as well.
Michael Peach: Right.
Ms Page: That is a document which sets out a sort of a form of access that had developed because it needed to be; is that a fair way of describing it?
Michael Peach: Yes.
Ms Page: When that was put to Alan d’Alvarez, he testified that it had gone entirely against what the access control policy says should happen.
Michael Peach: Yes.
Ms Page: So this was in 2002. You were obviously aware of it. Was your line manager aware of it, this needs must access?
Michael Peach: Yes.
Ms Page: Who would that have been in 2002?
Michael Peach: I don’t think that I was working direct to Stephen Muchow at that point. It would probably have been Peter Burden.
Ms Page: Peter?
Michael Peach: Burden.
Ms Page: Burden. Presumably there were some less than ideal discussions around that issue. I mean, it can’t have been something people were happy with?
Michael Peach: No, none of us were happy with it. The original access control policy, from my reading of it now and my vague memories of it at the time, assumed that the SSC would be providing detailed code support to the data centre boxes and I believe the assumption was that, in the majority of cases of any problem on the counter, then the counter would be replaced: a straight hardware swap.
So the SSC responsibilities with regard to counter were very limited in the beginning, according to the access control policy. My recollection is that there were one or two problems on counters which affected a number of counters and for which hardware swaps were clearly not going to be an option and, therefore, SSC were allowed access to the counters in order to fix those problems.
Immediately that that became obvious and that we were contravening the terms of the access control policy, the security architect, who was Glenn Stephens (now, sadly dead), started looking for a solution that would give the SSC the access they needed to fix the system but in a controlled and auditable way, and that’s where SSC came from.
Ms Page: What I’d quite like to understand – and thank you for that – is who – where did this land at the most senior level? Who was the most senior person who knew about this problem?
Michael Peach: I don’t know.
Ms Page: Was it seen as sufficiently serious for this to have got to the board?
Michael Peach: I’m sorry, I don’t know that either.
Ms Page: Your evidence about what then happened in terms of the secure shell system, the SSH, was that you understood that that meant that, from that point onwards, there was a full keystroke audit trail for everything that SSC members did when accessing subpostmaster accounts?
Michael Peach: When accessing any part of the live system.
Ms Page: But certainly subpostmasters accounts?
Michael Peach: Certainly access to counters. Indeed, SSC staff used their PCs to get into the SAS servers and, from that moment onwards, whenever they went into the live estate it is my understanding that all keystrokes were logged.
Ms Page: That was your understanding. You told us, however, that, although that was your understanding, you hadn’t actually seen evidence of that, as such, but that was your understanding. You never saw any printouts that showed full keystroke logging?
Michael Peach: I did not but then, to be honest, I don’t think I ever saw a message store. That was just too technical for me. I know from discussions with one or two SSC staff that the logging was there but only because they said it was absolute torture to look at it, because if you typed “T-E-H” instead of “the” and then went back, you would actually see “T-E-H, backspace, backspace, delete, delete, T-H-E”. So to read it was very difficult.
Ms Page: That suggests that definitely somebody saw it at some stage?
Michael Peach: Yes.
Ms Page: If I told you that there was a document considerably later which says that it wasn’t happening, it certainly wasn’t happening by that stage, you’re not aware, are you, of any decision that was taken to stop doing it?
Michael Peach: No, I would question the date on that subsequent document.
Ms Page: It was after you left. It was into the next decade.
Michael Peach: Okay. Then –
Ms Page: But you weren’t aware of a decision during your time to stop doing keystroke logging?
Michael Peach: No.
Ms Page: We’ve also been told that there was a four-eyes process and it was important to record the fact that one person was going into the message store had another person was watching them do so, and that there was then a narrative account of what the insertion into the message store had been and what it was for put into the PEAK.
Michael Peach: Yes.
Ms Page: So some effort would go into that and, indeed, it would involve a doubling up of manpower?
Michael Peach: Yes, and it wasn’t just for corrections being made in the counters. Corrections made on the data centres with any financial impact would also be subject to four eyes.
Ms Page: If you believed there was a full keystroke audit trail, what was the point? Why did you bother with the four eyes process?
Michael Peach: That’s a good question. Because it never occurred to me to remove the process.
Ms Page: Finally, if I may turn to the subject of Anne Chambers and her evidence in the Castleton trial, it’s your evidence, if I understand it correctly, that Ms Chambers was put into a difficult position when she was asked to give evidence in the Castleton trial; is that right?
Michael Peach: Yes.
Ms Page: According to her document after the event, she really identified various failings of others within Fujitsu who had failed to take full investigative steps ahead of the trial, failed to do a proper investigation ahead of the trial, yes?
Michael Peach: Yes. Again, I would stress that the first time I can recall seeing that document was early this morning.
Ms Page: But the result of that really, according to those two things put together, her being put in a difficult position and her understanding that others should have done a better investigation, is that she was really rather unfairly put into the firing line.
Michael Peach: I completely agree.
Ms Page: Who do you hold responsible for that?
Michael Peach: No, I couldn’t say. I really …
Ms Page: You had an argument about this. You were angry with somebody about this.
Michael Peach: Oh, yes.
Ms Page: Who was that?
Michael Peach: As I said this morning, I know that I had a stand-up argument and I know it was one of three people but what I can’t recall is which one of those it was.
Ms Page: Were you angry with all three of them?
Michael Peach: Very probably.
Ms Page: So one of those people was the Head of Security, Mr Utting – so sorry, not Mr Uttering, that was POL’s Security – Brian Pinder, Fujitsu’s Security, and one of them was a person who may or may not at that time have been the Customer Services Director –
Michael Peach: Yes.
Ms Page: – her name being Naomi Elliot?
Michael Peach: Yes.
Ms Page: Did she come from a Security background before she became Customer Services Director? I ask that because the document that we just looked at with Ms Patrick named her alongside Mr Pinder?
Michael Peach: I don’t know Naomi’s background. What I can tell you is that she was very astute technically, so I would have thought she would have come from an engineering programming background, rather than Security.
Ms Page: Was Security under the same director as your team, in the sense that they were also under Customer Services?
Michael Peach: Yes.
Ms Page: So we can take it, can we, that either the Head of Security or the director who was in charge of Security was ultimately responsible for putting Anne Chambers in the firing line?
Michael Peach: Ultimately responsible, yes.
Ms Page: Thank you. I have no further questions.
Questioned by Sir Wyn Williams
Sir Wyn Williams: I’m sorry, just to revisit what Ms Page has asked you but I wanted to be clear that I got your earlier evidence correct.
At paragraph 154 of your witness statement – it doesn’t have to come up, I can just read the relevant words – you say:
“I was instructed by the Director of Customer Services at the time, whose name I cannot recall, to detail someone from SSC.”
Michael Peach: Yes.
Sir Wyn Williams: You actually recall it was the Director of Customer Services but you couldn’t identify that person by name. When Mr Beer was asking you questions, you appeared to narrow it down from three to two, in that the relevant Director of Customer Services was either Mr Pinder or Ms Elliot, all right? Now, you have said something slightly different to Ms Page, so I just want to be clear what it is that I’m left with on this topic, which is obviously of some importance.
So am I correct to keep to my earlier note that it was the Director of Customer Services who instructed you and that, by a process of elimination, was either Mr Pinder or Ms Elliot, or do you want me to amend that?
Michael Peach: My recollection, sir, is that it was the Customer Services Director who instructed me. Brian Pinder was Head of Security reporting to the Customer Service Director. He was not Customer Service Director. At some point, Naomi Elliot, I believe, was Customer Service Director and at a different time. Dave Baldwin was Customer Service Director, and I cannot recall which of those two was in that post at that time.
Sir Wyn Williams: I’m sorry I misread my own note. That is, in fact, what you said to Mr Beer. I take it that this conversation, this instruction, must have been happening some time in the autumn of 2006?
Michael Peach: I have deduced that from the same documents without direct memory.
Sir Wyn Williams: So that if we were to ask Fujitsu who was the Director of Customer Services between, shall we say, August 2006 and December 2006, we’d probably deduce who that person was?
Michael Peach: Indeed.
Sir Wyn Williams: Fine. All right thank you very much.
I think that concludes the questioning, am I right in that, Mr Beer.
Mr Beer: Yes, you are, sir.
Sir Wyn Williams: All right. So thank you very much, Mr Peach, for coming to give evidence to the Inquiry and for answering very many questions over the course of the day and also thank you too for a detailed witness statement in advance.
So that I think, Mr Beer, just leaves tomorrow and, ordinarily, the assessors and I would appear tomorrow but, unfortunately, because of an appointment which I can’t change, which is early in the morning in South Wales, we’ll be appearing remotely. So that’s the reason why I have asked for a 10.30 start tomorrow.
My understanding is that we’ve got two short witnesses and then you may wish to say something about the written evidence relating to this phase, Mr Beer, and then we’re going to hear closing oral submissions.
Now, just for the avoidance of any doubt, there will be – this is not the complete closure of Phase 3 tomorrow because we have Mrs Chambers to return and Mr Gareth Jenkins to appear, and that’s going to happen I believe, in July so that those who are making final submissions, whether orally or in writing, can rest assured that if either of those persons say anything when they appear which they wish to address me about they can do it when they are making submissions at the end of Phase 4.
Does that sound reasonable?
Mr Beer: Sir, yes, that’s all correct and it does sound reasonable. Thank you.
Sir Wyn Williams: Fine. Well, then I will see you remotely tomorrow.
Mr Beer: 10.30, sir, thank you.
(3.52 pm)
(Adjourned until 10.30 am the following day)