POL00344065
POL00344065
Message
From: Dave Hulbert [IMCEAEX-
_O=MMS_OU=EXCHANGE+20ADMINISTRATIVE+20GROUP+20+28FYDIBOHF23SPDLT+29_CN=RECIPIENTS_CN=DAVE+2EHULB
ERT8FF12B5A-AD11-4323-AA83-E0874E17B20A@C72A47.ingest.local]
on Dave Hulbert <IMCEAEX-
behalf _O=MMS_OU=EXCHANGE+20ADMINISTRATIVE+20GROUP+20+28FYDIBOHF 23SPDLT+29_CN=RECIPIENTS_CN=DAVE+2EHULB
of ERT8FF12BSA-AD11-4323-AA83-E0874E17B20A@C72A47.ingest.local> [IMCEAEX-
_O=MMS_OU=EXCHANGE+20ADMINISTRATIVE+20GROUP+20+28FYDIBOHF 23SPDLT+29_CN=RECIPIENTS_CN=DAVE+2EHULB
ERT8FF12BSA-AD11-4323-AA83-E0874E17B20A@C72A47.ingest.local]
Sent: 22/01/2014 17:35:08
To: Lesley J Sewell
Subjec Fw: Update Wednesday
t
in post office INCO003137
Lesley
For info only
Dave.
“Yead of IT Services
Post office Ltd
aaa Original Message -----
From: Dave Hulbert
Sent: Wednesday, January 22, 2014 05:10 PM
To: Nick Beal; Steve Beddoe
Subject: Re: Update Wednesday 09:00am - Histon post office INC0003137
Nick
Yes, we can provide this. I'd also like us to get together to talk about the opportunities to improve so
that we do this jointly.
Can you advise who in Network (2 or 3 in addition to you) that we could do this with. From my team, it
would most likely be me, Steve and Tim Green.
Steve - can you get the stats together and share them before we get together. Could you also draw. Up a
list of areas we know are an issue eg BT, cabling, keeping branches informed etc,
Could you also get Amy to set the meeting up.
Thanks
Head of IT Services
Post Office Ltd
as Original Message -----
From: Nick Beal
Sent: Wednesday, January 22, 2014 05:00 PM
To: Dave Hulbert
Subject: RE: Update Wednesday 09:00am - Histon post office INC0003137
Hi Dave
So I can gauge the scale of this kind of problem, it would be very helpful if I could have some data
that summarises service issues - how many branches experience and outage of some sort, how long for, how
many get a payment etc.
Hopefully this is available without any extra work.
Happy to discuss.
Rgds
Nick
POL00344065
POL00344065
Nick Beal I Head of Network Development
oh SE... Floor. 148 old Street — EC1V 9HQ
GRO I
This email and any attachments are confidential and intended for the addressee only. If you are not the
named recipient, you must not use, disclose, reproduce, copy or distribute the contents of this
communication.
If you have received this in error, please contact the sender and then delete this email from your
system.
-----Original Message-----
From: Dave Hulbert
Sent: 22 January 2014 16:17
To: Kevin Gilliland
Cc: Lesley J Sewell; Jackie Meylak; Rebecca Barker; Steve Beddoe; Nick Beal; Scott Somerside
Subject: Re: Update Wednesday 09:00am - Histon post office INC0003137
kevin,
I recognise that the time taken to restore service at Histon is well below acceptable, particularly as
we're trying to grow our income and customer base.
So, in addition to Steve's note below, we will look at the information provided to branches about their
incidents. we know they need to have a good understanding for their own purposes and so that they can
advise their customers accordingly.
I also want to look at the processes we have operating between Fujitsu and BT to make sure they are slick
and robust. I know this issue has been impacted by some elements outside of our direct control, but I
recognise that overall we have improve; looking at different approaches etc, if we are to avoid similar
from occurring.
Regards
Head of IT Services
Post Office Ltd
an Original Message -----
From: Steve Beddoe
Sent: Wednesday, January 22, 2014 03:16 PM
To: Duty Manager; Nick Beal; Kevin Gilliland
Cc: Dave Hulbert; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley; Scott Somerside
Subject: RE: Update Wednesday 09:00am - Histon post office INC0003137
Rebecca,
Thanks for the update.
In addition :-
1. I will raise via the Branch Support Programme activity that we need to look at linking the IT / RoMEC
processes to better support branch issues where multiple services are affected, ie Horizon, Phones,
Alarms & Cabling etc., as these do not interact smoothly at present, as evidenced during this incident.
2. We have put Histon to the top of the list with regard to the Aerial service improvement, in order
that we can improve the Wireless back-up service that is currently very intermittent and as such has not
been fit for purpose during this incident.
Regards,
Steve
Senior IT Services Manager
Post Office IT
1 Future walk
Chesterfield
S49 1PF
-Original Message-----
From: Duty Manager
POL00344065
POL00344065
Sent: 22 January 2014 15:08
To: Nick Beal; Kevin Gilliland
Cc: Dave Hulbert; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: RE: Update Wednesday 09:00am - Histon post office INC0003137
Good Afternoon,
Update
Steve Beddoe has been in contact with the branch and senior escalation points within Fujitsu.
Nick Crowe (senior Fujitsu engineer) will be attending site with BT tomorrow - ETA to be confirmed.
The branch have opted out of the PSTN option as this affects their phone line which they use for other
means of custom and trade.
Thanks
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
i "GRO_; Postline:
duty_manager{
wore Original Message-----
: Duty Manager
22 January 2014 14:03
To: Nick Beal; Kevin Gilliland
Cc: Dave Hulbert; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: RE: Update Wednesday 09:00am - Histon post office INC0003137
Hi Nick,
I have been the one contacting branch and explaining what is happening at each stage, what the issues
are, and how we intend to progress.
Thank you
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
w----1 Original Message-----
: Nick Beal
: 22 January 2014 14:00
To: Duty Manager; Kevin Gilliland
Cc: Dave Hulbert; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: RE: Update Wednesday 09:00am - Histon post office INC0003137
Hi Rebecca
POL00344065
POL00344065
who is liaising with the branch - someone from the PO team needs to be working with them to explain this
- please do not leave it to the engineer.
Please confirm this will be the case.
Thx
Nick
Nick Beal I Head of Network Development
This email and any attachments are confidential and intended for the addressee only. If you are not the
named recipient, you must not use, disclose, reproduce, copy or distribute the contents of this
communication.
If you have received this in error, please contact the sender and then delete this email from your
system.
-----Original Message-----
From: Duty Manager
Sent: 22 January 2014 13:49
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: RE: Update Wednesday 09:00am - Histon post office INC0003137
Good Afternoon,
Update
Upon contacting the branch I was advised WWAN had in fact become unavailable again.
I have spoken to both the branch and our escalation contact within Fujitsu to arrange for a PSTN modem to
be installed at site. This was initially refused by branch, but at present this is the only way for the
branch to provide a service to customers.
The PSTN modem allows for a ‘dial up’ connection allowing the branch to trade effectively.
The engineer will be on site to complete this activity this afternoon.
Next Update 16:30pm
An interim update will be provided should Fujitsu engineers have completed the PSTN connection before
this time
Thanks
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
Postline: I
duty_manageri
-----original Message-----
From: Duty Manager
Sent: 22 January 2014 12:15
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: RE: Update Wednesday 09:00am - Histon post office INC0003137
POL00344065
POL00344065
Good Morning,
Update
The Fujitsu engineer on site has been able to resolve the WWAN and create a strong connection, from which
the branch should be able to trade without intermittent drops
Duty Manager will keep in regular contact with the branch to ensure service on WWAN remains stable, and
should any issue occur, we will raise immediately to Fujitsu.
Further to this, the branch have been added to the Aerial Roll Out Project as explained on a previous e-
mail.
The site will also be added to our list of 'Hot Sites’ to ensure that any future call raised will be
tracked immediately.
BT Issue and Router Fix
The main issue that required a router re-build cannot be completed at this time. Fujitsu have located
several BT wiring issues in branch that prevent the Horizon Connection. The branch can still trade on
WWAN as mentioned above.
The main connection point for the equipment in branch needs moving to the wall, and the face plate for
the router needs re-cabling.
we have escalated back to Fujitsu and through to BT as a result of the impact this is having on the
branch, and the expectation that this should have been resolved yesterday.
A work request has been raised to fix the incorrect wiring. The engineer is unable to establish the ADSL
horizon connection with the router until this has been completed.
Next Update; 15:00pm
Thank you
Kind Regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
From: Duty Manager
Sent: 22 January 2014 11:05
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: RE: Update Wednesday 09:00am - Histon post office INC0003137
Good Morning,
Update
The Fujitsu engineer was on site for 10:20am and is completing the full reboot of the router and
establishing sync. Fujitsu expect another update from the engineer at 11:30-11:45am.
I have contacted branch and left 2 voicemails to ensure they are kept up to date from POL, as well as the
engineer on site.
Fujitsu advise that a previous router swap was completed, but they were not able to gain sync as a result
of the BT Exchange fault. The Fujitsu engineer can now complete all activities to establish service now
the main fault with BT is resolved.
Next Update; 12:00pm
Thank you
POL00344065
POL00344065
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
riginal Message-----
Duty Manager
22 January 2014 08:55
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: Update wednesday 09:00am - Histon post office INC0003137
Good Morning,
Update
BT have completed all works successfully and there is no outstanding fault at the exchange.
Fujitsu attempted to reboot the router in order to gain ‘sync’, which would allow for the branch to
trade. Fujitsu were unable to complete this activity, and so a router swap will be taking place in branch
this morning. The SLA for the engineer to be on site is 10:20am. Duty Manager will contact the branch to
update, and we have requested that Fujitsu contact us as soon as the engineer is on site.
In the meantime, we have challenged Fujitsu as to why a router swap was not completed during initial
engineer visits, and we await their response,
Next Update; llam.
Thank you
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
w---- Original Message-----
From: Duty Manager
Sent: 21 January 2014 16:38
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: Update Tuesday 16:30pm - Histon post office INC0003137
Good Afternoon,
Update
BT have completed the lift and shift successfully,
Fujitsu are now on the phone to branch completing a system reboot to restore full service.
Next Update; 09:15am Wednesday 22nd January
POL00344065
POL00344065
Thanks
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
Postline:
duty_managert
-----Original Message-----
From: Duty Manager
Sent: 21 January 2014 13:06
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: Update Tuesday 13:00pm - Histon post office INC0003137
Good Afternoon,
Update
Fujitsu have advised that BT have this prioritised and should be resolved for end of day.
An engineer will be resolving the fault at the exchange, rather than in branch. The PM is aware of
engineer activity,
Once Fujitsu have received the notes from the BT visit, another update will be provided.
Next Update; 16:30pm
Thank you
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
Duty Manager
: 20 January 2014 16:14
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: Update 16:15pm - Histon post office INC0003137
Good Afternoon,
Update
BT have advised that the fault affects the entire area, and not just the branch.
An engineer will be attending the exchange tomorrow between 08am - 1pm to resolve.
I will contact the branch to advise of progress,
Next Update; Tuesday 21/01/2014 13:00pm
POL00344065
POL00344065
Thank you
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
a---- Original Message-----
From: Duty Manager
Sent: 20 January 2014 14:49
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: Interim Update 14:50pm - Histon post office INCO003137
Good Afternoon,
Interim Update
Fujitsu have found an alternative branch line, and offered the branch a PSTN modem. A PSTN modem offers a
dial up connection to the branch, allowing them to trade. However, a PSTN modem uses the branch phone
line, and therefore the branch have refused this option on the basis that they need to make outbound
calls.
The BT engineer was advised to complete a 'lift and shift’ - taking service off one line, and putting
this on another. The BT engineer did not complete this activity as requested this morning. BT have
confirmed that the issue is within their domain as the port in the exchange is faulty.
Fujitsu expect confirmation of a BT Engineer ETA within the next hour.
Thank you
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
Postline
GRO
From: Duty Manager
Sent: 20 January 2014 14:16
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: RE: 9:30am Update - Histon post office INC0003137
Good Afternoon,
UPDATE
The BT engineer has advised that there is neither sync nor noise on the Horizon line, and therefore this
needs to be defaulted back to BT wholesale.
Fujitsu have escalated within BT to obtain an engineer date, and we are awaiting BT's response.
POL00344065
POL00344065
We have reinforced the SLA deadline to ensure that this is met.
Going forwards I have requested that the branch be added to the ‘Aerial Roll out Project’. The project
aims to increase availability and signal to branches running on wireless back up. Roll out should
commence within the next 14 days.
Next Update; 16:30pm
Thanks
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
etait Original Message-----
From: Duty Manager
Sent: 20 January 2014 13:08
To: Kevin Gilliland
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: RE: 9:30am Update - Hilton post office INCO003137
Good Afternoon,
Fujitsu are confident that this issue is definitely within BT's domain, and are awaiting the notes from
the engineer visit this morning to understand why BT have advised this is outside of their domain. The
notes are expected within the next 45 minutes, at which point we can act based on BT's findings.
Both BT and Fujitsu's SLA contract allows them until end of day tomorrow to have this resolved, but
please be assured that we have escalated within Fujitsu, and requested that they do the same within BT.
I have contacted branch to keep them updated.
Unfortunately in terms of compensation, this is not something that POL offer. In accordance with process,
the branch will be put through the automated Goodwill procedure. The branch will qualify for goodwill
payment if they have been solid down for 48 hours, without the service being restored in that time.
Next Update; 14:00pm
Thank you
Kind regards
Rebecca Morton
Duty Manager I Live Service Team
Dearne House, Cortonwood Drive, Brampton, Barnsley, S73 OUF
From: Kevin Gilliland
Sent: 20 January 2014 12:46
To: Duty Manager
POL00344065
POL00344065
Cc: Dave Hulbert; Nick Beal; Steve Beddoe; Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: Re: 9:30am Update - Hilton post office INC0003137
I've spoken the agent twice, the office is still not trading and wasn't up and running either over the
weekend.
BT attended the branch this morning and said there was no problem from their side. A Fujitsu engineer has
now turned up and said it’s a BT issue.
As you might imagine the subpostmistess is very upset as she has not been able to trade for days - she
has a history of this type of failure and she believes we have never got to the bottom of this.
We are driving her out of business and to compound matters she became a grandmother for the first time at
the weekend. At a time that should be one of the happiest times of her life, she is beside herself
because of the poor service she continues to receive from pol.
she has kept the horizon engineer in branch because she doesn't know where to turn to next.
Can someone contact the branch urgently please and get this resolved.
Nick - I'd like us to compensate her when this is finally resolved please.
Kevin
sent from my iPhone
On 18 Jan 2014, at 11:57, “Duty Manager" ;
Good Morning,
>
>
>
>
> Update
>
> CMT have managed to get BT to bring forward the works in the exchange and it is now booked to take
place on Monday AM (08:00 - 12:00).
>
> The branch have been informed.
>
> Next Update: Monday 13:00pm
>
> Thanks
>
> Kind regards
>
> Rebecca Morton
> This has been.sent.fcam.gut of hours Duty Manager mobile. Please report all live service impacting
incidents to GRO i
>
v
From: Duty Manager
Sent: Saturday, January 18, 2014 09:29 AM
To: Dave Hulbert; Kevin Gilliland; Nick Beal; Steve Beddoe
Cc: Lesley J Sewell; Jackie Meylak; Rebecca Barker; Emily Harley
Subject: 9:30am Update - Hilton post office INC0003137
Good Morning All,
Update
BT have advised that the SLA for this work is 3 working days (excluding weekends).
VVVVVVVV¥VVVY¥
Fujitsu are escalating now to get a fixed engineer date at the earliest opportunity and will provide a
further update at 12:00.
v
> The WWAN back up seems to have improved over night and Fujitsu can see that the branch are connected
and able to trade this morning.
I will contact the branch to update.
Next Update; 12:30pm
Thanks
Kind regards
Rebecca Morton
This has been.sent.from.qut of hours Duty Manager mobile. Please report all live service impacting
ncidents to!
>
>
>
>
>
>
>
>
>
>
>
i
>
POL00344065
POL00344065
Original Message
Duty Manager
Friday, January 17, 2014 0
To: Dave Hulbert; Kevin Gilliland; Nick Beal; Steve _Beddoe
Lesley J_ Sewell; Jackie Meylak; Rebecca Barker }
Emily Harley
> Histon post office INC0003137
>
> Good Evening,
>
>
Fujitsu have confirmed that the branch raised the issue yesterday at 15:30pm, and a Fujitsu engineer
attended site at 12:30pm today to try and establish a WWAN connection.
>
> The WWAN connection is intermittent, and the underlying issue is believed to be in BT's domain. This
was faulted over to BT at 16:50pm today, and we are awaiting confirmation of an engineer date.
>
> Please be aware that due to the recent weather issues, BT are struggling to meet all SLA's, and issues
are being resolved on an area priority basis.
>
> Fujitsu have assured us that they will escalate within BT tomorrow morning to try an obtain engineer
resource ASAP.
>
> Next Update; 10am
>
> Thanks
>
> Kind regards
>
> Rebecca Morton
>
> Duty Manager I Live Service Team
Original Message-----
Duty Manager
17 January 2014 21:21
> To: Dave Hulbert; Kevin Gilliland; Nick Beal; steve Beddoe
> Cc: Lesley J Sewell; Jackie Meylak; Rebecca Barker; ‘Emily Harley’
> Subject: RE: Histon post office INC0003137
>
> Good Evening All,
>
> I have checked our incident records, and the branch has not yet been escalated to us.
>
> I have therefore raised to Fujitsu OOH and requested that the logs be checked, and actions taken by
Fujitsu confirmed back to us.
>
> Once Fujitsu have confirmed details of the issue, I will work with them to devise an action plan to
organise engineer visits (if not already completed), and to move forward. The branch will also be
contacted tomorrow to confirm that we are aware and are pushing for urgent resolution.
>
> I will provide an update by 10am tomorrow.
>
> Thank you
>
> Kind regards
>
> Rebecca Morton
>
> Duty Manager I Live Service Team
POL00344065
POL00344065
Original Message-----
Dave Hulbert
Sent: 17 January 2014 21:05
To: Kevin Gilliland; Nick Beal; Duty Manager; Steve Beddoe
Cc: Lesley J Sewell; Jackie Meylak
Subject: RE: Histon post office
Duty Manager
Can you check with Fujitsu where they are with restoring service at the branch (I've made the
assumption it's a Horizon problem, so please check facts); and push for a resolution please. Could you
provide an update to those on this email tomorrow. Thanks.
>
> Nick,
> Once we established the facts about the service issue and restored it, we'll work a response, but as
you know only too well we only have the goodwill payment process and this has conditions, so it could be
that no payment is due. We'll get the facts first and take it from there.
>
VVVVVV VV VV VV VY
Steve,
Can you stay on top of this with the team, please
Regards
Dave
Dave Hulbert I Head of IT Services
No.1 li wal
riginal Message-
Kevin Gilliland
17 January 2014 17:46
To: Dave Hulbert; Nick Beal
Cc: Lesley J Sewell; Jackie Meylak
Subject: Histon post office
Hi both,
Please see the text below from a subpostmaster. Can you investigate and see what we can do please:
VVVVVVVVVVVYVVVVVVVVYVVV¥ VV VV VY
Hello This is Rakhee from Histon Post Office , FAD 1011138. My post office network is down since
yesterday 3.30 pm. I'm waiting it to be fixed ! Is there any thing I can claim if the post office is not
open for more than 48 hrs, as this is loss if income , with no fault of mine.
>
> Regards,
>
> Kevin
>
>
Sent from my iPhone