FUJ00086490
FUJ00086490
Peak Incident Management System
Call Reference PC0146170 Call Logger _ Customer Call_ -- EDSC
Release Targeted At -- T70 Top Ref FSTK 2 0 WP25065
Call Type Live Incidents Priority B -- Business restricted
Contact EDSC Call Status Closed -- S/W Fix Available to Call Logger
Target Date 21/05/2007 Effort (Man Days) 4.00
Summary FAD080940 discrepancies when declaring Euros/cash-Code
All References Type Value
Fast Track Fix FSTK_2 0 WP24992
Powerhelp E-0705180512
Release PinICL PCO150797
Clone Call PCO148615
Fast Track Fix FSTK_2_0_WP25065
Work Package PWY WP 24992
Clone Call PC0150233
SSCKEL KEL MScardifield2219S
Release PinICL PC0148739
Work Package PWY_WP_ 25065
Clone Call
Release PinICL _ PCO148766
Progress Narrative
oate:18-May-2007 13:29:29 User: Customer Call_
CALL PCO146170 opened
Details entered are:~
Summary:openeing call as advised by pse suki
call Type:L
all Priority:B
lfarget Release: T50
outed
Unassigned
Date/Time Raised: May 18 2007 1
Priority: B
contact Name:
contact Phone
iginator: Phelp
riginator ference: E-0705180512
product Serial No:
Product Site: 080940
Product Si:
18/05/07 1
18/05/07 13:09 uk959335
information: This office had a problem when declaring Euros on Wednesday
6/05/07. She did a currency holding report and this showed
the Bu: e had but when she declared that figure the
system showed a discrepancy of 2000 Euros.
She contacted HSH but was advised to contact us. She says
this problem has occurred previously over the last 12 months
land she is always told to reboot which corrects it, but
she wants HSH to investigate why it keeps happening.
@ was working on the gateway and her user
the reboot again rectified the problem.
18/05/07 13:11 uk959335
information: Key Strokes:
fl transactions
r3 remmitences
lr9 pouch delivery
lscan bar codes on pouches
lt then prints
18/05/07 13:13 uk959335
[Information: vents are fine
FUJ00086490
FUJ00086490
fe7O5/07 1s:14 ukISSS35
Information: events are fine
18/05/07 13:14 uk959335
information:
full error message is:
there is no error message it is when you do the cash
Jdecleration they are getting a 2000 euro discrepency on wednesday
18/05/07 13:16 k959335
information: the error occured when they done the cash discrepency
18/05/07 13:17 uk959335
information: node affected: all nodes mgr thinks but is not 100% sure
[18/05/07 13:17 uk959335
Information: user: Cbi¢
18/05/07 13:18 uk959335
information: TIME OF ERROR: about 1500 wednesday 16th MAY 2007
18/05/07 13:19 uk959335
information: TRADING PERIOD: 1
BALANCE PERIOD: 8
IsTOCK UNIT: MAIN SAFE STOCK UNIT (MS)
H8/05/07 13:20 uk959335
lAccess Times: MON FRI 0900 1730
0 LUNCH
18/05/07 13:28 uk959335
KEL Ref No.: JBallantyne5245K
8/05/07 13:28 SYSADM
open OTI: Automatic Open OTT
t**Updated by Tracy Scott at 18/05/2007 13:2
[18/05/07 13:28 uk959335
IREASSIGN: Call # E-0705180512 was Reassigned from Tracy Scott, Group
listic to Group EDSC1
18-May-2007 13:31:08 User:Lorraine Guiblin
all summary has been changed rai
lopeneing call as advised by pse suki
{fhe call summary is now:
IFAD080940 problem when declaring Euros
jate:18-May-2007 13:31:21 User:Lorraine Guiblin
Reference Adde i
Jbatc:18-May-2007 13:31:33 Uscr:Lorraine Guiblin
Product: EPOSS & DeskTop -- Counter Common (version unspecified) added.
jate:18-May-2007 13:31:39 User:Lorraine Guiblin
the Call record has been assigned to the Team Member: Anne Chambers
Progress was delivered to Provider
jDate:18-May-2007 16:41:11 User:Anne Chambers
[Start of Response]
This looks very similar to the problem described in PC0145558. However the code found to have caused that problem is not yet live
(and won't be live; fixed in COUNTER EPOSS 36 3 which is about to go out everywhere.)
so far I haven't reproduced the problem on our test counter.
Phoned the branch but PM not available - I'll ring her Monday pm. They should not have to reboot if the problem happens, logging
loff and on should cause the data tree to be rebuilt correctly.
[End of Response]
lreaponse code to call type L as Category 40 -~ Pending -- Incident Under Investigation
lkesponse was delivered to Consumer
lDate:21-May-2007 18:00:22 User:Anne Chambers
spoke to PM, not happy, it happened again this morning (and they now have COUNTER EPOSS 36 3). Says it has been happening since
Hast March, not fixed by log off/ on but needs a reboot. She's concerned that it could have security implications if someone sees
that the figures are apparently adrift by several thousand pounds and decides to make off with the difference.
ill investigate some more and contact her Thursday.
IDate:24-May-2007 10:40:39 User: Customer Call_
IeMPTY 24/05/07 10:34 uk952601 HSH1 Repeat Call: pm states that this issue has happened again today. about
iS mins ago . they were inputting cash in to system and pm
states it doesnt add this to holdings so now looks like pm is
lk 5000 pounds up in cash when she isnt . pm is rebooting gw
node as pm states this sorts the issue out and figures are
fine after reboot. pm states this had been happening for a
lvear or more. not every month but alot. and pm has had
previous checks done with nbse and is doing all procedures
FUJ00086490
FUJ00086490
forrectly. 24/05/07 10:36 uk952601 SHI Information: advised pm this issue is Still under investigation as was
lresent on 18th may. awaiting updates. pm is happy. but states
if the reboot doesnt rectify issue pm is going to through
the horizon kit out the window.
lOaté:24-May-2007 10:45:06 User:Anne Chambers
[Start of Response]
[his branch reports that they have been having problems since March 2006 where they do declarations, do further transactions
(usually transfers or rems), redeclare then get a discrepancy equal to the value of the recent transactions.
[two recent examples (all times U'C)
led 16th May: 2000 Euros transferred out at 14:23 then reported as a discrepancy during trial balance at 15:54.
fon 21st May: 8 cash transactions totalling £2465 between 9:35 and 10:16, ignored when cash declared subsequently at 10:20.
[the problems are always in stock unit MS, which is an individual stock unit (hence a variance check is run automatically after a
cash declaration). Almost always using the gateway for this stock unit.
[these type of errors are not that unusual these days (I would say much less rare than a couple of years ago?). We usually quote
KEL MScardifield2219S - I think the outcome of those investigations was that the Riposte message port was possibly sometimes
failing.
liowever given that this problem is not uncommon at this branch, and also that PC0145558 found a problem within Horizon that gave
very similar consequences, I think this might be worth a recheck. I appreciate that with HNG-X coming up, it may not be worth
ursuing this, but this problem is causing a number of Postmasters to have serious doubts about the reliability of their systems,
since they are very obviously displaying incorrect figures.
[End of Response]
lkesponse code to call type L as Category 40 -- Pending -- Incident Under Investigation
esponse was delivered to Consumer
jbate:24-May-2007 11:35:50 User:Anne Chambers
rvidence Added - Full messagestore and 111111111
[Date:24-May-2007 1:
JEvidence Added -
6:14 User:Anne Chambers
1 audit logs
jate:24-May-2007 1:
evidence Added = Summ
rate worksheets)
y of 3 recent problems
Date: 24-May-2007 11:37:07 User:Anne Chambers
fhe call summary has been changed from:-
faD080940 problem when declaring Euros
[fhe call summary is now
JFAD080940 discrepancies when declaring Euros/cash
Jbate:24-May-2007 1
[Start of Response]
nother problem with cash reported this morning. PM threatening to throw kit out of window. I will inform her of the workround
from KEL MScardifield2219S which should avoid reboot.
(0:06 User:Anne Chambers
Please pass call to EPOSS Dev.
[End of Response]
Response code to call type L as Category 40 -- Pending -~ Incident Under Investigation
Response was delivered to Consumer
Date:24-May-2007 11:40:28 Usor:Anne Chambers
fhe Call record has been transferred to the team: QFP
Progress was delivered to Provider
bate:24-May-2007 11:45:16 User:Lionel Higman
[the Call record has been assigned to the Team Member: Mark Scardifield
Progress was delivered to Provider
IDate:25-May-2007 14:06:32 User:Steve Evans
[the Call record has been transferred to the team: EPOSS-Dev
the Call record has been assigned to the Team Member: Mark Scardifield
Progress was delivered to Provider
jDate:25-May-2007 14:27:59 User:Mark Scardifield
this is Steve Evans area. He is on Leave now until 6th June.
Date:29-May-2007 14:15:56 User:Anne Chambers
Reference Deleted: SSCKEL JBallantyne5245K
ltop Reference automatically set to:Powerhelp E-0705180512
FUJ00086490
FUJ00086490
Jbate:29-May-2007 1.
Reference Adde ie1d22193
bate:07-gun-2007 13:54:39 User:Mark Scardifield
{the Call record has been assigned to the Team Member: Steve Evans
Progress was delivered to Provider
jbate:11-Jun-2007 10:47:10 User:_Customer Call_
leMpTY 11/06/07 10:44 UK955761 HSH8 Repeat Call: pm calling to say that she is having to reboot again
joate:25-gun-2007 11:29:41 User:_Customer Call_
EMPTY EMPTY EMPTY OTI Astea OTI Success: An add has been sent to PINICL 25/06/07 1
current problem has also now occured on node
3, PM states node 3 is not adjusting the holdings
6 UK959245 HSH5 Repeat Call: PM states this
Date:27-dun-2007 08:37:31 User: Customer Call_
EMPTY EMPTY EMPTY OTI Astea OTT Success: An add has been sent to PINICL 27/06/07 0:
inform that node 3 did not adjust its
holdings yesterday, PO did not reboot but has cleared itself
lover night.
5 UK959656 HSH2 Repeat Cal
: PO called in to
jDate:18-dul-2007 11:16:12 User:Anne Chambers
[Start of Response]
[this problem is continuing to affect a number of branches, especially larger ones where they have individual stock units and
commonly where they have transferred out cash (which then appears as if it is still in the stock unit - hence they think there is
something wrong with the transfer mechanism). The PMs are not at all happy that the system is giving them inc: figures,
causing much confusion and potentially creating opportunities for fraud.
see PC0147959 for a recent example. I am putting this call up to A priority.
[End of Response]
lkesponse code to call type L as Category 40 -- Pending -- Incident Under Investigation
Response was delivered to Consumer
Jbate:18-dul-2007 11:16:16 User:Anne Chambers
\The call Priority has been changed from B
fhe call Priority is now A
bate:18-dul-2007 11:24:38 User:Steve Evans
[the Call record has been assigned to the Team Member: Gerald Barnes
progress was delivered to Provider
IDate:25-Jul-2007 15:09:48 User:Gerald Barnes
[Start of Response]
lanne documents two instances of the problem. In addition she has added evidence of a third problem
[this is a T30i1 office.
[there was a change at T30i1 in which a cash data tree is maintained which is very similar to the full data tree which already
existed but is only for cash. I suspect the increases of incidence of this problem occurred when the cash tree was added.
Ii could see no logic flaws with the cash tree.
the cash tree logic is in fact straighter forward than the full tree logic and so is easier to validate.
Janne says these problems usually occur with transfers.
in all three cases reported here the first transaction for which a message port notification did not occur was a transfer.
Ifransfers also use a message port notification mechanism similar to the notification mechanism used in the stock and cash trees.
tn all cases I could see from the audit log that the notification of the transfer mechanism, which occurs first, had worked but
that the tree notification had not. I can see that the notify from the transfer mechanism itself did occur
ly present thought is that this intermittent problem occurs because of the use of multiple notifys at the same time. The number
of instances of the problem has gone up because one extra message notification part has been added.
[End of Response]
lkesponse code to call type L as Category 40 -- Pending -- Incident Under Investigation
Ibate:26-gu1-2007 0:
[Start of Response]
IU have a correction from my previous response. Infact the Cash Only tree was introduced at 12013. Other bugs of a similar nature
ave been investigated before e.g. PC0141483.
[End of Response]
Response code to call type L as Category 40 -- Pending ~~ Incident Under Investigation
9:45 User:Gerald Barnes
FUJ00086490
FUJ00086490
Date: 03-Aug-2007 09:
[Start of Response]
Ir noticed that the software had been upgraded to T40il for the last two cases.
7:57 UscriGerald Barnes
I tried to duplicate the third instance. I installed T40i1 on a dual counter system. I imported the supplied message store to a
jdate of 24 May 07:12:17 and, on counter 1 and where necessary counter 2 (to do a transfer out required by counter 1 for a
transfer in) duplicated all transactions done for the day. This time where the discrepancy of £6972 occurred before there was no
roblem.
cheryl Card produced a table of incedences of the problem and one thing to note was that the minimum number of counters the
rablem has occurred is 4.
ly present thought is that it will only be possible to duplicate the bug if you have at least 4 counters and that it is why it
nas never been replicated before. Cheryly noticed also that transfers had beeing going on in all failing cases. I will
investigate this aspect as well.
[End of Response]
Response code to call type L as Category 40 -- Pending -- Incident Under Investigation
Date:03-Aug-2007 14:56:10 User:Gerald Barnes
[Start of Response]
lat a recent meeting between Chris Bailey, Mark Scardifield and I it was decided that I would investigate the feasiblity of
eliminating the use of TALOP_CREATE MESSAGE PORT (which is the call which causes the rebuilding of the trees on the fly by a
notify mechanism) and instead incrementally rebuild the tree when necessary. This will neatly side step the missing notify issue.
[End of Response]
Response code to call type L as Category 40 -~ Pending -- Incident Under Investigation
bate: 08-Aug-2007 11:18:21 User:Gerald Barnes
[Start of Response]
It attach my design proposal for the speculative fix for data server in evidence labelled “Data Server Design Modification" and a
Zip of the source that I have been running and appears to be working fine as evidence labelled "Source modification to Data
server for Tree rescans". All modification to DataServer from the currently released one are marked up with the line containing
~GJB PCO146170 03/08/2007".
[End of Response]
Response code to call type L as Category 40 -- Pending -~ Incident Under Investigation
jbate:08-Aug-2007 11:19:06 User:Gerald Barnes
Evidence Added - Data Server De’
gh Modification
Jat c:08-Aug-2007 11:19:33 User:Gerald Barnes
Evidence Added - Source modification to Dat.
Server
jDate:08-Aug-2007 11:50:34 User:Chris Bailey
jcerald, I have reviewed your proposed changes and this seems to capture the proposed change as discussed between ourselves and
Mark last Friday.
Jbate: 08-Aug-2007 1:
lrarget Date/Time updatec
Development Cost update:
(Start. of Response]
this problem has not yet been reproduced and so it is impossible to be certain that what I propose will fix anything.
9:56 User:Gerald Barnes
new value is 15/08/2007 13:29
new cost is 2 (Man Days)
liowever the problem, which has been around for a very long time, appears to be that the software gets in a state that the notify
echansim for stock trees (and now as a result of a recent change cash trees as well) stops getting though.
the speculative fix, which has already been coded and had initial module testing done on it, is to replace the Notify mechanism
by a method whereby the trees are incrementally rebuilt as required. The mechanism will be switched on by a piece or reference
lata. The order of delivery of code and reference data does not matter but only when both are in place will the change take
effect.
[the new reference data is shown below
kObjectName:
kData:
<NotifyReplacement:
<Enabled:True>
<MinimumRebui ldGapSeconds : 4>
>
in addition the existing trace has been subtly enhanced to include the instance number of the tree used in data server which may
help track down the problem if the speculative fix to DataServer does not fix the problem. Care has been taken so that the
Jdditional trace will have small volume and therefore negligible impact on performance. In addition some trace which was
lspuriously apearing by accident before has been taken out which should normally more than compensate for the trace added.
etx IMPACT
complete Forecast Date and Development (man days) fields below this text box.
Include a brief statement for each of the headings below these instructions.
jon return to Details window Set Target Release Type to “Proposed for" and Target Release to that proposed.
{to the Developer:
(2) Put yourself in the shoes of people downstream and provide information that they are likely to need to process this fix. eg
the testing and rollout costs may add significantly to the COST of the fix
FUJ00086490
FUJ00086490
(2) check that the statements are still accurate post-implementation
HMPACT ON DEVELOPMENT:
lsffort in mandays.
Coding is complete. However checking the source into VSS ready to build and running futher module tests will take me another 2
Inan days. In addition the code and test plan will need to be inspected which will take another half a man day.
[there will be an expectation at RMF that this approximates to the timescale for delivery so if there are reasons why this might
Jnct be the case please note them here.
IMPACT ON TEST:
what independent test coverage does development recommend?
[This will often be about the level of regression testing required.
[his fix affects balancing. I recommend it is given at least two man days of regression testing in the following areas.
boing transactions particularly of stamps and redeclaring cash and stamps often; mainly for individual stock units but also of
shared stock units. Rolling over stock units and the office with some of the rollovers including discrepancies. Doing that
kiescribed before on at least. a two counter office and doing thing on both counters simultaeneously.
Ihe fix potentially could affect reports; whilst doing the above check a few reports.
IMPACT ON USER:
Benefit of making the fix.
if the fix works (and it is a speculative fix so we can not be absolutely certain) bigger office will no longer get into a state
here there cash declarations and rollovers incorrectly report discrepancies.
what does the user have to do to get this problem?
le have never duplicated the problem so we are not sure. However the problem occurs both in cash declaration and rolling over.
[transfers between stock units appears to be a prerequisite for the problem to occur and the problem only appears to occur in
ffices with at least 4 counters.
low does it affect them when it occurs?
Itt is a very irritating problem. Spurious discrepancies are reported and the work around is now quite tedious. At least one post
aster has threatened to throw the kit out of the window!
IMPACT ON OPERATIONS:
Isenefit of fix that may not visible to end user.
le will have less support calls because there will no longer be calls about spurious discrepancies.
IkISKS (of releasing and of not releasing proposed fix):
hat Live problems will there be if we do not issue this fix?
spurious discrepancies will most certainly continue to occur.
hat are the risks of this fix having unexpected interactions with other areas?
Ithe fix does affect balancing and so it is conceivable that a problem will crop up in this area.
Its this a high-risk area in which changes have caused problems in the past?
Definitely yes!
should we consider a pilot rollout and of what sort?
It think this is a very sensible idea. Just put the fix out to a small sub set of offices and if they report no problems after at
least a month including an office rollover send it out everywhere. Although the problem we are trying to fix only occurs in big
offices; the successful running of the software on offices with only two counters is quite sufficient to prove it.
LIST OF LIKELY DELIVERABLES:
lepossDataserver
LIST OF THE ABOVE ALREADY DELIVERED FOR THE PROPOSED RELEASE:
jone
I.Isv OF THE ABOVE ALREADY DELIVERED TO A RELEASE LATER THAN THAT PROPOSED:
None
LIST OF THE ABOVE LIKELY TO BE REDELIVERED INTO THE PROPOSED OR A LATER RELEASE:
lepossDataserver
IANYTHING ELSE THAT SHOULD BE KNOWN ABOUT THIS CHANGE:
[End of Response]
Response code to call type L as Category 55 -- Pending -- Live Fix Impact Supplied
JDate:08-Aug-2007 12:41:08 Uscr:Geraid Barnes
[the call Target Release has been moved to Proposed For ~~ 150
FUJ00086490
FUJ00086490
[Date:08-Aug-2007 12:43:44 Uscr:Gerald Barnes
the Call record has been transferred to the team: RelMngmntForum
Progress was delivered to Provider
[Date:09-Aug-2007 17:06:46 User:John Budworth
the call Target Release has been moved to Targeted At —- T60
lDate:09-Aug-2007 17:16:03 User:John Budworth
[Start of Response]
targeted at
IsROSS-Dev to deliver by close of play on Tuesday August
aperwork for LST on the morning of Wednesday August 15th latest.
outing to RDT in the first instance to create a clone for the associated Ref Data.
Iapt to route to EPOSS-Dev.
[End of Response]
Response code to call
type L as Category 56 -- Pending -- Live Fix Authorised
14th if possible as Release Management would like
to raise release
[Date:09-Aug-2007 17:17:57 User:dohn Budworth
[the Call record has been transferred to the team: Ref-DataCS-Liv
Progress was delivered to Provider
Dat e:09-Aug-2007 17:18:23 User:John Budworth
Product DevintRel-Director -- Live Supp.Test
(version unspecified) added.
[Date:09-Aug-2007 17:20:01 Uscr:Rob Gelder
taking Clone for provision of Ref Data
Date:09-Aug-2007 17:20:11 User:Rob Gelder
{the call summary has been changed from:—
fAD080940 discrepancies when declaring Euros/cash
the call summary is now
IFADO80940 discrepancies when declaring Euros/cash-Code
Dat ¢:09-Aug-2007 17:20:15 Uscr:Rob Gelder
call has been cloned to Call:PC0148615 by User:Rob Gelder
Date: 09-Aug-2007 17:20:39 User:Rob Gelder
Defect cause updated to 40: General - User
[Dat e:09-Aug-2007 1°
the call
progress was delivered to Provider
0:46 Uscr:Rob Gelder
record has been transferred to the team: EPOSS~Dev
Jbate:10-Aug-2007 14:11:57 User:Kath Greenwood
fhe Call record has been assigned to the Team Member:
Progress was delivered to Provider
Gerald Barnes
bate:13-Aug-2007 1
[Start of Respons
1:15 User:Gerald Barnes
J
Unit Testing has been performed as specified in the attached Unit Test Plan
ip file which also contains two spreadsheets to which it refers. The zip file
Done
[End of Response]
Response code to call type L as Category 44 -- Pending -- Fix in Progress
(UTP146170.doc). The test plan is contained within a
is attached as evidence
labelled "Unit Testing
JDate:13-Aug-2007 1!
User:Gerald Barnes
Evidence Added ~ Un: ting
at e:14-Aug-2007 1:
JEPOSS-DEV
6:04
Solution Review
User:Steve Evans
Prior to internal handover and final unit testing, a review of the solution is required.
lin addition the proposed local Unit Test Plan should be attached, to aid the review.
(To be completed by the reviewer)
viable and comprehensive local Unit Test Plan which is sufficient to test the solution,
ji am satisfied that the proposed solution has been;
has been attached
(UTP146170.doc)
FUJ00086490
FUJ00086490
agreed with and underwritten by Design (¥)
Implemented according to this agreement in the proposed fix for this fault.
Notes:
Proviso: This PEAK has not been reproduced and so the UIP does not directly test the
the new mechanism works.
[this design and code change clearly achieves this.
‘fix' against the live problem; only that
Jbate:14-Aug-2007 13:08:03 User:Gerald Barnes
[Start of Response]
lrixed (hopefully - it is a speculative fix) by a new release of DataServer.
[End of Response]
Response code to call type L as Category 46 -~ Pending -~ Product Error Fixed
Date:44-Aug-2007 13:08:14 Uscr:Gerald Barnes
the Call record has been transferred to the team: EPOSS-Rel
progress was delivered to Provider
Date: 14-Aug-2007 16:53:44 User
Reference Adde
‘Tyrone Cozens
0148739
Release Pi
jDate:14-Aug-2007 16:55:22 User:Phil Budd
IEPOSS-REL HANDOVER QUALITY AUDIT CHECKLIST
Prior to formal Handover to the Testing function, an audit of required processes should be completed.
(fo be completed by the release-builder)
ji. Design/Solution Documentation attached or In-Line (¥)
2. EPOSS-DEV Solution Review complete (¥)
3. EPOSS-DEV Unit Test Notification complete (¥)
4. EPOSS-DEV Reference Data Notification complete (N)
Is. Reference Data change attached (N, see PC0148615)
JI am satisfied that the proposed solution has been processed correctly (Y)
jotes:
ix released in WP24992 for Té60il
JDate:14-Aug-2007 16:59:51 User:Phil Budd
Reference Added: Work Package PWY WP 24992
[Date:14-Aug-2007 1
TOP Reference set t
9:54 User:Phil Budd
Work Package PWY WP 24992
Jbate:14-Aug-2007 17:00:32 User:Phil Budd
[the Call record has been transferred to the team: Dev-Int-Rel
Progr was delivered to Provider
oate:14-Aug-2007 17:53:55 User:PIT Automated User
keference Added: Fast Track Fix FSTK_2_0 WP24992 (TOP Reference)
Jbate:14-Aug-2007 17:53:56 User:PIT Automated User
(Start of Response]
Fasttrack fix released, now ready for test
[End of Response]
Response code to call type L as Category 46 (Product Error Fixed)
the incident has been transferred to the team: Live Supp.Test
Progress was delivered to Powerhelp
[Date:45-Aug-2007
reference Added:
User:Bdward Willis
inTCL PCOIAaT 6G
JDate:17-Sep-2007 09:54:34 User:John Budworth
[Start of Response]
FUJ00086490
FUJ00086490
Routing by to EPOSS-Dev for attention of Gerald Barnes.
the fix for this PEAK was reversed in live (549 Branches) via Ref-Data
4/9/07. See associated PEAKs 159506 and 149540.
[End of Response]
Response code to call type L as Category 50 -- Pending -- Fix Failed
[Date:17-Sep-2007 09:54:52 User:John Budworth
the Call record has been transferred to the team: EPOSS-Dev
Progress was delivered to Provider
bate:17-Sep-2007 10:30:34 User:Mark Scardifield
the Call record has been assigned to the Team Member: Gerald Barnes
progress was delivered to Provider
bate:18-Sep-2007 1:
[Start of Response]
It have found an intermittent problem with the fix in PWY_WP_24992. It occurs for me about 1 in 5 stock unit rollovers into a new
trading period when there is a discrepancy. It was not spotted in module testing prior to release because I did not perform a
sufficient number of rollovers into a new trading period with a discrepancy.
0:48 User:Gerald Barnes
[the problem occurred because of my change from a notify mechanism to an incremental build mechanism.
[fhe basic change worked fine. No problems were reported with cash declarations (the key area effected) prior to the rollover day.
However on rolling over into a new trading period problems were detected with discrepancies on some counters.
the problem was tracked down to my implementation of the bIgnoreMsgPort private variable of DataServer.
ising the Notify mechanism this variable when set caused transactions not to go into the tree.
ly implentation checked the blgnoreMsqPort when incremntally building the tree. This is however flawed logic. The correct mapping
lof the blgnoreMsgPort behaviour to the new incremental build mechanism is to build up a list of message ranges for when it is
switched off and switched on and then when doing the incremental build checking each transaction against these ranges before
jputting the transaction in the tree.
[fhe bIgnoreMsgPort functionality is only used when rolling over with discrepancies and hence that is why the intermittent problem
loccurred on that day.
{End of Response]
Response code to call type L as Category 40 -- Pending -- Incident Under Investigation
bate:18-Sep-2007 15:56:12 User:Gerald Barnes
Target Date/Time updated: new value is 28/09/2007 13:29
Development Cost updated: new cost is 4.0 (Man Days)
[Start of Response]
Ir have module tested a new release of dataserver which (with incremental building enabled) handles blgnoreMsgPort in the manner
ldescribed in my response “Date: 2007-09-18 12:10:48 User:Gerald Barnes". No problems were observed in 12 trading period rollovers
ith discrepancy.
lat a meeting between Chris Bailey, Mark Scardifield and myself in Airmail today between lpm and 2pm it was decided this was the
forward path on this problem. However it was noted that there may well be other yet unforeseen problems lurking with the proposed
fix and so a thorough testing is essential.
JeTX IMPACT
complete Forecast Date and Development (man days) fields below this text box.
include a brief statement for each of the headings below these instructions.
n return to Details window Set Target Release Type to "Proposed for" and Target Release to that proposed.
Ho the Developer:
(Q) Put yourself in the shoes of people downstream and provide information that they are likely to need to process this fix. eg
the testing and rollout costs may add significantly to the COST of the fix
(2) Check that the statements are still accurate post-implementation
IMPACT ON DEVELOPMENT:
leffort in mandays.
[there will be an expectation at RMF that this approximates to the timescale for delivery so if there are reasons why this might
not be the case please note them here.
Ja. man days mostly of further module testing. The coding has already been done.
fhe further testing is to be broken down as follows
5 day reviewing each line changed whilst running EPOSSDataServer in the IDE whilst doing cash declarations and rollovers and
checking each such line behaves as expected.
5 day doing rollovers on two counters side by side (mimicking all transactions done) from a scratch build one with the enabling
jreference data and one without and checking that all printed results come out the same.
2 days doing rollovers with discrepancies (and a broad range of transactions between) on two counters side by side with and
without the enabling reference data starting with a messagestore imported from live mimicking the key strokes done on each
counter and checking all printed reports are the same.
i day for administration (checking the code into VSS, link testing, preparing handover forms etc.)
HMPACT ON TEST:
FUJ00086490
FUJ00086490
hat independent test coverage does development recommend?
[this will often be about the level of regression testing required.
two days of further testing mainly of cash declarations and rollovers with discrepancies preferably using as a start point an
imported live message store. As a variation from the module testing done this should be done on at least a dual counter office.
the testing should be done mainly with the enabling reference data. However a small amount of testing should be done with the
jenabling reference data switched off such that in the event the fix needs to be reversed again by reference data there is
confidence in advance for the mechanism.
HMPACT ON USER:
Benefit of making the fix.
Ite the fix works (and it is a speculative fix so we can not be absolutely certain) bigger office will no longer get into a state
here there cash declarations and rollovers incorrectly report discrepancies.
hat does the user have to do to get this problem?
le have never duplicated the problem so we are not sure. However the problem occurs both in cash declaration and rolling over.
lfransfers between stock units appears to be a prerequisite for the problem to occur and the problem only appears to occur in
offices with at least 4 counters.
lHow does it affect them when it occurs?
Itt is a very irritating problem. Spurious discrepancies are reported and the work around is now quite tedious. At least one post
master has threatened to throw the kit out of the window!
IMPACT ON OPERATIONS
lsenefit of fix that may not visible to end user.
le will have less support calls because there will no longer be calls about spurious discrepancies.
IRISKS (of releasing and of not releasing proposed fix):
lwhat live problems will there be if we do not issue this fix?
spurious discrepancies will most certainly continue to occur.
hat are the risks of this fix having unexpected interactions with other areas?
the fix does affect balancing and so it is conceivable that a problem will crop up in this area.
Its this a high-risk area in which changes have caused problems in the past?
Definitely yes!
should we consider a pilot rollout and of what sort?
[there must be a pilot. Put the fix out to a small number of Office and see whether there are any problems on trading period
jrollover. If there are none then release the fix to more offices and see whether there are any problems with trading period
rollover etc. Although the problem we are trying to fix only occurs in big offices; the successful running of the software on
offices with only two counters is quite sufficient to prove it.
LIST OF LIKELY DELIVERABLES:
lepossDataserver
LIS? OF THE ABOVE ALREADY DELIVERED FOR THE PROPOSED RELEASE:
lsPoSsDataserver
LIST OF THE ABOVE ALREADY DELIVERED TO A RELEASE LATER THAN THAT PROPOSED:
lone
LIST OF THE ABOVE LIKELY TO BE REDELIVERED INTO THE PROPOSED OR A LATER RELEASE:
lePossDataserver
IANYTHING ELSE THAT SHOULD BE KNOWN ABOUT THIS CHANGE:
fhe change will be able to be disabled by reference data if nec
[End of Response]
Response code to call type L as Category 42 -- Pending -- Product Error Diagnosed
sary. The disabling will only take effect after reboot though.
jDate:18-Sep-2007 16:00:40 Uscr:Gerald Barnes
[fhe Call record has been transferred to the team: RelMngmntForum
Progress was delivered to Provider
Joate:21-Sep-2007 09:15:14 User:Mark Scardifield
It notice that Gerald hasn't set a proposed target release. Logically this is a fix on top of the earlier T60 "attempt" however we
lare nearly out of 160 so I suspect T70 is more realistic.
jDate:04-Oct-2007 16:00:33 User: Tyrone Cozens
the call Target Release has been moved to Targeted At -- T70
FUJ00086490
FUJ00086490
IDate:04-Oct-=2007 16:01:07 User:Tyrone Cozens
[Start of Response]
kM authorise a fix for 170.
{End of Response]
Response code to call type L as Category 56 -- Pending -- Live Fix Authorised
[Date:04-Oct-2007 16:01:26 User: Tyrone Cozens
the Call record has been transferred to the team: EPOSS-Dev
JDate:04-Oct-2007 16:03:34 Uscr:Gerald Barnes
[the Call record has been assigned to the Team Member: Gerald Barnes
lDate:05-Oct-2007 17:27:04 Uscr:Gerald Barnes
the Call record has been transferred to the team: Ref—DataCS-Liv
the Call record has been assigned to the Team Member: Rob Gelder
Dat e:08-Oct-2007 09:36:45 User:Rob Gelder
faking Clone for Revised Ref Data
jDate:08-Oct-2007 09:36:50 Uscr:Rob Gelder
call has been cloned to Call:PC0150233 by User:Rob Gelder
Jbate:08-Oct-2007 09:37:42 Uscr:Rob Gelder
the Call record has been transferred to the team: EPOSS-Dey
fhe Call record has been assigned to the Team Member: Gerald Barnes
Date:10-Oct-2007 14:
[Start of Response]
jodule testing has now been completed on the T70i1 DataServer source with the label "Module Tested PC0146170".
6:39 User:Gerald Barnes
fhe module testing material is attached in a zip labelled "Test Material for initial failure".
[End of Response]
Response code to call type L as Category 44 -- Pending -- Fix in Progress
JDate:10-Oct-2007 1
Evidence Added - 1
User:Gerald Barnes
fal
[Dat e:10-Oct-2007 16:42:37 User:Steve Evans
[Start of Response]
IEPOSS-DEV Solution Review
prior to internal handover and final unit testing, a review of the solution is required.
tn addition the proposed local Unit Test Plan should be attached, to aid the review.
(To be completed by the reviewer)
[within PC0146170_175256[1].zip] - attached as ‘Test Material for initial failure’)
It am satisfied that the proposed solution has been;
Agreed with and underwritten by Design (NA)
I-Implemented according to this agreement in the proposed fix for this fault.
ote:
the tests look thorough, and the code has no obvious flaws.
[End of Response]
Response code to call type L as Category 44 -- Pending -- Fix in Progress
la viable and comprehensive local Unit Test Plan which is sufficient to test the solution, has been attached (UIP146170.doc
lDate:10-Oct-2007 17:00:49 User:Steve Evans
[Start of Response]
Y “AMENDED
IEPOSS-DEV Solution Review
prior to internal handover and final unit testing, a review of the solution is required.
tn addition the proposed local Unit Test Plan should be attached, to aid the review.
(To be completed by the reviewer)
FUJ00086490
FUJ00086490
Viable and conprehensive local Unit Test Plan which is sufficient to test the solution, has been attached (UIPI46170.doe
[within PC0146170_175256[1].zip] - attached as ‘Test Material for initial failure’)
It am satisfied that the proposed solution has beens
Agreed with and underwritten by Design (Y)
see entry above: Date:18-Sep-2007 15:56:12 User:Gerald Barnes
Gareth Jenkins was also consulted on, and sanctioned this further change.
I-Implemented according to this agreement in the proposed fix for this fault.
Notes:
fhe tests look thorough, and the code has no obvious flaws.
{End of Response]
lkesponse code to call type L as Category 44 -~ Pending -- Fix in Progress
[End of Response]
kesponse code to call type L as Category 44 -- Pending -- Fix in Progress
lbate:10-Oct=2007 1
[Start of Response]
Fixed by a new release of EPOSSDataServer.dll.
[End of Response]
Response code to call type L as Category 46 -- Pending -- Product Error Fixed
0:50 Uscr:Gerald Barnes
Jbate:10-Oct-2007 17:21:01 User:Gerald Barnes
Ithe Call record has been transferred to the team: EPQSS-Rel
jOate:17-Oct-2007 10:05:40 User:Gerald Barnes
[Start of Response]
Ii proceeded with my Link Test Phase. I imported the message store attached to PC0149476 and did parallel runs on two counters -
lone with the fix enabled and one with the fix disabled. When I rolled over stock unit ATM a difference was observed. The fix
Inceds to be studied again.
[End of Response]
lkesponse code to call type L as Category 38
Pending -- Potential Problem Identified
date:18-Oct-2007 09:40:58 Uscr:Gerald Barnes
[the Call record has been transferred to the team: EPOSS-Dev
[fhe Call record has been assigned to the Team Member: Gerald Barnes
jDate:18-Oct-2007 10:17:35 Uscr:Gerald Barnes
[Start of Response]
It attach my analysis of the link test failure as evidence labelled "Proposed Design Modification for Link Test Failure”.
[End of Response]
Response code to call type L as Category 38 -- Pending -- Potential Problem Identified
[Date:18-Oct-2007 1!
Jsvidence Added - F
0:17 Uscr:Gerald Barnes
for Link 1 Failure
IDate: 22-Oct-2007 0
[Start of Response]
S a result of testing and as anticipated in the previously attached design it was discovered that EPOSSDeclaration needs to be
reissued as well. It is a one line change to tell Dataserver to rescan incrementally just before calculating discrepancies.
[End of Response]
Response code to call type L as Category 40 -- Pending -- Incident Under Investigation
8:15 User:Gerald Barnes
bate:22-Oct-2007 1
[Start of Response]
Further testing has revealed a one line modification is required to EPOSSStockUnit as well of the same nature as the change to
leposSbeclare.
6:21 User:Gerald Barnes
IepoSsStockUnit is in the HNGX stream as well. I shall clone the PEAK for this reason.
[End of Response]
Response code to call type L as Category 44 -- Pending -~ Fix in Progress
jate:22-Oct-2007 16:36:26 Uscr:Gerald Barnes
call has been cloned to Call:PC0150568 by User:Gerald Barnes
lDate:23-Oct-2007 1
[Start of Response]
fodule testing is complete and the change is ready for a solution review.
8:49 Uscr:Gerald Barnes
[fhe changed modules are now EPOSSDataServer.dll, EPOSSDeclare.dll and EPOSSStockUnit.dl1. All source modules module tested have
been labelled in VSS “PC0146170 Solution Review Two".
FUJ00086490
FUJ00086490
jodule tests material is attached in the zip file labelled “Module Test Material 2”
[End of Response]
Response code to call type L as Category 44
Pending -- Fix in Progress
jbate:23-Oct-2007 16:39:44 Uscr:Gerald Barnes
evidence Added ~ Module Test Material 2
Joate:25-Oct-2007 18:06:15 User:Miriam Bell
[Start of Response]
POSS-DEV Solution Review
rior to internal handover and final unit testing, a review of the solution is required.
in addition the proposed local Unit Test Plan should be attached, to aid the review.
(To be
completed by the reviewer)
A viable and comprehensive local Unit Test Plan which is sufficient to test the solution, has been attached (UTP146170_2.doc)
Ii am satisfied that the proposed solution has been;
Agreed with and underwritten by Design (¥)
lcerald tells me that he discussed the solution to part 2 of this Peak fix with Gareth.
Implemented according to this agreement in the proposed fix for this fault.
Notes:
8 Gerald has pointed out, this is a high risk fix in that is is possible that an incremental scan has not been invoked from
Jsomewhere which needs to do it. Geral has addressed this by coding in such a way that the fix can be reversed by changing
lreference data. The tests look thorough. A very minor comment is that I happened to notice a misleading comment in
lsPOSsDataServer/clsInternalSession.cls - ResetAllNodes, which says before the call to fForceRescan 'It would be better to have
made a direct call from EPOSSStockUnit but then EPOSSStockUnit would have had to be released as well.' EPOSSStockUnit IS now
being released as part of the latest fix.
[End of Response
Response code to call type L as Category 44 -- Pending -- Fix in Progress
Date:25-Oct-2007 18:11:33 User:Miriam Bell
[Start of Response]
viously T cannot check that this fix is correc’
[End of Response]
lkesponse code to call type L as Category 44 -- Pending -- Fix in Progress
I have diff'ed the code changed and it doesn't look wrong so far as I can see.
JDate:25-Oct-2007 18:44:07 Uscr:Gerald Barnes
[Start of Response]
Fixed by new releases of EPOSSDataServer, EPOSSDeclare and EPOSSStockUnit.
he fix is labelled "Handover Number 2 for PCO146170" in VSS.
jote that although I have tested this fix to the best of my ability and am about to conduct a further thorough link test
involving parallel runs rolling over stock units on machines side by side with and without the fix enabled from an imported
essage store it is still a relatively high risk one.
kor this reason the fix has also been thoroughly tested without the enabling reference data. Should there be a problem the fix
can be disabled by end dating or deleting the enabling reference data EPOSSDataServer Parameters.
[this approach is recommended because it will avoid creating a log jam of other fixes on top of these dlls. In addition even with
the fix disabled there is useful extra diagnostics within the fix.
[End of Response]
Response code to call type L as Category 44 -- Pending -- Fix in Progress
Date:25-Oct-2007 18:44:35 Uscr:Gerald Barnes
[the Call record has been transferred to the team: EPOSS-Rel
0:12 User:Phil Budd
[Start
lcerald is still finishing off his thorough testing on the link test rig but I expect him to have an answer today, I will then
progress the Peak.
[End of Response]
Ikesponse code to call type L as Category 44 -- Pending -- Fix in Progress
Date: 30-Oct-2007 09:27:06 User:Gerald Barnes
[Start of Response]
I have conducted a link test consisting of importing the message store attached to PC0149476 to a date and time of <Date:12-Sep-
2007><Time:12:18:59> on two counters one with the fix enabled and one with it disabled and then rolling over stock units 01, 02
land 04 and checking all figures displayed on the screen and printed. They were all the same.
[End of Response]
Response code to call type L as Category 44 -- Pending -- Fix in Progress
FUJ00086490
FUJ00086490
IDate:30-Oct-2007 09:44:28 Us
[start of Response]
IEPOSS-REL HANDOVER QUALITY AUDIT CHECKLIST
2r?Phil Budd
Prior to formal Handover to the Testing function, an audit of required proc:
‘a
hould be completed.
(To be completed by the release-builder)
LH
Design/Solution Documentation attached or In-line (¥)
2. EPOSS-DEV Solution Review complete (¥)
3. EPOSS-DEV Unit Test Notification complete (¥)
4. EPOSS-DEV Reference Data Notification complete (N)
ls. Reference Data change attached (Not here, see PC0150233)
Ir am satisfied that the proposed solution has been processed correctly (Y)
jotes:
lrix released in WP25065 for T70i1. <Collection:EPOSSDataServer><ObjectName:Parameters> is required to enable the fix. The order
lof delivery of code and reference data does not matter but only when both are in place will the new functionality take effect.
IPCO150233 is the clone of the PEAK for the provision of this reference data.
[End of Response]
Response code to call type L as Category 44 -- Pending -- Fix in Progress
lbate:30-Oct-2007 09:44:56 Uscr:Phil Budd
Reference Added: Work Package PWY WP 25065
Dat e:30-Oct-2007 0:
fOr Reference set t
5:01 User:Phil Budd
Work Package PWY_WP_25065
JDate:30-Oct-2007 09:45:15 User:Phil Budd
[fhe Call record has been transferred to the team: Dev-Int-Rel
bate: 30-Oct-2007 15:24:52 Uscr:PIT Automated User
lkeference Added: Fast Track Fix FSTK 20 WP25065 (TOP Reference)
pate:30-Oct-2007 15:24:34 Uscr:PIT Automated User
{Start of Response]
MFasttrack fix released, now ready for test
[End of Response]
Response code to call type L as Category 46 (Product Error Fixed)
[the incident has been transferred to the team: Live Supp.Test
jDate:30-Oct-2007 1!
Reference Adde:
2:29 User: Tyrone Cozens
le.
JDate:12-Nov-2007 10:27:29 User:Tyrone Cozens
[Start of Response]
24908, 25061, 25065 and 25087 passed test on 09/11/07 via RNT7018. Routing for closure.
[End of Response]
Response code to call type L as Category 60 -- Final -- S/W Fix Released to Call Logger
Routing to Call Logger following Final Progress update.
Date:12-Nov-2007 10:27:59 Uscr:Lorraine Guiblin
{fhe Call record has been assigned to the Team Member: Anne Chambers
jDate:19-Dec-2007 1:
[Start of Response]
[this fix has been on pilot at 50 branches for some weeks now, with no problems reported. It will be fully rolled out in the new
lyear. I'm reducing the priority as requested.
[End of Response]
Response code to call type L as Category 46 -- Pending -~ Product
9:29 User:Anne Chambers
vor Fixed
jbate:19-Dec-2007 11:09:39 Uscr:Anne Chambers
[the call Priority has been changed from A
[the call Priority is now B
FUJ00086490
FUJ00086490
[bate:17-dan-2008 10:15:15 User:Anne Chambers
(Start of Response]
Fix now out everywhere (COUNTER EPOSS 39 3). Closing call.
[End of Response]
Response code to call type L as Category 60 -- Final -- S/W Fix Released to Call Logger
outing to Call Logger following Final Progress update.
jbate:17-Jan-2008 10:15:22 User:Anne Chambers
CALL PC0146170 closed: Category 60 Type L
Root Cause General - User
Logger _ Customer Call_ -- EDSC
Subject Product EPOSS & DeskTop -- Counter Common (version unspecified)
Assignee _Customer Call_ -- EDSC
Last Progress 17-Jan-2008 10:15 -- Anne Chambers