The DOBlog

Author: Daragh

  • Excel – some noteworthy c*ckups…

    The nice people over at the European Spreadsheet Risks Interest Group have an excellent website with good examples of spreadsheets gone wrong.

    Here’s one about sorting errors…

    And another… here

    And another… here

    and another… here

    and another … here

    and another… here (this one relates to electoral roll data)

    and another… here

    Some of these links might be broken now as some of the articles are a bit old but the EUSPRIG site has summaries of the stories.

    My finger is sore from cutting and pasting links… if you want to see more on the risks of relying on spreadsheets, check out http://www.eusprig.org/stories.htm

  • Finger print scanning and Indelible Ink in the Electoral Process

    I have been distracted from compiling my post on the use of excel as a data interchange mechanism by some other snippets of what Dick Roche said at yesterday’s Dail committee meeting as reported in the Irish Times (page 9) and Dublin’s freesheet Metro (page 5)…

    1. The Census Enumerators haven’t gone away… €5 million to €6 million is being provided to Local Authorities to hire temporary staff “such as census enumerators” to carry out door to door inquiries. The Unions haven’t gone away either though and I suspect that Local Authority staff currently doing those jobs might be offended by this idea. And at the end of the day it is €6 million between 114 Local Government agencies (29 County Councils, 10 City/Borough Councils and 75 Town Councils) or an average of €52632 per agency to pay temporary staff for the duration of the scrap and rework. (that calculation assumes that Town Councils have a role to play in the Electoral Register – if Town councils can be excluded that amounts to just under €154k per Local Authority). Of course, this is still just scrap and rework – the process of gathering the information will use the same apparently broken processes to capture the new register. Once the cleanup is finished, unless there is a study of the root causes, the same defective processes will operate to corrupt the register almost immediately and taxpayers will find themselves having to fund another €6million clean up in the not to distant future.
    2. The Minister has put forward a proposal to avoid voter fraud at the Polling stations in the next election. “Indelible Ink or fingerprint scanners could be used in polling booths at the next poll” (source: Metro). Of the ink, the Minister is quoted as saying “it woudl be a badge of pride that you had participated in democracy”. Allllllrrrrrriiiiiigggggghhhhhtttttyyyyy then.

    My first recommendation is to ban CSI and its spin off shows from the Roche household. Capturing a fingerprint at the time of a ballot would be totally useless unless there was a master data source of citizen biometric data that could be referenced – even Gil Grissom has to have a finger print in the database before he can nab the criminal.

    This is a proposal that I would suggest Digitial Rights Ireland jump all over quick smart as, in my view, this combined with the call to use PPS numbers brings us one step from a ‘big brother’ single view of citizen with biometric data. And as the Government seems to be incapable of properly managing complex technology projects I would be very concerned by this.

    I’ll get my post on Excel done over the coming days – unfortunately pesky day job is getting in the way. ;(

  • Electoral Register on Oireachtas Report

    The coverage of the Dail Committee meeting on Oireachtas Report this evening was a little disappointing. They only covered the Minister’s speechifying on the topic and didn’t show if there had been any debate or challenge on the topic from any of the Opposition politicians who sit on the Committee such as Labour’s Eamon Gilmore.

    My feeling is that anyone who has been following my posts on this site and is familiar at any level with the scrap & rework concept would have found a few open goals in what Dick Roche was putting forward.

    1. Use of ‘relevant databases’ within Local Authorities – this amounts to a process change to address a root cause (well done Minister). However there may be data protection issues given that data can only be used for the purposes for which it is captured – this fact was glossed over by the Minister. In addition, just because one has a second surrogate source for reality it doesn’t mean that it is any more accurate or complete than your ‘problem’ dataset. Furthermore, what tools do Local Authorities have available to them to actually match across these datasets quickly and accurately (short of manual efforts)? The Minister has been somewhat quiet on the ‘how’.
    2. The Minister referred to using ESB databases to update the register on the basis that we all use the ‘leccy. Again, this is not a complete surrogate for reality as it would hold the details of only one, at most two, individuals at an address. That data may not match against the Electoral Register for a variety of reasons (different spelling of names, use of aliases or akas (e.g. John Vivian Smith may be listed on the ESB bill as J.Vivian Smith or just plain old Vivian Smith or just J Smith)) – Again Data Protection issues (and commercial considerations) exist here but were glossed over by the Minister.
    3. Finally, Minister Roche is plowing ahead with the plan to give records of deceased persons to Local Authorities. The data will be sent in Excel format. There are issues with that idea that I will address in a later post (once I have gotten my thoughts on it ironed out) but in the meantime the question that arises is simply this – once the Local Authorities have their Excel spreadsheets of names and addresses of the deceased, how will they go about matching them against the Electoral Register and flagging matches for action? What is the process that will operate to convert raw data into actionable information?

    This issue is a national crisis. The Government seems content to operate on a scrap and rework basis with no real thought leadership or strategy. It is all well and good to identify potential sources of data against which the accuracy of the register can be checked, but in the absence of clear processes to use that information or the capability to process it all that Local Authorities will have is more data and a Minister who’s approach has been to criticise and berate and to effectively manage by sound-bite. Provision of ‘ring fenced’ budget and 3rd party data sets will not enable Local Authorities to perform their duties re the Electoral Register any better if there is no understanding of what needs to be done to address these problems in the long term.

    Where is the action plan to assess root causes and address the core deficiencies in the Electoral Register process? Where is the leadership in terms of thinking and in terms of control of the processes?

    I wonder what the boys and girls over at Digital Rights Ireland would have to say about the ESB providing its billing database or customer relationship management database to EVERY Local Authority (in excel format perhaps)?

    My suggestion of 2 weeks ago still stands…

    1. Scrap the current register
    2. Mail everyone on the current register and request that they re-register
    3. Use that register as the appropriate data set for the Draft Register
    4. In parallel assess Root Causes – involve Local Authority staff (not County Managers but Revenue Collectors, Librarians – anyone with a role in the process) to identify the root causes – even go as far as to talk to (dare I say it) Citizens to find out what is broken in the processes
    5. Develop a plan to address those root causes, addressing governance issues and Information Architecture issues as well as cultural issues.

    Scrap and Rework without corresponding review and improvement of processes will not solve the problems in the Register, it will merely postpone them until a later date.

  • Dail Committee Discussion on Electoral Register issues

    There is a Dail Committee discussion on the Electoral Register issue being held this afternoon at 16:15. The Minister for the Environment will be facing questioning on what is being done to fix our shambles of an electoral register.

    According to the published schedule, this discussion is to be televised. I’m not sure exactly where on the Oirechtas.ie website the Dail committee discussion will be televised, but the link to the general page is here.

    Oireachtas Report is on RTE1 at 23:40 tonight so it might carry coverage there.

     

  • Conflict of Expectations

    A few weeks back, I was at dinner with Larry English and some people from a large consulting firm. It was a social occasion and as the food and informality took hold, the discussion turned to the pillars of the Total Information Quality Management methodology and ethical/moral concerns.

    Mr English draws heavily on Deming and the TQM gurus in his view of quality – quality is defined as meeting or execeeding customer expectations and the most important person in the ‘production line’ is the customer. The challenge to this was what would one do if your customer wanted something illegal or immoral to meet their expectation or if, in order to meet or exceed their expectation, you had to do something illegal or immoral.

    My response to this argument was to point out that the expectation of society as to the acceptable standards of conduct are expressed in law, both statute and judicial. Just because a product or service meets or exceeds the expectations of a targetted customer it does not mean that it can be produced if the laws of society would preclude it. A good example of this is the location tracking tools that were being deployed by a Dublin security company. The prompt action of Digital Rights Ireland in alerting the Data Protection Commissioner resulted in the expectation of society of privacy (albeit curtailed somewhat for children) was asserted and the product had to be withdrawn.

    Where individuals or organisations choose to offer services that are illegal or immoral, and where those services are their business, then it can be argued that their obligation to their customer is the same as a business operating within the law. If a drug dealer (for example) provides low quality product then they will not survive long. Either in business or life.

    The choice is whether to operate within the law or not, not whether you need to produce a quality product that meets or exceeds your customer expectations. If you choose to operate within the law, then the legal process will operate (albeit at times slowly) to assert the expectation of Society as a customer. If you choose to act outside the law, producing or selling a product that is illegal, then your obligation to your chosen customer remains the same.

     

  • Amazon-inania update on Michele Neylon’s blog…

    His posted update… http://www.mneylon.com/blog/archives/2006/05/07/amazon-uk-shift-the-goalposts/#comment-7815

    my comment : http://www.mneylon.com/blog/archives/2006/05/07/amazon-uk-shift-the-goalposts/#comment-7818

  • Electoral Information Quality – A Consolidating post

    As the blog is getting legs a bit now, I thought it best to consolidate the posts of the last few weeks on the Electoral Register issues into one point of reference, particularly for readers new to the site.

    I am also taken the opportunity to upload a few additional articles etc. that I have written on the issue to the blog for reference.

    Articles:

    First up is a draft paper I have put together on the proposed solutions and why they are likely to be inadequate. 

    Next up is a link to an article I have had published in an International newsletter for Information Quality Management Professionals.

    Finally there is an article based on my post on what scrap and rework is of earlier this month. This article was submitted to national newspapers as an opinion piece – and I should acknowledge the assistance of Simon over on Tuppenceworth with whipping it into shape. Click here to download Scrap and Rework article. The article is also reproduced as an appendix in the previously mentioned report.

    As regards posts – pretty much any of the posts in the Information Quality/Electoral Data Quality category are relevant. I will double check all the post categorisations to make sure that nothing is missing.

    That’s my update for today.

     

     

  • Amazon-inania update

    I used the contact form on the Dept of Enterprise site last night to alert them to the Amazon-inania…

     Here is their response… which includes the email and name of the Private Secretary to the Minister, which I’ve cut out for sake of privacy. If anyone wants them, comment and request (I will not post this information publicly at this time). I’ll ignore the fact that they spelled my name wrong even though all they had to do was cut and paste it.

    Dear Darragh

    Your correspondence has been forwarded to [name witheld by webmaster], Private Secretary to the Minister for Enterprise, Trade and Employment, Micheál Martin, T.D. and she will reply directly to you in due course.

    The Private Secretary can be contacted by email at [email address removed by webmaster to avoid spammers – available on request] 

    If you require further information, please don’t hesitate to contact me.

    Regards

    [A Civil Servant – name excised for privacy]

    —

    Information Resource Unit

    Aonad Acmhainní Faisnéise

    Department of Enterprise, Trade and Employment, Kildare Street, Dublin 2, Ireland An Roinn Fiontar, Trádála, agus Fostaíochta, Sráid Chill Dara, Baile Átha Cliath 2, Éire

    T: + 353 1 631 2100; LoCall: 1890 220 222

    F: + 353 1 631 2827; www.entemp.ie

    Not sure if this will actually amount to anything, but some follow up contacts via the dept of enterprise website might get someone’s attention…. http://www.entemp.ie/contact/index.htm

     

     

      

     

     

     

     

  • Amazon-inania

    Tuppenceworth directed me to a worrying bit of news – Amazon have gone totally insane.

    Having read the posts and comments over at Michele’s blog, I had to get my writing fingers out.

    Let’s start with the issues in the order they come to me:

    1. The point was made that if you contacted a baker and asked for their cheap bread, they would be entitled to decline your offer of slang-bread for eating-bread on the grounds that they couldn’t reasonably meet your request without additional costs. Amazon don’t make bread. Amazon is an incredible logistics machine with a proven capability to get packages from Dusseldorf to Dublin – as long as you are willing to pay the shipping costs.
    2. There is a quasi-Irish Amazon site – it’s called shopireland.ie and it is run in association with Amazon. Prices are in euro and are quoted at Irish VAT rates. It is very nice. I have used it a lot. (Michele Neylon has informed me that this is an Amazon Associates site – thanks for the clarification.) This is what happened today when I tried to buy from an ostensibly IRISH website. Apparently Ireland is ‘International’ on an Irish site… I wonder if Dermot Ahern knows we’re not in the EU anymore? I wonder how the owners of shopireland.ie feel about their business being impacted by Amazon’s unilateral decision?
    3. Somewhat ironically (as anyone who has read the comment over on Michele’s site) Amazon’s customer service centre is not in the UK but…. in Ireland (Cork), as is their Systems, Networks and Operating Centre (Dublin). So our transactions are being captured (technically speaking) within the jurisdiction of the Irish Republic. But as we don’t live in the Irish Republic anymore we are “International”….
    4. Companies may try to segment their regional markets for a variety of reasons: operational reporting, segmented pricing (ie different prices in different countries). However if they attempt to enforce those prices when a citizen of the EU ‘shops around’ and puts a retailer in a role of ‘passive-seller’ then any attempt to prevent that Retailer from selling to that consumer is an illegal act – either under Free Movement of goods or Competition Law (abuse of a dominant position in Market A to protect/grow Market B).
    5. Warranty issues may be a valid argument – but most of the appliances I’ve bought in recently have either come with warranty information for EVERY EU country or just for the UK (because we speak English and they used to live here we must be English apparently). Therefore, the warranty issue is not as strong an arguement. Also, Caveat Emptor is a defence available to the manufacturer – the buyer would take on the risk of not being covered by a warranty – no reason not to sell it.

    So it looks like we are no longer in the EU because Amazon has decided so. I can see how that might be the case, what with the CIA allegedly using Shannon as a carpark and Airforce 1 and Airforce 2 stopping off for Duty Free every few months. Combine that with a national development plan that has put the car (single occupant preferred) at the centre of transport planning and I’d be forgiven for thinking we were the 52nd State of the US.

    It is either that or Amazon has just bungled into a whopping Information Quality problem. (Well, we do sit between Iran and Iraq at the UN, we’re obviously Middle Eastern).

    Given the impact this has on Irish Consumers, and given that Amazon’s Customer Service call centre is based in Cork, I would suggest letting Eddie Hobbs know…. he can drive down and ask them why they are letting us get ripped off!

  • Process Design & Quality

    Quality is defined as the ability of a product or piece of information to meet or exceed the expectations of its customers/consumers.

    Quality begins in the design stage, at the white board when you are figuring out how your process should work. I won’t waste my energy today rattling on about our Electoral Register issues, rather I’ll take a different example…

    http://www.theregister.co.uk/2006/05/10/ms_messenger_paradox/

    This is an example of poor quality information. The instructions presented to the customer are illogical and set up a logical recursion that would stump many a Dalek.

    Reminds me of the joke about the computer programmer who was found dead in the shower. The shampoo bottle instructions read “Wash, Rinse, Repeat”.

    Â