The DOBlog

Category: Information Quality

  • Please buy Expedia an Atlas…

    Following on from Michel Neylon’s on-going battle with Amazon, it looks like the illness has begun to affect Expedia (who may need to buy an atlas from Amazon).

    A colleague of mine just tried to book a hotel room in London for a weekend away. She got her itinerary number and had confirmed availability and price and was trying to give her credit card details to pay for the booking.

    On Expedia, you have to tell them if you are a UK address or a non-UK address (I suspect that this is to present different address format templates). My colleague selected “Non-UK” and proceeded to fill in her address details.

    Until she got to the part where they wanted to capture Country. Ireland wasn’t listed. Neither was Éire, Republic of Ireland, Irish Republic or Southern Ireland (all common alternatives that are sometimes used).

    Nepal and the South Mariana Islands were available options though. Lucky for them.

    Let me put it another way… the drop down list of countries was significantly incomplete for a company that is operating within the European Union (25 states and counting). Ireland hasn’t been part of the UK since 1922.

    My colleague rang Expedia to find out what was going on and to see if the order could be completed over the phone. To her surprise she was told that “expedia can’t take orders from Ireland”. Which is the equivalent of “the computer says no” from Little Britain.

    I wonder if the legal eagles who hang out over at tuppenceworth would have an opinion on the legality of Expedia’s business model, which to my mind smacks of an unjustified (and unjustifiable) restriction on free movement of services within the European Union and the European Free Trade Area.

    In the mean time, my colleague will be using a different site to book her accomodation in London. Until, of course, “the computer says no”.

    (editor’s note: I’ll stick the links ‘n’ stuff into this later).

  • Propogation of information errors and the risks of using surrogate sources

    ….ye wha’?

    There has been a lot written in relation to the electoral register and other matters about using information from other sources to improve the quality of information that you have or to create a new set of information.

    This makes sense, other people may already have done much of the work for you and, effectively, all you need to do is to copy their work and edit it to meet your needs. In most cases it may be faster and cheaper to use such ‘surrogates’ for reality to meet your information needs than to go to the effort of going to the real-world things (people, stock-rooms where ever) and actually starting from scratch to build exactly the information you need in the format you require to exactly your standards and formats.

    There is, however, a price to pay for having such surrogate sources available to you. You need to accept that

    1. The format and structure of the information may need to be changed to fit your systems or processes
    2. The information you are using may itself be innaccurate, incomplete or inconsistent.
    3. If you are combining it with other information, it will require investment in tools and skills to properly match and consolidate your information into a valid version of the truth.

    These risks apply to organisations buying marketing lists to integrate with their CRM systems but also could be applied to students relying on the Internet to present them with the content for their academic projects or journalists trawling for content for newspaper articles or reviews.

    Recurrence of common errors, phrases or inaccuracies in term papers is one way that academia has of identifying academic fraud. Similar techniques might be applied in other arenas to identify and track instances of copyright infringement.

    In businesses dealing with thousands of records, the cost/risk analysis is relatively straightforward. The recommendation I would make is that clear processes to manage suppliers and to measure the quality of the information they provide you based on a defined standard for completeness, consistency, duplication, conformity etc. is essential. Random sampling of surrogate data sources for accuracy (not every 100th record but a truly random sample) is also strongly recommended.

    These are EXACTLY the same techniques that manufacturing industries use to ensure the quality of the raw material inputs to their processes. If it works for industries where low quality can kill (such as pharmaceuticals), why shouldn’t work for you?

    For students, journalists and those of us hacking away in the blogosphere the recommendation is simple. Only rely on surrogate sources if you absolutely have to. If you use someone elses work as your source, credit them. If you don’t want to credit them then make sure you verify the accuracy of their work either by actually verifying against reality or by checking with at least one other source.

    That way you avoid having the errors of your source become your errors also and you don’t run the risk of someone crying foul and either suing you for stealing their copyright (and copyright does apply to content posted on the internet and in blogs) or taking whatever other sanctions might apply (such as kicking you off your college course).

    In many cases the costs and effort involved in double checking (particularly for a once of piece of writing) are neglibily different to the costs of actually starting from scratch and building your information up yourself. And, depending on the context, it may even be more enjoyable.

    The New York Times not so long ago had to relearn the lessons of checking stories with at least one other source for accuracy.

    Horatio Caine in CSI:Miami always tells his team to “trust, but verify”.

    When using surrogate sources for real-world information in any arena you must assess the risk of doing so and put in place the necessary controls so that you can trust that you have verified.

    (c) Daragh O Brien 2006 (just in case)

  • The real cost to business of poor quality Information

    The Irish Independent, the Irish Times and Silicon Republic have all carried coverage over the last days about TalkTalk, the CarphoneWarehouse fixed line subsidiary’s operation in Ireland (recently acquired from Tele2).

    According to Silicon Republic:

    “ Talk Talk has been ordered by the Commission for Communications Regulation (ComReg) and the Data Protection Commissioner to make a public apology over complaints by consumers who received cold calls despite recording their preference not to receive unsolicited marketing calls.”

    In addition, they have been asked by BOTH regulators (Comreg and the Data Protection Commissioner) to immediately cease all direct marketing until an audit has been carried out.

    The root of the problem is that TalkTalk talked to people who had opted out of direct telemarketing on the National Directory Database. As such TalkTalk should not have been talktalking to these people. And some of them complained, to both the Data Protection Commissioner and the Communications Regulator.

    TalkTalk have pointed the finger of blame at “data integrity issues in their internal processes” and gaps in the data that they acquired from Tele2 when they purchased it.

    In the increasingly comptetive telecommunications market, not being able to direct market to prospective customers effectively puts you out of the game, with an increased reliance on indirect marketing such as posters or TV ads, none of which match the conversion rate of outbound telemarketing.

    The Information Quality lessons here are simple:

    1. Ensure that your critical core processes (such as marketing database maintenance) are defined, measured and controlled in an environment that supports Quality information.
    2. Make sure that your Information Architecture is capable of meeting the needs of your knowledge workers. If a key fact needs to be known about a customer or potential customer (such as their telemarketing preferences) this should be clearly defined and maintained and accessible.
    3. When you are buying a new business or merging with another organisation, an important element of due diligence should be to look at the quality of their information assets. If you were buying a grocery store you would look at the quality of their perishable goods (are you buying a shop full of rotten tomatoes?). Buying the information assets of a business should be no different.
    4. “The obligation to the customer never ceases”. At some point somebody must have berated a TalkTalk Customer Service/Sales rep for ringing them during Corrie when they had opted out of direct marketing. Why was this not captured? Toyota’s Quality management method allows any employee to ‘stop the line’ if a quality problem is identified. In the context of a Call Centre, staff should have the ability to at least log where the information they have been provided doesn’t match with reality and to act on that. If these call outcomes weren’t being logged there is an absence of a valid component in the process. If the call outcomes were being logged but were not being acted on by Management there is an absence of control in the process.
    5. “Cease management by Quota”. My guess is that all the staff in the call centres were being measured on how many calls they made and how many contacts they converted. Where these measures were not met I would suspect that there was a culture that made failure to hit targets unacceptable. Unfortunately taking time out to figure out why a customer’s view of their suppressions is different to what is on the screen impacts call duration and the number of calls you can make in a night. Also, removing records from calling lists as scrap and rework slows down the campaign management lifecycle (if the processes aren’t in place to do this as par for the course).

    So now TalkTalk’s call centres are lying idle. TalkTalk has joined Irish Psychics Live as being among the first businesses to have a substantial penalty in terms of fines or interruption of business imposed on them by the Regulatory authorities for Data Protection issues. There’s a lot of call quotas not being met at the moment.

    I will be interested to hear what the audit of TalkTalk brings to light.

  • Something’s wrong – I find myself agreeing (in general) with a PD

    http://www.progressivedemocrats.ie/press_room/1812/

    Senator Minihan makes a good deal of sense in his speech to the Seanad (Irish Senate, second house of our executive).

    His is the closest I have seen in the debates and coverage thus far to an apolitical statement of purpose. Also, he has touched on a number of potential root causes for the current state of the register.

    He is espousing a long term strategy (good) and a short term scrap and rework (not so good, but necessary at this stage). He questions why there is so much variation in ‘quality’ between local authority areas and what the motivation might be for local authorities to manage the register in the current modus operandi.

    Significantly he states that the long term planning shouldn’t be put off until after the election but should start now. This is in keeping with good Information Quality Management practice where scrap and rework is commenced in parallel with process review  and improvement (ideally process improvement should start first).

    My recommendation is that the root cause analysis that is currently at anecdotal level should be formalised into a format similar to that outlined in my Draft Paper on Electoral Register Information Quality Approaches. The root causes should then be prioritised in terms of their frequency of occurence and their ease of remediation.

    Senator Minihan correctly points out that you need to provide more information when setting up an ESB account or a phone account than you do when registering to vote. Is that not a telling root cause?

    However, the challenge now is to ensure that the constancy of purpose that Senator Minihan calls for is achieved as if the governemnt believes that some scrap and rework is all there is to solving this problem they are sorely mistaken.

    I’ll have to read this speech a bit closer to find exactly what it is I can disagree with.

  • Electoral Register Forms

    I previously looked at the issue of the Electoral Register forms – the first point in the information chain that feeds the electoral register. On a whim this morning, I printed a copy out – my intention being to take a closer look at the structure of the form and its layout to see if anything there might contribute to problems in the Electoral register. Attached is a graphic showing what I found….  

    electoral_register1.jpg

    This form cannot be printed completely on a standard printer. The black areas show the sections of the form (instructions and data) that are missing when the form is printed on a standard office laserjet. The clear text area is what is actually printed, which is incomplete data.

    I would have thought this should have been a basic acceptance test  for a downloadable form. As the downloaded form loses important data, it may in itself be contributing to the problem.

    As it stands, my review of the structure of the Electoral Register form will now have to work off the PDF file.

    May 18, 2006

  • Excel – some noteworthy c*ckups…

    The nice people over at the European Spreadsheet Risks Interest Group have an excellent website with good examples of spreadsheets gone wrong.

    Here’s one about sorting errors…

    And another… here

    And another… here

    and another… here

    and another … here

    and another… here (this one relates to electoral roll data)

    and another… here

    Some of these links might be broken now as some of the articles are a bit old but the EUSPRIG site has summaries of the stories.

    My finger is sore from cutting and pasting links… if you want to see more on the risks of relying on spreadsheets, check out http://www.eusprig.org/stories.htm

  • Finger print scanning and Indelible Ink in the Electoral Process

    I have been distracted from compiling my post on the use of excel as a data interchange mechanism by some other snippets of what Dick Roche said at yesterday’s Dail committee meeting as reported in the Irish Times (page 9) and Dublin’s freesheet Metro (page 5)…

    1. The Census Enumerators haven’t gone away… €5 million to €6 million is being provided to Local Authorities to hire temporary staff “such as census enumerators” to carry out door to door inquiries. The Unions haven’t gone away either though and I suspect that Local Authority staff currently doing those jobs might be offended by this idea. And at the end of the day it is €6 million between 114 Local Government agencies (29 County Councils, 10 City/Borough Councils and 75 Town Councils) or an average of €52632 per agency to pay temporary staff for the duration of the scrap and rework. (that calculation assumes that Town Councils have a role to play in the Electoral Register – if Town councils can be excluded that amounts to just under €154k per Local Authority). Of course, this is still just scrap and rework – the process of gathering the information will use the same apparently broken processes to capture the new register. Once the cleanup is finished, unless there is a study of the root causes, the same defective processes will operate to corrupt the register almost immediately and taxpayers will find themselves having to fund another €6million clean up in the not to distant future.
    2. The Minister has put forward a proposal to avoid voter fraud at the Polling stations in the next election. “Indelible Ink or fingerprint scanners could be used in polling booths at the next poll” (source: Metro). Of the ink, the Minister is quoted as saying “it woudl be a badge of pride that you had participated in democracy”. Allllllrrrrrriiiiiigggggghhhhhtttttyyyyy then.

    My first recommendation is to ban CSI and its spin off shows from the Roche household. Capturing a fingerprint at the time of a ballot would be totally useless unless there was a master data source of citizen biometric data that could be referenced – even Gil Grissom has to have a finger print in the database before he can nab the criminal.

    This is a proposal that I would suggest Digitial Rights Ireland jump all over quick smart as, in my view, this combined with the call to use PPS numbers brings us one step from a ‘big brother’ single view of citizen with biometric data. And as the Government seems to be incapable of properly managing complex technology projects I would be very concerned by this.

    I’ll get my post on Excel done over the coming days – unfortunately pesky day job is getting in the way. ;(

  • Electoral Register on Oireachtas Report

    The coverage of the Dail Committee meeting on Oireachtas Report this evening was a little disappointing. They only covered the Minister’s speechifying on the topic and didn’t show if there had been any debate or challenge on the topic from any of the Opposition politicians who sit on the Committee such as Labour’s Eamon Gilmore.

    My feeling is that anyone who has been following my posts on this site and is familiar at any level with the scrap & rework concept would have found a few open goals in what Dick Roche was putting forward.

    1. Use of ‘relevant databases’ within Local Authorities – this amounts to a process change to address a root cause (well done Minister). However there may be data protection issues given that data can only be used for the purposes for which it is captured – this fact was glossed over by the Minister. In addition, just because one has a second surrogate source for reality it doesn’t mean that it is any more accurate or complete than your ‘problem’ dataset. Furthermore, what tools do Local Authorities have available to them to actually match across these datasets quickly and accurately (short of manual efforts)? The Minister has been somewhat quiet on the ‘how’.
    2. The Minister referred to using ESB databases to update the register on the basis that we all use the ‘leccy. Again, this is not a complete surrogate for reality as it would hold the details of only one, at most two, individuals at an address. That data may not match against the Electoral Register for a variety of reasons (different spelling of names, use of aliases or akas (e.g. John Vivian Smith may be listed on the ESB bill as J.Vivian Smith or just plain old Vivian Smith or just J Smith)) – Again Data Protection issues (and commercial considerations) exist here but were glossed over by the Minister.
    3. Finally, Minister Roche is plowing ahead with the plan to give records of deceased persons to Local Authorities. The data will be sent in Excel format. There are issues with that idea that I will address in a later post (once I have gotten my thoughts on it ironed out) but in the meantime the question that arises is simply this – once the Local Authorities have their Excel spreadsheets of names and addresses of the deceased, how will they go about matching them against the Electoral Register and flagging matches for action? What is the process that will operate to convert raw data into actionable information?

    This issue is a national crisis. The Government seems content to operate on a scrap and rework basis with no real thought leadership or strategy. It is all well and good to identify potential sources of data against which the accuracy of the register can be checked, but in the absence of clear processes to use that information or the capability to process it all that Local Authorities will have is more data and a Minister who’s approach has been to criticise and berate and to effectively manage by sound-bite. Provision of ‘ring fenced’ budget and 3rd party data sets will not enable Local Authorities to perform their duties re the Electoral Register any better if there is no understanding of what needs to be done to address these problems in the long term.

    Where is the action plan to assess root causes and address the core deficiencies in the Electoral Register process? Where is the leadership in terms of thinking and in terms of control of the processes?

    I wonder what the boys and girls over at Digital Rights Ireland would have to say about the ESB providing its billing database or customer relationship management database to EVERY Local Authority (in excel format perhaps)?

    My suggestion of 2 weeks ago still stands…

    1. Scrap the current register
    2. Mail everyone on the current register and request that they re-register
    3. Use that register as the appropriate data set for the Draft Register
    4. In parallel assess Root Causes – involve Local Authority staff (not County Managers but Revenue Collectors, Librarians – anyone with a role in the process) to identify the root causes – even go as far as to talk to (dare I say it) Citizens to find out what is broken in the processes
    5. Develop a plan to address those root causes, addressing governance issues and Information Architecture issues as well as cultural issues.

    Scrap and Rework without corresponding review and improvement of processes will not solve the problems in the Register, it will merely postpone them until a later date.

  • Dail Committee Discussion on Electoral Register issues

    There is a Dail Committee discussion on the Electoral Register issue being held this afternoon at 16:15. The Minister for the Environment will be facing questioning on what is being done to fix our shambles of an electoral register.

    According to the published schedule, this discussion is to be televised. I’m not sure exactly where on the Oirechtas.ie website the Dail committee discussion will be televised, but the link to the general page is here.

    Oireachtas Report is on RTE1 at 23:40 tonight so it might carry coverage there.

     

  • Conflict of Expectations

    A few weeks back, I was at dinner with Larry English and some people from a large consulting firm. It was a social occasion and as the food and informality took hold, the discussion turned to the pillars of the Total Information Quality Management methodology and ethical/moral concerns.

    Mr English draws heavily on Deming and the TQM gurus in his view of quality – quality is defined as meeting or execeeding customer expectations and the most important person in the ‘production line’ is the customer. The challenge to this was what would one do if your customer wanted something illegal or immoral to meet their expectation or if, in order to meet or exceed their expectation, you had to do something illegal or immoral.

    My response to this argument was to point out that the expectation of society as to the acceptable standards of conduct are expressed in law, both statute and judicial. Just because a product or service meets or exceeds the expectations of a targetted customer it does not mean that it can be produced if the laws of society would preclude it. A good example of this is the location tracking tools that were being deployed by a Dublin security company. The prompt action of Digital Rights Ireland in alerting the Data Protection Commissioner resulted in the expectation of society of privacy (albeit curtailed somewhat for children) was asserted and the product had to be withdrawn.

    Where individuals or organisations choose to offer services that are illegal or immoral, and where those services are their business, then it can be argued that their obligation to their customer is the same as a business operating within the law. If a drug dealer (for example) provides low quality product then they will not survive long. Either in business or life.

    The choice is whether to operate within the law or not, not whether you need to produce a quality product that meets or exceeds your customer expectations. If you choose to operate within the law, then the legal process will operate (albeit at times slowly) to assert the expectation of Society as a customer. If you choose to act outside the law, producing or selling a product that is illegal, then your obligation to your chosen customer remains the same.

    Â