The DOBlog

Category: Information/Data Quality Issues

  • Triskaidekaphobia Cars and Information Economics

    So, the Irish Government has decided – based it would seem solely on the analysis and advice of the Society for the Irish Motor Industry- to introduce a revised licence plate system for Irish cars starting from January of next year.

    The reasoning put forward is that fear of the number 13 will hamper car sales (superstition) and people don’t like the current system because they don’t know for certain when a car was manufactured (snobbery).

    Snobbery

    To address the snobbery element first, according to comments from SIMI quoted in the Irish Independent:

    Even though 70pc of new cars are bought during the first four months of the year, some consumers believe that it doesn’t accurately reflect the real age of a new car since cars bought in January are obviously manufactured the previous year while those bought later in the year are actually made in the same year

    So. 70% of all new cars are purchased in the first four months of the year. That’s a good statistic. It means that, on average, 3.75% of all new cars are sold in each of the remaining 8 months of the year. From that a reasonable guesstimate of the value at risk in each month can be worked out.

    What is not a good statistic is “some consumers”. Is that one consumer, one consumer and their friend from the gym, 1000 consumers, or every consumer who buys a car in the first 4 months of the year? If is the latter it obviously doesn’t bother them that much or they wouldn’t buy until later in the year.

    Surely a better and more cost effective approach would be for the SIMI to educate purchasers about the manufacture and supply chain processes that apply to vehicles. Bluntly – car manufacturers don’t build cars in the hope they will sell them. That’s too expensive. They apply logistics principles to build enough to just about meet forecast demand. And no more. So a car purchased in January will not have been sitting in a storage facility for a dozen months. It will be relatively recent.

    And does the fact that it was manufactured in the previous calendar year actually matter if features, specifications, and price are the same in December 2012 versus January 2013. I know from experience that the announcement of a new model of a car affects book value, but, excluding the change of model for a moment, logistics need to be considered when we think about the idea of the year of manufacture being a real decision point for people. After all, a car manufactured in January 2013 will be using parts that were on-hand at end December 2012, that were probably ordered at the start of December 2012, and were probably being manufactured by the downstream supplier from October 2012 in anticipation of a glut of orders from car manufacturers in December/January 2012.

    The new iPhone isn’t due out for a while yet, but already there are rumours of supply chains having been ramping up for months… that’s how logistics works.

    And as the supply chain for vehicles is largely a pull supply chain (building to respond to demand), the easiest way to avoid having a car that was assembled in 2012 delivered to you as a new car in 2013 is to order it in Month 2 or 3 of 2013.

    But even then it doesn’t matter as the actual age of components going into the car will depend on the vagaries of supply chain management down the line from the dealership to the nice man in Schenzen whose company makes the screws that hold your sun visor in place.

    I can remember a few years ago looking to buy a particular model of car. The dealership didn’t have any in stock and when they (and this is the CSI moment) looked at the logistics system from the manufacturer they were able to tell me when the next one of the model I wanted would be manufactured. There was no great holding pen of stocks waiting for me to turn up and buy.

    So… I would really like to see some objective evidence that people actually give a rats ass about when their car is assembled, given that the majority of new cars are purchased in a time period when it would be logical that the supply chain inputs to the delivery of that car would have taken place in the previous year. The data does not correlate.

    Superstition

    It’s a number. Currently there are vehicles on the roads in Ireland with the number 13 in their license plate. Not in the year, but in the other element of the license plate.

    Surely insurance companies can provide data on the number of claims involving vehicles registered within the past 10 years with the number 13 in their license plate against which we can determine if superstition is borne out by evidence. If it is… brilliant, we can establish an economic value case for changing an otherwise logical and straight forward system.

    The National Vehicle database (where registration numbers come from) would likewise have data on how many cars currently have a 13 in their license plate. If people are already avoiding it then the data will be there… lots of 12s, lots of 14s, no 13s.

    If not. Then there’s no actual reason to change other than a vague (and quantified) assertion that people won’t buy new cars because they have a 13 in the license plate.

    Reality

    This sounds like a simple change. But it isn’t. Many of the systems that your licence plate goes into are old and could require systems changes to accommodate the new format. Many of these are government departments. For example:

    • National Vehicle Driver File (Dept of Transport)-  reg number and registered owner
    • VRT tax systems (Revenue Commissioners)
    • Gardaí (PULSE system, asset registers for garda vehicles)
    • Insurers
    • Car park ticketing systems such as the Pay-by-SMS service in Dublin (Local Authorities)
    • Car clamping operator systems
    • CIE (they need to log busses)
    • Car Rental operators

    It would be interesting to know if the Government commissioned any form of economic impact assessment to off-set the cost of catering to one industry lobby group for a problem that would exist in one year against the costs to the State and other private sector organisations of making systems changes to support the new format.

    Particularly given that the changes would need to be implemented before mid December to allow for them to be in place for cars being registered in January.

    The reality is that life is not like Star Trek and data is not well managed. I would doubt if there is the required metadata available to do a quick Impact Assessment on the change. At a minimum you would need to know the maximum field lengths for reg numbers in key systems. Other data required would be information on data transfers, batch processing functionality, or edit checking that might be applied to make sure that the full extent of the changes is understood and addressed to avoid any systems or process failures.

    I was involved in a lot of that kind of activity in Call Centre systems for Y2K in a former life. It is not easy if things aren’t documented. And they are never documented.

    My prediction: It this suggestion goes ahead without any rigorous impact assessment here will be at least one major process failure in January/February 2013 arising from this. It is an idea that, while it may have merits, risks being rushed in without proper impact assessment being performed or any examination of the costs of implementation across the public sector or other private sector users of this information.

    In reality there has been a tentative Value case put forward with no corresponding assessment of the costs associated with delivering that value. And a horrendously ambitious time scale to make what is actually a deceptively complicated change.

  • Lies, damned lies, and statistics

    On Monday the 16th January 2012 the Irish Examiner ran a story that purported to have found that 93% of the Irish public “decried” the decision of the Minister for Foreign Affairs to close Ireland’s embassy in the Vatican City State. The article detailed how they had undertaken a review of correspondence released under the Freedom Of Information Act which showed that 93% of people in Ireland were against the closure. To cap it off, the article was picked up in the Editorial as well.

    Except that that isn’t what they had uncovered. The setting out of the statistics they had found in the sensationalised way they presented them was a gross distortion of the facts. A distortion that would, to paraphrase Winston Churchill, “be half way around the world before the truth had its boots on”).

    Demotivational poster about data

    What they had uncovered is that of the 102 people who wrote in to the Minister for Foreign Affairs about the issue, 93% of them expressed a negative opinion about the closure. The population of Ireland is approximately 4.5 million people. 95 people is closer to 0.000021%. While I may not have the academic qualifications in Mathematical physics that my famous comedian namesake has but I know that 95 people (that’s 93% of 102) is slightly less than 93% of the Irish public

    Or, to put it another way, significantly and substantially below the statistical margin for error usually applied in political opinion research by professional research companies.

    Or to put it another way, over 99% of the population cared so little about the closure of the Vatican Embassy that they couldn’t be bothered expressing an opinion to the Minister.

    Of course, the fact is that there were letters written about this issue. And the people who wrote them were expressing their opinion. And 93% of them were against the closure.  In fact, in defending themselves on Twitter against an onslaught of people who spotted the primary school maths level of error in the misuse of statistics in the article, the Irish Examiner twitter account repeatedly states that (and I’m paraphrasing the actual tweets here slightly) “for clarification we did point out that the analysis was based on the letters and emails”. But it is inaccurate and incorrect to conflate the 93% of negative comment in those letters to the entire population as the sample size is not statistically valid or representative being

    1. Too small (for a statistically valid sample of the Irish public you would need between 384 and 666 people selected RANDOMLY, not from a biased population. That’s why RED C and others use sample sizes of around 1000 people at least for phone surveys etc
    2. Inherently biased. 93% of cranky people were very cranky is not a headline. The population set is skewed towards one end of the distribution curve of opinion you would likely find in the wider population.

    Then today we see a story in the Examiner about how Lucinda Creighton, a Junior Minister in the Dept of Foreign Affairs is backing a campaign to reopen the embassy because

    there’s a very strong, and important and sizeable amount of people who are disappointed with the decision and want to see it overturned and who clearly aren’t happy

    What? Like 93% of the Public Lucinda? Where is your data to show the size, strength, and importance of this group? Have you done a study? What was the sample size?

    As a benchmark reference for what is needed for an Opinion Poll to validly represent the opinions of the Irish Public, here’s what a reputable polling company says on their website:

    For all national population opinion polls RED C interview a random sample of 1,000+ adults aged 18+ by telephone. This sample size is the recognised sample required by polling organisations for ensuring accuracy on political voting intention surveys. The accuracy level is estimated to be approximately plus or minus 3 per cent on any given result at 95% confidence levels.

    Anything less than that is not statistically valid data and can’t be held out as representing the opinion of the entire public.

    As an Information Quality Certified Professional and an active member of the Information Quality Profession on an International level for nearly a decade I am ethically bound to cry “BULLSHIT!!” on inaccuracies and errors in  information and in how it is presented. The comments from Ms Creighton are a good example of what that is important in the Information Quality and wider Information Management profession. If bullshit analysis or analysis based on flawed or inherently poor quality data is relied upon to make strategic decisions then we invariably wind up with bullshit decisions and flawed actions.

    And that effects everything from conversation with family, chats in the pub, business investment decisions, political decision making, through to social policy. Data, Information, and Statistics are COOL and are powerful. They should be treated with respect. People publishing them should take time to understand them so that their readers won’t be mislead. And care should be taken in compiling them so that bias does not skew the results.

    So, having had no joy or actual engagement from the Irish Examiner on the issue I forwarded my complaint to the Press Ombudsman yesterday pointing out that the article would seem, based on the disconnect between the headline, the leading paragraph, and the general thrust of it, to be in breach of the Code of Practice of Press Council of Ireland.

    I just hope they can tell the difference between lies, damned lies, and fudged statistics. (This Yes Minister clip about Opinion Polls shows how even validly sampled ones can be biased by question format and structure in the survey design).

  • Laser-like accuracy

    Word reaches me this morning of yet another incident of Bank Of Ireland double-dipping laser card transactions on or around a Bank Holiday.

    BOI will, doubtless, claim that this is a once off and hasn’t happened before. That’s what they said the last time (when it had actually happened before). Furthermore, I hope that BOI are more certain this time as to the root cause (last time out it was variously “retailer error” or “a software upgrade glitch”).

    And hopefully their process for catching “shadow transactions” which lead to the double-dipping will kick into play and actually refund the customers affected  (which if this glitch is on the scale of their 2009 one could be up to 200,000 card holders).

    For reference the relevant blog posts are:

    http://obriend.info/2009/09/09/bank-of-ireland-double-charging/

    http://obriend.info/2009/09/09/bank-of-ireland-double-charging-a-clarifying-post/

    http://obriend.info/2009/09/10/bank-of-ireland-overcharging-another-follow-up/

    http://obriend.info/2009/10/28/bank-of-ireland-again/

    The issue also featured over on IQTrainwrecks.com.

    My €0.02: This issue appears to manifest itself around Bank Holidays. This suggests a batch load process or some human triggered action doesn’t work correctly when there is a Bank Holiday. Having a process to detect the double-dipped transactions is not a fix, as if it doesn’t work (as seems might be the case here) then the incorrect data gets through.

    BOI might want to pay attention to Ferguson v British Gas, which while a UK case, could be arguable precedent for the view that Irish Courts won’t care how complex your IT systems are if a customer is impacted through a failure of your systems to process information correctly.

    BOI need to identify the precipitating root cause of this problem, based on the data they have available… I’d start with looking at the dates of incidents (BOI should have more data than newspaper headlines to go on) and seeking to confirm or disprove the ‘Bank Holiday hypothesis’.

    Relying on a ‘scrap and rework’ kludge that might itself fail is not a sustainable approach to ensuring information quality or quality of customer service.

  • In the interest of Electoral Balance

    I’ve written previously about Fine Gael and their issues with avoiding Data Protection pitfalls during this current General Election.

    Some people might have gotten the impression that I’m obsessed with Fine Gael. I’m not. I’m obsessed with Data, specifically the management of data and information in manner that ensures quality outcomes through quality data governed with due regard to relevant legislation.

    On courses I teach on Data Protection and Information Quality I often make reference to “The Joe Duffy Effect” to describe the brand impacts that can arise if organisations don’t take care to manage information as a complex and valuable asset. The term refers to Joe Duffy, a talk radio host on Irish radio. Joe enjoys taking the side of the common man, usually. Occasionally he makes a jape of not getting the point, whether by accident or design we may never know. But organisations who fall foul of the “Joe Duffy Effect” can find themselves fighting rear guard actions against an often intractable foe.

    Last week Joe spoke with Jacob, a South African living in Ireland who had received a pre-recorded voicemail to his phone from Michael Martin. Jacob’s tale can be heard in Technicolour on the RTE website.

    From the call we glean that:

    1. A voicemail was received by Jacob on the 9th of February with a pre-recorded message (which Jacob played)
    2. He has apparently received SMS messages from Fianna Fail with calls for volunteering and campaigning.
    3. He is not a member of Fianna Fail
    4. He has not asked for Fianna Fail to contact him and does not know where they got his number.
    5. The mobile in question is used as an internal work mobile and is not listed. His number is only listed with the Road Safety Authority.

    In the broadcast Joe tells Jacob that we live in a democracy.

    Correct. We live in a democracy. Specifically we live in democracy where we have decided that the Right to Privacy, while not absolute, is a right that must be defended. Just because we are a democracy it does not give politicians an automatic carte blanche to process data regardless of where or how it has been obtained. These rights to privacy are enshrined in law, in the Constitution and in EU Treaty obligations. Yes, there are balances, mitigations and exemptions with respect to how that right is exercised and protected – but it is still a democratic right of the individual.

    During the course of the call, a comment from Fianna Fail was read out saying that they didn’t have Jacob’s number. That is at odds with the evidence – to whit: one recording. And if I’ve learned one thing from watching CSI is that evidence trumps counter claim every day.

    So, what is the Data Protection issue here:

    • Fair Obtaining – Jacob is not a member of the party and was not aware of how his number came to be called and texted. Granted his phone seems to be for work purposes, but the electronic Privacy regulations apply to business as well as personal data. Also, while he may use the phone for work purposes a big question to ask here is who is paying the bill – him, or a company. If he pays the bill the phone may actually be a personal phone used for business purposes (Sole Trader data is a tricky area in Data Protection land).
    • Governance and control of data and/or data processors – Fianna Fail claimed not to have Jacob’s number. The fact that a Fianna Fail party message was left by voicemail and various SMS messages were sent to him suggests that they do. Or if not them then someone working on their behalf. Under the Data Protection Acts, the Data Controller is responsible for the actions of the Data Processor unless the Data Processor acts outside the parameters of the formal contract in writing that governs the Data Controller/Data Processor relationship. So… while it may be true that FF HQ don’t have Jabob’s number, someone processing data on behalf of Fianna Fail does. Fianna Fail not knowing whether or not they had the data suggests a weakness in internal control and governance.
    • Accuracy – Joe D. suggested to Jacob that maybe the messages were being sent because of a wrong number. Personal data needs to be kept accurate and up to date. FF should have taken steps to correct the error rather than denying that they have the data. Ultimately FF carry the can for the actions of the Data Processor.

    Of course, there is the distinction to be made between normal “direct marketing” and the processing of personal data by a candidate for elected office. Basically during an election personal data is “fair game” for politicians, provided they have obtained it correctly first and have clear consents for contact. Which puts the discussion of “auto dialling” or “power dialling” on the table. According to the Data Protection Commissioner’s website:

    The use of automatic dialling machines, to call individual subscribers at random for direct marketing purposes, is prohibited, unless subscribers’ consent has been obtained in advance.  Unsolicited fax messages to individual subscribers are likewise prohibited.

    That is why it is important to know who the “subscriber” is to Jacob’s phone. If it is a limited company or similar legal entity, then it is not a call to an “individual” subscriber. If it is his phone or he is a sole trader or part of a partnership, then it is possible that he is an “individual subscriber” and as such the use of an autodialler to RANDOMLY call numbers for direct marketing would be illegal. Dialling from a preloaded list is OK. So long as the list has been fairly obtained and takes into account NDD Opt-out requests etc. And then there is the grey area of the Political exemptions from the Data Protection Acts.

    The DPC has issued guidelines to all political parties before the election. My sense is that these guidelines may have been breached in this case.

    During previous election campaigns, the Commissioner received numerous complaints from individuals in receipt of unsolicited SMS (text) messages, emails and phone calls from political parties and candidates for election.  In many cases, the individual had no previous contact with the political party or candidate and was concerned at the manner in which their details were sourced.  Subsequent investigations revealed that contact details were obtained from sources such as sports clubs, friends, colleagues and schools.  Obtaining personal data in such   circumstances would constitute a breach of the Data Protection Acts, as there would be no consent from the individual for their details to be obtained and used in this way.

    So.. Fianna Fail need to know where their Data Processors are getting their data from. The evidence says they have Jacob’s phone (and who knows who elses’) but don’t know they have Jacob’s phone. That suggests that the Data Controller is not in Control of the Data. Which is a problem in and of itself.

    Fine Gael are not the only Data protection flaunters in this election. Fianna Fail have had their moments too. The Green Party STILL don’t have a Privacy statement. And I’m sure the others have slipped up along the way as well. But that is a discussion for another day.

  • There is oft a slip twixt tweet and twolicy

    This blog post is basically the text of an audioboo I recorded at 9:30 this morning which has disappeared into the ether ne’er to be found.

    Fine Gael have launched their “Twolicy Page”. I won’t comment on the hideous neologistic portmanteau that is “Twolicy”, other than to say it that seems to have been dreamed up by a pat.

    What strikes me about the “Twolicy” page is that it is yet another import of an American election campaign tool into Irish Politics, particularly with the concept of the “E-Canvasser”. Fine Gael dynamically tell us that the E-Canvasser (perhaps some distant cousin of the “Cyber Reporter” who has emerged as the colour piece of the day on certain Irish current affairs shows?) will

    “knock on all cyber doors by delving into the depths of Facebook, Twitter, Youtube, Flickr and more! Through the simple medium of sending e-mails, facebooking and tweeting messages of support for Fine Gael you can pledge your commitment to fixing the Irish economy.”

    This is a strategy which exists to some extent in Irish politics even today. Many of the letters to Madame Editor are crafted examples of “Astroturfing” – something that appears to be a grass roots movement but is not. I first became aware of the concept back in 2002 when I spotted the Republican Party in the US running “GOPTeamLeader.com” (which, thanks to the interweb waybackmachine I can bring to you in hideous technicolour). Basically the party recruits a team of volunteers who are tasked with sending “on-message” communications to the media (which in 2001 was the newspapers, TV, and radio). In return, the GOP provided a set of reward points (like Green Shield Stamps) which could be saved up and exchanged for rewards such as barbecues, autographed photographs of the Reichsfuerher candidate, and (if memory serves me correctly, an RV.

    Fine Gael liken this to door to door canvassing. However that analogy does not hold true because the Internet is not a housing estate or public street. Drop a bus load of eager canvassers on my door step and they will be able to

    1. See my house
    2. See my neighbours’ houses

    They will not need to ask my neighbour to throw leaflets over my back wall. They will see the big sign in my hall window warning them of the fate that will befall them should they ring the bell and seek discourse (“Warning – political nut lives here”). And most of them are clued in enough to know that the “no canvassers” sticker in the window means that stuffing my letter box with bumph will just be providing stimulus to the paper recycling industry.

    The Internet is different. Social media is different. Whoring out your personal contact list to a political party is different. And because it is different, we find ourselves to an extent in uncharted territory with regard to the Data Protection implications of Social Media driven Astroturfing.

    Right now I have a contact list of 413 followers on Twitter for my personal account. I have a second twitter account that is for my business. People who follow me know (from my profile and what I tweet about) that I’m a Data nut and I do data protection and information quality training so content about those things will pop up in my timeline. People who follow me also know I’m a bit of a politics geek and enjoy holding our leaders to account. But I try and keep my business tweeting separate from my personal tweeting. And when I whore myself out too much on Twitter, I get friendly DMs from people or I get unfollowed.

    This is because the contact details of my friends are information I have gathered for domestic purposes. As such the Data Protection Acts don’t apply. If I was to sign up to be an e-Canvasser (and I can’t get the image of a canvasser handing out bags of yokes out of my head) we would then face the question of whether I was still processing that data for Domestic use or whether I had become a Data Processor working on behalf of Fine Gael, a Data Controller.

    The key question would seem to be how much control Fine Gael are exerting over the content and communication from their e-Canvasser Astroturfers, and whether they are offering any form of reward or incentive for people to encourage them to pimp out their domestic contact lists.

    If Fine Gael are simply being “passive” and are relying on individuals to act on content that is made available, then there is probably no substantial issue here. It is a case of a person finding content on the web that they think would be of interest to their personal network. We do this every day. It is the way the social web works. Of course, that then raises the question of why they would need you to sign up to their team for this purpose… surely the type of political nut blogger who would retweet or repost their bumph would do so anyway without having to be officially flagged as an “E-Canvasser”?

    If Fine Gael are being “neutral” and are simply flagging content to people who have signed up and asking them to do what they see fit with it, then this too is probably OK. The analogy would be the charity that Tweets out a fundraising message and asks their followers to retweet it to send the fundraising virally. The charity has not asked you to commit to being an active fundraiser on their behalf.

    However, if Fine Gael are specifying specific content into specific constituencies at specific times and are exercising control over the content of the messages that are being sent, then we are into a potentially problematic area.

    The e-Canvasser would not on the Fine Gael payroll. But they would be, in effect, processing personal data on behalf of Fine Gael as part of the “Fine Gael Team”. It would be interesting to find out how much direct “editorial” control that FG are placing on the Facebook Statuses that people are “donating” (and where does this fit in SIPO? What is the monetary value of a person’s Facebook status?) or the emails to “family and friends”. This is personal data that was given to them for a domestic purpose, not for the purposes of canvassing for Fine Gael. Once they commence a “active” canvassing then the use of the data has likely changed from “domestic” to political and the Data Protection Acts would apply. If Fine Gael are directing the timing of messages, the content of messages, and/or the audiences for messages then the e-Canvasser is being directed in their processing by the Data Controller, Fine Gael. And, as Data Controller, Fine Gael would need to ensure that there was clarity about the new political use of the personal data and a clear mechanism for the Data Subject (the canvasser’s family and friends) to opt-out would need to be in place – and FG would, of necessity, need to push this responsibility down to the Canvasser.

    Otherwise, FG would not have obtained the data fairly for the purposes of electoral canvassing. It would be no different than if they had asked the local GAA club to email all their members to let them know about Fine Gael’s new policy on tax relief on sliotars and faceguards for hurlers. And that is the kind of thing that the Data Protection Commissioner has already warned against.

    Things become an order of magnitude more complicated if Fine Gael are running any kind of incentive scheme for e-Canvassers to drive up the publication of their AstroTurf message.

    Of course, Fine Gael have probably thought this through and will have the necessary protocols in place to ensure that there is a mechanism for a Canvasser’s friends to opt out of receiving Fine Gael campaign materials by email, Facebook or Twitter. They have probably realised that people have the same reaction to junk mail on-line as they do at their door step and need to have the ability to put up an on-line “No Canvassers” sign.

    Currently the only opt-out mechanism I can see is to unfriend people, unfollow them or block them. Which is exactly what I would do in the physical world if a friend of mine kept ramming leaflets and policy statements from a political party into my face.

    Of course, in the absence of such an opt-out facility, Fine Gael (as Data Controller) and the e-Canvasser (as Data Processor) would need to be cautious of falling foul of SI526 2008 (the e-Privacy regulations) which carry a fine of €5000 per breach, capped at €50,000 for an individual. While Twitter and Facebook might not be mentioned in the legislation, email is in section 13(1).

    b) A person shall not use or cause to be used any publicly available electronic communications service to send an unsolicited communication for the purpose of direct marketing by means of electronic mail, to a subscriber, who is a natural person, unless the person has been notified by that subscriber that for the time being he or she consents to the receipt of such a communication. 

    [edit to clarify some points raised by @tjmcintyre]

    Now, the DPC has ruled in the past that there is an exemption covering the Direct Mail (including email and texting)

    carried out in the course of political activities by a political party or its members, or by a candidate for election to, or a holder of, elective political office

    Question: is the eCanvasser the political party (I would argue yes if FG are exerting sufficient control that they would become a Data Controller)? In which case, the processing is possibly covered.

    But I would suggest that this exemption assumes that the email or tweet would be clearly coming from Xyz@partyname.ie or an individual clearly identifying themselves as a member of the party or publicly known to be a candidate for election or an elected official. Getting an email from “yourbestmate@gmail.com’ telling you to go and look at Fine Gael policies, where that email has been sent on the instruction of and under the Control of the party or candidate would seem to me to fall outside the scope of issues already decided.

    [/edit]

    So, the upshot is that while physical world canvassers have to be careful of yappy dogs, cats that bite and political nuts who have hard questions, eCanvassers need to consider both the social acceptability and potential legality of pimping out their personal contact lists on behalf of a political party. Such tactics are de rigeur in the US. But the US does not operate with the same privacy legislation as Ireland, so ideas imported from overseas must be vetted properly to ensure that no Compliance risks arise.

    I would be interested to see what the Data Protection Commissioner’s response to or advice on formal ecanvassing that places the data at arms length but creates a de facto Data Processor/Data Controller relationship would be, particularly if that relationship is not obvious to the recipient of the email or tweet. [update] Perhaps it would be sufficient for the emailer or tweeter to clearly flag that they are part of a formal eCanvassing team acting on behalf of and under the instruction of Fine Gael?[/update]

    [update] But the issue of whether the change of use of the data from domestic to overtly political will, in my personal view, give rise to questions of whether the data has been obtained fairly for that new purpose, which is a point already clearly settled in the mind of the DPC.[/update]

     

     

  • Personal Data – an Asset we hold on Trust

    There has been a bit of a scandal in Ireland with the discovery that Temple St Children’s Hospital has been retaining blood samples from children indefinitely without the consent of parents.

    The story broke in the Sunday Times just after Christmas and has been picked up as a discussion point on sites such as Boards.ie.  TJ McIntyre has also written about some of the legal issues raised by this.

    Ultimately, at the heart of the issue is a fundamental issue of Data Protection Compliance and a failure to treat Personal Data (and Sensitive Personal Data at that) as an asset (something of value) that the Hospital held and holds on trust for the data subject. It is not the Hospital’s data. It is not the HSE’s data. It is my child’s data, and (as I’m of a certain age) probably my data and my wife’s data and my brothers’ data and my sisters-in-laws’ data…..

    It’s of particular interest to me as I’m in the process of finishing off a tutorial course on Data Protection and Information Quality for a series of conferences at the end of February (if you are interested in coming, use the discount code “EARLYBIRD” up to the end of January to get a whopper of a discount). So many of the issues that this raises are to the front of my mind.

    Rather than simply write another post about Data Protection issues, I’m going to approach this from the perspective of Information as an Asset which has a readily definable Life Cycle at various points in which key decisions should be taken by responsible and accountable people to ensure that the asset continues to have value.

    Another aspect of how I’m going to discuss this is that, after over a decade working in Information Quality and Governance, I am a firm believer in the mantra: “Just because you can doesn’t mean you should“. I’m going to show how an Asset Life Cycle perspective can help you develop some robust structures to ensure your data is of high quality and you are less likely to fall foul of Data Protection issues.

    And for anyone who thinks that Data Protection and Data Quality are unrelated issues, I direct you to the specific wording in the heading of Chapter 2, Section 1 of the Directive 95/46/EC. (more…)

  • Who then is my customer?

    Two weeks ago I had the privilege of taking part in the IAIDQ’s Ask the Expert Webinar for World Quality Day (or as it will now be know, World Information Quality Day).

    The general format of the event was that a few of the IAIDQ Directors shared stories from their personal experiences or professional insights and extrapolated out what the landscape might be like in 2014 (the 10th anniversary of the IAIDQ).

    A key factor in all of the stories that were shared was the need to focus on the needs of your information customer, and the fact that the information customer may not be the person who you think they are. More often than not, failing to consider the needs of your information customers can result in outcomes that are significantly below expectations.

    One of my favourite legal maxims is Lord Atkin’s definition of who your ‘neighbour’ is who you owe legal duties of care to. He describes your ‘neighbour’ as being anyone who you should reasonably have in your mind when undertaking any action, or deciding not to take any action. While this defines a ‘neighbour’ from the point of view of litigation, I think it is also a very good definition of your “customer” in any process.

    Recently I had the misfortune to witness first hand what happens when one part of an organisation institutes a change in a process without ensuring that the people who they should have reasonably had in their mind when instituting the change were aware that the change was coming.

    My wife had a surgical procedure and a drain was inserted for a few days. After about 2 days, the drain was full and needed to be changed. The nurses on the ward couldn’t figure out how to change my wife’s drain because the drain that had been inserted was a new type which the surgical teams had elected to go with but which the ward nurses had never seen before.

    For a further full day my wife suffered the indignity of various medical staff attempting to figure out how to change the drain.

    1. There was no replacement drain of that type available on the ward. The connections were incompatible with the standard drain that was readily available to staff on the ward and which they were familiar with.
    2. When a replacement drain was sourced and fitted, no-one could figure out how to actually activate the magic vacuum function of it that made it work. The instructions on the device itself were incomplete.

    When the mystery of the drain fitting was eventually solved, the puzzle of how to actually read the amount of fluid being drained presented itself, which was only of importance as the surgeon had left instructions that the drain was to be removed once the output had dropped below a certain amount. The device itself presented misleading information, appearing to be filled to one level but when emptied out in fact containing a lesser amount (an information presentation quality problem one might say).

    The impacts of all this were:

    • A distressed and disturbed patient increasingly worried about the quality of care she was receiving.
    • Wasted time and resources pulling medical staff from other duties to try and solve the mystery of the drain
    • A very peeved and increasingly irate quality management blogger growing more annoyed at the whole situation.
    • Medical staff feeling and looking incompetent in front of a patient (and the patient’s family)

    Eventually the issues were sorted out and the drain was removed, but the outcome was a decidedly sub-optimal one for all involved. And it could have been easily avoided had there been proper communication about the change to the ward nurses and the doctors in the department from the surgical teams when they changed their standard. Had the surgical teams asked the question of who should they have in their minds to communicate with when taking an action, surely the post-op nurses should have featured in there somewhere?

    I would be tempted to say “silly Health Service” if I hadn’t seen exactly this type of scenario play out in day to day operations and flagship IT projects during the course of my career. Whether it is changing the format of a spreadsheet report so it can’t be loaded into a database or filtered, changing a reporting standard, changing meta-data or reference data, or changing process steps, each of these can result in poor quality information outcomes and irate information customers.

    So, while information quality is defined from the perspective of your information customers, you should take the time to step back and ask yourself who those information customers actually are before making changes that impact on the downstream ability of those customers to meet the needs of their customers.

  • Bank of Ireland – again

    The Irish Times today reports that Bank of Ireland are again investigating incidents of double charging of customers who use LASER cards.

    I wrote about this last month (see the archives here), picking up on a post from Tuppenceworth.ie earlier in the summer. I won’t be writing anything more about the issue (at least not for now).

    Looking back through my archives I found the picture below in a post that I’d written back in May when Simon on Tuppenceworth first raised his issue with BOI’s Laser Cards.

  • What’s in a name?

    Mrs DoBlog and I are anxiously awaiting the arrival of a mini-DoBlog any day now. So we have spent some time flicking through baby name books seeking inspiration for a name other than DoBlog 2.0.

    In doing so I have been yet again reminded of the challenges faced by information quality professionals when trying to unpick a concatenated string of text in a field that is labelled “Name”. The challenges are manifold:

    • Name formats differ from  to culture to culture – and it is not a Western/Asian divide as some people might assume at first.
    • Master Data for name spellings is notoriously difficult to obtain. My wife and I compared spellings of some common names in two books of baby names and the variations were staggering, with a number of spellings we are very familiar with (including my own name) not listed in either.
    • Often Family Names (surnames) can be used as Given Names (first names) such as Darcy (D’Arcy) or Jackson (Jackson) or Casey.
    • Often people pick names for their children based on where they were born or where they were conceived (Brooklyn Beckham, the son of footballer David Beckham is a good example).
    • Non-name words can appear in names, such as “Meat Loaf” or “Bear Grylls“
    • Douglas Adams famously named a character in the Hitchhiker’s Guide to the Galaxy after one of the “dominant life forms” – a car called a “Ford Prefect“
    • Names don’t always fit into an assumed varchar(30) or even varchar(100) field.
    • It is possible to have a one character Given name and a one character Family name.
    • Two character Family names are more common than we think.
    • Unicode characters, hyphens, spaces, apostrophes are all VALID in names – particularly if they are diacritical marks which change the meaning of words in particular languages.
    • And then you have people who change their names to silly things to be “different” or “special”,  but who create interesting statistical challenges for data profilers and parsing tools.

    Among the examples I found flicking through one of our baby name books last evening where “Alpha” and “Beta”. Personally I think it sends the wrong signals to name your children after letters of the Greek alphabet, but I’m sure it is helpful if you have had twins to keep them in order.

    I also found “Bairn” given as a Scots Gaelic name for a baby girl. I had to laugh at this as “Bairn” is actually a Scots dialect word for Child. Even Wikipedia recognises this and has a redirect from “Bairn” to “child“.  But it does remind me of the terribly sexist “joke” where the father asks the doctor after the birth whether it is a boy or a child his wife has just delivered. (more…)

  • A game changer – Ferguson v British Gas

    Back in April I wrote an article for the IAIDQ’s Quarterly Member Newsletter picking up on my niche theme, Common Law liability for poor quality information – in other words, the likelihood that poor quality information and poor quality information management practices will result in your organisation (or you personally) being sued.

    I’ve written and presented on this theme many times over the past few years and it always struck me how people started off being in the “that’s too theoretical” camp but by the time I (and occasionally my speaking/writing partner on this stuff, Mr Fergal Crehan) had finished people were all but phoning their company lawyers to have a chat.

    To an extent, I have to admit that in the early days much of this was theoretical, taking precedents from other areas of law and trying to figure out how they fit together in an Information Quality context. However, in January 2009 a case was heard in the Court of Appeal in England and Wales which has significant implications for the Information Quality profession and which has had almost no coverage (other than coverage via the IAIDQ and myself). My legal colleagues describe it as “ground breaking” for the profession because of the simple legal principle it creates regarding complex and silo’d computing environments and the impact of disparate and plain crummy data. I see it as a clear rallying cry that makes it crystal clear that poor information quality will get you sued.

    Recent reports (here and here) and anecdotal evidence suggest that in the current economic climate, the risk to companies of litigation is increasing. Simply put, the issues that might have been brushed aside or resolved amicably in the past are now life and death issues, at least in the commercial sense. As a result there is now a trend to “lawyer up” at the first sign of trouble. This trend is likely to accelerate in the context of issues involving information, and I suspect, particularly in financial services.

    A recent article in the Commercial Litigation Journal (Frisby & Morrison, 2008) supports this supposition. In that article, the authors conclude:

    “History has shown that during previous downturns in market conditions, litigation has been a source of increased activity in law firms as businesses fight to hold onto what they have or utilise it as a cashflow tool to avoid paying money out.”

    The Case that (should have) shook the Information Quality world

    The case of Ferguson v British Gas was started by Ms. Ferguson, a former customer of British Gas who had transferred to a new supplier but to whom British Gas continued to send invoices and letters with threats to cut off her supply, start legal proceedings, and report her to credit rating agencies.

    Ms Ferguson complained and received assurances that this would stop but the correspondence continued. Ms Ferguson then sued British Gas for harassment.

    Among the defences put forward by British Gas were the arguments that:

    (a) correspondence generated by automated systems did not amount to harassment, and (b) for the conduct to amount to harassment, Ms Ferguson would have to show that the company had “actual knowledge” that its behaviour was harassment.

    The Court of Appeal dismissed both these arguments. Lord Justice Breen, one of the judges on the panel for this appeal, ruled that:

    “It is clear from this case that a corporation, large or small, can be responsible for harassment and can’t rely on the argument that there is no ‘controlling mind’ in the company and that the left hand didn’t know what the right hand was doing,” he said.

    Lord Justice Jacob, in delivering the ruling of the Court, dismissed the automated systems argument by saying:

    “[British Gas] also made the point that the correspondence was computer generated and so, for some reason which I do not really follow, Ms. Ferguson should not have taken it as seriously as if it had come from an individual. But real people are responsible for programming and entering material into the computer. It is British Gas’s system which, at the very least, allowed the impugned conduct to happen.”

    So what does this mean?

    In this ruling, the Court of Appeal for England and Wales has effectively indicated a judicial dismissal of a ‘silo’ view of the organization when a company is being sued. The courts attribute to the company the full knowledge it ought to have had if the left hand knew what the right hand was doing. Any future defence argument grounded on the silo nature of organizations will likely fail. If the company will not break down barriers to ensure that its conduct meets the reasonable expectations of its customers, the courts will do it for them.

    Secondly, the Court clearly had little time or patience for the argument that correspondence generated by a computer was any less weighty or worrisome than a letter written by a human being. Lord Justice Jacob’s statement places the emphasis on the people who program the computer and the people who enter the information. The faulty ‘system’ he refers to includes more than just the computer system; arguably, it also encompasses the human factors in the systemic management of the core processes of British Gas.

    Thirdly, the Court noted that perfectly good and inexpensive avenues to remedy in this type of case exist through the UK’s Trading Standards regulations. Thus from a risk management perspective, the probability of a company being prosecuted for this type of error will increase.

    British Gas settled with Ms Ferguson for an undisclosed amount and was ordered to pay her costs.

    What does it mean from an Information Quality perspective?

    From an Information Quality perspective, this case clearly shows the legal risks that arise from (a) disconnected and siloed systems, and (b) inconsistencies between the facts about real world entities that are contained in these systems.

    It would appear that the debt recovery systems in British Gas were not updated with correct customer account balances (amongst other potential issues).

    Ms. Ferguson was told repeatedly by one part of British Gas that the situation was resolved, while another part of British Gas rolled forward with threats of litigation. The root cause here would appear to be an incomplete or inaccurate record or a failure of British Gas’ systems. The Court’s judgment implies that that poor quality data isn’t a defence against litigation.

    The ruling’s emphasis on the importance of people in the management of information, in terms of programming computers (which can be interpreted to include the IT tasks involved in designing and developing systems) and inputting data (which can be interpreted as defining the data that the business uses, and managing the processes that create, maintain, and apply that data) is likewise significant.

    Clearly, an effective information quality strategy and culture, implemented through people and systems, could have avoided the customer service disaster and litigation that this case represents.  The court held the company accountable for not breaking down barriers between departments and systems so that the left-hand of the organization knows what the right-hand is doing.

    Furthermore, it is now more important than ever that companies ensure the accuracy of information about customers, their accounts, and their relationship with the company, as well as ensuring the consistency of that information between systems. The severity of impact of the risk is relatively high (reputational loss, cost of investigations, cost of refunds) and the likelihood of occurrence is also higher in today’s economic climate.

    Given the importance of information in modern businesses, and the likelihood of increased litigation during a recession, it is inevitable: poor quality information will get you sued.