Except that that isn’t what they had uncovered. The setting out of the statistics they had found in the sensationalised way they presented them was a gross distortion of the facts. A distortion that would, to paraphrase Winston Churchill, “be half way around the world before the truth had its boots on”).

BOI will, doubtless, claim that this is a once off and hasn’t happened before. That’s what they said the last time (when it had actually happened before). Furthermore, I hope that BOI are more certain this time as to the root cause (last time out it was variously “retailer error” or “a software upgrade glitch”).

And hopefully their process for catching “shadow transactions” which lead to the double-dipping will kick into play and actually refund the customers affected  (which if this glitch is on the scale of their 2009 one could be up to 200,000 card holders).

For reference the relevant blog posts are:

http://obriend.info/2009/09/09/bank-of-ireland-double-charging/

http://obriend.info/2009/09/09/bank-of-ireland-double-charging-a-clarifying-post/

http://obriend.info/2009/09/10/bank-of-ireland-overcharging-another-follow-up/

http://obriend.info/2009/10/28/bank-of-ireland-again/

The issue also featured over on IQTrainwrecks.com.

My €0.02: This issue appears to manifest itself around Bank Holidays. This suggests a batch load process or some human triggered action doesn’t work correctly when there is a Bank Holiday. Having a process to detect the double-dipped transactions is not a fix, as if it doesn’t work (as seems might be the case here) then the incorrect data gets through.

BOI might want to pay attention to Ferguson v British Gas, which while a UK case, could be arguable precedent for the view that Irish Courts won’t care how complex your IT systems are if a customer is impacted through a failure of your systems to process information correctly.

BOI need to identify the precipitating root cause of this problem, based on the data they have available… I’d start with looking at the dates of incidents (BOI should have more data than newspaper headlines to go on) and seeking to confirm or disprove the ‘Bank Holiday hypothesis’.

Relying on a ‘scrap and rework’ kludge that might itself fail is not a sustainable approach to ensuring information quality or quality of customer service.

Some people might have gotten the impression that I’m obsessed with Fine Gael. I’m not. I’m obsessed with Data, specifically the management of data and information in manner that ensures quality outcomes through quality data governed with due regard to relevant legislation.

On courses I teach on Data Protection and Information Quality I often make reference to “The Joe Duffy Effect” to describe the brand impacts that can arise if organisations don’t take care to manage information as a complex and valuable asset. The term refers to Joe Duffy, a talk radio host on Irish radio. Joe enjoys taking the side of the common man, usually. Occasionally he makes a jape of not getting the point, whether by accident or design we may never know. But organisations who fall foul of the “Joe Duffy Effect” can find themselves fighting rear guard actions against an often intractable foe.

Last week Joe spoke with Jacob, a South African living in Ireland who had received a pre-recorded voicemail to his phone from Michael Martin. Jacob’s tale can be heard in Technicolour on the RTE website.

From the call we glean that:

1. A voicemail was received by Jacob on the 9^th of February with a pre-recorded message (which Jacob played)
2. He has apparently received SMS messages from Fianna Fail with calls for volunteering and campaigning.
3. He is not a member of Fianna Fail
4. He has not asked for Fianna Fail to contact him and does not know where they got his number.
5. The mobile in question is used as an internal work mobile and is not listed. His number is only listed with the Road Safety Authority.

In the broadcast Joe tells Jacob that we live in a democracy.

Correct. We live in a democracy. Specifically we live in democracy where we have decided that the Right to Privacy, while not absolute, is a right that must be defended. Just because we are a democracy it does not give politicians an automatic carte blanche to process data regardless of where or how it has been obtained. These rights to privacy are enshrined in law, in the Constitution and in EU Treaty obligations. Yes, there are balances, mitigations and exemptions with respect to how that right is exercised and protected – but it is still a democratic right of the individual.

During the course of the call, a comment from Fianna Fail was read out saying that they didn’t have Jacob’s number. That is at odds with the evidence – to whit: one recording. And if I’ve learned one thing from watching CSI is that evidence trumps counter claim every day.

So, what is the Data Protection issue here:

• Fair Obtaining – Jacob is not a member of the party and was not aware of how his number came to be called and texted. Granted his phone seems to be for work purposes, but the electronic Privacy regulations apply to business as well as personal data. Also, while he may use the phone for work purposes a big question to ask here is who is paying the bill – him, or a company. If he pays the bill the phone may actually be a personal phone used for business purposes (Sole Trader data is a tricky area in Data Protection land).
• Governance and control of data and/or data processors – Fianna Fail claimed not to have Jacob’s number. The fact that a Fianna Fail party message was left by voicemail and various SMS messages were sent to him suggests that they do. Or if not them then someone working on their behalf. Under the Data Protection Acts, the Data Controller is responsible for the actions of the Data Processor unless the Data Processor acts outside the parameters of the formal contract in writing that governs the Data Controller/Data Processor relationship. So… while it may be true that FF HQ don’t have Jabob’s number, someone processing data on behalf of Fianna Fail does. Fianna Fail not knowing whether or not they had the data suggests a weakness in internal control and governance.
• Accuracy – Joe D. suggested to Jacob that maybe the messages were being sent because of a wrong number. Personal data needs to be kept accurate and up to date. FF should have taken steps to correct the error rather than denying that they have the data. Ultimately FF carry the can for the actions of the Data Processor.

Of course, there is the distinction to be made between normal “direct marketing” and the processing of personal data by a candidate for elected office. Basically during an election personal data is “fair game” for politicians, provided they have obtained it correctly first and have clear consents for contact. Which puts the discussion of “auto dialling” or “power dialling” on the table. According to the Data Protection Commissioner’s website:

The use of automatic dialling machines, to call individual subscribers at random for direct marketing purposes, is prohibited, unless subscribers’ consent has been obtained in advance.  Unsolicited fax messages to individual subscribers are likewise prohibited.

That is why it is important to know who the “subscriber” is to Jacob’s phone. If it is a limited company or similar legal entity, then it is not a call to an “individual” subscriber. If it is his phone or he is a sole trader or part of a partnership, then it is possible that he is an “individual subscriber” and as such the use of an autodialler to RANDOMLY call numbers for direct marketing would be illegal. Dialling from a preloaded list is OK. So long as the list has been fairly obtained and takes into account NDD Opt-out requests etc. And then there is the grey area of the Political exemptions from the Data Protection Acts.

The DPC has issued guidelines to all political parties before the election. My sense is that these guidelines may have been breached in this case.

During previous election campaigns, the Commissioner received numerous complaints from individuals in receipt of unsolicited SMS (text) messages, emails and phone calls from political parties and candidates for election.  In many cases, the individual had no previous contact with the political party or candidate and was concerned at the manner in which their details were sourced.  Subsequent investigations revealed that contact details were obtained from sources such as sports clubs, friends, colleagues and schools.  Obtaining personal data in such   circumstances would constitute a breach of the Data Protection Acts, as there would be no consent from the individual for their details to be obtained and used in this way.

So.. Fianna Fail need to know where their Data Processors are getting their data from. The evidence says they have Jacob’s phone (and who knows who elses’) but don’t know they have Jacob’s phone. That suggests that the Data Controller is not in Control of the Data. Which is a problem in and of itself.

Fine Gael are not the only Data protection flaunters in this election. Fianna Fail have had their moments too. The Green Party STILL don’t have a Privacy statement. And I’m sure the others have slipped up along the way as well. But that is a discussion for another day.

Fine Gael have launched their “Twolicy Page”. I won’t comment on the hideous neologistic portmanteau that is “Twolicy”, other than to say it that seems to have been dreamed up by a pat.

What strikes me about the “Twolicy” page is that it is yet another import of an American election campaign tool into Irish Politics, particularly with the concept of the “E-Canvasser”. Fine Gael dynamically tell us that the E-Canvasser (perhaps some distant cousin of the “Cyber Reporter” who has emerged as the colour piece of the day on certain Irish current affairs shows?) will

“knock on all cyber doors by delving into the depths of Facebook, Twitter, Youtube, Flickr and more! Through the simple medium of sending e-mails, facebooking and tweeting messages of support for Fine Gael you can pledge your commitment to fixing the Irish economy.”

This is a strategy which exists to some extent in Irish politics even today. Many of the letters to Madame Editor are crafted examples of “Astroturfing” – something that appears to be a grass roots movement but is not. I first became aware of the concept back in 2002 when I spotted the Republican Party in the US running “GOPTeamLeader.com” (which, thanks to the interweb waybackmachine I can bring to you in hideous technicolour). Basically the party recruits a team of volunteers who are tasked with sending “on-message” communications to the media (which in 2001 was the newspapers, TV, and radio). In return, the GOP provided a set of reward points (like Green Shield Stamps) which could be saved up and exchanged for rewards such as barbecues, autographed photographs of the Reichsfuerher candidate, and (if memory serves me correctly, an RV.

Fine Gael liken this to door to door canvassing. However that analogy does not hold true because the Internet is not a housing estate or public street. Drop a bus load of eager canvassers on my door step and they will be able to

1. See my house
   
2. See my neighbours’ houses
   

They will not need to ask my neighbour to throw leaflets over my back wall. They will see the big sign in my hall window warning them of the fate that will befall them should they ring the bell and seek discourse (“Warning – political nut lives here”). And most of them are clued in enough to know that the “no canvassers” sticker in the window means that stuffing my letter box with bumph will just be providing stimulus to the paper recycling industry.

The Internet is different. Social media is different. Whoring out your personal contact list to a political party is different. And because it is different, we find ourselves to an extent in uncharted territory with regard to the Data Protection implications of Social Media driven Astroturfing.

Right now I have a contact list of 413 followers on Twitter for my personal account. I have a second twitter account that is for my business. People who follow me know (from my profile and what I tweet about) that I’m a Data nut and I do data protection and information quality training so content about those things will pop up in my timeline. People who follow me also know I’m a bit of a politics geek and enjoy holding our leaders to account. But I try and keep my business tweeting separate from my personal tweeting. And when I whore myself out too much on Twitter, I get friendly DMs from people or I get unfollowed.

This is because the contact details of my friends are information I have gathered for domestic purposes. As such the Data Protection Acts don’t apply. If I was to sign up to be an e-Canvasser (and I can’t get the image of a canvasser handing out bags of yokes out of my head) we would then face the question of whether I was still processing that data for Domestic use or whether I had become a Data Processor working on behalf of Fine Gael, a Data Controller.

The key question would seem to be how much control Fine Gael are exerting over the content and communication from their e-Canvasser Astroturfers, and whether they are offering any form of reward or incentive for people to encourage them to pimp out their domestic contact lists.

If Fine Gael are simply being “passive” and are relying on individuals to act on content that is made available, then there is probably no substantial issue here. It is a case of a person finding content on the web that they think would be of interest to their personal network. We do this every day. It is the way the social web works. Of course, that then raises the question of why they would need you to sign up to their team for this purpose… surely the type of political nut blogger who would retweet or repost their bumph would do so anyway without having to be officially flagged as an “E-Canvasser”?

If Fine Gael are being “neutral” and are simply flagging content to people who have signed up and asking them to do what they see fit with it, then this too is probably OK. The analogy would be the charity that Tweets out a fundraising message and asks their followers to retweet it to send the fundraising virally. The charity has not asked you to commit to being an active fundraiser on their behalf.

However, if Fine Gael are specifying specific content into specific constituencies at specific times and are exercising control over the content of the messages that are being sent, then we are into a potentially problematic area.

The e-Canvasser would not on the Fine Gael payroll. But they would be, in effect, processing personal data on behalf of Fine Gael as part of the “Fine Gael Team”. It would be interesting to find out how much direct “editorial” control that FG are placing on the Facebook Statuses that people are “donating” (and where does this fit in SIPO? What is the monetary value of a person’s Facebook status?) or the emails to “family and friends”. This is personal data that was given to them for a domestic purpose, not for the purposes of canvassing for Fine Gael. Once they commence a “active” canvassing then the use of the data has likely changed from “domestic” to political and the Data Protection Acts would apply. If Fine Gael are directing the timing of messages, the content of messages, and/or the audiences for messages then the e-Canvasser is being directed in their processing by the Data Controller, Fine Gael. And, as Data Controller, Fine Gael would need to ensure that there was clarity about the new political use of the personal data and a clear mechanism for the Data Subject (the canvasser’s family and friends) to opt-out would need to be in place – and FG would, of necessity, need to push this responsibility down to the Canvasser.

Otherwise, FG would not have obtained the data fairly for the purposes of electoral canvassing. It would be no different than if they had asked the local GAA club to email all their members to let them know about Fine Gael’s new policy on tax relief on sliotars and faceguards for hurlers. And that is the kind of thing that the Data Protection Commissioner has already warned against.

Things become an order of magnitude more complicated if Fine Gael are running any kind of incentive scheme for e-Canvassers to drive up the publication of their AstroTurf message.

Of course, Fine Gael have probably thought this through and will have the necessary protocols in place to ensure that there is a mechanism for a Canvasser’s friends to opt out of receiving Fine Gael campaign materials by email, Facebook or Twitter. They have probably realised that people have the same reaction to junk mail on-line as they do at their door step and need to have the ability to put up an on-line “No Canvassers” sign.

Currently the only opt-out mechanism I can see is to unfriend people, unfollow them or block them. Which is exactly what I would do in the physical world if a friend of mine kept ramming leaflets and policy statements from a political party into my face.

Of course, in the absence of such an opt-out facility, Fine Gael (as Data Controller) and the e-Canvasser (as Data Processor) would need to be cautious of falling foul of SI526 2008 (the e-Privacy regulations) which carry a fine of €5000 per breach, capped at €50,000 for an individual. While Twitter and Facebook might not be mentioned in the legislation, email is in section 13(1).

b) A person shall not use or cause to be used any publicly available electronic communications service to send an unsolicited communication for the purpose of direct marketing by means of electronic mail, to a subscriber, who is a natural person, unless the person has been notified by that subscriber that for the time being he or she consents to the receipt of such a communication. 

[edit to clarify some points raised by @tjmcintyre]

Now, the DPC has ruled in the past that there is an exemption covering the Direct Mail (including email and texting)

carried out in the course of political activities by a political party or its members, or by a candidate for election to, or a holder of, elective political office

Question: is the eCanvasser the political party (I would argue yes if FG are exerting sufficient control that they would become a Data Controller)? In which case, the processing is possibly covered.

But I would suggest that this exemption assumes that the email or tweet would be clearly coming from Xyz@partyname.ie or an individual clearly identifying themselves as a member of the party or publicly known to be a candidate for election or an elected official. Getting an email from “yourbestmate@gmail.com’ telling you to go and look at Fine Gael policies, where that email has been sent on the instruction of and under the Control of the party or candidate would seem to me to fall outside the scope of issues already decided.

[/edit]

So, the upshot is that while physical world canvassers have to be careful of yappy dogs, cats that bite and political nuts who have hard questions, eCanvassers need to consider both the social acceptability and potential legality of pimping out their personal contact lists on behalf of a political party. Such tactics are de rigeur in the US. But the US does not operate with the same privacy legislation as Ireland, so ideas imported from overseas must be vetted properly to ensure that no Compliance risks arise.

I would be interested to see what the Data Protection Commissioner’s response to or advice on formal ecanvassing that places the data at arms length but creates a de facto Data Processor/Data Controller relationship would be, particularly if that relationship is not obvious to the recipient of the email or tweet. [update] Perhaps it would be sufficient for the emailer or tweeter to clearly flag that they are part of a formal eCanvassing team acting on behalf of and under the instruction of Fine Gael?[/update]

[update] But the issue of whether the change of use of the data from domestic to overtly political will, in my personal view, give rise to questions of whether the data has been obtained fairly for that new purpose, which is a point already clearly settled in the mind of the DPC.[/update]

The story broke in the Sunday Times just after Christmas and has been picked up as a discussion point on sites such as Boards.ie.  TJ McIntyre has also written about some of the legal issues raised by this.

Ultimately, at the heart of the issue is a fundamental issue of Data Protection Compliance and a failure to treat Personal Data (and Sensitive Personal Data at that) as an asset (something of value) that the Hospital held and holds on trust for the data subject. It is not the Hospital’s data. It is not the HSE’s data. It is my child’s data, and (as I’m of a certain age) probably my data and my wife’s data and my brothers’ data and my sisters-in-laws’ data…..

It’s of particular interest to me as I’m in the process of finishing off a tutorial course on Data Protection and Information Quality for a series of conferences at the end of February (if you are interested in coming, use the discount code “EARLYBIRD” up to the end of January to get a whopper of a discount). So many of the issues that this raises are to the front of my mind.

Rather than simply write another post about Data Protection issues, I’m going to approach this from the perspective of Information as an Asset which has a readily definable Life Cycle at various points in which key decisions should be taken by responsible and accountable people to ensure that the asset continues to have value.

Another aspect of how I’m going to discuss this is that, after over a decade working in Information Quality and Governance, I am a firm believer in the mantra: “Just because you can doesn’t mean you should“. I’m going to show how an Asset Life Cycle perspective can help you develop some robust structures to ensure your data is of high quality and you are less likely to fall foul of Data Protection issues.

And for anyone who thinks that Data Protection and Data Quality are unrelated issues, I direct you to the specific wording in the heading of Chapter 2, Section 1 of the Directive 95/46/EC.

The Information Asset Life Cycle

Information, just like any other asset, has a life cycle through which it needs to be managed. Just because you can keep throwing files into a filing cabinet (or, in Data Protection terms, a “relevant filing system“) or hold it forever in electronic storage doesn’t mean you should.

POSMAD model (thanks to Danette McGilvray)

The key stages in the Information Asset Life Cycle are listed below, and are mapped to the 8 Data Protection Principles in the diagram opposite as well.

• Plan
• Obtain
• Store and Share
• Maintain
• Apply
• Dispose

The answers posed to the questions asked at each of these stages affect the ability of the organisation to meet its obligations under the Data Protection Act, as can be seen by the mapping of DP Principles to POSMAD Life Cycle stages.

Just like any other asset, information needs to be managed.

When you are hiring new staff, you plan for what type of people you will need in your business. You’ll need to have strategies for obtaining those staff, planning in place for how to store them (offices, desks etc.). You will (ideally) want to maintain them (training, rewards and recognition, staff retention), and be sure you can Apply them to their job (right tools, adequate resources etc.). Inevitably, you are also going to have to have answers to the question of what you will do with your staff when you no longer need them and need to dispose of them (retirement, redundancy, etc.).

With Information, you need to Plan what type of data you will be capturing and why, and who you’ll share it with. You’ll need to define policies and methods for obtaining that data (e.g. standardised testing protocols such as the Guthrie Card and the structured information captured on hospital test request forms). You’ll need to plan how that data will be stored and shared so it can be found when needed, and can be readily linked to other data etc. You’ll need to have some  consideration to how you will maintain that data (e.g. keeping personal data up to date for as long as you are holding it), and you’ll need to have a clear plan and protocol for how to dispose of the data when it is no longer needed.

Note that in this context, disposal does not necessarily mean the destruction of the data. It could simply be a process or policy that defines when data has become “excessive” and mandates the anonymising of all the data to a level suitable for statistical reporting and study but that is no longer “personal data” in the meaning of the Data Protection Acts.

Of course, just as you have measures that help you track and manage the effectiveness with which you are managing other assets (e.g. tracking equipment services and outage statistics for office equipment or purchase volumes for stationery and paper clips), you should ideally look to develop some metrics that help you know how well you are answering the various questions at each stage in the Information Asset Life Cycle.

Finally, you need to bear in mind that the Data Protection Acts now cover personal data held in formats other than electronic files. So paper based data (e.g. patient information associated with a Guthrie Card) is protected by the Data Protection Act when it is held in a “Relevant Filing System”. So your paper filing needs to bear up to the scrutiny of the Information Asset Life Cycle

The Temple Street Situation

What appears to have happened in the Temple St situation is that there was a failure to properly plan for the management of a key information asset.

• While there was a plan to provide a national scheme for testing, some key questions were not asked at the planning stage. As a result, the answers are not necessarily forthcoming to parents when faced with the test.
  

  Good Data Protection requires Planning

  For example, parents do not always know that the test is being done in Temple St. Certainly, parents were not aware that the data was to be held indefinitely. Ultimately, if Aldi (see image) are able to tell me what my personal data (CCTV images) is to be used for and who I can contact if I have a query, then the HSE and its sub-division and constituent hospitals should have been able to do the same.

• There appears to have been no thought given to the conditions or criteria for reasonable disposal (either outright disposal or anonymising) of the data. This gave rise to a situation where a Data Protection breach was inevitable.
• Parents (acting as legal guardians of their children) were not given the opportunity to opt in or opt out of being part of further scientific research using the blood samples taken from children.
• Likewise, children and adults (and the DPA does not require someone to be an adult to be protected) who have data on file in Temple St. do not appear to have a clear mechanism available whereby they can request that their data be blocked from use in processes other than that for which it was initially provided. (I’ve looked at the list of Data Controllers and Data Processors registered with the Data Protection Commissioner and Temple St. doesn’t seem to be listed in its own right is listed as the “Children’s University Hospital” and it’s not immediately clear from the register whether HSE Dublin North East or Dublin Mid-Leinster covers Temple St.) [Thanks to Hugh Jones, the lead trainer on the Irish Computer Society's Data Protection course for the correction]
• The personal data attached to the blood samples will, in a large number of cases, be woefully out of date and inaccurate – in itself a breach of the Data Protection Act – as there is likely no process to keep that data up to date. If the data is not being maintained, it is not needed and is excessive to the stated purposes for which it is being used. If the personal data (e.g. name and address) on samples that are 26 years old is being used for specific purposes, then one would suggest that any action taken on foot of such analysis is likely to be wrong.

A number of commenters on Boards.ie and elswhere have commented to the effect that the personal data could be excised out when the blood test data or samples were being used. That misses the fundamental point. There is a point in time beyond which having my daughter’s name and address cease to have an actionable value in any analysis of blood tests taken from her. If you don’t need it, then the data you are holding is excessive (in the meaning of the DPA and the EU Data Protection Directives) and should be permenantly removed. Aggregation of samples into clusters based on geographic region would likely allow for data sets suitable for scientific and statistical analysis.

But then we are right back to the Plan stage of the POSMAD model. What was intended to be done with this data? What are the stated purposes for which it is being captured? At what point does the data become excessive for those purposes? What is the plan to dispose of the excessive data (while retaining data appropriate to the stated purposes?

Conclusion

Information is an Asset. It needs to be managed and protected as such. Adopting an Asset Life Cycle approach to planning, with your Data Protection Duties forming the basis for your Key Questions can help your organisation properly plan to manage your Asset in a compliant manner. Furthermore, the Data Protection rules are enablers rather than restrictions as they provide a clear framework within which you can manage the expectations and worries of Data Subjects so that you get better up-take and happy and informed consent to you having the relevant data for as long as you plan to need it.

A key message coming from “DPA-Aware” bloggers affected by this issue is that if they had been asked if they minded anonymised data being held for research purposes they would not necessarily have objected. Investing time in planning and in valuing the Personal Data Asset which the HSE holds on Trust for the affected Data Subjects (they don’t own it) would have avoided the negative reaction and push back from rightly concerned parents and Civil Rights groups.

Want to learn practical approaches to avoiding this type of boo boo in your organisation? Why not come along to my tutorial at the 2010 IDQ Seminar Series event in Dublin on the 22nd and 23rd of February. Use the code “EarlyBird” before Jan 31st to get big savings on the event. If you are interested in just my tutorial, use the code “DoBlog” when registering.

The general format of the event was that a few of the IAIDQ Directors shared stories from their personal experiences or professional insights and extrapolated out what the landscape might be like in 2014 (the 10th anniversary of the IAIDQ).

A key factor in all of the stories that were shared was the need to focus on the needs of your information customer, and the fact that the information customer may not be the person who you think they are. More often than not, failing to consider the needs of your information customers can result in outcomes that are significantly below expectations.

One of my favourite legal maxims is Lord Atkin’s definition of who your ‘neighbour’ is who you owe legal duties of care to. He describes your ‘neighbour’ as being anyone who you should reasonably have in your mind when undertaking any action, or deciding not to take any action. While this defines a ‘neighbour’ from the point of view of litigation, I think it is also a very good definition of your “customer” in any process.

Recently I had the misfortune to witness first hand what happens when one part of an organisation institutes a change in a process without ensuring that the people who they should have reasonably had in their mind when instituting the change were aware that the change was coming.

My wife had a surgical procedure and a drain was inserted for a few days. After about 2 days, the drain was full and needed to be changed. The nurses on the ward couldn’t figure out how to change my wife’s drain because the drain that had been inserted was a new type which the surgical teams had elected to go with but which the ward nurses had never seen before.

For a further full day my wife suffered the indignity of various medical staff attempting to figure out how to change the drain.

1. There was no replacement drain of that type available on the ward. The connections were incompatible with the standard drain that was readily available to staff on the ward and which they were familiar with.
2. When a replacement drain was sourced and fitted, no-one could figure out how to actually activate the magic vacuum function of it that made it work. The instructions on the device itself were incomplete.

When the mystery of the drain fitting was eventually solved, the puzzle of how to actually read the amount of fluid being drained presented itself, which was only of importance as the surgeon had left instructions that the drain was to be removed once the output had dropped below a certain amount. The device itself presented misleading information, appearing to be filled to one level but when emptied out in fact containing a lesser amount (an information presentation quality problem one might say).

The impacts of all this were:

• A distressed and disturbed patient increasingly worried about the quality of care she was receiving.
• Wasted time and resources pulling medical staff from other duties to try and solve the mystery of the drain
• A very peeved and increasingly irate quality management blogger growing more annoyed at the whole situation.
• Medical staff feeling and looking incompetent in front of a patient (and the patient’s family)

Eventually the issues were sorted out and the drain was removed, but the outcome was a decidedly sub-optimal one for all involved. And it could have been easily avoided had there been proper communication about the change to the ward nurses and the doctors in the department from the surgical teams when they changed their standard. Had the surgical teams asked the question of who should they have in their minds to communicate with when taking an action, surely the post-op nurses should have featured in there somewhere?

I would be tempted to say “silly Health Service” if I hadn’t seen exactly this type of scenario play out in day to day operations and flagship IT projects during the course of my career. Whether it is changing the format of a spreadsheet report so it can’t be loaded into a database or filtered, changing a reporting standard, changing meta-data or reference data, or changing process steps, each of these can result in poor quality information outcomes and irate information customers.

So, while information quality is defined from the perspective of your information customers, you should take the time to step back and ask yourself who those information customers actually are before making changes that impact on the downstream ability of those customers to meet the needs of their customers.

I wrote about this last month (see the archives here), picking up on a post from Tuppenceworth.ie earlier in the summer. I won’t be writing anything more about the issue (at least not for now).

Looking back through my archives I found the picture below in a post that I’d written back in May when Simon on Tuppenceworth first raised his issue with BOI’s Laser Cards.

In doing so I have been yet again reminded of the challenges faced by information quality professionals when trying to unpick a concatenated string of text in a field that is labelled “Name”. The challenges are manifold:

• Name formats differ from  to culture to culture – and it is not a Western/Asian divide as some people might assume at first.
• Master Data for name spellings is notoriously difficult to obtain. My wife and I compared spellings of some common names in two books of baby names and the variations were staggering, with a number of spellings we are very familiar with (including my own name) not listed in either.
• Often Family Names (surnames) can be used as Given Names (first names) such as Darcy (D’Arcy) or Jackson (Jackson) or Casey.
• Often people pick names for their children based on where they were born or where they were conceived (Brooklyn Beckham, the son of footballer David Beckham is a good example).
• Non-name words can appear in names, such as “Meat Loaf” or “Bear Grylls“
• Douglas Adams famously named a character in the Hitchhiker’s Guide to the Galaxy after one of the “dominant life forms” – a car called a “Ford Prefect“
• Names don’t always fit into an assumed varchar(30) or even varchar(100) field.
• It is possible to have a one character Given name and a one character Family name.
• Two character Family names are more common than we think.
• Unicode characters, hyphens, spaces, apostrophes are all VALID in names – particularly if they are diacritical marks which change the meaning of words in particular languages.
• And then you have people who change their names to silly things to be “different” or “special”,  but who create interesting statistical challenges for data profilers and parsing tools.

Among the examples I found flicking through one of our baby name books last evening where “Alpha” and “Beta”. Personally I think it sends the wrong signals to name your children after letters of the Greek alphabet, but I’m sure it is helpful if you have had twins to keep them in order.

I also found “Bairn” given as a Scots Gaelic name for a baby girl. I had to laugh at this as “Bairn” is actually a Scots dialect word for Child. Even Wikipedia recognises this and has a redirect from “Bairn” to “child“.  But it does remind me of the terribly sexist “joke” where the father asks the doctor after the birth whether it is a boy or a child his wife has just delivered.

The trouble with names, from an information quality point of view, is that they are inherently personal things which people have a strong attachment to. So getting spellings wrong can have negative effects on your business and your relationship with your customers (like my on-going gripe with Vodafone). But often companies need to accept the “fuzziness” of identity in order to match records and meet the needs of Anti-money laundering or similar regulations or simply to create a single view of their customers. But the EU Data Protection regulations require organisations to hold data accurately – with accuracy being defined from the point of view of the data subject.

So, when you head has stopped spinning from managing all the Alphas, Betas, Brooklyns, and Ford Prefects, as an Information Quality practitioner you are faced with juggling the needs of Customer Intimacy, the demands of Data Protection, and a range of other legal requirements when you are deciding how to clean your name data up.

Jim Harris’ excellent series of posts on Data Profiling  gives a great run through of how data profiling tools can help you figure out what is in those strings of text in that field labelled “Name”. However, you should exercise caution in your assumptions about what a name might be and might look like.

For example, allegedly the longest Name in the world is

Mr. Adolph Blaine Charles David Earl Frederick Gerald Hubert Irvin John Kenneth Lloyd Martin Nero Oliver Paul Quincy Randolph Sherman Thomas Uncas Victor William Xerxes Yancy Wolfeschlegelsteinhausenbergerdorffwelchevoralternwarengewissenschaftschafe rswessenschafewarenwohlgepflegeundsorgfaltigkeitbeschutzenvonangreifeudurch ihrraubgierigfeindewelchevoralternzwolftausendjahresvorandieerscheinenersch einenvanderersteerdemenschderraumschiffgebrauchlichtalsseinursprungvonkraft gestartseinlangefahrthinzwischensternaitigraumaufdersuchenachdiesternwelche gehabtbewohnbarplanetenkreisedrehensichundwohinderneurassevonverstandigmens chlichkeitkonntefortpflanzenundsicherfeuenanlebenslanglichfreudeundruhemitn icheinfurchtvorangreifenvonandererintelligentgeschopfsvonhinzwischenternart Zeus igraum Senior

Now, that’s 802 character’s long (including the Mr). It also doesn’t fit very easily into a <given name><middle_initial><Family_name> format which most of us would probably start with as our template for parsing a name string. Note also, that he was a “senior”, so there is another one of this name out there somewhere. Perhaps he just goes by the name “Mr Adolph Ingram”. I’d also hate to see what a matching process would make of that name (how many match keys would need to be created?)

There are some interesting comments about this name on the Stackoverflow.com website. Some of them are helpful pointers to the different structures of names that exist out there. Others show the risks that are run in designing and developing systems based on a particular cultural bias or perception of what a name is (a lot of commenters refer to US government forms and how people with long names usually have a form that they use for “official purposes”. This is not necessarily a “safe” assumption… not every government form in each country is the same. Indeed, many Irish Government forms don’t have enough space for my address or my wife’s first name and compound family name).

In a case that will be put up on IQTrainwrecks.com, the impact of cultural assumptions about what a “valid” name is can be seen in this story of a woman who trouble boarding a plane because of her name.

Fans of Star Trek:Deep Space 9 will recall how the actor who played  Doctor Bashir changed his credit name from Siddig el Fadil to “Alexander Siddig” because (it is claimed) fans couldn’t pronounce Siddig el Fadil properly. The full version of his name would be an interesting challenge for a data profiling tool  in the hands of an Information Quality professional and certainly challenges the <Given_name><middle_name><family_name> format used in Anglo-Saxon cultures.

Wikipedia has an interesting page of references for unusual and long names which I would recommend at the very least as a tool to blow away any assumptions you have of what’s in a name.

No set of name Master Data in a reference dictionary will ever be complete or fully accurate. When balancing the needs for accuracy and correctness of data versus the needs to match and consolidate data (either for internal business purposes like CRM or for legally mandated purposes such as AML or PEP processes), you need to give some thought to how you will weight and manage your priorities within the data. Furthermore, assumptions you might make about the “correct” structure of a name could actually create information quality problems for you.

For now, Mrs DoBlog and I will continue to see if we can find a name that fits the impending arrival. But it has been made a lot harder because of my insights into the fun a name can cause for an information quality team.

I’m angling for something very traditional and Irish…. just to really confuse people and break Soundex keys.

I’ve written and presented on this theme many times over the past few years and it always struck me how people started off being in the “that’s too theoretical” camp but by the time I (and occasionally my speaking/writing partner on this stuff, Mr Fergal Crehan) had finished people were all but phoning their company lawyers to have a chat.

To an extent, I have to admit that in the early days much of this was theoretical, taking precedents from other areas of law and trying to figure out how they fit together in an Information Quality context. However, in January 2009 a case was heard in the Court of Appeal in England and Wales which has significant implications for the Information Quality profession and which has had almost no coverage (other than coverage via the IAIDQ and myself). My legal colleagues describe it as “ground breaking” for the profession because of the simple legal principle it creates regarding complex and silo’d computing environments and the impact of disparate and plain crummy data. I see it as a clear rallying cry that makes it crystal clear that poor information quality will get you sued.

Recent reports (here and here) and anecdotal evidence suggest that in the current economic climate, the risk to companies of litigation is increasing. Simply put, the issues that might have been brushed aside or resolved amicably in the past are now life and death issues, at least in the commercial sense. As a result there is now a trend to “lawyer up” at the first sign of trouble. This trend is likely to accelerate in the context of issues involving information, and I suspect, particularly in financial services.

A recent article in the Commercial Litigation Journal (Frisby & Morrison, 2008) supports this supposition. In that article, the authors conclude:

“History has shown that during previous downturns in market conditions, litigation has been a source of increased activity in law firms as businesses fight to hold onto what they have or utilise it as a cashflow tool to avoid paying money out.”

The Case that (should have) shook the Information Quality world

The case of Ferguson v British Gas was started by Ms. Ferguson, a former customer of British Gas who had transferred to a new supplier but to whom British Gas continued to send invoices and letters with threats to cut off her supply, start legal proceedings, and report her to credit rating agencies.

Ms Ferguson complained and received assurances that this would stop but the correspondence continued. Ms Ferguson then sued British Gas for harassment.

Among the defences put forward by British Gas were the arguments that:

(a) correspondence generated by automated systems did not amount to harassment, and (b) for the conduct to amount to harassment, Ms Ferguson would have to show that the company had “actual knowledge” that its behaviour was harassment.

The Court of Appeal dismissed both these arguments. Lord Justice Breen, one of the judges on the panel for this appeal, ruled that:

“It is clear from this case that a corporation, large or small, can be responsible for harassment and can’t rely on the argument that there is no ‘controlling mind’ in the company and that the left hand didn’t know what the right hand was doing,” he said.

Lord Justice Jacob, in delivering the ruling of the Court, dismissed the automated systems argument by saying:

“[British Gas] also made the point that the correspondence was computer generated and so, for some reason which I do not really follow, Ms. Ferguson should not have taken it as seriously as if it had come from an individual. But real people are responsible for programming and entering material into the computer. It is British Gas’s system which, at the very least, allowed the impugned conduct to happen.”

So what does this mean?

In this ruling, the Court of Appeal for England and Wales has effectively indicated a judicial dismissal of a ‘silo’ view of the organization when a company is being sued. The courts attribute to the company the full knowledge it ought to have had if the left hand knew what the right hand was doing. Any future defence argument grounded on the silo nature of organizations will likely fail. If the company will not break down barriers to ensure that its conduct meets the reasonable expectations of its customers, the courts will do it for them.

Secondly, the Court clearly had little time or patience for the argument that correspondence generated by a computer was any less weighty or worrisome than a letter written by a human being. Lord Justice Jacob’s statement places the emphasis on the people who program the computer and the people who enter the information. The faulty ‘system’ he refers to includes more than just the computer system; arguably, it also encompasses the human factors in the systemic management of the core processes of British Gas.

Thirdly, the Court noted that perfectly good and inexpensive avenues to remedy in this type of case exist through the UK’s Trading Standards regulations. Thus from a risk management perspective, the probability of a company being prosecuted for this type of error will increase.

British Gas settled with Ms Ferguson for an undisclosed amount and was ordered to pay her costs.

What does it mean from an Information Quality perspective?

From an Information Quality perspective, this case clearly shows the legal risks that arise from (a) disconnected and siloed systems, and (b) inconsistencies between the facts about real world entities that are contained in these systems.

It would appear that the debt recovery systems in British Gas were not updated with correct customer account balances (amongst other potential issues).

Ms. Ferguson was told repeatedly by one part of British Gas that the situation was resolved, while another part of British Gas rolled forward with threats of litigation. The root cause here would appear to be an incomplete or inaccurate record or a failure of British Gas’ systems. The Court’s judgment implies that that poor quality data isn’t a defence against litigation.

The ruling’s emphasis on the importance of people in the management of information, in terms of programming computers (which can be interpreted to include the IT tasks involved in designing and developing systems) and inputting data (which can be interpreted as defining the data that the business uses, and managing the processes that create, maintain, and apply that data) is likewise significant.

Clearly, an effective information quality strategy and culture, implemented through people and systems, could have avoided the customer service disaster and litigation that this case represents.  The court held the company accountable for not breaking down barriers between departments and systems so that the left-hand of the organization knows what the right-hand is doing.

Furthermore, it is now more important than ever that companies ensure the accuracy of information about customers, their accounts, and their relationship with the company, as well as ensuring the consistency of that information between systems. The severity of impact of the risk is relatively high (reputational loss, cost of investigations, cost of refunds) and the likelihood of occurrence is also higher in today’s economic climate.

Given the importance of information in modern businesses, and the likelihood of increased litigation during a recession, it is inevitable: poor quality information will get you sued.

Finding Red Herrings or Missing a Trick?

For most businesses there are major advantages to investing money in doing direct research with your customer base In theory it’s a ready built list of people who are familiar with your business – so they can speak with authority on their experience as your customer.

The value of customer research to business should be by now fairly obvious, but there’s an old saying in research (and elsewhere) – “garbage in, garbage out”. The insights built off the data generated from your customer list is only as relevant as the list of people you ask to participate in the research.

However if, for example, they are lapsed customers then researching them is going to give you a picture of what your past customers wanted from you (unless these people are the focus of your research of course). Is this the same as what your present customers want? And if you are looking for why past customers stopped dealing with you and use a list full of current customers you end up with either few people able to answer the questions you set or …worse….data from people who shouldn’t have answered the question – which leads to another scenario.

Picture an important piece of research done with a list of past and present customers mixed in together with no way to tell who is who. Do current and ex-customers differ in their wants and needs from your business? I don’t know – and neither do you. So how useful are any insights generated from this research? Not being able to separate these two groups gives rise to two potential scenarios. Either the excess numbers in there are throwing up ‘clear’ results that are not applicable to your current customers or the combination of both bodies is adding noise which stops you uncovering real insights about the customers you’re interested in – you’re either finding red herrings or you’re missing a trick!

I’ve used just one scenario here to make a point that can be applied to lots of customer data stored by companies – be it incorrect regional information, incorrect gender, you can add whatever block of data is relevant to your own company here and the story is the same. If the data is not accurate then any use it is put to suffers.