The DOBlog

Category: Ethics & Law of Information

A category dealing with the ethics and legalities of the management of Information and Information Quality.

  • Fair use/Specified purpose and the IBTS

    I am a blood donor. I am proud of it. I have provided quite a lot of sensitive personal data to the IBTS over the years that I’ve been donating.

    The specific purposes for which I believed I was providing the information was to allow the IBTS to administer communications with me as a donor (so I know when clinics are on so I can donate), to allow the IBTS to identify me and track my donation patterns, and to alert IBTS staff to any reasons why I cannot donate on a given occasion (donated too recently in the past, I’ve had an illness etc.). I accepted as implied purposes the use of my information for internal reporting and statistical purposes.

    I did not provide the information for the purposes of testing software developed by a 3rd party, particularly when that party is in a foreign country.

    The IBTS’s website (www.ibts.ie) has a privacy policy which relates to data captured through their website. It tells me that

    The IBTS does not collect any personal data about you on this website apart from information which you volunteer (for example by emailing us or by using our on line contact forms). Any information which you provide in this way is not made available to any third parties, and is used by the IBTS only for the purpose for which you provided it.

    So, if any information relating to my donor record was captured via the website, the IBTS is in breach of their own privacy policy. So if you register to be a donor… using this link… http://www.ibts.ie/register.cfm?mID=2&sID=77 then that information is covered by their Privacy policy and you would not be unreasonable in assuming that your data wouldn’t wind up on a laptop in a crackhouse in New York.

    In the IBTS’s Donor Charter, they assure potential Donors that:

    The IBTS guarantees that all personal information about donors is kept in the strictest confidence

    Hmm… so no provision here for production data to be used in testing. Quite the contrary.

    However, it gets even better… in the Donor Information Leaflet on the IBTS’s website, in the Data Protection section (scroll down… it’s right at the bottom), current and potential donors the IBTS tells us that (emphasis is mine throughout):

    The IBTS holds donor details, donation details and test results on a secure computerised database. This database is used by the IBTS to communicate with donors and to record their donation details, including all blood sample test results. It is also used for the proper and necessary administration of the IBTS. All the information held is treated with the strictest confidence.

    This information may also be used for research in order to improve our knowledge about the blood donor population, and for clinical audit, to assess and improve the quality of our service. Wherever possible, all such information will be anonymised.

    Right.. so from their policy and their statement of fair use and specified purposes we learn that:

    1. They can use it for communication with donors and for tracking donation details and results of tests (as expected)
    2. They can use it for necessary administration. Which covers internal reporting but, I would argue, not giving it to other organisations to lose on their behalf.
    3. They can use it for research about the blood donor population, auditing clinical practices. This is OK… and expected.
    4. They are also permitted to use the data to “improve the quality of [their] service”. That might cover the use of the data for testing…

    Until you read that last bit… the data would be anonymised whenever possible. That basically means the creation of dummy data as described towards the end of my last post on this topic.

    So, the IBTS did not specify at any time that they would use the information I had provided to them for the purposes of software development by 3rd parties. It did specify a purpose for using the information for the improvement of service quality. But only if it was anonymised.

    Section 2 of the Data Protection Act says that data can only be used by a Data Controller for the specific purposes for which it has been gathered. As the use of un-anonymised personal data for the purposes of software development by agencies based outside of the EU (or in the EU for that matter) was not a specified use, the IBTS is, at this point, in breach of the Data Protection Act. If the data had been anonymised (ie if ‘fictional’ test data had been used or if the identifying elements of the personal data had been muddled up before being transferred) there would likely be no issue.

    • Firstly, the data would have been provided in a manner consistent with the specified use of the data
    • Secondly, there would have been no risk to personal data security as the data on the stolen laptop would not have related to an identifiable person in the real world.

    Of course, that would have cost a few euros to do so it was probable de-scoped from the project.

    If I get a letter and my data was not anonymised I’ll be raising a specific complaint under Section 2 of the Data Protection Act. If the data was not anonymised (regardless of the security precautions applied) then the IBTS is in breach of their specified purposes for the collection of the data and are in breach of the Data Protection Act.

    Billy Hawkes, if you are reading this I’ve just saved your team 3 weeks work.

  • Irish Blood Transfusion Service loses data..

    Why is it that people never learn? Only months after the debacle of HMRC sending millions of records of live confidential data whizzing around in the post on 2 CDs (or DVDs), the Irish Blood Transfusion Service (IBTS) has had 171,000 records of blood tests and blood donors stolen.

    The data was on a laptop (bad enough from a security point of view). The data was (apparently) secured with 256bit AES encryption (happy days if true). The laptop was taken in a mugging (unfortunate). The mugging took place in New York (WTF!?!?)

    Why was the data in New York?
    It would seem that the IBTS had contracted with the New York Blood Centre (NYBC) for the customisation of some software that the NYBC had developed to better manage information on donors and blood test results. To that end the IBTS gave a copy of ‘live’ (or what we call in the trade ‘production’) data to the NYBC for them to use in developing the customisations.

    So, personal data, which may contain ‘sensitive’ data relating to sexual activity, sexual behaviour, medicial conditions etc. was sent to the US. But it was encrypted, we are assured.

    A quick look at the Safe Harbor list of the US Dept of Commerce reveals that the NYBC is not registered as being a ‘Safe Harbor’ for personal data from within the EU. Facebook is however (and we all know how compliant Facebook is with basic rules of data protection).

    Apparently the IBTS relied on provisions of their contract with the NYBC to ensure and assure the security of the data relating to REAL people. As yet no information has come to light regarding whether any audits or checks were performed to ensure that those contractual terms were being complied with or were capable of being complied with.

    How did the data get to New York?
    From the IBTS press release it is clear that the data got to New York in a controlled manner.
    An employee of NYBC took the disc back from Ireland and placed it in secure storage.

    Which is a lot better than sticking two CDs in the post, like the UK Revenue services did not so long ago.

    What about sending the data by email? Hmmm… nope, not secure enough and the file sizes might be to big. A direct point to point FTP between two servers? that would work as well, assuming that the FTP facilities were appropriately secured by firewalls and a healthy sense of paranoia.

    Why was the data needed in New York?
    According to the Irish Times

    The records were in New York, the blood service said, “because we are upgrading the software that we use to analyse our data to provide a better service to donors, patients and the public service”.

    Cool. So the data was needed in New York to let the developers make the necessary modifications to code.

    Nice sound bite. Hangs together well. Sounds reasonable.

    Unfortunately it is total nonsense.

    For the developers to make modifications to an existing application, what was required in New York was

    • A detailed specification of what the modifications needed to be to enable the software to function for Irish datasets and meet Irish requirements. Eg. if the name/address data capture screens needed to change they should have been specified in a document. If validation routines for zip cods/postcodes needed to be turned off, that should have been specified. If base data/reference data needed to be change – specify it in a document. Are we seeing a trend here?
    • Definition of the data formats used in Ireland. by this I mean the definition of the formats of data such as “social security number”. We call it a PPSN and it has a format nnnnnnnA as opposed to the US format which has dashes in the middle. A definition of the data formats that would be used in Ireland and a mapping to/from the US formats would possibly be required… this is (wait for it) another document. NOT THE DATA ITSELF
    • Some data for testing. Ok, so this is why all 171000+ records were on a laptop in New York. ehh… NO. What was required was a sample data set that replicates the formats and patterns of data found in the IBTS production data. This does not mean a cut of production data. What this means is that the IBTS should have created dummy data that was a replica of production data (warts and all – so if there are 10% of their records that have text values in fields where numbers would be expected, then 10% of the test data should reflect this). The test data should also be tied to specific test cases (experiments to prove or disprove functionality in the software).

    At no time was production data needed for development or developer testing activities in New York. Clear project specification and requirements documentation, documents about data formatting and ‘meta-data’ (data about data), Use Cases (walk throughs of how the software would be used in a given process – like a movie script) and either a set of dummy sample data that looks and smells like you production data or a ‘recipe’ for how the developer can create that data.

    But the production data would be needed for Acceptance testing by IBTS?
    eh… nope. And even if it was it would not need to be sent to New York for the testing.

    User Acceptance testing is a stage of testing in software development AFTER the developer swears blind that the software works as it should and BEFORE the knowledge workers in your organisation bitch loudly that the software is buggered up beyond all recognition.

    As with all testing it does not require a the use of production data is not required, and indeed is often a VERY BAD IDEA (except in certain extreme circumstances such as the need for volume stress testing or testing of very complex software solutions that need data that is exactly like production to be tested effectively… eg. a complex parsing/matching/loading process on a multi-million record database – and even at that, key data not relevant to the specific process being tested ought to be ‘obscured’ to ensure data protection compliance ).

    What is required is that your test environment is as close a copy to the reality you are testing for as possible. So, from a test data point of view, creating test data that looks like your production data is the ideal. One way is to do data profiling, develop an understanding of the ‘patterns’ and statistical trends in your data and then hand carve a set of test data that looks and smells like your production data but is totally fake and fraudulent and safe. Another approach is to take a copy of your production data and bugger around with it to mix names and addresses up, replace certain words in address data with different words (e.g. “Park” with “Grove” or “Leitrim” with “Carialmeg” or “@obriend.info” with “obriend.fakedatapeople” – whatever works). So long as the test data is representative of the structure and content of your production data set and can support the test scenarios you wish to perform then you are good to go.

    So, was the production data needed in New York – Nope. Would it be needed for testing in a test event for User Acceptance testing? Nope.

    And who does the ‘User Acceptance testing’? Here’s a hint… whats the first word? User Acceptance testing is done by representatives of the people who will be using the software. They usually follow test scripts to make sure that specific functionality is tested for, but importantly they can also highlight were things are just wrong.

    So, were there any IBTS ‘users’ (knowledge workers/clerical staff) in New York to support testing? We don’t know. But it sounds like the project was at the software development stage so it is unlikely. So why the heck was production data being used for development tasks?

    So… in conclusion
    The data was stolen in New York. It may or may not have been encrypted (the IBTS has assured the public that the data was encrypted on the laptop… perhaps I am cynical but someone who takes data from a client in another nation home for the weekend might possibly have decrypted the data to make life easier during development). We’re not clear (at this point) how the data got to New York – we’re assuming that an IBTS employee accompanied it to NY stored on physical media (the data, not the employee).

    However, there is no clear reason why PRODUCTION data needed to be in New York. Details of how the IBTS’s current data formats might map to the new system, details of requirements for changes to the NYBC’s current system to meet the needs of the IBTS, details of the data formats in the IBTS’s current data sets (both field structues and, ideally, a ‘profile’ of the structure of the data and any common errors that occur) and DUMMY data might be required for design, development and developer testing are all understandable. Production data is not.

    There is no evidence, other than the existence of a contractual arrangement, that the NYBC had sufficient safeguards in place to ensure the safety of personal data from Ireland. The fact that an NYBC employee decided to take the data out of the office into an unsecure environment (down town New York) and bring it home with them would evidence that, perhaps, there is a cultural and procedural gap in NYBC’s processes that might have meant they either couldn’t comply or didnt’ understand what the expectation of the clauses in those contracts actually meant.

    For testing, what is required is a model of production. A model. A fake. A facsimile NOT PRODUCTION. The more accurate your fake is the better. But it doesn’t need to be a carbon copy of your production data with exactly the same ‘data DNA’… indeed it can be a bad idea to test with ‘live’ data. Just like it is often dangerous to play with ‘live’ grenades or grab a ‘live’ power line to see what will happen.

    The loss of our IBTS data in New York evidences a failure of governance and a ‘happy path’ approach to risk planning, and a lack of appreciation of the governance and control of software development projects to ensure the protection of live data.

    As this was a project for the development of a software solution there was no compelling reason that I can identify for production data to have been sent from Ireland to New York when dummy data and project documentation would have sufficed.

    The press release from the IBTS about this incident can be found here..

    [Update – Simon over at Tuppenceworth has noted my affiliation to the IAIDQ. Just to clarify, 99% of this post is about basic common sense. 1% is about Information Management/Information Quality Management. And as this post is appearing here and not on the IAIDQ’s website it goes without saying that my comments here may not match exactly the position of the IAIDQ on this issue. I’m also a member of the ICS, who offer a Data Protection certification course which I suspect will be quite heavily subscribed the next time it runs.]

    [Update 2: This evening RTE News interviewed Dr David Gray from DCU who is somewhat of an expert on IT security. The gist of Dr Gray’s comments were that software controls to encrypt data are all well and good, but you would have to question the wisdom of letting the information wander around a busy city and not having it under tight physical control… which is pretty much the gist of some of my comments below. No one has (as yet) asked why the hell production data rather than ‘dummy’ data was being used during the development phase of a project.]

  • Facebook & Data Protection

    The Younger McGarr (Simon that is) has a very detailed and well written post on the data protection issues that arise (and seemingly are ignored) by Facebook. It can be found over at the McGarr Solicitors website. He has already picked up some complimentary comments, including one from Thomas Otter (who has written on these issues previously). (Surely a reply from Robert Scoble is only a mouse-click away?)

    I’ve been scratching away on some notes for a post on Facebook myself (never one to miss a rolling bandwagon me). Expect more on this soon. (ie as soon as I’ve written the buggering thing).

  • What the…? – Irish Political coverage ignores the Elephant in the room

    I’m frankly baffled. We are in the run up to a General Election here in Ireland. All the media pundits are quoting 24th May as the date of the (as yet unannounced) election. This would require our parliament to be dissolved at the latest next week.

    Ireland runs a Proportional Representation/Single Transferable Vote system. It is built into our Consitution. There is a large body of legal opinion around the thresholds at which the ratio of elected representatives to number of people in a constituency breaches the Constitution. We are, it seems, at that point in 10 consituencies out of a total of 43. This has resulted in a Constitutional challenge in the High Court by two Independent TDs (Members of Parliament) to the holding of any election until the balance of Proportional Representation is restored through changes to the make up of Consituencies.

    The fact that key demographics had changed and there was a risk that the Electoral Constituency boundaries or numbers of representatives in each consituency might need to be altered was identified in September 2006 when the preliminary figures from our Census were published. There is no legal obligation on the Government to act or react to these however. The final Census figures were published on the 29th of March. These should be acted on or else there is the risk of any election being declared unconstitutional.

    The risk is that if the Dáil (our parliament) is dissolved prior to an election the running of which is declared unconstitutional until the parliament (the one that has been dissolved) addresses the issue of the Electoral contituencies then we could find ourselves with a bit of a governmental and Constitutional crisis.

    Yet the media continue to focus on the dog and pony show but ignore the Gorrilla in the room. The Executive arm of Government continues to barrel down the path to an election without any apparent appreciation of the risk that exists, both to the simple fact of an election and to the essence of our Constitution. Why has the existence of this Constitutional challenge not been publicised more? Why are the media giving the politicians sound-bite time to puff their agendas ahead of an election being called but they don’t ask the relevant politicians why we find ourselves at a juncture where the Constitutionality of our Electoral system is being challenged due a disproportionality in representation?

    The chronic lack of leadership and accountability on the part of the Government Minister charged with monitoring and managing how our Electoral Register and our Electoral Processes operate is shocking. However at least it is consistent with his lack of leadership and lack of willingness to be accountable for anything other than a soundbite on the news (he was going to ‘bash some heads together’ over the Galway water crisis apparently).

    To tie this back to my theme of Information Quality Management, Deming called on management to adopt a “constancy of purpose” and to wholeheartedly take on “the new philosophy” while breaking down barriers between people/organisations and driving out the fear that prevents the delivery of quality.

    Why does the relevant Minister seem to act in a manner that could only serve to drive in fear and increase the barriers that might exist that would prevent a good job being done? What is our incumbent Government’s purpose that they are constant to? What is the philosophy that they are pursuing?

    I’m off to Paddy Powers to place a bet that the Election won’t be called this side of June. Congratulations to Catherine Murphy and Finian McGrath for taking a stand on this issue.

  • An idea pinched from Tuppenceworth

    Tuppenceworth have started doing ‘Tag Clouds’ to visualise the frequency of words etc in political manifestos and speeches in the run up to the Irish General Election.

    Ho-hum I thought… wouldn’t it be interesting to do the same on the emerging commercial Information Quality Blogs (and perhaps other commentary in the area – it works on any text) to see what sort of themes emerge in the tags (ie commonly occuring words). The bigger the font the more frequent the occurence. I’m making no inferences based on the ‘cloud’ – I’ll leave that up to you. This one is based on a post by Garry Moroney of Informatica on their blog site.

    created at TagCrowd.com

  • The Tuppenceworth Paper Round (a slight return)

    Oh good grief, Hendrix is probably spinning in his grave…

    Simon over at Tuppenceworth has announced he’s been invited to take part in Leviathan this Thursday. This is a nice ‘attaboy’ for the gang at Tuppenceworth from David McWilliams et al. Or else it is a trap given the number of journos who’ll be in the room.

    Just in case it is a trap, I thought I’d trundle out the Journo Code of Practice – the Code of Conduct of the NUJ – for Simon to recite in the manner of Indiana Jones in “Indy and James Bond save the world“ or whatever that movie was called.

    The full code of conduct can be found HERE. For the purposes of this post, I’ll only focus on the salient points…

    From the Code of Conduct:

    2.) A journalist shall at all times defend the principle of the freedom of the press and other media in relation to the collection of information and the expression of comment and criticism. He/she shall strive to eliminate distortion, news suppression and censorship.

    The reaction of some journalists to the Paper Round (ie ‘whatwouldbloggersknow’) is contrary to the spirit if not the letter of this section of the NUJ code of conduct. The Bloggers in the Paper Round were operating in another medium (not print or traditional broadcast) but they were open about how they collected their information and presented fact-based criticisms based on what they found, as well as inviting comment and a right to reply. Comments on the professionalism of bloggers vs that of professional journalists missed the point entirely – a better response more in keeping with the Code of Conduct might have been to get involved, help refine the methodology and publicise the process.

    The fact that much of what the Paper Round found given the reliance of some newspapers on advertorial or reguritated press releases could viewed as “distortion” or “news suppression” is worthy of mention. However, point number 9 of the Code of Conduct smacks one more blatantly between the eyes, given the prevalance of advertorial:

    9.) A journalist shall not lend himself/herself to the distortion or suppression of the truth because of advertising or other considerations.

    Given the findings of the Paper Round in relation to Opinion pieces masquerading as ‘real news’, imagine my surprise when I found the following in the NUJ Code of Conduct:

    3.) A journalist shall strive to ensure that the information he/she disseminates is fair and accurate, avoid the expression of comment and conjecture as established fact and falsification by distortion, selection or misrepresentation.

    Based on my reading of the Code of Conduct (and I am just an ignorant blogger), the Tuppenceworth Paper Round raise some interesting questions about the state of Irish journalism and print media in particular. I am heartened that it would seem that the Tuppenceworth approach is actually in alignment with the spirit of the NUJ code of conduct in that they published facts and provided a right of reply.

    Those in print media who interpreted the Paper Round as an attack on journalists by bloggers missed the point. Those who attacked bloggers, or the paper round, in print may have acted in breach of their own code of conduct.

    More worryingly for me is the question of, if journalists and print media aren’t producing a ‘product’ for the public (the customer) or one that conforms to their own Union’s code of conduct just who is the piper calling the tune?

    Perhaps in 2007 the NUJ might collaborate with Tupp’worth to devise a more structured methodology for measuring standards in Irish journalism and the quality of what is actually printed in Irish print media against both the expectation of the consumer (Seamus Q Newspaperreader) and the NUJ Code of Conduct.

    In the meantime, I hope that Simon over at Tupp’worth makes use of the Code of Conduct if he is ambushed. 

    And as for the DOB-log… from January I’ll be adopting the NUJ Code of Conduct for all posts and comments on this blog. If it’s good enough for the journos, it’s good enough for a humble blogger like me.

  • Customer focus in the mee-ja

    The PaperRound over at Tuppenceworth has stirred up a hornet’s nest of phlegm and brimstone from at least two sources – the Indo article by Niall Byrne that I mentioned previously and a mysterious comment from ‘Soontobe’ on the Tuppenceworth Blog.

    The team over at Tupp’worth may take issue with my view that the Paper Round study was a valiant first attempt at to measure how closely the Irish newspaper industry meets the expectation that papers contain news and as such is a form of Information Quality Audit of the Irish print media. However, if we consider the Paper round in that light, the responses, particularly the comment from ‘soontobe’ show a significant disconnect in the mindset of Irish print journalism from core principles of quality.

    The basic gist of the responses seems to be along the lines of ‘how dare bloggers criticise journos because bloggers don’t know anything, don’t have real lives and aren’t skilled enough to write for print media‘. We’ll ignore the fact that I know a of number of bloggers who write or have written for print media – Karlin Lillington anyone?

    I will, however, pay attention to what this type of response means.

    1. A study was done by a set of information consumers who had, I assume, paid for their newspapers and hadn’t stolen them and were therefore customers of the Irish print media houses.
    2. This study (admittedly unscientific in its rigour but better than nothing) showed a very mixed bag of results across the papers examined.
    3. These findings were published (along with the methodology)

    In a Quality management context, what has happened here is that a group of customers have identified that their expectation of the product purchased is not being met and have produced some data to support their opinion.

    Quality management advises (or rather mandates) that the focus of all processes and quality measures should be the consumer of your goods or information. Leading companies such as Toyota take the view that a customer complaint is an opportunity to improve their product. Where a customer or group of customers comes to them with actual DATA to backup the frequency or volume of defect, it is the equivalent of Christmas in a Quality manager’s world.

    So, where sit the journos or the ‘journo-aligned’? Have they said “gee, thanks for doing this, now we have something to throw back at our editor when they insist on doing a half-page black and white piece on Claire Byrne’s orange dress”? ehhh…no. Have they said “good grief, what has happened to us? When we were in journo-school we wanted to be Woodward or Bernstein, or at least Robert Redford”? ehhhh…. no.

    What has happened is a response that, to return to an automotive analogy, would be like General Motors telling you to f*ck off and stop bothering them about your dodgy gear box because you knew f*ck all about building cars or running a big company (with optional comments about your level of gratitude and parts of the male anatomy).

    Which is not too dissimilar what GM used to do until the 1980s when they woke up one morning to find they had lost more money in a year than they had made for most of the previous decade. They don’t do that anymore, but have still had their arse kicked by Toyota and other Asian Tiger auto companies.

    Quality is about meeting or exceeding your customer’s expectations. If the customers of Irish newspapers (or papers that are sold in Ireland with Irish-ish content) expect a bit of journalism as opposed to press-releases dressed up as reportage or opinion then the paper round has shown that we are a long way from quality in many cases.

    Unless of course the people who buy the newspapers aren’t the customer. By “buy the newspaper” I mean, of course, the individual buying one copy of the paper. But what if some people have confused that with people who buy the newspaper?

    If you are a consumer of a product that is increasingly failing to meet your expectations because a more powerful group is exerting influence to have their expectations met, then you will switch product or supplier. In the auto industry this happened when cash-strapped students spurnned the gas-guzzling, sometimes patchy quality cars produced by Detroit and opted for the lower cost, more reliable and more fuel efficient Japanese imports in the 1970s. The oil crisis of the 1970s accelerated that trend. By the 1980s, many of these students were trading up and simply bought the newer model from Toyota or Nissan because they knew it met their expectations of quality and cost-effectiveness better than a Chevy or an Oldsmobile.

    In the Information Age, those of us who seek an alternative to the print media sources will increasingly look to the Internet, where peer comment and review and a wide array of varying opinion allow Citizen Blogger to make up their minds. Credibility and status will come not from the weight of a backer’s bank account but from the how consistently the information provider meets or exceeds their reader’s expecations in terms of incisiveness of comment, depth of analysis and the ability to take a story and peer behind the press release to question what is actually happening.

    The fact that this is currently being done mainly by hobbyists is irrelevant. Increasingly organisations are looking to blogs and wikis as ways of improving interaction with their customers. It is inevitable that eventually people will be paid to perform blogalism, either through a corporate entity or through advertising on their sites that pays them more the more people visit. At which point…other than the medium what is the difference between a blogger and a ‘traditional’ journalist?

    A specialist blogger in a niche area who provides reliable, well written, well researched pieces giving a different angle on topical issues will get hits and will become part of a network of ‘go-to’ people for opinion. We can see this happening already with extremely co-incidental similarities between blog posts and pieces printed in some newspapers which I’ve posted about here a while back). But how does this differ from a specialist investigative reporter?

    Conclusion

    If the critical comments posted on Tuppenceworth and elsewhere are indicative of the response by one or more ‘print meeja’ people to the Paper Round then the industry is in a much worse state then the survey shows.

    Either there is a fundamental disconnect between the journalists view of their work or role and the expectation of the newspaper reader, or the reactions are suggestive of the existence of a more powerful ‘customer’ group who have more highly prioritised expectations of the content and editorial policies of our media. If the former is the case, then there may be some hope, as it may be that the Paper Round pricks the slumbering customer-focus of the tired and cynical hacks and prompts some push back on advertorialising and press-release reporting.

    If the latter is the case then the the role and mandate of those pioneering ‘blogalism’ will become increasingly important as Information Consumers seek out sources of news and information that more closely match their expectations of reporting.

    The fact that, despite our growing population, a report at the 57th World Newspaper Congress in 2004 showed a decline in newspaper circulation in Ireland of 7.8% would suggest that there is a shift taking place (source : Wikipedia, accessed 19:04 UTC, 28 Nov 2006)

    When Johann Carolus printed the first newspaper in 1605, chances are the towncriers of the day dismissed him as a hobbyist who had no place disseminating news.

     

    Update—

    Just read over one of the later PaperRound posts on Tupp’worth.. some interesting points made. http://www.tuppenceworth.ie/blog/index.php/2006/11/24/sunday-independent-12th-nov-2006/

  • Tuppenceworth…

    Simon over at Tuppenceworth is getting a bee in his bonnet about the standard of Irish journalism. I have to agree. I am a Director of publicity for an international association for Information Quality professionals. Over the past year I have submitted a number of commentaries on issues such as the Electoral Register.
    Not ONCE has there been a journalist who has contacted me back on any topic, not even to say thanks but no thanks. It seems to be easier to trot out the easy soundbite than to actually research a topic (such as the Electoral Register issue – despite what Dick Roche says it is still an unmitigated disaster area and will NOT be clean come election because the fundamental root causes have not been addressed) and be in a position to ask hard questions.

    For example – with Electronic voting, journalists have swallowed the line that the e-voting systems have been fully tested because that was the response to a Parlimentary Question (dail reportage). Of course, the question was badly put… if didn’t ask how many machines actually passed the tests… (answer is approximately none of them).

    Furthermore, no newspaper I’ve seen in the last month has picked up on the link between the State Claims Agency report on Injuries arising from Treatment errors in the Irish Healthcare system and the number of articles that appeared in Oct and Nov of this year about people who’d had horrendous harm inflicted on them by unnecessary surgery because their patient information had been mixed up with someone elses… I’ve got an article on that drafted that links to the situation in the US… any takers?

    I look forward to a day when I can pick up a respected newspaper and not be accosted by obvious press-release fodder, commercial features or limp-wristed reporting. Hopefully some journalist will pick up on the story at Tuppenceworth and run with it…

    …now wouldn’t that be ironic?

    Update….. Indo picked up on the Tuppenceworth story but not in the way you’d like. Apparently if you don’t like the message and can’t directly attack the messenger you should attack the credibility of the medium the message travels in. Which is ironic given that it is the credibility of print journalism that the Tuppenceworth Paper Round calls into such stark question.

  • Propogation of information errors and the risks of using surrogate sources

    ….ye wha’?

    There has been a lot written in relation to the electoral register and other matters about using information from other sources to improve the quality of information that you have or to create a new set of information.

    This makes sense, other people may already have done much of the work for you and, effectively, all you need to do is to copy their work and edit it to meet your needs. In most cases it may be faster and cheaper to use such ‘surrogates’ for reality to meet your information needs than to go to the effort of going to the real-world things (people, stock-rooms where ever) and actually starting from scratch to build exactly the information you need in the format you require to exactly your standards and formats.

    There is, however, a price to pay for having such surrogate sources available to you. You need to accept that

    1. The format and structure of the information may need to be changed to fit your systems or processes
    2. The information you are using may itself be innaccurate, incomplete or inconsistent.
    3. If you are combining it with other information, it will require investment in tools and skills to properly match and consolidate your information into a valid version of the truth.

    These risks apply to organisations buying marketing lists to integrate with their CRM systems but also could be applied to students relying on the Internet to present them with the content for their academic projects or journalists trawling for content for newspaper articles or reviews.

    Recurrence of common errors, phrases or inaccuracies in term papers is one way that academia has of identifying academic fraud. Similar techniques might be applied in other arenas to identify and track instances of copyright infringement.

    In businesses dealing with thousands of records, the cost/risk analysis is relatively straightforward. The recommendation I would make is that clear processes to manage suppliers and to measure the quality of the information they provide you based on a defined standard for completeness, consistency, duplication, conformity etc. is essential. Random sampling of surrogate data sources for accuracy (not every 100th record but a truly random sample) is also strongly recommended.

    These are EXACTLY the same techniques that manufacturing industries use to ensure the quality of the raw material inputs to their processes. If it works for industries where low quality can kill (such as pharmaceuticals), why shouldn’t work for you?

    For students, journalists and those of us hacking away in the blogosphere the recommendation is simple. Only rely on surrogate sources if you absolutely have to. If you use someone elses work as your source, credit them. If you don’t want to credit them then make sure you verify the accuracy of their work either by actually verifying against reality or by checking with at least one other source.

    That way you avoid having the errors of your source become your errors also and you don’t run the risk of someone crying foul and either suing you for stealing their copyright (and copyright does apply to content posted on the internet and in blogs) or taking whatever other sanctions might apply (such as kicking you off your college course).

    In many cases the costs and effort involved in double checking (particularly for a once of piece of writing) are neglibily different to the costs of actually starting from scratch and building your information up yourself. And, depending on the context, it may even be more enjoyable.

    The New York Times not so long ago had to relearn the lessons of checking stories with at least one other source for accuracy.

    Horatio Caine in CSI:Miami always tells his team to “trust, but verify”.

    When using surrogate sources for real-world information in any arena you must assess the risk of doing so and put in place the necessary controls so that you can trust that you have verified.

    (c) Daragh O Brien 2006 (just in case)

  • Conflict of Expectations

    A few weeks back, I was at dinner with Larry English and some people from a large consulting firm. It was a social occasion and as the food and informality took hold, the discussion turned to the pillars of the Total Information Quality Management methodology and ethical/moral concerns.

    Mr English draws heavily on Deming and the TQM gurus in his view of quality – quality is defined as meeting or execeeding customer expectations and the most important person in the ‘production line’ is the customer. The challenge to this was what would one do if your customer wanted something illegal or immoral to meet their expectation or if, in order to meet or exceed their expectation, you had to do something illegal or immoral.

    My response to this argument was to point out that the expectation of society as to the acceptable standards of conduct are expressed in law, both statute and judicial. Just because a product or service meets or exceeds the expectations of a targetted customer it does not mean that it can be produced if the laws of society would preclude it. A good example of this is the location tracking tools that were being deployed by a Dublin security company. The prompt action of Digital Rights Ireland in alerting the Data Protection Commissioner resulted in the expectation of society of privacy (albeit curtailed somewhat for children) was asserted and the product had to be withdrawn.

    Where individuals or organisations choose to offer services that are illegal or immoral, and where those services are their business, then it can be argued that their obligation to their customer is the same as a business operating within the law. If a drug dealer (for example) provides low quality product then they will not survive long. Either in business or life.

    The choice is whether to operate within the law or not, not whether you need to produce a quality product that meets or exceeds your customer expectations. If you choose to operate within the law, then the legal process will operate (albeit at times slowly) to assert the expectation of Society as a customer. If you choose to act outside the law, producing or selling a product that is illegal, then your obligation to your chosen customer remains the same.

    Â