The DOBlog

Category: Uncategorized

  • Arise DeskZilla

    I use a standing desk when working in my office (and if I could find a light weight portable option I’d use one on client sites as well). Many of the greatest leaders have used standing desks.

    There are proven medical benefits to getting off your backside when working. It’s worth bearing in mind that sitting for a living is an invention of the late 19th and early 20th century. Prior to that most people did have to move around a lot. But standing desks can be expensive. So a theme has developed over the past few years of hacking functional standing desks that are ergonomically aligned using a low capital investment model (for which we must read “it don’t cost much if you make a mess of it”). The source of raw materials is a certain Swedish home improvements store famed for their meatballs that I won’t name here because they are very protective of their brand name. But a good source of ideas for how to repurpose their stuff can be found here.

    About 18 months ago, after a flare up of back trouble, I did a bit of research (using the hacks site linked to above and a few others) to see how I might best build a standing desk on a near-zero budget. I started with a few basic design principles:

    1. Aim for “minimum viable product” – it had to meet ergonomic requirements for me and my height, but I guessed it how I worked, laid out my work, and how the desk would need to function would evolve as I changed from sitting on my ass to moving around.
    2. Reuse or recycle things I already had – I had a desk already. I wasn’t going to junk that. I also had a pretty cool laptop stand with cooling fan and USB ports.
    3. Kaizen principles – I’d look to find ways to reduce waste of effort and time when working, and accept that the desk would not be perfect as I’d always find something else to improve how it works for me and with me.
    4. MacGyver rocks.

    Ergonomics

    Some basics. If you don’t have your standing desk set up correctly you will simply make things worse for yourself. Do some research. Buy a measuring tape. Think about posture, stance and positioning. I train (as often as I can, which isn’t often enough) in Aikido so I am very concious of my centre point (hara) and the need to have hips and back aligned correctly for good movement and energy flow.

    Some good resources for standing desk ergonomics I found during my research are here, here, and here. A recent resource that covers off some good “dos and don’ts” can be found here.

    Introducing DeskZilla

    DeskZilla was the result of my research and my design principles. It was built entirely from parts purchased from Ikea (oops I’ve named them), with a few extra bits thrown in to make minor adjustments.

    Picture of deskzilla standing desk
    First iteration of DeskZilla.

    The parts I used were:

    1. A Vika/Amon desktop (no longer available). It is 100cm wide and 50 cm deep. For alternative table tops, see here: Ikea TableTops
    2. An Ekby Jarpen shelf for the monitor and laptop level, with three Ekby Tore clamp brackets (3 ensures shelf doesn’t bow in the middle). Ikea actually illustrate the use of the brackets on a desktop on their website now.
    3. Capita legs for the desk (which require a little MacGyvering with a drill to make some new screw holes for them as they are not meant as desk legs). I went for these as they could be adjusted up to 17cm high. Note that the Capita legs aren’t MASSIVELY extendy, they adjustable to compensate for uneven floors in the furniture they are supposed to be used on. But a centimetre or two can make all the difference.
    4. Two power blocks from Aldi that bolted onto the desk. I put them on the rear edge to stop DeskZilla from sliding backwards.

    Total cost, a little over €70.

    A key point… it is really important to measure your existing desk and the height/depth of each component to make sure things are going to be at the right height.

    What I have with Deskzilla is a modular system where I can move the monitor and laptop down on to the lower level and move the keyboard and mouse down to the lower desk and use it as a sitting desk. The monitor is almost exactly perfectly positioned for a sitting desk when on the first level.

    I had to add a pencil box under the keyboard to move it up a centimetre and a half or so for better ergonomics when typing. The monitor is now raised up on a hardback books to improve positioning (more on that in a moment).

    Evolution, Phase 1

    Almost immediately DeskZilla began to evolve. While the monitor was almost perfectly aligned, I found that video conferencing was a great way to double check.

    Rule of thumb: if you have a webcam in your monitor your eyes should be in line with the lens. A hardback book fixed that.

    After a few weeks of use I noticed I was getting stiffness. Some gym mats from Argos on the floor provides an anti-fatigue feature, and I still have my chair and can switch to sitting if I get too stiff and sore any day. The body is a bugger and some days you can stand without issue for hours (I pulled a 27 hour straight working day on a project last year… standing almost the entire time) and other days it hurts like heck after a few hours.

    Second rule of thumb: listen to your body and adapt each day.

    Evolution Phase 2

    DeskZilla will evolve again soon. Experience with the monitor, and the hassle of bending to get pens, post-it notes etc,  tells me that it might make sense to swap the Ekby shelf for one with drawers that has the same length and a bit more height. The Ekby Alex shelf looks like a contender. The only reason I rejected it in Phase 1 was cost – it would have been over 50% additional on the budget.

    I also need to think about raising the desktop a little to remove the need for a pencil case under the keyboard. That could be achieved through castor guards or something like that (the things that you put on furniture that is going on a wooden floor), another option is some half-inch wooden blocks  between the Ekby desktop and the Capita legs to give a small height boost. That last option would be a good call for anyone over 6ft 2″ who wanted to use this recipe, and could be a way to incrementally tweak the height to what you need rather than relying on just the leg extendibility.

    Finally, I’ll probably invest in a folding bar stool type chair, or an ironing board chair to use when fatigue kicks in to take the weight off my ankles and knees.

    Some key lessons about standing while working

    1. Think zen and do yoga. Simple stretching movements keeps fatigue at bay and helps strengthen core.
    2. Don’t stand still… move around and shift posture.
    3. Get used to working in shorter bursts and then changing position. I used to sit motionless for hours, now I work in 10 to 15 minute bursts and then switch posture or position… any longer and I stiffen up, which can hurt and break concentration any way. Movement keeps the brain awake!
    4. Two monitors makes a massive difference, but only if you aren’t having to crane your neck to see it.
    5. Your workspace will evolve around you. Find a natural movement and flow for you and settle into it. If you force it you’ll find it just doesn’t click for you.
    6. Breathe. Take advantage of your posture and position to take deep breaths and relax into your work.
    7. Each day you will need to improvise something to tweak a factor to improve comfort and flow. Accept that and get on with it
    8. The Desk is NEVER finished if you are building your own, (and it’s never perfect if you bought it off the shelf)

  • Serendipity

    So, within hours of me blogging about data protection consent issues in the Facebook mood manipulation study, the Register has the EXCLUSIVE that Facebook is being investigated by the irish DPC with specific questions around the consent relied upon. http://www.theregister.co.uk/2014/07/01/uk_and_irish_data_watchdogs_wade_in_on_facebook_messin_with_your_head_scandal/

    I’m not saying anyone in an office above a Centra in Portarlington reads this blog but it is a serendipitous co-incidence.

    And it may turn out that manipulating user timelines to provoke emotional responses could make Facebook management very sad.

  • Facebook Research, Timeline Manipulation, & EU Data Protection Law

    This is an initial post based on the information I have to hand today (1st July 2014). I’ve written it because I’ve had a number of queries this morning about the Data Protection implications of Facebook’s research activity. I’m writing it here and not on my company’s website because it is a work in progress and is my personal view. I may be wrong on some or all of these questions.

    Question 1: Can (or should) the Data Protection Commissioner in Ireland get involved?

    Facebook operates worldwide. However, for Facebook users outside the US and Canada, the Data Controller is Facebook Ireland, based in Dublin. Therefore EU Data Protection laws, in the form of the Irish Data Protection Acts 1988 and 2003 applies to the processing of personal data by Facebook. As a result, the Irish Data Protection Commissioner is the relevant regulator for all Facebook users outside the US and Canada. The key question then is whether or not Facebook constrained their research population to data subjects (users) within the US and Canada.

    • If yes, then this is not a matter for investigation by EU data protection authorities (i.e. the Data Protection Commissioner).
    • If no, then the Irish Data Protection Commissioner and EU Data Protection laws come into play.

    If Facebook didn’t constrain their population set, it is therefore possible for Facebook users outside of the US and Canada to make a complaint to the DPC about the processing and to have it investigated. However, the DPC does not have to wait for a complaint. Section 10 of the Data Protection Acts empowers the Commissioner to undertake “such investigations as he or she considers appropriate” to ensure compliance with legislation and to “identify any contravention” of the Data Protection Acts 1988 and 2003.

    [update] So, it is clear that the data was obtained from a random sample of facebook users. Which raises the question of the sampling method used – was it stratified random sampling (randomised within a sub-set of the total user base) or random sampling across the entire user base? If the former then the data might have been constrained. If the latter, the data inevitably will contain data subjects from outside the US/Canada region. [/update]

    Answer: If Facebook hasn’t constrained their population to just North America (US/Canada) then… Yes.

    Question 2: If Irish/EU Data Protection Law applies, has Facebook done anything wrong?

    Tricky question, and I wouldn’t want to prejudge any possible investigation by the Data Protection Commissioner (assuming the answer to Question 1 would get them involved).  However, based on the information that is available a number of potential issues arise, most of them centred on the question of consent. Consent is a tricky issue in academic research, market research, or clinical research. The study which was conducted related to the psychological state of data subjects. That is categorised as “Sensitive Personal Data” under the Data Protection Acts. As such, the processing of that data requires explicit consent under Section 2B of the Acts. Beyond the scope of the Data Protection Acts, clinical research is governed by ethical standards such as the Nuremburg Code which also requires a focus on voluntary and informed consent:

    The voluntary consent of the human subject is absolutely essential… and should have sufficient knowledge and comprehension of the elements of the subject matter involved as to enable him to make an understanding and enlightened decision. This latter element requires that before the acceptance of an affirmative decision by the experimental subject there should be made known to him the nature, duration, and purpose of the experiment

    Question 2A: Was Consent Required? Consent is required for processing of sensitive personal data. For that data to be sensitive personal data it needs to be data that is identifiable to an individual and is sensitive in nature. However, if the data being processed was anonymised or pseudonymised then it falls outside the scope of personal data, assuming appropriate controls are in place to prevent re-identification. The Irish Data Protection Commissioner has published guidance in 2007 on Clinical Research in the Healthcare sector which provides some guidance on the question of consent, albeit from the perspective of a pure clinical healthcare perspective. A key point in the guidance is that while anonymising data may remove the Data Protection question around consent, it doesn’t preclude the ethical questions around conducting research using patient data. These kind of questions are the domain of Ethics Committees in Universities or commercial research organisations. Research of this kind are governed by Institutional Review Boards (IRB) (aka Ethics Committees).

    Apparently Cornell University took the view that, as their researchers were not actually looking at the original raw data and were basing their analysis of results produced by the Facebook Data Science team they were not conducting human research and as such the question of whether consent was required for the research wasn’t considered. The specifics of the US rules and regulations on research ethics are too detailed for me to go into here. There is a great post on the topic here which concludes that, in a given set of circumstances, it is possible that an IRB might have been able to approve the research as it was conducted given that Facebook manipulates timelines and algorithms all the time. However, the article concludes that some level of information about the research, over and above the blanket “research” term contained in Facebook’s Data Use policy would likely have been required (but not to the level of biasing the study by putting all cards on the table), and it would have been preferable if the subjects had received a debrief from Facebook rather than the entire user population wondering if it was them who had been manipulated. Interestingly, the authors of the paper point to Facebook’s Data Use Policy as the basis of their “informed consent” for this study:

    As such, it was consistent with Facebook’s Data Use Policy, to which all users agree prior to creating an account on Facebook, constituting informed consent for this research.

    Answer: This is a tricky one. For the analysis of aggregate data no consent is required under DP laws and, it appears, it raises no ethical issues. However, the fact that the researchers felt they needed to clarify that they had consent under Facebook’s Data Use policy to conduct the data gathering experiments suggests that they felt they needed to have consent for the specific experimentation they were undertaking, notwithstanding that they might have been able to clear ethical hurdles over the use of the data once it had been obtained legally.

    Question 2b: If consent exists, is it valid? The only problem with the assertion by the researchers that the research was governed by Facebook’s Data Use policy is that, at the time of the study (January 2012) there was no such specified purpose in Facebook’s Data use policy. This has been highlighted by Forbes writer Kashmir Hill.

    The text covering research purposes was added in May 2012. It may well have been a proposed change that was working its way through internal reviews within Facebook, but it is impossible for someone to give informed consent for a purpose about which they have not been informed. Therefore, if Facebook are relying on a term in their Data Use Policy which hadn’t been introduced at the time of the study, then there is no valid consent in place, even if we can assume that implied consent would be sufficient for the purposes of conducting psychological research. If we enter into a degree of speculation and assume that, through some wibbly-wobbly timey-wimey construct (or Kashmir Hill having made an unlikely error in her analysis), there was a single word in the Data Use Policy for Facebook that permitted “research”, is that sufficient?

    For consent to be valid it must be specific, informed, unambiguous, and freely given. I would argue that “research” is too broad a term and could be interpreted as meaning just internal research about service functionality and operations, particularly in the context in which it appears in the Facebook Data Use Policy where it is lumped in as part of “internal operations”. Is publishing psychological and sociological research part of Facebook’s “internal operations”? Is it part of Facebook’s “internal operations” to try to make their users sad? Interestingly, a review of the Irish Data Protection Commissioner’s Audit of Facebook in 2012 reveals no mention of “Research” as a stated purpose for Facebook to be processing personal data. There is a lot of information about how the Facebook Ireland User Operations team process data such as help-desk queries etc. But there is nothing about conducting psychometric analysis of users through manipulation of their timelines. Perhaps the question was not asked by the DPC?

    So, it could be argued by a Data Protection regulator (or an aggrieved research subject) that the consent was insufficiently specific or unambiguous to be valid. And, lest we forget it, processing of data relating to Sensitive personal data such as psychological health, philosophical opinions etc. requires explicit consent under EU law. The direct manipulation of a data subject’s news feed to test if it made them happier or sadder or had no effect might therefore require a higher level of disclosure and a more positive and direct confirmation/affirmation of consent other than “they read the document and used the service”. There are other reasons people would use Facebook other than to be residents of a petri dish.

    Does this type of research differ from A/B testing in user interface design or copywriting? Arguably no, as it is a tweak to a thing to see if people respond differently. However A/B testing isn’t looking for a profound correlation over a long term between changes to content and how a person feels. A/B testing is simply asking, at a point in time, whether someone liked presentation A of content versus presentation B. It is more functionally driven market research than psychological or sociological analysis.

    Answer: I’d have to come down on the negative here. If consent to the processing of personal data in the manner described was required, it is difficult for me to see how it could be validly given, particularly as the requirement is for EXPLICIT consent. On one hand it appears that the magic words being relied up on by the researchers didn’t exist at the time of the research being conducted. Therefore there can be no consent. Assuming some form of fudged retroactivity of consents given to cover processing in the past, it is still difficult to see how “research” for “internal operations” purposes meets the requirement  of explicit consent necessary for psychological research of this kind. It differs to user experience testing which is more “market research” than psychological and therefore is arguably subject to a higher standard.

    Question 3: Could it have been done differently to avoid Data Protection Risks

    Short answer: yes. A number of things could have been done differently.

    1. Notification of inclusion in a research study to assess user behaviours, with an option to opt-out, would have provided clarity on consent.
    2. Analysis of anonymised data sets without directed manipulation of specific users timelines would not have raised any DP issues.
    3. Ensure validity of consent. Make sure the text includes references to academic research activities and the potential psychological analysis of user responses to changes in Facebook environment. Such text should be clearly highlighted and, ideally, the consent to that element should be by a positive act to either opt-in (preferred) or to opt-out
    4. Anonymise data sets during study.
    5. Restrict population for study to US/Canada only – removes EU Data Protection issues entirely (but is potentially a cynical move).

    Long Answer: It will depend on whether there is any specific finding by a Data Protection Authority against Facebook on this. It does, however, highlight the importance of considering Data Protection compliance concerns as well as ethical issues when designing studies, particularly in the context of Big Data. There have been comparisons between this kind of study and other sociological research such as researchers walking up to random test subjects and asking them to make a decision subject to a particular test condition. Such comparisons have merit, but only if we break them down to assess what is happening. With those studies there is a test subject who is anonymous, about whom data is recorded for research purposes, often in response to a manipulated stimulus to create a test condition. The volume of test subjects will be low. The potential impact will be low. And the opportunity to decline to participate exists (the test subject can walk on by… as I often did when faced with undergrad psychology students in University) With “Big Data” research, the subject is not anonymous, even if they can be anonymised. The volume of test subjects is high. Significantly (particularly in this case) there is no opportunity to decline to participate. By being a participant in the petri-dish system you are part of the experiment without your knowledge. I could choose to go to the University coffee shop without choosing to be surveyed and prodded by trainee brain monkeys. I appear to have no such choice with Data Scientists. The longer answer is that a proper consideration of the ethics and legal positioning of this kind of research is important.

  • Examples of poor Data Protection Practice in Public Sector

    Earlier this week the Data Protection Commissioner bemoaned the lack of attention to detail and the poor culture of Data Protection compliance practices in the Irish Public Service.

    He was right to do so. My experience as both a service user and as a consultant has been that there are excellent people making excellent efforts to swim against a tide of indifference and short-cutting that results in breaches of legislation and puts personal data of citizens at risk.

    In a “brain fart” moment yesterday I googled the words “Election”, “Training” and “Ireland” by accident. It brought back a website called ElectionTrainingIreland.ie. This website announces itself to be the “Official Presiding Officer Online Training “. Apparently Presiding Officers in this year’s Local and European Elections are required to complete this training, which I understand consists of a series of videos. It’s actually a rather good idea.

    However it has been badly implemented from a Data Protection perspective.

    1. It requires a PPS Number to login. This is not a permitted use of the PPS Number. For a start, ElectionTrainingIreland is not registered as a Registered User of the PPSN under the 2005 Social Welfare Consolidation Act.
    2. Using PPS Numbers as a login is not good information security practice.
    3. As I understand it, Presiding Officers receive a letter that contains their PPS Number and a password for this site – which suggests that passwords are stored somewhere in an unencrypted freetext format (again BAD Information Security practice)
    4. There is no information about who Election Training Ireland are. They are NOT an official state body or division of the Department of the Environment. There is no privacy statement on the website that explains the purposes of processing of data, the retention of data, or (for that matter) where Election Training Ireland got the PPSN that they are using in the background to verify your identity.
    5. The website, which asks you to key in your PPS Number, does not have a valid SSL certificate. There is no encrypted transfer of information. Given the value of the PPS Number, that’s simply not good enough from a Data Protection point of view.

    Looking at the process from the outside, armed only with nearly two decades of experience in designing and reviewing information management processes for regulatory compliance, I suspect that this might be the underlying process:

    1. A list of all people who registered to be Presiding officers was provided to Election Training Ireland. This list included PPS Numbers, names, and home addresses. [Issue #1 below]
    2. This list was used to create a database of Presiding Officers which in turn was used to create a list of user logins for the website. These user logins used PPSN as the user id [issue #2 below]
    3. This list was used to generate a mailmerge to each Presiding Officer at the address provided by them for contact (which is almost inevitably a home address) which contained their password [Issue #3 below]
    4. The website is not encrypted. [Issue #4 below]
    5. This list was provided to and processed by Election Training Ireland, who are an external contractor working (one assumes) for the Department of the Environment [See: “Who are ETI?” below]

    Issue #1: Transfer of data about candidate Presiding Officers

    Data constituting a significant portion of what is defined in the 2005 Social Welfare Consolidation Act as the “Public Service Identity” has been transferred to a 3rd party by local authorities and/or the Dept of Environment. What was the lawful basis for this transfer? Was there a statutory basis (which is the DPC’s favoured basis for data transfers in the Public Sector)? What was the protocols regarding security of transfer, retention, security of storage, restrictions on use etc? Is there a Data Processor contract in place (if there is it will be a doozy IMHO because of questions under “Who is ETI” below)?

    As ETI is not registered as a User of the PPSN with the Department of Social Protection, issues potentially arise with the legality of transfer here. And even assuming that ETI has a valid contract etc. with either EVERY local authority or the Dept of Environment, the PPS numbers would have been obtained originally from Presiding Officers for the purposes of processing their payments and making appropriate deductions for taxation and PRSI etc. Not for using them as a unique identifier in a system hosted by a 3rd party.

    Issue #2: Creation of lists and user logins

    As mentioned above, the creation of a central database of presiding officers and the use of their PPS Number as an identifier in that database constitutes a new purpose within the context of the Data Protection Acts. Using PPS Number as a login is just dumb (a proxy could easily have been created). This database has PPS Numbers, names, and addresses of Presiding Officers. Where is it being stored? Under what security protocols? Who has access to it? How long will it be retained for? (Please don’t let them have saved it to Google Docs, otherwise I’ll have to get cross).

    Issue #3 Mail merge and posting out passwords

    Passwords are stored in plaintext if they could be mailed out in a mail merge. Being able to do a mail merge means that who ever sent the letters to Presiding Officers has their PPS Number, name, and addresss. That’s a heck of a lot of personal data. And if they are not thinking of the implications of storing passwords in an encrypted for and not sending them out in unsecured plain text, what’s the level of confidence in back-end security and security of related data transfers?

    Issue #4 No SSL on the site

    Using PPSN as a login is not great. Doing it in a way that could result in the data being intercepted by anyone minded to do so compounds the issue. Some might say it’s overkill for “just a login”, but the PPS Number is a significant identifier for people using public services in Ireland.

    Who are ETI?

    The site is not owned or operated by any Local Authority or Government Department. It is owned and operated by a partnership of three people based in Co. Meath. It took me 90 seconds to find that information on the CRO website, a basic due diligence test. If they are a partnership, each individual member of that partnership is a Data Processor acting on behalf of the Data Controller that engaged them (which might wind up being EVERY local authority or the Dept of the Environment – that is still unclear to me). There is nothing on the website identifying that the holder of the data doing this processing is not a government body.

    So a private sector organization has been given a chunk of the Public Service identifier for a defined population of people, has implemented a solution that is arguably illegal in its design and is certainly not good information security practice. There is a questionable lawful basis for the transfer of data to this 3rd party. (I haven’t looked for the tender for this training solution, I’m assuming it went to tender and there was some specification of information security and data protection standards in that document. But I’ve got a day job to do).

    What could be done better/differently?

    Lots of stuff.

    1. Use a proxy identifier. If the data controller holding the PPSN had created a proxy identifier (an alphanumeric string that was NOT the PPSN) and provided that to ETI to use as a login the PPSN issue would not arise.
    2. Ensure proper contracts in place with the data processor
    3. Use SSL by default.
    4. Use an encrypted (salted and hashed) password that could be generated from a link that a user could follow that would bring them to a page where they set their own password, rather than having a plaintext password sent in the post.
    5. Improve transparency and communication about who the actors are in the process and their role.

    That’ just my first four. Depending on the detail of the processes etc. there could be a LOT more. But none of them would cost a heck of lot of money and would result in a more compliant and less insecure processing of personal data.

  • The Strife of Reilly (Tone at the Top revisited)

    Scanning twitter over my post-breakfast intra-planning pre-work coffee this morning I noticed tweets that were agog at a Minister for Health who is a medical doctor asking non-medical doctor political colleagues for lists of people who should have been given a medical card. The agogness was triggered by this news story on the RTE website.

    Yes. It is a cause for agogness.

    However my gog was a’d by one line in the middle of that story that actually links into a story covered (briefly) by Irish media yesterday. Minister Reilly has also asked for a list of names of people who have given information to the Primary Care Reimbursement Service who have had their information “misplaced”.

    Only yesterday the Data Protection Commissioner was scathing in his comments about the level of “sloppiness” around the handling of personal and sensitive personal data in the Public Sector.

    Today, buried in a story that was likely sourced from the Office of the Minister for Health himself, we find a disclosure that sensitive personal data and potentially personal financial data have been “misplaced” by a unit of the HSE.

    However, the Minister is asking his colleagues for the names of people who might be affected. So that’s OK then.

    No. It’s not.

    If the PCRS has “misplaced” information that was provided to them in either electronic or hard copy form this constitutes a breach of Section 2(d) of the Data Protection Acts 1988 and 2003. Under the Voluntary Code of Practice for Data Security Breach Notification, the HSE is required to notify the Data Protection Commissioner where there is a risk that Personal Data, Sensitive Personal Data, or Personal Financial Data have been lost, or accessed or disclosed without authorization. The affected Data Subjects are supposed to be notified (unless it affects less than 100 people and doesn’t relate to Sensitive Personal Data or Personal Financial Data). The HSE, as Data Controller, is required to maintain a Data Breach Register for any reported incidents where the security of personal data has been put at risk. If the Minister is having to effectively do a ring around his mates in the Dáil to find out what the scale of the problem is, that should be a bit of a worry.

    So. Riddle me this…

    1. Why is the Minister asking for a list of names of people whose data has been “misplaced”?
    2. Why is he asking for this list if the HSE PCRS has been maintaining a Register of incidents of reported loss of data?
    3. Why has the Minister not referred the issue to the Data Protection Commissioner?

    The answer is, as ever, the Tone at the Top in the Public Service in Ireland. Unerringly it is a discordant “BLAAARRRRRRPPPPP” when it comes to matters of Data Protection. Organisation restructurings are undertaken without consideration for effective Data Governance, Information Quality, or Data Protection controls. Training in these things is seen as an overhead not an investment. Kludged manual processes are put in place without documentation and standardization (sure documentation takes AGES), and Ministers give undertakings to do things RIGHT NOW (immediately) rather than doing them RIGHT NOW (error proofed, proper controls, designed for efficiency, consistently executed).

    This problem is not confined to the Public Sector. However the Public Sector is, as Billy Hawkes has pointed out many times, the one actor that processes our personal data who can REQUIRE us to provide information by law and which requires us to provide information to avail of key Public Services and functions.

    “BLLLAAAAARRRRRPPPPP” is an insufficient response from the leadership.

  • Tone at the Top revisited

    I’ve written in the past about the problems with the “tone at the top” around Data Protection and Data Privacy in Irish government and political circles.

    Bluntly, with few exceptions, it seems they don’t get it, don’t like it, and would rather it go away. That attitude cascades down into government departments where it is almost (again with few exceptions) impossible to sack staff members who blatantly breach basic Data Privacy rights of individuals. Whether it is Social Welfare staff providing information to private detectives without authority or snooping on the personal details of lotto winners or celebrities, or formal political policies that presume a panopticon and damn the data privacy implications and risks.

    The icing on the cake for me was Minister for Justice not seeing the problem with disclosing sensitive personal data about a political opponent on national television. I’ve written about that last year.

    But the bum notes at the top are resonating higher and further. Our candidates for the European Parliament, the body which only recently has managed to push back against a wholesale dilution of data privacy rights of over 500 million people, are almost unanimous in their silence on the issue of Data Protection. Even those such as Sean Kelly who were active in European Parliament committees looking at the Draft Data Protection Regulation are silent on the issue. (And remember it was only last year that Sean Kelly won an award from the IAB Europe (an internet advertising representative body) for his work on the Data Protection Regulation, and only a few weeks ago that he was voted “MEP of the Year for the Digital Agenda”)

    Of a total of 37 candidates only THREE have signed up to support a Charter to oppose any measure “that removes the power to make decisions on matters that affect European citizen’s fundamental rights from the judiciary or democratically-elected policy-makers”.

    Three. Out of 37. Less than 10%. 8.1% to be precise. And every single one of them in Dublin.

    But the Charter is not perfect. I would have an issue with item 10 which calls for the promotion of Free/Open Source software in public sector environments if was to be interpreted as a “thou shalt not purchase commercial technologies that have warranties and approvals and proper liability if things go tits up”, but that’s not the interpretation I’d put on it – I read that as a “closed source is not the only fruit” and personally think that individuals pushing an ‘Open Source or bust’ approach to things are often missing the total cost of ownership (risk, compliance, learning curves, stability, longevity etc.). However, as @Treasa pointed out to me on Twitter, FOSS ideologies are often presented and pushed as dogmatic mantras which brook no potential use of closed source or proprietary technologies for any reason. She is right of course. That is a risk. And perhaps it is a barrier to candidates endorsing the Charter, in which case candidates should flag that they can do anything for love (or votes) but they won’t do that.

    (disclosure: my company makes minimal use of some Open Source technologies, but only where the platform is stable, reasonably standardized, and interoperable. We advise clients to consider total cost of operation and governance if putting Open Source in the mix on projects, based in part on personal experience of having to abandon tools that became unsupported or just unworkable)

    And the use of a list (albeit one which is non-exhaustive and not overly prescriptive) might be off putting. But I’m not sure what other mechanism might have been used for the effective presentation of the key points of the charter. Perhaps keeping it focused on privacy issues might have been enough?

    But overall the Charter is a positive statement of position and opinion on a range of fundamental information rights. It doesn’t mandate any other position other than to ensure that the Parliament have a democratic decision making role and that the Judiciary have an oversight function in the development of policy and legislation in this space.

    And only 8.1% of Irish candidates have expressed a preference in favour of their having a role in defending data privacy rights or ensuring oversight on the selection of suppliers to EU projects, and all of them representing Dublin.

    Our political classes don’t appear to care enough – either to sign the charter or explain why they won’t and what their position is on key principles.

    The “tone at the top” drones on it seems.

  • My letter to Brendan Howlin re: FOI Fees

    Below is an edited version of the letter I faxed to Brendan Howlin today regarding Freedom of Information Act fees proposed last Friday at the end of legislative drafting before the Committee Stage in the Irish Parliament.

    While I agree that public service resources need to be utilised efficiently, particularly in the current (apparently getting better) economic context, I disagree that putting a paywall up (which is the practical effect of the fees proposed) is the solution. Better results could be achieved by actually managing information as an asset and ensuring appropriate governance and joined up thinking.

    Dear Minister Howlin,

    It was with dismay that I learned of the proposals in the current draft Freedom of Information Bill regarding the charging of fees. Simply put: the proposal regarding fees is dangerously retrograde, belies a failure of customer/citizen-centricity in Public Sector thinking and a missed opportunity to mandate improvements in Data Governance, runs counter to your own initiatives in relation to “Open Data”, and may indeed serve to weaken any strategy to break down ‘silo thinking’ in Public Service organisations to achieve operational efficiencies through better use of data internally.

    · Dangerously Retrograde:

    Creating an uncapped initial application fees structure for simple exercise of Freedom of Information rates is a dangerously retrograde step. Much of the waste in Public Service organisations over the past few years has been uncovered through journalists and others using FOI rights carefully.

    An uncapped application fees structure, particularly the provisions which give rise to additional charges where data requests span multiple “administrative units”, creates a financial disincentive for budget-conscious editors or freelance investigative journalists to seek information which might be in the Public Interest.

    By effectively curtailing the avenues of information access for citizens to only those who have resources to take an unknown punt on the final costs or to the official press releases our FOI regime that was weakened in 2003 will have been replaced with a model for ‘mushroom management’ in which citizens will be kept in the dark and fed what’s good for mushrooms.

    · Failure of Customer/Citizen-centricity and Data Governance:

    The erection of a “paywall” that will inevitably act as a disincentive to exercise of FOI rights belies a failure of Customer/Citizen-centricity in the Irish Public Sector. In tandem it highlights a failure to seize a valuable opportunity to drive strategic change in Data Governance processes, practices, and methodologies in the Public Service.

    Rather than seeing the challenges raised by requests as an issue which must be curtailed through charges, the Irish Government can choose to invest some effort to understand the root causes of the issues that are reported. For example

    o Could it be that multi-part requests being submitted under the current system are likely a function of journalists needing to maximize the ‘bang for their buck’ on each individual request. Removing this may remove the multi-part queries?

    o How many queries relate to ‘standard’ information or reports which might be ‘pre-packaged’, perhaps in formats that require additional analysis by the requestor, but which meet the requirement for access to information?

    o Are FOI access and accessibility a primary consideration in the design of new systems and processes? If not, should the Data Governance structures of departments be addressed to ensure “FOI-by-design” (producing standardized core reports etc), in the same way as “Privacy by Design” will be a requirement under the forthcoming EU Data Protection Regulation?

    I am currently engaged in a project [details of project redacted for publication] where FOI requirements under regulations such as the Aarhus Convention as well as their voluntary compliance with various regulatory standards have been identified as key strategic drivers for precisely this kind of Data Governance change and re-alignment. The project team has identified further substantial benefits arising from the improvements in Data Governance and Information Quality in this organization that go beyond simple Freedom of Information capabilities.

    By opting for a “paywall” that will keep enquiries out the Government is ignoring an internal dissatisfaction with status quo that can be leveraged to trigger and hopefully sustain Data Governance change in the Irish Public Sector. By knowing the cost of everything and the value of nothing, an opportunity to improve is being foregone.

    · Free Information, Open Data: Fees are inconsistent with Open Data Strategy

    Frankly my head is spinning trying to figure out what the strategic position regarding information is from the Government.

    On one hand you are promoting “Open Data”, while on the other you are proposing changes that will make it harder for citizens (not just journalists) to get access to information in accordance with their rights.

    Ultimately, it is my professional view that the very Data Governance and Information Quality benefits that the proposed “paywall” is forgoing in the context of FOI will inevitably emerge as challenges and barriers to producing Open Data that can be relied upon for service planning, development of applications, guiding investment strategies etc.

    The “pre-packed” reporting solutions outlined above (which I believe were raised by Gavin Sheridan during the legislative consultation period) are Open Data. By implementing these and addressing root causes for current issues and inefficiencies in the FOI model in Ireland the Government has an opportunity for a double-win. Instead we are presented with cognitive dissonance where the Government trumpets Open Data with one hand but claws back Information Freedom with the other, while forgoing any operational efficiency benefits which would arise from tackling root causes in Data Governance practices.

    · Breaking Down Silo Thinking in the Public Service

    The Haddington Road Agreement aims to improve efficiency and effectiveness in the Public Sector and to reduce costs. Ultimately it will need to break down the traditional “silo” thinking that exists in all large organisations, and the provisions regarding staff mobility hint to that.

    However the proposed charging structures under the FOI proposals run counter to this strategic vision. If one was cynical it could be described as a “Silos Charter” given that additional charging will be tied to the number of administrative units in which data is being processed.

    What controls are being implemented to ensure that there is no fragmentation of administrative units to split data processing within FOI-able entities? In the absence of controls it is inevitable that fragmentation will exist, particularly in the context of processes, projects, or functions that might be of notable interest to people seeking to exercise their FOI rights.

    On the other hand, improving Data Governance (data standards, meta-data, master data, clarification of data ownership rights, rules and accountabilities) and seeking to identify common methods for developing and delivering standardized reports would inevitably result in a breaking down of silos and the promotion of cross-functional ways of working within the Public Service.

    However, hiding the silos behind a paywall appears to be the easy path which the preferred choice of the Government.

    Conclusion

    There is a significant potential opportunity to drive change in the governance and management of information in the Irish Public Sector. This change aligns with the objectives of Open Data and has potentially far reaching benefits beyond just FOI effectiveness.

    A PayWall, which is what an uncapped open-ended application fee is in practice, removes this driver and allows both current inefficiencies to fester and current efficiencies to remain siloed, and deprives the citizen of their opportunity to find out answers to their questions of Government. Raising the paywall potentially beyond the reach of individuals, freelance journalists, and mainstream media is a dangerous retrograde step for transparent democracy.

    FOI is the Parliamentary Question for the individual – it should not be locked away behind a paywall

  • More unchecked Data Protection guff in the media

    Today’s Irish Times carried a story in the Business section that the ASAI, self-described on their website as the "self regulatory body" for the advertising industry in Ireland, have issued guidelines on the use of cookies in behavioural advertising which will come into effect from September.

    Great news but for a few minor facts that seem to have eluded the fact checking doubtless done by the journalist taking the by-line.

    • The ASAI is a voluntary self regulatory body. It is not a statutory agency
    • The use of cookies, especially for online behavioural advertising or tracking is covered under SI336, a piece of Data Protection legislation that came into effect in July. 2011. (i.e. 23 months ago).
    • The DPC has already begun enforcement proceedings to encourage compliance. Among the organisations written to late last year was the Irish Times

    So, the ASAI is essentially claiming credit for encouraging its members to comply with the law of the land 27 months late. This is presented unquestioningly in the article as a "good thing" being done by a responsible self-regulating body. But the ASAI is just moving to bring their members into line with the law. Late.

    In doing so they muddy the waters for consumers by making it seem that they are the entity to complain to (they’re not – it’s the DPC, who can levy actual criminal penalties and fines). While the ASAI’s move to regulate the on-line data gathering practices of its members is laudable, responsible journalism would have pointed out that that is what the law actually is and this is not a proactive industry response.

    "Look at us! Self regulation can work!" is the implied message. (That’s exactly the message by the way that has emerged from lobbyists who campaigned to dilute the protections for individual rights in the Draft EU Data Protection Regulation, and also the message that was trotted out in other industries in recent years with less than stellar results).

    Taken in combination with a number of "data protection kills puppies" stories that the Irish Times has been running recently one can’t help but form the view that, in the absence of proper fact checking by journalists someone is st00ging the Irish Times and distorting the paper of record.

    After all, this publication of unchecked errors as fact couldn’t possibly be editorial policy (could it?)

  • The Risk of Poor Information Quality #nama

    I thought it timely to add an Information Quality perspective to the debate and discussion on NAMA. So, for tweeters the hashtag is #NAMAInfoQuality.

    The title of this post (less the Hashtag) is, co-incidentally, the title of a set of paired conferences I’m helping to organise in Dublin and Cardiff in a little over a week.

    It is a timely topic given the contribution that poor quality information played in the sub-prime mortgage collapse in the US. While a degree of ‘magical thinking’ is also to blame (“what, I can just say I’m a CEO with €1million and you’ll take my word for it?”) and , ultimately the risks that poor quality information posed to down stream processes and decisions  were not effectively managed even if they were actually recognised.

    Listening to the NAMA (twitter hash-tag #nama) debate on-line yesterday (and following it on the excellent liveblog.ie I couldn’t help but think about the “Happy path” thinking that seems to be prevailing and how similar it is to the Happy Path thinking that pervaded the CRM goldrush of the late 1990s and early 2000’s, and the ERP and MDM bandwagons that have trundled through a little place I call “ProjectsVille” in the intervening years.

    (note to people checking Wikipedia links above… Wikipedia, in its wisdom, seems to class CRM, ERP and MDM as “IT” issues. That’s bullshit frankly and doesn’t reflect the key lessons learned from painful failures over the years in many companies around the world. While there is an IT component to implementing solutions and excuting  projects, these are all fundamentally part of core business strategy and are a business challenge. )

    But I digress….

    Basically, at the heart of every CRM project, ERP project or MDM project is the need to create a “Single View of Something”, be it this bizarre creature called a “Customer” (they are like Yeti.. we all believe they exist but no-one can precisely describe or define them), or “Widget” or other things that the Business needs to know about to, well… run the business and survive.

    This involves taking data from multiple sources and combining them together in a single repository of facts. So if you have  999 seperate Access databases and 45000 spreadsheets with customer  data on them and data about what products your customers have bought, ideally you want to be boiling them down to one database of customers and one database of products with links between them that tell you that Customer 456  has bought 45000 of Widget X in the last 6 months and likes to be phoned after 4:30pm on Thursdays and prefers to be called ‘Dave’ instead of “Mr Rodgers”, oh… and they haven’t got around to paying you for 40,000 of those widgets yet.

    (This is the kind of thing that Damien Mulley referred to recently as a “Golden Database“.)

    NAMA proposes to basically take the facts that are known about a load of loans from multiple lenders, put them all together in a “Single View of Abyss” (they’d probably call it something else) and from that easily and accurately identify under-performing and nonperforming loans and put the State in the position where it can ultimately take the assets on which loans were secured or for which loans were acquired if the loans aren’t being repaid.

    Ignoring the economists’ arguments about the merits and risks of this approach, this sounds very much like a classic CRM/MDM problem where you have lots of source data sets and want to boil them down to three basic sets of facts, in this case:

    • Property or other assets affected by loans (either used as security or purchased using loans)

    • People or companies who borrowed those monies

    • Information about the performance of those loans.

    Ideally then you should be able to ask the magic computermebob to tell you exactly what loans Developer X has, and what assets are those loans secured on. Somewhere in that process there is some magic that happens that turns crud into gold and the Irish taxpayer comes out a winner (at least that’s the impression I’m getting).

    This is Happy Path.

    The Crappy Path

    Some statistics now to give you an insight into just how crappy the crappy path can be.

    • Various published studies have found that over 70% of CRM implementations had failed to deliver on the promised “Single View of Customer”

    • In 2007 Bloor Research found that 84% of all ERP data migrations fail (either run over time, over budget or fail to integrate all the data) because of problems with the quality of the data

    • As recently as last month, Gartner Group reported that 75% of CFOs surveyed felt that poor quality information was a direct impediment to achieving business goals.

    • A study by IBM found that the average “knowledge worker” can spend up to 30% of their time rechecking information and correcting errors.

    Translating this to NAMA’s potential Information Management Challenge:

    1. The probability of the information meeting expectations is about the same as the discount tthat has been applied on the loans. (30%).
    2. The probability of the migration and consolidation of information happening on time, on budget and to the level of quality required is slightly better than the forecast growth rate in property prices once the economy recovers (16% versus 10%)
    3. Around 30% of the time of staff in NAMA will likely be spent checking errors, seeking information, correcting and clarifying facts etc.

    There is a whole lot more to this than just taking loans and pressing a button on a money machine for the banks.

    Ultimately the loans are described in the abstract by Information, the assets which were used as security or which were purchased with those loans are defined by data, and the people and businesses servicing those loans (or not as the case may be) are represented by facts and attributes like “Firstname/LastName”, “Company Registration Number”. Much as we taxpayers might like it, Liam Carroll will not be locked in a dungeon in the basement of Treasury Buildings while NAMA operates. However, the facts and attributes that describe the commercial entity “Liam Carroll” and the businesses he operated will be stored in a database (which could very well be in

    This ultimate reliance on ephemeral information brings with it some significant risks across a number of areas, all of which could signpost the detour from Happy Path to Crappy Path.

    Rather than bore readers with a detailed thesis on the types of problems that might occur (I’ve written it and it runs to many words), I’ve decided to run a little series over the next few days which is drawing on some of the topics I and other speakers will be covering at the IAIDQ/ICS IQ Network Conference on the 28th of September.

    Examples of problems that might occur (Part 1)

    Address Data (also known as “Postcode postcode wherefore art thou postcode?”)

    Ireland is one of the few countries that lacks a postcode system. This means that postal addresses in Ireland are, for want of a better expression, fuzzy.

    Take for example one townland in Wexford called Murrintown. only it’s not. It has been for centuries as far as the locals are concerned but according to the Ordnance Survey and the Place Names commission, the locals don’t know how to spell. All the road signs have “Murntown”.

    Yes,  An Post has the *koff* lovely */koff* Geodirectory system which is the nearest thing to an address standard database we have in Ireland. Of course, it is designed and populated to supprt the delivery of letter post. As a result, many towns and villages have been transposed around the country as their “Town” from a postal perspective is actually their nearest main sorting office.

    Ballyhaunis in County  Mayo is famously logged in Geodirectory as being in Co. Roscommon. This results in property being occasionally misfiled.

    There are also occasionally typographical errors and transcription errors in data in data. For example, some genius put an accented character into the name of the development I live in in Wexford which means that Google Maps, Satnavs and other cleverness can’t find my address unless I actually screw it up on purpose.

    Of course, one might assume that the addresses given in the title deeds to properties would be accurate and correct (and, for the most part, they are I believe). However there is still the issue of transcription errors and mis-reading of handwriting on deeds which can introduce small and insidious errors.

    It is an interesting fact that the Land Registry has moved to computerised registers in recent years but the Property Registration Authority trusts still to the trusty quill and only recently moved to put the forms for registering property deeds on-line. Please let me know what you think of the layout of their web form.

    I am Spartacus (No, I am Spartacus. No I’m Brian Spartacus).

    Identity is a somewhat fluid thing. When seeking to build their consolidated view of borrowings, NAMA will need to create a “single view of borrower”. This will require them to match names of companies to create a single view of businesses who have borrowed (and then that will likely need to have some input from the CRO to flag where such companies have a change in status such as being wound up or bought).

    The process will also likely need to have a Single View of Borrower down to the level of a person a) because some loans may be out to people and b) because the link between some borrowings and various companies who would have borrowed will likely turn out to be an individual.

    Now. Are these people the same:

    • Daragh O Brien

    • Dara O’Brien

    • Daire O Brian

    • Daragh Ó Briain

    • Dara Ó Briain

    • Darach O Brien

    • D Patrick O Brien

    • Pat O’Brien

    The answer is that they are. They are variations in spelling on my name, one possible variation in use of my middle name, and the last one is what a basketball coach I had decades ago called me because he couldn’t pronounce Daragh.

    However, the process of matching personal data is very complex and requires great care be taken, particularly given the implications under the Data Protection Act of making an error.

    The challenge NAMA potentially faces is identifying if Joe Bloggs in Bank A is Joseph Bloggs in Bank B or J.P Bloggs in Bank C or both or none at all. Recommended practice is to have name plus at least two other ‘facts’ to feed your matching processes. And at that the process inevitably requires human review.

    However, the problem is relatively solvable if you invest in the right tools and are willing to invest in the people necessary to use the tools and (unfortunately) take care of the manual review and approval that would be required.

    A related risk is the risk of not having the customer’s name correct. Simply put, where that happens the lender or controller of the loans effectively hands the borrower a “get out of jail” card as they can simply claim that the named person is not them. Courts are pedantically persnickety about accuracy of information in these types of matters.

    A corollary of this is where the lender or controller of the loans (in this case NAMA) starts chasing the wrong person for the payment of a loan due to a mismatch of data about people. Here, the aggrieved party could ultimately sue the controller of the loans for libel if they publish to anyone the allegation that the person owed money and wasn’t paying it back.

    While each of these are risks that the banks individually manage on their own at the moment, the simple fact of pulling all this information together under NAMA increases the risk factor here. Each bank may have individually had problems with errors and mismatching etc. but resolved them quietly and locally. The root causes of those errors may not be addressed and may not be possible to address in a bulk data migration. Therefore, the data may get muddled again, leading to the issues outlined above.

    Conclusion (for Part 1)

    NAMA will not be managing physical loans or physical assets. What NAMA will work with is the information about those things, the facts and figures that describe the attributes of those things in the abstract.

    To assume that individually Irish banks have levels of data quality that are as pure as the driven snow is naive. To assume that taking a load of crummy data from multiple sources and mixing it together to create a new “Single View” of anything without first understanding the quality of that information and ensuring that you have steps in place to manage and mitigate the risks posed by non-quality information means you risk failure rates of between 70% and 84%.

    To put it another way, there is only between 16% and 30% of a chance that NAMA will be able to deliver the kind of robust management of all loans out to individual property developers and property development companies that would be necessary to ensure that the risk to the taxpayer is properly managed.

    The key to managing that risk of failure will be discussed in my next post and at the IAIDQ/ICS IQ Network conference on the 28th of September

  • Is Info Quality Management a Recession Proof Profession?

    Over the past few weeks I’ve been pondering whether or not Information Quality Management is a recession proof profession. Those of you who know me will probably guess that my recent departure from “big company” employment was one of the seeds to this line of thought. Another was the interesting findings contained in the IAIDQ’s recent report on Salary and Job Satisfaction in the Information/Data Quality profession (you can find a copy of the report here).

    First off, the salary survey made for interesting reading because it pegged the average salary (in US dollars) for an Information Quality professional at just over $95000 (EUR 72k approx).  In Europe, the average was $85000 (EUR 65k approx). Cripes, I was a bit less well paid in the old job than I had thought. At those salary levels, the information quality professionals were, overall, satisfied with their lot.

    78% of IDQ professionals say they feel either secure or very secure about their current position, indicating remarkable confidence despite the current difficult economic times

    So… is this one of those mythical recession proof professions?

    Data Data Everywhere….

    We’re fond of saying it but it is true. We live in an increasingly “informationalized” world. Strip away most business models now and you will find that the real value is generated by the smooth flow of information around an organisation. Buying a laptop from Dell? That’s an information flow that needs to pass with out glitch to a factory in Poland (alas no longer Ireland) and also out to suppliers in China and elsewhere to ensure that the bits all arrive together so you receive delivery of a laptop to your door. And let’s not forget about the flow of information about your finance arrangement to fund the purchase. Try to get a phone line connected and you are relying on the quality of information that the call centre agent has about what services are available in your area. Buy a coffee on your debit card… The list goes on.

    Information and data are increasingly being recognised as critical assets to the organisation. Whether it is in Tom Redman’s “Data Driven”, Tom Fisher’s “The Data Asset” or on blogs or webinars, we see an increasing presentation of data as an asset in terms that C-level executives should get. But this isn’t enough (for reasons we’ll come to in a minute)

    But on a more personal level I’ve been busier since I left my old job then I have been at almost any time in my career. I am finding more people connecting with me through the IAIDQ (and other forums) and I am sensing a strong feeling of postive attitude which is far removed from “magical thinking” but is instead grounded on a very clear understanding of how poor quality information contributed to the mess we are in and an equally clear vision of how effective management of the quality of information can help get us out of this situation and, more importantly, help us to better manage the risk of it happening again.

    The Problem…

    The problem we face now as a profession, and this was highlighted very clearly by the IAIDQ’s study, is that of clearly communicating to our employers, customers, and wider audiences, the value of good quality information. 84% of the IAIDQ’s respondents said that this was their biggest challenge. If we face that challenge in a downturn that caused us to look at the relative importance of the assets in our organisation, how can we hope to overcome that challenge now that greenshoots are breaking out all over?  (As I write this Germany is now out of recession, France is on the way, Eurozone is heading positive)

    But Gartner recently shared (well, in 2006) with us this prediction:

    Through 2011, 75 percent of organizations will experience significantly reduced revenue growth potential and increased costs due to the failure to introduce data quality assurance and coordinate it with their data integration and metadata management strategies (0.7 probability).

    In physics, “friction” is the name given to the opposing force that slows the movement of a body. The problem with friction is that it requires you to expend greater effort to achieve the same result. The laws of Conservation of Energy tell us that that extra energy is lost in the form of heat. (Think about the last time you watched your local scout troop light a fire by rubbing two sticks together. Didn’t they look out of breath when they’d finished?)

    More recent Gartner research, published on the 11th of August 2009 [2009 Gartner FEI Technology Study Reveals FinanceManagers’ Perspectives on Data Quality, www.gartner.com] finds that:

    Three-quarters of the respondents consider data quality problems a constraint on, or a barrier to achieving, business success. Even so, only 41% of their organizations have a formal improvement program — the rest are doing nothing formally to improve matters.

    So. There we have it. Confirmation that poor quality information is adding friction to businesses. And only 41% have a formal programme in place to reduce that friction (and even then, a programme does not equate to successful outcomes).

    While there are some signs of the global economy recovering, it is clear that poor quality information will add friction to the mix, potentially slowing down the pace of recovery. And the last thing you need when trying to push uphill is friction working against you. Organisations who have to carry the non-value-adding costs of poor quality information will be unable to reap the “first mover” advantages or seize the “low cost operator” niches in the post melt-down market place. Organisations which have invested in reducing the friction will benefit.

    In my opinion, there is an opportunity right now for information quality professionals to develop some clear messages about the importance of information quality and its value to your organisation and the wider economy.

    • To compete in a “lean” way , organisations are investing in Business Intelligence. Without regard to the underlying quality of the information being pulled together, this can rapidly descend into “Business UNtelligence”. Issues such as missing or incomplete data, or even the existence of “non-standard” characters like apostrophes in surnames or email addresses can cause problems in your BI reporting.
    • To ensure compliance with current and as yet emerging regulations, organisations will need to pay closer attention to the information flows within their walls and between them and their partners. Closer validation of data, increased focus on internal integrity of facts (e.g. does the salary figure on the loan application align with other credit information available to a lender) will likely become more important. These are all information quality based initiatives.
    • Risk Management – a colleague who specialises in Risk Management consulting shared with me recently that “Can’t rely on our information” is a risk that keeps cropping up again and again in his risk workshops with large businesses. This is borne out by an Information Age survey (referenced in Tom Fisher’s new book) which found that 32% of companies who responded cited Risk Management (compliance and regulatory issues) as a key driver of their Information Quality initiatives.
    • Changes in the quality of information only take place through effective management decisions. Either you decided to invest in managing your information quality effectively, or you effectively decide NOT to manage your information assets.

    These are just a few areas where there is friction caused by poor quality information – I welcome suggestions for others.

    By removing or reducing the friction, the information quality expert and their team can help businesses seize new ground or at least hold their own as the global economy recovers slowly. By reducing friction, you reduce the amount of wasted energy that is lost in the form of heat.

    Conversely, if you are trying to get your information quality programme jumpstarted, one good way is to figure out how to focus all that lost heat in one place to start a small fire under someone.

    Just answer the question!

    But back to the question at hand… is Information Quality Management a Recession Proof profession?

    I think the answer is yes and no.

    It is yes in that, insofar as any profession can be recession proof, information quality practitioners and vendors have seemed to weather the storm quite well recently. Furthermore, down turns inevitably focus attention on areas of avoidable cost and waste within organisations. Sensible ones look to remove that cost surgically – a process that in and of itself requires sound information. Ultimately, if a business is trading it is creating and consuming information in order to make or deliver its products. Therefore, even in a down turn there is a role ofr the information quality professional. The relatively high job satisfaction ratings in the IAIDQ’s survey suggest that IQ professionals may have been biding their time in organisations and building their value cases slowly.

    However, the answer is no if we think to what might happen once recovery sets in. In the absence of a crisis, how do we present the value case for continued or renewed investment in information quality? In order to ensure success in the good times we as a profession must convince senior management of the value of reducing information friction in our businesses. While it is easy to point at the pile of rubble and say “If we’d had better quality information we could have avoided that”, it is more challenging to show how those same skills, tools and approaches can build a shiny new edifice on the foundations of that rubble.

    So, information quality management is recession proof, but only if we continue to define and refine the value proposition for better quality information within our organisations and in the wider global context as well.