Why Scrap and Rework isn’t good enough

Simon has thrown down a bit of a challenge…  can I show why Information Scrap and Rework isn’t good enough because it seems like a sensible starting point…

First off… let me provide a reference that should educate and delight (at least some of you) that explains what this Information Quality yoke is all about… THERE we go. The reference is a little old (2002) but for an update come to ICTEXPO on Friday.

Now… why isn’t Scrap and Rework good enough?

Who likes chocolate cake? Isn’t it a pain when your face gets covered in chocolate from mashing handfuls of cake into your gob? But you can wipe your face (usually in your sleeve) and carry on. That’s scrap and rework. A better solution is to wipe your face and take a smaller division of cake (a forkful). That is a change in the process based on an analysis of why you keep getting a chocolatey face, coupled with a scrap and rework task to set a baseline of cleanliness for your face that you will seek to maintain.

Simon is right – scrap and rework looks like a good place to start, and when you say “Data Quality” to most people that’s what they think, under the labels “data scrubbing”, “data cleansing” or similar. However, it doesn’t address the actual source of the poor information quality, much as wiping your face in your sleeve doesn’t stop your face getting covered in chocolate.

Therefore, once you clean your database, you will very quickly find it filling up with duff data again. Which eventually results in another round of scrap and rework to fix things again. Which then leads people to say that Information Quality management doesn’t work and costs lots of money. But scrap and rework isn’t information quality management. It is a process step to improving the quality of your information but it is just one step in many that range from culture change (from apathy to active interest) to process change to training etc.

Tom Redman is one of the co-founders of the IAIDQ. His metaphor is that databases are like lakes. No matter how many times you clean the lake, if you don’t address the sources of ‘pollution’ (root causes, cake-eating processes) then you will never achieve good quality.

To put it in professional terms that Simon (law-talking boyo that he is) might understand, scrap and rework is like apologising and offering some compensation everytime you punch a complete stranger in the face. A far better solution is to examine why it is you punch strangers in the face and stop doing it. Your apologies and offers of money to the injured fix the historical damage but do not prevent future occurences. And I doubt Simon would counsel any of his firm’s clients to continue punching strangers in the face.

Scrap and rework is costly. Scrap and rework on a repetitive institutionalised basis is futile, creating a sense of doing something about your Information quality without actually getting anywhere but burning a pile of cash to stand still. It is an important step in any information quality management programme. However, understanding your data capture processes and the root causes of your poor quality data and then acting to improve those processes to address those root causes are the components that contribute to a sustained improvement in quality.

Scrap and rework solves the problems of today at a short-term economic cost. However, it serves to bury the problems of tomorrow unless it takes place in tandem with process improvement to address root cause and the development of a ‘Quality culture’.

To tie this back to the Electoral Register, to rely on scrap and rework would mean that we would get a clean register this time around at a point in time. However, over time the register would degrade in quality again, in the same way as your face gets dirty again if you don’t change the way you eat your cake.

Now put that chocolate cake down and get a fork!

An Irish Open University….

Saw this on the Labour party website:

Coughlan calls for the establishment of Irish Open University

Interesting. What about:

Oscail (www.oscail.ie) DCU’s distance learning school?

What about UL’s courses in Project management that are offered over distance learning.

And above all else… what about the Open University in Ireland?

Perhaps what is needed is some funding to develop Irish universitites existing distance learning programmes?

Oh, hang on.. the speech/press release was posted on 1st April 2006. I hope I’ve been sucked in by a prank. I really do.

Information non-quality on the blog

Simon over at tuppenceworth has been on to me. Apparently my blog isn’t quite meeting his expectations.

Can’t have that.

Simon’s problem is that he isn’t quite sure what scrap and rework is. He says that it sounds bad to him, and I use it in a way that suggests it is a thing we shouldn’t do… but what exactly is it?

Can’t have that. Must meet or exceed expecation… so what I will do is put together a page in the blog that will contain a lexicon of terms that I can link to… I’ll try to make it alphabetical as it would be ironic if I couldn’t manage that simple an Information process.

Any terms that I use (and I suspect there may be many) that require definition please let me know. I sometimes speak a different language (dataqualitarian) and I acknowledge that I need to translate sometimes to communicate clearly.

Thanks Simon.

 

Reponse to Damien Blakes’ post on irishelection.com

Damien Blake written a good piece on IrishElection.com regarding his view on how to address the Electoral Register.

Much of what he says has merit as a short term solution. He suggests that we scrap the register (no problem there) and rework it from scratch (again, no problem there) rather than limp on with a defective register.

Damien suggests that my writings will eventually find their way into undergraduate or post graduate theses.  I too look forward to the day that clear thinking about the fundamental best practices of Quality Management applied to Information form part of University curricula at undergraduate level and post-graduate level, much like the practices of Manufacturing Quality Management gained acceptance. In the US this is already beginning to happen, and I have been involved in curriculum development work with an Irish University in a similar vein.

What I have written is based on a number of years (best part of a decade) working in complex Information Management environments and on the shared experiences of other practitioners in the Information Quality Management space with whom I have spoken at conferences (internationally), and with whom I work on a regular basis as a Director of the International Association for Information & Data Quality. The techniques, methodologies and approaches I put forward are based on my real world,practical experiences in applying best practices that have been proven in other industries and disciplines.

Damien’s further goes on to suggest using the PPS Number and associated data to register people – preferably automatically. What Damien has suggested here is a process change to address root causes of poor data quality. Excellent. That is what I have been writing on… well at least as far at the review/change of processes goes. I’ll come ot my concerns with his proposal in a moment. Well done for thinking about the root causes of the problem and how the processes can be changed to address it. Top of the class that man.

Damien’s suggestion doesn’t address the fact that there is no legal obligation on anyone to register to vote, and it could even be argued that one has a constitutional right not to register to vote. Automatic voter registration based on a “Single View of Voter” may not be a runner. Also, the Data Protection Commissioner has limited the uses that a PPS number can be put to – however I am sure legislation could get around that. The Digital Rights Ireland site has a nice paper on it about the ‘scope creep’ in the use of PPS Numbers that I’ve referenced in an earlier post.

Ultimately, even if we scrapped the Register in the morning and rebuilt it in a shining pristine form, the simple fact is that name and address data degrades at a significant rate. In the absence of clear controls and processes to manage and maintain that data at an acceptable level of quality we will find ourselves rapidly returning to a situation where the Register is unreliable – Scrap and Rework is not the route to a high quality Information asset. It is a step on the journey, and an important process – but in the absence of process re-engineering to address root causes of defects and deficiencies the inevitable result is more scrap and rework.

Damien’s proposal to use the name/address/citzenship data associated with the PPS number would serve to reduce redundancy of data (multiple copies of the same data held in multiple data stores) but it may run into Data Protection issues. But as a change of process for the management of Electoral Register data to address deficiencies in the existing process it has merit. But should you have to be a tax payer or a recipeint of State benefits in order to vote?

Damien’s suggestion for a Statutory agency or a reallocation of resources/roles to task someone with maintaining the Register. Again, I am in wholehearted agreement. What Damien proposes here is a review of the Governance model for this data to give clear accountability, authority and mandate and (I would surmise) a standardisation of processes, controls and toolsets for managing and measuring the quality of Electoral Register data. I fully agree with the general thrust of Damien’s proposal here, although the specifics of what that Governance model in my view should be aligned with the requirements of the process and the requirements of the controls necessary to ensure the quality of the electoral register – simply assigning a role with a stroke of a pen does not deliver quality improvements.

I agree with Damien that the process for voter registration and for maintaining that data should be a simple as possible. Clear definition of processes and business rules to support ‘flow-through’ registration and data maintenance are part of the Information Architecture design that should underpin any long term solution. Simplicity of process  could be part of the ‘customer expectation’ against which the quality of the process (and the information it produces) could be measured. However a simple ‘customer’ interface that sits on top of chaotic processes riddled with deficiencies and absences of controls to ensure the quality of the Information will not achieve the full objective of a simple to operate set of processes or functions that deliver reliable and high-quality Electoral Register information.

Damien is right. We need to start again. We need to start again in terms of the information in the Register. We need to start again in terms of the Governance model that is put in place to manage this Information Asset. We need to start again in terms of the processes that people follow to create, update and maintain that information to ensure that we achieve our objective of a reliable, accurate (within a margin for error) Electoral Register. We need to start again in terms of how we think about the ‘architecture’ that this Electoral Information is held. We need to start again in terms of ensuring that we adopt appropriate technologies and strategies to address identified weaknesses in the processes for managing our Electoral Register data.

However, to focus just on scrap and rework simply solves the problem of today. Addressing the root causes in the processes and governance as well as conducting scrap and rework on the data solves today’s problems and prevents those of tomorrow.

I’m glad Damien and I are in such agreement on the principles, even though we may differ in our view on specifics (specific solutions aren’t my goal here – raising awareness of Best Practices was my intention). I hope he can find the time to attend the Information Quality Master Class that is being held in the RDS on Friday where a person with even more years experience in this domain than I will be sharing his knowledge with delegates

Use of PPS Numbers..

The literati over at Digital Rights Ireland have a nice posting on their blog about the issues surrounding the use of PPS numbers and the overwhelming need for clarity of governance in their use to avoid ‘scope creep’.

Here is the link to that post… http://www.digitalrights.ie/2006/04/20/pps-numbers-and-cradle-to-grave-tracking/

Defined processes that address identified root causes, operating in the context of a clear Governance model supported by appropriate technology and Information Architecture, building from a baseline that may be set through initial scrap and rework to recover quality is the only sustainable solution to the Electoral Register issues.

The proposals to extend the use of PPS Numbers without a clearly established Governance raises the spectre of scope creep in the use of both the PPS and Electoral register data.

Electoral Register Forms

I had a look at the Reach Service Portal ( a central portal for citizen information).

They had a helpful link to a form to register for the Register of Electors. Very helpful. My thoughts are as follows:

  1.  If there is no standard template in existence amongst local authorities ( to the Minister’s surprise), why has the Dun Laoghaire Rathdown County Council form been picked as the standard one to select through a central information portal? Is it in some way superior to the other forms? If so, should this be the standard form?
  2. The form does not allow for deleting people from the register (such as might need to be done if a person moved out of a residence) nor does it seem to allow for flagging a change in name – which could lead to additional people being added to the register just because they got married and their names changed.

The form does state that it is an offence to provide false information. What processes are in place for checking for false/inaccurate information?

Perhaps Irish citizens visiting the Blogosphere could check with their local authorities to see if they can register to vote through an ‘on-line’ facility. Thus far all I have found is a link to Dun Laoghaire/Rathdown County Council. I may be missing something on www.wexford.ie (Wexford County Council’s site) but I can’t even find a link there to an application form.

If you do find a form or on-line service, post a comment here with your assssment of the form – is it easy to understand/use? Is the information capture process meeting your expectations?

Dunlaoghaire Rathdown Councils’ form for Electoral Register… the apparent standard form.

From the Minister’s Mind…

I have just come across Dick Roche’s website (www.dickroche.com). It’s not bad as politico sites go. I was particularly interested in his post re: the Electoral Register issues. To an extent it is more telling in what it lacks than what it contains.

1)      The use of Census Enumerators or other temporary personnel to support local authorities in preparing the next Register as part of an intensive registration campaign to be conducted this summer“.

It is May now. As per my previous posts, Census Enumerators cease to be employed on the 21st of this month. I haven’t seen or heard any adverts recruiting for Electoral Register data gatherers. Irrespective of the lead times involved, recruitment of additional people to run the process is simply scrap and rework and will not deliver a sustained improvement in the quality and control of our Electoral Register. Unless the deficiencies in the processes are addressed any cleansed register will rapidly degrade in terms of its quality. Scrap and rework is not quality improvement, it is not value-adding. It is just scrap and rework – a costly exercise in standing still at best.

2)      Additional ring fenced financial resources will be made available to local authorities to update the Register“.

Excellent. Money will be available to throw at the problem. But in order for that money to be spent wisely, there must be a clear strategy across the local authorities and the Dept of Environment & Local Government as to how that money should be spent. Leadership is required. My suggestion for a quick hit:

·        Select a standard Information Quality toolkit to perform standardisation and matching within local authority datasets and match local authority Electoral registers against the register of Deaths and also the Register of marriages.

Standardisation on a single toolset allows for greater reuse of common functions (such as the standardisation of Irish addresses), the development of transferable skillsets between Local Authorities and economies of scale when negotiating with vendors.

This process can then be automated and an initial level of quality scorecarding can be put in place. However, updating the Register is only the start. It is still Scrap and Rework, albeit with some fancy toys in the mix to improve efficiencies. What is required is the expenditure of resources to identify root causes, assess and improve processes and in general conduct a root and branch overhaul of our Electoral Register processes to bring them to a standard that meets the expectations and needs of 21st Century post-Celtic Tiger Ireland.If the core processes are not addressed, everything else is just scrap and rework which will need to be redone again in a few years (if not months).

3)      Updated and consolidated guidelines to ensure all Local Authorities work to the same template

Standardisation of Information capture processes and procedures is an important first step. Reducing variation in the process is important. This is the nearest thing to a Process Improvement initiative that the Minister has mentioned thus far. But it isn’t enough. It addresses one potential root cause for the dismal level of quality in our electoral register – what are the others?? Will the consolidated guidelines make provision for people who want to update their name without moving address?

Fixing the templates without examining other root causes and reviewing the process and information architecture simply means that the scrap and rework will be better structured. Scrap and rework is non-value adding and does not improve quality in a sustainable way. Repetitive cycles of scrap and rework are costly exercises in admitting failure of governance.

Does the Minister propose here to establish a consolidated Governance Board to ‘own’ the standards around templates, toolsets, Information Architecture design etc. or is this just a memo that will be issued? If it is the former – then this is a positive step. The latter is just lip-service to Quality.

4)      An early start to be made to the local Authority Register Campaign for 2007/2008

Data-cleanse early, data-cleanse often. No matter how early you start your scrap and rework or how much time you allow for it, it remains Scrap and Rework unless there is a review of processes to remove sources of error and root causes of non-quality.

All this proposal does is push the ‘cycle time’ for an Electoral Register refresh out a bit and increase the cost of doing so. That is fine as a once-off to establish a baseline level of quality in your data, but should it become the ‘way things are done around here’ it is simply institutionalised scrap and rework.

Where are the Minister’s proposals for a strategic, long-term plan for continuous improvement of quality in the Electoral Register?

5)      “Local Authorities have been instructed to make use of relevant databases to cross check the Register; Councils in particular will be encouraged to use their own databases more effectively than they have to date;”
Cross-checking = matching. Matching of data between datasets requires an investment in tools and techniques to standardise the data and then match on name-plus-2 identifiers.

If the existing databases that the Councils have were not designed for the purposes of matching against other datasets, they may not hold sufficient data items to support a match. In addition, they may not be complete as surrogate views of reality – for example I don’t think Wexford County Council have any record of me living at my current address as my wife pays the Bin Charges and the car is in her name.

Additionally, the quality of these ‘surrogate sources’ for reality might also be questionable in terms of completeness, consistency, conformity, accuracy etc. and could lead to false positive matches or false negative matches.

Data Protection issues come into play here also and could land Councils in legal bothers if they use a data set for a purpose for which it was not provided.

Furthermore, continuous clashing of multiple data sets doesn’t improve the source processes – it is just another flavour of scrap and rework unless it is in a clear process context. Conducting these clashes using ‘traditional’ IT approaches will likely be costly and time consuming. This raises the need for a common “Second Generation” Information Quality management toolset across Local Authorities that can standardise and match across multiple data source types. There are a number of such tools on the market.

6)      “The possibility to allow local authorities access other databases to cross check the Register is now under consideration”
Access to more data does not lead inexorably to better quality Information. A clear process for what information is to be used and how is needed. Furthermore, technical issues about getting access to the data and integrating it into an Electoral Register master file would need to be considered.

Again – this raises the question of process purpose and function and the tools that would be available to Local Authorities to conduct the processes.

It smells like a Process Improvement, but unless the additional information can be used to address an identified Root Cause of process failure and Information Non-quality, it is simply information for information’s sake and would be another non value-adding form of scrap and rework.

7)      “Approximately 30,000 deaths occur each year: new arrangements to delete deceased persons from the Register have been agreed”
This is a process improvement piece that, at first glance, would address another obvious root cause of error in the Electoral Register.

What are these arrangements? Do they address the technical challenges of standardising data and matching across multiple data sets, particularly given the ‘interesting’ nature of Irish addresses?

Do the arrangements include recommendations on toolsets, techniques and processes for standardisation and matching of data?

If the arrangement is simply to share “electronic files” of data without addressing the process of making use of this information then we return to the point that access to more data does not lead inexorably to better quality Information.

8)      “An intensive information campaign will be mounted later this year”
What information will be given in this campaign? Will the Minister spend taxpayer funds educating people on an Electoral Register process that would seem to have some key deficiencies?

Will the Minister be conducting a Root Cause analysis and Process Improvement project to identify and address deficiencies in the Electoral Register process before engaging in “an intensive information campaign”? If not, then we must accept that we may simply be teaching people how to operate a broken process more efficiently.

Yes, an information campaign is required but in the absence of a review of the processes to identify and address root causes the information campaign would be unlikely to lead to significant sustained improvements in the quality of Electoral Register information.

9)      “Better ‘on-line’ facilities to help  people check the voting Register, including an additional on line facility through the Department of the Environment website”

What are the processes that would sit behind these on-line facilities? How will citizens be able to correct their Electoral Register data through these on-line facilities? How will on-line facilities serve to improve the quality of Electoral data?

One of the lessons from the Business world and initiatives such as customer relationship management (CRM) is that bolting a web front end on defective processes simply exposes the defective processes and information to your customers and the public.

There are any number of technical and detailed questions on the proposed information architecture for these on-line facilities that I could ask… but my reader’s eyes are probably glazing over a bit already…

10)  “A strengthening of the controls at polling stations at election time. New guidelines in this area will issue before the next Election”.
 The focus of quality management and quality improvement should be on building quality into the process, not inspecting defects out. While an improvement on the controls at the polling station is to be welcomed, a root and branch reform of the processes and governance for the Electoral Register would deliver more sustained quality improvements.

Catching someone personating on the day is good. But denying them the opportunity in the first place is even better. A statistical report showing the continued increase in accuracy of the register is a nicer headline than a report of the numbers of persons caught in the act of corrupting our democracy. Apart from anything else, the question people will ask is how many weren’t caught for each one that was?

The Minister pats himself on the back for his practicality, sense and realism. I question the completeness of the thinking and the long-term sustainability of any quality that would arise from his proposals. Nothing that is proposed looks to identify and eliminate root causes of error. Nothing that is proposed seems, to my reading, to amount to anything more than reactive scrap and rework.

Minister – prove me wrong.
 

Sampling of Constituency shows 90% error rate

Vote loss scandal as 90% names are ‘wrong’ (requires registration with www.unison.ie)

OK, I am taking this headline with a small pinch of salt, not having had any visibility of the methodology used, sample size taken etc. There are lies, damn lies and statistics after all. 52.5% of cats surveyed said that they knew that ;-). Also – what is the definition of “wrong” in this case?

However – if it is representative of the trend in the Register nationally then a whopping great problem exists that needs some sort of concerted plan to be put in place to address it.

My question is this… what study has been done to identify the potential root causes of this error?

One anecdotal cause I have heard about just today concerns people who are changing their names for reasons of marriage, divorce or other. The only form that is issued is a change of address form (or so I’ve been told). But if you have been co-habiting and have then married, you will not necessarily be changing address. Therefore if you opt to register in your married name (Ms. Jones becomes Mrs Smith at the same address) the process throws, for want of a better term, at total wobbly and it is possible that Ms. Jones would remain on the register while a new entry for Mrs. Smith is created.

This process probably worked fine back in pre-Celtic Tiger Ireland where people co-habited less than they do now before marriage. However it has failed to keep pace with the expectations of the Information Consumers and other actors in the process.

To the political classes… please stop playing political ping pong with this issue. Take action NOW to identify the root causes of these errors, address deficienies in process.

One good starting point would be to attend the Master Class on Information Quality that the Irish Computer Society and the IAIDQ are hosting in the RDS on Friday… http://www.ics.ie/expo/iq/index.htm

Short term actions that could be taken:

1) Invest in software (ideally a standard software tool across all Local Authorities) to clash Electoral register data with the Register of Deaths and the Register of Marriages to cut out noise and reduce the incidence of dead men voting or blushing brides voting early and often. This investment in a standard toolset could leverage economies of scale and would also standardise approaches across local authorities while supporting reuse of clean-up processes. Examples of such tools would include Informatica Data Quality (formerly Similarity Systems Athanor) -which is a IRISH made product (nice Celtic Tiger success story there).

2) Commence a root cause analysis review across ALL Local Authorities under the management of a central governance board to identify the process failures

3) Fix process gaps, introduce standardisation, amend processes, work practices, information management approaches etc. to enable the process to deliver a product that meets or exceeds expectations.

To build the business case for this work, I would call on the statistically minded politicos to work out what the cost to the State is from process failures in other areas that rely on the Electoral Register – for example the Jury Duty process – how many letters are sent to people who are dead or no longer exist at an address each year.

From the data complied by Ogra Fianna Fail in Trinity up to 90% of the spend in that downstream process could be wasted.

What is the COST to the Exchequer of doing nothing to improve the quality of these processes on a continuous basis?

The Week in Politics…

I caught some of The Week in Politics last night. The video stream of the discussion can be found here (requires RealPlayer).

What interested me was Noel Ahern, Junior Minister for the Environment (and sibling of our Prime Minister) seeking to distract attention from the electoral register fiasco by implying (please note that he didn’t actually say that it was happening) that there was nefarious activity on behalf of Sinn Fein registering people without their knowledge whilly-nilly.

Very “these aren’t the droids you’re looking for”… but unfortunately Junior Minister Ahern’s comments further highlight the need to do more than just scrap and rework our Electoral Register. If there is the potential for such abuse and manipulation of our Electoral Register then there is a further compelling case for a root and branch review of the processes for collating, maintaining and updating our Electoral Register to improve the accuracy of the register, reduce the scope for malicious abuse of the register and ensure that the information on the register meets or exceeds our expectations.

The revelation during the week that different Local Authorities were using different format templates for capturing Electoral registration data apparently came as a shock to the Minister. I would ask WHY? The report by the C&AG into the PPARS implementation cited divergences in work practices and basic formats for information capture as being a key element in the Information quality issues that contributed to the massive cost over runs there.

This is a symptom of the same issue -divergence of work practices arising from slio’d processes and an absence of co-ordinated governance and control.

The roadmap to a solution is simple – assess processes, seek root causes, use statistical data to measure and prioritise root causes, refine processes to address root causes and institute leadership and governance to ensure quality and a focus on continuous improvement of process and information quality to meet the needs and expectations of the electorate.

If certain political parties are taking advantage of control deficienies in the Electoral Register process, then that is disturbing but unsurprising. The solution is not to foist blame for the overstatement on to these parties but to refine the processes to improve the controls and quality of the processes and to protect our democracy.

 Minister Ahern… yup, them’s the droids we are looking for.

 For any media persons looking for a piece based on my posting to date – please leave a request in the comments and I will respond.

For anyone looking to learn more about Total Information Quality Management, check out the IQ Master Class on Friday 5th May at ICTEXPO

An aside…

http://uk.news.yahoo.com/28042006/344/fallout-slur.html

In the snippet above, Irish Minister for the Environment Dick Roche ( see Electoral Register posts below) suggests that the portrayal in a TV drama about the impact of an incident at Sellafield on the east coast of Ireland was “bizarre” because it suggested that there would be riots on the streets of Dublin.

Yes minister. That is bizarre. It has never happened before…

http://www.indymedia.ie/article/74507