The DOBlog

I previously looked at the issue of the Electoral Register forms - the first point in the information chain that feeds the electoral register. On a whim this morning, I printed a copy out - my intention being to take a closer look at the structure of the form and its layout to see if anything there might contribute to problems in the Electoral register. Attached is a graphic showing what I found….  

The nice people over at the European Spreadsheet Risks Interest Group have an excellent website with good examples of spreadsheets gone wrong.

Here’s one about sorting errors…

And another… here

And another… here

and another… here

and another … here

and another… here (this one relates to electoral roll data)

and another… here

Some of these links might be broken now as some of the articles are a bit old but the EUSPRIG site has summaries of the stories.

My finger is sore from cutting and pasting links… if you want to see more on the risks of relying on spreadsheets, check out http://www.eusprig.org/stories.htm

I have been distracted from compiling my post on the use of excel as a data interchange mechanism by some other snippets of what Dick Roche said at yesterday’s Dail committee meeting as reported in the Irish Times (page 9) and Dublin’s freesheet Metro (page 5)…

1. The Census Enumerators haven’t gone away… €5 million to €6 million is being provided to Local Authorities to hire temporary staff “such as census enumerators” to carry out door to door inquiries. The Unions haven’t gone away either though and I suspect that Local Authority staff currently doing those jobs might be offended by this idea. And at the end of the day it is €6 million between 114 Local Government agencies (29 County Councils, 10 City/Borough Councils and 75 Town Councils) or an average of €52632 per agency to pay temporary staff for the duration of the scrap and rework. (that calculation assumes that Town Councils have a role to play in the Electoral Register - if Town councils can be excluded that amounts to just under €154k per Local Authority). Of course, this is still just scrap and rework - the process of gathering the information will use the same apparently broken processes to capture the new register. Once the cleanup is finished, unless there is a study of the root causes, the same defective processes will operate to corrupt the register almost immediately and taxpayers will find themselves having to fund another €6million clean up in the not to distant future.
2. The Minister has put forward a proposal to avoid voter fraud at the Polling stations in the next election. “Indelible Ink or fingerprint scanners could be used in polling booths at the next poll” (source: Metro). Of the ink, the Minister is quoted as saying “it woudl be a badge of pride that you had participated in democracy”. Allllllrrrrrriiiiiigggggghhhhhtttttyyyyy then.

My first recommendation is to ban CSI and its spin off shows from the Roche household. Capturing a fingerprint at the time of a ballot would be totally useless unless there was a master data source of citizen biometric data that could be referenced - even Gil Grissom has to have a finger print in the database before he can nab the criminal.

This is a proposal that I would suggest Digitial Rights Ireland jump all over quick smart as, in my view, this combined with the call to use PPS numbers brings us one step from a ‘big brother’ single view of citizen with biometric data. And as the Government seems to be incapable of properly managing complex technology projects I would be very concerned by this.

I’ll get my post on Excel done over the coming days - unfortunately pesky day job is getting in the way. ;(

The coverage of the Dail Committee meeting on Oireachtas Report this evening was a little disappointing. They only covered the Minister’s speechifying on the topic and didn’t show if there had been any debate or challenge on the topic from any of the Opposition politicians who sit on the Committee such as Labour’s Eamon Gilmore.

My feeling is that anyone who has been following my posts on this site and is familiar at any level with the scrap & rework concept would have found a few open goals in what Dick Roche was putting forward.

1. Use of ‘relevant databases’ within Local Authorities - this amounts to a process change to address a root cause (well done Minister). However there may be data protection issues given that data can only be used for the purposes for which it is captured - this fact was glossed over by the Minister. In addition, just because one has a second surrogate source for reality it doesn’t mean that it is any more accurate or complete than your ‘problem’ dataset. Furthermore, what tools do Local Authorities have available to them to actually match across these datasets quickly and accurately (short of manual efforts)? The Minister has been somewhat quiet on the ‘how’.
2. The Minister referred to using ESB databases to update the register on the basis that we all use the ‘leccy. Again, this is not a complete surrogate for reality as it would hold the details of only one, at most two, individuals at an address. That data may not match against the Electoral Register for a variety of reasons (different spelling of names, use of aliases or akas (e.g. John Vivian Smith may be listed on the ESB bill as J.Vivian Smith or just plain old Vivian Smith or just J Smith)) - Again Data Protection issues (and commercial considerations) exist here but were glossed over by the Minister.
3. Finally, Minister Roche is plowing ahead with the plan to give records of deceased persons to Local Authorities. The data will be sent in Excel format. There are issues with that idea that I will address in a later post (once I have gotten my thoughts on it ironed out) but in the meantime the question that arises is simply this - once the Local Authorities have their Excel spreadsheets of names and addresses of the deceased, how will they go about matching them against the Electoral Register and flagging matches for action? What is the process that will operate to convert raw data into actionable information?

This issue is a national crisis. The Government seems content to operate on a scrap and rework basis with no real thought leadership or strategy. It is all well and good to identify potential sources of data against which the accuracy of the register can be checked, but in the absence of clear processes to use that information or the capability to process it all that Local Authorities will have is more data and a Minister who’s approach has been to criticise and berate and to effectively manage by sound-bite. Provision of ‘ring fenced’ budget and 3rd party data sets will not enable Local Authorities to perform their duties re the Electoral Register any better if there is no understanding of what needs to be done to address these problems in the long term.

Where is the action plan to assess root causes and address the core deficiencies in the Electoral Register process? Where is the leadership in terms of thinking and in terms of control of the processes?

I wonder what the boys and girls over at Digital Rights Ireland would have to say about the ESB providing its billing database or customer relationship management database to EVERY Local Authority (in excel format perhaps)?

My suggestion of 2 weeks ago still stands…

1. Scrap the current register
2. Mail everyone on the current register and request that they re-register
3. Use that register as the appropriate data set for the Draft Register
4. In parallel assess Root Causes - involve Local Authority staff (not County Managers but Revenue Collectors, Librarians - anyone with a role in the process) to identify the root causes - even go as far as to talk to (dare I say it) Citizens to find out what is broken in the processes
5. Develop a plan to address those root causes, addressing governance issues and Information Architecture issues as well as cultural issues.

Scrap and Rework without corresponding review and improvement of processes will not solve the problems in the Register, it will merely postpone them until a later date.