Double counting can be one of the most problematic data quality issues in a development programme. While it’s important not to lose sight of the bigger social change we are trying to create, this issue is at the heart of accurately measuring indicators that are important to many donors. This blog post follows on from a previous post on data quality more generally. It takes a practical look at how to prevent double counting and lists some ways in which others have avoided it. It explores which of these options are best suited to your programme.
What is double counting?
Double counting relates to measuring how many unique people, organisations, sites or other ‘things’ your programme has helped in some way. Let’s look at some example indicators to illustrate the problem:
- Number of mothers receiving ante-natal care
- Number of households surveyed
- Number of organisations provided with technical assistance
- Number of sites offering health services
- Number of people receiving cash transfers
Often assumed (but not always clearly stated) with indicators like these is that we mean ‘unique’ number of X. Note with the examples above that double counting can relate to any 'thing' that you are counting in your programme.
Why is this a problem?
Double counting can be problematic in two situations. First, if you are measuring indicators where you must be sure that you are counting how many unique 'things' your programme has helped. If not tackled in your monitoring processes, you risk inflating your numbers and producing in-accurate data.
Second, if you are delivering services where you must be certain that each person only receives the service once. For example, in the context of health-care it could be dangerous if the same person received the same medication twice. Or with social assistance programmes, you are open to fraud if the same person can receive the same payment twice or more.
How does double counting happen?
This depends on what you are measuring. Let’s take another example to illustrate what can happen.
Number of people reached with educational messages
To measure this indicator you need to consider the following things in your monitoring system:
(1) What is our definition of an educational message? - This will depend on your programme and should link to how providing people with educational messages will achieve positive social change. Let’s assume that your programme is disseminating educational pamphlets, organising public viewings of educational videos and broadcasting educational messages on public radio. Our monitoring and evaluation plan will then use this definition.
(2) What tools will we use to track how many people are reached by each educational message? - This will again depend on the types of messages you are sending and by which channel. For this example we will need forms for each of the following:
- How many people received pamphlets?
- How many people attended the video viewing?
- How many people heard the radio broadcast?
(3) Who is measuring this indicator? - In this example, different people might be responsible for collecting this data. Perhaps there is one field worker who organises the video viewings and hands out pamphlets? Maybe other staff also give out pamphlets during their own meetings. Meanwhile someone in the office organises the radio broadcasts, working with different radio stations?
There are three types of double counting that could happen in this example.
Double counting over time
Imagine that the field worker organises a video viewing in the same village on both Friday and Saturday. She completes the form to note down how many people attended each viewing and how many pamphlets she gave out.
If the same person attends both viewings or takes more than one pamphlet they will be counted as two.
Double counting across sites
Now imagine that the field worker organises the same viewing in a neighbouring village. Again, she notes on the form how many people attended each viewing and how many pamphlets she gave out.
Again, if the same person attended viewings in both villages they will be counted as two.
Double counting across partners
But what if this is a large programme with four partners? Each are running the same activities, just in different parts of the country. As with double counting across sites, two partners may both assist the same person and count them separately. This results in double counting at the programme level.
How do we prevent double counting?
There are two ways to prevent double counting. The first is simply to modify your indicators to accept that there is a risk of double counting. Instead of counting unique number of X you instead count total number of X. In the above examples this may be the best approach. Reaching people with educational messages (via radio, video or pamphlets) does not necessarily mean that they will read them. Or if they do read them that it will result in any change in their actions. This type of indicator is helpful to assess the overall scale and progress of a programme, but is not particularly helpful in learning if you are achieving your goals.
If you are measuring indicators where you do need to prevent double counting you must have tools to track which unique people, sites, organisations or other things you are counting. This is clearly considerably more work and hence should only be used in cases where the effort is worth it.
Unique identification and registration processes
Tracking unique people or other 'things' in your programme requires several changes to the examples above. First, the programme must agree on which criteria will be used to identify people. More on this shortly. Second, the programme must implement a 'master list'. This provides a definitive list of people, sites or whatever is being measured. Third, the programme must agree on a registration workflow. This is a process whereby site level staff can review the master list and add new people to it if necessary. Finally, in most cases, a data audit is also needed to check for cases where duplicates do make it onto the master list.
Changes at the site level
Let’s return briefly to our earlier example. The field worker has a form that she uses to record how many people attend the viewing and how many receive a pamphlet. This form would now need to be modified to list which people attended the viewing and received the pamphlet.
However, it would also need to include a ‘unique identifier’ to ensure that we can identify if the same person attended a later viewing or received two pamphlets. She would also need access to the master list of people, to check if they were registered or not. Finally, she would need a registration process to collect information on new people who are not on that list.
Multi-site or multi-partner programmes
In larger programmes working with many sites or several partners there are two additional challenges. First, the procedures used to monitor activities and register people need to be the same across each site and partner. Second, each site needs instant access to the master list so they can see people registered on other sites.
Types of unique identifiers
We mentioned earlier the concept of a unique identifier for each person, site or other thing. What options do we have for this?
Broadly speaking there are two:
(1) Existing standards - In some cases other organisations (typically government, United Nations or other NGOs) will have agreed and implemented an existing standard that you can use.
Here are some examples:
- National identification numbers for people
- Registration numbers for NGOs and companies
- Code numbers for hospitals, schools and other state facilities
- GPS coordinates for site or organisation locations
- IATI codelists for organisations and projects
If these exist, are used widely and are reliable then you should strongly consider using them. Consult with others doing similar work to see what they are using and if it makes sense for you to use the same approach. However, there are many situations where these do not yet exist or are not feasible to use.
(2) Custom options - Where you cannot use existing standards you must implement your own. If you need to develop your own unique identifier, here are some options you can draw from. Each have advantages and disadvantages.
|Unique identifier||Type||Accuracy||Set-up cost||Running cost||Comments|
|Programme specific number||All||High||Low||Low||This is a number of code devised by the programme itself. Can be hard for participants to remember programme specific numbers.|
|Biometric - Voice recognition||People||Varies. Some studies quote 80%||Medium||Low||Raises privacy issues for people from marginalized groups. Limited examples of use at scale in international development|
|Biometric - Voice and facial recognition together||People||99%||Medium||Low||Close to 99% accurate when used together. Minimal requirements fo data audit to review low confidence matches. Raises privacy issues for people from marginalized groups. Limited examples of use at scale in international development.|
|Biometric - Fingerprint recognition||People||99%||Medium||Medium||Accurate field devices can be expensive|
|Biometric - Iris recognition||People||95%+||High||High||Field devices are very expensive. Only really suitable for use in a facility|
|Date of birth||People||Low||Low||Low||Not enough by itself as several people may share the same date of birth, must be used with another criteria. Some people may not know their date of birth.|
|Email address||People or organisations||Low||Low||Low||Many people involved in development programmes don’t have an email. Common also for people to share emails|
|Phone number||People or organisations||Low||Low||Low||Many people involved in development programmes don’t have a phone number. Common also for people to share emails|
|Smart cards||People||100%||High||High||Can be expensive to issue and requires custom hardware. Not really suitable to use in the field. Registration can only be done at a facility.|
|Unique identifier code (UIC) system||People||98%||Low||Low||Ideal for use on smaller scale programmes and/or with marginalized groups, who are uncomfortable sharing identifiable details. 6|
|National ID card||People||High||Low||Low||Great option if usage is widespread. May allow verification against government databases. Raises privacy issues for marginalised groups|
We'll write more in future blog posts about some of the biometrics options mentioned above. These are complex areas, but are well suited to larger programmes, particularly those distributing cash transfers or other payments. Of the options above, the Unique Identifier Code (UIC) system may be of particular interest. This works by creating a code drawn from the following information:
- First two letters of mother’s first name
- First two letters of father’s first name
- Gender (single letter M/F or number 1/2)
- Year of birth (last two digits)
Other variations exist using different criteria. The UIC offers the advantage of using unique and easy to recall criteria that are also confidential. This is critical when dealing with marginalised groups like injecting drug users, commercial sex workers or men who have sex with men. More on this approach here.
A statistical exercise determined that the likelihood that two or more individuals would wind up with the same code was less than 2%. However, in cultures using patronymic naming systems and/or polygamous marriages, the likelihood of duplicates rose.
Which option should I use?
It's impossible to make any recommendation without knowing more about your context. However, there are some criteria that can help narrow the choices. These are:
Why do you need to avoid double counting? If you are doing this to prevent fraud then the accuracy levels you need are higher, as presumably, is the budget you can afford to invest in systems. If you are monitoring only to measure indicators then cost is likely to be more of an issue.
What are you counting? As per the table above, the unique identifier you use will depend on if you are working with people, sites, projects or something else. If a national standard exists (and you don't need to worry about confidentiality) then using it makes sense.
What scale are you working at? As your programme gets larger the challenges grow. If you are working on one site with 20 people this is easy to manage. If you are working with tens of thousands of people across hundreds of sites (and possibly with multiple partners) you need to consider web-based technologies. Ideally these should include workflow to manage registration processes.
How will you register new people? Will new people be registered at the site level or will you have registration centres? What process will you follow to register new people on the master list? How will they be approved and shared with other sites? Some unique verifiers are dependent on certain infrastructure (smart cards for example) and would preclude site level registration in many cases.
Are you working with marginalised groups? As mentioned above, the question of confidentiality becomes important in this context. Consider the UIC in this case.
We hope that this post has helped to answer your questions on this important issue. More importantly that it provides some ideas for how to prevent double counting in your own work. If you know of other useful unique identifiers or helpful resources please mention them in the comments below.
Our experience in this area relates to both large-scale social protection programmes and monitoring citizen engagement processes. For more information on this work see the links below: