Basic Income Pilots: Uses, Limitations and Design Principles

2.1 The Pioneering Pilot

Although there were negative income tax (NIT) experiments in the United States in the 1970s, the first pilot in the spirit of basic income was launched in 1975 in the province of Manitoba, Canada, including a ‘saturation’ variant in Dauphin, a small town. The Manitoba Basic Annual Income Experiment (Mincome) is notorious for the fact that, after it was ended prematurely due to a change of government, all the evaluation data were lost. Nearly 40 years later, they were discovered in the national archives in 1800 dusty boxes.^[4]

The payments were targeted at recipients having an income below a pre-specified threshold, so this was not a true basic income pilot. However, the data clearly showed that receiving a guaranteed income led to improved health, with fewer hospital visits, accidents and injuries, including from domestic violence, and a considerable ‘reduction in physician claims for mental health disorders’. This suggested that basic income would ‘improve health and social outcomes at the community level’ (Forget, 2011)

There was also a peer-group effect in education. Youths in recipient families, especially boys, were less likely to drop out of high school following introduction of the guaranteed income – but so were their classmates from non-recipient families. Recipients felt a greater sense of economic security and the scheme de-stigmatised income assistance, because it was seen as for everybody; anybody whose income fell below the threshold could receive it, with no other conditions (Calnitsky, 2016). The community-level benefits demonstrated the desirability of designing basic income pilots to cover everyone in a community rather than just a sample of individuals. Spill-over and communal effects matter.

2.2 Recent Demonstration Projects, Experiments and Pilots

Particularly since the onset of COVID-19, there has been a surge of interest in basic income and a corresponding surge in the number of experiments and pilots. By early 2021, no fewer than 43 US cities and towns had announced plans for some sort of experiment. In the UK, the councils of 32 cities and towns had voted in favour of doing pilots and requested government approval and assistance. Experiments described as pilots had recently been completed in Barcelona, Finland and the Dutch city of Utrecht, while there were ongoing experiments in Germany and Spain.

In Africa, there have been completed pilots in Namibia and Uganda, with an ambitious 12-year pilot underway in Kenya. In Asia, there have been completed pilots in Madhya Pradesh and West Delhi, a proposed scheme in Odisha, and schemes that merit inclusion in Mongolia, Hong Kong, Iran, Saudi Arabia, Macao, Japan and South Korea. In Latin America, Brazil has been the centre for small-scale pilots and the town of Maricá is paying a modest guaranteed income that reaches 42,000 residents, a quarter of the population. And the Renda Cidada introduced in 2020 for the pandemic was a very popular and socially helpful policy until abruptly ended by President Jair Bolsonaro.

A cursory review suggests that most of the initiatives in the USA and Europe are best described as demonstration projects or experiments. One concern is the focus on whether they are achieving poverty relief, while depending on charity and enthusiastic volunteers. A bigger danger is that they will allow policymakers to divert attention away from the ethical principles and to hope that interest in basic income will fade away with ‘economic recovery’. The challenge for the basic income community is to prevent that happening. It is therefore most encouraging that during the pandemic various opinion polls have shown that, for example, over two-thirds of adults in France, Germany, Italy, Poland, Portugal and Spain support a long-term basic income system, replicating equally strong support in earlier polls.

The remainder of this paper considers what would be closest to the ideal design for basic income pilots. There is no ideal, but we can surely learn a great deal if the design principles are respected.

2.3 Guidelines for Useful Pilots

We can identify 19 principles or guidelines that should be considered and if possible followed in any pilot. They fall into three groups: embedding key characteristics of BI; designing the pilot sample and testing methodologies; and evaluating the outcomes. Failure or inability to follow strictly all the guidelines would not necessarily invalidate the results, but that possibility should be considered at the start.

3.1 The Basic Income Payments Must be Proper Basic Incomes

For a pilot to test a true basic income, the payments should have the following features:

1. The amount paid should be basic, that is, sufficient to be meaningful for the recipient but not so high as to provide total security. There is no ideal level, but it should be an amount that is regarded as reasonable and fiscally feasible in the country. It is fair to say that many of the pilots and experiments have met this most basic of criteria.

2. The income should be paid in cash, or in a form easily convertible into cash, via a bank account, debit card or mobile phone, for example. The payment should be regular, predictable and stable, preferably paid monthly over a sustained period. And it should also be non-withdrawable, so participants know that they will continue to receive it for the full duration of the pilot.

3. It should not be paid as a lump sum, mainly because of the ‘weakness of will’ effect; people confronted by a large windfall may be tempted to ‘blow it’ on ill-advised ventures or otherwise squander it.^[5] The US charity GiveDirectly is currently testing lump-sum payments in Kenya to see if this promotes entrepreneurial activity. However, capital grants, which by their nature are one-offs, cannot replicate what happens with a regular and stable basic income.

4. The basic income should be universal, paid to all those usually resident in the pilot community at the outset. This will enable detection of community effects, including non-economic effects, that paying isolated individuals will not. Only a handful of pilots have complied with this principle. Apart from the Manitoba Mincome pilot mentioned earlier, they include the pilots carried out in Namibia in 2008–09 and Madhya Pradesh, India, in 2011–12, the aborted pilot in Ontario, Canada, in 2017–19, and the ongoing pilot funded by GiveDirectly in Kenya, launched in 2017. All of these (except Ontario where the data were suppressed) have shown significant and beneficial community effects.

5. It follows from the principle of universal payments that there should be no targeting. The basic income should not be given only to those deemed to be ‘poor’, however poverty is defined. Basic income is intended to be a right and rights are universal. This contrasts with targeting, which creates poverty traps because the loss of benefits as income rises is a disincentive to earning extra. Most so-called BI pilots and experiments have nevertheless adopted targeting, in what are really tests of a ‘guaranteed income’. Almost all the city experiments, in the United States and elsewhere in the world, have selected only participants with low incomes (indeed, most of the US experiments are part of an initiative called ‘Mayors for a Guaranteed Income’). Other pilots and experiments have made payments on a sliding scale so that the full payment is made only for those with incomes below a certain threshold and above that tapers to zero as other income rises. This was the case for the negative income tax experiments in the United States in the 1970s, the Manitoba Mincome experiment, the Ontario pilot launched in 2017 and abruptly terminated a year later, and the Barcelona B-Mincome pilot between 2017 and 2019.

6. It also follows that there should be no selectivity. Giving to one ‘deserving’ group rather than another, on the basis of age, ethnicity, family or work status, may put the selected group under pressure to share with others, diluting the effect of the basic income. Paying an income just to mothers, for example, risks generating tensions within households. The city of Columbia, South Carolina, is planning to make payments just to a small number of black fathers. Cambridge, Massachusetts is only paying single ‘caretakers’ with children. While these results can be expected to show improvements in recipients’ wellbeing, they will not tell us much about how basic income paid to all might work in practice.

A widely reported pilot in Finland in 2017–18^[6] also involved regrettable selectivity, though it did produce some useful results. Although the originators wished to test a basic income, applied to whole communities and to every resident, regardless of work status, this was whittled down by political and budgetary pressures. Instead, a basic income of €560 a month for two years was provided to 2000 unemployed people from January 2017 to December 2018, selected from across the country. The final evaluation found that, compared with the control group, recipients were more satisfied with their lives, felt better about their income and economic wellbeing, experienced less mental strain, depression, sadness and loneliness, showed a greater degree of trust in other people and institutions, and were more confident in their own future and their ability to influence things.^[7]

However, to help obtain political acquiescence to the pilot, it was stated before it began that “the primary goal” was “promoting employment” (Kela, 2016). So, the pilot could be judged a ‘failure’ if employment was unchanged, even if the basic income clearly improved wellbeing and agency. In fact, the evaluation found that basic income recipients had on average slightly more days in employment than the control group, though the findings were complicated by the introduction of new ‘work activation’ policies for the unemployed in 2018. These findings, first reported in a preliminary evaluation, did not stop cynics from dismissing their relevance. An article in The Economist was entitled ‘A Finnish trail is as clear as mud’.^[8] What was unclear?

As mentioned later, interviews and case studies can make a valuable contribution to explaining the mechanisms behind the statistical results. The Finnish evaluation, for example, included interviews with 80 participants. One of the recipients, interviewed by a British newspaper, had used the time and money to build up a workshop for making and selling shaman drums.^[9] As reported, it was not the money that had made that possible but the lack of behavioural conditions that had previously forced him to look for jobs and use up time to satisfy the employment bureau’s demands. Another recipient, an artist, took part in the BIEN Congress held in Tampere, Finland in August 2018. He said in a statement: “We are all emotional beings as well as rational ones. My basic income payment was the day I became free – a signal I could dream of something better.”

The Left Alliance, in its manifesto for Finland’s 2019 General Election, included a commitment to a basic income of €800 a month, phasing out means-tested social assistance while leaving housing allowance unchanged. The Alliance gained votes and parliamentary seats and won two ministries in the subsequent coalition government, including the Ministry of Social Affairs and Health. The preliminary pilot results surely helped.

1. The basic income should be unconditional, without requiring some pre-specified behaviour. This principle has been generally observed in pilots and experiments.

2. The basic income should be paid individually and equally to each man and each woman. Basic income security is an individual right and should not be applied to some notion of the ‘family’ or ‘household’, whose size and composition may be affected by the policy itself. For children, or adults unable to collect their own income because of disability or frailty, a surrogate (the mother in the case of children) can be designated to receive it on their behalf.

One of the drawbacks of an overly complex pilot in Barcelona, dubbed B-Mincome, which ran from 2017 to the end of 2019, was that the money was provided on a family basis, with 1000 households selected from three of the city’s poorest districts to receive a maximum of €1675 a month. Preliminary results showed that, compared with the control households, BI recipients reported greater financial security, higher wellbeing, better mental health, less stress and more engagement with and participation in neighbourhood and community life (Laín, 2019). When launched, it was stated that the main objective was “to analyse the most effective way of reducing inequality and breaking the poverty cycle”, with a view to implementing a more permanent municipal programme. However, the pilot unwisely tried to test several policy interventions within the sample, clouding interpretation of the statistical results.

1. No other policy change should be made at the start of or during the pilot, to ensure a fair assessment of its effects. As mentioned above, introduction of new ‘work activation’ policies half-way through the Finnish pilot muddied the results on labour supply that politicians had wanted to be the primary focus.

A basic income pilot that respected some but not all of these criteria, perhaps because of resource constraints, might be acceptable to test the validity of certain features. But in evaluating and publicising the outcomes, researchers would need to make clear that the results did not reflect the effects of a full or proper basic income.

4.1 There Should be a Legally Binding Commitment that the Pilot will be Conducted in the Way Stated at the Outset to Completion

This procedural rule should seem obvious but was shockingly abused in the pilot launched by the provincial government of Ontario in April 2017. Some 4000 low-income individuals in the areas of Hamilton, Lindsay and Thunder Bay were to be paid C$17,000 a year, if single, and C$24,000 a year if they were a couple, with a supplement for those with disabilities. Recipients were told that the payments would continue for three years.

In August 2018, Ontario voted in a new right-wing government, which promptly terminated the pilot, despite having promised in the election campaign to continue it to completion.^[10] Many recipients had made long-term plans based on the reasonable expectation that they would continue to receive their ‘basic income’ payments. Adding insult to injury, the Progressive Conservative government disbanded the evaluation team and called in all the data that had been collected, preventing statistical analysis of the results and denying information to the public on how the pilot had been working. That gap has been partially filled by more anecdotal evidence, by journalists who interviewed participants before the pilot was cancelled,^[11] by the Basic Income Canada Network in a survey of over 400 participants after news of the cancellation (BICN, 2019), and by a survey of 217 former recipients after payments ended in March 2019 by academic researchers at McMaster and Ryerson universities (Ferdosi et al., 2020). Inevitably, these survey results lack the statistical weight that would have come from data for 4000 recipients over a three-year period. Nevertheless, they suggest potentially life-changing benefits.

In the university survey, 80% reported better physical health while taking part in the pilot, 83% said they were less stressed or anxious, 81% said they felt more self-confident, and 66% said they got on better with family members. Many said they were more financially self-reliant, less dependent on help from family and friends, and able to pay down debts. They also socialised more, participated more frequently in extracurricular activities, and did more volunteering. Of those who were in jobs at the start of the pilot, three-quarters continued in employment, with a third of those saying they had moved to a higher-paying job. Of those who stopped working, about half went into education or training, some took the opportunity to leave a bad job and search for a better one, and others quit their jobs to care for a family member or because of their own disability that made paid work difficult.

In the midst of the furore around the pilot’s cancellation, Canada’s Prime Minister, Justin Trudeau, and the social development minister, Jean-Yves Duclos, said in interviews that a guaranteed national minimum income was an option in helping Canadians adapt to an unstable labour market. Again, it shows how pilots can have political outcomes. Trudeau’s Liberal Party, the opposition New Democratic Party and the Greens all now back some form of basic income.

4.2 The Pilot Design Must be Clear and Sustainable

It is easy to overlook the need for clarity about the design of a pilot and its sustainability, that is, why a particular design is adopted rather than alternative feasible designs. This applies to all the key aspects of the pilot, including the basis for deciding on the amount of the basic income and the duration of the pilot. All the design features should be recorded at the outset. To ensure sustainability, there should also be a clear workplan and budget recorded in writing before the pilot starts.

The design should avoid over-complication and capture by diverse interests with separate research agendas. The much-touted pilot set for Oakland, California, with considerable funding from Y-Combinator, was eventually abandoned after so many established economists and other social scientists became involved in ‘advisory’ capacities that the complexity of design became enormous. The management team later announced that the pilot would be relocated to two regions of two other states, but at the time of writing this article there was no sign it had started. Ironically, the most tangible outcome of what began as the Oakland pilot project, which was rebranded ‘Making Ends Meet’, is that the core team produced a technical guide on basic income pilots – before they had done one (Stanford Basic Income Lab and National League of Cities, 2018).

4.3 The Design Must be Kept Constant

Once a pilot has been launched, its design should not be altered unless it becomes essential to do so. In such cases, the evaluators should make it clear what change has been made and give the reason for it. For example, if there is considerable consumer price inflation during the year or two of the pilot, there may be justification for raising the monetary amount so that the basic income represents a constant amount in real terms. This was done in the Madhya Pradesh basic income pilot (Davala, Mehta, Jhabvala & Standing, 2015).

4.4 The Size of Pilot, in Terms of Number of Recipients, Should be Adequately Large

A very large pilot cannot be sensibly managed, but if it is too small it will not produce reliable statistical results and better fits the description of an experiment or demonstration project. The point of a pilot is to identify statistically significant differences between those receiving and not receiving a basic income that are unlikely to have arisen by chance. The minimum size of the sample will depend on what hypotheses the pilot aims to test and the significance level chosen (the probability that the results could reflect the ‘null hypothesis’ – that the basic income made no difference). This is most commonly set at 5%; in other words, there would need to be at least a 95% probability that the results shown by the pilot reflect the basic income. But the calculation of statistical significance also depends on sample size and the results will not be reliable if the sample is too small.

Pilots will often want to test hypotheses that need to differentiate between sub-groups in the overall sample. Behavioural and attitudinal changes may be tendencies; some people change, others hesitate, and others do not change. The pilot might want to identify the characteristics of those who change and those who do not. Often the hypothetical effects are relevant only to some of the people in the selected sample (teenagers, people with disabilities, etc.). It is essential to have enough people with the potential characteristics of interest to estimate effects (or lack of them) with confidence. As a general rule, the minimum sample size should be 1000 individuals, and a bigger sample would be preferable.

4.5 The Duration Should be Greater than One Year, if Possible

Obviously, the duration of the pilot should not be very short, for then it would be hardly more than a one-off capital grant. The effects of a basic income can be expected to manifest themselves over time, as people learn and adapt. There may be an ‘impact effect’ immediately after the grant is first received, and an ‘assimilation effect’ as individuals become used to receiving the basic income. There may also be a ‘wearing off’ effect in some respects and a ‘learning effect’ in others. For these reasons, the duration of the pilot should be more than a year. Two years is reasonable, though some would argue for longer.

One practical consideration is project fatigue, easily ignored in the first flush of enthusiasm. However, any pilot scheme requires regular evaluation and the construction and maintenance of a team of researchers and fieldworkers. There is also respondent fatigue, since evaluations involve asking respondents’ questions or watching them in some way. In this respect, as in several others, the best may be the enemy of the good.

This may prove to be the case in the much advertised and well-funded pilot GiveDirectly is implementing in Kenya. One of the three variants it is testing involves provision of a small basic income to 5000 villagers in 44 villages for a period of 12 years. Those advising the project are well-established US academics with no known experience of running basic income pilots. One can predict with confidence that some will have retired before the pilot is completed, and others will have taken their consultancy money and moved on.

Despite such reservations, one unique long-term ‘accidental’ pilot in North Carolina deserves more attention than it has received. Shortly after a long-term longitudinal study of child development in the area was launched, the local Native American community decided to convert the profits of its casino into basic income payments for community members. This enabled researchers to compare children in families receiving the basic income with other children enrolled in the study. Children in ‘basic income’ families did much better in school and were less likely to be involved in petty crime, due primarily to better family relations attributable to less financial stress. Although the casino pilot is scarcely replicable (see below), it showed long-term feedback effects that should be incorporated into any sensible estimation of the cost of a basic income. Claims that basic income is unaffordable have singularly omitted such effects as savings on health spending or higher tax revenues from better-paid jobs. This pilot demonstrates that they should do so.^[12]

4.6 A Pilot Must be Replicable and Up-Scalable

Pilots should be designed to be replicable and up-scalable. In other words, it should be possible to conduct a similar pilot somewhere else and it should be feasible in principle to scale up the pilot to a larger community and to national level.

Many of the recent experiments have been demonstration projects that are not really up-scalable. This probably applies to the small-scale schemes being implemented or planned in a number of US cities under the banner of ‘Mayors for a Guaranteed Income’. As of early 2021, 43 cities had signed up to the initiative, founded in 2020 by Michael Tubbs, then mayor of Stockton, California, and the privately funded Economic Security Project, which sponsors basic income projects in the United States. As a new young mayor, Tubbs launched a ‘basic income’ trial in Stockton in February 2019, which gave 125 low-income residents $500 each month for two years, no strings attached. The $3 million initial funding for the trial, named the Stockton Economic Empowerment Demonstration (SEED), came from foundations and individuals, including a $1 million grant from Facebook co-founder Chris Hughes, co-chair of the Economic Security Project.

The individuals provided with the basic income had to be at least 18 years old and live in a neighbourhood in the run-down city with an average income below the city median of $46,000. The sample and a control group were selected randomly from 1000 applicants who responded to the invitation to participate sent out to 4200 randomly selected residences. The preliminary evaluation, covering the first (pre-pandemic) year of the experiment, showed as expected that recipients of the basic income reported less pain, anxiety and fatigue, spent more time with their children, and felt more in control of their lives than the control group (West, Baker, Samra, & Coltrera, 2021). But one interesting finding was a big (and statistically significant) increase in full-time employment, from 28% when the payments began to 40% a year later. Full-time employment among the control group also rose, but by much less, from 32% to 37%. Interviews with recipients showed the basic income had enabled people to find better jobs, for example, by quitting second or third jobs to gain qualifications or time to go for interviews, or by having the wherewithal to buy interview-suitable clothes, pay for transport or arrange childcare.

Mayor Tubbs said at the start of the project: “The most important thing the pilot can do is spark a conversation that might translate down the road into state or nationwide change.” And it can certainly be said that the Stockton experiment, which has been extensively reported in the USA and around the world, has succeeded in that aim, aided by Andrew Yang’s run for the Democratic presidential nomination in 2020 and the experience of and responses to the COVID-19 pandemic. Local demonstration projects of these types, including those financed by private donors, can thus serve a useful purpose in helping the public conversation and legitimation of more ambitious schemes.

Among other highly targeted experiments with unconditional cash transfers in the USA is the project run by the Magnolia Mother’s Trust, which pays just over 100 African-American single mothers in Jackson, Mississippi, $1000 a month for 12 months. A more rigorous study was launched in 2018 in four American cities that pays 1000 low-income mothers an unconditional $333 per month for the first 40 months of the child’s life. A control group is being given $20 a month. The study is called ‘Baby’s First Years’, because the focus is on the expected link between basic income security in early life and brain development.

The primary hypothesis is that the steady stream of income will make a difference to the cognitive and emotional development of the children whose mothers receive it. A professor of neuroscience connected to the project who had been studying the links for over 15 years wrote, “Cash transfers, as opposed to counselling, childcare and other services, have the potential to empower families to make the financial decisions they deem best for themselves and their children.”(Noble, 2017).

Developing this project took a long time, largely because the project team had to raise the funds and obtain approval for it through new legislation in the two states where it is being conducted. These are issues almost all pilots must resolve at the planning stage.

4.7 Random Control Groups Should be Used

Determining the impact of a basic income requires comparing outcomes, both with the behaviour and attitudes of people beforehand (before the ‘treatment’ began) and with ‘control groups’, otherwise similar, who do not receive the ‘treatment’ or come under the policy. This is to ensure that outcomes are the result of the basic income and not a change coming from the outside.

While randomised control trials (RCTs) are currently in vogue, they have severe limitations when it comes to evaluating basic income. The principle behind RCTs, as the name implies, is that those receiving the ‘treatment’ should be selected ‘randomly’ from a wider population, and the control group should also be selected ‘randomly’. This would typically mean listing households in an area and drawing a sample, so that one family (or individual within a family) would receive the basic income while the family next door would not.

Apart from the risk of generating inter-community and inter-familial tensions, one likely outcome would be some sharing of the basic income between households and between family members, on a non-random basis. This would negate the universality of the basic income, dilute its impact and make it hard to evaluate outcomes. In the Indian pilot in Madhya Pradesh, 18 similar villages were selected randomly from a wide area: in eight of these, everyone received the basic income and in the remaining 12 villages no one did. It was thus possible to explore a range of effects, both at micro-level and community level, including attitudinal and behavioural changes.

5.1 There Should be Baseline Surveys

For a proper evaluation, a baseline census or survey of the community is needed to provide detailed information on the prospective respondents, covering all aspects likely to be assessed by the pilot. (If a census is not practicable a large sample should be chosen.) This should be complemented by a baseline community survey that collects data on structural characteristics of the community – demographic breakdown, infrastructure, distance to schools and clinics, and so on. The idea of both baseline surveys is to identify the conditions, behaviour and attitudes that exist before the impact of the basic income.

Preferably, the intended recipients of the cash transfers should not be informed of the plans at the time the baseline survey data are gathered – although of course cooperation in the survey is more likely if they do know!

Since the evaluators will not know in advance what effects the basic income will have, and cannot realistically anticipate all of them, the baseline surveys need to collect a rich array of data. They should be conducted about one month before the first grant pay-outs, to capture the patterns prevailing at the time of the launch.

One of the key decisions for the baseline census is the selection of respondent or respondents. Asking, say, the nominated ‘household head’ is likely to elicit different responses on some issues from those that would be given by another household member. Bearing in mind the individual nature of a basic income, responses can be expected to differ by gender, age and other personal factors.

The ideal would be to collect background factual data on the household and individual data from each member separately. This may be impractical and too costly in terms of fieldwork and data processing. However, the number of women selected as respondents should be equal to the number of men. The Madhya Pradesh pilot interviewed one male and one female respondent per household.

After the baseline survey, it is advisable to conduct a public information campaign to explain what will happen in the course of the pilot, to overcome suspicions, and to make sure respondents know what will be involved, including the fact that they will receive the payment regularly, without spending or behavioural conditions.

5.2 There Should be Regular Evaluation Surveys

The idea of a pilot is to test effects, and to do so it should build in a series of surveys, starting with the baseline survey. Another survey should be conducted about six months after the start of the pilot, allowing enough time to detect an impact on behaviour and attitudes, and further interim surveys every six months or so, depending on the length of the pilot. At the end of the pilot, preferably within a month of the last cash transfer, there should be a final evaluation survey covering the same individuals and households questioned in the baseline survey. In each case, the questionnaire should be as close as possible to the one used in the baseline, with similar reference periods.

It is important that the survey questions are designed in a way that does not prejudge or influence the answers, otherwise respondents may answer in the way they think is wanted. It is also important that questions are answerable. This may seem obvious, but it is a frequent pitfall, for example, when respondents are asked to recall events or attitudes some time in the past. Another issue for pilots is the extent to which recipients can be publicly identified, which could change the way they respond to the basic income. The SEED project in Stockton dealt with this issue by separating a 100-strong anonymous ‘core’ for analysis from 25 recipients who agreed to talk about their experiences to the press. In the Madhya Pradesh pilot, the test villages were not identified publicly to prevent outside influence.

5.3 There Should be Legacy Evaluation Surveys

One methodological tool too rarely used in basic income pilots – or in other social policy experiments – is the follow-up survey, to assess the lasting effects of what is inevitably a short-term scheme. The Madhya Pradesh pilot conducted not only a post-evaluation survey six months after the basic income payments ended, but also a legacy survey three years later, designed to see what observed effects of the basic income continued and where there was recidivism. Identifying such legacy effects can be important in considering what other measures are needed to prevent recidivism or to ensure that beneficial effects persist.

5.4 Key Informants Should be Used

While the focus of attention should be on the individuals receiving the basic income, it is desirable to involve ‘key informants’ in the evaluation process. Extra information should be collected from local authorities, teachers and medical personnel, and others with knowledge of the community. They should be asked for information that is not available from recipients and for information on the way a basic income might influence behaviour. A good pilot evaluation should build on a structured questionnaire addressed to key informants with questions expressed in a neutral way.

If the pilot covers several communities, the areas chosen should be structurally similar, particularly for comparing basic income areas and control areas. But the evaluation design should also take into account possible exogenous changes during the course of the pilot that may change the structure of a community so that it is no longer similar – the building of a new school, or the introduction of an irrigation scheme, for example. For this reason, it is desirable to conduct a simple community survey at each evaluation point to check on structural changes that may influence outcomes.

5.5 Analysis Should Address Multi-Level Effects, including Feedback Effects

The pilot should include adequate techniques to assess and evaluate effects on individuals, on households and families, and on the community, such as economic multiplier effects. Looking solely at effects on individuals can be misleading.^[13]

Some of the richest effects may be at the community level. If they are not studied at the same time, an evaluation may conclude that, because the effect at individual level was good or bad, basic income is good or bad. There may be feedback effects at community level on how individuals behave and interact with one another. One of the claims made by advocates of basic income is that it would foster altruism and social responsibility within communities. Results from the few pilots that have studied community effects appear to support these claims.

It may also be desirable to look at what happens in nearby communities which do not receive a basic income. The experience of GiveDirectly’s various projects in East Africa is informative here. One large-scale village-based cash transfer project in an area of Kenya near Lake Victoria generated a ‘fiscal multiplier’ of 2.7 (Egger, Haushofer, Miguel, Niehaus, & Walker, 2019). Each dollar of cash paid out generated $2.7 extra dollars in income for the wider area, as recipients spent their money in local shops and businesses, also raising incomes in nearby villages that did not receive the cash transfers. However, an earlier project, also in Kenya, found that giving cash to individuals on a randomised basis helped the individuals but adversely affected the wellbeing of their neighbours who did not receive cash.^[14] Such negative spill-over effects from randomised trials have also been found in studies of cash transfers elsewhere, underlining the need for basic income pilots to make payments universal.

5.6 Evaluations Should Cover Both Attitudinal (Psychological) and Behavioural Effects

A basic income can be expected to affect behaviour, physical and emotional ‘wellbeing’, and attitudes. So a pilot must aim to look beyond directly measurable behavioural effects and include attitudinal questions, posed in as neutral a way as possible. To permit numerical evaluation, respondents should be asked to choose from a range of five possible answers on a ‘likert scale’ (e.g. (1) strongly disagree, (2) disagree, (3) neither agree nor disagree, (4) agree, (5) strongly agree).

5.7 Hypotheses Should be Made Explicit Before the Pilot is Launched

There should be a clear list of hypotheses to be tested, established before the pilot is launched and before the baseline survey is carried out. Too often pilot schemes are launched with only vague ideas of what to expect, perhaps because they are implemented under pressure or as a result of a conflicting array of claims and counterclaims. Another problem, less often noted, is launching a pilot with only one or two hypotheses to be tested (the impact on labour supply, for example), as has tended to be the case with RCTs. Consider a few standard hypotheses:

1. A basic income enables the household or family to provide children with more food, and this leads to better nutrition and better health.

2. A basic income reduces the pressure on the household to oblige children to labour and increases the probability of them attending school.

3. A basic income enables the household to pay off debt.

These hypotheses require benchmark data, from the baseline survey and baseline community survey.

Then consider another set of hypotheses:

1. A basic income granted to person X leads person Y to reduce the amount of time devoted to income-earning activity, and/or leads person Y to alter his/her consumption.

2. A basic income scheme leads to the establishment of a local group to advise recipients on how to spend their money.

3. A basic income scheme leads to social community pressure on recipients to share with non-recipients outside the community.

These are indirect and external effects that may require information from the recipient, from the household and from outside the household. While this has implications for the design of a baseline household survey, it also requires a community-level benchmark and monitoring survey, and matching household and community evaluation surveys conducted over the period of the pilot.

Then there are hypotheses that relate to the effects on the local economy and local society, such as the following:

1. A basic income scheme leads to an improvement in income distribution, lessening income inequality, and it does so by more than a simple addition of the cash transfer.

2. A basic income scheme leads to the creation of local financial agencies, increasing the extent of financial intermediation.

3. A basic income scheme leads to the development of local businesses and more employment in the community.

Hypothesis (g), for instance, is crucial to a proper evaluation of a basic income system. If the pilot design cannot measure the impact on income distribution, it will fail to evaluate a key aspect of basic income. However, this crucial hypothesis cannot be tested at all if only a minority of the community are given the basic income. It is again essential that the basic income is paid to all residents.

5.8 Cost and Budgeting Must be Realistic

It is important to devise realistic cost estimates at the outset of the pilot. Proper evaluation requires decent funding and technical expertise. And besides costing for the basic incomes themselves, for administration and for the evaluation, some money must be set aside for contingencies. In any empirical study involving surveys, the only certain rule is that mistakes will occur, not necessarily because of errors in the design or conduct of the pilot but because inevitably there will be unexpected events and setbacks.

5.9 The Sample Must be as Constant as Possible

It is an essential but hard-to-maintain principle of a basic income pilot that the sample must be as constant as possible. However regrettable from a social point of view to deny a basic income to a newcomer, nobody not covered at the outset of the pilot should be subsequently included, with the exception of new babies. Migrants or returnees who enter the community after the start cannot be included. However, it is desirable to include them in the survey, since their presence may have effects on attitudes and behaviour that should be taken into account.

Out-migration is another issue. The solution will depend on the hypotheses to be tested. In the Madhya Pradesh pilot, people who moved away no longer received the basic income. However, the GiveDirectly pilot in Kenya will continue to pay the basic income to out-migrants present in the basic income villages at the start, because one hypothesis to be tested is the incentive to move to find paid work. Obviously, in a prolonged pilot there will also be some deaths. So, keeping the sample constant will rarely be entirely maintained. The point is to try to come as close to it as possible.

5.10 The Transfer Mechanisms Must be Monitored

Particularly in developing countries, one of the biggest challenges for any cash transfer scheme, including basic income schemes, is the lack of sophisticated financial institutions and widespread ignorance of banking. Cash transfers inevitably involve a learning function, and institutional failure can distort the effects on behaviour and attitudes (for example, if it is difficult to access cash paid into a bank account). In Madhya Pradesh, the majority of villagers did not have bank accounts. But in rich countries too, many low-income people may not have or use a bank account. In Stockton, for example, nearly 10% of the adult population do not have a bank account.

There are several methods for distributing cash transfers, none of which is entirely adequate. The important issue is to make sure people receive the money directly, without the involvement of an intermediary. However, bearing in mind the need for replicability and up-scaling, another question is: What option would be the most desirable and most easily replicable at national level, in terms of cost, transparency and user-friendliness?^[15] Pilots are a good way of answering such questions, and could save a great deal of public money if a basic income system were introduced nationally.

While the method chosen will depend on local circumstances, most pilots will need to make provision for costs and time involved in enabling people to receive the money and ensuring that the money can be accessed quickly and easily. In Madhya Pradesh, recipients were helped to set up bank accounts; in Stockton recipients were given debit cards that were then loaded with $500 each month; in GiveDirectly’s Kenya pilot, money is transferred through a mobile phone payment system. For monitoring purposes, however, all transactions should pass through the banking system, as may also be required by law.

5.11 Agency or Voice Effects Should be Considered

No sensible advocate of basic income believes it to be a panacea. Many vulnerable people given a basic income would remain vulnerable and liable to be oppressed or exploited. To combat that vulnerability, they need agency, the capacity to exercise effective ‘Voice’ in their defence. This writer has long taken the view that basic income will work optimally only if those receiving it have both individual agency and some form of collective Voice to defend their interests. Basic income pilots could show what type of agency should be encouraged and supported to minimise the vulnerability of basic income recipients.

A pilot social policy project should always take account of agency effects. Agency or Voice may exist prior to the experiment or may emerge during and/or as a result of the intervention. In the Madhya Pradesh pilot, SEWA – the Self-Employed Women’s Association – was already present in some of the recipient and control villages, making it possible to compare SEWA and non-SEWA villages in both groups. In the Namibian basic income pilot, within months of starting, the villagers formed a committee to advise on good use of the money and to defend the more vulnerable against anybody trying to take advantage of them. To what extent did that affect the outcomes? The evaluation team had the strong impression that the impact was positive. But it was unable to tell how important it was.

Other pilot cash transfer schemes have picked up a positive effect of agency. For instance, a study in Nicaragua found that in areas with a relatively high proportion of ‘community leaders’, the effects of conditional cash transfers were stronger (Vakis and Macours, 2009). That is only one type of agency that could be measured in a pilot. Others should be taken into account in the selection of sampled areas and in the design of the evaluation instruments.