JPAL Executive Education : Evaluating Social Programs

Do you know about Confirmation Bias ? People more likely to believe information that fits their pre-existing beliefs, but they’re also more likely to go looking for such information. Hence, we are stuck with wrong design of the development programs dooming millions of investment. So, we do not let evidence from the ground guide the policy? Unfortunately, it is hard to get clear-cut evidence of causality. Using evidence to guide aid and social work is crucial to ensuring the efficient use of limited resources. For years, policymakers have debated different approaches to helping the poor and now they have published research paper after a nine-year, six-country study, offers resounding evidence for a strategy that works. Proponents of randomized program evaluation argued more field experiments were needed to learn what worked.

There are also critiques like economist Angus Deaton who suspects that an average bureaucrat might take the results in controlled environment too serious. Any aid to poor really ought to be decided by democratic discussion  between stakeholders while RCTs are often done on the poor without any partnership is hardly an encouraging sign.

This led my interest in the executive education program offered by Abdul Latif Jameel Poverty Action Lab (J-PAL). I was among 31 executives selected for the course held on July 2015. This five-day program on evaluating social programs provided a thorough understanding of randomized evaluations and pragmatic step-by-step training for conducting one’s own evaluation. RCT measures impact of the program by comparing a treatment group to a control group, where the people who get the treatment are drawn randomly by lottery. There was emphasis on building theory of change for seeking an impact of the program.

I enjoyed great atmosphere of learning in those five days. Thanks to Sharanya Chandran, Dechen Zem, Diwaker Basnet, Chandra Shekhar Gowda, Rajeev Kumar and Rajesh Jain. And I learn a valuable lesson that ideas should be funded based on evidence that they work — not hope.

How to build a theory of change for an impact evaluation

SECC data is out.

As per Prof Cornel West, Democracy always raises the fundamental question: What is the role of the most disadvantaged in the public interest? The catch in the valid question lies in the definition of the 'Disadvantage Class/Caste'. The definition of "Who are they" varies among politicians, academics, and policymakers. Any major program in any sector can't be effective without a robust information system. The road to social justice for all should be based on solid data for understanding ground realities.

SECC data is out. This census takes caste into account for the first time in any such exercise since 1931. But the government is hiding OBC numbers. The previous government led by the Congress had also decided to conduct a caste-based census, but not to make the data public. The reasons for withholding this data can only be political.  Data on economic indicators, with special enumeration of SCs and STs, is done by the regular census as well. SECC was commissioned because many parties wanted similar numbers for the OBCs to pitch for political mileage. However, it is always tough to digest that SECC was done to mitigate the absence of large-scale, credible, and empirical data for public policy. But, it is a matter of time before the numbers are made public.

There was once a huge hue and cry over caste census as being labeled as divisive by upper caste-dominated groups. Prof. Kancha Ilaiah had explained this in his article: Who’s afraid of caste census? --- "Caste culture is all around us. In the dalit-bahujan discourse, the upper castes are being shown as constituting less than 15 per cent. This could be totally wrong. Even within the lower castes there are several false claims about numbers. Every caste claims that it is numerically the strongest and keeps asking for its “rightful” share. How to tell them that their claims are wrong? When caste has become such an important category of day-to-day reckoning it is important to have proper data at hand to tell communities that they constitute this much and cannot ask for more than their share. It is true that we cannot distribute everything based on caste. But caste census is the right basis for statistics such as literacy rate and issues like the proportion of representation. Once we cite the Census data there cannot be any authentic opposition to that evidence."

There is unquestionable value in a general policy of reservation as merit can't be the only criteria for granting subsidies and quotas. Social Justice and economic barriers also need to be fulfilled for deciding such national/state-level policies. Despite deep affiliation with caste groups, the public is ready and eager to advance toward a more transparent and accountable society. And, making SECC data publicly available would improve transparency in the political debates if not direct action leading to better governance.

The bigger question isn't the availability of the data but the lack of willpower to implement the policy. There is all the required data available for STs in the census yet there is a siphoning of funds allocated to SC/STs. (Systemic denial and diversion of budgetary allocations). Many departments have failed miserably in allocating funds and even spending that minor fund for the TSP as per the proportion of the tribal population of the state. The manipulation made by the bureaucracy and political leaders in making the decision to divert the TSP fund showed a huge contrast between public policy and data-based evidence. Also, there is a deafening silence on Gender Budgeting (GB) in the corridors of power. So we have a country where no one pays heed to independent minds seeking evidence-based policy in the social sector. In such a political system, one has to learn to manipulate rather than simply put the case.

A pure functioning meritocracy would produce a society with growing inequality, but that inequality would come along with a correlated increase in social mobility. Is this really happening across caste and religion? We can only guess policy as there is not yet sufficient data for planning on caste-based issues. SECC data would even regenerate healthy debate between two groups proposing - either 'caste-blind &class-based' or 'caste & class based' type of affirmative action. The best way to improve the quality of the debate is by providing individuals with the data to make informed choices in a democratic country. SECC data is crucial for making sound evidence-based plans, it helps us understand what justice is so that appropriate policy can be made for the Disadvantaged class/caste. 

Migration Series -3

Let me move ahead in the last part of Migrant Series. (1 and 2) .Whenever someone comes with an idea on research on the issue of migration and informal sector, there is complete lack of valid data on the nature and terms of employment, particularly in the informal sector. This is directly linked with the state’s unwillingness or inability to use resources for counting the large number of informal sector workers including child workers, bonded labour, migrant workers, home-based workers, domestic workers, and manual scavengers.

A person migrate to a prosperous region due to low employment opportunities at home, low agricultural productivity, deeply entrenched feudalism, and negative industrial growth of the region. Due to irregular cash flow nature of agriculture, a household has no other option but to seek credit at exorbitant interest rates from the local money lender during an emergency situation. The household requires huge money for repayment that isn't much available in returns from the agriculture. That is one of the reasons of the migration in any part of the India. This migration is facilitated by contractors and relatives. They are even willing to accept any distress wages that are offered as long as they have access to employment. In the dire need, they neglect the problems of not having an identity cards, lack of awareness about potential employers or favourable working conditions. The share of agricultural households among rural households varied from 27 per cent in Kerala to 78 per cent in Rajasthan. And, an accurate mapping of agricultural household and migrants can show the true picture of migration in India.

According to a recent census report, three out of five people in Indian cities live in slums; the increasing migration will make that four out of five within the decade. Covering over 4,000 towns, the report reveals a figure of 65 million living in unauthorised squalor. "Unrecognised" slums have no access to safe drinking water, electricity and sanitation. Chakarpur village in Gurgaon is a hub of migrants from Bangladesh and West bengal. Such migrants always have a sense of vulnerability and social isolation that is exacerbated by their ignorance, illiteracy and the alien environment. Urban middle class populations who generally hold negative views on migrants. Local Rajasthani and Punjabi is considered as “indigenous and authentic” whereas the migrant from Bihar is a sinister criminal in New Delhi. Low skilled workers are utilized in construction sector, domestic servants, chauffeurs and gardeners while women and child are working as maids, rag pickers, and scavengers with low and irregular income. These slum residents provide cheap labour as their average wages are just 40-70% of the local labour. There is a CARE report on such pathetic living conditions.

Mosse et al (2005) highlight the fact that most formal channels for protection of seasonal migrant workers such as the Minimum Wages Act (1948), the Inter-State Migrant Workmen (sic) Act (1979) and the Construction Workers Act (1996) among many others have demonstrated a bias towards formal sector workers and hence failed to deliver. As a result most informal sector migrant workers rely on contractors and agents to cope with socio-economic risks. Most of these insurance products sold by agents (usually friends and acquaintances of the migrant) have limited knowledge of the product themselves. The reliability of such agents is always in doubt and rarely case is heard about claim settlements through employer. Exhaustive working hours, poor nutrition and occupational hazards (injuries occurring at the workplace) always push migrants back to the source. Early return is a complex and multi-layered issue that often destabilizes the social, economic and psychological well-being of a migrant.

Bonded labour is rampant in brick kilns, stone quarries, beedi manufacturing, carpet weaving and construction, and child bonded labour in the silk industry. Bondage has been rampant across the state, but the government denies the fact. While thousands of bonded labourers have over the years been released, only very few of them were issued release certificate by the authorities concerned. The bigger question is “whether the state is willing to abolish the bonded labour system?” Since no release certificates are issued to those released and no legal case filed against those who keep bonded labourers, hence there is no persecution.

In this last part of the series, we will give focus on Distress Migration and Human Trafficking. It must be kept in mind that trafficking is different from migration. There are important fundamental differences between migration and trafficking. Trafficking of women and children is one of the gravest organized crimes and violations of human rights done either through deception, coercion or debt bondage situation. Human trafficking is also up in India for the similar reasons -- a lack of economic opportunities, a corrupt state, and the rise of terrifying social pathologies. There were hundreds of thousands of migrants, nearly all male, who did not have wives or girlfriends with them, and having an expendable income. This made a much more fertile environment for prostitution, and hence human trafficking in India. There are various pull factors for destination are Jobs, promise of marriage and a better life, demand for young girls and finally access to sex trade.

Immigration has become symbolic of the disruption of communities, the undermining of identities and the fraying of the family values. It is an outright violation of the right to mobility when government stop people from emigrating for better life. But it is a quite impossible task and also not desirable to stop people from migrating voluntarily. However, there are strong voices that objects to migration. Immigration is clearly one of the most fiercely debated and toxic issues of today. Even in policy circles, there is a disconnected emphasis on remittances alone and less about social factors. In the short term migration may result in the loss of local financial and human capital, but it can also be beneficial and contribute to the long-term development of rural areas. Remittances generate significant indirect benefits to the community at large.

Do migrants really have a choice between two such different worlds, between aspiration and deprivation, power and powerlessness? There are no words for me to write on anger, violence and frustration of the migrant worker. The topic of gender, trafficking and migration has been around for a while, but substantive work on this topic that go beyond the anecdotal is yet hard to find. Those who are interested in basic study can look upto : An overview of migration in India, its impacts and key issue.

Migration Series - 2

Let us start from Migration Series - 1. Once there was a popular myth: in the Government and Development Sector: Migration of 'poor' rural population is bad idea. Even when higher caste population was migrating towards cities for education, the government was launching various schemes on poverty reduction in the villages to prevent people from moving to the urban areas. So, why is the Migration Issue ? Migration – when it is safe, legal, and voluntary – is the oldest poverty-reduction and human-development strategy [Migration, Development and Poverty Reduction in Asia] . As per Wiggins & Keats [Stepping out of agriculture] - "Migration takes various forms, distinguished by:  Destination – international versus domestic, rural to urban, rural to rural and urban to urban; and, Length of absence – permanent moves of a year or more, and seasonal moves – to which might even be added daily commuting."

Many moves are not permanent, but vary from seasonal and circular. The reasons for out-migration can vary as a result of debt at home combined with high unemployment level and poor wages for jobs in the village. The availability of temporary jobs in the nearest vicinity with boom in urban development leads a huge circular migration pattern daily. Temporary migration is a routine livelihood strategy for the poor in India  rather than coping strategy to “keep the wolves at bay".

Migration Pattern in India

There is migration from landlocked BIMARU state towards places having either industrial hubs or agricultural prosperity.The tussle between migrants and ‘people of the soil’ has given rise to political right parties in Maharashtra.

Migration Pattern International 

Taking the estimates available, it seems that just over 3% of the world’s population are international migrants (UN Population Division, 2013), while domestic migrants are at least 12% of world population (Bell and Charles-Edwards, 2013).

Remittance Market

Remittances provide the most tangible link between migration and development, a relationship that has only increased in importance since the economic slump since 2008. Let us compare the Official remittance flows compared to other large monetary flows in 1990–2016 projection for India. The graph is constructed with World Development Indicators and World Bank Development Prospects Group. The remittances from the migrated Indians have played a major role in the development of India from 1990's to present day. Personal remittances are estimated towards value of above 70 Billion Dollars leading to major boost in local consumption. Even when FDI and portfolio equity has dipped during recession era of 2008, the personal remittance has grown in a major way in India.

Internal remittances are part and parcel of livelihoods for many poor families in the developing world with migrant members working in big cities. Internal migrants within far outnumber international migrants but the internal remittances, however, are often small.  Rural areas often receive the lions’ share of remittances. As rural-urban wage differentials grow, the returns from migration increase. India has the second largest domestic remittance market in the world (Tumbe 2011). It is also estimated that of the total domestic remittance flows in India only 30% are routed through formal channels. This is in stark contrast with China where 75% of the remittances are formally routed (ibid).

Are internal remittances contributing to poverty reduction? Remittances from urban employment are mainly used for such purposes as immediate consumption, repayment of loans, health care expenses, education and meeting other social obligations. Investments by migrant households in housing, land and consumer durable are common, and migrant income is also used to finance working capital requirements in agriculture as well as small businesses. . Those who are interested can  read World Bank report on The Remittance Market in India(PDF).

Remittances need to flow directly into the hands of the people who need it most. There is a lot of policy gap for this goal that must be addressed on urgent basis by the government.  Policy initiatives by the government and banking institutions have achieved an important result - Most remittances is flowing  through formal channels. India need to revamp their apparatus for issuing passports and regulate agencies that recruit unskilled workers. And internal migrants also need a lot of entitlements and services from the government and better mechanism for fund transfer through financial institutions. The social impact on the lives of migrants will be discussed in the last post of the migration series.

Capacity building of capacity builders

The heading of the article doesn't seems to confuse reader who are working in development sector. Its a fancy word on the training of field staff. The major component of any development project is institutional and human capacity development in the community. Training the larger development community to more effectively support community-driven efforts requires a field staff with local networking, trust and suitable skills. There are six factors to be considered while managing field staff in the development project.

1. Recruitment of LRPs - In this phase, mostly NGOs go either for references or prefer an experienced candidate. The most preferable candidate is not the most smart one from the community. Honesty, non political nature, perception in public and hard work are the traits to be seen in the recruitment phase. With some expectations the leadership is concentrated in the hands of elderly people. The style of functioning of these elderly people exhibits authoritarianism and frustrates younger generation. 25-45 is the most suitable age for the field worker as the community have a certain level of trust and sees maturity in these candidates. Conflict of Interest must be considered before hiring of any worker. The experience of person who has worked recently in fraudulent chit fund will make NGO dubious for the community. Job profile, terms of the payment, and attendance must be clarified in the interview.

2. Knowledge Transfer - Knowledge is a powerful tool. Knowledge transfer requires a detail knowledge on the name of the Project, the Implementing agency, the Funding agency, Area of operation, deliverable of the project, total projected outreach, role of field staff, organization hierarchy and a brief project note. The major hurdles in information dissemination is the language of the medium. This is a huge problem at all India level as the necessary level of English is not known to the field staff. All the training modules and IEC material must be in local language.

Trainers used custom charts, posters, pamphlets and a video for the training session. Short movie clips is the best medium for the transfer of the knowledge. Digital Green has done significant works in this area. Training session in leadership, team-building and core objectives of project require a detail article in itself. There are ways like role play, puzzle solving, group discussion & storytelling depending on the skills of the trainer for knowledge transfer in the training session. Through exposure visits also, field staff gains a lot of insights on hurdles in implementing project.

3. Data Gathering - Honest data collection is one of the rigorous task performed by field staff. Its easy to  criticize but difficult to gather data in rural India.  Hence, the continuous monitoring of processes of change, and scientific evaluation to track the progress of the project depends on the shoulders of the field staff . While going in for a survey it is always a good idea to get to take input  of the local staff as they are the one needing most clarity. Perhaps one of the most common mistakes is not to understand that the rate of collection of data in the rural area differ from an urban setting. The whole process is a lot slower as the villages are really really spread out, migration issue and the low connectivity. The scope of data which has been actually and more importantly properly collected in our villages very low.

4. Gender Issues - ‘Feminisation of development’ is a fancy phrase referring to the recent trend of seeing women as both beneficiaries and agents of change in development. There must be combination of male and female staff to provide capacity building support to the community. Effective two-way communication engage in dialogue and debate on issues ensures proper outreach.

5. Transparency & Ethics - Transparency International's website and their definition of corruption is "Corruption is the abuse of entrusted power for private gain" and it "depends on the integrity of people in a position of authority". A disgruntled field staff can go back and sow seeds of distrust in the community. Hence, transparency in decision making with community involvement is a better option rather than handling a post crisis situation. One of the instruments for achieving trust of the community is more transparency. Right to know rules & tell rules are pillars of ethical high ground for any person and institution. 

6. Incentive & Rewards - Employee motivation is a continual challenge in traditional ‘command and control' structure of NGOs. ; Maslow’s Hierarchy of Needs applies for small NGOs. All field workers aspire for recognition of work. They look for responsibility when they can either see advancement in salary or non monetary reward for good work.

7. Monitoring -  The reporting officer must  visit their operational sites, observed their activities, witness their implemented project, participate in their committee meetings and interact with numerous ordinary villagers – both beneficiaries and non-beneficiaries as well as village leaders. This community-level information will immensely effective in analyzing the implications of these organizations at the grassroots level. Not only this will give the hold of grassroots by these interactions, but also minimize chances of bogus reporting by the field staff.

Working in non profit sector doesn't give us excuse of inefficient manner. Nonprofits often have limited resources to invest in staff training on effective project management. They are also in fear of personnel who may shift to another big NGO after taking training from it. The flip side of inadequate trained staffl due to lack of funds will take toll on the community as well institution. Last of all, never pretend to know great fundas of development in front of field staff. You will have a bad time.