Fundación de Estudios de Economía Aplicada

What Works Best For Getting the Unemployed Back to Work: Employment Services or Small-Business Assistance Programmes? Evidence from Romania by Nuria Rodriguez-Planas DOCUMENTO DE TRABAJO 2007-32

Serie 3 CÁTEDRA Fedea-Santander

October 2007

* Universitat Autònoma de Barcelona, IZA and FEDEA.

3. Serie Capital Humano y Empleo

Nuria Rodriguez-Planas¹ Universitat Autònoma de Barcelona, IZA and FEDEA

October 2007

Abstract

Recent empirical evidence has found that employment services and small-business assistance programmes are often successful at getting the unemployed back to work. One important concern of policy makers is to decide which of these two programmes is more effective and for whom. Using unusually rich (for transition economies) survey data and matching methods, I evaluate the relative effectiveness of these two programmes in Romania. While I find that employment services (ES) are, on average, more successful than a small-business assistance programme (SBA), estimation of heterogeneity effects reveals that, compared to non-participation, ES are effective for workers with little access to informal search channels, and SBA works for workers with less access to the primary sector. Finally, if the policy decision is whether to offer ES or SBA to unemployed workers, ES is more effective in getting low-skilled individuals and young workers out of unemployment, while SBA works best for more educated workers.

Key words: Active labour market programmes, evaluation, propensity score matching, transition economies, and treatment effects.

JEL classification: J21, J23, J31, J64, J65, J68

Contact e-mail: Nuria.Rodriguez@uab.es

¹ I am extremely grateful to Jacob Benus for designing the evaluation and making the data available. I am also grateful to participants of the 3^rd Annual IZA Conference on the Evaluation of Labour Market Programmes for helpful comments. Financial support from the Romanian Ministry of Labour and Social Protection, the National Agency for Employment and Vocational Training, the Spanish ministry of Education and Science (grant SEJ2006-712) and the Generalitat de Catalunya (grant SGR2005-712) is also gratefully acknowledged.

I. Introduction

In recent years, there has been a substantial increase in the empirical evidence on the effectiveness of active labour market programmes (ALMPs) in developed, developing and transition economies. This improvement can be explained by the increased availability of data, the improvements on data quality, and recent developments on evaluation methodology. However, most of these studies focus on comparing the labour market outcomes of unemployed individuals who participate in an ALMP with other unemployed individuals who do not participate in an ALMP—at least during a pre-determined span of time.

While evaluating the effects of ALMPs relative to non-participation is an interesting question per se, policy makers may well be more interested on what are the relative effects of two different types of programmes, as well as the suitability of these programmes for different target groups. More specifically, for the two ALMPs—that is, small-business assistance programmes and employment services—that recent empirical studies indicate are often successful at getting the unemployed back to work, a useful policy question may be: which one is more effective and for whom. This paper provides some evidence on which of these two types of programmes’ works best for different population subgroups in Romania. The results offer interesting policy recommendations for implementing these programmes both in transition economies, and in countries with large informal search sectors and dual labor markets, such as developing countries.

There are considerable differences in the design of these two types of programmes. On the one hand, small-business assistance programmes are usually intended to support the startup and development of self-employment endeavours or micro-enterprises. They usually provide counselling and assistance in developing and implementing a business plan, and often include some form of financial assistance. Although the use of these programmes has been limited compared to other ALMPs, their popularity (as well as the number of empirical evaluations available) has recently increased (Kluve, 2006). Employment services, on the other hand, include different types of measures aimed at improving job search efficiency. They usually include the following types of services: job clubs, job-search courses, counselling, testing, and assessment. Moreover, because of their relatively low costs, employment services tend to be the most cost-effective (Martin and Grubb, 2001, and Dar and Tzannatos, 1999, among others).

To my knowledge, there is no theoretical and comparative empirical research devoted to analysing the relative effects of employment services and small-business assistance programmes.² The main reason for this is that the latter type of programmes has, until recently, been seldom used, and thus data allowing comparison of these two types of programmes in the same country are rarely available. While cross-country studies of these two types of programmes are possible, differences in labour market conditions, institutions, evaluation designs, availability of outcome variables, and time periods seriously complicate the analysis.

Romania can be used to study the differences between employment services and smallbusiness assistance programmes because these two programmes were the first major ALMPs implemented in Romania on a large national scale after the 1989 Revolution. Moreover, these programmes were targeted at more or less the same population of unemployed. Furthermore, the experience of Romania ought to be of interest to policymakers of other countries, especially transition economies, which have suffered soaring labour surplus after social, economic, and political reforms, and developing countries that, like transition economies, tend to have large informal search sectors and segmented labor markets. Finally, a rich data set provided good quality data on key variables—such as earnings for both the employed and the self-employed, and allowed me to track individuals’ earnings and employment status at different points in time over a four-year period. This database was previously used by Benus and Rodriguez-Planas, 2007, (BR, hereafter) for a microeconomic evaluation study of several active labour market policies. Their study focuses on the effects of ALMPs relative to nonparticipation, and finds that both programmes improved participants' economic outcomes compared to non-participants. However, their paper does not address the relative effectiveness of these two programmes, nor does it discuss theoretical implications of both programmes and contrasts them with heterogeneity results.

¹ The focus is on the direct effects of the programmes; no attempt is made to assess the general equilibrium implications.

² Several microeconometric studies have looked at the relative effects of programmes in one country (Bonnal, et al., 1997, Gerfin and Lechner, 2002, Larsson, 2003, Carling and Richardson, 2004, Gerfin, et al., 2005, and Sianesi, 2005, among others), however, none of these studies compares the relative effect of small business assistance programs compared to employment services.

The analysis, based on matching methods, reveals that average effects for the population as a whole may hide statistically and economically significant differences across subgroups. I relate these differences to the different institutional set-ups and discuss theoretical implications, which are then empirically contrasted with the heterogeneity effects. While, I find that employment services (ES) are, on average, more successful than small-business assistance programmes (SBA), estimation of heterogeneity effects reveals that, compared to non-participation, ES are effective for workers with little access to informal search channels—such as young workers, and those living in rural areas, and SBA works for workers with less access to the primary sector, such as the less-qualified workers or those living in rural areas. Finally, if the policy decision is whether to offer ES or SBA to unemployed workers, ES is more effective in getting low-skilled individuals and young workers out of unemployment, while SBA works best for more educated workers.

This paper is organized as follows. The next section presents an overview of institutional environement. Section three summarises previous empirical findings. Section four describes the data, and displays the descriptive statistics. Section five discusses the economic evaluation strategy and displays the results from a multiple treatment evaluation framework using a ‘matching propensity score’ estimator. Section seven concludes with a discussion on policy implications and cost-effectiveness.

II. The Institutional Environment

In the late 1990s, the Romanian government launched the real start of active programmes on a large national scale by signing a loan agreement with the World Bank. The two major programmes offered were employment services (ES), and small business assistance (SBA). Altogether, these two programmes served more than 80% of the unemployed who received some kind of active labour market programme in Romania during that period. To understand the effects of these programmes and the composition of the different groups of participants, it is necessary to understand the specifics of the programme as well as the institutional environment in which they operated.

The Romanian Unemployment Programme

Unemployed individuals were eligible for financial support through unemployment benefits, allowance for vocational integration and support allowance. To be eligible, an individual had to: be registered at the local Labour Office, be aged eighteen and over, have an income less than half of the indexed national minimum wage, be unemployed due to liquidation or a lay-off, be employed at least six months during the last twelve months, or be a recent graduate from school or university unable to find suitable employment. Unemployment benefits were paid for a maximum duration of nine months. The level of these benefits ranged from 50 to 60 percent of the average monthly salary during the last three months of employment for laid-off workers. For new entrants, benefits varied by the level of education and years of experience for those with prior work experience. After exhausting unemployment benefits, those who remained unemployed received a support allowance (of 60 percent of the indexed minimum wage) for a maximum period of 18 months.

Employment Services

Clients eligible for these services were offered a variety of employment services, including job and social counselling, labour market information, job search assistance, job placement services, and relocation assistance. The duration of these services was limited to 9 months per individual. In addition, those clients receiving relocation assistance could be reimbursed for expenses associated with moving to another community—up to $500 U.S. dollars equivalent in lei per family (based on submission of receipts). The programme also offered up to two months of salary at the minimum wage. Service providers had to agree to a negotiated job placement rate of at least 10 percent.

Small Business Assistance Programmes

Provision of these services included initial assessment of the aptitude and skills of unemployed persons to start businesses, developing business plans, advising on legal, accounting, financial, marketing and sales services issues, assistance in the dialogue with local authorities, short-term entrepreneurial courses and training and other consulting services to unemployed entrepreneurs who intended to start, or who had started businesses during the past 12 months. There were also provisions for short-term working capital loans of up to $25,000 U.S. dollars to programme participants. Service providers had to agree to a negotiated business start-up rate of at least 5 percent of clients initially contacted. Maximum length of initial contract was 12 months.

III. Previous Empirical Evidence

Recent empirical evidence highlights the success of employment services and small business assistance programmes at getting the unemployed back to work. According to Kluve, 2006, a consistent result for both Europe and the US are the positive effects for both programmes. Martin and Grubb, 2001, also find that these programmes are successful at getting the unemployed back to work in developed countries. In addition, Dar and Tzannatos, 1999, and Betcherman, Olivas and Dar, 2004, find that both of these programmes tend to be successful in developing, and transition countries. Below, I briefly describe the results for developing and transition economies, and refer to the previously cited authors for thorough reviews of the literature in developed countries.

³ See Earle and Pauna (1998) for a detailed description and thourough analysis of this programme in Romania.

Employment Services

The evidence on the effectiveness of employment services in transition and developing countries is scarce. To my knowledge, five non-experimental studies have evaluated employment services in the following transition countries: Bulgaria (Walsh, et al., 2001), Czech Republic (Terrel and Sorm, 1999), Hungary and Poland (O’Leary et al., 1998), Macedonia (World Bank, 2002), and Romania (BR). All of these programmes found positive impacts of this type of services on improving employment prospects of participants. However, of the three evaluations that estimated the impact of these services on earnings, only in Poland and Romania a positive effect was found. According to Betcherman, et al., 2004, the two studies that evaluate employment services in developing countries show mixed results, finding limited impact in Brazil and positive impact for only some subgroups in Uruguay.

When looking at studies that evaluate heterogeneity results, employment services seem to be more beneficial for the less educated workers in developed countries. In contrast, the opposite seems to be true in developing countries. For instance, Fawcett, 2001, found that employment services mainly worked for better-educated workers. Finally, the evidence from transition economies, such as Hungary and Poland, indicates positive effects of these services for women.

Small Business Assistance

The results from non-experimental evaluations in transition countries are also consistent with small business assistance programmes increasing the probability of reemployment. These studies found that self-employment assistance programmes in Bulgaria (Walsh, et al., 2001), Hungary and Poland (O’Leary, 1998a), and Romania (BR) were successful at getting the unemployed back to work. However, the evidence on earnings is mixed. While the Romanian study found no effect on earnings, the study from Hungary found a negative effect, and the study from Poland found a positive one. The Bulgarian programme did not estimate the impact of the programme on earnings. The little empirical evidence found in developing countries seems to indicate that small business assistance improves the outcomes of its participants with entrepreneurial skills and motivation (Almeida and Galasso, 2007).

Heterogeneity analysis suggests that these programmes work best for unemployed workers who have entrepreneurial skills and the motivation to survive in a competitive environment, such as, highly educated prime-aged males in developed countries or young and more educated workers in Argentina. The evidence in transition economies finds that these programmes are beneficial for women and older workers (in Hungary, and Poland).

IV. The Data and Descriptive Statistics

This study uses data from a follow-up survey of registered unemployed that has already been used by BR. I shied away from using existing data from the Ministry of Labor and Social Protection because I was concerned on the quantity and quality of the existing data, and, more importantly, data from the Ministry lacked several key variables needed for the analysis, such as earnings for both the employed and the self-employed, or detailed characteristics of the unemployed.

The data, a random sample of 3,357 persons who registered at the Employment Bureau during 1999, was collected during January and February 2002. Thus, I observe individuals at least 24 months after the programmes started. About two fifth of this sample (1,408 individuals) were ALMP participants of either ES or SBA whose ALMP contract began in 1999. The rest of the sample—the potential comparison group—were 1,949 persons who were registered at the Employment Bureau around the same time and in the same county than participants but who had not participated in an ALMP.

⁴ No impact on earnings was found in Hungary.

The sample contains detailed information on individual labour market histories and earnings prior to 1999 unemployment spell such as: level of education, experience, usual monthly earnings, unemployment history, participation in a training programme; as well as socio-demographic information, such as age, gender, family composition and whether the person was the family’s main wage earner. It also contains variables that capture the local labour market conditions. These variables measure the different employment opportunities in the judets (counties) Since differences in labour market conditions may favour a different mix of programme and unemployment policies, these variables are also used as a proxy for different policy approaches across judets. Finally, the analysis includes judet dummies to capture unobserved local aspects that were likely to be correlated with programme implementation and utilization, or local offices’ placement policies, and thus relevant for programme-joining decisions and individuals’ potential labour market performance.

Although at first sight and relative to studies conducted in developed countries, these data may not seem sufficiently rich to observe all relevant factors, I believe that these data are unusually rich for studies conducted in transition economies—see Kluve, et al., 1999, or Earle and Pauna, 1996, among others, for discussion on the poor quality of ALMPs’ data in transition economies. For instance, one could argue that, even though I can control for employment history during 1998, I lack of information on employment history prior to 1998 (Heckman and Smith, 1999, point to the importance of controlling for employment dynamics prior to programme participation.) Rich data on employment dynamics prior to programme participation in studies on transition economies is unusual. Many of these studies do not have any employment information prior to participation (Lubyova and Van Ours, 1999, Puhani and Steiner, 1997, and Vodopic, 1999, among others). Others have limited information on employment history prior to participation. For instance, while O’Leary 1998, have information on prior employment status and whether the individual was a long-term unemployed, they do not control for months employed or unemployed prior to programme participation. A study that has information on the unemployment spell that took place right before programme participation is Terrel and Sorm’s JCE 1999 paper. However, their information is limited to the year of participants’ unemployment registration, which is, at most, the year prior to programme entrance. Finally, to our knowledge, Kluve, et al., 1999 and 2002, have the most thorough information on employment dynamics prior to participation in transition economies, and again, this information is limited to 12-month prior to entering the programme.

Restriction that all data be available led to a sample of 2,610 individuals (1,109 participants and 1,501 non-participants). All the results presented below are robust to using all of the observations available for each of the different outcome variables. However, in order to work with the same sample in the whole paper I restricted our sample to have all data available. More details on the data, the sample selection, and full descriptive statistics can be found in Appendix A, and Table A.1.

Table 1 displays selected descriptive statistics for socio-economic variables for participants of employment services and small-business assistance, and for non-participants. Although participants in ES and SBA had similar and relatively stable employment histories during 1998—as reflected by the fact that about three fourths of them reported working during 1998, of which more than four fifths did so for at least 7 months—SBA participants seemed to be slightly more advantaged workers than those participating in ES. For example, participants in SBA were more educated, employed for a longer share of the year and worked in better paid jobs than ES participants. Another difference is that ES participants were more likely to live in large urban areas than SBA participants.

When comparing participants to non-participants, the main finding is that non-participants tended to have more stable employment histories despite living, on average, in less dynamic areas. Other differences worth highlighting are that non-participants had a higher share of men in their group, were more likely to live in rural areas, and were much less likely to have received training during 1998 than participants of any of the other two programmes.

V. Empirical Estimates of the Effects

V.1. Estimation Methodology

I analyse the effects of the K = 2 different ALMPs (ES and SBA) on employment outcomes at the individual level. In a situation where individuals have multiple treatment options, I estimate the average treatment effect on the treated (ATET) of one ALMP against non-participation in any ALMP and of pair wise comparisons of the two programmes. I also analyse the heterogeneity of the estimated ATET by various socio-economic characteristics of the treated individuals as explained in Section V.3.2.

The empirical approach follows the framework suggested by Rubin (1974), and extended by Imbens (2000) and Lechner (2001) to multiple, mutually exclusive states. Let the potential outcome , denote the outcome when a person gets the treatment (in this case, participates in one of the two ALMPs described above), and denote the outcome when a person does not participate in any ALMP. For any individual, only one component of is observable.

Participation in a particular treatment k is indicated by the realization of the random variable S, . This notation allows under the usual assumptions (see Rubin, 1974) to define average treatment effect on the treated for pair-wise comparisons of the effect of different states:

\[\begin{array}{l} \theta^ {k, l} = E (Y ^ {k} - Y ^ {l} | S = k) = E (Y ^ {k} | S = k) - E (Y ^ {l} | S = k) \\ k \neq l; k, l \in \{0, 1, 2 \} \end{array}\tag{1}\]

The shorthand notation denotes the mean in the population of all individuals who participate in an ALMP, denoted by

shows the expected effect of the programme for those persons who actually participated (average treatment on the treated, ATET). However, we cannot observe the counterfactual, ., the average outcome of those persons who participated in the programme had they not participated. Thus, without further assumptions, ATETs are not identified. Lechner, 2001, shows that if we can observe all factors that jointly influence outcomes and participation decision, then—conditional on those factors (call them X), the participation decision and the outcomes are independent. This property is exploited by the conditional independence assumption. Note that the ATETs are not symmetric, if participants in treatment k and l differ in a way that is related to the distribution of X and if the treatment effects vary with X.

To identify the effects, I rely on the highly informative data I have at hand for the estimation, that is, I assume that selection is on observables only. The crucial question—that is left to the reader to decide—is whether there is sufficient information to justify the conditional independence assumption. I believe that the data used frequently provides variables that contain some of this needed key information, and is at least qualitatively equal (if not superior) to data used in other evaluations of ALMPs in transition economies (see earlier discussion).

Empirical Implementation

I used propensity scores to select a group of participants for each treatment group, according to the following three steps. First, I fitted a probit model for the choice between the two programmes and non-participation. Table A.2. in the appendix displays the estimation results and provides a more exact description of the variables used in the analysis. Second, I used the output from this selection model to estimate choice probabilities conditional on X (the so-called propensity scores) for each treatment comparison pair. I then imposed the common-support requirement to guarantee that there is an overlap between the propensity scores for each pair (see column 9 of Table 2 for number of treated observations lost due to this requirement). Third, for each treatment group member, I selected potential comparison group members based on their propensity scores and their judet. The selection process was done with replacement, so that a potential comparison group member could have been matched to more than one treatment group member. In addition, the selection method used was kernel-based matching, which uses all of the comparison units within a predefined propensity score radius (or “caliper of 0.01”).⁵ When there were multiple matches, each nonparticipant received a weight that reflects the number of successful matches within the caliper range. To adjust for the additional sources of variability introduced by the estimation of the propensity score as well as by the matching process itself, bootstrapped confidence intervals have been calculated based on 1,000 resamples.⁶

Matching Quality

The results in Table 2 show indicators on the quality of the match for each of the two ALMPs and for each programme compared to non-participation. Overall, matching on the estimated propensity score balances the X’s in the matched samples extremely well (and better than the other versions of matching I experienced with). First, to test if the matching procedure is able to balance all the covariates, I estimated the median absolute standardized bias before and after the matching (Rosembaum and Rubin, 1983). This indicator assesses the distance in marginal distributions of the X-variables, and is commonly used to evaluate the validity of the match (Sianesi, 2004, Caliendo et al., 2005, among others). Columns 7 and 8 show the median standardized difference over all covariates before and after the matching took place. The matching procedure balances the distribution of covariates very well since the median absolute standardized bias drops from a range between 9.36% and 18.56% before the matching to a range between 2.29% to 4.19% after the match. Second, comparison of the pseudo-R²’s before and after the matching indicates that after the matching there are no systematic differences in the distribution of the covariates between the two groups (columns 4 and 5). Similarly, the P-value of the likelihood test after the matching rejects joint significance of the regressors, while the opposite was true before the matching (column 6).

⁵ I tried alternative matching methods and caliper sizes of 0.05, 0.02, and 0.01. To ensure a good quality match, I implemented a caliper of 0.01.

⁶ Heckman et al., 1997, derive the asymptotic distribution of kernel-based matching estimators and show that bootstrapping is valid to draw inference. This is an additional advantage of this matching method compared to alternative methods, such as nearest neighbour matching, since it allows to circumvent the issues regarding nearest neighbour matching raised by Abadie and Imbens, 2006.

V.2. Average Results

Impacts were estimated as the difference in average outcomes between the treatment and the comparison groups (a definition of the outcomes can be found in Appendix Table A.3). The estimated ATET and their bootstrapped 95 percent confidence intervals are shown in Table 3. Results are robust to the sensitivity analysis, where various estimators were applied to the same problem to see whether and how the results differed (discussed in Appendix B).

The last two columns of Table 3 show the impacts of participation in ES and SBA, respectively, compared to non-participation (these results were already found and discussed by BR). Overall, participation in either programme is successful into getting the unemployed back to work compared to non-participation. In addition, ES had a positive impact on earnings.

Finding that participation in ES or SBA increases the employment prospects of its participants compared to attending no-programme, does not address the question of which of these two programmes is the most effective for getting unemployed workers back to work. The first two columns of Table 3 show the pair-wise average outcome differences between ES and SBA, which allow us to answer this question. Overall, the estimated effects show that in terms of the accumulated employment effects, ES was superior to SBA.

The first column of Table 3 shows that ES was more effective for individuals receiving this type of service than if they had participated in a SBA programme instead. For instance, I find that participating in ES increased by 17.28 percentage points (or 34.18%)⁷ the likelihood of being employed for at least 12 months in the two-year period 2000-2001, and reduced by over 3 months (or 27.48%) the spell of unemployment during the same period compared to participating in SBA.

Moreover, SBA participants would have been better off had they participated in ES instead. For example, when the treatment was the SBA programme (column 2 of Table 3), its participants had 9.86 percentage points (or 11.53%) and 17.02 percentage points (or 24.65%) lower probability of being employed for at least 6 and 12 months, respectively, within the years 2000 and 2001. Moreover, although not statistically significant, the impacts on outcomes measured at the time of the survey suggest that SBA participants were less likely to be employed at wage and salary jobs than if they had participated in ES instead.

⁷ This result is calculated by dividing the ATET estimate (in this case, 17.28) by the percent of matched nonparticipants employed at the time of the survey, which is 50.56 percent.

V.3. Heterogeneity among Individuals

So far, I have considered the average effects for the participants in the different programmes, and have found that ES was superior to SBA. However, this average analysis does not provide any guidance on why ES might work better than SBA, nor does it explore whether the impacts vary with the socio-economic characteristics of its participants. In this section, I discuss competing theories explaining why the different programmes may have different effects, and then explore the compatibility of the estimated effect heterogeneity with the discussed theories.

V.3.1. Theoretical Considerations

Improved Job Matching

Both programmes may improve job matching for different reasons. On the one hand, the main objective of ES is to improve job search efficiency by increasing the information available to potential employers on the amount and quality of the applicants, and by improving unemployed workers’ knowledge about potential new employers and new occupations. On the other hand, SBA offers a network and contacts to unemployed workers that could (and sometimes does) result in wage and salary job offers. For instance, according to a study by Kosavonich, et al., 2001, more than 45% of participants of a self-employment assistance programme in New Jersey and close to 60% of participants of self-employment assistance programmes in Maine and New York ended up working in full-time wage and salary jobs, often, in the same industries as those in which participants initially became selfemployed.

It is unclear which of these two programmes would work best at improving job matching. A priori, one would think that ES should be more efficient at improving job search then SBA since the job matching mechanism for the latter programme is the result of an indirect, and thus, secondary effect of the programme. However, theoretical findings on search channels find that the efficiency of employment counselling services alone (that is, without monitoring) is seriously questionable. For instance, Van der Berg and Van der Klaauvw, 2006, developed a job search model with two search channels—the formal and informal one—and endogenous job search, and found that low-intensity job search programmes were useless. Moreover, empirical evidence on employment services in countries with large informal search sectors, such as transition economies and developing countries, have shown that public employment services may have limited reach as workers may prefer other channels of job search (Woltermann, 2002).

The empirical literature on the use of different search channels by different types of workers indicates that workers with characteristics such that their chances to find a job are low because of little access to informal search channels rely to a relatively large extent on formal search (Van der Berg and Van der Klaauw, 2006). Because ES facilitate job finding through the formal channel, I would expect it to have stronger effects for workers with little access to the informal search channels (such as young workers) or workers for whom their informal search channels have dried up (such as workers living in depressed areas) than for those with access to informal search channels (such as older workers and workers living in more dynamic areas).

Segmented Labour Markets

The dualistic view perceives the labor market in developing countries as segmented by two sectors: the modern or primary sector, characterized by high productivity growth and job benefits, and the traditional or secondary sector. This dualistic view also seems to apply to Romania’s labor market, where in the late 1990s, there was a two-tier system consisting of a large number of people involved in low productivity jobs and subsistence agriculture coexisting with large, potentially profitable but unreformed farms. This two-tier system was partly the result of Romania’s 1990s policy approach of limiting job destruction by adjusting through real wages, combined with a series of early retirement programmes, that pushed workers out of the labour force and into low productivity jobs, primarily in agriculture. This implied that by 2001 a high share of employment (42%) was in subsistence agriculture (up from 28 percent of total employment in 1989).

According to the dualistic view, the unregulated self-employed sectors are frequently seen as the disadvantaged segment rationed out of salaried employment (Fajnzylber et al., 2006). Those workers with little access to the primary labor market enter self-employment while queing for salaried jobs. This view predicts that SBA will have stronger effects on individuals with little access to the primary labor market, such as younger workers, less educated workers and those living in depressed areas, because it will give its participants a comparative advantage relative to other individuals that enter self-employment without assistance. In contrast, individuals with access to the primary labor market are likely to be little interested, and therefore motivated, in entering self-employment.

Human Capital

The impact of ES on human capital is likely to be small since the programme does not incorporate explicit training. In contrast, SBA offers some training through the form of advising on legal, accounting, financial, marketing and sales services issues, and some short-term entrepreneurial courses. Moreover, there is some empirical evidence that business-training programmes improved participants’ business knowledge and productivity, measured by revenues, repayment, and client retention rates (Karlan and Valdivia, 2006).

Assuming that human capital may be a complement to managerial activity (as argued by Rees and Shah, 1986, and Cressy, 1996), SBA will work best for more educated workers. This is consistent with empirical evidence from the US that finds that the probability of survival of SME is positively related to the level of education of their owners (Bates, 1999). In addition, if individuals acquire more capital, knowledge of business opportunities, and managerial ability while working, SBA ought to also have more of an impact with older and therefore more experienced workers.

Signalling

Participating in SBA may have a signalling value to prospective clients and contractors. Given the little entrepreneurial tradition in Romania, it is likely that prospective clients and contractors conclude that individuals who have participated in SBA are better entrepreneurs, and are more reliable since they have institutional support than those who did not participate in SBA.⁸ Moreover, in order to be a credible signal, participating in SBA must be more costly for the less productive workers than ES. Given that SBA involves entrepreneurial courses, and writing a business plan, it is likely that participating in SBA is more costly for less capable workers than ES.

⁸ For thourough studies explaining why Romanian SMEs’ sector has been slow to develop, see Ahrend and Martins, 2003, Brown et al., 2004, Dochia, 2000, Mitrut and Constantin, 2006, among others.

According to this view, SBA should be more effective for those workers for whom the costs of participating in SBA would be lower. Because older workers are likely to have more experience, networks and contacts than younger ones, this should lower their costs of starting a business compared to those of younger workers. A similar prediction would hold for more educated workers since they have lower costs of acquiring entrepreneurial skills than less educated ones. For instance, according to Costariol, 1993, in the case of Romania, where two generations of artisan tradition were lost during communism, the typical private entrepreneur is a first-generation person, middle-aged, mainly with previous experience in a managerial position with large scale state-owned companies or, if he is young, usually with a University education. This description of the typical Romanian entrepreneur indicates that being more experienced, and educated facilitates access to entrepreneurial activities. Thus, if signalling is important, I expect SBA to have more of an impact compared to non-participation for older and more qualified workers.

V.3.2. Empirical evidence

It is not possible to derive strict tests for the relative importance of these explanations. However, systematic heterogeneity of the effects between different groups of unemployed will provide evidence consistent with one theory but not with another. I use non-participation in any programme as a benchmark because non-participation will neither have job matching, human capital, or signalling effects. In addition, subgroup estimates comparing the impacts of ES versus SBA participation are also provided and discussed.

The subgroup impact estimates have been estimated by previously stratifiying the sample along the dimensions age, type of region, and education matching within strata. Additional heterogeneity analysis by gender and prior-unemployment duration can be found in Appendix B.

Heterogeneity with Respect to Age

I find that, with respect to non-participation, ES improved the economic outcomes of younger workers compared to older ones (estimates shown in Table 4). These results are consistent with the improved matching theory. For instance, I find that younger workers had 26.20 percentage points (or 61.02%) higher likelihood of being employed for at least 12 months within the two-year period 2000-2001 than non-participants. In the case of older ES participants, this likelihood was increased by only 4.12 percentage points (or 7.16%) and the estimate was not statistically significant. This finding is explained by considerably higher likelihood of employment for older non-participants (57.58%) compared to younger ones (42.94%), suggesting that the latter may find it more difficult to find work through alternative job search channels.

When comparing SBA participants to non-participants, I find that the impact estimates on current employment and earnings are larger for older than younger workers (although the differences between the two subgroups are not statistically significant). This result would be consistent with the human capital and signaling view.

Pair-wise comparison of current employment outcomes between ES and SBA shows that, ES are more effective than SBA for younger workers than for older ones. I find that participating in ES increases by 26.52 percentage points (or 46.86%) the likelihood of being employed at the time of the survey for younger workers, and that it increases their current earnings by 129 thousand lei (or 43.37%). The estimates for older workers are negative although not statistically significant.

Heterogeneity with Respect to Type of Region

In Romania in the late 1990s, rural areas tended to be more economically depressed than urban areas. Therefore to test the predictions on less depressed areas I will compare the impact of both programs in rural versus urban areas. The evidence found in Table 5 is consistent with the view that ES ought to work better for individuals living in depressed areas where informal search channels have most likely dried up. For instance, I find that ES increases the average wage of its participants in rural areas by 144.24 thousand lei, and reduces their unemployment spell by almost 5 months over the period 2000-2001.

In addition, I find that, compared to non-participation, SBA is more successful for workers living in rural areas than those living in urban ones. For instance, participating in SBA increased the likelihood of employment for at least 12 months of the two-year period 2000-2001 by 19.06 percentage points (or 49.16%) and reduced the accumulated spell of UB receipt by 3.61 months (or 86.57%) for workers living in rural areas. No statistically significant effects were found for SBA participants living in urban area. These results are compatible with the segmented labor market view.

Heterogeneity with Respect to Education

As a measure of skill I have used whether the worker has a high-school degree or not. When comparing SBA to non-participation, the results are consistent with the segmented labor market view and contrast with the human capital and signaling hypothesis. The estimates in Table 6 show that SBA is beneficial with respect to non-participation for workers without a high-school diploma. For instance, I find that, with respect to non-participation, SBA increased the probability of being employed for at least 12 months within the two-year period 2000-2001 by 19.35 percentage points (or 48.51%) for the lower educated subgroup— compared to a non-statistically significant increase of 1.45 percentage points (or 2.44%) for the higher educated one. This large difference seems to be explained by the scarce employment chances among the group of less educated workers, as illustrated by a considerably lower average employment likelihood for the lower educated workers’ comparison group (of 40%) as compared to the one for the higher educated group of nonparticipants (60%), and indicative of less skilled workers having less chances in the primary labour market.

When comparing ES and SBA, I find that ES works best for workers with less than a high-school degree, and that the opposite is true for SBA. I interpret this result in the following way: while compared to non-participation, SBA works best for low-skilled workers, given a choice between participating in ES or SBA, low-skilled workers are better off participating in ES, while more educated workers gain by participating in SBA. This result is consistent with the fact that in an economy with segmented labor markets, offering SBA to unemployed individuals is better than offering no program since participation in SBA improves their success chances in the secondary segment. However, offering ES instead of SBA is more effective in getting low-skilled individuals out of unemployment. Moreover, for workers with access to the primary sector, such as the more educated workers, SBA improves their chances of success compared to ES, consistent with earlyer findings (Almeida and Galasso, 2007).

V.3.3. Sensitivity Analysis

One way to check the robustness of the results is to apply various estimators to the same problem to see whether the results differ. I compared the results obtained by matching to some alternative estimators. Tables 7, 8, and 9 present average impact estimates on various employment outcomes and earnings in Romania using four alternative estimators. The first set of results (first column of Tables 7 through 9) is gross impact estimates, which were not adjusted for observable differences between the participant and non-participants, that is, I use the whole sample of non-participants regardless of whether their baseline characteristics resembled to those of participants. The second set of results (second column of Tables 7 through 9) is net impact estimates, which were adjusted for demographic and regional differences, and earnings, employment, unemployment and training experiences in 1998 using multivariate ordinary least squares regression (when the dependent variable was continuous) or probit regression (when the dependent variable was a binary variable). The covariates included in the OLS and the probit estimations are the same as those used to estimate the propensity scores in Tables 3 through 6, and column 4 of Tables 7 through 9. The third set of results are net impact estimates that were computed as simple differences between the mean outcome of interest for the participant group and the mean outcome for a non-experimentally matched comparison group selected by the same propensity score method described in section IV, however, I did not use any of the pre-earnings, pre-employment, and pre-unemployment history to match participants to non-participants. The fourth set of results is the estimators presented in section V.2. and Table 3.

Below, I summarize the main findings from analysing the sensitivity of the results of participating in either ES or SBA compared to non-participation (Tables 8 and 9, respectively). The most obvious overall result in Table 9 is that the unadjusted impact estimates (column 1) are generally different from the other estimates (columns 2 through 4). In general, the unadjusted impact estimates of SBA were better than the other ones, suggesting that operators “cream off” the most qualified candidates among the unemployed for this programme. This finding is consistent with other analyses of ALMP in transition economies (O’Leary, 1998 and Kluve, Lehmann, and Schmidt, 2001, among others). Moreover, I find that comparing the gross impact estimates with the regression-adjusted estimates (column 1 versus column 2 of Table 9) clearly reduces the positive impact of most of SBA estimates, reflecting that there is an over-representation of individuals with “better” observable characteristics in this group. Comparing columns 2 and 4 provides us with a comparison between results obtained by matching with the standard OLS regression for the continuous dependent variables, and a probit model for the discrete dependent variables. I observe that the results obtained by matching reduce the significance of the SBA estimates. These differences are presumably explained by the parametric restrictions underlying the OLS and probit estimations. Matching allows for heterogeneity in the treatment effect in a more flexible way. In the case of ES participants, I find that although some “cream off” also seems to take place for ES participants, the results are not as strong as those observed for the SBA participants (see Table 8). For instance, regression-adjusting the estimates has little effect on ES findings, leaving most estimates unchanged (columns 1 and 2 of Table 8). Finally, comparing estimates from column 3 and 4 enable us to explore the importance of controlling for pre-earnings, pre-employment, and pre-unemployment history. I find that these variables are important when measuring the effect of the different programmes, as reflected by the fact that excluding them changes the size of impact estimates of both programmes.

⁹ Note that the simple size of participants and non-participants vary because, in the first two columns the whole sample is used, while in the third and fourth columns, the size will depend on the propensity score matching.

I now proceed to summarize the main findings from the sensitivity analysis of the relative impacts of participating in ES compared to SBA (Table 7). Again I find that the unadjusted impact estimates (column 1) are generally different from the other estimates. The unadjusted impact estimates indicate very few differences in the outcomes of ES and SBA participants. The only statistically significant difference is that ES reduced the length of UB receipt by a bit more than half a month. Since the other estimates (columns 2 through 4) reflect a superiority of ES, this result suggests that programme operators are more selective when choosing SBA participants than ES participants. This implies that there is an overrepresentation of individuals with “better” observable characteristics in the group of SBA participants, which is consistent with earlier findings from Tables 8 and 9. In addition, the differences between the OLS and probit estimators on the one hand (column 2), and matching on the other (column 4) are not too large, and overall, both types of estimates are consistent with ES being more successful than SBA. Finally, comparing estimates from the last two columns indicates that controlling for employment and earnings baseline characteristics is relevant.

VI. Conclusion

Recent empirical evidence has found that employment services and small-business assistance programmes are useful active labour market programmes to help get the unemployed back to work. In this paper, I investigate which of these two programmes works best and for whom in Romania. The results indicate that, on average, ES is more successful than SBA in getting the unemployed back to work.

I have explored four alternative theoretical explanations for these findings, and have found empirical evidence from the heterogeneity analysis compatible with: (1) improved job matching theory for ES (based on the results for the younger workers and those living in rural areas); and (2) segmented labor market theory for SBA (based on the results for the loweducated workers and rural workers).

These results suggest the following policy implications. First, I find that offering ES to unemployed workers with good access to the informal job search channel is not a good idea. This finding is consistent with earlier findings (Van der Berg and Van der Klaauv, 2006, among others). However, the novelty of this paper is to provide some guidance on which populations would benefit from ES in economies with large informal sectors. In such countries, ES ought to be targeted to displaced workers with little access to the informal job search channel (such as young workers) or those for whom the informal channel has dried up (such as those living in depressed areas.) Another policy implication is that, in economies with segmented labor markets, offering SBA to unemployed individuals is better than offering no program since participation in SBA improves their success chances in the secondary segment. However, if the policy decision is whether to offer ES or SBA to unemployed workers, ES is more effective in getting low-skilled individuals and young workers out of unemployment, while SBA works best for more educated workers.

To address the question of whether these two programmes were cost-effective from society’s perspective, I can compare the costs per client of each programme with the economic benefits, as reflected in predicted earnings.¹⁰ I estimated that the average cost per client was 123.74 thousand lei for ES and 179.15 thousand lei for SBA.¹¹ To estimate the benefits of the policy, I used the estimated impact of these ALMPs on the usual average monthly earnings of their participants. I preferred using the earnings estimates over the 2000- 2001 period because they are more likely to represent individuals’ earnings than those observed at one point in time. This amounts to an annual sum of 1,047.84 thousand lei for ES, and 4,783.20 thousand lei for SBA (although this estimate was not statistically significant).¹² In both cases, benefits cover by far the cost per client served, indicating that both programmes were cost-effective.¹³ Given that the cost differences between the two programmes were small, and that the heterogeneity analysis suggested that SBA was superior for some subgroups and ES for others, the policy recommendation is to target each programme to those sub-populations most likely to benefit the most from participation.

A caveat in my cost-benefit analysis is that I did not include several potential benefits, such as the possible effects on labour market behaviour of the unemployed prior to participation, or savings in the deadweight losses due to reduced taxes required to pay participants’ future unemployment benefits, among others. Another caveat is that I did not considered the deadweight loss of taxation to finance benefits, subsidies, and operation of programmes, the cost of the leisure forgone while participants are in the programme or employed, and possible displacement effects of non-subsidized workers. Given that I ignore the above mentioned benefits and costs, the cost-effectiveness results have to be taken with some caution. This is especially true for the SBA programme, since this type of programmes can presumably have significant deadweight and displacement costs (although, according to Betcherman, Olivas and Dar, 2004, very few evaluations tend to provide any empirical evidence of them). Unfortunately, I cannot address this question with the data at hand. Future research ought to be directed towards this end.

¹⁰ When measuring cost-effectiveness from society’s perspective, I measure whether aggregate benefits from implementing the policy are greater than the aggregate resources spent by the policy, abstracting from who enjoys its benefits and who bears its costs.

¹¹ This was calculated from data available from USDOL Technical Assistance Support Team.

¹² Estimates were calculated with respect to non-participation.

¹³ Given that benefits that accrue within the observation period are above the costs, I did not use a long-term perspective to estimate cost-effectiveness.

REFERENCES

1. Abadie A., and G. Imbens. 2006. “On the Failure of the Bootstrap for Matching Estimators.” Working Paper, Harvard University.
2. Almeida, R., and E. Galasso. 2007. “Jump-Starting Self-Employment? Evidence Among Welfare Participants in Argentina.” IZA Discussion Paper No. 2902.
3. Ahrend, R., and J.O. Martins. 2003. “Creative Destruction or Destructive Perpetuation: The Role of Large State-Owned Enterprises and SMEs in Romania During Transition.” Post-Communist Economies, 15:3, 331-356.
4. Bates, T. 1999. ‘‘Entrepreneur Human Capital Inputs and Small Business Longevity.’’ The Review of Economics and Statistics 72(4):551–59.
5. Betcherman, G., K. Olivar, and A. Dar. “Impact of Active Labor Market Programmes: New Evidence from Evaluations with Particular Attention to Developing and Transition Countries”. 2004. Social Protection Discussion Paper Series, No. 0402.
6. Benus J., and N. Rodriguez-Planas. “Evaluating Active Labor Markets in Romania.” 2007. UFAE and IAE Working Papers 699-07. Also IZA Discussion Papers 2464.
7. Bonnal, L., D. Fougère, and A. Sérandon. 1997. “Evaluating the Impact of French Employment Policies on Individual Labour Market Histories.” Review of Economic Studies 64 (4), 683-713.
8. Brown, D., J. Earle, D. Lup. 2004. “What Makes Small Firms Grow? Finance, Human Capital, Technical Assistance, and the Business Environment in Romania.” W.E. Upjohn Institute for Employment Research, Staff Working Papers.
9. Caliendo M., R. Hujer, and S. Thomsen. 2005. “The Employment Effects of Job Creation Schemes in Germany- A Microeconomic Evaluation.” IZA Discussion Paper No. 1512, Bonn.
10. Carling, K., and K. Richardson, “The Relative Efficiency of Labor Market Programmes: Swedish Experience from the 1900s,” 2004. Labour Economics, 11: 335-354.
11. Cressy, Robert. 1996. ‘‘Are Business Startups Debt-Rationed?’’ Economic Journal 106(438):1253–70.
12. Costariol, M. 1993. “The Small and Medium-Sized Enterprises in the Transition Process: the Case of Romania.” CNA Veneto and EURO-IN Library, Venice.
13. Dar A., and Z. Tzannatos. “Active Labor Market Programmes: A Review of the Evidence from Evaluations,” 1999. Social Protection Discussion Paper no. 9901. The World Bank, Washington, D.C.
14. Dochia, A. 2000. “New Private Firms Contributions to Structural Change in the Romanian Economy.” RCEP Working Paper No. 12.
15. Earle, J. and C. Pauna. 1996. “Incidence and Duration of Unemployment in Romania.” European Economic Review 49: 829-837.
16. Earle, J. and C. Pauna. 1998. “Long-term Unemployment, Social Assistance and Labour Market Policies in Romania.” Empirical Economics, 23:203-235.
17. Fajnzylber, Maloney, and Montes Rojas, 2006. “Microenterprise Dynamics in Developing Countries: How Similar are They to Those in the Industrialized World? Evidence from México.” The World Bank Economic Review, vol. 20: 389-419.
18. Fawcett, C. 2001. “Latin American Youth in Transition: A Policy Paper on Youth Unemployment in Latin America”. Paper prepared for the “First Technical Meeting on Labor Market Issues.” Panama City, Panama.
19. Gerfin M., and M. Lechner. “Microeconometric Evaluation of the Active Labour Market Policy in Switzerland.” 2002. Economic Journal 112: 854-893.
20. Gerfin M., and M. Lechner, and H. Steiger. 2005. “Does Subsidised Temporary Employment Get the Unemployed Back to Work? An Econometric Analysis of Two Different Schemes.” Labour Economics 12: 807-835.
21. Heckman, J., H. Ichimura, and P. Todd. 1997. “Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme.” Review of Economic Studies, 64(4), 605-654.
22. Heckman, J., and J. Smith. “The Preprogramme Earnings Dip and the Determinants of Participation in a Social Programme: Implications for Simple Programme Evaluation Strategies,” 1999. Economic Journal, 108: 313-348.
23. Imbens G.W. “The Role of the Propensity Score in Estimaing Dose—Response Functions.” 2000. Biometrika 87: 706-710.
24. Karlan D., and M. Valdivia. 2006. "Teaching Entrepreneurship: Impact of Business Training on Microfinance Clients and Institutions," Working Papers 941, Economic Growth Center, Yale University.
25. Kluve, J. 2006. “The Effectivenes of European Active Labor Market Policy.” IZA Discussion Paper No. 2018.
26. Kluve, J., H. Lehmann, and C. Schmidt, “Active Labor Market Policies in Poland: Human Capital Enhancement, Stigmatisation, or Benefit Churning?,” 1999. Journal of Comparative Economics, Vol 27.
27. Kluve, J., H. Lehmann, and C. Schmidt, “Disentangling Treatment Effects of Polish Active Labour Market Policies: Evidence from Matched Samples,” 2002. Discussion Paper no. 3298, April London, UK: Center for Economic Policy Research.
28. Kosanovich, W., and H. Fleck. 2001. “Comprehensive Assessment of Self-Employment Assistance Programmes.” DTI Associates, Inc. Prepared for :US Department of Labor. Office of Workforce Security.
29. Larsson, L., “Evaluation of Swedish youth labour market programmes,” 2003. Journal of Human Resources, 38, 891-927.
30. Lechner, M. “Identification and Estimation of Causal Effects of Multiple Treatments Under the Conditional Independence Assumption.” 2001. In: Lechner, M. Pfeiffer, F. (Eds.), Econometric Evaluation of Active Labour Market Policies.” Physica, Heidelberg, pp. 43-58.
31. Leuven, E., and B. Sianesi. 2003. psmatch2: Stata module to perform Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing, available at: http://ideas.repec.org/c/boc/bocode/s432001.html
32. Lubyova, M., van Ours, J.C. “Effects of Active Labor Market Programmes on the Transition Rate from Unemployment into Regular Jobs in the Slovak Republic”, 1999. Journal of Comparative Economics 27, no. 466:805-37.
33. Martin, J.,and D. Grubb. 2001. “What Works and For Whom: A Review of OECD Countries’ Experiences with Active Labour Market Policies.” Office of Labour Market Policy Evaluation. Working Paper 2001:14.
34. Mitrut, C., and D-L Constantin. 2006. “Current Issues Concerning Regional Policy and SMEs in Romania.” Southern Eastern European Journal of Economics 2: 209-221.
35. O’Leary, C. J., P. Kolodziejczyk, and G. Lazar. “The Net Impact of Active Labour Programmes in Hungary and Poland,” 1998. International Labour Review, 137(3), 321-346.
36. Puhani P. and V. Steiner. “The Effectiveness and Efficiency of ALMP in Poland,” 1997. Empirica 24, p. 209-231.
37. Rees, Hedley, and Anup Shah. 1986. ‘‘An Empirical Analysis of Self-Employment in the U.K.’’ Journal of Applied Econometrics 1(1):95–108.
38. Rosenbaum P., and Rubin, D. “The Central Role of the Propensity Score in Observational Studies for Causal Effects,” 1983. Biometrika 27, no.1:33-60.
39. Rubin, D. "Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies.” 1974. Journal of Educational Psychology 66: 688-701.
40. Sianesi, B. 2004. “An Evaluation of the Swedish System of Active Labour Market Programmemes in the 1990s.” The Review of Economics and Statistics, 86(1), 133-155.
41. Sianesi, B., “Differential Effects of Swedish Active Labour Market Programmes for Unemployed Adults During the 1990s.” 2005. The Institute of Fiscal Studies Working Paper 01/25.
42. Terrell, K., and V. Sorm. “Labor Market Policies and Unemployment in the Czech Republic,” 1999. Journal of Comparative Economics 27, no. 1:33-60.
43. Van der Berg and Van der Klaauw, 2006. “Counselling and Monitoring of Unemployed Workers: Theory and Evidence from a Controlled Social Experiment.” International Economic Review, 47, 895-936
44. Vodopivec, M. “Does the Slovenian Public Work Programme Increase Participants’ Chances to Find a Job?,”1999. Journal of Comparative Economics 27, no. 1:113-30.
45. Walsh, K; M. Kotzeva, E. Dolle, and R. Dorenbos. “Evaluation of the Net Impact of Active Labour Market Programmes in Bulgaria”, 2001. Ministry of Labour and Social Policy, Bulgaria.
46. Woltermann, S. 2002. “Job.Search Methods and Labor Market Transitions.” University of Gottingen. Denmark.
47. World Bank. 2002. “Evaluation of Active Labor Programs in Macedonia. Final Report.” Social Reform and Technical Assistance Project. IDA-2722-MK. World Bank.

Selected Characteristics of ALMP Participants and Non-Participants (Percentages except where noted) Table 1

Standard deviation in parenthesis for continuous variables.

Table 2 Indicators on the quality of the match by ALMP

aThe difference in the number of treated and non-treated in the different rows is ex lained b the fact that I restricted the sam le to have treated and non-treated units come from the same local area (judet) and the two ALMPs under study were not implemented in all of the same judets ( 1 ) Number of treated that is j oining an ALMP programme in 1 999 (2) Number of potential comparisons that is persons who had registered at the Employment Bureau in 1 999 but did not participate in an ALMP (3) Treated as a percentage of potential comparisons (4) Pseudo-R2 from probit estimation of the j oining probability on X giving an indication of how well the regressors X explain the participants probability (5) (6) (7) and ( 1 0) are postmatching indicators on kernel-based matching ( 1 % caliper)

(5) Pseudo-R2 from probit estimation of the j oining probability on X on the matched samples

(6) P-value of the likelihood ratio test after matching After matching the j oint significance of the regres sors is always rej ected B efore matching the j oint significance of the regressors was never rej ected at any significance level with

(7) and (8) Median absolute standardized bias before and after matching median taken over all regressors X Following Rosembaum and Rubin 1 983 for a given covariate X the standardized difference before matching is the difference of the sample means in the full treated and nontreated subsamples as a percentage of the square root of the average of the sample variances in the full treated and nontreated groups The standardized difference after matching is the difference of the sample means in the matched treated that is, the common support and matched nontreated subsamples as a percentage of the square root of the average of the sample variances in the full nontreated groups :

\[B _ {b e f o r e} (X) \equiv 1 0 0. \frac {\overline {{{X}}} _ {1} - \overline {{{X}}} _ {0}}{\sqrt {\left[ V _ {1} (X) + V _ {0} (X) \right] / 2}} \text {and} B _ {a f t e r} (X) \equiv 1 0 0. \frac {\overline {{{X}}} _ {1 M} - \overline {{{X}}} _ {0 M}}{\sqrt {\left[ V _ {1} (X) + V _ {0} (X) \right] / 2}}\]

Note that the standardization allows comparisons between variables X and for a given X comparisons before and after matching.

(9) Number of treated individuals falling outside of the common support (based on a caliper of 1 % )

Table 3 Average Treatment Effects of Programmes on the Employment Experience of their Participants (Percentage points except where noted)

Monthly earnings have been deflated using 1 998 deflator. Monthly earnings are coded as zero if person reported not working at the time of the survey. B old numbers indicate significance at the 5 % level (two- sided test)

Table 4 Average Treatment Effects according to Age (Percentage points except where noted)

Monthly earnings have been deflated using 1 998 deflator. Monthly earnings are coded as zero if person reported not working at the time of the survey . B old numbers indicate significance at the 5 % level (two- sided test) 9 indicates that the difference of the two estimated effects is significant at the 5 % level

Average Treatment Effects according to Type of Region (Percentage points except where noted) Table 5

Monthly earnings have been deflated using 1 998 deflator. Monthly earnings are coded as zero if person reported not working at the time of the survey . B old numbers indicate significance at the 5 % level (two- sided test) 9 indicates that the difference of the two estimated effects is significant at the 5 % level

Table 6 Average Treatment Effects according to Education Achievement (Percentage points except where noted)

Monthly earnings have been deflated using 1 998 deflator. Monthly earnings are coded as zero if person reported not working at the time of the survey . B old numbers indicate significance at the 5 % level (two- sided test) 9 indicates that the difference of the two estimated effects is significant at the 5 % level

Table 7 Sensitivity Analysis Impacts of Employment Services Compared to Small Business Assistance (Percentage points except where noted)

Monthly earnings have been deflated using 1 998 deflator B old numbers indicate significance at the 5 % level (two-sided test)

TABLE 8 SENSITIVITY ANALYSIS Impacts of Employment Services Compared to Non-Participation (Percentage points except where noted)

Monthly earnings have been deflated using 1 998 deflator B old numbers indicate significance at the 5 % level (two-sided test)

TABLE 9 Sensitivity Analysis Impacts of Small-Business Assistance Programme Compared to Non-Participation (Percentage points except where noted)

Monthly earnings have been deflated using 1 998 deflator B old numbers indicate significance at the 5 % level (two-sided test)

APPENDIX A.

The Data

A Romanian private survey firm, Institute of Marketing and Polls (IMAS), was contracted to conduct field surveys during January-February 2002. The study goal was to achieve over 4,000 respondents. Of the 4,839 individuals contacted for interviewing, about 70 percent responded, leaving us with a sample of 3,357 persons. As is common in these type of studies, response rate was slightly higher for participants (72 percent) than for non-participants (68 percent).

Sample Selection

The data used in this study, a random sample of almost 3,357 persons who registered at the Employment Bureau during 1999, was collected during January and February 2002. About two fifth of this sample (1,408 individuals) were ALMP participants of either employment services (ES) or small-business assistance (SBA) whose ALMP contract began in 1999.¹

To obtain a representative sample of ALMP participants, we randomly selected, for each of the programmes, 10% of clients served in the fifteen counties with the largest number of clients served in 1999. These fifteen counties represented 86% of all clients served in 1999. Furthermore, an analysis of the economies of these fifteen judets indicates that they represented a broad spectrum of the Romanian economy with many sectors represented, including heavy industry, mining, agriculture and other sectors. Moreover, these fifteen judets included some of the poorest judets in Romania (Botosoni and Vaslui—north-east region) as well as some judets with substantial natural resources and highly developed industries (Cluj and Maramures—north-west region).

The rest of the sample—the potential comparison group—were 1,949 persons who were registered at the Employment Bureau around the same time and in the same county than participants but who had not participated in an ALMP.² To select non-participants, we first determined, for each of the two ALMPs, the number of participants that were selected for the participant sample in each of the counties. Next, in each county and for each programme, we randomly selected a similar number of non-participants from the same Employment Bureau register list. Following recommendations from Heckman, Ichimura, and Todd, 1997, in addition to draw the comparison group from the same local labour market with respect to participants, the same questionnaire was used for both participants and non-participants.

The timing of events (illustrated by Figure A.1) goes as follows. Some of the workers who registered at the Employment Bureau during 1999 received services from one of the two programmes. The rest of the workers did not receive any of these services. Although it is possible that some of the programme participants may have continued to receive services during the year 2000 (since the maximum duration of the programmes varied between 9 and 12 months), this is quite unlikely since, in practice, the length of these programmes was considerably shorter. During January and February of 2002, we interviewed the selected sample of participants and non-participants. All interviewed persons were asked three types of questions: (1) questions on employment and earnings at the time of the survey, (2) retrospective questions on employment and earnings during the years 2000 and 2001, and (3) retrospective questions on employment and earnings during 1998, prior to participating in the ALMPs.

¹ Based on discussions with programme implementation staff, we determined that contracts that begun in 1999 most accurately reflect the operations of the programmemes. Prior to 1999, the programmemes were new and some of the procedures were not fully implemented. Contracts that begun after 1999 may not be suitable for the evaluation since some may still be in operation or recently finished at the time of the survey and impacts from these contracts may not yet be fully reflected in participants’ outcomes. Thus, our sample was drawn from contracts that started during 1999.

² Non-participants did not receive employment services, nor did they participate in small-business assistance. In addition, non-participants did not participate in the two other ALMPs—training and public services, which were offered at the same time but in a much smaller scale (see BR for more details on these two programmes).

Figure A.1

TIMING OF EVENTS

APPENDIX B

Sensitivity Analysis

One way to check the robustness of the results is to apply various estimators to the same problem to see whether the results differ. I compared the results obtained by matching to some alternative estimators. Tables A.4, A.5, and A.6 present average impact estimates on various employment outcomes and earnings in Romania using four alternative estimators. The first set of results (first column of Tables A.4 through A.6) is gross impact estimates, which were not adjusted for observable differences between the participant and non-participants, that is, I use the whole sample of non-participants regardless of whether their baseline characteristics resembled to those of participants. The second set of results (second column of Tables A.4 through A.6) is net impact estimates, which were adjusted for demographic and regional differences, and earnings, employment, unemployment and training experiences in 1998 using multivariate ordinary least squares regression (when the dependent variable was continuous) or probit regression (when the dependent variable was a binary variable). The covariates included in the OLS and the probit estimations are the same as those used to estimate the propensity scores in Tables 3 through 6, and column 4 of Tables A.4 through A.6. The third set of results are net impact estimates that were computed as simple differences between the mean outcome of interest for the participant group and the mean outcome for a non-experimentally matched comparison group selected by the same propensity score method described in section IV, however, I did not use any of the pre-earnings, pre-employment, and pre-unemployment history to match participants to non-participants.³ The fourth set of results is the estimators presented in section V.2. and Table 3.

Note that the simple size of participants and non-participants vary because, in the first two columns the whole sample is used, while in the third and fourth columns, the size will depend on the propensity score

Below, I summarize the main findings from analysing the sensitivity of the results of participating in either ES or SBA compared to non-participation (Tables A.5 and A.6, respectively). The most obvious overall result in Table A.6 is that the unadjusted impact estimates (column 1) are generally different from the other estimates (columns 2 through 4). In general, the unadjusted impact estimates of SBA were better than the other ones, suggesting that operators “cream off” the most qualified candidates among the unemployed for this programme. This finding is consistent with other analyses of ALMP in transition economies (O’Leary, 1998 and Kluve, Lehmann, and Schmidt, 2001, among others). Moreover, I find that comparing the gross impact estimates with the regression-adjusted estimates (column 1 versus column 2 of Table A.6) clearly reduces the positive impact of most of SBA estimates, reflecting that there is an overrepresentation of individuals with “better” observable characteristics in this group. Comparing columns 2 and 4 provides us with a comparison between results obtained by matching with the standard OLS regression for the continuous dependent variables, and a probit model for the discrete dependent variables. I observe that the results obtained by matching reduce the significance of the SBA estimates. These differences are presumably explained by the parametric restrictions underlying the OLS and probit estimations. Matching allows for heterogeneity in the treatment effect in a more flexible way. In the case of ES participants, I find that although some “cream off” also seems to take place for ES participants, the results are not as strong as those observed for the SBA participants (see Table A.5). For instance, regression-adjusting the estimates has little effect on ES findings, leaving most estimates unchanged (columns 1 and 2 of Table A.5). Finally, comparing estimates from column 3 and 4 enable us to explore the importance of controlling for pre-earnings, pre-employment, and preunemployment history. I find that these variables are important when measuring the effect of the different programmes, as reflected by the fact that excluding them changes the size of impact estimates of both programmes.

I now proceed to summarize the main findings from the sensitivity analysis of the relative impacts of participating in ES compared to SBA (Table A.4). Again I find that the unadjusted impact estimates (column 1) are generally different from the other estimates. The unadjusted impact estimates indicate very few differences in the outcomes of ES and SBA participants. The only statistically significant difference is that ES reduced the length of UB receipt by a bit more than half a month. Since the other estimates (columns 2 through 4) reflect a superiority of ES, this result suggests that programme operators are more selective when choosing SBA participants than ES participants. This implies that there is an over-representation of individuals with “better” observable characteristics in the group of SBA participants, which is consistent with earlier findings from Tables A.5 and A.6. In addition, the differences between the OLS and probit estimators on the one hand (column 2), and matching on the other (column 4) are not too large, and overall, both types of estimates are consistent with ES being more successful than SBA. Finally, comparing estimates from the last two columns indicates that controlling for employment and earnings baseline characteristics is relevant.

APPENDIX C

Additional Heterogeneity Analysis

I have also estimated the effects of ES and SBA with respect to non-participation for the following sub-populations: males, females, with prior unemployment spell below 6 months, and with prior unemployment spell longer than 5 months (shown in the Appendix Tables A.4 and A.5).

I find that, with respect to non-participation, ES improves economic outcomes of participating workers with histories of short-term unemployment compared to those with histories of long-term unemployment. In the case of short-term unemployed workers, ES reduced the likelihood of being employed 6 months during the 2000-2001 period by 7.55 percentage points (or 10.25%). In contrast, ES were not beneficial for the long-term unemployed since it did not affect the employment likelihood (although the impact was a negative 5.02 percentage points, it was not statistically significant).

Finally, I also find that the impact estimates on accumulated employment are larger for females than for males (although the differences between the two subgroups are not statistically significant).

Table A.1 Baseline Demographic and Regional Characteristics of ALMP Participants and Non-Participants, 1998 (Percentages except where noted)

Standard deviation in parenthesis for continuous variables. Monthly earnings have been deflated using 1998 deflator.

Table A.2 Estimation of the Propensity Score (marginal effects)

* significant at 5% level; ** significant at 1% level. Standard errors in parentheses. Pseudo-R² for all 4 specifications are presented in Table column 4.

Table A.3 Description of outcome variables

Note: Earnings are deflated by gross domestic product (base=1998). Earnings are coded as zero if person reported not working at the time of the survey.

Table A.4 Average Treatment Effects according to Pre-Unemployment History (Percentage points except where noted)

Monthly earnings have been deflated using 1 998 deflator B old numbers indicate significance at the 5 % level (two- sided test) 9 indicates that the difference of the two estimated effects is significant at the 5 % level

Table A.5 Average Treatment Effects according to Gender (Percentage points except where noted)

Monthly earnings have been deflated using 1 998 deflator B old numbers indicate significance at the 5 % level (two- sided test) 9 indicates that the difference of the two estimated effects is significant at the 5 % level