Fundación de Estudios de Economía Aplicada

Differential Grading Standards and University Funding: Evidence from Italy by * Manuel Bagues Mauro Sylos Labini Natalia Zinovyeva *** DOCUMENTO DE TRABAJO 2008-07 Serie Capital Humano y Empleo CÁTEDRA Fedea – Santander

February 2008

* Universidad Carlos III ** IMT Lucca Institute for Advanced Studies *** BETA at University of Strasbourg and FEDEA

Los Documentos de Trabajo se distribuyen gratuitamente a las Universidades e Instituciones de Investigación que lo solicitan. No obstante están disponibles en texto completo a través de Internet: http://www.fedea.es. These Working Paper are distributed free of charge to University Department and other Research Centres. They are also available through Internet: http://www.fedea.es.

ISSN:1696-750X

Jorge Juan, 46 28001 Madrid -España Tel.: +34 914 359 020 Fax: +34 915 779 575 infpub@fedea.es

Di§erential Grading Standards and University Funding: Evidence from Italy

Manuel Baguesy Mauro Sylos Labiniz Natalia Zinovyevax

January 31, 2008

Abstract

This paper documents that grades vary signiÖcantly across Italian public universities and degrees. We provide evidence suggesting that these di§erences reáect the heterogeneity of grading standards. A straightforward implication of this result is that university funding schemes based on studentsíacademic performance do not necessary favour universities that generate higher value added. We test this for the case of the Italian funds allocation system, which rewards universities according to the number of exams passed by their students. We Önd that university departments that rank higher according to this indicator actually tend to be signiÖcantly worse in terms of their graduatesíperformance in the labour market.

JEL ClassiÖcation: I2, J31, J64.

Keywords: Higher Education, Grading Standards.

yUniversidad Carlos III

zIMT Lucca Institute for Advanced Studies

We would like to thank Stijn Kelchtermans, Reinhilde Veugelers and participants at seminars at the UniversitÈ Louis Pasteur, SantíAnna School of Advance Studies, 2nd EIASM workshop on Process of Reform of University Systems, XXII Jornadas de Economia Industrial, University of Alicante, the V Brucchi Luchino workshop and the CESifo Workshop on Innovation and Higher Education for their comments on an earlier draft of this paper. The empirical analysis of this paper would not have been possible without the data and the help provided by ISTAT (the Italian Statistical O¢ce). The econometric analysis was carried out at the ADELE Laboratory in Rome.

xBETA at University of Strasbourg and FEDEA.

1 Introduction

In a number of European countries - including Italy, Spain and France - university grading standards are presumed to be similar across institutions. This presumption justiÖes the legal value that is typically given to university titles and explains why public funding of universities is increasingly related to the number of diplomas or grade points assigned by universities.

This paper empirically investigates the existence of di§erences in grades and grading standards across Italian universities. It exploits three editions of a survey run on a representative sample of Italian graduates. The survey contains information about graduatesí academic and labour market performance, as well as a large set of individual characteristics, including high school grade, province of origin and various measures of family background. Conditional on this extensive set of controls, we Önd that grades vary considerably across universities and disciplines. Evidence from graduatesílabour market performance and post-university external professional qualiÖcation exams (îabilitazione pro fessionaleî) suggests that these variations in grades do actually reáect di§erences in grading standards and not true changes in studentsíquality. Indeed, we Önd a signiÖcant negative correlation between departmentsíaverage grades and the labour market outcomes of their graduates, i.e. graduating from a high grading department leads to a higher unemployment probability and lower wages. As well, graduates from departments with high average grades do not have higher chances to get professional qualiÖcations in external examinations.

A straightforward policy implication follows from the above results. Policy makers should be very cautious about using studentsí academic performance as a proxy for university value added. If, as shown in this paper, grading standards vary signiÖcantly across departments and universities, rewarding universities with high graduating rates may lead to undesirable consequences.

We test the relevance of this hypothesis using the main output variable that the Italian government takes into account in order to Önance universities: the number of full-time equivalent students (FTE), which measures the number of exams that students have passed in a given year. Consistently with our predictions, we Önd that graduates from universities with a relatively higher number of FTE perform signiÖcantly worse in the labour market and do not obtain better results in professional qualiÖcation exams. The evidence thus suggests that a Önancing scheme which is meant to reward those universities that produce higher value added is, instead, favouring those universities with lower standards.

The rest of the paper is organized as follows. Section 2 presents the existent literature on grading standards. Section 3 describes the data and the main variables use in the empirical part. Section 4 presents the empirical analysis and discusses policy implications. Finally, Section 5 summarizes the main results and provides the conclusions.

2 Background

The issue of educational standards has been widely discussed in the economics literature both from a theoretical and an empirical perspective. Grading standards may vary over time and across higher education institutions for a number of reasons. Standards may adjust to the quality of students (Strenta and Elliott, 1987). As well, professors may ináate grades to escape negative evaluations by students, whose opinions matter for tenure and promotion decisions (Nelson and Lynch, 1984; Siegfried and Fels, 1979). Some departments may also increase their grades to Öll poorly attended courses that might otherwise be canceled (Dickson, 1984; Staples, 1998). In addition, Freeman (1999) argues that given the institutional constraints that prevent, within each university, a system of áexible money pricing for those courses with di§erent expected earnings, instructors and departments may act strategically to manage enrolment by adjusting the time and the e§ort cost of achieving a given grade. More generally, Costrell (1994) notes that if institutions choose grading standards in a decentralized way a free rider problem may arise, as high standards might not be fully appropriated by each institution. De Paola and Scoppa (2007) point out that, in a decentralized setting, educational standards might be also ináuenced by the existence of labour market distorsions.

An extensive empirical literature has documented the existence of variations in grades over time across American universities and colleges. In particular, there has been, at least since the 1960ís, an increase in the grades issued by American universities, coupled with the perception of a deteri oration in academic standards (Kolevzon, 1980; Sabot and Wakeman-Linn, 1991; Anglin and Meng, 2000). As well, there exists a line of studies which provide evidence on divergence in grades across di§erent disciplines (Sabot and Wakeman-Linn, 1991; Freeman, 1999; Dickson, 1984). For instance, Sabot and Wakeman-Linn (1991) report average grades received by students in several disciplines in eight American colleges and universities, Önding a clear division of colleges into high and low grading departments.

Di§erences in grades are also observed in Europe. A report on the development of exam grades in Germany Önds that the average grades vary widely across universities (Wissenschaftsrat, 2004). Several authors also observe that in the UK degree results vary according to institution. For example, Chapman (1997) studies the degree results from 1973 to 1993 for eight disciplines and Önds a clear tendency for certain universities to award consistently higher percentages of top degrees in all disciplines with respect to the corresponding national average. As far as Italy is concerned, Boero et al. (2001) report that grades tend to vary signiÖcantly across degrees and regions.

Unfortunately, in most of these studies it is di¢cult to disentangle whether the observed di§erences in grades reáect di§erent qualiÖcations and performance or, conversely, di§erences in teaching and assessment practices. As Boero et al. (2001) put it, whether the observed di§erences îindicate use of di§erential standards across the di§erent institutions or genuine institutional di§erences in valueadded cannot be identiÖed from the data.î (Boero et al., 2001) (p.27). However, assessing whether the observed heterogeneity in grades stems form di§erent grading standards or from di§erences in graduatesítrue performance might be important for a number of reasons. Both in Europe and in the US variations in grading standards might be problematic in the presence of informational asymmetries about the quality of graduates and/or institutions. Most importantly, in many European countries the institutional design of higher education typically requires the homogeneity of grading standards across institutions. This assumption explains why titles have a legal value and are legally required for many occupations and, as well, why several countries, such as Italy and Denmark, have adopted output funding schemes based on the number of diplomas or grade points each higher education institutions delivers.

3 Data

We investigate the potential existence of di§erences in grading standards across Italian universities and Öelds of studies using a very detailed dataset concerning Italian university graduates which allows to observe their socioeconomic background, high school grades, university performance and, Önally, their outcomes in the labour market and in professional qualiÖcation exams.

More speciÖcally, our main data is drawn from three distinct but almost identical surveys named Indagine Inserimento Professionale Laureati (Survey on University-to-Work Transition) run in 1998, 2001, and 2004 on individuals that graduated in 1995, 1998, and 2001 respectively.¹

The target samples consist of 25716 individuals in 1998, 36373 individuals in 2001, and 38470 individuals in 2004. They represent respectively the 25%, 28.1%, and 24.7% of the total population of university graduates in Italian universities. The response rates were of 64.7%, 53.3%, and 67.6% for a total of 17326, 20844 and 26006 respondents. In all three years the sample is stratiÖed according to sex, university and obtained degree and in the analysis below all estimations are performed using stratiÖcation weights. We exclude from the sample graduates from physical education studies and from the so-called ìlaurea primo livelloî, since they were surveyed only in the 2001 edition (501 and 475 observations respectively).

¹ Di§erences may stem from the di§erent interviewing technologies used in the surveys: in 1998 ISTAT mailed paperbased questionnaires, while in 2001 and 2004 graduates were Örst contacted by mail and then questions were asked following the so-called C.A.T.I. (Computer Assisted Telephone Interview) technique.

As other European continental countries, Italy has a system of open admission into public universities: most departments are obliged to admit every applicant, without being allowed to set up any entry restrictions. This system is common to all public universities and all disciplines except medicine, veterinary and architecture (84% of our sample). Only 7% of graduates of our sample attended a private university. For a number of reasons, grading standards are likely to be di§erent in those universities and Öelds of study that can select their students; thus, we further restrict our sample to those departments that cannot select students. This reduces the total size of the sample to 61844 observations.

The surveys provide information on (i) individual characteristics that are predetermined with respect to college choices and outcomes, (ii) college-related individual indicators and (iii) labour market outcomes. The Örst set of variables includes information on the individual socio-demographic background such as gender, nationality, number of siblings, province of residence before college enrolment, parentsíeducation and employment when respondent was around 14 years old, the situation of military service obligations before attending university and self-reported high school curricula - high school grade and type of school attended. The second includes university-related indicators: the type of de gree and university attended, educational outcomesói.e. Önal grade obtained and the number of years spent for the completion of the degree²óand additional information such as occupational status during studies, changes in the degree followed, attainment of an other degree and whether the respondent originated from a town or province other than the one where her university was located. O¢ cial grades range from 66 points and a maximum of 110 e lode. Third, the survey collects self-reported information about a number of occupational outcomes three years after graduation. Among others, it is possible to observe whether the graduate is employed, whether the job requires a university degree, her wage and several indexes of job satisfaction. Table 1 depicts descriptive statistics for the key individual variables.³

In addition to the individual information, we use data on several college characteristics. Fields of study are aggregated in twelve di§erent disciplines.⁴ Table 2 displays descriptive statistics at the department level on the share of Full Time Equivalent Students (FTE) - the main measure used by the Ministry for distribution of ordinary Önancial funds across universities - and ordinary Önancial funds themselves.⁵ Finally, we also consider a number of demographic and economic indicators at the provincial level such as GDP, total population and unemployment.

4 Empirical Analysis

To begin with, we investigate whether grades vary signiÖcantly across disciplines and universities. Then, we analyze whether the potential existence of di§erences in grades across institutions stems from di§erences in grading standards or, rather, it reáects genuine di§erences in institutional value added. Finally, we analyze how the existence of di§erential grading standards a§ects the funding of Italian universities.

² In Italy the Önal grade is calculated as the sum of the grades obtained by the graduate during her courses plus the grade received for the so-called degree dissertation (tesi di laurea). Any student whose Önal grade is higher than 110 obtains what is known as ì110 e lodeî. For simplicity, in the empirical analysis reported bellow the potential existence of grades above 110 has been disregarded. The results obtained using a tobit regression, available upon request, are very similar to the ones reported here. Also note than in the Italian education system in the analyzed period students were not constrained either in time or in the number of trials taken for passing exams.

³ The unemployment rate for graduates in our sample is 14.7 percent. It is consistent with the OECD 2003 data suggesting that 13.6 percent of Italian graduates aged 25-29 not being in education are unemployed. Italian graduates experience disadvantage in terms of early performance in the labour market as the overall unemployment rate among individuals aged 25-29 is 10.4 percent (OECD, Education at a Glance 2005)

⁴ The aggregated disciplines are Agriculture, Architecture, Chemistry and Pharmacy, Economics and Statistics, Engineering, Law, Literature, Medicine and Surgery, Pedagogy, Political and Sociological Studies, Sciences, Veterinary. In what follows the term department stands for the corresponding disciplinary unit within a particular university.

⁵ Information on the number of FTEs comes from the Osservatorio per la valutazione del sistema universitario (1998). See Perotti (2005) for detailed information on how the number of FTEs a§ects universitiesífunding.

4.1 Grades

The grades obtained by a university graduate are likely to be related to a number of personal characteristics including parental background and pre-university ability. We estimate the following model:

\[G _ {i} = \beta X _ {i} + \gamma D _ {f} + \delta D _ {u} + \alpha_ {t} + \varepsilon_ {i t f u},\tag{1}\]

where is a measure of the academic results obtained by individual i and is a set of individual characteristics, as described in Table 1, including dummies for the province where the individual lived before attending university and gdp and unemployment rates at the provincial level. and are the sets of dummies corresponding, respectively, to the Öeld of study (or discipline) and university. The time dummy controls general changes across time. Finally, the error term captures any remaining factor a§ecting academic performance.

Column 1 of Table 3 shows the results of an OLS estimation of equation (1) where the dependent variable is the Önal aggregate grade obtained by the individual during her studies. In addition to individual predetermined characteristics, the regression also controls for the number of extra years taken to graduate.⁶ The e§ect of individual characteristics is largely consistent with those obtained by previous studies.⁷ We also observe that grades are positively correlated with unemployment rates. This is consistent with the work of Dornbusch et al. (2000) and Di Pietro (2006), who point out that local labor market conditions may ináuence studentsídecisions. Lower unemployment rate may encourage a number of students to devote less e§ort to university in order to take advantage of the improved labor market conditions.

Moreover, grades tend to vary to a large extent both across universities and across faculties.⁸ Figure 1 shows the set of estimated university dummy coe¢cients, i.e. the component of an individualís grade that is statistically explained by her attendance to a given institution, conditional on her observable characteristics, discipline, geographical origin and the time she took to graduate. Universities are ordered from left to right according to their o¢cial university code, lower codes corresponding in general to northern locations and bigger codesíto southern ones. The positive slope suggests that, as one moves across universities from the north to the south of Italy, grades - conditional on individualís observable characteristics - tend to increase. Similarly, Figure 2 shows how grades vary across disciplines. This Ögure suggests that there are notable di§erences in the size of discipline Öxed-e§ects on grades with Engineering, Economics and Statistics, Chemistry and Pharmacy, and Law being among the lowest grading and Agriculture, Literature, Pedagogy and Architecture among the highest grading disciplines.

The second column of Table 3 displays the results of the above model when we use as dependent variable the number of extra years taken to graduate. The previous Öndings are largely conÖrmed. Results concerning the variation of university and discipline dummy coe¢cients in this case are quali tatively very similar to the ones of Figures 1 and 2 and are available upon request.

Two important caveats apply to the above estimations. First, note that the estimation builds on the information provided by individuals with similar characteristics, including geographic origin, but who decide to attend di§erent departments. This strategy provides consistent estimates as long as these individuals do not di§er signiÖcantly in their unobservable characteristics. Second, another important concern regards the endogeneity of the sample. In fact, we observe only graduates, but not drop-outs.⁹

⁶ The di¢culty of each particular program could be described in two ways: as the time that is necessary in order to complete a program and obtain a certain grade or as the Önal grade that an individual will obtain if she takes a given period time to graduate.

⁷ See, for instance, Boero et al. (2001) who studies the determinants of academic success using the ISTAT survey corresponding to year 1998.

⁸ The inclusion in equation (1) of university and discipline Öxed a§ects signiÖcantly the explanatory of the model. Including university dummies increases the R-square from 21.68 percent to 28.54 percent. The subsequent inclusion of the discipline Öxed e§ects raises R-square to 39.82 percent.

⁹ This shortcoming generates two problems. First, the factors that a§ect the grades obtained by those students that do not manage to graduate could di§er from the factors a§ecting the grades obtained by graduates. A key assumption is, therefore, that the grades obtained by graduates consistently reáect, conditional on observables, the grades obtained

4.2 Di§erences in Quality or Di§erences in Grading Standards?

The above results show that grades, conditional on graduatesípredetermined characteristics, tend to vary greatly across universities and Öelds. In principle, these di§erences could be due either to the value added by universities or to their grading standards. To investigate these alternative explanations we use two additional proxies of quality. First, we exploit the indicators detecting graduatesílabour market performance. If higher grades reáect higher value added, graduates from high grading departments should perform better in the labour market. Second, we use the outcomes of external professional qualifying exams. In Italy, they are compulsory for a number of professional occupations. If higher grades reáect higher quality, graduates from high grading institutions should display higher passing rates.

4.2.1 Labour Market Outcomes

Graduatesílabour market performance is likely to be a§ected by a number of socioeconomic characteristics by their Öeld of study and by the university attended : Equation (2) analyzes this relationship:

\[L _ {i} = \alpha_ {t} + \beta X _ {i} + \gamma D _ {f} + \delta D _ {u} + \varepsilon_ {i t f u},\tag{2}\]

Table 3 presents the estimation results of this model when labour market performance is measured, three years after graduation, by the employment status (column 3), the wage (column 4) and the probability of Önding a job which requires a university degree (column 4) of those individuals who are in the labour force.¹⁰ If, on the one hand, female perform better in terms of grades, on the other, they exhibit a worse performance in the labour market. Similarly, foreign graduates are not able to transform their higher academic performance into better labour market outcomes.

In addition to personal characteristics, the institution attended is a key predictor of future labour market performance. Figure 3 depicts the estimates of universitiesíÖxed e§ects on wages conditional on the individualsíobservable characteristics, their geographical origin and discipline. Again, universities are ordered from left to right according to their o¢cial ISTAT code number, which increases as we move from the north to the south of Italy. Thus, the negative slope observed in Figure 3 suggests that northern universitiesí graduates tend to earn higher wages than southern universitiesí ones. A similar pattern is observed if we restrict our analysis to graduates who Önished their studies on time. Including the region of actual residence does not a§ect the pattern observed in the histogram. Thus, our results are not driven by unobserved labour market conditions. The picture is similar if we use as dependent variable graduatesíemployment status: given two students with similar socioeconomic backgrounds and geographical origins, those who graduate from a northern university are more likely to be successful in the labour market than those who graduates from a southern university, even if they end up working in the same region. This result is consistent with previous studies that also Önd a premium for graduating in the North (Brunello and Cappellari, 2005; Pozzoli, 2006; Makovec, 2007). Moreover, we observe signiÖcant di§erences across disciplines in terms of wages. In particular, conditional on graduating in the same university, high school grades, individual background and province of origin, graduates in Engineering, Economics and Statistics, Chemistry and Pharmacy and Medicine are likely to have higher wage with respect to graduates in Veterinary, Literature, Law and Pedagogy (Figure 4).

by those students that dropped out before graduation. Second, a more subtle problem is related to the fact that the very same unobservable characteristics ñ i.e. talent or grading standardsñ that a§ect grades do also a§ect selection into the sample, this is, graduation. This makes the usual selection-based-on-observables assumption likely to fail. Still, the nature of the problem allows us to make some predictions about the direction of the bias, at least among the cohort of students that graduate on time. Any factor that generates an increase in grades would presumably increase the size of this cohort. The new sample would include individuals which are, conditional on observables, relatively worse in unobservables. This suggests that the e§ect of factors that generate an increase in grades will tend to be underestimated or, in other words, that the estimated coe¢cients will tend to be a lower bound of their true value.

^{1 0} Results are essentially unchanged if we consider instead the whole population of graduates, including also those graduates that do not look for a job.

As shown in Figures 1 and Figure 3, while grades tend to be higher in southern universities, labour market outcomes tend to be better for those that graduate in the North. With the exception of Law departments¹¹ , the same pattern generally holds at the discipline level: high grading disciplines tend to provide lower labour market opportunities for their student. This descriptive evidence suggests that there exists a negative correlation between departmentsí grades and their graduatesí labour market outcomes, both across universities and across Öelds of study. Bellow we formally test this statistical relationship.

First, we estimate an equation, in which -as in equation (1)- we analyze the determinants of grades, but we substitute the discipline and university dummies with a set of dummies speciÖc to each university department separately for 1995, 1998 and 2001

\[G _ {i} = \beta X _ {i} + \gamma D _ {t d} + \alpha_ {t} + \varepsilon_ {i t d}.\tag{3}\]

Second, using the department dummies coe¢cients ( ), we decompose individualsí grades into two components: b; reáecting the (conditional) average grade obtained by individuals that bgraduated within the same cohort and department and (2) the relative grade obtained by the individual, calculated as a di§erence between the actual grade and the estimated grade conditional on personal characteristics . Third, we study how these components a§ect labour market eperformance measures:

\[L _ {i} = \alpha_ {t} + \beta X _ {i} + \eta \widetilde {G} _ {i} + \mu \widehat {\gamma} _ {t d} + \varepsilon_ {i t d},\tag{4}\]

Table 4 presents the estimation results of equation (4) using three di§erent measures of graduates labour market performance: the probability of being employed (columns 1 & 2), the probability of Önding a job that requires a university degree (columns 3 & 4) and the expected wage (columns 5 & 6). Conditional on their observable personal characteristics, the number of years spent in university and the discipline chosen, students that obtain higher grades relative to their classmates are more likely to be employed three years after graduation and, if employed, tend to earn a signiÖcantly higher wage. However, the departmentís average grade has the opposite e§ect. Students that graduated from universities where average grades were higher are signiÖcantly less likely to be employed (column 1) and, if employed, they are not more likely to have a job that requires a degree (column 3) and do not tend to earn more (column 5). Results remain essentially the same if we include among the controls graduatesíclass size or the region of graduatesíresidence when being interviewed. In columns 2, 4 and 6 we compare individuals who graduated in the same university but who had enrolled into di§erent Öelds. We Önd that those individuals who obtained their degree in departments with relatively higher average grades are signiÖcantly less likely to Önd a job which requires being a graduate (column 4) and actually tend to earn signiÖcantly less (column 6). As in the previous analysis, controlling for the region of current residence does not have a signiÖcant e§ect on the estimates.

The above results may help to rationalize the puzzling correlation that arises when we compare the academic performance of Italian graduates with their performance in the labour market. A simple descriptive analysis of the data provided by the ISTAT surveys on year 1995, 1998 and 2001 graduates reveals that those individuals that had obtained higher grades in university do not obtain higher wages later on (see Table 5, columns 1, 2 & 3).¹² In the last edition of the survey, it turns out that grades are negatively correlated with earnings: graduates who obtained lower grades tend to earn relatively more. Of course, as our above results suggested, this negative relationship is driven by the di§erent grading standards that departments apply. As expected, once we take into account the university and the department from which an individual has graduated the expected positive relationship between grades and salary is re-established (though, signiÖcant only at 11%).

^{1 1} Law is a quite particular case. Note that in Italy graduates in Law must spend at least two years as interns before taking professional qualiÖcation exams and becoming lawyers.

^{1 2} Boero et al. (2001) already point out that the grades of 1998 graduates show no correlation with their wages

4.2.2 Professional QualiÖcation Exams

An additional way to test whether higher grades reáect higher quality or simply di§erent standards is to exploit the outcomes of post-university professional qualiÖcation exams (îabilitazione professionaleî). These exams are granted by o¢cial professional organizations and are meant to certify that a given graduate holds a minimal set of competencies for a given profession. They are not compulsory but are required in order to perform legally a number of professions. The set of professions for which an exam is required includes Architects, Chemists, Accountants, Physicians, Psychologists or Engineers.¹³

The ISTAT survey allows to observe whether a given graduate has passed the corresponding external qualiÖcation exam within three years of her graduation. A potential source of bias of this measure might arise from the fact that we only observe whether individuals succeeded in the professional qualiÖcation exam, but not whether they took it and failed. This problem is likely to be bigger in those disciplines where graduates have other professional possibilities that do not require an o¢cial qualiÖcation.

As it is shown in Table 1, about half of respondents have passed an external qualiÖcation exam after graduation. However, the distribution of this percentage across Öelds is not homogenous¹⁴ : the probability that a graduate pass the qualiÖcation exam ranges from 0 to 40 percent in 66 percent of courses, from 40 to 60 in 4 percent of courses and from 60 to 100 percent in 30 percent of courses. In other words, there exist a big group of courses in which more than 60 percent of graduates do not ever pass the exam, another group of courses in which more than 60 percent of graduates pass the exam and very few courses that could not be attributed either to the Örst or to the second group. In order to minimize the problem of self-selection described above we restrict the analysis to those occupations where graduates have a very limited scope for professional possibilities unless they pass the external qualiÖcation exam. In what follows only the latter group of courses, namely, the one in which more than 60 percent of graduates passed the exam (mainly Engineering and Chemistry courses), is considered.

Column 6 of Table 3 shows the relationship between individual characteristics and the probability of success in qualiÖcation exams. As expected, success in this exam is closely related to graduates quality, as measured by high school grades and other socioeconomic characteristics.

In Table 6 we analyze the relationship between university grades and performance in external qual iÖcation exams. We Önd that conditional on the department and university attended, those graduates that obtained relatively better grades than their classmates in university are signiÖcantly more likely to pass the qualiÖcation exams. Then we investigate whether the (conditional) average grade of all individuals that graduated within the same cohort and department , as deÖned in the previous subsection, has a similar positive e§ect on graduatesíperformance in professional qualiÖcation exams estimating the following regression:

\[A _ {i} = \alpha_ {t} + \beta X _ {i} + \eta \widetilde {G} _ {i} + \mu \widehat {\gamma} _ {t d} + \varepsilon_ {i t d}.\tag{5}\]

As shown in column 2 of Table 6, while we still Önd that within each department better students are more likely to succeed in professional qualiÖcation exams, in general graduates from departments with higher average grades tend to be less successful in professional qualiÖcation exams. Given that in these Öelds the lack of success in external exams is associated with signiÖcantly lower employment rates and with signiÖcantly lower probabilities of Önding a job which requires a degree, our results suggest that the variations in the department-average component of grades are not likely to reáect better quality.¹⁵

^{1 3} For a complete list of Italian professional organizations and details of respective exams see http://it.wikipedia.org/wiki/Albo_professionale.

^{1 4} Degree course deÖnes graduatesí specialization within a certain discipline. Each disciplinary Öeld on average o§ers around 10 di§erent degree courses.

^{1 5} Those graduates who passed the professional exam have a probability of Önding a job that matches the knowledge acquired in university which is 11 percentage points higher than the rest of individuals in the sample. Note also that if individualsíunobserved ability in university performance was positively correlated to individualsíunobserved ability in professional qualiÖcation exams, the estimated coe¢cient must be considered as an upper bound of its true value.

4.3 Di§erential Grading Standards and the Funding of Italian Universities

Before 1993 the Italian national ministry of education was in charge of Öxing the total amount of funds, their shares across public universities and their allocation across disciplines. Its decisions were largely made on historical bases and were sometimes a§ected by distinct deals with single institutions and faculties within institutions. In 1993 a reform was approved allowing each university to become an autonomous entity with its own budget to be allocated across distinct disciplines (law n.537/1993). Moreover, discretion was replaced by a complex set of rules, which in the short run left about 90 per cent of the big bulk of public funding to be assigned on historical basis and the rest to be allocated via an equalization component (EC). The latter is supposed to progressively substitute the former. The EC objective is two fold: Örst, to reduce public funding disparities across university and across disciplines and, second, to incentivate quality. On the incentives side, the EC seeks to reward the quality of teaching linking funding to the number of exams passed by enrolled students. Technically, the funds depend positively on the share of full-time equivalent students (FTE), which is deÖned as the ratio between the number of exams that students passed and the number of exams that students should have taken. See Perotti (2002) for details.

In principle, the measure of quality based on the share of FTE students might be subject to at least two problems. First, it fails to take into account the initial quality of students. Universities that attract students of better quality will tend to perform relatively better even if they fail to provide better education. Second, in the absence of quality assurance mechanisms, the FTE might capture both the students true quality and the easiness (or grading standards) of a given institution.¹⁶ In fact, the evidence provided in the previous section suggests that the relationship between the average grades issued by di§erent universities and the performance of their graduates in the labour market or in qualiÖcation exams is, if any, negative. A straightforward implication of this result is that Önancing universities based on their self-evaluated academic performance does not necessary reward those universities that generate a higher value.

Table 7 shows the relationship between graduatesílabour market outcomes and the share FTE students in the department where they graduated, conditional on graduatesísocioeconomic background and pre-university measures of quality. While the number of FTE students is meant to proxy the quality of a department, we Önd a strong and signiÖcant negative relationship between this measure and graduatesílabour market outcomes, as measured by occupation rates (column 1) and obtaining a job which requires a university degree (column 2). We also Önd no signiÖcant relationship whatsoever between the share of FTE students and graduatesíwages (column 3) or their performance in professional qualiÖcation exams and (column 4).

To sum up, FTE fails to capture quality of institutions, at least as measured by graduatesíperformance in the labour market and in professional qualiÖcation exams.

5 Conclusion

In recent years a number of European Countries, including Italy, have adopted output funding schemes based on the number of diplomas or grade points each institution delivers. One of the preconditions for such systems to be e§ective in providing quality enhancing incentives is ensuring homogeneity of educational quality and grading standards across institutions. Otherwise, as noted by Jacobs and Van der Ploeg (2005), this practice might undermine incentives to improve educational quality, as in most cases the quantity rather than the quality of output is rewarded due to the di¢ culties in measuring the later.

In this paper we analyze grading standards across Italian universities and disciplines. More specifically, we study the performance of several cohorts of Italian graduates in the labour market and in external qualiÖcation exams and analyze how it relates to their performance in university. We Önd that, conditional on a large set of individualsí observable characteristics that includes geographical origin, high school grade, and socioeconomic background, graduates from high grading departments tend to perform signiÖcantly worse in the labour market. Moreover, graduates from high grading uni versities are less likely to succeed in external qualifying exams that are required for many professional activities. These results suggest that the signiÖcant variations in grades that can be observed in Italy across disciplines and universities reáect to a large extent di§erences in grading standards.

^{1 6} In 1996 and 1999 two distinct kinds of evaluating committees were established to preserve quality: a National Committee (Comitato Nazionale per la Valutazione del Sistema Universitario) and several Internal Committees (Nucle di Valutazione Interna). However, as convincingly argued by Perotti (2002), their objectives are too vague and they turned out to be to be largely ine§ective.

In line with this evidence, we also Önd that the output measure of university quality that has been adopted by the Italian Ministry of Education to allocate funds across universities - i.e. the number of full-time equivalent students (FTE) deÖned as the ratio between the number of exams that students passed and the total number of exams that they should have taken - is negatively correlated with graduatesílabour market outcomes.

This Önding rises concerns on the e§ectiveness of such funding mechanisms. In the light of this evidence, the implementation of quality ensuring mechanisms - such as a system of external examiners as in the UK - should be seriously considered as a necessary complement to any output funding scheme. Additionally, given that obtaining objective evaluations of external examiners might be itself problem atic and costly, a more radical policy option may involve fostering reputation e§ects in the market for higher education. This goal may be approached in di§erent ways, for instance, by allowing universities to select their students and, simultaneously, promoting student mobility, by letting universities set tuition fees and introducing e¢cient student loan systems.¹⁷

References

1. Anglin, P. M. and R. Meng (2000). "Evidence on Grades and Grade Ináation in Ontarioís Universities". Canadian Public Policy, XXVI(3), 361-368.
2. Boero, G., A. McKnight, R. Naylor and J. Smith (2001). "Graduates and the Graduate Labour Market: Evidence from the UK and Italy". Lavoro e Relazioni Industriali, 131-172, 2001.
3. Brunello, G. and L. Cappellari (2005). "The Labour Market E§ects of Alma Mater: Evidence from Italy". IZA Discussion Paper, No. 1562. Forthcoming in Economics of Education Review.
4. Chapman, K. (1997). "Degrees of Di§erence: Variability of Degree Results in UK Universities". Higher Education, 33(2), pp. 137-153.
5. Costrell, R. M. (1994). "A Simple Model of Educational Standards". The American Economic Review, 84(4), pp. 956-971.
6. De Paola, M. and V. Scoppa (2007). "Returns to Skills, Incentives to Study and Optimal Educational Standards". Journal of Economics, 92(3), 229-262.
7. Dickson, V.A. (1984). "An Economic Model of Faculty Grading Practices". Journal of Economic Education, 6 (Summer),197-203.
8. Di Pietro, G. (2006). "Regional labour market conditions and university dropout rates: Evidence from Italy". Regional Studies: The Journal of the Regional Studies Association, 40(6), 617ñ630.
9. Dornbush, R., A. Gentilini and F. Giavazzi (2000). "Italian Labour Force Participation: disguised unemployment on campus". Paper presented at the Annual Meeting of the European Economic Association, Bolzano.
10. Freeman, D.G. (1999). "Grade Divergence as a Market Outcome". Journal of Economic Education, 30(4), 344-351.

^{1 7} See Mas-Colell (2003) and Mas-Colell (2004) for a thorough discussion on reforms that might foster competition and reputation e§ects in the European higher education space.

1. Jacobs, B. and F. Van der Ploeg (2005). "Guide to Reform of Higher Education: A European Perspective". Economic Policy, 535-592.
2. Kolevzon, M.S. (1981). "Grade Ináation in Higher Education: A Comparative Study". Research in Higher Education, 15(3), 195-211.
3. Makovec, M. (2007). "Does it Pay to Study Far from Home? Explaining the Returns to Geographic Mobility of Italian College Graduates". Universitat de Alicante, mimeo.
4. Mas-Colell, A. (2003). "The European Space of Higher Education: Incentive and Governance Issues." Rivista di Politica Economica, November-Dicember, 9-27.
5. Mas-Colell, A. (2004). "The European Higher Education Space: Funding Issues". Third MEMS Lecture, Masterís Program in Economics and Management Science, Humboldt University, Berlin, 30 April 2004.
6. Nelson, J. P. and K. A. Lynch (1984). "Grade Ináation, Real Income, Simultaneity, and Teaching Evaluations". Journal of Economic Education, Winter, 21-37.
7. OECD (2005). Education at a Glance. Paris, Organization for Economic Co-operation and Development.
8. Osservatorio per la valutazione del sistema universitario (1998). "Líevoluzione della domanda di for mazione universitaria: studenti, laureati e studenti equivalenti". Rome, Italy.
9. Perotti, R. (2002). "The Italian University System: Rules vs. Incentives". Paper presented at the Örst conference on Monitoring Italy, ISAE, Rome, January.
10. Pozzoli, D. (2006). "The Transition to Work for Italian University Graduates: Determinants of the Time to Obtain the First Job". Milan Catholic University, mimeo.
11. Sabot, R. and J. Wakeman-Linn (1991). "Grade Ináation and Course Choice". Journal of Economic Perspectives, 5(1), 159-170.
12. Siegfried, J. J. and R. Fels (1979). "Research on Teaching College Economics: A Survey". Journal of Economic Literature, September, 923-969.
13. Staples, B. (1998). "Why Colleges Shower Their Students With Aís". New York Times, March 8.
14. Strenta, A. C. and R. Elliott (1987). "Di§erential Grading Standards Revisited". Journal of Educational Measurement, 24(4), 281-291.
15. Wissenschaftsrat (2004). Empfehlungen zur Reform des Hochschulzugangs [On-line]. http://www.wissenschaftsrat.de/texte/5920-04.pdf.

Table 1: Descriptive Statistics - Individual Characteristics

Notes:The number of observations is 61844. (*) In this case the median value is reported instead of the mean. Value 4 means that 4 or more extra years to graduate have been employed. (**) The number of observations with non-missing wage is 37552.

Table 2: Descriptive Statistics - Department Characteristics

Notes: In 2001 there were 410 di§erent departments. (*) In this case the statistics are reported at the university level in billions of lire. Note that the ordinary Önancial funds are only available for public universities. (**) This is the ratio of the number of professors to the total number of non-delayed students.

haracteristics and Performance in University, Labour Market and External Professiona

le _3:(con^tinu Notes:signiÖcantat10%;signiÖcantat5%;signiÖcantat1%.Forprobitregressionsmarginale§ectsatmeanvaluesarereported.Standarderrorsin

The _E§ect of Grades on Labour M^arket private universities and departments with constrained adm^ission are excluded. ^z Variables listed in Table ; signiÖcant at _5%; signiÖcant at _1%. For probit regressions marginal ^e§ects at mean values are re

Table 5: The (Puzzling) Relationship between Grades and Wages

Notes: signiÖcant at 10%; signiÖcant at 5%; signiÖcant at 1%. Standard errors in parentheses. Students from private universities and departments with constrained admission are excluded. y The coe¢cient shows the e§ect of a 10 point increase in grade. z Variables listed in Table 3 are included among the regressors.

Table 6: The E§ect of Grades on Perfomance in External QualiÖcation Exams

Notes : signiÖcant at 10%; signiÖcant at 5%; signiÖcant at 1%. Marginal e§ects at mean values are reported. Standard errors in parentheses. Students from private universities and departments with constrained admission are excluded. z Variables listed in Table 3 are included among the regressors.

Table 7: The Relationship between the share of Full Time Equivalent Students (FTEs) and Labour Market Performance

Notes: signiÖcant at signiÖcant at 5%; signiÖcant at 1%. Standard errors in parentheses. The coe¢cient shows the e§ect of an increase of 10 Full Time Equivalent Students in 1995. All variables listed in Table 3 are included among the regressors. Only students that graduated in 1998 from a public universities and from a department with open entry have been considered

mark. The error bars indicate _th^e conÖdence intervals at the ^{5% s} controls include individual characteristics, university and tim^e ta represent discipline dumm^ies obtained from ^a OLS regression where dependen