Universities from Australia and New Zealand in the 2015 edition of the Shanghai ranking

Transcription

1 DC Technical Report 02/2015 Universities from Australia and New Zealand in the 2015 edition of the Shanghai ranking D. Docampo L. Cram October 24, Abstract The emergence of international academic rankings is one of the most interesting phenomena in the field of comparative analysis of higher education. The growing influence of the Shanghai ranking led its many critics to show strong reluctance in using it as a source of analysis and improvement, mainly because it was generally thought that its results were not reproducible. Once we have found a way to accurately replicate the results of the ranking, we are in a position to shed light into the performance of whole Higher Education systems. This technical report presents the results of Australian and New Zealand universities in the 2015 edition of the Shanghai ranking. Keywords Academic rankings Shanghai arwu world class universities Australian universities Australia Australian Higher Education System New Zealand universities New Zealand New Zealand Higher Education System 1 Introduction The Academic Ranking of World Universities (ARWU) a.k.a the Shanghai Ranking, evaluates the research performance of academic institutions on the basis of numerical measures related to individual outstanding achievements and institutional research throughput. The accessibility of the sources of the raw data and the fact that the hierarchy of universities generated by ARWU roughly aligns with perceptions of the historical and recent performance of elite research universities have contributed to its acceptance and success (Docampo and Cram, 2014). A first hand account of the Shanghai ranking indicators can be found in Liu and Cheng (2005), and is summarized in Section 2 following Docampo (2013). A reliable account of the ranking procedures can be found in Docampo and Cram (2014). For the sake of clarity, a summary of the procedure is also reproduced in section 3. Domingo Docampo Universidade de Vigo, Atlantic Research Center for Information and Communication Technologies; Campus Universitario, Vigo, Spain. ddocampo@uvigo.es On sabatical at Charles Darwin University, Northern Institute, Ellengowan Drive, Brinkin NT 0909 Australia. ddocampo@cdu.edu.au Lawrence Cram Research School of Physics and Engineering, Australian National University, Acton ACT 0200, Australia. Lawrence.Cram@anu.edu.au Charles Darwin University, Northern Institute, Ellengowan Drive, Brinkin NT Lawrence.Cram@cdu.edu.au

2 2 This report analyzes the performance of Higher Education institutions from Australia and New Zealand on the indicators measured by the Shanghai ranking (2015 edition). We have not included three institutions with very small publication counts: Avondale College, Melbourne College of Divinity and the Batchelor Institute of Indigenous Tertiary Education. To collect the necessary data for the computation of the ARWU raw scores, we have checked the electronic sources shown in Section 4. The list of special affiliation names and Web of Knowledge corresponding clues are shown in Section 5. Section 6 shows the procedure for the computation of the ARWU indicators. It is divided into two subsections, the first one dealing with the basic five indicators and the second one explaining how to compute the composed indicator (PCP). Finally, we present in Section 7 the results of 47 universities from Australia and New Zealand on the Shanghai ranking indicators. Data for the 47 universities under analysis have been estimated using the procedures described in Section 6. 2 ARWU ranking ARWU ranks universities on six indicators of academic performance. Individual indicators Alumni Total number of graduates from an institution winning Nobel Prizes or Fields Medals in Mathematics. Award Total number of the staff working at an institution at the time of winning Nobel prizes in the sciences, or Fields Medals in Mathematics. HiCi An equally weighted combination of two sub-indicators: old HiCi and new Hici, for the Highly cited researchers listed in the 2014 and 2001 version of Thompson Reuters Highly Cited Researchers, respectively 1. For ARWU 2015 the score on the old HiCi was the ARWU 2013 score. The highlycited.com web site indicates that highly cited researchers may change their affiliations monthly by changing their ResearcherID affiliation list. Collective indicators N&S Total number of articles published in Science and Nature in the past five years. PUB Total number of articles indexed by Science Citation Index-Expanded and Social Science Citation Index in the previous year. PCP Total scores of the previous five indicators divided by the number of full-time equivalent academic staff. 3 ARWU ranking methodology ARWU ranks universities individually or into bands by sorting on the total score, i.e. the linearly weighted sum of 6 indicator scores derived from the corresponding raw data by transformations that have been calibrated and explained by Docampo (2013). To produce the annual Shanghai ranking report the following steps are required: 1

3 3 1. Assemble new raw data Scan the relevant data sources (see Section 4) for updated information on new Nobel prize winners (every year) and Fields medalists (every four years), new awardees (expected in 2015) and revised affiliations of the current list of HCR for Thompson ISI Highly Cited researchers, and the institutional details of Nature & Science publications and ISI indexed publications in the prior year. Do your best to obtain the institutional FTE (full-time equivalent) faculty numbers used in the calculation of the PCP (per-capita performance) indicator. 2. Assess and determine exceptions Once per decade, rescale indicators for historical Nobel prize winners and Fields medalists according to the decadal aging procedure. Allocate institutions specialized in the humanities and the social sciences to the class that uses special weights to exclude Nature & Science publications. Decide which entire countries and which special institutions will be excluded from the determination of PCP because the relevant values of FTE have not been found. 3. Combine raw data and apply scaling Sum and/or average raw data over the relevant time windows according to the published algorithms. For every indicator other than PCP, multiply each value of the processed raw data by a fixed scaling factor so that Harvard University has a scaled raw score of Calculate an intermediate quantity for PCP by dividing the weighted sum of the scaled raw scores by the FTE for the California Institute of Technology, and apply a scaling factor so that this intermediate quantity for CalTech is Compress the scaled raw data For each indicator, including PCP, compress the dynamic range of the scaled raw data by taking its square root to form the indicator score. 5. Calculate the total score and determine ranking/banding Combine the indicator scores using the relevant indicator weights, and linearly rescale so that the total score for Harvard University is 100. Rank institutions by total score, and determine for publication the ranking sequence, the membership of bands, and the excluded institutions. 4 Electronic sources Data were gathered from the following Electronic Sources: Nobel Prize Laureates available at Data on Field Medalists available at Medal Data on Highly Cited Authors available at Data on scientific production available at Preprint copies of published articles related to this technical report are available at Cram/contributions/ Egret/contributions/ Docampo/contributions/ 5 Affiliation names We have conducted electronic searches using the organizations enhanced (og) names feature of the Web of Knowledge. According to Thomson Reuters (Thomson Reuters,

4 4 2013), og names precisely identify research published from a specific organization using the Organization - Enhanced searching to quantify an organizations output including naming variants. We used those og identifiers with most universities, but made special calls in some cases, as shown in Table 1. It is not unlikely that some disambiguation problems with affiliations may still persist, so we warmly invite comments on any apparent errors or uncertainties. Web of Knowledge call oo=australian catholic univ og=(flinders university south australia or flinders univ south australia) oo=monash univ oo=(rmit univ or rmit or rmit univ melbourne) oo=swinburne univ technol og=(university of canterbury or univ canterbury) og=(university of melbourne or univ melbourne or royal children s hospital melbourne) og=(university of notre dame australia or notre dame univ) og=(university of queensland or univ queensland or queensland ctr mental hlth res) og=(university of sydney or univ sydney or sydney med sch) oo=univ tasmania oo=univ technol sydney oo=(univ western australia or uwa or uwa ocean inst) oo=victoria univ and ad=melbourne Table 1: Special cases : WOK organization (oo) or organization-enhanced (og) names 6 Computation of the indicator scores Final scores depend on raw scores, specifically on their ratio: universities to the best performer institution. Hence, the first step is to evaluate those raw scores on the five non composed indicators: Alumni, Award, HICI, S&N and PUB. For a complete account of the computation of raw scores the reader is referred to Docampo (2013). 6.1 Computation of the five basic indicators For the computation of the non composed indicators (Alumni, Staff, HiCi, S&N, and PUB) we need the raw scores of the best performer. Harvard University achieves the maximum score on the five non composed indicators in the 2015 edition of ARWU. Harvard s (approximate in the case of the S&N and PUB indicators) raw scores are exhibited in Table 2. Let us now summarize the precise operations with the raw scores Alumni Award newhici oldhici S&N PUB Table 2: Harvard University raw scores in 2015 described in steps 4 and 5 of Section 3. Let H be the number of points (raw score) of Harvard on any of the five indicators, and X the number of points achieved by any other institution. Estimated scores are computed through the same formula 2 : X EST = 100 (1) H 2 In the case of the HiCi indicator the equation applies to both the old and new HCR counting separately.

5 5 Institution FTE Faculty ASL LLC FTE Australian Catholic University Australian National University Bond University Central Queensland University Charles Darwin University Charles Sturt University Curtin University of Technology Deakin University Edith Cowan University Federation University Australia Flinders University Griffith University James Cook University La Trobe University Macquarie University Monash University Murdoch University Queensland University of Technology Royal Melbourne institute of Technology (RMIT) Southern Cross University Swinburne University of Technology University of Adelaide University of Canberra University of Melbourne University of New England University of New South Wales University of Newcastle, Australia University of Notre Dame Australia University of Queensland University of South Australia University of Southern Queensland University of Sydney University of Tasmania University of Technology, Sydney University of the Sunshine Coast University of Western Australia University of Western Sydney University of Wollongong Victoria University Table 3: Full Time Equivalent Staff of Australian universities in ARWU Source: Australian Government, Department of Education. 6.2 PCP computation There is no easy way to access reliable data on equivalent full-time faculty for all the institutions listed in ARWU. However, in the case of Australian universities, there are public and reliable data published by the Department of Education. The number of Full Time Equivalent Academic Staff used in ARWU for Australian universities is computed as the aggregate number of Full Time Equivalent Faculty at the levels above Senior Lecturer and Lecturer (Level C). For the 2015 edition ARWU has apparently made use of FTE data from the following document: Selected Higher Education Statistics Staff 2013 Full-time Equivalence (AUSTRALIAN-GOVERNMENT, 2014).

6 6 Data from the 39 Australian universities included in the analysis are shown in Table 3, where the acronyms of the three columns stand for: ASL: Full Time Equivalent number of Faculty above Senior Lecturer. LLC: Full Time Equivalent number of Faculty at Lecturer Level C. FTE: Full Time Academic Staff for the calculations of the ARWU data. FTE figures for New Zealand universities have been left unchanged from ARWU 2013 to ARWU As of the estimation of FTE values for universities not included in the ARWU 500 list we refer the reader to our previous report (Docampo and Cram, 2013). The FTE values are shown in Table 4. Institution year Academic Staff FTE Staff Massey University , The University of Auckland , University of Canterbury University of Otago , Victoria University of Wellington Auckland University of Technology , Lincoln University University of Waikato Table 4: Institutional figures for FTE academic and total staff: universities from New Zealand. Source: Annual reports of New Zealand universities Now, to get the PCP values for all the universities included in the analysis, both from Australia and New Zealand, we first compute the weighted sum of the squares of the estimated scores. The procedure has changed due to the introduction of the old and new Hici sub-indicators, as documented in Docampo et al (2015). In the 2014 and 2015 editions of ARWU, the weighted sum is evaluated through Equation 2; let s call that value WSS. WSS = 0.1 ( Alumni 2 + newhici 2 + oldhici 2) ( Award 2 + N&S 2 + PUB 2) (2) Let s call WSSCT the value of the squared sums for Caltech, the highest score in the indicator PCP. Let FTECT be the Full Time Equivalent Staff of Caltech. To estimate the PCP indicator of the university X, with a Full Academic Staff of FTEX and weighted scores sum of WSSX, the following operation will be carried out: PCP = 100 WSSX FTEX WSSCT FTECT = 100 FTECT WSSX WSSCT FTEX (3) To compute formula 3 we introduce the value of 278 for the FTE Academic Staff of Caltech. In 2015, WSSCT , hence we have FTECT W SSX = PCP = WSSCT F T EX (4)

7 7 7 Results We show in Figure 1 the results of the 47 institutions under analysis 3. It is perhaps worth pointing out that the entrance score in ARWU 2015 was close to 9.5. We show in Figure 2 the the 47 institutions ranked by PCP. References AUSTRALIAN-GOVERNMENT (2014) Selected Higher Education Statistics Staff 2013 Full-time Equivalences. TRIM Reference D14/199109, URL full-time-equivalence, downloaded from the Department of Education and Training server on August 19th Docampo D (2013) Reproducibility of the Shanghai academic ranking of world universities results. Scientometrics 94(2): Docampo D, Cram L (2013) Universities from Australia and New Zealand and the 2013 edition of the Shanghai ranking. ATLANTIC Technical Report 2/2013, URL Docampo D, Cram L (2014) On the Internal Dynamics of the Shanghai ranking. Scientometrics 98(2): Docampo D, Egret D, Cram L (2015) The effect of university mergers on the Shanghai ranking. Scientometrics 104(1): Liu NC, Cheng Y (2005) Academic ranking of world universities: Methodologies and problems. Higher Education in Europe, 30(2): Thomson Reuters (2013) Web of knowledge v5.9 release notes. URL 3 To test the accuracy of our estimation we provide in the table our figures for the universities in ARWU 500, so the reader may check that the errors are almost negligible

8 8 ranking ARWU ARWU INDICATORS local ranking score Universities world Fig. 1: Universities from Australia alu awd and newhnew oldhzealand hici S&N ranked PUB by pcptotal ARWU2013 score 2015 RDIF SDIF 44 yes yes Univ Melbourne yes yes Australian Natl Univ yes yes Univ Queensland yes yes Univ Western Australia yes yes Monash Univ yes yes Univ Sydney yes yes Univ New South Wales yes yes Univ Adelaide yes yes Univ Auckland yes yes Macquarie Univ yes yes Univ Wollongong yes yes Univ Otago yes yes Curtin Univ yes yes Univ Tasmania yes yes James Cook Univ yes yes Univ Newcastle yes yes Griffith Univ yes yes Flinders Univ yes yes Univ Technol Sydney yes yes Swinburne Univ Tech no yes Deakin Univ no yes Queensland Univ Tech yes no La Trobe Univ yes no Univ Canterbury no no Univ New England yes no Victoria Univ Wellington yes no Massey Univ no no Univ Western Sydney no no Univ South Australia no no Murdoch Univ no no RMIT no no Southern Cross Univ no no Charles Darwin Univ no no Univ Sunshine Coast no no Univ Waikato no no Charles Sturt Univ no no Univ Canberra no no Edith Cowan Univ no no Victoria Univ Melbourne no no Central Queensland Univ no no Australian Catholic Univ no no Lincoln Univ no no Auckland Univ Tech no no Fed Univ Australia no no Bond Univ no no Univ S Queensland no no Univ Notre Dame SCORE FTE

9 9 ranking ARWU ARWU INDICATORS pcp ranking pcp score Universities world 2013 Fig : Universities from Australiaalu andawd Newnewh Zealand oldhranked hici S&N by the PUBscore on the 2013 PCP 2015 indicator rdif sdif 44 yes yes Univ Melbourne yes yes Univ Queensland yes yes Australian Natl Univ yes yes Univ Western Australia yes yes Univ Otago yes yes Univ Adelaide yes yes Monash Univ yes yes Univ Sydney yes yes Macquarie Univ yes yes James Cook Univ yes yes Univ Wollongong yes yes Univ New South Wales yes yes Curtin Univ no no Charles Darwin Univ yes yes Univ Tasmania yes yes Swinburne Univ Tech yes yes Univ Newcastle yes yes Flinders Univ yes yes Univ Auckland no no Univ Sunshine Coast no no Univ New England no yes Deakin Univ yes yes Griffith Univ no no Southern Cross Univ yes yes Univ Technol Sydney yes no La Trobe Univ no no Univ South Australia yes no Univ Canterbury no yes Queensland Univ Tech no no Murdoch Univ no no Fed Univ Australia no no Univ Western Sydney yes no Victoria Univ Wellington no no Victoria Univ Melbourne no no Univ Canberra no no Charles Sturt Univ no no Edith Cowan Univ no no Univ Waikato no no RMIT yes no Massey Univ no no Australian Catholic Univ no no Central Queensland Univ no no Bond Univ no no Univ S Queensland no no Lincoln Univ no no Univ Notre Dame no no Auckland Univ Tech PCP FTE