METHODOLOGY OF RANKING
The main goal of determining the ranking of higher education organizations is to create a healthy competitive environment among the HEI, stimulate the improvement of the level and quality of their scientific and pedagogical activity, effectively use the scientific potential of the teaching staff, achieve high indicators in accordance with international criteria and standards of teaching quality and to create a basis for training highly qualified personnel for the industries of the real sector of the economy.
ARIA’s ranking based on this methodology manual and the criteria for determining the ranking of the activities of higher educational organizations is determined based on 15 indicators distributed among 5 areas:
- TEACHING— The quality (competence) indicator of professors and teachers is 15%.
- RESEARCH & EDUCATION IMPACT — The effectiveness of scientific and pedagogical activities — 20%
- TRANSFER OF KNOWLEDGE — 20%
- INTERNATIONAL ENGAGEMENT — 20%
- SUSTAINABLITY AND EMPLOYABALITY — Conditions created for training and quality of graduates — 25%
In the process of designing ARIA’s methodology, the practice of world’s leading ranking agencies was carefully analyzed. In particular, through the analysis of the methodologies of such international ranking agencies as QS, THE, ARWU, best practice has been incorporated into ranking indicators as well as specific indicators were worked out.
For example, according to the updated methodology, professional development of the teaching staff indicator included the number professors and teachers with master’s degrees obtained in foreign universities that are included in the TOP-1000 list of prestigious international ranking organizations. Such indicators as survey of professors and teachers about conditions for scientific and creative activity and number of intellectual property license agreements have been introduced.
The main objectives of determining the ranking of higher education institutions include: training qualified personnel who are in high demand in the labor market, based on regular improvement of higher education; provide the competent education authorities, personnel customers and the public with objective information on the quality of educational services provided in higher education institutions, their scientific and educational processes and capabilities; ensure cooperation between higher education institutions and other highly ranked higher education institutions in the world and help them achieve high results in accordance with international standards; gradually implement the necessary measures to ensure that higher education institutions that have recorded high indicators in the national ranking system are included in the list of 1000 prestigious higher education institutions in the world.
Formation of a database for determining the ranking of higher education institutions
Formation of a database for determining the HEI ranking is the main process, and its formation is carried out before September 1 of the year following the reporting year. Accreditation and Rating International Agency (ARIA) will create a special information system integrated into the relevant databases, which will eliminate human intervention in maintaining the ranking of higher educational organizations and will also ensure automatic collection and analysis of data in real time.
Ranking data is entered by responsible department and personnel of each Higher Educational Institution. The survey of staff and students will be conducted remotely in order to assess the quality of education, spiritual and educational environment, conditions for learning and engaging in scientific, creative and sports activities among teachers and students in an educational organization. Also, employers of organizations will be surveyed in order to assess the knowledge and skills of higher education graduates.
The teaching staff questionnaire will consist of the following four parts:
- a fair and transparent management system of the educational organization;
- a creative environment created in the educational organization;
- conditions for effective educational and scientific work have been created;
- social and spiritual environment in the higher education organization.
The student questionnaire will consist of the following four parts:
• conditions created for education;
• quality of classes;
• objective assessment of students’ knowledge;
• social and spiritual environment in the higher education organization.
Online survey will be conducted among employers to determine the knowledge, qualifications and skills of graduates of higher education institutions. The survey results are taken into account in the process of ranking.
Calculation and publication of the results of the HEI ranking
The results of ranking of higher education organizations are determined by the ARIA Ranking Committee based on the collected data to determine the HEI ranking and the results of the surveys. The calculation of ranking points is carried out based on the criteria for determining the ranking of higher education institutions.
Methodology for calculating national ranking points for educational organizations the calculation of points for ranking indicators is carried out as follows: the educational organizations with the highest index for the relevant ranking indicator are determined;
the educational organizations with the highest performance are awarded the highest score indicated for this indicator;
the scores of all other educational organizations are calculated relative to the indicator of educational organizations with the highest score for this indicator;
the sum of points calculated for each indicator determines the ranking of educational organizations.
The ranking of educational organizations with the same sum of points is determined by priority based on the points received for the criterion with the highest number of points in the section of approved criteria. If the points for the priority criterion are equal, each of them will be assigned a place.
For educational organizations that do not have a graduate or the indicators of the evaluation criteria do not correspond to the activities of the educational organization, the corresponding indicators are not taken into account by adding a coefficient corresponding to the value of the total score.
If during the analysis cases of falsification of ranking data posted in the information system by educational organizations are identified, penalty points in the amount of 5% of the sum of points awarded to the educational organization for this ranking indicator is applied for each identified case.
