Performance Evaluation of Indian Technical Institutions Using Promethee-gaia Approach

It has now become an important issue to evaluate the performance of technical institutions to develop better research and enrich the existing teaching processes. The results of such performance appraisal would serve as a reference point for decisions to choose a particular institution, hire manpower, and provide financial support for the betterment of students and underperforming institutions. The performance of institutions responsible for promotion of technical education in a vast country, like India also needs to be assessed for its journey to share a major role in global economy. In this paper, an integrated approach combining PROMETHEE (Preference Ranking Organization METHod for Enrichment Evaluation) and GAIA (Geometrical Analysis for Interactive Aid) methods is applied for evaluating the performance of 20 National Institutes of Technology (NITs) in India with respect to nine pivotal criteria. It is observed that NIT, Tiruchirappalli and NIT, Warangal are the two best performers based on the last five years data. On the other hand, NIT, Patna takes the last position in the ranking list of 20 NITs suggesting allocation of more resources for its efficient management and subsequent development. Placement of the graduated students from those NITs is found to be the most important criterion.


Introduction
In the age of mechanics, technical education is essential to run our factories and fields of production.The need for technical education in India is no longer a subject of debate as it plays a vital role in our national and global development.Indian technical education system is one of the largest educational systems in the world.After independence in 1947, based on the recommendations of the Planning Commission in 1955 to promote technical education in India and develop India as a leader in science and technology, 14 Regional Engineering Colleges (RECs) were first established between 1959 and 1965.The main aim behind setting up those technical institutions was to provide the required technical manpower for the industrial projects being contemplated during the 2 nd Five-Year Plan (1956)(1957)(1958)(1959)(1960)(1961).Later, three other new RECs were established.In 2003, all RECs became National Institutes of Technology (NITs).In addition, the Central Government of India had taken over three other institutes and converted them into NITs.In 2010, the government had announced setting up ten more new NITs in the remaining states/union territories, leading to establishment of NITs in every state in India.It means that there are now altogether 30 NITs in India.
As the uniform quality output has become the prime concern today, therefore, performance evaluation and ranking of these NITs have now been a key research issue.The mechanism used to evaluate NITs plays an important role in guiding the development of and the government's financial support for each NIT.The evaluation results would serve as a reference point for decisions to choose a NIT, employ suitable resources, identify underperforming NITs and provide direction for subsequent improvement.It would also help the NITs develop better research and teaching processes.As all the stakeholders (students as well as researchers) want to get the optimal benefits at the shortest period of time and at an economical cost to have quality education, this is the high time to analyze and evaluate the performance of these NITs.With this aim, this paper focuses on the application of integrated PROMETHEE (Preference Ranking Organization METHod for Enrichment Evaluation) and GAIA (Geometrical Analysis for Interactive Aid) methods for assessing the performance of 20 NITs in India with respect to nine evaluation criteria.The weights of the considered criteria are determined using Shannon's entropy method.Here, PROMETHEE II method provides the total ranking of all the 20 considered NITs, and GAIA method acts as a graphical decision aid to identify the best NIT along with its positive dimensions and show the effects of changing criteria weights on the final ranking of these NITs.It also searches out the worst performing NIT along with its major weaknesses where special emphasize needs to be provided.

Review of the Past Researches
Johnes and Johnes (1995) applied data envelopment analysis (DEA) to investigate the technical efficiency of UK university departments of Economics as producers of research, and concluded that DEA had a positive contribution to make in the development of meaningful indicators of university performance.Johnes (1996) examined the possibility of constructing measures of performance of UK universities, and developed a methodology in the framework of production theory using multiple regression techniques to estimate the relationship between the outputs and inputs of the universities.Athanassopoulos and Shale (1997) examined the comparative efficiency of higher education institutions in the UK using DEA model.The application of DEA model to 45 universities in the UK shortlisted a subset of six institutions showing satisfactory performance across alternative efficiency tests.McMillan and Datta (1998) reported the results of using DEA model to assess the relative efficiency of 45 Canadian universities.Korhonen et al. (2001) proposed a systematic approach to analyze academic research performance at universities and research institutes.It was concluded that the developed approach would enable a university to allocate resources more efficiently to its research units.Avkiran (2001) used DEA 105 method to examine the relative efficiency of Australian universities, and concluded that the university sector was performing well on technical and scale efficiency, but there was scope for improving performance on fee-paying enrolments.Moreno and Tadepalli (2002) used DEA technique for evaluating the efficiency of academic departments at a public university.Using DEA method, Lopes and Lanzer (2002) evaluated the performance of 58 departments in a Brazilian university and showed that 15 of those departments had low performance.Abbott and Doucouliagos (2003) applied non-parametric techniques to estimate technical and scale efficiency of individual Australian universities.Flegg et al. (2004) used DEA method to examine the technical efficiency of 45 British universities in the period 1980/81-1992/93.Casu and Thanassoulis (2006) applied DEA model to identify practices leading to cost-efficient central administrative services in UK universities.Reichmann and Sommersguter-Reichmann (2006) proposed a framework for assessing the technical efficiency of 118 university libraries from Australia, Austria, Canada, Germany, Switzerland and the United States.The DEA model was chosen to analyze the performance differentials of university libraries.Johnes (2006) applied DEA method to 2547 Economics graduates from UK universities in 1993 in order to assess teaching efficiency.Fandel (2007) presented a solution for redistributing funds for teaching and research among the universities in North Rhine-Westphalia in Germany according to some specific criteria.Kao and Hung (2008) applied DEA method to assess the relative efficiency of the academic departments at National Cheng Kung University in Taiwan, considering total credit hours, publications and external grants as outputs, and number of personnel, operating expenses and floor space as inputs.Worthington and Lee (2008) investigated the productivity growth in 35 Australian universities using nonparametric frontier techniques, taking into account full time equivalent academic and non-academic staff, non-labor expenditure, and undergraduate (UG) and postgraduate (PG) student load as the inputs, and UG, PG and Ph.D. completions, national competitive, industry grants and publications as the outputs.Tyagi et al. (2009) evaluated the performance efficiencies of 19 academic departments of Indian Institute of Technology (IIT), Roorkee through DEA technique.Chen and Chen (2010) developed an innovation support system for Taiwanese higher education institutes to comprehensively evaluate their innovation performance.The decision-making trial and evaluation laboratory (DE-MATEL), fuzzy analytic network process (FANP) and technique for order preference by similarity to an ideal solution (TOPSIS) were utilized considering the interdependence and relative weight of each measurement criterion.Giannoulis and Ishizaka (2010) developed a three-tier Web-system for customized ranking of British universities using ELECTRE III (ELimination and Et Choice Translating REality) method.Lee (2010) developed an intellectual capital evaluation model as a university assessment scheme in Taiwan.Pouris and Pouris (2010) identified the international standing of the South African universities in various scientific disciplines, compared them with their standing over time and focused on the consequences relevant to higher education policy.Kuah and Wong (2011) presented a DEA model for jointly evaluating the relative teaching and research efficiencies of universities.Abramo et al. (2011) proposed an application of DEA methodology for measurement of technical and allocative efficiency of university research activity.Agha et al. (2011) applied DEA model to assess the relative technical efficiency of the academic departments, considering operating expenses, credit hours and training resources as the inputs, and number of graduates, promotions and public service activities as the outputs.Li (2011) used DEA method to analyze and evaluate the output efficiency, scale efficiency and deficiency of 42 universities' human resources.Wu et al. (2012) determined the performance evaluation indices for higher education for ranking 12 private universities in Taiwan.A combined approach using analytic hierarchy process (AHP) and VIKOR (VlseKriterijumska Optimizacija I Kompromisno Resenje) method was proposed.Das et al. (2012) focused on the performance evaluation and ranking of seven IITs in respect to stakeholders' preference using an integrated model of fuzzy AHP and COPRAS (COmplex PRoportional ASsessment).It was observed that the performance of two IITs would need considerable improvement.Hashemkhani Zolfani and Safaei Ghadikolaei (2013) applied DEMATEL for research on cause and effect relations of perspectives of balanced scorecard, ANP to calculate weights of indices in perspectives and VIKOR for ranking five universities in Iran.In Table 1, a concise representation of the performance evaluation methodologies adopted, sample where the evaluation was performed and criteria/sub-criteria as considered by the past researchers is provided.It is found that most of the past researchers adopted DEA methodology for assessing the performance and ranking of various universities/institutions/libraries/research centers.But the mathematical models as required for deployment of DEA technique are too complex to comprehend and it is able to shortlist only the efficient decision-making units/ alternatives.When a complete ranking preorder of the considered alternatives/options is desired, it fails to fulfill this requirement.Lengthy linear programming formulations often need to develop for its solution.For this reason, an attempt is made in this paper to augment a simple integrated approach combining PROMETHEE and GAIA methods to evaluate the performance of 20 NITs in India.

PROMETHEE Method
The PROMETHEE (Preference Ranking Organization METHod for Enrichment Evaluation) is a multi-criteria decision-making (MCDM) method developed by Brans et al. (Brans and Vincke, 1985;Brans et al., 1986;Brans and Mareschal, 1994).It is a simple ranking method in conception and application compared to other MCDM techniques.

109
The PROMETHEE method can be applied to various decision-making situations where the best alternative/option needs to be chosen from a finite set of feasible alternatives based on several conflicting criteria.An excellent review on the application of PROMETHEE method in various domains of engineering and management can be available in (Behzadian et al., 2010).The PROMETHEE method starts with the following decision (evaluation) matrix: (1) The PROMETHEE method can be applied to various decision-making situations where the best alternative/option needs to be chosen from a finite set of feasible alternatives based on several conflicting criteria.An excellent review on the application of PROMETHEE method in various domains of engineering and management can be available in (Behzadian et al., 2010).The PROMETHEE method starts with the following decision (evaluation) matrix: where g j ( a i ) shows the performance of i th alternative on j th criterion, m is the number of alternatives and n is the number of criteria.Implementation of PROMETHEE method requires two additional information, i.e. a) relative importance or weights/priorities of the considered criteria, and b) the decision maker's preference function for comparing the contribution of the alternatives with respect to each criterion.The AHP or Shannon's entropy method can be applied for determination of the criteria weights.The preference structure of PROMETHEE method is based on pair-wise comparisons.In this case, the deviation between the evaluations of two alternatives on a particular criterion is considered.For small deviation, the decision maker would allocate a small preference to the best alternative and even possibly no preference if the decision maker considers that this deviation is negligible.The larger the deviation, the larger is the preference.These preferences are real numbers varying between 0 and 1.This means that for each criterion, the decision maker considers the following preference function: For beneficial criteria, this function gives the preference of 'a' over 'b' for the observed deviations between their evaluations on criterion g j ( .).The preference equals to 0 when the deviations are negative.
For non-beneficial criteria, the preference function should be reversed as follows: A generalized criterion, {g j ( .), P j ( a, b )} is associated to criterion g j ( . ) and it is to be defined for each criterion.In order to facilitate selection of a specific preference function, six basic types of preference function, i.e. usual criterion, U-shape criterion, where g j ( a i ) shows the performance of i th alternative on j th criterion, m is the number of alternatives and n is the number of criteria.
Implementation of PROMETHEE method requires two additional information, i.e. a) relative importance or weights/priorities of the considered criteria, and b) the decision maker's preference function for comparing the contribution of the alternatives with respect to each criterion.The AHP or Shannon's entropy method can be applied for determination of the criteria weights.The preference structure of PROMETHEE method is based on pair-wise comparisons.In this case, the deviation between the evaluations of two alternatives on a particular criterion is considered.For small deviation, the decision maker would allocate a small preference to the best alternative and even possibly no preference if the decision maker considers that this deviation is negligible.The larger the deviation, the larger is the preference.These preferences are real numbers varying between 0 and 1.This means that for each criterion, the decision maker considers the following preference function: For beneficial criteria, this function gives the preference of 'a' over 'b' for the observed deviations between their evaluations on criterion g j ( .).The preference equals to 0 when the deviations are negative.
For non-beneficial criteria, the preference function should be reversed as follows: A generalized criterion, {g j ( .), P j ( a, b )} is associated to criterion g j ( . ) and it is to be defined for each criterion.In order to facilitate selection of a specific preference function, six basic types of preference function, i.e. usual criterion, U-shape criterion, V-shape criterion, level shape criterion, V-shape with indifference criterion and Gauss-ian criterion are proposed to the decision maker.In some of these preference functions, different threshold parameters ( p, q or s ) need to be set by the decision maker.As the shapes of preference functions and their parameters are chosen by the decision maker, it exerts clear advantages and features of PROMETHEE method (Podvezko and Podviezko, 2010).
As soon as the evaluation matrix, g j ( . ) is developed, and the relative importance (weight), w j and generalized criterion, {g j ( .), P j ( a, b )} are defined, PROMETHEE method becomes ready for implementation.PROMETHEE method is based on pair-wise comparisons where the aggregated preference indices are defined as follows (Brans and Vincke, 1985): Page 8 of 25 V-shape criterion, level shape criterion, V-shape with indifference criterion and Gaussian criterion are proposed to the decision maker.In some of these preference functions, different threshold parameters ( p, q or s ) need to be set by the decision maker.As the shapes of preference functions and their parameters are chosen by the decision maker, it exerts clear advantages and features of PROMETHEE method (Podvezko and Podviezko, 2010).
As soon as the evaluation matrix, g j ( . ) is developed, and the relative importance (weight), w j and generalized criterion, {g j ( .), P j ( a, b )} are defined, PROMETHEE method becomes ready for implementation.PROMETHEE method is based on pairwise comparisons where the aggregated preference indices are defined as follows (Brans and Vincke, 1985):  are computed for each pair of alternatives in the evaluation matrix, a complete outranking graph, including two arcs between each pair of nodes (alternatives), can be obtained.In this method, each alternative 'a' faces ( m -1 ) number of other alternatives in the evaluation matrix.Now, the following two outranking flows can be defined: Positive outranking flow: Negative outranking flow: The positive outranking flow expresses how an alternative 'a' outranks all other alternatives.The higher the value of φ + ( a ), the better is the alternative.The negative outranking flow expresses how an alternative 'a' is being outranked by the others.Lower value of φ -( a ) signifies better alternative.In PROMETHEE I method, the partial ranking ( P I , I I , R I ) is obtained from the positive and negative outranking flow where π ( a, b ) expresses the degree with which 'a' is preferred to 'b' over all the criteria and π ( b, a ) represents how 'b' is preferred to 'a'.In most of the cases, there are criteria for which 'a' is better than 'b', and criteria for which 'b' is better than 'a', and consequently, π ( a, b ) and π ( b, a ) are usually positive.
As soon as the values of π ( a, b ) and π ( b, a ) are computed for each pair of alternatives in the evaluation matrix, a complete outranking graph, including two arcs between each pair of nodes (alternatives), can be obtained.In this method, each alternative 'a' faces ( m -1 ) number of other alternatives in the evaluation matrix.Now, the following two outranking flows can be defined: Page 8 of 25 V-shape criterion, level shape criterion, V-shape with indifference criterion and Gaussian criterion are proposed to the decision maker.In some of these preference functions, different threshold parameters ( p, q or s ) need to be set by the decision maker.As the shapes of preference functions and their parameters are chosen by the decision maker, it exerts clear advantages and features of PROMETHEE method (Podvezko and Podviezko, 2010).
As soon as the evaluation matrix, g j ( . ) is developed, and the relative importance (weight), w j and generalized criterion, {g j ( .), P j ( a, b )} are defined, PROMETHEE method becomes ready for implementation.PROMETHEE method is based on pairwise comparisons where the aggregated preference indices are defined as follows (Brans and Vincke, 1985):


are computed for each pair of alternatives in the evaluation matrix, a complete outranking graph, including two arcs between each pair of nodes (alternatives), can be obtained.In this method, each alternative 'a' faces ( m -1 ) number of other alternatives in the evaluation matrix.Now, the following two outranking flows can be defined: Positive outranking flow: Negative outranking flow: The positive outranking flow expresses how an alternative 'a' outranks all other alternatives.The higher the value of φ + ( a ), the better is the alternative.The negative outranking flow expresses how an alternative 'a' is being outranked by the others.Lower value of φ -( a ) signifies better alternative.In PROMETHEE I method, the partial ranking ( P I , I I , R I ) is obtained from the positive and negative outranking flow V-shape criterion, level shape criterion, V-shape with indifference criterion and Gaussian criterion are proposed to the decision maker.In some of these preference functions, different threshold parameters ( p, q or s ) need to be set by the decision maker.As the shapes of preference functions and their parameters are chosen by the decision maker, it exerts clear advantages and features of PROMETHEE method (Podvezko and Podviezko, 2010).
As soon as the evaluation matrix, g j ( . ) is developed, and the relative importance (weight), w j and generalized criterion, {g j ( .), P j ( a, b )} are defined, PROMETHEE method becomes ready for implementation.PROMETHEE method is based on pairwise comparisons where the aggregated preference indices are defined as follows (Brans and Vincke, 1985):


are computed for each pair of alternatives in the evaluation matrix, a complete outranking graph, including two arcs between each pair of nodes (alternatives), can be obtained.In this method, each alternative 'a' faces ( m -1 ) number of other alternatives in the evaluation matrix.Now, the following two outranking flows can be defined: Positive outranking flow: Negative outranking flow: The positive outranking flow expresses how an alternative 'a' outranks all other alternatives.The higher the value of φ + ( a ), the better is the alternative.The negative outranking flow expresses how an alternative 'a' is being outranked by the others.Lower value of φ -( a ) signifies better alternative.In PROMETHEE I method, the partial ranking ( P I , I I , R I ) is obtained from the positive and negative outranking flow (6) The positive outranking flow expresses how an alternative 'a' outranks all other alternatives.The higher the value of φ + ( a ), the better is the alternative.The negative outranking flow expresses how an alternative 'a' is being outranked by the others.Lower value of φ -( a ) signifies better alternative.In PROMETHEE I method, the partial ranking ( P I , I I , R I ) is obtained from the positive and negative outranking flow values where P I , I I and R I respectively stand for preference, indifference and incomparability relations.
Both the flows do not usually induce the same rankings.PROMETHEE II method can provide a complete preorder of the alternatives by using a net flow, though it loses much information of preference relations.In this method, there is a balance between the positive and negative outranking flows.The net outranking flow for each alternative can be obtained using the following equation: The higher the value of φ ( a ), the better is the alternative.Thus, the best alternative is the one having the highest φ ( a ) value.PROMETHEE II provides a complete ranking of the alternatives from the best to the worst one.Since PROMETHEE I does not provide a complete ranking, the resulting ranking cannot be compared with the ranking provided by PROMETHEE II method.PROMETHEE I ensures creation of indifferent and incomparable alternatives.Other versions of PROMETHEE method, such as PROMETHEE III for ranking based on interval, PROMETHEE IV for complete or partial ranking of the alternatives when the set of viable solutions is continuous, PROMETHEE V for problems with segmentation constraints, PROMETHEE VI for human brain representation, PRO-METHEE GDSS for group decision-making and the visual interactive module GAIA for graphical representation are also available to help the decision makers to solve more complicated decision-making problems (Behzadian et al., 2010).

GAIA Method
From the positive and negative outranking flow values, as expressed in Eqns.( 5) and ( 6) respectively, the net outranking flow value can be estimated, as given below: Consequently, φ ( a ) = Page 9 of 25 values where P I , I I and R I respectively stand for preference, indifference and incomparability relations.Both the flows do not usually induce the same rankings.PROMETHEE II method can provide a complete preorder of the alternatives by using a net flow, though it loses much information of preference relations.In this method, there is a balance between the positive and negative outranking flows.The net outranking flow for each alternative can be obtained using the following equation: The higher the value of φ ( a ), the better is the alternative.Thus, the best alternative is the one having the highest φ (a ) value.PROMETHEE II provides a complete ranking of the alternatives from the best to the worst one.Since PROMETHEE I does not provide a complete ranking, the resulting ranking cannot be compared with the ranking provided by PROMETHEE II method.PROMETHEE I ensures creation of indifferent and incomparable alternatives.Other versions of PROMETHEE method, such as PROMETHEE III for ranking based on interval, PROMETHEE IV for complete or partial ranking of the alternatives when the set of viable solutions is continuous, PROMETHEE V for problems with segmentation constraints, PROMETHEE VI for human brain representation, PROMETHEE GDSS for group decision-making and the visual interactive module GAIA for graphical representation are also available to help the decision makers to solve more complicated decision-making problems (Behzadian et al., 2010).

GAIA Method
From the positive and negative outranking flow values, as expressed in Eqns.( 5) and ( 6) respectively, the net outranking flow value can be estimated, as given below: φ j ( a ) is the single criterion net flow obtained when only criterion g j ( . ) is considered.It expresses how an alternative 'a' is outranking (φ j ( a ) > 0) or outranked (φ j ( a ) < 0) by all other alternatives on criterion g j ( .).From Eqn. (9), it is observed that the global net flow of an alternative is the scalar product between the vector of criteria weights and the profile vector of that alternative.This property is extremely useful while developing the GAIA plane.Now, the following matrix M ( m × n ) can be defined based on the single criterion net flows of all the alternatives.M = Page 10 of 25 φ j ( a ) is the single criterion net flow obtained when only criterion g j ( . ) is considered.It expresses how an alternative 'a' is outranking (φ j ( a ) > 0) or outranked (φ j ( a ) < 0) by all other alternatives on criterion g j ( .).From Eqn. (9), it is observed that the global net flow of an alternative is the scalar product between the vector of criteria weights and the profile vector of that alternative.This property is extremely useful while developing the GAIA plane.Now, the following matrix M ( m × n ) can be defined based on the single criterion net flows of all the alternatives.
The information included in matrix M is more extensive than the one in the decision/evaluation matrix, as given in Eqn.(1), because the degrees of preference given by the generalized criteria are taken into consideration in M.Moreover, the g j ( a i ) values are expressed on their own scale, while the φ j ( a i ) values are dimensionless.In addition, the matrix M is not dependant on the weights of the considered criteria.
Consequently, the set of m alternatives can be represented as a cloud of m points in a n-dimensional space.As the number of criteria is usually larger than two, it is impossible to obtain a clear view of the relative positions of the points with regard to the criteria.The GAIA plane is obtained by projection of this on a plane such that as few information as possible get lost.In this plane, alternatives ( a 1 , a 2 , … a m ) are represented by points and the criteria ( c 1 , c 2 , …, c n ) by axes, as shown in Fig. 1. (11) The information included in matrix M is more extensive than the one in the decision/ evaluation matrix, as given in Eqn.(1), because the degrees of preference given by the generalized criteria are taken into consideration in M.Moreover, the g j ( a i ) values are expressed on their own scale, while the φ j ( a i ) values are dimensionless.In addition, the matrix M is not dependant on the weights of the considered criteria.
Consequently, the set of m alternatives can be represented as a cloud of m points in a n-dimensional space.As the number of criteria is usually larger than two, it is impossible to obtain a clear view of the relative positions of the points with regard to the criteria.The GAIA plane is obtained by projection of this on a plane such that as few information as possible get lost.In this plane, alternatives ( a 1 , a 2 , … a m ) are represented by points and the criteria ( c 1 , c 2 , …, c n ) by axes, as shown in Fig. 1.From Eqn. (9), it can be said that the PROMETHEE net flow of a i is the projection of the vector of its single criterion net flows on w.Consequently, the relative positions of the projections of all the alternatives on w provides the PROMETHEE ranking of the alternatives.Clearly, the vector w plays a pivotal role.It is represented in GAIA plane by the projection of the unit vector of the weights.This projection is referred to as the PROMETHEE decision axis ( π ).This axis shows the direction of the compromise solution resulting from the weights allocated to the criteria.If the weights are concentrated on one criterion, it is clear that the π axis will coincide with the axis of that criterion in GAIA plane.When the weights are distributed over all the criteria, the π axis appears as a weighted resultant of all the criteria axes.
If π is long, the PROMETHEE decision axis has a strong decision power and the decision maker is invited to select alternatives as far as possible on its direction.On the other hand, when π is short, it has no strong decision power, which means that according to the weights, the criteria are strongly conflicting and the selection of a good compromise solution is a hard problem.In this plane, criteria expressing similar preferences on the evaluation data are oriented in the same direction, and the conflicting criteria are pointing in the opposite directions.When the weights are modified, the positions of the alternatives and of the criteria remain unchanged in GAIA plane.The weight vector appears as a decision stick that the decision maker can move according to the preference in favor of a particular criterion.When a sensitivity analysis is performed by modifying the weights, the PROMETHEE decision stick ( w ) and the π axis are moving in such a way that the consequences for decision-making are easily observed in GAIA plane.

Shannon's Entropy Method
Although there are several distinct methods to determine weights/relative importance of criteria as involved in a decision-making problem, like AHP, Delphi method and Shannon's information entropy method, the latter one is usually adopted as it can avoid the effect of subjective judgments while pair-wise comparing the relative importance of the considered criteria (Shanon, 1948).Entropy is a measure that uses probability theory to measure the uncertainty of information.It shows that the more dispersive the data, the bigger the uncertainty.If there is a decision matrix, A consisting of m number of alternatives and n number of criteria, then the ratio of index of j th criterion in i th alternative can be calculated as follows: Page 11 of 25 From Eqn. ( 9), it can be said that the PROMETHEE net flow of a i is the projection of the vector of its single criterion net flows on w.Consequently, the relative positions of the projections of all the alternatives on w provides the PROMETHEE ranking of the alternatives.Clearly, the vector w plays a pivotal role.It is represented in GAIA plane by the projection of the unit vector of the weights.This projection is referred to as the PROMETHEE decision axis ( )  .This axis shows the direction of the compromise solution resulting from the weights allocated to the criteria.If the weights are concentrated on one criterion, it is clear that the  axis will coincide with the axis of that criterion in GAIA plane.When the weights are distributed over all the criteria, the π axis appears as a weighted resultant of all the criteria axes.
If  is long, the PROMETHEE decision axis has a strong decision power and the decision maker is invited to select alternatives as far as possible on its direction.On the other hand, when  is short, it has no strong decision power, which means that according to the weights, the criteria are strongly conflicting and the selection of a good compromise solution is a hard problem.In this plane, criteria expressing similar preferences on the evaluation data are oriented in the same direction, and the conflicting criteria are pointing in the opposite directions.When the weights are modified, the positions of the alternatives and of the criteria remain unchanged in GAIA plane.The weight vector appears as a decision stick that the decision maker can move according to the preference in favor of a particular criterion.When a sensitivity analysis is performed by modifying the weights, the PROMETHEE decision stick ( w ) and the  axis are moving in such a way that the consequences for decision-making are easily observed in GAIA plane.

Shannon's Entropy Method
Although there are several distinct methods to determine weights/relative importance of criteria as involved in a decision-making problem, like AHP, Delphi method and Shannon's information entropy method, the latter one is usually adopted as it can avoid the effect of subjective judgments while pair-wise comparing the relative importance of the considered criteria (Shanon, 1948).Entropy is a measure that uses probability theory to measure the uncertainty of information.It shows that the more dispersive the data, the bigger the uncertainty.If there is a decision matrix, A consisting of m number of alternatives and n number of criteria, then the ratio of index of j th criterion in i th alternative can be calculated as follows: where x ij is the performance of i th alternative with respect to j th criterion.Now, the decision information of each index can be expressed by the entropy value (e j ) using the following equation: where x ij is the performance of i th alternative with respect to j th criterion.Now, the decision information of each index can be expressed by the entropy value ( e j ) using the following equation: The Shannon's entropy weight for j th criterion can now be obtained employing the following expression: The Shannon's entropy weight for j th criterion can now be obtained employing the following expression: Entropy method has several advantages over the others, i.e. it can be applied in case of fuzzy data, it can be adopted for problems with high complexity involving a large number of criteria, and it can be modified and used for non-deterministic data, such as interval data while producing interval weights.

NITs in India
After independence, Jawaharlal Nehru sought to develop India as a leader in science and technology.On the recommendations of Engineering Personnel Committee (EPC) set up by the Planning Commission in 1955, the Government of India started 14 RECs between 1959and 1965at Bhopal (in 1960), Allahabad (in 1961), Kozhikode (Calicut) (in 1961), Durgapur (in 1960), Kurukshetra (in 1963), Jamshedpur (in 1960), Jaipur (in 1963), Nagpur (in 1960), Rourkela (in 1961), Srinagar (in 1960), Surathkal (in 1960), Surat (in 1961), Tiruchirappalli (in 1964) and Warangal (in 1959).Those RECs were set up as joint and co-operative ventures of the Central and State Governments concerned with a view to create the required technical manpower by providing undergraduate education and training in different branches of engineering and technology for the industrial projects being contemplated during the 2 nd Five-Year Plan (1956)(1957)(1958)(1959)(1960)(1961).Later, three other new RECs in Silchar (in 1967), Hamirpur (in 1986) and Jalandhar (in 1987) were established.Those institutes were registered as autonomous bodies under the Society Registration Act, 1860 and affiliated to the State Universities in their respective regions.The success of technology-based industry led to high demand for more technical and scientific education.Further, the RECs were also envisaged to function as pace setters and to provide academic leadership to the technical institutions.RECs were basically founded to promote regional diversity and multi-cultural understanding in India.
Due to enormous cost and infrastructure involved in creating globally respected IITs, in 2002, Ministry of Human Resources and Development (MHRD) decided to upgrade RECs to NITs instead of creating new IITs.The Central Government controls those NITs and provides all funding.Thus, the NITs became a group of public engineering institutes of India.In 2003, all RECs became NITs.The upgrade was designed along the lines of the prestigious IITs after it was concluded that Entropy method has several advantages over the others, i.e. it can be applied in case of fuzzy data, it can be adopted for problems with high complexity involving a large number of criteria, and it can be modified and used for non-deterministic data, such as interval data while producing interval weights.

NITs in India
After independence, Jawaharlal Nehru sought to develop India as a leader in science and technology.On the recommendations of Engineering Personnel Committee (EPC) set up by the Planning Commission in 1955, the Government of India started 14 RECs between 1959and 1965at Bhopal (in 1960), Allahabad (in 1961), Kozhikode (Calicut) (in 1961), Durgapur (in 1960), Kurukshetra (in 1963), Jamshedpur (in 1960), Jaipur (in 1963), Nagpur (in 1960), Rourkela (in 1961), Srinagar (in 1960), Surathkal (in 1960), Surat (in 1961), Tiruchirappalli (in 1964) and Warangal (in 1959).Those RECs were set up as joint and co-operative ventures of the Central and State Governments concerned with a view to create the required technical manpower by providing undergraduate education and training in different branches of engineering and technology for the industrial projects being contemplated during the 2 nd Five-Year Plan (1956)(1957)(1958)(1959)(1960)(1961).Later, three other new RECs in Silchar (in 1967), Hamirpur (in 1986) and Jalandhar (in 1987) were established.Those institutes were registered as autonomous bodies under the Society Registration Act, 1860 and affiliated to the State Universities in their respective regions.The success of technology-based industry led to high demand for more technical and scientific education.Further, the RECs were also envisaged to function as pace setters and to provide academic leadership to the technical institutions.RECs were basically founded to promote regional diversity and multi-cultural understanding in India.
Due to enormous cost and infrastructure involved in creating globally respected IITs, in 2002, Ministry of Human Resources and Development (MHRD) decided to upgrade RECs to NITs instead of creating new IITs.The Central Government controls those NITs and provides all funding.Thus, the NITs became a group of public engineering institutes of India.In 2003, all RECs became NITs.The upgrade was designed along the lines of the prestigious IITs after it was concluded that RECs had potential as proven by the success of their alumni and their contributions in the field of technical education.Subsequently, funding and autonomy for NITs increased, and those were taken over as fully funded institutes of the Central Government of India and granted deemed university status.In addition, the Central Government had also taken over three other institutes, i.e.Bihar Engineering College, Patna; Government Engineering Col-lege, Raipur and Tripura Engineering College, Agartala, and converted them into NITs in 2004, 2005 and 2006 respectively.Thus, the total number of NITs had gone up to 20.Those institutes were expected to be at par with other national level technical institutes and be able to fulfill the demand of high quality undergraduate and postgraduate level of education in engineering and technology.An Act, i.e. the NIT Act, 2007 has since been enacted by the Parliament of India so as to provide a common statutory framework for all NITs and the Indian government has declared those schools as Institutes of National Importance.
In 2010, the Government of India announced setting up ten new NITs in the remaining states/union territories, i.e.Delhi, Goa, Puducherry, Uttarakhand, Mizozam, Meghalaya, Manipur, Nagaland, Arunachal Pradesh and Sikkim.Comprising 30 autonomous institutes, this would lead to every state in India having its own NIT.Right from the beginning, NITs in Delhi, Goa, Puducherry and Uttarakhand share their place with the top NITs leaving behind old and private universities.Apart from their geographical advantages, they also have top mentoring institutes.
NITs offer degree courses at bachelors, masters and doctorate levels in various branches of engineering and technology.All NITs are autonomous which enables them to set up their own curriculum.Average NIT funding increased to Rs. 100 crores by year 2011.On average, each NIT also receives Rs. 20-25 crores under the World Bank funded Technical Education Quality Improvement Program (TEQIP).
All NITs have a central library equipped with technical books, literature, fiction, scientific journals and other electronic materials.Most have digitized their libraries.Some provide an intranet library facility.Every departmental library has high speed connectivity.Electronic libraries allow students access to online journals and other periodicals through the AICTE-INDEST consortium.While some have video conferencing facilities, others are upgrading under the World Bank funded TEQIP scheme.

Ranking of NITs Using PROMETHEE-GAIA Method
It has already been observed that MCDM methods have immense potential in performance evaluation of competing alternatives (Zavadskas and Turskis, 2011;Zavadskas et al., 2014).Although, at present, there are 30 NITs in India, i.e. each state/union territory has its own NIT, in this paper, the performance of 20 NITs is evaluated using PRO-METHEE-GAIA approach.Those ten NITs which were established during the year 2009-2010 are not considered here due to non-availability of relevant data.From those NITs, students would be awarded with their UG/PG degrees only in year 2013-2014.So, some of the information as necessary for this evaluation process would not be available from those newly started NITs.The performance appraisal of the remaining 20 NITs is carried out based on nine evaluation criteria, as enlisted in Table 2.All these nine criteria are so selected that they are entirely uncorrelated.Faculty strength is the total number of teaching faculties, including professors, associate professors and assistant professors in each NIT.Teacher student ratio is the ratio of the number of teaching faculties to the total number of enrolled students (UG, PG and doctoral students).The MHRD has already recommended to maintain a teacher student ratio of 1:12 (0.083) in the NITs.The total number of enrolled students may be another important criterion, but as it will have a strong correlation with the teacher student ratio, it is discarded from consideration.The total number of international/national conferences/seminars/ symposiums organized by the NITs during the span of last 5 years (2008)(2009)(2010)(2011)(2012) and the total number of papers published in the International Journals of high repute during the last 5 years are taken as two other criteria.Research grant is the total amount of grants in crores of rupees sanctioned by the MHRD of Government of India to each of the NITs during the last 5 years to support conducting research activities.Campus area (in acres) is the total area of each NIT within which all its facilities are situated.Placement of UG and PG students (in %) is the percentage of passed out students who had been absorbed in different industries/institutions in the last year (2011)(2012).The total number of books and online Journals available in the libraries of the considered NITs is taken into account as another criterion.The last criterion is considered as the course fee (in Rs.) for each student required to complete his/her target UG degree.Among these nine criteria, all except course fee are beneficial attributes where higher values are always required.For course fee, being a non-beneficial criterion, its minimum value is desired.Table 3 shows the decision matrix as developed for performance evaluation of 20 NITs based on nine criteria where the relevant information about the NITs with respect to different criteria are accumulated from various websites (NITs' own websites, www.scopus.com,www.wikipedia.orgetc.).The criteria weights are calculated using Shannon's entropy method, as given in Table 4. Now, this performance appraisal problem of 20 NITs is solved using Visual PRO-METHEE which can be easily downloadable from www.promethee-gaia.net/software.htmlwebsite.It is an interactive PROMETHEE-GAIA software.Fig. 2 shows the input window of Visual PROMETHEE where the related criteria values for 20 NITs have already been entered.In this window, the user can choose the number of actions (alternatives), number of criteria, number of scenarios (for group decision-making problem), unit for each criterion, type of the criterion (beneficial or non-beneficial) and type of the preference function.Here, the simplest form of the preference function (i.e.usual criterion) is selected.The user can also select other types of preference function, the corresponding threshold values for those functions would then need to be provided.After entering all the relevant criteria values for 20 NITs, the corresponding maximum, minimum, average and standard deviation values for each of the criteria are automatically displayed.In Fig. 3, the PROMETHEE II complete ranking of the considered NITs is provided, based on the net outranking flow values.In this figure, the top half of the scale (in green) corresponds to positive net outranking scores and the bottom half (in red) to negative scores.It is clearly observed that alternative 19 (NIT, Tiruchirappalli) tops the ranking list, followed by alternatives 20 (NIT, Warangal) and 17 (NIT, Surat).It signifies that these three NITs are the best performers with respect to nine evaluation criteria.The performance of alternative 12 (NIT, Patna) is not at all satisfactory.It is also interesting to observe that alternatives 6, 8 and 13 (NIT, Hamirpur; NIT, Jalandhar and NIT, Raipur) also perform unsatisfactorily.The developed GAIA plane is shown in Fig. 4 where the positions of the criteria, alternative NITs and π axis are exhibited.The direction of the π The PROMETHEE rainbow diagram for performance evaluation of 20 NITs is exhibited in Fig. 5.In this figure, each bar for each NIT is composed of different slices, and the height of each slice is proportional to the contribution of one criterion on the final ranking of NITs.Positive (upward) slices correspond to good features while negative (downward) slices represent the weaknesses.The 20 NITs are positioned from left to right according to the PROMETHEE II ranking.From this figure, it is clear that NIT, Tiruchirappalli is excellent in performance with respect to all the nine criteria, and criterion C1 (faculty strength) is its most favorable feature, followed by criterion C7 (placement of UG and PG students).On the other hand, the performance of NIT, Warangal is quite comparable with that of NIT, Tiruchirappalli, except for criterion C6 (campus area).NIT, Warangal has a total campus area of 248 acres as compared to NIT, Tiruchirappalli (800 acres).Criterion C7 (placement of UG and PG students) is the strongest feature for NIT, Warangal, followed by criterion C8 (number of library books and online Journals).According to the PROMETHEE rainbow diagram, NIT, Patna has the weakest performance amongst the 20 NITs and its only good feature is its relatively low course fee structure.Fig. 6 exhibits the best to worst ranking of 20 NITs and also the weights allocated to nine selected criteria.Clearly in this figure, the considered 20 NITs form two separate clusters based on their net outranking flow values.NITs 19,20,17,14,15,18,5,3,4 and 10 are in one cluster having positive flow values, whereas, NITs 9, 11, 1, 2, 16, 7, 13, 8, 6 and 12 form another group with negative flow scores.It signifies that amongst 20 NITs, exactly ten are satisfactory performers and the remaining ten NITs are underperformers having deficiency with respect to some of the considered criteria.In   this window, by changing the criteria weights (walking weights), the user can also visualize the effects of those changing weights on the subsequent ranking of NITs.It basically acts as a visual sensitivity analysis tool.In Fig. 7, the visual stability interval of the most important criterion C7 (placement of UG and PG students) is provided.It shows how the multi-criteria net flow scores change as a function of the weight of criterion C7.In this figure, the horizontal dimension corresponds to the weight of criterion C7 and the vertical dimension relates to the net flow scores.For each NIT, a line is drawn showing the net flow value as a function of the weight of criterion C7.At the right edge of this figure, the weight of the criterion is 100% and the NITs are ranked here with respect to criterion C7.At the left edge, the weight of the criterion is equal to 0%.The position of the vertical green and red bar corresponds to the current weight of criterion C7 at 14.08%.The intersections of different NIT lines with the vertical bar also provide the complete ranking of NITs.It is observed that the weight stability interval of C7 criterion ranges between 13.90% and 14.86%, which signifies that within this interval, there would be no change in the positions of the top-ranked NITs.It is also quite interesting to observe that at the higher weight of C7 criterion, NIT, Warangal supercedes NIT, Tiruchirappalli, and NIT, Patna remains as the underperformer along the entire range of C7 criterion weight.Similar types of weight stability intervals can also be derived for other non-important criteria to search out the possibility of any rank reversal among the considered NITs.Performance Evaluation of Indian Technical Institutions Using ... 123

Conclusions
In this paper, an attempt is made to integrate PROMETHEE and GAIA methods to evaluate the comparative performance of 20 NITs in India based on nine uncorrelated criteria.In order to avoid subjective judgments, Shannon's entropy method is used for computing the criteria weights.It is observed from the analysis that NIT, Tiruchirappalli is the best performer followed by NIT, Warangal.NIT, Warangal is lagging behind NIT, Tiruchirappalli only with respect to its total campus area.NIT, Patna obtains the last position in the ranking list among 20 NITs and it has only one favorable feature as its relatively low course fee structure.The walking weights analysis helps the user to observe the effects of changing criteria weights on the final ranking of NITs.On the other hand, the user can also determine the weight stability intervals of different criteria to identify the possibility of any rank reversal among the considered NITs.This performance evaluation analysis will help the MHRD of Government of India to identify the strengths and weaknesses of NITs, and take proper remedial measures to better the positions of the underperforming NITs.The performance of other higher education institutions, like IITs and Indian universities can also be evaluated using the adopted approach.R. Ranjan is a post graduate student in Department of Production Engineering of Jadavpur University.He received his Bachelor of Technology in Mechanical Engineering from Sikkim Manipal Institute of Technology.He is now pursuing his research work in the area of applications of PROMETHEE method in decision making.

S.
Chakraborty is an Associate Professor in Production Engineering Department of Jadavpur University of India.He graduated in 1986 from University of Calcutta and obtained his post-graduate degree from Jadavpur University in 1989.He was awarded with PhD (Engineering) degree from Jadavpur University in 1994.His research interests include applications of multi-criteria decision-making methods in manufacturing environment, control chart pattern recognition, and development of MIS systems for engineering applications.
with which 'a' is preferred to 'b' over all the criteria and ( , ) b a  represents how 'b' is preferred to 'a'.In most of the cases, there are criteria for which 'a' is better than 'b', and criteria for which 'b' is better than 'a', and consequently, with which 'a' is preferred to 'b' over all the criteria and ( , ) b a  represents how 'b' is preferred to 'a'.In most of the cases, there are criteria for which 'a' is better than 'b', and criteria for which 'b' is better than 'a', and consequently, with which 'a' is preferred to 'b' over all the criteria and ( , ) b a  represents how 'b' is preferred to 'a'.In most of the cases, there are criteria for which 'a' is better than 'b', and criteria for which 'b' is better than 'a', and consequently,

Table 1
Methodology, sample and criteria/sub-criteria considered by the past researchers

Table 2
Selected criteria for performance appraisal of 20 NITs indicates the superiority of NIT, Tiruchirappalli and NIT, Warangal over the others.But in the π axis direction, as the position of NIT, Tiruchirappalli is the farthest from the origin of the axis, it is the best choice.A recent report (http://www.kinindia.com/university/top-10-nit-colleges-in-india-2013/)makesalist of the top ten NIT performers in the session 2012-2013 as Warangal, Tiruchirapalli, Surathkal, Delhi, Allahabad, Jaipur, Calicut, Nagpur, Kurukshetra and Rourkela which also matches with the observations from the developed GAIA plane regarding the positions of the top two performing NITs.It is worthwhile to note here that in the GAIA plane, criteria C 4 , C 5 and C 8 (number of papers published in Journals in the last 5 years, research grant, and number of books and online Journals available in library) form a cluster.On the other hand, criteria C 1 , C 2 , C 3 , C 6 and C 7 (faculty strength, teacher student ratio, number of conferences held during the last 5 years, campus area, and placement of UG and PG students) are in another cluster.The identified criteria in both the clusters have the same effects on this evaluation process, expressing similar preferences.As these two clusters of criteria are represented by orthogonal axes in the GAIA plane, they are also not related to each other with respect to preferences.As criterion C 9 (course fee) is a non-beneficial attribute, its position is totally in the opposite direction in the GAIA plane, as observed in Fig.4.NITs 3, 9, 10, 17, 19and 20 are quite strong with respect to criteria C 1 , C 2 , C 3 , C 6 and C 7 .On the other hand, the values of criteria C 4 , C 5 and C 8 are favorable for NITs 4, 5 and 14.The performances of NITs 4 and 5 (NIT, Calicut and NIT, Durgapur) are quite similar in nature.While using PROMETHEE-GAIA method, this performance evaluation problem of NITs has a medium quality of solution (reliability) of 52.6%, which indicates that it is hard problem to solve.
Fig. 4. GAIA plane for performance evaluation of 20 NITs.axis