Nowadays, universities and higher education institutions have a significant role in the development of countries. For various reasons such as widespread demand for higher education, distance learning, internationalization of higher education, entrepreneurship and innovation universities, there is a close competition in higher education industry. This has resulted in improvement of educational quality and performance, knowledge-based developments, and access to more resources, recruit of more effective scientists in order to enhance competitive status by universities. University ranking systems are one of the most reliable tools in defining universities status while these systems themselves are facing serious changes. In this article, the way of the development and the features of the refereed ranking systems have been investigated with respect to the literature. Then, with respect to the merits of the universities’ and higher education institutes’ rankings, the fundamental challenges in ranking systems, indices, and data measurement instruments have been investigated with a critical perspective. The results indicated that the ranking systems mainly encounter some problems in order to consider parameters such as quality measurement, growth or decline diagrams, criteria weighting, the Meteo effect, comprehensibility, clarity, dynamicity, generalizability, and language limitations. Also, the ranking criteria of the Islamic Scientific Citation have been criticized due to the fact that Iranian universities are ranked on these bases and hence some suggestions are recommended with regards to its limitations.