Edu-leaders

According to MeritTrac, an independent skill assessment company, every year the Indian corporate sector requires 128,000 MBAs. However, Indian B-schools collectively produce about 100,000 MBAs annually. Merit-Trac also points out that, of those thousands, only 23 percent are employable—increasing the already-existing gap between demand and supply. Another MeritTrac report on engineering states that India produces over 400,000 engineering graduates. But, only 15 percent of these 400,000 are industry-ready. It is apparent that some of the colleges (and universities) are failing to equip students for the industry.

Exactly why this happens needs to be examined—which an audit can do. However, in India, an audit is often seen as a facilitator for accreditation, and rarely as a means, to improve an institution’s functions.

Examining differences

The word “audit” comes from the Latin word auditus, meaning “to hear." By definition, an audit entails an examination and verification of records- originally an oral process. In context, of an institution, the word means the process of assessing whether a college or university is indeed fulfilling its purpose of disseminating knowledge, as well as contributing to the creation of a talent pool

In a report titled Quality Audit in the Universities, C. Webb writes that in context of higher education the word (audit) was first used by Professor Stewart Sutherland, Vice Chancellor, University of London. (Sutherland was also the chairman of the academic standards group at the Committee of Vice Chancellor and Principals and was a member of the management board of UK’s Academic Audit Unit between 1990 and 1992.) Sutherland tried to distinguish an academic audit from other kinds of peer reviews, and hinted at its intended methodology. Webb, on his part, explains, “Academic audit would seek to borrow techniques from the financial audit.” It would not mean that a university’s functioning would be akin to the balance sheet of a company. It just means that the level of scrutiny would be the same.

Dr David Woodhouse, executive director of Australian Universities Quality Agency, clears the matter further. In Putting ‘A’ into Quality (a paper that he authored) Woodhouse distinguishes between audit, assessment and accreditation. While an audit, he says, “is a check on an organisation’s explicit or implicit claims about itself”, an assessment is more result-oriented. It answers the question “how good are your outputs?”

“According to the International Standards’ Organisation (ISO), quality audit is a threepronged process—checking suitability of the planned quality procedures in relation to the stated objectives; conformity of actual quality activities with plans; and effectiveness of the activities in achieving the stated objectives,” Woodhouse explains.

Accreditation is the outcome of the audit and assessment processes. Thus; audit is a process, assessment is the result and accreditation is the qualified result.

The proliferation of institutions offering higher education in India and increased student mobility and globalisation has made a quality compass critical. An audit is that compass. With foreign institutions making their foray into India, now there is also the need for a standardised assessment. As Professor Shashi Gulhati, points out, “The concept of an academic audit does not exist in India. An institution such as the IIT is better because there is the pressure to perform. We ensure that curriculum is revised, faculty is assessed—even by students—and teaching assignments and assessments are regularly scrutinised.” Gulhati has been a professor at IIT Delhi for 40 years.

There are several types of academic audits, based on levels of assessment and on people carrying out this assessment.

Brigadier Rajiv Divekar (retired), director, Symbiosis Institute of Management Studies, says, “Recently we received our ISO certification. It made us confident of our abilities and qualities. In today’s academic world transparency and consistency are important. Run an honest office with regular audits, then every person understands what is expected from him, or her. Accountability increases and processes become standardised.” In India an audit is rarely seen as an ongoing exercise. If that is the case then Andhra University (AU), is one of the few exceptions. The university will be conducting audits from the 2009 to 2010 session. Of course, the need for a mandatory audit was driven by the quest for an ISO rating.

But, soon, Beela Satyanarayan, the VC admits, the process was seen as a proactive internal measure. Osmania University, Hyderabad, is a pioneer because of its internal audit cell. Professor Lingamurthy, Vice Chancellor, Kakatiya University, tells us, “Osmania was probably the first university in the country to start an audit cell. We followed that model. Professor G. Ram Reddy, the then Vice Chancellor, created the Director Academic Audit Cell. Later, when Professor Vidyavathi became the Vice Chancellor of Kakatiya University, I asked her to create a similar cell here.” Curriculum watch: Keeping a curricula updated is a challenge. Usually, there isn’t a fixed way to arrive at this. Professor Gulhati says, “At IIT we try and update the curriculum frequently. The usual way is when a faculty member realises from his or her research that there is something that can be developed as a new programme they present a proposal in individual capacity to the departmental board. It is then put forth for approval.” This ensures frequent upgrade. But, the process is limited to the IITs. In universities, curriculum upgrade is a long-drawn process, because it has to be acceptable to all colleges falling under its jurisdiction before change is internalised. Regular curriculum audits are required to understand what the focus is, and if that focus is relevant. There are no fixed ways to conduct a curriculum audit—an external agency can do it, or it could be an internal audit. If you choose the latter, there are templates and guidelines available on the internet from institutions that help the process. Some (sites) even give a step-by-step approach. But the core idea is to start with objectives, obviously. The next step is to set standards against which curriculum may be measured—this can be developed after interviews with stakeholders and representatives—management, teachers, students, parents, industry and, of course, experts. For developing such a survey questionnaire, resources available on the web can guide you. Then the actual design of the audit begins, again involving stakeholders. For internal audits, guidelines are necessary, as is a schedule. And the process needs to be monitored as it happens. Then data have to be collated and analysed, with a final report on findings. The process is straightforward enough. Of course, the answers come when you ask the right questions, so the challenge is in the design.

Pedagogical process: It is not just the content that needs to be audited, but also delivery. Thomas Angelo, the co-author of Classroom Assessment Techniques, said, “Teaching in the absence of learning is just talking.” The effectiveness of teaching and pedagogy is something that needs to be audited from time to time, given the deluge of new tools for improving curriculum delivery. In our universities, however, the pedagogical processes have remained tethered to age-old practices. The National Knowledge Commission (NKC) in its recommendation document to the ministry, says, “The university syllabi remain unchanged for decades. It needs to be upgraded constantly and revised frequently. Universities should revise or restructure at least once in three years. Revisions should be subjected to peer review before implementation. The process should be streamlined and de-centralised, with teachers’ autonomy and statute changes. There should be a mode of censure for departments or universities that do not upgrade.”

Evaluating the teaching-learning process needs a three-pronged approach—evaluation by management, peer evaluation and student-led evaluation. Eliminating even one will result in a skewed process.

Secondly, standards need to be set for disciplines. In terms of methodology, there are a number of ways that can be used independently, or in combination. There is the questionnaire approach. How it is designed is critical. Then, there is the focus-group approach, where a group of peers, superiors and students are asked to discuss the effectiveness of a class, course, programme or department.

There is peer evaluation methodology in which an auditor or a group attend classes and evaluate it. Auditing teacher performance is also a way for a university to exercise control over the education quality. Professor Lingamurthy says, “In a way, it serves as a monitor. When a university gives affiliation, it sets certain conditions. Whether these conditions are being followed is periodically checked by an academic audit cell. A team goes and checks teachers’ qualifications, numbers and recruitment.”

Evaluation may be developed at a government-level as well. Professor Gulhati tells us that the Ministry of Human Resources Development had created a Quality Improvement Programme, specifically focused on engineering.

Teachers from smaller engineering colleges can come to the IIT for their PhD and then start research at their respective institutions. IIT teams will sometimes arrive to assess their programmes. Evaluating evaluation: Quality of learning is often tested by student performance. However, the NKC states, “Learning and creativity are at a discount in a system of assessment that places a premium on memory rather than understanding.” Therefore, evaluation needs to audited to bring out a correct picture of student performance.

Before the institution adopts any way, it needs to audit (check) its process of evaluation and see if there is a gap between student evaluation according to university and industry’s expectations. Students, too, should receive the opportunity to give their feedback.