CAT 2014 - Attempts and Percentiles

Let us list and bust a few of the myths regarding CAT attempts and percentiles

1. Number of attempts and percentiles have a strong correlation
This is of course true. But the correlation is extremely overstated. Every year, there are lots and lots of students who attempt 65% of the total questions and end up with less than 85th percentile. The far deeper correlation is likely to be between accuracy and percentile. The number of attempts is merely the most visible metric and therefore the most discussed. Unfortunately, quite a few students chase magic attempts numbers and end up shooting themselves in the foot. Come results date, we will hear a lot of students cribbing that the scoring, normalization, etc were incorrect.

In CAT 2013, a few of our students secured more than 90th percentile in a section having attempted only ~8 questions in the section. If you had read any of the forums, you would believe that guys who attempted less than 15 in a section should basically throw in the towel.

2. All students have 85% accuracy
The only thing the attempts and percentiles tell us is that there are a LOT of people who have accuracy rate of less than 50%. They get more than half the questions wrong. The CAT is a high quality exam. Pausing for thought and worrying about detail pays off in this exam. Guys who have cracked this exam will tell you that being in a frenzy never helps.

3. If you have attempted less than 65 questions in CAT 2014 you have no chance
At least half the students who are going to join IIM ABC are going to be guys who attempted less than 60 questions. The vast majority of guys who attempt 65+ questions are going to end up with less than 95th percentile. There are lots of students who appear for CAT every year, attempt lots of questions, and consistently get between 85th and 90th percentile. Its a shame that they do not change their strategy.

So, where does this leave us.
If you have attempted fewer questions but are sure about most of them. the score might surprise you to the upside.

As far as percentiles go, I would guess that they would be in the following range
Overall attempts ~ 40  - ~90th percentile
Overall attempts ~ 45  - ~94th percentile
Overall attempts ~ 50  - ~98th percentile
Overall attempts ~ 55  - ~99th percentile
Overall attempts ~ 60  - ~99.4 percentile
Overall attempts ~ 65  - ~99.7 percentile
Overall attempts ~ 70  - ~99.9 percentile
Overall attempts ~ 75  - Higher than this

Two caveats
1. These are at-best informed guesses. At worst, they are random numbers in a sequence.
2. All of these numbers are based on the assumption that at worst 2-3 questions are wrong. I do not understand the term "around 70% accuracy". Any paper where you believe you will have around 70% accuracy, you are basically leaving to the machinations of fate. If you think around 3 out of every 10 could be wrong, then very easily 5 out of 10 could be wrong.

Note that the density of scores is usually very low right at the top. Between 100th percentile and 99th percentile, the difference in number of attempts (correct answers) will be around 20 (perhaps even more). It is fairly crowded in the 90th-98th percentile range. With the same number of attempts, 4-5 more correct answers might determine the difference between a middling 89th percentile and sure-fire 96th percentile. This is why we ask students to take gazillion mock CATs and fine-tune everything.

Those students still waiting to take the exam, the one suggestion I would make strongly is to not get carried away by all the attempts numbers you are hearing. An enormous number of students live for the kick of saying that they attempted a lot of questions on D-day. A great proportion of this "huge number of attempts" gang end up having attempted 10 questions too many for their own good.

CAT GMAT XAT SNAP IIFT NMAT etc

There  are so many of these entrance exams that it is all very overwhelming. So, let us see if we can get some clarity out of these.

As far as India is concerned, the biggest fish is CAT
Give or take 90 out of the top 100 select based on CAT, with quite of a few of these selecting based only on CAT. This includes the IIMs (13 of these, soon to be 18), IITs, SP Jain, FMS, Bajaj, NITIE, MDI, IMT, among a great many others

For international MBAs, it is GMAT
Almost all universities in the US, UK, Australia, Singapore, Europe select based on GMAT. In India ISB and Great Lakes select based on GMAT. All the IIMs (+ MDI, XLRI etc) use the GMAT score for admissions into their executive MBA programs.

CAT or GMAT, which one should I take?
There are two simple rules
1. If you want to do in India, take CAT. If you want to go abroad, take GMAT.
2. If you are a fresher, take CAT. If you have more than 8 years of experience, give preference to GMAT

If you have 2-4 years of experience, and not sure whether you want to do MBA in India or abroad, then the decision is even simpler. Take both.

What about XAT?
There are around 30 decent colleges that select based on XAT. Within this, probably XLRI is the only college that selects only through XAT. Almost everyone else accepts CAT also (XIMB, LIBA GIM etc)

The big two are CAT and GMAT really, XAT is important because it can give you a second option.

What about these other exams?
Now, IIFT selects based on a separate exam. Symbiosis uses SNAP, NMIMS has NMAT. These three give you access to narrow groups of colleges, so write these based on whether you like that particular college. There is a government-mandated CMAT and then MAT. Of these, CMAT is more or less useless. MAT is accepted in a lot of private colleges, but there is practically no college that accepts MAT that will not give you a seat based on a good CAT score. If you are confident of getting 90th percentile in CAT, then skip MAT.

What about the quality of the exam?
CAT is of a good quality (note high quality, not difficulty). The quality of the GMAT exam is even better. Now, what do I mean by quality? The questions are challenging but not impossible, the level of difficulty is consistent, the framework is well defined and the exam is generally well conducted. Among the others, IIFT and NMAT are alright, at least they care enough to set decent papers.

The really worst papers are XAT and SNAP. XAT papers usually carry errors and have wild variances in difficulty across sections. SNAP also do not care much about the way they set their papers. Symbiosis probably thinks it is a good idea to sell some applications and create some excitement around the college name. SNAP has been accused of even picking some questions from freely available sources on the web.

The following chart should give an idea about all these exams, their level of difficulty, their quality (subjective opinion, bear this in mind) and how many people take these exams.

Note that the bubble size broadly outlines how many prepare for these exams

So, how does one prepare for all of these?
Prepare for CAT, forget all other Indian exams. If you are taking any of the other Indian exams, all you will need to do is make are a few some adjustments that shouldn't take more than 3-4 days. Focus slightly more on XAT; because even though the paper is inconsistent XAT can open more doors than most of the other exams.

Plan for GMAT separately. Since it is an American exam, you will need to make some adjustments regarding what is more vital. GMAT tests concepts better, and de-emphasizes computation. GMAT verbal is also tougher than that of CAT. So, even though more than 75% of the syllabus is common across CAT and GMAT, a few adjustments need to be made.

How CAT, XAT and GMAT are different, syllabus-wise.

Explaining the abbreviations used in the diagram
FIJ - Fact, Inference Judgement. Even CAT Seems to be phasing this out. You should see a maximum of 1-2 questions from this in the exam.
SE - Sentence Elimination
LR - Logical Reasoning (puzzle)
WU - Word Usage
TC - Text Completion (Fill in the blanks)
PC - Paragraph Completion
DI - Data Interpretation
Trig - Trigonometry
BDM - Business Decision Making
RC - Reading Comprehension
AWA - Analytical Writing Assessment
IR - Integrated Reasoning
CR - Critical Reasoning
SC - Sentence Correction
GK - General Knowledge
P&C - Permutation and Combination.

Note that this is a broadly indicative chart. Kindly do more research on your own after keeping this as a starting point. Dont get all officious about it if I have missed some topic or category.

Mock CATs, Mock CAT percentiles, predictions etc

I am going to start with a series of quotes.

There are two ways of lying. One, not telling the truth; the other, making up Statistics. 

There are lies, damn lies, and statistics.

Statistics are used much like a drunk uses a lamppost: for support, not illumination.

Replace Statistics with Percentile analytics and this would fit in perfectly in the context of CAT. No matter how many fancy charts are shown, it is the responsibility of the students to not over-interpret the results. When you take a mock CAT, the review happens on three planks -

1) Learn better methods by looking at the solutions
2) Think about how you could have improved your attempts by selecting better questions.
3) Evaluate your percentile.

Of the three, the least useful is no 3 (probably also the most harmful). Because it is the least reliable. Almost all mock CAT providers use look-up tables to give percentile scores. Gone are the days where a mock CAT provider could get 10,000 people to beat themselves into a frenzy on a given Sunday to take a mock CAT at 100 god-forsaken locations in the Country.

These days, proctored CATs probably have a sample size in the range of 1% of CAT takers. These are likely to be the most serious, most experienced, most-driven of the CAT takers. So, the test providers "adjust" the look-up table to factor this in. In layman terms, this boils down to - they look at the actual numbers, feed in some other numbers, and pat themselves on the back.

So, the percentiles are fairly unpredictable. And contrary to what all the service providers claim, there is no way to back-verify that database. The only ones who will call a coaching institute and tell them that they scored a decent percentile are the ones who got a decent percentile. So, the reverse-verification process is bogus. So, no one actually knows whether any percentile number has a serious correlation with the actual numbers. There is way more anecdotal evidence to say that the percentile numbers are erratic, than to say that they are accurate.

I recently heard that there was a percentile predictor that predicts percentiles based on numbers you feed in based on the scores you have got on other exams (some of my students called me to give me an update on their predicted results.). I almost fell from my chair laughing. Students who believe that there is a scientific basis for this kind of predictor should re-examine their brains. This level of gullibility cannot be good for the body.

The thought process behind the predictor is as follows - Take three not scientifically proven indicators with no established proof from back-verification, define arbitrary weightages based on your genius to these, come up with a fourth metric and call it a prediction. And supposedly students are flocking to this predictor engine. No wonder soothsaying, tarot card reading are all flourishing industries in our Country. If smart, ambitious people can be made to believe there is such a thing called a percentile predictor, god save the masses.

Anxiety does strange things to the mind. Fretting and fuming about this exam makes you do all kinds of over-analysis. The mind needs something to cling on to, to feel that you are on the right track. Ergo, predictor.

Think of all the guys who have completed their MBAs from IIM A, B, C and think about how each of them would react if you said "percentile predictor" in their midst. I will give you a hint. I spoke to a bunch of them and the only common factor was laughter.

So, what should we do?

I am going to say something that is difficult to implement. The last 2-3 weeks, you must focus on three things

1. Chuck the percentiles. Seriously, chuck them. Forget about them. Banish them.

2. Focus on still being curious and not getting sucked into this pressure vortex. The first 2-3 mocks that you would have taken, you would have enjoyed grappling with the questions. Slowly, the pressure takes over and you stop having fun in the mocks. You are too worried about the score (and that damned percentile). The ones who crack this exam are the ones who are likely to look at a question and go "Wow. Super question. I think I will enjoy solving this". If that joie de vivre is gone, you should rekindle it.

3. Mentally prepare for a paper that is far simpler than your mocks. In our Country, the service providers have a feeling that to create a great mock CAT, you must make it very tough. So, they create phantom tough questions. CAT does not do that. Lot of students panic in a paper because it is way easier than all the mocks they have done. They have difficulty adjusting to the fact that they can attempt 34 questions in Quant in CAT, whereas they never managed more than 22 in any mock. Finding a paper that is easier than mocks can disorient you. You end up shooting for too less, end up adjusting too late, and end up with a just-miss scoreline.

We at 2IIM have resisted every urge to make our mocks tougher and fought tooth-and-nail to keep it representative. This is why good students can sometimes attempt 75-80 questions in our mocks. Remember, the very best ones can and will attempt that many in actual CAT. So, it is very important to keep in mind that the level of difficulty will not be outlandish. If you are really good, but have never attempted more than 60 questions overall in a CAT paper, how will you adjust on the day of the exam to have a crack at 84 questions?

Try your best to have some fun in the paper. be very zen and forget the percentile, and go in thinking "this exam is going to be easy". Best wishes for CAT.