Mock CATs, Mock CAT percentiles, predictions etc

I am going to start with a series of quotes.

There are two ways of lying. One, not telling the truth; the other, making up Statistics. 

There are lies, damn lies, and statistics.

Statistics are used much like a drunk uses a lamppost: for support, not illumination.

Replace Statistics with Percentile analytics and this would fit in perfectly in the context of CAT. No matter how many fancy charts are shown, it is the responsibility of the students to not over-interpret the results. When you take a mock CAT, the review happens on three planks -

1) Learn better methods by looking at the solutions
2) Think about how you could have improved your attempts by selecting better questions.
3) Evaluate your percentile.

Of the three, the least useful is no 3 (probably also the most harmful). Because it is the least reliable. Almost all mock CAT providers use look-up tables to give percentile scores. Gone are the days where a mock CAT provider could get 10,000 people to beat themselves into a frenzy on a given Sunday to take a mock CAT at 100 god-forsaken locations in the Country.

These days, proctored CATs probably have a sample size in the range of 1% of CAT takers. These are likely to be the most serious, most experienced, most-driven of the CAT takers. So, the test providers "adjust" the look-up table to factor this in. In layman terms, this boils down to - they look at the actual numbers, feed in some other numbers, and pat themselves on the back.

So, the percentiles are fairly unpredictable. And contrary to what all the service providers claim, there is no way to back-verify that database. The only ones who will call a coaching institute and tell them that they scored a decent percentile are the ones who got a decent percentile. So, the reverse-verification process is bogus. So, no one actually knows whether any percentile number has a serious correlation with the actual numbers. There is way more anecdotal evidence to say that the percentile numbers are erratic, than to say that they are accurate.

I recently heard that there was a percentile predictor that predicts percentiles based on numbers you feed in based on the scores you have got on other exams (some of my students called me to give me an update on their predicted results.). I almost fell from my chair laughing. Students who believe that there is a scientific basis for this kind of predictor should re-examine their brains. This level of gullibility cannot be good for the body.

The thought process behind the predictor is as follows - Take three not scientifically proven indicators with no established proof from back-verification, define arbitrary weightages based on your genius to these, come up with a fourth metric and call it a prediction. And supposedly students are flocking to this predictor engine. No wonder soothsaying, tarot card reading are all flourishing industries in our Country. If smart, ambitious people can be made to believe there is such a thing called a percentile predictor, god save the masses.

Anxiety does strange things to the mind. Fretting and fuming about this exam makes you do all kinds of over-analysis. The mind needs something to cling on to, to feel that you are on the right track. Ergo, predictor.

Think of all the guys who have completed their MBAs from IIM A, B, C and think about how each of them would react if you said "percentile predictor" in their midst. I will give you a hint. I spoke to a bunch of them and the only common factor was laughter.

So, what should we do?

I am going to say something that is difficult to implement. The last 2-3 weeks, you must focus on three things

1. Chuck the percentiles. Seriously, chuck them. Forget about them. Banish them.

2. Focus on still being curious and not getting sucked into this pressure vortex. The first 2-3 mocks that you would have taken, you would have enjoyed grappling with the questions. Slowly, the pressure takes over and you stop having fun in the mocks. You are too worried about the score (and that damned percentile). The ones who crack this exam are the ones who are likely to look at a question and go "Wow. Super question. I think I will enjoy solving this". If that joie de vivre is gone, you should rekindle it.

3. Mentally prepare for a paper that is far simpler than your mocks. In our Country, the service providers have a feeling that to create a great mock CAT, you must make it very tough. So, they create phantom tough questions. CAT does not do that. Lot of students panic in a paper because it is way easier than all the mocks they have done. They have difficulty adjusting to the fact that they can attempt 34 questions in Quant in CAT, whereas they never managed more than 22 in any mock. Finding a paper that is easier than mocks can disorient you. You end up shooting for too less, end up adjusting too late, and end up with a just-miss scoreline.

We at 2IIM have resisted every urge to make our mocks tougher and fought tooth-and-nail to keep it representative. This is why good students can sometimes attempt 75-80 questions in our mocks. Remember, the very best ones can and will attempt that many in actual CAT. So, it is very important to keep in mind that the level of difficulty will not be outlandish. If you are really good, but have never attempted more than 60 questions overall in a CAT paper, how will you adjust on the day of the exam to have a crack at 84 questions?

Try your best to have some fun in the paper. be very zen and forget the percentile, and go in thinking "this exam is going to be easy". Best wishes for CAT.

Labels: , , , , , , , ,

IIM - CAT Coaching: Experts' Insights: Mock CATs, Mock CAT percentiles, predictions etc

Tuesday, November 04, 2014

Mock CATs, Mock CAT percentiles, predictions etc

I am going to start with a series of quotes.

There are two ways of lying. One, not telling the truth; the other, making up Statistics. 

There are lies, damn lies, and statistics.

Statistics are used much like a drunk uses a lamppost: for support, not illumination.

Replace Statistics with Percentile analytics and this would fit in perfectly in the context of CAT. No matter how many fancy charts are shown, it is the responsibility of the students to not over-interpret the results. When you take a mock CAT, the review happens on three planks -

1) Learn better methods by looking at the solutions
2) Think about how you could have improved your attempts by selecting better questions.
3) Evaluate your percentile.

Of the three, the least useful is no 3 (probably also the most harmful). Because it is the least reliable. Almost all mock CAT providers use look-up tables to give percentile scores. Gone are the days where a mock CAT provider could get 10,000 people to beat themselves into a frenzy on a given Sunday to take a mock CAT at 100 god-forsaken locations in the Country.

These days, proctored CATs probably have a sample size in the range of 1% of CAT takers. These are likely to be the most serious, most experienced, most-driven of the CAT takers. So, the test providers "adjust" the look-up table to factor this in. In layman terms, this boils down to - they look at the actual numbers, feed in some other numbers, and pat themselves on the back.

So, the percentiles are fairly unpredictable. And contrary to what all the service providers claim, there is no way to back-verify that database. The only ones who will call a coaching institute and tell them that they scored a decent percentile are the ones who got a decent percentile. So, the reverse-verification process is bogus. So, no one actually knows whether any percentile number has a serious correlation with the actual numbers. There is way more anecdotal evidence to say that the percentile numbers are erratic, than to say that they are accurate.

I recently heard that there was a percentile predictor that predicts percentiles based on numbers you feed in based on the scores you have got on other exams (some of my students called me to give me an update on their predicted results.). I almost fell from my chair laughing. Students who believe that there is a scientific basis for this kind of predictor should re-examine their brains. This level of gullibility cannot be good for the body.

The thought process behind the predictor is as follows - Take three not scientifically proven indicators with no established proof from back-verification, define arbitrary weightages based on your genius to these, come up with a fourth metric and call it a prediction. And supposedly students are flocking to this predictor engine. No wonder soothsaying, tarot card reading are all flourishing industries in our Country. If smart, ambitious people can be made to believe there is such a thing called a percentile predictor, god save the masses.

Anxiety does strange things to the mind. Fretting and fuming about this exam makes you do all kinds of over-analysis. The mind needs something to cling on to, to feel that you are on the right track. Ergo, predictor.

Think of all the guys who have completed their MBAs from IIM A, B, C and think about how each of them would react if you said "percentile predictor" in their midst. I will give you a hint. I spoke to a bunch of them and the only common factor was laughter.

So, what should we do?

I am going to say something that is difficult to implement. The last 2-3 weeks, you must focus on three things

1. Chuck the percentiles. Seriously, chuck them. Forget about them. Banish them.

2. Focus on still being curious and not getting sucked into this pressure vortex. The first 2-3 mocks that you would have taken, you would have enjoyed grappling with the questions. Slowly, the pressure takes over and you stop having fun in the mocks. You are too worried about the score (and that damned percentile). The ones who crack this exam are the ones who are likely to look at a question and go "Wow. Super question. I think I will enjoy solving this". If that joie de vivre is gone, you should rekindle it.

3. Mentally prepare for a paper that is far simpler than your mocks. In our Country, the service providers have a feeling that to create a great mock CAT, you must make it very tough. So, they create phantom tough questions. CAT does not do that. Lot of students panic in a paper because it is way easier than all the mocks they have done. They have difficulty adjusting to the fact that they can attempt 34 questions in Quant in CAT, whereas they never managed more than 22 in any mock. Finding a paper that is easier than mocks can disorient you. You end up shooting for too less, end up adjusting too late, and end up with a just-miss scoreline.

We at 2IIM have resisted every urge to make our mocks tougher and fought tooth-and-nail to keep it representative. This is why good students can sometimes attempt 75-80 questions in our mocks. Remember, the very best ones can and will attempt that many in actual CAT. So, it is very important to keep in mind that the level of difficulty will not be outlandish. If you are really good, but have never attempted more than 60 questions overall in a CAT paper, how will you adjust on the day of the exam to have a crack at 84 questions?

Try your best to have some fun in the paper. be very zen and forget the percentile, and go in thinking "this exam is going to be easy". Best wishes for CAT.

Labels: , , , , , , , ,

0 Comments:

Post a Comment

Subscribe to Post Comments [Atom]

<< Home