Friday, December 2, 2016

Online Test Prep for COGAT and NNAT

I get a steady stream of really great questions from readers some to my email, but many just in the comments to random pages I've posted over the years.  Someone asked about the differences between the COGAT and the NNAT, but deleted it before I could write my 3 page answer.  I think covered this before because it's such a fascinating topic to insane people like me.

Here's a great question about online material and about what to do if you've exhausted all the material on the market.   If your child is in the unfortunate range of 2nd grade to 4th grade, you're going to exhaust the material on the market very quickly, because there is very little of it.  I think the question of online material is a really good one.

Here's the question.

My child is in 1st grade and we are exploring materials for NNAT. We have completed working through a couple of NNAT prep books (smart cookie, building thinking skills level 1), but I am looking for online practice tests since the test will be online. I looked at testing mom, but getting suspicious after reading your review and also after getting multiple emails a day urging me to pay after registering with them. I looked at mathtosuccess and testpreponline websites, but confused about which one to use. Any recommendations on online NNAT testing materials?

Here's the answer.

There are four reasons why I don't recommend online test prep websites.

The first reason is that putting either of my sons in front of a computer is like giving them addictive drugs. They're intellects are just going to end up looking like those before and after people on websites that show the effects of crack cocaine.  The key to the last 15% of the score on any type of test is concentration, patience, and checking the answer, but that submit button sits there on the screen like the devil whispering "click the mouse click the mouse click the mouse click the mouse.

I'm a business consultant at a boutique IT consulting firm.  All of our business consulting goes into producing websites, and everyone on staff, including me, is an IT engineer with about 25 years experience.   So 5 years ago I created a website that had hundreds of really clever questions and sat my child in front of it.  I even hooked up a credit card service which would not bill people automatically every month even if they asked for it because I hate sites that do that.

I sat my kid down in front of our computer and watched him go through the first 100 questions.   Then I printed each of these questions and forbid him from using the computer again.   I am convinced that the education industry doesn't subject their own children to the material they publish. It works great for kids below the 40% mark, but to get to 99%, you have to look really hard.

The second reason why I don't need to use a computer is that the NNAT book, is that after we've spent 3 months going through the NNAT book, we're at about 75% and then we can go through it again properly over the next 9 months to get to 99%.  Think about that.  It's actually two books, one to go through once the normal way, and one to go through it again properly.  If you just bought this book and you only have two months before the test, you can read what follows and just do things right the first time with your 2 months, which could be enough anyway.  My very first GAT test effort was a 2 month crash course in desperate test prep.

For the second pass, I went through the book and marked all answer choices in a sloppy way so my child wouldn't know which one he marked the first time.  I tried erasing but you could still see the original answer.

Instead of asking for a correct answer, I asked him to explain what is going on in the problem and why the author chose the 4 possible answers to confuse him.  What makes each wrong answer a good candidate for a wrong answer, and what is aspect of the problem is missing from each incorrect answer so that it is incorrect.   At the end of each answer, we categorize the trick that the author used to try to make the student get the incorrect answer.   This is a piece of paper with a list on it.  As a bonus, you should write down the question numbers next to each trick on your list.

To do things right the first time, you should not ask for an answer, but go through the 3-5 questions you are going to do that day and just ask for a description of the problem. Cover the answers.  Then go back again and ask for a description of each answer choice and how it relates to each question.  The go back a third time and ask for the correct answer.  After a month of doing this, you can graduate to this exercise on each problem - description of the question, analysis of each answer choice, and then pick the right answer.  It takes a long time (which it should because the test will), enforces the right skills, increases learning, and makes an expensive book last a long time.  Which is good because there are so few books to use.

The tests aren't looking for a child who is adept at NNAT or COGAT type questions.  Instead the test is looking for a child who thinks through things properly.  This is where my child academic skills list came from and the GAT parenting skills required to support these academic skills.   The actual test, whether it's NNAT or COGAT is going to be way harder than the practice material anyway, and mastering practice questions is not a predictor of success on the actual test, but mastering the academic skills during the practicing is.

The last thing we do is to take the list we created and I ask my child to create their own question for each item on the list.  If the first draft stinks, and it usually does, we can refer to the list to see what question numbers were for this trick, reread these questions, and then try again.  The end result is usually subpar, because I'm dealing with a 5 or 6 year old, but even in making a subpar attempt that is perhaps wrong, this child just went through a learning exercise that is way, way beyond the 99th percentile and the benefits are extraordinary.  I have know of two readers who figured this out on their own without 3,000 hours of test research and one author (me) who had to figure it out the hard way.

The third reason why I don't use online test prep material when I've exhausted other avenues is that an online company would have to be insane to write their questions at the proper level, which is 85% to 99%.   Most of the market needs things in the range of 40% to 75% with some spoon feeding. There is no point in exposing any child to a question at the 50% level.  It teaches nothing important to that child. There's no skill in doing something that a child can do.  Would you make an online course for 5% of the potential market when you can make money off of the other 95%?

I would not have explained in detail my full approach to test prep which I just presented above 5 years ago.  If you read the articles in depth (there are now 475 of them and counting), you'd see it, but I didn't spell it out in a convenient formula.  My official reason is that I don't write a lot about my experiments until I know which ones will pay off and why.  This saves me the trouble of having to go back through my blog and delete articles that would steer parents in the wrong direction.   My unofficial reason is that I would be nuts to subject my children to unnecessary competition from children who are way smarter and parents who read about my diabolical cheating methods.

On that note, I am getting questions from readers of my 6th parenting graders peers about what to do during this super critical period of testing and grades.   I don't have much to say right now, but when I'm out of the danger zone, I'm going to refer back to this article and point out that the answer was already there.  It was there 5 years ago as well, but one would actually have to read my articles to see it.

For the 2nd to 4th grade range, the material is so bad that I created the Test Prep Math series.   As a test prep author, I felt an obligation to get a copy of everything on the market, and we exhausted everything through the six grade before my child turned 8.   I really need to spend more time explaining how awesome this series is.  My older child demanded that I change the cover.  "Add a stick dog with brown hears, some swirly read at the head, different font sizes, and other useful stuff."

I set out to create a course that hammers away at cognitive skills from about the 75% to 99.999% level.  My 10 year old co-author was in charge of goofy names and making sure the questions "aren't lame".  Both of these kids are now 3 years in math, a point which I didn't think was possible, especially with children who aren't especially gifted or exceptional in any way.   As predicted by the COGAT author, cognitive skills are really powerful.

Back to first grade.   With everything I mentioned above about using test prep material, and the real skill set behind succeeding on the test (which I've covered extensively in the last few years), I found that a really great approach is to use reading comprehension books.  They don't have figure matrices, obviously, but they have the exact same skill set required, and for us, they're really hard if you get material 2 years advanced, even material that the parent has to read because the material exceeds the child's reading level, and the parent has to explain word definitions for the child to get through the question.

That is the forth reason why I never got around to using online courses.  Reading comprehension is way better test prep, and it takes a whole session to get through one question.  That's also why the questions in Test Prep math get longer and more confusing as you progress through the book.   My the half way point of Level 3, the questions are downright convoluted, and both my child and I had to read the question a few times to figure out what the heck I was asking.  That's the #1 skill you want your child to have going into the test.   He should mistrust his first reading of the question.   I also designed each question so that an incorrect answer is inevitable on the first try.  I rewrote any question that my test group got correct the first time in the last half of the book.  That's the #2 skill.  A child should mistrust their first answer.

Reading comprehension is designed specifically to require these two skills.  That's where I learned it. Without these two skills, a bright student is going to miss 15% of the questions on a both a cognitive skills test and a standardized test.   This exercise is so powerful, and so painful, that we only did a single question from a reading comp book (2 years ahead) every other weekend.   And that is my final answer to the question above, reason #4.

While Test Prep Math directly steals these aspects from reading comprehension, I wrote it so that we could do a question each day from each section.   We couldn't do this with a challenging reading comprehension book.  Getting through a single question from a reading comprehension book is like taking a practice test; this works every other week but not every day.   If I were desperate and the test was in 10 days, we would do reading comprehension every other day or every day, but it would be painful.  Also, Test Prep Math targets working memory and number sense skills, starting at a failry reasonable level and then going to a ridiculously high level in a more step-by-step, manageable approach, because, after all, a reading comprehension book is not going to help with the quantitative section.  There is no Level 1 to this series, because most of my readers are not struggling at age 6 with quantitative issues and I there is good material at this age.

Finally, I need to address the part of the question "because the test is online".   The answer is the same if the question said "because the test is the COGAT" or "because the test is the NNAT" online or in a classroom.   The last 5% of every score depends on the child being familiar with the format of the test.  This has been well documented by the test authors themselves.  You need a practice test, and I strongly recommend simulating a short test as close as possible to the test your child will take.  

I went through the more thorough answer to distinguish the practice effect from real learning (as in the real learning of the cognitive skills the test is measuring).   If the test is online, you have to get an online practice test, or an online test prep service, choose 20 questions, and make it a formal practice test exercise, preferably in a class room setting with reasonable distractions.  Even with practice COGAT books, this wasn't our primary learning exercise (a formal practice test session) and I don't see learning or skills improvement happening during this process, nor would I recommend doing it a lot, but the 5% is real.

The study did not use really hard test questions.  It's the format that helps with the 5%.  Your child's performance on this practice test will not indicate how they do on the real test, and because of that, I don't think which online service you choose really matters.

1 comment:

  1. Norwood, thank you so much for taking the time to post such a well thought and articulated response! I completely agree with your views on computers akin to addictive drugs. But such is the current state of affairs. Not to say I am succumbing to this, but NNAT test is solely administered online. So there is no way around that enticing "Submit" button. I have had friends whose kids could not perform at their level on the test because of these distractions - clicking the bubble correctly, being able to look at a pattern on the computer and matching it with the correct answer etc.