Do students spend more time solving appropriate challenges?

Originally posted on the Grockit blog at http://grockit.com/blog/main/2010/05/18/do-students-spending-more-time-solving-appropriate-challenges/

When you play a solo game in Grockit, the questions that you see are selected based on your recent performance. Our goal in doing this is to offer you challenges that are neither too easy nor too hard for you, in order to provide better opportunities for learning.

Our concept of how difficult you’ll find a question is based on our estimate of the likelihood that you will answer the question correctly. If, for instance, we estimate that there is a 90% chance that you will answer the question correctly, we think that you’ll consider that question to be pretty easy. Others may find the same question difficult, but you’ll probably find it to be straightforward.

Today’s question: Does a subjective measure of difficulty have any relationship with the length of time it takes a student to solve the problem?

Think for a moment about your own problem-solving experiences. Do you think that you spend more time on a harder question than on an easier question? Less time?

We record a lot of interesting data when students answer questions on Grockit, on some of this data can help us answer the question at hand. Below are two plots of the average time taken by Grockit students to answer GMAT Quantitative and ACT Math questions, as a function of how likely we believe the student to correctly solve the question. When you look at the plots, the first thing to note is that the relationship between subjective difficulty and response time isn’t linear.

These plots indicates that, on average, students are spending less time answering each question that are subjectively very easy or very difficult than on the questions that are in between. Do you find this surprising? Interesting? Does this reflect your own experiences studying? How would you interpret these plots?