This article was published in our December 2014 Newsletter, by Tom Gottschalk, COO
Many EMS teachers and program directors will seek out help from our customer service advocates around the time they need help defending their testing practices. In fact, I believe this is one of the main motivators for adopting our system for their students to use. Therefore, I thought it might be a perfect time to dust off and re-gift an incredible document created by my mentor Douglas Smith sometime ago. While the pictures are mine, the expertise is all his brainpower via my keyboard today. I hope I do his work justice here.
Class test results (Upper third)
Part A is called the Cut-score Average for the class. Based on the actual cut score for the exam, this is the average of all the students grades. NOTE: Zeros and incomplete tests may skew results.
Part B is called the Raw-score Average and is just that. This is the average class scores based correct answers and incorrect answers without factoring the tests Cut score. Frequently this outcome is less than the cut score average because exams tend to be harder than the required passing score for the class.
Part C is the KR20 or Kuder–Richardson Formula 20, which indicates how well the exam performed based on time and likelihood of abnormalities during testing (I.E. cheating). This number should be equal to or greater than 0.70 to be satisfactory.
Part D and E are rarely referenced by faculty because the input from student is often not utilized during testing. Part D reflects how confident students are answering questions while Part E identifies when student indicates a desire to review a topic area by objectives.
Individual student results (Middle Third)
Now we report the student’s individual test results. Column 1 is the student’s grade based on the tests cut score, Column 2 is the percentage correct (raw points/score), and then of course column 3 is the number of questions right versus number of items on the test.
Section 4 allows you to see a breakdown of the students results based on the four levels of feedback we share with them.
Class detailed results (Bottom Third)
Then there are the brass tacks. In the upper left hand corner of this image is a re-grade tests button. If for some reason (other than your grade book not being set up correctly) the test grades do not look accurate, you can force the system to re-grade the entire exam. IMPORTANT NOTE: I do not recommend pressing this button indiscriminately, particularly during testing or after a class has graduated. While it may only seem time consuming to re-grade a test, statistically it is also possible to change outcomes slightly based on the latest item response numbers at the time of pressing re-grade. Therefore I recommend you use the feature wisely or ask for help if you wonder about it.
In the upper right corner of the last image is the export answer to excel button, which allows you to see the actual testing experience of all the students in an Excel table. As mentioned in another article, this may help you identify if there is any testing inconsistency that may have occurred like cheating.
In the Answers column you will see the correct answer shown in green. To the right of that column are number of times students asked to review a question during testing taking.
Then continuing to move left to right takes us to the scientific numbers for the exam by nation and your particular class. Callout I, II, and III emphasize there is a P value, Difficulty, Discrimination, and Point Biserial reference for every question when available.
And finally on the far left side (highlighted in pink), is an indication that an instructor can click on the red item number to hyperlink to the actual questions on the test to review.
That is it, in a not so small nutshell. Granted there are some very heady topics within each area, but in the interest of time and the average attention span I purposely glazed over some of them. If a term or some topic needs some enhancing, I would suggest joining us at one of our statewide instructor education sessions. Just check out https://www.platinumed.com/events to see one coming to your area soon.