It recently came to our attention that our content and testing validation was put into question. We take great measures, and great pride, into having the most valid and reliable testing on the market to date. Quite simply, we have the best products and customer service bar none. We would like to take the time and explain our processes below. If you have any questions please give us a call or email us. We value you whether you’re a friend, business partner, or customer.
Each item was originally created based on the Registry Practice Analysis, the Educational Standards, and the National Curriculum. All available commonly used textbooks were referenced and when there was a conflict, common practice or the most common response was used. The gold standard reference for drug dosages and indications as well as Cardiac algorithms come from the AHA Guidelines and is updated with each AHA update. Reading levels are reviewed and we attempt to keep each item within 2 grade levels of the expected reading level for each provider level. Each question is assigned a Bloom level and estimated cut score. Each item is annotated to explain why the correct answer is correct and why the good distracters are not correct. Other content experts, educators, and editors review items. Our Medical Director reviews all high stakes items, such as those that might be found on a final exam.
All questions are reviewed for cut scores once over 100 candidates have responded and are adjusted based on actual results and the Nedelsky method of calculating cut scores.
The results you receive if you use the full system and the students take the tests on line are:
- A KR20 Reliability if over 4 students have participated
- The class average for the test
- The Registry, Educational Standard, and National Curricular reference for each question if applicable (not all items have representation in all three categories)
- The National Difficulty and Discrimination (If available-must have over 40 results in our system) and your classes Difficulty and Discrimination (if over 4 responded) results for comparison (Calculated by taking the upper and lower quarter performers and identifying Difficulty = (U-missed + L-Missed)/(Total U and L), Discrimination = (L-missed – U missed)/(Total U)
- The breakdown of responses for each question
- The number of responses from the students for items they would like to discuss in class
- The student reported confidence level and success rate
- Results for each student (available for the student and instructor)
- Individual student start and end times
- Individual average performance by reading level
- Individual average performance by cut score
- Individual average performance by Bloom level
- Individual average performance by Registry Reference
- Individual average performance by Educational Standard
- Individual average performance by National Objective
- An opportunity for students to review each question with annotation
Our final exams and computer adaptive tests provide similar results and have been validated the same way our item bank was validated.
Available upon request are monthly reports of the difficulty/discrimination numbers from our test bank. On the second Sunday of each month we receive a printout of the results of our Computer Adaptive Testing (EMSCAT) results. As of June 2015, we have had over 25.8 million responses of our 4,000+ question bank to aid in our validation process. In addition, there are numerous places for instructors and students to provide feedback. Each of these submissions is reviewed and, if appropriate, acted upon.
Our final exams and other high stakes exams are also blueprinted, reviewed, and evaluated for difficulty and discrimination and piloted to over 100 individuals prior to implementation with program and medical director feedback as well.
It is true you can have a valid question bank and yet still create a test that is not appropriate, such as pulling 50 valid OB questions from a bank and offer that as your cardiology exam. Some programs out on the market do not trust you the educator to create valid exams and therefore do not give you the opportunity. We believe you can create valid tests and we also believe it is critical that you do so as you are the most knowledgeable person on where your students need to be evaluated. We always recommend that you blueprint the test, review the test prior to administering to your students, and have someone else review as well, such as other educators within your program and your medical director.
Thank you for giving us the opportunity to provide this to you. We hope this answers your questions. Please let us know if there are other specific questions you would like answered.
Platinum Educational Group