(ISC)² Moves to Computer Adaptive Testing (CAT)

Dec 12 / Gwen Bettwy
You may already know that on December 18th, 2017 (ISC)2 moved English language CISSP exams to Computer Adaptive Testing (CAT). The question posed by many, is “What does this mean to me?” Let’s take a look at that and other related questions.

Will the changes mean that it is more difficult to pass the exam? Am I going to pass?

With the exam changing to the CAT format, the content of the Common Body of Knowledge (CBK)® will not change, only the methodology of how the candidate is screened for the knowledge will change. However, with a CAT exam, the candidate may feel that the exam is more difficult than it was before.
This is due, not to a change in the content or the questions, but how the exam is administered. With CAT the candidate will be given questions with an increasing level of difficulty, until the candidate has a 50% chance of getting the next topic question correct. At that point, the exam has determined the candidate's level of mastery for that topic, however the candidate will be feeling like they have missed more questions. Think of it as a hearing exam. The goal of the exam is not to show that you have hearing loss (failure) but it measures your level of hearing until the edge of your level of hearing is found. The important thing is to NOT give up when you get several hard questions in a row.

What is Computer Adaptive Testing (CAT)?

Computer Adaptive Testing (CAT) is a methodology whereby a candidate can be accurately tested to identify their level of proficiency in any given topic with a minimal number of questions being asked. Historically CISSP ® questions were individually scored and a number was assigned to each question indicating the percentage of candidates who answered the question correctly. The questions were sorted to create a “scaled score” so that each exam (collection of questions) was weighted equally, that is, no exam was full of “easy” or “hard” questions, but the questions were distributed evenly throughout all the concurrent versions of the exam. With CAT the same questions can be used, and the same weights can be applied to each question. The difference will be in how the questions are selected for each candidate.

Let us say that we want to see if a candidate has a 70% understanding of a given topic. A relatively easy question (one that 80% of former candidates have chosen the correct answer for) is given to the candidate. Assuming that the candidate answers the question correctly, a more difficult question is given on the same topic. Perhaps this question has a 60% rate of having been answered correctly. The candidate gets this question correct. A third question is posed, this time with a 40% pass rate. The candidate misses this question. Their level of knowledge is now considered to be somewhere between 40 and 60 percent. More questions will be given within this range of expertise until the candidate is getting approximately half of the questions correct. Once the calculations performed based on the questions given show that the candidate is assumed to have more than the minimum level of knowledge on a specific topic then the exam will not need to ask any further question on this topic. Keep in mind that the topics will still be mixed up, so the candidate will not get 14 questions on the same topic until their level of mastery is identified.

Is Computer Adaptive Testing (CAT) new?

No. Adaptive testing has been around for over 100 years. Obviously with the increased technology available with modern computing power, the mathematical calculations can be done by the computer at a much higher rate of speed than by humans. This increases the ease with which the questions can be selected.

Are there still going to be Beta questions?

Yes, there still will be beta questions which are being evaluated in order to establish a baseline difficulty for that question. This is required before the question can be used to measure a candidates’ relative knowledge of a topic. There will always be 25 beta questions within an exam.

How will the Beta questions affect my score?

The Beta questions will not be scored during the exam. Your responses will be noted and averaged with the responses of others until the question is evaluated as being properly vetted at which time it will be added to the pool of possible questions.

How many questions will I get?

That depends….gotta love that answer. (ISC)2 says that there will be between 100 and 150 items on the exam. Each candidate will receive a minimum of 75 and a maximum of 125 “counted” questions and 25 beta questions on the exam. How many questions you get will depend on how well you answer the questions. A lot of guessing on the exam can lead to more questions being asked to eliminate the “luck” factor. Once the system has consistently identified your knowledge level on a topic, there will be no more further questions on that topic.

How much time will the test take?

That also depends. This will vary depending on the amount of questions required for the candidate to prove their level of mastery of the CBK. The maximum amount of time for the CISSP CAT is three (3) hours.

Do I need to study more/less/differently?

As the CBK has not changed, candidates should not change their review strategy. Keep in mind that the CISSP exam has always been a management exam and is about how to manage security, NOT how to configure specific technology.

Why this change to the exam format?
The three primary reasons for changing the exam format are:
CAT can get a more accurate evaluation of a candidate’s competency with less questions.
Shorter test sessions.
Enhanced Exam security – less questions per candidate means less likelihood that any particular question can be compromised.

Where can the CISSP CAT exams be taken?

The CISSP CAT exams will ONLY be administered at Pearson Professional Centers (PPCs) and (ISC)2 authorized Pearson VUE Select Test Centers. This means that (ISC)2 is no longer allowing the exam to be given at mobile testing centers and will no longer be given by (ISC)2 Authorized Training Partners (ATP). Please plan accordingly as this is designed to prevent the CISSP exam being given at the end of an ATP training session. All exams will now have to be scheduled via Pearson VUE.