How to write better test questions for any assessment

What makes a good question?

Quizzes, tests and exams can yield impressive results and give you a wealth of information about their respondents.

They are irreplaceable in business standardized testing, especially in recruitment, HR departments, training, or sales and customer service team evaluations. They are also priceless at all levels of education, where they not only measure but also facilitate teaching and learning processes.

To get reliable and valid information from your respondents, you must ensure that you phrase correctly the questions you ask. They must be free from any bias, leading or other issues which obscure the result, making it unactionable and worthless.

A good question serves its purpose. It gives you, as a test author and your respondents, the information you need. For the former, it means qualitative and/or quantitative insights into test-takers' skills and knowledge. For the latter, an objective assessment or evaluation of their performance and indication of strong and weak spots.

Qualitative vs quantitative test questions

We use quantitative questions to measure skills and knowledge.

That is, to quantify and very often grade it in the evaluation-type quizzes, tests and exams. It is intended as an objective way of establishing respondents' level of performance, often versus thresholds. Examples of multiple choice questions have several essential applications in the business world.

We use qualitative questions to explore skills and knowledge.

Here, the purpose of the assessment is to dive deep into the topic and find out more about a person than just the thresholds of their abilities and performance.

The results of such assessments are often subjective. Ultimately, it is at the test organizer's discretion to determine if the outcome is positive or negative.

Is making test questions difficult?

This problem bothers aspiring recruiters, trainers and teachers. The answer is: it doesn't have to be!

Testing is a crucial part of business and educational processes. When done right, it offers fail-proof data, eliminates bias, and helps to set the course for success. It is also a form of feedback that lets teachers and trainers measure the result of their efforts and tells them what to change to be even more effective.

To create accurate and reliable tests, you should follow basic rules and read the questioning methods you should be using. Assessing or evaluating knowledge also requires skillful use of the three most common question types: multiple-choice, short open-ended, and descriptive.

Preparing questions for assessments and evaluations can be challenging. However, you will get the most out of your tests when you learn how to phrase them right. Following some simple rules will give you the actionable insights you want in less time with less effort.

Things to consider before you start

Your tests must be consistent with the requirement of the offered position, training plan or course curriculum. In the case of training and education, it must also include the exact information you shared with your learners. That is why in most cases, you should write your own tests rather than trust ready-made ones you can find online.

How to write quiz questions that guarantee all the necessary insights? Begin with a detailed plan of the assessment. Review the content of your course or training. Consider the most important aspects of the job you’re recruiting for. Then decide which types of questions will suit your purpose.

Consider the age of your respondents, their professional experience or the level of the job opening. Use different questions when recruiting for junior, senior and executive positions. Choose between short open-ended, descriptive and multiple-choice questions.

We will help you author any evaluation for any kind of respondent and always get exactly the insights you need. With our 101 on creating test questions for on-site and online evaluations, you will avoid all the possible pitfalls.

How to write good test questions?

Start with understanding what a test is. Writing good test questions can be difficult and time-consuming. Fortunately for you, we compiled a list of shortcuts to take when you need to save time and create quality questions that will give you actionable data.

And just in case you think you’re not skilled enough to write questions, trust us, you are. All you need to do is follow some simple steps, and your school efforts, hiring processes, training or workforce management processes will make your work easier and more effective.

Creating a test for both on-site and online use starts with five points, all of which you must carefully consider to succeed with your recruitment process, training or teaching. Remember that they apply to all kinds of questions.

When deciding on the source of your test questions, their number and types, you must carefully consider all the five measures mentioned above.

Start preparing even before you write your first question

No one is born with the ability to create good questions. But it can be done, and in fact, it is much easier than you might think!

All you need to challenge your respondents, engage them and produce actionable insights for training, recruitment or teaching is a bit of practice. That's why we explain step-by-step what to do to maximize your chances of creating an excellent online quiz, test or exam.

Make sure you tell your job candidates, trainees or students, what kind of questions to expect. You might even show them examples of test questions you are going to use. Surprises can cause excessive stress and have a negative impact on their performance.

To ensure your results are valid and reflect real skills and knowledge, you can also practice answering the types of questions with your groups before scheduling the test. You can use the initial call to tell candidates about the test for hiring processes.

In any case, don’t forget to highlight the benefits of such tests for the other party. Your respondents will:

• save time, especially in the case of remote assessment on a professional online testing platform, and get results faster,

• have their answers graded objectively, without bias (including unconscious bias, which is hard to avoid without tests).

Creating effective open-ended, short-answer test questions

If you want to know how to write good short-answer test questions, you need to start by learning about the advantages and disadvantages of this type of test task.

Short-answer questions require respondents to provide their own answers rather than choose one from the available options. Business professionals, teachers and professors use them because:

• They check knowledge more thoroughly than true/false (T/F) or other types of multiple-choice questions (MCQs).

• Respondents can't just recognize the correct answer among those provided. They must know it.

• Successful blind guessing is impossible without at least some knowledge of the topic.

However, short-answer questions come with some drawbacks:

They are much more difficult to grade than MCQs. You need to grade them by hand unless you use a professional online platform that supports the automatic grading of short-answer questions.

Test authors must write their questions correctly to avoid causing confusion by allowing more than one correct answer. When incorrectly phrased, they might encourage respondents to start guessing what the test author had in mind.

When you create short-answer questions, make sure you phrase the problem in such a way that the respondents only have to answer with one or two words.

It is a perfect way to verify knowledge of dates, names, etc. Remember also that if you decide to run an online test, with the help of a good remote assessment platform, it will be able to grade short-ended questions for you. It is a great help and a time-saver.

Creating good descriptive (a.k.a. long answer or essay) questions

To make your online quizzes, tests and exams more thorough and varied, you can enrich them with descriptive questions. They are even more demanding than short-answer ones and can give you deeper insights than multiple-choice questions.

First of all, you must be proficient with the knowledge you want to test. If you make mistakes when formulating your descriptive questions, don't expect correct answers from your respondents. Also, you should always double-check the content for any mistakes or, even better, have someone else verify it for you.

Keep your stems as short as possible. Use only those words that provide context to your respondents. Make sure none of your phrases is ambiguous. If a sentence has two meanings, be sure that test-takers will interpret it the wrong way, and the validity of your result will suffer.

Make time to check your tests. Descriptive questions take less time to formulate than, for example, multiple-choice ones, but they are very time-consuming to grade. Here, you can't use any answer key, so it will take a lot of effort if you want to get real insights.

You can still use online test platforms to give your assessments and evaluations in a remote or hybrid environment. It will be convenient for both you and your respondents, but still; you will have to grade all your descriptive questions yourself.

Creating good multiple-choice questions

Multiple-choice questions (also called items), or MCQs, are the most common type of questions. You can find them in on-site and online tests used to streamline recruitment, offer better professional training, evaluate employee performance and, of course, in education.

Before we start discussing the best practices for creating, we will briefly discuss what they are and what purpose they serve in assessment and evaluations.

A multiple-choice question consists of two parts: the stem and the alternatives. We'll dive into that subject in the next part of this article.

The stem is the first part which identifies the problem and ends with the question. It should be brief and coherent, so that test-takers don't have to waste time trying to guess what the authors had in mind.

Alternatives, in the form of possible answers, form the second part. They consist of one short sentence or phrase. There are usually four of them, although it is ultimately the organizers' decision if they introduce more or less of them.

Alternatives themselves also consist of two parts: the distractors and the correct answer.

The distractors should be fairly believable, although incorrect answers related to the test question. They should be of similar length, word choice and style as the correct answer and not different among themselves.

The correct answer is, hopefully, one that your test-takers will choose the most often.

Multiple-choice question writing best practices

Before we start explaining what a good multiple-choice question should look like, let's see an example of a poorly created one.

Did you notice the problems with the question above? See the list below and find out if you managed to spot all of them:

1. The stem (the problem posed by a test question) is too long, contains irrelevant information and has a double negative.

2. The alternatives are of different lengths, which might confuse respondents.

3. The 3rd alternative contains capital letters, which might lead test-takers to think it is the correct answer

4. The "None of the above" is the 4th alternative of 5, so it makes no sense when only one answer is correct.

5. There is a typo in the last alternative.

Now, let's take a look at the same question written according to good practices.

To know exactly how to write meaningful, effective tests, you must understand what a question is made of. There are 3 main components of every good MCQ:

1. The stem

Learn how to write a good test question, write it in a clear and consistent manner and do not try to trick and confuse your test-takers with your wording, etc. The stem should be short and concise, but you should avoid using acronyms and abbreviations at all costs.

Create your stems in such a way that the alternatives are positives, not negatives. Avoid phrasing such as "What is the least common type of...". Instead, choose "What is the most common type of...", etc. If your question requires respondents to indicate the incorrect answer, use capital letters and write: "Which of the following answers is FALSE?".

Remember, your respondents will probably be stressed and in a hurry, especially in cases of hiring processes or high-stakes exams. Make sure all instructions are clear and easy to understand. Avoid using unnecessary and irrelevant words in your question stem, as it can confuse readers and potentially impact their performance.

Finally, your stems should include full sentences. Using gaps in the test question and possible words to fill in those gaps significantly increases the cognitive load of the problem, creating an additional level of confusion, which is absolutely unnecessary. It might severely impede the validity of your test results.

In layman's terms, create a quiz with challenging but not overly complicated questions. Keep in mind that to effectively check your learners' skills and knowledge, you should create the stem in accordance with the advice from this article.

When asking the question after explaining the problem, use constructions such as:

• "All of the following answers are correct, EXCEPT..."

• "Which of the following answers is CORRECT?"

• "Which of the following answers is INCORRECT?"

• "Choose the CORRECT answer."

• "Choose the INCORRECT answer."

2. The alternatives

Keep in mind that the order of your alternatives should not in any way suggest which answer is the correct one. To achieve that, you can arrange them numerically or in alphabetical order.

Also, to get better result reliability, all the alternatives you choose for your test questions should be plausible. The goal should be to force test-takers to either employ their knowledge and point to the correct option or use their intelligence to rule out incorrect answers.

If you choose to include an obviously incorrect alternative, it might trick some students into selecting that one, thinking it is too weird not to be correct.

Use present tense for all possible answers. Avoid using double negatives at all costs. Phrases such as "Anna wasn't unhappy about his presence" or "He couldn't find his car keys nowhere" are extremely confusing, especially for younger learners, but might also be tricky for job applicants or during professional training tests.

3. The distractors

Your distractors should be mutually exclusive. Selecting any one option, correct or otherwise, should be tantamount to considering all other answers incorrect. Only one correct answer per question is a way of making sure respondents concentrate on finding the correct solution, not wondering if the test author is trying to outsmart them.

Your distractors should not give any clues as to which answer is correct. The way you phrase each alternative should be precisely the same. Also, you shouldn't include any hints and clues, such as hidden indicators (logical errors, obvious mistakes), that a certain alternative is incorrect.

Own test questions vs ready to use content

Creating tests is a time-consuming process, especially for those business professionals and teachers who don’t have a lot of experience under their belts. It is no wonder that they often prefer to take a shortcut and look for ready-made tests which are available online.

Unfortunately, in most cases, it is a bad idea. The more complex the subject of your tests, the higher the stakes, the more useless all the pre-generated tests will be. Also, the content you can find online can have little to do with the position you have to fill, your training or the subject of your lessons.

You might sometimes struggle to come up with question ideas for your business-related test. When in doubt, add real-life scenarios from work! This method is perfect for measuring how a job candidate might perform when faced with typical situations they will encounter if hired.

Another reason why you should concentrate on creating tests yourself is that saving time by using ready-made content is an illusion. On many occasions, you will find that you can spend more time browsing websites in search of the right test than you would if you just sat down and created one yourself.

Tests created by other authors might be of poor quality. They might have accuracy problems, incorrect answers shown as correct, etc. This might lead to serious problems, such as mishires, awarding certificates to trainees who didn’t perform or giving good students bad grades.

Even if you decide to use ready-made tests from a trusted source, proofread them before you use them. Remember, you’re going to use a test authored by someone else, but in case of problems, you will pay for their mistakes. Always check the accuracy of questions and answers before you give them to your learners! Also, remember that if you find a test online, your respondents can find it as well.

So, why should you avoid ready-made content and create your test questions?

Benefits of creating own test questions

Pre-generated tests would be a great idea, but only if all people were identical. Unfortunately, they aren’t. It means that if you want to get real, actionable insights from your candidates, trainees or students, you should tailor-make your evaluations for them.

You’re a teacher? No one knows your learners better than you. No one knows how you teach and how often you decide to go beyond the textbook to include additional relevant information during your lessons and training sessions.

You’re a recruiter? You probably look for the perfect candidates for various positions with different responsibilities. Write the questions yourself, and you will mitigate the risk of a mishire or time wasted during a pointless interview.

You’re a trainer? Working in professional development is demanding. What if you could easily prove the value of your efforts with insights from custom-made pre- and post-training tests?

Ready-made online evaluations are designed for the average test-takers. Your respondents are never average. For example, that kind of content will not take into account trainees, employees, candidates or students with special needs, who require a bit of a personalized approach to grow.

When you create your own tests, you have full control over them. Tailor those evaluations to your needs and to your respondents’ capabilities. Depending on the task, use only one kind of question or mix and match all of them to create a diverse test.

You can increase engagement with pictures, formulas or diagrams. Phrase your questions in a way that will not confuse respondents. Use enlarged print or other ways to make the content more accessible.

That extra time you spend preparing tests will definitely pay off. With better insights, you will be able to pick a real talent for the job opening, precisely measure how your training increased your team members’ performance or which students require more time to embrace the curriculum.

Making test questions is a skill. You can master it too.

To create a question and answer quiz, a test or an exam that ends with success, you must keep in mind the most important factors of every evaluation: validity and reliability.

The first piece of advice is that you should make your own quiz questions whenever possible. Ready-made assessments or quiz creators have the advantage of being available and easy to deploy but will most likely fail you when it comes to efficiency and result validity.

You should also learn how to write test questions and always consider your candidates’, employees’, trainees’ or students’ ability to solve problems before deciding whether to include multiple-choice, fill-in-the-blanks, descriptive or other types of problems.

Creating assessment questions is a skill you can learn and master if you follow the advice from professionals in on-site and online tests.

Many authors consider multiple-choice questions the most suitable to create quiz, test and exam content. It is correct, but to go one step further and check your test-takers more thoroughly, do more than just create MCQ test. Add short open-ended, or descriptive questions. This way, you minimize the luck factor and increase result validity.