DIGITAL QUESTIONNAIRE DESIGN - Psychometric's and Traditional
Best practices for designing online questionnaires require attention to format, medium, layout, and cadence. Our findings indicate that while the cadence of questions and layout of the questionnaire are similar for both traditional and psychometric testing, best practices for format and medium differ. Specifically, the most widely used and best format for psychometric questionnaires is to use multi-item psychometric scales, while best practices for cadence include grouping questions by topic, using images when appropriate, being specific, not including jargon, avoiding leading questions, and employing positive wording. Mobile technology is becoming the preferred medium for many psychometric questionnaires, and the layout of questionnaires should be short. Below you will find an overview of this topic, as well as specific findings.
OVERVIEW OF TESTING
When finding employees, training employees, or finding the best role for employees, including promotion opportunities, companies employ a variety of methods. While the process starts with an application submission and usually includes vetting and human interviews, companies are increasingly using tests and questionnaires to evaluate employees.
Traditional tests and questionnaires are used to get a sense of an employees abilities in comparison to other employees or applicants. These tests are normative and the outcomes depend on the whole cohort of those tested, i.e. a great score for an individual in one group of testers may become a low score when compared to another group of testers.
Psychometric testing is part of Quantitative Psychology and objectively measures, “skills and knowledge, abilities, attitudes, personality traits, and educational achievement.” Essentially, psychometric tests do not pit people against each other but instead analyze people within themselves.
Therefore, the cadence of questions and best practices for layouts are similar for traditional and psychometric testing, although the best practices for format and medium differ.
The most widely used and best format for psychometric questionnaires is to use multi-item psychometric scales. Psychometric scales use their own rating scale and that is applied to multiple questions. While a rating scale measures a participant’s response to a single question, a psychometric scale uses a single focal variable to measure participants over multiple questions or items. This format often asks testers to rate responses on a scale of, for example, 1-5, with each numeral corresponding to a degree of feeling about the answer.
There have been huge developments in the best practices for designing and administering psychometric tests in the past 30 years largely due to technological advances and changes in society. Pen and paper was the favored method before computers gained ground, but now mobile technology is becoming the preferred format for many psychometric questionnaires.
While mobile may not always be the medium used, it has become best practices to design questionnaires for mobile-first as this will guarantee the greatest consistency across mediums. According to PwC, millennials will make up 50% of the workforce by 2020 and for them mobile is the preferred device, so companies trying to cater to millennials should take that into consideration.
- Mobile first design is the best practice as it can be easily adapted to computer or converted into pen and paper as needed.
- Using mobile for testing is quicker, candidate-centric, and more inclusive (some candidates may not have access to a computer but almost everyone has a smartphone now).
- Mobile lends itself to gamification a popular way to administer psychometric questionnaires.
Whether mobile or computer based, gamification is a popular method and among the best practices used by many companies. Among the benefits of gaming are the fact that it lowers test anxiety, reduces the effects of a lack of self awareness, and it minimizes the likelihood that candidates will respond based on what they believe the employer wants them to say or do.
- CodeFight pits applicants against Company Bots to test them on coding challenges and evaluate their skill level.
- Or for example, the global marketing firm Ketchum's app LaunchPad, which gamifies the application process to measure, written, digital, creative, and communications skills.
- In the US, Deloitte created a 20-minute customized “game” that puts potential employees into real work situations. Deloitte explains, “It's highly competitive, and we feel this is not innovating for the sake of innovating, but we do feel that it's reflective of our environment. Having a boring, lengthy process that has no real connectivity to what they'll be doing at the firm is not ideal. We absolutely wanted to address that.”
Layout is how the tester interacts with the test and it is important to make sure the layout is conducive to the tester and encourages them to complete the entire test. As such, keeping questionnaires short and simple with only necessary details used to clarify complicated questions or concepts. Using images can help boost completion rates, one company, Traitify, which uses images in their testing claims a 98% completion rate.
Cadence, the manner in which the questions are presented to the tester, is another important element to consider when designing psychometric tests. Among the best practices for structuring the cadence of a psychometric questionnaire:
- Questions should be grouped by topic and format
- Use images where possible and appropriate
- Use specificity and avoid ambiguity
- Do not use jargon or specialist terminology unless you need to
- Avoid bias and leading questions
- Use positive wording as negatively worded questions can be confusing
Many companies (and almost all the top companies) use psychometric testing, and many companies also use the same test providers. The most popular assessment companies include SHL, Kenexa, Saville, Talent Q, Cut-e, and Cubiks. Many of these tests can be found online with links to examples of their tests. For example, JobTestPrep offers access to many of the psychometric tests used by top companies for a fee.
Among the companies for which you can locate sample tests are:
J P Morgan
Ernst & Young
Bank of America (Merrill Lynch)
In summary, while the cadence of questions and layout of the questionnaire are similar for both traditional and psychometric testing, best practices for format and medium differ