Let’s start out by telling you what a psychometric test is not!
You may have come across various different online tests. Not all of them are psychometric. Whether they are or not will depend on how they were designed, for what purpose and what they intend to measure. For example, you may have come across the Thematic Apperception Test or the Rorshach Inkblot Test. These tests are used mainly in clinical settings. The client is asked to look at scenes or pictures and to articulate what they see. Such tests are supposed to be able to assess the unconscious mind. However, interpretation is not as objective as we would like to see in occupational testing and assessment. Furthermore, what the client “sees” may be based on external factors such as culture, upbringing or season of the year.
Psychometric Tests are different! They are defined as quantitative (numerical) assessments of one or more psychological (in the head) attributes. So, psychometric tests are used to assess in a quantitative way things like numerical reasoning skills, verbal aptitude, extroversion, conscientiousness and so on. That alone does not make a test psychometric of course. In order to be psychometric the test must have been designed to be:
1. Administered in a standardised manner
2. Scored in a standardised manner
3. Interpreted in a standardised manner
4. Constructed according to psychometric principles
You’ll see later how important standardisation is throughout the use of psychometric tools. In fact standardisation is a critical element of all scientific HR processes.
As for being constructed according the psychometric principles, what we mean here is that the test must have gone through each aspect of a very scientific development process. It is not good enough to simply write down a few questions, produce a flashy report and start selling a new test! This is the process it must go through:
a. Development of a rationale behind the test that is supported by research. For example – if I decide to develop a new test of extroversion, the scientific research literature and models must inform the design of my questions and the aspects of extroversion that I attempt to assess.
b. Next I need to write some experimental questions. At this stage I’ll write more questions than I intend to have in the final version of my test because I am prepared to throw some out based on feedback.
c. Now I’ll find a sample of people (who represent the group I am designing the test for) and I’ll ask this sample to complete my experimental test. In other words, I pilot my test.
d. Next I head back to my office and assess the responses and how they relate to each other statistically. At this stage I am running item analysis to test that similar questions (items) are indeed related to each other in the way I would expect. More on this later. The point to grasp now is that this is an iterative process. Things won’t be perfect the first time around. Based on the statistics I’ll need to remove some questions, refine others and then go back to point C. I’ll do this again and again until I am happy with the statistics I get at point D!
e. Now that my test questions are performing well I need to enter a standardisation phase. Here, a larger group of people will complete my test and that will show me where people tend to score on the test. This group will become my benchmark or norm group later and add meaning to the scores of future test-takers.
f. The next question is “Does my new test actually assess what it is supposed to assess and/or does it predict something meaningful?”. As an example, you would expect my numerical reasoning test to predict success of accountant trainees. This stage is called Validation, we are assessing the validity of the test or whether it is fit for purpose.
g. Now, we all know from science classes at school that all good experiments end with a write-up! That’s exactly what we do at the final stage of psychometric test development. We need to write up all of the above stages in a long document which is called the test’s technical manual. It is this manual that prospective clients with reputable training in psychometrics will consult before purchasing a psychometric test. So, if your test publisher tells you they don’t have such a document, it might be wise to stay away from them. On the other hand – do expect to pay for the manual. Some publishers will offer them free of charge but others will require a fee.
Based on the above, hopefully you can see that developing tests well takes time and effort. It is for this reason that good tests are usually not cheap! Not only that. The test is not static. People change, norms change and so validities may even change. The publisher cannot put the test on the shelf and forget as if it were a book they wrote years ago. This is why usually clients will pay a fee per test report or per candidate whenever they use the test. If you are attracted by free or very cheap internet-based tests you could be making a costly mistake. Particularly if you plan using the test in candidate selection for your business.
