At TTI SI, we’re highly focused on research. We believe that research is our best way to keep our assessments up to date and competitive in the industry, but we also know that it can be difficult to understand some of the concepts that are crucial to utilizing research to it’s highest degree.
Here are three misconceptions about research and what you need to know instead.
What does population refer to?
In some of the assessments at TTI SI, we share information that relies on using a general population, like the means of data and national norms.
However, it’s important for you to know that when TTI SI refers to a population, we’re not talking about the entire population of the country in question. Rather, we are referring to the population of those people who have taken the assessment in that country.
“This distinction is crucially important,” said Dr. Eric Gehrig, Vice President of Research & Development. “We don’t have access to the kind of data needed to say we have the results of an entire population, but we do have an extensive, live data base with the results of people who have used our assessments.”
Use this information while explaining the process to clients or to simply broaden your own understanding!
Are reliability and validity the same thing?
Reliability and validity are both incredibly important concepts to develop a working knowledge for, but they are not the same thing.
A great basic example is a thermometer. Let’s say you are taking your temperature multiple times throughout a day. If you get the same temperature every time, that means you are getting a reliable reading.
However, the thermometer might be malfunctioning in some way, giving you an inaccurate reading. If the thermometer is broken, then your reading is not valid— you might still have a temperature, but you continually get that reliable reading that says otherwise.
These concepts are important for assessments because they affect the impact and understanding of results. If you’re using an assessment provider that says they’re assessments are reliable, that just means that they give consistent results. It doesn’t mean that they are accurate!
The internal consistency argument is that people respond to TTI SI assessments very consistently. The other side that we work to maintain is to make sure that our assessments are both reliable and valid. Validity and reliability ensure that the results are both consistent and accurate.
What is test-retest reliability?
Test-retest reliability is the idea that there should be close or similar results in an assessment that is re-taken in the time period of two months but not more than 13 or 14 months after the original test date.
“When we’re talking about this type of reliability, we’re not talking about an individual,” said Dr. Gehrig. “We’re talking about an entire population. What research cares about is the changes of a group over time.”
By looking at the test-retest reliability of a group instead of just one person, it shows that an assessment is consistently ranking a population. Research isn’t seeking out individual scores— it’s less about individuals being exactly the same each time. Instead, it’s about the group. The high correlation between the two groups of scores indicates that the assessment has temporal consistency.
This is important for you to know because assessment results do change after significant emotional events.To ensure that you are getting the most accurate results, make sure your team is not taking assessments too frequently but is retesting after major life events.
It makes sense to have questions about research, and TTI SI is here to help you get the answers you need! If you’re interested in learning more, check out these additional resources.
