A solid understanding of statistics, I believe, is crucial for researchers to master. Having a good grasp of statistics will lead us to choose the appropriate statistical methods in research.
Using the correct statistical methods will bring us closer to drawing research conclusions that represent the real conditions in the field. One of the questions that I am frequently asked is about the use of non-parametric variables measured using a Likert scale.
They ask, why and how are variables measured using a Likert scale need to undergo validity and reliability testing before the questionnaire is used in the field? This has prompted me to write an article on why variables using a Likert scale need to be followed up with validity and reliability testing of the questionnaire.
Measurement of variables using Likert scale
Currently, it’s quite common for researchers to measure non-parametric variables using the Likert scale. Several variables measured using ordinal scales are commonly measured using the Likert scale.
In principle, variables measured using the Likert scale undergo categorization where there are levels or degrees within it. This is exactly the same as the principle of an ordinal scale. There are several variations of the Likert scale commonly used by researchers, but generally, the most commonly used is the scale of 1 – 5.
To facilitate understanding of the measurement of ordinal scale variables using the Likert scale, let’s take an example of a researcher measuring the relationship between employee competency and company performance. Both variables are non-parametric variables measured using an ordinal scale.
Why do we call them non-parametric variables? This is because to measure competency and performance variables, researchers cannot directly obtain numerical data. Therefore, the researcher measures them using non-parametric variables, namely using an ordinal scale. To facilitate the measurement of competency and performance variables, the researcher then uses the Likert scale approach.
The researcher will create a number of statement items representing both competency and performance variables. The researcher can elaborate on as many statement items as possible that can represent the measurement of competency variables and also the measurement of performance variables.
Next, the researcher can select statement items that have been created, based on theory or previous research that can be considered to represent the measured variables.
To test each of these items, the researcher creates answer choices using the Likert scale, for example, using a scale of 1 to 5. Based on this scale, answer choices can be made such as “Strongly Disagree, Disagree, Neutral, Agree, and Strongly Agree.”
For each measured variable, statement items representing the measured variable can be created. The number of statement items can be 10 items, 15 items, or others for each measured variable.
Next, to ensure that the questionnaire or instrument is valid and reliable, the researcher needs to conduct validity and reliability tests before the questionnaire is used to collect sample data in the field.
Sampling for validity and reliability testing
In testing the validity and reliability of a questionnaire, researchers need to collect sample data that will be used for these tests. The sample of respondents used for validity and reliability testing is not the main sample used in the study.
Researchers should preferably take samples outside of the main research sample. The sample used for validity and reliability testing should have characteristics that are almost similar to the population observed in the study.
For example, a researcher who intends to observe a research sample of 150 respondents may try to select 15 individuals for the purpose of testing the validity and reliability of the questionnaire.
Many also ask, why not simultaneously take the validity and reliability test samples along with the main sample of 150 respondents mentioned earlier? Because this is a trial of the questionnaire, where it cannot be certain whether the questionnaire is truly valid and reliable.
In reality, not all statement items representing the measured variables, such as competency and performance variables, are valid and reliable. Sometimes researchers have to remove several statement items that are not valid and reliable.
Imagine if we directly collected data from 150 main respondents, and it turns out that the questionnaire used is not valid and reliable. This would require researchers to work twice: correcting the questionnaire and then collecting data again using a valid and reliable questionnaire.
However, if researchers have previously conducted validity and reliability tests using only 15 samples, this will certainly save time, costs, and energy. Subsequently, researchers can go into the field to collect data from the 150 respondents using a questionnaire or instrument that has been tested for its validity and reliability.
So from here, we can conclude that validity and reliability testing needs to be done before researchers go into the field to collect data.
Testing validity and reliability
After successfully collecting data for the purpose of validity and reliability testing, the next step for the researcher is to input the data and tabulate it. The researcher needs to convert the scores from each respondent’s answer choices.
For example, from measuring variables using a Likert scale ranging from 1 to 5, the scores can be converted as follows:
Strongly Agree = Score 5
Agree = Score 4
Neutral = Score 3
Disagree = Score 2
Strongly Disagree = Score 1
Based on the score conversion above, the researcher only needs to change each respondent’s answer choices to scores for each statement item. The next step is for the researcher to conduct testing on each statement item using validity and reliability tests.
Validity testing can use Pearson correlation, while reliability testing can use Cronbach’s alpha test. For a tutorial on how to conduct and interpret both validity and reliability tests, I have created a video tutorial on the Kanda Data YouTube channel.
The Importance of Validity and Reliability Testing in Questionnaires
In the research world, the validity and reliability of questionnaires play a crucial role. When measuring non-parametric variables using ordinal scales such as the Likert scale, it is important to test the quality of the questionnaire before using it in research.
Questionnaire validity reflects how well the measuring tool can measure what it is supposed to measure. Meanwhile, questionnaire reliability indicates how consistent the measuring tool is in measuring the same phenomenon over time. In other words, validity concerns accuracy, while reliability relates to consistency.
Through validity and reliability testing, researchers can ensure that the instruments used are trustworthy and provide accurate results. This is important to avoid bias in data collection and ensure that research findings can be justified.
Although definitions and testing methods may vary, the ultimate goal of validity and reliability testing remains the same: to ensure that the questionnaire can accurately measure variables and can be relied upon.
Therefore, for researchers using the Likert scale to measure non-parametric variables, validity and reliability testing of the questionnaire are crucial steps before proceeding with further research.
That concludes a brief discussion on the importance of validity and reliability testing of questionnaires in research. Hopefully, this article helps broaden your understanding in designing quality research.
Hence, it is reiterated that to ensure measurement results can represent the measured variables, researchers need to conduct validity and reliability testing. For non-parametric variables using ordinal scales and measured using the Likert scale, researchers should conduct questionnaire validity and reliability testing.
That’s the article I can write at this opportunity, hopefully providing added value of knowledge and sparking discussions in the comment section below. See you in another article in the next opportunity.
