Scale Development and Validation - Resources and Tools
Introduction and problem statement
A
pivotal aspect of empirical social science is the reliable and valid
measurement of the concepts of interest. Before we can study the things
that we are interested in (e.g., attitudes, emotions, opinions,
behavior…), we must define the construct of interest and build
instruments that measure it accurately. Scale development and validation
is its own large research field and much advances have been made with
regard to how we should go about developing and validating instruments
and scales. Developing a proper scale is its own research project, often
spanning several studies from initial item development through
qualitative work, cognitive pretesting and formative evaluation by
experts, pilot test, initial empirical studies to refine and reduce the
item pool, to a final confirmation of the factorial structure based on a
representative sample.
Yet, the scholarly practice often looks quite different. A lot of the published studies either a) used “existing” scales, whose validity is often unknown, or b) developed adhoc-scales which are often comparatively short and specific to the purpose of the study. On top, these “existing” or adhoc-scales are not much scrutinized with regard to their reliability and validity. A short descriptive analysis of the resulting mean indices and Cronbach’s alpha are often the only aspects that are reported. Comprehensive descriptive analyses of individual items, correlations between them, confirmatory factor analyses, as well as psychometric properties such as composite reliability (McDonald’s omega), average variance extracted (AVE) and validity analysis (incl. convergent, discriminant, and criterion validity) are mostly missing.
The goal of this repository
On this page, we aim to collect resources related to scale development and validation (incl. overall guidelines, literature, tutorials, R scripts, etc.) to improve the scholarly practices related to the reporting of measures, the development of instruments and scales, and the knowledge and awareness of challenges, problems and issues. The goal is to organize relevant information and resources in an easy-to-access type of way.
Overview
Getting Started: A first introduction to scale development and validation consisting of 6 important steps
Guidelines: A list of aspects that are important to take into account and report in a scientific study (independent of whether a scale was developed or not)
R Resources: A collection of links to relevant packages and tutorials that are useful to conduct psychometric analyses, confirmatory and exploratory factor analyses, as well as alternative approaches such as item reponse theory
Literature: A growing list of relevant articles and books on various topics related to scale development and validation
We hope this repository will help interested scholars to improve their reporting of measures and learn more about the complex task of developing proper scales and measures.
Note: The image was generated using DALL-E with the prompt “Item and Scale development and validation, 3d render, digital art”
This repository is published under the following license.