Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Requirements for development and adaptation of complex measurement tools in sociology: reliability, validity and trustworthiness

236
2
Article(UKR)(.pdf)

The article dwells upon main characteristics of sociological research quality (reliability, validity and trustworthiness) as well as general logic of their testing. Complex measurement instruments divided into scales and indexes are in the focus. Scales aim to measure certain qualities that are not explicitly observed (latent variables) and determine characteristics, which underlie the indicators of the measurement technique. When using integral indices, latent variables or social phenomenon are determined by the factors based on which indicators are formulated. Reliability is viewed as repeatability of measurement or absence of random errors.

Authors distinguish such approaches towards reliability testing as a test-retest (level of rigidity of the measurement in some time), parallel forms (level of equivalence of two forms of the same scale), and internal consistency (consistency of results of different indicators). In the most general perspective, validity is viewed as correspondence degree of the research process results to the reality. Integrated approach towards validation of measurement scales and integral indices includes two stages – theoretical and empirical validation.

Theoretical validation implies reaching the sufficient level of correspondence of theoretical construct to phenomenon under study, while empirical validation implies testing the accordance of measurement technique and results that were obtained while using it with the theoretical construct. Thus, theoretical validity is achieved first, and then empirical validity is tested. Trustworthiness of measurement is viewed as a respondent’s capability and desire to report information in which the researcher is interested. Usage of reliable and valid measurement technique does not guarantee obtaining required data when it is not possible to assure trustworthiness of measurement.

It is recommended to use index of subjective reliability to test trustworthiness. Recommendations regarding general logic of validity and reliability testing of complex measurement techniques and requirements towards design and sample size of the survey are presented in the article.

  1. DeVellis, R. (2012). Scale Development: Theory and Applications. Los Angeles; London; New Delhi; Singapore; Washington DC: SAGE

  2. Alreck, P.L., Settle, R.B. (1985). The Survey Research Handbook. Homewood, IL: Dow Jones-Irwin

  3. Carmines, E.G., Zeller, R.A. (1979). Reliability and Validity Assessment. Beverly Hills, California: SAGE Publications

  4. Paniotto, V. (1986). Quality of sociological information. Kyiv: Naukova dumka [in Russian]

  5. Trochim, W. Types of Reliability. URL: http://www.socialresearchmethods.net/kb/reltypes.php

  6. Mueller, R. (1996). Basic Principles of Structural Equation Modeling. An Introduction to LISREL and EQS. New York: Springer-Verlag New York, Inc.

  7. Johnston, J., Pennypacker, H. (1980). Strategies and tactics of human behavioral research. New Jersey: Erlbaum

  8. Dembitskii, S. (2014). To the problem of statistical verification of theoretical constructs. Sotsiologiya: teoriya, metody, marketing – Sociology: theory, methods, marketing, 2, 190-194 [in Russian]

  9. Dembitskii, S. (2016). Theoretical validation in sociological research. Moscow: LENAND [in Russian]

  10. Dembitskii, S. (2012). Theorethical validation of measurement scales. Sotsiologiya: teoriya, metody, marketing – Sociology: theory, methods, marketing, 2, 53-65 [in Russian]

  11. Yin, R. (2009). Case Study Research: Design and Methods. London; Thousand Oaks; New Delhi: Sage

  12. Tilli, Ch. (2009). Coercion, Capital, and European States, AD 990–1990. 1990–1992. Moscow: Territory of the Future [in Russian]

  13. Anderson, B. (2001). Imagined Communities: Reflections on the Origin and Spread of Nationalism. Moscow: Kanon-press-Ts: Kuchkovo pole [in Russian]

  14. Derogatis, L., Fitzpatrick, M. (2004). The SCL-90-R, the Brief Symptom Inventory (BSI), and the BSI-18. In M.E. Maruish (Ed.), The use of Psychological Testing for Treatment Planning and Outcomes Assessment. Vol. 3: Instruments for Adults (pp. 1–42). New Jersey; London: Lawrence Erlbaum Associated Publishers

  15. Golovakha, E., Panina, N. (1997). Integral index of social well-being (IISWB): design and application of socio-logical test in surveys. Kyiv: Institute of Sociology of NAS of Ukraine [in Russian]

  16. Campbell, D., Fiske, D. (1959). Convergent and Discriminant Validation by the Multitrait-multimethod Matrix. Psychological Bulletin, 2, 81-105

  17. Volovich, V. (1976). Reliability of information in a sociological research. Problems of methodology and techniques. Kyiv [in Russian]

  18. Shmelev, A. (1996). Fundamentals of Psychodiagnostics. Rostov-na-Donu: Feniks [in Russian]

  19. McNabb, D. (2014). Nonsampling error in social surveys. Los Angeles, London, New Delhi, Singapore, Washington DC: SAGE

  20. Golovakha, Ye., Gorbachyk, A., Lyubyva, T., Panіna, N., Sereda, V., Ursulenko, K. (2008). Subjective Reliability: Theory and Method of Measurement (ISR). Sotsіolohіia: teorіia, metody, marketynh – Sociology: theory, methods, marketing, 1, 166-188 [in Ukrainian]

Full text