Cheng, Y;
Pérez-Díaz, PA;
Petrides, KV;
Li, J;
(2023)
Monte Carlo simulation with confusion matrix paradigm – A sample of internal consistency indices.
Frontiers in Psychology
, 14
, Article 1298534. 10.3389/fpsyg.2023.1298534.
Preview |
PDF
Petrides_Monte Carlo simulation with confusion matrix paradigm_VoR.pdf - Published Version Download (586kB) | Preview |
Abstract
Monte Carlo simulation is a common method of providing empirical evidence to verify statistics used in psychological studies. A representative set of conditions should be included in simulation studies. However, several recently published Monte Carlo simulation studies have not included the conditions of the null distribution of the statistic in their evaluations or comparisons of statistics and, therefore, have drawn incorrect conclusions. This present study proposes a design based on a common statistic evaluation procedure in psychology and machine learning, using a confusion matrix with four cells: true positive, true negative, false negative modified, and false positive modified. To illustrate this design, we employ an influential Monte Carlo simulation study by Trizano-Hermosilla and Alvarado (2016), which concluded that the Omega-indexed internal consistency should be preferred over other alternatives. Our results show that Omega can report an acceptable level of internal consistency (i.e., > 0.7) in a population with no relationship between every two items in some conditions, providing novel empirical evidence for comparing internal consistency indices.
Type: | Article |
---|---|
Title: | Monte Carlo simulation with confusion matrix paradigm – A sample of internal consistency indices |
Location: | Switzerland |
Open access status: | An open access version is available from UCL Discovery |
DOI: | 10.3389/fpsyg.2023.1298534 |
Publisher version: | http://dx.doi.org/10.3389/fpsyg.2023.1298534 |
Language: | English |
Additional information: | © 2023 Cheng, Pérez-Díaz, Petrides and Li. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/). |
Keywords: | Monte Carlo simulation, confusion matrix, internal consistency, null distribution, statistic comparison |
URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10185987 |
Archive Staff Only
![]() |
View Item |