What is Bifactor Analysis?

Bifactor analysis is a versatile tool that allows researchers to answer a host of crucial questions about the social scientific instruments they use in their research.

On November 1, 2016 at the University of Kentucky, I presented a 50-minute talk with Dr. Michael Toland on Bifactor Analysis in Mplus.  We walked the audience through what a bifactor model is, what follow-up ancillary bifactor measures are, what questions can be answered by using these techniques, how to run everything using Mplus, how to interpret results from these techniques, and what conclusions you can draw from these results.

We used a real life example from our 2016 Stigma and Health paper on the bifactor analysis of the Internalized Stigma of Mental Illness Scale.  You can download the PowerPoint Slideshow as well as sample data and Mplus syntax we used for this talk, so that you can replicate our results.  You are encouraged to adapt this syntax for your own confirmatory bifactor analyses and ancillary bifactor measure calculations, but please cite us:

Hammer, J. H., & Toland, M. D. (2016, November).  Bifactor analysis in Mplus. [Video file]. Retrieved from http://sites.education.uky.edu/apslab/upcoming-events/

Here’s the video of our talk, which included Q&A at the end:

Bifactor-Related Calculators

Here are some useful bifactor-related calculators and resources.  I discuss the PUC and ARPB Calculators in my Bifactor Analysis in Mplus talk.

Bifactor Indices Calculator – This comprehensive Microsoft Excel-based calculator can be used to calculate IECV, Relative Parameter Bias, Absolute Relative Parameter Bias, IECV, ECV, Omega, OmegaH, OmegaHS, Relative Omega (PRV), and H, as well as model-level PUC, ECV, and Average Relative Parameter Bias.  I recommend double-checking results from this calculator with results calculated via Mplus syntax and/or the specific calculators below.  The creator of the calculator, David Dueber, provides a helpful 8-minute tutorial video on how to use the calculator.

Construct Replicability (H index) Calculator – This Microsoft Excel-based calculator can be used to calculate the H index.  Construct Replicability, as measured by the H index of Hancock and Mueller (2001),  is a statistical approach to evaluate how well a set of items represent a latent variable.  It provides information on whether the SEM measurement model is suitable and replicable across studies. High H values (>.80) suggest a well-defined latent variable, which is more likely to be stable across studies.  (Description adapted from p. 230; Rodriguez et al., 2016).

Percent of Uncontaminated Correlations (PUC) Calculator – This Microsoft Excel-based calculator can be used to calculate the PUC statistic.  “A concern researchers often have pertains to the biasing effects of forcing a unidimensional model to multidimensional data. When this is at issue, important diagnostic information can be derived from examining both ECV and PUC. Reise et al., (2013) and Bonifay et al. (2015) demonstrated that parameter bias is directly related to ECV, which, in turn, is moderated by the PUC…  Simply stated, the research cited earlier indicates that as PUC increases, the magnitude of the ECV value becomes less and less important in determining the potential for bias when a unidimensional model is fit to multidimensional data.” (from p. 231-232; Rodriguez et al., 2016).  When Percent of Uncontaminated Correlations (PUC) values are higher than .80, general ECV values are less important in predicting bias; when PUC values are lower than .80, general ECV values greater than .60 and ωH > .70 suggest that the presence of some multidimensionality is not severe enough to disqualify the interpretation of the instrument as primarily unidimensional (p. 22, Reise, Scheines, Widaman, & Haviland, 2013).  Rodriguez et al., (2016) says on page 232 that “when ECV is > .70 and PUC > .70, relative bias will be slight, and the common variance can be regarded as essentially unidimensional”.

Average Relative Parameter Bias (ARPB) Calculator – This Microsoft Excel-based calculator can be used to calculate the ARPB statistic. Rodriguez, Reise, & Haviland (2016) say the following about the ARPB: “We then computed the relative parameter bias as the difference between an item’s loading in the unidimensional solution and its general factor loading in the bifactor (i.e., the truer model), divided by the general factor loading in the bifactor. We found that the average relative bias across items was 2%. According to Muthén, Kaplan, and Hollis (1987), parameter bias less than 10-15% is acceptable and poses no serious concern.” (p. 145).

Key Bifactor References

Bonifay, W., Lane, S. P., & Reise, S. P. (2017). Three concerns with applying a bifactor model as a structure of psychopathology. Clinical Psychological Science5(1), 184-186. 10.1177/2167702616657069

Gignac, G. E. (2016). The higher-order model imposes a proportionality constraint: That is why the bifactor model tends to fit better. Intelligence55, 57-68. 10.1016/j.intell.2016.01.006

Howard, J. L., Gagné, M., Morin, A. J., & Forest, J. (2016). Using bifactor exploratory structural equation modeling to test for a continuum structure of motivation. Journal of Management. 10.1177/0149206316645653

Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696. doi: 10.1080/00273171.2012.715555

Reise, S. P., Moore, T. N., & Haviland, M. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544–559. doi: 10.1080/00223891.2010.496477

Reise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective. Educational and Psychological Measurement, 73(1), 5–26. doi: 10.1177/0013164412449831

Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95, 129-140. doi: 10.1080/00223891.2012.725437

Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016a). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98, 223-237. doi: 10.1080/00223891.2015.1089249

Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016b). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21, 137-150. doi: 10.1037/met0000045

Stucky, B. D., Edelen, M. O., Vaughan, C. A., Tucker, J. S., & Butler, J. (2014). The Psychometric Development and Initial Validation of the DCI-A Short Form for Adolescent Therapeutic Community Treatment Process. Journal of Substance Abuse Treatment46, 516–521. doi: 10.1016/j.jsat.2013.12.005

Stucky, B. D., & Edelen, M. O. (2014). Using hierarchical IRT models to create unidimensional measures from multidimensional data. In S. P. Reise & D. A. Revicki (Eds.), Handbook of item response theory modeling: Applications to typical performance assessment (183-206). London, UK: Taylor & Francis. doi: 10.4324/9781315736013.ch9

Toland, M. D., Sulis, I., Giambona, F., Porcu, M., & Campbell, J. M. (2017). Introduction to bifactor polytomous item response theory analysis. Journal of school psychology60, 41-63. doi: 10.1016/j.jsp.2016.11.001

Yang, R., Spirtes, P., Scheines, R., Reise, S. P., & Mansoff, M. (2017). Finding Pure Submodels for Improved Differentiation of Bifactor and Second-Order Models. Structural equation modeling: A multidisciplinary journal24(3), 402-413. doi: 10.1080/10705511.2016.1261351

Example Papers that Apply Bifactor Analysis

Hammer, J. H., Heath, P. J., & Vogel, D. L. (in press). Fate of the total score: Dimensionality of the Conformity to Masculine Role Norms Inventory (CMNI-46). Psychology of Men & Masculinity. DOI forthcoming. *PDF* >Video Explanation<

Hammer, J. H., & McDermott, R. C., Levant, R. F., & McKelvey, D. K. (in press). Dimensionality, Reliability, and Validity of the Gender Role Conflict Scale – Short Form (GRCS-SF). Psychology of Men & Masculinity. Advance online publication. doi: 10.1037/men0000131g *PDF*

Hammer, J. H., & Lazar, A. (in press). Internal Structure and criterion relationships for long and brief versions of the Intratextual Fundamentalism Scale (IFS) among Israeli Jews. Psychology of Religion and Spirituality. DOI forthcoming *PDF*

Hammer, J. H., & Brenner, R. E. (2017). Disentangling gratitude: A theoretical and psychometric examination of the Gratitude Resentment and Appreciation Test- Revised Short (GRAT-RS). Journal of Personality Assessment, 14, 1-10. doi: 10.1080/00223891.2017.1344986 *PDF*

McDermott, R., Levant, R. F., Hammer, J. H., Hall, R., McKelvey, D., & Jones, Z. (2017). Further examination of the factor structure of the Male Role Norms Inventory-Short Form (MRNI-SF): Measurement considerations for women, men of color, and gay men. Journal of Counseling Psychology, 64, 724-738. doi: 10.1037/cou0000225 *PDF*

Hammer, J. H., & Toland, M. D. (2017). Internal structure and reliability of the Internalized Stigma of Mental Illness Scale (ISMI-29) and brief versions (ISMI-10, ISMI-9) among Americans with depression. Stigma and Health, 2, 159-174. doi: 10.1037/sah0000049 *PDF*

Hammer, J. H., & Vogel, D. V. (2017). Development of the Help-Seeker Stereotype Scale. Stigma and Health, 2, 121-136. doi: 10.1037/sah0000048 *PDF*

Brewster, M. E., Hammer, J. H., Sawyer, J., Eklund, A., & Palamar, J. (2016). Perceived experiences of atheist discrimination: Instrument development and evaluation. Journal of Counseling Psychology, 63, 557-570. doi: 10.1037/cou0000156 *PDF*

Here’s a video of an invited talk I gave on applying bifactor analysis to real world contexts to facilitate more accurate measurement:

Bifactor Consultation services

Visit my consultation page to learn more about the consultation services I provide related to bifactor analysis and scale development.