Structural equation modeling: Difference between revisions

Revision as of 11:44, 23 June 2025

Latest revision as of 17:03, 8 August 2025

(2 intermediate revisions by 2 users not shown)

Line 17:

== History ==

Structural equation modeling (SEM) began differentiating itself from correlation and regression when [[Sewall Wright]] provided explicit causal interpretations for a set of regression-style equations based on a solid understanding of the physical and physiological mechanisms producing direct and indirect effects among his observed variables.<ref name="Wright21">{{cite journal |last1=Wright |first1=Sewall |date=1921 |title=Correlation and causation |journal=Journal of Agricultural Research |volume=20 |pages=557–585 }}</ref><ref name="Wright34">{{cite journal | doi=10.1214/aoms/1177732676 | title=The Method of Path Coefficients | date=1934 | last1=Wright | first1=Sewall | journal=The Annals of Mathematical Statistics | volume=5 | issue=3 | pages=161–215 }}</ref><ref name="Wolfle99">{{cite journal |last1=Wolfle |first1=Lee M. |title=Sewall wright on the method of path coefficients: An annotated bibliography |journal=Structural Equation Modeling: A Multidisciplinary Journal |date=January 1999 |volume=6 |issue=3 |pages=280–291 |doi=10.1080/10705519909540134 }}</ref> The equations were estimated like ordinary regression equations but the substantive context for the measured variables permitted clear causal, not merely predictive, understandings. O. D. Duncan introduced SEM to the social sciences in his 1975 book,<ref name="Duncan75">Duncan, Otis Dudley. (1975). Introduction to Structural Equation Models. New York: Academic Press. ISBN 0-12-224150-9.{{pn|date=June 2025}}</ref> and SEM blossomed in the late 1970's and 1980's when increasing computing power permitted practical model estimation. In 1987 Hayduk<ref name="Hayduk87"/> provided the first book-length introduction to structural equation modeling with latent variables, and this was soon followed by Bollen's popular text (1989).<ref name="Bollen89">Bollen, K. (1989). Structural Equations with Latent Variables. New York, Wiley. ISBN 0-471-01171-1.{{pn|date=June 2025}}</ref>

Different yet mathematically related modeling approaches developed in psychology, sociology, and economics. Early [[Cowles Foundation|Cowles Commission]] work on [[Simultaneous equations model|simultaneous equations]] estimation centered on Koopman and Hood's (1953) algorithms from [[transport economics]] and optimal routing, with [[maximum likelihood estimation]], and closed form algebraic calculations, as iterative solution search techniques were limited in the days before computers. The convergence of two of these developmental streams (factor analysis from psychology, and path analysis from sociology via Duncan) produced the current core of SEM. One of several programs Karl Jöreskog developed at Educational Testing Services, LISREL<ref name="JGvT70">Jöreskog, Karl; Gruvaeus, Gunnar T.; van Thillo, Marielle. (1970) ACOVS: A General Computer Program for Analysis of Covariance Structures. Princeton, N.J.; Educational Testing Services.{{pn|date=June 2025}}</ref><ref name=":0">{{cite journal |last1=Jőreskog |first1=Karl G. |last2=van Thiilo |first2=Marielle |title=Lisrel a General Computer Program for Estimating a Linear Structural Equation System Involving Multiple Indicators of Unmeasured Variables |journal=ETS Research Bulletin Series |date=December 1972 |volume=1972 |issue=2 |id={{ERIC|ED073122}} |doi=10.1002/j.2333-8504.1972.tb00827.x }}</ref><ref name="JS76">Jöreskog, Karl; Sorbom, Dag. (1976) LISREL III: Estimation of Linear Structural Equation Systems by Maximum Likelihood Methods. Chicago: National Educational Resources, Inc.{{pn|date=June 2025}}</ref> embedded latent variables (which psychologists knew as the latent factors from factor analysis) within path-analysis-style equations (which sociologists inherited from Wright and Duncan). The factor-structured portion of the model incorporated measurement errors which permitted measurement-error-adjustment, though not necessarily error-free estimation, of effects connecting different postulated latent variables.

Traces of the historical convergence of the factor analytic and path analytic traditions persist as the distinction between the measurement and structural portions of models; and as continuing disagreements over model testing, and whether measurement should precede or accompany structural estimates.<ref name="HG00a">{{cite journal |last1=Hayduk |first1=Leslie A. |last2=Glaser |first2=Dale N. |title=Jiving the Four-Step, Waltzing Around Factor Analysis, and Other Serious Fun |journal=Structural Equation Modeling: A Multidisciplinary Journal |date=January 2000 |volume=7 |issue=1 |pages=1–35 |doi=10.1207/s15328007sem0701_01 }}</ref><ref name="HG00b">{{cite journal |last1=Hayduk |first1=Leslie A. |last2=Glaser |first2=Dale N. |title=Doing the Four-Step, Right-2-3, Wrong-2-3: A Brief Reply to Mulaik and Millsap; Bollen; Bentler; and Herting and Costner |journal=Structural Equation Modeling: A Multidisciplinary Journal |date=January 2000 |volume=7 |issue=1 |pages=111–123 |doi=10.1207/S15328007SEM0701_06 }}</ref> Viewing factor analysis as a data-reduction technique deemphasizes testing, which contrasts with path analytic appreciation for testing postulated causal connections – where the test result might signal model misspecification. The friction between factor analytic and path analytic traditions continue to surface in the literature.

Wright's path analysis influenced Hermann Wold, Wold's student Karl Jöreskog, and Jöreskog's student Claes Fornell, but SEM never gained a large following among U.S. econometricians, possibly due to fundamental differences in modeling objectives and typical data structures. The prolonged separation of SEM's economic branch led to procedural and terminological differences, though deep mathematical and statistical connections remain.<ref name="Westland15">{{cite book |doi=10.1007/978-3-030-12508-0 |title=Structural Equation Models |series=Studies in Systems, Decision and Control |date=2019 |volume=22 |isbn=978-3-030-12507-3 }}{{pn|date=June 2025}}</ref><ref>{{cite journal |last1=Christ |first1=Carl F. |title=The Cowles Commission's Contributions to Econometrics at Chicago, 1939-1955 |journal=Journal of Economic Literature |date=1994 |volume=32 |issue=1 |pages=30–59 |jstor=2728422 }}</ref> Disciplinary differences in approaches can be seen in SEMNET discussions of endogeneity, and in discussions on causality via directed acyclic graphs (DAGs).<ref name="Pearl09"/> Discussions comparing and contrasting various SEM approaches are available<ref name="Imbens20">{{cite journal |last1=Imbens |first1=Guido W. |title=Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics |journal=Journal of Economic Literature |date=December 2020 |volume=58 |issue=4 |pages=1129–1179 |doi=10.1257/jel.20191597 }}</ref><ref name="BP13">{{cite book | doi=10.1007/978-94-007-6094-3_15 | chapter=Eight Myths About Causality and Structural Equation Models | title=Handbook of Causal Analysis for Social Research | series=Handbooks of Sociology and Social Research | date=2013 | last1=Bollen | first1=Kenneth A. | last2=Pearl | first2=Judea | pages=301–328 | isbn=978-94-007-6093-6 }}</ref> highlighting disciplinary differences in data structures and the concerns motivating economic models.

Line 63:

a) the coefficients' locations in the model (e.g. which variables are connected/disconnected),

b) the nature of the connections between the variables (covariances or effects; with effects often assumed to be linear),

c) the nature of the error or residual variables (often assumed to be independent of, or causally- disconnected from, many variables),

and d) the measurement scales appropriate for the variables (interval level measurement is often assumed).

Line 90:

Replication is unlikely to detect misspecified models which inappropriately-fit the data. If the replicate data is within random variations of the original data, the same incorrect coefficient placements that provided inappropriate-fit to the original data will likely also inappropriately-fit the replicate data. Replication helps detect issues such as data mistakes (made by different research groups), but is especially weak at detecting misspecifications after exploratory model modification – as when confirmatory factor analysis is applied to a random second-half of data following exploratory factor analysis (EFA) of first-half data.

A modification index is an estimate of how much a model's fit to the data would "improve" (but not necessarily how much the model's structure would improve) if a specific currently- fixed model coefficient were freed for estimation. Researchers confronting data-inconsistent models can easily free coefficients the modification indices report as likely to produce substantial improvements in fit. This simultaneously introduces a substantial risk of moving from a causally-wrong-and-failing model to a causally-wrong-but-fitting model because improved data-fit does not provide assurance that the freed coefficients are substantively reasonable or world matching. The original model may contain causal misspecifications such as incorrectly directed effects, or incorrect assumptions about unavailable variables, and such problems cannot be corrected by adding coefficients to the current model. Consequently, such models remain misspecified despite the closer fit provided by additional coefficients. Fitting yet worldly-inconsistent models are especially likely to arise if a researcher committed to a particular model (for example a factor model having a desired number of factors) gets an initially-failing model to fit by inserting measurement error covariances "suggested" by modification indices. MacCallum (1986) demonstrated that "even under favorable conditions, models arising from specification serchers must be viewed with caution."<ref name="MacCallum1986" /> Model misspecification may sometimes be corrected by insertion of coefficients suggested by the modification indices, but many more corrective possibilities are raised by employing a few indicators of similar-yet-importantly-different latent variables.<ref name="HL12">{{cite journal | doi=10.1186/1471-2288-12-159 | doi-access=free | title=Should researchers use single indicators, best indicators, or multiple indicators in structural equation models? | date=2012 | last1=Hayduk | first1=Leslie A. | last2=Littvay | first2=Levente | journal=BMC Medical Research Methodology | volume=12 | page=159 | pmid=23088287 | pmc=3506474 }}</ref>

"Accepting" failing models as "close enough" is also not a reasonable alternative. A cautionary instance was provided by Browne, MacCallum, Kim, Anderson, and Glaser who addressed the mathematics behind why the {{math|χ²}} test can have (though it does not always have) considerable power to detect model misspecification.<ref name="BMKAG02">{{cite journal |last1=Browne |first1=Michael W. |last2=MacCallum |first2=Robert C. |last3=Kim |first3=Cheong-Tag |last4=Andersen |first4=Barbara L. |last5=Glaser |first5=Ronald |title=When fit indices and residuals are incompatible. |journal=Psychological Methods |date=2002 |volume=7 |issue=4 |pages=403–421 |doi=10.1037/1082-989x.7.4.403 |pmid=12530701 |pmc=2435310 }}</ref> The probability accompanying a {{math|χ²}} test is the probability that the data could arise by random sampling variations if the current model, with its optimal estimates, constituted the real underlying population forces. A small {{math|χ²}} probability reports it would be unlikely for the current data to have arisen if the current model structure constituted the real population causal forces – with the remaining differences attributed to random sampling variations. Browne, McCallum, Kim, Andersen, and Glaser presented a factor model they viewed as acceptable despite the model being significantly inconsistent with their data according to {{math|χ²}}. The fallaciousness of their claim that close-fit should be treated as good enough was demonstrated by Hayduk, Pazkerka-Robinson, Cummings, Levers and Beres<ref name="HP-RCLB05">{{cite journal |doi=10.1186/1471-2288-5-1|doi-access=free |title=Structural equation model testing and the quality of natural killer cell activity measurements |date=2005 |last1=Hayduk |first1=Leslie A. |last2=Pazderka-Robinson |first2=Hannah |last3=Cummings |first3=Greta G. |last4=Levers |first4=Merry-Jo D. |last5=Beres |first5=Melanie A. |journal=BMC Medical Research Methodology |volume=5 |page=1 |pmid=15636638 |pmc=546216 }} Note the correction of .922 to .992, and the correction of .944 to .994 in the Hayduk, et al. Table 1.</ref> who demonstrated a fitting model for Browne, et al.'s own data by incorporating an experimental feature Browne, et al. overlooked. The fault was not in the math of the indices or in the over-sensitivity of {{math|χ²}} testing. The fault was in Browne, MacCallum, and the other authors forgetting, neglecting, or overlooking, that the amount of ill fit cannot be trusted to correspond to the nature, ___location, or seriousness of problems in a model's specification.<ref name="Hayduk14a">{{cite journal | doi=10.1177/0013164414527449 | title=Seeing Perfectly Fitting Factor Models That Are Causally Misspecified | date=2014 | last1=Hayduk | first1=Leslie | journal=Educational and Psychological Measurement | volume=74 | issue=6 | pages=905–926 }}</ref>

Line 194:

=== Interpretation ===

Causal interpretations of SE models are the clearest and most understandable but those interpretations will be fallacious/wrong if the model’s structure does not correspond to the world’s causal structure. Consequently, interpretation should address the overall status and structure of the model, not merely the model’s estimated coefficients. Whether a model fits the data, and/or how a model came to fit the data, are paramount for interpretation. Data fit obtained by exploring, or by following successive modification indices, does not guarantee the model is wrong but raises serious doubts because these approaches are prone to incorrectly modeling data features. For example, exploring to see how many factors are required preempts finding the data are not factor structured, especially if the factor model has been “persuaded” to fit via inclusion of measurement error covariances. Data’s ability to speak against a postulated model is progressively eroded with each unwarranted inclusion of a “modification index suggested” effect or error covariance. It becomes exceedingly difficult to recover a proper model if the initial/base model contains several misspecifications.<ref name="HC00">{{cite journal |last1=Herting |first1=Jerald R. |last2=Costner |first2=Herbert L. |title=Another Perspective on 'The Proper Number of Factors' and the Appropriate Number of Steps |journal=Structural Equation Modeling: A Multidisciplinary Journal |date=January 2000 |volume=7 |issue=1 |pages=92–110 |doi=10.1207/S15328007SEM0701_05 }}</ref>

Direct-effect estimates are interpreted in parallel to the interpretation of coefficients in regression equations but with causal commitment. Each unit increase in a causal variable’s value is viewed as producing a change of the estimated magnitude in the dependent variable’s value given control or adjustment for all the other operative/modeled causal mechanisms. Indirect effects are interpreted similarly, with the magnitude of a specific indirect effect equaling the product of the series of direct effects comprising that indirect effect. The units involved are the real scales of observed variables’ values, and the assigned scale values for latent variables. A specified/fixed 1.0 effect of a latent on a specific indicator coordinates that indicator’s scale with the latent variable’s scale. The presumption that the remainder of the model remains constant or unchanging may require discounting indirect effects that might, in the real world, be simultaneously prompted by a real unit increase. And the unit increase itself might be inconsistent with what is possible in the real world because there may be no known way to change the causal variable’s value. If a model adjusts for measurement errors, the adjustment permits interpreting latent-level effects as referring to variations in true scores.<ref name="BMvH03"/>

Line 202:

SE model interpretation should connect specific model causal segments to their variance and covariance implications. A single direct effect reports that the variance in the independent variable produces a specific amount of variation in the dependent variable’s values, but the causal details of precisely what makes this happens remains unspecified because a single effect coefficient does not contain sub-components available for integration into a structured story of how that effect arises. A more fine-grained SE model incorporating variables intervening between the cause and effect would be required to provide features constituting a story about how any one effect functions. Until such a model arrives each estimated direct effect retains a tinge of the unknown, thereby invoking the essence of a theory. A parallel essential unknownness would accompany each estimated coefficient in even the more fine-grained model, so the sense of fundamental mystery is never fully eradicated from SE models.

Even if each modeled effect is unknown beyond the identity of the variables involved and the estimated magnitude of the effect, the structures linking multiple modeled effects provide opportunities to express how things function to coordinate the observed variables – thereby providing useful interpretation possibilities. For example, a common cause contributes to the covariance or correlation between two effected variables, because if the value of the cause goes up, the values of both effects should also go up (assuming positive effects) even if we do not know the full story underlying each cause.<ref name="Duncan75"/> (A correlation is the covariance between two variables that have both been standardized to have variance 1.0). Another interpretive contribution might be made by expressing how two causal variables can both explain variance in a dependent variable, as well as how covariance between two such causes can increase or decrease explained variance in the dependent variable. That is, interpretation may involve explaining how a pattern of effects and covariances can contribute to decreasing a dependent variable’s variance.<ref name="Hayduk87p20">Hayduk, L. (1987) Structural Equation Modeling with LISREL: Essentials and Advances, page 20. Baltimore, Johns Hopkins University Press. ISBN 0-8018-3478-3 Page 20</ref> Understanding causal implications implicitly connects to understanding “controlling”, and potentially explaining why some variables, but not others, should be controlled.<ref name="Pearl09"/><ref name="HCSNGDGP-R03">{{cite journal |last1=Hayduk |first1=Leslie |last2=Cummings |first2=Greta |last3=Stratkotter |first3=Rainer |last4=Nimmo |first4=Melanie |last5=Grygoryev |first5=Kostyantyn |last6=Dosman |first6=Donna |last7=Gillespie |first7=Michael |last8=Pazderka-Robinson |first8=Hannah |last9=Boadu |first9=Kwame |title=Pearl's D-Separation: One More Step Into Causal Thinking |journal=Structural Equation Modeling: A Multidisciplinary Journal |date=April 2003 |volume=10 |issue=2 |pages=289–311 |doi=10.1207/S15328007SEM1002_8 }}</ref> As models become more complex these fundamental components can combine in non-intuitive ways, such as explaining how there can be no correlation (zero covariance) between two variables despite the variables being connected by a direct non-zero causal effect.<ref name="Duncan75"/><ref name="Bollen89"/><ref name="Hayduk87"/><ref name="Hayduk96"/>

The statistical insignificance of an effect estimate indicates the estimate could rather easily arise as a random sampling variation around a null/zero effect, so interpreting the estimate as a real effect becomes equivocal. As in regression, the proportion of each dependent variable’s variance explained by variations in the modeled causes are provided by ''R''², though the Blocked-Error ''R''² should be used if the dependent variable is involved in reciprocal or looped effects, or if it has an error variable correlated with any predictor’s error variable.<ref name="Hayduk06">{{cite journal |last1=Hayduk |first1=Leslie A. |title=Blocked-Error-R 2: A Conceptually Improved Definition of the Proportion of Explained Variance in Models Containing Loops or Correlated Residuals |journal=Quality & Quantity |date=August 2006 |volume=40 |issue=4 |pages=629–649 |doi=10.1007/s11135-005-1095-4 }}</ref>

Line 246:

* [[Measurement invariance]] models <ref>{{cite journal |last1=Leitgöb |first1=Heinz |last2=Seddig |first2=Daniel |last3=Asparouhov |first3=Tihomir |last4=Behr |first4=Dorothée |last5=Davidov |first5=Eldad |last6=De Roover |first6=Kim |last7=Jak |first7=Suzanne |last8=Meitinger |first8=Katharina |last9=Menold |first9=Natalja |last10=Muthén |first10=Bengt |last11=Rudnev |first11=Maksim |last12=Schmidt |first12=Peter |last13=van de Schoot |first13=Rens |title=Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives |journal=Social Science Research |date=February 2023 |volume=110 |pages=102805 |doi=10.1016/j.ssresearch.2022.102805 |pmid=36796989 |hdl=1874/431763 |hdl-access=free }}</ref>

* [[Mixture model]] {{citation needed|date=July 2023}}

* [[Multilevel models]], hierarchical models (e.g. people nested in groups) <ref>{{Citation |last1=Sadikaj |first1=Gentiana |title=Multilevel structural equation modeling for intensive longitudinal data: A practical guide for personality researchers |date=2021 |url=https://linkinghub.elsevier.com/retrieve/pii/B9780128139950000339 |work=The Handbook of Personality Dynamics and Processes |pages=855–885 |access-date=2023-11-03 |publisher=Elsevier |language=en |doi=10.1016/b978-0-12-813995-0.00033-9 |isbn=978-0-12-813995-0 |last2=Wright |first2=Aidan G.C. |last3=Dunkley |first3=David M. |last4=Zuroff |first4=David C. |last5=Moskowitz |first5=D.S.|url-access=subscription }}</ref>

* Multiple group modelling with or without constraints between groups (genders, cultures, test forms, languages, etc.) {{citation needed|date=July 2023}}

* Multi-method multi-trait models {{citation needed|date=July 2023}}

* Random intercepts models {{citation needed|date=July 2023}}

* Structural Equation Model Trees {{citation needed|date=July 2023}}

* Structural Equation [[Multidimensional scaling]]<ref>{{cite journal |last1=Vera |first1=José Fernando |last2=Mair |first2=Patrick |title=SEMDS: An R Package for Structural Equation Multidimensional Scaling |journal=Structural Equation Modeling: A Multidisciplinary Journal |date=3 September 2019 |volume=26 |issue=5 |pages=803–818 |doi=10.1080/10705511.2018.1561292 }}</ref>

== Software ==

Line 401:

<!--<ref name="Boslaugh2008">{{cite book |doi=10.4135/9781412953948.n443 |chapter=Structural Equation Modeling |title=Encyclopedia of Epidemiology |year=2008 |isbn=978-1-4129-2816-8 |last1=Boslaugh |first1=Sarah |last2=McNutt |first2=Louise-Anne |hdl=2022/21973 }}</ref> -->

<ref name="Ing2024">{{cite report |type=Preprint |doi=10.1101/2024.06.13.598616 |title=Integrating Multi-Modal Cancer Data Using Deep Latent Variable Path Modelling |date=2024 |last1=Ing |first1=Alex |last2=Andrades |first2=Alvaro |last3=Cosenza |first3=Marco Raffaele |last4=Korbel |first4=Jan O. |doi-access=free }}</ref>

}}