Supertype of all objects that can serve as the observed field of a SEM. Pre-processes data and computes sufficient statistics for example. If you have a special kind of data, e.g. ordinal data, you should implement a subtype of SemObserved.

For observed data without missings.

Constructor

SemObservedData(;
    data,
    observed_vars = nothing,
    specification = nothing,
    kwargs...)

Arguments

• data: observed data – DataFrame or Matrix
• observed_vars::Vector{Symbol}: column names of the data (if the object passed as data does not have column names, i.e. is not a data frame)
• specification: optional SEM specification (SemSpecification)

Extended help

Interfaces

• nsamples(::SemObservedData) -> number of observed data points

• nobserved_vars(::SemObservedData) -> number of observed (manifested) variables

• samples(::SemObservedData) -> observed data

• obs_cov(::SemObservedData) -> observed covariance matrix

• obs_mean(::SemObservedData) -> observed mean vector

Type alias for SemObservedData that has mean and covariance, but no actual data.

For instances of SemObservedCovariance samples returns nothing.

For observed data with missing values.

Constructor

SemObservedMissing(;
    data,
    observed_vars = nothing,
    specification = nothing,
    kwargs...)

Arguments

• data: observed data
• observed_vars::Vector{Symbol}: column names of the data (if the object passed as data does not have column names, i.e. is not a data frame)
• specification: optional SEM model specification (SemSpecification)

Extended help

Interfaces

• nsamples(::SemObservedMissing) -> number of samples (data points)

• nobserved_vars(::SemObservedMissing) -> number of observed variables

• samples(::SemObservedMissing) -> data matrix (contains both measured and missing values)

Expectation maximization

em_mvn!(::SemObservedMissing) can be called to fit a covariance matrix and mean vector to the data using an expectation maximization (EM) algorithm under the assumption of multivariate normality. After, the following methods are available:

• em_model(::SemObservedMissing) -> EmMVNModel that contains the covariance matrix and mean vector found via EM
• obs_cov(::SemObservedData) -> EM covariance matrix
• obs_mean(::SemObservedData) -> EM mean vector

samples(observed::SemObservedData)

Gets the matrix of observed data samples. Rows are samples, columns are observed variables.

See Also

nsamples, observed_vars.

observed_vars(semobj) -> Vector{Symbol}

Return the vector of SEM model observed variable in the order specified by the model, which also should match the order of variables in SemObserved.

Base type for all SEM specifications.

Supertype of all objects that can serve as the implied field of a SEM. Computes model-implied values that should be compared with the observed data to find parameter estimates, e. g. the model implied covariance or mean. If you would like to implement a different notation, e.g. LISREL, you should implement a subtype of SemImplied.

Model implied covariance and means via RAM notation.

Constructor

RAM(;specification,
    meanstructure = false,
    gradient = true,
    kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object
• meanstructure::Bool: does the model have a meanstructure?
• gradient::Bool: is gradient-based optimization used

Extended help

RAM notation

The model implied covariance matrix is computed as

\[ \Sigma = F(I-A)^{-1}S(I-A)^{-T}F^T\]

and for models with a meanstructure, the model implied means are computed as

\[ \mu = F(I-A)^{-1}M\]

Interfaces

• param_labels(::RAM)-> vector of parameter labels

• nparams(::RAM) -> number of parameters

• ram.Σ -> model implied covariance matrix

• ram.μ -> model implied mean vector

RAM matrices for the current parameter values:

• ram.A
• ram.S
• ram.F
• ram.M

Jacobians of RAM matrices w.r.t to the parameter vector θ

• ram.∇A -> $∂vec(A)/∂θᵀ$
• ram.∇S -> $∂vec(S)/∂θᵀ$
• ram.∇M = $∂M/∂θᵀ$

Vector of indices of each parameter in the respective RAM matrix:

• ram.A_indices
• ram.S_indices
• ram.M_indices

Additional interfaces

• ram.F⨉I_A⁻¹ -> $F(I-A)^{-1}$
• ram.F⨉I_A⁻¹S -> $F(I-A)^{-1}S$
• ram.I_A -> $I-A$

Only available in gradient! calls:

• ram.I_A⁻¹ -> $(I-A)^{-1}$

Subtype of SemImplied that implements the RAM notation with symbolic precomputation.

Constructor

RAMSymbolic(;
    specification,
    vech = false,
    gradient = true,
    hessian = false,
    approximate_hessian = false,
    meanstructure = false,
    kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object
• meanstructure::Bool: does the model have a meanstructure?
• gradient::Bool: is gradient-based optimization used
• hessian::Bool: is hessian-based optimization used
• approximate_hessian::Bool: for hessian based optimization: should the hessian be approximated
• vech::Bool: should the half-vectorization of Σ be computed (instead of the full matrix) (automatically set to true if any of the loss functions is SemWLS)

Extended help

Interfaces

• param_labels(::RAMSymbolic)-> vector of parameter ids

• nparams(::RAMSymbolic) -> number of parameters

• ram.Σ -> model implied covariance matrix

• ram.μ -> model implied mean vector

Jacobians (only available in gradient! calls)

• ram.∇Σ -> $∂vec(Σ)/∂θᵀ$

• ram.∇μ -> $∂μ/∂θᵀ$

• ∇Σ_eval!(::RAMSymbolic) -> function to evaluate ∇Σ in place, i.e. ∇Σ_eval!(∇Σ, θ). Typically, you do not want to use this but simply query ram.∇Σ.

Hessians The computation of hessians is more involved. Therefore, we desribe it in the online documentation, and the respective interfaces are omitted here.

RAM notation

The model implied covariance matrix is computed as

\[ \Sigma = F(I-A)^{-1}S(I-A)^{-T}F^T\]

and for models with a meanstructure, the model implied means are computed as

\[ \mu = F(I-A)^{-1}M\]

Empty placeholder for models that don't need an implied part. (For example, models that only regularize parameters.)

Constructor

ImpliedEmpty(;specification, kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object

Examples

A multigroup model with ridge regularization could be specified as a SemEnsemble with one model per group and an additional model with ImpliedEmpty and SemRidge for the regularization part.

Extended help

Interfaces

• param_labels(::ImpliedEmpty)-> Vector of parameter labels
• nparams(::ImpliedEmpty) -> Number of parameters

SemLoss(args...; loss_weights = nothing, ...)

Constructs the loss field of a SEM. Can contain multiple SemLossFunctions, the model is optimized over their sum. See also SemLossFunction.

Arguments

• args...: Multiple SemLossFunctions.
• loss_weights::Vector: Weights for each loss function. Defaults to unweighted optimization.

Examples

my_ml_loss = SemML(...)
my_ridge_loss = SemRidge(...)
my_loss = SemLoss(SemML, SemRidge; loss_weights = [1.0, 2.0])

Supertype for all loss functions of SEMs. If you want to implement a custom loss function, it should be a subtype of SemLossFunction.

Maximum likelihood estimation.

Constructor

SemML(;observed, meanstructure = false, approximate_hessian = false, kwargs...)

Arguments

• observed::SemObserved: the observed part of the model
• meanstructure::Bool: does the model have a meanstructure?
• approximate_hessian::Bool: if hessian-based optimization is used, should the hessian be swapped for an approximation

Examples

my_ml = SemML(observed = my_observed)

Interfaces

Analytic gradients are available, and for models without a meanstructure and RAMSymbolic implied type, also analytic hessians.

Full information maximum likelihood estimation. Can handle observed data with missings.

Constructor

SemFIML(;observed, specification, kwargs...)

Arguments

• observed::SemObservedMissing: the observed part of the model
• specification: either a RAMMatrices or ParameterTable object

Examples

my_fiml = SemFIML(observed = my_observed, specification = my_parameter_table)

Interfaces

Analytic gradients are available.

Weighted least squares estimation. At the moment only available with the RAMSymbolic implied type.

Constructor

SemWLS(;
    observed,
    meanstructure = false,
    wls_weight_matrix = nothing,
    wls_weight_matrix_mean = nothing,
    approximate_hessian = false,
    kwargs...)

Arguments

• observed: the SemObserved part of the model
• meanstructure::Bool: does the model have a meanstructure?
• approximate_hessian::Bool: should the hessian be swapped for an approximation
• wls_weight_matrix: the weight matrix for weighted least squares. Defaults to GLS estimation ($0.5*(D^T*kron(S,S)*D)$ where D is the duplication matrix and S is the inverse of the observed covariance matrix)
• wls_weight_matrix_mean: the weight matrix for the mean part of weighted least squares. Defaults to GLS estimation (the inverse of the observed covariance matrix)

Examples

my_wls = SemWLS(observed = my_observed)

Interfaces

Analytic gradients are available, and for models without a meanstructure also analytic hessians.

Ridge regularization.

Constructor

SemRidge(;α_ridge, which_ridge, nparams, parameter_type = Float64, implied = nothing, kwargs...)

Arguments

• α_ridge: hyperparameter for penalty term
• which_ridge::Vector: Vector of parameter labels (Symbols) or indices that indicate which parameters should be regularized.
• nparams::Int: number of parameters of the model
• implied::SemImplied: implied part of the model
• parameter_type: type of the parameters

Examples

my_ridge = SemRidge(;α_ridge = 0.02, which_ridge = [:λ₁, :λ₂, :ω₂₃], nparams = 30, implied = my_implied)

Interfaces

Analytic gradients and hessians are available.

Constant loss term. Can be used for comparability to other packages.

Constructor

SemConstant(;constant_loss, kwargs...)

Arguments

• constant_loss::Number: constant to add to the objective

Examples

    my_constant = SemConstant(constant_loss = 42.0)

Interfaces

Analytic gradients and hessians are available.

SemOptimizer(args...; engine::Symbol = :Optim, kwargs...)

Constructs a SemOptimizer object that can be passed to fit for specifying aspects of the numerical optimization involved in fitting a SEM.

The keyword engine controlls which Julia package is used, with :Optim being the default.

• optimizer_engines() prints a list of currently available engines.
• optimizer_engine_doc(EngineName) prints information on the usage of a specific engine.

More engines become available if specific packages are loaded, for example NLopt.jl (also see Constrained optimization in the online documentation) or ProximalAlgorithms.jl (also see Regularization in the online documentation).

The additional arguments args... and kwargs... are engine-specific and control further aspects of the optimization process, such as the algorithm, convergence criteria or constraints. Information on those can be accessed with optimizer_engine_doc.

To connect the SEM package to a completely new optimization backend, you can implement a new subtype of SemOptimizer.

Uses NLopt.jl as the optimization engine. For more information on the available algorithms and options, see the NLopt.jl package and the NLopt docs.

Constructor

SemOptimizer(;
    engine = :NLopt,
    algorithm = :LD_LBFGS,
    options = Dict{Symbol, Any}(),
    local_algorithm = nothing,
    local_options = Dict{Symbol, Any}(),
    equality_constraints = nothing,
    inequality_constraints = nothing,
    constraint_tol::Number = 0.0,
    kwargs...)

Arguments

• algorithm: optimization algorithm.
• options::Dict{Symbol, Any}: options for the optimization algorithm
• local_algorithm: local optimization algorithm
• local_options::Dict{Symbol, Any}: options for the local optimization algorithm
• `equality_constraints: optional equality constraints
• `inequality_constraints:: optional inequality constraints
• constraint_tol::Number: default tolerance for constraints

Constraints specification

Equality and inequality constraints arguments could be a single constraint or any iterable constraints container (e.g. vector or tuple). Each constraint could be a function or any other callable object that takes the two input arguments:

• the vector of the model parameters;
• the array for the in-place calculation of the constraint gradient.

To override the default tolerance, the constraint can be specified as a pair of the function and its tolerance: constraint_func => tol. For information on how to use inequality and equality constraints, see Constrained optimization in our online documentation.

Example

my_optimizer = SemOptimizer(engine = :NLopt)

# constrained optimization with augmented lagrangian
my_constrained_optimizer = SemOptimizer(;
    engine = :NLopt,
    algorithm = :AUGLAG,
    local_algorithm = :LD_LBFGS,
    local_options = Dict(:ftol_rel => 1e-6),
    inequality_constraints = (my_constraint => tol),
)

Interfaces

• algorithm(::SemOptimizerNLopt)
• local_algorithm(::SemOptimizerNLopt)
• options(::SemOptimizerNLopt)
• local_options(::SemOptimizerNLopt)
• equality_constraints(::SemOptimizerNLopt)
• inequality_constraints(::SemOptimizerNLopt)

Connects to ProximalAlgorithms.jl as the optimization backend. For more information on the available algorithms and options, see the online docs on Regularization and the documentation of ProximalAlgorithms.jl / ProximalOperators.jl.

Constructor

SemOptimizerProximal(;
    algorithm = ProximalAlgorithms.PANOC(),
    operator_g,
    operator_h = nothing,
    kwargs...,

Arguments

• algorithm: optimization algorithm.
• operator_g: proximal operator (e.g., regularization penalty)
• operator_h: optional second proximal operator