Supertype of all objects that can serve as the observed field of a SEM. Pre-processes data and computes sufficient statistics for example. If you have a special kind of data, e.g. ordinal data, you should implement a subtype of SemObserved.

For observed data without missings.

Constructor

SemObservedData(;
    data,
    observed_vars = nothing,
    specification = nothing,
    kwargs...)

Arguments

• data: observed data – DataFrame or Matrix
• observed_vars::Vector{Symbol}: column names of the data (if the object passed as data does not have column names, i.e. is not a data frame)
• specification: optional SEM specification (SemSpecification)

Extended help

Interfaces

• nsamples(::SemObservedData) -> number of observed data points

• nobserved_vars(::SemObservedData) -> number of observed (manifested) variables

• samples(::SemObservedData) -> observed data

• obs_cov(::SemObservedData) -> observed covariance matrix

• obs_mean(::SemObservedData) -> observed mean vector

Type alias for SemObservedData that has mean and covariance, but no actual data.

For instances of SemObservedCovariance samples returns nothing.

For observed data with missing values.

Constructor

SemObservedMissing(;
    data,
    observed_vars = nothing,
    specification = nothing,
    kwargs...)

Arguments

• data: observed data
• observed_vars::Vector{Symbol}: column names of the data (if the object passed as data does not have column names, i.e. is not a data frame)
• specification: optional SEM model specification (SemSpecification)

Extended help

Interfaces

• nsamples(::SemObservedMissing) -> number of samples (data points)

• nobserved_vars(::SemObservedMissing) -> number of observed variables

• samples(::SemObservedMissing) -> data matrix (contains both measured and missing values)

Expectation maximization

em_mvn!(::SemObservedMissing) can be called to fit a covariance matrix and mean vector to the data using an expectation maximization (EM) algorithm under the assumption of multivariate normality. After, the following methods are available:

• em_model(::SemObservedMissing) -> EmMVNModel that contains the covariance matrix and mean vector found via EM
• obs_cov(::SemObservedData) -> EM covariance matrix
• obs_mean(::SemObservedData) -> EM mean vector

samples(observed::SemObservedData)

Gets the matrix of observed data samples. Rows are samples, columns are observed variables.

See Also

nsamples, observed_vars.

observed_vars(semobj) -> Vector{Symbol}

Return the vector of SEM model observed variable in the order specified by the model, which also should match the order of variables in SemObserved.

Base type for all SEM specifications.

Supertype of all objects that can serve as the implied field of a SEM. Computes model-implied values that should be compared with the observed data to find parameter estimates, e. g. the model implied covariance or mean. If you would like to implement a different notation, e.g. LISREL, you should implement a subtype of SemImplied.

Model implied covariance and means via RAM notation.

Constructor

RAM(;specification,
    meanstructure = false,
    gradient = true,
    kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object
• meanstructure::Bool: does the model have a meanstructure?
• gradient::Bool: is gradient-based optimization used

Extended help

RAM notation

The model implied covariance matrix is computed as

\[ \Sigma = F(I-A)^{-1}S(I-A)^{-T}F^T\]

and for models with a meanstructure, the model implied means are computed as

\[ \mu = F(I-A)^{-1}M\]

Interfaces

• param_labels(::RAM)-> vector of parameter labels

• nparams(::RAM) -> number of parameters

• ram.Σ -> model implied covariance matrix

• ram.μ -> model implied mean vector

RAM matrices for the current parameter values:

• ram.A
• ram.S
• ram.F
• ram.M

Jacobians of RAM matrices w.r.t to the parameter vector θ

• ram.∇A -> $∂vec(A)/∂θᵀ$
• ram.∇S -> $∂vec(S)/∂θᵀ$
• ram.∇M = $∂M/∂θᵀ$

Vector of indices of each parameter in the respective RAM matrix:

• ram.A_indices
• ram.S_indices
• ram.M_indices

Additional interfaces

• ram.F⨉I_A⁻¹ -> $F(I-A)^{-1}$
• ram.F⨉I_A⁻¹S -> $F(I-A)^{-1}S$
• ram.I_A -> $I-A$

Only available in gradient! calls:

• ram.I_A⁻¹ -> $(I-A)^{-1}$

Subtype of SemImplied that implements the RAM notation with symbolic precomputation.

Constructor

RAMSymbolic(;
    specification,
    vech = false,
    gradient = true,
    hessian = false,
    approximate_hessian = false,
    meanstructure = false,
    kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object
• meanstructure::Bool: does the model have a meanstructure?
• gradient::Bool: is gradient-based optimization used
• hessian::Bool: is hessian-based optimization used
• approximate_hessian::Bool: for hessian based optimization: should the hessian be approximated
• vech::Bool: should the half-vectorization of Σ be computed (instead of the full matrix) (automatically set to true if any of the loss functions is SemWLS)

Extended help

Interfaces

• param_labels(::RAMSymbolic)-> vector of parameter ids

• nparams(::RAMSymbolic) -> number of parameters

• ram.Σ -> model implied covariance matrix

• ram.μ -> model implied mean vector

Jacobians (only available in gradient! calls)

• ram.∇Σ -> $∂vec(Σ)/∂θᵀ$

• ram.∇μ -> $∂μ/∂θᵀ$

• ram.∇Σ_function -> function to overwrite ∇Σ in place, i.e. ∇Σ_function(∇Σ, θ). Typically, you do not want to use this but simply query ram.∇Σ.

Hessians The computation of hessians is more involved. Therefore, we desribe it in the online documentation, and the respective interfaces are omitted here.

RAM notation

The model implied covariance matrix is computed as

\[ \Sigma = F(I-A)^{-1}S(I-A)^{-T}F^T\]

and for models with a meanstructure, the model implied means are computed as

\[ \mu = F(I-A)^{-1}M\]

Empty placeholder for models that don't need an implied part. (For example, models that only regularize parameters.)

Constructor

ImpliedEmpty(;specification, kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object

Examples

A multigroup model with ridge regularization could be specified as a SemEnsemble with one model per group and an additional model with ImpliedEmpty and SemRidge for the regularization part.

Extended help

Interfaces

• param_labels(::ImpliedEmpty)-> Vector of parameter labels
• nparams(::ImpliedEmpty) -> Number of parameters

SemLoss(args...; loss_weights = nothing, ...)

Constructs the loss field of a SEM. Can contain multiple SemLossFunctions, the model is optimized over their sum. See also SemLossFunction.

Arguments

• args...: Multiple SemLossFunctions.
• loss_weights::Vector: Weights for each loss function. Defaults to unweighted optimization.

Examples

my_ml_loss = SemML(...)
my_ridge_loss = SemRidge(...)
my_loss = SemLoss(SemML, SemRidge; loss_weights = [1.0, 2.0])

Supertype for all loss functions of SEMs. If you want to implement a custom loss function, it should be a subtype of SemLossFunction.

Maximum likelihood estimation.

Constructor

SemML(;observed, meanstructure = false, approximate_hessian = false, kwargs...)

Arguments

• observed::SemObserved: the observed part of the model
• meanstructure::Bool: does the model have a meanstructure?
• approximate_hessian::Bool: if hessian-based optimization is used, should the hessian be swapped for an approximation

Examples

my_ml = SemML(observed = my_observed)

Interfaces

Analytic gradients are available, and for models without a meanstructure and RAMSymbolic implied type, also analytic hessians.

Full information maximum likelihood estimation. Can handle observed data with missings.

Constructor

SemFIML(;observed, specification, kwargs...)

Arguments

• observed::SemObservedMissing: the observed part of the model
• specification: either a RAMMatrices or ParameterTable object

Examples

my_fiml = SemFIML(observed = my_observed, specification = my_parameter_table)

Interfaces

Analytic gradients are available.

Weighted least squares estimation. At the moment only available with the RAMSymbolic implied type.

Constructor

SemWLS(;
    observed,
    meanstructure = false,
    wls_weight_matrix = nothing,
    wls_weight_matrix_mean = nothing,
    approximate_hessian = false,
    kwargs...)

Arguments

• observed: the SemObserved part of the model
• meanstructure::Bool: does the model have a meanstructure?
• approximate_hessian::Bool: should the hessian be swapped for an approximation
• wls_weight_matrix: the weight matrix for weighted least squares. Defaults to GLS estimation ($0.5*(D^T*kron(S,S)*D)$ where D is the duplication matrix and S is the inverse of the observed covariance matrix)
• wls_weight_matrix_mean: the weight matrix for the mean part of weighted least squares. Defaults to GLS estimation (the inverse of the observed covariance matrix)

Examples

my_wls = SemWLS(observed = my_observed)

Interfaces

Analytic gradients are available, and for models without a meanstructure also analytic hessians.

Ridge regularization.

Constructor

SemRidge(;α_ridge, which_ridge, nparams, parameter_type = Float64, implied = nothing, kwargs...)

Arguments

• α_ridge: hyperparameter for penalty term
• which_ridge::Vector: Vector of parameter labels (Symbols) or indices that indicate which parameters should be regularized.
• nparams::Int: number of parameters of the model
• implied::SemImplied: implied part of the model
• parameter_type: type of the parameters

Examples

my_ridge = SemRidge(;α_ridge = 0.02, which_ridge = [:λ₁, :λ₂, :ω₂₃], nparams = 30, implied = my_implied)

Interfaces

Analytic gradients and hessians are available.

Constant loss term. Can be used for comparability to other packages.

Constructor

SemConstant(;constant_loss, kwargs...)

Arguments

• constant_loss::Number: constant to add to the objective

Examples

    my_constant = SemConstant(constant_loss = 42.0)

Interfaces

Analytic gradients and hessians are available.

Supertype of all objects that can serve as the optimizer field of a SEM. Connects the SEM to its optimization backend and controls options like the optimization algorithm. If you want to connect the SEM package to a new optimization backend, you should implement a subtype of SemOptimizer.

SemOptimizerOptim{A, B} <: SemOptimizer{:Optim}

Connects to Optim.jl as the optimization backend.

Constructor

SemOptimizerOptim(;
    algorithm = LBFGS(),
    options = Optim.Options(;f_reltol = 1e-10, x_abstol = 1.5e-8),
    kwargs...)

Arguments

• algorithm: optimization algorithm from Optim.jl
• options::Optim.Options: options for the optimization algorithm

Usage

All algorithms and options from the Optim.jl library are available, for more information see the Optim.jl online documentation.

Examples

my_optimizer = SemOptimizerOptim()

# hessian based optimization with backtracking linesearch and modified initial step size
using Optim, LineSearches

my_newton_optimizer = SemOptimizerOptim(
    algorithm = Newton(
        ;linesearch = BackTracking(order=3),
        alphaguess = InitialHagerZhang()
    )
)

Extended help

Constrained optimization

When using the Fminbox or SAMIN constrained optimization algorithms, the vector or dictionary of lower and upper bounds for each model parameter can be specified via lower_bounds and upper_bounds keyword arguments. Alternatively, the lower_bound and upper_bound keyword arguments can be used to specify the default bound for all non-variance model parameters, and the variance_lower_bound and variance_upper_bound keyword – for the variance parameters (the diagonal of the S matrix).

Interfaces

• algorithm(::SemOptimizerOptim)
• options(::SemOptimizerOptim)

Implementation

Subtype of SemOptimizer.

Connects to NLopt.jl as the optimization backend. Only usable if NLopt.jl is loaded in the current Julia session!

Constructor

SemOptimizerNLopt(;
    algorithm = :LD_LBFGS,
    options = Dict{Symbol, Any}(),
    local_algorithm = nothing,
    local_options = Dict{Symbol, Any}(),
    equality_constraints = Vector{NLoptConstraint}(),
    inequality_constraints = Vector{NLoptConstraint}(),
    kwargs...)

Arguments

• algorithm: optimization algorithm.
• options::Dict{Symbol, Any}: options for the optimization algorithm
• local_algorithm: local optimization algorithm
• local_options::Dict{Symbol, Any}: options for the local optimization algorithm
• equality_constraints::Vector{NLoptConstraint}: vector of equality constraints
• inequality_constraints::Vector{NLoptConstraint}: vector of inequality constraints

Example

my_optimizer = SemOptimizerNLopt()

# constrained optimization with augmented lagrangian
my_constrained_optimizer = SemOptimizerNLopt(;
    algorithm = :AUGLAG,
    local_algorithm = :LD_LBFGS,
    local_options = Dict(:ftol_rel => 1e-6),
    inequality_constraints = NLoptConstraint(;f = my_constraint, tol = 0.0),
)

Usage

All algorithms and options from the NLopt library are available, for more information see the NLopt.jl package and the NLopt online documentation. For information on how to use inequality and equality constraints, see Constrained optimization in our online documentation.

Extended help

Interfaces

• algorithm(::SemOptimizerNLopt)
• local_algorithm(::SemOptimizerNLopt)
• options(::SemOptimizerNLopt)
• local_options(::SemOptimizerNLopt)
• equality_constraints(::SemOptimizerNLopt)
• inequality_constraints(::SemOptimizerNLopt)

Connects to ProximalAlgorithms.jl as the optimization backend. Can be used for regularized SEM, for a tutorial see the online docs on Regularization.

Constructor

SemOptimizerProximal(;
    algorithm = ProximalAlgorithms.PANOC(),
    operator_g,
    operator_h = nothing,
    kwargs...,

Arguments

• algorithm: optimization algorithm.
• operator_g: proximal operator (e.g., regularization penalty)
• operator_h: optional second proximal operator

Usage

All algorithms and operators from ProximalAlgorithms.jl are available, for more information see the online docs on Regularization and the documentation of ProximalAlgorithms.jl / ProximalOperators.jl.