Supertype of all objects that can serve as the observed field of a SEM. Pre-processes data and computes sufficient statistics for example. If you have a special kind of data, e.g. ordinal data, you should implement a subtype of SemObserved.

For observed data without missings.

Constructor

SemObservedData(;
    data,
    observed_vars = nothing,
    specification = nothing,
    kwargs...)

Arguments

• specification: optional SEM specification (SemSpecification)
• data: observed data – DataFrame or Matrix
• observed_vars::Vector{Symbol}: column names of the data (if the object passed as data does not have column names, i.e. is not a data frame)

Extended help

Interfaces

• nsamples(::SemObservedData) -> number of observed data points

• nobserved_vars(::SemObservedData) -> number of observed (manifested) variables

• samples(::SemObservedData) -> observed data

• obs_cov(::SemObservedData) -> observed.obs_cov

• obs_mean(::SemObservedData) -> observed.obs_mean

Implementation

Subtype of SemObserved

Type alias for SemObservedData that has mean and covariance, but no actual data.

For instances of SemObservedCovariance samples returns nothing.

For observed data with missing values.

Constructor

SemObservedMissing(;
    data,
    observed_vars = nothing,
    specification = nothing,
    kwargs...)

Arguments

• specification: optional SEM model specification (SemSpecification)
• data: observed data
• observed_vars::Vector{Symbol}: column names of the data (if the object passed as data does not have column names, i.e. is not a data frame)

Extended help

Interfaces

• nsamples(::SemObservedMissing) -> number of samples (data points)

• nobserved_vars(::SemObservedMissing) -> number of observed variables

• samples(::SemObservedMissing) -> data matrix (contains both measured and missing values)

• em_model(::SemObservedMissing) -> EmMVNModel that contains the covariance matrix and mean vector found via expectation maximization

Implementation

Subtype of SemObserved

samples(observed::SemObservedData)

Gets the matrix of observed data samples. Rows are samples, columns are observed variables.

See Also

nsamples, observed_vars.

observed_vars(semobj) -> Vector{Symbol}

Return the vector of SEM model observed variable in the order specified by the model, which also should match the order of variables in SemObserved.

Base type for all SEM specifications.

Supertype of all objects that can serve as the implied field of a SEM. Computed model-implied values that should be compared with the observed data to find parameter estimates, e. g. the model implied covariance or mean. If you would like to implement a different notation, e.g. LISREL, you should implement a subtype of SemImplied.

Model implied covariance and means via RAM notation.

Constructor

RAM(;
    specification,
    meanstructure = false,
    gradient = true,
    kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object
• meanstructure::Bool: does the model have a meanstructure?
• gradient::Bool: is gradient-based optimization used

Extended help

Implementation

Subtype of SemImplied.

RAM notation

The model implied covariance matrix is computed as

\[ \Sigma = F(I-A)^{-1}S(I-A)^{-T}F^T\]

and for models with a meanstructure, the model implied means are computed as

\[ \mu = F(I-A)^{-1}M\]

Interfaces

• params(::RAM)-> vector of parameter labels

• nparams(::RAM) -> number of parameters

• Σ(::RAM) -> model implied covariance matrix

• μ(::RAM) -> model implied mean vector

RAM matrices for the current parameter values:

• A(::RAM)
• S(::RAM)
• F(::RAM)
• M(::RAM)

Jacobians of RAM matrices w.r.t to the parameter vector θ

• ∇A(::RAM) -> $∂vec(A)/∂θᵀ$
• ∇S(::RAM) -> $∂vec(S)/∂θᵀ$
• ∇M(::RAM) = $∂M/∂θᵀ$

Vector of indices of each parameter in the respective RAM matrix:

• A_indices(::RAM)
• S_indices(::RAM)
• M_indices(::RAM)

Additional interfaces

• F⨉I_A⁻¹(::RAM) -> $F(I-A)^{-1}$
• F⨉I_A⁻¹S(::RAM) -> $F(I-A)^{-1}S$
• I_A(::RAM) -> $I-A$
• has_meanstructure(::RAM) -> Val{Bool} does the model have a meanstructure?

Only available in gradient! calls:

• I_A⁻¹(::RAM) -> $(I-A)^{-1}$

Subtype of SemImplied that implements the RAM notation with symbolic precomputation.

Constructor

RAMSymbolic(;specification,
    vech = false,
    gradient = true,
    hessian = false,
    approximate_hessian = false,
    meanstructure = false,
    kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object
• meanstructure::Bool: does the model have a meanstructure?
• gradient::Bool: is gradient-based optimization used
• hessian::Bool: is hessian-based optimization used
• approximate_hessian::Bool: for hessian based optimization: should the hessian be approximated
• vech::Bool: should the half-vectorization of Σ be computed (instead of the full matrix) (automatically set to true if any of the loss functions is SemWLS)

Extended help

Implementation

Subtype of SemImplied.

Interfaces

• params(::RAMSymbolic)-> vector of parameter ids

• nparams(::RAMSymbolic) -> number of parameters

• Σ(::RAMSymbolic) -> model implied covariance matrix

• μ(::RAMSymbolic) -> model implied mean vector

Jacobians (only available in gradient! calls)

• ∇Σ(::RAMSymbolic) -> $∂vec(Σ)/∂θᵀ$

• ∇μ(::RAMSymbolic) -> $∂μ/∂θᵀ$

• ∇Σ_function(::RAMSymbolic) -> function to overwrite ∇Σ in place, i.e. ∇Σ_function(∇Σ, θ). Normally, you do not want to use this but simply query ∇Σ(::RAMSymbolic).

Hessians The computation of hessians is more involved, and uses the "chain rule for hessian matrices". Therefore, we desribe it at length in the mathematical appendix of the online documentation, and the relevant interfaces are omitted here.

Additional interfaces

• has_meanstructure(::RAMSymbolic) -> Val{Bool} does the model have a meanstructure?

RAM notation

The model implied covariance matrix is computed as

\[ \Sigma = F(I-A)^{-1}S(I-A)^{-T}F^T\]

and for models with a meanstructure, the model implied means are computed as

\[ \mu = F(I-A)^{-1}M\]

Empty placeholder for models that don't need an implied part. (For example, models that only regularize parameters.)

Constructor

ImpliedEmpty(;specification, kwargs...)

Arguments

• specification: either a RAMMatrices or ParameterTable object

Examples

A multigroup model with ridge regularization could be specified as a SemEnsemble with one model per group and an additional model with ImpliedEmpty and SemRidge for the regularization part.

Extended help

Interfaces

• params(::RAMSymbolic)-> Vector of parameter labels
• nparams(::RAMSymbolic) -> Number of parameters

Implementation

Subtype of SemImplied.

SemLoss(args...; loss_weights = nothing, ...)

Constructs the loss field of a SEM. Can contain multiple SemLossFunctions, the model is optimized over their sum. See also SemLossFunction.

Arguments

• args...: Multiple SemLossFunctions.
• loss_weights::Vector: Weights for each loss function. Defaults to unweighted optimization.

Examples

my_ml_loss = SemML(...)
my_ridge_loss = SemRidge(...)
my_loss = SemLoss(SemML, SemRidge; loss_weights = [1.0, 2.0])

Supertype for all loss functions of SEMs. If you want to implement a custom loss function, it should be a subtype of SemLossFunction.

Maximum likelihood estimation.

Constructor

SemML(;observed, meanstructure = false, approximate_hessian = false, kwargs...)

Arguments

• observed::SemObserved: the observed part of the model
• meanstructure::Bool: does the model have a meanstructure?
• approximate_hessian::Bool: if hessian-based optimization is used, should the hessian be swapped for an approximation

Examples

my_ml = SemML(observed = my_observed)

Interfaces

Analytic gradients are available, and for models without a meanstructure, also analytic hessians.

Extended help

Implementation

Subtype of SemLossFunction.

Full information maximum likelihood estimation. Can handle observed data with missings.

Constructor

SemFIML(;observed, specification, kwargs...)

Arguments

• observed::SemObservedMissing: the observed part of the model
• specification: either a RAMMatrices or ParameterTable object

Examples

my_fiml = SemFIML(observed = my_observed, specification = my_parameter_table)

Interfaces

Analytic gradients are available.

Extended help

Implementation

Subtype of SemLossFunction.

Weighted least squares estimation.

Constructor

SemWLS(;
    observed,
    meanstructure = false,
    wls_weight_matrix = nothing,
    wls_weight_matrix_mean = nothing,
    approximate_hessian = false,
    kwargs...)

Arguments

• observed: the SemObserved part of the model
• meanstructure::Bool: does the model have a meanstructure?
• approximate_hessian::Bool: should the hessian be swapped for an approximation
• wls_weight_matrix: the weight matrix for weighted least squares. Defaults to GLS estimation ($0.5*(D^T*kron(S,S)*D)$ where D is the duplication matrix and S is the inverse of the observed covariance matrix)
• wls_weight_matrix_mean: the weight matrix for the mean part of weighted least squares. Defaults to GLS estimation (the inverse of the observed covariance matrix)

Examples

my_wls = SemWLS(observed = my_observed)

Interfaces

Analytic gradients are available, and for models without a meanstructure, also analytic hessians.

Extended help

Implementation

Subtype of SemLossFunction.

Ridge regularization.

Constructor

SemRidge(;α_ridge, which_ridge, nparams, parameter_type = Float64, implied = nothing, kwargs...)

Arguments

• α_ridge: hyperparameter for penalty term
• which_ridge::Vector: Vector of parameter labels (Symbols) or indices that indicate which parameters should be regularized.
• nparams::Int: number of parameters of the model
• implied::SemImplied: implied part of the model
• parameter_type: type of the parameters

Examples

my_ridge = SemRidge(;α_ridge = 0.02, which_ridge = [:λ₁, :λ₂, :ω₂₃], nparams = 30, implied = my_implied)

Interfaces

Analytic gradients and hessians are available.

Extended help

Implementation

Subtype of SemLossFunction.

Constant loss term. Can be used for comparability to other packages.

Constructor

SemConstant(;constant_loss, kwargs...)

Arguments

• constant_loss::Number: constant to add to the objective

Examples

    my_constant = SemConstant(constant_loss = 42.0)

Interfaces

Analytic gradients and hessians are available.

Extended help

Implementation

Subtype of SemLossFunction.

Supertype of all objects that can serve as the optimizer field of a SEM. Connects the SEM to its optimization backend and controls options like the optimization algorithm. If you want to connect the SEM package to a new optimization backend, you should implement a subtype of SemOptimizer.

SemOptimizerOptim{A, B} <: SemOptimizer{:Optim}

Connects to Optim.jl as the optimization backend.

Constructor

SemOptimizerOptim(;
    algorithm = LBFGS(),
    options = Optim.Options(;f_tol = 1e-10, x_tol = 1.5e-8),
    kwargs...)

Arguments

• algorithm: optimization algorithm.
• options::Optim.Options: options for the optimization algorithm

Usage

All algorithms and options from the Optim.jl library are available, for more information see the Optim.jl online documentation.

Examples

my_optimizer = SemOptimizerOptim()

# hessian based optimization with backtracking linesearch and modified initial step size
using Optim, LineSearches

my_newton_optimizer = SemOptimizerOptim(
    algorithm = Newton(
        ;linesearch = BackTracking(order=3),
        alphaguess = InitialHagerZhang()
    )
)

Extended help

Constrained optimization

When using the Fminbox or SAMIN constrained optimization algorithms, the vector or dictionary of lower and upper bounds for each model parameter can be specified via lower_bounds and upper_bounds keyword arguments. Alternatively, the lower_bound and upper_bound keyword arguments can be used to specify the default bound for all non-variance model parameters, and the variance_lower_bound and variance_upper_bound keyword – for the variance parameters (the diagonal of the S matrix).

Interfaces

• algorithm(::SemOptimizerOptim)
• options(::SemOptimizerOptim)

Implementation

Subtype of SemOptimizer.

Connects to NLopt.jl as the optimization backend. Only usable if NLopt.jl is loaded in the current Julia session!

Constructor

SemOptimizerNLopt(;
    algorithm = :LD_LBFGS,
    options = Dict{Symbol, Any}(),
    local_algorithm = nothing,
    local_options = Dict{Symbol, Any}(),
    equality_constraints = Vector{NLoptConstraint}(),
    inequality_constraints = Vector{NLoptConstraint}(),
    kwargs...)

Arguments

• algorithm: optimization algorithm.
• options::Dict{Symbol, Any}: options for the optimization algorithm
• local_algorithm: local optimization algorithm
• local_options::Dict{Symbol, Any}: options for the local optimization algorithm
• equality_constraints::Vector{NLoptConstraint}: vector of equality constraints
• inequality_constraints::Vector{NLoptConstraint}: vector of inequality constraints

Example

my_optimizer = SemOptimizerNLopt()

# constrained optimization with augmented lagrangian
my_constrained_optimizer = SemOptimizerNLopt(;
    algorithm = :AUGLAG,
    local_algorithm = :LD_LBFGS,
    local_options = Dict(:ftol_rel => 1e-6),
    inequality_constraints = NLoptConstraint(;f = my_constraint, tol = 0.0),
)

Usage

All algorithms and options from the NLopt library are available, for more information see the NLopt.jl package and the NLopt online documentation. For information on how to use inequality and equality constraints, see Constrained optimization in our online documentation.

Extended help

Interfaces

• algorithm(::SemOptimizerNLopt)
• local_algorithm(::SemOptimizerNLopt)
• options(::SemOptimizerNLopt)
• local_options(::SemOptimizerNLopt)
• equality_constraints(::SemOptimizerNLopt)
• inequality_constraints(::SemOptimizerNLopt)

Implementation

Subtype of SemOptimizer.

Connects to ProximalAlgorithms.jl as the optimization backend.

Constructor

SemOptimizerProximal(;
    algorithm = ProximalAlgorithms.PANOC(),
    operator_g,
    operator_h = nothing,
    kwargs...,

Arguments

• algorithm: optimization algorithm.
• operator_g: gradient of the objective function
• operator_h: optional hessian of the objective function