fab.hme.learner package¶
Subpackages¶
- fab.hme.learner.init package
- fab.hme.learner.opt package
Submodules¶
fab.hme.learner.bern_gate_mixin module¶
- class fab.hme.learner.bern_gate_mixin.HMEBernGateMixin¶
Bases:
object
Mix-in class for the latent variable prior with Bernoulli-gates.
This mix-in is add a feature of the latent variable prior with Bernoulli-gates for concrete classes of HMEBinaryTreeSupervisedLearner.
Methods
binarize_lvprior
(model, data, context)Binarizes all Bernoulli-gate functions in the latent variable prior.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior.
- binarize_lvprior(model, data, context)¶
Binarizes all Bernoulli-gate functions in the latent variable prior.
- Parameters:
- modelHMESupervisedModel
Model whose parameters will be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- None
- opt_lvprior(model, vposterior, data, context)¶
Optimizes parameters of a latent variable prior.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of the latent variable prior are updated. It must have HMEBernGateLVPrior as a prior.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context information. Some pre-calculated properties at other processes could be used if needed.
- Returns:
- None
fab.hme.learner.bspline_cl_mixin module¶
- class fab.hme.learner.bspline_cl_mixin.HMEBinaryTreeBSplineClMixin¶
Bases:
object
Mix-in class for B-spline classifications.
This mix-in is add a feature of B-spline classification component for concrete classes of HMEBinaryTreeSupervisedLearner. The learner inheriting this mix-in must define self.comp_opt_params as a dict type in which ‘with_scaled_l0_regularize’ is contained.
Methods
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
opt_comps
(model, vposterior, data, context)Optimizes parameters of B-spline non-linear classification components.
- calc_comp_fisher_coeffs(model, data, context)¶
Calculates the Fisher coefficients of components.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- comp_fisher_coeffsfloat or np.ndarray,
size = (num_samples, num_comps)
If 1.0 [default], the number of expected samples is simply applied. For calculating scaled FIC value of component complexity penalty, coefficient matrix or scalar value to vposterior must be given.
- opt_comps(model, vposterior, data, context)¶
Optimizes parameters of B-spline non-linear classification components.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of components are updated.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context information. Some pre-calculated properties at other processes could be used if needed.
- Returns:
- None
fab.hme.learner.bspline_rg_mixin module¶
- class fab.hme.learner.bspline_rg_mixin.HMEBinaryTreeBSplineRgMixin¶
Bases:
object
Mix-in class for B-spline regressions.
This mix-in is add a feature of B-spline regression component for concrete classes of HMEBinaryTreeSupervisedLearner. The learner inheriting this mix-in must define self.comp_opt_params as a dict type in which ‘with_scaled_l0_regularize’ is contained.
Methods
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
opt_comps
(model, vposterior, data, context)Optimizes parameters of B-spline non-linear regression components.
- calc_comp_fisher_coeffs(model, data, context)¶
Calculates the Fisher coefficients of components.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- comp_fisher_coeffsfloat or np.ndarray,
size = (num_samples, num_comps)
If 1.0 [default], the number of expected samples is simply applied. For calculating scaled FIC value of component complexity penalty, coefficient matrix or scalar value to vposterior must be given.
- opt_comps(model, vposterior, data, context)¶
Optimizes parameters of B-spline non-linear regression components.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of components are updated.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context information. Some pre-calculated properties at other processes could be used if needed.
- Returns:
- None
fab.hme.learner.hme_bern_gate_bspline_cl_learner module¶
- class fab.hme.learner.hme_bern_gate_bspline_cl_learner.HMEBernGateBSplineClLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, comp_bspline_degree, comp_bspline_basis_dim, comp_bspline_knot_vecs, gate_opt_mode, gate_max_bins, comp_opt_mode, post_comp_opt_type, comp_l2_regularize, comp_pspline, with_comp_scaled_l0_regularize, max_comp_relevant_features, comp_svd_threshold, num_threads_gates, num_threads_gate_features, num_threads_comps)¶
Bases:
HMEBernGateMixin
,HMEBinaryTreeBSplineClMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME B-spline non-linear classification learner with Bernoulli-gates.
Notes
Use init_random(), init_with_posterior() or init_with_model_dict() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all Bernoulli-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficient of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME Bernoulli gates B-spline non-linear classification learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME Bernoulli gates B-spline non-linear classification learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME Bernoulli gates B-spline non-linear classification learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of B-spline non-linear classification components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, tree_depth=5, comp_bspline_degree=3, comp_bspline_basis_dim=10, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, gate_max_bins=None, post_comp_opt_type='standard', comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates B-spline non-linear classification learner where the all parameters are initialized randomly.
- Parameters:
- comp_bspline_degreeint, optional [default: 3]
Degree of B-spline function.
- comp_bspline_basis_dimint, optional [default: 10]
The number of B-spline basis functions to be generated for each feature.
- post_comp_opt_type{‘standard’, ‘quadratic’}, optional
Algorithm of component parameter optimization in the post-processing. The domain and default value depends on each learner type and described in the following section. [default: ‘standard’]
- comp_psplinefloat, optional [default: 1.0]
P-spline penalized coefficient parameter for component parameter optimization.
- Returns:
- learnerHMEBernGateBSplineClLearner
An instance of FAB/HME Bernoulli gates B-spline non-linear classification learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, gate_opt_mode='opt', gate_max_bins=None, comp_opt_mode='opt', post_comp_opt_type='standard', comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates B-spline non-linear classification learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’]. Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMEBernGateBSplineClLearner
An instance of FAB/HME Bernoulli gates B-spline non-linear classification learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_bspline_degree=3, comp_bspline_basis_dim=10, gate_max_bins=None, post_comp_opt_type='standard', comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates B-spline non-linear classification learner where the initial posterior distribution is given.
- Returns:
- learnerHMEBernGateBSplineClLearner
An instance of FAB/HME Bernoulli gates B-spline non-linear classification learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_bern_gate_bspline_rg_learner module¶
- class fab.hme.learner.hme_bern_gate_bspline_rg_learner.HMEBernGateBSplineRgLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, comp_bspline_degree, comp_bspline_basis_dim, comp_bspline_knot_vecs, gate_opt_mode, gate_max_bins, comp_opt_mode, comp_l2_regularize, comp_pspline, with_comp_scaled_l0_regularize, max_comp_relevant_features, comp_svd_threshold, num_threads_gates, num_threads_gate_features, num_threads_comps)¶
Bases:
HMEBernGateMixin
,HMEBinaryTreeBSplineRgMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME B-spline non-linear regression learner with Bernoulli-gates.
Notes
Use init_random(), init_with_posterior() or init_with_model_dict() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all Bernoulli-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficient of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME Bernoulli gates B-spline non-linear regression learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME Bernoulli gates B-spline non-linear regression learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME Bernoulli gates B-spline non-linear regression learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of B-spline non-linear regression components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, tree_depth=5, comp_bspline_degree=3, comp_bspline_basis_dim=10, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, comp_variance_min_scale=0.1, comp_variance_max_scale=0.25, gate_max_bins=None, comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates B-spline non-linear regression learner where the all parameters are initialized randomly.
- Parameters:
- comp_bspline_degreeint, optional [default: 3]
Degree of B-spline function.
- comp_bspline_basis_dimint, optional [default: 10]
The number of B-spline basis functions to be generated for each feature.
- comp_psplinefloat, optional [default: 1.0]
P-spline penalized coefficient parameter for component parameter optimization.
- Returns:
- learnerHMEBernGateBSplineRgLearner
An instance of FAB/HME Bernoulli gates B-spline non-linear regression learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, gate_opt_mode='opt', gate_max_bins=None, comp_opt_mode='opt', comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates B-spline non-linear regression learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMEBernGateLeastSquaresRgLearner
An instance of FAB/HME least-squares regression learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_bspline_degree=3, comp_bspline_basis_dim=10, gate_max_bins=None, comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates B-spline non-linear regression learner where the initial posterior distribution is given.
- Returns:
- learnerHMEBernGateBSplineRgLearner
An instance of FAB/HME Bernoulli gates B-spline non-linear regression learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_bern_gate_logit_rg_learner module¶
- class fab.hme.learner.hme_bern_gate_logit_rg_learner.HMEBernGateLogisticRgLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, comp_positive_feature_ids, comp_negative_feature_ids, gate_opt_mode, gate_max_bins, comp_foba_skip, comp_foba_skip_max_interval, comp_opt_mode, comp_backward_step, comp_opt_type, post_comp_opt_type, comp_l2_regularize, with_comp_scaled_l0_regularize, max_comp_relevant_features, max_comp_foba_iterations, comp_svd_threshold, num_threads_gates, num_threads_gate_features, num_threads_comps)¶
Bases:
HMEBernGateMixin
,HMEBinaryTreeLogisticRgMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME logistic regression learner with Bernoulli-gates.
Notes
Use init_random(), init_with_posterior() or init_with_model_dict() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all Bernoulli-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficient of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME Bernoulli gates logistic regression learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME Bernoulli gates logistic regression learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME Bernoulli gates logistic regression learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of logistic regression components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, tree_depth=5, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, gate_max_bins=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_backward_step=False, comp_opt_type='quadratic', post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates logistic regression learner where the all parameters are initialized randomly.
- Parameters:
- comp_opt_type{‘quadratic’, ‘standard’, ‘quadratic_svd’}, optional
[default: ‘quadratic’] Algorithm of component parameter optimization:
‘standard’: using logit function explicitly; liblinear-weights.
- ‘quadratic’: using quadratic upper bound approximation with matrix
inversion lemma.
- ‘quadratic_svd’: using quadratic upper bound approximation with
singular value decomposition.
- post_comp_opt_type{‘standard’, ‘quadratic’}, optional
[default: ‘standard’] Algorithm of component parameter optimization in post-processing.
- Returns:
- learnerHMEBernGateLogisticRgLearner
An instance of FAB/HME Bernoulli gates logistic regression learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_opt_mode='opt', gate_max_bins=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_opt_mode='opt', comp_backward_step=False, comp_opt_type='quadratic', post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates logistic regression learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMEBernGateLogisticRgLearner
An instance of FAB/HME Bernoulli gates logistic regression learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_max_bins=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_backward_step=False, comp_opt_type='quadratic', post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates logistic regression learner where the initial posterior distribution is given.
- Returns:
- learnerHMEBernGateLogisticRgLearner
An instance of FAB/HME Bernoulli gates logistic regression learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_bern_gate_lsqr_rg_learner module¶
- class fab.hme.learner.hme_bern_gate_lsqr_rg_learner.HMEBernGateLeastSquaresRgLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, comp_positive_feature_ids, comp_negative_feature_ids, gate_opt_mode, gate_max_bins, comp_foba_skip, comp_foba_skip_max_interval, comp_opt_mode, comp_two_stage_opt, comp_backward_step, comp_opt_type, comp_l2_regularize, with_comp_scaled_l0_regularize, max_comp_relevant_features, max_comp_foba_iterations, comp_svd_threshold, num_threads_gates, num_threads_gate_features, num_threads_comps)¶
Bases:
HMEBernGateMixin
,HMEBinaryTreeLeastSquaresRgMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME least-squares regression learner with Bernoulli-gates.
Notes
Use init_random(), init_with_posterior() or init_with_model_dict() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all Bernoulli-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficient of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME Bernoulli gates least-squares regression learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME least-squares regression learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME least-squares regression learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of least-squares regression components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, tree_depth=5, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, comp_variance_min_scale=0.1, comp_variance_max_scale=0.25, gate_max_bins=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_two_stage_opt=False, comp_backward_step=False, comp_opt_type='mil', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates least-squares regression learner where the all parameters are initialized randomly.
- Parameters:
- max_fab_iterationsint, optional [default: 100]
Maximum number of FAB-iterations. Domain = [1, inf).
- start_from_mstepbool, optional [default: False]
If True, the first iteration starts with M-step; otherwise, E-step.
- num_acceleration_stepsint, optional [default: 0]
The number of steps of acceleration algorithm for each FAB iteration. If 0, the acceleration algorithm is disabled. Domain = [1, inf).
- repeat_until_convergencebool, optional [default: False]
If False, FAB-iterations and the post-processing are executed only once even if the FAB-iterations are stopped not by convergence condition but by max_fab_iterations condition.
- projection_estepbool, optional [default: False]
If True, the projection E-step algorithm is enabled, otherwise the normal E-step method is executed. It results in higher accuracy in some cases by considering tree structure in the E-step.
- shrink_thresholdfloat or str, optional [default: 1.0]
Threshold value for shrinkage. If a str in percentage value (ex. ‘1.0%’) is specified, shrinkage is executed according to relative value (= num_samples * shrink_threshold). Domain = [1, inf) for absolute value, (0%, 100%) for relative value.
- fab_stop_thresholdfloat or str, optional [default: 0.001]
Threshold value for FAB-iterations: the increase of FIC value is less than the threshold, the FAB-iterations is considered to be converged. If a str in percentage value (ex. ‘1.0%’) is specified, convergence check is executed according to relative value (= threshold > \((FIC^{(t)} - FIC^{(t-1)}) / | FIC^{(t-1)} |\). Domain = (0, inf).
- hard_gatebool, optional [default: True]
If True, hard-gate post-processing is enabled.
- gate_feature_idsNone or list[int], optional [default: None]
List of feature IDs which are applied to gate parameter optimizations. If None, all features are used.
- comp_feature_idsNone or list[int], optional [default: None]
List of feature IDs which are applied to component parameter optimizations. If None, all features are used. If empty list, model is learned as decision tree (not supported in learners consisting of B-spline Rg/Cl components).
- comp_mandatory_feature_idsNone or list[int], optional
List of feature IDs which non-L0-regularize constraints are applied to. It means the specified features will always be relevant for all components. If None, no features are specified for non-L0-regularization. [default: None]
- comp_positive_feature_idsNone or list[int], optional
List of feature IDs whose weight values for all components are constrained to positive values. If None, all features are optimized with no constraints.
- comp_negative_feature_idsNone or list[int], optional
List of feature IDs whose weight values for all components are constrained to negative values. If None, all features are optimized with no constraint.
- tree_depthint, optional [default: 5]
Initial depth of the gate-tree structure of latent variable prior. The initial number of components is 2^tree_depth. If 0, the optimization with only one component will be executed. Domain = [0, inf).
- comp_weights_min_scale, comp_weights_max_scalefloat, optional
Scale value for the initialization of weight values of components. Domain = (-inf, inf). [default: -0.5, 0.5, respectively]
- comp_bias_min_scale, comp_bias_max_scalefloat, optional
Scale value for the initialization of bias values of components. Domain = (-inf, inf). [default: 0.25, 0.75, respectively]
- comp_variance_min_scale, comp_variance_max_scalefloat, optional
Scale value for the initialization of variance values of components. Domain = (0, inf). [default: 0.1, 0.25, respectively]
- gate_max_binsNone or int, optional [default: None]
Maximum number of binning for each feature, which is used for gate parameter optimization. If None, all unique samples for each feature are used; otherwise, the equal-width binning algorithm is adopted. Domain = [1, inf).
- comp_foba_skip{‘power_of_two’, ‘quarter_square’, ‘none’},
optional [default: ‘power_of_two’] The checking function for the FoBa algorithm skipping. If ‘none’, FoBa algorithm is executed for all steps.
- comp_foba_skip_max_intervalint, optional [default: 25]
The maximum interval for the FoBa algorithm skipping. If comp_foba_skip is ‘none’, this value is ignored. Domain = [2, inf)
- comp_two_stage_optbool, optional [default: False]
Whether the two-stage optimization is enabled. If True, the first stage performs the parameter optimization on user-specified base features (comp_mandatory_feature_ids), and the second stage carries out parameter optimization to the residual of the first stage for only the relevant non-base features.
- comp_backward_stepbool, optional [default: False]
Whether the backward-steps are enabled in FoBa algorithm. In the post-process the backward-step is forcibly enabled regardless of this value.
- comp_opt_type{‘mil’, ‘svd’}, optional [default: ‘mil’]
Algorithm of component parameter optimization; ‘svd’: using singular value decomposition, ‘mil’: using matrix inversion lemma.
- comp_l2_regularizefloat, optional [default: 0.0]
L2-regularization hyper-parameter for component parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled. Domain = [0, inf).
- with_comp_scaled_l0_regularizebool, optional [default: True]
Whether with scaled L0-regularization using a tighter lower bound of FIC for components; approximation of det(F) is refined, where F is a empirical Fisher information matrix.
- max_comp_relevant_featuresint, optional [default: 100]
Maximum number of the relevant features for each component.
- max_comp_foba_iterationsint, optional [default: 100]
Maximum number of the FoBa-iterations for each component. Domain = [1, inf).
- comp_svd_thresholdfloat, optional [default: 0.00001]
Threshold value for singular value decomposition (SVD) in component parameter optimization with FoBa-algorithm. Modes having eigen values smaller than the threshold are ignored. When comp_opt_type is not ‘svd’, this value is ignored. Domain = [0, inf).
- num_threads_gatesint, optional [default: 1]
Maximum number of OpenMP threads of gate parameter optimization where tasks for all gates are divided into. If the number of gates is less than num_threads_gates, the former is used. Domain = [1, inf).
- num_threads_gate_featuresint, optional [default: 1]
Maximum number of OpenMP threads of gate parameter optimization where tasks for all features are divided into. If the number of features is less than num_threads_gate_features, the former is used. Domain = [1, inf).
- num_threads_compsint, optional [default: 1]
Maximum number of OpenMP threads of component parameter optimization. If the number of components is less than num_threads_comps, the former is used. Domain = [1, inf).
- Returns:
- learnerHMEBernGateLeastSquaresRgLearner
An instance of FAB/HME Bernoulli gates least-squares regression learner.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_opt_mode='opt', gate_max_bins=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_opt_mode='opt', comp_two_stage_opt=False, comp_backward_step=False, comp_opt_type='mil', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME least-squares regression learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMEBernGateLeastSquaresRgLearner
An instance of FAB/HME least-squares regression learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_max_bins=None, comp_two_stage_opt=False, comp_backward_step=False, comp_opt_type='mil', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME least-squares regression learner where the initial posterior distribution is given.
Initial tree depth is calculated from comp_ids. The FAB-iteration will be started from M-step in order to use the specified initial posterior. Parameter values of components are initialize randomly but those values are not essential. The values do not affect to the learning result because they are updated at first M-step with specified initial posterior distribution.
- Parameters:
- posterior_probnp.ndarray, size = (num_samples, num_comps)
Initial posterior distribution. The number of samples (rows) must be consistent with that for input data given at
learn()
.- comp_idslist[int], size = (num_comps)
List of component ID numbers, whose IDs are assigned the same as the components with corresponding locations in a complete binary tree numbered from left to right (0 to 2^tree_depth - 1). Initial tree structure is defined from this parameter.
- Returns:
- learnerHMEBernGateLeastSquaresRgLearner
An instance of FAB/HME least-squares regression learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_bern_gate_softmax_cl_learner module¶
- class fab.hme.learner.hme_bern_gate_softmax_cl_learner.HMEBernGateSoftmaxClLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, gate_opt_mode, gate_max_bins, comp_foba_skip, comp_foba_skip_max_interval, comp_opt_mode, comp_backward_step, post_comp_opt_type, comp_l2_regularize, with_comp_scaled_l0_regularize, max_comp_relevant_features, max_comp_foba_iterations, comp_svd_threshold, num_threads_gates, num_threads_gate_features, num_threads_comps)¶
Bases:
HMEBernGateMixin
,HMEBinaryTreeSoftmaxClMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME softmax classification learner with Bernoulli-gates.
Notes
Use init_random(), init_with_posterior() or init_with_model_dict() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all Bernoulli-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficient of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME Bernoulli gates softmax classification learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME Bernoulli gates softmax classification learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME Bernoulli gates softmax classification learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of softmax classification components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, tree_depth=5, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, gate_max_bins=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_backward_step=False, post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates softmax classification learner where the all parameters are initialized randomly.
- Parameters:
- post_comp_opt_type{‘standard’, ‘quadratic’}, optional
[default: ‘standard’] Algorithm of component parameter optimization in post-processing.
- Returns:
- learnerHMEBernGateSoftmaxClLearner
An instance of FAB/HME Bernoulli gates softmax classification learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, gate_opt_mode='opt', gate_max_bins=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_opt_mode='opt', comp_backward_step=False, post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates softmax classification learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMEBernGateSoftmaxClLearner
An instance of FAB/HME Bernoulli gates softmax classification learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, gate_max_bins=None, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_backward_step=False, post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_gate_features=1, num_threads_comps=1)¶
Creates an instance of FAB/HME Bernoulli gates softmax classification learner where the initial posterior distribution is given.
- Returns:
- learnerHMEBernGateSoftmaxClLearner
An instance of FAB/HME Bernoulli gates softmax classification learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples, num_targets)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_binary_tree_supervised_learner module¶
- class fab.hme.learner.hme_binary_tree_supervised_learner.HMEBinaryTreeSupervisedLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_opt_mode='opt', comp_opt_mode='opt')¶
Bases:
HMESupervisedLearner
A base class of learners for FAB/HME supervised learning with prior having binary tree gate structure.
Methods
binarize_lvprior
(model, data, context)Binarizes all gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficient of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficient of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- abstract binarize_lvprior(model, data, context)¶
Binarizes all gate functions in the latent variable prior.
- Parameters:
- modelHMESupervisedModel
Model object whose parameters will be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- calc_comp_fisher_coeffs(model, data, context)¶
Calculates the Fisher coefficient of components.
If the component specific coefficient value instead of the default one (= 1.0), calculate the value from the diagonal elements of the empirical Fisher information matrix here, and set it in the context.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- comp_fisher_coeffsfloat or np.ndarray
Coefficient for calculating the scaled number of expected samples.
- calc_gate_fisher_coeffs(model, data, context)¶
Calculates the Fisher coefficient of gates.
If the gate specific coefficient value instead of the default one (= 1.0), calculate the value from the diagonal elements of the empirical Fisher information matrix here, and set it in the context.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- gate_fisher_coeffsfloat or np.ndarray
Coefficient for calculating the scaled number of expected samples.
- calc_loglikelihood_compwise(model, data, context)¶
Calculates the component-wise log-likelihood values.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- loglikelihood_compwisenp.ndarray, size = (num_samples, num_comps)
Component-wise log-likelihood values for all samples.
- do_post_process(model, vposterior, data, context)¶
Executes post processing.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- finalize(model, vposterior, data, context)¶
Executes finalizing the learning.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- init_model(data)¶
Creates model, variational posterior, and context objects and initializes their parameter values.
The learner must have a HMESupervisedModelInitializer object as self.model_initializer.
- Parameters:
- dataHMESupervisedData
Input data.
- Returns:
- modelHMESupervisedModel
Initialized model object.
- vposteriorHMEBinaryTreeVPosterior
Initialized variational posterior.
- contextHMELearningContext
Initialized context object.
- is_stop(model, vposterior, data, context)¶
Judges whether the FAB iteration is converged.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object to be used to be judged for convergence.
- Returns:
- is_stopbool
If True, the FAB iteration is converged, thus fulfilling the stopping condition.
- make_hard_gates(model, vposterior, data, context)¶
Converts soft-gates to hard-gates by modifying gate parameters.
- Parameters:
- modelHMESupervisedModel
Model object whose parameters will be updated.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior to be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- operate_shrinkage(model, vposterior, data, context)¶
Operates shrinkage process.
The model (components and tree structure of the latent variable prior) and variational posterior (the number of columns of probability matrix) are updated accompanying with shrinkage operation.
- Parameters:
- modelHMESupervisedModel
Model object to be updated.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior to be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
- update_context_fic(model, vposterior, data, context)¶
Calculates the FIC value.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object in which the newly calculated FIC value and the number of components will be appended.
- Returns:
- None
- update_vposterior(model, vposterior, data, context)¶
Updates the variational posterior distribution.
Projection E-step algorithm is supported in this method.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior to be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- update_vposterior_plain(model, vposterior, data, context)¶
Updates the variational posterior distribution by the plain method.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior to be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- update_vposterior_projection(model, vposterior, data, context)¶
Calculates the variational posterior distribution using projection E-step algorithm.
- Parameters:
- modelHMESupervisedModel
The model object and it must have HMEBernGateLVPrior as a prior.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior to be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context information. Some pre-calculated properties at other processes could be used if needed.
- Returns:
- None
fab.hme.learner.hme_binary_tree_vposterior module¶
- class fab.hme.learner.hme_binary_tree_vposterior.HMEBinaryTreeVPosterior(prob)¶
Bases:
HMEVPosterior
Variational posterior class for FAB/HME supervised model with binary tree latent variable prior.
- Parameters:
- probnp.ndarray, size = (num_samples, num_comps)
Initial distribution of variational posterior.
Methods
cumulate_vposterior_on_gates
(lvprior)Cumulates variational posterior for each gate-node.
normalize
()Normalizes the variational posterior matrix so that the sum of samples for each row is 1.0.
- cumulate_vposterior_on_gates(lvprior)¶
Cumulates variational posterior for each gate-node.
The posterior will be calculated based on the variational posterior and tree-structure of the prior. The threshold value and the probability of gating-function in the prior will not be used.
- Parameters:
- lvpriorHMEBinaryTreeLVPrior
Latent variable prior object for FAB/HME.
- Returns:
- vposterior_on_gatesnp.ndarray, size = (num_samples, num_gates)
Variational posterior on each gate-node.
fab.hme.learner.hme_context module¶
- class fab.hme.learner.hme_context.HMELearningContext(num_comps)¶
Bases:
LearningContext
Context class for FAB/HME learning.
It holds context information in the HME/FAB learning process.
- Parameters:
- num_compsint
Number of components at the initial state.
- Attributes:
- fic_historylist[float], size = (num_fab_steps)
History of FIC values.
- num_comps_historylist[float], size = (num_fab_steps)
History of the number of remained (un-shrunk) components.
- is_exist_loglikelihood_compwisebool
The flag whether component-wise log-likelihood is calculated.
- loglikelihood_compwisenp.ndarray, size = (num_samples, num_comps)
Component-wise log-likelihood values for each sample. For these values, log-likelihood of both components and the prior are considered.
- is_exist_gate_fisher_coeffsbool
The flag whether Fisher coefficients for gates are calculated.
- gate_fisher_coeffsfloat or np.ndarray, size = (num_samples, num_gates)
If 1.0 [default], the number of expected samples is simply applied. For calculating scaled FIC value of gate complexity penalty, coefficient matrix or scalar value to vposterior must be given.
- is_exist_comp_fisher_coeffsbool
The flag whether Fisher coefficients for components are calculated.
- comp_fisher_coeffsfloat or np.ndarray, size = (num_samples, num_comps)
Same as gate_fisher_coeffs.
- within_post_processbool
The flag whether the FAB-iterations is in the post-processing
Methods
append
(fic, num_comps)Appends FIC and the number of components to corresponding history variables.
post_process_context
- property num_fab_steps¶
- Returns:
- num_fab_stepsint
Executed FAB-iterations steps. Range = [0, inf).
- post_process_context()¶
fab.hme.learner.hme_data module¶
- class fab.hme.learner.hme_data.HMESupervisedData(X, Y, feature_ids, gate_feature_ids=None, comp_feature_ids=None, mandatory_feature_mask=None, constraint_signs=None)¶
Bases:
SupervisedData
Input-data container class for FAB/HME supervised learnings.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.array, size = (num_samples)
np.ndarray, size = (num_samples, num_targets) Input target data.
- feature_idslist[int]
Feature ID number for each feature.
- gate_feature_idsNone or list[int], optional [default: None]
List of indices of feature data used for optimization of gate parameters. If None, all indices will be used.
- comp_feature_idsNone or list[int], optional [default: None]
List of indices of feature data used for optimization of component parameters. If None, all indices will be used.
- mandatory_feature_maskNone or np.array(bool), optional [default: None]
The mask of the mandatory relevant features for component. size = (num_features).
- constraint_signsNone or np.array({1, 0, -1}), size = (num_features)
Sign constraints for weight values. For instance, 1 means positive constraint. 0 means no-constraint.
- Attributes:
num_features
Returns the number of features.
num_samples
Returns the number of samples.
num_targets
Returns ——- num_targets : int Number of targets.
fab.hme.learner.hme_logit_gate_bspline_cl_learner module¶
- class fab.hme.learner.hme_logit_gate_bspline_cl_learner.HMELogitGateBSplineClLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, comp_bspline_degree, comp_bspline_basis_dim, comp_bspline_knot_vecs, gate_opt_mode, gate_opt_type, gate_l2_regularize, with_gate_scaled_l0_regularize, max_gate_relevant_features, gate_svd_threshold, comp_opt_mode, post_comp_opt_type, comp_l2_regularize, comp_pspline, with_comp_scaled_l0_regularize, max_comp_relevant_features, comp_svd_threshold, num_threads_gates, num_threads_comps)¶
Bases:
HMELogitGateMixin
,HMEBinaryTreeBSplineClMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME B-spline non-linear classification learner with logistic-gates.
Notes
Use init_random() or init_with_posterior() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all logistic-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME logistic gates B-spline non-linear classification learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME logistic gates B-spline non-linear classification learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME logistic gates B-spline non-linear classification learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of B-spline non-linear classification components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior with logistic-gates.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, tree_depth=5, comp_bspline_degree=3, comp_bspline_basis_dim=10, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, post_comp_opt_type='standard', comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates B-spline non-linear classification learner where the all parameters are initialized randomly.
- Parameters:
- comp_bspline_degreeint, optional [default: 3]
Degree of B-spline function.
- comp_bspline_basis_dimint, optional [default: 10]
The number of B-spline basis functions to be generated for each feature.
- gate_opt_type{‘quadratic’, ‘quadratic_svd’}, optional [default: ‘quadratic’]
Algorithm of gate parameter optimization:
- ‘quadratic’: using quadratic upper bound approximation with matrix
inversion lemma.
- ‘quadratic_svd’: using quadratic upper bound approximation with
singular value decomposition.
- gate_l2_regularizefloat, optional [default: 0.0]
L2-regularization hyper-parameter for gate-parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled.
- with_gate_scaled_l0_regularizebool, optional [default: True]
Whether scale factor is applied to the gate L0-regularize term.
- max_gate_relevant_featuresint, optional [default: 3]
Maximum number of the relevant features for each gate.
- gate_svd_thresholdfloat, optional [default: 0.00001]
Threshold value for singular value decomposition (SVD) in gate parameter optimization with FoBa-algorithm. Modes having eigen values smaller than the threshold are ignored. When gate_opt_type is not ‘quadratic_svd’, this value is ignored. Domain = [0, inf).
- post_comp_opt_type{‘standard’, ‘quadratic’}, optional
Algorithm of component parameter optimization in the post-processing. The domain and default value depends on each learner type and described in the following section. [default: ‘standard’]
- comp_psplinefloat, optional [default: 1.0]
P-spline penalized coefficient parameter for component parameter optimization.
- Returns:
- learnerHMELogitGateBSplineClLearner
An instance of FAB/HME logistic gates B-spline non-linear classification learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, gate_opt_mode='opt', gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_opt_mode='opt', post_comp_opt_type='standard', comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates B-spline non-linear classification learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features),
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMELogitGateBSplineClLearner
An instance of FAB/HME logistic gates B-spline non-linear classification learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_bspline_degree=3, comp_bspline_basis_dim=10, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, post_comp_opt_type='standard', comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates B-spline non-linear classification learner where the initial posterior distribution is given.
- Returns:
- learnerHMELogitGateBSplineClLearner
An instance of FAB/HME logistic gates B-spline non-linear classification learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
Basis function matrices are calculated and set them in the data object.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_logit_gate_bspline_rg_learner module¶
- class fab.hme.learner.hme_logit_gate_bspline_rg_learner.HMELogitGateBSplineRgLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, comp_bspline_degree, comp_bspline_basis_dim, comp_bspline_knot_vecs, gate_opt_mode, gate_opt_type, gate_l2_regularize, with_gate_scaled_l0_regularize, max_gate_relevant_features, gate_svd_threshold, comp_opt_mode, comp_l2_regularize, comp_pspline, with_comp_scaled_l0_regularize, max_comp_relevant_features, comp_svd_threshold, num_threads_gates, num_threads_comps)¶
Bases:
HMELogitGateMixin
,HMEBinaryTreeBSplineRgMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME B-spline non-linear regression learner with logistic-gates.
Notes
Use init_random() or init_with_posterior() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all logistic-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME logistic gates B-spline non-linear regression learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME logistic gates B-spline non-linear regression learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME logistic gates B-spline non-linear regression learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of B-spline non-linear regression components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior with logistic-gates.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, tree_depth=5, comp_bspline_degree=3, comp_bspline_basis_dim=10, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, comp_variance_min_scale=0.1, comp_variance_max_scale=0.25, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates B-spline non-linear regression learner where the all parameters are initialized randomly.
- Parameters:
- comp_bspline_degreeint, optional [default: 3]
Degree of B-spline function.
- comp_bspline_basis_dimint, optional [default: 10]
The number of B-spline basis functions to be generated for each feature.
- gate_opt_type{‘quadratic’, ‘quadratic_svd’}, optional [default: ‘quadratic’]
Algorithm of gate parameter optimization:
- ‘quadratic’: using quadratic upper bound approximation with matrix
inversion lemma.
- ‘quadratic_svd’: using quadratic upper bound approximation with
singular value decomposition.
- gate_l2_regularizefloat, optional [default: 0.0]
L2-regularization hyper-parameter for gate-parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled.
- with_gate_scaled_l0_regularizebool, optional [default: True]
Whether scale factor is applied to the gate L0-regularize term.
- max_gate_relevant_featuresint, optional [default: 3]
Maximum number of the relevant features for each gate.
- gate_svd_thresholdfloat, optional [default: 0.00001]
Threshold value for singular value decomposition (SVD) in gate parameter optimization with FoBa-algorithm. Modes having eigen values smaller than the threshold are ignored. When gate_opt_type is not ‘quadratic_svd’, this value is ignored. Domain = [0, inf).
- comp_psplinefloat, optional [default: 1.0]
P-spline penalized coefficient parameter for component parameter optimization.
- Returns:
- learnerHMELogitGateBSplineRgLearner
An instance of FAB/HME logistic gates B-spline non-linear regression learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, gate_opt_mode='opt', gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_opt_mode='opt', comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates B-spline non-linear regression learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMELogitGateBSplineRgLearner
An instance of FAB/HME logistic gates B-spline non-linear regression learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_bspline_degree=3, comp_bspline_basis_dim=10, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_l2_regularize=0.0, comp_pspline=1.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates B-spline non-linear regression learner where the initial posterior distribution is given.
- Returns:
- learnerHMELogitGateBSplineRgLearner
An instance of FAB/HME logistic gates B-spline non-linear regression learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_logit_gate_logit_rg_learner module¶
- class fab.hme.learner.hme_logit_gate_logit_rg_learner.HMELogitGateLogisticRgLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, comp_positive_feature_ids, comp_negative_feature_ids, gate_opt_mode, gate_opt_type, gate_l2_regularize, with_gate_scaled_l0_regularize, max_gate_relevant_features, gate_svd_threshold, comp_foba_skip, comp_foba_skip_max_interval, comp_opt_mode, comp_backward_step, comp_opt_type, post_comp_opt_type, comp_l2_regularize, with_comp_scaled_l0_regularize, max_comp_relevant_features, max_comp_foba_iterations, comp_svd_threshold, num_threads_gates, num_threads_comps)¶
Bases:
HMELogitGateMixin
,HMEBinaryTreeLogisticRgMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME logistic regression learner with logistic-gates.
Notes
Use init_random() or init_with_posterior() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all logistic-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME logistic gates logistic regression learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME logistic gates logistic regression learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME logistic gates logistic regression learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of logistic regression components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior with logistic-gates.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, tree_depth=5, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_backward_step=False, comp_opt_type='quadratic', post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates logistic regression learner where the all parameters are initialized randomly.
- Parameters:
- gate_opt_type{‘quadratic’, ‘quadratic_svd’}, optional [default: ‘quadratic’]
Algorithm of gate parameter optimization:
- ‘quadratic’: using quadratic upper bound approximation with matrix
inversion lemma.
- ‘quadratic_svd’: using quadratic upper bound approximation with
singular value decomposition.
- gate_l2_regularizefloat, optional [default: 0.0]
L2-regularization hyper-parameter for gate-parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled.
- with_gate_scaled_l0_regularizebool, optional [default: True]
Whether scale factor is applied to the gate L0-regularize term.
- max_gate_relevant_featuresint, optional [default: 3]
Maximum number of the relevant features for each gate.
- gate_svd_thresholdfloat, optional [default: 0.00001]
Threshold value for singular value decomposition (SVD) in gate parameter optimization with FoBa-algorithm. Modes having eigen values smaller than the threshold are ignored. When gate_opt_type is not ‘quadratic_svd’, this value is ignored. Domain = [0, inf).
- comp_opt_type{‘quadratic’, ‘standard’, ‘quadratic_svd’}, optional
[default: ‘quadratic’] Algorithm of component parameter optimization:
‘standard’: using logit function explicity; liblinear-weights.
- ‘quadratic’: using quadratic upper bound approximation with matrix
inversion lemma.
- ‘quadratic_svd’: using quadratic upper bound approximation with
singular value decomposition.
- post_comp_opt_type{‘standard’, ‘quadratic’}, optional
[default: ‘standard’] Algorithm of component parameter optimization in post-processing.
- Returns:
- learnerHMELogitGateLogisticRgLearner
An instance of FAB/HME logistic gates logistic regression learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_opt_mode='opt', gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_opt_mode='opt', comp_backward_step=False, comp_opt_type='quadratic', post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates logistic regression learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMELogitGateLogisticRgLearner
An instance of FAB/HME logistic gates logistic regression learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_backward_step=False, comp_opt_type='quadratic', post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates logistic regression learner where the initial posterior distribution is given.
- Returns:
- learnerHMELogitGateLogisticRgLearner
An instance of FAB/HME logistic gates logistic regression learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_logit_gate_lsqr_rg_learner module¶
- class fab.hme.learner.hme_logit_gate_lsqr_rg_learner.HMELogitGateLeastSquaresRgLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, comp_positive_feature_ids, comp_negative_feature_ids, gate_opt_mode, gate_opt_type, gate_l2_regularize, with_gate_scaled_l0_regularize, max_gate_relevant_features, gate_svd_threshold, comp_foba_skip, comp_foba_skip_max_interval, comp_opt_mode, comp_two_stage_opt, comp_backward_step, comp_opt_type, comp_l2_regularize, with_comp_scaled_l0_regularize, max_comp_relevant_features, max_comp_foba_iterations, comp_svd_threshold, num_threads_gates, num_threads_comps)¶
Bases:
HMELogitGateMixin
,HMEBinaryTreeLeastSquaresRgMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME least-squares regression learner with logistic-gates.
Notes
Use init_random() or init_with_posterior() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all logistic-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME logistic gates least-squares regression learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME logistic gates least-squares regression learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME logistic gates least-squares regression learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of least-squares regression components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior with logistic-gates.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, tree_depth=5, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, comp_variance_min_scale=0.1, comp_variance_max_scale=0.25, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_two_stage_opt=False, comp_backward_step=False, comp_opt_type='mil', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates least-squares regression learner where the all parameters are initialized randomly.
- Parameters:
- gate_opt_type{‘quadratic’, ‘quadratic_svd’}, optional [default: ‘quadratic’]
Algorithm of gate parameter optimization:
- ‘quadratic’: using quadratic upper bound approximation with matrix
inversion lemma.
- ‘quadratic_svd’: using quadratic upper bound approximation with
singular value decomposition.
- gate_l2_regularizefloat, optional [default: 0.0]
L2-regularization hyper-parameter for gate-parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled.
- with_gate_scaled_l0_regularizebool, optional [default: True]
Whether scale factor is applied to the gate L0-regularize term.
- max_gate_relevant_featuresint, optional [default: 3]
Maximum number of the relevant features for each gate.
- gate_svd_thresholdfloat, optional [default: 0.00001]
Threshold value for singular value decomposition (SVD) in gate parameter optimization with FoBa-algorithm. Modes having eigen values smaller than the threshold are ignored. When gate_opt_type is not ‘quadratic_svd’, this value is ignored. Domain = [0, inf).
- Returns:
- learnerHMELogitGateLeastSquaresRgLearner
An instance of FAB/HME logistic gates least-squares regression learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_opt_mode='opt', gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_opt_mode='opt', comp_two_stage_opt=False, comp_backward_step=False, comp_opt_type='mil', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates least-squares regression learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMELogitGateLeastSquaresRgLearner
An instance of FAB/HME logistic gates least-squares regression learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, comp_positive_feature_ids=None, comp_negative_feature_ids=None, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_two_stage_opt=False, comp_backward_step=False, comp_opt_type='mil', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates least-squares regression learner where the initial posterior distribution is given.
- Returns:
- learnerHMELogitGateLeastSquaresRgLearner
An instance of FAB/HME logistic gates least-squares regression learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_logit_gate_softmax_cl_learner module¶
- class fab.hme.learner.hme_logit_gate_softmax_cl_learner.HMELogitGateSoftmaxClLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence, projection_estep, shrink_threshold, fab_stop_threshold, hard_gate, model_initializer, gate_feature_ids, comp_feature_ids, comp_mandatory_feature_ids, gate_opt_mode, gate_opt_type, gate_l2_regularize, with_gate_scaled_l0_regularize, max_gate_relevant_features, gate_svd_threshold, comp_foba_skip, comp_foba_skip_max_interval, comp_opt_mode, comp_backward_step, post_comp_opt_type, comp_l2_regularize, with_comp_scaled_l0_regularize, max_comp_relevant_features, max_comp_foba_iterations, comp_svd_threshold, num_threads_gates, num_threads_comps)¶
Bases:
HMELogitGateMixin
,HMEBinaryTreeSoftmaxClMixin
,HMEBinaryTreeSupervisedLearner
FAB/HME softmax classification learner with logistic-gates.
Notes
Use init_random() or init_with_posterior() to create an instance.
Methods
binarize_lvprior
(model, data, context)Binarizes all logistic-gate functions in the latent variable prior.
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of gates.
calc_loglikelihood_compwise
(model, data, context)Calculates the component-wise log-likelihood values.
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
init_random
([max_fab_iterations, ...])Creates an instance of FAB/HME logistic gates softmax classification learner where the all parameters are initialized randomly.
init_with_model_dict
(model_dict[, ...])Creates an instance of FAB/HME logistic gates softmax classification learner where the information on FAB/HME supervised model is given.
init_with_posterior
(posterior_prob, comp_ids)Creates an instance of FAB/HME logistic gates softmax classification learner where the initial posterior distribution is given.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
make_hard_gates
(model, vposterior, data, context)Converts soft-gates to hard-gates by modifying gate parameters.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of softmax classification components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior with logistic-gates.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
update_vposterior_plain
(model, vposterior, ...)Updates the variational posterior distribution by the plain method.
update_vposterior_projection
(model, ...)Calculates the variational posterior distribution using projection E-step algorithm.
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
- Parameters:
- modelHMESupervisedModel
Model whose parameters are optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- data: HMESupervisedData
Input data.
- contextHMELearningContext
Context information.
- Returns:
- None
- classmethod init_random(max_fab_iterations=100, start_from_mstep=False, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, tree_depth=5, comp_weights_min_scale=-0.5, comp_weights_max_scale=0.5, comp_bias_min_scale=0.25, comp_bias_max_scale=0.75, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_backward_step=False, post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates softmax classification learner where the all parameters are initialized randomly.
- Parameters:
- gate_opt_type{‘quadratic’, ‘quadratic_svd’}, optional [default: ‘quadratic’]
Algorithm of gate parameter optimization:
- ‘quadratic’: using quadratic upper bound approximation with matrix
inversion lemma.
- ‘quadratic_svd’: using quadratic upper bound approximation with
singular value decomposition.
- gate_l2_regularizefloat, optional [default: 0.0]
L2-regularization hyper-parameter for gate-parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled.
- with_gate_scaled_l0_regularizebool, optional [default: True]
Whether scale factor is applied to the gate L0-regularize term.
- max_gate_relevant_featuresint, optional [default: 3]
Maximum number of the relevant features for each gate.
- gate_svd_thresholdfloat, optional [default: 0.00001]
Threshold value for singular value decomposition (SVD) in gate parameter optimization with FoBa-algorithm. Modes having eigen values smaller than the threshold are ignored. When gate_opt_type is not ‘quadratic_svd’, this value is ignored. Domain = [0, inf).
- post_comp_opt_type{‘standard’, ‘quadratic’}, optional
[default: ‘standard’] Algorithm of component parameter optimization in post-processing.
- Returns:
- learnerHMELogitGateSoftmaxClLearner
An instance of FAB/HME logistic gates softmax classification learner.
Notes
For other parameters, please refer to ones of
HMEBernGateLeastSquaresRgLearner
class.
- classmethod init_with_model_dict(model_dict, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, gate_opt_mode='opt', gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_opt_mode='opt', comp_backward_step=False, post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates softmax classification learner where the information on FAB/HME supervised model is given.
Initial model is restored from model_dict. The FAB-iteration will be started from E-step in order to use the specified initial model.
- Parameters:
- model_dict: dict
Information on FAB/HME supervised model.
- gate_opt_mode{‘opt’, ‘refit’, ‘keep’}
optional [default: ‘opt’] Mode of gate parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
‘keep’: keeping all parameter values.
- comp_opt_mode{‘opt’, ‘refit’}
optional [default: ‘opt’] Mode of component parameter optimization:
‘opt’: optimizing with all features (selecting and fitting the features).
‘refit’: only fitting with relevant features.
- Returns:
- learnerHMELogitGateSoftmaxClLearner
An instance of FAB/HME logistic gates softmax classification learner.
Notes
For other parameters, please refer to ones of
init_random()
method.
- classmethod init_with_posterior(posterior_prob, comp_ids, max_fab_iterations=100, num_acceleration_steps=0, repeat_until_convergence=False, projection_estep=False, shrink_threshold=1.0, fab_stop_threshold=0.001, hard_gate=True, gate_feature_ids=None, comp_feature_ids=None, comp_mandatory_feature_ids=None, gate_opt_type='quadratic', gate_l2_regularize=0.0, with_gate_scaled_l0_regularize=True, max_gate_relevant_features=3, gate_svd_threshold=1e-05, comp_foba_skip='power_of_two', comp_foba_skip_max_interval=25, comp_backward_step=False, post_comp_opt_type='standard', comp_l2_regularize=0.0, with_comp_scaled_l0_regularize=True, max_comp_relevant_features=100, max_comp_foba_iterations=100, comp_svd_threshold=1e-05, num_threads_gates=1, num_threads_comps=1)¶
Creates an instance of FAB/HME logistic gates softmax classification learner where the initial posterior distribution is given.
- Returns:
- learnerHMELogitGateSoftmaxClLearner
An instance of FAB/HME logistic gates softmax classification learner.
Notes
For parameters, please refer to ones of
init_random()
method andHMEBernGateLeastSquaresRgLearner
class.
- setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples, num_targets)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
fab.hme.learner.hme_supervised_learner module¶
- class fab.hme.learner.hme_supervised_learner.HMESupervisedLearner(max_fab_iterations, start_from_mstep, num_acceleration_steps, repeat_until_convergence)¶
Bases:
object
A Base class of learners for FAB/HME supervised learning.
An interface method
learn(X, Y)
is defined in this class, and it requires sub-classes to implement some methods which are declared as abstract methods in this class.Methods
do_fab_iterations
(model, vposterior, data, ...)Executes FAB iterations.
do_fab_single
(model, vposterior, data, context)Executes the single component optimization.
do_mstep
(model, vposterior, data, context)Executes M-step.
do_post_process
(model, vposterior, data, context)Executes post processing.
finalize
(model, vposterior, data, context)Executes finalizing processes of the learning.
init_model
(data)Creates model, variational posterior, and context objects and initializes their parameter values.
is_stop
(model, vposterior, data, context)Judges whether the FAB iteration is converged.
learn
(X, Y)Executes a learning process.
log_stats
(context)Records statistics of the FAB-iterations.
operate_shrinkage
(model, vposterior, data, ...)Operates shrinkage process.
opt_comps
(model, vposterior, data, context)Optimizes parameters of components.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior.
setup_data
(X, Y)Sets up data.
update_context_fic
(model, vposterior, data, ...)Calculates the FIC value.
update_vposterior
(model, vposterior, data, ...)Updates the variational posterior distribution.
- do_fab_iterations(model, vposterior, data, context)¶
Executes FAB iterations.
This method behaves as a generator, therefore it should be kept being called repeatedly until a StopIteration exception is raised.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- do_fab_single(model, vposterior, data, context)¶
Executes the single component optimization.
This is a learning process if at most one component remains after shrinkage mechanism. If multiple components do not exist, FAB- iterations is not suitable.
- Process to optimize a single component is repeating the follows:
optimize the component parameters
calculate FIC value
check stopping condition of FIC convergence
- The following conditions must be satisfied before calling this method:
model has a component: len(model.comps) = 1
vposterior is for a component: vposterior.prob.shape[1] = 1
lvprior has no gate node: type(lvprior.root_node) = BinaryTreeComponentNode
- Parameters:
- modelHMESupervisedModel
Model whose parameter values will be optimized.
- vposteriorHMEVPosterior
Variational posterior. The column size of the matrix must be 1.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- do_mstep(model, vposterior, data, context)¶
Executes M-step.
Acceleration algorithm is supported in this method.
- Parameters:
- modelHMESupervisedModel
Model whose parameter values will be optimized.
- vposteriorHMEVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- abstract do_post_process(model, vposterior, data, context)¶
Executes post processing.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- abstract finalize(model, vposterior, data, context)¶
Executes finalizing processes of the learning.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- abstract init_model(data)¶
Creates model, variational posterior, and context objects and initializes their parameter values.
- Parameters:
- dataHMESupervisedData
Input data.
- Returns:
- modelHMESupervisedModel
Initialized model object.
- vposteriorHMEVPosterior
Initialized variational posterior.
- contextHMELearningContext
Initialized context object.
- abstract is_stop(model, vposterior, data, context)¶
Judges whether the FAB iteration is converged.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object to be used to be judged for convergence.
- Returns:
- is_stopbool
If True, the FAB iteration is converged, thus fulfilling the stopping condition.
- learn(X, Y)¶
Executes a learning process.
- This method generates a model using input data with FAB algorithm:
initialize of the model and the variational posterior
perform FAB-iterations
execute post-processing
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- modelHMESupervisedModel
Model whose parameters have been optimized.
- vposteriorHMEVPosterior
Variational posterior which has been used in the learning process.
- contextHMELearningContext
Context object. Statistics of the learning process are contained in the object: the history of FIC values, etc.
- log_stats(context)¶
Records statistics of the FAB-iterations.
- Parameters:
- contextHMELearningContext
Context object in which FIC and the number of components on each iteration are stored.
- Returns:
- None
- abstract operate_shrinkage(model, vposterior, data, context)¶
Operates shrinkage process.
The model (components and tree structure of the latent variable prior) and variational posterior (the number of columns of probability matrix) are updated accompanying with shrinkage operation.
- Parameters:
- modelHMESupervisedModel
Model object to be updated.
- vposteriorHMEVPosterior
Variational posterior to be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- abstract opt_comps(model, vposterior, data, context)¶
Optimizes parameters of components.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of components will be updated.
- vposteriorHMEVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- abstract opt_lvprior(model, vposterior, data, context)¶
Optimizes parameters of a latent variable prior.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of the latent variable prior will be updated.
- vposteriorHMEVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
- abstract setup_data(X, Y)¶
Sets up data.
- Parameters:
- Xnp.ndarray, size = (num_samples, num_features)
Input feature data.
- Ynp.ndarray, size = (num_samples,)
Input target data.
- Returns:
- dataHMESupervisedData
Input feature and target data set.
- abstract update_context_fic(model, vposterior, data, context)¶
Calculates the FIC value.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object in which the newly calculated FIC value and the number of components will be appended.
- Returns:
- None
- abstract update_vposterior(model, vposterior, data, context)¶
Updates the variational posterior distribution.
- Parameters:
- modelHMESupervisedModel
Model object.
- vposteriorHMEVPosterior
Variational posterior to be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context object.
- Returns:
- None
fab.hme.learner.hme_vposterior module¶
- class fab.hme.learner.hme_vposterior.HMEVPosterior(prob)¶
Bases:
object
An abstract class for the variational posterior of FAB/HME supervised models.
This class holds a variational posterior matrix. It also provides methods to calculate posterior on each gate-node and branches. The calculation procedure depends on the type of the latent variable prior, therefore specific processes must be implemented in sub-classes.
- Parameters:
- probnp.ndarray, size = (num_samples, num_comps), optional
Posterior matrix. [default: None]
Methods
cumulate_vposterior_on_gates
(lvprior)Cumulates variational posterior for each gate-node.
Normalizes the variational posterior matrix so that the sum of samples for each row is 1.0.
- abstract cumulate_vposterior_on_gates(lvprior)¶
Cumulates variational posterior for each gate-node.
The posterior will be calculated based on the variational posterior and tree-structure of the prior. The threshold value and the probability of gating-function in the prior will not be used.
- Parameters:
- lvpriorHMELVPrior
Latent variable prior. Its tree structure is used for the calculation.
- Returns:
- vposterior_on_gatesnp.ndarray, size = (num_samples, num_gates)
Variational posterior on each gate-node.
- normalize()¶
Normalizes the variational posterior matrix so that the sum of samples for each row is 1.0.
- Returns:
- vposteriorHMEVPosterior
This object.
fab.hme.learner.logit_gate_mixin module¶
- class fab.hme.learner.logit_gate_mixin.HMELogitGateMixin¶
Bases:
object
Mix-in class for the latent variable prior with logistic-gates.
This mix-in is add a feature of the latent variable prior with logistic-gates for concrete classes of HMEBinaryTreeSupervisedLearner. The learner inheriting this mix-in must define self.gate_opt_params as a dict type in which ‘with_scaled_l0_regularize’ is contained.
Methods
binarize_lvprior
(model, data, context)Binarizes all logistic-gate functions in the latent variable prior.
calc_gate_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of gates.
opt_lvprior
(model, vposterior, data, context)Optimizes parameters of a latent variable prior with logistic-gates.
- binarize_lvprior(model, data, context)¶
Binarizes all logistic-gate functions in the latent variable prior.
- Parameters:
- modelHMESupervisedModel
Model whose parameters will be updated.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- None
- calc_gate_fisher_coeffs(model, data, context)¶
Calculates the Fisher coefficients of gates.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- None
- opt_lvprior(model, vposterior, data, context)¶
Optimizes parameters of a latent variable prior with logistic-gates.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of the latent variable prior are updated. It must have HMEBernGateLVPrior as a prior.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context information. Some pre-calculated properties at other processes could be used if needed.
- Returns:
- None
fab.hme.learner.logit_rg_mixin module¶
- class fab.hme.learner.logit_rg_mixin.HMEBinaryTreeLogisticRgMixin¶
Bases:
object
Mix-in class for logistic regressions.
This mix-in is add a feature of logistic regression component for concrete classes of HMEBinaryTreeSupervisedLearner. The learner inheriting this mix-in must define self.comp_opt_params as a dict type in which ‘with_scaled_l0_regularize’ is contained.
Methods
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
opt_comps
(model, vposterior, data, context)Optimizes parameters of logistic regression components.
- calc_comp_fisher_coeffs(model, data, context)¶
Calculates the Fisher coefficients of components.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- comp_fisher_coeffsfloat or np.ndarray,
size = (num_samples, num_comps)
If 1.0 [default], the number of expected samples is simply applied. For calculating scaled FIC value of component complexity penalty, coefficient matrix or scalar value to vposterior must be given.
- opt_comps(model, vposterior, data, context)¶
Optimizes parameters of logistic regression components.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of components are updated.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context information. Some pre-calculated properties at other processes could be used if needed.
- Returns:
- None
fab.hme.learner.lsqr_rg_mixin module¶
- class fab.hme.learner.lsqr_rg_mixin.HMEBinaryTreeLeastSquaresRgMixin¶
Bases:
object
Mix-in class for least-squares regressions.
This mix-in is add a feature of least-squares regression component for concrete classes of HMEBinaryTreeSupervisedLearner. The learner inheriting this mix-in must define self.comp_opt_params as a dict type in which ‘with_scaled_l0_regularize’ is contained.
Methods
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
opt_comps
(model, vposterior, data, context)Optimizes parameters of least-squares regression components.
- calc_comp_fisher_coeffs(model, data, context)¶
Calculates the Fisher coefficients of components.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- comp_fisher_coeffsfloat or np.ndarray,
size = (num_samples, num_comps)
If 1.0 [default], the number of expected samples is simply applied. For calculating scaled FIC value of component complexity penalty, coefficient matrix or scalar value to vposterior must be given.
- opt_comps(model, vposterior, data, context)¶
Optimizes parameters of least-squares regression components.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of components are updated.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context information. Some pre-calculated properties at other processes could be used if needed.
- Returns:
- None
fab.hme.learner.softmax_cl_mixin module¶
- class fab.hme.learner.softmax_cl_mixin.HMEBinaryTreeSoftmaxClMixin¶
Bases:
object
Mix-in class for softmax classification.
This mix-in is add a feature of softmax classification component for concrete classes of HMEBinaryTreeSupervisedLearner. The learner inheriting this mix-in must define self.comp_opt_params as a dict type in which ‘with_scaled_l0_regularize’ is contained.
Methods
calc_comp_fisher_coeffs
(model, data, context)Calculates the Fisher coefficients of components.
opt_comps
(model, vposterior, data, context)Optimizes parameters of softmax classification components.
- calc_comp_fisher_coeffs(model, data, context)¶
Calculates the Fisher coefficients of components.
- Parameters:
- modelHMESupervisedModel
Model object.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context to be updated.
- Returns:
- comp_fisher_coeffsfloat or np.ndarray,
size = (num_samples, num_targets, num_comps)
If 1.0 [default], the number of expected samples is simply applied. For calculating scaled FIC value of component complexity penalty, coefficient matrix or scalar value to vposterior must be given.
- opt_comps(model, vposterior, data, context)¶
Optimizes parameters of softmax classification components.
- Parameters:
- modelHMESupervisedModel
Model object in which parameter values of components are updated.
- vposteriorHMEBinaryTreeVPosterior
Variational posterior.
- dataHMESupervisedData
Input data.
- contextHMELearningContext
Context information. Some pre-calculated properties at other processes could be used if needed.
- Returns:
- None