FABHMELogitGateLinearCl Component Specification¶
Contents
Overview¶
FABHMELogitGateLinearCl component is a linear binary classification component with FAB/HME algorithm. This component learns a tree-structured model in which each sample is assigned to a component according to Logistic gating functions.
Note
FAB engine uses the word ‘component’ with a different meaning from that of SAMPO. Each component in FAB/HME is a prediction formula, and each sample data is assigned to a specific component for prediction.
Example:
SPD:
# fabhmecl.spd dl1 -> fab1 --- components: dl1: component: DataLoader fab1: component: FABHMELogitGateLinearClComponent features: name != 'class' tree_depth: 3 target: name == 'class' positive_label: 'Iris-setosa' global_settings: keep_attributes: - class feature_exclude: - class
Input of the component:
_sid
sepal_length_ in_cm
sepal_width_ in_cm
class
0
4.9
2.5
Iris-versicolor
1
6.2
2.8
Iris-versicolor
2
7.2
3.6
Iris-versicolor
…
…
…
…
28
6.2
2.9
Iris-setosa
29
6.7
3.1
Iris-setosa
Output of the component:
_sid
fab1_ actual
fab1_ predict
fab1_score
fab1_ assigned_comp_id
0
-1
1
2.657069e+00
2
1
-1
1
6.524541e-01
2
2
-1
-1
-1.600153e+00
0
…
…
…
…
…
28
1
1
6.524541e-01
2
29
1
-1
-1.080094e+00
0
_sid
fab1_ predict_c0
fab1_ score_c0
fab1_ predict_c1
fab1_ score_c1
fab1_ predict_c2
fab1_ score_c2
0
1
7.921206e-01
-1
-1.028756e+00
1
2.657069e+00
1
-1
-5.600341e-01
-1
-2.346818e+01
1
6.524541e-01
2
-1
-1.600153e+00
-1
-1.082974e+01
-1
-8.895575e-01
…
…
…
…
…
…
…
28
-1
-5.600341e-01
-1
-1.821556e+01
1
6.524541e-01
29
-1
-1.080094e+00
-1
-2.240158e+01
-1
-1.185517e-01
This component has component-specific external formats for model and prediction result evaluation.
See also
Component-common external format files in convert_process
Parameters¶
This component has the following component-specific parameters.
SPD¶
The following parameters are for “components” section of SPD.
Parameter Name |
Type |
Domain |
Default Value |
Description |
---|---|---|---|---|
positive_label 1 |
str |
See Description |
– |
A value chosen from the target attributes to be set as positive label. The domain of this parameter corresponds to that of the target attribute. |
max_fab_iterations |
int |
[1, inf) |
100 |
Maximum number of FAB-iterations. |
bool |
True / False |
False |
If True, the first iteration starts with M-step; otherwise, E-step. |
|
num_acceleration_steps |
int |
[0, inf) |
0 |
The number of steps of acceleration algorithm for each FAB-iteration. If 0, the acceleration algorithm is disabled. |
repeat_until_convergence |
bool |
True / False |
False |
If False, FAB-iterations and the post-processing are executed only once
even if the FAB-iterations are stopped not by convergence condition but
by |
projection_estep |
bool |
True / False |
False |
Whether the projection E-step algorithm is enabled. |
shrink_threshold |
float or str |
[1, inf) or (0%, 100%) |
1.0 |
Threshold value for shrinkage. If a percentage value (e.g. |
fab_stop_threshold |
float or str |
(0, inf) or (0%, inf%) |
0.001 |
Threshold value for FAB-iterations: if the increase of FIC value
is less than the threshold, the FAB-iterations is considered to
be converged. If a percentage value (e.g. |
gate_features |
str |
Query format |
all() |
Features which are applied to gate parameter optimizations. If not specified, all features are used. |
comp_features |
str |
Query format |
all() |
Features which are applied to component parameter optimizations. If not specified, all features are used. If empty, the model is learned as a decision tree. |
comp_mandatory_features |
str |
Query format |
See Description |
Features which non-L0-regularize constraints are applied to. It means the specified features will always be relevant for all components. If not specified, no features are specified for non-L0-regularization, which implies all relevant features are selected by FoBa algorithm. |
comp_positive_features |
str |
Query format |
See Description |
Features whose weight values for all components are constrained to positive values. If not specified, all features are optimized with no constraints. |
comp_negative_features |
str |
Query format |
See Description |
Features whose weight values for all components are constrained to negative values. If not specified, all features are optimized with no constraints. |
int |
[0, inf) |
5 |
Initial depth of the gate-tree structure of latent variable prior. The initial number of components is \(2^d\) where \(d\) is tree depth. If 0, the optimization with only one component will be executed. |
|
float |
(-inf, inf) |
-0.5 |
Scale value for the initialization of weight values of components. |
|
float |
(-inf, inf) |
0.5 |
Scale value for the initialization of weight values of components. |
|
float |
(-inf, inf) |
0.25 |
Scale value for the initialization of bias values of components. |
|
float |
(-inf, inf) |
0.75 |
Scale value for the initialization of bias values of components. |
|
gate_l2_regularize |
float |
[0, inf) |
0.0 |
L2-regularization hyper-parameter for gate-parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled. |
with_gate_scaled_l0_regularize |
bool |
True / False |
True |
Whether with scaled L0-regularization using a tighter lower bound of FIC for gate parameter optimization; approximation of det(F) is refined, where F is a Fisher matrix. |
max_gate_relevant_features |
int |
[1, inf) |
3 |
Maximum number of the relevant features for each gate. |
comp_foba_skip |
str |
{‘power_of_two’, ‘quarter_square’, ‘none’} |
‘power_of_two’ |
The judging function type for the FoBa algorithm skipping. If ‘none’, FoBa is executed for all FAB-iteration steps. FoBa is skipped at \({\rm log}_{2}t \ne {\rm ceil}({\rm log}_{2}t)\) if ‘power_of_two’, or \(t \bmod {\rm ceil}(\sqrt{t}) \ne 0\) if ‘quarter_square’. \(t\) is FAB-iteration step index (\(t\) starts from 1). |
comp_foba_skip_max_interval |
int |
[2, inf) |
25 |
The maximum interval for the FoBa algorithm skipping. If comp_foba_skip is ‘none’, this value is ignored. |
comp_backward_step |
bool |
True / False |
False |
Whether the backward-steps of FoBa algorithm are enabled. In the post-process, backward-steps are carried out regardless of this argument value. |
comp_l2_regularize |
float |
[0, inf) |
0.0 |
L2-regularization hyper-parameter for component parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled. |
with_comp_scaled_l0_regularize |
bool |
True / False |
True |
Whether with scaled L0-regularization using a tighter lower bound of FIC for component parameter optimization; approximation of det(F) is refined, where F is a Fisher matrix. |
max_comp_relevant_features |
int |
[1, inf) |
100 |
Maximum number of the relevant features for each component. |
max_comp_foba_iterations |
int |
[1, inf) |
100 |
Maximum number of the FoBa-iterations for each component. |
num_threads_gates |
int |
[1, inf) |
1 |
Maximum number of OpenMP threads of gate parameter optimization where tasks for all gates are divided into. |
num_threads_comps |
int |
[1, inf) |
1 |
Maximum number of OpenMP threads of component parameter optimization. |
SRC¶
The following parameter is for “hotstart” section of SRC.
Parameter Name |
Type |
Domain |
Default Value |
Description |
---|---|---|---|---|
type |
str |
{‘posterior’, ‘mh_refit_comp’, ‘mh_opt_comp’, ‘mh_refit_gate_and_refit_comp’, ‘mh_refit_gate_and_opt_comp’, ‘mh_opt_gate_and_opt_comp’} |
The hot-start type. If ‘posterior’, FAB learns with posterior hot-start which use the initial model whose tree structure is generated by base model and data. Each gate and component parameters are initialized randomly. ‘mh_XXX’ means FAB learns with model hot-start which uses base model as initial model. ‘refit_{gate, comp}’ means refitting the gate functions or prediction formulas with current data. ‘opt_{gate, comp}’ means optimizing (feature selection and fitting) the gate functions or prediction formulas with current data. |
Utilizable Sample Metadata¶
Warning
_fabhme_assigned_comp_id is deprecated. Use hotstart section of SRC instead of _fabhme_assigned_comp_id data column.
This component can utilize the _fabhme_assigned_comp_id attribute of the sample metadata to hot-start with posterior. When the attribute _fabhme_assigned_comp_id attribute is specified in the input data, this component will start the FAB/HME algorithm with the _fabhme_assigned_comp_id attribute values as its initial posterior.
To create the attribute _fabhme_assigned_comp_id, see the specification of the command sampo_ps_fabhme export_assigned_comp_id.
Output Attributes¶
This component generates the following attributes.
Attribute Name |
Scale |
Description |
---|---|---|
<component_id>_actual |
INTEGER |
Values of target attribute. |
<component_id>_predict |
INTEGER |
Predicted values. |
<component_id>_score |
REAL |
A prediction result is obtained by classifying this value according to a boundary. |
<component_id>_assigned_comp_id |
INTEGER |
Component IDs assigned by gating functions. |
<component_id>_predict_c<hme_comp_id> |
INTEGER |
Predicted values for the prediction formula of component id, <hme_comp_id>. |
<component_id>_score_c<hme_comp_id> |
REAL |
Score values for the prediction formula of component id, <hme_comp_id>. |
These attributes are in the component output data. These can be loaded in SAMPO API.
See also
Obtaining process results via ProcessResultLoader.
When convert_process is executed, the component output data will be saved in <component_id>_predict_result.csv.
This file describes the prediction result of the component.
_sid,fab1_actual,fab1_predict,fab1_score,fab1_assigned_comp_id,fab1_predict_c0,fab1_score_c0,fab1_predict_c1,fab1_score_c1,fab1_predict_c2,fab1_score_c2
0,-1,1,2.657069e+00,2,1,7.921206e-01,-1,-1.028756e+00,1,2.657069e+00
1,-1,1,6.524541e-01,2,-1,-5.600341e-01,-1,-2.346818e+01,1,6.524541e-01
2,-1,-1,-1.600153e+00,0,-1,-1.600153e+00,-1,-1.082974e+01,-1,-8.895575e-01
...
28,1,1,6.524541e-01,2,-1,-5.600341e-01,-1,-1.821556e+01,1,6.524541e-01
29,1,-1,-1.080094e+00,0,-1,-1.080094e+00,-1,-2.240158e+01,-1,-1.185517e-01
Attribute Metadata¶
The metadata of the output attributes is created with the following rules.
Context Rule¶
Attribute Name |
Context Name |
Description |
---|---|---|
All the output attributes of this component |
field_path |
List of the superordinate concepts of each output attribute based on the following hierarchical structure of the output attributes: root
├── fabhmecl
│ ├── assigned_comp_id
│ └── component
│ ├── 0
│ │ ├── predict
│ │ └── score
│ ├── 1
│ │ ├── predict
│ │ └── score
│ .
│ .
│ .
│
└── binary_classification
├── actual
├── predict
└── score
|
<component_id>_actual, <component_id>_predict, <component_id>_predict_c<hme_comp_id> |
positive_map |
Mapping between a positive value and a positive label. |
<component_id>_actual, <component_id>_predict, <component_id>_predict_c<hme_comp_id> |
negative_map |
Mapping between a negative value and a negative label. |
<component_id>_assigned_comp_id |
active_comp_ids |
List of component IDs corresponding to each prediction formula. |
Derivation Rule¶
Attribute Name |
Derived From |
---|---|
<component_id>_actual |
Derived from the target attribute. |
<component_id>_predict |
Derived from the attributes which have non-zero coefficients in any prediction formula. |
<component_id>_score |
Derived from the attributes which have non-zero coefficients in any prediction formula. |
<component_id>_assigned_comp_id |
Derived from the attributes used in the gating functions. |
<component_id>_predict_c<hme_comp_id> |
Derived from the attributes which have non-zero coefficients in the prediction formula of component id, <hme_comp_id>. |
<component_id>_score_c<hme_comp_id> |
Derived from the attributes which have non-zero coefficients in the prediction formula of component id, <hme_comp_id>. |
Example¶
{
"nodes": [
{"aid": "_sid", "name": "_sid", ... },
{"aid": "dl1[0]", "name": "sepal_length_in_cm", ... },
{"aid": "dl1[1]", "name": "sepal_width_in_cm", ... },
{"aid": "dl1[2]", "name": "petal_length_in_cm", ... },
{"aid": "dl1[3]", "name": "petal_width_in_cm", ... },
{"aid": "dl1[4]", "name": "class", ... },
{"aid": "fab1[0]", "name": "fab1_actual", "scale": "integer", "is_excluded": false,
"cid": "fab1", "cindex": 0, "values": null, "is_kept": false,
"context": {
"positive_map": {"1": ["Iris-setosa"]},
"field_path": ["binary_classification", "actual"],
"negative_map": {"-1": ["Iris-versicolor"]}
}
},
{"aid": "fab1[1]", "name": "fab1_predict", "scale": "integer", "is_excluded": false,
"cid": "fab1", "cindex": 1, "values": null, "is_kept": false,
"context": {
"positive_map": {"1": ["Iris-setosa"]},
"field_path": ["binary_classification", "predict"],
"negative_map": {"-1": ["Iris-versicolor"]}
}
},
{"aid": "fab1[2]", "name": "fab1_score", "scale": "real", "is_excluded": false,
"cid": "fab1", "cindex": 2, "values": null, "is_kept": false,
"context": {
"field_path": ["binary_classification", "score"]
}
},
{"aid": "fab1[3]", "name": "fab1_assigned_comp_id", "scale": "integer",
"is_excluded": false, "cid": "fab1", "cindex": 3, "values": null, "is_kept": false,
"context": {
"active_comp_ids": [7, 13, 17, 19, 22], "field_path": ["fabhmecl", "assigned_comp_id"]
}
},
{"aid": "fab1[4]", "name": "fab1_predict_c7", "scale": "integer", "is_excluded": false,
"cid": "fab1", "cindex": 4, "values": null, "is_kept": false,
"context": {
"positive_map": {"1": ["Iris-setosa"]},
"field_path": ["fabhmecl", "component", 7, "predict"],
"negative_map": {"-1": ["Iris-versicolor"]}
}
},
{"aid": "fab1[5]", "name": "fab1_score_c7", "scale": "real", "is_excluded": false,
"cid": "fab1", "cindex": 5, "values": null, "is_kept": false,
"context": {
"field_path": ["fabhmecl", "component", 7, "score"]
}
},
...
],
"links": [
{"source": "dl1[1]", "target": "fab1[2]"},
{"source": "dl1[1]", "target": "fab1[1]"},
{"source": "dl1[1]", "target": "fab1[3]"},
{"source": "dl1[0]", "target": "fab1[5]"},
{"source": "dl1[0]", "target": "fab1[12]"},
{"source": "dl1[0]", "target": "fab1[4]"},
{"source": "dl1[0]", "target": "fab1[8]"},
{"source": "dl1[0]", "target": "fab1[2]"},
{"source": "dl1[0]", "target": "fab1[9]"},
{"source": "dl1[0]", "target": "fab1[3]"},
{"source": "dl1[0]", "target": "fab1[6]"},
{"source": "dl1[0]", "target": "fab1[13]"},
{"source": "dl1[0]", "target": "fab1[7]"},
{"source": "dl1[0]", "target": "fab1[10]"},
{"source": "dl1[0]", "target": "fab1[1]"},
{"source": "dl1[0]", "target": "fab1[11]"},
{"source": "dl1[2]", "target": "fab1[2]"},
{"source": "dl1[2]", "target": "fab1[1]"},
{"source": "dl1[2]", "target": "fab1[3]"},
{"source": "dl1[3]", "target": "fab1[2]"},
{"source": "dl1[3]", "target": "fab1[1]"},
{"source": "dl1[3]", "target": "fab1[3]"},
{"source": "dl1[4]", "target": "fab1[0]"}
]
}
See also
Attribute metadata file format in Attribute Metadata File Specification
Model¶
The model of this component can be described by the following parameters.
Model Parameter |
Type |
Domain |
Description |
---|---|---|---|
fic |
float |
(-inf, inf) |
Factorized Information Criterion. The asymptotic approximation value used by FAB/HME. |
num_initial_comps |
int |
[0, inf) |
The initial number of components before iterations. |
num_active_comps |
int |
[0, inf) |
The terminal number of active components after iterations. |
gate_tree |
dict |
See Description |
Dictionary form of the gating tree structure. |
prediction_formulas |
pandas.DataFrame |
See Description |
Component weights and bias for each prediction formula. |
The gate_tree
dictionary keys are described below:
Gate Tree Dictionary Key |
Type |
Domain |
Description |
---|---|---|---|
gate_type |
str |
‘logit’ |
The type of gate. |
hard_gate |
bool |
true / false |
Whether the gate is hard_gate or not. |
nodes |
list of dict |
See Description |
List of node dictionaries. |
edges |
list of dict |
See Description |
List of edge dictionaries. |
The keys of each node dictionary in nodes
are described below:
Node Dictionary Key |
Type |
Domain |
Description |
---|---|---|---|
node_id |
int |
[0, inf) |
The node ID. |
node_type |
str |
{‘gate’, ‘component’} |
The node type. |
gate_func |
dict |
See Description |
The |
comp_id |
int |
[0, inf) |
The component ID. Specifiable if |
The keys of each edge dictionary in edges
are described below:
Edge Dictionary Key |
Type |
Domain |
Description |
---|---|---|---|
source |
int |
[0, inf) |
The |
target |
int |
[0, inf) |
The |
is_left |
bool |
true / false |
Whether the target node is the left-child of the source. |
The keys of the gate_func
dictionary are described below:
Gate Function Dictionary Key |
Type |
Domain |
Description |
---|---|---|---|
bias |
float |
(-inf, inf) |
The gate function bias. |
weights |
list of dict |
See Description |
Lists weights dictionaries mapping each attribute to its corresponding weight. |
When the model is loaded in the SAMPO API, the model parameters will be output as a single dictionary.
See also
Obtaining process results via ProcessResultLoader
{'fic': -23.832958802449035,
'num_initial_comps': 32
'num_active_comps': 2,
'gate_tree':
{'gate_type': 'logit',
'hard_gate': True,
'nodes': [
{'node_type': 'gate',
'node_id': 0,
'gate_func':
{'bias': 3.42356291005949,
'weights': [
{'aid': 'dl[0]', 'attr_name': 'sepal_length_in_cm', 'weight': -2.3734711943189515},
{'aid': 'dl[1]', 'attr_name': 'petal_length_in_cm', 'weight': 3.129271251371417}]}},
{'comp_id': 12, 'node_type': 'component', 'node_id': 1},
{'comp_id': 16, 'node_type': 'component', 'node_id': 2}],
'edges': [
{'source': 0, 'target': 1, 'is_left': True},
{'source': 0, 'target': 2, 'is_left': False}]}},
'prediction_formulas':
prediction_formula_12 prediction_formula_16
attr_name
sepal_length_in_cm 0 0
petal_length_in_cm 0 0
bias 1 -1}
External Format¶
When convert_process is executed, the model parameters are saved into different files and are grouped as: general information, gating function, and prediction formula.
General Information¶
This file describes \(FIC\) after learning the model, initial number of components, and the terminal number of components.
fic,num_initial_comps,num_active_comps
-1.294308e+02,8,3
Gate Tree¶
This file describes the structure and parameters of the gate-tree of the model.
{
"gate_tree": {
"gate_type": "logit",
"hard_gate": true,
"nodes": [
{
"node_id": 11,
"node_type": "gate",
"gate_func": {
"weights": [
{
"aid": "std1[4]",
"attr_name": "std1_RM",
"weight": -3.6682658685673992e+00
},
{
"aid": "std1[7]",
"attr_name": "std1_TAX",
"weight": -5.8122016705226542e+00
},
{
"aid": "std1[10]",
"attr_name": "std1_LSTAT",
"weight": 1.0537643144910271e+01
}
],
"bias": 1.2740926133353371e+01
}
},
{
"node_id": 10,
"node_type": "gate",
"gate_func": {
"weights": [
{
"aid": "std1[2]",
"attr_name": "std1_INDUS",
"weight": 7.6493213521271874e-01
},
{
"aid": "std1[4]",
"attr_name": "std1_RM",
"weight": 2.5021103534594329e+00
},
{
"aid": "std1[10]",
"attr_name": "std1_LSTAT",
"weight": -2.8529074313420657e+00
}
],
"bias": -2.3621841358789547e-01
}
},
...
{
"node_id": 13,
"node_type": "component",
"comp_id": 13
},
{
"node_id": 12,
"node_type": "component",
"comp_id": 12
},
...
],
"edges": [
{
"source": 11,
"target": 13,
"is_left": false
},
{
"source": 11,
"target": 12,
"is_left": true
},
...
]
}
}
Prediction Formulas¶
This file describes parameters of prediction formulas: weights and bias values.
aid,attr_name,prediction_formula_0,prediction_formula_5
dl1[0],sepal_length_in_cm,0.0000000000000000e+00,0.0000000000000000e+00
dl1[2],petal_length_in_cm,-5.4812912357444157e+00,0.0000000000000000e+00
,bias,1.4422594205941337e+01,9.2027352072284625e+00
Prediction Result Evaluation¶
The indices used in evaluating prediction results of this component are described below.
Evaluation Index |
Type |
Description |
---|---|---|
true_positive |
int |
Number of samples determined as positive correctly (TP). |
false_positive |
int |
Number of samples determined as positive incorrectly (FP). |
true_negative |
int |
Number of samples determined as negative correctly (TN). |
false_negative |
int |
Number of samples determined as negative incorrectly (FN). |
accuracy |
float |
Proportion of true results in the population as shown below:
\(\frac{\mbox{TP} + \mbox{TN}}{\mbox{TP} + \mbox{FP} + \mbox{TN} + \mbox{FN}}\)
|
classification_error |
float |
Proportion of false results in the population as shown below:
\(\frac{\mbox{FP} + \mbox{FN}}{\mbox{TP} + \mbox{FP} + \mbox{TN} + \mbox{FN}} = 1 - \mbox{accuracy}\)
|
precision |
float |
Proportion of the
true_positive against all samples determined as positive as shown below:\(\frac{\mbox{TP}}{\mbox{TP} + \mbox{FP}}\)
|
recall |
float |
Proportion of the
true_positive against all the actual positive samples as shown below:\(\frac{\mbox{TP}}{\mbox{TP} + \mbox{FN}}\)
|
specificity |
float |
Proportion of the
true_negative against all the actual negative samples as shown below:\(\frac{\mbox{TN}}{\mbox{TN} + \mbox{FP}}\)
|
false_positive_rate |
float |
Proportion of the
false_positive against all the actual negative samples as shown below:\(\frac{\mbox{FP}}{\mbox{TN} + \mbox{FP}} = 1 - \mbox{specificity}\)
|
false_negative_rate |
float |
Proportion of the
false_negative against all the actual positive samples as shown below:\(\frac{\mbox{FN}}{\mbox{TP} + \mbox{FN}} = 1 - \mbox{recall}\)
|
f_measure |
float |
Harmonic mean of
precision and recall as shown below:\(\frac{2 \times \mbox{precision} \times \mbox{recall}}{\mbox{precision} + \mbox{recall}}\)
|
auc |
float |
Area under ROC (Receiver Operating Characteristic) curve. |
area_under_precision_recall |
float |
Area under PR (Precision-Recall) curve. |
When obtaining these evaluation results in SAMPO API, a pandas.DataFrame is loaded with the evaluation indices as the columns of the DataFrame.
See also
Obtaining process results via ProcessResultLoader
External Format¶
When convert_process is executed, the evaluation results are saved as a CSV file with the evaluation indices as the header of the CSV.
This file describes the evaluation of the prediction result of the component.
true_positive,false_positive,true_negative,false_negative,accuracy,classification_error,precision,recall,specificity,false_positive_rate,false_negative_rate,f_measure,auc,area_under_precision_recall
6,3,14,7,6.666667e-01,3.333333e-01,6.666667e-01,4.615385e-01,8.235294e-01,1.764706e-01,5.384615e-01,5.454545e-01,6.696833e-01,5.715832e-01