FABHMEBernGateBSplineRg Component Specification¶
Contents
Overview¶
FABHMEBernGateBSplineRg component is a B-spline non-linear regression component with FAB/HME algorithm. This component learns a tree-structured model in which each sample is assigned to a component according to Bernoulli gating functions.
Note
FAB engine uses the word ‘component’ with a different meaning from that of SAMPO. Each component in FAB/HME is a prediction formula, and each sample data is assigned to a specific component for prediction.
Example:
SPD:
# fabhmerg.spd dl1 -> std1 -> fab1 --- components: dl1: component: DataLoader std1: component: StandardizeFDComponent features: scale == 'real' or scale == 'integer' fab1: component: FABHMEBernGateBSplineRgComponent features: name != 'Concrete_compressive_strength_MPa' standardize_target: True target: name == 'Concrete_compressive_strength_MPa' tree_depth: 5 shrink_threshold: 1.0% global_settings: keep_attributes: - Concrete_compressive_strength_MPa feature_exclude: - Concrete_compressive_strength_MPa
Input of the component:
_sid
std1_Superplasticizer_ kg_in_a_m3_mixture
std1_Coarse_Aggregate_ kg_in_a_m3_mixture
Concrete_compressive_ strength_MPa
0
0.729738484
0.705292074
9
1
1.413868312
1.434904563
9
2
-1.387806224
-1.321409287
4
…
…
…
…
8
-0.019546567
0.016213611
8
9
-0.736254006
-0.835000961
6
Output of the component:
_sid
fab1_ actual
fab1_ std_actual
fab1_ predict
fab1_ std_predict
fab1_ assigned_comp_id
0
9
-4.686873e-01
1.193986e+01
3.613565e-01
2
1
9
-4.686873e-01
1.487464e+01
1.189968e+00
2
2
4
-1.880396e+00
5.744445e+00
-1.387866e+00
0
…
…
…
…
…
…
8
8
-7.510290e-01
9.614527e+00
-2.951807e-01
0
9
6
-1.315712e+00
7.587341e+00
-8.675398e-01
0
_sid
fab1_ predict_c0
fab1_ std_predict_c0
fab1_ predict_c1
fab1_ std_predict_c1
fab1_ predict_c2
fab1_ std_predict_c2
0
1.173386e+01
3.031946e-01
1.285257e+01
6.190547e-01
1.193986e+01
3.613565e-01
1
1.366890e+01
8.495373e-01
1.503635e+01
1.235628e+00
1.487464e+01
1.189968e+00
2
5.744445e+00
-1.387866e+00
6.786511e+00
-1.093648e+00
3.787681e+00
-1.940342e+00
…
…
…
…
…
…
…
8
9.614527e+00
-2.951807e-01
1.079011e+01
3.673590e-02
9.168116e+00
-4.212211e-01
9
7.587341e+00
-8.675398e-01
8.242365e+00
-6.825991e-01
5.744203e+00
-1.387935e+00
This component has component-specific external formats for model and prediction result evaluation.
See also
Component-common external format files in convert_process
Parameters¶
This component has the following component-specific parameters.
SPD¶
The following parameters are for “components” section of SPD.
Parameter Name |
Type |
Domain |
Default Value |
Description |
---|---|---|---|---|
standardize_target |
bool |
True / False |
False |
If this parameter is True, the target attribute is standardized. |
max_fab_iterations |
int |
[1, inf) |
100 |
Maximum number of FAB-iterations. |
bool |
True / False |
False |
If True, the first iteration starts with M-step; otherwise, E-step. |
|
num_acceleration_steps |
int |
[0, inf) |
0 |
The number of steps of acceleration algorithm for each FAB-iteration. If 0, the acceleration algorithm is disabled. |
repeat_until_convergence |
bool |
True / False |
False |
If False, FAB-iterations and the post-processing are executed only once
even if the FAB-iterations are stopped not by convergence condition but
by |
projection_estep |
bool |
True / False |
False |
Whether the projection E-step algorithm is enabled. |
shrink_threshold |
float or str |
[1, inf) or (0%, 100%) |
1.0 |
Threshold value for shrinkage. If a percentage value (e.g. |
fab_stop_threshold |
float or str |
(0, inf) or (0%, inf%) |
0.001 |
Threshold value for FAB-iterations: if the increase of FIC value
is less than the threshold, the FAB-iterations is considered to
be converged. If a percentage value (e.g. |
gate_features |
str |
Query format |
all() |
Features which are applied to gate parameter optimizations. If not specified, all features are used. |
comp_features |
str |
Query format |
all() |
Features which are applied to component parameter optimizations. If not specified, all features are used. If empty, the model is learned as a decision tree. |
comp_mandatory_features |
str |
Query format |
See Description |
Features which non-L0-regularize constraints are applied to. It means the specified features will always be relevant for all components. If not specified, no features are specified for non-L0-regularization, which implies all relevant features are selected by FoBa algorithm. |
int |
[0, inf) |
5 |
Initial depth of the gate-tree structure of latent variable prior. The initial number of components is \(2^d\) where \(d\) is tree depth. If 0, the optimization with only one component will be executed. |
|
comp_bspline_degree 3 |
int |
[0, inf) |
3 |
Degree of B-spline function. |
comp_bspline_basis_dim 3 |
int |
[4, inf) |
10 |
The number of B-spline basis functions to be generated for each feature. |
float |
(-inf, inf) |
-0.5 |
Scale value for the initialization of weight values of components. |
|
float |
(-inf, inf) |
0.5 |
Scale value for the initialization of weight values of components. |
|
float |
(-inf, inf) |
0.25 |
Scale value for the initialization of bias values of components. |
|
float |
(-inf, inf) |
0.75 |
Scale value for the initialization of bias values of components. |
|
float |
(0, inf) |
0.1 |
Scale value for the initialization of variance values of components. |
|
float |
(0, inf) |
0.25 |
Scale value for the initialization of variance values of components. |
|
gate_max_bins |
int |
[1, inf) |
See Description |
Maximum number of binning for each feature, which is used for gate parameter optimization. If not specified, all unique samples for each feature are used; otherwise, the equal-width binning algorithm is adopted. |
comp_l2_regularize |
float |
[0, inf) |
0.0 |
L2-regularization hyper-parameter for component parameter optimization. The larger the specified value, the stronger the regularization effect is. If 0.0, L2-regularization is disabled. |
comp_pspline |
float |
[0, inf) |
1.0 |
L2-regularization coefficient value for penalized B-spline function (P-spline). |
with_comp_scaled_l0_regularize |
bool |
True / False |
True |
Whether with scaled L0-regularization using a tighter lower bound of FIC for component parameter optimization; approximation of det(F) is refined, where F is a Fisher matrix. |
max_comp_relevant_features |
int |
[1, inf) |
100 |
Maximum number of the relevant features for each component. |
num_threads_gates |
int |
[1, inf) |
1 |
Maximum number of OpenMP threads of gate parameter optimization where tasks for all gates are divided into. |
num_threads_gate_features |
int |
[1, inf) |
1 |
Maximum number of OpenMP threads of gate parameter optimization where tasks for all features are divided into. |
num_threads_comps |
int |
[1, inf) |
1 |
Maximum number of OpenMP threads of component parameter optimization. |
SRC¶
The following parameter is for “hotstart” section of SRC.
Parameter Name |
Type |
Domain |
Default Value |
Description |
---|---|---|---|---|
type |
str |
{‘posterior’, ‘mh_refit_comp’, ‘mh_opt_comp’, ‘mh_refit_gate_and_refit_comp’, ‘mh_refit_gate_and_opt_comp’, ‘mh_opt_gate_and_opt_comp’} |
The hot-start type. If ‘posterior’, FAB learns with posterior hot-start which use the initial model whose tree structure is generated by base model and data. Each gate and component parameters are initialized randomly. ‘mh_XXX’ means FAB learns with model hot-start which uses base model as initial model. ‘refit_{gate, comp}’ means refitting the gate functions or prediction formulas with current data. ‘opt_{gate, comp}’ means optimizing (feature selection and fitting) the gate functions or prediction formulas with current data. |
Utilizable Sample Metadata¶
Warning
_fabhme_assigned_comp_id is deprecated. Use hotstart section of SRC instead of _fabhme_assigned_comp_id data column.
This component can utilize the _fabhme_assigned_comp_id attribute of the sample metadata to hot-start with posterior. When the attribute _fabhme_assigned_comp_id attribute is specified in the input data, this component will start the FAB/HME algorithm with the _fabhme_assigned_comp_id attribute values as its initial posterior.
To create the attribute _fabhme_assigned_comp_id, see the specification of the command sampo_ps_fabhme export_assigned_comp_id.
Output Attributes¶
This component generates the following attributes.
Attribute Name |
Scale |
Description |
---|---|---|
<component_id>_actual |
INTEGER/REAL (depend on target attribute) |
Values of target attribute. |
<component_id>_std_actual |
REAL |
Standardized values of |
<component_id>_predict |
REAL |
Predicted values. |
<component_id>_std_predict |
REAL |
Standardized values of |
<component_id>_assigned_comp_id |
INTEGER |
Component IDs assigned by gating functions. |
<component_id>_predict_c<hme_comp_id> |
REAL |
Predicted values for the prediction formula of component id, <hme_comp_id>. |
<component_id>_std_predict_c<hme_comp_id> |
REAL |
Standardized predicted values for the prediction formula of component id, <hme_comp_id>. |
<component_id>_basisfunc_<feature_attr_name>:<basis_func_index> |
REAL |
Basis function values. |
These attributes are in the component output data. These can be loaded in SAMPO API.
See also
Obtaining process results via ProcessResultLoader.
When convert_process is executed, the component output data will be saved in two separate files:
All non-basis function value attributes will be saved as <component_id>_predict_result.csv.
This file describes the prediction result of the component.
_sid,fab1_actual,fab1_std_actual,fab1_predict,fab1_std_predict,fab1_assigned_comp_id,fab1_predict_c0,fab1_std_predict_c0,fab1_predict_c1,fab1_std_predict_c1,fab1_predict_c2,fab1_std_predict_c2 0,9,-4.686873e-01,1.193986e+01,3.613565e-01,2,1.173386e+01,3.031946e-01,1.285257e+01,6.190547e-01,1.193986e+01,3.613565e-01 1,9,-4.686873e-01,1.487464e+01,1.189968e+00,2,1.366890e+01,8.495373e-01,1.503635e+01,1.235628e+00,1.487464e+01,1.189968e+00 2,4,-1.880396e+00,5.744445e+00,-1.387866e+00,0,5.744445e+00,-1.387866e+00,6.786511e+00,-1.093648e+00,3.787681e+00,-1.940342e+00 ... 8,8,-7.510290e-01,9.614527e+00,-2.951807e-01,0,9.614527e+00,-2.951807e-01,1.079011e+01,3.673590e-02,9.168116e+00,-4.212211e-01 9,6,-1.315712e+00,7.587341e+00,-8.675398e-01,0,7.587341e+00,-8.675398e-01,8.242365e+00,-6.825991e-01,5.744203e+00,-1.387935e+00
Basis function value attributes will be saved as basis_func_values.csv.
This file describes the basis function values of B-spline functions.
_sid,fab1_basisfunc_std1_CRIM:0,fab1_basisfunc_std1_CRIM:1,fab1_basisfunc_std1_CRIM:2,fab1_basisfunc_std1_CRIM:3,fab1_basisfunc_std1_CRIM:4,fab1_basisfunc_std1_CRIM:5,fab1_basisfunc_std1_CRIM:6,fab1_basisfunc_std1_CRIM:7,fab1_basisfunc_std1_CRIM:8,fab1_basisfunc_std1_CRIM:9,fab1_basisfunc_std1_ZN:0,fab1_basisfunc_std1_ZN:1,fab1_basisfunc_std1_ZN:2,fab1_basisfunc_std1_ZN:3,fab1_basisfunc_std1_ZN:4,fab1_basisfunc_std1_ZN:5,fab1_basisfunc_std1_ZN:6,fab1_basisfunc_std1_ZN:7,fab1_basisfunc_std1_ZN:8,fab1_basisfunc_std1_ZN:9,fab1_basisfunc_std1_NOX:0,fab1_basisfunc_std1_NOX:1,fab1_basisfunc_std1_NOX:2,fab1_basisfunc_std1_NOX:3,fab1_basisfunc_std1_NOX:4,fab1_basisfunc_std1_NOX:5,fab1_basisfunc_std1_NOX:6,fab1_basisfunc_std1_NOX:7,fab1_basisfunc_std1_NOX:8,fab1_basisfunc_std1_NOX:9,fab1_basisfunc_bin1(0)_CHAS:0,fab1_basisfunc_bin1(0)_CHAS:1,fab1_basisfunc_bin1(0)_CHAS:2,fab1_basisfunc_bin1(0)_CHAS:3,fab1_basisfunc_bin1(0)_CHAS:4,fab1_basisfunc_bin1(0)_CHAS:5,fab1_basisfunc_bin1(0)_CHAS:6,fab1_basisfunc_bin1(0)_CHAS:7,fab1_basisfunc_bin1(0)_CHAS:8,fab1_basisfunc_bin1(0)_CHAS:9,fab1_basisfunc_bin1(1)_RAD:0,fab1_basisfunc_bin1(1)_RAD:1,fab1_basisfunc_bin1(1)_RAD:2,fab1_basisfunc_bin1(1)_RAD:3,fab1_basisfunc_bin1(1)_RAD:4,fab1_basisfunc_bin1(1)_RAD:5,fab1_basisfunc_bin1(1)_RAD:6,fab1_basisfunc_bin1(1)_RAD:7,fab1_basisfunc_bin1(1)_RAD:8,fab1_basisfunc_bin1(1)_RAD:9,fab1_basisfunc_std1_LSTAT:0,fab1_basisfunc_std1_LSTAT:1,fab1_basisfunc_std1_LSTAT:2,fab1_basisfunc_std1_LSTAT:3,fab1_basisfunc_std1_LSTAT:4,fab1_basisfunc_std1_LSTAT:5,fab1_basisfunc_std1_LSTAT:6,fab1_basisfunc_std1_LSTAT:7,fab1_basisfunc_std1_LSTAT:8,fab1_basisfunc_std1_LSTAT:9,fab1_basisfunc_std1_TAX:0,fab1_basisfunc_std1_TAX:1,fab1_basisfunc_std1_TAX:2,fab1_basisfunc_std1_TAX:3,fab1_basisfunc_std1_TAX:4,fab1_basisfunc_std1_TAX:5,fab1_basisfunc_std1_TAX:6,fab1_basisfunc_std1_TAX:7,fab1_basisfunc_std1_TAX:8,fab1_basisfunc_std1_TAX:9,fab1_basisfunc_bin1(3)_RAD:0,fab1_basisfunc_bin1(3)_RAD:1,fab1_basisfunc_bin1(3)_RAD:2,fab1_basisfunc_bin1(3)_RAD:3,fab1_basisfunc_bin1(3)_RAD:4,fab1_basisfunc_bin1(3)_RAD:5,fab1_basisfunc_bin1(3)_RAD:6,fab1_basisfunc_bin1(3)_RAD:7,fab1_basisfunc_bin1(3)_RAD:8,fab1_basisfunc_bin1(3)_RAD:9,fab1_basisfunc_std1_DIS:0,fab1_basisfunc_std1_DIS:1,fab1_basisfunc_std1_DIS:2,fab1_basisfunc_std1_DIS:3,fab1_basisfunc_std1_DIS:4,fab1_basisfunc_std1_DIS:5,fab1_basisfunc_std1_DIS:6,fab1_basisfunc_std1_DIS:7,fab1_basisfunc_std1_DIS:8,fab1_basisfunc_std1_DIS:9,fab1_basisfunc_std1_PTRATIO:0,fab1_basisfunc_std1_PTRATIO:1,fab1_basisfunc_std1_PTRATIO:2,fab1_basisfunc_std1_PTRATIO:3,fab1_basisfunc_std1_PTRATIO:4,fab1_basisfunc_std1_PTRATIO:5,fab1_basisfunc_std1_PTRATIO:6,fab1_basisfunc_std1_PTRATIO:7,fab1_basisfunc_std1_PTRATIO:8,fab1_basisfunc_std1_PTRATIO:9,fab1_basisfunc_std1_B:0,fab1_basisfunc_std1_B:1,fab1_basisfunc_std1_B:2,fab1_basisfunc_std1_B:3,fab1_basisfunc_std1_B:4,fab1_basisfunc_std1_B:5,fab1_basisfunc_std1_B:6,fab1_basisfunc_std1_B:7,fab1_basisfunc_std1_B:8,fab1_basisfunc_std1_B:9,fab1_basisfunc_std1_INDUS:0,fab1_basisfunc_std1_INDUS:1,fab1_basisfunc_std1_INDUS:2,fab1_basisfunc_std1_INDUS:3,fab1_basisfunc_std1_INDUS:4,fab1_basisfunc_std1_INDUS:5,fab1_basisfunc_std1_INDUS:6,fab1_basisfunc_std1_INDUS:7,fab1_basisfunc_std1_INDUS:8,fab1_basisfunc_std1_INDUS:9,fab1_basisfunc_std1_RM:0,fab1_basisfunc_std1_RM:1,fab1_basisfunc_std1_RM:2,fab1_basisfunc_std1_RM:3,fab1_basisfunc_std1_RM:4,fab1_basisfunc_std1_RM:5,fab1_basisfunc_std1_RM:6,fab1_basisfunc_std1_RM:7,fab1_basisfunc_std1_RM:8,fab1_basisfunc_std1_RM:9,fab1_basisfunc_std1_AGE:0,fab1_basisfunc_std1_AGE:1,fab1_basisfunc_std1_AGE:2,fab1_basisfunc_std1_AGE:3,fab1_basisfunc_std1_AGE:4,fab1_basisfunc_std1_AGE:5,fab1_basisfunc_std1_AGE:6,fab1_basisfunc_std1_AGE:7,fab1_basisfunc_std1_AGE:8,fab1_basisfunc_std1_AGE:9 0,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.197819e-01,0.000000e+00,4.056000e-01,5.621333e-01,3.226667e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,2.848299e-01,3.540404e-01,5.969046e-01,4.905502e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-1.287909e+00,0.000000e+00,0.000000e+00,3.657979e-01,5.893919e-01,4.481024e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-1.442174e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,3.173806e-01,6.185440e-01,6.407538e-02,0.000000e+00,0.000000e+00,4.136719e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.810398e-01,4.160185e-01,2.941641e-03,0.000000e+00,-1.200134e-01,0.000000e+00,0.000000e+00,5.681834e-01,4.278832e-01,3.933446e-03,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,1.402136e-01,0.000000e+00,2.974283e-01,6.290620e-01,7.350970e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-6.666082e-01,0.000000e+00,0.000000e+00,4.828731e-01,5.023389e-01,1.478799e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-1.459000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,4.410519e-01,2.741603e-01,6.400531e-01,8.578652e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-1.075562e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00 1,6.654090e-01,3.345904e-01,5.937007e-07,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.173393e-01,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.877224e-01,0.000000e+00,1.878266e-01,6.654025e-01,1.467709e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-5.933810e-01,0.000000e+00,4.359346e-01,5.396535e-01,2.441193e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-7.402622e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,4.352526e-01,5.401738e-01,2.457365e-02,0.000000e+00,0.000000e+00,1.942745e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.036805e-01,4.849149e-01,1.140454e-02,3.671664e-01,0.000000e+00,0.000000e+00,2.433332e-01,6.522031e-01,1.044637e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.571599e-01,2.243847e-01,6.581007e-01,1.175145e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-9.873295e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,4.131583e-01,5.566621e-01,3.017957e-02,0.000000e+00,0.000000e+00,-3.030941e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,4.410519e-01,0.000000e+00,3.101911e-01,6.224435e-01,6.736547e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.924394e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00 2,6.654102e-01,3.345892e-01,5.925699e-07,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.173416e-01,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.877224e-01,0.000000e+00,1.878266e-01,6.654025e-01,1.467709e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-5.933810e-01,0.000000e+00,4.359346e-01,5.396535e-01,2.441193e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-7.402622e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,3.479622e-01,6.006842e-01,5.135363e-02,0.000000e+00,1.282714e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,2.419814e-01,6.526661e-01,1.053525e-01,0.000000e+00,0.000000e+00,-2.658118e-01,0.000000e+00,0.000000e+00,2.433332e-01,6.522031e-01,1.044637e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.571599e-01,2.243847e-01,6.581007e-01,1.175145e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-9.873295e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,4.131583e-01,5.566621e-01,3.017957e-02,0.000000e+00,0.000000e+00,-3.030941e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,1.951574e-01,5.942084e-01,3.964270e-01,3.711468e-01,5.858889e-01,4.296433e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-1.208727e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00 3,6.651060e-01,3.348931e-01,9.144462e-07,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.167504e-01,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.877224e-01,3.728038e-01,5.847932e-01,4.240303e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-1.306878e+00,0.000000e+00,5.390128e-01,4.542103e-01,6.776858e-03,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-8.352838e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,4.997154e-01,4.882744e-01,1.201021e-02,0.000000e+00,1.016303e+00,0.000000e+00,0.000000e+00,0.000000e+00,3.579481e-01,5.944367e-01,4.761513e-02,0.000000e+00,0.000000e+00,0.000000e+00,-8.098885e-01,0.000000e+00,0.000000e+00,0.000000e+00,3.321228e-01,6.101833e-01,5.769394e-02,0.000000e+00,0.000000e+00,0.000000e+00,1.077737e+00,3.580211e-01,5.943904e-01,4.758852e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-1.106115e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.451185e-01,4.487702e-01,6.111363e-03,0.000000e+00,1.130321e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,1.822800e-01,5.900916e-01,4.161628e-01,5.004857e-01,4.876232e-01,1.189113e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-1.361517e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00 ... 504,6.604905e-01,3.394952e-01,1.437113e-05,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.077641e-01,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.877224e-01,0.000000e+00,0.000000e+00,0.000000e+00,4.462810e-01,5.316804e-01,2.203857e-02,0.000000e+00,0.000000e+00,0.000000e+00,1.157384e-01,0.000000e+00,0.000000e+00,0.000000e+00,6.050596e-01,3.934473e-01,1.493110e-03,0.000000e+00,0.000000e+00,0.000000e+00,1.581241e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,1.817470e-01,6.660136e-01,1.522394e-01,0.000000e+00,0.000000e+00,7.256721e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.900481e-01,4.064453e-01,7.369964e-01,1.958019e-01,6.643210e-01,1.398771e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-6.684368e-01,0.000000e+00,4.743410e-01,5.093332e-01,1.632578e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-8.032117e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.710729e-01,4.233816e-01,1.176466e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,1.906723e-01,5.929144e-01,4.032249e-01,0.000000e+00,6.346830e-01,3.649239e-01,3.930952e-04,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-8.653016e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00 505,6.642058e-01,3.357919e-01,2.275176e-06,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.150002e-01,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-4.877224e-01,0.000000e+00,0.000000e+00,0.000000e+00,4.462810e-01,5.316804e-01,2.203857e-02,0.000000e+00,0.000000e+00,0.000000e+00,1.157384e-01,0.000000e+00,0.000000e+00,0.000000e+00,6.050596e-01,3.934473e-01,1.493110e-03,0.000000e+00,0.000000e+00,0.000000e+00,1.581241e-01,0.000000e+00,0.000000e+00,0.000000e+00,2.461861e-01,6.512057e-01,1.026082e-01,0.000000e+00,0.000000e+00,0.000000e+00,-3.627671e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,4.170544e-01,5.538074e-01,2.913818e-02,4.347315e-01,0.000000e+00,6.662848e-01,3.337151e-01,5.470025e-08,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-6.132465e-01,0.000000e+00,4.743410e-01,5.093332e-01,1.632578e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-8.032117e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.710729e-01,4.233816e-01,1.176466e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,4.410519e-01,0.000000e+00,4.495709e-01,5.291142e-01,2.131485e-02,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,-6.690583e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00,6.666667e-01,3.333333e-01,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,0.000000e+00,5.000000e-01,1.000000e+00
Attribute Metadata¶
The metadata of the output attributes is created with the following rules.
Context Rule¶
Attribute Name |
Context Name |
Description |
---|---|---|
All the output attributes of this component |
field_path |
List of the superordinate concepts of each output attribute based on the following hierarchical structure of the output attributes: root
├── fabhmerg
│ ├── assigned_comp_id
│ └── component
│ ├── 0
│ │ ├── predict
│ │ └── std_predict
│ ├── 1
│ │ ├── predict
│ │ └── std_predict
│ .
│ .
│ .
│
└── regression
├── actual
├── std_actual
├── predict
└── std_predict
|
<component_id>_std_actual, <component_id>_std_predict, <component_id>_std_predict_c<hme_comp_id> |
mean |
Mean of the target values for learning. |
<component_id>_std_actual, <component_id>_std_predict, <component_id>_std_predict_c<hme_comp_id> |
std |
Standard deviation of the target values for learning. |
<component_id>_assigned_comp_id |
active_comp_ids |
List of component IDs corresponding to each prediction formula. |
Derivation Rule¶
Attribute Name |
Derived From |
---|---|
<component_id>_actual, <component_id>_std_actual |
Derived from the target attribute. |
<component_id>_predict, <component_id>_std_predict |
Derived from the attributes which have non-zero coefficients in any prediction formula. |
<component_id>_assigned_comp_id |
Derived from the attributes used in the gating functions. |
<component_id>_predict_c<hme_comp_id>, <component_id>_std_predict_c<hme_comp_id> |
Derived from the attributes which have non-zero coefficients in the prediction formula of component id, <hme_comp_id>. |
<component_id>_basisfunc_<feature_attr_name>:<basis_function_index> |
Derived from the attribute of the name of <feature_attr_name>. |
Example¶
{
"nodes": [
{
"aid": "fab1[152]",
"name": "fab1_basisfunc_std1_AGE:3",
"scale": "real",
"is_excluded": false,
"cid": "fab1",
"cindex": 152,
"values": null,
"is_kept": false,
"context": null
},
{
"aid": "std1[9]",
"name": "std1_B",
"scale": "real",
"is_excluded": false,
"cid": "std1",
"cindex": 9,
"values": null,
"is_kept": false,
"context": {
"std": 9.1204607452175566e+01,
"mean": 3.5667403162055257e+02
}
},
...
],
"links": [
{
"source": "std1[9]",
"target": "fab1[122]"
},
{
"source": "std1[9]",
"target": "fab1[120]"
},
...
]
}
See also
Attribute metadata file format in Attribute Metadata File Specification
Model¶
The model of this component can be described by the following parameters.
Model Parameter |
Type |
Domain |
Description |
---|---|---|---|
fic |
float |
(-inf, inf) |
Factorized Information Criterion. The asymptotic approximation value used by FAB/HME. |
num_initial_comps |
int |
[0, inf) |
The initial number of components before iterations. |
num_active_comps |
int |
[0, inf) |
The terminal number of active components after iterations. |
standardize_mean |
float |
(-inf, inf) |
Mean value used for standardizing the target attribute during learning. |
standardize_std |
float |
(-inf, inf) |
Standard deviation value used for standardizing the target attribute during learning. |
gate_tree |
dict |
See Description |
Dictionary form of the gating tree structure. |
prediction_formulas |
pandas.DataFrame |
See Description |
Component weights and bias for each prediction formula. |
bspline_params |
pandas.DataFrame |
See Description |
Degree and basis dimensionality of the B-spline function. |
bspline_knot_vecs |
pandas.DataFrame |
See Description |
Knot vectors for all features for all knots in the B-spline function. |
The gate_tree
dictionary keys are described below:
Gate Tree Dictionary Key |
Type |
Domain |
Description |
---|---|---|---|
gate_type |
str |
‘bern’ |
The type of gate. |
hard_gate |
bool |
true / false |
Whether the gate is hard_gate or not. |
nodes |
list of dict |
See Description |
List of node dictionaries. |
edges |
list of dict |
See Description |
List of edge dictionaries. |
The keys of each node dictionary in nodes
are described below:
Node Dictionary Key |
Type |
Domain |
Description |
---|---|---|---|
node_id |
int |
[0, inf) |
The node ID. |
node_type |
str |
{‘gate’, ‘component’} |
The node type. |
gate_func |
dict |
See Description |
The |
comp_id |
int |
[0, inf) |
The component ID. Specifiable if |
The keys of each edge dictionary in edges
are described below:
Edge Dictionary Key |
Type |
Domain |
Description |
---|---|---|---|
source |
int |
[0, inf) |
The |
target |
int |
[0, inf) |
The |
is_left |
bool |
true / false |
Whether the target node is the left-child of the source. |
The keys of the gate_func
dictionary are described below:
Gate Function Dictionary Key |
Type |
Domain |
Description |
---|---|---|---|
attr_name |
str |
See Description |
The attribute name. |
aid |
str |
See Description |
The attribute ID. |
threshold |
float |
(-inf, inf) |
Threshold value of the Bernoulli-gating function. |
prob_left_smaller_than_threshold |
float |
[0.0, 1.0] |
Probability that the value of left-child node is smaller than the |
When the model is loaded in the SAMPO API, the model parameters will be output as a single dictionary.
See also
Obtaining process results via ProcessResultLoader
{'fic': -60.32621126381368,
'num_initial_comps': 8,
'num_active_comps': 5,
'standardize_mean': 1.1303215277777777e+04,
'standardize_std': 5.7343353765366674e+03,
'gate_tree':
{'gate_type': 'bern',
'hard_gate': True,
'edges': [
{'source': 1,
'target': 2,
'is_left': True},
{'source': 1,
'target': 5,
'is_left': False},
{'source': 0,
'target': 1,
'is_left': True},
{'source': 0,
'target': 6,
'is_left': False},
{'source': 2,
'target': 3,
'is_left': True},
{'source': 2,
'target': 4,
'is_left': False},
{'source': 6,
'target': 8,
'is_left': False},
{'source': 6,
'target': 7,
'is_left': True}],
'nodes': [
{'node_type': 'gate',
'node_id': 1,
'gate_func':
{'threshold': 0.7952709366877533,
'aid': 'std1[5]',
'attr_name': 'std1_Coarse_Aggregate_kg_in_a_m3_mixture',
'prob_left_smaller_than_threshold': 1.0}},
{'node_type': 'gate',
'node_id': 0,
'gate_func':
{'threshold': -0.995668911902564,
'aid': 'std1[5]',
'attr_name': 'std1_Coarse_Aggregate_kg_in_a_m3_mixture',
'prob_left_smaller_than_threshold': 0.0}},
{'node_type': 'gate',
'node_id': 2,
'gate_func':
{'threshold': -0.1580241042873804,
'aid': 'std1[5]',
'attr_name': 'std1_Coarse_Aggregate_kg_in_a_m3_mixture',
'prob_left_smaller_than_threshold': 0.0}},
{'node_type': 'gate',
'node_id': 6,
'gate_func':
{'threshold': 1.0926798109856788,
'aid': 'std1[6]',
'attr_name': 'std1_Fine_Aggregate_kg_in_a_m3_mixture',
'prob_left_smaller_than_threshold': 1.0}},
{'comp_id': 0, 'node_type': 'component', 'node_id': 3},
{'comp_id': 1, 'node_type': 'component', 'node_id': 4},
{'comp_id': 3, 'node_type': 'component', 'node_id': 5},
{'comp_id': 4, 'node_type': 'component', 'node_id': 7},
{'comp_id': 6, 'node_type': 'component', 'node_id': 8}]}},
'prediction_formulas':
prediction_formula_0 prediction_formula_1 prediction_formula_3 prediction_formula_4 prediction_formula_6
attr_name basis_function_index
std1_Cement_kg_in_a_m3_mixture 0 0.000000 0.000000 0.025867 0.000000 0.000000
1 0.000000 0.000000 -0.334484 0.000000 0.000000
2 0.000000 0.000000 -0.612774 0.000000 0.000000
3 0.000000 0.000000 -0.597176 0.000000 0.000000
4 0.000000 0.000000 -0.133172 0.000000 0.000000
5 0.000000 0.000000 0.510668 0.000000 0.000000
6 0.000000 0.000000 0.612924 0.000000 0.000000
7 0.000000 0.000000 1.020042 0.000000 0.000000
8 0.000000 0.000000 0.650793 0.000000 0.000000
9 0.000000 0.000000 0.384896 0.000000 0.000000
std1_Blast_Furnace_Slag_kg_in_a_m3_mixture 0 0.000000 0.030223 0.000000 0.000000 0.000000
1 0.000000 0.068615 0.000000 0.000000 0.000000
2 0.000000 0.080770 0.000000 0.000000 0.000000
3 0.000000 0.040618 0.000000 0.000000 0.000000
4 0.000000 -0.019869 0.000000 0.000000 0.000000
5 0.000000 -0.051048 0.000000 0.000000 0.000000
6 0.000000 -0.068837 0.000000 0.000000 0.000000
7 0.000000 -0.119049 0.000000 0.000000 0.000000
8 0.000000 -0.174934 0.000000 0.000000 0.000000
9 0.000000 -0.187765 0.000000 0.000000 0.000000
std1_Fly_Ash_kg_in_a_m3_mixture 0 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.000000 0.000000 0.000000 0.000000 0.000000
2 0.000000 0.000000 0.000000 0.000000 0.000000
3 0.000000 0.000000 0.000000 0.000000 0.000000
4 0.000000 0.000000 0.000000 0.000000 0.000000
5 0.000000 0.000000 0.000000 0.000000 0.000000
6 0.000000 0.000000 0.000000 0.000000 0.000000
7 0.000000 0.000000 0.000000 0.000000 0.000000
8 0.000000 0.000000 0.000000 0.000000 0.000000
9 0.000000 0.000000 0.000000 0.000000 0.000000
... ... ... ... ... ...
std1_Superplasticizer_kg_in_a_m3_mixture 2 0.000000 0.064056 -0.867979 0.000000 0.000000
3 0.000000 0.021742 -1.189430 0.000000 0.000000
4 0.000000 -0.082330 -1.120923 0.000000 0.000000
5 0.000000 -0.187887 -0.563142 0.000000 0.000000
6 0.000000 -0.262108 -0.002287 0.000000 0.000000
7 0.000000 -0.285473 0.222134 0.000000 0.000000
8 0.000000 -0.304922 0.345827 0.000000 0.000000
9 0.000000 -0.303971 0.338419 0.000000 0.000000
std1_Coarse_Aggregate_kg_in_a_m3_mixture 0 0.000000 0.000000 0.000000 0.000000 0.000000
1 0.000000 0.000000 0.000000 0.000000 0.000000
2 0.000000 0.000000 0.000000 0.000000 0.000000
3 0.000000 0.000000 0.000000 0.000000 0.000000
4 0.000000 0.000000 0.000000 0.000000 0.000000
5 0.000000 0.000000 0.000000 0.000000 0.000000
6 0.000000 0.000000 0.000000 0.000000 0.000000
7 0.000000 0.000000 0.000000 0.000000 0.000000
8 0.000000 0.000000 0.000000 0.000000 0.000000
9 0.000000 0.000000 0.000000 0.000000 0.000000
std1_Fine_Aggregate_kg_in_a_m3_mixture 0 0.000000 -0.029913 0.000000 0.000000 0.000000
1 0.000000 0.024779 0.000000 0.000000 0.000000
2 0.000000 0.105099 0.000000 0.000000 0.000000
3 0.000000 0.128117 0.000000 0.000000 0.000000
4 0.000000 0.140744 0.000000 0.000000 0.000000
5 0.000000 0.159559 0.000000 0.000000 0.000000
6 0.000000 0.179275 0.000000 0.000000 0.000000
7 0.000000 0.197711 0.000000 0.000000 0.000000
8 0.000000 0.215885 0.000000 0.000000 0.000000
9 0.000000 0.210016 0.000000 0.000000 0.000000
bias -0.160285 0.420209 -0.057683 -0.350672 0.072509
variance 0.125835 0.008705 0.333139 0.096289 0.038456
[72 rows x 5 columns],
'bspline_params':
degree basis_dim
0 3 10,
'bspline_knot_vecs':
knot_value_0 knot_value_1 knot_value_2 knot_value_3 knot_value_4 knot_value_5 knot_value_6 knot_value_7 knot_value_8 knot_value_9 knot_value_10 knot_value_11 knot_value_12
attr_name
std1_Cement_kg_in_a_m3_mixture -2.545730 -2.545730 -2.054368 -1.563006 -1.071643 -0.580281 -0.088919 0.402444 0.893806 1.385169 1.876531 2.367893 2.367893
std1_Blast_Furnace_Slag_kg_in_a_m3_mixture -1.917254 -1.917254 -1.485959 -1.054663 -0.623367 -0.192071 0.239225 0.670521 1.101817 1.533113 1.964409 2.395704 2.395704
std1_Fly_Ash_kg_in_a_m3_mixture -2.548438 -2.548438 -2.175280 -1.802123 -1.428965 -1.055808 -0.682650 -0.309493 0.063665 0.436822 0.809980 1.183138 1.183138
std1_Water_kg_in_a_m3_mixture -1.083007 -1.083007 -0.598550 -0.114093 0.370364 0.854821 1.339278 1.823735 2.308191 2.792648 3.277105 3.761562 3.761562
std1_Superplasticizer_kg_in_a_m3_mixture -2.049753 -2.049753 -1.460760 -0.871767 -0.282774 0.306219 0.895212 1.484206 2.073199 2.662192 3.251185 3.840178 3.840178
std1_Coarse_Aggregate_kg_in_a_m3_mixture -1.513754 -1.513754 -1.047173 -0.580592 -0.114011 0.352569 0.819150 1.285731 1.752312 2.218893 2.685473 3.152054 3.152054
std1_Fine_Aggregate_kg_in_a_m3_mixture -1.202162 -1.202162 -0.813606 -0.425050 -0.036494 0.352062 0.740618 1.129174 1.517730 1.906286 2.294842 2.683398 2.683398}
External Format¶
When convert_process is executed, the model parameters are saved into different files and are grouped as: general information, gating function, prediction formula, B-spline parameters, and B-spline knot vectors.
General Information¶
This file describes \(FIC\) after learning the model, initial number of components, and the terminal number of components.
fic,num_initial_comps,num_active_comps
-1.294308e+02,8,3
Gate Tree¶
This file describes the structure and parameters of the gate-tree of the model.
{
"gate_tree": {
"gate_type": "bern",
"hard_gate": true,
"nodes": [
{
"node_id": 1,
"node_type": "gate",
"gate_func": {
"aid": "dl1[1]",
"attr_name": "sepal_width_in_cm",
"threshold": 2.5499999999999998e+00,
"prob_left_smaller_than_threshold": 1.0000000000000000e+00
}
},
{
"node_id": 0,
"node_type": "gate",
"gate_func": {
"aid": "dl1[1]",
"attr_name": "sepal_width_in_cm",
"threshold": 3.7500000000000000e+00,
"prob_left_smaller_than_threshold": 1.0000000000000000e+00
}
},
...
{
"node_id": 2,
"node_type": "component",
"comp_id": 2
},
{
"node_id": 5,
"node_type": "component",
"comp_id": 12
},
...
],
"edges": [
{
"source": 1,
"target": 3,
"is_left": false
},
{
"source": 1,
"target": 2,
"is_left": true
},
...
]
}
}
Prediction Formulas¶
This file describes parameters of prediction formulas: weights, bias and variance values.
aid,attr_name,basis_function_index,prediction_formula_12
std1[0],std1_CRIM,0,1.2530167969423509e-02
std1[0],std1_CRIM,1,-7.8803447268247659e-02
std1[0],std1_CRIM,2,-2.3929690668681283e-01
...
,bias,,-1.6628387490763341e-02
,variance,,4.5299007118344711e-02
B-spline Parameters¶
This file describes parameters of B-spline type prediction formulas: degree and the number of basis function for each feature.
degree,basis_dim
3,10
B-spline Knot Vectors¶
This file describes knot vectors of B-spline’s prediction formula.
aid,attr_name,knot_value_0,knot_value_1,knot_value_2,knot_value_3,knot_value_4,knot_value_5,knot_value_6,knot_value_7,knot_value_8,knot_value_9,knot_value_10,knot_value_11,knot_value_12
std1[0],std1_CRIM,-1.2341486543554931e+00,-1.2341486543554931e+00,-8.2810505758699771e-01,-4.2206146081850215e-01,-1.6017864050006603e-02,3.9002573271848906e-01,7.9606932948698450e-01,1.2021129262554799e+00,1.6081565230239758e+00,2.0142001197924713e+00,2.4202437165609667e+00,2.8262873133294621e+00,2.8262873133294621e+00
std1[1],std1_ZN,-1.3831265020520607e+00,-1.3831265020520607e+00,-1.0478983552351340e+00,-7.1267020841820727e-01,-3.7744206160128058e-01,-4.2213914784353879e-02,2.9301423203257282e-01,6.2824237884949952e-01,9.6347052566642621e-01,1.2986986724833531e+00,1.6339268193002798e+00,1.9691549661172065e+00,1.9691549661172065e+00
std1[2],std1_INDUS,-1.9600846596365655e+00,-1.9600846596365655e+00,-1.6593508630904183e+00,-1.3586170665442712e+00,-1.0578832699981242e+00,-7.5714947345197714e-01,-4.5641567690583007e-01,-1.5568188035968311e-01,1.4505191618646407e-01,4.4578571273261125e-01,7.4651950927875799e-01,1.0472533058249049e+00,1.0472533058249049e+00
...
Prediction Result Evaluation¶
The indices used in evaluating prediction results of this component are described below.
Formula |
Description |
---|---|
\(X_{\mbox{p}}\) |
Array of predicted value. |
\(X_{\mbox{a}}\) |
Array of actual value. |
\(\mbox{mean}(X)\) |
Mean of \(X\). |
\(\mbox{median}(X)\) |
Median value in \(X\). |
\(\mbox{max}(X)\) |
Maximum value in \(X\). |
\([\cdot]_+\) |
A function which returns the argument directly if it is greater than \(0\), otherwise returns \(0\). |
Evaluation Index |
Type |
Description |
---|---|---|
root_mean_squared_error |
float |
RMSE (Root Mean Square Error), which is the square root of the mean squared error as shown below:
\(\sqrt{\mbox{mean}((X_{\mbox{p}} - X_{\mbox{a}})^2)}\)
|
root_median_squared_error |
float |
RMdSE (Root Median Square Error), which is the square root of the median squared error as shown below:
\(\sqrt{\mbox{median}((X_{\mbox{p}} - X_{\mbox{a}})^2)}\)
|
mean_abs_error |
float |
Mean of absolute error as shown below:
\(\mbox{mean}(|X_{\mbox{p}} - X_{\mbox{a}}|)\)
|
median_abs_error |
float |
Median of absolute error as shown below:
\(\mbox{median}(|X_{\mbox{p}} - X_{\mbox{a}}|)\)
|
max_abs_error |
float |
Maximum value of absolute error.
\(\mbox{max}(|X_{\mbox{p}} - X_{\mbox{a}}|)\)
|
relative_root_mean_squared_error |
float |
The square root of the mean squared relative error as shown below:
\(\sqrt{\mbox{mean}((\frac{{\large X}_{\mbox{p}} {\large - X}_{\mbox{a}}}{ {\large X}_{\mbox{a}}})^2)}\)
|
relative_root_median_squared_error |
float |
The square root of the median squared relative error as shown below:
\(\sqrt{\mbox{median}((\frac{{\large X}_{\mbox{p}} {\large - X}_{\mbox{a}}}{ {\large X}_{\mbox{a}}})^2)}\)
|
relative_mean_abs_error |
float |
The mean abs relative error as shown below:
\(\mbox{mean}(|\frac{{\large X}_{\mbox{p}} {\large - X}_{\mbox{a}}}{ {\large X}_{\mbox{a}}}|)\)
|
relative_median_abs_error |
float |
The median abs relative error as shown below:
\(\mbox{median}(|\frac{{\large X}_{\mbox{p}} {\large - X}_{\mbox{a}}}{ {\large X}_{\mbox{a}}}|)\)
|
relative_max_abs_error |
float |
The maximum abs relative error as shown below:
\(\mbox{max}(|\frac{{\large X}_{\mbox{p}} {\large - X}_{\mbox{a}}}{ {\large X}_{\mbox{a}}}|)\)
|
positive_side_root_mean_squared_error |
float |
|
positive_side_root_median_squared_error |
float |
|
positive_side_mean_abs_error |
float |
|
positive_side_median_abs_error |
float |
|
positive_side_max_abs_error |
float |
|
negative_side_root_mean_squared_error |
float |
|
negative_side_root_median_squared_error |
float |
|
negative_side_mean_abs_error |
float |
|
negative_side_median_abs_error |
float |
|
negative_side_max_abs_error |
float |
|
max_upside_err_mean_obs |
float |
Proportion of the maximum error for samples that satisfy the condition, \(X_{\mbox{p}} > X_{\mbox{a}}\) against the mean of actual values as shown below:
\(\frac{\mbox{max}({\large X}_{\mbox{p}} {\large - X}_{\mbox{a}})}{\mbox{mean}({\large X}_{\mbox{a}})}\)
|
mean_upside_err_mean_obs |
float |
Proportion of the mean error whose value is only available if it satisfies the condition, \(X_{\mbox{p}} > X_{\mbox{a}}\) (otherwise \(0\)) against the mean of actual values as shown below:
\(\frac{\mbox{mean}([{\large X}_{\mbox{p}} {\large - X}_{\mbox{a}}]_+)}{\mbox{mean}({\large X}_{\mbox{a}})}\)
|
max_downside_err_mean_obs |
float |
Proportion of the maximum error for samples that satisfy the condition, \(X_{\mbox{a}} \geq X_{\mbox{p}}\) against the mean of actual values as shown below:
\(\frac{\mbox{max}({\large X}_{\mbox{a}} {\large - X}_{\mbox{p}})}{\mbox{mean}({\large X}_{\mbox{a}})}\)
|
mean_downside_err_mean_obs |
float |
Proportion of the mean error whose value is only available if it satisfies the condition, \(X_{\mbox{a}} \geq X_{\mbox{p}}\) (otherwise \(0\)) against the mean of actual values as shown below:
\(\frac{\mbox{mean}([{\large X}_{\mbox{a}} - {\large X}_{\mbox{p}}]_+)}{\mbox{mean}({\large X}_{\mbox{a}})}\)
|
negative_pred_num |
int |
The number of the samples that satisfy the condition, \(X_{\mbox{p}} < 0\). |
std_root_mean_squared_error |
float |
|
std_root_median_squared_error |
float |
|
std_mean_abs_error |
float |
|
std_median_abs_error |
float |
|
std_max_abs_error |
float |
|
std_relative_root_mean_squared_error |
float |
|
std_relative_root_median_squared_error |
float |
|
std_relative_mean_abs_error |
float |
|
std_relative_median_abs_error |
float |
|
std_relative_max_abs_error |
float |
|
std_positive_side_root_mean_squared_error |
float |
|
std_positive_side_root_median_squared_error |
float |
|
std_positive_side_mean_abs_error |
float |
|
std_positive_side_median_abs_error |
float |
|
std_positive_side_max_abs_error |
float |
|
std_negative_side_root_mean_squared_error |
float |
|
std_negative_side_root_median_squared_error |
float |
|
std_negative_side_mean_abs_error |
float |
|
std_negative_side_median_abs_error |
float |
|
std_negative_side_max_abs_error |
float |
|
std_max_upside_err_mean_obs |
float |
|
std_mean_upside_err_mean_obs |
float |
|
std_max_downside_err_mean_obs |
float |
|
std_mean_downside_err_mean_obs |
float |
|
std_negative_pred_num |
int |
|
When obtaining these evaluation results in SAMPO API, a pandas.DataFrame is loaded with the evaluation indices as the columns of the DataFrame.
See also
Obtaining process results via ProcessResultLoader
External Format¶
When convert_process is executed, the evaluation results are saved as a CSV file with the evaluation indices as the header of the CSV.
This file describes the evaluation of the prediction result of the component.
root_mean_squared_error,root_median_squared_error,mean_abs_error,median_abs_error,max_abs_error,relative_root_mean_squared_error,relative_root_median_squared_error,relative_mean_abs_error,relative_median_abs_error,relative_max_abs_error,positive_side_root_mean_squared_error,positive_side_root_median_squared_error,positive_side_mean_abs_error,positive_side_median_abs_error,positive_side_max_abs_error,negative_side_root_mean_squared_error,negative_side_root_median_squared_error,negative_side_mean_abs_error,negative_side_median_abs_error,negative_side_max_abs_error,max_upside_err_mean_obs,mean_upside_err_mean_obs,max_downside_err_mean_obs,mean_downside_err_mean_obs,negative_pred_num,std_root_mean_squared_error,std_root_median_squared_error,std_mean_abs_error,std_median_abs_error,std_max_abs_error,std_relative_root_mean_squared_error,std_relative_root_median_squared_error,std_relative_mean_abs_error,std_relative_median_abs_error,std_relative_max_abs_error,std_positive_side_root_mean_squared_error,std_positive_side_root_median_squared_error,std_positive_side_mean_abs_error,std_positive_side_median_abs_error,std_positive_side_max_abs_error,std_negative_side_root_mean_squared_error,std_negative_side_root_median_squared_error,std_negative_side_mean_abs_error,std_negative_side_median_abs_error,std_negative_side_max_abs_error,std_max_upside_err_mean_obs,std_mean_upside_err_mean_obs,std_max_downside_err_mean_obs,std_mean_downside_err_mean_obs,std_negative_pred_num
1.350699e+01,8.838062e+00,1.036833e+01,8.837361e+00,3.377979e+01,8.686816e-01,3.416881e-01,5.519667e-01,3.413176e-01,3.334933e+00,1.356254e+01,9.506539e+00,1.098259e+01,9.506539e+00,3.377979e+01,1.338252e+01,5.174490e+00,9.001098e+00,5.174490e+00,2.990477e+01,1.240240e+00,2.782290e-01,1.097967e+00,1.024486e-01,2,1.135126e+00,7.427497e-01,8.713529e-01,7.426909e-01,2.838850e+00,2.076069e+00,6.108897e-01,1.178233e+00,6.105078e-01,9.166376e+00,1.139795e+00,7.989285e-01,9.229754e-01,7.989285e-01,2.838850e+00,1.124665e+00,4.348636e-01,7.564513e-01,4.348636e-01,2.513193e+00,-2.609989e+00,-5.855117e-01,-2.310587e+00,-2.155952e-01,74