Simple pipeline
You can execute this notebook at: Google collab notebook
!pip install pinard
!pip install scikeras
Collecting pinard
Downloading pinard-0.9.5-py3-none-any.whl (38 kB)
Requirement already satisfied: pandas in c:\workspace\ml\pynirs_env\lib\site-packages (from pinard) (1.3.5)
Requirement already satisfied: scipy in c:\workspace\ml\pynirs_env\lib\site-packages (from pinard) (1.7.3)
Requirement already satisfied: tensorflow in c:\workspace\ml\pynirs_env\lib\site-packages (from pinard) (2.8.0)
Requirement already satisfied: PyWavelets in c:\workspace\ml\pynirs_env\lib\site-packages (from pinard) (1.3.0)
Requirement already satisfied: scikit-learn in c:\workspace\ml\pynirs_env\lib\site-packages (from pinard) (1.0.2)
Requirement already satisfied: numpy in c:\workspace\ml\pynirs_env\lib\site-packages (from pinard) (1.21.6)
Requirement already satisfied: pytz>=2017.3 in c:\workspace\ml\pynirs_env\lib\site-packages (from pandas->pinard) (2022.1)
Requirement already satisfied: python-dateutil>=2.7.3 in c:\workspace\ml\pynirs_env\lib\site-packages (from pandas->pinard) (2.8.2)
Requirement already satisfied: joblib>=0.11 in c:\workspace\ml\pynirs_env\lib\site-packages (from scikit-learn->pinard) (1.1.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from scikit-learn->pinard) (3.1.0)
Requirement already satisfied: absl-py>=0.4.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (1.0.0)
Requirement already satisfied: keras<2.9,>=2.8.0rc0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (2.8.0)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (0.24.0)
Requirement already satisfied: gast>=0.2.1 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (0.5.3)
Requirement already satisfied: termcolor>=1.1.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (1.1.0)
Requirement already satisfied: tf-estimator-nightly==2.8.0.dev2021122109 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (2.8.0.dev2021122109)
Requirement already satisfied: h5py>=2.9.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (3.6.0)
Requirement already satisfied: flatbuffers>=1.12 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (2.0)
Requirement already satisfied: google-pasta>=0.1.1 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (0.2.0)
Requirement already satisfied: protobuf>=3.9.2 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (3.20.0)
Requirement already satisfied: tensorboard<2.9,>=2.8 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (2.8.0)
Requirement already satisfied: setuptools in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (65.5.0)
Requirement already satisfied: typing-extensions>=3.6.6 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (4.1.1)
Requirement already satisfied: libclang>=9.0.1 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (13.0.0)
Requirement already satisfied: six>=1.12.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (1.16.0)
Requirement already satisfied: keras-preprocessing>=1.1.1 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (1.1.2)
Requirement already satisfied: opt-einsum>=2.3.2 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (3.3.0)
Requirement already satisfied: astunparse>=1.6.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (1.6.3)
Requirement already satisfied: wrapt>=1.11.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (1.14.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorflow->pinard) (1.44.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from astunparse>=1.6.0->tensorflow->pinard) (0.37.1)
Requirement already satisfied: cached-property in c:\workspace\ml\pynirs_env\lib\site-packages (from h5py>=2.9.0->tensorflow->pinard) (1.5.2)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorboard<2.9,>=2.8->tensorflow->pinard) (0.4.6)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorboard<2.9,>=2.8->tensorflow->pinard) (0.6.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorboard<2.9,>=2.8->tensorflow->pinard) (1.8.1)
Requirement already satisfied: werkzeug>=0.11.15 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorboard<2.9,>=2.8->tensorflow->pinard) (2.1.1)
Requirement already satisfied: requests<3,>=2.21.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorboard<2.9,>=2.8->tensorflow->pinard) (2.27.1)
Requirement already satisfied: google-auth<3,>=1.6.3 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorboard<2.9,>=2.8->tensorflow->pinard) (2.6.4)
Requirement already satisfied: markdown>=2.6.8 in c:\workspace\ml\pynirs_env\lib\site-packages (from tensorboard<2.9,>=2.8->tensorflow->pinard) (3.3.6)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow->pinard) (5.0.0)
Requirement already satisfied: rsa<5,>=3.1.4 in c:\workspace\ml\pynirs_env\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow->pinard) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in c:\workspace\ml\pynirs_env\lib\site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow->pinard) (0.2.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow->pinard) (1.3.1)
Requirement already satisfied: importlib-metadata>=4.4 in c:\workspace\ml\pynirs_env\lib\site-packages (from markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow->pinard) (4.11.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\workspace\ml\pynirs_env\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow->pinard) (1.26.9)
Requirement already satisfied: certifi>=2017.4.17 in c:\workspace\ml\pynirs_env\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow->pinard) (2021.10.8)
Requirement already satisfied: idna<4,>=2.5 in c:\workspace\ml\pynirs_env\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow->pinard) (3.3)
Requirement already satisfied: charset-normalizer~=2.0.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow->pinard) (2.0.12)
Requirement already satisfied: zipp>=0.5 in c:\workspace\ml\pynirs_env\lib\site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow->pinard) (3.8.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in c:\workspace\ml\pynirs_env\lib\site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow->pinard) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in c:\workspace\ml\pynirs_env\lib\site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow->pinard) (3.2.0)
Installing collected packages: pinard
Successfully installed pinard-0.9.5
[notice] A new release of pip available: 22.3 -> 22.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
Load data
initialize random variables
create a data set to train the pipelines
from pinard import utils
from pinard.model_selection import train_test_split_idx
from sklearn.model_selection import train_test_split
import numpy as np
# Init basic random
rd_seed = 42
np.random.seed(rd_seed)
xcal_csv = "https://raw.githubusercontent.com/GBeurier/pinard/main/examples/Xcal.csv"
ycal_csv = "https://raw.githubusercontent.com/GBeurier/pinard/main/examples/Ycal.csv"
# Create a set named data
x, y = utils.load_csv(xcal_csv, ycal_csv, x_hdr=0, y_hdr=0, remove_na=True)
train_index, test_index = train_test_split_idx(x, y=y, method="random", test_size=0.25, random_state=rd_seed)
X_train, y_train, X_test, y_test = x[train_index], y[train_index], x[test_index], y[test_index]
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape)
(270, 2151) (270,) (91, 2151) (91,)
Declare preprocessing operators
Here we declare the list of preprocessings that will be applied either in FeatureUnion or FeatureAugmentation.
from pinard import preprocessing as pp
from sklearn.pipeline import Pipeline
### Declare preprocessing pipeline components
preprocessing = [ ('id', pp.IdentityTransformer()),
('savgol', pp.SavitzkyGolay()),
('gaussian1', pp.Gaussian(order = 1, sigma = 2)),
]
Simple PLS regression
Here we create a pipeline with a FeatureUnion preprocessing. Then we train the pipeline and display results
from sklearn.pipeline import FeatureUnion
from sklearn.preprocessing import MinMaxScaler
from sklearn.compose import TransformedTargetRegressor
from sklearn.metrics import mean_absolute_error, mean_squared_error, mean_absolute_percentage_error, r2_score
from sklearn.cross_decomposition import PLSRegression
# Simple PLS pipeline declaration
pipeline = Pipeline([
('scaler', MinMaxScaler()),
('preprocessing', FeatureUnion(preprocessing)),
('pls', PLSRegression(n_components=10))
])
# TransformedTargetRegressor is used to apply scaling to Y within the pipeline
estimator = TransformedTargetRegressor(regressor = pipeline, transformer = MinMaxScaler())
# # Train the model
estimator.fit(X_train, y_train)
# # Compute metrics on the test set
Y_preds = estimator.predict(X_test)
print("MAE", mean_absolute_error(y_test, Y_preds))
print("MSE", mean_squared_error(y_test, Y_preds))
print("MAPE", mean_absolute_percentage_error(y_test, Y_preds))
print("R²", r2_score(y_test, Y_preds))
# print(estimator.get_params())
MAE 1.1567468539928398
MSE 2.5117852372966643
MAPE 0.0254993046121325
R² 0.7367788435855969
Same pipeline with a XGBoost estimator
from xgboost import XGBRegressor
x_pipeline = Pipeline([
('scaler', MinMaxScaler()),
('preprocessing', FeatureUnion(preprocessing)),
('XGB', XGBRegressor(n_estimators=50, max_depth=10))
])
x_estimator = TransformedTargetRegressor(regressor = x_pipeline, transformer = MinMaxScaler())
x_estimator.fit(X_train, y_train)
Y_preds = x_estimator.predict(X_test)
print("MAE", mean_absolute_error(y_test, Y_preds))
print("MSE", mean_squared_error(y_test, Y_preds))
print("MAPE", mean_absolute_percentage_error(y_test, Y_preds))
print("R²", r2_score(y_test, Y_preds))
# print(x_estimator.get_params())
MAE 1.2912766
MSE 3.8444414
MAPE 0.028730346
R² 0.5971238611860624
Same pipeline with simple KerasRegressor
A more detailed and complete example is provided in keras_regressor.ipynb.
from pinard.sklearn import FeatureAugmentation
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Conv1D, SpatialDropout1D,BatchNormalization,Flatten, Dropout, Input
from scikeras.wrappers import KerasRegressor
from typing import Dict, Iterable, Any
# Create the keras model in the scikeras wrapper format (meta arg)
def keras_model(meta: Dict[str, Any]):
input_shape = meta["X_shape_"][1:]
model = Sequential()
model.add(Input(shape=input_shape))
model.add(SpatialDropout1D(0.08))
model.add(Conv1D (filters=8, kernel_size=15, strides=5, activation='selu'))
model.add(Dropout(0.2))
model.add(Conv1D (filters=64, kernel_size=21, strides=3, activation='relu'))
model.add(BatchNormalization())
model.add(Conv1D (filters=32, kernel_size=5, strides=3, activation='elu'))
model.add(BatchNormalization())
model.add(Flatten())
model.add(Dense(16, activation='sigmoid'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss = 'mean_squared_error', optimizer = 'adam')
model.summary()
return model
# Create the KerasRegressor
k_regressor = KerasRegressor(model = keras_model,
epochs=400,
fit__batch_size=50,
fit__validation_split=0.2,
verbose = 2)
# Declare the pipeline with a FeatureAugmentation (2D)
k_pipeline = Pipeline([
('scaler', MinMaxScaler()),
('preprocessing', FeatureAugmentation(preprocessing)),
('KerasNN', k_regressor)
])
# Train and predict same as usual
k_estimator = TransformedTargetRegressor(regressor = k_pipeline, transformer = MinMaxScaler())
k_estimator.fit(X_train, y_train)
# print(k_estimator.regressor_[2].history_)
Y_preds = k_estimator.predict(X_test)
print("MAE", mean_absolute_error(y_test, Y_preds))
print("MSE", mean_squared_error(y_test, Y_preds))
print("MAPE", mean_absolute_percentage_error(y_test, Y_preds))
print("R²", r2_score(y_test, Y_preds))
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
spatial_dropout1d (SpatialD (None, 2151, 3) 0
ropout1D)
conv1d (Conv1D) (None, 428, 8) 368
dropout (Dropout) (None, 428, 8) 0
conv1d_1 (Conv1D) (None, 136, 64) 10816
batch_normalization (BatchN (None, 136, 64) 256
ormalization)
conv1d_2 (Conv1D) (None, 44, 32) 10272
batch_normalization_1 (Batc (None, 44, 32) 128
hNormalization)
flatten (Flatten) (None, 1408) 0
dense (Dense) (None, 16) 22544
dense_1 (Dense) (None, 1) 17
=================================================================
Total params: 44,401
Trainable params: 44,209
Non-trainable params: 192
_________________________________________________________________
Epoch 1/400
5/5 - 19s - loss: 0.0294 - val_loss: 0.0220 - 19s/epoch - 4s/step
Epoch 2/400
5/5 - 0s - loss: 0.0266 - val_loss: 0.0192 - 197ms/epoch - 39ms/step
Epoch 3/400
5/5 - 0s - loss: 0.0218 - val_loss: 0.0190 - 157ms/epoch - 31ms/step
Epoch 4/400
5/5 - 0s - loss: 0.0213 - val_loss: 0.0189 - 145ms/epoch - 29ms/step
Epoch 5/400
5/5 - 0s - loss: 0.0208 - val_loss: 0.0176 - 147ms/epoch - 29ms/step
Epoch 6/400
5/5 - 0s - loss: 0.0210 - val_loss: 0.0193 - 147ms/epoch - 29ms/step
Epoch 7/400
5/5 - 0s - loss: 0.0179 - val_loss: 0.0182 - 145ms/epoch - 29ms/step
Epoch 8/400
5/5 - 0s - loss: 0.0164 - val_loss: 0.0184 - 147ms/epoch - 29ms/step
Epoch 9/400
5/5 - 0s - loss: 0.0174 - val_loss: 0.0204 - 165ms/epoch - 33ms/step
Epoch 10/400
5/5 - 0s - loss: 0.0162 - val_loss: 0.0175 - 147ms/epoch - 29ms/step
Epoch 11/400
5/5 - 0s - loss: 0.0156 - val_loss: 0.0177 - 161ms/epoch - 32ms/step
Epoch 12/400
5/5 - 0s - loss: 0.0139 - val_loss: 0.0185 - 144ms/epoch - 29ms/step
Epoch 13/400
5/5 - 0s - loss: 0.0152 - val_loss: 0.0159 - 147ms/epoch - 29ms/step
Epoch 14/400
5/5 - 0s - loss: 0.0141 - val_loss: 0.0164 - 163ms/epoch - 33ms/step
Epoch 15/400
5/5 - 0s - loss: 0.0137 - val_loss: 0.0162 - 164ms/epoch - 33ms/step
Epoch 16/400
5/5 - 0s - loss: 0.0127 - val_loss: 0.0143 - 147ms/epoch - 29ms/step
Epoch 17/400
5/5 - 0s - loss: 0.0142 - val_loss: 0.0141 - 146ms/epoch - 29ms/step
Epoch 18/400
5/5 - 0s - loss: 0.0136 - val_loss: 0.0150 - 147ms/epoch - 29ms/step
Epoch 19/400
5/5 - 0s - loss: 0.0109 - val_loss: 0.0134 - 145ms/epoch - 29ms/step
Epoch 20/400
5/5 - 0s - loss: 0.0105 - val_loss: 0.0140 - 161ms/epoch - 32ms/step
...
Epoch 400/400
5/5 - 0s - loss: 0.0012 - val_loss: 0.0132 - 143ms/epoch - 29ms/step
3/3 - 0s - 361ms/epoch - 120ms/step
MAE 1.4564642
MSE 3.9834785
MAPE 0.031864382
R² 0.5825535320021759
Save and load pipeline
There is two ways to save a pipeline using either pickle or joblib. If a KerasRegressor is used only the pickle method works.
import joblib
import pickle
# save xgb estimator
Y_preds = estimator.predict(X_test)
print("R²", r2_score(y_test, Y_preds))
joblib.dump(estimator, 'xgb_estimator.pkl')
# load xgb estimator
loaded_estimator = joblib.load('xgb_estimator.pkl')
Y_preds = loaded_estimator.predict(X_test)
print("R²", r2_score(y_test, Y_preds))
# save keras estimator
Y_preds = k_estimator.predict(X_test)
print("R²", r2_score(y_test, Y_preds))
with open('keras_estimator.pickle', 'wb') as handle:
pickle.dump(k_estimator, handle, protocol=pickle.HIGHEST_PROTOCOL)
# byte_model = pickle.dumps(estimator, 'xgb_estimator.pkl')
# load keras estimator
with open('keras_estimator.pickle', 'rb') as handle:
loaded_estimator = pickle.load(handle)
# loaded_estimator = pickle.loads(bytes_model)
Y_preds = loaded_estimator.predict(X_test)
print("R²", r2_score(y_test, Y_preds))
R² 0.7367788435855969
R² 0.7367788435855969
3/3 - 0s - 60ms/epoch - 20ms/step
R² 0.5825535320021759
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 3 of 3). These functions will not be directly callable after loading.
INFO:tensorflow:Assets written to: C:\Users\grego\AppData\Local\Temp\tmp53x6_gz7\assets
INFO:tensorflow:Assets written to: C:\Users\grego\AppData\Local\Temp\tmp53x6_gz7\assets
3/3 - 0s - 254ms/epoch - 85ms/step
R² 0.5825535320021759
Simple cross validation with sklearn
from sklearn.model_selection import cross_val_score, cross_val_predict, cross_validate
print("CV_scores", cross_val_score(estimator, x, y, cv=3))
print("-- CV predict --")
Y_preds = cross_val_predict(estimator, x, y, cv=3)
print("MAE", mean_absolute_error(y, Y_preds))
print("MSE", mean_squared_error(y, Y_preds))
print("MAPE", mean_absolute_percentage_error(y, Y_preds))
print("R²", r2_score(y, Y_preds))
print("-- Cross Validate --")
cv_results = cross_validate(estimator, x, y, cv=3, return_train_score=True, n_jobs=3)
for key in cv_results.keys():
print(key, cv_results[key])
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_23356/3318979689.py in <module>
1 from sklearn.model_selection import cross_val_score, cross_val_predict, cross_validate
2
----> 3 print("CV_scores", cross_val_score(estimator, X, y, cv=3))
4 print("-- CV predict --")
5 Y_preds = cross_val_predict(estimator, X, y, cv=3)
NameError: name 'X' is not defined
Same with Keras Regressor
n_jobs parameter seems to deactivate gpu in tensorflow * The CV do not take the best model but the last model. A better way would be to handle cv set by hand and to score the best model per fold. (see keras_regressor)
print("-- Cross Validate --")
cv_results = cross_validate(k_estimator, x, y, cv=3, return_train_score=True)
for key in cv_results.keys():
print(key, cv_results[key])
-- Cross Validate --
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
spatial_dropout1d_1 (Spatia (None, 2151, 9) 0
lDropout1D)
conv1d_3 (Conv1D) (None, 428, 8) 1088
dropout_1 (Dropout) (None, 428, 8) 0
conv1d_4 (Conv1D) (None, 136, 64) 10816
batch_normalization_2 (Batc (None, 136, 64) 256
hNormalization)
conv1d_5 (Conv1D) (None, 44, 32) 10272
batch_normalization_3 (Batc (None, 44, 32) 128
hNormalization)
flatten_1 (Flatten) (None, 1408) 0
dense_2 (Dense) (None, 16) 22544
dense_3 (Dense) (None, 1) 17
=================================================================
Total params: 45,121
Trainable params: 44,929
Non-trainable params: 192
_________________________________________________________________
Epoch 1/400
4/4 - 3s - loss: 0.0364 - val_loss: 0.0542 - 3s/epoch - 679ms/step
Epoch 2/400
4/4 - 0s - loss: 0.0246 - val_loss: 0.0490 - 129ms/epoch - 32ms/step
Epoch 3/400
4/4 - 0s - loss: 0.0213 - val_loss: 0.0432 - 116ms/epoch - 29ms/step
Epoch 4/400
4/4 - 0s - loss: 0.0191 - val_loss: 0.0467 - 126ms/epoch - 31ms/step
Epoch 5/400
4/4 - 0s - loss: 0.0208 - val_loss: 0.0308 - 115ms/epoch - 29ms/step
Epoch 6/400
4/4 - 0s - loss: 0.0168 - val_loss: 0.0369 - 114ms/epoch - 29ms/step
Epoch 7/400
4/4 - 0s - loss: 0.0155 - val_loss: 0.0361 - 122ms/epoch - 31ms/step
Epoch 8/400
4/4 - 0s - loss: 0.0155 - val_loss: 0.0292 - 119ms/epoch - 30ms/step
Epoch 9/400
4/4 - 0s - loss: 0.0140 - val_loss: 0.0319 - 121ms/epoch - 30ms/step
Epoch 10/400
4/4 - 0s - loss: 0.0137 - val_loss: 0.0277 - 127ms/epoch - 32ms/step
Epoch 11/400
4/4 - 0s - loss: 0.0134 - val_loss: 0.0313 - 119ms/epoch - 30ms/step
Epoch 12/400
4/4 - 0s - loss: 0.0130 - val_loss: 0.0346 - 117ms/epoch - 29ms/step
Epoch 13/400
4/4 - 0s - loss: 0.0123 - val_loss: 0.0248 - 122ms/epoch - 31ms/step
Epoch 14/400
4/4 - 0s - loss: 0.0123 - val_loss: 0.0289 - 121ms/epoch - 30ms/step
Epoch 15/400
4/4 - 0s - loss: 0.0122 - val_loss: 0.0331 - 114ms/epoch - 29ms/step
Epoch 16/400
4/4 - 0s - loss: 0.0127 - val_loss: 0.0232 - 130ms/epoch - 33ms/step
Epoch 17/400
4/4 - 0s - loss: 0.0119 - val_loss: 0.0296 - 128ms/epoch - 32ms/step
Epoch 18/400
4/4 - 0s - loss: 0.0117 - val_loss: 0.0258 - 134ms/epoch - 33ms/step
Epoch 19/400
4/4 - 0s - loss: 0.0116 - val_loss: 0.0249 - 129ms/epoch - 32ms/step
Epoch 20/400
4/4 - 0s - loss: 0.0106 - val_loss: 0.0288 - 130ms/epoch - 33ms/step
...
Epoch 400/400
4/4 - 0s - loss: 4.0711e-04 - val_loss: 0.0043 - 142ms/epoch - 36ms/step
TRANSFORM
ahahah (121, 2151, 9)
4/4 - 0s - 401ms/epoch - 100ms/step
TRANSFORM
ahahah (240, 2151, 9)
8/8 - 0s - 156ms/epoch - 20ms/step
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
spatial_dropout1d_2 (Spatia (None, 2151, 9) 0
lDropout1D)
conv1d_6 (Conv1D) (None, 428, 8) 1088
dropout_2 (Dropout) (None, 428, 8) 0
conv1d_7 (Conv1D) (None, 136, 64) 10816
batch_normalization_4 (Batc (None, 136, 64) 256
hNormalization)
conv1d_8 (Conv1D) (None, 44, 32) 10272
batch_normalization_5 (Batc (None, 44, 32) 128
hNormalization)
flatten_2 (Flatten) (None, 1408) 0
dense_4 (Dense) (None, 16) 22544
dense_5 (Dense) (None, 1) 17
=================================================================
Total params: 45,121
Trainable params: 44,929
Non-trainable params: 192
_________________________________________________________________
Epoch 1/400
4/4 - 2s - loss: 0.0347 - val_loss: 0.0205 - 2s/epoch - 613ms/step
Epoch 2/400
4/4 - 0s - loss: 0.0266 - val_loss: 0.0216 - 149ms/epoch - 37ms/step
Epoch 3/400
4/4 - 0s - loss: 0.0203 - val_loss: 0.0194 - 133ms/epoch - 33ms/step
Epoch 4/400
4/4 - 0s - loss: 0.0189 - val_loss: 0.0173 - 124ms/epoch - 31ms/step
Epoch 5/400
4/4 - 0s - loss: 0.0175 - val_loss: 0.0177 - 130ms/epoch - 32ms/step
Epoch 6/400
4/4 - 0s - loss: 0.0161 - val_loss: 0.0193 - 123ms/epoch - 31ms/step
Epoch 7/400
4/4 - 0s - loss: 0.0169 - val_loss: 0.0174 - 133ms/epoch - 33ms/step
Epoch 8/400
4/4 - 0s - loss: 0.0151 - val_loss: 0.0163 - 124ms/epoch - 31ms/step
Epoch 9/400
4/4 - 0s - loss: 0.0149 - val_loss: 0.0164 - 136ms/epoch - 34ms/step
Epoch 10/400
4/4 - 0s - loss: 0.0132 - val_loss: 0.0167 - 137ms/epoch - 34ms/step
Epoch 11/400
4/4 - 0s - loss: 0.0141 - val_loss: 0.0161 - 140ms/epoch - 35ms/step
Epoch 12/400
4/4 - 0s - loss: 0.0134 - val_loss: 0.0158 - 120ms/epoch - 30ms/step
Epoch 13/400
4/4 - 0s - loss: 0.0129 - val_loss: 0.0164 - 123ms/epoch - 31ms/step
Epoch 14/400
4/4 - 0s - loss: 0.0107 - val_loss: 0.0163 - 124ms/epoch - 31ms/step
Epoch 15/400
4/4 - 0s - loss: 0.0119 - val_loss: 0.0166 - 128ms/epoch - 32ms/step
Epoch 16/400
4/4 - 0s - loss: 0.0123 - val_loss: 0.0167 - 140ms/epoch - 35ms/step
Epoch 17/400
4/4 - 0s - loss: 0.0109 - val_loss: 0.0166 - 133ms/epoch - 33ms/step
Epoch 18/400
4/4 - 0s - loss: 0.0102 - val_loss: 0.0161 - 136ms/epoch - 34ms/step
Epoch 19/400
4/4 - 0s - loss: 0.0105 - val_loss: 0.0158 - 143ms/epoch - 36ms/step
Epoch 20/400
4/4 - 0s - loss: 0.0111 - val_loss: 0.0156 - 135ms/epoch - 34ms/step
...
Epoch 400/400
4/4 - 0s - loss: 4.4512e-04 - val_loss: 0.0034 - 125ms/epoch - 31ms/step
TRANSFORM
ahahah (120, 2151, 9)
4/4 - 0s - 364ms/epoch - 91ms/step
TRANSFORM
ahahah (241, 2151, 9)
8/8 - 0s - 132ms/epoch - 16ms/step
Model: "sequential_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
spatial_dropout1d_3 (Spatia (None, 2151, 9) 0
lDropout1D)
conv1d_9 (Conv1D) (None, 428, 8) 1088
dropout_3 (Dropout) (None, 428, 8) 0
conv1d_10 (Conv1D) (None, 136, 64) 10816
batch_normalization_6 (Batc (None, 136, 64) 256
hNormalization)
conv1d_11 (Conv1D) (None, 44, 32) 10272
batch_normalization_7 (Batc (None, 44, 32) 128
hNormalization)
flatten_3 (Flatten) (None, 1408) 0
dense_6 (Dense) (None, 16) 22544
dense_7 (Dense) (None, 1) 17
=================================================================
Total params: 45,121
Trainable params: 44,929
Non-trainable params: 192
_________________________________________________________________
Epoch 1/400
4/4 - 2s - loss: 0.0220 - val_loss: 0.0157 - 2s/epoch - 560ms/step
Epoch 2/400
4/4 - 0s - loss: 0.0210 - val_loss: 0.0163 - 174ms/epoch - 43ms/step
Epoch 3/400
4/4 - 0s - loss: 0.0188 - val_loss: 0.0164 - 161ms/epoch - 40ms/step
Epoch 4/400
4/4 - 0s - loss: 0.0205 - val_loss: 0.0162 - 114ms/epoch - 29ms/step
Epoch 5/400
4/4 - 0s - loss: 0.0182 - val_loss: 0.0161 - 128ms/epoch - 32ms/step
Epoch 6/400
4/4 - 0s - loss: 0.0161 - val_loss: 0.0156 - 117ms/epoch - 29ms/step
Epoch 7/400
4/4 - 0s - loss: 0.0156 - val_loss: 0.0157 - 120ms/epoch - 30ms/step
Epoch 8/400
4/4 - 0s - loss: 0.0145 - val_loss: 0.0155 - 128ms/epoch - 32ms/step
Epoch 9/400
4/4 - 0s - loss: 0.0142 - val_loss: 0.0149 - 157ms/epoch - 39ms/step
Epoch 10/400
4/4 - 0s - loss: 0.0139 - val_loss: 0.0152 - 123ms/epoch - 31ms/step
Epoch 11/400
4/4 - 0s - loss: 0.0133 - val_loss: 0.0151 - 137ms/epoch - 34ms/step
Epoch 12/400
4/4 - 0s - loss: 0.0125 - val_loss: 0.0144 - 139ms/epoch - 35ms/step
Epoch 13/400
4/4 - 0s - loss: 0.0120 - val_loss: 0.0151 - 134ms/epoch - 34ms/step
Epoch 14/400
4/4 - 0s - loss: 0.0118 - val_loss: 0.0148 - 127ms/epoch - 32ms/step
Epoch 15/400
4/4 - 0s - loss: 0.0122 - val_loss: 0.0151 - 124ms/epoch - 31ms/step
Epoch 16/400
4/4 - 0s - loss: 0.0117 - val_loss: 0.0151 - 125ms/epoch - 31ms/step
Epoch 17/400
4/4 - 0s - loss: 0.0118 - val_loss: 0.0140 - 123ms/epoch - 31ms/step
Epoch 18/400
4/4 - 0s - loss: 0.0113 - val_loss: 0.0150 - 128ms/epoch - 32ms/step
Epoch 19/400
4/4 - 0s - loss: 0.0109 - val_loss: 0.0142 - 124ms/epoch - 31ms/step
Epoch 20/400
4/4 - 0s - loss: 0.0101 - val_loss: 0.0136 - 133ms/epoch - 33ms/step
...
Epoch 400/400
4/4 - 0s - loss: 3.2399e-04 - val_loss: 0.0033 - 141ms/epoch - 35ms/step
TRANSFORM
ahahah (120, 2151, 9)
4/4 - 0s - 256ms/epoch - 64ms/step
TRANSFORM
ahahah (241, 2151, 9)
8/8 - 0s - 106ms/epoch - 13ms/step
fit_time [58.65734673 60.80831957 58.15930986]
score_time [0.65953374 0.64429283 0.53594303]
test_score [0.61488104 0.64633656 0.71660909]
train_score [0.92478667 0.9227233 0.95562093]