Data Generation¶
使用 Coefficient 和 Intercept 計算的受試者間 Euclidean Distance¶
Distance matrix shape: (20, 20) Minimum non-zero distance: 0.048 Maximum distance: 1.826 Mean distance: 0.724 Standard deviation: 0.369 Closest subject pair: S001 - S011 (distance: 0.048) Farthest subject pair: S007 - S010 (distance: 1.826)
Subject 的實際資料¶
- Coefficient Matrix in all regimes
- MSVAR series
Train¶
2025/07/21 07:07:07 WARNING mlflow.utils.autologging_utils: MLflow pytorch autologging is known to be compatible with 2.1.0 <= torch <= 2.7.1, but the installed version is 2.7.1+cu128. If you encounter errors during autologging, try upgrading / downgrading torch to a compatible version, or try upgrading MLflow.
INFO:pytorch_lightning.utilities.rank_zero:GPU available: True (cuda), used: True
INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
Metric val_loss improved. New best score: 1.126
Metric val_loss improved by 0.029 >= min_delta = 0.0. New best score: 1.097
Metric val_loss improved by 0.002 >= min_delta = 0.0. New best score: 1.095
Metric val_loss improved by 0.015 >= min_delta = 0.0. New best score: 1.080
Metric val_loss improved by 0.002 >= min_delta = 0.0. New best score: 1.078
Metric val_loss improved by 0.003 >= min_delta = 0.0. New best score: 1.075
Metric val_loss improved by 0.006 >= min_delta = 0.0. New best score: 1.069
Metric val_loss improved by 0.001 >= min_delta = 0.0. New best score: 1.067
Metric val_loss improved by 0.004 >= min_delta = 0.0. New best score: 1.063
Metric val_loss improved by 0.002 >= min_delta = 0.0. New best score: 1.062
Monitored metric val_loss did not improve in the last 10 records. Best score: 1.062. Signaling Trainer to stop.
2025/07/21 07:07:33 WARNING mlflow.utils.requirements_utils: Found torch version (2.7.1+cu128) contains a local version label (+cu128). MLflow logged a pip requirement for this package as 'torch==2.7.1' without the local version label to make it installable from PyPI. To specify pip requirements containing local version labels, please use `conda_env` or `pip_requirements`.
2025/07/21 07:07:36 WARNING mlflow.utils.environment: Failed to resolve installed pip version. ``pip`` will be added to conda.yaml environment spec without a version specifier.
2025/07/21 07:07:36 WARNING mlflow.models.model: Model logged without a signature and input example. Please set `input_example` parameter when logging the model to auto infer the model signature.
Downloading artifacts: 0%| | 0/1 [00:00<?, ?it/s]
Downloading artifacts: 0%| | 0/1 [00:00<?, ?it/s]
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
🏃 View run lstm_wo_subject at: http://localhost:8093/#/experiments/3/runs/423962bbb60f4b70ad7d04c74c855f24 🧪 View experiment at: http://localhost:8093/#/experiments/3
2025/07/21 07:07:39 WARNING mlflow.utils.autologging_utils: MLflow pytorch autologging is known to be compatible with 2.1.0 <= torch <= 2.7.1, but the installed version is 2.7.1+cu128. If you encounter errors during autologging, try upgrading / downgrading torch to a compatible version, or try upgrading MLflow.
INFO:pytorch_lightning.utilities.rank_zero:GPU available: True (cuda), used: True
INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores
INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
Metric val_loss improved. New best score: 1.082
Metric val_loss improved by 0.005 >= min_delta = 0.0. New best score: 1.076
Metric val_loss improved by 0.000 >= min_delta = 0.0. New best score: 1.076
Metric val_loss improved by 0.013 >= min_delta = 0.0. New best score: 1.064
Metric val_loss improved by 0.005 >= min_delta = 0.0. New best score: 1.059
Metric val_loss improved by 0.001 >= min_delta = 0.0. New best score: 1.058
Monitored metric val_loss did not improve in the last 10 records. Best score: 1.058. Signaling Trainer to stop.
2025/07/21 07:07:57 WARNING mlflow.utils.requirements_utils: Found torch version (2.7.1+cu128) contains a local version label (+cu128). MLflow logged a pip requirement for this package as 'torch==2.7.1' without the local version label to make it installable from PyPI. To specify pip requirements containing local version labels, please use `conda_env` or `pip_requirements`.
2025/07/21 07:07:59 WARNING mlflow.utils.environment: Failed to resolve installed pip version. ``pip`` will be added to conda.yaml environment spec without a version specifier.
2025/07/21 07:07:59 WARNING mlflow.models.model: Model logged without a signature and input example. Please set `input_example` parameter when logging the model to auto infer the model signature.
Downloading artifacts: 0%| | 0/1 [00:00<?, ?it/s]
Downloading artifacts: 0%| | 0/1 [00:00<?, ?it/s]
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
🏃 View run lstm_w_subject at: http://localhost:8093/#/experiments/3/runs/9fb125aa12e34ad783389839a637d544 🧪 View experiment at: http://localhost:8093/#/experiments/3
MSE Comparison:¶
- 有做 test split，數值為 test set 上的 MSE (100 time points)
整體 MSE 結果¶
- 原始 MSE df 的三個欄位是 subject_id, variable, MSE, method
- 將 MSE 直接 group by method 後平均的結果
Overall MSE statistics by model:
| model | mean | median | std | min | max | |
|---|---|---|---|---|---|---|
| 0 | naive | 1.416494 | 1.391464 | 0.266860 | 0.983082 | 2.317245 |
| 1 | var_wo_subject | 1.305654 | 1.144451 | 0.481530 | 0.814947 | 3.218416 |
| 2 | lstm_wo_subject | 1.131959 | 1.113514 | 0.182017 | 0.820668 | 1.577590 |
| 3 | var_w_subject | 1.167422 | 1.126004 | 0.274256 | 0.760216 | 2.157579 |
| 4 | lstm_w_subject | 1.145369 | 1.100247 | 0.256644 | 0.784450 | 2.261144 |
Subject MSE 結果¶
- 原始 MSE df 的三個欄位是 subject_id, variable, MSE, method
- 將 MSE group by method, subject_id 後平均的結果
MSE by Subject and Model (Mean values):
| model | naive | var_wo_subject | lstm_wo_subject | var_w_subject | lstm_w_subject | is_w_better_than_wo (lstm) | is_w_better_than_wo (var) | is_lstm_better_than_var (w) |
|---|---|---|---|---|---|---|---|---|
| subject_id | ||||||||
| S001 | 1.511340 | 1.385207 | 1.258406 | 1.314772 | 1.221242 | True | True | True |
| S002 | 1.172132 | 0.961877 | 1.017518 | 0.935402 | 0.959393 | True | True | False |
| S003 | 1.416455 | 1.304144 | 1.403278 | 1.278769 | 1.299473 | True | True | False |
| S004 | 1.239320 | 1.093261 | 1.095139 | 1.108836 | 1.121217 | False | False | False |
| S005 | 1.350614 | 1.064754 | 1.006570 | 0.980729 | 0.977733 | True | True | True |
| S006 | 1.340440 | 1.066818 | 0.953785 | 0.912325 | 0.931094 | True | True | False |
| S007 | 1.748385 | 1.402694 | 1.181220 | 1.289086 | 1.180533 | True | True | True |
| S008 | 1.092009 | 0.992594 | 1.083802 | 0.996418 | 1.036352 | True | False | False |
| S009 | 1.396575 | 1.111734 | 1.048756 | 1.032752 | 1.048022 | True | True | False |
| S010 | 1.362703 | 2.900806 | 1.278660 | 1.812166 | 1.923106 | False | True | False |
| S011 | 1.343167 | 1.164834 | 1.043238 | 1.160499 | 1.073715 | False | True | True |
| S012 | 1.555270 | 1.225907 | 1.085683 | 1.107338 | 1.088226 | False | True | True |
| S013 | 1.216118 | 1.062747 | 1.196036 | 1.080677 | 1.112442 | True | False | False |
| S014 | 1.660973 | 2.198284 | 1.425555 | 1.704362 | 1.377083 | True | True | True |
| S015 | 1.334488 | 1.080425 | 1.028847 | 1.090936 | 1.094756 | False | False | False |
| S016 | 1.724558 | 1.353274 | 1.198341 | 1.142323 | 1.158910 | True | True | False |
| S017 | 1.318283 | 1.077215 | 1.024428 | 1.030014 | 1.017098 | True | True | True |
| S018 | 1.192183 | 0.982516 | 0.981181 | 0.945795 | 0.971863 | True | True | False |
| S019 | 1.726348 | 1.387481 | 1.233545 | 1.292828 | 1.235911 | False | True | True |
| S020 | 1.628521 | 1.296504 | 1.095190 | 1.132414 | 1.079204 | True | True | True |
Subject Time series 實際預測的表現¶
- 可以在圖片上拉來放大。
- 有做 test split，以上 MSE 數值為 test set 上的 MSE (100 time points)。
- 畫圖全部畫出來。