Note
Go to the end to download the full example code
Comparing Custom Lasso Regression with scikit-learn’s Implementation#
This script demonstrates how to implement and train a Lasso regression model using PyTorch Lightning and compares it with scikit-learn’s built-in Lasso regression.
Lasso regression is a linear regression variant that incorporates L1 regularization, leading to sparse weight vectors. In other words, many weights become exactly zero, allowing for simpler and more interpretable models.
In this example:
A custom LassoRegression class is defined using PyTorch Lightning, which incorporates the L1 regularization via a proximal gradient method.
Synthetic data is generated where the ground truth weights are partly set to zero to mimic sparse structures.
Both the custom Lasso model and scikit-learn’s Lasso model are trained on the synthetic data.
The models’ performances are compared based on their mean squared error (MSE) on a test set.
The predicted values of both models are visualized against the true values for a comparative look.
Finally, the learned weights from both models are compared with the true weights through bar plots.
By the end of this script, you should have insights into how Lasso regression can be implemented in PyTorch Lightning and how its performance matches up against traditional implementations in packages like scikit-learn.
Dependencies: - torch - torch.nn - torch.optim - sklearn.model_selection - sklearn.linear_model - numpy - matplotlib - pytorch_lightning - proxtorch
Training: 0it [00:00, ?it/s]
Training: 0%| | 0/70 [00:00<?, ?it/s]
Epoch 0: 0%| | 0/70 [00:00<?, ?it/s] /home/docs/checkouts/readthedocs.org/user_builds/proxtorch/envs/stable/lib/python3.11/site-packages/torch/nn/modules/loss.py:536: UserWarning: Using a target size (torch.Size([1])) that is different to the input size (torch.Size([])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
return F.mse_loss(input, target, reduction=self.reduction)
Epoch 0: 1%|▏ | 1/70 [00:00<00:00, 96.51it/s]
Epoch 0: 1%|▏ | 1/70 [00:00<00:00, 94.14it/s, v_num=2]
Epoch 0: 3%|▎ | 2/70 [00:00<00:00, 160.57it/s, v_num=2]
Epoch 0: 3%|▎ | 2/70 [00:00<00:00, 157.69it/s, v_num=2]
Epoch 0: 4%|▍ | 3/70 [00:00<00:00, 208.25it/s, v_num=2]
Epoch 0: 4%|▍ | 3/70 [00:00<00:00, 205.12it/s, v_num=2]
Epoch 0: 6%|▌ | 4/70 [00:00<00:00, 246.14it/s, v_num=2]
Epoch 0: 6%|▌ | 4/70 [00:00<00:00, 242.90it/s, v_num=2]
Epoch 0: 7%|▋ | 5/70 [00:00<00:00, 275.86it/s, v_num=2]
Epoch 0: 7%|▋ | 5/70 [00:00<00:00, 272.64it/s, v_num=2]
Epoch 0: 9%|▊ | 6/70 [00:00<00:00, 300.85it/s, v_num=2]
Epoch 0: 9%|▊ | 6/70 [00:00<00:00, 297.65it/s, v_num=2]
Epoch 0: 10%|█ | 7/70 [00:00<00:00, 321.05it/s, v_num=2]
Epoch 0: 10%|█ | 7/70 [00:00<00:00, 317.95it/s, v_num=2]
Epoch 0: 11%|█▏ | 8/70 [00:00<00:00, 337.51it/s, v_num=2]
Epoch 0: 11%|█▏ | 8/70 [00:00<00:00, 334.50it/s, v_num=2]
Epoch 0: 13%|█▎ | 9/70 [00:00<00:00, 352.43it/s, v_num=2]
Epoch 0: 13%|█▎ | 9/70 [00:00<00:00, 349.50it/s, v_num=2]
Epoch 0: 14%|█▍ | 10/70 [00:00<00:00, 366.10it/s, v_num=2]
Epoch 0: 14%|█▍ | 10/70 [00:00<00:00, 363.30it/s, v_num=2]
Epoch 0: 16%|█▌ | 11/70 [00:00<00:00, 378.21it/s, v_num=2]
Epoch 0: 16%|█▌ | 11/70 [00:00<00:00, 375.49it/s, v_num=2]
Epoch 0: 17%|█▋ | 12/70 [00:00<00:00, 388.39it/s, v_num=2]
Epoch 0: 17%|█▋ | 12/70 [00:00<00:00, 385.72it/s, v_num=2]
Epoch 0: 19%|█▊ | 13/70 [00:00<00:00, 397.67it/s, v_num=2]
Epoch 0: 19%|█▊ | 13/70 [00:00<00:00, 394.99it/s, v_num=2]
Epoch 0: 20%|██ | 14/70 [00:00<00:00, 405.48it/s, v_num=2]
Epoch 0: 20%|██ | 14/70 [00:00<00:00, 403.01it/s, v_num=2]
Epoch 0: 21%|██▏ | 15/70 [00:00<00:00, 413.36it/s, v_num=2]
Epoch 0: 21%|██▏ | 15/70 [00:00<00:00, 410.96it/s, v_num=2]
Epoch 0: 23%|██▎ | 16/70 [00:00<00:00, 419.70it/s, v_num=2]
Epoch 0: 23%|██▎ | 16/70 [00:00<00:00, 417.32it/s, v_num=2]
Epoch 0: 24%|██▍ | 17/70 [00:00<00:00, 426.07it/s, v_num=2]
Epoch 0: 24%|██▍ | 17/70 [00:00<00:00, 423.82it/s, v_num=2]
Epoch 0: 26%|██▌ | 18/70 [00:00<00:00, 431.07it/s, v_num=2]
Epoch 0: 26%|██▌ | 18/70 [00:00<00:00, 428.88it/s, v_num=2]
Epoch 0: 27%|██▋ | 19/70 [00:00<00:00, 436.22it/s, v_num=2]
Epoch 0: 27%|██▋ | 19/70 [00:00<00:00, 434.10it/s, v_num=2]
Epoch 0: 29%|██▊ | 20/70 [00:00<00:00, 441.14it/s, v_num=2]
Epoch 0: 29%|██▊ | 20/70 [00:00<00:00, 438.72it/s, v_num=2]
Epoch 0: 30%|███ | 21/70 [00:00<00:00, 445.28it/s, v_num=2]
Epoch 0: 30%|███ | 21/70 [00:00<00:00, 443.28it/s, v_num=2]
Epoch 0: 31%|███▏ | 22/70 [00:00<00:00, 449.51it/s, v_num=2]
Epoch 0: 31%|███▏ | 22/70 [00:00<00:00, 447.56it/s, v_num=2]
Epoch 0: 33%|███▎ | 23/70 [00:00<00:00, 453.13it/s, v_num=2]
Epoch 0: 33%|███▎ | 23/70 [00:00<00:00, 451.24it/s, v_num=2]
Epoch 0: 34%|███▍ | 24/70 [00:00<00:00, 456.86it/s, v_num=2]
Epoch 0: 34%|███▍ | 24/70 [00:00<00:00, 455.01it/s, v_num=2]
Epoch 0: 36%|███▌ | 25/70 [00:00<00:00, 460.05it/s, v_num=2]
Epoch 0: 36%|███▌ | 25/70 [00:00<00:00, 458.25it/s, v_num=2]
Epoch 0: 37%|███▋ | 26/70 [00:00<00:00, 463.29it/s, v_num=2]
Epoch 0: 37%|███▋ | 26/70 [00:00<00:00, 461.53it/s, v_num=2]
Epoch 0: 39%|███▊ | 27/70 [00:00<00:00, 466.03it/s, v_num=2]
Epoch 0: 39%|███▊ | 27/70 [00:00<00:00, 464.31it/s, v_num=2]
Epoch 0: 40%|████ | 28/70 [00:00<00:00, 468.81it/s, v_num=2]
Epoch 0: 40%|████ | 28/70 [00:00<00:00, 467.14it/s, v_num=2]
Epoch 0: 41%|████▏ | 29/70 [00:00<00:00, 471.31it/s, v_num=2]
Epoch 0: 41%|████▏ | 29/70 [00:00<00:00, 469.67it/s, v_num=2]
Epoch 0: 43%|████▎ | 30/70 [00:00<00:00, 473.96it/s, v_num=2]
Epoch 0: 43%|████▎ | 30/70 [00:00<00:00, 472.37it/s, v_num=2]
Epoch 0: 44%|████▍ | 31/70 [00:00<00:00, 476.43it/s, v_num=2]
Epoch 0: 44%|████▍ | 31/70 [00:00<00:00, 474.88it/s, v_num=2]
Epoch 0: 46%|████▌ | 32/70 [00:00<00:00, 478.52it/s, v_num=2]
Epoch 0: 46%|████▌ | 32/70 [00:00<00:00, 477.00it/s, v_num=2]
Epoch 0: 47%|████▋ | 33/70 [00:00<00:00, 480.75it/s, v_num=2]
Epoch 0: 47%|████▋ | 33/70 [00:00<00:00, 479.26it/s, v_num=2]
Epoch 0: 49%|████▊ | 34/70 [00:00<00:00, 482.52it/s, v_num=2]
Epoch 0: 49%|████▊ | 34/70 [00:00<00:00, 481.06it/s, v_num=2]
Epoch 0: 50%|█████ | 35/70 [00:00<00:00, 484.60it/s, v_num=2]
Epoch 0: 50%|█████ | 35/70 [00:00<00:00, 483.17it/s, v_num=2]
Epoch 0: 51%|█████▏ | 36/70 [00:00<00:00, 486.21it/s, v_num=2]
Epoch 0: 51%|█████▏ | 36/70 [00:00<00:00, 484.82it/s, v_num=2]
Epoch 0: 53%|█████▎ | 37/70 [00:00<00:00, 488.13it/s, v_num=2]
Epoch 0: 53%|█████▎ | 37/70 [00:00<00:00, 486.75it/s, v_num=2]
Epoch 0: 54%|█████▍ | 38/70 [00:00<00:00, 489.60it/s, v_num=2]
Epoch 0: 54%|█████▍ | 38/70 [00:00<00:00, 488.25it/s, v_num=2]
Epoch 0: 56%|█████▌ | 39/70 [00:00<00:00, 491.34it/s, v_num=2]
Epoch 0: 56%|█████▌ | 39/70 [00:00<00:00, 490.02it/s, v_num=2]
Epoch 0: 57%|█████▋ | 40/70 [00:00<00:00, 492.91it/s, v_num=2]
Epoch 0: 57%|█████▋ | 40/70 [00:00<00:00, 491.62it/s, v_num=2]
Epoch 0: 59%|█████▊ | 41/70 [00:00<00:00, 494.24it/s, v_num=2]
Epoch 0: 59%|█████▊ | 41/70 [00:00<00:00, 492.98it/s, v_num=2]
Epoch 0: 60%|██████ | 42/70 [00:00<00:00, 495.77it/s, v_num=2]
Epoch 0: 60%|██████ | 42/70 [00:00<00:00, 494.52it/s, v_num=2]
Epoch 0: 61%|██████▏ | 43/70 [00:00<00:00, 496.95it/s, v_num=2]
Epoch 0: 61%|██████▏ | 43/70 [00:00<00:00, 495.73it/s, v_num=2]
Epoch 0: 63%|██████▎ | 44/70 [00:00<00:00, 498.33it/s, v_num=2]
Epoch 0: 63%|██████▎ | 44/70 [00:00<00:00, 497.14it/s, v_num=2]
Epoch 0: 64%|██████▍ | 45/70 [00:00<00:00, 499.48it/s, v_num=2]
Epoch 0: 64%|██████▍ | 45/70 [00:00<00:00, 498.29it/s, v_num=2]
Epoch 0: 66%|██████▌ | 46/70 [00:00<00:00, 500.79it/s, v_num=2]
Epoch 0: 66%|██████▌ | 46/70 [00:00<00:00, 499.63it/s, v_num=2]
Epoch 0: 67%|██████▋ | 47/70 [00:00<00:00, 501.86it/s, v_num=2]
Epoch 0: 67%|██████▋ | 47/70 [00:00<00:00, 500.72it/s, v_num=2]
Epoch 0: 69%|██████▊ | 48/70 [00:00<00:00, 503.03it/s, v_num=2]
Epoch 0: 69%|██████▊ | 48/70 [00:00<00:00, 501.91it/s, v_num=2]
Epoch 0: 70%|███████ | 49/70 [00:00<00:00, 503.91it/s, v_num=2]
Epoch 0: 70%|███████ | 49/70 [00:00<00:00, 502.64it/s, v_num=2]
Epoch 0: 71%|███████▏ | 50/70 [00:00<00:00, 504.82it/s, v_num=2]
Epoch 0: 71%|███████▏ | 50/70 [00:00<00:00, 503.74it/s, v_num=2]
Epoch 0: 73%|███████▎ | 51/70 [00:00<00:00, 500.38it/s, v_num=2]
Epoch 0: 73%|███████▎ | 51/70 [00:00<00:00, 499.32it/s, v_num=2]
Epoch 0: 74%|███████▍ | 52/70 [00:00<00:00, 501.44it/s, v_num=2]
Epoch 0: 74%|███████▍ | 52/70 [00:00<00:00, 500.42it/s, v_num=2]
Epoch 0: 76%|███████▌ | 53/70 [00:00<00:00, 502.38it/s, v_num=2]
Epoch 0: 76%|███████▌ | 53/70 [00:00<00:00, 501.36it/s, v_num=2]
Epoch 0: 77%|███████▋ | 54/70 [00:00<00:00, 503.40it/s, v_num=2]
Epoch 0: 77%|███████▋ | 54/70 [00:00<00:00, 502.41it/s, v_num=2]
Epoch 0: 79%|███████▊ | 55/70 [00:00<00:00, 504.41it/s, v_num=2]
Epoch 0: 79%|███████▊ | 55/70 [00:00<00:00, 503.43it/s, v_num=2]
Epoch 0: 80%|████████ | 56/70 [00:00<00:00, 505.19it/s, v_num=2]
Epoch 0: 80%|████████ | 56/70 [00:00<00:00, 504.23it/s, v_num=2]
Epoch 0: 81%|████████▏ | 57/70 [00:00<00:00, 506.17it/s, v_num=2]
Epoch 0: 81%|████████▏ | 57/70 [00:00<00:00, 505.22it/s, v_num=2]
Epoch 0: 83%|████████▎ | 58/70 [00:00<00:00, 506.88it/s, v_num=2]
Epoch 0: 83%|████████▎ | 58/70 [00:00<00:00, 505.94it/s, v_num=2]
Epoch 0: 84%|████████▍ | 59/70 [00:00<00:00, 507.76it/s, v_num=2]
Epoch 0: 84%|████████▍ | 59/70 [00:00<00:00, 506.84it/s, v_num=2]
Epoch 0: 86%|████████▌ | 60/70 [00:00<00:00, 508.47it/s, v_num=2]
Epoch 0: 86%|████████▌ | 60/70 [00:00<00:00, 507.55it/s, v_num=2]
Epoch 0: 87%|████████▋ | 61/70 [00:00<00:00, 509.33it/s, v_num=2]
Epoch 0: 87%|████████▋ | 61/70 [00:00<00:00, 508.44it/s, v_num=2]
Epoch 0: 89%|████████▊ | 62/70 [00:00<00:00, 509.99it/s, v_num=2]
Epoch 0: 89%|████████▊ | 62/70 [00:00<00:00, 509.10it/s, v_num=2]
Epoch 0: 90%|█████████ | 63/70 [00:00<00:00, 510.81it/s, v_num=2]
Epoch 0: 90%|█████████ | 63/70 [00:00<00:00, 509.93it/s, v_num=2]
Epoch 0: 91%|█████████▏| 64/70 [00:00<00:00, 511.55it/s, v_num=2]
Epoch 0: 91%|█████████▏| 64/70 [00:00<00:00, 510.69it/s, v_num=2]
Epoch 0: 93%|█████████▎| 65/70 [00:00<00:00, 512.10it/s, v_num=2]
Epoch 0: 93%|█████████▎| 65/70 [00:00<00:00, 511.24it/s, v_num=2]
Epoch 0: 94%|█████████▍| 66/70 [00:00<00:00, 512.83it/s, v_num=2]
Epoch 0: 94%|█████████▍| 66/70 [00:00<00:00, 511.98it/s, v_num=2]
Epoch 0: 96%|█████████▌| 67/70 [00:00<00:00, 513.37it/s, v_num=2]
Epoch 0: 96%|█████████▌| 67/70 [00:00<00:00, 512.53it/s, v_num=2]
Epoch 0: 97%|█████████▋| 68/70 [00:00<00:00, 514.08it/s, v_num=2]
Epoch 0: 97%|█████████▋| 68/70 [00:00<00:00, 513.26it/s, v_num=2]
Epoch 0: 99%|█████████▊| 69/70 [00:00<00:00, 514.62it/s, v_num=2]
Epoch 0: 99%|█████████▊| 69/70 [00:00<00:00, 513.81it/s, v_num=2]
Epoch 0: 100%|██████████| 70/70 [00:00<00:00, 515.30it/s, v_num=2]
Epoch 0: 100%|██████████| 70/70 [00:00<00:00, 514.50it/s, v_num=2]
Epoch 0: 100%|██████████| 70/70 [00:00<00:00, 512.92it/s, v_num=2]
Epoch 0: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 1: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 1: 1%|▏ | 1/70 [00:00<00:00, 600.39it/s, v_num=2]
Epoch 1: 1%|▏ | 1/70 [00:00<00:00, 531.73it/s, v_num=2]
Epoch 1: 3%|▎ | 2/70 [00:00<00:00, 569.30it/s, v_num=2]
Epoch 1: 3%|▎ | 2/70 [00:00<00:00, 536.42it/s, v_num=2]
Epoch 1: 4%|▍ | 3/70 [00:00<00:00, 566.31it/s, v_num=2]
Epoch 1: 4%|▍ | 3/70 [00:00<00:00, 544.50it/s, v_num=2]
Epoch 1: 6%|▌ | 4/70 [00:00<00:00, 565.35it/s, v_num=2]
Epoch 1: 6%|▌ | 4/70 [00:00<00:00, 545.78it/s, v_num=2]
Epoch 1: 7%|▋ | 5/70 [00:00<00:00, 561.83it/s, v_num=2]
Epoch 1: 7%|▋ | 5/70 [00:00<00:00, 548.73it/s, v_num=2]
Epoch 1: 9%|▊ | 6/70 [00:00<00:00, 557.52it/s, v_num=2]
Epoch 1: 9%|▊ | 6/70 [00:00<00:00, 546.64it/s, v_num=2]
Epoch 1: 10%|█ | 7/70 [00:00<00:00, 555.74it/s, v_num=2]
Epoch 1: 10%|█ | 7/70 [00:00<00:00, 546.47it/s, v_num=2]
Epoch 1: 11%|█▏ | 8/70 [00:00<00:00, 554.43it/s, v_num=2]
Epoch 1: 11%|█▏ | 8/70 [00:00<00:00, 546.34it/s, v_num=2]
Epoch 1: 13%|█▎ | 9/70 [00:00<00:00, 553.74it/s, v_num=2]
Epoch 1: 13%|█▎ | 9/70 [00:00<00:00, 546.53it/s, v_num=2]
Epoch 1: 14%|█▍ | 10/70 [00:00<00:00, 551.63it/s, v_num=2]
Epoch 1: 14%|█▍ | 10/70 [00:00<00:00, 542.47it/s, v_num=2]
Epoch 1: 16%|█▌ | 11/70 [00:00<00:00, 546.16it/s, v_num=2]
Epoch 1: 16%|█▌ | 11/70 [00:00<00:00, 540.42it/s, v_num=2]
Epoch 1: 17%|█▋ | 12/70 [00:00<00:00, 541.81it/s, v_num=2]
Epoch 1: 17%|█▋ | 12/70 [00:00<00:00, 536.61it/s, v_num=2]
Epoch 1: 19%|█▊ | 13/70 [00:00<00:00, 542.31it/s, v_num=2]
Epoch 1: 19%|█▊ | 13/70 [00:00<00:00, 537.49it/s, v_num=2]
Epoch 1: 20%|██ | 14/70 [00:00<00:00, 543.58it/s, v_num=2]
Epoch 1: 20%|██ | 14/70 [00:00<00:00, 539.08it/s, v_num=2]
Epoch 1: 21%|██▏ | 15/70 [00:00<00:00, 543.90it/s, v_num=2]
Epoch 1: 21%|██▏ | 15/70 [00:00<00:00, 539.70it/s, v_num=2]
Epoch 1: 23%|██▎ | 16/70 [00:00<00:00, 544.90it/s, v_num=2]
Epoch 1: 23%|██▎ | 16/70 [00:00<00:00, 540.97it/s, v_num=2]
Epoch 1: 24%|██▍ | 17/70 [00:00<00:00, 545.86it/s, v_num=2]
Epoch 1: 24%|██▍ | 17/70 [00:00<00:00, 541.68it/s, v_num=2]
Epoch 1: 26%|██▌ | 18/70 [00:00<00:00, 546.25it/s, v_num=2]
Epoch 1: 26%|██▌ | 18/70 [00:00<00:00, 542.71it/s, v_num=2]
Epoch 1: 27%|██▋ | 19/70 [00:00<00:00, 543.77it/s, v_num=2]
Epoch 1: 27%|██▋ | 19/70 [00:00<00:00, 540.62it/s, v_num=2]
Epoch 1: 29%|██▊ | 20/70 [00:00<00:00, 544.84it/s, v_num=2]
Epoch 1: 29%|██▊ | 20/70 [00:00<00:00, 541.67it/s, v_num=2]
Epoch 1: 30%|███ | 21/70 [00:00<00:00, 545.75it/s, v_num=2]
Epoch 1: 30%|███ | 21/70 [00:00<00:00, 542.72it/s, v_num=2]
Epoch 1: 31%|███▏ | 22/70 [00:00<00:00, 545.96it/s, v_num=2]
Epoch 1: 31%|███▏ | 22/70 [00:00<00:00, 543.07it/s, v_num=2]
Epoch 1: 33%|███▎ | 23/70 [00:00<00:00, 543.72it/s, v_num=2]
Epoch 1: 33%|███▎ | 23/70 [00:00<00:00, 540.97it/s, v_num=2]
Epoch 1: 34%|███▍ | 24/70 [00:00<00:00, 544.06it/s, v_num=2]
Epoch 1: 34%|███▍ | 24/70 [00:00<00:00, 541.43it/s, v_num=2]
Epoch 1: 36%|███▌ | 25/70 [00:00<00:00, 544.84it/s, v_num=2]
Epoch 1: 36%|███▌ | 25/70 [00:00<00:00, 542.32it/s, v_num=2]
Epoch 1: 37%|███▋ | 26/70 [00:00<00:00, 545.02it/s, v_num=2]
Epoch 1: 37%|███▋ | 26/70 [00:00<00:00, 542.60it/s, v_num=2]
Epoch 1: 39%|███▊ | 27/70 [00:00<00:00, 545.79it/s, v_num=2]
Epoch 1: 39%|███▊ | 27/70 [00:00<00:00, 543.44it/s, v_num=2]
Epoch 1: 40%|████ | 28/70 [00:00<00:00, 543.04it/s, v_num=2]
Epoch 1: 40%|████ | 28/70 [00:00<00:00, 540.79it/s, v_num=2]
Epoch 1: 41%|████▏ | 29/70 [00:00<00:00, 543.74it/s, v_num=2]
Epoch 1: 41%|████▏ | 29/70 [00:00<00:00, 541.56it/s, v_num=2]
Epoch 1: 43%|████▎ | 30/70 [00:00<00:00, 544.25it/s, v_num=2]
Epoch 1: 43%|████▎ | 30/70 [00:00<00:00, 541.93it/s, v_num=2]
Epoch 1: 44%|████▍ | 31/70 [00:00<00:00, 528.77it/s, v_num=2]
Epoch 1: 44%|████▍ | 31/70 [00:00<00:00, 526.81it/s, v_num=2]
Epoch 1: 46%|████▌ | 32/70 [00:00<00:00, 529.18it/s, v_num=2]
Epoch 1: 46%|████▌ | 32/70 [00:00<00:00, 527.30it/s, v_num=2]
Epoch 1: 47%|████▋ | 33/70 [00:00<00:00, 530.15it/s, v_num=2]
Epoch 1: 47%|████▋ | 33/70 [00:00<00:00, 528.32it/s, v_num=2]
Epoch 1: 49%|████▊ | 34/70 [00:00<00:00, 530.64it/s, v_num=2]
Epoch 1: 49%|████▊ | 34/70 [00:00<00:00, 528.88it/s, v_num=2]
Epoch 1: 50%|█████ | 35/70 [00:00<00:00, 531.34it/s, v_num=2]
Epoch 1: 50%|█████ | 35/70 [00:00<00:00, 529.61it/s, v_num=2]
Epoch 1: 51%|█████▏ | 36/70 [00:00<00:00, 529.78it/s, v_num=2]
Epoch 1: 51%|█████▏ | 36/70 [00:00<00:00, 528.08it/s, v_num=2]
Epoch 1: 53%|█████▎ | 37/70 [00:00<00:00, 530.57it/s, v_num=2]
Epoch 1: 53%|█████▎ | 37/70 [00:00<00:00, 528.93it/s, v_num=2]
Epoch 1: 54%|█████▍ | 38/70 [00:00<00:00, 531.04it/s, v_num=2]
Epoch 1: 54%|█████▍ | 38/70 [00:00<00:00, 529.44it/s, v_num=2]
Epoch 1: 56%|█████▌ | 39/70 [00:00<00:00, 531.82it/s, v_num=2]
Epoch 1: 56%|█████▌ | 39/70 [00:00<00:00, 530.27it/s, v_num=2]
Epoch 1: 57%|█████▋ | 40/70 [00:00<00:00, 530.78it/s, v_num=2]
Epoch 1: 57%|█████▋ | 40/70 [00:00<00:00, 529.27it/s, v_num=2]
Epoch 1: 59%|█████▊ | 41/70 [00:00<00:00, 531.54it/s, v_num=2]
Epoch 1: 59%|█████▊ | 41/70 [00:00<00:00, 530.07it/s, v_num=2]
Epoch 1: 60%|██████ | 42/70 [00:00<00:00, 532.26it/s, v_num=2]
Epoch 1: 60%|██████ | 42/70 [00:00<00:00, 530.73it/s, v_num=2]
Epoch 1: 61%|██████▏ | 43/70 [00:00<00:00, 532.72it/s, v_num=2]
Epoch 1: 61%|██████▏ | 43/70 [00:00<00:00, 531.32it/s, v_num=2]
Epoch 1: 63%|██████▎ | 44/70 [00:00<00:00, 532.11it/s, v_num=2]
Epoch 1: 63%|██████▎ | 44/70 [00:00<00:00, 530.69it/s, v_num=2]
Epoch 1: 64%|██████▍ | 45/70 [00:00<00:00, 532.50it/s, v_num=2]
Epoch 1: 64%|██████▍ | 45/70 [00:00<00:00, 531.16it/s, v_num=2]
Epoch 1: 66%|██████▌ | 46/70 [00:00<00:00, 533.14it/s, v_num=2]
Epoch 1: 66%|██████▌ | 46/70 [00:00<00:00, 531.82it/s, v_num=2]
Epoch 1: 67%|██████▋ | 47/70 [00:00<00:00, 533.54it/s, v_num=2]
Epoch 1: 67%|██████▋ | 47/70 [00:00<00:00, 532.24it/s, v_num=2]
Epoch 1: 69%|██████▊ | 48/70 [00:00<00:00, 534.15it/s, v_num=2]
Epoch 1: 69%|██████▊ | 48/70 [00:00<00:00, 532.88it/s, v_num=2]
Epoch 1: 70%|███████ | 49/70 [00:00<00:00, 534.47it/s, v_num=2]
Epoch 1: 70%|███████ | 49/70 [00:00<00:00, 533.21it/s, v_num=2]
Epoch 1: 71%|███████▏ | 50/70 [00:00<00:00, 535.01it/s, v_num=2]
Epoch 1: 71%|███████▏ | 50/70 [00:00<00:00, 533.79it/s, v_num=2]
Epoch 1: 73%|███████▎ | 51/70 [00:00<00:00, 535.33it/s, v_num=2]
Epoch 1: 73%|███████▎ | 51/70 [00:00<00:00, 534.12it/s, v_num=2]
Epoch 1: 74%|███████▍ | 52/70 [00:00<00:00, 535.82it/s, v_num=2]
Epoch 1: 74%|███████▍ | 52/70 [00:00<00:00, 534.64it/s, v_num=2]
Epoch 1: 76%|███████▌ | 53/70 [00:00<00:00, 536.36it/s, v_num=2]
Epoch 1: 76%|███████▌ | 53/70 [00:00<00:00, 535.20it/s, v_num=2]
Epoch 1: 77%|███████▋ | 54/70 [00:00<00:00, 536.64it/s, v_num=2]
Epoch 1: 77%|███████▋ | 54/70 [00:00<00:00, 535.49it/s, v_num=2]
Epoch 1: 79%|███████▊ | 55/70 [00:00<00:00, 537.04it/s, v_num=2]
Epoch 1: 79%|███████▊ | 55/70 [00:00<00:00, 535.91it/s, v_num=2]
Epoch 1: 80%|████████ | 56/70 [00:00<00:00, 536.95it/s, v_num=2]
Epoch 1: 80%|████████ | 56/70 [00:00<00:00, 535.84it/s, v_num=2]
Epoch 1: 81%|████████▏ | 57/70 [00:00<00:00, 537.43it/s, v_num=2]
Epoch 1: 81%|████████▏ | 57/70 [00:00<00:00, 536.34it/s, v_num=2]
Epoch 1: 83%|████████▎ | 58/70 [00:00<00:00, 537.55it/s, v_num=2]
Epoch 1: 83%|████████▎ | 58/70 [00:00<00:00, 536.49it/s, v_num=2]
Epoch 1: 84%|████████▍ | 59/70 [00:00<00:00, 538.00it/s, v_num=2]
Epoch 1: 84%|████████▍ | 59/70 [00:00<00:00, 536.96it/s, v_num=2]
Epoch 1: 86%|████████▌ | 60/70 [00:00<00:00, 538.23it/s, v_num=2]
Epoch 1: 86%|████████▌ | 60/70 [00:00<00:00, 537.19it/s, v_num=2]
Epoch 1: 87%|████████▋ | 61/70 [00:00<00:00, 538.66it/s, v_num=2]
Epoch 1: 87%|████████▋ | 61/70 [00:00<00:00, 537.65it/s, v_num=2]
Epoch 1: 89%|████████▊ | 62/70 [00:00<00:00, 538.37it/s, v_num=2]
Epoch 1: 89%|████████▊ | 62/70 [00:00<00:00, 537.23it/s, v_num=2]
Epoch 1: 90%|█████████ | 63/70 [00:00<00:00, 538.63it/s, v_num=2]
Epoch 1: 90%|█████████ | 63/70 [00:00<00:00, 537.64it/s, v_num=2]
Epoch 1: 91%|█████████▏| 64/70 [00:00<00:00, 539.02it/s, v_num=2]
Epoch 1: 91%|█████████▏| 64/70 [00:00<00:00, 538.04it/s, v_num=2]
Epoch 1: 93%|█████████▎| 65/70 [00:00<00:00, 539.19it/s, v_num=2]
Epoch 1: 93%|█████████▎| 65/70 [00:00<00:00, 538.23it/s, v_num=2]
Epoch 1: 94%|█████████▍| 66/70 [00:00<00:00, 539.54it/s, v_num=2]
Epoch 1: 94%|█████████▍| 66/70 [00:00<00:00, 538.59it/s, v_num=2]
Epoch 1: 96%|█████████▌| 67/70 [00:00<00:00, 539.46it/s, v_num=2]
Epoch 1: 96%|█████████▌| 67/70 [00:00<00:00, 538.54it/s, v_num=2]
Epoch 1: 97%|█████████▋| 68/70 [00:00<00:00, 539.80it/s, v_num=2]
Epoch 1: 97%|█████████▋| 68/70 [00:00<00:00, 538.90it/s, v_num=2]
Epoch 1: 99%|█████████▊| 69/70 [00:00<00:00, 539.95it/s, v_num=2]
Epoch 1: 99%|█████████▊| 69/70 [00:00<00:00, 539.04it/s, v_num=2]
Epoch 1: 100%|██████████| 70/70 [00:00<00:00, 540.28it/s, v_num=2]
Epoch 1: 100%|██████████| 70/70 [00:00<00:00, 539.40it/s, v_num=2]
Epoch 1: 100%|██████████| 70/70 [00:00<00:00, 537.66it/s, v_num=2]
Epoch 1: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 2: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 2: 1%|▏ | 1/70 [00:00<00:00, 602.98it/s, v_num=2]
Epoch 2: 1%|▏ | 1/70 [00:00<00:00, 533.97it/s, v_num=2]
Epoch 2: 3%|▎ | 2/70 [00:00<00:00, 570.58it/s, v_num=2]
Epoch 2: 3%|▎ | 2/70 [00:00<00:00, 537.94it/s, v_num=2]
Epoch 2: 4%|▍ | 3/70 [00:00<00:00, 568.90it/s, v_num=2]
Epoch 2: 4%|▍ | 3/70 [00:00<00:00, 547.11it/s, v_num=2]
Epoch 2: 6%|▌ | 4/70 [00:00<00:00, 567.55it/s, v_num=2]
Epoch 2: 6%|▌ | 4/70 [00:00<00:00, 544.17it/s, v_num=2]
Epoch 2: 7%|▋ | 5/70 [00:00<00:00, 549.25it/s, v_num=2]
Epoch 2: 7%|▋ | 5/70 [00:00<00:00, 536.64it/s, v_num=2]
Epoch 2: 9%|▊ | 6/70 [00:00<00:00, 551.65it/s, v_num=2]
Epoch 2: 9%|▊ | 6/70 [00:00<00:00, 541.05it/s, v_num=2]
Epoch 2: 10%|█ | 7/70 [00:00<00:00, 550.96it/s, v_num=2]
Epoch 2: 10%|█ | 7/70 [00:00<00:00, 541.84it/s, v_num=2]
Epoch 2: 11%|█▏ | 8/70 [00:00<00:00, 552.63it/s, v_num=2]
Epoch 2: 11%|█▏ | 8/70 [00:00<00:00, 544.58it/s, v_num=2]
Epoch 2: 13%|█▎ | 9/70 [00:00<00:00, 552.15it/s, v_num=2]
Epoch 2: 13%|█▎ | 9/70 [00:00<00:00, 545.01it/s, v_num=2]
Epoch 2: 14%|█▍ | 10/70 [00:00<00:00, 552.92it/s, v_num=2]
Epoch 2: 14%|█▍ | 10/70 [00:00<00:00, 546.46it/s, v_num=2]
Epoch 2: 16%|█▌ | 11/70 [00:00<00:00, 531.86it/s, v_num=2]
Epoch 2: 16%|█▌ | 11/70 [00:00<00:00, 526.32it/s, v_num=2]
Epoch 2: 17%|█▋ | 12/70 [00:00<00:00, 534.15it/s, v_num=2]
Epoch 2: 17%|█▋ | 12/70 [00:00<00:00, 529.14it/s, v_num=2]
Epoch 2: 19%|█▊ | 13/70 [00:00<00:00, 535.09it/s, v_num=2]
Epoch 2: 19%|█▊ | 13/70 [00:00<00:00, 530.37it/s, v_num=2]
Epoch 2: 20%|██ | 14/70 [00:00<00:00, 536.72it/s, v_num=2]
Epoch 2: 20%|██ | 14/70 [00:00<00:00, 532.33it/s, v_num=2]
Epoch 2: 21%|██▏ | 15/70 [00:00<00:00, 535.91it/s, v_num=2]
Epoch 2: 21%|██▏ | 15/70 [00:00<00:00, 531.84it/s, v_num=2]
Epoch 2: 23%|██▎ | 16/70 [00:00<00:00, 537.46it/s, v_num=2]
Epoch 2: 23%|██▎ | 16/70 [00:00<00:00, 533.63it/s, v_num=2]
Epoch 2: 24%|██▍ | 17/70 [00:00<00:00, 537.98it/s, v_num=2]
Epoch 2: 24%|██▍ | 17/70 [00:00<00:00, 534.37it/s, v_num=2]
Epoch 2: 26%|██▌ | 18/70 [00:00<00:00, 539.33it/s, v_num=2]
Epoch 2: 26%|██▌ | 18/70 [00:00<00:00, 535.90it/s, v_num=2]
Epoch 2: 27%|██▋ | 19/70 [00:00<00:00, 540.42it/s, v_num=2]
Epoch 2: 27%|██▋ | 19/70 [00:00<00:00, 536.62it/s, v_num=2]
Epoch 2: 29%|██▊ | 20/70 [00:00<00:00, 540.90it/s, v_num=2]
Epoch 2: 29%|██▊ | 20/70 [00:00<00:00, 537.50it/s, v_num=2]
Epoch 2: 30%|███ | 21/70 [00:00<00:00, 541.59it/s, v_num=2]
Epoch 2: 30%|███ | 21/70 [00:00<00:00, 538.63it/s, v_num=2]
Epoch 2: 31%|███▏ | 22/70 [00:00<00:00, 542.08it/s, v_num=2]
Epoch 2: 31%|███▏ | 22/70 [00:00<00:00, 539.24it/s, v_num=2]
Epoch 2: 33%|███▎ | 23/70 [00:00<00:00, 542.88it/s, v_num=2]
Epoch 2: 33%|███▎ | 23/70 [00:00<00:00, 540.15it/s, v_num=2]
Epoch 2: 34%|███▍ | 24/70 [00:00<00:00, 543.17it/s, v_num=2]
Epoch 2: 34%|███▍ | 24/70 [00:00<00:00, 540.57it/s, v_num=2]
Epoch 2: 36%|███▌ | 25/70 [00:00<00:00, 543.38it/s, v_num=2]
Epoch 2: 36%|███▌ | 25/70 [00:00<00:00, 540.88it/s, v_num=2]
Epoch 2: 37%|███▋ | 26/70 [00:00<00:00, 543.68it/s, v_num=2]
Epoch 2: 37%|███▋ | 26/70 [00:00<00:00, 541.26it/s, v_num=2]
Epoch 2: 39%|███▊ | 27/70 [00:00<00:00, 544.38it/s, v_num=2]
Epoch 2: 39%|███▊ | 27/70 [00:00<00:00, 542.05it/s, v_num=2]
Epoch 2: 40%|████ | 28/70 [00:00<00:00, 544.60it/s, v_num=2]
Epoch 2: 40%|████ | 28/70 [00:00<00:00, 542.34it/s, v_num=2]
Epoch 2: 41%|████▏ | 29/70 [00:00<00:00, 545.26it/s, v_num=2]
Epoch 2: 41%|████▏ | 29/70 [00:00<00:00, 543.09it/s, v_num=2]
Epoch 2: 43%|████▎ | 30/70 [00:00<00:00, 545.92it/s, v_num=2]
Epoch 2: 43%|████▎ | 30/70 [00:00<00:00, 543.64it/s, v_num=2]
Epoch 2: 44%|████▍ | 31/70 [00:00<00:00, 546.03it/s, v_num=2]
Epoch 2: 44%|████▍ | 31/70 [00:00<00:00, 543.98it/s, v_num=2]
Epoch 2: 46%|████▌ | 32/70 [00:00<00:00, 546.61it/s, v_num=2]
Epoch 2: 46%|████▌ | 32/70 [00:00<00:00, 543.80it/s, v_num=2]
Epoch 2: 47%|████▋ | 33/70 [00:00<00:00, 545.97it/s, v_num=2]
Epoch 2: 47%|████▋ | 33/70 [00:00<00:00, 544.04it/s, v_num=2]
Epoch 2: 49%|████▊ | 34/70 [00:00<00:00, 546.47it/s, v_num=2]
Epoch 2: 49%|████▊ | 34/70 [00:00<00:00, 544.61it/s, v_num=2]
Epoch 2: 50%|█████ | 35/70 [00:00<00:00, 546.72it/s, v_num=2]
Epoch 2: 50%|█████ | 35/70 [00:00<00:00, 544.89it/s, v_num=2]
Epoch 2: 51%|█████▏ | 36/70 [00:00<00:00, 547.24it/s, v_num=2]
Epoch 2: 51%|█████▏ | 36/70 [00:00<00:00, 545.47it/s, v_num=2]
Epoch 2: 53%|█████▎ | 37/70 [00:00<00:00, 547.48it/s, v_num=2]
Epoch 2: 53%|█████▎ | 37/70 [00:00<00:00, 545.75it/s, v_num=2]
Epoch 2: 54%|█████▍ | 38/70 [00:00<00:00, 547.98it/s, v_num=2]
Epoch 2: 54%|█████▍ | 38/70 [00:00<00:00, 546.28it/s, v_num=2]
Epoch 2: 56%|█████▌ | 39/70 [00:00<00:00, 548.43it/s, v_num=2]
Epoch 2: 56%|█████▌ | 39/70 [00:00<00:00, 546.53it/s, v_num=2]
Epoch 2: 57%|█████▋ | 40/70 [00:00<00:00, 548.56it/s, v_num=2]
Epoch 2: 57%|█████▋ | 40/70 [00:00<00:00, 546.95it/s, v_num=2]
Epoch 2: 59%|█████▊ | 41/70 [00:00<00:00, 548.97it/s, v_num=2]
Epoch 2: 59%|█████▊ | 41/70 [00:00<00:00, 547.40it/s, v_num=2]
Epoch 2: 60%|██████ | 42/70 [00:00<00:00, 549.08it/s, v_num=2]
Epoch 2: 60%|██████ | 42/70 [00:00<00:00, 547.55it/s, v_num=2]
Epoch 2: 61%|██████▏ | 43/70 [00:00<00:00, 549.39it/s, v_num=2]
Epoch 2: 61%|██████▏ | 43/70 [00:00<00:00, 547.89it/s, v_num=2]
Epoch 2: 63%|██████▎ | 44/70 [00:00<00:00, 549.16it/s, v_num=2]
Epoch 2: 63%|██████▎ | 44/70 [00:00<00:00, 547.08it/s, v_num=2]
Epoch 2: 64%|██████▍ | 45/70 [00:00<00:00, 546.48it/s, v_num=2]
Epoch 2: 64%|██████▍ | 45/70 [00:00<00:00, 545.20it/s, v_num=2]
Epoch 2: 66%|██████▌ | 46/70 [00:00<00:00, 545.62it/s, v_num=2]
Epoch 2: 66%|██████▌ | 46/70 [00:00<00:00, 544.22it/s, v_num=2]
Epoch 2: 67%|██████▋ | 47/70 [00:00<00:00, 545.99it/s, v_num=2]
Epoch 2: 67%|██████▋ | 47/70 [00:00<00:00, 544.63it/s, v_num=2]
Epoch 2: 69%|██████▊ | 48/70 [00:00<00:00, 546.07it/s, v_num=2]
Epoch 2: 69%|██████▊ | 48/70 [00:00<00:00, 544.74it/s, v_num=2]
Epoch 2: 70%|███████ | 49/70 [00:00<00:00, 546.40it/s, v_num=2]
Epoch 2: 70%|███████ | 49/70 [00:00<00:00, 545.09it/s, v_num=2]
Epoch 2: 71%|███████▏ | 50/70 [00:00<00:00, 545.32it/s, v_num=2]
Epoch 2: 71%|███████▏ | 50/70 [00:00<00:00, 543.95it/s, v_num=2]
Epoch 2: 73%|███████▎ | 51/70 [00:00<00:00, 545.60it/s, v_num=2]
Epoch 2: 73%|███████▎ | 51/70 [00:00<00:00, 544.36it/s, v_num=2]
Epoch 2: 74%|███████▍ | 52/70 [00:00<00:00, 545.66it/s, v_num=2]
Epoch 2: 74%|███████▍ | 52/70 [00:00<00:00, 544.43it/s, v_num=2]
Epoch 2: 76%|███████▌ | 53/70 [00:00<00:00, 545.94it/s, v_num=2]
Epoch 2: 76%|███████▌ | 53/70 [00:00<00:00, 544.73it/s, v_num=2]
Epoch 2: 77%|███████▋ | 54/70 [00:00<00:00, 546.29it/s, v_num=2]
Epoch 2: 77%|███████▋ | 54/70 [00:00<00:00, 545.11it/s, v_num=2]
Epoch 2: 79%|███████▊ | 55/70 [00:00<00:00, 544.92it/s, v_num=2]
Epoch 2: 79%|███████▊ | 55/70 [00:00<00:00, 543.75it/s, v_num=2]
Epoch 2: 80%|████████ | 56/70 [00:00<00:00, 545.27it/s, v_num=2]
Epoch 2: 80%|████████ | 56/70 [00:00<00:00, 543.63it/s, v_num=2]
Epoch 2: 81%|████████▏ | 57/70 [00:00<00:00, 544.55it/s, v_num=2]
Epoch 2: 81%|████████▏ | 57/70 [00:00<00:00, 543.44it/s, v_num=2]
Epoch 2: 83%|████████▎ | 58/70 [00:00<00:00, 544.87it/s, v_num=2]
Epoch 2: 83%|████████▎ | 58/70 [00:00<00:00, 543.78it/s, v_num=2]
Epoch 2: 84%|████████▍ | 59/70 [00:00<00:00, 544.01it/s, v_num=2]
Epoch 2: 84%|████████▍ | 59/70 [00:00<00:00, 542.90it/s, v_num=2]
Epoch 2: 86%|████████▌ | 60/70 [00:00<00:00, 544.33it/s, v_num=2]
Epoch 2: 86%|████████▌ | 60/70 [00:00<00:00, 543.28it/s, v_num=2]
Epoch 2: 87%|████████▋ | 61/70 [00:00<00:00, 537.43it/s, v_num=2]
Epoch 2: 87%|████████▋ | 61/70 [00:00<00:00, 536.41it/s, v_num=2]
Epoch 2: 89%|████████▊ | 62/70 [00:00<00:00, 537.53it/s, v_num=2]
Epoch 2: 89%|████████▊ | 62/70 [00:00<00:00, 536.52it/s, v_num=2]
Epoch 2: 90%|█████████ | 63/70 [00:00<00:00, 537.69it/s, v_num=2]
Epoch 2: 90%|█████████ | 63/70 [00:00<00:00, 536.17it/s, v_num=2]
Epoch 2: 91%|█████████▏| 64/70 [00:00<00:00, 537.45it/s, v_num=2]
Epoch 2: 91%|█████████▏| 64/70 [00:00<00:00, 536.31it/s, v_num=2]
Epoch 2: 93%|█████████▎| 65/70 [00:00<00:00, 537.61it/s, v_num=2]
Epoch 2: 93%|█████████▎| 65/70 [00:00<00:00, 536.66it/s, v_num=2]
Epoch 2: 94%|█████████▍| 66/70 [00:00<00:00, 538.00it/s, v_num=2]
Epoch 2: 94%|█████████▍| 66/70 [00:00<00:00, 537.07it/s, v_num=2]
Epoch 2: 96%|█████████▌| 67/70 [00:00<00:00, 538.06it/s, v_num=2]
Epoch 2: 96%|█████████▌| 67/70 [00:00<00:00, 537.15it/s, v_num=2]
Epoch 2: 97%|█████████▋| 68/70 [00:00<00:00, 537.49it/s, v_num=2]
Epoch 2: 97%|█████████▋| 68/70 [00:00<00:00, 536.57it/s, v_num=2]
Epoch 2: 99%|█████████▊| 69/70 [00:00<00:00, 537.66it/s, v_num=2]
Epoch 2: 99%|█████████▊| 69/70 [00:00<00:00, 536.77it/s, v_num=2]
Epoch 2: 100%|██████████| 70/70 [00:00<00:00, 538.01it/s, v_num=2]
Epoch 2: 100%|██████████| 70/70 [00:00<00:00, 537.13it/s, v_num=2]
Epoch 2: 100%|██████████| 70/70 [00:00<00:00, 535.41it/s, v_num=2]
Epoch 2: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 3: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 3: 1%|▏ | 1/70 [00:00<00:00, 605.06it/s, v_num=2]
Epoch 3: 1%|▏ | 1/70 [00:00<00:00, 534.99it/s, v_num=2]
Epoch 3: 3%|▎ | 2/70 [00:00<00:00, 572.95it/s, v_num=2]
Epoch 3: 3%|▎ | 2/70 [00:00<00:00, 539.95it/s, v_num=2]
Epoch 3: 4%|▍ | 3/70 [00:00<00:00, 569.98it/s, v_num=2]
Epoch 3: 4%|▍ | 3/70 [00:00<00:00, 547.77it/s, v_num=2]
Epoch 3: 6%|▌ | 4/70 [00:00<00:00, 554.47it/s, v_num=2]
Epoch 3: 6%|▌ | 4/70 [00:00<00:00, 533.54it/s, v_num=2]
Epoch 3: 7%|▋ | 5/70 [00:00<00:00, 553.09it/s, v_num=2]
Epoch 3: 7%|▋ | 5/70 [00:00<00:00, 540.22it/s, v_num=2]
Epoch 3: 9%|▊ | 6/70 [00:00<00:00, 552.54it/s, v_num=2]
Epoch 3: 9%|▊ | 6/70 [00:00<00:00, 541.74it/s, v_num=2]
Epoch 3: 10%|█ | 7/70 [00:00<00:00, 554.45it/s, v_num=2]
Epoch 3: 10%|█ | 7/70 [00:00<00:00, 545.26it/s, v_num=2]
Epoch 3: 11%|█▏ | 8/70 [00:00<00:00, 555.67it/s, v_num=2]
Epoch 3: 11%|█▏ | 8/70 [00:00<00:00, 547.53it/s, v_num=2]
Epoch 3: 13%|█▎ | 9/70 [00:00<00:00, 555.31it/s, v_num=2]
Epoch 3: 13%|█▎ | 9/70 [00:00<00:00, 548.08it/s, v_num=2]
Epoch 3: 14%|█▍ | 10/70 [00:00<00:00, 555.93it/s, v_num=2]
Epoch 3: 14%|█▍ | 10/70 [00:00<00:00, 549.42it/s, v_num=2]
Epoch 3: 16%|█▌ | 11/70 [00:00<00:00, 555.66it/s, v_num=2]
Epoch 3: 16%|█▌ | 11/70 [00:00<00:00, 549.68it/s, v_num=2]
Epoch 3: 17%|█▋ | 12/70 [00:00<00:00, 556.45it/s, v_num=2]
Epoch 3: 17%|█▋ | 12/70 [00:00<00:00, 550.99it/s, v_num=2]
Epoch 3: 19%|█▊ | 13/70 [00:00<00:00, 554.42it/s, v_num=2]
Epoch 3: 19%|█▊ | 13/70 [00:00<00:00, 549.41it/s, v_num=2]
Epoch 3: 20%|██ | 14/70 [00:00<00:00, 555.28it/s, v_num=2]
Epoch 3: 20%|██ | 14/70 [00:00<00:00, 550.60it/s, v_num=2]
Epoch 3: 21%|██▏ | 15/70 [00:00<00:00, 554.72it/s, v_num=2]
Epoch 3: 21%|██▏ | 15/70 [00:00<00:00, 550.27it/s, v_num=2]
Epoch 3: 23%|██▎ | 16/70 [00:00<00:00, 555.39it/s, v_num=2]
Epoch 3: 23%|██▎ | 16/70 [00:00<00:00, 551.28it/s, v_num=2]
Epoch 3: 24%|██▍ | 17/70 [00:00<00:00, 555.88it/s, v_num=2]
Epoch 3: 24%|██▍ | 17/70 [00:00<00:00, 551.28it/s, v_num=2]
Epoch 3: 26%|██▌ | 18/70 [00:00<00:00, 555.52it/s, v_num=2]
Epoch 3: 26%|██▌ | 18/70 [00:00<00:00, 551.86it/s, v_num=2]
Epoch 3: 27%|██▋ | 19/70 [00:00<00:00, 555.91it/s, v_num=2]
Epoch 3: 27%|██▋ | 19/70 [00:00<00:00, 552.46it/s, v_num=2]
Epoch 3: 29%|██▊ | 20/70 [00:00<00:00, 555.62it/s, v_num=2]
Epoch 3: 29%|██▊ | 20/70 [00:00<00:00, 552.33it/s, v_num=2]
Epoch 3: 30%|███ | 21/70 [00:00<00:00, 555.77it/s, v_num=2]
Epoch 3: 30%|███ | 21/70 [00:00<00:00, 552.62it/s, v_num=2]
Epoch 3: 31%|███▏ | 22/70 [00:00<00:00, 555.56it/s, v_num=2]
Epoch 3: 31%|███▏ | 22/70 [00:00<00:00, 552.59it/s, v_num=2]
Epoch 3: 33%|███▎ | 23/70 [00:00<00:00, 556.02it/s, v_num=2]
Epoch 3: 33%|███▎ | 23/70 [00:00<00:00, 553.16it/s, v_num=2]
Epoch 3: 34%|███▍ | 24/70 [00:00<00:00, 555.89it/s, v_num=2]
Epoch 3: 34%|███▍ | 24/70 [00:00<00:00, 553.15it/s, v_num=2]
Epoch 3: 36%|███▌ | 25/70 [00:00<00:00, 555.49it/s, v_num=2]
Epoch 3: 36%|███▌ | 25/70 [00:00<00:00, 552.86it/s, v_num=2]
Epoch 3: 37%|███▋ | 26/70 [00:00<00:00, 555.45it/s, v_num=2]
Epoch 3: 37%|███▋ | 26/70 [00:00<00:00, 552.89it/s, v_num=2]
Epoch 3: 39%|███▊ | 27/70 [00:00<00:00, 555.80it/s, v_num=2]
Epoch 3: 39%|███▊ | 27/70 [00:00<00:00, 553.35it/s, v_num=2]
Epoch 3: 40%|████ | 28/70 [00:00<00:00, 556.14it/s, v_num=2]
Epoch 3: 40%|████ | 28/70 [00:00<00:00, 553.78it/s, v_num=2]
Epoch 3: 41%|████▏ | 29/70 [00:00<00:00, 553.19it/s, v_num=2]
Epoch 3: 41%|████▏ | 29/70 [00:00<00:00, 550.92it/s, v_num=2]
Epoch 3: 43%|████▎ | 30/70 [00:00<00:00, 553.37it/s, v_num=2]
Epoch 3: 43%|████▎ | 30/70 [00:00<00:00, 551.19it/s, v_num=2]
Epoch 3: 44%|████▍ | 31/70 [00:00<00:00, 553.33it/s, v_num=2]
Epoch 3: 44%|████▍ | 31/70 [00:00<00:00, 551.24it/s, v_num=2]
Epoch 3: 46%|████▌ | 32/70 [00:00<00:00, 553.77it/s, v_num=2]
Epoch 3: 46%|████▌ | 32/70 [00:00<00:00, 551.75it/s, v_num=2]
Epoch 3: 47%|████▋ | 33/70 [00:00<00:00, 553.79it/s, v_num=2]
Epoch 3: 47%|████▋ | 33/70 [00:00<00:00, 551.81it/s, v_num=2]
Epoch 3: 49%|████▊ | 34/70 [00:00<00:00, 554.12it/s, v_num=2]
Epoch 3: 49%|████▊ | 34/70 [00:00<00:00, 552.20it/s, v_num=2]
Epoch 3: 50%|█████ | 35/70 [00:00<00:00, 553.87it/s, v_num=2]
Epoch 3: 50%|█████ | 35/70 [00:00<00:00, 551.99it/s, v_num=2]
Epoch 3: 51%|█████▏ | 36/70 [00:00<00:00, 554.17it/s, v_num=2]
Epoch 3: 51%|█████▏ | 36/70 [00:00<00:00, 552.36it/s, v_num=2]
Epoch 3: 53%|█████▎ | 37/70 [00:00<00:00, 554.26it/s, v_num=2]
Epoch 3: 53%|█████▎ | 37/70 [00:00<00:00, 552.47it/s, v_num=2]
Epoch 3: 54%|█████▍ | 38/70 [00:00<00:00, 552.75it/s, v_num=2]
Epoch 3: 54%|█████▍ | 38/70 [00:00<00:00, 551.02it/s, v_num=2]
Epoch 3: 56%|█████▌ | 39/70 [00:00<00:00, 553.07it/s, v_num=2]
Epoch 3: 56%|█████▌ | 39/70 [00:00<00:00, 551.16it/s, v_num=2]
Epoch 3: 57%|█████▋ | 40/70 [00:00<00:00, 553.06it/s, v_num=2]
Epoch 3: 57%|█████▋ | 40/70 [00:00<00:00, 551.42it/s, v_num=2]
Epoch 3: 59%|█████▊ | 41/70 [00:00<00:00, 546.02it/s, v_num=2]
Epoch 3: 59%|█████▊ | 41/70 [00:00<00:00, 544.45it/s, v_num=2]
Epoch 3: 60%|██████ | 42/70 [00:00<00:00, 546.41it/s, v_num=2]
Epoch 3: 60%|██████ | 42/70 [00:00<00:00, 544.89it/s, v_num=2]
Epoch 3: 61%|██████▏ | 43/70 [00:00<00:00, 545.29it/s, v_num=2]
Epoch 3: 61%|██████▏ | 43/70 [00:00<00:00, 543.80it/s, v_num=2]
Epoch 3: 63%|██████▎ | 44/70 [00:00<00:00, 545.71it/s, v_num=2]
Epoch 3: 63%|██████▎ | 44/70 [00:00<00:00, 544.26it/s, v_num=2]
Epoch 3: 64%|██████▍ | 45/70 [00:00<00:00, 546.02it/s, v_num=2]
Epoch 3: 64%|██████▍ | 45/70 [00:00<00:00, 544.61it/s, v_num=2]
Epoch 3: 66%|██████▌ | 46/70 [00:00<00:00, 546.04it/s, v_num=2]
Epoch 3: 66%|██████▌ | 46/70 [00:00<00:00, 544.66it/s, v_num=2]
Epoch 3: 67%|██████▋ | 47/70 [00:00<00:00, 546.46it/s, v_num=2]
Epoch 3: 67%|██████▋ | 47/70 [00:00<00:00, 545.10it/s, v_num=2]
Epoch 3: 69%|██████▊ | 48/70 [00:00<00:00, 546.53it/s, v_num=2]
Epoch 3: 69%|██████▊ | 48/70 [00:00<00:00, 545.21it/s, v_num=2]
Epoch 3: 70%|███████ | 49/70 [00:00<00:00, 546.90it/s, v_num=2]
Epoch 3: 70%|███████ | 49/70 [00:00<00:00, 545.60it/s, v_num=2]
Epoch 3: 71%|███████▏ | 50/70 [00:00<00:00, 546.88it/s, v_num=2]
Epoch 3: 71%|███████▏ | 50/70 [00:00<00:00, 545.61it/s, v_num=2]
Epoch 3: 73%|███████▎ | 51/70 [00:00<00:00, 547.18it/s, v_num=2]
Epoch 3: 73%|███████▎ | 51/70 [00:00<00:00, 545.93it/s, v_num=2]
Epoch 3: 74%|███████▍ | 52/70 [00:00<00:00, 545.97it/s, v_num=2]
Epoch 3: 74%|███████▍ | 52/70 [00:00<00:00, 544.74it/s, v_num=2]
Epoch 3: 76%|███████▌ | 53/70 [00:00<00:00, 545.42it/s, v_num=2]
Epoch 3: 76%|███████▌ | 53/70 [00:00<00:00, 543.82it/s, v_num=2]
Epoch 3: 77%|███████▋ | 54/70 [00:00<00:00, 545.20it/s, v_num=2]
Epoch 3: 77%|███████▋ | 54/70 [00:00<00:00, 544.02it/s, v_num=2]
Epoch 3: 79%|███████▊ | 55/70 [00:00<00:00, 544.46it/s, v_num=2]
Epoch 3: 79%|███████▊ | 55/70 [00:00<00:00, 543.30it/s, v_num=2]
Epoch 3: 80%|████████ | 56/70 [00:00<00:00, 544.50it/s, v_num=2]
Epoch 3: 80%|████████ | 56/70 [00:00<00:00, 543.36it/s, v_num=2]
Epoch 3: 81%|████████▏ | 57/70 [00:00<00:00, 544.78it/s, v_num=2]
Epoch 3: 81%|████████▏ | 57/70 [00:00<00:00, 543.68it/s, v_num=2]
Epoch 3: 83%|████████▎ | 58/70 [00:00<00:00, 544.04it/s, v_num=2]
Epoch 3: 83%|████████▎ | 58/70 [00:00<00:00, 542.75it/s, v_num=2]
Epoch 3: 84%|████████▍ | 59/70 [00:00<00:00, 544.23it/s, v_num=2]
Epoch 3: 84%|████████▍ | 59/70 [00:00<00:00, 543.15it/s, v_num=2]
Epoch 3: 86%|████████▌ | 60/70 [00:00<00:00, 544.49it/s, v_num=2]
Epoch 3: 86%|████████▌ | 60/70 [00:00<00:00, 543.44it/s, v_num=2]
Epoch 3: 87%|████████▋ | 61/70 [00:00<00:00, 544.56it/s, v_num=2]
Epoch 3: 87%|████████▋ | 61/70 [00:00<00:00, 543.52it/s, v_num=2]
Epoch 3: 89%|████████▊ | 62/70 [00:00<00:00, 543.86it/s, v_num=2]
Epoch 3: 89%|████████▊ | 62/70 [00:00<00:00, 542.83it/s, v_num=2]
Epoch 3: 90%|█████████ | 63/70 [00:00<00:00, 544.01it/s, v_num=2]
Epoch 3: 90%|█████████ | 63/70 [00:00<00:00, 543.01it/s, v_num=2]
Epoch 3: 91%|█████████▏| 64/70 [00:00<00:00, 544.34it/s, v_num=2]
Epoch 3: 91%|█████████▏| 64/70 [00:00<00:00, 543.35it/s, v_num=2]
Epoch 3: 93%|█████████▎| 65/70 [00:00<00:00, 544.42it/s, v_num=2]
Epoch 3: 93%|█████████▎| 65/70 [00:00<00:00, 543.45it/s, v_num=2]
Epoch 3: 94%|█████████▍| 66/70 [00:00<00:00, 544.08it/s, v_num=2]
Epoch 3: 94%|█████████▍| 66/70 [00:00<00:00, 542.83it/s, v_num=2]
Epoch 3: 96%|█████████▌| 67/70 [00:00<00:00, 543.98it/s, v_num=2]
Epoch 3: 96%|█████████▌| 67/70 [00:00<00:00, 543.03it/s, v_num=2]
Epoch 3: 97%|█████████▋| 68/70 [00:00<00:00, 544.26it/s, v_num=2]
Epoch 3: 97%|█████████▋| 68/70 [00:00<00:00, 543.33it/s, v_num=2]
Epoch 3: 99%|█████████▊| 69/70 [00:00<00:00, 544.39it/s, v_num=2]
Epoch 3: 99%|█████████▊| 69/70 [00:00<00:00, 543.46it/s, v_num=2]
Epoch 3: 100%|██████████| 70/70 [00:00<00:00, 544.61it/s, v_num=2]
Epoch 3: 100%|██████████| 70/70 [00:00<00:00, 543.71it/s, v_num=2]
Epoch 3: 100%|██████████| 70/70 [00:00<00:00, 541.81it/s, v_num=2]
Epoch 3: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 4: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 4: 1%|▏ | 1/70 [00:00<00:00, 606.03it/s, v_num=2]
Epoch 4: 1%|▏ | 1/70 [00:00<00:00, 521.68it/s, v_num=2]
Epoch 4: 3%|▎ | 2/70 [00:00<00:00, 572.33it/s, v_num=2]
Epoch 4: 3%|▎ | 2/70 [00:00<00:00, 533.49it/s, v_num=2]
Epoch 4: 4%|▍ | 3/70 [00:00<00:00, 565.27it/s, v_num=2]
Epoch 4: 4%|▍ | 3/70 [00:00<00:00, 543.47it/s, v_num=2]
Epoch 4: 6%|▌ | 4/70 [00:00<00:00, 551.36it/s, v_num=2]
Epoch 4: 6%|▌ | 4/70 [00:00<00:00, 534.61it/s, v_num=2]
Epoch 4: 7%|▋ | 5/70 [00:00<00:00, 550.74it/s, v_num=2]
Epoch 4: 7%|▋ | 5/70 [00:00<00:00, 538.12it/s, v_num=2]
Epoch 4: 9%|▊ | 6/70 [00:00<00:00, 553.13it/s, v_num=2]
Epoch 4: 9%|▊ | 6/70 [00:00<00:00, 542.43it/s, v_num=2]
Epoch 4: 10%|█ | 7/70 [00:00<00:00, 552.15it/s, v_num=2]
Epoch 4: 10%|█ | 7/70 [00:00<00:00, 543.01it/s, v_num=2]
Epoch 4: 11%|█▏ | 8/70 [00:00<00:00, 546.88it/s, v_num=2]
Epoch 4: 11%|█▏ | 8/70 [00:00<00:00, 539.83it/s, v_num=2]
Epoch 4: 13%|█▎ | 9/70 [00:00<00:00, 548.62it/s, v_num=2]
Epoch 4: 13%|█▎ | 9/70 [00:00<00:00, 541.53it/s, v_num=2]
Epoch 4: 14%|█▍ | 10/70 [00:00<00:00, 550.28it/s, v_num=2]
Epoch 4: 14%|█▍ | 10/70 [00:00<00:00, 543.94it/s, v_num=2]
Epoch 4: 16%|█▌ | 11/70 [00:00<00:00, 550.57it/s, v_num=2]
Epoch 4: 16%|█▌ | 11/70 [00:00<00:00, 544.72it/s, v_num=2]
Epoch 4: 17%|█▋ | 12/70 [00:00<00:00, 551.80it/s, v_num=2]
Epoch 4: 17%|█▋ | 12/70 [00:00<00:00, 546.45it/s, v_num=2]
Epoch 4: 19%|█▊ | 13/70 [00:00<00:00, 547.69it/s, v_num=2]
Epoch 4: 19%|█▊ | 13/70 [00:00<00:00, 542.80it/s, v_num=2]
Epoch 4: 20%|██ | 14/70 [00:00<00:00, 548.98it/s, v_num=2]
Epoch 4: 20%|██ | 14/70 [00:00<00:00, 544.44it/s, v_num=2]
Epoch 4: 21%|██▏ | 15/70 [00:00<00:00, 550.07it/s, v_num=2]
Epoch 4: 21%|██▏ | 15/70 [00:00<00:00, 545.53it/s, v_num=2]
Epoch 4: 23%|██▎ | 16/70 [00:00<00:00, 550.23it/s, v_num=2]
Epoch 4: 23%|██▎ | 16/70 [00:00<00:00, 546.19it/s, v_num=2]
Epoch 4: 24%|██▍ | 17/70 [00:00<00:00, 547.18it/s, v_num=2]
Epoch 4: 24%|██▍ | 17/70 [00:00<00:00, 543.38it/s, v_num=2]
Epoch 4: 26%|██▌ | 18/70 [00:00<00:00, 547.35it/s, v_num=2]
Epoch 4: 26%|██▌ | 18/70 [00:00<00:00, 543.83it/s, v_num=2]
Epoch 4: 27%|██▋ | 19/70 [00:00<00:00, 548.13it/s, v_num=2]
Epoch 4: 27%|██▋ | 19/70 [00:00<00:00, 544.80it/s, v_num=2]
Epoch 4: 29%|██▊ | 20/70 [00:00<00:00, 548.39it/s, v_num=2]
Epoch 4: 29%|██▊ | 20/70 [00:00<00:00, 545.21it/s, v_num=2]
Epoch 4: 30%|███ | 21/70 [00:00<00:00, 523.61it/s, v_num=2]
Epoch 4: 30%|███ | 21/70 [00:00<00:00, 520.79it/s, v_num=2]
Epoch 4: 31%|███▏ | 22/70 [00:00<00:00, 525.22it/s, v_num=2]
Epoch 4: 31%|███▏ | 22/70 [00:00<00:00, 522.55it/s, v_num=2]
Epoch 4: 33%|███▎ | 23/70 [00:00<00:00, 526.30it/s, v_num=2]
Epoch 4: 33%|███▎ | 23/70 [00:00<00:00, 523.14it/s, v_num=2]
Epoch 4: 34%|███▍ | 24/70 [00:00<00:00, 527.26it/s, v_num=2]
Epoch 4: 34%|███▍ | 24/70 [00:00<00:00, 524.79it/s, v_num=2]
Epoch 4: 36%|███▌ | 25/70 [00:00<00:00, 526.44it/s, v_num=2]
Epoch 4: 36%|███▌ | 25/70 [00:00<00:00, 523.72it/s, v_num=2]
Epoch 4: 37%|███▋ | 26/70 [00:00<00:00, 527.47it/s, v_num=2]
Epoch 4: 37%|███▋ | 26/70 [00:00<00:00, 525.19it/s, v_num=2]
Epoch 4: 39%|███▊ | 27/70 [00:00<00:00, 528.74it/s, v_num=2]
Epoch 4: 39%|███▊ | 27/70 [00:00<00:00, 526.54it/s, v_num=2]
Epoch 4: 40%|████ | 28/70 [00:00<00:00, 529.36it/s, v_num=2]
Epoch 4: 40%|████ | 28/70 [00:00<00:00, 527.21it/s, v_num=2]
Epoch 4: 41%|████▏ | 29/70 [00:00<00:00, 529.07it/s, v_num=2]
Epoch 4: 41%|████▏ | 29/70 [00:00<00:00, 526.49it/s, v_num=2]
Epoch 4: 43%|████▎ | 30/70 [00:00<00:00, 528.82it/s, v_num=2]
Epoch 4: 43%|████▎ | 30/70 [00:00<00:00, 526.81it/s, v_num=2]
Epoch 4: 44%|████▍ | 31/70 [00:00<00:00, 529.85it/s, v_num=2]
Epoch 4: 44%|████▍ | 31/70 [00:00<00:00, 527.92it/s, v_num=2]
Epoch 4: 46%|████▌ | 32/70 [00:00<00:00, 530.09it/s, v_num=2]
Epoch 4: 46%|████▌ | 32/70 [00:00<00:00, 528.24it/s, v_num=2]
Epoch 4: 47%|████▋ | 33/70 [00:00<00:00, 530.51it/s, v_num=2]
Epoch 4: 47%|████▋ | 33/70 [00:00<00:00, 527.65it/s, v_num=2]
Epoch 4: 49%|████▊ | 34/70 [00:00<00:00, 529.65it/s, v_num=2]
Epoch 4: 49%|████▊ | 34/70 [00:00<00:00, 527.89it/s, v_num=2]
Epoch 4: 50%|█████ | 35/70 [00:00<00:00, 530.59it/s, v_num=2]
Epoch 4: 50%|█████ | 35/70 [00:00<00:00, 528.88it/s, v_num=2]
Epoch 4: 51%|█████▏ | 36/70 [00:00<00:00, 531.22it/s, v_num=2]
Epoch 4: 51%|█████▏ | 36/70 [00:00<00:00, 529.57it/s, v_num=2]
Epoch 4: 53%|█████▎ | 37/70 [00:00<00:00, 532.12it/s, v_num=2]
Epoch 4: 53%|█████▎ | 37/70 [00:00<00:00, 530.49it/s, v_num=2]
Epoch 4: 54%|█████▍ | 38/70 [00:00<00:00, 531.60it/s, v_num=2]
Epoch 4: 54%|█████▍ | 38/70 [00:00<00:00, 529.97it/s, v_num=2]
Epoch 4: 56%|█████▌ | 39/70 [00:00<00:00, 532.38it/s, v_num=2]
Epoch 4: 56%|█████▌ | 39/70 [00:00<00:00, 530.83it/s, v_num=2]
Epoch 4: 57%|█████▋ | 40/70 [00:00<00:00, 533.12it/s, v_num=2]
Epoch 4: 57%|█████▋ | 40/70 [00:00<00:00, 531.60it/s, v_num=2]
Epoch 4: 59%|█████▊ | 41/70 [00:00<00:00, 533.59it/s, v_num=2]
Epoch 4: 59%|█████▊ | 41/70 [00:00<00:00, 532.12it/s, v_num=2]
Epoch 4: 60%|██████ | 42/70 [00:00<00:00, 532.97it/s, v_num=2]
Epoch 4: 60%|██████ | 42/70 [00:00<00:00, 531.52it/s, v_num=2]
Epoch 4: 61%|██████▏ | 43/70 [00:00<00:00, 533.44it/s, v_num=2]
Epoch 4: 61%|██████▏ | 43/70 [00:00<00:00, 532.03it/s, v_num=2]
Epoch 4: 63%|██████▎ | 44/70 [00:00<00:00, 534.10it/s, v_num=2]
Epoch 4: 63%|██████▎ | 44/70 [00:00<00:00, 532.73it/s, v_num=2]
Epoch 4: 64%|██████▍ | 45/70 [00:00<00:00, 534.47it/s, v_num=2]
Epoch 4: 64%|██████▍ | 45/70 [00:00<00:00, 533.12it/s, v_num=2]
Epoch 4: 66%|██████▌ | 46/70 [00:00<00:00, 535.15it/s, v_num=2]
Epoch 4: 66%|██████▌ | 46/70 [00:00<00:00, 533.83it/s, v_num=2]
Epoch 4: 67%|██████▋ | 47/70 [00:00<00:00, 535.55it/s, v_num=2]
Epoch 4: 67%|██████▋ | 47/70 [00:00<00:00, 534.25it/s, v_num=2]
Epoch 4: 69%|██████▊ | 48/70 [00:00<00:00, 536.08it/s, v_num=2]
Epoch 4: 69%|██████▊ | 48/70 [00:00<00:00, 534.81it/s, v_num=2]
Epoch 4: 70%|███████ | 49/70 [00:00<00:00, 536.66it/s, v_num=2]
Epoch 4: 70%|███████ | 49/70 [00:00<00:00, 535.19it/s, v_num=2]
Epoch 4: 71%|███████▏ | 50/70 [00:00<00:00, 536.96it/s, v_num=2]
Epoch 4: 71%|███████▏ | 50/70 [00:00<00:00, 535.74it/s, v_num=2]
Epoch 4: 73%|███████▎ | 51/70 [00:00<00:00, 537.56it/s, v_num=2]
Epoch 4: 73%|███████▎ | 51/70 [00:00<00:00, 536.36it/s, v_num=2]
Epoch 4: 74%|███████▍ | 52/70 [00:00<00:00, 537.73it/s, v_num=2]
Epoch 4: 74%|███████▍ | 52/70 [00:00<00:00, 536.51it/s, v_num=2]
Epoch 4: 76%|███████▌ | 53/70 [00:00<00:00, 538.18it/s, v_num=2]
Epoch 4: 76%|███████▌ | 53/70 [00:00<00:00, 537.01it/s, v_num=2]
Epoch 4: 77%|███████▋ | 54/70 [00:00<00:00, 538.47it/s, v_num=2]
Epoch 4: 77%|███████▋ | 54/70 [00:00<00:00, 537.33it/s, v_num=2]
Epoch 4: 79%|███████▊ | 55/70 [00:00<00:00, 538.98it/s, v_num=2]
Epoch 4: 79%|███████▊ | 55/70 [00:00<00:00, 537.86it/s, v_num=2]
Epoch 4: 80%|████████ | 56/70 [00:00<00:00, 539.24it/s, v_num=2]
Epoch 4: 80%|████████ | 56/70 [00:00<00:00, 538.14it/s, v_num=2]
Epoch 4: 81%|████████▏ | 57/70 [00:00<00:00, 539.69it/s, v_num=2]
Epoch 4: 81%|████████▏ | 57/70 [00:00<00:00, 538.60it/s, v_num=2]
Epoch 4: 83%|████████▎ | 58/70 [00:00<00:00, 540.10it/s, v_num=2]
Epoch 4: 83%|████████▎ | 58/70 [00:00<00:00, 538.85it/s, v_num=2]
Epoch 4: 84%|████████▍ | 59/70 [00:00<00:00, 540.36it/s, v_num=2]
Epoch 4: 84%|████████▍ | 59/70 [00:00<00:00, 539.31it/s, v_num=2]
Epoch 4: 86%|████████▌ | 60/70 [00:00<00:00, 540.75it/s, v_num=2]
Epoch 4: 86%|████████▌ | 60/70 [00:00<00:00, 539.71it/s, v_num=2]
Epoch 4: 87%|████████▋ | 61/70 [00:00<00:00, 540.76it/s, v_num=2]
Epoch 4: 87%|████████▋ | 61/70 [00:00<00:00, 539.73it/s, v_num=2]
Epoch 4: 89%|████████▊ | 62/70 [00:00<00:00, 541.18it/s, v_num=2]
Epoch 4: 89%|████████▊ | 62/70 [00:00<00:00, 540.17it/s, v_num=2]
Epoch 4: 90%|█████████ | 63/70 [00:00<00:00, 541.30it/s, v_num=2]
Epoch 4: 90%|█████████ | 63/70 [00:00<00:00, 540.31it/s, v_num=2]
Epoch 4: 91%|█████████▏| 64/70 [00:00<00:00, 541.69it/s, v_num=2]
Epoch 4: 91%|█████████▏| 64/70 [00:00<00:00, 540.72it/s, v_num=2]
Epoch 4: 93%|█████████▎| 65/70 [00:00<00:00, 541.72it/s, v_num=2]
Epoch 4: 93%|█████████▎| 65/70 [00:00<00:00, 540.76it/s, v_num=2]
Epoch 4: 94%|█████████▍| 66/70 [00:00<00:00, 542.11it/s, v_num=2]
Epoch 4: 94%|█████████▍| 66/70 [00:00<00:00, 541.15it/s, v_num=2]
Epoch 4: 96%|█████████▌| 67/70 [00:00<00:00, 542.29it/s, v_num=2]
Epoch 4: 96%|█████████▌| 67/70 [00:00<00:00, 541.36it/s, v_num=2]
Epoch 4: 97%|█████████▋| 68/70 [00:00<00:00, 542.60it/s, v_num=2]
Epoch 4: 97%|█████████▋| 68/70 [00:00<00:00, 541.68it/s, v_num=2]
Epoch 4: 99%|█████████▊| 69/70 [00:00<00:00, 542.96it/s, v_num=2]
Epoch 4: 99%|█████████▊| 69/70 [00:00<00:00, 542.05it/s, v_num=2]
Epoch 4: 100%|██████████| 70/70 [00:00<00:00, 543.13it/s, v_num=2]
Epoch 4: 100%|██████████| 70/70 [00:00<00:00, 542.24it/s, v_num=2]
Epoch 4: 100%|██████████| 70/70 [00:00<00:00, 537.15it/s, v_num=2]
Epoch 4: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 5: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 5: 1%|▏ | 1/70 [00:00<00:00, 606.11it/s, v_num=2]
Epoch 5: 1%|▏ | 1/70 [00:00<00:00, 536.01it/s, v_num=2]
Epoch 5: 3%|▎ | 2/70 [00:00<00:00, 576.62it/s, v_num=2]
Epoch 5: 3%|▎ | 2/70 [00:00<00:00, 542.95it/s, v_num=2]
Epoch 5: 4%|▍ | 3/70 [00:00<00:00, 572.37it/s, v_num=2]
Epoch 5: 4%|▍ | 3/70 [00:00<00:00, 549.81it/s, v_num=2]
Epoch 5: 6%|▌ | 4/70 [00:00<00:00, 570.32it/s, v_num=2]
Epoch 5: 6%|▌ | 4/70 [00:00<00:00, 553.41it/s, v_num=2]
Epoch 5: 7%|▋ | 5/70 [00:00<00:00, 566.38it/s, v_num=2]
Epoch 5: 7%|▋ | 5/70 [00:00<00:00, 553.05it/s, v_num=2]
Epoch 5: 9%|▊ | 6/70 [00:00<00:00, 566.25it/s, v_num=2]
Epoch 5: 9%|▊ | 6/70 [00:00<00:00, 555.12it/s, v_num=2]
Epoch 5: 10%|█ | 7/70 [00:00<00:00, 563.63it/s, v_num=2]
Epoch 5: 10%|█ | 7/70 [00:00<00:00, 554.13it/s, v_num=2]
Epoch 5: 11%|█▏ | 8/70 [00:00<00:00, 564.06it/s, v_num=2]
Epoch 5: 11%|█▏ | 8/70 [00:00<00:00, 555.74it/s, v_num=2]
Epoch 5: 13%|█▎ | 9/70 [00:00<00:00, 562.29it/s, v_num=2]
Epoch 5: 13%|█▎ | 9/70 [00:00<00:00, 554.88it/s, v_num=2]
Epoch 5: 14%|█▍ | 10/70 [00:00<00:00, 562.52it/s, v_num=2]
Epoch 5: 14%|█▍ | 10/70 [00:00<00:00, 555.85it/s, v_num=2]
Epoch 5: 16%|█▌ | 11/70 [00:00<00:00, 561.48it/s, v_num=2]
Epoch 5: 16%|█▌ | 11/70 [00:00<00:00, 555.43it/s, v_num=2]
Epoch 5: 17%|█▋ | 12/70 [00:00<00:00, 562.14it/s, v_num=2]
Epoch 5: 17%|█▋ | 12/70 [00:00<00:00, 556.61it/s, v_num=2]
Epoch 5: 19%|█▊ | 13/70 [00:00<00:00, 562.45it/s, v_num=2]
Epoch 5: 19%|█▊ | 13/70 [00:00<00:00, 557.33it/s, v_num=2]
Epoch 5: 20%|██ | 14/70 [00:00<00:00, 561.76it/s, v_num=2]
Epoch 5: 20%|██ | 14/70 [00:00<00:00, 557.02it/s, v_num=2]
Epoch 5: 21%|██▏ | 15/70 [00:00<00:00, 562.18it/s, v_num=2]
Epoch 5: 21%|██▏ | 15/70 [00:00<00:00, 557.75it/s, v_num=2]
Epoch 5: 23%|██▎ | 16/70 [00:00<00:00, 561.72it/s, v_num=2]
Epoch 5: 23%|██▎ | 16/70 [00:00<00:00, 557.56it/s, v_num=2]
Epoch 5: 24%|██▍ | 17/70 [00:00<00:00, 561.82it/s, v_num=2]
Epoch 5: 24%|██▍ | 17/70 [00:00<00:00, 557.89it/s, v_num=2]
Epoch 5: 26%|██▌ | 18/70 [00:00<00:00, 561.15it/s, v_num=2]
Epoch 5: 26%|██▌ | 18/70 [00:00<00:00, 557.47it/s, v_num=2]
Epoch 5: 27%|██▋ | 19/70 [00:00<00:00, 561.49it/s, v_num=2]
Epoch 5: 27%|██▋ | 19/70 [00:00<00:00, 557.96it/s, v_num=2]
Epoch 5: 29%|██▊ | 20/70 [00:00<00:00, 560.25it/s, v_num=2]
Epoch 5: 29%|██▊ | 20/70 [00:00<00:00, 556.92it/s, v_num=2]
Epoch 5: 30%|███ | 21/70 [00:00<00:00, 560.48it/s, v_num=2]
Epoch 5: 30%|███ | 21/70 [00:00<00:00, 557.34it/s, v_num=2]
Epoch 5: 31%|███▏ | 22/70 [00:00<00:00, 560.78it/s, v_num=2]
Epoch 5: 31%|███▏ | 22/70 [00:00<00:00, 557.26it/s, v_num=2]
Epoch 5: 33%|███▎ | 23/70 [00:00<00:00, 560.48it/s, v_num=2]
Epoch 5: 33%|███▎ | 23/70 [00:00<00:00, 557.58it/s, v_num=2]
Epoch 5: 34%|███▍ | 24/70 [00:00<00:00, 560.71it/s, v_num=2]
Epoch 5: 34%|███▍ | 24/70 [00:00<00:00, 557.94it/s, v_num=2]
Epoch 5: 36%|███▌ | 25/70 [00:00<00:00, 560.50it/s, v_num=2]
Epoch 5: 36%|███▌ | 25/70 [00:00<00:00, 557.86it/s, v_num=2]
Epoch 5: 37%|███▋ | 26/70 [00:00<00:00, 560.63it/s, v_num=2]
Epoch 5: 37%|███▋ | 26/70 [00:00<00:00, 558.08it/s, v_num=2]
Epoch 5: 39%|███▊ | 27/70 [00:00<00:00, 560.19it/s, v_num=2]
Epoch 5: 39%|███▊ | 27/70 [00:00<00:00, 557.74it/s, v_num=2]
Epoch 5: 40%|████ | 28/70 [00:00<00:00, 560.45it/s, v_num=2]
Epoch 5: 40%|████ | 28/70 [00:00<00:00, 558.07it/s, v_num=2]
Epoch 5: 41%|████▏ | 29/70 [00:00<00:00, 560.27it/s, v_num=2]
Epoch 5: 41%|████▏ | 29/70 [00:00<00:00, 557.97it/s, v_num=2]
Epoch 5: 43%|████▎ | 30/70 [00:00<00:00, 560.49it/s, v_num=2]
Epoch 5: 43%|████▎ | 30/70 [00:00<00:00, 558.28it/s, v_num=2]
Epoch 5: 44%|████▍ | 31/70 [00:00<00:00, 560.70it/s, v_num=2]
Epoch 5: 44%|████▍ | 31/70 [00:00<00:00, 558.16it/s, v_num=2]
Epoch 5: 46%|████▌ | 32/70 [00:00<00:00, 560.51it/s, v_num=2]
Epoch 5: 46%|████▌ | 32/70 [00:00<00:00, 558.44it/s, v_num=2]
Epoch 5: 47%|████▋ | 33/70 [00:00<00:00, 560.72it/s, v_num=2]
Epoch 5: 47%|████▋ | 33/70 [00:00<00:00, 558.71it/s, v_num=2]
Epoch 5: 49%|████▊ | 34/70 [00:00<00:00, 560.53it/s, v_num=2]
Epoch 5: 49%|████▊ | 34/70 [00:00<00:00, 558.57it/s, v_num=2]
Epoch 5: 50%|█████ | 35/70 [00:00<00:00, 560.74it/s, v_num=2]
Epoch 5: 50%|█████ | 35/70 [00:00<00:00, 558.84it/s, v_num=2]
Epoch 5: 51%|█████▏ | 36/70 [00:00<00:00, 560.45it/s, v_num=2]
Epoch 5: 51%|█████▏ | 36/70 [00:00<00:00, 558.58it/s, v_num=2]
Epoch 5: 53%|█████▎ | 37/70 [00:00<00:00, 560.62it/s, v_num=2]
Epoch 5: 53%|█████▎ | 37/70 [00:00<00:00, 558.81it/s, v_num=2]
Epoch 5: 54%|█████▍ | 38/70 [00:00<00:00, 560.33it/s, v_num=2]
Epoch 5: 54%|█████▍ | 38/70 [00:00<00:00, 558.58it/s, v_num=2]
Epoch 5: 56%|█████▌ | 39/70 [00:00<00:00, 560.58it/s, v_num=2]
Epoch 5: 56%|█████▌ | 39/70 [00:00<00:00, 558.88it/s, v_num=2]
Epoch 5: 57%|█████▋ | 40/70 [00:00<00:00, 560.83it/s, v_num=2]
Epoch 5: 57%|█████▋ | 40/70 [00:00<00:00, 558.89it/s, v_num=2]
Epoch 5: 59%|█████▊ | 41/70 [00:00<00:00, 560.72it/s, v_num=2]
Epoch 5: 59%|█████▊ | 41/70 [00:00<00:00, 559.09it/s, v_num=2]
Epoch 5: 60%|██████ | 42/70 [00:00<00:00, 560.88it/s, v_num=2]
Epoch 5: 60%|██████ | 42/70 [00:00<00:00, 559.30it/s, v_num=2]
Epoch 5: 61%|██████▏ | 43/70 [00:00<00:00, 560.78it/s, v_num=2]
Epoch 5: 61%|██████▏ | 43/70 [00:00<00:00, 559.24it/s, v_num=2]
Epoch 5: 63%|██████▎ | 44/70 [00:00<00:00, 560.96it/s, v_num=2]
Epoch 5: 63%|██████▎ | 44/70 [00:00<00:00, 559.46it/s, v_num=2]
Epoch 5: 64%|██████▍ | 45/70 [00:00<00:00, 560.05it/s, v_num=2]
Epoch 5: 64%|██████▍ | 45/70 [00:00<00:00, 558.57it/s, v_num=2]
Epoch 5: 66%|██████▌ | 46/70 [00:00<00:00, 560.14it/s, v_num=2]
Epoch 5: 66%|██████▌ | 46/70 [00:00<00:00, 558.70it/s, v_num=2]
Epoch 5: 67%|██████▋ | 47/70 [00:00<00:00, 560.03it/s, v_num=2]
Epoch 5: 67%|██████▋ | 47/70 [00:00<00:00, 558.61it/s, v_num=2]
Epoch 5: 69%|██████▊ | 48/70 [00:00<00:00, 560.22it/s, v_num=2]
Epoch 5: 69%|██████▊ | 48/70 [00:00<00:00, 558.83it/s, v_num=2]
Epoch 5: 70%|███████ | 49/70 [00:00<00:00, 560.15it/s, v_num=2]
Epoch 5: 70%|███████ | 49/70 [00:00<00:00, 558.79it/s, v_num=2]
Epoch 5: 71%|███████▏ | 50/70 [00:00<00:00, 560.15it/s, v_num=2]
Epoch 5: 71%|███████▏ | 50/70 [00:00<00:00, 558.51it/s, v_num=2]
Epoch 5: 73%|███████▎ | 51/70 [00:00<00:00, 549.87it/s, v_num=2]
Epoch 5: 73%|███████▎ | 51/70 [00:00<00:00, 548.59it/s, v_num=2]
Epoch 5: 74%|███████▍ | 52/70 [00:00<00:00, 550.14it/s, v_num=2]
Epoch 5: 74%|███████▍ | 52/70 [00:00<00:00, 548.90it/s, v_num=2]
Epoch 5: 76%|███████▌ | 53/70 [00:00<00:00, 550.22it/s, v_num=2]
Epoch 5: 76%|███████▌ | 53/70 [00:00<00:00, 549.01it/s, v_num=2]
Epoch 5: 77%|███████▋ | 54/70 [00:00<00:00, 550.54it/s, v_num=2]
Epoch 5: 77%|███████▋ | 54/70 [00:00<00:00, 549.35it/s, v_num=2]
Epoch 5: 79%|███████▊ | 55/70 [00:00<00:00, 550.52it/s, v_num=2]
Epoch 5: 79%|███████▊ | 55/70 [00:00<00:00, 549.35it/s, v_num=2]
Epoch 5: 80%|████████ | 56/70 [00:00<00:00, 550.74it/s, v_num=2]
Epoch 5: 80%|████████ | 56/70 [00:00<00:00, 549.60it/s, v_num=2]
Epoch 5: 81%|████████▏ | 57/70 [00:00<00:00, 550.80it/s, v_num=2]
Epoch 5: 81%|████████▏ | 57/70 [00:00<00:00, 549.67it/s, v_num=2]
Epoch 5: 83%|████████▎ | 58/70 [00:00<00:00, 551.03it/s, v_num=2]
Epoch 5: 83%|████████▎ | 58/70 [00:00<00:00, 549.91it/s, v_num=2]
Epoch 5: 84%|████████▍ | 59/70 [00:00<00:00, 551.19it/s, v_num=2]
Epoch 5: 84%|████████▍ | 59/70 [00:00<00:00, 550.10it/s, v_num=2]
Epoch 5: 86%|████████▌ | 60/70 [00:00<00:00, 551.24it/s, v_num=2]
Epoch 5: 86%|████████▌ | 60/70 [00:00<00:00, 550.17it/s, v_num=2]
Epoch 5: 87%|████████▋ | 61/70 [00:00<00:00, 551.48it/s, v_num=2]
Epoch 5: 87%|████████▋ | 61/70 [00:00<00:00, 550.42it/s, v_num=2]
Epoch 5: 89%|████████▊ | 62/70 [00:00<00:00, 551.37it/s, v_num=2]
Epoch 5: 89%|████████▊ | 62/70 [00:00<00:00, 550.32it/s, v_num=2]
Epoch 5: 90%|█████████ | 63/70 [00:00<00:00, 551.60it/s, v_num=2]
Epoch 5: 90%|█████████ | 63/70 [00:00<00:00, 550.58it/s, v_num=2]
Epoch 5: 91%|█████████▏| 64/70 [00:00<00:00, 551.62it/s, v_num=2]
Epoch 5: 91%|█████████▏| 64/70 [00:00<00:00, 550.61it/s, v_num=2]
Epoch 5: 93%|█████████▎| 65/70 [00:00<00:00, 551.81it/s, v_num=2]
Epoch 5: 93%|█████████▎| 65/70 [00:00<00:00, 550.82it/s, v_num=2]
Epoch 5: 94%|█████████▍| 66/70 [00:00<00:00, 551.85it/s, v_num=2]
Epoch 5: 94%|█████████▍| 66/70 [00:00<00:00, 550.87it/s, v_num=2]
Epoch 5: 96%|█████████▌| 67/70 [00:00<00:00, 552.09it/s, v_num=2]
Epoch 5: 96%|█████████▌| 67/70 [00:00<00:00, 551.12it/s, v_num=2]
Epoch 5: 97%|█████████▋| 68/70 [00:00<00:00, 552.29it/s, v_num=2]
Epoch 5: 97%|█████████▋| 68/70 [00:00<00:00, 551.29it/s, v_num=2]
Epoch 5: 99%|█████████▊| 69/70 [00:00<00:00, 552.31it/s, v_num=2]
Epoch 5: 99%|█████████▊| 69/70 [00:00<00:00, 551.38it/s, v_num=2]
Epoch 5: 100%|██████████| 70/70 [00:00<00:00, 552.49it/s, v_num=2]
Epoch 5: 100%|██████████| 70/70 [00:00<00:00, 551.57it/s, v_num=2]
Epoch 5: 100%|██████████| 70/70 [00:00<00:00, 549.70it/s, v_num=2]
Epoch 5: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 6: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 6: 1%|▏ | 1/70 [00:00<00:00, 605.59it/s, v_num=2]
Epoch 6: 1%|▏ | 1/70 [00:00<00:00, 536.01it/s, v_num=2]
Epoch 6: 3%|▎ | 2/70 [00:00<00:00, 574.05it/s, v_num=2]
Epoch 6: 3%|▎ | 2/70 [00:00<00:00, 541.03it/s, v_num=2]
Epoch 6: 4%|▍ | 3/70 [00:00<00:00, 571.30it/s, v_num=2]
Epoch 6: 4%|▍ | 3/70 [00:00<00:00, 549.18it/s, v_num=2]
Epoch 6: 6%|▌ | 4/70 [00:00<00:00, 566.45it/s, v_num=2]
Epoch 6: 6%|▌ | 4/70 [00:00<00:00, 549.98it/s, v_num=2]
Epoch 6: 7%|▋ | 5/70 [00:00<00:00, 566.17it/s, v_num=2]
Epoch 6: 7%|▋ | 5/70 [00:00<00:00, 552.81it/s, v_num=2]
Epoch 6: 9%|▊ | 6/70 [00:00<00:00, 563.35it/s, v_num=2]
Epoch 6: 9%|▊ | 6/70 [00:00<00:00, 552.40it/s, v_num=2]
Epoch 6: 10%|█ | 7/70 [00:00<00:00, 564.01it/s, v_num=2]
Epoch 6: 10%|█ | 7/70 [00:00<00:00, 554.50it/s, v_num=2]
Epoch 6: 11%|█▏ | 8/70 [00:00<00:00, 563.79it/s, v_num=2]
Epoch 6: 11%|█▏ | 8/70 [00:00<00:00, 554.44it/s, v_num=2]
Epoch 6: 13%|█▎ | 9/70 [00:00<00:00, 561.51it/s, v_num=2]
Epoch 6: 13%|█▎ | 9/70 [00:00<00:00, 554.21it/s, v_num=2]
Epoch 6: 14%|█▍ | 10/70 [00:00<00:00, 562.18it/s, v_num=2]
Epoch 6: 14%|█▍ | 10/70 [00:00<00:00, 555.57it/s, v_num=2]
Epoch 6: 16%|█▌ | 11/70 [00:00<00:00, 561.62it/s, v_num=2]
Epoch 6: 16%|█▌ | 11/70 [00:00<00:00, 555.62it/s, v_num=2]
Epoch 6: 17%|█▋ | 12/70 [00:00<00:00, 562.11it/s, v_num=2]
Epoch 6: 17%|█▋ | 12/70 [00:00<00:00, 556.60it/s, v_num=2]
Epoch 6: 19%|█▊ | 13/70 [00:00<00:00, 561.55it/s, v_num=2]
Epoch 6: 19%|█▊ | 13/70 [00:00<00:00, 556.44it/s, v_num=2]
Epoch 6: 20%|██ | 14/70 [00:00<00:00, 561.70it/s, v_num=2]
Epoch 6: 20%|██ | 14/70 [00:00<00:00, 556.97it/s, v_num=2]
Epoch 6: 21%|██▏ | 15/70 [00:00<00:00, 561.17it/s, v_num=2]
Epoch 6: 21%|██▏ | 15/70 [00:00<00:00, 556.76it/s, v_num=2]
Epoch 6: 23%|██▎ | 16/70 [00:00<00:00, 561.67it/s, v_num=2]
Epoch 6: 23%|██▎ | 16/70 [00:00<00:00, 557.51it/s, v_num=2]
Epoch 6: 24%|██▍ | 17/70 [00:00<00:00, 561.41it/s, v_num=2]
Epoch 6: 24%|██▍ | 17/70 [00:00<00:00, 557.45it/s, v_num=2]
Epoch 6: 26%|██▌ | 18/70 [00:00<00:00, 561.72it/s, v_num=2]
Epoch 6: 26%|██▌ | 18/70 [00:00<00:00, 558.03it/s, v_num=2]
Epoch 6: 27%|██▋ | 19/70 [00:00<00:00, 561.95it/s, v_num=2]
Epoch 6: 27%|██▋ | 19/70 [00:00<00:00, 558.46it/s, v_num=2]
Epoch 6: 29%|██▊ | 20/70 [00:00<00:00, 561.63it/s, v_num=2]
Epoch 6: 29%|██▊ | 20/70 [00:00<00:00, 558.28it/s, v_num=2]
Epoch 6: 30%|███ | 21/70 [00:00<00:00, 561.93it/s, v_num=2]
Epoch 6: 30%|███ | 21/70 [00:00<00:00, 558.75it/s, v_num=2]
Epoch 6: 31%|███▏ | 22/70 [00:00<00:00, 561.46it/s, v_num=2]
Epoch 6: 31%|███▏ | 22/70 [00:00<00:00, 558.43it/s, v_num=2]
Epoch 6: 33%|███▎ | 23/70 [00:00<00:00, 561.71it/s, v_num=2]
Epoch 6: 33%|███▎ | 23/70 [00:00<00:00, 558.81it/s, v_num=2]
Epoch 6: 34%|███▍ | 24/70 [00:00<00:00, 561.24it/s, v_num=2]
Epoch 6: 34%|███▍ | 24/70 [00:00<00:00, 558.46it/s, v_num=2]
Epoch 6: 36%|███▌ | 25/70 [00:00<00:00, 561.53it/s, v_num=2]
Epoch 6: 36%|███▌ | 25/70 [00:00<00:00, 558.87it/s, v_num=2]
Epoch 6: 37%|███▋ | 26/70 [00:00<00:00, 561.40it/s, v_num=2]
Epoch 6: 37%|███▋ | 26/70 [00:00<00:00, 558.83it/s, v_num=2]
Epoch 6: 39%|███▊ | 27/70 [00:00<00:00, 561.61it/s, v_num=2]
Epoch 6: 39%|███▊ | 27/70 [00:00<00:00, 559.14it/s, v_num=2]
Epoch 6: 40%|████ | 28/70 [00:00<00:00, 561.85it/s, v_num=2]
Epoch 6: 40%|████ | 28/70 [00:00<00:00, 559.48it/s, v_num=2]
Epoch 6: 41%|████▏ | 29/70 [00:00<00:00, 561.64it/s, v_num=2]
Epoch 6: 41%|████▏ | 29/70 [00:00<00:00, 559.33it/s, v_num=2]
Epoch 6: 43%|████▎ | 30/70 [00:00<00:00, 561.83it/s, v_num=2]
Epoch 6: 43%|████▎ | 30/70 [00:00<00:00, 559.60it/s, v_num=2]
Epoch 6: 44%|████▍ | 31/70 [00:00<00:00, 552.99it/s, v_num=2]
Epoch 6: 44%|████▍ | 31/70 [00:00<00:00, 550.58it/s, v_num=2]
Epoch 6: 46%|████▌ | 32/70 [00:00<00:00, 551.98it/s, v_num=2]
Epoch 6: 46%|████▌ | 32/70 [00:00<00:00, 549.84it/s, v_num=2]
Epoch 6: 47%|████▋ | 33/70 [00:00<00:00, 551.86it/s, v_num=2]
Epoch 6: 47%|████▋ | 33/70 [00:00<00:00, 549.91it/s, v_num=2]
Epoch 6: 49%|████▊ | 34/70 [00:00<00:00, 552.27it/s, v_num=2]
Epoch 6: 49%|████▊ | 34/70 [00:00<00:00, 550.37it/s, v_num=2]
Epoch 6: 50%|█████ | 35/70 [00:00<00:00, 552.31it/s, v_num=2]
Epoch 6: 50%|█████ | 35/70 [00:00<00:00, 550.46it/s, v_num=2]
Epoch 6: 51%|█████▏ | 36/70 [00:00<00:00, 552.73it/s, v_num=2]
Epoch 6: 51%|█████▏ | 36/70 [00:00<00:00, 550.94it/s, v_num=2]
Epoch 6: 53%|█████▎ | 37/70 [00:00<00:00, 552.68it/s, v_num=2]
Epoch 6: 53%|█████▎ | 37/70 [00:00<00:00, 550.93it/s, v_num=2]
Epoch 6: 54%|█████▍ | 38/70 [00:00<00:00, 553.02it/s, v_num=2]
Epoch 6: 54%|█████▍ | 38/70 [00:00<00:00, 551.31it/s, v_num=2]
Epoch 6: 56%|█████▌ | 39/70 [00:00<00:00, 552.44it/s, v_num=2]
Epoch 6: 56%|█████▌ | 39/70 [00:00<00:00, 550.78it/s, v_num=2]
Epoch 6: 57%|█████▋ | 40/70 [00:00<00:00, 552.79it/s, v_num=2]
Epoch 6: 57%|█████▋ | 40/70 [00:00<00:00, 551.18it/s, v_num=2]
Epoch 6: 59%|█████▊ | 41/70 [00:00<00:00, 553.09it/s, v_num=2]
Epoch 6: 59%|█████▊ | 41/70 [00:00<00:00, 551.23it/s, v_num=2]
Epoch 6: 60%|██████ | 42/70 [00:00<00:00, 553.01it/s, v_num=2]
Epoch 6: 60%|██████ | 42/70 [00:00<00:00, 551.46it/s, v_num=2]
Epoch 6: 61%|██████▏ | 43/70 [00:00<00:00, 553.33it/s, v_num=2]
Epoch 6: 61%|██████▏ | 43/70 [00:00<00:00, 551.83it/s, v_num=2]
Epoch 6: 63%|██████▎ | 44/70 [00:00<00:00, 553.34it/s, v_num=2]
Epoch 6: 63%|██████▎ | 44/70 [00:00<00:00, 551.87it/s, v_num=2]
Epoch 6: 64%|██████▍ | 45/70 [00:00<00:00, 553.61it/s, v_num=2]
Epoch 6: 64%|██████▍ | 45/70 [00:00<00:00, 552.17it/s, v_num=2]
Epoch 6: 66%|██████▌ | 46/70 [00:00<00:00, 553.64it/s, v_num=2]
Epoch 6: 66%|██████▌ | 46/70 [00:00<00:00, 552.23it/s, v_num=2]
Epoch 6: 67%|██████▋ | 47/70 [00:00<00:00, 553.91it/s, v_num=2]
Epoch 6: 67%|██████▋ | 47/70 [00:00<00:00, 552.53it/s, v_num=2]
Epoch 6: 69%|██████▊ | 48/70 [00:00<00:00, 553.93it/s, v_num=2]
Epoch 6: 69%|██████▊ | 48/70 [00:00<00:00, 552.57it/s, v_num=2]
Epoch 6: 70%|███████ | 49/70 [00:00<00:00, 554.18it/s, v_num=2]
Epoch 6: 70%|███████ | 49/70 [00:00<00:00, 552.85it/s, v_num=2]
Epoch 6: 71%|███████▏ | 50/70 [00:00<00:00, 554.36it/s, v_num=2]
Epoch 6: 71%|███████▏ | 50/70 [00:00<00:00, 552.92it/s, v_num=2]
Epoch 6: 73%|███████▎ | 51/70 [00:00<00:00, 554.45it/s, v_num=2]
Epoch 6: 73%|███████▎ | 51/70 [00:00<00:00, 553.16it/s, v_num=2]
Epoch 6: 74%|███████▍ | 52/70 [00:00<00:00, 554.69it/s, v_num=2]
Epoch 6: 74%|███████▍ | 52/70 [00:00<00:00, 553.43it/s, v_num=2]
Epoch 6: 76%|███████▌ | 53/70 [00:00<00:00, 554.67it/s, v_num=2]
Epoch 6: 76%|███████▌ | 53/70 [00:00<00:00, 553.44it/s, v_num=2]
Epoch 6: 77%|███████▋ | 54/70 [00:00<00:00, 554.94it/s, v_num=2]
Epoch 6: 77%|███████▋ | 54/70 [00:00<00:00, 553.73it/s, v_num=2]
Epoch 6: 79%|███████▊ | 55/70 [00:00<00:00, 554.83it/s, v_num=2]
Epoch 6: 79%|███████▊ | 55/70 [00:00<00:00, 553.56it/s, v_num=2]
Epoch 6: 80%|████████ | 56/70 [00:00<00:00, 554.79it/s, v_num=2]
Epoch 6: 80%|████████ | 56/70 [00:00<00:00, 553.62it/s, v_num=2]
Epoch 6: 81%|████████▏ | 57/70 [00:00<00:00, 554.77it/s, v_num=2]
Epoch 6: 81%|████████▏ | 57/70 [00:00<00:00, 553.62it/s, v_num=2]
Epoch 6: 83%|████████▎ | 58/70 [00:00<00:00, 554.97it/s, v_num=2]
Epoch 6: 83%|████████▎ | 58/70 [00:00<00:00, 553.84it/s, v_num=2]
Epoch 6: 84%|████████▍ | 59/70 [00:00<00:00, 554.99it/s, v_num=2]
Epoch 6: 84%|████████▍ | 59/70 [00:00<00:00, 553.86it/s, v_num=2]
Epoch 6: 86%|████████▌ | 60/70 [00:00<00:00, 555.20it/s, v_num=2]
Epoch 6: 86%|████████▌ | 60/70 [00:00<00:00, 554.12it/s, v_num=2]
Epoch 6: 87%|████████▋ | 61/70 [00:00<00:00, 555.32it/s, v_num=2]
Epoch 6: 87%|████████▋ | 61/70 [00:00<00:00, 554.25it/s, v_num=2]
Epoch 6: 89%|████████▊ | 62/70 [00:00<00:00, 555.32it/s, v_num=2]
Epoch 6: 89%|████████▊ | 62/70 [00:00<00:00, 554.27it/s, v_num=2]
Epoch 6: 90%|█████████ | 63/70 [00:00<00:00, 555.47it/s, v_num=2]
Epoch 6: 90%|█████████ | 63/70 [00:00<00:00, 554.43it/s, v_num=2]
Epoch 6: 91%|█████████▏| 64/70 [00:00<00:00, 555.50it/s, v_num=2]
Epoch 6: 91%|█████████▏| 64/70 [00:00<00:00, 554.47it/s, v_num=2]
Epoch 6: 93%|█████████▎| 65/70 [00:00<00:00, 555.71it/s, v_num=2]
Epoch 6: 93%|█████████▎| 65/70 [00:00<00:00, 554.71it/s, v_num=2]
Epoch 6: 94%|█████████▍| 66/70 [00:00<00:00, 555.70it/s, v_num=2]
Epoch 6: 94%|█████████▍| 66/70 [00:00<00:00, 554.71it/s, v_num=2]
Epoch 6: 96%|█████████▌| 67/70 [00:00<00:00, 555.89it/s, v_num=2]
Epoch 6: 96%|█████████▌| 67/70 [00:00<00:00, 554.92it/s, v_num=2]
Epoch 6: 97%|█████████▋| 68/70 [00:00<00:00, 555.92it/s, v_num=2]
Epoch 6: 97%|█████████▋| 68/70 [00:00<00:00, 554.95it/s, v_num=2]
Epoch 6: 99%|█████████▊| 69/70 [00:00<00:00, 556.09it/s, v_num=2]
Epoch 6: 99%|█████████▊| 69/70 [00:00<00:00, 555.14it/s, v_num=2]
Epoch 6: 100%|██████████| 70/70 [00:00<00:00, 556.01it/s, v_num=2]
Epoch 6: 100%|██████████| 70/70 [00:00<00:00, 555.08it/s, v_num=2]
Epoch 6: 100%|██████████| 70/70 [00:00<00:00, 553.09it/s, v_num=2]
Epoch 6: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 7: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 7: 1%|▏ | 1/70 [00:00<00:00, 602.98it/s, v_num=2]
Epoch 7: 1%|▏ | 1/70 [00:00<00:00, 517.11it/s, v_num=2]
Epoch 7: 3%|▎ | 2/70 [00:00<00:00, 571.20it/s, v_num=2]
Epoch 7: 3%|▎ | 2/70 [00:00<00:00, 538.46it/s, v_num=2]
Epoch 7: 4%|▍ | 3/70 [00:00<00:00, 568.87it/s, v_num=2]
Epoch 7: 4%|▍ | 3/70 [00:00<00:00, 546.77it/s, v_num=2]
Epoch 7: 6%|▌ | 4/70 [00:00<00:00, 564.74it/s, v_num=2]
Epoch 7: 6%|▌ | 4/70 [00:00<00:00, 548.33it/s, v_num=2]
Epoch 7: 7%|▋ | 5/70 [00:00<00:00, 565.10it/s, v_num=2]
Epoch 7: 7%|▋ | 5/70 [00:00<00:00, 551.93it/s, v_num=2]
Epoch 7: 9%|▊ | 6/70 [00:00<00:00, 562.40it/s, v_num=2]
Epoch 7: 9%|▊ | 6/70 [00:00<00:00, 551.34it/s, v_num=2]
Epoch 7: 10%|█ | 7/70 [00:00<00:00, 562.80it/s, v_num=2]
Epoch 7: 10%|█ | 7/70 [00:00<00:00, 553.36it/s, v_num=2]
Epoch 7: 11%|█▏ | 8/70 [00:00<00:00, 560.43it/s, v_num=2]
Epoch 7: 11%|█▏ | 8/70 [00:00<00:00, 552.20it/s, v_num=2]
Epoch 7: 13%|█▎ | 9/70 [00:00<00:00, 561.02it/s, v_num=2]
Epoch 7: 13%|█▎ | 9/70 [00:00<00:00, 553.68it/s, v_num=2]
Epoch 7: 14%|█▍ | 10/70 [00:00<00:00, 560.21it/s, v_num=2]
Epoch 7: 14%|█▍ | 10/70 [00:00<00:00, 553.53it/s, v_num=2]
Epoch 7: 16%|█▌ | 11/70 [00:00<00:00, 521.23it/s, v_num=2]
Epoch 7: 16%|█▌ | 11/70 [00:00<00:00, 515.89it/s, v_num=2]
Epoch 7: 17%|█▋ | 12/70 [00:00<00:00, 523.03it/s, v_num=2]
Epoch 7: 17%|█▋ | 12/70 [00:00<00:00, 518.20it/s, v_num=2]
Epoch 7: 19%|█▊ | 13/70 [00:00<00:00, 526.05it/s, v_num=2]
Epoch 7: 19%|█▊ | 13/70 [00:00<00:00, 521.54it/s, v_num=2]
Epoch 7: 20%|██ | 14/70 [00:00<00:00, 527.80it/s, v_num=2]
Epoch 7: 20%|██ | 14/70 [00:00<00:00, 523.62it/s, v_num=2]
Epoch 7: 21%|██▏ | 15/70 [00:00<00:00, 530.28it/s, v_num=2]
Epoch 7: 21%|██▏ | 15/70 [00:00<00:00, 526.31it/s, v_num=2]
Epoch 7: 23%|██▎ | 16/70 [00:00<00:00, 531.43it/s, v_num=2]
Epoch 7: 23%|██▎ | 16/70 [00:00<00:00, 527.69it/s, v_num=2]
Epoch 7: 24%|██▍ | 17/70 [00:00<00:00, 533.41it/s, v_num=2]
Epoch 7: 24%|██▍ | 17/70 [00:00<00:00, 529.88it/s, v_num=2]
Epoch 7: 26%|██▌ | 18/70 [00:00<00:00, 535.15it/s, v_num=2]
Epoch 7: 26%|██▌ | 18/70 [00:00<00:00, 531.18it/s, v_num=2]
Epoch 7: 27%|██▋ | 19/70 [00:00<00:00, 536.03it/s, v_num=2]
Epoch 7: 27%|██▋ | 19/70 [00:00<00:00, 532.82it/s, v_num=2]
Epoch 7: 29%|██▊ | 20/70 [00:00<00:00, 537.42it/s, v_num=2]
Epoch 7: 29%|██▊ | 20/70 [00:00<00:00, 534.33it/s, v_num=2]
Epoch 7: 30%|███ | 21/70 [00:00<00:00, 538.20it/s, v_num=2]
Epoch 7: 30%|███ | 21/70 [00:00<00:00, 535.27it/s, v_num=2]
Epoch 7: 31%|███▏ | 22/70 [00:00<00:00, 539.33it/s, v_num=2]
Epoch 7: 31%|███▏ | 22/70 [00:00<00:00, 536.49it/s, v_num=2]
Epoch 7: 33%|███▎ | 23/70 [00:00<00:00, 539.87it/s, v_num=2]
Epoch 7: 33%|███▎ | 23/70 [00:00<00:00, 537.18it/s, v_num=2]
Epoch 7: 34%|███▍ | 24/70 [00:00<00:00, 540.92it/s, v_num=2]
Epoch 7: 34%|███▍ | 24/70 [00:00<00:00, 538.34it/s, v_num=2]
Epoch 7: 36%|███▌ | 25/70 [00:00<00:00, 541.30it/s, v_num=2]
Epoch 7: 36%|███▌ | 25/70 [00:00<00:00, 538.82it/s, v_num=2]
Epoch 7: 37%|███▋ | 26/70 [00:00<00:00, 542.21it/s, v_num=2]
Epoch 7: 37%|███▋ | 26/70 [00:00<00:00, 539.83it/s, v_num=2]
Epoch 7: 39%|███▊ | 27/70 [00:00<00:00, 542.94it/s, v_num=2]
Epoch 7: 39%|███▊ | 27/70 [00:00<00:00, 540.39it/s, v_num=2]
Epoch 7: 40%|████ | 28/70 [00:00<00:00, 542.96it/s, v_num=2]
Epoch 7: 40%|████ | 28/70 [00:00<00:00, 540.72it/s, v_num=2]
Epoch 7: 41%|████▏ | 29/70 [00:00<00:00, 543.79it/s, v_num=2]
Epoch 7: 41%|████▏ | 29/70 [00:00<00:00, 541.64it/s, v_num=2]
Epoch 7: 43%|████▎ | 30/70 [00:00<00:00, 544.12it/s, v_num=2]
Epoch 7: 43%|████▎ | 30/70 [00:00<00:00, 542.04it/s, v_num=2]
Epoch 7: 44%|████▍ | 31/70 [00:00<00:00, 544.66it/s, v_num=2]
Epoch 7: 44%|████▍ | 31/70 [00:00<00:00, 542.63it/s, v_num=2]
Epoch 7: 46%|████▌ | 32/70 [00:00<00:00, 544.85it/s, v_num=2]
Epoch 7: 46%|████▌ | 32/70 [00:00<00:00, 542.88it/s, v_num=2]
Epoch 7: 47%|████▋ | 33/70 [00:00<00:00, 545.49it/s, v_num=2]
Epoch 7: 47%|████▋ | 33/70 [00:00<00:00, 543.57it/s, v_num=2]
Epoch 7: 49%|████▊ | 34/70 [00:00<00:00, 545.66it/s, v_num=2]
Epoch 7: 49%|████▊ | 34/70 [00:00<00:00, 543.80it/s, v_num=2]
Epoch 7: 50%|█████ | 35/70 [00:00<00:00, 546.26it/s, v_num=2]
Epoch 7: 50%|█████ | 35/70 [00:00<00:00, 544.45it/s, v_num=2]
Epoch 7: 51%|█████▏ | 36/70 [00:00<00:00, 546.51it/s, v_num=2]
Epoch 7: 51%|█████▏ | 36/70 [00:00<00:00, 544.73it/s, v_num=2]
Epoch 7: 53%|█████▎ | 37/70 [00:00<00:00, 547.02it/s, v_num=2]
Epoch 7: 53%|█████▎ | 37/70 [00:00<00:00, 545.30it/s, v_num=2]
Epoch 7: 54%|█████▍ | 38/70 [00:00<00:00, 547.52it/s, v_num=2]
Epoch 7: 54%|█████▍ | 38/70 [00:00<00:00, 545.83it/s, v_num=2]
Epoch 7: 56%|█████▌ | 39/70 [00:00<00:00, 547.71it/s, v_num=2]
Epoch 7: 56%|█████▌ | 39/70 [00:00<00:00, 546.06it/s, v_num=2]
Epoch 7: 57%|█████▋ | 40/70 [00:00<00:00, 548.20it/s, v_num=2]
Epoch 7: 57%|█████▋ | 40/70 [00:00<00:00, 546.62it/s, v_num=2]
Epoch 7: 59%|█████▊ | 41/70 [00:00<00:00, 548.33it/s, v_num=2]
Epoch 7: 59%|█████▊ | 41/70 [00:00<00:00, 546.78it/s, v_num=2]
Epoch 7: 60%|██████ | 42/70 [00:00<00:00, 548.76it/s, v_num=2]
Epoch 7: 60%|██████ | 42/70 [00:00<00:00, 547.25it/s, v_num=2]
Epoch 7: 61%|██████▏ | 43/70 [00:00<00:00, 548.89it/s, v_num=2]
Epoch 7: 61%|██████▏ | 43/70 [00:00<00:00, 547.39it/s, v_num=2]
Epoch 7: 63%|██████▎ | 44/70 [00:00<00:00, 549.29it/s, v_num=2]
Epoch 7: 63%|██████▎ | 44/70 [00:00<00:00, 547.83it/s, v_num=2]
Epoch 7: 64%|██████▍ | 45/70 [00:00<00:00, 549.40it/s, v_num=2]
Epoch 7: 64%|██████▍ | 45/70 [00:00<00:00, 547.96it/s, v_num=2]
Epoch 7: 66%|██████▌ | 46/70 [00:00<00:00, 549.77it/s, v_num=2]
Epoch 7: 66%|██████▌ | 46/70 [00:00<00:00, 548.38it/s, v_num=2]
Epoch 7: 67%|██████▋ | 47/70 [00:00<00:00, 550.15it/s, v_num=2]
Epoch 7: 67%|██████▋ | 47/70 [00:00<00:00, 548.79it/s, v_num=2]
Epoch 7: 69%|██████▊ | 48/70 [00:00<00:00, 550.32it/s, v_num=2]
Epoch 7: 69%|██████▊ | 48/70 [00:00<00:00, 548.98it/s, v_num=2]
Epoch 7: 70%|███████ | 49/70 [00:00<00:00, 549.86it/s, v_num=2]
Epoch 7: 70%|███████ | 49/70 [00:00<00:00, 548.55it/s, v_num=2]
Epoch 7: 71%|███████▏ | 50/70 [00:00<00:00, 549.92it/s, v_num=2]
Epoch 7: 71%|███████▏ | 50/70 [00:00<00:00, 548.64it/s, v_num=2]
Epoch 7: 73%|███████▎ | 51/70 [00:00<00:00, 550.21it/s, v_num=2]
Epoch 7: 73%|███████▎ | 51/70 [00:00<00:00, 548.96it/s, v_num=2]
Epoch 7: 74%|███████▍ | 52/70 [00:00<00:00, 550.30it/s, v_num=2]
Epoch 7: 74%|███████▍ | 52/70 [00:00<00:00, 549.06it/s, v_num=2]
Epoch 7: 76%|███████▌ | 53/70 [00:00<00:00, 550.58it/s, v_num=2]
Epoch 7: 76%|███████▌ | 53/70 [00:00<00:00, 549.36it/s, v_num=2]
Epoch 7: 77%|███████▋ | 54/70 [00:00<00:00, 550.64it/s, v_num=2]
Epoch 7: 77%|███████▋ | 54/70 [00:00<00:00, 549.45it/s, v_num=2]
Epoch 7: 79%|███████▊ | 55/70 [00:00<00:00, 550.91it/s, v_num=2]
Epoch 7: 79%|███████▊ | 55/70 [00:00<00:00, 549.74it/s, v_num=2]
Epoch 7: 80%|████████ | 56/70 [00:00<00:00, 551.14it/s, v_num=2]
Epoch 7: 80%|████████ | 56/70 [00:00<00:00, 549.78it/s, v_num=2]
Epoch 7: 81%|████████▏ | 57/70 [00:00<00:00, 551.16it/s, v_num=2]
Epoch 7: 81%|████████▏ | 57/70 [00:00<00:00, 550.03it/s, v_num=2]
Epoch 7: 83%|████████▎ | 58/70 [00:00<00:00, 551.39it/s, v_num=2]
Epoch 7: 83%|████████▎ | 58/70 [00:00<00:00, 550.28it/s, v_num=2]
Epoch 7: 84%|████████▍ | 59/70 [00:00<00:00, 551.19it/s, v_num=2]
Epoch 7: 84%|████████▍ | 59/70 [00:00<00:00, 550.09it/s, v_num=2]
Epoch 7: 86%|████████▌ | 60/70 [00:00<00:00, 551.44it/s, v_num=2]
Epoch 7: 86%|████████▌ | 60/70 [00:00<00:00, 550.36it/s, v_num=2]
Epoch 7: 87%|████████▋ | 61/70 [00:00<00:00, 547.50it/s, v_num=2]
Epoch 7: 87%|████████▋ | 61/70 [00:00<00:00, 546.44it/s, v_num=2]
Epoch 7: 89%|████████▊ | 62/70 [00:00<00:00, 547.78it/s, v_num=2]
Epoch 7: 89%|████████▊ | 62/70 [00:00<00:00, 546.75it/s, v_num=2]
Epoch 7: 90%|█████████ | 63/70 [00:00<00:00, 547.76it/s, v_num=2]
Epoch 7: 90%|█████████ | 63/70 [00:00<00:00, 546.75it/s, v_num=2]
Epoch 7: 91%|█████████▏| 64/70 [00:00<00:00, 548.02it/s, v_num=2]
Epoch 7: 91%|█████████▏| 64/70 [00:00<00:00, 547.02it/s, v_num=2]
Epoch 7: 93%|█████████▎| 65/70 [00:00<00:00, 548.10it/s, v_num=2]
Epoch 7: 93%|█████████▎| 65/70 [00:00<00:00, 547.11it/s, v_num=2]
Epoch 7: 94%|█████████▍| 66/70 [00:00<00:00, 548.16it/s, v_num=2]
Epoch 7: 94%|█████████▍| 66/70 [00:00<00:00, 547.19it/s, v_num=2]
Epoch 7: 96%|█████████▌| 67/70 [00:00<00:00, 547.99it/s, v_num=2]
Epoch 7: 96%|█████████▌| 67/70 [00:00<00:00, 547.03it/s, v_num=2]
Epoch 7: 97%|█████████▋| 68/70 [00:00<00:00, 548.22it/s, v_num=2]
Epoch 7: 97%|█████████▋| 68/70 [00:00<00:00, 547.29it/s, v_num=2]
Epoch 7: 99%|█████████▊| 69/70 [00:00<00:00, 548.34it/s, v_num=2]
Epoch 7: 99%|█████████▊| 69/70 [00:00<00:00, 547.42it/s, v_num=2]
Epoch 7: 100%|██████████| 70/70 [00:00<00:00, 548.59it/s, v_num=2]
Epoch 7: 100%|██████████| 70/70 [00:00<00:00, 547.68it/s, v_num=2]
Epoch 7: 100%|██████████| 70/70 [00:00<00:00, 545.86it/s, v_num=2]
Epoch 7: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 8: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 8: 1%|▏ | 1/70 [00:00<00:00, 601.85it/s, v_num=2]
Epoch 8: 1%|▏ | 1/70 [00:00<00:00, 533.08it/s, v_num=2]
Epoch 8: 3%|▎ | 2/70 [00:00<00:00, 582.99it/s, v_num=2]
Epoch 8: 3%|▎ | 2/70 [00:00<00:00, 548.78it/s, v_num=2]
Epoch 8: 4%|▍ | 3/70 [00:00<00:00, 572.50it/s, v_num=2]
Epoch 8: 4%|▍ | 3/70 [00:00<00:00, 550.12it/s, v_num=2]
Epoch 8: 6%|▌ | 4/70 [00:00<00:00, 570.65it/s, v_num=2]
Epoch 8: 6%|▌ | 4/70 [00:00<00:00, 553.81it/s, v_num=2]
Epoch 8: 7%|▋ | 5/70 [00:00<00:00, 565.85it/s, v_num=2]
Epoch 8: 7%|▋ | 5/70 [00:00<00:00, 552.48it/s, v_num=2]
Epoch 8: 9%|▊ | 6/70 [00:00<00:00, 565.89it/s, v_num=2]
Epoch 8: 9%|▊ | 6/70 [00:00<00:00, 554.78it/s, v_num=2]
Epoch 8: 10%|█ | 7/70 [00:00<00:00, 563.19it/s, v_num=2]
Epoch 8: 10%|█ | 7/70 [00:00<00:00, 553.69it/s, v_num=2]
Epoch 8: 11%|█▏ | 8/70 [00:00<00:00, 563.51it/s, v_num=2]
Epoch 8: 11%|█▏ | 8/70 [00:00<00:00, 555.19it/s, v_num=2]
Epoch 8: 13%|█▎ | 9/70 [00:00<00:00, 562.16it/s, v_num=2]
Epoch 8: 13%|█▎ | 9/70 [00:00<00:00, 554.75it/s, v_num=2]
Epoch 8: 14%|█▍ | 10/70 [00:00<00:00, 553.59it/s, v_num=2]
Epoch 8: 14%|█▍ | 10/70 [00:00<00:00, 547.05it/s, v_num=2]
Epoch 8: 16%|█▌ | 11/70 [00:00<00:00, 553.68it/s, v_num=2]
Epoch 8: 16%|█▌ | 11/70 [00:00<00:00, 547.70it/s, v_num=2]
Epoch 8: 17%|█▋ | 12/70 [00:00<00:00, 554.58it/s, v_num=2]
Epoch 8: 17%|█▋ | 12/70 [00:00<00:00, 549.18it/s, v_num=2]
Epoch 8: 19%|█▊ | 13/70 [00:00<00:00, 550.64it/s, v_num=2]
Epoch 8: 19%|█▊ | 13/70 [00:00<00:00, 544.94it/s, v_num=2]
Epoch 8: 20%|██ | 14/70 [00:00<00:00, 550.74it/s, v_num=2]
Epoch 8: 20%|██ | 14/70 [00:00<00:00, 546.17it/s, v_num=2]
Epoch 8: 21%|██▏ | 15/70 [00:00<00:00, 551.54it/s, v_num=2]
Epoch 8: 21%|██▏ | 15/70 [00:00<00:00, 547.27it/s, v_num=2]
Epoch 8: 23%|██▎ | 16/70 [00:00<00:00, 551.80it/s, v_num=2]
Epoch 8: 23%|██▎ | 16/70 [00:00<00:00, 547.79it/s, v_num=2]
Epoch 8: 24%|██▍ | 17/70 [00:00<00:00, 549.68it/s, v_num=2]
Epoch 8: 24%|██▍ | 17/70 [00:00<00:00, 545.10it/s, v_num=2]
Epoch 8: 26%|██▌ | 18/70 [00:00<00:00, 548.46it/s, v_num=2]
Epoch 8: 26%|██▌ | 18/70 [00:00<00:00, 544.86it/s, v_num=2]
Epoch 8: 27%|██▋ | 19/70 [00:00<00:00, 549.20it/s, v_num=2]
Epoch 8: 27%|██▋ | 19/70 [00:00<00:00, 545.82it/s, v_num=2]
Epoch 8: 29%|██▊ | 20/70 [00:00<00:00, 546.12it/s, v_num=2]
Epoch 8: 29%|██▊ | 20/70 [00:00<00:00, 542.89it/s, v_num=2]
Epoch 8: 30%|███ | 21/70 [00:00<00:00, 547.00it/s, v_num=2]
Epoch 8: 30%|███ | 21/70 [00:00<00:00, 543.95it/s, v_num=2]
Epoch 8: 31%|███▏ | 22/70 [00:00<00:00, 547.33it/s, v_num=2]
Epoch 8: 31%|███▏ | 22/70 [00:00<00:00, 544.45it/s, v_num=2]
Epoch 8: 33%|███▎ | 23/70 [00:00<00:00, 548.21it/s, v_num=2]
Epoch 8: 33%|███▎ | 23/70 [00:00<00:00, 545.44it/s, v_num=2]
Epoch 8: 34%|███▍ | 24/70 [00:00<00:00, 548.99it/s, v_num=2]
Epoch 8: 34%|███▍ | 24/70 [00:00<00:00, 545.88it/s, v_num=2]
Epoch 8: 36%|███▌ | 25/70 [00:00<00:00, 546.99it/s, v_num=2]
Epoch 8: 36%|███▌ | 25/70 [00:00<00:00, 544.43it/s, v_num=2]
Epoch 8: 37%|███▋ | 26/70 [00:00<00:00, 547.73it/s, v_num=2]
Epoch 8: 37%|███▋ | 26/70 [00:00<00:00, 545.28it/s, v_num=2]
Epoch 8: 39%|███▊ | 27/70 [00:00<00:00, 548.04it/s, v_num=2]
Epoch 8: 39%|███▊ | 27/70 [00:00<00:00, 545.69it/s, v_num=2]
Epoch 8: 40%|████ | 28/70 [00:00<00:00, 546.94it/s, v_num=2]
Epoch 8: 40%|████ | 28/70 [00:00<00:00, 544.42it/s, v_num=2]
Epoch 8: 41%|████▏ | 29/70 [00:00<00:00, 546.97it/s, v_num=2]
Epoch 8: 41%|████▏ | 29/70 [00:00<00:00, 544.77it/s, v_num=2]
Epoch 8: 43%|████▎ | 30/70 [00:00<00:00, 547.61it/s, v_num=2]
Epoch 8: 43%|████▎ | 30/70 [00:00<00:00, 545.50it/s, v_num=2]
Epoch 8: 44%|████▍ | 31/70 [00:00<00:00, 547.83it/s, v_num=2]
Epoch 8: 44%|████▍ | 31/70 [00:00<00:00, 545.79it/s, v_num=2]
Epoch 8: 46%|████▌ | 32/70 [00:00<00:00, 548.33it/s, v_num=2]
Epoch 8: 46%|████▌ | 32/70 [00:00<00:00, 546.32it/s, v_num=2]
Epoch 8: 47%|████▋ | 33/70 [00:00<00:00, 548.46it/s, v_num=2]
Epoch 8: 47%|████▋ | 33/70 [00:00<00:00, 546.53it/s, v_num=2]
Epoch 8: 49%|████▊ | 34/70 [00:00<00:00, 548.98it/s, v_num=2]
Epoch 8: 49%|████▊ | 34/70 [00:00<00:00, 547.08it/s, v_num=2]
Epoch 8: 50%|█████ | 35/70 [00:00<00:00, 547.48it/s, v_num=2]
Epoch 8: 50%|█████ | 35/70 [00:00<00:00, 545.63it/s, v_num=2]
Epoch 8: 51%|█████▏ | 36/70 [00:00<00:00, 547.98it/s, v_num=2]
Epoch 8: 51%|█████▏ | 36/70 [00:00<00:00, 546.20it/s, v_num=2]
Epoch 8: 53%|█████▎ | 37/70 [00:00<00:00, 548.50it/s, v_num=2]
Epoch 8: 53%|█████▎ | 37/70 [00:00<00:00, 546.77it/s, v_num=2]
Epoch 8: 54%|█████▍ | 38/70 [00:00<00:00, 548.73it/s, v_num=2]
Epoch 8: 54%|█████▍ | 38/70 [00:00<00:00, 547.04it/s, v_num=2]
Epoch 8: 56%|█████▌ | 39/70 [00:00<00:00, 549.13it/s, v_num=2]
Epoch 8: 56%|█████▌ | 39/70 [00:00<00:00, 547.50it/s, v_num=2]
Epoch 8: 57%|█████▋ | 40/70 [00:00<00:00, 549.31it/s, v_num=2]
Epoch 8: 57%|█████▋ | 40/70 [00:00<00:00, 547.69it/s, v_num=2]
Epoch 8: 59%|█████▊ | 41/70 [00:00<00:00, 524.48it/s, v_num=2]
Epoch 8: 59%|█████▊ | 41/70 [00:00<00:00, 523.02it/s, v_num=2]
Epoch 8: 60%|██████ | 42/70 [00:00<00:00, 525.04it/s, v_num=2]
Epoch 8: 60%|██████ | 42/70 [00:00<00:00, 523.63it/s, v_num=2]
Epoch 8: 61%|██████▏ | 43/70 [00:00<00:00, 525.89it/s, v_num=2]
Epoch 8: 61%|██████▏ | 43/70 [00:00<00:00, 524.52it/s, v_num=2]
Epoch 8: 63%|██████▎ | 44/70 [00:00<00:00, 526.79it/s, v_num=2]
Epoch 8: 63%|██████▎ | 44/70 [00:00<00:00, 525.37it/s, v_num=2]
Epoch 8: 64%|██████▍ | 45/70 [00:00<00:00, 527.22it/s, v_num=2]
Epoch 8: 64%|██████▍ | 45/70 [00:00<00:00, 525.89it/s, v_num=2]
Epoch 8: 66%|██████▌ | 46/70 [00:00<00:00, 527.64it/s, v_num=2]
Epoch 8: 66%|██████▌ | 46/70 [00:00<00:00, 526.35it/s, v_num=2]
Epoch 8: 67%|██████▋ | 47/70 [00:00<00:00, 528.14it/s, v_num=2]
Epoch 8: 67%|██████▋ | 47/70 [00:00<00:00, 526.86it/s, v_num=2]
Epoch 8: 69%|██████▊ | 48/70 [00:00<00:00, 528.87it/s, v_num=2]
Epoch 8: 69%|██████▊ | 48/70 [00:00<00:00, 527.64it/s, v_num=2]
Epoch 8: 70%|███████ | 49/70 [00:00<00:00, 529.21it/s, v_num=2]
Epoch 8: 70%|███████ | 49/70 [00:00<00:00, 527.98it/s, v_num=2]
Epoch 8: 71%|███████▏ | 50/70 [00:00<00:00, 529.84it/s, v_num=2]
Epoch 8: 71%|███████▏ | 50/70 [00:00<00:00, 528.64it/s, v_num=2]
Epoch 8: 73%|███████▎ | 51/70 [00:00<00:00, 530.20it/s, v_num=2]
Epoch 8: 73%|███████▎ | 51/70 [00:00<00:00, 529.02it/s, v_num=2]
Epoch 8: 74%|███████▍ | 52/70 [00:00<00:00, 530.75it/s, v_num=2]
Epoch 8: 74%|███████▍ | 52/70 [00:00<00:00, 529.59it/s, v_num=2]
Epoch 8: 76%|███████▌ | 53/70 [00:00<00:00, 531.29it/s, v_num=2]
Epoch 8: 76%|███████▌ | 53/70 [00:00<00:00, 530.03it/s, v_num=2]
Epoch 8: 77%|███████▋ | 54/70 [00:00<00:00, 531.76it/s, v_num=2]
Epoch 8: 77%|███████▋ | 54/70 [00:00<00:00, 530.65it/s, v_num=2]
Epoch 8: 79%|███████▊ | 55/70 [00:00<00:00, 532.35it/s, v_num=2]
Epoch 8: 79%|███████▊ | 55/70 [00:00<00:00, 531.26it/s, v_num=2]
Epoch 8: 80%|████████ | 56/70 [00:00<00:00, 532.72it/s, v_num=2]
Epoch 8: 80%|████████ | 56/70 [00:00<00:00, 531.64it/s, v_num=2]
Epoch 8: 81%|████████▏ | 57/70 [00:00<00:00, 533.27it/s, v_num=2]
Epoch 8: 81%|████████▏ | 57/70 [00:00<00:00, 532.22it/s, v_num=2]
Epoch 8: 83%|████████▎ | 58/70 [00:00<00:00, 533.63it/s, v_num=2]
Epoch 8: 83%|████████▎ | 58/70 [00:00<00:00, 532.59it/s, v_num=2]
Epoch 8: 84%|████████▍ | 59/70 [00:00<00:00, 534.08it/s, v_num=2]
Epoch 8: 84%|████████▍ | 59/70 [00:00<00:00, 533.05it/s, v_num=2]
Epoch 8: 86%|████████▌ | 60/70 [00:00<00:00, 534.37it/s, v_num=2]
Epoch 8: 86%|████████▌ | 60/70 [00:00<00:00, 533.36it/s, v_num=2]
Epoch 8: 87%|████████▋ | 61/70 [00:00<00:00, 534.85it/s, v_num=2]
Epoch 8: 87%|████████▋ | 61/70 [00:00<00:00, 533.85it/s, v_num=2]
Epoch 8: 89%|████████▊ | 62/70 [00:00<00:00, 535.19it/s, v_num=2]
Epoch 8: 89%|████████▊ | 62/70 [00:00<00:00, 534.20it/s, v_num=2]
Epoch 8: 90%|█████████ | 63/70 [00:00<00:00, 535.64it/s, v_num=2]
Epoch 8: 90%|█████████ | 63/70 [00:00<00:00, 534.67it/s, v_num=2]
Epoch 8: 91%|█████████▏| 64/70 [00:00<00:00, 536.08it/s, v_num=2]
Epoch 8: 91%|█████████▏| 64/70 [00:00<00:00, 535.13it/s, v_num=2]
Epoch 8: 93%|█████████▎| 65/70 [00:00<00:00, 536.32it/s, v_num=2]
Epoch 8: 93%|█████████▎| 65/70 [00:00<00:00, 535.38it/s, v_num=2]
Epoch 8: 94%|█████████▍| 66/70 [00:00<00:00, 536.76it/s, v_num=2]
Epoch 8: 94%|█████████▍| 66/70 [00:00<00:00, 535.83it/s, v_num=2]
Epoch 8: 96%|█████████▌| 67/70 [00:00<00:00, 537.02it/s, v_num=2]
Epoch 8: 96%|█████████▌| 67/70 [00:00<00:00, 536.11it/s, v_num=2]
Epoch 8: 97%|█████████▋| 68/70 [00:00<00:00, 537.41it/s, v_num=2]
Epoch 8: 97%|█████████▋| 68/70 [00:00<00:00, 536.50it/s, v_num=2]
Epoch 8: 99%|█████████▊| 69/70 [00:00<00:00, 537.68it/s, v_num=2]
Epoch 8: 99%|█████████▊| 69/70 [00:00<00:00, 536.78it/s, v_num=2]
Epoch 8: 100%|██████████| 70/70 [00:00<00:00, 537.93it/s, v_num=2]
Epoch 8: 100%|██████████| 70/70 [00:00<00:00, 537.06it/s, v_num=2]
Epoch 8: 100%|██████████| 70/70 [00:00<00:00, 535.36it/s, v_num=2]
Epoch 8: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 9: 0%| | 0/70 [00:00<?, ?it/s, v_num=2]
Epoch 9: 1%|▏ | 1/70 [00:00<00:00, 608.49it/s, v_num=2]
Epoch 9: 1%|▏ | 1/70 [00:00<00:00, 538.42it/s, v_num=2]
Epoch 9: 3%|▎ | 2/70 [00:00<00:00, 575.78it/s, v_num=2]
Epoch 9: 3%|▎ | 2/70 [00:00<00:00, 542.36it/s, v_num=2]
Epoch 9: 4%|▍ | 3/70 [00:00<00:00, 572.76it/s, v_num=2]
Epoch 9: 4%|▍ | 3/70 [00:00<00:00, 550.53it/s, v_num=2]
Epoch 9: 6%|▌ | 4/70 [00:00<00:00, 567.91it/s, v_num=2]
Epoch 9: 6%|▌ | 4/70 [00:00<00:00, 548.53it/s, v_num=2]
Epoch 9: 7%|▋ | 5/70 [00:00<00:00, 559.17it/s, v_num=2]
Epoch 9: 7%|▋ | 5/70 [00:00<00:00, 546.28it/s, v_num=2]
Epoch 9: 9%|▊ | 6/70 [00:00<00:00, 560.49it/s, v_num=2]
Epoch 9: 9%|▊ | 6/70 [00:00<00:00, 549.59it/s, v_num=2]
Epoch 9: 10%|█ | 7/70 [00:00<00:00, 558.87it/s, v_num=2]
Epoch 9: 10%|█ | 7/70 [00:00<00:00, 549.49it/s, v_num=2]
Epoch 9: 11%|█▏ | 8/70 [00:00<00:00, 559.36it/s, v_num=2]
Epoch 9: 11%|█▏ | 8/70 [00:00<00:00, 551.16it/s, v_num=2]
Epoch 9: 13%|█▎ | 9/70 [00:00<00:00, 558.48it/s, v_num=2]
Epoch 9: 13%|█▎ | 9/70 [00:00<00:00, 551.21it/s, v_num=2]
Epoch 9: 14%|█▍ | 10/70 [00:00<00:00, 558.54it/s, v_num=2]
Epoch 9: 14%|█▍ | 10/70 [00:00<00:00, 551.93it/s, v_num=2]
Epoch 9: 16%|█▌ | 11/70 [00:00<00:00, 557.90it/s, v_num=2]
Epoch 9: 16%|█▌ | 11/70 [00:00<00:00, 551.92it/s, v_num=2]
Epoch 9: 17%|█▋ | 12/70 [00:00<00:00, 558.48it/s, v_num=2]
Epoch 9: 17%|█▋ | 12/70 [00:00<00:00, 552.97it/s, v_num=2]
Epoch 9: 19%|█▊ | 13/70 [00:00<00:00, 558.03it/s, v_num=2]
Epoch 9: 19%|█▊ | 13/70 [00:00<00:00, 552.92it/s, v_num=2]
Epoch 9: 20%|██ | 14/70 [00:00<00:00, 558.09it/s, v_num=2]
Epoch 9: 20%|██ | 14/70 [00:00<00:00, 551.62it/s, v_num=2]
Epoch 9: 21%|██▏ | 15/70 [00:00<00:00, 553.81it/s, v_num=2]
Epoch 9: 21%|██▏ | 15/70 [00:00<00:00, 548.77it/s, v_num=2]
Epoch 9: 23%|██▎ | 16/70 [00:00<00:00, 553.61it/s, v_num=2]
Epoch 9: 23%|██▎ | 16/70 [00:00<00:00, 549.51it/s, v_num=2]
Epoch 9: 24%|██▍ | 17/70 [00:00<00:00, 553.85it/s, v_num=2]
Epoch 9: 24%|██▍ | 17/70 [00:00<00:00, 550.02it/s, v_num=2]
Epoch 9: 26%|██▌ | 18/70 [00:00<00:00, 553.67it/s, v_num=2]
Epoch 9: 26%|██▌ | 18/70 [00:00<00:00, 550.06it/s, v_num=2]
Epoch 9: 27%|██▋ | 19/70 [00:00<00:00, 554.13it/s, v_num=2]
Epoch 9: 27%|██▋ | 19/70 [00:00<00:00, 550.70it/s, v_num=2]
Epoch 9: 29%|██▊ | 20/70 [00:00<00:00, 554.07it/s, v_num=2]
Epoch 9: 29%|██▊ | 20/70 [00:00<00:00, 550.81it/s, v_num=2]
Epoch 9: 30%|███ | 21/70 [00:00<00:00, 542.30it/s, v_num=2]
Epoch 9: 30%|███ | 21/70 [00:00<00:00, 539.31it/s, v_num=2]
Epoch 9: 31%|███▏ | 22/70 [00:00<00:00, 542.46it/s, v_num=2]
Epoch 9: 31%|███▏ | 22/70 [00:00<00:00, 539.61it/s, v_num=2]
Epoch 9: 33%|███▎ | 23/70 [00:00<00:00, 543.38it/s, v_num=2]
Epoch 9: 33%|███▎ | 23/70 [00:00<00:00, 540.68it/s, v_num=2]
Epoch 9: 34%|███▍ | 24/70 [00:00<00:00, 543.83it/s, v_num=2]
Epoch 9: 34%|███▍ | 24/70 [00:00<00:00, 541.22it/s, v_num=2]
Epoch 9: 36%|███▌ | 25/70 [00:00<00:00, 541.45it/s, v_num=2]
Epoch 9: 36%|███▌ | 25/70 [00:00<00:00, 538.94it/s, v_num=2]
Epoch 9: 37%|███▋ | 26/70 [00:00<00:00, 541.66it/s, v_num=2]
Epoch 9: 37%|███▋ | 26/70 [00:00<00:00, 539.26it/s, v_num=2]
Epoch 9: 39%|███▊ | 27/70 [00:00<00:00, 542.39it/s, v_num=2]
Epoch 9: 39%|███▊ | 27/70 [00:00<00:00, 540.06it/s, v_num=2]
Epoch 9: 40%|████ | 28/70 [00:00<00:00, 542.71it/s, v_num=2]
Epoch 9: 40%|████ | 28/70 [00:00<00:00, 540.48it/s, v_num=2]
Epoch 9: 41%|████▏ | 29/70 [00:00<00:00, 543.41it/s, v_num=2]
Epoch 9: 41%|████▏ | 29/70 [00:00<00:00, 541.26it/s, v_num=2]
Epoch 9: 43%|████▎ | 30/70 [00:00<00:00, 544.17it/s, v_num=2]
Epoch 9: 43%|████▎ | 30/70 [00:00<00:00, 541.69it/s, v_num=2]
Epoch 9: 44%|████▍ | 31/70 [00:00<00:00, 543.74it/s, v_num=2]
Epoch 9: 44%|████▍ | 31/70 [00:00<00:00, 541.69it/s, v_num=2]
Epoch 9: 46%|████▌ | 32/70 [00:00<00:00, 544.35it/s, v_num=2]
Epoch 9: 46%|████▌ | 32/70 [00:00<00:00, 542.39it/s, v_num=2]
Epoch 9: 47%|████▋ | 33/70 [00:00<00:00, 544.62it/s, v_num=2]
Epoch 9: 47%|████▋ | 33/70 [00:00<00:00, 542.71it/s, v_num=2]
Epoch 9: 49%|████▊ | 34/70 [00:00<00:00, 543.09it/s, v_num=2]
Epoch 9: 49%|████▊ | 34/70 [00:00<00:00, 541.24it/s, v_num=2]
Epoch 9: 50%|█████ | 35/70 [00:00<00:00, 542.92it/s, v_num=2]
Epoch 9: 50%|█████ | 35/70 [00:00<00:00, 541.12it/s, v_num=2]
Epoch 9: 51%|█████▏ | 36/70 [00:00<00:00, 543.43it/s, v_num=2]
Epoch 9: 51%|█████▏ | 36/70 [00:00<00:00, 541.68it/s, v_num=2]
Epoch 9: 53%|█████▎ | 37/70 [00:00<00:00, 543.54it/s, v_num=2]
Epoch 9: 53%|█████▎ | 37/70 [00:00<00:00, 541.85it/s, v_num=2]
Epoch 9: 54%|█████▍ | 38/70 [00:00<00:00, 544.06it/s, v_num=2]
Epoch 9: 54%|█████▍ | 38/70 [00:00<00:00, 542.41it/s, v_num=2]
Epoch 9: 56%|█████▌ | 39/70 [00:00<00:00, 544.28it/s, v_num=2]
Epoch 9: 56%|█████▌ | 39/70 [00:00<00:00, 542.25it/s, v_num=2]
Epoch 9: 57%|█████▋ | 40/70 [00:00<00:00, 543.01it/s, v_num=2]
Epoch 9: 57%|█████▋ | 40/70 [00:00<00:00, 541.43it/s, v_num=2]
Epoch 9: 59%|█████▊ | 41/70 [00:00<00:00, 543.23it/s, v_num=2]
Epoch 9: 59%|█████▊ | 41/70 [00:00<00:00, 541.70it/s, v_num=2]
Epoch 9: 60%|██████ | 42/70 [00:00<00:00, 542.32it/s, v_num=2]
Epoch 9: 60%|██████ | 42/70 [00:00<00:00, 540.82it/s, v_num=2]
Epoch 9: 61%|██████▏ | 43/70 [00:00<00:00, 542.46it/s, v_num=2]
Epoch 9: 61%|██████▏ | 43/70 [00:00<00:00, 540.99it/s, v_num=2]
Epoch 9: 63%|██████▎ | 44/70 [00:00<00:00, 542.97it/s, v_num=2]
Epoch 9: 63%|██████▎ | 44/70 [00:00<00:00, 541.56it/s, v_num=2]
Epoch 9: 64%|██████▍ | 45/70 [00:00<00:00, 543.48it/s, v_num=2]
Epoch 9: 64%|██████▍ | 45/70 [00:00<00:00, 542.09it/s, v_num=2]
Epoch 9: 66%|██████▌ | 46/70 [00:00<00:00, 543.70it/s, v_num=2]
Epoch 9: 66%|██████▌ | 46/70 [00:00<00:00, 542.33it/s, v_num=2]
Epoch 9: 67%|██████▋ | 47/70 [00:00<00:00, 544.11it/s, v_num=2]
Epoch 9: 67%|██████▋ | 47/70 [00:00<00:00, 542.77it/s, v_num=2]
Epoch 9: 69%|██████▊ | 48/70 [00:00<00:00, 543.06it/s, v_num=2]
Epoch 9: 69%|██████▊ | 48/70 [00:00<00:00, 541.74it/s, v_num=2]
Epoch 9: 70%|███████ | 49/70 [00:00<00:00, 543.53it/s, v_num=2]
Epoch 9: 70%|███████ | 49/70 [00:00<00:00, 542.25it/s, v_num=2]
Epoch 9: 71%|███████▏ | 50/70 [00:00<00:00, 543.65it/s, v_num=2]
Epoch 9: 71%|███████▏ | 50/70 [00:00<00:00, 542.39it/s, v_num=2]
Epoch 9: 73%|███████▎ | 51/70 [00:00<00:00, 544.06it/s, v_num=2]
Epoch 9: 73%|███████▎ | 51/70 [00:00<00:00, 542.50it/s, v_num=2]
Epoch 9: 74%|███████▍ | 52/70 [00:00<00:00, 543.00it/s, v_num=2]
Epoch 9: 74%|███████▍ | 52/70 [00:00<00:00, 541.78it/s, v_num=2]
Epoch 9: 76%|███████▌ | 53/70 [00:00<00:00, 543.38it/s, v_num=2]
Epoch 9: 76%|███████▌ | 53/70 [00:00<00:00, 542.18it/s, v_num=2]
Epoch 9: 77%|███████▋ | 54/70 [00:00<00:00, 543.54it/s, v_num=2]
Epoch 9: 77%|███████▋ | 54/70 [00:00<00:00, 542.37it/s, v_num=2]
Epoch 9: 79%|███████▊ | 55/70 [00:00<00:00, 543.90it/s, v_num=2]
Epoch 9: 79%|███████▊ | 55/70 [00:00<00:00, 542.75it/s, v_num=2]
Epoch 9: 80%|████████ | 56/70 [00:00<00:00, 544.23it/s, v_num=2]
Epoch 9: 80%|████████ | 56/70 [00:00<00:00, 543.11it/s, v_num=2]
Epoch 9: 81%|████████▏ | 57/70 [00:00<00:00, 544.32it/s, v_num=2]
Epoch 9: 81%|████████▏ | 57/70 [00:00<00:00, 543.22it/s, v_num=2]
Epoch 9: 83%|████████▎ | 58/70 [00:00<00:00, 543.86it/s, v_num=2]
Epoch 9: 83%|████████▎ | 58/70 [00:00<00:00, 542.77it/s, v_num=2]
Epoch 9: 84%|████████▍ | 59/70 [00:00<00:00, 544.04it/s, v_num=2]
Epoch 9: 84%|████████▍ | 59/70 [00:00<00:00, 542.97it/s, v_num=2]
Epoch 9: 86%|████████▌ | 60/70 [00:00<00:00, 544.39it/s, v_num=2]
Epoch 9: 86%|████████▌ | 60/70 [00:00<00:00, 543.35it/s, v_num=2]
Epoch 9: 87%|████████▋ | 61/70 [00:00<00:00, 544.55it/s, v_num=2]
Epoch 9: 87%|████████▋ | 61/70 [00:00<00:00, 543.51it/s, v_num=2]
Epoch 9: 89%|████████▊ | 62/70 [00:00<00:00, 544.89it/s, v_num=2]
Epoch 9: 89%|████████▊ | 62/70 [00:00<00:00, 543.88it/s, v_num=2]
Epoch 9: 90%|█████████ | 63/70 [00:00<00:00, 545.01it/s, v_num=2]
Epoch 9: 90%|█████████ | 63/70 [00:00<00:00, 544.00it/s, v_num=2]
Epoch 9: 91%|█████████▏| 64/70 [00:00<00:00, 545.27it/s, v_num=2]
Epoch 9: 91%|█████████▏| 64/70 [00:00<00:00, 544.28it/s, v_num=2]
Epoch 9: 93%|█████████▎| 65/70 [00:00<00:00, 544.05it/s, v_num=2]
Epoch 9: 93%|█████████▎| 65/70 [00:00<00:00, 543.07it/s, v_num=2]
Epoch 9: 94%|█████████▍| 66/70 [00:00<00:00, 544.23it/s, v_num=2]
Epoch 9: 94%|█████████▍| 66/70 [00:00<00:00, 543.28it/s, v_num=2]
Epoch 9: 96%|█████████▌| 67/70 [00:00<00:00, 544.52it/s, v_num=2]
Epoch 9: 96%|█████████▌| 67/70 [00:00<00:00, 543.39it/s, v_num=2]
Epoch 9: 97%|█████████▋| 68/70 [00:00<00:00, 544.57it/s, v_num=2]
Epoch 9: 97%|█████████▋| 68/70 [00:00<00:00, 543.64it/s, v_num=2]
Epoch 9: 99%|█████████▊| 69/70 [00:00<00:00, 544.86it/s, v_num=2]
Epoch 9: 99%|█████████▊| 69/70 [00:00<00:00, 543.94it/s, v_num=2]
Epoch 9: 100%|██████████| 70/70 [00:00<00:00, 544.94it/s, v_num=2]
Epoch 9: 100%|██████████| 70/70 [00:00<00:00, 544.05it/s, v_num=2]
Epoch 9: 100%|██████████| 70/70 [00:00<00:00, 535.85it/s, v_num=2]
Epoch 9: 100%|██████████| 70/70 [00:00<00:00, 528.24it/s, v_num=2]
Custom Lasso Test MSE: 0.0188
scikit-learn Lasso Test MSE: 0.0836
import matplotlib.pyplot as plt
import numpy as np
import pytorch_lightning as pl
import torch
import torch.nn as nn
import torch.optim as optim
from pytorch_lightning import seed_everything
from sklearn.linear_model import Lasso
from sklearn.model_selection import train_test_split
from torch.utils.data import DataLoader, TensorDataset
from proxtorch.operators import L1
seed_everything(42)
class LassoRegression(pl.LightningModule):
def __init__(self, input_size, lasso_param):
super(LassoRegression, self).__init__()
self.input_size = input_size
self.lasso_param = lasso_param
self.linear = nn.Linear(input_size, 1)
self.l1_prox = L1(alpha=lasso_param)
def forward(self, x):
return self.linear(x)
def proximal_step(self, w, parameter):
return self.l1_prox.prox(w, parameter)
def training_step(self, batch, batch_idx):
x, y = batch
y_pred = self(x).squeeze()
loss = nn.MSELoss()(y_pred, y)
self.log("train_loss", loss)
return loss
def on_train_batch_end(self, outputs, batch, batch_idx) -> None:
with torch.no_grad():
for param in self.parameters():
if param.requires_grad:
param.data = self.proximal_step(
param.data, self.trainer.optimizers[0].param_groups[0]["lr"]
)
def configure_optimizers(self):
optimizer = optim.SGD(self.parameters(), lr=0.01)
return optimizer
# Generate synthetic data
np.random.seed(42)
n_samples, n_features = 100, 10
X = np.random.randn(n_samples, n_features)
w_true = np.random.randn(n_features)
# make 50% of the weights zero
w_true[np.random.choice(range(n_features), int(n_features * 0.5), replace=False)] = 0
y = X.dot(w_true) + np.random.normal(0, 0.1, n_samples)
# Split the data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42
)
# Convert to PyTorch tensors
x_train_tensor = torch.tensor(X_train, dtype=torch.float32)
y_train_tensor = torch.tensor(y_train, dtype=torch.float32)
x_test_tensor = torch.tensor(X_test, dtype=torch.float32)
y_test_tensor = torch.tensor(y_test, dtype=torch.float32)
# Train the custom Lasso regression model
lasso_param = 0.1
lasso_model = LassoRegression(input_size=n_features, lasso_param=lasso_param)
trainer = pl.Trainer(max_epochs=10)
trainer.fit(lasso_model, DataLoader(TensorDataset(x_train_tensor, y_train_tensor)))
# Train scikit-learn's Lasso regression model
sklearn_lasso = Lasso(alpha=lasso_param)
sklearn_lasso.fit(X_train, y_train)
# Compare the models' performance using MSE on the test set
with torch.no_grad():
y_pred_custom = lasso_model(x_test_tensor).numpy().flatten()
mse_custom = np.mean((y_pred_custom - y_test) ** 2)
y_pred_sklearn = sklearn_lasso.predict(X_test)
mse_sklearn = np.mean((y_pred_sklearn - y_test) ** 2)
print(f"Custom Lasso Test MSE: {mse_custom:.4f}")
print(f"scikit-learn Lasso Test MSE: {mse_sklearn:.4f}")
# Create a plot comparing predictions
plt.figure()
plt.scatter(y_test, y_pred_custom, label="Custom Lasso")
plt.scatter(y_test, y_pred_sklearn, label="scikit-learn Lasso")
plt.plot(y_test, y_test, color="black", linestyle="--", label="Ground Truth")
plt.xlabel("True Values")
plt.ylabel("Predicted Values")
plt.title("Lasso Regression Comparison on Test Data")
plt.legend()
plt.show()
# Compare weights
w_custom = lasso_model.linear.weight.data.numpy().flatten()
w_sklearn = sklearn_lasso.coef_
plt.figure(figsize=(15, 5))
x = np.arange(n_features)
plt.bar(x - 0.2, w_true, 0.2, label="True Weights", align="center", alpha=0.8)
plt.bar(x, w_custom, 0.2, label="Custom Lasso Weights", align="center", alpha=0.8)
plt.bar(
x + 0.2, w_sklearn, 0.2, label="sklearn Lasso Weights", align="center", alpha=0.8
)
plt.xlabel("Features")
plt.ylabel("Weights")
plt.title("Weight Comparison")
plt.legend()
plt.show()
Total running time of the script: (0 minutes 1.730 seconds)