How to Use ENAS/ProxylessNAS in Ten Minutes

What is the Key Idea of ENAS and ProxylessNAS?

Traditional reinforcement learning-based neural architecture search learns an architecture controller by iteratively sampling the architecture and training the model to get final reward to update the controller. It is extremely expensive process due to training CNN.

https://raw.githubusercontent.com/zhanghang1989/AutoGluonWebdata/master/docs/tutorial/proxyless.png

Fig. 1 ProxylessNAS

Recent work of ENAS and ProxylessNAS construct an over-parameterized network (supernet) and share the weights across different architecture to speed up the search speed. The reward is calculated every few iterations instead of every training period.

Import MXNet and AutoGluon:

import autogluon as ag
import mxnet as mx
import mxnet.gluon.nn as nn

How to Construct a SuperNet

Basic NN blocks for CNN.

class Identity(mx.gluon.HybridBlock):
    def hybrid_forward(self, F, x):
        return x

class ConvBNReLU(mx.gluon.HybridBlock):
    def __init__(self, in_channels, channels, kernel, stride):
        super().__init__()
        padding = (kernel - 1) // 2
        self.conv = nn.Conv2D(channels, kernel, stride, padding, in_channels=in_channels)
        self.bn = nn.BatchNorm(in_channels=channels)
        self.relu = nn.Activation('relu')
    def hybrid_forward(self, F, x):
        return self.relu(self.bn(self.conv(x)))

AutoGluon ENAS Unit

from autogluon.contrib.enas import *

@enas_unit()
class ResUnit(mx.gluon.HybridBlock):
    def __init__(self, in_channels, channels, hidden_channels, kernel, stride):
        super().__init__()
        self.conv1 = ConvBNReLU(in_channels, hidden_channels, kernel, stride)
        self.conv2 = ConvBNReLU(hidden_channels, channels, kernel, 1)
        if in_channels == channels and stride == 1:
            self.shortcut = Identity()
        else:
            self.shortcut = nn.Conv2D(channels, 1, stride, in_channels=in_channels)
    def hybrid_forward(self, F, x):
        return self.conv2(self.conv1(x)) + self.shortcut(x)

AutoGluon Sequntial

Creating a ENAS network using Sequential Block:

mynet = ENAS_Sequential(
    ResUnit(1, 8, hidden_channels=ag.space.Categorical(4, 8), kernel=ag.space.Categorical(3, 5), stride=2),
    ResUnit(8, 8, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=2),
    ResUnit(8, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=2),
    ResUnit(16, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=1, with_zero=True),
    ResUnit(16, 16, hidden_channels=8, kernel=ag.space.Categorical(3, 5), stride=1, with_zero=True),
    nn.GlobalAvgPool2D(),
    nn.Flatten(),
    nn.Activation('relu'),
    nn.Dense(10, in_units=16),
)

mynet.initialize()

#mynet.graph

Evaluate Network Latency and Define Reward Function

x = mx.nd.random.uniform(shape=(1, 1, 28, 28))
y = mynet.evaluate_latency(x)

Show the latencies:

print('Average latency is {:.2f} ms, latency of the current architecture is {:.2f} ms'.format(mynet.avg_latency, mynet.latency))
Average latency is 2.30 ms, latency of the current architecture is 2.54 ms

We also provide number of params

mynet.nparams
8714

Define the reward function:

reward_fn = lambda metric, net: metric * ((net.avg_latency / net.latency) ** 0.1)

Start the Training

Construct experiment scheduler, which automatically creates an RL controller based on user-defined search space.

scheduler = ENAS_Scheduler(mynet, train_set='mnist',
                           reward_fn=reward_fn, batch_size=128, num_gpus=1,
                           warmup_epochs=0, epochs=1, controller_lr=3e-3,
                           plot_frequency=10, update_arch_frequency=5)
get_built_in_dataset mnist
get_built_in_dataset mnist

Start the training:

scheduler.run()
  0%|          | 0/1 [00:00<?, ?it/s]
  0%|          | 0/468 [00:00<?, ?it/s]
avg reward: 0.13:   0%|          | 0/468 [00:00<?, ?it/s]
avg reward: 0.13:   0%|          | 1/468 [00:00<02:38,  2.94it/s]
avg reward: 0.13:   0%|          | 1/468 [00:00<02:38,  2.94it/s]
avg reward: 0.13:   0%|          | 1/468 [00:00<02:38,  2.94it/s]
avg reward: 0.13:   0%|          | 1/468 [00:00<02:38,  2.94it/s]
avg reward: 0.13:   0%|          | 1/468 [00:00<02:38,  2.94it/s]
avg reward: 0.12:   0%|          | 1/468 [00:00<02:38,  2.94it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:56,  3.97it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:56,  3.97it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:56,  3.97it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:56,  3.97it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:56,  3.97it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:56,  3.97it/s]
avg reward: 0.12:   2%|▏         | 11/468 [00:00<01:26,  5.26it/s]
avg reward: 0.12:   2%|▏         | 11/468 [00:00<01:26,  5.26it/s]
avg reward: 0.12:   2%|▏         | 11/468 [00:00<01:26,  5.26it/s]
avg reward: 0.12:   2%|▏         | 11/468 [00:00<01:26,  5.26it/s]
avg reward: 0.12:   2%|▏         | 11/468 [00:00<01:26,  5.26it/s]
avg reward: 0.10:   2%|▏         | 11/468 [00:01<01:26,  5.26it/s]
avg reward: 0.10:   3%|▎         | 16/468 [00:01<01:06,  6.75it/s]
avg reward: 0.10:   3%|▎         | 16/468 [00:01<01:06,  6.75it/s]
avg reward: 0.10:   3%|▎         | 16/468 [00:01<01:06,  6.75it/s]
avg reward: 0.10:   3%|▎         | 16/468 [00:01<01:06,  6.75it/s]
avg reward: 0.10:   3%|▎         | 16/468 [00:01<01:06,  6.75it/s]
avg reward: 0.11:   3%|▎         | 16/468 [00:01<01:06,  6.75it/s]
avg reward: 0.11:   4%|▍         | 21/468 [00:01<00:52,  8.48it/s]
avg reward: 0.11:   4%|▍         | 21/468 [00:01<00:52,  8.48it/s]
avg reward: 0.11:   4%|▍         | 21/468 [00:01<00:52,  8.48it/s]
avg reward: 0.11:   4%|▍         | 21/468 [00:01<00:52,  8.48it/s]
avg reward: 0.11:   4%|▍         | 21/468 [00:01<00:52,  8.48it/s]
avg reward: 0.12:   4%|▍         | 21/468 [00:01<00:52,  8.48it/s]
avg reward: 0.12:   6%|▌         | 26/468 [00:01<00:42, 10.39it/s]
avg reward: 0.12:   6%|▌         | 26/468 [00:01<00:42, 10.39it/s]
avg reward: 0.12:   6%|▌         | 26/468 [00:01<00:42, 10.39it/s]
avg reward: 0.12:   6%|▌         | 26/468 [00:01<00:42, 10.39it/s]
avg reward: 0.12:   6%|▌         | 26/468 [00:01<00:42, 10.39it/s]
avg reward: 0.12:   6%|▌         | 26/468 [00:01<00:42, 10.39it/s]
avg reward: 0.12:   7%|▋         | 31/468 [00:01<00:35, 12.22it/s]
avg reward: 0.12:   7%|▋         | 31/468 [00:01<00:35, 12.22it/s]
avg reward: 0.12:   7%|▋         | 31/468 [00:01<00:35, 12.22it/s]
avg reward: 0.12:   7%|▋         | 31/468 [00:01<00:35, 12.22it/s]
avg reward: 0.12:   7%|▋         | 31/468 [00:01<00:35, 12.22it/s]
avg reward: 0.11:   7%|▋         | 31/468 [00:02<00:35, 12.22it/s]
avg reward: 0.11:   8%|▊         | 36/468 [00:02<00:31, 13.92it/s]
avg reward: 0.11:   8%|▊         | 36/468 [00:02<00:31, 13.92it/s]
avg reward: 0.11:   8%|▊         | 36/468 [00:02<00:31, 13.92it/s]
avg reward: 0.11:   8%|▊         | 36/468 [00:02<00:31, 13.92it/s]
avg reward: 0.11:   8%|▊         | 36/468 [00:02<00:31, 13.92it/s]
avg reward: 0.11:   8%|▊         | 36/468 [00:02<00:31, 13.92it/s]
avg reward: 0.11:   9%|▉         | 41/468 [00:02<00:27, 15.65it/s]
avg reward: 0.11:   9%|▉         | 41/468 [00:02<00:27, 15.65it/s]
avg reward: 0.11:   9%|▉         | 41/468 [00:02<00:27, 15.65it/s]
avg reward: 0.11:   9%|▉         | 41/468 [00:02<00:27, 15.65it/s]
avg reward: 0.11:   9%|▉         | 41/468 [00:02<00:27, 15.65it/s]
avg reward: 0.13:   9%|▉         | 41/468 [00:02<00:27, 15.65it/s]
avg reward: 0.13:  10%|▉         | 46/468 [00:02<00:25, 16.86it/s]
avg reward: 0.13:  10%|▉         | 46/468 [00:02<00:25, 16.86it/s]
avg reward: 0.13:  10%|▉         | 46/468 [00:02<00:25, 16.86it/s]
avg reward: 0.13:  10%|▉         | 46/468 [00:02<00:25, 16.86it/s]
avg reward: 0.13:  10%|▉         | 46/468 [00:02<00:25, 16.86it/s]
avg reward: 0.17:  10%|▉         | 46/468 [00:02<00:25, 16.86it/s]
avg reward: 0.17:  11%|█         | 51/468 [00:02<00:23, 17.71it/s]
avg reward: 0.17:  11%|█         | 51/468 [00:02<00:23, 17.71it/s]
avg reward: 0.17:  11%|█         | 51/468 [00:02<00:23, 17.71it/s]
avg reward: 0.17:  11%|█         | 51/468 [00:02<00:23, 17.71it/s]
avg reward: 0.17:  11%|█         | 51/468 [00:02<00:23, 17.71it/s]
avg reward: 0.22:  11%|█         | 51/468 [00:02<00:23, 17.71it/s]
avg reward: 0.22:  12%|█▏        | 56/468 [00:02<00:21, 18.89it/s]
avg reward: 0.22:  12%|█▏        | 56/468 [00:02<00:21, 18.89it/s]
avg reward: 0.22:  12%|█▏        | 56/468 [00:02<00:21, 18.89it/s]
avg reward: 0.22:  12%|█▏        | 56/468 [00:03<00:21, 18.89it/s]
avg reward: 0.22:  12%|█▏        | 56/468 [00:03<00:21, 18.89it/s]
avg reward: 0.27:  12%|█▏        | 56/468 [00:03<00:21, 18.89it/s]
avg reward: 0.27:  13%|█▎        | 61/468 [00:03<00:20, 19.67it/s]
avg reward: 0.27:  13%|█▎        | 61/468 [00:03<00:20, 19.67it/s]
avg reward: 0.27:  13%|█▎        | 61/468 [00:03<00:20, 19.67it/s]
avg reward: 0.27:  13%|█▎        | 61/468 [00:03<00:20, 19.67it/s]
avg reward: 0.27:  13%|█▎        | 61/468 [00:03<00:20, 19.67it/s]
avg reward: 0.32:  13%|█▎        | 61/468 [00:03<00:20, 19.67it/s]
avg reward: 0.32:  14%|█▍        | 66/468 [00:03<00:19, 20.32it/s]
avg reward: 0.32:  14%|█▍        | 66/468 [00:03<00:19, 20.32it/s]
avg reward: 0.32:  14%|█▍        | 66/468 [00:03<00:19, 20.32it/s]
avg reward: 0.32:  14%|█▍        | 66/468 [00:03<00:19, 20.32it/s]
avg reward: 0.32:  14%|█▍        | 66/468 [00:03<00:19, 20.32it/s]
avg reward: 0.35:  14%|█▍        | 66/468 [00:03<00:19, 20.32it/s]
avg reward: 0.35:  15%|█▌        | 71/468 [00:03<00:19, 20.29it/s]
avg reward: 0.35:  15%|█▌        | 71/468 [00:03<00:19, 20.29it/s]
avg reward: 0.35:  15%|█▌        | 71/468 [00:03<00:19, 20.29it/s]
avg reward: 0.35:  15%|█▌        | 71/468 [00:03<00:19, 20.29it/s]
avg reward: 0.35:  15%|█▌        | 71/468 [00:03<00:19, 20.29it/s]
avg reward: 0.38:  15%|█▌        | 71/468 [00:03<00:19, 20.29it/s]
avg reward: 0.38:  16%|█▌        | 76/468 [00:03<00:19, 20.52it/s]
avg reward: 0.38:  16%|█▌        | 76/468 [00:03<00:19, 20.52it/s]
avg reward: 0.38:  16%|█▌        | 76/468 [00:03<00:19, 20.52it/s]
avg reward: 0.38:  16%|█▌        | 76/468 [00:03<00:19, 20.52it/s]
avg reward: 0.38:  16%|█▌        | 76/468 [00:03<00:19, 20.52it/s]
avg reward: 0.41:  16%|█▌        | 76/468 [00:04<00:19, 20.52it/s]
avg reward: 0.41:  17%|█▋        | 81/468 [00:04<00:18, 20.63it/s]
avg reward: 0.41:  17%|█▋        | 81/468 [00:04<00:18, 20.63it/s]
avg reward: 0.41:  17%|█▋        | 81/468 [00:04<00:18, 20.63it/s]
avg reward: 0.41:  17%|█▋        | 81/468 [00:04<00:18, 20.63it/s]
avg reward: 0.41:  17%|█▋        | 81/468 [00:04<00:18, 20.63it/s]
avg reward: 0.41:  17%|█▋        | 81/468 [00:04<00:18, 20.63it/s]
avg reward: 0.41:  18%|█▊        | 86/468 [00:04<00:18, 20.59it/s]
avg reward: 0.41:  18%|█▊        | 86/468 [00:04<00:18, 20.59it/s]
avg reward: 0.41:  18%|█▊        | 86/468 [00:04<00:18, 20.59it/s]
avg reward: 0.41:  18%|█▊        | 86/468 [00:04<00:18, 20.59it/s]
avg reward: 0.41:  18%|█▊        | 86/468 [00:04<00:18, 20.59it/s]
avg reward: 0.44:  18%|█▊        | 86/468 [00:04<00:18, 20.59it/s]
avg reward: 0.44:  19%|█▉        | 91/468 [00:04<00:18, 20.81it/s]
avg reward: 0.44:  19%|█▉        | 91/468 [00:04<00:18, 20.81it/s]
avg reward: 0.44:  19%|█▉        | 91/468 [00:04<00:18, 20.81it/s]
avg reward: 0.44:  19%|█▉        | 91/468 [00:04<00:18, 20.81it/s]
avg reward: 0.44:  19%|█▉        | 91/468 [00:04<00:18, 20.81it/s]
avg reward: 0.46:  19%|█▉        | 91/468 [00:04<00:18, 20.81it/s]
avg reward: 0.46:  21%|██        | 96/468 [00:04<00:17, 20.68it/s]
avg reward: 0.46:  21%|██        | 96/468 [00:04<00:17, 20.68it/s]
avg reward: 0.46:  21%|██        | 96/468 [00:04<00:17, 20.68it/s]
avg reward: 0.46:  21%|██        | 96/468 [00:04<00:17, 20.68it/s]
avg reward: 0.46:  21%|██        | 96/468 [00:04<00:17, 20.68it/s]
avg reward: 0.49:  21%|██        | 96/468 [00:05<00:17, 20.68it/s]
avg reward: 0.49:  22%|██▏       | 101/468 [00:05<00:18, 19.89it/s]
avg reward: 0.49:  22%|██▏       | 101/468 [00:05<00:18, 19.89it/s]
avg reward: 0.49:  22%|██▏       | 101/468 [00:05<00:18, 19.89it/s]
avg reward: 0.49:  22%|██▏       | 101/468 [00:05<00:18, 19.89it/s]
avg reward: 0.49:  22%|██▏       | 101/468 [00:05<00:18, 19.89it/s]
avg reward: 0.53:  22%|██▏       | 101/468 [00:05<00:18, 19.89it/s]
avg reward: 0.53:  23%|██▎       | 106/468 [00:05<00:18, 20.04it/s]
avg reward: 0.53:  23%|██▎       | 106/468 [00:05<00:18, 20.04it/s]
avg reward: 0.53:  23%|██▎       | 106/468 [00:05<00:18, 20.04it/s]
avg reward: 0.53:  23%|██▎       | 106/468 [00:05<00:18, 20.04it/s]
avg reward: 0.53:  23%|██▎       | 106/468 [00:05<00:18, 20.04it/s]
avg reward: 0.55:  23%|██▎       | 106/468 [00:05<00:18, 20.04it/s]
avg reward: 0.55:  24%|██▎       | 111/468 [00:05<00:17, 20.08it/s]
avg reward: 0.55:  24%|██▎       | 111/468 [00:05<00:17, 20.08it/s]
avg reward: 0.55:  24%|██▎       | 111/468 [00:05<00:17, 20.08it/s]
avg reward: 0.55:  24%|██▎       | 111/468 [00:05<00:17, 20.08it/s]
avg reward: 0.55:  24%|██▎       | 111/468 [00:05<00:17, 20.08it/s]
avg reward: 0.55:  25%|██▍       | 115/468 [00:05<00:15, 23.49it/s]
avg reward: 0.57:  25%|██▍       | 115/468 [00:05<00:15, 23.49it/s]
avg reward: 0.57:  25%|██▍       | 115/468 [00:05<00:15, 23.49it/s]
avg reward: 0.57:  25%|██▍       | 115/468 [00:05<00:15, 23.49it/s]
avg reward: 0.57:  25%|██▌       | 118/468 [00:05<00:17, 19.87it/s]
avg reward: 0.57:  25%|██▌       | 118/468 [00:05<00:17, 19.87it/s]
avg reward: 0.57:  25%|██▌       | 118/468 [00:05<00:17, 19.87it/s]
avg reward: 0.59:  25%|██▌       | 118/468 [00:06<00:17, 19.87it/s]
avg reward: 0.59:  26%|██▌       | 121/468 [00:06<00:19, 17.71it/s]
avg reward: 0.59:  26%|██▌       | 121/468 [00:06<00:19, 17.71it/s]
avg reward: 0.59:  26%|██▌       | 121/468 [00:06<00:19, 17.71it/s]
avg reward: 0.59:  26%|██▌       | 121/468 [00:06<00:19, 17.71it/s]
avg reward: 0.59:  26%|██▌       | 121/468 [00:06<00:19, 17.71it/s]
avg reward: 0.61:  26%|██▌       | 121/468 [00:06<00:19, 17.71it/s]
avg reward: 0.61:  27%|██▋       | 126/468 [00:06<00:18, 18.38it/s]
avg reward: 0.61:  27%|██▋       | 126/468 [00:06<00:18, 18.38it/s]
avg reward: 0.61:  27%|██▋       | 126/468 [00:06<00:18, 18.38it/s]
avg reward: 0.61:  27%|██▋       | 126/468 [00:06<00:18, 18.38it/s]
avg reward: 0.61:  27%|██▋       | 126/468 [00:06<00:18, 18.38it/s]
avg reward: 0.64:  27%|██▋       | 126/468 [00:06<00:18, 18.38it/s]
avg reward: 0.64:  28%|██▊       | 131/468 [00:06<00:17, 18.95it/s]
avg reward: 0.64:  28%|██▊       | 131/468 [00:06<00:17, 18.95it/s]
avg reward: 0.64:  28%|██▊       | 131/468 [00:06<00:17, 18.95it/s]
avg reward: 0.64:  28%|██▊       | 131/468 [00:06<00:17, 18.95it/s]
avg reward: 0.64:  28%|██▊       | 131/468 [00:06<00:17, 18.95it/s]
avg reward: 0.64:  29%|██▉       | 135/468 [00:06<00:14, 22.48it/s]
avg reward: 0.68:  29%|██▉       | 135/468 [00:06<00:14, 22.48it/s]
avg reward: 0.68:  29%|██▉       | 135/468 [00:06<00:14, 22.48it/s]
avg reward: 0.68:  29%|██▉       | 135/468 [00:06<00:14, 22.48it/s]
avg reward: 0.68:  29%|██▉       | 138/468 [00:06<00:17, 19.19it/s]
avg reward: 0.68:  29%|██▉       | 138/468 [00:06<00:17, 19.19it/s]
avg reward: 0.68:  29%|██▉       | 138/468 [00:06<00:17, 19.19it/s]
avg reward: 0.71:  29%|██▉       | 138/468 [00:07<00:17, 19.19it/s]
avg reward: 0.71:  30%|███       | 141/468 [00:07<00:18, 17.77it/s]
avg reward: 0.71:  30%|███       | 141/468 [00:07<00:18, 17.77it/s]
avg reward: 0.71:  30%|███       | 141/468 [00:07<00:18, 17.77it/s]
avg reward: 0.71:  30%|███       | 141/468 [00:07<00:18, 17.77it/s]
avg reward: 0.71:  30%|███       | 141/468 [00:07<00:18, 17.77it/s]
avg reward: 0.76:  30%|███       | 141/468 [00:07<00:18, 17.77it/s]
avg reward: 0.76:  31%|███       | 146/468 [00:07<00:17, 18.60it/s]
avg reward: 0.76:  31%|███       | 146/468 [00:07<00:17, 18.60it/s]
avg reward: 0.76:  31%|███       | 146/468 [00:07<00:17, 18.60it/s]
avg reward: 0.76:  31%|███       | 146/468 [00:07<00:17, 18.60it/s]
avg reward: 0.76:  31%|███       | 146/468 [00:07<00:17, 18.60it/s]
avg reward: 0.79:  31%|███       | 146/468 [00:07<00:17, 18.60it/s]
avg reward: 0.79:  32%|███▏      | 151/468 [00:07<00:16, 18.94it/s]
avg reward: 0.79:  32%|███▏      | 151/468 [00:07<00:16, 18.94it/s]
avg reward: 0.79:  32%|███▏      | 151/468 [00:07<00:16, 18.94it/s]
avg reward: 0.79:  32%|███▏      | 151/468 [00:07<00:16, 18.94it/s]
avg reward: 0.79:  32%|███▏      | 151/468 [00:07<00:16, 18.94it/s]
avg reward: 0.82:  32%|███▏      | 151/468 [00:07<00:16, 18.94it/s]
avg reward: 0.82:  33%|███▎      | 156/468 [00:07<00:16, 19.49it/s]
avg reward: 0.82:  33%|███▎      | 156/468 [00:07<00:16, 19.49it/s]
avg reward: 0.82:  33%|███▎      | 156/468 [00:07<00:16, 19.49it/s]
avg reward: 0.82:  33%|███▎      | 156/468 [00:07<00:16, 19.49it/s]
avg reward: 0.82:  33%|███▎      | 156/468 [00:07<00:16, 19.49it/s]
avg reward: 0.84:  33%|███▎      | 156/468 [00:08<00:16, 19.49it/s]
avg reward: 0.84:  34%|███▍      | 161/468 [00:08<00:15, 19.75it/s]
avg reward: 0.84:  34%|███▍      | 161/468 [00:08<00:15, 19.75it/s]
avg reward: 0.84:  34%|███▍      | 161/468 [00:08<00:15, 19.75it/s]
avg reward: 0.84:  34%|███▍      | 161/468 [00:08<00:15, 19.75it/s]
avg reward: 0.84:  34%|███▍      | 161/468 [00:08<00:15, 19.75it/s]
avg reward: 0.86:  34%|███▍      | 161/468 [00:08<00:15, 19.75it/s]
avg reward: 0.86:  35%|███▌      | 166/468 [00:08<00:15, 19.91it/s]
avg reward: 0.86:  35%|███▌      | 166/468 [00:08<00:15, 19.91it/s]
avg reward: 0.86:  35%|███▌      | 166/468 [00:08<00:15, 19.91it/s]
avg reward: 0.86:  35%|███▌      | 166/468 [00:08<00:15, 19.91it/s]
avg reward: 0.86:  35%|███▌      | 166/468 [00:08<00:15, 19.91it/s]
avg reward: 0.88:  35%|███▌      | 166/468 [00:08<00:15, 19.91it/s]
avg reward: 0.88:  37%|███▋      | 171/468 [00:08<00:15, 19.76it/s]
avg reward: 0.88:  37%|███▋      | 171/468 [00:08<00:15, 19.76it/s]
avg reward: 0.88:  37%|███▋      | 171/468 [00:08<00:15, 19.76it/s]
avg reward: 0.88:  37%|███▋      | 171/468 [00:08<00:15, 19.76it/s]
avg reward: 0.88:  37%|███▋      | 171/468 [00:08<00:15, 19.76it/s]
avg reward: 0.89:  37%|███▋      | 171/468 [00:08<00:15, 19.76it/s]
avg reward: 0.89:  38%|███▊      | 176/468 [00:08<00:15, 19.42it/s]
avg reward: 0.89:  38%|███▊      | 176/468 [00:08<00:15, 19.42it/s]
avg reward: 0.89:  38%|███▊      | 176/468 [00:08<00:15, 19.42it/s]
avg reward: 0.89:  38%|███▊      | 176/468 [00:08<00:15, 19.42it/s]
avg reward: 0.89:  38%|███▊      | 176/468 [00:08<00:15, 19.42it/s]
avg reward: 0.89:  38%|███▊      | 180/468 [00:08<00:12, 22.78it/s]
avg reward: 0.90:  38%|███▊      | 180/468 [00:09<00:12, 22.78it/s]
avg reward: 0.90:  38%|███▊      | 180/468 [00:09<00:12, 22.78it/s]
avg reward: 0.90:  38%|███▊      | 180/468 [00:09<00:12, 22.78it/s]
avg reward: 0.90:  39%|███▉      | 183/468 [00:09<00:14, 19.35it/s]
avg reward: 0.90:  39%|███▉      | 183/468 [00:09<00:14, 19.35it/s]
avg reward: 0.90:  39%|███▉      | 183/468 [00:09<00:14, 19.35it/s]
avg reward: 0.90:  39%|███▉      | 183/468 [00:09<00:14, 19.35it/s]
avg reward: 0.90:  40%|███▉      | 186/468 [00:09<00:15, 18.01it/s]
avg reward: 0.90:  40%|███▉      | 186/468 [00:09<00:15, 18.01it/s]
avg reward: 0.90:  40%|███▉      | 186/468 [00:09<00:15, 18.01it/s]
avg reward: 0.90:  40%|███▉      | 186/468 [00:09<00:15, 18.01it/s]
avg reward: 0.90:  40%|███▉      | 186/468 [00:09<00:15, 18.01it/s]
avg reward: 0.90:  41%|████      | 190/468 [00:09<00:13, 21.38it/s]
avg reward: 0.92:  41%|████      | 190/468 [00:09<00:13, 21.38it/s]
avg reward: 0.92:  41%|████      | 190/468 [00:09<00:13, 21.38it/s]
avg reward: 0.92:  41%|████      | 190/468 [00:09<00:13, 21.38it/s]
avg reward: 0.92:  41%|████      | 193/468 [00:09<00:15, 18.13it/s]
avg reward: 0.92:  41%|████      | 193/468 [00:09<00:15, 18.13it/s]
avg reward: 0.92:  41%|████      | 193/468 [00:09<00:15, 18.13it/s]
avg reward: 0.93:  41%|████      | 193/468 [00:09<00:15, 18.13it/s]
avg reward: 0.93:  42%|████▏     | 196/468 [00:09<00:15, 17.44it/s]
avg reward: 0.93:  42%|████▏     | 196/468 [00:09<00:15, 17.44it/s]
avg reward: 0.93:  42%|████▏     | 196/468 [00:09<00:15, 17.44it/s]
avg reward: 0.93:  42%|████▏     | 196/468 [00:09<00:15, 17.44it/s]
avg reward: 0.93:  42%|████▏     | 196/468 [00:09<00:15, 17.44it/s]
avg reward: 0.93:  42%|████▏     | 196/468 [00:10<00:15, 17.44it/s]
avg reward: 0.93:  43%|████▎     | 201/468 [00:10<00:14, 18.30it/s]
avg reward: 0.93:  43%|████▎     | 201/468 [00:10<00:14, 18.30it/s]
avg reward: 0.93:  43%|████▎     | 201/468 [00:10<00:14, 18.30it/s]
avg reward: 0.93:  43%|████▎     | 201/468 [00:10<00:14, 18.30it/s]
avg reward: 0.93:  43%|████▎     | 201/468 [00:10<00:14, 18.30it/s]
avg reward: 0.93:  43%|████▎     | 201/468 [00:10<00:14, 18.30it/s]
avg reward: 0.93:  44%|████▍     | 206/468 [00:10<00:14, 18.21it/s]
avg reward: 0.93:  44%|████▍     | 206/468 [00:10<00:14, 18.21it/s]
avg reward: 0.93:  44%|████▍     | 206/468 [00:10<00:14, 18.21it/s]
avg reward: 0.93:  44%|████▍     | 206/468 [00:10<00:14, 18.21it/s]
avg reward: 0.93:  44%|████▍     | 206/468 [00:10<00:14, 18.21it/s]
avg reward: 0.94:  44%|████▍     | 206/468 [00:10<00:14, 18.21it/s]
avg reward: 0.94:  45%|████▌     | 211/468 [00:10<00:13, 18.59it/s]
avg reward: 0.94:  45%|████▌     | 211/468 [00:10<00:13, 18.59it/s]
avg reward: 0.94:  45%|████▌     | 211/468 [00:10<00:13, 18.59it/s]
avg reward: 0.94:  45%|████▌     | 211/468 [00:10<00:13, 18.59it/s]
avg reward: 0.94:  45%|████▌     | 211/468 [00:10<00:13, 18.59it/s]
avg reward: 0.93:  45%|████▌     | 211/468 [00:10<00:13, 18.59it/s]
avg reward: 0.93:  46%|████▌     | 216/468 [00:10<00:13, 19.16it/s]
avg reward: 0.93:  46%|████▌     | 216/468 [00:10<00:13, 19.16it/s]
avg reward: 0.93:  46%|████▌     | 216/468 [00:10<00:13, 19.16it/s]
avg reward: 0.93:  46%|████▌     | 216/468 [00:10<00:13, 19.16it/s]
avg reward: 0.93:  46%|████▌     | 216/468 [00:11<00:13, 19.16it/s]
avg reward: 0.94:  46%|████▌     | 216/468 [00:11<00:13, 19.16it/s]
avg reward: 0.94:  47%|████▋     | 221/468 [00:11<00:13, 18.90it/s]
avg reward: 0.94:  47%|████▋     | 221/468 [00:11<00:13, 18.90it/s]
avg reward: 0.94:  47%|████▋     | 221/468 [00:11<00:13, 18.90it/s]
avg reward: 0.94:  47%|████▋     | 221/468 [00:11<00:13, 18.90it/s]
avg reward: 0.94:  47%|████▋     | 221/468 [00:11<00:13, 18.90it/s]
avg reward: 0.95:  47%|████▋     | 221/468 [00:11<00:13, 18.90it/s]
avg reward: 0.95:  48%|████▊     | 226/468 [00:11<00:12, 19.06it/s]
avg reward: 0.95:  48%|████▊     | 226/468 [00:11<00:12, 19.06it/s]
avg reward: 0.95:  48%|████▊     | 226/468 [00:11<00:12, 19.06it/s]
avg reward: 0.95:  48%|████▊     | 226/468 [00:11<00:12, 19.06it/s]
avg reward: 0.95:  48%|████▊     | 226/468 [00:11<00:12, 19.06it/s]
avg reward: 0.95:  48%|████▊     | 226/468 [00:11<00:12, 19.06it/s]
avg reward: 0.95:  49%|████▉     | 231/468 [00:11<00:12, 19.46it/s]
avg reward: 0.95:  49%|████▉     | 231/468 [00:11<00:12, 19.46it/s]
avg reward: 0.95:  49%|████▉     | 231/468 [00:11<00:12, 19.46it/s]
avg reward: 0.95:  49%|████▉     | 231/468 [00:11<00:12, 19.46it/s]
avg reward: 0.95:  49%|████▉     | 231/468 [00:11<00:12, 19.46it/s]
avg reward: 0.95:  49%|████▉     | 231/468 [00:11<00:12, 19.46it/s]
avg reward: 0.95:  50%|█████     | 236/468 [00:11<00:11, 19.57it/s]
avg reward: 0.95:  50%|█████     | 236/468 [00:11<00:11, 19.57it/s]
avg reward: 0.95:  50%|█████     | 236/468 [00:12<00:11, 19.57it/s]
avg reward: 0.95:  50%|█████     | 236/468 [00:12<00:11, 19.57it/s]
avg reward: 0.95:  50%|█████     | 236/468 [00:12<00:11, 19.57it/s]
avg reward: 0.95:  50%|█████     | 236/468 [00:12<00:11, 19.57it/s]
avg reward: 0.95:  51%|█████▏    | 241/468 [00:12<00:11, 20.01it/s]
avg reward: 0.95:  51%|█████▏    | 241/468 [00:12<00:11, 20.01it/s]
avg reward: 0.95:  51%|█████▏    | 241/468 [00:12<00:11, 20.01it/s]
avg reward: 0.95:  51%|█████▏    | 241/468 [00:12<00:11, 20.01it/s]
avg reward: 0.95:  51%|█████▏    | 241/468 [00:12<00:11, 20.01it/s]
avg reward: 0.95:  51%|█████▏    | 241/468 [00:12<00:11, 20.01it/s]
avg reward: 0.95:  53%|█████▎    | 246/468 [00:12<00:11, 20.13it/s]
avg reward: 0.95:  53%|█████▎    | 246/468 [00:12<00:11, 20.13it/s]
avg reward: 0.95:  53%|█████▎    | 246/468 [00:12<00:11, 20.13it/s]
avg reward: 0.95:  53%|█████▎    | 246/468 [00:12<00:11, 20.13it/s]
avg reward: 0.95:  53%|█████▎    | 246/468 [00:12<00:11, 20.13it/s]
avg reward: 0.95:  53%|█████▎    | 250/468 [00:12<00:09, 23.52it/s]
avg reward: 0.96:  53%|█████▎    | 250/468 [00:12<00:09, 23.52it/s]
avg reward: 0.96:  53%|█████▎    | 250/468 [00:12<00:09, 23.52it/s]
avg reward: 0.96:  53%|█████▎    | 250/468 [00:12<00:09, 23.52it/s]
avg reward: 0.96:  54%|█████▍    | 253/468 [00:12<00:10, 20.22it/s]
avg reward: 0.96:  54%|█████▍    | 253/468 [00:12<00:10, 20.22it/s]
avg reward: 0.96:  54%|█████▍    | 253/468 [00:12<00:10, 20.22it/s]
avg reward: 0.95:  54%|█████▍    | 253/468 [00:12<00:10, 20.22it/s]
avg reward: 0.95:  55%|█████▍    | 256/468 [00:12<00:11, 18.03it/s]
avg reward: 0.95:  55%|█████▍    | 256/468 [00:12<00:11, 18.03it/s]
avg reward: 0.95:  55%|█████▍    | 256/468 [00:12<00:11, 18.03it/s]
avg reward: 0.95:  55%|█████▍    | 256/468 [00:13<00:11, 18.03it/s]
avg reward: 0.95:  55%|█████▍    | 256/468 [00:13<00:11, 18.03it/s]
avg reward: 0.96:  55%|█████▍    | 256/468 [00:13<00:11, 18.03it/s]
avg reward: 0.96:  56%|█████▌    | 261/468 [00:13<00:11, 18.67it/s]
avg reward: 0.96:  56%|█████▌    | 261/468 [00:13<00:11, 18.67it/s]
avg reward: 0.96:  56%|█████▌    | 261/468 [00:13<00:11, 18.67it/s]
avg reward: 0.96:  56%|█████▌    | 261/468 [00:13<00:11, 18.67it/s]
avg reward: 0.96:  56%|█████▌    | 261/468 [00:13<00:11, 18.67it/s]
avg reward: 0.96:  57%|█████▋    | 265/468 [00:13<00:09, 22.00it/s]
avg reward: 0.95:  57%|█████▋    | 265/468 [00:13<00:09, 22.00it/s]
avg reward: 0.95:  57%|█████▋    | 265/468 [00:13<00:09, 22.00it/s]
avg reward: 0.95:  57%|█████▋    | 265/468 [00:13<00:09, 22.00it/s]
avg reward: 0.95:  57%|█████▋    | 268/468 [00:13<00:10, 19.10it/s]
avg reward: 0.95:  57%|█████▋    | 268/468 [00:13<00:10, 19.10it/s]
avg reward: 0.95:  57%|█████▋    | 268/468 [00:13<00:10, 19.10it/s]
avg reward: 0.95:  57%|█████▋    | 268/468 [00:13<00:10, 19.10it/s]
avg reward: 0.95:  58%|█████▊    | 271/468 [00:13<00:11, 17.38it/s]
avg reward: 0.95:  58%|█████▊    | 271/468 [00:13<00:11, 17.38it/s]
avg reward: 0.95:  58%|█████▊    | 271/468 [00:13<00:11, 17.38it/s]
avg reward: 0.95:  58%|█████▊    | 271/468 [00:13<00:11, 17.38it/s]
avg reward: 0.95:  58%|█████▊    | 271/468 [00:13<00:11, 17.38it/s]
avg reward: 0.96:  58%|█████▊    | 271/468 [00:13<00:11, 17.38it/s]
avg reward: 0.96:  59%|█████▉    | 276/468 [00:13<00:10, 18.21it/s]
avg reward: 0.96:  59%|█████▉    | 276/468 [00:13<00:10, 18.21it/s]
avg reward: 0.96:  59%|█████▉    | 276/468 [00:14<00:10, 18.21it/s]
avg reward: 0.96:  59%|█████▉    | 276/468 [00:14<00:10, 18.21it/s]
avg reward: 0.96:  59%|█████▉    | 276/468 [00:14<00:10, 18.21it/s]
avg reward: 0.96:  59%|█████▉    | 276/468 [00:14<00:10, 18.21it/s]
avg reward: 0.96:  60%|██████    | 281/468 [00:14<00:09, 19.05it/s]
avg reward: 0.96:  60%|██████    | 281/468 [00:14<00:09, 19.05it/s]
avg reward: 0.96:  60%|██████    | 281/468 [00:14<00:09, 19.05it/s]
avg reward: 0.96:  60%|██████    | 281/468 [00:14<00:09, 19.05it/s]
avg reward: 0.96:  60%|██████    | 281/468 [00:14<00:09, 19.05it/s]
avg reward: 0.96:  60%|██████    | 281/468 [00:14<00:09, 19.05it/s]
avg reward: 0.96:  61%|██████    | 286/468 [00:14<00:09, 19.66it/s]
avg reward: 0.96:  61%|██████    | 286/468 [00:14<00:09, 19.66it/s]
avg reward: 0.96:  61%|██████    | 286/468 [00:14<00:09, 19.66it/s]
avg reward: 0.96:  61%|██████    | 286/468 [00:14<00:09, 19.66it/s]
avg reward: 0.96:  61%|██████    | 286/468 [00:14<00:09, 19.66it/s]
avg reward: 0.96:  61%|██████    | 286/468 [00:14<00:09, 19.66it/s]
avg reward: 0.96:  62%|██████▏   | 291/468 [00:14<00:08, 20.21it/s]
avg reward: 0.96:  62%|██████▏   | 291/468 [00:14<00:08, 20.21it/s]
avg reward: 0.96:  62%|██████▏   | 291/468 [00:14<00:08, 20.21it/s]
avg reward: 0.96:  62%|██████▏   | 291/468 [00:14<00:08, 20.21it/s]
avg reward: 0.96:  62%|██████▏   | 291/468 [00:14<00:08, 20.21it/s]
avg reward: 0.96:  62%|██████▏   | 291/468 [00:14<00:08, 20.21it/s]
avg reward: 0.96:  63%|██████▎   | 296/468 [00:14<00:08, 20.26it/s]
avg reward: 0.96:  63%|██████▎   | 296/468 [00:14<00:08, 20.26it/s]
avg reward: 0.96:  63%|██████▎   | 296/468 [00:14<00:08, 20.26it/s]
avg reward: 0.96:  63%|██████▎   | 296/468 [00:14<00:08, 20.26it/s]
avg reward: 0.96:  63%|██████▎   | 296/468 [00:14<00:08, 20.26it/s]
avg reward: 0.96:  63%|██████▎   | 296/468 [00:15<00:08, 20.26it/s]
avg reward: 0.96:  64%|██████▍   | 301/468 [00:15<00:08, 20.27it/s]
avg reward: 0.96:  64%|██████▍   | 301/468 [00:15<00:08, 20.27it/s]
avg reward: 0.96:  64%|██████▍   | 301/468 [00:15<00:08, 20.27it/s]
avg reward: 0.96:  64%|██████▍   | 301/468 [00:15<00:08, 20.27it/s]
avg reward: 0.96:  64%|██████▍   | 301/468 [00:15<00:08, 20.27it/s]
avg reward: 0.96:  64%|██████▍   | 301/468 [00:15<00:08, 20.27it/s]
avg reward: 0.96:  65%|██████▌   | 306/468 [00:15<00:08, 20.22it/s]
avg reward: 0.96:  65%|██████▌   | 306/468 [00:15<00:08, 20.22it/s]
avg reward: 0.96:  65%|██████▌   | 306/468 [00:15<00:08, 20.22it/s]
avg reward: 0.96:  65%|██████▌   | 306/468 [00:15<00:08, 20.22it/s]
avg reward: 0.96:  65%|██████▌   | 306/468 [00:15<00:08, 20.22it/s]
avg reward: 0.96:  65%|██████▌   | 306/468 [00:15<00:08, 20.22it/s]
avg reward: 0.96:  66%|██████▋   | 311/468 [00:15<00:07, 20.16it/s]
avg reward: 0.96:  66%|██████▋   | 311/468 [00:15<00:07, 20.16it/s]
avg reward: 0.96:  66%|██████▋   | 311/468 [00:15<00:07, 20.16it/s]
avg reward: 0.96:  66%|██████▋   | 311/468 [00:15<00:07, 20.16it/s]
avg reward: 0.96:  66%|██████▋   | 311/468 [00:15<00:07, 20.16it/s]
avg reward: 0.96:  66%|██████▋   | 311/468 [00:15<00:07, 20.16it/s]
avg reward: 0.96:  68%|██████▊   | 316/468 [00:15<00:07, 20.33it/s]
avg reward: 0.96:  68%|██████▊   | 316/468 [00:15<00:07, 20.33it/s]
avg reward: 0.96:  68%|██████▊   | 316/468 [00:15<00:07, 20.33it/s]
avg reward: 0.96:  68%|██████▊   | 316/468 [00:15<00:07, 20.33it/s]
avg reward: 0.96:  68%|██████▊   | 316/468 [00:15<00:07, 20.33it/s]
avg reward: 0.96:  68%|██████▊   | 320/468 [00:15<00:06, 23.84it/s]
avg reward: 0.96:  68%|██████▊   | 320/468 [00:16<00:06, 23.84it/s]
avg reward: 0.96:  68%|██████▊   | 320/468 [00:16<00:06, 23.84it/s]
avg reward: 0.96:  68%|██████▊   | 320/468 [00:16<00:06, 23.84it/s]
avg reward: 0.96:  69%|██████▉   | 323/468 [00:16<00:06, 20.73it/s]
avg reward: 0.96:  69%|██████▉   | 323/468 [00:16<00:06, 20.73it/s]
avg reward: 0.96:  69%|██████▉   | 323/468 [00:16<00:06, 20.73it/s]
avg reward: 0.97:  69%|██████▉   | 323/468 [00:16<00:06, 20.73it/s]
avg reward: 0.97:  70%|██████▉   | 326/468 [00:16<00:07, 19.16it/s]
avg reward: 0.97:  70%|██████▉   | 326/468 [00:16<00:07, 19.16it/s]
avg reward: 0.97:  70%|██████▉   | 326/468 [00:16<00:07, 19.16it/s]
avg reward: 0.97:  70%|██████▉   | 326/468 [00:16<00:07, 19.16it/s]
avg reward: 0.97:  70%|██████▉   | 326/468 [00:16<00:07, 19.16it/s]
avg reward: 0.97:  70%|██████▉   | 326/468 [00:16<00:07, 19.16it/s]
avg reward: 0.97:  71%|███████   | 331/468 [00:16<00:07, 19.56it/s]
avg reward: 0.97:  71%|███████   | 331/468 [00:16<00:07, 19.56it/s]
avg reward: 0.97:  71%|███████   | 331/468 [00:16<00:07, 19.56it/s]
avg reward: 0.97:  71%|███████   | 331/468 [00:16<00:07, 19.56it/s]
avg reward: 0.97:  71%|███████   | 331/468 [00:16<00:07, 19.56it/s]
avg reward: 0.97:  72%|███████▏  | 335/468 [00:16<00:05, 23.10it/s]
avg reward: 0.97:  72%|███████▏  | 335/468 [00:16<00:05, 23.10it/s]
avg reward: 0.97:  72%|███████▏  | 335/468 [00:16<00:05, 23.10it/s]
avg reward: 0.97:  72%|███████▏  | 335/468 [00:16<00:05, 23.10it/s]
avg reward: 0.97:  72%|███████▏  | 338/468 [00:16<00:06, 19.43it/s]
avg reward: 0.97:  72%|███████▏  | 338/468 [00:16<00:06, 19.43it/s]
avg reward: 0.97:  72%|███████▏  | 338/468 [00:16<00:06, 19.43it/s]
avg reward: 0.97:  72%|███████▏  | 338/468 [00:17<00:06, 19.43it/s]
avg reward: 0.97:  73%|███████▎  | 341/468 [00:17<00:07, 18.11it/s]
avg reward: 0.97:  73%|███████▎  | 341/468 [00:17<00:07, 18.11it/s]
avg reward: 0.97:  73%|███████▎  | 341/468 [00:17<00:07, 18.11it/s]
avg reward: 0.97:  73%|███████▎  | 341/468 [00:17<00:07, 18.11it/s]
avg reward: 0.97:  73%|███████▎  | 341/468 [00:17<00:07, 18.11it/s]
avg reward: 0.97:  73%|███████▎  | 341/468 [00:17<00:07, 18.11it/s]
avg reward: 0.97:  74%|███████▍  | 346/468 [00:17<00:06, 18.93it/s]
avg reward: 0.97:  74%|███████▍  | 346/468 [00:17<00:06, 18.93it/s]
avg reward: 0.97:  74%|███████▍  | 346/468 [00:17<00:06, 18.93it/s]
avg reward: 0.97:  74%|███████▍  | 346/468 [00:17<00:06, 18.93it/s]
avg reward: 0.97:  74%|███████▍  | 346/468 [00:17<00:06, 18.93it/s]
avg reward: 0.97:  74%|███████▍  | 346/468 [00:17<00:06, 18.93it/s]
avg reward: 0.97:  75%|███████▌  | 351/468 [00:17<00:06, 18.96it/s]
avg reward: 0.97:  75%|███████▌  | 351/468 [00:17<00:06, 18.96it/s]
avg reward: 0.97:  75%|███████▌  | 351/468 [00:17<00:06, 18.96it/s]
avg reward: 0.97:  75%|███████▌  | 351/468 [00:17<00:06, 18.96it/s]
avg reward: 0.97:  75%|███████▌  | 351/468 [00:17<00:06, 18.96it/s]
avg reward: 0.98:  75%|███████▌  | 351/468 [00:17<00:06, 18.96it/s]
avg reward: 0.98:  76%|███████▌  | 356/468 [00:17<00:05, 18.97it/s]
avg reward: 0.98:  76%|███████▌  | 356/468 [00:17<00:05, 18.97it/s]
avg reward: 0.98:  76%|███████▌  | 356/468 [00:17<00:05, 18.97it/s]
avg reward: 0.98:  76%|███████▌  | 356/468 [00:17<00:05, 18.97it/s]
avg reward: 0.98:  76%|███████▌  | 356/468 [00:17<00:05, 18.97it/s]
avg reward: 0.98:  76%|███████▌  | 356/468 [00:18<00:05, 18.97it/s]
avg reward: 0.98:  77%|███████▋  | 361/468 [00:18<00:05, 19.49it/s]
avg reward: 0.98:  77%|███████▋  | 361/468 [00:18<00:05, 19.49it/s]
avg reward: 0.98:  77%|███████▋  | 361/468 [00:18<00:05, 19.49it/s]
avg reward: 0.98:  77%|███████▋  | 361/468 [00:18<00:05, 19.49it/s]
avg reward: 0.98:  77%|███████▋  | 361/468 [00:18<00:05, 19.49it/s]
avg reward: 0.97:  77%|███████▋  | 361/468 [00:18<00:05, 19.49it/s]
avg reward: 0.97:  78%|███████▊  | 366/468 [00:18<00:05, 19.98it/s]
avg reward: 0.97:  78%|███████▊  | 366/468 [00:18<00:05, 19.98it/s]
avg reward: 0.97:  78%|███████▊  | 366/468 [00:18<00:05, 19.98it/s]
avg reward: 0.97:  78%|███████▊  | 366/468 [00:18<00:05, 19.98it/s]
avg reward: 0.97:  78%|███████▊  | 366/468 [00:18<00:05, 19.98it/s]
avg reward: 0.98:  78%|███████▊  | 366/468 [00:18<00:05, 19.98it/s]
avg reward: 0.98:  79%|███████▉  | 371/468 [00:18<00:04, 20.28it/s]
avg reward: 0.98:  79%|███████▉  | 371/468 [00:18<00:04, 20.28it/s]
avg reward: 0.98:  79%|███████▉  | 371/468 [00:18<00:04, 20.28it/s]
avg reward: 0.98:  79%|███████▉  | 371/468 [00:18<00:04, 20.28it/s]
avg reward: 0.98:  79%|███████▉  | 371/468 [00:18<00:04, 20.28it/s]
avg reward: 0.98:  79%|███████▉  | 371/468 [00:18<00:04, 20.28it/s]
avg reward: 0.98:  80%|████████  | 376/468 [00:18<00:04, 20.07it/s]
avg reward: 0.98:  80%|████████  | 376/468 [00:18<00:04, 20.07it/s]
avg reward: 0.98:  80%|████████  | 376/468 [00:18<00:04, 20.07it/s]
avg reward: 0.98:  80%|████████  | 376/468 [00:18<00:04, 20.07it/s]
avg reward: 0.98:  80%|████████  | 376/468 [00:18<00:04, 20.07it/s]
avg reward: 0.98:  80%|████████  | 376/468 [00:19<00:04, 20.07it/s]
avg reward: 0.98:  81%|████████▏ | 381/468 [00:19<00:04, 20.70it/s]
avg reward: 0.98:  81%|████████▏ | 381/468 [00:19<00:04, 20.70it/s]
avg reward: 0.98:  81%|████████▏ | 381/468 [00:19<00:04, 20.70it/s]
avg reward: 0.98:  81%|████████▏ | 381/468 [00:19<00:04, 20.70it/s]
avg reward: 0.98:  81%|████████▏ | 381/468 [00:19<00:04, 20.70it/s]
avg reward: 0.97:  81%|████████▏ | 381/468 [00:19<00:04, 20.70it/s]
avg reward: 0.97:  82%|████████▏ | 386/468 [00:19<00:03, 20.91it/s]
avg reward: 0.97:  82%|████████▏ | 386/468 [00:19<00:03, 20.91it/s]
avg reward: 0.97:  82%|████████▏ | 386/468 [00:19<00:03, 20.91it/s]
avg reward: 0.97:  82%|████████▏ | 386/468 [00:19<00:03, 20.91it/s]
avg reward: 0.97:  82%|████████▏ | 386/468 [00:19<00:03, 20.91it/s]
avg reward: 0.97:  82%|████████▏ | 386/468 [00:19<00:03, 20.91it/s]
avg reward: 0.97:  84%|████████▎ | 391/468 [00:19<00:03, 20.93it/s]
avg reward: 0.97:  84%|████████▎ | 391/468 [00:19<00:03, 20.93it/s]
avg reward: 0.97:  84%|████████▎ | 391/468 [00:19<00:03, 20.93it/s]
avg reward: 0.97:  84%|████████▎ | 391/468 [00:19<00:03, 20.93it/s]
avg reward: 0.97:  84%|████████▎ | 391/468 [00:19<00:03, 20.93it/s]
avg reward: 0.98:  84%|████████▎ | 391/468 [00:19<00:03, 20.93it/s]
avg reward: 0.98:  85%|████████▍ | 396/468 [00:19<00:03, 20.96it/s]
avg reward: 0.98:  85%|████████▍ | 396/468 [00:19<00:03, 20.96it/s]
avg reward: 0.98:  85%|████████▍ | 396/468 [00:19<00:03, 20.96it/s]
avg reward: 0.98:  85%|████████▍ | 396/468 [00:19<00:03, 20.96it/s]
avg reward: 0.98:  85%|████████▍ | 396/468 [00:19<00:03, 20.96it/s]
avg reward: 0.97:  85%|████████▍ | 396/468 [00:20<00:03, 20.96it/s]
avg reward: 0.97:  86%|████████▌ | 401/468 [00:20<00:03, 20.77it/s]
avg reward: 0.97:  86%|████████▌ | 401/468 [00:20<00:03, 20.77it/s]
avg reward: 0.97:  86%|████████▌ | 401/468 [00:20<00:03, 20.77it/s]
avg reward: 0.97:  86%|████████▌ | 401/468 [00:20<00:03, 20.77it/s]
avg reward: 0.97:  86%|████████▌ | 401/468 [00:20<00:03, 20.77it/s]
avg reward: 0.98:  86%|████████▌ | 401/468 [00:20<00:03, 20.77it/s]
avg reward: 0.98:  87%|████████▋ | 406/468 [00:20<00:03, 20.56it/s]
avg reward: 0.98:  87%|████████▋ | 406/468 [00:20<00:03, 20.56it/s]
avg reward: 0.98:  87%|████████▋ | 406/468 [00:20<00:03, 20.56it/s]
avg reward: 0.98:  87%|████████▋ | 406/468 [00:20<00:03, 20.56it/s]
avg reward: 0.98:  87%|████████▋ | 406/468 [00:20<00:03, 20.56it/s]
avg reward: 0.98:  87%|████████▋ | 406/468 [00:20<00:03, 20.56it/s]
avg reward: 0.98:  88%|████████▊ | 411/468 [00:20<00:02, 20.82it/s]
avg reward: 0.98:  88%|████████▊ | 411/468 [00:20<00:02, 20.82it/s]
avg reward: 0.98:  88%|████████▊ | 411/468 [00:20<00:02, 20.82it/s]
avg reward: 0.98:  88%|████████▊ | 411/468 [00:20<00:02, 20.82it/s]
avg reward: 0.98:  88%|████████▊ | 411/468 [00:20<00:02, 20.82it/s]
avg reward: 0.98:  88%|████████▊ | 411/468 [00:20<00:02, 20.82it/s]
avg reward: 0.98:  89%|████████▉ | 416/468 [00:20<00:02, 20.77it/s]
avg reward: 0.98:  89%|████████▉ | 416/468 [00:20<00:02, 20.77it/s]
avg reward: 0.98:  89%|████████▉ | 416/468 [00:20<00:02, 20.77it/s]
avg reward: 0.98:  89%|████████▉ | 416/468 [00:20<00:02, 20.77it/s]
avg reward: 0.98:  89%|████████▉ | 416/468 [00:20<00:02, 20.77it/s]
avg reward: 0.98:  89%|████████▉ | 416/468 [00:20<00:02, 20.77it/s]
avg reward: 0.98:  90%|████████▉ | 421/468 [00:20<00:02, 20.61it/s]
avg reward: 0.98:  90%|████████▉ | 421/468 [00:21<00:02, 20.61it/s]
avg reward: 0.98:  90%|████████▉ | 421/468 [00:21<00:02, 20.61it/s]
avg reward: 0.98:  90%|████████▉ | 421/468 [00:21<00:02, 20.61it/s]
avg reward: 0.98:  90%|████████▉ | 421/468 [00:21<00:02, 20.61it/s]
avg reward: 0.98:  91%|█████████ | 425/468 [00:21<00:01, 24.06it/s]
avg reward: 0.98:  91%|█████████ | 425/468 [00:21<00:01, 24.06it/s]
avg reward: 0.98:  91%|█████████ | 425/468 [00:21<00:01, 24.06it/s]
avg reward: 0.98:  91%|█████████ | 425/468 [00:21<00:01, 24.06it/s]
avg reward: 0.98:  91%|█████████▏| 428/468 [00:21<00:01, 20.57it/s]
avg reward: 0.98:  91%|█████████▏| 428/468 [00:21<00:01, 20.57it/s]
avg reward: 0.98:  91%|█████████▏| 428/468 [00:21<00:01, 20.57it/s]
avg reward: 0.98:  91%|█████████▏| 428/468 [00:21<00:01, 20.57it/s]
avg reward: 0.98:  92%|█████████▏| 431/468 [00:21<00:01, 18.77it/s]
avg reward: 0.98:  92%|█████████▏| 431/468 [00:21<00:01, 18.77it/s]
avg reward: 0.98:  92%|█████████▏| 431/468 [00:21<00:01, 18.77it/s]
avg reward: 0.98:  92%|█████████▏| 431/468 [00:21<00:01, 18.77it/s]
avg reward: 0.98:  92%|█████████▏| 431/468 [00:21<00:01, 18.77it/s]
avg reward: 0.98:  92%|█████████▏| 431/468 [00:21<00:01, 18.77it/s]
avg reward: 0.98:  93%|█████████▎| 436/468 [00:21<00:01, 19.01it/s]
avg reward: 0.98:  93%|█████████▎| 436/468 [00:21<00:01, 19.01it/s]
avg reward: 0.98:  93%|█████████▎| 436/468 [00:21<00:01, 19.01it/s]
avg reward: 0.98:  93%|█████████▎| 436/468 [00:21<00:01, 19.01it/s]
avg reward: 0.98:  93%|█████████▎| 436/468 [00:21<00:01, 19.01it/s]
avg reward: 0.98:  93%|█████████▎| 436/468 [00:21<00:01, 19.01it/s]
avg reward: 0.98:  94%|█████████▍| 441/468 [00:21<00:01, 19.56it/s]
avg reward: 0.98:  94%|█████████▍| 441/468 [00:22<00:01, 19.56it/s]
avg reward: 0.98:  94%|█████████▍| 441/468 [00:22<00:01, 19.56it/s]
avg reward: 0.98:  94%|█████████▍| 441/468 [00:22<00:01, 19.56it/s]
avg reward: 0.98:  94%|█████████▍| 441/468 [00:22<00:01, 19.56it/s]
avg reward: 0.98:  94%|█████████▍| 441/468 [00:22<00:01, 19.56it/s]
avg reward: 0.98:  95%|█████████▌| 446/468 [00:22<00:01, 19.76it/s]
avg reward: 0.98:  95%|█████████▌| 446/468 [00:22<00:01, 19.76it/s]
avg reward: 0.98:  95%|█████████▌| 446/468 [00:22<00:01, 19.76it/s]
avg reward: 0.98:  95%|█████████▌| 446/468 [00:22<00:01, 19.76it/s]
avg reward: 0.98:  95%|█████████▌| 446/468 [00:22<00:01, 19.76it/s]
avg reward: 0.98:  96%|█████████▌| 450/468 [00:22<00:00, 23.16it/s]
avg reward: 0.98:  96%|█████████▌| 450/468 [00:22<00:00, 23.16it/s]
avg reward: 0.98:  96%|█████████▌| 450/468 [00:22<00:00, 23.16it/s]
avg reward: 0.98:  96%|█████████▌| 450/468 [00:22<00:00, 23.16it/s]
avg reward: 0.98:  97%|█████████▋| 453/468 [00:22<00:00, 19.78it/s]
avg reward: 0.98:  97%|█████████▋| 453/468 [00:22<00:00, 19.78it/s]
avg reward: 0.98:  97%|█████████▋| 453/468 [00:22<00:00, 19.78it/s]
avg reward: 0.98:  97%|█████████▋| 453/468 [00:22<00:00, 19.78it/s]
avg reward: 0.98:  97%|█████████▋| 456/468 [00:22<00:00, 18.05it/s]
avg reward: 0.98:  97%|█████████▋| 456/468 [00:22<00:00, 18.05it/s]
avg reward: 0.98:  97%|█████████▋| 456/468 [00:22<00:00, 18.05it/s]
avg reward: 0.98:  97%|█████████▋| 456/468 [00:22<00:00, 18.05it/s]
avg reward: 0.98:  97%|█████████▋| 456/468 [00:22<00:00, 18.05it/s]
avg reward: 0.98:  98%|█████████▊| 460/468 [00:22<00:00, 21.52it/s]
avg reward: 0.98:  98%|█████████▊| 460/468 [00:22<00:00, 21.52it/s]
avg reward: 0.98:  98%|█████████▊| 460/468 [00:22<00:00, 21.52it/s]
avg reward: 0.98:  98%|█████████▊| 460/468 [00:23<00:00, 21.52it/s]
avg reward: 0.98:  99%|█████████▉| 463/468 [00:23<00:00, 19.31it/s]
avg reward: 0.98:  99%|█████████▉| 463/468 [00:23<00:00, 19.31it/s]
avg reward: 0.98:  99%|█████████▉| 463/468 [00:23<00:00, 19.31it/s]
avg reward: 0.98:  99%|█████████▉| 463/468 [00:23<00:00, 19.31it/s]
avg reward: 0.98: 100%|█████████▉| 466/468 [00:23<00:00, 17.96it/s]
avg reward: 0.98: 100%|█████████▉| 466/468 [00:23<00:00, 17.96it/s]
avg reward: 0.98: 100%|██████████| 468/468 [00:23<00:00, 20.12it/s]

  0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.96875:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.9765625:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.9791666666666666:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.970703125:   0%|          | 0/79 [00:00<?, ?it/s]       
Val Acc: 0.965625:   0%|          | 0/79 [00:00<?, ?it/s]   
Val Acc: 0.96484375:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.9654017857142857:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.9658203125:   0%|          | 0/79 [00:00<?, ?it/s]      
Val Acc: 0.9658203125:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]
Val Acc: 0.9696180555555556:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]
Val Acc: 0.96875:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]           
Val Acc: 0.9694602272727273:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]
Val Acc: 0.9694010416666666:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]
Val Acc: 0.96875:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]           
Val Acc: 0.9693080357142857:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]
Val Acc: 0.9692708333333333:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]
Val Acc: 0.96923828125:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]     
Val Acc: 0.9682904411764706:  10%|█         | 8/79 [00:00<00:00, 74.05it/s]
Val Acc: 0.9682904411764706:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]
Val Acc: 0.9665798611111112:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]
Val Acc: 0.9666940789473685:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]
Val Acc: 0.9671875:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]         
Val Acc: 0.9683779761904762:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]
Val Acc: 0.96875:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]           
Val Acc: 0.9697690217391305:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]
Val Acc: 0.9700520833333334:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]
Val Acc: 0.97:  22%|██▏       | 17/79 [00:00<00:00, 76.72it/s]              
Val Acc: 0.97:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]
Val Acc: 0.9696514423076923:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]
Val Acc: 0.9701967592592593:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]
Val Acc: 0.9690290178571429:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]
Val Acc: 0.9698275862068966:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]
Val Acc: 0.96875:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]           
Val Acc: 0.9695060483870968:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]
Val Acc: 0.969482421875:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]    
Val Acc: 0.9692234848484849:  32%|███▏      | 25/79 [00:00<00:00, 76.04it/s]
Val Acc: 0.9692234848484849:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s]
Val Acc: 0.9696691176470589:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s]
Val Acc: 0.9698660714285714:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s]
Val Acc: 0.9700520833333334:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s]
Val Acc: 0.9700168918918919:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s]
Val Acc: 0.9695723684210527:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s]
Val Acc: 0.969551282051282:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s] 
Val Acc: 0.9689453125:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s]     
Val Acc: 0.9681783536585366:  42%|████▏     | 33/79 [00:00<00:00, 76.89it/s]
Val Acc: 0.9681783536585366:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9685639880952381:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9685683139534884:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9683948863636364:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9685763888888889:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9680706521739131:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9680851063829787:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9684244791666666:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9685905612244898:  52%|█████▏    | 41/79 [00:00<00:00, 76.87it/s]
Val Acc: 0.9685905612244898:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]
Val Acc: 0.96875:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]           
Val Acc: 0.9689031862745098:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]
Val Acc: 0.9684495192307693:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]
Val Acc: 0.9688974056603774:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]
Val Acc: 0.96875:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]           
Val Acc: 0.96875:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]
Val Acc: 0.9688895089285714:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]
Val Acc: 0.9692982456140351:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]
Val Acc: 0.9688846982758621:  62%|██████▏   | 49/79 [00:00<00:00, 76.99it/s]
Val Acc: 0.9688846982758621:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]
Val Acc: 0.9691472457627118:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]
Val Acc: 0.96875:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]           
Val Acc: 0.9691342213114754:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]
Val Acc: 0.9691280241935484:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]
Val Acc: 0.9689980158730159:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]
Val Acc: 0.9688720703125:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]   
Val Acc: 0.9691105769230769:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]
Val Acc: 0.9693418560606061:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]
Val Acc: 0.9689832089552238:  73%|███████▎  | 58/79 [00:00<00:00, 79.70it/s]
Val Acc: 0.9689832089552238:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s]
Val Acc: 0.96875:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s]           
Val Acc: 0.9682971014492754:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s]
Val Acc: 0.9684151785714286:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s]
Val Acc: 0.968199823943662:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s] 
Val Acc: 0.9684244791666666:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s]
Val Acc: 0.9684289383561644:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s]
Val Acc: 0.9685388513513513:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s]
Val Acc: 0.96875:  85%|████████▍ | 67/79 [00:00<00:00, 79.03it/s]           
Val Acc: 0.96875:  95%|█████████▍| 75/79 [00:00<00:00, 78.41it/s]
Val Acc: 0.96875:  95%|█████████▍| 75/79 [00:00<00:00, 78.41it/s]
Val Acc: 0.968648538961039:  95%|█████████▍| 75/79 [00:00<00:00, 78.41it/s]
Val Acc: 0.9689503205128205:  95%|█████████▍| 75/79 [00:00<00:00, 78.41it/s]
Val Acc: 0.969: 100%|██████████| 79/79 [00:01<00:00, 76.79it/s]
epoch 0, val_acc: 0.97, avg reward: 0.98: 100%|██████████| 1/1 [00:24<00:00, 24.33s/it]

The resulting architecture is:

mynet.graph
../../_images/output_enas_mnist_1830e6_21_0.svg

Change the reward trade-off:

reward_fn = lambda metric, net: metric * ((net.avg_latency / net.latency) ** 0.8)
mynet.initialize(force_reinit=True)
scheduler = ENAS_Scheduler(mynet, train_set='mnist',
                           reward_fn=reward_fn, batch_size=128, num_gpus=1,
                           warmup_epochs=0, epochs=1, controller_lr=3e-3,
                           plot_frequency=10, update_arch_frequency=5)
scheduler.run()
get_built_in_dataset mnist
get_built_in_dataset mnist
  0%|          | 0/1 [00:00<?, ?it/s]
  0%|          | 0/468 [00:00<?, ?it/s]
avg reward: 0.11:   0%|          | 0/468 [00:00<?, ?it/s]
avg reward: 0.11:   0%|          | 1/468 [00:00<01:59,  3.91it/s]
avg reward: 0.11:   0%|          | 1/468 [00:00<01:59,  3.91it/s]
avg reward: 0.11:   0%|          | 1/468 [00:00<01:59,  3.91it/s]
avg reward: 0.11:   0%|          | 1/468 [00:00<01:59,  3.91it/s]
avg reward: 0.11:   0%|          | 1/468 [00:00<01:59,  3.91it/s]
avg reward: 0.12:   0%|          | 1/468 [00:00<01:59,  3.91it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:29,  5.16it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:29,  5.16it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:29,  5.16it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:29,  5.16it/s]
avg reward: 0.12:   1%|▏         | 6/468 [00:00<01:29,  5.16it/s]
avg reward: 0.11:   1%|▏         | 6/468 [00:00<01:29,  5.16it/s]
avg reward: 0.11:   2%|▏         | 11/468 [00:00<01:08,  6.69it/s]
avg reward: 0.11:   2%|▏         | 11/468 [00:00<01:08,  6.69it/s]
avg reward: 0.11:   2%|▏         | 11/468 [00:00<01:08,  6.69it/s]
avg reward: 0.11:   2%|▏         | 11/468 [00:00<01:08,  6.69it/s]
avg reward: 0.11:   2%|▏         | 11/468 [00:00<01:08,  6.69it/s]
avg reward: 0.11:   2%|▏         | 11/468 [00:00<01:08,  6.69it/s]
avg reward: 0.11:   3%|▎         | 16/468 [00:00<00:54,  8.37it/s]
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:54,  8.37it/s]
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:54,  8.37it/s]
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:54,  8.37it/s]
avg reward: 0.11:   3%|▎         | 16/468 [00:01<00:54,  8.37it/s]
avg reward: 0.12:   3%|▎         | 16/468 [00:01<00:54,  8.37it/s]
avg reward: 0.12:   4%|▍         | 21/468 [00:01<00:43, 10.17it/s]
avg reward: 0.12:   4%|▍         | 21/468 [00:01<00:43, 10.17it/s]
avg reward: 0.12:   4%|▍         | 21/468 [00:01<00:43, 10.17it/s]
avg reward: 0.12:   4%|▍         | 21/468 [00:01<00:43, 10.17it/s]
avg reward: 0.12:   4%|▍         | 21/468 [00:01<00:43, 10.17it/s]
avg reward: 0.12:   5%|▌         | 25/468 [00:01<00:33, 13.10it/s]
avg reward: 0.12:   5%|▌         | 25/468 [00:01<00:33, 13.10it/s]
avg reward: 0.12:   5%|▌         | 25/468 [00:01<00:33, 13.10it/s]
avg reward: 0.12:   5%|▌         | 25/468 [00:01<00:33, 13.10it/s]
avg reward: 0.12:   6%|▌         | 28/468 [00:01<00:32, 13.45it/s]
avg reward: 0.12:   6%|▌         | 28/468 [00:01<00:32, 13.45it/s]
avg reward: 0.12:   6%|▌         | 28/468 [00:01<00:32, 13.45it/s]
avg reward: 0.14:   6%|▌         | 28/468 [00:01<00:32, 13.45it/s]
avg reward: 0.14:   7%|▋         | 31/468 [00:01<00:31, 14.06it/s]
avg reward: 0.14:   7%|▋         | 31/468 [00:01<00:31, 14.06it/s]
avg reward: 0.14:   7%|▋         | 31/468 [00:01<00:31, 14.06it/s]
avg reward: 0.14:   7%|▋         | 31/468 [00:01<00:31, 14.06it/s]
avg reward: 0.14:   7%|▋         | 31/468 [00:01<00:31, 14.06it/s]
avg reward: 0.16:   7%|▋         | 31/468 [00:01<00:31, 14.06it/s]
avg reward: 0.16:   8%|▊         | 36/468 [00:01<00:27, 15.52it/s]
avg reward: 0.16:   8%|▊         | 36/468 [00:01<00:27, 15.52it/s]
avg reward: 0.16:   8%|▊         | 36/468 [00:02<00:27, 15.52it/s]
avg reward: 0.16:   8%|▊         | 36/468 [00:02<00:27, 15.52it/s]
avg reward: 0.16:   8%|▊         | 36/468 [00:02<00:27, 15.52it/s]
avg reward: 0.18:   8%|▊         | 36/468 [00:02<00:27, 15.52it/s]
avg reward: 0.18:   9%|▉         | 41/468 [00:02<00:25, 16.78it/s]
avg reward: 0.18:   9%|▉         | 41/468 [00:02<00:25, 16.78it/s]
avg reward: 0.18:   9%|▉         | 41/468 [00:02<00:25, 16.78it/s]
avg reward: 0.18:   9%|▉         | 41/468 [00:02<00:25, 16.78it/s]
avg reward: 0.18:   9%|▉         | 41/468 [00:02<00:25, 16.78it/s]
avg reward: 0.22:   9%|▉         | 41/468 [00:02<00:25, 16.78it/s]
avg reward: 0.22:  10%|▉         | 46/468 [00:02<00:23, 18.05it/s]
avg reward: 0.22:  10%|▉         | 46/468 [00:02<00:23, 18.05it/s]
avg reward: 0.22:  10%|▉         | 46/468 [00:02<00:23, 18.05it/s]
avg reward: 0.22:  10%|▉         | 46/468 [00:02<00:23, 18.05it/s]
avg reward: 0.22:  10%|▉         | 46/468 [00:02<00:23, 18.05it/s]
avg reward: 0.25:  10%|▉         | 46/468 [00:02<00:23, 18.05it/s]
avg reward: 0.25:  11%|█         | 51/468 [00:02<00:22, 18.76it/s]
avg reward: 0.25:  11%|█         | 51/468 [00:02<00:22, 18.76it/s]
avg reward: 0.25:  11%|█         | 51/468 [00:02<00:22, 18.76it/s]
avg reward: 0.25:  11%|█         | 51/468 [00:02<00:22, 18.76it/s]
avg reward: 0.25:  11%|█         | 51/468 [00:02<00:22, 18.76it/s]
avg reward: 0.29:  11%|█         | 51/468 [00:02<00:22, 18.76it/s]
avg reward: 0.29:  12%|█▏        | 56/468 [00:02<00:21, 19.26it/s]
avg reward: 0.29:  12%|█▏        | 56/468 [00:02<00:21, 19.26it/s]
avg reward: 0.29:  12%|█▏        | 56/468 [00:02<00:21, 19.26it/s]
avg reward: 0.29:  12%|█▏        | 56/468 [00:03<00:21, 19.26it/s]
avg reward: 0.29:  12%|█▏        | 56/468 [00:03<00:21, 19.26it/s]
avg reward: 0.34:  12%|█▏        | 56/468 [00:03<00:21, 19.26it/s]
avg reward: 0.34:  13%|█▎        | 61/468 [00:03<00:20, 19.61it/s]
avg reward: 0.34:  13%|█▎        | 61/468 [00:03<00:20, 19.61it/s]
avg reward: 0.34:  13%|█▎        | 61/468 [00:03<00:20, 19.61it/s]
avg reward: 0.34:  13%|█▎        | 61/468 [00:03<00:20, 19.61it/s]
avg reward: 0.34:  13%|█▎        | 61/468 [00:03<00:20, 19.61it/s]
avg reward: 0.38:  13%|█▎        | 61/468 [00:03<00:20, 19.61it/s]
avg reward: 0.38:  14%|█▍        | 66/468 [00:03<00:20, 20.03it/s]
avg reward: 0.38:  14%|█▍        | 66/468 [00:03<00:20, 20.03it/s]
avg reward: 0.38:  14%|█▍        | 66/468 [00:03<00:20, 20.03it/s]
avg reward: 0.38:  14%|█▍        | 66/468 [00:03<00:20, 20.03it/s]
avg reward: 0.38:  14%|█▍        | 66/468 [00:03<00:20, 20.03it/s]
avg reward: 0.38:  15%|█▍        | 70/468 [00:03<00:16, 23.54it/s]
avg reward: 0.39:  15%|█▍        | 70/468 [00:03<00:16, 23.54it/s]
avg reward: 0.39:  15%|█▍        | 70/468 [00:03<00:16, 23.54it/s]
avg reward: 0.39:  15%|█▍        | 70/468 [00:03<00:16, 23.54it/s]
avg reward: 0.39:  16%|█▌        | 73/468 [00:03<00:19, 19.90it/s]
avg reward: 0.39:  16%|█▌        | 73/468 [00:03<00:19, 19.90it/s]
avg reward: 0.39:  16%|█▌        | 73/468 [00:03<00:19, 19.90it/s]
avg reward: 0.39:  16%|█▌        | 73/468 [00:03<00:19, 19.90it/s]
avg reward: 0.39:  16%|█▌        | 76/468 [00:03<00:21, 18.31it/s]
avg reward: 0.39:  16%|█▌        | 76/468 [00:03<00:21, 18.31it/s]
avg reward: 0.39:  16%|█▌        | 76/468 [00:03<00:21, 18.31it/s]
avg reward: 0.39:  16%|█▌        | 76/468 [00:03<00:21, 18.31it/s]
avg reward: 0.39:  16%|█▌        | 76/468 [00:03<00:21, 18.31it/s]
avg reward: 0.46:  16%|█▌        | 76/468 [00:04<00:21, 18.31it/s]
avg reward: 0.46:  17%|█▋        | 81/468 [00:04<00:20, 18.98it/s]
avg reward: 0.46:  17%|█▋        | 81/468 [00:04<00:20, 18.98it/s]
avg reward: 0.46:  17%|█▋        | 81/468 [00:04<00:20, 18.98it/s]
avg reward: 0.46:  17%|█▋        | 81/468 [00:04<00:20, 18.98it/s]
avg reward: 0.46:  17%|█▋        | 81/468 [00:04<00:20, 18.98it/s]
avg reward: 0.49:  17%|█▋        | 81/468 [00:04<00:20, 18.98it/s]
avg reward: 0.49:  18%|█▊        | 86/468 [00:04<00:19, 19.57it/s]
avg reward: 0.49:  18%|█▊        | 86/468 [00:04<00:19, 19.57it/s]
avg reward: 0.49:  18%|█▊        | 86/468 [00:04<00:19, 19.57it/s]
avg reward: 0.49:  18%|█▊        | 86/468 [00:04<00:19, 19.57it/s]
avg reward: 0.49:  18%|█▊        | 86/468 [00:04<00:19, 19.57it/s]
avg reward: 0.53:  18%|█▊        | 86/468 [00:04<00:19, 19.57it/s]
avg reward: 0.53:  19%|█▉        | 91/468 [00:04<00:19, 19.32it/s]
avg reward: 0.53:  19%|█▉        | 91/468 [00:04<00:19, 19.32it/s]
avg reward: 0.53:  19%|█▉        | 91/468 [00:04<00:19, 19.32it/s]
avg reward: 0.53:  19%|█▉        | 91/468 [00:04<00:19, 19.32it/s]
avg reward: 0.53:  19%|█▉        | 91/468 [00:04<00:19, 19.32it/s]
avg reward: 0.53:  20%|██        | 95/468 [00:04<00:16, 22.70it/s]
avg reward: 0.54:  20%|██        | 95/468 [00:04<00:16, 22.70it/s]
avg reward: 0.54:  20%|██        | 95/468 [00:04<00:16, 22.70it/s]
avg reward: 0.54:  20%|██        | 95/468 [00:04<00:16, 22.70it/s]
avg reward: 0.54:  21%|██        | 98/468 [00:04<00:19, 19.08it/s]
avg reward: 0.54:  21%|██        | 98/468 [00:04<00:19, 19.08it/s]
avg reward: 0.54:  21%|██        | 98/468 [00:05<00:19, 19.08it/s]
avg reward: 0.58:  21%|██        | 98/468 [00:05<00:19, 19.08it/s]
avg reward: 0.58:  22%|██▏       | 101/468 [00:05<00:21, 17.23it/s]
avg reward: 0.58:  22%|██▏       | 101/468 [00:05<00:21, 17.23it/s]
avg reward: 0.58:  22%|██▏       | 101/468 [00:05<00:21, 17.23it/s]
avg reward: 0.58:  22%|██▏       | 101/468 [00:05<00:21, 17.23it/s]
avg reward: 0.58:  22%|██▏       | 101/468 [00:05<00:21, 17.23it/s]
avg reward: 0.58:  22%|██▏       | 105/468 [00:05<00:17, 20.56it/s]
avg reward: 0.62:  22%|██▏       | 105/468 [00:05<00:17, 20.56it/s]
avg reward: 0.62:  22%|██▏       | 105/468 [00:05<00:17, 20.56it/s]
avg reward: 0.62:  22%|██▏       | 105/468 [00:05<00:17, 20.56it/s]
avg reward: 0.62:  23%|██▎       | 108/468 [00:05<00:19, 18.04it/s]
avg reward: 0.62:  23%|██▎       | 108/468 [00:05<00:19, 18.04it/s]
avg reward: 0.62:  23%|██▎       | 108/468 [00:05<00:19, 18.04it/s]
avg reward: 0.66:  23%|██▎       | 108/468 [00:05<00:19, 18.04it/s]
avg reward: 0.66:  24%|██▎       | 111/468 [00:05<00:21, 16.55it/s]
avg reward: 0.66:  24%|██▎       | 111/468 [00:05<00:21, 16.55it/s]
avg reward: 0.66:  24%|██▎       | 111/468 [00:05<00:21, 16.55it/s]
avg reward: 0.66:  24%|██▎       | 111/468 [00:05<00:21, 16.55it/s]
avg reward: 0.66:  24%|██▎       | 111/468 [00:05<00:21, 16.55it/s]
avg reward: 0.66:  24%|██▎       | 111/468 [00:05<00:21, 16.55it/s]
avg reward: 0.66:  25%|██▍       | 116/468 [00:05<00:20, 17.27it/s]
avg reward: 0.66:  25%|██▍       | 116/468 [00:06<00:20, 17.27it/s]
avg reward: 0.66:  25%|██▍       | 116/468 [00:06<00:20, 17.27it/s]
avg reward: 0.66:  25%|██▍       | 116/468 [00:06<00:20, 17.27it/s]
avg reward: 0.66:  25%|██▍       | 116/468 [00:06<00:20, 17.27it/s]
avg reward: 0.66:  26%|██▌       | 120/468 [00:06<00:16, 20.78it/s]
avg reward: 0.68:  26%|██▌       | 120/468 [00:06<00:16, 20.78it/s]
avg reward: 0.68:  26%|██▌       | 120/468 [00:06<00:16, 20.78it/s]
avg reward: 0.68:  26%|██▌       | 120/468 [00:06<00:16, 20.78it/s]
avg reward: 0.68:  26%|██▋       | 123/468 [00:06<00:19, 18.07it/s]
avg reward: 0.68:  26%|██▋       | 123/468 [00:06<00:19, 18.07it/s]
avg reward: 0.68:  26%|██▋       | 123/468 [00:06<00:19, 18.07it/s]
avg reward: 0.70:  26%|██▋       | 123/468 [00:06<00:19, 18.07it/s]
avg reward: 0.70:  27%|██▋       | 126/468 [00:06<00:19, 17.25it/s]
avg reward: 0.70:  27%|██▋       | 126/468 [00:06<00:19, 17.25it/s]
avg reward: 0.70:  27%|██▋       | 126/468 [00:06<00:19, 17.25it/s]
avg reward: 0.70:  27%|██▋       | 126/468 [00:06<00:19, 17.25it/s]
avg reward: 0.70:  27%|██▋       | 126/468 [00:06<00:19, 17.25it/s]
avg reward: 0.71:  27%|██▋       | 126/468 [00:06<00:19, 17.25it/s]
avg reward: 0.71:  28%|██▊       | 131/468 [00:06<00:18, 18.22it/s]
avg reward: 0.71:  28%|██▊       | 131/468 [00:06<00:18, 18.22it/s]
avg reward: 0.71:  28%|██▊       | 131/468 [00:06<00:18, 18.22it/s]
avg reward: 0.71:  28%|██▊       | 131/468 [00:06<00:18, 18.22it/s]
avg reward: 0.71:  28%|██▊       | 131/468 [00:06<00:18, 18.22it/s]
avg reward: 0.80:  28%|██▊       | 131/468 [00:06<00:18, 18.22it/s]
avg reward: 0.80:  29%|██▉       | 136/468 [00:06<00:17, 19.06it/s]
avg reward: 0.80:  29%|██▉       | 136/468 [00:06<00:17, 19.06it/s]
avg reward: 0.80:  29%|██▉       | 136/468 [00:07<00:17, 19.06it/s]
avg reward: 0.80:  29%|██▉       | 136/468 [00:07<00:17, 19.06it/s]
avg reward: 0.80:  29%|██▉       | 136/468 [00:07<00:17, 19.06it/s]
avg reward: 0.82:  29%|██▉       | 136/468 [00:07<00:17, 19.06it/s]
avg reward: 0.82:  30%|███       | 141/468 [00:07<00:17, 19.14it/s]
avg reward: 0.82:  30%|███       | 141/468 [00:07<00:17, 19.14it/s]
avg reward: 0.82:  30%|███       | 141/468 [00:07<00:17, 19.14it/s]
avg reward: 0.82:  30%|███       | 141/468 [00:07<00:17, 19.14it/s]
avg reward: 0.82:  30%|███       | 141/468 [00:07<00:17, 19.14it/s]
avg reward: 0.88:  30%|███       | 141/468 [00:07<00:17, 19.14it/s]
avg reward: 0.88:  31%|███       | 146/468 [00:07<00:16, 19.64it/s]
avg reward: 0.88:  31%|███       | 146/468 [00:07<00:16, 19.64it/s]
avg reward: 0.88:  31%|███       | 146/468 [00:07<00:16, 19.64it/s]
avg reward: 0.88:  31%|███       | 146/468 [00:07<00:16, 19.64it/s]
avg reward: 0.88:  31%|███       | 146/468 [00:07<00:16, 19.64it/s]
avg reward: 0.85:  31%|███       | 146/468 [00:07<00:16, 19.64it/s]
avg reward: 0.85:  32%|███▏      | 151/468 [00:07<00:16, 19.73it/s]
avg reward: 0.85:  32%|███▏      | 151/468 [00:07<00:16, 19.73it/s]
avg reward: 0.85:  32%|███▏      | 151/468 [00:07<00:16, 19.73it/s]
avg reward: 0.85:  32%|███▏      | 151/468 [00:07<00:16, 19.73it/s]
avg reward: 0.85:  32%|███▏      | 151/468 [00:07<00:16, 19.73it/s]
avg reward: 0.88:  32%|███▏      | 151/468 [00:07<00:16, 19.73it/s]
avg reward: 0.88:  33%|███▎      | 156/468 [00:07<00:15, 20.02it/s]
avg reward: 0.88:  33%|███▎      | 156/468 [00:07<00:15, 20.02it/s]
avg reward: 0.88:  33%|███▎      | 156/468 [00:08<00:15, 20.02it/s]
avg reward: 0.88:  33%|███▎      | 156/468 [00:08<00:15, 20.02it/s]
avg reward: 0.88:  33%|███▎      | 156/468 [00:08<00:15, 20.02it/s]
avg reward: 0.91:  33%|███▎      | 156/468 [00:08<00:15, 20.02it/s]
avg reward: 0.91:  34%|███▍      | 161/468 [00:08<00:15, 20.11it/s]
avg reward: 0.91:  34%|███▍      | 161/468 [00:08<00:15, 20.11it/s]
avg reward: 0.91:  34%|███▍      | 161/468 [00:08<00:15, 20.11it/s]
avg reward: 0.91:  34%|███▍      | 161/468 [00:08<00:15, 20.11it/s]
avg reward: 0.91:  34%|███▍      | 161/468 [00:08<00:15, 20.11it/s]
avg reward: 0.94:  34%|███▍      | 161/468 [00:08<00:15, 20.11it/s]
avg reward: 0.94:  35%|███▌      | 166/468 [00:08<00:15, 19.60it/s]
avg reward: 0.94:  35%|███▌      | 166/468 [00:08<00:15, 19.60it/s]
avg reward: 0.94:  35%|███▌      | 166/468 [00:08<00:15, 19.60it/s]
avg reward: 0.94:  35%|███▌      | 166/468 [00:08<00:15, 19.60it/s]
avg reward: 0.94:  35%|███▌      | 166/468 [00:08<00:15, 19.60it/s]
avg reward: 0.95:  35%|███▌      | 166/468 [00:08<00:15, 19.60it/s]
avg reward: 0.95:  37%|███▋      | 171/468 [00:08<00:15, 19.64it/s]
avg reward: 0.95:  37%|███▋      | 171/468 [00:08<00:15, 19.64it/s]
avg reward: 0.95:  37%|███▋      | 171/468 [00:08<00:15, 19.64it/s]
avg reward: 0.95:  37%|███▋      | 171/468 [00:08<00:15, 19.64it/s]
avg reward: 0.95:  37%|███▋      | 171/468 [00:08<00:15, 19.64it/s]
avg reward: 0.99:  37%|███▋      | 171/468 [00:08<00:15, 19.64it/s]
avg reward: 0.99:  38%|███▊      | 176/468 [00:08<00:14, 20.23it/s]
avg reward: 0.99:  38%|███▊      | 176/468 [00:08<00:14, 20.23it/s]
avg reward: 0.99:  38%|███▊      | 176/468 [00:09<00:14, 20.23it/s]
avg reward: 0.99:  38%|███▊      | 176/468 [00:09<00:14, 20.23it/s]
avg reward: 0.99:  38%|███▊      | 176/468 [00:09<00:14, 20.23it/s]
avg reward: 1.03:  38%|███▊      | 176/468 [00:09<00:14, 20.23it/s]
avg reward: 1.03:  39%|███▊      | 181/468 [00:09<00:14, 19.79it/s]
avg reward: 1.03:  39%|███▊      | 181/468 [00:09<00:14, 19.79it/s]
avg reward: 1.03:  39%|███▊      | 181/468 [00:09<00:14, 19.79it/s]
avg reward: 1.03:  39%|███▊      | 181/468 [00:09<00:14, 19.79it/s]
avg reward: 1.03:  39%|███▊      | 181/468 [00:09<00:14, 19.79it/s]
avg reward: 1.03:  39%|███▊      | 181/468 [00:09<00:14, 19.79it/s]
avg reward: 1.03:  40%|███▉      | 186/468 [00:09<00:13, 20.27it/s]
avg reward: 1.03:  40%|███▉      | 186/468 [00:09<00:13, 20.27it/s]
avg reward: 1.03:  40%|███▉      | 186/468 [00:09<00:13, 20.27it/s]
avg reward: 1.03:  40%|███▉      | 186/468 [00:09<00:13, 20.27it/s]
avg reward: 1.03:  40%|███▉      | 186/468 [00:09<00:13, 20.27it/s]
avg reward: 1.05:  40%|███▉      | 186/468 [00:09<00:13, 20.27it/s]
avg reward: 1.05:  41%|████      | 191/468 [00:09<00:13, 20.52it/s]
avg reward: 1.05:  41%|████      | 191/468 [00:09<00:13, 20.52it/s]
avg reward: 1.05:  41%|████      | 191/468 [00:09<00:13, 20.52it/s]
avg reward: 1.05:  41%|████      | 191/468 [00:09<00:13, 20.52it/s]
avg reward: 1.05:  41%|████      | 191/468 [00:09<00:13, 20.52it/s]
avg reward: 1.08:  41%|████      | 191/468 [00:09<00:13, 20.52it/s]
avg reward: 1.08:  42%|████▏     | 196/468 [00:09<00:13, 20.30it/s]
avg reward: 1.08:  42%|████▏     | 196/468 [00:09<00:13, 20.30it/s]
avg reward: 1.08:  42%|████▏     | 196/468 [00:09<00:13, 20.30it/s]
avg reward: 1.08:  42%|████▏     | 196/468 [00:10<00:13, 20.30it/s]
avg reward: 1.08:  42%|████▏     | 196/468 [00:10<00:13, 20.30it/s]
avg reward: 1.12:  42%|████▏     | 196/468 [00:10<00:13, 20.30it/s]
avg reward: 1.12:  43%|████▎     | 201/468 [00:10<00:13, 20.51it/s]
avg reward: 1.12:  43%|████▎     | 201/468 [00:10<00:13, 20.51it/s]
avg reward: 1.12:  43%|████▎     | 201/468 [00:10<00:13, 20.51it/s]
avg reward: 1.12:  43%|████▎     | 201/468 [00:10<00:13, 20.51it/s]
avg reward: 1.12:  43%|████▎     | 201/468 [00:10<00:13, 20.51it/s]
avg reward: 1.15:  43%|████▎     | 201/468 [00:10<00:13, 20.51it/s]
avg reward: 1.15:  44%|████▍     | 206/468 [00:10<00:12, 20.65it/s]
avg reward: 1.15:  44%|████▍     | 206/468 [00:10<00:12, 20.65it/s]
avg reward: 1.15:  44%|████▍     | 206/468 [00:10<00:12, 20.65it/s]
avg reward: 1.15:  44%|████▍     | 206/468 [00:10<00:12, 20.65it/s]
avg reward: 1.15:  44%|████▍     | 206/468 [00:10<00:12, 20.65it/s]
avg reward: 1.16:  44%|████▍     | 206/468 [00:10<00:12, 20.65it/s]
avg reward: 1.16:  45%|████▌     | 211/468 [00:10<00:12, 20.27it/s]
avg reward: 1.16:  45%|████▌     | 211/468 [00:10<00:12, 20.27it/s]
avg reward: 1.16:  45%|████▌     | 211/468 [00:10<00:12, 20.27it/s]
avg reward: 1.16:  45%|████▌     | 211/468 [00:10<00:12, 20.27it/s]
avg reward: 1.16:  45%|████▌     | 211/468 [00:10<00:12, 20.27it/s]
avg reward: 1.17:  45%|████▌     | 211/468 [00:10<00:12, 20.27it/s]
avg reward: 1.17:  46%|████▌     | 216/468 [00:10<00:12, 20.00it/s]
avg reward: 1.17:  46%|████▌     | 216/468 [00:10<00:12, 20.00it/s]
avg reward: 1.17:  46%|████▌     | 216/468 [00:10<00:12, 20.00it/s]
avg reward: 1.17:  46%|████▌     | 216/468 [00:11<00:12, 20.00it/s]
avg reward: 1.17:  46%|████▌     | 216/468 [00:11<00:12, 20.00it/s]
avg reward: 1.18:  46%|████▌     | 216/468 [00:11<00:12, 20.00it/s]
avg reward: 1.18:  47%|████▋     | 221/468 [00:11<00:12, 19.66it/s]
avg reward: 1.18:  47%|████▋     | 221/468 [00:11<00:12, 19.66it/s]
avg reward: 1.18:  47%|████▋     | 221/468 [00:11<00:12, 19.66it/s]
avg reward: 1.18:  47%|████▋     | 221/468 [00:11<00:12, 19.66it/s]
avg reward: 1.18:  47%|████▋     | 221/468 [00:11<00:12, 19.66it/s]
avg reward: 1.18:  47%|████▋     | 221/468 [00:11<00:12, 19.66it/s]
avg reward: 1.18:  48%|████▊     | 226/468 [00:11<00:12, 19.56it/s]
avg reward: 1.18:  48%|████▊     | 226/468 [00:11<00:12, 19.56it/s]
avg reward: 1.18:  48%|████▊     | 226/468 [00:11<00:12, 19.56it/s]
avg reward: 1.18:  48%|████▊     | 226/468 [00:11<00:12, 19.56it/s]
avg reward: 1.18:  48%|████▊     | 226/468 [00:11<00:12, 19.56it/s]
avg reward: 1.20:  48%|████▊     | 226/468 [00:11<00:12, 19.56it/s]
avg reward: 1.20:  49%|████▉     | 231/468 [00:11<00:12, 19.22it/s]
avg reward: 1.20:  49%|████▉     | 231/468 [00:11<00:12, 19.22it/s]
avg reward: 1.20:  49%|████▉     | 231/468 [00:11<00:12, 19.22it/s]
avg reward: 1.20:  49%|████▉     | 231/468 [00:11<00:12, 19.22it/s]
avg reward: 1.20:  49%|████▉     | 231/468 [00:11<00:12, 19.22it/s]
avg reward: 1.20:  50%|█████     | 235/468 [00:11<00:10, 22.45it/s]
avg reward: 1.22:  50%|█████     | 235/468 [00:11<00:10, 22.45it/s]
avg reward: 1.22:  50%|█████     | 235/468 [00:12<00:10, 22.45it/s]
avg reward: 1.22:  50%|█████     | 235/468 [00:12<00:10, 22.45it/s]
avg reward: 1.22:  51%|█████     | 238/468 [00:12<00:12, 19.16it/s]
avg reward: 1.22:  51%|█████     | 238/468 [00:12<00:12, 19.16it/s]
avg reward: 1.22:  51%|█████     | 238/468 [00:12<00:12, 19.16it/s]
avg reward: 1.22:  51%|█████     | 238/468 [00:12<00:12, 19.16it/s]
avg reward: 1.22:  51%|█████▏    | 241/468 [00:12<00:12, 17.63it/s]
avg reward: 1.22:  51%|█████▏    | 241/468 [00:12<00:12, 17.63it/s]
avg reward: 1.22:  51%|█████▏    | 241/468 [00:12<00:12, 17.63it/s]
avg reward: 1.22:  51%|█████▏    | 241/468 [00:12<00:12, 17.63it/s]
avg reward: 1.22:  51%|█████▏    | 241/468 [00:12<00:12, 17.63it/s]
avg reward: 1.22:  51%|█████▏    | 241/468 [00:12<00:12, 17.63it/s]
avg reward: 1.22:  53%|█████▎    | 246/468 [00:12<00:12, 18.42it/s]
avg reward: 1.22:  53%|█████▎    | 246/468 [00:12<00:12, 18.42it/s]
avg reward: 1.22:  53%|█████▎    | 246/468 [00:12<00:12, 18.42it/s]
avg reward: 1.22:  53%|█████▎    | 246/468 [00:12<00:12, 18.42it/s]
avg reward: 1.22:  53%|█████▎    | 246/468 [00:12<00:12, 18.42it/s]
avg reward: 1.24:  53%|█████▎    | 246/468 [00:12<00:12, 18.42it/s]
avg reward: 1.24:  54%|█████▎    | 251/468 [00:12<00:11, 18.91it/s]
avg reward: 1.24:  54%|█████▎    | 251/468 [00:12<00:11, 18.91it/s]
avg reward: 1.24:  54%|█████▎    | 251/468 [00:12<00:11, 18.91it/s]
avg reward: 1.24:  54%|█████▎    | 251/468 [00:12<00:11, 18.91it/s]
avg reward: 1.24:  54%|█████▎    | 251/468 [00:12<00:11, 18.91it/s]
avg reward: 1.25:  54%|█████▎    | 251/468 [00:12<00:11, 18.91it/s]
avg reward: 1.25:  55%|█████▍    | 256/468 [00:12<00:10, 19.52it/s]
avg reward: 1.25:  55%|█████▍    | 256/468 [00:12<00:10, 19.52it/s]
avg reward: 1.25:  55%|█████▍    | 256/468 [00:13<00:10, 19.52it/s]
avg reward: 1.25:  55%|█████▍    | 256/468 [00:13<00:10, 19.52it/s]
avg reward: 1.25:  55%|█████▍    | 256/468 [00:13<00:10, 19.52it/s]
avg reward: 1.25:  55%|█████▍    | 256/468 [00:13<00:10, 19.52it/s]
avg reward: 1.25:  56%|█████▌    | 261/468 [00:13<00:10, 20.03it/s]
avg reward: 1.25:  56%|█████▌    | 261/468 [00:13<00:10, 20.03it/s]
avg reward: 1.25:  56%|█████▌    | 261/468 [00:13<00:10, 20.03it/s]
avg reward: 1.25:  56%|█████▌    | 261/468 [00:13<00:10, 20.03it/s]
avg reward: 1.25:  56%|█████▌    | 261/468 [00:13<00:10, 20.03it/s]
avg reward: 1.25:  56%|█████▌    | 261/468 [00:13<00:10, 20.03it/s]
avg reward: 1.25:  57%|█████▋    | 266/468 [00:13<00:10, 20.13it/s]
avg reward: 1.25:  57%|█████▋    | 266/468 [00:13<00:10, 20.13it/s]
avg reward: 1.25:  57%|█████▋    | 266/468 [00:13<00:10, 20.13it/s]
avg reward: 1.25:  57%|█████▋    | 266/468 [00:13<00:10, 20.13it/s]
avg reward: 1.25:  57%|█████▋    | 266/468 [00:13<00:10, 20.13it/s]
avg reward: 1.24:  57%|█████▋    | 266/468 [00:13<00:10, 20.13it/s]
avg reward: 1.24:  58%|█████▊    | 271/468 [00:13<00:09, 20.38it/s]
avg reward: 1.24:  58%|█████▊    | 271/468 [00:13<00:09, 20.38it/s]
avg reward: 1.24:  58%|█████▊    | 271/468 [00:13<00:09, 20.38it/s]
avg reward: 1.24:  58%|█████▊    | 271/468 [00:13<00:09, 20.38it/s]
avg reward: 1.24:  58%|█████▊    | 271/468 [00:13<00:09, 20.38it/s]
avg reward: 1.23:  58%|█████▊    | 271/468 [00:13<00:09, 20.38it/s]
avg reward: 1.23:  59%|█████▉    | 276/468 [00:13<00:09, 20.11it/s]
avg reward: 1.23:  59%|█████▉    | 276/468 [00:13<00:09, 20.11it/s]
avg reward: 1.23:  59%|█████▉    | 276/468 [00:13<00:09, 20.11it/s]
avg reward: 1.23:  59%|█████▉    | 276/468 [00:14<00:09, 20.11it/s]
avg reward: 1.23:  59%|█████▉    | 276/468 [00:14<00:09, 20.11it/s]
avg reward: 1.23:  60%|█████▉    | 280/468 [00:14<00:08, 23.49it/s]
avg reward: 1.24:  60%|█████▉    | 280/468 [00:14<00:08, 23.49it/s]
avg reward: 1.24:  60%|█████▉    | 280/468 [00:14<00:08, 23.49it/s]
avg reward: 1.24:  60%|█████▉    | 280/468 [00:14<00:08, 23.49it/s]
avg reward: 1.24:  60%|██████    | 283/468 [00:14<00:09, 19.08it/s]
avg reward: 1.24:  60%|██████    | 283/468 [00:14<00:09, 19.08it/s]
avg reward: 1.24:  60%|██████    | 283/468 [00:14<00:09, 19.08it/s]
avg reward: 1.24:  60%|██████    | 283/468 [00:14<00:09, 19.08it/s]
avg reward: 1.24:  61%|██████    | 286/468 [00:14<00:10, 17.70it/s]
avg reward: 1.24:  61%|██████    | 286/468 [00:14<00:10, 17.70it/s]
avg reward: 1.24:  61%|██████    | 286/468 [00:14<00:10, 17.70it/s]
avg reward: 1.24:  61%|██████    | 286/468 [00:14<00:10, 17.70it/s]
avg reward: 1.24:  61%|██████    | 286/468 [00:14<00:10, 17.70it/s]
avg reward: 1.24:  62%|██████▏   | 290/468 [00:14<00:08, 21.13it/s]
avg reward: 1.25:  62%|██████▏   | 290/468 [00:14<00:08, 21.13it/s]
avg reward: 1.25:  62%|██████▏   | 290/468 [00:14<00:08, 21.13it/s]
avg reward: 1.25:  62%|██████▏   | 290/468 [00:14<00:08, 21.13it/s]
avg reward: 1.25:  63%|██████▎   | 293/468 [00:14<00:09, 18.92it/s]
avg reward: 1.25:  63%|██████▎   | 293/468 [00:14<00:09, 18.92it/s]
avg reward: 1.25:  63%|██████▎   | 293/468 [00:14<00:09, 18.92it/s]
avg reward: 1.25:  63%|██████▎   | 293/468 [00:14<00:09, 18.92it/s]
avg reward: 1.25:  63%|██████▎   | 296/468 [00:14<00:09, 17.92it/s]
avg reward: 1.25:  63%|██████▎   | 296/468 [00:14<00:09, 17.92it/s]
avg reward: 1.25:  63%|██████▎   | 296/468 [00:15<00:09, 17.92it/s]
avg reward: 1.25:  63%|██████▎   | 296/468 [00:15<00:09, 17.92it/s]
avg reward: 1.25:  63%|██████▎   | 296/468 [00:15<00:09, 17.92it/s]
avg reward: 1.25:  64%|██████▍   | 300/468 [00:15<00:07, 21.41it/s]
avg reward: 1.26:  64%|██████▍   | 300/468 [00:15<00:07, 21.41it/s]
avg reward: 1.26:  64%|██████▍   | 300/468 [00:15<00:07, 21.41it/s]
avg reward: 1.26:  64%|██████▍   | 300/468 [00:15<00:07, 21.41it/s]
avg reward: 1.26:  65%|██████▍   | 303/468 [00:15<00:08, 18.87it/s]
avg reward: 1.26:  65%|██████▍   | 303/468 [00:15<00:08, 18.87it/s]
avg reward: 1.26:  65%|██████▍   | 303/468 [00:15<00:08, 18.87it/s]
avg reward: 1.26:  65%|██████▍   | 303/468 [00:15<00:08, 18.87it/s]
avg reward: 1.26:  65%|██████▌   | 306/468 [00:15<00:09, 17.89it/s]
avg reward: 1.26:  65%|██████▌   | 306/468 [00:15<00:09, 17.89it/s]
avg reward: 1.26:  65%|██████▌   | 306/468 [00:15<00:09, 17.89it/s]
avg reward: 1.26:  65%|██████▌   | 306/468 [00:15<00:09, 17.89it/s]
avg reward: 1.26:  65%|██████▌   | 306/468 [00:15<00:09, 17.89it/s]
avg reward: 1.25:  65%|██████▌   | 306/468 [00:15<00:09, 17.89it/s]
avg reward: 1.25:  66%|██████▋   | 311/468 [00:15<00:08, 18.62it/s]
avg reward: 1.25:  66%|██████▋   | 311/468 [00:15<00:08, 18.62it/s]
avg reward: 1.25:  66%|██████▋   | 311/468 [00:15<00:08, 18.62it/s]
avg reward: 1.25:  66%|██████▋   | 311/468 [00:15<00:08, 18.62it/s]
avg reward: 1.25:  66%|██████▋   | 311/468 [00:15<00:08, 18.62it/s]
avg reward: 1.25:  66%|██████▋   | 311/468 [00:15<00:08, 18.62it/s]
avg reward: 1.25:  68%|██████▊   | 316/468 [00:15<00:07, 19.46it/s]
avg reward: 1.25:  68%|██████▊   | 316/468 [00:15<00:07, 19.46it/s]
avg reward: 1.25:  68%|██████▊   | 316/468 [00:15<00:07, 19.46it/s]
avg reward: 1.25:  68%|██████▊   | 316/468 [00:16<00:07, 19.46it/s]
avg reward: 1.25:  68%|██████▊   | 316/468 [00:16<00:07, 19.46it/s]
avg reward: 1.25:  68%|██████▊   | 320/468 [00:16<00:06, 22.94it/s]
avg reward: 1.25:  68%|██████▊   | 320/468 [00:16<00:06, 22.94it/s]
avg reward: 1.25:  68%|██████▊   | 320/468 [00:16<00:06, 22.94it/s]
avg reward: 1.25:  68%|██████▊   | 320/468 [00:16<00:06, 22.94it/s]
avg reward: 1.25:  69%|██████▉   | 323/468 [00:16<00:07, 19.99it/s]
avg reward: 1.25:  69%|██████▉   | 323/468 [00:16<00:07, 19.99it/s]
avg reward: 1.25:  69%|██████▉   | 323/468 [00:16<00:07, 19.99it/s]
avg reward: 1.24:  69%|██████▉   | 323/468 [00:16<00:07, 19.99it/s]
avg reward: 1.24:  70%|██████▉   | 326/468 [00:16<00:07, 18.90it/s]
avg reward: 1.24:  70%|██████▉   | 326/468 [00:16<00:07, 18.90it/s]
avg reward: 1.24:  70%|██████▉   | 326/468 [00:16<00:07, 18.90it/s]
avg reward: 1.24:  70%|██████▉   | 326/468 [00:16<00:07, 18.90it/s]
avg reward: 1.24:  70%|██████▉   | 326/468 [00:16<00:07, 18.90it/s]
avg reward: 1.25:  70%|██████▉   | 326/468 [00:16<00:07, 18.90it/s]
avg reward: 1.25:  71%|███████   | 331/468 [00:16<00:07, 19.44it/s]
avg reward: 1.25:  71%|███████   | 331/468 [00:16<00:07, 19.44it/s]
avg reward: 1.25:  71%|███████   | 331/468 [00:16<00:07, 19.44it/s]
avg reward: 1.25:  71%|███████   | 331/468 [00:16<00:07, 19.44it/s]
avg reward: 1.25:  71%|███████   | 331/468 [00:16<00:07, 19.44it/s]
avg reward: 1.25:  71%|███████   | 331/468 [00:16<00:07, 19.44it/s]
avg reward: 1.25:  72%|███████▏  | 336/468 [00:16<00:06, 19.75it/s]
avg reward: 1.25:  72%|███████▏  | 336/468 [00:16<00:06, 19.75it/s]
avg reward: 1.25:  72%|███████▏  | 336/468 [00:16<00:06, 19.75it/s]
avg reward: 1.25:  72%|███████▏  | 336/468 [00:16<00:06, 19.75it/s]
avg reward: 1.25:  72%|███████▏  | 336/468 [00:16<00:06, 19.75it/s]
avg reward: 1.26:  72%|███████▏  | 336/468 [00:17<00:06, 19.75it/s]
avg reward: 1.26:  73%|███████▎  | 341/468 [00:17<00:06, 20.27it/s]
avg reward: 1.26:  73%|███████▎  | 341/468 [00:17<00:06, 20.27it/s]
avg reward: 1.26:  73%|███████▎  | 341/468 [00:17<00:06, 20.27it/s]
avg reward: 1.26:  73%|███████▎  | 341/468 [00:17<00:06, 20.27it/s]
avg reward: 1.26:  73%|███████▎  | 341/468 [00:17<00:06, 20.27it/s]
avg reward: 1.27:  73%|███████▎  | 341/468 [00:17<00:06, 20.27it/s]
avg reward: 1.27:  74%|███████▍  | 346/468 [00:17<00:06, 20.29it/s]
avg reward: 1.27:  74%|███████▍  | 346/468 [00:17<00:06, 20.29it/s]
avg reward: 1.27:  74%|███████▍  | 346/468 [00:17<00:06, 20.29it/s]
avg reward: 1.27:  74%|███████▍  | 346/468 [00:17<00:06, 20.29it/s]
avg reward: 1.27:  74%|███████▍  | 346/468 [00:17<00:06, 20.29it/s]
avg reward: 1.27:  74%|███████▍  | 346/468 [00:17<00:06, 20.29it/s]
avg reward: 1.27:  75%|███████▌  | 351/468 [00:17<00:05, 20.15it/s]
avg reward: 1.27:  75%|███████▌  | 351/468 [00:17<00:05, 20.15it/s]
avg reward: 1.27:  75%|███████▌  | 351/468 [00:17<00:05, 20.15it/s]
avg reward: 1.27:  75%|███████▌  | 351/468 [00:17<00:05, 20.15it/s]
avg reward: 1.27:  75%|███████▌  | 351/468 [00:17<00:05, 20.15it/s]
avg reward: 1.27:  75%|███████▌  | 351/468 [00:17<00:05, 20.15it/s]
avg reward: 1.27:  76%|███████▌  | 356/468 [00:17<00:05, 20.37it/s]
avg reward: 1.27:  76%|███████▌  | 356/468 [00:17<00:05, 20.37it/s]
avg reward: 1.27:  76%|███████▌  | 356/468 [00:17<00:05, 20.37it/s]
avg reward: 1.27:  76%|███████▌  | 356/468 [00:17<00:05, 20.37it/s]
avg reward: 1.27:  76%|███████▌  | 356/468 [00:17<00:05, 20.37it/s]
avg reward: 1.27:  76%|███████▌  | 356/468 [00:18<00:05, 20.37it/s]
avg reward: 1.27:  77%|███████▋  | 361/468 [00:18<00:05, 20.54it/s]
avg reward: 1.27:  77%|███████▋  | 361/468 [00:18<00:05, 20.54it/s]
avg reward: 1.27:  77%|███████▋  | 361/468 [00:18<00:05, 20.54it/s]
avg reward: 1.27:  77%|███████▋  | 361/468 [00:18<00:05, 20.54it/s]
avg reward: 1.27:  77%|███████▋  | 361/468 [00:18<00:05, 20.54it/s]
avg reward: 1.27:  77%|███████▋  | 361/468 [00:18<00:05, 20.54it/s]
avg reward: 1.27:  78%|███████▊  | 366/468 [00:18<00:04, 20.62it/s]
avg reward: 1.27:  78%|███████▊  | 366/468 [00:18<00:04, 20.62it/s]
avg reward: 1.27:  78%|███████▊  | 366/468 [00:18<00:04, 20.62it/s]
avg reward: 1.27:  78%|███████▊  | 366/468 [00:18<00:04, 20.62it/s]
avg reward: 1.27:  78%|███████▊  | 366/468 [00:18<00:04, 20.62it/s]
avg reward: 1.27:  78%|███████▊  | 366/468 [00:18<00:04, 20.62it/s]
avg reward: 1.27:  79%|███████▉  | 371/468 [00:18<00:04, 20.44it/s]
avg reward: 1.27:  79%|███████▉  | 371/468 [00:18<00:04, 20.44it/s]
avg reward: 1.27:  79%|███████▉  | 371/468 [00:18<00:04, 20.44it/s]
avg reward: 1.27:  79%|███████▉  | 371/468 [00:18<00:04, 20.44it/s]
avg reward: 1.27:  79%|███████▉  | 371/468 [00:18<00:04, 20.44it/s]
avg reward: 1.27:  79%|███████▉  | 371/468 [00:18<00:04, 20.44it/s]
avg reward: 1.27:  80%|████████  | 376/468 [00:18<00:04, 20.17it/s]
avg reward: 1.27:  80%|████████  | 376/468 [00:18<00:04, 20.17it/s]
avg reward: 1.27:  80%|████████  | 376/468 [00:18<00:04, 20.17it/s]
avg reward: 1.27:  80%|████████  | 376/468 [00:18<00:04, 20.17it/s]
avg reward: 1.27:  80%|████████  | 376/468 [00:18<00:04, 20.17it/s]
avg reward: 1.27:  81%|████████  | 380/468 [00:18<00:03, 23.50it/s]
avg reward: 1.27:  81%|████████  | 380/468 [00:19<00:03, 23.50it/s]
avg reward: 1.27:  81%|████████  | 380/468 [00:19<00:03, 23.50it/s]
avg reward: 1.27:  81%|████████  | 380/468 [00:19<00:03, 23.50it/s]
avg reward: 1.27:  82%|████████▏ | 383/468 [00:19<00:04, 20.31it/s]
avg reward: 1.27:  82%|████████▏ | 383/468 [00:19<00:04, 20.31it/s]
avg reward: 1.27:  82%|████████▏ | 383/468 [00:19<00:04, 20.31it/s]
avg reward: 1.28:  82%|████████▏ | 383/468 [00:19<00:04, 20.31it/s]
avg reward: 1.28:  82%|████████▏ | 386/468 [00:19<00:04, 17.96it/s]
avg reward: 1.28:  82%|████████▏ | 386/468 [00:19<00:04, 17.96it/s]
avg reward: 1.28:  82%|████████▏ | 386/468 [00:19<00:04, 17.96it/s]
avg reward: 1.28:  82%|████████▏ | 386/468 [00:19<00:04, 17.96it/s]
avg reward: 1.28:  82%|████████▏ | 386/468 [00:19<00:04, 17.96it/s]
avg reward: 1.27:  82%|████████▏ | 386/468 [00:19<00:04, 17.96it/s]
avg reward: 1.27:  84%|████████▎ | 391/468 [00:19<00:04, 18.72it/s]
avg reward: 1.27:  84%|████████▎ | 391/468 [00:19<00:04, 18.72it/s]
avg reward: 1.27:  84%|████████▎ | 391/468 [00:19<00:04, 18.72it/s]
avg reward: 1.27:  84%|████████▎ | 391/468 [00:19<00:04, 18.72it/s]
avg reward: 1.27:  84%|████████▎ | 391/468 [00:19<00:04, 18.72it/s]
avg reward: 1.27:  84%|████████▍ | 395/468 [00:19<00:03, 22.16it/s]
avg reward: 1.27:  84%|████████▍ | 395/468 [00:19<00:03, 22.16it/s]
avg reward: 1.27:  84%|████████▍ | 395/468 [00:19<00:03, 22.16it/s]
avg reward: 1.27:  84%|████████▍ | 395/468 [00:19<00:03, 22.16it/s]
avg reward: 1.27:  85%|████████▌ | 398/468 [00:19<00:03, 19.74it/s]
avg reward: 1.27:  85%|████████▌ | 398/468 [00:19<00:03, 19.74it/s]
avg reward: 1.27:  85%|████████▌ | 398/468 [00:19<00:03, 19.74it/s]
avg reward: 1.26:  85%|████████▌ | 398/468 [00:20<00:03, 19.74it/s]
avg reward: 1.26:  86%|████████▌ | 401/468 [00:20<00:03, 18.53it/s]
avg reward: 1.26:  86%|████████▌ | 401/468 [00:20<00:03, 18.53it/s]
avg reward: 1.26:  86%|████████▌ | 401/468 [00:20<00:03, 18.53it/s]
avg reward: 1.26:  86%|████████▌ | 401/468 [00:20<00:03, 18.53it/s]
avg reward: 1.26:  86%|████████▌ | 401/468 [00:20<00:03, 18.53it/s]
avg reward: 1.26:  86%|████████▌ | 401/468 [00:20<00:03, 18.53it/s]
avg reward: 1.26:  87%|████████▋ | 406/468 [00:20<00:03, 19.05it/s]
avg reward: 1.26:  87%|████████▋ | 406/468 [00:20<00:03, 19.05it/s]
avg reward: 1.26:  87%|████████▋ | 406/468 [00:20<00:03, 19.05it/s]
avg reward: 1.26:  87%|████████▋ | 406/468 [00:20<00:03, 19.05it/s]
avg reward: 1.26:  87%|████████▋ | 406/468 [00:20<00:03, 19.05it/s]
avg reward: 1.26:  88%|████████▊ | 410/468 [00:20<00:02, 22.51it/s]
avg reward: 1.28:  88%|████████▊ | 410/468 [00:20<00:02, 22.51it/s]
avg reward: 1.28:  88%|████████▊ | 410/468 [00:20<00:02, 22.51it/s]
avg reward: 1.28:  88%|████████▊ | 410/468 [00:20<00:02, 22.51it/s]
avg reward: 1.28:  88%|████████▊ | 413/468 [00:20<00:02, 19.66it/s]
avg reward: 1.28:  88%|████████▊ | 413/468 [00:20<00:02, 19.66it/s]
avg reward: 1.28:  88%|████████▊ | 413/468 [00:20<00:02, 19.66it/s]
avg reward: 1.28:  88%|████████▊ | 413/468 [00:20<00:02, 19.66it/s]
avg reward: 1.28:  89%|████████▉ | 416/468 [00:20<00:02, 18.15it/s]
avg reward: 1.28:  89%|████████▉ | 416/468 [00:20<00:02, 18.15it/s]
avg reward: 1.28:  89%|████████▉ | 416/468 [00:20<00:02, 18.15it/s]
avg reward: 1.28:  89%|████████▉ | 416/468 [00:20<00:02, 18.15it/s]
avg reward: 1.28:  89%|████████▉ | 416/468 [00:20<00:02, 18.15it/s]
avg reward: 1.28:  89%|████████▉ | 416/468 [00:21<00:02, 18.15it/s]
avg reward: 1.28:  90%|████████▉ | 421/468 [00:21<00:02, 19.02it/s]
avg reward: 1.28:  90%|████████▉ | 421/468 [00:21<00:02, 19.02it/s]
avg reward: 1.28:  90%|████████▉ | 421/468 [00:21<00:02, 19.02it/s]
avg reward: 1.28:  90%|████████▉ | 421/468 [00:21<00:02, 19.02it/s]
avg reward: 1.28:  90%|████████▉ | 421/468 [00:21<00:02, 19.02it/s]
avg reward: 1.29:  90%|████████▉ | 421/468 [00:21<00:02, 19.02it/s]
avg reward: 1.29:  91%|█████████ | 426/468 [00:21<00:02, 19.75it/s]
avg reward: 1.29:  91%|█████████ | 426/468 [00:21<00:02, 19.75it/s]
avg reward: 1.29:  91%|█████████ | 426/468 [00:21<00:02, 19.75it/s]
avg reward: 1.29:  91%|█████████ | 426/468 [00:21<00:02, 19.75it/s]
avg reward: 1.29:  91%|█████████ | 426/468 [00:21<00:02, 19.75it/s]
avg reward: 1.28:  91%|█████████ | 426/468 [00:21<00:02, 19.75it/s]
avg reward: 1.28:  92%|█████████▏| 431/468 [00:21<00:01, 19.95it/s]
avg reward: 1.28:  92%|█████████▏| 431/468 [00:21<00:01, 19.95it/s]
avg reward: 1.28:  92%|█████████▏| 431/468 [00:21<00:01, 19.95it/s]
avg reward: 1.28:  92%|█████████▏| 431/468 [00:21<00:01, 19.95it/s]
avg reward: 1.28:  92%|█████████▏| 431/468 [00:21<00:01, 19.95it/s]
avg reward: 1.28:  92%|█████████▏| 431/468 [00:21<00:01, 19.95it/s]
avg reward: 1.28:  93%|█████████▎| 436/468 [00:21<00:01, 20.18it/s]
avg reward: 1.28:  93%|█████████▎| 436/468 [00:21<00:01, 20.18it/s]
avg reward: 1.28:  93%|█████████▎| 436/468 [00:21<00:01, 20.18it/s]
avg reward: 1.28:  93%|█████████▎| 436/468 [00:21<00:01, 20.18it/s]
avg reward: 1.28:  93%|█████████▎| 436/468 [00:21<00:01, 20.18it/s]
avg reward: 1.29:  93%|█████████▎| 436/468 [00:22<00:01, 20.18it/s]
avg reward: 1.29:  94%|█████████▍| 441/468 [00:22<00:01, 20.35it/s]
avg reward: 1.29:  94%|█████████▍| 441/468 [00:22<00:01, 20.35it/s]
avg reward: 1.29:  94%|█████████▍| 441/468 [00:22<00:01, 20.35it/s]
avg reward: 1.29:  94%|█████████▍| 441/468 [00:22<00:01, 20.35it/s]
avg reward: 1.29:  94%|█████████▍| 441/468 [00:22<00:01, 20.35it/s]
avg reward: 1.29:  94%|█████████▍| 441/468 [00:22<00:01, 20.35it/s]
avg reward: 1.29:  95%|█████████▌| 446/468 [00:22<00:01, 20.01it/s]
avg reward: 1.29:  95%|█████████▌| 446/468 [00:22<00:01, 20.01it/s]
avg reward: 1.29:  95%|█████████▌| 446/468 [00:22<00:01, 20.01it/s]
avg reward: 1.29:  95%|█████████▌| 446/468 [00:22<00:01, 20.01it/s]
avg reward: 1.29:  95%|█████████▌| 446/468 [00:22<00:01, 20.01it/s]
avg reward: 1.29:  95%|█████████▌| 446/468 [00:22<00:01, 20.01it/s]
avg reward: 1.29:  96%|█████████▋| 451/468 [00:22<00:00, 20.37it/s]
avg reward: 1.29:  96%|█████████▋| 451/468 [00:22<00:00, 20.37it/s]
avg reward: 1.29:  96%|█████████▋| 451/468 [00:22<00:00, 20.37it/s]
avg reward: 1.29:  96%|█████████▋| 451/468 [00:22<00:00, 20.37it/s]
avg reward: 1.29:  96%|█████████▋| 451/468 [00:22<00:00, 20.37it/s]
avg reward: 1.29:  96%|█████████▋| 451/468 [00:22<00:00, 20.37it/s]
avg reward: 1.29:  97%|█████████▋| 456/468 [00:22<00:00, 20.94it/s]
avg reward: 1.29:  97%|█████████▋| 456/468 [00:22<00:00, 20.94it/s]
avg reward: 1.29:  97%|█████████▋| 456/468 [00:22<00:00, 20.94it/s]
avg reward: 1.29:  97%|█████████▋| 456/468 [00:22<00:00, 20.94it/s]
avg reward: 1.29:  97%|█████████▋| 456/468 [00:22<00:00, 20.94it/s]
avg reward: 1.28:  97%|█████████▋| 456/468 [00:22<00:00, 20.94it/s]
avg reward: 1.28:  99%|█████████▊| 461/468 [00:22<00:00, 21.39it/s]
avg reward: 1.28:  99%|█████████▊| 461/468 [00:22<00:00, 21.39it/s]
avg reward: 1.28:  99%|█████████▊| 461/468 [00:22<00:00, 21.39it/s]
avg reward: 1.28:  99%|█████████▊| 461/468 [00:23<00:00, 21.39it/s]
avg reward: 1.28:  99%|█████████▊| 461/468 [00:23<00:00, 21.39it/s]
avg reward: 1.28:  99%|█████████▊| 461/468 [00:23<00:00, 21.39it/s]
avg reward: 1.28: 100%|█████████▉| 466/468 [00:23<00:00, 21.28it/s]
avg reward: 1.28: 100%|█████████▉| 466/468 [00:23<00:00, 21.28it/s]
avg reward: 1.28: 100%|██████████| 468/468 [00:23<00:00, 20.15it/s]

  0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.96875:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.96875:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.9739583333333334:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.974609375:   0%|          | 0/79 [00:00<?, ?it/s]       
Val Acc: 0.971875:   0%|          | 0/79 [00:00<?, ?it/s]   
Val Acc: 0.9739583333333334:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.9743303571428571:   0%|          | 0/79 [00:00<?, ?it/s]
Val Acc: 0.9736328125:   0%|          | 0/79 [00:00<?, ?it/s]      
Val Acc: 0.9736328125:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]
Val Acc: 0.9739583333333334:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]
Val Acc: 0.97265625:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]        
Val Acc: 0.9708806818181818:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]
Val Acc: 0.970703125:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]       
Val Acc: 0.9711538461538461:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]
Val Acc: 0.9704241071428571:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]
Val Acc: 0.9708333333333333:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]
Val Acc: 0.97021484375:  10%|█         | 8/79 [00:00<00:00, 72.43it/s]     
Val Acc: 0.97021484375:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]
Val Acc: 0.9696691176470589:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]
Val Acc: 0.9704861111111112:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]
Val Acc: 0.9691611842105263:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]
Val Acc: 0.9703125:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]         
Val Acc: 0.9713541666666666:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]
Val Acc: 0.9715909090909091:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]
Val Acc: 0.9718070652173914:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]
Val Acc: 0.9716796875:  20%|██        | 16/79 [00:00<00:00, 73.38it/s]      
Val Acc: 0.9716796875:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]
Val Acc: 0.9721875:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]   
Val Acc: 0.9723557692307693:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]
Val Acc: 0.9710648148148148:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]
Val Acc: 0.9701450892857143:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]
Val Acc: 0.9709051724137931:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]
Val Acc: 0.9705729166666667:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]
Val Acc: 0.9712701612903226:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]
Val Acc: 0.97119140625:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]     
Val Acc: 0.9713541666666666:  30%|███       | 24/79 [00:00<00:00, 74.15it/s]
Val Acc: 0.9713541666666666:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.9710477941176471:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.9709821428571429:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.9704861111111112:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.9708614864864865:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.9708059210526315:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.9707532051282052:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.971484375:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]       
Val Acc: 0.9714176829268293:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.9715401785714286:  42%|████▏     | 33/79 [00:00<00:00, 76.70it/s]
Val Acc: 0.9715401785714286:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9712936046511628:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9708806818181818:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9710069444444445:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9707880434782609:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9707446808510638:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9710286458333334:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9708227040816326:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9709375:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]         
Val Acc: 0.9713541666666666:  53%|█████▎    | 42/79 [00:00<00:00, 77.23it/s]
Val Acc: 0.9713541666666666:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9716045673076923:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9714033018867925:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9716435185185185:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9710227272727273:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9711216517857143:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9709429824561403:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9709051724137931:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9710010593220338:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9709635416666667:  65%|██████▍   | 51/79 [00:00<00:00, 80.42it/s]
Val Acc: 0.9709635416666667:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.9711834016393442:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.9712701612903226:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.9713541666666666:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.971435546875:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]    
Val Acc: 0.9713942307692308:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.9711174242424242:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.9711986940298507:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.9712775735294118:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.9713541666666666:  76%|███████▌  | 60/79 [00:00<00:00, 77.02it/s]
Val Acc: 0.9713541666666666:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.9713169642857142:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.9713908450704225:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.9714626736111112:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.9716395547945206:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.9714949324324325:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.9716666666666667:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.9715254934210527:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.971489448051948:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s] 
Val Acc: 0.9713541666666666:  87%|████████▋ | 69/79 [00:00<00:00, 79.59it/s]
Val Acc: 0.9713541666666666:  99%|█████████▊| 78/79 [00:00<00:00, 79.20it/s]
Val Acc: 0.9714: 100%|██████████| 79/79 [00:00<00:00, 79.21it/s]
epoch 0, val_acc: 0.97, avg reward: 1.28: 100%|██████████| 1/1 [00:24<00:00, 24.26s/it]

The resulting architecture is:

mynet.graph
../../_images/output_enas_mnist_1830e6_25_0.svg

Reference

[1] Efficient Neural Architecture Search via Parameter Sharing H Pham, MY Guan, B Zoph, QV Le, J Dean International Conference on Machine Learning (ICML)

[3] ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware Han Cai, Ligeng Zhu, Song Han International Conference on Learning Representations (ICLR), 2019.