Demo RL Searcher

In this tutorial, we are going to compare RL searcher with random search in a simulation environment.

A Toy Reward Space

Input Space x = [0: 99], y = [0: 99]. The rewards are a combination of 2 gaussians as shown in the following figure:

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

Generate the simulation rewards as a mixture of 2 gaussians:

def gaussian(x, y, x0, y0, xalpha, yalpha, A):
    return A * np.exp( -((x-x0)/xalpha)**2 -((y-y0)/yalpha)**2)

x, y = np.linspace(0, 99, 100), np.linspace(0, 99, 100)
X, Y = np.meshgrid(x, y)

Z = np.zeros(X.shape)
ps = [(20, 70, 35, 40, 1),
      (80, 40, 20, 20, 0.7)]
for p in ps:
    Z += gaussian(X, Y, *p)

Visualize the reward space:

fig = plt.figure()
ax = fig.gca(projection='3d')
ax.plot_surface(X, Y, Z, cmap='plasma')
ax.set_zlim(0,np.max(Z)+2)
plt.show()
../../_images/output_rl_searcher_ecab16_5_0.png

Simulation Experiment

Customize Train Function

We can define any function with a decorator @ag.args, which converts the function to AutoGluon searchable. The reporter is used to communicate with AutoGluon search algorithms.

import autogluon as ag

@ag.args(
    x=ag.space.Categorical(*list(range(100))),
    y=ag.space.Categorical(*list(range(100))),
)
def rl_simulation(args, reporter):
    x, y = args.x, args.y
    reporter(accuracy=Z[y][x])
/var/lib/jenkins/miniconda3/envs/autogluon_docs/lib/python3.7/site-packages/sklearn/utils/deprecation.py:144: FutureWarning: The sklearn.metrics.classification module is  deprecated in version 0.22 and will be removed in version 0.24. The corresponding classes / functions should instead be imported from sklearn.metrics. Anything that cannot be imported from sklearn.metrics is now part of the private API.
  warnings.warn(message, FutureWarning)

Random Search Baseline

random_scheduler = ag.scheduler.FIFOScheduler(rl_simulation, rl_simulation.args,
                                              resource={'num_cpus': 1, 'num_gpus': 0},
                                              num_trials=300,
                                              reward_attr="accuracy",
                                              resume=False)
random_scheduler.run()
random_scheduler.join_jobs()
print('Best config: {}, best reward: {}'.format(random_scheduler.get_best_config(), random_scheduler.get_best_reward()))
Starting Experiments
Num of Finished Tasks is 0
Num of Pending Tasks is 300
HBox(children=(FloatProgress(value=0.0, max=300.0), HTML(value='')))
Finished Task with config: {'x.choice': 0, 'y.choice': 0} and reward: 0.033741370964360654
Finished Task with config: {'x.choice': 29, 'y.choice': 13} and reward: 0.12302503912251365
Finished Task with config: {'x.choice': 43, 'y.choice': 71} and reward: 0.6509766462937238
Finished Task with config: {'x.choice': 1, 'y.choice': 28} and reward: 0.24728999880361294
Finished Task with config: {'x.choice': 10, 'y.choice': 62} and reward: 0.8854745836123944
Finished Task with config: {'x.choice': 17, 'y.choice': 7} and reward: 0.08308054023116862
Finished Task with config: {'x.choice': 74, 'y.choice': 41} and reward: 0.692846703047434
Finished Task with config: {'x.choice': 50, 'y.choice': 93} and reward: 0.3446837276184217
Finished Task with config: {'x.choice': 79, 'y.choice': 82} and reward: 0.06179731027126467
Finished Task with config: {'x.choice': 93, 'y.choice': 48} and reward: 0.4004860397163461
Finished Task with config: {'x.choice': 87, 'y.choice': 15} and reward: 0.13367872682578
Finished Task with config: {'x.choice': 23, 'y.choice': 61} and reward: 0.9437454090924291
Finished Task with config: {'x.choice': 78, 'y.choice': 24} and reward: 0.3825329734250292
Finished Task with config: {'x.choice': 18, 'y.choice': 89} and reward: 0.7954158970191254
Finished Task with config: {'x.choice': 60, 'y.choice': 27} and reward: 0.2540631287220444
Finished Task with config: {'x.choice': 13, 'y.choice': 73} and reward: 0.9554007850670236
Finished Task with config: {'x.choice': 13, 'y.choice': 97} and reward: 0.6091900472888774
Finished Task with config: {'x.choice': 64, 'y.choice': 98} and reward: 0.1262164615398585
Finished Task with config: {'x.choice': 49, 'y.choice': 70} and reward: 0.509995662527042
Finished Task with config: {'x.choice': 82, 'y.choice': 8} and reward: 0.05749960605191611
Finished Task with config: {'x.choice': 2, 'y.choice': 9} and reward: 0.0750114442919767
Finished Task with config: {'x.choice': 20, 'y.choice': 84} and reward: 0.8847065880090775
Finished Task with config: {'x.choice': 58, 'y.choice': 17} and reward: 0.1087846652032669
Finished Task with config: {'x.choice': 85, 'y.choice': 84} and reward: 0.03331381821823629
Finished Task with config: {'x.choice': 82, 'y.choice': 33} and reward: 0.6315655803061208
Finished Task with config: {'x.choice': 48, 'y.choice': 91} and reward: 0.4003475343565478
Finished Task with config: {'x.choice': 82, 'y.choice': 63} and reward: 0.2267349351666571
Finished Task with config: {'x.choice': 80, 'y.choice': 17} and reward: 0.19567415733695726
Finished Task with config: {'x.choice': 27, 'y.choice': 22} and reward: 0.22791533124450625
Finished Task with config: {'x.choice': 3, 'y.choice': 92} and reward: 0.583670675609728
Finished Task with config: {'x.choice': 13, 'y.choice': 19} and reward: 0.189075558317391
Finished Task with config: {'x.choice': 2, 'y.choice': 68} and reward: 0.7656809089448
Finished Task with config: {'x.choice': 44, 'y.choice': 20} and reward: 0.14106617320214818
Finished Task with config: {'x.choice': 92, 'y.choice': 97} and reward: 0.009354976304869554
Finished Task with config: {'x.choice': 3, 'y.choice': 67} and reward: 0.7854147830268986
Finished Task with config: {'x.choice': 23, 'y.choice': 48} and reward: 0.7337362600095223
Finished Task with config: {'x.choice': 98, 'y.choice': 44} and reward: 0.30375685016509923
Finished Task with config: {'x.choice': 23, 'y.choice': 48} and reward: 0.7337362600095223
Finished Task with config: {'x.choice': 68, 'y.choice': 93} and reward: 0.10997807863457479
Finished Task with config: {'x.choice': 66, 'y.choice': 63} and reward: 0.2866659941103633
Finished Task with config: {'x.choice': 74, 'y.choice': 95} and reward: 0.06292969694631459
Finished Task with config: {'x.choice': 9, 'y.choice': 52} and reward: 0.7398755445448225
Finished Task with config: {'x.choice': 69, 'y.choice': 36} and reward: 0.5653871439201965
Finished Task with config: {'x.choice': 12, 'y.choice': 12} and reward: 0.1159337021935752
Finished Task with config: {'x.choice': 51, 'y.choice': 95} and reward: 0.30882795704109445
Finished Task with config: {'x.choice': 14, 'y.choice': 84} and reward: 0.8590848063963619
Finished Task with config: {'x.choice': 2, 'y.choice': 41} and reward: 0.4537928718809114
Finished Task with config: {'x.choice': 57, 'y.choice': 11} and reward: 0.059920214795622054
Finished Task with config: {'x.choice': 55, 'y.choice': 4} and reward: 0.02991989745871368
Finished Task with config: {'x.choice': 70, 'y.choice': 74} and reward: 0.15892781528405858
Finished Task with config: {'x.choice': 47, 'y.choice': 68} and reward: 0.55660833487708
Finished Task with config: {'x.choice': 62, 'y.choice': 39} and reward: 0.44057058538182536
Finished Task with config: {'x.choice': 65, 'y.choice': 59} and reward: 0.3392720689841773
Finished Task with config: {'x.choice': 58, 'y.choice': 22} and reward: 0.16575088934785642
Finished Task with config: {'x.choice': 11, 'y.choice': 43} and reward: 0.5934871967485988
Finished Task with config: {'x.choice': 59, 'y.choice': 96} and reward: 0.18944632279651397
Finished Task with config: {'x.choice': 66, 'y.choice': 57} and reward: 0.36815443831693867
Finished Task with config: {'x.choice': 29, 'y.choice': 32} and reward: 0.38050016743072923
Finished Task with config: {'x.choice': 43, 'y.choice': 81} and reward: 0.6023633986483371
Finished Task with config: {'x.choice': 37, 'y.choice': 53} and reward: 0.6638309413877119
Finished Task with config: {'x.choice': 5, 'y.choice': 15} and reward: 0.1256446546902696
Finished Task with config: {'x.choice': 16, 'y.choice': 72} and reward: 0.9845611583109444
Finished Task with config: {'x.choice': 74, 'y.choice': 61} and reward: 0.3003690856955615
Finished Task with config: {'x.choice': 44, 'y.choice': 6} and reward: 0.049829385470345024
Finished Task with config: {'x.choice': 3, 'y.choice': 12} and reward: 0.09648014973742529
Finished Task with config: {'x.choice': 97, 'y.choice': 5} and reward: 0.016460108808583444
Finished Task with config: {'x.choice': 16, 'y.choice': 37} and reward: 0.4997549934621582
Finished Task with config: {'x.choice': 15, 'y.choice': 36} and reward: 0.4757458036372656
Finished Task with config: {'x.choice': 73, 'y.choice': 90} and reward: 0.07982086011318584
Finished Task with config: {'x.choice': 35, 'y.choice': 32} and reward: 0.34128118043365707
Finished Task with config: {'x.choice': 88, 'y.choice': 63} and reward: 0.181200613796082
Finished Task with config: {'x.choice': 0, 'y.choice': 98} and reward: 0.44196233644183475
Finished Task with config: {'x.choice': 16, 'y.choice': 0} and reward: 0.04616417077649875
Finished Task with config: {'x.choice': 81, 'y.choice': 81} and reward: 0.05490407675355491
Finished Task with config: {'x.choice': 70, 'y.choice': 97} and reward: 0.08253942860835017
Finished Task with config: {'x.choice': 50, 'y.choice': 57} and reward: 0.46739553115286414
Finished Task with config: {'x.choice': 86, 'y.choice': 56} and reward: 0.36259919294403065
Finished Task with config: {'x.choice': 30, 'y.choice': 57} and reward: 0.829886169891878
Finished Task with config: {'x.choice': 14, 'y.choice': 59} and reward: 0.9003183316659336
Finished Task with config: {'x.choice': 97, 'y.choice': 7} and reward: 0.022995081918957477
Finished Task with config: {'x.choice': 36, 'y.choice': 22} and reward: 0.19470823967605447
Finished Task with config: {'x.choice': 43, 'y.choice': 33} and reward: 0.29617841663181754
Finished Task with config: {'x.choice': 73, 'y.choice': 51} and reward: 0.5382042316686375
Finished Task with config: {'x.choice': 1, 'y.choice': 61} and reward: 0.7079945928222398
Finished Task with config: {'x.choice': 17, 'y.choice': 16} and reward: 0.1604462590521143
Finished Task with config: {'x.choice': 80, 'y.choice': 11} and reward: 0.09151504811429659
Finished Task with config: {'x.choice': 53, 'y.choice': 86} and reward: 0.3508654094060075
Finished Task with config: {'x.choice': 46, 'y.choice': 98} and reward: 0.35281473637019606
Finished Task with config: {'x.choice': 6, 'y.choice': 17} and reward: 0.14724998418414145
Finished Task with config: {'x.choice': 22, 'y.choice': 22} and reward: 0.23622470581721916
Finished Task with config: {'x.choice': 13, 'y.choice': 43} and reward: 0.6091991952591707
Finished Task with config: {'x.choice': 83, 'y.choice': 26} and reward: 0.4309759173661692
Finished Task with config: {'x.choice': 89, 'y.choice': 14} and reward: 0.10837612871155422
Finished Task with config: {'x.choice': 33, 'y.choice': 65} and reward: 0.8582148651879908
Finished Task with config: {'x.choice': 21, 'y.choice': 88} and reward: 0.8160204383304456
Finished Task with config: {'x.choice': 97, 'y.choice': 30} and reward: 0.2676044025350516
Finished Task with config: {'x.choice': 76, 'y.choice': 66} and reward: 0.20063463240282592
Finished Task with config: {'x.choice': 73, 'y.choice': 7} and reward: 0.049143163838836046
Finished Task with config: {'x.choice': 42, 'y.choice': 81} and reward: 0.6248312828863247
Finished Task with config: {'x.choice': 88, 'y.choice': 34} and reward: 0.5553674220127564
Finished Task with config: {'x.choice': 19, 'y.choice': 14} and reward: 0.14075526057959076
Finished Task with config: {'x.choice': 26, 'y.choice': 46} and reward: 0.6779080466565959
Finished Task with config: {'x.choice': 64, 'y.choice': 65} and reward: 0.2800675466288951
Finished Task with config: {'x.choice': 58, 'y.choice': 10} and reward: 0.05442745595426403
Finished Task with config: {'x.choice': 99, 'y.choice': 59} and reward: 0.12081487603082532
Finished Task with config: {'x.choice': 70, 'y.choice': 54} and reward: 0.44469248457817034
Finished Task with config: {'x.choice': 70, 'y.choice': 27} and reward: 0.39820915463788653
Finished Task with config: {'x.choice': 22, 'y.choice': 45} and reward: 0.6745744317860629
Finished Task with config: {'x.choice': 14, 'y.choice': 20} and reward: 0.20354581430297314
Finished Task with config: {'x.choice': 73, 'y.choice': 49} and reward: 0.5824053095779436
Finished Task with config: {'x.choice': 29, 'y.choice': 22} and reward: 0.22223523352202473
Finished Task with config: {'x.choice': 65, 'y.choice': 50} and reward: 0.4597345732556535
Finished Task with config: {'x.choice': 60, 'y.choice': 96} and reward: 0.17763017250127666
Finished Task with config: {'x.choice': 91, 'y.choice': 17} and reward: 0.14065899437870624
Finished Task with config: {'x.choice': 64, 'y.choice': 82} and reward: 0.19265677919271945
Finished Task with config: {'x.choice': 8, 'y.choice': 76} and reward: 0.8693139460134877
Finished Task with config: {'x.choice': 87, 'y.choice': 53} and reward: 0.4272731949683524
Finished Task with config: {'x.choice': 34, 'y.choice': 81} and reward: 0.790129687954288
Finished Task with config: {'x.choice': 45, 'y.choice': 60} and reward: 0.5760424427609301
Finished Task with config: {'x.choice': 21, 'y.choice': 46} and reward: 0.6972133342511547
Finished Task with config: {'x.choice': 34, 'y.choice': 39} and reward: 0.47089464693609545
Finished Task with config: {'x.choice': 41, 'y.choice': 56} and reward: 0.6254749052678589
Finished Task with config: {'x.choice': 22, 'y.choice': 16} and reward: 0.16113123351480665
Finished Task with config: {'x.choice': 4, 'y.choice': 9} and reward: 0.07929310011549053
Finished Task with config: {'x.choice': 90, 'y.choice': 57} and reward: 0.281175275705863
Finished Task with config: {'x.choice': 93, 'y.choice': 88} and reward: 0.011984349174564691
Finished Task with config: {'x.choice': 75, 'y.choice': 10} and reward: 0.07823002434693944
Finished Task with config: {'x.choice': 72, 'y.choice': 97} and reward: 0.06991768141649937
Finished Task with config: {'x.choice': 58, 'y.choice': 42} and reward: 0.3951388302806597
Finished Task with config: {'x.choice': 2, 'y.choice': 80} and reward: 0.7210911042406029
Finished Task with config: {'x.choice': 26, 'y.choice': 36} and reward: 0.4719345830199497
Finished Task with config: {'x.choice': 62, 'y.choice': 60} and reward: 0.3371309272109969
Finished Task with config: {'x.choice': 30, 'y.choice': 99} and reward: 0.5448431952103038
Finished Task with config: {'x.choice': 85, 'y.choice': 83} and reward: 0.03505559268841201
Finished Task with config: {'x.choice': 50, 'y.choice': 30} and reward: 0.23391370765054031
Finished Task with config: {'x.choice': 80, 'y.choice': 25} and reward: 0.4137777109214211
Finished Task with config: {'x.choice': 1, 'y.choice': 1} and reward: 0.03799431222673659
Finished Task with config: {'x.choice': 53, 'y.choice': 92} and reward: 0.3039026228040344
Finished Task with config: {'x.choice': 21, 'y.choice': 80} and reward: 0.9386486383843003
Finished Task with config: {'x.choice': 45, 'y.choice': 78} and reward: 0.577717739735392
Finished Task with config: {'x.choice': 86, 'y.choice': 92} and reward: 0.021842966709407264
Finished Task with config: {'x.choice': 85, 'y.choice': 30} and reward: 0.5238214302202387
Finished Task with config: {'x.choice': 30, 'y.choice': 45} and reward: 0.6248622673008052
Finished Task with config: {'x.choice': 52, 'y.choice': 40} and reward: 0.345588192998109
Finished Task with config: {'x.choice': 79, 'y.choice': 7} and reward: 0.0507640619472289
Finished Task with config: {'x.choice': 83, 'y.choice': 17} and reward: 0.18914528559759028
Finished Task with config: {'x.choice': 9, 'y.choice': 2} and reward: 0.05034911458601803
Finished Task with config: {'x.choice': 91, 'y.choice': 15} and reward: 0.11089194375401298
Finished Task with config: {'x.choice': 9, 'y.choice': 2} and reward: 0.05034911458601803
Finished Task with config: {'x.choice': 25, 'y.choice': 29} and reward: 0.34292308114614106
Finished Task with config: {'x.choice': 10, 'y.choice': 19} and reward: 0.18136358228461952
Finished Task with config: {'x.choice': 63, 'y.choice': 77} and reward: 0.22546877215751743
Finished Task with config: {'x.choice': 55, 'y.choice': 31} and reward: 0.262015928613347
Finished Task with config: {'x.choice': 32, 'y.choice': 14} and reward: 0.1256435478077466
Finished Task with config: {'x.choice': 92, 'y.choice': 31} and reward: 0.4044621461066644
Finished Task with config: {'x.choice': 68, 'y.choice': 70} and reward: 0.2039397738677381
Finished Task with config: {'x.choice': 81, 'y.choice': 43} and reward: 0.713121092868598
Finished Task with config: {'x.choice': 7, 'y.choice': 92} and reward: 0.6437407592890484
Finished Task with config: {'x.choice': 89, 'y.choice': 20} and reward: 0.2146099882765347
Finished Task with config: {'x.choice': 88, 'y.choice': 51} and reward: 0.45910498463188143
Finished Task with config: {'x.choice': 9, 'y.choice': 41} and reward: 0.5355847571240971
Finished Task with config: {'x.choice': 98, 'y.choice': 46} and reward: 0.28945966697478853
Finished Task with config: {'x.choice': 17, 'y.choice': 75} and reward: 0.9772915134829253
Finished Task with config: {'x.choice': 62, 'y.choice': 81} and reward: 0.22432882046861616
Finished Task with config: {'x.choice': 69, 'y.choice': 53} and reward: 0.4566085299883192
Finished Task with config: {'x.choice': 32, 'y.choice': 4} and reward: 0.058509072002078895
Finished Task with config: {'x.choice': 51, 'y.choice': 92} and reward: 0.33732921921590503
Finished Task with config: {'x.choice': 24, 'y.choice': 47} and reward: 0.709395204415485
Finished Task with config: {'x.choice': 78, 'y.choice': 78} and reward: 0.08040771369425029
Finished Task with config: {'x.choice': 33, 'y.choice': 38} and reward: 0.46211174959160545
Finished Task with config: {'x.choice': 37, 'y.choice': 62} and reward: 0.7609263958754803
Finished Task with config: {'x.choice': 60, 'y.choice': 46} and reward: 0.4243299658509876
Finished Task with config: {'x.choice': 84, 'y.choice': 14} and reward: 0.12907237805643534
Finished Task with config: {'x.choice': 77, 'y.choice': 31} and reward: 0.5862067716993843
Finished Task with config: {'x.choice': 1, 'y.choice': 49} and reward: 0.5653452298173441
Finished Task with config: {'x.choice': 40, 'y.choice': 20} and reward: 0.15593488990234233
Finished Task with config: {'x.choice': 49, 'y.choice': 11} and reward: 0.06488295711418514
Finished Task with config: {'x.choice': 13, 'y.choice': 46} and reward: 0.670328599493369
Finished Task with config: {'x.choice': 38, 'y.choice': 44} and reward: 0.5112631870303967
Finished Task with config: {'x.choice': 41, 'y.choice': 74} and reward: 0.691602455550991
Finished Task with config: {'x.choice': 46, 'y.choice': 65} and reward: 0.575117300536389
Finished Task with config: {'x.choice': 98, 'y.choice': 56} and reward: 0.17036319395622118
Finished Task with config: {'x.choice': 86, 'y.choice': 91} and reward: 0.02263556196844715
Finished Task with config: {'x.choice': 19, 'y.choice': 92} and reward: 0.7383655688318538
Finished Task with config: {'x.choice': 80, 'y.choice': 43} and reward: 0.7179865334557008
Finished Task with config: {'x.choice': 40, 'y.choice': 6} and reward: 0.05648190259458204
Finished Task with config: {'x.choice': 90, 'y.choice': 9} and reward: 0.05112221118662261
Finished Task with config: {'x.choice': 43, 'y.choice': 4} and reward: 0.04356125419886951
Finished Task with config: {'x.choice': 39, 'y.choice': 97} and reward: 0.4722190572177832
Finished Task with config: {'x.choice': 65, 'y.choice': 10} and reward: 0.062218308685217126
Finished Task with config: {'x.choice': 71, 'y.choice': 75} and reward: 0.14452333090741692
Finished Task with config: {'x.choice': 89, 'y.choice': 44} and reward: 0.5627112015317133
Finished Task with config: {'x.choice': 60, 'y.choice': 53} and reward: 0.3948842256197249
Finished Task with config: {'x.choice': 7, 'y.choice': 59} and reward: 0.8076846282610318
Finished Task with config: {'x.choice': 50, 'y.choice': 83} and reward: 0.4322980014612179
Finished Task with config: {'x.choice': 81, 'y.choice': 10} and reward: 0.0786493614781162
Finished Task with config: {'x.choice': 63, 'y.choice': 12} and reward: 0.07487520002055202
Finished Task with config: {'x.choice': 6, 'y.choice': 65} and reward: 0.8389326904030064
Finished Task with config: {'x.choice': 78, 'y.choice': 81} and reward: 0.06986838445933771
Finished Task with config: {'x.choice': 74, 'y.choice': 94} and reward: 0.06498051764762768
Finished Task with config: {'x.choice': 31, 'y.choice': 56} and reward: 0.8024084642184506
Finished Task with config: {'x.choice': 25, 'y.choice': 70} and reward: 0.9798370077917251
Finished Task with config: {'x.choice': 40, 'y.choice': 43} and reward: 0.46995462696706974
Finished Task with config: {'x.choice': 49, 'y.choice': 97} and reward: 0.3191491581292934
Finished Task with config: {'x.choice': 10, 'y.choice': 89} and reward: 0.7354610909760133
Finished Task with config: {'x.choice': 42, 'y.choice': 44} and reward: 0.4596828490814722
Finished Task with config: {'x.choice': 21, 'y.choice': 40} and reward: 0.5694342045978888
Finished Task with config: {'x.choice': 32, 'y.choice': 2} and reward: 0.04947221138187314
Finished Task with config: {'x.choice': 21, 'y.choice': 5} and reward: 0.07126392906111646
Finished Task with config: {'x.choice': 4, 'y.choice': 31} and reward: 0.31361046654204716
Finished Task with config: {'x.choice': 10, 'y.choice': 17} and reward: 0.15925444888355014
Finished Task with config: {'x.choice': 84, 'y.choice': 80} and reward: 0.04548610644055609
Finished Task with config: {'x.choice': 17, 'y.choice': 13} and reward: 0.13029823486120112
Finished Task with config: {'x.choice': 58, 'y.choice': 82} and reward: 0.28371295663184704
Finished Task with config: {'x.choice': 85, 'y.choice': 86} and reward: 0.030394871693420502
Finished Task with config: {'x.choice': 20, 'y.choice': 80} and reward: 0.9394146450440607
Finished Task with config: {'x.choice': 18, 'y.choice': 39} and reward: 0.5467275596132934
Finished Task with config: {'x.choice': 60, 'y.choice': 22} and reward: 0.17873412172437247
Finished Task with config: {'x.choice': 35, 'y.choice': 81} and reward: 0.7716589683888583
Finished Task with config: {'x.choice': 98, 'y.choice': 30} and reward: 0.24508218569398543
Finished Task with config: {'x.choice': 58, 'y.choice': 99} and reward: 0.18191607551749792
Finished Task with config: {'x.choice': 32, 'y.choice': 83} and reward: 0.7999957039636457
Finished Task with config: {'x.choice': 39, 'y.choice': 56} and reward: 0.6644142807408967
Finished Task with config: {'x.choice': 9, 'y.choice': 66} and reward: 0.8969321813783663
Finished Task with config: {'x.choice': 51, 'y.choice': 40} and reward: 0.345527271124613
Finished Task with config: {'x.choice': 41, 'y.choice': 48} and reward: 0.5288716811907156
Finished Task with config: {'x.choice': 97, 'y.choice': 87} and reward: 0.007958491574296973
Finished Task with config: {'x.choice': 46, 'y.choice': 24} and reward: 0.17397015672037075
Finished Task with config: {'x.choice': 45, 'y.choice': 85} and reward: 0.5218203688785258
Finished Task with config: {'x.choice': 81, 'y.choice': 92} and reward: 0.036244584064054314
Finished Task with config: {'x.choice': 50, 'y.choice': 45} and reward: 0.39385834533335085
Finished Task with config: {'x.choice': 21, 'y.choice': 9} and reward: 0.09765313356470433
Finished Task with config: {'x.choice': 50, 'y.choice': 45} and reward: 0.39385834533335085
Finished Task with config: {'x.choice': 41, 'y.choice': 39} and reward: 0.39823509070580687
Finished Task with config: {'x.choice': 68, 'y.choice': 1} and reward: 0.01867612589820212
Finished Task with config: {'x.choice': 76, 'y.choice': 94} and reward: 0.054392588811343984
Finished Task with config: {'x.choice': 68, 'y.choice': 71} and reward: 0.19656394760261928
Finished Task with config: {'x.choice': 56, 'y.choice': 47} and reward: 0.39615617027831773
Finished Task with config: {'x.choice': 16, 'y.choice': 18} and reward: 0.18213259850683555
Finished Task with config: {'x.choice': 78, 'y.choice': 0} and reward: 0.01569493869627951
Finished Task with config: {'x.choice': 75, 'y.choice': 98} and reward: 0.0519970517313986
Finished Task with config: {'x.choice': 63, 'y.choice': 21} and reward: 0.18712922268701532
Finished Task with config: {'x.choice': 77, 'y.choice': 89} and reward: 0.05794674914573816
Finished Task with config: {'x.choice': 3, 'y.choice': 0} and reward: 0.03694155386395536
Finished Task with config: {'x.choice': 77, 'y.choice': 96} and reward: 0.0464708502272967
Finished Task with config: {'x.choice': 3, 'y.choice': 59} and reward: 0.7323159549303344
Finished Task with config: {'x.choice': 35, 'y.choice': 6} and reward: 0.0645798319630886
Finished Task with config: {'x.choice': 22, 'y.choice': 8} and reward: 0.09020848854087422
Finished Task with config: {'x.choice': 14, 'y.choice': 49} and reward: 0.7371246306323758
Finished Task with config: {'x.choice': 2, 'y.choice': 91} and reward: 0.5826813593530957
Finished Task with config: {'x.choice': 86, 'y.choice': 17} and reward: 0.17540788647266262
Finished Task with config: {'x.choice': 9, 'y.choice': 23} and reward: 0.2277753389733464
Finished Task with config: {'x.choice': 41, 'y.choice': 18} and reward: 0.133392876401845
Finished Task with config: {'x.choice': 99, 'y.choice': 5} and reward: 0.013714738601385047
Finished Task with config: {'x.choice': 86, 'y.choice': 40} and reward: 0.6560220573153291
Finished Task with config: {'x.choice': 93, 'y.choice': 62} and reward: 0.149206452090719
Finished Task with config: {'x.choice': 87, 'y.choice': 63} and reward: 0.18986685502904702
Finished Task with config: {'x.choice': 94, 'y.choice': 83} and reward: 0.014512466050164489
Finished Task with config: {'x.choice': 68, 'y.choice': 39} and reward: 0.5707766307776807
Finished Task with config: {'x.choice': 23, 'y.choice': 64} and reward: 0.9706433042491662
Finished Task with config: {'x.choice': 22, 'y.choice': 86} and reward: 0.8493666022807681
Finished Task with config: {'x.choice': 5, 'y.choice': 22} and reward: 0.19717330114728845
Finished Task with config: {'x.choice': 19, 'y.choice': 59} and reward: 0.9264331576743946
Finished Task with config: {'x.choice': 53, 'y.choice': 53} and reward: 0.4172934508658618
Finished Task with config: {'x.choice': 99, 'y.choice': 71} and reward: 0.03181480137127428
Finished Task with config: {'x.choice': 29, 'y.choice': 93} and reward: 0.6725047816970916
Finished Task with config: {'x.choice': 67, 'y.choice': 40} and reward: 0.552661805928224
Finished Task with config: {'x.choice': 74, 'y.choice': 43} and reward: 0.6841760343140092
Finished Task with config: {'x.choice': 75, 'y.choice': 26} and reward: 0.42809488832599496
Finished Task with config: {'x.choice': 87, 'y.choice': 99} and reward: 0.015247454264708742
Finished Task with config: {'x.choice': 96, 'y.choice': 66} and reward: 0.07697683174389325
Finished Task with config: {'x.choice': 39, 'y.choice': 26} and reward: 0.2284998924011785
Finished Task with config: {'x.choice': 14, 'y.choice': 92} and reward: 0.7175678767943938
Finished Task with config: {'x.choice': 19, 'y.choice': 4} and reward: 0.06565915418868676
Finished Task with config: {'x.choice': 6, 'y.choice': 43} and reward: 0.5403038762738565
Finished Task with config: {'x.choice': 75, 'y.choice': 49} and reward: 0.6012916464391939
Finished Task with config: {'x.choice': 82, 'y.choice': 32} and reward: 0.6081547654159908
Finished Task with config: {'x.choice': 22, 'y.choice': 72} and reward: 0.9942633286025881
Finished Task with config: {'x.choice': 89, 'y.choice': 46} and reward: 0.5367904795387466
Finished Task with config: {'x.choice': 41, 'y.choice': 35} and reward: 0.3391236686676639
Finished Task with config: {'x.choice': 71, 'y.choice': 32} and reward: 0.535674693463521
Finished Task with config: {'x.choice': 36, 'y.choice': 82} and reward: 0.7416418216113805
Finished Task with config: {'x.choice': 50, 'y.choice': 48} and reward: 0.4173186181878839
Finished Task with config: {'x.choice': 86, 'y.choice': 68} and reward: 0.11859827258893028
Finished Task with config: {'x.choice': 86, 'y.choice': 98} and reward: 0.017636059554661514
Finished Task with config: {'x.choice': 61, 'y.choice': 52} and reward: 0.4051224114464923
Finished Task with config: {'x.choice': 73, 'y.choice': 91} and reward: 0.07756490833021186
Finished Task with config: {'x.choice': 73, 'y.choice': 18} and reward: 0.20330035162896018
Finished Task with config: {'x.choice': 82, 'y.choice': 69} and reward: 0.127998796844626
Finished Task with config: {'x.choice': 99, 'y.choice': 69} and reward: 0.040802481115714426
Finished Task with config: {'x.choice': 8, 'y.choice': 37} and reward: 0.4501508890323689
Finished Task with config: {'x.choice': 74, 'y.choice': 37} and reward: 0.6723574205676176
Finished Task with config: {'x.choice': 35, 'y.choice': 17} and reward: 0.14498546581437186
Finished Task with config: {'x.choice': 44, 'y.choice': 94} and reward: 0.43597901158822167
Finished Task with config: {'x.choice': 11, 'y.choice': 20} and reward: 0.19620140736873487
Finished Task with config: {'x.choice': 22, 'y.choice': 11} and reward: 0.11318607783174965
Finished Task with config: {'x.choice': 93, 'y.choice': 54} and reward: 0.29205963504103777
Finished Task with config: {'x.choice': 89, 'y.choice': 26} and reward: 0.3563445217873367
Finished Task with config: {'x.choice': 30, 'y.choice': 47} and reward: 0.6633491670786982
Finished Task with config: {'x.choice': 51, 'y.choice': 46} and reward: 0.3965324326933921
Finished Task with config: {'x.choice': 37, 'y.choice': 76} and reward: 0.772541508368217
Finished Task with config: {'x.choice': 79, 'y.choice': 5} and reward: 0.036817616720617986
Finished Task with config: {'x.choice': 76, 'y.choice': 79} and reward: 0.08849655314973445
Best config: {'x.choice': 22, 'y.choice': 72}, best reward: 0.9942633286025881

Reinforcement Learning

rl_scheduler = ag.scheduler.RLScheduler(rl_simulation,
                                        resource={'num_cpus': 1, 'num_gpus': 0},
                                        num_trials=300,
                                        reward_attr="accuracy",
                                        controller_batch_size=4,
                                        controller_lr=5e-3,
                                        checkpoint='./rl_exp/checkerpoint.ag',
                                        resume=False,
                                        sync=True)
rl_scheduler.run()
rl_scheduler.join_jobs()
print('Best config: {}, best reward: {}'.format(rl_scheduler.get_best_config(), rl_scheduler.get_best_reward()))
Reserved DistributedResource(
    Node = Remote REMOTE_ID: 0,
    <Remote: 'inproc://172.31.45.231/20881/1' processes=1 threads=8, memory=33.24 GB>
    nCPUs = 0) in Remote REMOTE_ID: 0,
    <Remote: 'inproc://172.31.45.231/20881/1' processes=1 threads=8, memory=33.24 GB>
Starting Experiments
Num of Finished Tasks is 0
Num of Pending Tasks is 300
  0%|          | 0/76 [00:00<?, ?it/s]Finished Task with config: {'x.choice': 54, 'y.choice': 60} and reward: 0.4131320021194963
Finished Task with config: {'x.choice': 59, 'y.choice': 85} and reward: 0.25248241948989986
Finished Task with config: {'x.choice': 71, 'y.choice': 54} and reward: 0.452177348673922
Finished Task with config: {'x.choice': 84, 'y.choice': 84} and reward: 0.03655423885462522
  1%|▏         | 1/76 [00:00<00:37,  2.01it/s]Finished Task with config: {'x.choice': 62, 'y.choice': 29} and reward: 0.3129734289166378
Finished Task with config: {'x.choice': 42, 'y.choice': 43} and reward: 0.44561928050295485
Finished Task with config: {'x.choice': 64, 'y.choice': 89} and reward: 0.16521724999224768
Finished Task with config: {'x.choice': 38, 'y.choice': 5} and reward: 0.055140459703497166
  3%|▎         | 2/76 [00:00<00:33,  2.20it/s]Finished Task with config: {'x.choice': 27, 'y.choice': 80} and reward: 0.9025895808289532
Finished Task with config: {'x.choice': 96, 'y.choice': 79} and reward: 0.01675323386032184
Finished Task with config: {'x.choice': 38, 'y.choice': 52} and reward: 0.6328227532925973
Finished Task with config: {'x.choice': 47, 'y.choice': 47} and reward: 0.4369372160417949
  4%|▍         | 3/76 [00:01<00:30,  2.38it/s]Finished Task with config: {'x.choice': 56, 'y.choice': 7} and reward: 0.039952445147950336
Finished Task with config: {'x.choice': 39, 'y.choice': 33} and reward: 0.3257993021856096
Finished Task with config: {'x.choice': 92, 'y.choice': 8} and reward: 0.03906803139561688
Finished Task with config: {'x.choice': 83, 'y.choice': 64} and reward: 0.2004520333101668
  5%|▌         | 4/76 [00:01<00:28,  2.49it/s]Finished Task with config: {'x.choice': 2, 'y.choice': 77} and reward: 0.7444461288479184
Finished Task with config: {'x.choice': 36, 'y.choice': 14} and reward: 0.11531548824101008
Finished Task with config: {'x.choice': 83, 'y.choice': 86} and reward: 0.03682398109240761
Finished Task with config: {'x.choice': 95, 'y.choice': 87} and reward: 0.010053264415937721
  7%|▋         | 5/76 [00:01<00:27,  2.55it/s]Finished Task with config: {'x.choice': 97, 'y.choice': 46} and reward: 0.3161396814775167
Finished Task with config: {'x.choice': 46, 'y.choice': 52} and reward: 0.4974644053536026
Finished Task with config: {'x.choice': 79, 'y.choice': 77} and reward: 0.07935556115551717
Finished Task with config: {'x.choice': 79, 'y.choice': 67} and reward: 0.17085551866906823
  8%|▊         | 6/76 [00:02<00:26,  2.62it/s]Finished Task with config: {'x.choice': 11, 'y.choice': 15} and reward: 0.14131830986160224
Finished Task with config: {'x.choice': 71, 'y.choice': 53} and reward: 0.47455256379247157
Finished Task with config: {'x.choice': 11, 'y.choice': 15} and reward: 0.14131830986160224
Finished Task with config: {'x.choice': 63, 'y.choice': 94} and reward: 0.15444995894372754
Finished Task with config: {'x.choice': 57, 'y.choice': 75} and reward: 0.33073294541320397
  9%|▉         | 7/76 [00:02<00:25,  2.67it/s]Finished Task with config: {'x.choice': 51, 'y.choice': 26} and reward: 0.18846597355335637
Finished Task with config: {'x.choice': 10, 'y.choice': 19} and reward: 0.18136358228461952
Finished Task with config: {'x.choice': 40, 'y.choice': 76} and reward: 0.7058736551652897
Finished Task with config: {'x.choice': 46, 'y.choice': 72} and reward: 0.5774605629951269
 11%|█         | 8/76 [00:02<00:25,  2.72it/s]Finished Task with config: {'x.choice': 44, 'y.choice': 1} and reward: 0.03249007863969159
Finished Task with config: {'x.choice': 21, 'y.choice': 33} and reward: 0.4247735686936988
Finished Task with config: {'x.choice': 55, 'y.choice': 60} and reward: 0.39956895655479624
Finished Task with config: {'x.choice': 13, 'y.choice': 16} and reward: 0.1552861522645142
 12%|█▏        | 9/76 [00:03<00:24,  2.77it/s]Finished Task with config: {'x.choice': 60, 'y.choice': 94} and reward: 0.18915413038019477
Finished Task with config: {'x.choice': 21, 'y.choice': 90} and reward: 0.77816551129998
Finished Task with config: {'x.choice': 60, 'y.choice': 67} and reward: 0.3109689510968434
Finished Task with config: {'x.choice': 37, 'y.choice': 45} and reward: 0.5408988749006151
 13%|█▎        | 10/76 [00:03<00:23,  2.78it/s]Finished Task with config: {'x.choice': 34, 'y.choice': 69} and reward: 0.8520424635717397
Finished Task with config: {'x.choice': 60, 'y.choice': 11} and reward: 0.06220932769716972
Finished Task with config: {'x.choice': 42, 'y.choice': 6} and reward: 0.05312575413313245
Finished Task with config: {'x.choice': 89, 'y.choice': 97} and reward: 0.013178123417393045
 14%|█▍        | 11/76 [00:04<00:23,  2.82it/s]Finished Task with config: {'x.choice': 66, 'y.choice': 23} and reward: 0.2529084027182172
Finished Task with config: {'x.choice': 65, 'y.choice': 38} and reward: 0.49583631011448265
Finished Task with config: {'x.choice': 66, 'y.choice': 14} and reward: 0.10416741370431903
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 16%|█▌        | 12/76 [00:04<00:22,  2.83it/s]Finished Task with config: {'x.choice': 30, 'y.choice': 57} and reward: 0.829886169891878
Finished Task with config: {'x.choice': 59, 'y.choice': 3} and reward: 0.025054518512878866
Finished Task with config: {'x.choice': 35, 'y.choice': 46} and reward: 0.5846609186365146
Finished Task with config: {'x.choice': 31, 'y.choice': 62} and reward: 0.8709395257996267
 17%|█▋        | 13/76 [00:04<00:22,  2.82it/s]Finished Task with config: {'x.choice': 98, 'y.choice': 25} and reward: 0.17939595099821065
Finished Task with config: {'x.choice': 95, 'y.choice': 63} and reward: 0.11610891252206063
Finished Task with config: {'x.choice': 9, 'y.choice': 20} and reward: 0.18989747667591955
Finished Task with config: {'x.choice': 64, 'y.choice': 99} and reward: 0.12178118688087612
 18%|█▊        | 14/76 [00:05<00:22,  2.81it/s]Finished Task with config: {'x.choice': 55, 'y.choice': 52} and reward: 0.4028107985736251
Finished Task with config: {'x.choice': 63, 'y.choice': 50} and reward: 0.4368457895739434
Finished Task with config: {'x.choice': 24, 'y.choice': 31} and reward: 0.3817091079995325
Finished Task with config: {'x.choice': 40, 'y.choice': 62} and reward: 0.6969580893232793
 20%|█▉        | 15/76 [00:05<00:21,  2.82it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 67} and reward: 0.9814913209712314
Finished Task with config: {'x.choice': 32, 'y.choice': 40} and reward: 0.5087969233381394
Finished Task with config: {'x.choice': 10, 'y.choice': 20} and reward: 0.19318127651343833
Finished Task with config: {'x.choice': 65, 'y.choice': 76} and reward: 0.20282352738135428
 21%|██        | 16/76 [00:05<00:21,  2.79it/s]Finished Task with config: {'x.choice': 19, 'y.choice': 78} and reward: 0.9600071681988102
Finished Task with config: {'x.choice': 93, 'y.choice': 1} and reward: 0.010896047938056276
Finished Task with config: {'x.choice': 34, 'y.choice': 15} and reward: 0.12939423666456223
Finished Task with config: {'x.choice': 63, 'y.choice': 62} and reward: 0.31372800819230695
 22%|██▏       | 17/76 [00:06<00:21,  2.80it/s]Finished Task with config: {'x.choice': 81, 'y.choice': 96} and reward: 0.031703038768058744
Finished Task with config: {'x.choice': 65, 'y.choice': 84} and reward: 0.17254207043832434
Finished Task with config: {'x.choice': 9, 'y.choice': 52} and reward: 0.7398755445448225
Finished Task with config: {'x.choice': 97, 'y.choice': 54} and reward: 0.21495484912430388
 24%|██▎       | 18/76 [00:06<00:20,  2.80it/s]Finished Task with config: {'x.choice': 97, 'y.choice': 72} and reward: 0.03416132366430215
Finished Task with config: {'x.choice': 5, 'y.choice': 3} and reward: 0.05032274511975561
Finished Task with config: {'x.choice': 55, 'y.choice': 9} and reward: 0.04922766858232881
Finished Task with config: {'x.choice': 41, 'y.choice': 52} and reward: 0.5806808361616504
 25%|██▌       | 19/76 [00:06<00:20,  2.79it/s]Finished Task with config: {'x.choice': 27, 'y.choice': 45} and reward: 0.650688955916307
Finished Task with config: {'x.choice': 97, 'y.choice': 53} and reward: 0.22935714262309276
Finished Task with config: {'x.choice': 12, 'y.choice': 24} and reward: 0.2529076252482878
Finished Task with config: {'x.choice': 32, 'y.choice': 69} and reward: 0.8888090750132936
 26%|██▋       | 20/76 [00:07<00:20,  2.79it/s]Finished Task with config: {'x.choice': 29, 'y.choice': 18} and reward: 0.1730263130667964
Finished Task with config: {'x.choice': 85, 'y.choice': 40} and reward: 0.6756957290216257
Finished Task with config: {'x.choice': 37, 'y.choice': 69} and reward: 0.7901920138784992
Finished Task with config: {'x.choice': 88, 'y.choice': 58} and reward: 0.286327467312913
 28%|██▊       | 21/76 [00:07<00:19,  2.80it/s]Finished Task with config: {'x.choice': 48, 'y.choice': 55} and reward: 0.488952436424344
Finished Task with config: {'x.choice': 82, 'y.choice': 90} and reward: 0.03511541242031658
Finished Task with config: {'x.choice': 24, 'y.choice': 31} and reward: 0.3817091079995325
Finished Task with config: {'x.choice': 44, 'y.choice': 90} and reward: 0.48670585177293596
 29%|██▉       | 22/76 [00:07<00:19,  2.78it/s]Finished Task with config: {'x.choice': 48, 'y.choice': 50} and reward: 0.4527992474270069
Finished Task with config: {'x.choice': 89, 'y.choice': 62} and reward: 0.19018552350948792
Finished Task with config: {'x.choice': 8, 'y.choice': 4} and reward: 0.058422749634424065
Finished Task with config: {'x.choice': 26, 'y.choice': 74} and reward: 0.9614044025404535
 30%|███       | 23/76 [00:08<00:19,  2.78it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 46} and reward: 0.6886459183180196
Finished Task with config: {'x.choice': 63, 'y.choice': 45} and reward: 0.46885050272174905
Finished Task with config: {'x.choice': 54, 'y.choice': 49} and reward: 0.4009235733031741
Finished Task with config: {'x.choice': 77, 'y.choice': 52} and reward: 0.5350782292847762
 32%|███▏      | 24/76 [00:08<00:18,  2.78it/s]Finished Task with config: {'x.choice': 46, 'y.choice': 69} and reward: 0.5802833298863385
Finished Task with config: {'x.choice': 44, 'y.choice': 52} and reward: 0.5294533508693043
Finished Task with config: {'x.choice': 2, 'y.choice': 12} and reward: 0.0937625707459735
Finished Task with config: {'x.choice': 17, 'y.choice': 45} and reward: 0.6717131358286801
 33%|███▎      | 25/76 [00:09<00:18,  2.78it/s]Finished Task with config: {'x.choice': 53, 'y.choice': 69} and reward: 0.42463765730607606
Finished Task with config: {'x.choice': 43, 'y.choice': 52} and reward: 0.5462230869026983
Finished Task with config: {'x.choice': 25, 'y.choice': 74} and reward: 0.9700697273516173
Finished Task with config: {'x.choice': 55, 'y.choice': 52} and reward: 0.4028107985736251
 34%|███▍      | 26/76 [00:09<00:17,  2.79it/s]Finished Task with config: {'x.choice': 23, 'y.choice': 52} and reward: 0.8108532667658626
Finished Task with config: {'x.choice': 61, 'y.choice': 69} and reward: 0.2880558719901152
Finished Task with config: {'x.choice': 41, 'y.choice': 52} and reward: 0.5806808361616504
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
 36%|███▌      | 27/76 [00:09<00:17,  2.76it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 78, 'y.choice': 52} and reward: 0.5359258876938673
Finished Task with config: {'x.choice': 88, 'y.choice': 69} and reward: 0.09579273174126507
Finished Task with config: {'x.choice': 47, 'y.choice': 74} and reward: 0.5485750572832516
 37%|███▋      | 28/76 [00:10<00:17,  2.77it/s]Finished Task with config: {'x.choice': 46, 'y.choice': 52} and reward: 0.4974644053536026
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 27, 'y.choice': 52} and reward: 0.7850991782603625
Finished Task with config: {'x.choice': 55, 'y.choice': 52} and reward: 0.4028107985736251
 38%|███▊      | 29/76 [00:10<00:16,  2.78it/s]Finished Task with config: {'x.choice': 72, 'y.choice': 52} and reward: 0.5059934264424871
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 38, 'y.choice': 52} and reward: 0.6328227532925973
Finished Task with config: {'x.choice': 27, 'y.choice': 52} and reward: 0.7850991782603625
 39%|███▉      | 30/76 [00:10<00:16,  2.77it/s]Finished Task with config: {'x.choice': 46, 'y.choice': 74} and reward: 0.5723229680537476
Finished Task with config: {'x.choice': 40, 'y.choice': 52} and reward: 0.5981207041323738
Finished Task with config: {'x.choice': 35, 'y.choice': 52} and reward: 0.6827438813536933
Finished Task with config: {'x.choice': 33, 'y.choice': 52} and reward: 0.713395029404709
 41%|████      | 31/76 [00:11<00:16,  2.78it/s]Finished Task with config: {'x.choice': 30, 'y.choice': 52} and reward: 0.7536095770338388
Finished Task with config: {'x.choice': 37, 'y.choice': 52} and reward: 0.6498556852950799
Finished Task with config: {'x.choice': 37, 'y.choice': 52} and reward: 0.6498556852950799
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
 42%|████▏     | 32/76 [00:11<00:15,  2.79it/s]Finished Task with config: {'x.choice': 37, 'y.choice': 52} and reward: 0.6498556852950799
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 27, 'y.choice': 74} and reward: 0.9512641104822159
 43%|████▎     | 33/76 [00:11<00:15,  2.76it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
 45%|████▍     | 34/76 [00:12<00:15,  2.78it/s]Finished Task with config: {'x.choice': 31, 'y.choice': 52} and reward: 0.7410814359404405
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
Finished Task with config: {'x.choice': 80, 'y.choice': 52} and reward: 0.5316010536952981
 46%|████▌     | 35/76 [00:12<00:14,  2.77it/s]Finished Task with config: {'x.choice': 46, 'y.choice': 52} and reward: 0.4974644053536026
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 47%|████▋     | 36/76 [00:13<00:14,  2.78it/s]Finished Task with config: {'x.choice': 35, 'y.choice': 52} and reward: 0.6827438813536933
Finished Task with config: {'x.choice': 27, 'y.choice': 52} and reward: 0.7850991782603625
Finished Task with config: {'x.choice': 31, 'y.choice': 74} and reward: 0.8970279379062928
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
 49%|████▊     | 37/76 [00:13<00:13,  2.81it/s]Finished Task with config: {'x.choice': 27, 'y.choice': 52} and reward: 0.7850991782603625
Finished Task with config: {'x.choice': 93, 'y.choice': 74} and reward: 0.038273278217563975
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
 50%|█████     | 38/76 [00:13<00:13,  2.81it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 51%|█████▏    | 39/76 [00:14<00:13,  2.79it/s]Finished Task with config: {'x.choice': 27, 'y.choice': 74} and reward: 0.9512641104822159
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 53%|█████▎    | 40/76 [00:14<00:12,  2.78it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 54%|█████▍    | 41/76 [00:14<00:12,  2.81it/s]Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 55%|█████▌    | 42/76 [00:15<00:12,  2.80it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 57%|█████▋    | 43/76 [00:15<00:11,  2.76it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 58%|█████▊    | 44/76 [00:15<00:11,  2.76it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 59%|█████▉    | 45/76 [00:16<00:11,  2.78it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 61%|██████    | 46/76 [00:16<00:10,  2.78it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 62%|██████▏   | 47/76 [00:16<00:10,  2.77it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 63%|██████▎   | 48/76 [00:17<00:10,  2.78it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 64%|██████▍   | 49/76 [00:17<00:09,  2.77it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 66%|██████▌   | 50/76 [00:18<00:09,  2.78it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 67%|██████▋   | 51/76 [00:18<00:09,  2.76it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 68%|██████▊   | 52/76 [00:18<00:08,  2.73it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 70%|██████▉   | 53/76 [00:19<00:08,  2.75it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 71%|███████   | 54/76 [00:19<00:08,  2.73it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 72%|███████▏  | 55/76 [00:19<00:07,  2.71it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 74%|███████▎  | 56/76 [00:20<00:07,  2.76it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 75%|███████▌  | 57/76 [00:20<00:06,  2.74it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 76%|███████▋  | 58/76 [00:20<00:06,  2.74it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 78%|███████▊  | 59/76 [00:21<00:06,  2.67it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 79%|███████▉  | 60/76 [00:21<00:05,  2.70it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 80%|████████  | 61/76 [00:22<00:05,  2.71it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 82%|████████▏ | 62/76 [00:22<00:05,  2.74it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 83%|████████▎ | 63/76 [00:22<00:04,  2.72it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 84%|████████▍ | 64/76 [00:23<00:04,  2.75it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 86%|████████▌ | 65/76 [00:23<00:04,  2.72it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 87%|████████▋ | 66/76 [00:23<00:03,  2.75it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 88%|████████▊ | 67/76 [00:24<00:03,  2.76it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 89%|████████▉ | 68/76 [00:24<00:02,  2.75it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 91%|█████████ | 69/76 [00:25<00:02,  2.73it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 92%|█████████▏| 70/76 [00:25<00:02,  2.71it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 93%|█████████▎| 71/76 [00:25<00:01,  2.68it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 95%|█████████▍| 72/76 [00:26<00:01,  2.72it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 96%|█████████▌| 73/76 [00:26<00:01,  2.73it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 97%|█████████▋| 74/76 [00:26<00:00,  2.74it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
100%|██████████| 76/76 [00:27<00:00,  2.79it/s]
Best config: {'x.choice': 21, 'y.choice': 74}, best reward: 0.9892484241569526

Compare the Performance

Get the result history:

results_rl = [v[0]['accuracy'] for v in rl_scheduler.training_history.values()]
results_random = [v[0]['accuracy'] for v in random_scheduler.training_history.values()]

Average result every 10 trials:

import statistics
results1 = [statistics.mean(results_random[i:i+10]) for i in range(0, len(results_random), 10)]
results2 = [statistics.mean(results_rl[i:i+10]) for i in range(0, len(results_rl), 10)]

Plot the results:

plt.plot(range(len(results1)), results1, range(len(results2)), results2)
[<matplotlib.lines.Line2D at 0x7febc2e2da90>,
 <matplotlib.lines.Line2D at 0x7febc2cc39d0>]
../../_images/output_rl_searcher_ecab16_17_1.png