Demo RL Searcher

In this tutorial, we are going to compare RL searcher with random search in a simulation environment.

A Toy Reward Space

Input Space x = [0: 99], y = [0: 99]. The rewards are a combination of 2 gaussians as shown in the following figure:

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

Generate the simulation rewards as a mixture of 2 gaussians:

def gaussian(x, y, x0, y0, xalpha, yalpha, A):
    return A * np.exp( -((x-x0)/xalpha)**2 -((y-y0)/yalpha)**2)

x, y = np.linspace(0, 99, 100), np.linspace(0, 99, 100)
X, Y = np.meshgrid(x, y)

Z = np.zeros(X.shape)
ps = [(20, 70, 35, 40, 1),
      (80, 40, 20, 20, 0.7)]
for p in ps:
    Z += gaussian(X, Y, *p)

Visualize the reward space:

fig = plt.figure()
ax = fig.gca(projection='3d')
ax.plot_surface(X, Y, Z, cmap='plasma')
ax.set_zlim(0,np.max(Z)+2)
plt.show()
../../_images/output_rl_searcher_ecab16_5_0.png

Simulation Experiment

Customize Train Function

We can define any function with a decorator @ag.args, which converts the function to AutoGluon searchable. The reporter is used to communicate with AutoGluon search algorithms.

import autogluon as ag

@ag.args(
    x=ag.space.Categorical(*list(range(100))),
    y=ag.space.Categorical(*list(range(100))),
)
def rl_simulation(args, reporter):
    x, y = args.x, args.y
    reporter(accuracy=Z[y][x])

Random Search Baseline

random_scheduler = ag.scheduler.FIFOScheduler(rl_simulation, rl_simulation.args,
                                              resource={'num_cpus': 1, 'num_gpus': 0},
                                              num_trials=300,
                                              reward_attr="accuracy",
                                              resume=False)
random_scheduler.run()
random_scheduler.join_jobs()
print('Best config: {}, best reward: {}'.format(random_scheduler.get_best_config(), random_scheduler.get_best_reward()))
Starting Experiments
Num of Finished Tasks is 0
Num of Pending Tasks is 300
HBox(children=(FloatProgress(value=0.0, max=300.0), HTML(value='')))
Finished Task with config: {'x.choice': 0, 'y.choice': 0} and reward: 0.033741370964360654
Finished Task with config: {'x.choice': 88, 'y.choice': 85} and reward: 0.02370987044724916
Finished Task with config: {'x.choice': 52, 'y.choice': 65} and reward: 0.44742365278570423
Finished Task with config: {'x.choice': 25, 'y.choice': 97} and reward: 0.6212429874050642
Finished Task with config: {'x.choice': 58, 'y.choice': 9} and reward: 0.04895380267955919
Finished Task with config: {'x.choice': 76, 'y.choice': 59} and reward: 0.34443089888324385
Finished Task with config: {'x.choice': 12, 'y.choice': 84} and reward: 0.8396712480896082
Finished Task with config: {'x.choice': 92, 'y.choice': 49} and reward: 0.4098743899577231
Finished Task with config: {'x.choice': 29, 'y.choice': 5} and reward: 0.06680267225490555
Finished Task with config: {'x.choice': 21, 'y.choice': 75} and reward: 0.9836985346918046
Finished Task with config: {'x.choice': 15, 'y.choice': 28} and reward: 0.325344931407943
Finished Task with config: {'x.choice': 50, 'y.choice': 53} and reward: 0.44874444837595984
Finished Task with config: {'x.choice': 71, 'y.choice': 8} and reward: 0.05502004132780324
Finished Task with config: {'x.choice': 74, 'y.choice': 10} and reward: 0.07718012514180801
Finished Task with config: {'x.choice': 29, 'y.choice': 20} and reward: 0.1965858554699551
Finished Task with config: {'x.choice': 70, 'y.choice': 90} and reward: 0.10223623652174404
Finished Task with config: {'x.choice': 87, 'y.choice': 52} and reward: 0.4529881077611811
Finished Task with config: {'x.choice': 25, 'y.choice': 99} and reward: 0.5792430819086133
Finished Task with config: {'x.choice': 45, 'y.choice': 60} and reward: 0.5760424427609301
Finished Task with config: {'x.choice': 85, 'y.choice': 89} and reward: 0.026985361476694288
Finished Task with config: {'x.choice': 62, 'y.choice': 69} and reward: 0.2748175225458374
Finished Task with config: {'x.choice': 81, 'y.choice': 31} and reward: 0.5887866095009011
Finished Task with config: {'x.choice': 4, 'y.choice': 24} and reward: 0.2162157174809089
Finished Task with config: {'x.choice': 78, 'y.choice': 99} and reward: 0.03805523229113804
Finished Task with config: {'x.choice': 88, 'y.choice': 34} and reward: 0.5553674220127564
Finished Task with config: {'x.choice': 21, 'y.choice': 4} and reward: 0.06566120957793595
Finished Task with config: {'x.choice': 53, 'y.choice': 98} and reward: 0.25186058408155276
Finished Task with config: {'x.choice': 7, 'y.choice': 46} and reward: 0.6077707887503334
Finished Task with config: {'x.choice': 73, 'y.choice': 28} and reward: 0.4655885914629626
Finished Task with config: {'x.choice': 40, 'y.choice': 62} and reward: 0.6969580893232793
Finished Task with config: {'x.choice': 31, 'y.choice': 40} and reward: 0.5179232997264821
Finished Task with config: {'x.choice': 65, 'y.choice': 7} and reward: 0.04223211242114851
Finished Task with config: {'x.choice': 99, 'y.choice': 57} and reward: 0.14335297823558435
Finished Task with config: {'x.choice': 26, 'y.choice': 74} and reward: 0.9614044025404535
Finished Task with config: {'x.choice': 88, 'y.choice': 69} and reward: 0.09579273174126507
Finished Task with config: {'x.choice': 54, 'y.choice': 46} and reward: 0.38957919549700126
Finished Task with config: {'x.choice': 10, 'y.choice': 86} and reward: 0.7853446353989183
Finished Task with config: {'x.choice': 68, 'y.choice': 56} and reward: 0.3924028194544852
Finished Task with config: {'x.choice': 51, 'y.choice': 21} and reward: 0.1364395075598378
Finished Task with config: {'x.choice': 49, 'y.choice': 87} and reward: 0.42039801917967784
Finished Task with config: {'x.choice': 34, 'y.choice': 18} and reward: 0.15828957374871191
Finished Task with config: {'x.choice': 54, 'y.choice': 34} and reward: 0.2911834643280108
Finished Task with config: {'x.choice': 54, 'y.choice': 84} and reward: 0.3453448685349498
Finished Task with config: {'x.choice': 49, 'y.choice': 93} and reward: 0.3616785492144533
Finished Task with config: {'x.choice': 56, 'y.choice': 91} and reward: 0.2637797449288515
Finished Task with config: {'x.choice': 43, 'y.choice': 59} and reward: 0.6112852170454494
Finished Task with config: {'x.choice': 13, 'y.choice': 18} and reward: 0.17728720079030913
Finished Task with config: {'x.choice': 91, 'y.choice': 9} and reward: 0.048404473142230384
Finished Task with config: {'x.choice': 81, 'y.choice': 52} and reward: 0.5263159165670958
Finished Task with config: {'x.choice': 8, 'y.choice': 13} and reward: 0.11669708254177907
Finished Task with config: {'x.choice': 37, 'y.choice': 5} and reward: 0.05665090538536245
Finished Task with config: {'x.choice': 2, 'y.choice': 12} and reward: 0.0937625707459735
Finished Task with config: {'x.choice': 37, 'y.choice': 43} and reward: 0.5075292207663951
Finished Task with config: {'x.choice': 47, 'y.choice': 65} and reward: 0.5525975094496758
Finished Task with config: {'x.choice': 98, 'y.choice': 31} and reward: 0.2570095473514332
Finished Task with config: {'x.choice': 49, 'y.choice': 27} and reward: 0.19999150581974157
Finished Task with config: {'x.choice': 61, 'y.choice': 31} and reward: 0.32983957233017214
Finished Task with config: {'x.choice': 21, 'y.choice': 44} and reward: 0.6549832051085137
Finished Task with config: {'x.choice': 88, 'y.choice': 73} and reward: 0.062011635782821456
Finished Task with config: {'x.choice': 57, 'y.choice': 10} and reward: 0.0541338464455822
Finished Task with config: {'x.choice': 25, 'y.choice': 19} and reward: 0.1929340159663649
Finished Task with config: {'x.choice': 40, 'y.choice': 74} and reward: 0.7149565583173672
Finished Task with config: {'x.choice': 54, 'y.choice': 81} and reward: 0.36278004484185944
Finished Task with config: {'x.choice': 77, 'y.choice': 75} and reward: 0.10141092056783696
Finished Task with config: {'x.choice': 80, 'y.choice': 21} and reward: 0.29569116572893495
Finished Task with config: {'x.choice': 23, 'y.choice': 73} and reward: 0.9871254855101496
Finished Task with config: {'x.choice': 29, 'y.choice': 36} and reward: 0.45547903771119813
Finished Task with config: {'x.choice': 35, 'y.choice': 24} and reward: 0.22409324383047496
Finished Task with config: {'x.choice': 20, 'y.choice': 29} and reward: 0.34978294352039435
Finished Task with config: {'x.choice': 36, 'y.choice': 15} and reward: 0.12366504409612786
Finished Task with config: {'x.choice': 25, 'y.choice': 69} and reward: 0.9792309174863233
Finished Task with config: {'x.choice': 99, 'y.choice': 63} and reward: 0.0815915145879063
Finished Task with config: {'x.choice': 29, 'y.choice': 68} and reward: 0.9338269953586453
Finished Task with config: {'x.choice': 59, 'y.choice': 9} and reward: 0.04926591410979671
Finished Task with config: {'x.choice': 11, 'y.choice': 39} and reward: 0.5133803752616657
Finished Task with config: {'x.choice': 64, 'y.choice': 56} and reward: 0.37677913262287954
Finished Task with config: {'x.choice': 30, 'y.choice': 51} and reward: 0.7364596640362513
Finished Task with config: {'x.choice': 14, 'y.choice': 73} and reward: 0.9655939575002809
Finished Task with config: {'x.choice': 12, 'y.choice': 17} and reward: 0.1640048878597255
Finished Task with config: {'x.choice': 48, 'y.choice': 55} and reward: 0.488952436424344
Finished Task with config: {'x.choice': 55, 'y.choice': 13} and reward: 0.07199979992889341
Finished Task with config: {'x.choice': 22, 'y.choice': 97} and reward: 0.6319846121863306
Finished Task with config: {'x.choice': 45, 'y.choice': 31} and reward: 0.25878166625332877
Finished Task with config: {'x.choice': 58, 'y.choice': 40} and reward: 0.38403476087459665
Finished Task with config: {'x.choice': 7, 'y.choice': 37} and reward: 0.4410567870400008
Finished Task with config: {'x.choice': 46, 'y.choice': 32} and reward: 0.2667064686889594
Finished Task with config: {'x.choice': 98, 'y.choice': 88} and reward: 0.006671340230915448
Finished Task with config: {'x.choice': 2, 'y.choice': 78} and reward: 0.7374995583972943
Finished Task with config: {'x.choice': 97, 'y.choice': 6} and reward: 0.01950026396219131
Finished Task with config: {'x.choice': 69, 'y.choice': 82} and reward: 0.1350225092162868
Finished Task with config: {'x.choice': 80, 'y.choice': 16} and reward: 0.17440412191763857
Finished Task with config: {'x.choice': 52, 'y.choice': 2} and reward: 0.026758302210931156
Finished Task with config: {'x.choice': 41, 'y.choice': 22} and reward: 0.17224776710651427
Finished Task with config: {'x.choice': 59, 'y.choice': 73} and reward: 0.3025644755949059
Finished Task with config: {'x.choice': 63, 'y.choice': 50} and reward: 0.4368457895739434
Finished Task with config: {'x.choice': 32, 'y.choice': 46} and reward: 0.6223165662078063
Finished Task with config: {'x.choice': 85, 'y.choice': 46} and reward: 0.6231620164437511
Finished Task with config: {'x.choice': 13, 'y.choice': 65} and reward: 0.9458957413053012
Finished Task with config: {'x.choice': 64, 'y.choice': 47} and reward: 0.4744765792129547
Finished Task with config: {'x.choice': 33, 'y.choice': 35} and reward: 0.4077426725983319
Finished Task with config: {'x.choice': 49, 'y.choice': 13} and reward: 0.07630012880204198
Finished Task with config: {'x.choice': 73, 'y.choice': 56} and reward: 0.4158663026820698
Finished Task with config: {'x.choice': 43, 'y.choice': 82} and reward: 0.5937072165360853
Finished Task with config: {'x.choice': 8, 'y.choice': 3} and reward: 0.053762718274869094
Finished Task with config: {'x.choice': 3, 'y.choice': 64} and reward: 0.7722721316817742
Finished Task with config: {'x.choice': 18, 'y.choice': 5} and reward: 0.07108638701796187
Finished Task with config: {'x.choice': 65, 'y.choice': 21} and reward: 0.20444904912699213
Finished Task with config: {'x.choice': 78, 'y.choice': 5} and reward: 0.03699050838461438
Finished Task with config: {'x.choice': 73, 'y.choice': 51} and reward: 0.5382042316686375
Finished Task with config: {'x.choice': 81, 'y.choice': 81} and reward: 0.05490407675355491
Finished Task with config: {'x.choice': 5, 'y.choice': 68} and reward: 0.8301296574431464
Finished Task with config: {'x.choice': 80, 'y.choice': 84} and reward: 0.052362865447605315
Finished Task with config: {'x.choice': 72, 'y.choice': 36} and reward: 0.6265167389413449
Finished Task with config: {'x.choice': 98, 'y.choice': 12} and reward: 0.04471446103385546
Finished Task with config: {'x.choice': 53, 'y.choice': 75} and reward: 0.4099932625289486
Finished Task with config: {'x.choice': 52, 'y.choice': 22} and reward: 0.14656594976683673
Finished Task with config: {'x.choice': 60, 'y.choice': 47} and reward: 0.42243757919289426
Finished Task with config: {'x.choice': 66, 'y.choice': 37} and reward: 0.5092949807158434
Finished Task with config: {'x.choice': 16, 'y.choice': 75} and reward: 0.9717224882073473
Finished Task with config: {'x.choice': 53, 'y.choice': 3} and reward: 0.02854895567741269
Finished Task with config: {'x.choice': 44, 'y.choice': 10} and reward: 0.06875080289267363
Finished Task with config: {'x.choice': 74, 'y.choice': 70} and reward: 0.1599421562340529
Finished Task with config: {'x.choice': 52, 'y.choice': 76} and reward: 0.4276934939414273
Finished Task with config: {'x.choice': 66, 'y.choice': 50} and reward: 0.47241577066173623
Finished Task with config: {'x.choice': 72, 'y.choice': 23} and reward: 0.31727743298548305
Finished Task with config: {'x.choice': 38, 'y.choice': 7} and reward: 0.06480002998754321
Finished Task with config: {'x.choice': 49, 'y.choice': 55} and reward: 0.47338367365484746
Finished Task with config: {'x.choice': 35, 'y.choice': 89} and reward: 0.6641269398812268
Finished Task with config: {'x.choice': 0, 'y.choice': 28} and reward: 0.23954107281900225
Finished Task with config: {'x.choice': 22, 'y.choice': 3} and reward: 0.06027692379275232
Finished Task with config: {'x.choice': 93, 'y.choice': 73} and reward: 0.04297864339875762
Finished Task with config: {'x.choice': 53, 'y.choice': 16} and reward: 0.09324321501441603
Finished Task with config: {'x.choice': 90, 'y.choice': 96} and reward: 0.012218797109661481
Finished Task with config: {'x.choice': 3, 'y.choice': 73} and reward: 0.7854147585003671
Finished Task with config: {'x.choice': 46, 'y.choice': 46} and reward: 0.4373405560928681
Finished Task with config: {'x.choice': 58, 'y.choice': 85} and reward: 0.26861674760974447
Finished Task with config: {'x.choice': 94, 'y.choice': 73} and reward: 0.03955987603792764
Finished Task with config: {'x.choice': 80, 'y.choice': 66} and reward: 0.1815675014338308
Finished Task with config: {'x.choice': 47, 'y.choice': 53} and reward: 0.4905157991105475
Finished Task with config: {'x.choice': 81, 'y.choice': 18} and reward: 0.21706501550239402
Finished Task with config: {'x.choice': 74, 'y.choice': 53} and reward: 0.4965222636653659
Finished Task with config: {'x.choice': 85, 'y.choice': 8} and reward: 0.05371039932646394
Finished Task with config: {'x.choice': 85, 'y.choice': 11} and reward: 0.0839329437005921
Finished Task with config: {'x.choice': 57, 'y.choice': 92} and reward: 0.24191791334115323
Finished Task with config: {'x.choice': 92, 'y.choice': 99} and reward: 0.008668531750660007
Finished Task with config: {'x.choice': 13, 'y.choice': 67} and reward: 0.9554016826947737
Finished Task with config: {'x.choice': 85, 'y.choice': 48} and reward: 0.5838434796461214
Finished Task with config: {'x.choice': 90, 'y.choice': 79} and reward: 0.029576700229173733
Finished Task with config: {'x.choice': 96, 'y.choice': 50} and reward: 0.29443626563628245
Finished Task with config: {'x.choice': 27, 'y.choice': 97} and reward: 0.609190229739896
Finished Task with config: {'x.choice': 91, 'y.choice': 83} and reward: 0.019771870270097303
Finished Task with config: {'x.choice': 73, 'y.choice': 23} and reward: 0.326072881584614
Finished Task with config: {'x.choice': 57, 'y.choice': 62} and reward: 0.36987692800024347
Finished Task with config: {'x.choice': 90, 'y.choice': 0} and reward: 0.010841597566485417
Finished Task with config: {'x.choice': 5, 'y.choice': 44} and reward: 0.5454345262138888
Finished Task with config: {'x.choice': 22, 'y.choice': 20} and reward: 0.20898538871494066
Finished Task with config: {'x.choice': 23, 'y.choice': 12} and reward: 0.12128578802743897
Finished Task with config: {'x.choice': 26, 'y.choice': 37} and reward: 0.49210492949748
Finished Task with config: {'x.choice': 76, 'y.choice': 48} and reward: 0.6302372943352225
Finished Task with config: {'x.choice': 48, 'y.choice': 15} and reward: 0.09095201666775383
Finished Task with config: {'x.choice': 51, 'y.choice': 7} and reward: 0.04381113819238674
Finished Task with config: {'x.choice': 38, 'y.choice': 98} and reward: 0.4702523715227277
Finished Task with config: {'x.choice': 63, 'y.choice': 79} and reward: 0.21771769101633315
Finished Task with config: {'x.choice': 84, 'y.choice': 25} and reward: 0.3931677266907765
Finished Task with config: {'x.choice': 96, 'y.choice': 86} and reward: 0.0094952525595048
Finished Task with config: {'x.choice': 84, 'y.choice': 1} and reward: 0.016809160782026305
Finished Task with config: {'x.choice': 1, 'y.choice': 79} and reward: 0.7079945565148782
Finished Task with config: {'x.choice': 23, 'y.choice': 92} and reward: 0.7335594679353517
Finished Task with config: {'x.choice': 37, 'y.choice': 72} and reward: 0.7884048380310176
Finished Task with config: {'x.choice': 6, 'y.choice': 83} and reward: 0.7667266038705639
Finished Task with config: {'x.choice': 40, 'y.choice': 93} and reward: 0.5183349839621634
Finished Task with config: {'x.choice': 23, 'y.choice': 61} and reward: 0.9437454090924291
Finished Task with config: {'x.choice': 10, 'y.choice': 35} and reward: 0.4285918072702373
Finished Task with config: {'x.choice': 35, 'y.choice': 50} and reward: 0.6515745643475849
Finished Task with config: {'x.choice': 9, 'y.choice': 35} and reward: 0.42130625627627266
Finished Task with config: {'x.choice': 30, 'y.choice': 90} and reward: 0.7177535466994763
Finished Task with config: {'x.choice': 20, 'y.choice': 20} and reward: 0.20964316710193154
Finished Task with config: {'x.choice': 27, 'y.choice': 89} and reward: 0.7667281392359926
Finished Task with config: {'x.choice': 46, 'y.choice': 88} and reward: 0.4704450486651075
Finished Task with config: {'x.choice': 98, 'y.choice': 21} and reward: 0.1278435736139452
Finished Task with config: {'x.choice': 83, 'y.choice': 77} and reward: 0.06031601390966193
Finished Task with config: {'x.choice': 12, 'y.choice': 42} and reward: 0.5814481209783846
Finished Task with config: {'x.choice': 78, 'y.choice': 40} and reward: 0.7296013949338893
Finished Task with config: {'x.choice': 50, 'y.choice': 49} and reward: 0.4243575422139322
Finished Task with config: {'x.choice': 25, 'y.choice': 98} and reward: 0.6002506095606537
Finished Task with config: {'x.choice': 74, 'y.choice': 17} and reward: 0.18645972138249706
Finished Task with config: {'x.choice': 36, 'y.choice': 78} and reward: 0.779745613633179
Finished Task with config: {'x.choice': 3, 'y.choice': 94} and reward: 0.551056261514387
Finished Task with config: {'x.choice': 30, 'y.choice': 87} and reward: 0.7693181820344884
Finished Task with config: {'x.choice': 57, 'y.choice': 80} and reward: 0.3106794374970307
Finished Task with config: {'x.choice': 70, 'y.choice': 67} and reward: 0.21730334307817395
Finished Task with config: {'x.choice': 34, 'y.choice': 66} and reward: 0.8443160288386271
Finished Task with config: {'x.choice': 50, 'y.choice': 83} and reward: 0.4322980014612179
Finished Task with config: {'x.choice': 43, 'y.choice': 72} and reward: 0.6494598761654348
Finished Task with config: {'x.choice': 43, 'y.choice': 38} and reward: 0.36499333218830377
Finished Task with config: {'x.choice': 6, 'y.choice': 72} and reward: 0.8500161515751357
Finished Task with config: {'x.choice': 34, 'y.choice': 17} and reward: 0.14819020120457146
Finished Task with config: {'x.choice': 10, 'y.choice': 74} and reward: 0.9124404562856999
Finished Task with config: {'x.choice': 47, 'y.choice': 89} and reward: 0.4402252900465484
Finished Task with config: {'x.choice': 81, 'y.choice': 46} and reward: 0.6716095482523521
Finished Task with config: {'x.choice': 82, 'y.choice': 23} and reward: 0.34739845707338457
Finished Task with config: {'x.choice': 83, 'y.choice': 8} and reward: 0.05645336125521532
Finished Task with config: {'x.choice': 75, 'y.choice': 68} and reward: 0.17705230255611187
Finished Task with config: {'x.choice': 94, 'y.choice': 14} and reward: 0.08074119324477916
Finished Task with config: {'x.choice': 59, 'y.choice': 69} and reward: 0.3171228568606663
Finished Task with config: {'x.choice': 29, 'y.choice': 17} and reward: 0.16202259944625425
Finished Task with config: {'x.choice': 59, 'y.choice': 8} and reward: 0.04411185926125333
Finished Task with config: {'x.choice': 45, 'y.choice': 24} and reward: 0.1772436387368555
Finished Task with config: {'x.choice': 86, 'y.choice': 19} and reward: 0.21804248959690348
Finished Task with config: {'x.choice': 70, 'y.choice': 8} and reward: 0.053900378804893956
Finished Task with config: {'x.choice': 26, 'y.choice': 66} and reward: 0.9614659896874531
Finished Task with config: {'x.choice': 47, 'y.choice': 41} and reward: 0.3719250040103771
Finished Task with config: {'x.choice': 30, 'y.choice': 11} and reward: 0.10480210711397478
Finished Task with config: {'x.choice': 19, 'y.choice': 0} and reward: 0.04673362708207175
Finished Task with config: {'x.choice': 75, 'y.choice': 22} and reward: 0.31258660946844835
Finished Task with config: {'x.choice': 43, 'y.choice': 62} and reward: 0.6306666404036612
Finished Task with config: {'x.choice': 15, 'y.choice': 25} and reward: 0.2763752234523125
Finished Task with config: {'x.choice': 20, 'y.choice': 52} and reward: 0.8167467526672123
Finished Task with config: {'x.choice': 79, 'y.choice': 68} and reward: 0.15653941147067685
Finished Task with config: {'x.choice': 71, 'y.choice': 25} and reward: 0.35947985565642476
Finished Task with config: {'x.choice': 74, 'y.choice': 49} and reward: 0.592702917519141
Finished Task with config: {'x.choice': 21, 'y.choice': 5} and reward: 0.07126392906111646
Finished Task with config: {'x.choice': 28, 'y.choice': 57} and reward: 0.8543548707680503
Finished Task with config: {'x.choice': 5, 'y.choice': 52} and reward: 0.6796529980229132
Finished Task with config: {'x.choice': 62, 'y.choice': 70} and reward: 0.2697491453361137
Finished Task with config: {'x.choice': 28, 'y.choice': 99} and reward: 0.5610924131844697
Finished Task with config: {'x.choice': 56, 'y.choice': 1} and reward: 0.02141165344229046
Finished Task with config: {'x.choice': 15, 'y.choice': 26} and reward: 0.2921843921688608
Finished Task with config: {'x.choice': 81, 'y.choice': 39} and reward: 0.7228090068362416
Finished Task with config: {'x.choice': 52, 'y.choice': 17} and reward: 0.10117835846623884
Finished Task with config: {'x.choice': 62, 'y.choice': 47} and reward: 0.4457245546030484
Finished Task with config: {'x.choice': 21, 'y.choice': 24} and reward: 0.26631219551230567
Finished Task with config: {'x.choice': 58, 'y.choice': 7} and reward: 0.03946418102499037
Finished Task with config: {'x.choice': 1, 'y.choice': 70} and reward: 0.744759552543121
Finished Task with config: {'x.choice': 17, 'y.choice': 33} and reward: 0.42193671899267254
Finished Task with config: {'x.choice': 98, 'y.choice': 69} and reward: 0.04500072260113956
Finished Task with config: {'x.choice': 77, 'y.choice': 91} and reward: 0.05453732391423032
Finished Task with config: {'x.choice': 87, 'y.choice': 30} and reward: 0.49173081127907853
Finished Task with config: {'x.choice': 66, 'y.choice': 24} and reward: 0.2734894532564603
Finished Task with config: {'x.choice': 86, 'y.choice': 13} and reward: 0.10714541441968928
Finished Task with config: {'x.choice': 6, 'y.choice': 86} and reward: 0.7261490410748771
Finished Task with config: {'x.choice': 74, 'y.choice': 18} and reward: 0.2078426746909669
Finished Task with config: {'x.choice': 10, 'y.choice': 73} and reward: 0.9164411614995052
Finished Task with config: {'x.choice': 87, 'y.choice': 9} and reward: 0.05854419560698867
Finished Task with config: {'x.choice': 52, 'y.choice': 11} and reward: 0.06125982756685816
Finished Task with config: {'x.choice': 6, 'y.choice': 30} and reward: 0.3134867989459137
Finished Task with config: {'x.choice': 56, 'y.choice': 77} and reward: 0.34210464396292306
Finished Task with config: {'x.choice': 37, 'y.choice': 1} and reward: 0.0404478969636493
Finished Task with config: {'x.choice': 76, 'y.choice': 41} and reward: 0.7165747874234801
Finished Task with config: {'x.choice': 28, 'y.choice': 79} and reward: 0.9022624573472735
Finished Task with config: {'x.choice': 37, 'y.choice': 68} and reward: 0.7888420705989861
Finished Task with config: {'x.choice': 57, 'y.choice': 72} and reward: 0.3406826319385914
Finished Task with config: {'x.choice': 70, 'y.choice': 80} and reward: 0.13203595913287092
Finished Task with config: {'x.choice': 3, 'y.choice': 21} and reward: 0.17612826366950896
Finished Task with config: {'x.choice': 48, 'y.choice': 56} and reward: 0.4950322639677046
Finished Task with config: {'x.choice': 63, 'y.choice': 74} and reward: 0.2377348443899519
Finished Task with config: {'x.choice': 64, 'y.choice': 92} and reward: 0.15257487038322787
Finished Task with config: {'x.choice': 63, 'y.choice': 94} and reward: 0.15444995894372754
Finished Task with config: {'x.choice': 20, 'y.choice': 32} and reward: 0.40562811909195556
Finished Task with config: {'x.choice': 67, 'y.choice': 89} and reward: 0.1326157175613137
Finished Task with config: {'x.choice': 65, 'y.choice': 56} and reward: 0.3796978747046771
Finished Task with config: {'x.choice': 24, 'y.choice': 10} and reward: 0.10406057762408168
Finished Task with config: {'x.choice': 62, 'y.choice': 63} and reward: 0.31276022682897947
Finished Task with config: {'x.choice': 79, 'y.choice': 77} and reward: 0.07935556115551717
Finished Task with config: {'x.choice': 5, 'y.choice': 27} and reward: 0.2620297841595642
Finished Task with config: {'x.choice': 87, 'y.choice': 89} and reward: 0.021974242913101186
Finished Task with config: {'x.choice': 12, 'y.choice': 71} and reward: 0.9485040109963007
Finished Task with config: {'x.choice': 24, 'y.choice': 3} and reward: 0.05969329500226269
Finished Task with config: {'x.choice': 17, 'y.choice': 43} and reward: 0.6294438710927085
Finished Task with config: {'x.choice': 29, 'y.choice': 59} and reward: 0.8682661456788996
Finished Task with config: {'x.choice': 21, 'y.choice': 18} and reward: 0.18440364325580455
Finished Task with config: {'x.choice': 44, 'y.choice': 24} and reward: 0.18096488322740784
Finished Task with config: {'x.choice': 21, 'y.choice': 13} and reward: 0.13116515973198836
Finished Task with config: {'x.choice': 22, 'y.choice': 80} and reward: 0.9363534485833306
Finished Task with config: {'x.choice': 57, 'y.choice': 86} and reward: 0.2796594710653824
Finished Task with config: {'x.choice': 14, 'y.choice': 60} and reward: 0.9122123354951699
Finished Task with config: {'x.choice': 58, 'y.choice': 48} and reward: 0.40522239097256896
Finished Task with config: {'x.choice': 58, 'y.choice': 30} and reward: 0.2757454259325032
Finished Task with config: {'x.choice': 4, 'y.choice': 8} and reward: 0.0734258520828858
Finished Task with config: {'x.choice': 97, 'y.choice': 97} and reward: 0.005114350279263141
Finished Task with config: {'x.choice': 2, 'y.choice': 9} and reward: 0.0750114442919767
Finished Task with config: {'x.choice': 25, 'y.choice': 80} and reward: 0.9204423345850681
Finished Task with config: {'x.choice': 81, 'y.choice': 55} and reward: 0.4395136784554735
Finished Task with config: {'x.choice': 29, 'y.choice': 54} and reward: 0.7982635506321227
Finished Task with config: {'x.choice': 55, 'y.choice': 35} and reward: 0.30891800081427834
Finished Task with config: {'x.choice': 7, 'y.choice': 67} and reward: 0.8662480584181851
Finished Task with config: {'x.choice': 57, 'y.choice': 25} and reward: 0.19853744287793101
Finished Task with config: {'x.choice': 61, 'y.choice': 60} and reward: 0.34261273075965726
Finished Task with config: {'x.choice': 44, 'y.choice': 75} and reward: 0.6164691489006827
Finished Task with config: {'x.choice': 72, 'y.choice': 86} and reward: 0.09673647893582649
Finished Task with config: {'x.choice': 86, 'y.choice': 58} and reward: 0.31069619363448053
Finished Task with config: {'x.choice': 25, 'y.choice': 53} and reward: 0.8181236801040525
Finished Task with config: {'x.choice': 15, 'y.choice': 11} and reward: 0.11124578120698932
Finished Task with config: {'x.choice': 72, 'y.choice': 8} and reward: 0.056065669760406805
Finished Task with config: {'x.choice': 62, 'y.choice': 55} and reward: 0.3832771438934273
Finished Task with config: {'x.choice': 82, 'y.choice': 27} and reward: 0.4678752894461969
Finished Task with config: {'x.choice': 44, 'y.choice': 25} and reward: 0.1918744510740746
Finished Task with config: {'x.choice': 98, 'y.choice': 17} and reward: 0.0841823403591161
Finished Task with config: {'x.choice': 98, 'y.choice': 4} and reward: 0.012653484011151123
Finished Task with config: {'x.choice': 12, 'y.choice': 85} and reward: 0.8245892901752379
Best config: {'x.choice': 23, 'y.choice': 73}, best reward: 0.9871254855101496

Reinforcement Learning

rl_scheduler = ag.scheduler.RLScheduler(rl_simulation,
                                        resource={'num_cpus': 1, 'num_gpus': 0},
                                        num_trials=300,
                                        reward_attr="accuracy",
                                        controller_batch_size=4,
                                        controller_lr=5e-3,
                                        checkpoint='./rl_exp/checkerpoint.ag',
                                        resume=False,
                                        sync=True)
rl_scheduler.run()
rl_scheduler.join_jobs()
print('Best config: {}, best reward: {}'.format(rl_scheduler.get_best_config(), rl_scheduler.get_best_reward()))
Reserved DistributedResource(
    Node = Remote REMOTE_ID: 0,
    <Remote: 'inproc://172.31.35.213/1975/1' processes=1 threads=8, memory=33.24 GB>
    nCPUs = 0) in Remote REMOTE_ID: 0,
    <Remote: 'inproc://172.31.35.213/1975/1' processes=1 threads=8, memory=33.24 GB>
Starting Experiments
Num of Finished Tasks is 0
Num of Pending Tasks is 300
  0%|          | 0/76 [00:00<?, ?it/s]Finished Task with config: {'x.choice': 54, 'y.choice': 60} and reward: 0.4131320021194963
Finished Task with config: {'x.choice': 59, 'y.choice': 85} and reward: 0.25248241948989986
Finished Task with config: {'x.choice': 71, 'y.choice': 54} and reward: 0.452177348673922
Finished Task with config: {'x.choice': 84, 'y.choice': 84} and reward: 0.03655423885462522
  1%|▏         | 1/76 [00:00<00:29,  2.54it/s]Finished Task with config: {'x.choice': 42, 'y.choice': 43} and reward: 0.44561928050295485
Finished Task with config: {'x.choice': 62, 'y.choice': 29} and reward: 0.3129734289166378
Finished Task with config: {'x.choice': 64, 'y.choice': 89} and reward: 0.16521724999224768
Finished Task with config: {'x.choice': 38, 'y.choice': 5} and reward: 0.055140459703497166
  3%|▎         | 2/76 [00:00<00:28,  2.60it/s]Finished Task with config: {'x.choice': 96, 'y.choice': 79} and reward: 0.01675323386032184
Finished Task with config: {'x.choice': 27, 'y.choice': 80} and reward: 0.9025895808289532
Finished Task with config: {'x.choice': 38, 'y.choice': 52} and reward: 0.6328227532925973
Finished Task with config: {'x.choice': 47, 'y.choice': 47} and reward: 0.4369372160417949
  4%|▍         | 3/76 [00:01<00:28,  2.61it/s]Finished Task with config: {'x.choice': 92, 'y.choice': 8} and reward: 0.03906803139561688
Finished Task with config: {'x.choice': 56, 'y.choice': 7} and reward: 0.039952445147950336
Finished Task with config: {'x.choice': 39, 'y.choice': 33} and reward: 0.3257993021856096
Finished Task with config: {'x.choice': 83, 'y.choice': 64} and reward: 0.2004520333101668
  5%|▌         | 4/76 [00:01<00:28,  2.56it/s]Finished Task with config: {'x.choice': 2, 'y.choice': 77} and reward: 0.7444461288479184
Finished Task with config: {'x.choice': 36, 'y.choice': 14} and reward: 0.11531548824101008
Finished Task with config: {'x.choice': 95, 'y.choice': 87} and reward: 0.010053264415937721
Finished Task with config: {'x.choice': 83, 'y.choice': 86} and reward: 0.03682398109240761
  7%|▋         | 5/76 [00:01<00:28,  2.50it/s]Finished Task with config: {'x.choice': 97, 'y.choice': 46} and reward: 0.3161396814775167
Finished Task with config: {'x.choice': 46, 'y.choice': 52} and reward: 0.4974644053536026
Finished Task with config: {'x.choice': 79, 'y.choice': 77} and reward: 0.07935556115551717
Finished Task with config: {'x.choice': 79, 'y.choice': 67} and reward: 0.17085551866906823
  8%|▊         | 6/76 [00:02<00:28,  2.46it/s]Finished Task with config: {'x.choice': 11, 'y.choice': 15} and reward: 0.14131830986160224
Finished Task with config: {'x.choice': 71, 'y.choice': 53} and reward: 0.47455256379247157
Finished Task with config: {'x.choice': 63, 'y.choice': 94} and reward: 0.15444995894372754
Finished Task with config: {'x.choice': 57, 'y.choice': 75} and reward: 0.33073294541320397
  9%|▉         | 7/76 [00:02<00:27,  2.50it/s]Finished Task with config: {'x.choice': 51, 'y.choice': 26} and reward: 0.18846597355335637
Finished Task with config: {'x.choice': 10, 'y.choice': 19} and reward: 0.18136358228461952
Finished Task with config: {'x.choice': 40, 'y.choice': 76} and reward: 0.7058736551652897
Finished Task with config: {'x.choice': 46, 'y.choice': 72} and reward: 0.5774605629951269
 11%|█         | 8/76 [00:03<00:27,  2.46it/s]Finished Task with config: {'x.choice': 44, 'y.choice': 1} and reward: 0.03249007863969159
Finished Task with config: {'x.choice': 21, 'y.choice': 33} and reward: 0.4247735686936988
Finished Task with config: {'x.choice': 55, 'y.choice': 60} and reward: 0.39956895655479624
Finished Task with config: {'x.choice': 13, 'y.choice': 16} and reward: 0.1552861522645142
 12%|█▏        | 9/76 [00:03<00:27,  2.41it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 90} and reward: 0.77816551129998
Finished Task with config: {'x.choice': 60, 'y.choice': 94} and reward: 0.18915413038019477
Finished Task with config: {'x.choice': 60, 'y.choice': 67} and reward: 0.3109689510968434
Finished Task with config: {'x.choice': 37, 'y.choice': 45} and reward: 0.5408988749006151
 13%|█▎        | 10/76 [00:04<00:27,  2.44it/s]Finished Task with config: {'x.choice': 34, 'y.choice': 69} and reward: 0.8520424635717397
Finished Task with config: {'x.choice': 60, 'y.choice': 11} and reward: 0.06220932769716972
Finished Task with config: {'x.choice': 42, 'y.choice': 6} and reward: 0.05312575413313245
Finished Task with config: {'x.choice': 89, 'y.choice': 97} and reward: 0.013178123417393045
 14%|█▍        | 11/76 [00:04<00:26,  2.45it/s]Finished Task with config: {'x.choice': 66, 'y.choice': 23} and reward: 0.2529084027182172
Finished Task with config: {'x.choice': 65, 'y.choice': 38} and reward: 0.49583631011448265
Finished Task with config: {'x.choice': 66, 'y.choice': 14} and reward: 0.10416741370431903
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 16%|█▌        | 12/76 [00:04<00:26,  2.46it/s]Finished Task with config: {'x.choice': 30, 'y.choice': 57} and reward: 0.829886169891878
Finished Task with config: {'x.choice': 59, 'y.choice': 3} and reward: 0.025054518512878866
Finished Task with config: {'x.choice': 35, 'y.choice': 46} and reward: 0.5846609186365146
Finished Task with config: {'x.choice': 31, 'y.choice': 62} and reward: 0.8709395257996267
 17%|█▋        | 13/76 [00:05<00:25,  2.50it/s]Finished Task with config: {'x.choice': 98, 'y.choice': 25} and reward: 0.17939595099821065
Finished Task with config: {'x.choice': 95, 'y.choice': 63} and reward: 0.11610891252206063
Finished Task with config: {'x.choice': 9, 'y.choice': 20} and reward: 0.18989747667591955
Finished Task with config: {'x.choice': 64, 'y.choice': 99} and reward: 0.12178118688087612
 18%|█▊        | 14/76 [00:05<00:25,  2.46it/s]Finished Task with config: {'x.choice': 24, 'y.choice': 31} and reward: 0.3817091079995325
Finished Task with config: {'x.choice': 55, 'y.choice': 52} and reward: 0.4028107985736251
Finished Task with config: {'x.choice': 63, 'y.choice': 50} and reward: 0.4368457895739434
Finished Task with config: {'x.choice': 40, 'y.choice': 62} and reward: 0.6969580893232793
 20%|█▉        | 15/76 [00:06<00:24,  2.46it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 67} and reward: 0.9814913209712314
Finished Task with config: {'x.choice': 32, 'y.choice': 40} and reward: 0.5087969233381394
Finished Task with config: {'x.choice': 10, 'y.choice': 20} and reward: 0.19318127651343833
Finished Task with config: {'x.choice': 65, 'y.choice': 76} and reward: 0.20282352738135428
 21%|██        | 16/76 [00:06<00:24,  2.42it/s]Finished Task with config: {'x.choice': 19, 'y.choice': 78} and reward: 0.9600071681988102
Finished Task with config: {'x.choice': 93, 'y.choice': 1} and reward: 0.010896047938056276
Finished Task with config: {'x.choice': 34, 'y.choice': 15} and reward: 0.12939423666456223
Finished Task with config: {'x.choice': 63, 'y.choice': 62} and reward: 0.31372800819230695
 22%|██▏       | 17/76 [00:06<00:23,  2.46it/s]Finished Task with config: {'x.choice': 81, 'y.choice': 96} and reward: 0.031703038768058744
Finished Task with config: {'x.choice': 65, 'y.choice': 84} and reward: 0.17254207043832434
Finished Task with config: {'x.choice': 9, 'y.choice': 52} and reward: 0.7398755445448225
Finished Task with config: {'x.choice': 97, 'y.choice': 54} and reward: 0.21495484912430388
 24%|██▎       | 18/76 [00:07<00:25,  2.26it/s]Finished Task with config: {'x.choice': 97, 'y.choice': 72} and reward: 0.03416132366430215
Finished Task with config: {'x.choice': 5, 'y.choice': 3} and reward: 0.05032274511975561
Finished Task with config: {'x.choice': 55, 'y.choice': 9} and reward: 0.04922766858232881
Finished Task with config: {'x.choice': 41, 'y.choice': 52} and reward: 0.5806808361616504
 25%|██▌       | 19/76 [00:07<00:24,  2.32it/s]Finished Task with config: {'x.choice': 27, 'y.choice': 45} and reward: 0.650688955916307
Finished Task with config: {'x.choice': 97, 'y.choice': 53} and reward: 0.22935714262309276
Finished Task with config: {'x.choice': 27, 'y.choice': 45} and reward: 0.650688955916307
Finished Task with config: {'x.choice': 12, 'y.choice': 24} and reward: 0.2529076252482878
Finished Task with config: {'x.choice': 32, 'y.choice': 69} and reward: 0.8888090750132936
 26%|██▋       | 20/76 [00:08<00:23,  2.38it/s]Finished Task with config: {'x.choice': 29, 'y.choice': 18} and reward: 0.1730263130667964
Finished Task with config: {'x.choice': 85, 'y.choice': 40} and reward: 0.6756957290216257
Finished Task with config: {'x.choice': 37, 'y.choice': 69} and reward: 0.7901920138784992
Finished Task with config: {'x.choice': 88, 'y.choice': 58} and reward: 0.286327467312913
 28%|██▊       | 21/76 [00:08<00:23,  2.34it/s]Finished Task with config: {'x.choice': 48, 'y.choice': 55} and reward: 0.488952436424344
Finished Task with config: {'x.choice': 82, 'y.choice': 90} and reward: 0.03511541242031658
Finished Task with config: {'x.choice': 24, 'y.choice': 31} and reward: 0.3817091079995325
Finished Task with config: {'x.choice': 44, 'y.choice': 90} and reward: 0.48670585177293596
 29%|██▉       | 22/76 [00:09<00:23,  2.33it/s]Finished Task with config: {'x.choice': 48, 'y.choice': 50} and reward: 0.4527992474270069
Finished Task with config: {'x.choice': 8, 'y.choice': 4} and reward: 0.058422749634424065
Finished Task with config: {'x.choice': 26, 'y.choice': 74} and reward: 0.9614044025404535
Finished Task with config: {'x.choice': 89, 'y.choice': 62} and reward: 0.19018552350948792
 30%|███       | 23/76 [00:09<00:22,  2.35it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 46} and reward: 0.6886459183180196
Finished Task with config: {'x.choice': 54, 'y.choice': 49} and reward: 0.4009235733031741
Finished Task with config: {'x.choice': 63, 'y.choice': 45} and reward: 0.46885050272174905
Finished Task with config: {'x.choice': 77, 'y.choice': 52} and reward: 0.5350782292847762
 32%|███▏      | 24/76 [00:09<00:21,  2.39it/s]Finished Task with config: {'x.choice': 46, 'y.choice': 69} and reward: 0.5802833298863385
Finished Task with config: {'x.choice': 44, 'y.choice': 52} and reward: 0.5294533508693043
Finished Task with config: {'x.choice': 2, 'y.choice': 12} and reward: 0.0937625707459735
Finished Task with config: {'x.choice': 17, 'y.choice': 45} and reward: 0.6717131358286801
 33%|███▎      | 25/76 [00:10<00:21,  2.40it/s]Finished Task with config: {'x.choice': 53, 'y.choice': 69} and reward: 0.42463765730607606
Finished Task with config: {'x.choice': 43, 'y.choice': 52} and reward: 0.5462230869026983
Finished Task with config: {'x.choice': 25, 'y.choice': 74} and reward: 0.9700697273516173
Finished Task with config: {'x.choice': 55, 'y.choice': 52} and reward: 0.4028107985736251
 34%|███▍      | 26/76 [00:10<00:20,  2.39it/s]Finished Task with config: {'x.choice': 23, 'y.choice': 52} and reward: 0.8108532667658626
Finished Task with config: {'x.choice': 61, 'y.choice': 69} and reward: 0.2880558719901152
Finished Task with config: {'x.choice': 41, 'y.choice': 52} and reward: 0.5806808361616504
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
 36%|███▌      | 27/76 [00:11<00:20,  2.38it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 88, 'y.choice': 69} and reward: 0.09579273174126507
Finished Task with config: {'x.choice': 78, 'y.choice': 52} and reward: 0.5359258876938673
Finished Task with config: {'x.choice': 47, 'y.choice': 74} and reward: 0.5485750572832516
 37%|███▋      | 28/76 [00:11<00:20,  2.38it/s]Finished Task with config: {'x.choice': 46, 'y.choice': 52} and reward: 0.4974644053536026
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 27, 'y.choice': 52} and reward: 0.7850991782603625
Finished Task with config: {'x.choice': 55, 'y.choice': 52} and reward: 0.4028107985736251
 38%|███▊      | 29/76 [00:11<00:19,  2.43it/s]Finished Task with config: {'x.choice': 72, 'y.choice': 52} and reward: 0.5059934264424871
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 38, 'y.choice': 52} and reward: 0.6328227532925973
Finished Task with config: {'x.choice': 27, 'y.choice': 52} and reward: 0.7850991782603625
 39%|███▉      | 30/76 [00:12<00:18,  2.44it/s]Finished Task with config: {'x.choice': 46, 'y.choice': 74} and reward: 0.5723229680537476
Finished Task with config: {'x.choice': 40, 'y.choice': 52} and reward: 0.5981207041323738
Finished Task with config: {'x.choice': 33, 'y.choice': 52} and reward: 0.713395029404709
Finished Task with config: {'x.choice': 35, 'y.choice': 52} and reward: 0.6827438813536933
 41%|████      | 31/76 [00:12<00:18,  2.44it/s]Finished Task with config: {'x.choice': 30, 'y.choice': 52} and reward: 0.7536095770338388
Finished Task with config: {'x.choice': 37, 'y.choice': 52} and reward: 0.6498556852950799
Finished Task with config: {'x.choice': 37, 'y.choice': 52} and reward: 0.6498556852950799
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
 42%|████▏     | 32/76 [00:13<00:17,  2.45it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 37, 'y.choice': 52} and reward: 0.6498556852950799
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 27, 'y.choice': 74} and reward: 0.9512641104822159
 43%|████▎     | 33/76 [00:13<00:17,  2.48it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
 45%|████▍     | 34/76 [00:13<00:16,  2.48it/s]Finished Task with config: {'x.choice': 31, 'y.choice': 52} and reward: 0.7410814359404405
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
Finished Task with config: {'x.choice': 80, 'y.choice': 52} and reward: 0.5316010536952981
 46%|████▌     | 35/76 [00:14<00:16,  2.48it/s]Finished Task with config: {'x.choice': 46, 'y.choice': 52} and reward: 0.4974644053536026
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
 47%|████▋     | 36/76 [00:14<00:16,  2.45it/s]Finished Task with config: {'x.choice': 27, 'y.choice': 52} and reward: 0.7850991782603625
Finished Task with config: {'x.choice': 35, 'y.choice': 52} and reward: 0.6827438813536933
Finished Task with config: {'x.choice': 31, 'y.choice': 74} and reward: 0.8970279379062928
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 31, 'y.choice': 74} and reward: 0.8970279379062928
 49%|████▊     | 37/76 [00:15<00:16,  2.42it/s]Finished Task with config: {'x.choice': 27, 'y.choice': 52} and reward: 0.7850991782603625
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 93, 'y.choice': 74} and reward: 0.038273278217563975
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
 50%|█████     | 38/76 [00:15<00:15,  2.42it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 52} and reward: 0.8061063575714768
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 51%|█████▏    | 39/76 [00:16<00:15,  2.43it/s]Finished Task with config: {'x.choice': 27, 'y.choice': 74} and reward: 0.9512641104822159
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 53%|█████▎    | 40/76 [00:16<00:14,  2.46it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 52} and reward: 0.8161012247440166
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 54%|█████▍    | 41/76 [00:16<00:14,  2.42it/s]Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 32, 'y.choice': 52} and reward: 0.7276509087726585
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 55%|█████▌    | 42/76 [00:17<00:14,  2.42it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 57%|█████▋    | 43/76 [00:17<00:13,  2.42it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 58%|█████▊    | 44/76 [00:18<00:12,  2.48it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 59%|█████▉    | 45/76 [00:18<00:12,  2.48it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 61%|██████    | 46/76 [00:18<00:12,  2.48it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 62%|██████▏   | 47/76 [00:19<00:11,  2.55it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 63%|██████▎   | 48/76 [00:19<00:11,  2.51it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 64%|██████▍   | 49/76 [00:20<00:10,  2.47it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 66%|██████▌   | 50/76 [00:20<00:10,  2.49it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 67%|██████▋   | 51/76 [00:20<00:10,  2.44it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 68%|██████▊   | 52/76 [00:21<00:09,  2.45it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 70%|██████▉   | 53/76 [00:21<00:09,  2.47it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 71%|███████   | 54/76 [00:22<00:08,  2.45it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 72%|███████▏  | 55/76 [00:22<00:08,  2.47it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 74%|███████▎  | 56/76 [00:22<00:08,  2.49it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 75%|███████▌  | 57/76 [00:23<00:07,  2.48it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 76%|███████▋  | 58/76 [00:23<00:07,  2.43it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 78%|███████▊  | 59/76 [00:24<00:06,  2.43it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 79%|███████▉  | 60/76 [00:24<00:06,  2.38it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 80%|████████  | 61/76 [00:25<00:06,  2.34it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 82%|████████▏ | 62/76 [00:25<00:05,  2.37it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 83%|████████▎ | 63/76 [00:25<00:05,  2.39it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 84%|████████▍ | 64/76 [00:26<00:05,  2.34it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 86%|████████▌ | 65/76 [00:26<00:04,  2.34it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
 87%|████████▋ | 66/76 [00:27<00:04,  2.37it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 88%|████████▊ | 67/76 [00:27<00:03,  2.31it/s]Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 89%|████████▉ | 68/76 [00:28<00:03,  2.33it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 91%|█████████ | 69/76 [00:28<00:02,  2.35it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 92%|█████████▏| 70/76 [00:28<00:02,  2.41it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 93%|█████████▎| 71/76 [00:29<00:02,  2.44it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 95%|█████████▍| 72/76 [00:29<00:01,  2.43it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 16, 'y.choice': 74} and reward: 0.9772040425573253
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 96%|█████████▌| 73/76 [00:30<00:01,  2.40it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
 97%|█████████▋| 74/76 [00:30<00:00,  2.40it/s]Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
Finished Task with config: {'x.choice': 21, 'y.choice': 74} and reward: 0.9892484241569526
100%|██████████| 76/76 [00:30<00:00,  2.46it/s]
Best config: {'x.choice': 21, 'y.choice': 74}, best reward: 0.9892484241569526

Compare the Performance

Get the result history:

results_rl = [v[0]['accuracy'] for v in rl_scheduler.training_history.values()]
results_random = [v[0]['accuracy'] for v in random_scheduler.training_history.values()]

Average result every 10 trials:

import statistics
results1 = [statistics.mean(results_random[i:i+10]) for i in range(0, len(results_random), 10)]
results2 = [statistics.mean(results_rl[i:i+10]) for i in range(0, len(results_rl), 10)]

Plot the results:

plt.plot(range(len(results1)), results1, range(len(results2)), results2)
[<matplotlib.lines.Line2D at 0x7f9ced1ccad0>,
 <matplotlib.lines.Line2D at 0x7f9ced1cc5d0>]
../../_images/output_rl_searcher_ecab16_17_1.png