Some questions about how the optimizer works and best practice in usage

[Matthies]

No issue but some questions came to my mind while I'm using your very useful and promising tool (thanks for developing it)...

1. Lets say ran the optimizer for 100 budgets, each budget playing 500 games and I notice that I get high values for the loss function, most of them > 0.9 and all far from beating the 0.5. Is it okay to continue the optimizer on this data file for another 100 or even more budgets but now with reduced games per budget, e.g. 200 so save some time assuming that the optimizer needs a lot more runs to get even close to the 0.5 area? Or is it important to use a constant games_per_budget for the complete process?
2. Lets assume the optimizer found a very good set of values with a loss value of 0.45. And lets assume this good score was a fluke, maybe caused by too small games_per_budget. Will these "fluked" parameter set ever be retested? Will other value sets with a "real" loss value of 0.47 ever get a change to replace the fluked one?
3. Maybe related to the last question: When continuing an optimization, should I replace the original init values (I usually use the defaults of the engine when starting the optimization for the first time) with the best values found so far (e.g. the values from the fluked set in last question). Can I still use the same --input-data-file (= --output-data-file of last run) even if I change the inits?
4. Replacing the base engine... should I ever do that? Probably only when I found something that passes the verification test.

I could probably write some more questions but lets go step by step.
Thanks in advance.

[fsmosca]

No issue but some questions came to my mind while I'm using your very useful and promising tool (thanks for developing it)...

1. Lets say ran the optimizer for 100 budgets, each budget playing 500 games and I notice that I get high values for the loss function, most of them > 0.9 and all far from beating the 0.5. Is it okay to continue the optimizer on this data file for another 100 or even more budgets but now with reduced games per budget, e.g. 200 so save some time assuming that the optimizer needs a lot more runs to get even close to the 0.5 area? Or is it important to use a constant games_per_budget for the complete process?

The optimizer is dependent on the output of the objective that is the result of the match. You can adjust the number of games per budget.

2. Lets assume the optimizer found a very good set of values with a loss value of 0.45. And lets assume this good score was a fluke, maybe caused by too small games_per_budget. Will these "fluked" parameter set ever be retested?

This depends on the optimizer, optimizer oneplusone will retest promising param values.

Will other value sets with a "real" loss value of 0.47 ever get a change to replace the fluked one?

The optimizer does not know if the param of loss 0.45 is a fluke or not, all it does is minimize the loss. Since the best loss so far is 0.45 it will try to keep on reducing it (run more budgets) by suggesting param values that it deemed best according to its algorithm.

3. Maybe related to the last question: When continuing an optimization, should I replace the original init values (I usually use the defaults of the engine when starting the optimization for the first time) with the best values found so far (e.g. the values from the fluked set in last question). Can I still use the same --input-data-file (= --output-data-file of last run) even if I change the inits?

The init value is not part of nevergrad parametrization, I use it for base engine param. Yes you can change it. But do not change the upper and lower values as nevergrad knows about this range. There could be consequences too as the data recorded by optimizer is against different init values.

There is a way of using the best param found so far as opponent of optimizer suggested param, use the --use-best-param flag. By default Lakas uses the init or default param against the suggested param from optimizer. By using the --use-best-param lakas will use the current best param found so far against the optimizer suggested values. For example if the current best loss is 0.45, the param of this loss will play against the optimizer suggested param values. By default when optimizer param is better than base param by more than 0.5 or 50% the optimizer param will become the best param. You can control this 0.5 value by using the option --best-result-threshold <value>. This is used to give allowance for the noise generated from the test environment, tc conditions, number of games per budget and other noises. The idea is that when we use 0.6 as result threshold and test or optimizer param values wins against the init or current best param by only 0.55, the best param is not updated because 0.55 is not more than 0.6. The optimizer param values has to perform more than 0.6 for its suggested param values to become the best param. The 0.55 could be just a fluke and what we need is more than 0.6. This method is useful when we are running low number of game per budget too. Like for example.

--use-best-param --best-result-threshold 0.6 --games-per-budget 24

That would mean the optimizer param values has to perform a minimum of 14.5/24 for it to become the best param. The games per budget is low but we are not replacing the init or default values that easily. We need +5 points to replace our init value. If the optimizer param is more that 0.6 then it will become the current best param. Note the first current best param is the init param or default param. In the next budget, the opponent of optimizer suggested param values is now our current best param not the old init param.

Without using the --use-best-param flag lakas will always use the init param against the optimizer suggested param.

4. Replacing the base engine... should I ever do that? Probably only when I found something that passes the verification test.

Probably not a good idea as the optimizer is dependent also on the optimization records, the param values tried and its equivalent loss values.

[Matthies]

That helps a lot, thank you. Seems that --use-best-param is what I'm searching for to replace a "fluke" result.
I will leave this open for now, maybe I'll come back with more questions :-)

[fsmosca]

I have to correct myself, init is part of nevergrad, not sure if the old version does not consider init, but regardless this is the current info.