Great explanation, thanks! Agree on all your points about GAs. They're elegant for sure, but definitely cumbersome to truly get them to work. I know VNS but have never used it in practice. My go-to metaheuristic for highly structured problems is grammatical evolution, especially an EDA version we designed during my PhD that works like ✨ in program synthesis. We work on automated ML which is basically evolving ML pipelines, it's a very high dimensional and highly structured search space, because you have to pick which algorithms to use, in which order, and then hyperparameters, which can be other algorithms, so the search space is hierarchical. Also some changes are very non-local, suffice to say it's a hell of a search problem. So nothing based on changing solutions seems to work, and we had to resort to EDAs but there were no EDAs for such highly structured spaces, so we came up with a fairly straightforward way to turn grammatical evolution into an EDA using a probabilistic grammar and a bayesian learning framework to update production probabilities based on sampled pipeline' fitness. It was a really fun project to work on.
First I hope to write one article on each of the algorithms I mentioned, as a way to show the many ways in which computer science can be beautiful. Haven't considered writing about grammatical evolution, seems too niche, but now that you mentioned it, I might just do it ;)
Great explanation, thanks! Agree on all your points about GAs. They're elegant for sure, but definitely cumbersome to truly get them to work. I know VNS but have never used it in practice. My go-to metaheuristic for highly structured problems is grammatical evolution, especially an EDA version we designed during my PhD that works like ✨ in program synthesis. We work on automated ML which is basically evolving ML pipelines, it's a very high dimensional and highly structured search space, because you have to pick which algorithms to use, in which order, and then hyperparameters, which can be other algorithms, so the search space is hierarchical. Also some changes are very non-local, suffice to say it's a hell of a search problem. So nothing based on changing solutions seems to work, and we had to resort to EDAs but there were no EDAs for such highly structured spaces, so we came up with a fairly straightforward way to turn grammatical evolution into an EDA using a probabilistic grammar and a bayesian learning framework to update production probabilities based on sampled pipeline' fitness. It was a really fun project to work on.
Wow, it sounds super interesting! Now that you're talking about beautiful algorithms in your newsletter... Would you write about all that stuff? :-)
First I hope to write one article on each of the algorithms I mentioned, as a way to show the many ways in which computer science can be beautiful. Haven't considered writing about grammatical evolution, seems too niche, but now that you mentioned it, I might just do it ;)
Great article and excellent contributions.
Glad you liked it, Toni :-)