Programme

Abstract:
Expensive black-box optimization has produced several mature research traditions, including DIRECT-type partitioning methods, deterministic response-surface optimization, model-based derivative-free optimization, Bayesian optimization, and surrogate-assisted evolutionary algorithms. Although these approaches address closely related problems, they have developed largely within separate communities, with distinct terminology, software ecosystems, benchmark suites, and experimental conventions. This talk reviews their common principles, methodological differences, and current research directions, then examines the surprisingly limited evidence from direct cross-family comparisons.

Abstract:
Real-world optimization problems are frequently expensive, with objectives and constraints assessed through costly simulations or experiments. Conventional evolutionary computation treats each solution evaluation as a single, monolithic black box returning all objective and constraint values simultaneously. Yet in many practical settings, these functions can be evaluated independently, sometimes at different costs. This creates an opportunity for substantial savings through partial evaluation, that is, evaluating only a subset of constraints or objectives for a candidate solution to guide the search.

In this talk, I outline some of our work in this direction. This includes initial studies combining evolutionary constraint-handling schemes with random sequencing, followed by more principled approaches such as feasibility-ratio-based sequencing. I then discuss extensions to computationally expensive constrained optimization via surrogate-assisted algorithms, to settings with heterogeneous evaluation costs across constraints, and to multi-objective optimization where objectives are selectively evaluated. Together, this line of work highlights that deciding what to evaluate, not only where to sample, is a promising approach for reducing the cost of expensive optimization, with several open challenges remaining.

Abstract:
Large Language Models (LLMs) are increasingly being used not only to assist optimization, but to generate optimization algorithms themselves. But what do LLMs actually design when given this freedom? Do they discover genuinely new optimization strategies, intelligently recombine existing ideas, or simply exploit the benchmarks used to evaluate them?

In this talk, I use LLaMEA-BO, an evolutionary framework in which LLMs iteratively generate and improve Bayesian optimization code, as a case study to explore these questions. Rather than focusing only on performance, I look inside the generated algorithms, examining which design patterns emerge, how they vary across dimensions and tasks, and whether they rediscover, recombine, or depart from established BO principles. From these observations, I discuss the current capabilities and limitations of LLM-driven algorithm design, including novelty, generalization, benchmark overfitting, and interpretability.