Self-reinforcing feedback loops will become a cornerstone of superintelligence. Games offer key principles that could build such loops: competition and cooperation. In (non-cooperative) games, progress in one player creates new challenges, and hence, learning signals for others. From GANs, AlphaZero, and thousands of agents collaborating on a Millennium Prize Problem, we've seen a potential of this principle.
However, games are notoriously difficult to solve. Hence, scaling multi-agent learning will require more than just scaling experiments. In particular, games often exhibit limit cycles, chaotic behaviors, and intractable equilibria. Such extreme pathologies call for a theory that could identify a necessary structure for efficient learning.
As theoretical physics guided the discovery of Higgs boson, black holes, and gravitational waves, a predictive theory of games between multiple neural nets will reduce the cost of experiments by orders of magnitude. With mathematical superintelligence now in the hands of theorists, I believe this theory is coming within reach.
“ When can simple learning rules solve games efficiently? ”
A single basic question lies at the foundation of multi-agent learning: what makes a class of games computationally tractable, and what makes it intrinsically hard? Answering the question requires us to understand the obstacles that algorithms can overcome, and obstacles that require additional structures. Recent advances in self-play acceleration and decentralized learning in Markov games reveal how different types of interaction and game structures can enable efficient learning. Nevertheless, identifying a minimal structure for nonconvex games, understanding self-play acceleration under different types of feedbacks, and determining (sub)optimality of simple first-order methods, still remain open.
“ When would games induce sample-efficient learning dynamics for neural nets? ”
Interaction between multiple neural nets could make useful features easier to learn. Recent theory explains how useful features can be discovered sequentially, and under suitable conditions, learned efficiently in multi-index models. These results identify the structures that enable efficient feature learning, but whether dynamic interactions between multiple neural nets could create such structures, has been unexplored. For instance, whether collaboration could induce sample-efficient feature learning, and whether competition could induce efficient curricula, remain largely open.
“ When can non-cooperative games provably guarantee alignment and monitorability? ”
Carefully designed non-cooperative games between multiple neural nets could provide alignment guarantees without expecting every participants to be trustworthy. The goal is to identify incentive structures under which every equilibrium would satisfy human-alignment, pluralistic-alignment, or a guarantee that misbehaviors would remain detectable through CoT monitoring. AI safety via debate motivates this approach, while the emergence of concealed misbehavior shows why oversight incentives require careful design. Establishing provable guarantees under imperfect feedback, conflicting human preferences, and determining whether each player could actually reach a meaningful equilibrium, remain intriguing open problems.
I'm actively looking for a Ph.D. opportunity at the intersection of (non-cooperative) games and deep learning. I'd be happy to discuss research with anyone interested. Please feel free to reach out to me via email: junsoo.ha.contact@gmail.com