Projects
A Spectral Theory of Grokking
Grokking occurs when a neural network sharply transitions from a memorized to a generalized state. It first reaches almost perfect training accuracy while its validation accuracy remains near chance.11. Throughout this post, “validation” refers to the held-out part of the training distribution. After many more updates, validation accuracy rises abruptly, even though the training data has not changed. The network appears to move from memorizing examples to learning the “rule” behind them. We study what changes inside the network during that delay.
Evolutionary Processes
Work in Progress
Come back soon!
Foreknowledge
Work in Progress
Come back soon!
High Selection Evolution
Work in Progress
Come back soon!
Neural Operators
Work in Progress
Come back soon!
Supernash
Supernash Project
This figure represents the main method of the Supernash Project. One finds a Nash equilibrium in the Memory 1 Infinitely Repeated Game by finding a strategy that puts the blue dot into the dark region in the plane.
Introduction
We consider the infinitely repeated game. In each round, players can choose either to cooperate or defect. A strategy belongs to the Memory 1 space if it conditions its next move solely on the previous round. In our case, the strategy is parameterized by four probabilities representing the outcomes of the last round: