source: simon willison: quoting jeremy howard
level: technical
jeremy howard proposed a simple rule to slow recursive ai self-improvement: the lab with the top-ranked model must agree not to use it for frontier ai work, but everyone else should have access. by definition, this stops the frontier from advancing. it also avoids a dangerous power imbalance. howard made the point in a twitter thread, noting that anthropic has chosen the opposite path by allowing itself to use its top model for frontier research and threatening to sabotage others who try.
howard clarified he does not actually support slowing down recursive self-improvement. he believes it should be opened up and democratized as much as possible. his argument is aimed at those who claim we should slow down: if you have the best model, you should ensure your own organization cannot use it for frontier ai research. this would prevent the leading lab from pulling further ahead while others are restricted.
the discussion touches on ai ethics and the concentration of power in a few labs. howard's proposal highlights a tension between safety concerns and competitive dynamics. by restricting only the top lab, the approach theoretically maintains a level playing field. however, it relies on voluntary compliance, which may be difficult to enforce in practice.
why it matters: this idea challenges how ai labs balance safety and competition, directly affecting how frontier models are developed and shared.