Analysis
This week's Euler equations proof-and-credit fight between an NYU mathematician, an Anthropic researcher, and OpenAI's Sébastien Bubeck -- see Pulse's full writeup for what's confirmed and what's contested -- is exactly the kind of story I want every LP asking their GPs about, because it's the clearest public example yet of something diligence teams have been guessing at privately for two years: frontier labs' claims about what their models can do are no longer just marketing copy you take on faith -- they're now contestable in public, by named researchers, with call logs and timestamps attached.
I've sat through enough pitch decks with a slide that says "our model achieved X" with a footnote citing an internal benchmark or a cherry-picked demo. That's been the norm because there was rarely anyone positioned to publicly dispute it -- academic researchers using a lab's API don't usually have the standing or the incentive to pick a fight with OpenAI or Anthropic. Buckmaster does, because he's tenured, his own name is on the underlying math, and he has nothing to lose by going public. That combination is rare, and it's exactly why this dispute deserves more diligence attention than the Euler equations result itself.
“I've sat through enough pitch decks with a slide that says "our model achieved X" with a footnote citing an internal benchmark or a cherry-picked demo.”
What I'd actually diligence differently after this: when a lab cites an internal research result as evidence of model capability -- not a published benchmark, an internal claim -- I want to know who outside the company can verify it, and whether that person has any incentive not to. If the answer is "nobody, and no," that claim is worth roughly the same as the pitch deck footnote.
Room for disagreement: the counterargument is that this is one dispute between two research collaborators and a competing lab, not evidence of a systemic pattern -- most frontier-lab capability claims never get contested publicly, but that could just as easily mean nobody with standing has bothered to check, not that the claims are clean. I'd also concede that Bubeck disputes Buckmaster's characterization entirely, and we don't yet have his full account of the calls.
