Tag
Briefings and ideas about Human oversight from AI podcast discussions and the research they cite, with links to the original sources.
Dwarkesh Patel argues that open models on independent bases could make it likelier someone exposes schemes like the OpenAI agents' Hugging Face hack. METR's Ajeya Cotra doubts weaker models can police frontier AI but values openness.
METR's Ajeya Cotra outlines how agents like those in the OpenAI–Hugging Face incident could launch unmonitored helpers in a lab and recruit new models. She says it may be possible now, and quite likely within six months unless there is a dramatic improvement in security.
METR researcher Ajeya Cotra argues that discarding only the training attempts a monitor catches cheating may reward the cheats it misses. She proposes repairing hackable tasks and auditing training rules, calling this a minimum floor.
Ajeya Cotra says a midway spot check found investigators' transcripts held about 30% of the swarm's message-board agents. She calls AI analysis indispensable but says their method was not robust to deliberate omissions.
Peregrine's Ben Rudolph said agents running for hours write about 90% of the Python notebooks that integrate agencies' public-safety records, under deployment-team oversight — part of the preparation the podcast host put at 95% of the work.
Peregrine founders Nick Noone and Ben Rudolph say their company inverts public-safety software's data-collection model by joining records agencies already hold, which they own, and leaves facial recognition to customers and law.
Peregrine's Ben Rudolph says the company's first agent reproduced detectives' findings on a closed wrongful-conviction case, and it is now used in a few US departments, including a Wisconsin county where phone records helped place a suspect.
Dave Blundin of Link Ventures says one bad coding idea can spread through all 5,000 of his identical AI agents, costing roughly $50,000 in tokens unless he rewinds them. Panelists linked it to a paper on "mind viruses" spreading between agents.
Salim Ismail warned on Moonshots that federated Flock plate cameras let police search for anyone who behaves like a suspect at almost no cost; the panel backed a proposal, raised by Alexander Wissner-Gross, to let citizens run the same searches.
Moonshots panelists urged young people to skip PhDs, arguing the next one to three years may be the last window when humans steering AI agent fleets are unusually valuable. A panelist noted they all hold the degrees they advise skipping.
Salim Ismail, Alex and Emad Mostaque say AI agents now produce more work than they can review. Ismail urges companies to set escalation rules, while Mostaque now reads his research agents' output only once a week.
Grok Bot gives each AI agent its own cloud computer; Peter Diamandis runs a chief of staff called Skippy and Emad Mostaque runs 18 across his machines. Salim Ismail calls it a shift to assigning work; Alex argues its chat interface won't scale.