2 OpenAI researchers working on safety and governance have quit

• OpenAI researchers Daniel Kokotajlo and William Saunders recently left the company behind ChatGPT.

• Kokotajlo said on a forum he doesn't think OpenAI will "behave responsibly around the time of AGI."

• Kokotajlo was on the governance team, and Saunders worked on the Superalignment team at OpenAI.

Two OpenAI employees who worked on safety and governance recently resigned from the company behind ChatGPT.

Daniel Kokotajlo left last month and William Saunders departed OpenAI in February. The timing of their departures was confirmed by two people familiar with the situation. The people asked to remain anonymous in order to discuss the departures, but their identities are known to Business Insider.

Kokotajlo, who worked on the governance team, is listed as an adversarial tester of GPT-4, which was launched in March last year. Saunders had worked on the Alignment team, which became the Superalignment team, since 2021.

Kokotajlo wrote on his profile page on the online forum LessWrong that he quit "due to losing confidence that it would behave responsibly around the time of AGI."

In a separate post on the platform in April, he partially explained one of the reasons behind his decision to leave. He also weighed in on a discussion about pausing AGI development.

"I think most people pushing for a pause are trying to push against a 'selective pause' and for an actual pause that would apply to the big labs who are at the forefront of progress," Kokotajlo wrote.

He added: "The current overton window seems unfortunately centered around some combination of evals-and-mitigations that is at IMO high risk of regulatory capture (i.e. resulting in a selective pause that doesn't apply to the big corporations that most need to pause!) My disillusionment about this is part of why I left OpenAI."

Saunders said in a comment on his LessWrong profile page that he resigned that month after three years at the ChatGPT maker.

The Superalignment team, which was initially led by Ilya Sutskever and Jan Leike, is tasked with building safeguards to prevent artificial general intelligence (AGI) going rogue.

Sutskever and Leike have previously predicted that AGI could arrive within a decade. It's not clear if Sutskever is still at OpenAI following his participation in the brief ousting of Sam Altman as CEO last year.

Saunders was also a manager of the interpretability team, which researches how to make AGI safe and examines how and why models behave the way they do. He has co-authored several papers on AI models.

Kokotajlo and Saunders' resignations come amid other departures at OpenAI. Two executives, Diane Yoon and Chris Clark, quit last week, The Information reported. Yoon was the VP of people, and Clark was head of nonprofit and strategic initiatives.

OpenAI also parted ways with researchers Leopold Aschenbrenner and Pavel Izmailov, according to another report by The Information last month.

OpenAI, Kokotajlo, and Saunders did not respond to requests for comment from Business Insider.

Do you work for OpenAI? Got a tip? Contact this reporter at jmann@businessinsider.com for a nonwork device.

Read the original article on Business Insider