Here is the onboarding scenario I'm envisioning:

• The AI is put on a laptop/desktop/workstation that it has root permissions on

• It's given access to a shared work chat (Slack, Discord, whatever) where it can ask questions/request additional permissions/etc

• It needs to setup a workstation that can pull, build, and edit at least one of the org's primary codebases.

• It needs to make one accepted pull request (or equivalent concept).

• The workstation can be the local machine or a cloud instance

• Time limit of five business days.

In short: an AI that can go through the onboarding process for a software engineer.

Caveats:

• Resolves NO if no one tries to do this (which seems reasonably likely)

• The pull request does not need to be interesting or important. A "hello world" PR is fine.

• YES does not require that an AI be as good at this as a human engineer, but it can't be significantly specialized for one environment. It needs to work with at least three different environments (e.g. setup Python locally, setup typescript on AWS, and C++ on Azure).