When you no longer need humans to lead AI improvements, humans are already obsolete
we're not really there yet tho, current models still need humans to pick the data, write the evals and decide what counts as better. the scary point is when the evals themselves get automated, thats when it actually closes the loop imo