Currently we have an approval gate after the
--dry-run
and before the real upload where we check the output of the pipeline and ensure the dry-run looks correct. This is the type of check an AI could easily do.
How I envision this feature working:
  1. It would be an option on regular approval gates to allow the AI to also approve based on the pipeline logs (and previous approved pipeline logs as reference).
  2. You would be able to set the "Confidence Threshold" of the AI, things like "Only exact matches to previous runs." or "Allow similar outputs to previously approved runs." etc. This could be just be a freeform instruction field in the
    config.yml
    .
  3. A human would always still be able to manually approve the gate as before.
  4. The user context/permissions running the pipeline would not change when the gate is approved by an AI. (Currently when someone approves a pipeline started by someone else, the pipeline gets the permissions of the approver).