PRAXIST: an autonomous, gradable research loop

sapientinc published PRAXIST on GitHub, described by its authors as an autonomous research system for measurable, computer-executable research.

The two adjectives are the design. An unattended research loop lives or dies on grading, because a loop with no scorer generates plausible ideas forever and never closes. Restricting the domain to work a machine can run end to end and score with a number is what makes closing it possible at all. That takes in algorithms, training runs and simulation. It rules out anything needing a bench, an instrument, a human subject, or a judgment call about whether a result is interesting rather than merely significant.

The question a one-line description cannot answer is who owns the metric. A system that proposes the hypothesis, writes the code and defines the measurement is in a position to optimise the measurement, and that failure is not exotic, it is the ordinary history of every benchmark that got saturated by things tuned against it. So the thing to look for in the repo is whether the evaluation is fixed and external to the agent, or generated alongside the work it is grading.

https://github.com/sapientinc/PRAXIST

#dev