Running evaluations in the current neuron-core/neuron-ai repository is slow because the evaluation command processes evaluators and dataset items strictly sequentially.
Details
- The
EvaluationCommand runs each evaluator one by one.
- Within each evaluator, dataset items are processed in sequence via
EvaluatorRunner::run().
- For larger datasets or many evaluators, this becomes a bottleneck.
Running evaluations in the current
neuron-core/neuron-airepository is slow because the evaluation command processes evaluators and dataset items strictly sequentially.Details
EvaluationCommandruns each evaluator one by one.EvaluatorRunner::run().