Skip to content

Support parallel execution for evaluation command #485

Description

@Leward

Running evaluations in the current neuron-core/neuron-ai repository is slow because the evaluation command processes evaluators and dataset items strictly sequentially.

Details

  • The EvaluationCommand runs each evaluator one by one.
  • Within each evaluator, dataset items are processed in sequence via EvaluatorRunner::run().
  • For larger datasets or many evaluators, this becomes a bottleneck.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or improvement idea

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions