Skip to content

bug: add_images_and_text to HF dataset not filtering references #128

Open
Open
bug: add_images_and_text to HF dataset not filtering references#128
Labels
Team DatabasebugSomething isn't working
Milestone
Peer reviewed journal submission

Description

@georgia-hf
georgia-hf
opened on Jul 31, 2025

The current script to add images and text to the huggingface dataset from PDF links is not filtering the extracted text to remove references and appendices. This should be added.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Team DatabasebugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions