bg-remover-algos

Open-source background removal research workbench covering deterministic algorithms, classical computer vision, and ML-based matting/segmentation. The Rust core provides a fast library and CLI for production-friendly deterministic masks; the Python research layer is for OpenCV, ONNX, PyTorch, video, and model-comparison experiments.

This repo is intentionally multi-language. Rust is used where portability and speed matter; Python is used where the research ecosystem is strongest.

For background-removal terminology, research context, and future classical/deep-learning directions, see docs/research.md.

The generated LaTeX-style equation card gallery is in docs/algorithm-equations.md.

Features

• Black and white luma threshold removal
• RGB chroma key removal with tolerance and feathering
• Automatic background-color keying from histogram/border smoothness cues
• Edge-connected border flood-fill removal for plain backgrounds
• Otsu thresholding with blur and color binning mask generation
• Truncated luma thresholding
• HSV saturation/value masking
• Clean-background difference masking
• Transparent output or solid background compositing
• Preserves existing foreground alpha in PNG inputs
• Standalone mask generation for custom pipelines
• Mask export and post-processing from the CLI
• Optional Rayon-backed parallel mask application with the parallel feature
• Python research implementations for adaptive thresholding, region/cluster methods, watershed, GrabCut, active-contour/skimage adapters, U2-Net/MODNet ONNX adapters, DeepLab/U-Net-style segmentation adapters, and video experiments
• Library, CLI, and research entry points

Install

cargo install --path .

Or run from a checkout:

cargo run -- --input assets/test1.png --output out.png --mode otsu-binning

Enable parallel mask application when processing larger batches or high-resolution images:

cargo run --features parallel -- --input assets/test1.png --output out.png --mode otsu-binning

CLI Usage

bg-remover-algos --input input.png --output output.png --mode white --threshold 245

Available modes:

black
white
color
otsu-binning
threshold-trunc
hsv-mask
background-diff
auto-color
border-flood

Examples:

# Remove a white background into transparency.
bg-remover-algos -i product.png -o product-transparent.png -m white --threshold 245

# Remove a green-screen background with soft edges.
bg-remover-algos -i subject.png -o subject.png -m color --key-color 00FF00 --tolerance 0.12 --feather 0.08

# Estimate the border/background color automatically, then key it out.
bg-remover-algos -i product.png -o product-transparent.png -m auto-color --tolerance 0.12 --feather 0.04

# Remove only background-colored regions connected to the image border.
bg-remover-algos -i product.png -o product-transparent.png -m border-flood --tolerance 0.12

# Replace the removed background with a custom solid color.
bg-remover-algos -i product.png -o product-solid.png -m white --background-color "#F8F8F2"

# Compare against a clean background plate and save the processed mask too.
bg-remover-algos -i subject.png -o subject.png -m background-diff --background-image clean-bg.png --mask-output mask.png --mask-blur 1.0

# Clean up masks, keep the largest component, save trimap and JSON report.
bg-remover-algos -i product.png -o product.png -m white \
  --mask-open 1 --mask-close 1 --largest-component --fill-holes \
  --trimap-output product-trimap.png --json-report product-report.json

# Batch process a directory with masks, trimaps, and reports.
bg-remover-algos --input-dir datasets/sample/images --output-dir runs/rust-white --mode white

--background-color implies solid output. --solid without a color uses white.

Library Usage

use bg_remover_algos::{
    remove_white_background,
    remove_white_background_with_background,
    white_background_mask,
    Background,
};
use image::Rgb;

let img = image::open("input.png")?;

let transparent = remove_white_background(&img, 245);
transparent.save("transparent.png")?;

let mask = white_background_mask(&img, 245);
mask.save("mask.png")?;

let solid = remove_white_background_with_background(
    &img,
    245,
    Background::Solid(Rgb([248, 248, 242])),
);
solid.save("solid.png")?;
# Ok::<(), Box<dyn std::error::Error>>(())

Algorithm Behavior

The classic threshold methods classify pixels with simple luma comparisons. The chroma key method uses Euclidean RGB distance from the key color and can produce partial alpha in the feather band.

Otsu binning computes its mask from a blurred, binned copy of the image, then applies that mask to the original image so foreground colors are preserved.

Each algorithm exposes a mask function that returns a grayscale image where 0 means removed background, 255 means kept foreground, and intermediate values represent soft edges where supported.

When transparent output is selected, removed pixels get alpha 0 and kept pixels retain their original input alpha. When solid output is selected, pixels are composited over the chosen background color and the output is fully opaque.

The CLI can also save masks with --mask-output and post-process them with --mask-erode, --mask-dilate, and --mask-blur. It also supports --mask-open, --mask-close, --largest-component, --fill-holes, --trimap-output, and --json-report.

Research Layer

The research/python package contains prototypes and adapters for algorithms that are better researched in Python:

• OpenCV-style Otsu/binning, threshold truncation, HSV masking
• Kapur entropy, Ridler-Calvard, Niblack, Sauvola, and Bernsen threshold masks
• Chroma key, auto-chroma key, border flood-fill, k-means color clustering, mean-shift, split-and-merge, SRM-style merging, watershed, Canny contour masking, and clean-background difference masks
• GrabCut interactive foreground extraction
• Optional scikit-image adapters for Random Walker, Active Contours, and Geodesic Active Contours
• U2-Net/U2-NetP ONNX salient-object masks
• MODNet ONNX portrait matting
• DeepLab/U-Net-style ONNX semantic segmentation
• Frame-wise video experiments
• Evaluation metrics: IoU, Dice/F1, precision, recall, MAE, MSE, SAD, boundary F-score
• Contact-sheet visual reports and CSV/JSON experiment summaries

Model weights are not vendored. Experiments should document the exact model, license, preprocessing, dataset, and metrics used.

Example research run:

python -m bg_remover_research.synthetic
python -m bg_remover_research.run \
  --algo otsu-binning \
  --input datasets/sample/images \
  --masks datasets/sample/masks \
  --output runs/otsu-baseline

Compare Rust and Python/OpenCV implementations on the same fixtures:

cargo build
PYTHONPATH=research/python python -m bg_remover_research.compare \
  --input datasets/sample/images \
  --masks datasets/sample/masks \
  --backgrounds datasets/sample/backgrounds \
  --output runs/compare-baseline

Open runs/compare-baseline/index.html to inspect the visual comparison dashboard.

Roadmap

The next practical upgrades are GrabCut in Rust, Bayesian/Poisson matting prototypes, model-evaluation notebooks, and more datasets. Longer-term ML backends such as U2-Net, MODNet, BASNet, DeepLab, and U-Net should stay optional so the default Rust core remains lightweight and deterministic.

Benchmarks

cargo bench

Criterion benchmarks currently cover the Rust mask generators on a synthetic 640x480 image.

Development

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-targets --all-features

The integration tests also regenerate outputs from assets/test1.png and compare them against the checked-in golden assets.

License

MIT