Skip to content

CLAUDE.md

Latest commit

 

History

History
147 lines (113 loc) · 11.1 KB

CLAUDE.md

File metadata and controls

147 lines (113 loc) · 11.1 KB
Raw

paper-to-storyboard

This repo is the source of the paper-to-storyboard Claude Code Skill. The skill turns an academic PDF into a dark scroll-snap webpage (HTML + CSS + vanilla JS + transparent figures + AI cover).

Origin: https://github.com/MaoranSun/paper-to-storyboard (public, MIT). Always confirm git remote -v matches this before pushing — a divergent fork would be a sign the working tree was cloned elsewhere.

Layout

paper-to-storyboard/        (this repo — the canonical source of truth)
├── skill/                  # the skill itself; copied/symlinked to ~/.claude/skills/
│   ├── SKILL.md
│   ├── templates/          # index.html.tmpl, style.css.tmpl, script.js.tmpl, section_snippets.html
│   ├── scripts/            # extract_text, extract_figures, make_transparent, generate_cover, render, preview
│   ├── palettes/themes.json
│   ├── schemas/storyboard.schema.json
│   └── examples/reference_storyboard.json
├── examples/               # rendered storyboards (one self-contained dir per paper)
│   ├── README.md           # per-example layout convention + how to render/preview
│   └── SCS_storyboard/     # Sun & Bardhan 2024 (cool / light / modern)
│       ├── README.md       # paper citation + style combo + view instructions
│       ├── index.html      # rendered chassis
│       ├── style.css
│       ├── script.js
│       ├── storyboard.json # editable narrative — re-render after edits
│       ├── cover.png       # AI-generated title cover
│       ├── figureN.png     # transparent figures lifted from the paper
│       └── screenshot.png  # README hero image (1800px max, ~2 MB)
├── install.sh              # copies ./skill/ → ~/.claude/skills/paper-to-storyboard/
├── requirements.txt
├── README.md               # public-facing
├── LICENSE                 # MIT
└── CLAUDE.md               # this file

The installed skill at ~/.claude/skills/paper-to-storyboard/ is a SYMLINK to ./skill/ in this repo. ⚠️ Always edit ./skill/ here. Edits made through the symlinked path technically work (they hit the same files), but if a future contributor reinstalls with ./install.sh (copy mode) the symlink turns into a real directory and edits diverge silently. Verify with ls -la ~/.claude/skills/paper-to-storyboard — you should see -> /Users/maoransun/GitHub/paper-to-storyboard/skill. If it isn't a symlink, run ./install.sh --symlink once to restore.

What the skill produces

For any PDF, the pipeline emits:

<out_dir>/
├── content.json             # extracted text (sections, abstract, stats, DOI, tables)
├── figures/figureN.png      # raw figures + figures.json (captions, page, bbox)
├── figureN.png              # transparent versions
├── cover.png                # optional AI-generated title cover (OpenAI gpt-image-1)
├── storyboard.json          # 7-9 slot narrative (Claude writes this)
├── index.html               # rendered chassis
├── style.css                # palette + mode + typography injected
└── script.js                # IntersectionObserver, themes map

How to run

PY=.venv/bin/python3
SKILL=./skill
PDF=/path/to/paper.pdf
OUT=/path/to/out

$PY $SKILL/scripts/extract_text.py    $PDF $OUT/content.json
$PY $SKILL/scripts/extract_figures.py $PDF $OUT/figures/
for f in $OUT/figures/figure*.png; do
  $PY $SKILL/scripts/make_transparent.py "$f" "$OUT/$(basename "$f")"
done
$PY $SKILL/scripts/generate_cover.py --concept "..." --palette cool --mode dark --out $OUT/cover.png
# Claude builds $OUT/storyboard.json from content.json + figures.json
$PY $SKILL/scripts/render.py --storyboard $OUT/storyboard.json --palette cool --mode dark --typography academic --out $OUT
$PY $SKILL/scripts/preview.py $OUT 8765

Style knobs

  • palettes: warm | cool | earth | clinical | tech
  • modes: dark | light (each palette has both)
  • typography: editorial (Playfair + Inter), modern (Space Grotesk + Inter), tech (JetBrains Mono + Inter), academic (Crimson Pro + Source Sans)
  • layouts: title, split, split_reverse, split_no_image, stacked, quote, impact, impact_single, stats_grid, chart, comparison, insight, credits. (fullbleed exists but is reserved for atmospheric photos — do not use it for paper figures.)

Conventions

  • Chassis is fixed. Per-paper variation = palette + mode + typography + per-section content + AI cover. Do not regenerate style.css/script.js freeform; always go through render.py.
  • Narrative arc has 9 fixed slots: title → hook → problem → method → keyFinding → dataNarrative → secondaryFinding → insight → credits. Slot 6 is skippable when a paper has only one main result.
  • Figures are transparent PNGs so they blend into the dark/light section backgrounds. make_transparent.py uses corner flood-fill by default; pass --mode rembg for photos/schematics.
  • Body copy is rewritten to fit display type (1.2rem). Headline numbers and key definitions are lifted verbatim from the paper.
  • No fullbleed for paper figures — scientific charts don't read well at viewport scale; use split, split_reverse, or stacked instead.

Dependencies

pip install -r requirements.txt
# pdfplumber, pymupdf, pillow (required)
# openai (optional, for cover generation)
# rembg (optional, for complex-figure bg removal)

Adding an example to examples/

When the skill produces an output dir you want to publish, here's the cleanup recipe (we used it for SCS_storyboard/):

  1. Trim raw extraction artifacts. Delete content.json and figures/ from the output dir — they're for the pipeline, not the reader. Cuts size ~50%. 
    rm -rf <out>/content.json <out>/figures
  2. Move into examples/ as examples/<paper-shortname>_storyboard/. The .gitignore has an !examples/*_storyboard/ exception so the dir won't be silently excluded by the demo-output rule.
  3. Write a per-example README.md with the paper citation, style combo (palette / mode / typography), layouts used, and a "View it" command (python3 skill/scripts/preview.py examples/<name> 8765).
  4. Capture a screenshot — open via preview.py (not directly, so the cover image and fonts load), screenshot the title slot at ~1600×1000, save as examples/<name>/screenshot.png, downscale: sips -Z 1800 examples/<name>/screenshot.png --out examples/<name>/screenshot.png (keeps under ~2 MB).
  5. Add the screenshot to the repo's top-level README.md as the hero image (only for the first/canonical example; later examples don't need to appear at the top).
  6. git add examples/<name>/ && git commit && git push. The .gitignore exception will already pick up the new subdir.

Methods evaluated (and why we kept what we kept)

A running log of alternatives we tested and rejected, so future contributors don't burn cycles re-deriving the same conclusions.

Figure background removal — kept flood-fill

skill/scripts/make_transparent.py uses corner flood-fill as the only path. We tested three alternatives on a representative paper figure (a 3-branch deep-learning architecture diagram with text labels, color blocks, dense connections, and embedded house/aerial photos):

Method Verdict Why rejected
flood-fill (current default) ✅ Kept Sub-second, zero extra deps, preserves content exactly. Minor white halos near anti-aliased lines, acceptable.
rembg + u2net (the rembg default) ❌ Worse 96% partially-transparent output — entire diagram ghostly. U2Net is trained for salient-object segmentation (people, products); diagrams aren't objects to it.
rembg + birefnet-general ❌ Worth keeping in mind, but not default Actually preserves content well with proper anti-aliased edges. 973 MB model download + ~5–10 s per figure. The user evaluated and chose flood-fill anyway.
OpenAI gpt-image-1 image-edit with background="transparent" ❌ Unusable for scientific figures ~$0.04 per image. Background does come out transparent, but every text label gets garbled ("Conv + Pool" → "Grovan PPA", "Dense Block" → "Bilesk dolk", "Output" → "Outopf"). Topology stays roughly right but specifics drift. Could be useful as an atmospheric / decorative backdrop, not for primary figure delivery.

Recreating figures as interactive charts — kept a vanilla bar chart from ground-truth data only

We explored turning paper bar/line plots into interactive charts (the original ask: "can Claude/OpenAI recreate the plots interactively?"). Conclusion: rendering is the easy part; data provenance is the whole problem. A chart is only as trustworthy as the numbers behind it, and this is a real paper — eyeballing values off a figure with a vision model risks publishing fabricated data under a real citation.

What we shipped (chart layout, see section_snippets.html + render.py _chart_html()):

Decision Choice Why
Rendering Vanilla HTML/CSS bar chart, no CDN Keeps the "vanilla JS, no framework" chassis rule. Bars grow on scroll by reusing the existing IntersectionObserver .active hook — no JS added. Hover/focus reveals exact values.
Chart types bar only Bars from a handful of discrete values are reliably recoverable. Line/scatter/dense series are not — excluded in the schema on purpose.
Data source Ground-truth required; provenance is mandatory chart.data_source ∈ table | text | estimated. estimated (eyeballed) renders an "approximate, not exact" caption. We did not build vision-based extraction as a default path.
Auto table extraction Built (extract_text.py → tables[]) pdfplumber.find_tables() pulls ruled tables, tags numeric_columns, and sets a chart_ready flag, so a chart slot can be sourced from a real table (data_source: "table") instead of hand-authored. Best-effort: catches ruled tables well, may miss borderless ones, and can merge multi-line headers (the values survive — the storyboard step reads rows directly).

First applied to examples/SCS_storyboard/ (the keyFinding ablation, data_source: "text") — replaced its stats_grid. See examples/SCS_storyboard/chart.png.

Cover image generation — kept OpenAI gpt-image-1 (generate, not edit)

skill/scripts/generate_cover.py calls client.images.generate() (not .edit()) with a Claude-composed --concept. Outputs an abstract data-art cover. This works well precisely because there's nothing to be faithful to — we want a stylized hero image, not an accurate reproduction. The same --background="transparent" flag from the figure test isn't useful here (we want a full bleed image).

Layouts — fullbleed exists but is reserved

The fullbleed snippet/CSS exists in the skill but is documented as "atmospheric photos only — do not use for paper figures." Reason: scientific charts at viewport scale don't read well, and the glass-panel overlay competes with the data. See also the ## Conventions and ## Style knobs notes above.