Skip to content

PaperBanana 🍌

Resource Location

/labs/SysMedBio/Q/Resources/AI/PaperBanana

/Volumes/lab/SysMedBio/Resources/AI/PaperBanana

Q:\SysMedBio\Resources\AI\PaperBanana

PaperBanana is a reference-driven multi-agent framework for automated academic illustration generation.

Scientific Context

Academic illustration (such as conceptual methodology diagrams and data plots) plays a critical role in communicating scientific work. However, manually designing these figures is highly time-consuming, while general-purpose AI text-to-image generators (like Stable Diffusion or Midjourney) suffer from a lack of technical accuracy, scientific layout conventions, and semantic alignment with the underlying text.

PaperBanana addresses this "visualization gap" by formulating diagram generation as a reference-driven, multi-agent task. It uses generative retrieval of reference academic figures to guide the generation (in-context learning) and operates a structured pipeline of agents (Retriever, Planner, Stylist, Visualizer, Critic) that iteratively refines and evaluates illustrations based on faithfulness, conciseness, readability, and aesthetic appeal.

Biological Relevance

In systems biology, medicine, and bioinformatics, researchers frequently need to illustrate: 1. Biological Systems/Pathways: Mapping complex biological interactions (e.g., signal transduction pathways, metabolic cascades, genomic regulatory networks, and drug-target interactions). 2. Experimental Workflows: Designing visual protocols for clinical trials, sample sequencing pipelines, or cell culture assays. 3. Data Visualization (Plots): Constructing high-precision plots representing multi-omics data (e.g., survival curves, volcano plots, pathway enrichments, clustering visualizations).

PaperBanana enables biomedical scientists to turn text-based methodology descriptions into standardized, publication-ready diagrams and code-generated plots, accelerating research communication.

Main Publication

  • Title: PaperBanana: Automating Academic Illustration for AI Scientists
  • Authors: Dawei Zhu, Rui Meng, Yale Song, Xiyu Wei, Sujian Li, Tomas Pfister, and Jinsung Yoon
  • Journal/Archive: arXiv (Computation and Language cs.CL; Computer Vision and Pattern Recognition cs.CV)
  • Year: 2026
  • Identifier: arXiv:2601.23265

Available Files

The following key files and folders are available in the PaperBanana resource directory:

  • app.py: Entry point for the Gradio-based web application.
  • demo.py: Entry point for the Streamlit-based interactive web app.
  • main.py: Entry point for running PaperBanana through the command-line interface.
  • requirements.txt & Dockerfile: Python environment requirements and Docker container configurations.
  • agents/: Core multi-agent implementation modules:
    • retriever_agent.py: Identifies relevant references for downstream guidance.
    • planner_agent.py: Formulates the structural layout and planning of the figure.
    • stylist_agent.py: Enforces aesthetic guidelines and color palettes.
    • visualizer_agent.py: Handles drawing/rendering via vision-language and generation models.
    • critic_agent.py: Evaluates output accuracy and prompts iterative refinement.
    • polish_agent.py: Handles image-to-image visual refinement.
  • prompts/: Evaluation templates (diagram_eval_prompts.py and plot_eval_prompts.py).
  • configs/: Workspace configuration files (e.g., model_config.yaml for API keys and default models).
  • style_guides/: Code and markdown guidelines for standardizing aesthetics (e.g., neurips2025_diagram_style_guide.md and neurips2025_plot_style_guide.md).
  • utils/: Under-the-hood utility functions (e.g., eval_toolkits.py, generation_utils.py, image_utils.py).
  • visualize/: Analytical dashboards for viewing pipeline evolution and evaluation results.
  • data/PaperBananaBench/: Local evaluation benchmark dataset containing separate reference configurations, test cases, and target outputs.