import gradio as gr def build_page(): with gr.Column(elem_id="about-page-content-wrapper"): # --- Section 1: About OpenHands Index --- gr.HTML( """

About OpenHands Index

OpenHands Index is a comprehensive leaderboard that tracks the performance of AI coding agents across multiple software engineering benchmarks. It provides a unified view of agent capabilities in areas like code generation, bug fixing, repository-level tasks, and complex reasoning challenges. The index makes it easy to compare agents' performance in an apples-to-apples manner across diverse evaluation scenarios.

""" ) gr.Markdown("---", elem_classes="divider-line") # --- Section 2: Why OpenHands Index? --- gr.HTML( """

Why OpenHands Index?

Software engineering benchmarks are scattered across different platforms and evaluation frameworks, making it difficult to compare agent performance holistically. Agents may excel at one type of task but struggle with others. Understanding the true capabilities of coding agents requires comprehensive evaluation across multiple dimensions.


OpenHands Index fills this gap by providing a unified leaderboard aggregating results from diverse software engineering benchmarks. It helps developers and researchers identify which agents best suit their needs, while providing standardized metrics for comparing agent performance across tasks like repository-level editing, multimodal understanding, and commit message generation.

""" ) gr.Markdown("---", elem_classes="divider-line") # --- Section 3: What Does OpenHands Index Include? --- gr.HTML( """

What Does OpenHands Index Include?

OpenHands Index aggregates results from 6 key benchmarks for evaluating AI coding agents:

Plus: comprehensive leaderboards showing performance across models, agents, and configurations.

🔍 Learn more at github.com/OpenHands/OpenHands

""" ) gr.Markdown("---", elem_classes="divider-line") # --- Section 4: Understanding the Leaderboards --- gr.HTML( """

Understanding the Leaderboards

The OpenHands Index Overall Leaderboard provides a high-level view of agent performance and efficiency:

Individual benchmark pages provide:

""" ) gr.Markdown("---", elem_classes="divider-line") # --- Section 5: Scoring & Aggregation --- gr.HTML( """

Scoring & Aggregation

OpenHands Index provides transparent, standardized evaluation metrics:

Scores

Cost

Note: Cost values reflect API pricing at evaluation time and may vary based on provider, infrastructure, and usage patterns.

""" ) gr.Markdown("---", elem_classes="divider-line") # --- Section 6: Submitting Results & Accessing Raw Data --- gr.HTML( """

Submitting Results & Accessing Raw Data

How to Submit Your Agent Results

To submit your agent's evaluation results to the OpenHands Index:

  1. Run your agent on the supported benchmarks (SWE-bench, Multi-SWE-bench, SWE-bench Multimodal, SWT-bench, Commit0, GAIA)
  2. Format your results according to the data structure documented in the repository
  3. Submit a pull request to github.com/OpenHands/openhands-index-results
  4. Your submission should include:

Accessing Raw Results

All raw evaluation results displayed on this leaderboard are publicly available at:

📊 github.com/OpenHands/openhands-index-results

The repository contains:

You can clone the repository, analyze the data, or use it for your own research and comparisons.

""" ) gr.Markdown("---", elem_classes="divider-line") # --- Section 7: Acknowledgements --- gr.HTML( """

Acknowledgements

The OpenHands Index leaderboard interface and visualization components are adapted from the AstaBench Leaderboard developed by the Allen Institute for AI. We thank the AstaBench team for their excellent work in creating a clear and effective leaderboard design that we have customized for the software engineering domain.

Key aspects adapted from AstaBench include:

We have extended and modified this foundation to support software engineering benchmarks and the specific requirements of evaluating AI coding agents.

""" ) gr.Markdown("---", elem_classes="divider-line") # --- Section 8: Citation --- gr.HTML( """

Citation

If you use OpenHands or reference the OpenHands Index in your work, please cite:

@misc{openhands2024,
    title={OpenHands: An Open Platform for AI Software Developers as Generalist Agents},
    author={OpenHands Team},
    year={2024},
    howpublished={https://github.com/OpenHands/OpenHands}
}
""" )