SomeAI.org
  • Hot AI Tools
  • New AI Tools
  • AI Category
SomeAI.org
SomeAI.org

Discover 10,000+ free AI tools instantly. No login required.

About

  • Blog

© 2025 • SomeAI.org All rights reserved.

  • Privacy Policy
  • Terms of Service
Home
Model Benchmarking
OR-Bench Leaderboard

OR-Bench Leaderboard

Evaluate LLM over-refusal rates with OR-Bench

You May Also Like

View All
🦾

GAIA Leaderboard

Submit models for evaluation and view leaderboard

360
🚀

Can You Run It? LLM version

Determine GPU requirements for large language models

950
🥇

Arabic MMMLU Leaderborad

Generate and view leaderboard for LLM evaluations

15
🌎

Push Model From Web

Push a ML model to Hugging Face Hub

9
🥇

OpenLLM Turkish leaderboard v0.2

Browse and submit model evaluations in LLM benchmarks

51
📊

MEDIC Benchmark

View and compare language model evaluations

8
🥇

Pinocchio Ita Leaderboard

Display leaderboard of language model evaluations

11
🏅

Open Persian LLM Leaderboard

Open Persian LLM Leaderboard

61
📈

GGUF Model VRAM Calculator

Calculate VRAM requirements for LLM models

37
🚀

stm32 model zoo app

Explore and manage STM32 ML models with the STM32AI Model Zoo dashboard

2
🥇

GIFT Eval

GIFT-Eval: A Benchmark for General Time Series Forecasting

64
🚀

Model Memory Utility

Calculate memory needed to train AI models

922

OR-Bench Leaderboard is a benchmarking tool designed to evaluate the performance of large language models (LLMs) with a specific focus on their over-refusal rates. It provides a comprehensive platform to assess how often LLMs refuse to provide answers, even when they should be capable of doing so. This metric is crucial for understanding model reliability and effectiveness in real-world applications.

Features

• Over-refusal rate tracking: Measures how frequently LLMs decline to answer questions they should know. • Comparison across models: Allows users to compare multiple models based on refusal rates. • Real-time leaderboards: Provides up-to-date rankings of LLMs in a competitive format. • Interactive data exploration: Enables users to filter results by specific criteria like model size or dataset. • Transparency and reproducibility: Offers detailed methodologies and datasets for independent verification.

How to use OR-Bench Leaderboard ?

  1. Visit the OR-Bench Leaderboard platform and explore the available models and their performances.
  2. Use the filtering options to narrow down results by model, dataset, or other criteria.
  3. Analyze the refusal rates and compare them against other models.
  4. Access additional resources like datasets, evaluation metrics, and detailed reports.
  5. For submitting your own model, follow the submission guidelines provided on the platform.

Frequently Asked Questions

1. Why is OR-Bench Leaderboard important for evaluating LLMs?
OR-Bench Leaderboard is important because it helps identify models that are overly cautious, ensuring they provide meaningful answers rather than refusing when they have the capability to respond.

2. Can anyone submit their model to OR-Bench Leaderboard?
Yes, researchers and developers can submit their models for evaluation by following the submission guidelines provided on the platform.

3. How is the over-refusal rate calculated?
The over-refusal rate is calculated by evaluating how often a model refuses to answer questions it should reasonably be expected to answer, based on its training data and capabilities.

4. Does OR-Bench Leaderboard provide insights into model reliability?
Yes, the leaderboard offers insights into model reliability by highlighting how often models refuse to answer questions, helping users assess their practical effectiveness.

5. Are the datasets used for evaluation publicly accessible?
Yes, the datasets and evaluation methodologies used by OR-Bench Leaderboard are transparent and publicly accessible to ensure reproducibility and fairness.

Recommended Category

View All
🧑‍💻

Create a 3D avatar

📈

Predict stock market trends

🚫

Detect harmful or offensive content in images

🎬

Video Generation

🔍

Object Detection

🎥

Convert a portrait into a talking video

🗣️

Voice Cloning

📋

Text Summarization

🎵

Generate music for a video

💻

Code Generation

🎤

Generate song lyrics

🔇

Remove background noise from an audio

✨

Restore an old photo

🤖

Chatbots

🎎

Create an anime version of me