Track, rank and evaluate open Arabic LLMs and chatbots
Open LLM(CohereForAI/c4ai-command-r7b-12-2024) and RAG
A benchmark for open-source multi-dialect Arabic ASR models
Generate vector representations from text
Display and explore model leaderboards and chat history
Ask questions about air quality data with pre-built prompts or your own queries
Extract... key phrases from text
Parse and highlight entities in an email thread
Predict song genres from lyrics
Generate relation triplets from text
Explore and Learn ML basics
Compare different tokenizers in char-level and byte-level.
Identify AI-generated text
The Open Arabic LLM Leaderboard is a community-driven platform designed to track, rank, and evaluate open Arabic language models (LLMs) and chatbots. It provides a comprehensive framework for comparing the performance of various Arabic LLMs across different tasks and metrics, fostering transparency and innovation in the field of natural language processing (NLP) for Arabic.
• Evaluation Metrics: Comprehensive benchmarking of Arabic LLMs based on text generation, understanding, and conversation capabilities. • Model Submissions: Open submission process for developers to include their models in the leaderboard. • Performance Comparison: Side-by-side comparison of models based on accuracy, fluency, relevance, and contextual understanding. • Filtering Options: Customizable filters to sort models by specific criteria such as model size, training data, or use case. • Community Engagement: Discussion forums and resources for developers to share insights and improve models. • Open-Source Access: Transparent access to benchmarking tools and evaluation datasets. • Regular Updates: Continuous updates with new models, datasets, and evaluation metrics.
1. What is the purpose of the Open Arabic LLM Leaderboard?
The leaderboard aims to standardize evaluation practices for Arabic LLMs, promote transparency, and foster collaboration among researchers and developers in the NLP community.
2. Can anyone submit their model to the leaderboard?
Yes, any developer or researcher can submit their Arabic LLM or chatbot to the leaderboard, provided it meets the platform's submission criteria and guidelines.
3. How often are the leaderboards updated?
The leaderboards are regularly updated to include new models, improved evaluation metrics, and feedback from the community. Updates are typically announced on the platform's official channels.
4. Are the evaluation metrics customizable?
Yes, the platform offers customizable filters and comparison tools to allow users to evaluate models based on specific criteria relevant to their use cases.
5. Is the leaderboard open-source?
Yes, the Open Arabic LLM Leaderboard is open-source, providing transparent access to its benchmarking tools, evaluation datasets, and submission processes.