Pergel: A Unified Benchmark for Evaluating Turkish LLMs
Convert PaddleOCR models to ONNX format
Evaluate code generation with diverse feedback types
Measure execution times of BERT models using WebGPU and WASM
Benchmark models using PyTorch and OpenVINO
View RL Benchmark Reports
Evaluate LLM over-refusal rates with OR-Bench
Merge Lora adapters with a base model
Evaluate open LLMs in the languages of LATAM and Spain.
Convert PyTorch models to waifu2x-ios format
Measure BERT model performance using WASM and WebGPU
Upload a machine learning model to Hugging Face Hub
Find recent high-liked Hugging Face models
Cetvel is a benchmarking tool designed to evaluate the performance of Turkish Large Language Models (LLMs). It provides a comprehensive framework for assessing model capabilities across various natural language processing tasks. Cetvel automates the evaluation process, enabling users to compare and analyze the performance of different models efficiently.
• Task Coverage: Evaluate models on a wide range of Turkish NLP tasks, including text classification, summarization, and question answering.
• Customizable Benchmarks: Tailor evaluation metrics and tasks to specific use cases.
• Detailed Performance Reports: Generate in-depth analysis of model strengths and weaknesses.
• Cross-Model Comparison: Compare multiple models side-by-side to identify the best performer for your needs.
• Easy Integration: Seamlessly integrate with popular Turkish LLMs for quick and accurate benchmarking.
What models does Cetvel support?
Cetvel supports a wide range of Turkish LLMs, including popular models like BERTurk, TTUM, and others. For the full list of supported models, refer to the official documentation.
How do I customize the benchmarking tasks?
Customization options are available through the Cetvel interface, where you can select specific tasks, datasets, and evaluation metrics to tailor the benchmarking process to your needs.
Where can I find Cetvel?
Cetvel is available for download on its official GitHub repository. Ensure you follow the installation instructions carefully to set up the tool correctly.