B

benchmarking

Benchmarks data processing tools

An open-source observatory for reproducible Linux kernel performance research and systems investigation.

Python toolkit for benchmarking RAG quality on Salesforce Data Cloud. Includes retrieval metrics (Hit Rate, MRR, NDCG), generation evaluation, and optional Claude Code skills.

Automated LLM Benchmarking on GPU - tokens/sec, latency percentiles, VRAM profiling, multi-format support (HuggingFace, GGUF, GPTQ)

https://ayinedjimi-consultants.fr

MultiNativQA is Multilingual Native question-answering (QA) dataset consisting of 64k QA pairs in seven extremely low to high resource languages, covering 18 different topics from nine different regions. Paper: https://arxiv.org/pdf/2407.09823. Project: https://nativqa.gitlab.io

QAPerf is a tool for evaluating the performance of quality assessment (QA) methods. It automates the generation of standard reports, including correlation tables, leave-one-group-out correlation boxplots, and other benchmarking visualizations. Designed for researchers and practitioners, QAPerf streamlines the analysis of QA metrics, ensuring reliable and reproducible evaluations.

A Live Evaluation of Computational Methods for Metagenome Investigation

SecurityPerf is a tool designed for benchmarking production workloads. In doing so, it makes measuring the impact of security programs on production workloads easy.

VTmark results, Ansible roles and other scripts.

The project uses natural language processing to automatically assess causal diagrams filled out by students based on semantic distance from the model causal diagram provided.

Modular toolbox for performance recording, benchmarking and visualization. https://kpouget_spice.gitlab.io/streaming_stats/