AI benchmarking tool, MLPerf Client 1.0, unveiled; introduces graphical user interface, embraces a wider range of models and tasks, and boosts hardware acceleration capabilities
MLPerf Client 1.0, a new benchmark for client AI systems, has been released by the MLCommons consortium. This tool, developed in collaboration with major hardware and software vendors, aims to navigate the rapidly changing environment of client AI [1].
The benchmark focuses on local execution of AI models using GPUs and NPUs. It improves over earlier versions by supporting more AI models, such as Meta's Llama 2 7B Chat, Llama 3.1 8B Instruct, Microsoft's Phi 3.5 Mini Instruct, and the experimental Phi 4 Reasoning 14B [1]. It also tests a broader range of possible user interactions with large language models.
Regarding supported hardware execution paths, MLPerf Client 1.0 extends hardware acceleration support to devices from multiple major vendors including AMD, Intel, NVIDIA, Qualcomm, and Apple [1]. This includes compatibility with GPUs and dedicated AI accelerators (NPUs) from these vendors, optimized for inference workloads through vendor-specific APIs and frameworks.
- AMD supports their Instinct™ MI series GPUs with ROCm-based acceleration frameworks, enabling advanced MLPerf inference benchmarking, including optimizations for large language models and image generation [3].
- NVIDIA hardware uses Tensor Core GPUs, such as the HGX B200 platform, delivering strong AI training and inference throughput [2].
- Intel, Qualcomm, and Apple provide hardware acceleration paths leveraging their own NPU and GPU architectures optimized for local AI inference, supported through vendor SDKs integrated into the MLPerf benchmark suite [1].
Qualcomm Technologies NPU and CPU hybrid support is available via Qualcomm Genie and the QAIRT SDK. NVIDIA GPU support via Llama.cpp-CUDA is now available in MLPerf Client 1.0. AMD NPU and GPU hybrid support is available via ONNX Runtime GenAI and the Ryzen AI SDK [1].
MLPerf Client 1.0 also includes Intel NPU and GPU support via Microsoft Windows ML and the OpenVINO execution provider, though this is currently experimental [1]. Some experimental workloads in MLPerf Client 1.0 require a GPU with 16GB of VRAM to run, allowing for stress testing of higher-end hardware [1].
The benchmark includes a user-friendly graphical interface to make the test more accessible to casual users. This interface allows users to understand the full range of benchmarks they can run on their hardware and choose among them [1]. The GUI version of MLPerf Client 1.0 provides real-time monitoring of hardware resources.
For up-to-date news, analysis, and reviews related to MLPerf Client 1.0, users can follow Tom's Hardware on Google News. MLPerf Client 1.0 is available as a free download from GitHub [1].
[1] MLCommons. (2022). MLPerf Client 1.0. Retrieved from https://www.mlcommons.org/mlperf-client-10 [2] NVIDIA. (2021). NVIDIA Tensor Core GPUs. Retrieved from https://developer.nvidia.com/tensor-cores [3] AMD. (2021). ROCm. Retrieved from https://rocm.github.io/
Data-and-cloud-computing technology plays a significant role in the MLPerf Client 1.0, as it optimizes inference workloads through vendor-specific APIs and frameworks on diverse hardware, including GPUs and NPUs from AMD, Intel, NVIDIA, Qualcomm, and Apple. Users can take advantage of this by downloading MLPerf Client 1.0, which is available for free on GitHub.
