We conducted SPDK bdevperf tests on an AWS EC2 r8gd.metal-24xl instance, focusing on single CPU core performance under high I/O load. Our objective was to demonstrate a CPU-bound workload. Results show low I/O wait and high CPU utilization, confirming the CPU is the limiting factor. The 2-disk configuration achieved the highest throughput, indicating a CPU saturation point. 1. Performance Results Summary (100-second duration) Below is a consolidated view of our 100-second bdevperf runs across 1, 2, and 3 local NVMe disks. These figures include throughput, latency, CPU utilization (from mpstat), and Instructions Per Cycle (IPC, from perf stat) for the dedicated CPU core. 2. Test Setup and Environment Tests were conducted on an AWS EC2 r8gd.metal-24xl instance, a bare-metal machine. Processor: AWS Graviton4 (ARM-based). Local Storage: Three 1900 GiB NVMe SSDs (Instance Store). bdevperf parameters used to focus on CPU utilization include: SPDK Driver: AIO (Asynchronous I/O), which uses the Linux kernel's native AIO interfaces. Queue Depth (QD): 384, set high to keep storage busy. I/O Size (IO_SIZE): 4096 bytes (4 KiB), a block size for transactional workloads. Workload Type: randrw (Mixed Random Read/Write) with 70% Reads / 30% Writes. Test Duration: 100 seconds per run. CPU Core Dedication: bdevperf was affinity-set to a single CPU core (core 0) to measure that core's I/O processing capacity. 3. Our Script's Test Methodology Our custom automation script executes and monitors bdevperf tests as follows: 1. Device Identification & Selection: The script identifies available, unmounted NVMe block devices, excluding system partitions. We then select devices for testing. 2. SPDK AIO bdev Configuration: For each test (incrementally adding selected disks), a JSON configuration file is generated. This configures SPDK to use AIO block devices from the physical NVMe drives. 3. Performance Execution with Monitoring: bdevperf runs with the generated configuration. mpstat Monitoring: mpstat concurrently monitors the dedicated CPU core(s), capturing CPU utilization percentages (User, System, Idle, I/O Wait). perf stat Monitoring: perf stat wraps bdevperf, targeting the dedicated CPU core(s). It collects hardware performance counter data (instructions, cycles) and directly extracts Instructions Per Cycle (IPC), a measure of CPU efficiency. All raw outputs are saved to a unique, timestamped directory. 4. Results Aggregation & Summary: After each test, the script parses bdevperf (IOPS, throughput, latency) and CPU metrics. A summary table is presented, highlighting the configuration with the highest total CPU utilization. 4. Key Findings: CPU-Bound Workload Confirmation Our tests confirm that the workload is CPU-bound on the single dedicated Graviton4 core, not bottlenecked by NVMe storage. Low I/O Wait: Multi-disk configurations show I/O Wait at 0.02% to 0.06%, indicating NVMe storage provides data faster than the CPU can process it. The single disk I/O wait is 5.49%. High CPU Utilization: Total CPU Utilization on the dedicated core remained high (84.87% for 1-disk, nearly 99% for 2-disk and 3-disk), confirming the single core as the performance bottleneck. Dominant System CPU: High System CPU (76-87%) is expected with SPDK AIO bdevs under heavy load, reflecting kernel overhead in processing numerous asynchronous I/O requests. IPC Values: IPC values (2.21 to 2.53) indicate the Graviton4 core's efficiency. The slight IPC increase with more disks suggests improved pipeline utilization as CPU saturation increases. 5. Performance Dynamics: 2 Disks vs. 3 Disks - Optimal Point Identification The comparison between 2-disk and 3-disk scenarios shows the CPU's saturation point: 2 Disks: Optimal Throughput: This configuration achieved the highest throughput (658.59 kIOPS / 20.09 Gbps), with nearly 99% Total CPU Utilization and minimal I/O Wait. Two NVMe devices provide I/O that a single Graviton4 core can optimally handle, maximizing throughput without excessive contention. 3 Disks: Beyond Optimal Saturation: Adding a third disk maintained high CPU utilization (98.93%) and low I/O Wait (0.02%). However, total throughput slightly decreased (638.92 kIOPS / 19.50 Gbps), and average latency significantly increased to 1802.86 μs (from 1166.00 μs). This indicates that beyond CPU saturation, additional I/O sources increase contention and queueing, leading to higher latencies without throughput gain. 6. Limitations and Future Work: The Role of SPDK Drivers (VFIO/UIO) Our current methodology utilizes the SPDK AIO bdev driver, which passes all I/O through the Linux kernel's I/O stack. This incurs kernel overhead, contributing to our observed high System CPU utilization. SPDK offers VFIO (Virtual Function I/O) and UIO (Userspace I/O) drivers for direct, zero-copy access to NVMe devices from user space, bypassing kernel overhead. These drivers typically offer higher IOPS and lower latency. We were unable to utilize VFIO or UIO drivers in this test series due to setup constraints. Using these drivers could yield higher performance (more User CPU, less System CPU), further pushing the single Graviton4 core's capabilities. Future Work: Investigating SPDK performance with VFIO or UIO drivers to fully assess the r8gd.metal-24xl instance's potential by minimizing kernel involvement. Conclusion Our experiments confirm the CPU-bound nature of the SPDK AIO bdevperf workload on a single Graviton4 core. The r8gd.metal-24xl instance's local NVMe storage is suffcient to saturate a single CPU core with high-volume, small-block random I/O. The 2-disk configuration represents the optimal point for throughput before latency increases. Future tests with user-space drivers like VFIO or UIO could demonstrate even higher performance.

Complex computing environments, spanning cloud, HPC, AI, and edge workloads; observability is no longer optional. With multiple layers of hardware and software working together, traditional monitoring alone cannot surface the insights needed for optimizing performance or preventing downtime. At Whileone Techsoft Pvt. Ltd., we help companies go beyond monitoring by building deep observability frameworks that connect performance benchmarking, system analytics, telemetry, and profiling. This integrated approach helps engineering teams gain complete visibility into their systems, enabling faster debugging, reduced operational costs, and enhanced end-user experiences. Why Observability Matters As infrastructures scale and workloads diversify, blind spots emerge. This can lead to: Observability addresses these challenges by providing end-to-end visibility into the state and behavior of your systems. This means you can detect issues earlier, understand their root causes, and fix them before they impact users. Observability vs. Traditional Monitoring Traditional monitoring answers the “what”, for example, CPU utilisation or error counts. Observability goes deeper and answers the “why” behind performance issues. It focuses on three core pillars: Metrics – Quantifiable measurements (e.g., latency, throughput) Logs – Detailed event records for context Traces – Understanding requests as they travel across distributed systems At Whileone Techsoft, we layer these pillars with analytics, telemetry, and profiling to deliver actionable insights. Performance Benchmarking: The Foundation Our Performance Benchmarking Services form the cornerstone of observability. We help companies: This data-driven approach uncovers bottlenecks early before they become costly in production. System Analytics for Deeper Understanding Benchmarking generates performance data, but analytics transforms that data into insights. System analytics helps teams understand: How workloads utilize CPU, memory, I/O, and network resources Correlation between resource consumption and performance outcomes Trends and anomalies in system behavior over time Our analytics frameworks leverage advanced models to identify optimization opportunities, ensuring your workloads perform consistently and reliably. Telemetry for Real-Time Visibility Telemetry extends observability by collecting live data from hardware, firmware, middleware, and applications. It captures fine-grained performance metrics continuously Enables proactive alerts for deviations from benchmarks Allows visualization of live system health through unified dashboards Whileone’s use of open standards like OpenTelemetry makes this telemetry layer scalable and interoperable with your existing tools. Profiling for Root Cause Analysis Even the best benchmarking and telemetry setups cannot replace profiling when you need detailed root cause analysis. System-level profiling: Identifies hotspots in the kernel, drivers, or hardware interfaces Code-level profiling: Finds inefficient functions, loops, or algorithms in the application stack By correlating profiling data with benchmark and telemetry insights, we help engineering teams quickly diagnose and resolve performance regressions. An Integrated Observability Framework At Whileone Techsoft, we integrate benchmarking, analytics, telemetry, and profiling into a single observability framework: Unified dashboards to correlate data across layers Automated workflows for continuous testing and monitoring Cross-silo visibility that spans hardware, system software, and applications This holistic approach ensures reliable, high-performance outcomes for pre-silicon validation, cloud workload optimization, and edge deployments. Benefits of Observability-Driven Benchmarking Real-World Example One of our semiconductor customers was struggling with inconsistent performance in their post-silicon validation phase. By deploying Whileone’s observability-driven benchmarking framework, they were able to: Pinpoint compiler-level inefficiencies using code profiling: https://www.whileone.in/post/tuning-compiler-flags-for-custom-hardware Correlate memory bandwidth metrics from telemetry data with workload performance: https://www.whileone.in/post/investigating-performance-discrepancy-in-hpl-test-on-arm64-machines Best Practices for Building Observability Observability is no longer a “nice-to-have”, it’s essential for ensuring reliable, high-performance systems. Whileone Techsoft Pvt. Ltd. brings together performance benchmarking, system analytics, telemetry, and profiling to build observability frameworks tailored for semiconductor companies, cloud providers, and software enterprises. Ready to take your performance engineering efforts to the next level? Reach out to us to learn how our observability-driven services can help you reduce costs, accelerate time-to-market, and achieve industry-leading performance.

Overview MySQL is an open-source relational database management system (RDBMS) that stores and organizes data using tables, rows, and columns, and allows you to query and manage that data using SQL (Structured Query Language). MySQL Database Server is fast, reliable, scalable, and easy to use. It continues to rank highly in popularity among databases, according to DB-engines. SysBench is a multi-threaded benchmark tool. The tool can create a simple database schema, populate database tables with data, and generate multi-thread load (SQL queries) towards the database server.​ Sysbench works very well with MySQL because it was originally designed specifically to benchmark MySQL and MariaDB under various OLTP workloads. Sysbench comes with Lua scripts to simulate: Read/write transactions (oltp_read_write.lua) Read-only workloads (oltp_read_only.lua) Write-only workloads (oltp_write_only.lua) Point-select workloads (oltp_point_select.lua) Setup Details: MySQL Server Version: 8.0.36​ Sysbench Version: 1.0.20​ Example: MySQL Benchmark (OLTP) sysbench oltp_point_select\ --db-driver=mysql \ --mysql-user=root \ --mysql-password=yourpass \ --mysql-db=test \ prepare # load data sysbench oltp_point_select \ --db-driver=mysql \ --mysql-user=root \ --mysql-password=yourpass \ --mysql-db=test \ --tables=10 \ --table-size=100000 \ --threads=8 \ --time=60 run # run benchmark sysbench oltp_point_select \ --db-driver=mysql \ --mysql-user=root \ --mysql-password=yourpass \ --mysql-db=test \ cleanup # remove data MySQL - Competitive Analysis: Compare MySQL on AMD, Intel and ARM machines. On each platform using the same Linux distribution (Fedora 38) and the same kernel version (6.4.13-200.fc38), with 4K Page Size. Test Purpose: oltp_point_select.lua: This workload performs single-row SELECTs by primary key. It’s used to measure pure read throughput, memory/cache efficiency, and indexing performance. Sysbench Command: sysbench/oltp_point_select.lua --table-size=10000000 --tables=8 --mysql-port=3000 --mysql-db=sbtest --threads=64 --events=0 --time=600 --report-interval=10 --thread-init-timeout=5 --rate=0 --rand-type=uniform --rand-seed=1 --mysql-host=`ip_address` --mysql-user=sbtest --mysql-password=`yourpass` --mysql-ssl=REQUIRED --mysql-ssl-cipher=AES128-SHA256 --db-ps-mode=disable --mysql-ignore-errors=1213,1205,1020,2013 --db-driver=mysql run Observations: High QPS with very low average latency — typical of optimized read workloads. Performance scales well with threads up to a point. Depends heavily on index lookups and buffer pool hits. Minor spikes in latency may occur with disk I/O or buffer pool misses. Performance Insights Arm outperforms others in high-concurrency scenarios, ideal for thread-heavy OLTP workloads. Icelake performs best in low-to-mid thread counts, likely due to strong single-threaded or cache performance. Genoa is consistent, Good Compatibility with all MySQL features. Best Performance. Milan may need tuning or reflects a less optimized test environment. MySQL tunings for better performance: Tuning MySQL for better performance involves adjusting configuration settings based on your workload type (OLTP, analytics, mixed), system resources (RAM, CPU, disk), and traffic pattern (read-heavy, write-heavy, etc.). Also tune some parameters in my.cnf such as “innodb write io threads”, “innodb read io threads”, “max connections” etc. Also used PGO (Profile-Guided Optimization) is a compiler optimization technique used with GCC to improve binary performance by collecting execution profiles and optimizing accordingly. MySQL - Competitive Analysis over GCP cloud: Compare MySQL on AMD, Intel and ARM machines on GCP cloud. Performance Analysis: Performance Insights GCP-N2 is the best choice for maximum performance in read-intensive benchmarks like Sysbench OLTP Point Select. GCP-N2D and T2A offer a good balance of performance and likely cost. GCP-T2D may not scale well for high QPS workloads — best suited for light-duty use. Conclusion MySQL handled concurrent connections efficiently with steady TPS and low latency. The oltp_point_select.lua test demonstrated MySQL's ability to handle high-throughput primary key lookups with exceptional performance — achieving 18,000 QPS at ~0.5 ms average latency. The results indicate efficient use of the InnoDB buffer pool and minimal I/O waits. To sustain performance under higher concurrency, further tuning of buffer size, read threads, and CPU parallelism may be beneficial. This workload is a good indicator of how MySQL will perform in read-heavy applications like caching layers or real-time dashboards.

Benchmarking SPECint on FPGA: Introduction With the growing interest in AI hardware for high-performance and power-efficient computing, understanding how industry-standard benchmarks perform on such platforms is critical. In this paper, we focus on SPECrate®2017 Integer workloads, a widely-used CPU benchmark suite, and share a case study comparing various runs on an FPGA target: a base run and a tuned run that achieved better performance. This paper describes how the tuning and benchmarking procedure was executed, the challenges faced, and what we learned from this hands-on analysis. Why SPECint on FPGA? SPECrate®2017 integer evaluates the integer processing capabilities of a CPU through a set of compute-intensive, single-threaded programs. Running these on an FPGA (with soft or hardened CPU cores) helps evaluate and tune how custom logic performs in realistic software scenarios, especially in workloads like compilers, compression, and AI preprocessing. Benchmarking Setup Platform: FPGA Emulation: Run on QEMU for pre-validation, native execution on FPGA target Benchmark Suite: SPEC CPU2017 Cross-compilation: All benchmarks built using a target toolchain with specmake, applying -static, and a base set of flags Base Run: No tuning; baseline compiler flags, minimal memory tuning Optimized Run: Enhanced compiler flags, better memory layout, cache tuning Here’s how the benchmarking was carried out: Cross-Compilation of SPECrate®2017 Integer Benchmarks Ensured static linking for portability Verified ELF binaries using file and readelf Execution with runspec Invoked with runspec --config=target.cfg --tune=base --size=test,train.ref for initial testing Data Collection Captured runtime, SPEC score, and individual benchmark outputs Track CPU MHz, instruction counts using perf or counters Tricks Use math models to reduce run times of Spec Workloads Get a sense for Test, Train and Ref workloads and find a relation so there is no need to runref everytime.

Operational Excellence for Service-Driven Enterprises As businesses increasingly deploy services and in production environments, the reliability and uptime of servers have become a critical need. These workloads are often hosted in hybrid setups, including dedicated data centers and public clouds, where even brief outages can impact performance, user trust, and business outcomes. To meet these demands, a dedicated Site Reliability Engineering (SRE) team provides comprehensive support, combining real-time incident management, infrastructure optimization, and operational discipline to maintain high availability, typically targeting 99.9% uptime. The Whileone Approach to SRE Excellence At Whileone, we specialize in keeping critical system up and running with minimal disruption. Our team blends hands-on expertise in Linux, server management, and cloud platforms to deliver consistent, high-availability support. From alert response to root cause analysis and resolution, we follow a disciplined SRE approach that ensures incidents are handled swiftly and systematically. We take pride in being the steady hand behind your infrastructure proactive, and reliable. Core Capabilities and Technical Expertise The SRE team operates with a diverse skill set tailored to high-performance, always-on environments: Operating Systems & Systems-Level Engineering: Deep understanding of Linux-based systems including process management, disk and memory diagnostics, kernel tuning, system services, networking, and security configurations. Physical and Virtual Server Management: Experience with both bare-metal server environments and virtualized compute platforms, ensuring reliability from hardware up to the OS and service layer. Cloud and Hybrid Infrastructure: Proficient in managing cloud-native workloads and integrating cloud services with on-premise infrastructure across platforms such as AWS, Azure, Google Cloud, and Oracle Cloud. Monitoring and Observability: Skilled in leveraging observability stacks to monitor key metrics, application health, and system-level behavior, enabling proactive detection and rapid triage of issues. Process Engineering and Benchmarking: The team implements standardized incident handling workflows and continuously refines processes to improve detection, diagnosis, and recovery times. Full Stack of Operational Support (L1–L4): The team provides structured, in-house coverage across all support levels, from basic alert triage (L1), to systems analysis (L2), code-level debugging (L3), and infrastructure-level resolution or architectural remediation (L4). Cross-Functional Collaboration: Workflows are integrated with enterprise-grade tools that support alerting, team coordination, ticketing, documentation, and shift-based communication. Shift-Based Support and Observational Handoffs The team operates in rotating shifts to ensure 24/7 coverage. Each shift is responsible for ongoing incident management, proactive health checks, and noting key system behaviors or deviations. At the end of each shift, outgoing engineers document their observations. The first shift of each day consolidates these notes into a comprehensive report, highlighting unresolved issues, recurring patterns, and system performance trends. This ensures that both technical and leadership teams remain informed and aligned. Structured Incident Response Lifecycle Alert Detection & Acknowledgement: Monitoring tools flag anomalies; engineers acknowledge and initiate an investigation immediately. System Diagnosis & Log Review: Teams inspect logs, resource metrics, and system health to identify stalls, failures, or contention. Collaborative Communication: A live incident thread is established to coordinate response and ensure full team visibility. Corrective Actions: Engineers take steps like restarting services, isolating nodes, or reallocating load to stabilize systems. Documentation & Run-log Update: The incident is formally logged with actions and findings for traceability and future reference. Escalation When Required: Complex issues are smoothly handed off to higher-tier specialists with full context and diagnostics. Operational Readiness and In-House Autonomy All support services from the initial alert handling to the most advanced system-level debugging are managed by a fully autonomous in-house team. This includes: Immediate L1 triage and alert response. Deep L2 and L3 systems troubleshooting. L4 infrastructure decision-making and optimization. With expertise spanning operating systems, cloud platforms, observability, automation, and performance engineering, the team is self-sufficient and minimizes external dependencies. This allows for faster resolution times and better control over long-term infrastructure health. This Site Reliability Engineering function provides robust operational support across hybrid and cloud-native environments. With a combination of hands-on technical depth, well-defined processes, and structured escalation paths, the team ensures stability, uptime, and resilience for complex production systems.

Automation is essential for performance benchmarking because it ensures that results are reliable, repeatable, scalable, and comparable. Various open source benchmarking tools are used for Automation. Tools are essential for benchmarking because they bring standardization, accuracy, efficiency, and repeatability to performance evaluation. Open-Source Benchmarking Tools that support ad-hoc extensibility, meaning they can be customized or extended without rebuilding or heavily modifying the core codebase. These tools provide flexibility in creating custom test scenarios, simulating various workloads, and adapting to new APIs or environments. List of tools which we used for benchmarking: Phoronix Test Suite PerfKit Benchmarker Phoronix Test Suite: Phoronix Test Suite is the most comprehensive open-source benchmarking platform available for Linux, macOS, and windows systems. It is widely used for automated testing, performance analysis, and software comparisons. What is PTS Extension? A PTS Extension is a plugin or add-on for the Phoronix Test Suite (PTS) that extends its functionality. It allows users to add custom behaviors before, during, or after benchmark runs—ideal for automation, integration, or custom logging. PTS extensions are used to: Add full socket runs Add open source docker tests Integrate with other systems. System & hardware benchmarking Why Shift from PTS to PerfKit Benchmarker? Phoronix Test Suite (PTS) is primarily a single-node benchmarking tool, which runs on a single machine. To overcome this issue Perfkit Benchmarker tool is used. PKB is specifically built for cloud platforms. PKB handles provisioning, benchmarking, monitoring, and cleanup automatically. PTS requires manual test setup, especially for cloud VMs. PKB can push benchmark data to: InfluxDB Stackdriver Grafana JSON logs for CI/CD systems PTS does offer HTML/JSON/CSV output but lacks native telemetry integrations. Perfkit Benchmarker(PKB): PerfKit Benchmarker is an open-source tool developed by Google that automates the process of benchmarking cloud infrastructure across different cloud providers. Main Stages of a PerfKit Benchmarker Run: What Is a PerfKit Benchmarker Extension? Extensions allow users to define: Custom benchmark Flags Providers Workloads Top Benefits of PerfKit Benchmarker Extension: PerfKit Benchmarker can run distributed benchmarks involving multiple VMs across one or more cloud zones or providers. Automatically handles VM provisioning, software installation, test execution, teardown. Easily integrates with dashboards, analytics pipelines, or cost/performance reports. Useful in capacity planning, performance regression testing, or SLI validation. In addition PKB Extension supports Turbostat(useful for analyzing power and frequency behavior during benchmarks), Lm-Sensors(Linux utility used to monitor hardware sensors), and Sysstat(analyze CPU, memory, disk I/O, networking, and other system-level performance metrics.). PKB extension also support additional feature for Report generation, which is useful to generate report with all result and peripheral data. It supports various formats such as TXT, CSV and HTML. Here’s a set of workload charts for PerfKit Benchmarker (PKB) organized by category. These charts summarize the common benchmark workloads available in PKB, helping you choose the right tests for CPU, memory, disk, network, and database performance analysis across cloud platforms. Cloud Comparison Using PerfKit Benchmarker Here's a comprehensive comparison of cloud providers (GCP, Azure, OCI) using PerfKit Benchmarker (PKB) as a common benchmarking framework: Conclusion PTS is excellent for deep technical benchmarking of a single system. PKB is a robust choice for cloud performance comparisons, cost evaluation, and infrastructure benchmarking at scale.

In this case study, we describe our systematic approach to benchmarking and validating workloads on FPGA platforms using HAPS (High-performance ASIC Prototyping System) models. The workflow involves compiling and cross-compiling a diverse set of workloads using both native QEMU and the open source toolchain, executing them on FPGA hardware, and capturing detailed performance metrics such as instructions executed and cycle counts. 1. Benchmark Preparation and Build Process We classify our benchmarks into the following categories: High-Performance Computing (HPC) Benchmarks: Includes matrix multiplication, FFT, and other numerical kernels. Synthetic Benchmarks: Includes whetstone, dhrystone, and other CPU stress tests. Algorithmic Benchmarks: Includes sorting algorithms, graph traversal, and numerical integration. Cryptography and Security Benchmarks: AES, RSA, SHA-based microbenchmarks (in future pipeline). Memory and I/O Benchmarks: Includes stream, memcpy stressors, and file read/write tests. Industry-standard Benchmarks: SPEC CPU2017 for INT and FP tracks. All benchmarks are first built or cross-compiled depending on their compatibility: Native Build: Performed on QEMU-based emulation environment where toolchain compatibility allows. Cross Compilation: Done using toolchain targeting the architecture for cases where native build fails or is time-prohibitive. Application Categories Distribution 2. Deployment and Execution on FPGA The compiled binaries are deployed to the FPGA via HAPS models configured with a soft-core. Execution is controlled using a lightweight shell interface or boot script. We utilize a custom performance monitoring utility (akc_counter_capture) to gather the following metrics: Total instruction count Cycle count These values are stored for each benchmark run and are used in performance comparisons. 3. Workload Example 1: DGEMM (Double-Precision General Matrix Multiply) DGEMM is a key linear algebra kernel from the BLAS library. We compiled and executed the DGEMM kernel using double-precision arithmetic with matrix size NxN, where N=256. Performance was evaluated using instruction count, cycle count, and IPC (Instructions Per Cycle). 2: N-Queens Problem The N-Queens benchmark is a classic example of combinatorial search used to evaluate control-flow-heavy algorithm performance. It computes all valid arrangements of N queens on an N×N chessboard such that no two queens attack each other. We verified correctness by comparing the total number of valid solutions for standard board sizes (e.g., N=12 and N=14), which matched precisely across architectures. The benchmark’s output was deterministic, and no deviations were observed across multiple FPGA runs. 3: Red-Black Tree (RBTree) Manipulation Red-Black Tree (RBTree) manipulation represents a memory-bound and pointer-intensive workload that tests dynamic memory access patterns and data structure balancing algorithms. This benchmark was compiled using both the Embedded toolchain and natively on QEMU for consistency. Validation involved verifying the in-order traversal of the tree after bulk insertions and deletions. RBTree serves as a robust test of both instruction scheduling and memory subsystem behavior. Conclusion Our approach demonstrates that workloads can be effectively compiled, executed, and validated on FPGA platforms using HAPS models.

Delivering Modernization, Benchmarking & Cost Efficiency Migrating workloads from AMD64 to ARM64 allows organizations to harness the price-performance and scalability benefits of leading-edge ARM cloud platforms (Ampere, AWS Graviton, Oracle, etc.). We offer comprehensive containerized porting packages, including leading microservices benchmarks such as DeathStarBench (DSB), to accelerate your cloud adoption and benchmarking initiatives on next-generation ARM infrastructure. DSB serves as a prime example of these optimized workloads, enabling robust performance evaluation and seamless migration to ARM-based environments. Key benefits for your organization: Unlock significant cloud cost savings with ARM’s high-efficiency compute Run production-ready, optimized containers and DSB benchmarks on ARM64 Build technical credibility and customer trust through future-ready, open-source best practices Our Comprehensive Porting Services Applications & Database Containers MongoDB (Sharded/Replica), Redis Cluster, Memcached, Mcrouter Exporters (Prometheus-compatible), orchestration utilities Full Dockerfile, image, and chart refactoring for ARM64 DeathStarBench (DSB) Microservices Suite End-to-end ARM64 porting of all major DSB microservices (social network, media, graph, etc.) Validation of dependencies and orchestration scripts for ARM compatibility Helm chart adaptation and seamless CI/CD integration Kubernetes & Benchmarking Enablement Refactor Helm charts and manifests for ARM64 Secure deployment via RBAC, RoleBindings, and cluster automation Guidance on operationalizing DSB for robust ARM-native benchmarking How We Deliver: DSB Porting Workflow What We Port & Support MongoDB Sharded & Exporter (DSB-ready) Redis Cluster & Exporter (DSB-ready) Memcached & Exporter (DSB-ready) Mcrouter (DSB-ready) DeathStarBench Microservices (all major suites) Supporting K8s utilities (yq, kubectl, gosu, etc.) All solutions are validated for ARM64 compatibility, performance, and reliability, both as standalone services and within the DeathStarBench framework. Conclusion This enables organizations to accelerate their modernization and benchmarking initiatives with confidence. Our deep expertise in ARM-native cloud, SaaS, and benchmarking ecosystems ensures that you benefit from: Full lifecycle delivery: from binary integration to Docker builds, Helm/K8s enablement, and DSB orchestration Transparent and auditable processes, aligned with your compliance and operational requirements Demonstrated success across cloud and SaaS environments, validated by references and production deployments

Benchmarks are vital for evaluating High-Performance Computing (HPC) system performance, guiding hardware choices, and optimizing software. This whitepaper focuses on understanding and overcoming bottlenecks in HPC benchmarks for CPU environments, specifically considering ARM/AARCH64 architectures, using miniFE and GROMACS as examples. 1. Introduction to miniFE and GROMACS Benchmarks 1.1. miniFE: A Finite Element Mini-Application miniFE, part of the Mantevo suite, simulates implicit finite element applications. It solves sparse linear systems, with its core kernels focused on element-operator computation, assembly, sparse matrix-vector products (SpMV), and basic vector operations. It's excellent for benchmarking systems handling sparse linear algebra and iterative solvers. To run miniFE, you typically compile it with an MPI-enabled compiler. Execution involves specifying problem dimensions and MPI processes. # Example for a single node with 16 MPI tasks srun -N 1 -n 16 miniFE.x -nx 264 -ny 256 -nz 256 # Example for a multi-node run (adjust N and n) srun -N 4 -n 64 miniFE.x -nx 528 -ny 512 -nz 512 Note: srun is for SLURM; mpirun or similar for other systems. 1.2. GROMACS: Molecular Dynamics Simulation Software GROMACS (GROningen MAchine for Chemical Simulations) is a highly optimized open-source software for molecular dynamics (MD) simulations. It models atomic and molecular movements, particularly for biochemical systems, and is efficient in calculating non-bonded interactions. A typical GROMACS workflow prepares input files, then runs the simulation. # Step 1: Prepare the run input file (.tpr) gmx grompp -f pme.mdp -c conf.gro -p topol.top -o topol.tpr # Step 2: Run the molecular dynamics simulation mpirun -np 4 gmx_mpi mdrun -s topol.tpr -ntomp 4 # To run a specific benchmark system (e.g., 'benchPEP-h') mpirun -np gmx_mpi mdrun -s benchPEP-h.tpr -ntomp Note: Tune MPI processes (-np) and OpenMP threads (-ntomp) to your hardware. 1.2. GROMACS: Molecular Dynamics Simulation Software GROMACS (GROningen MAchine for Chemical Simulations) is a highly optimized open-source software for molecular dynamics (MD) simulations. It models atomic and molecular movements, particularly for biochemical systems, and is efficient in calculating non-bonded interactions. A typical GROMACS workflow prepares input files, then runs the simulation. # Step 1: Prepare the run input file (.tpr) gmx grompp -f pme.mdp -c conf.gro -p topol.top -o topol.tpr # Step 2: Run the molecular dynamics simulation mpirun -np 4 gmx_mpi mdrun -s topol.tpr -ntomp 4 # To run a specific benchmark system (e.g., 'benchPEP-h') mpirun -np gmx_mpi mdrun -s benchPEP-h.tpr -ntomp Note: Tune MPI processes (-np) and OpenMP threads (-ntomp) to your hardware. 2. Interpreting Performance Output (Benchmarking POV) Understanding benchmark output is crucial for evaluating HPC system throughput and efficiency. 2.1. miniFE Performance Metrics miniFE outputs performance data, primarily focused on: Total CG Mflops (Mega-Floating Point Operations per Second for Conjugate Gradient solve): The main Figure of Merit (FOM). Higher values indicate better performance, reflecting the system's efficiency in sparse linear algebra, often limited by memory bandwidth and FPU throughput. 2.2. GROMACS Performance Metrics GROMACS provides detailed output, with the key metric being: ns/day (nanoseconds per day): The standard performance metric for GROMACS. It shows how many nanoseconds of simulated time can be computed per real-world day. A higher ns/day means faster simulation. This metric is ideal for comparing different CPU architectures or configurations. Other useful outputs include Total Wall Time and a breakdown of time spent in different force calculations, which helps pinpoint specific bottlenecks. 3. Bottlenecks in Running HPC Benchmarks Achieving peak HPC performance requires identifying and mitigating bottlenecks that limit system throughput. 3.1. miniFE Specific Bottlenecks miniFE is particularly sensitive to: Memory Bandwidth: The Sparse Matrix-Vector Product (SpMV) is highly memory-bandwidth bound due to irregular memory access patterns. Cache Misses: Irregular accesses lead to frequent cache misses, increasing data retrieval latency. Inter-node Communication (for large problems): For distributed problems, communication during assembly and the Conjugate Gradient solver can be limited by network latency and bandwidth. 3.2. GROMACS Specific Bottlenecks For GROMACS, key bottlenecks include: CPU Core Performance & Threading: The number of cores and their individual performance (Instructions Per Cycle (IPC), clock speed) directly impact ns/day. Optimal balance between MPI ranks and OpenMP threads per rank is crucial. Memory Bandwidth: The CPU needs to access large datasets frequently for force calculations. SIMD Vectorization: GROMACS heavily relies on CPU SIMD instructions (e.g., NEON). If the CPU architecture or compiler doesn't fully exploit these, performance will suffer. Cache Utilization: Efficient cache usage is critical for the main simulation loop. Inter-node Communication: For large systems simulated across multiple nodes, MPI communication for domain decomposition and force summation can be a significant bottleneck, even with fast interconnects. NUMA Effects: Proper process and memory binding is crucial on multi-socket systems to minimize cross-socket memory access latency. Load Imbalance: Uneven workload distribution across PP and PME leads to idle compute units. 3.3. Dynamic Monitoring for Bottleneck Analysis (Frequency, Power, Temperature) Beyond static analysis, dynamic monitoring of CPU frequency, power consumption, and temperature during benchmark execution provides invaluable insights for root-causing performance bottlenecks. This data, when mapped over the run duration, can reveal transient issues that logs alone might miss. Application-Specific Context: For miniFE, if memory bandwidth is the primary bottleneck, the CPU might not be fully utilized, leading to lower-than-expected power consumption and temperatures, even if the frequency remains high. Conversely, if the SpMV operations push the CPU's compute capabilities, sustained high power and temperature could be observed. Any sudden dips in Mflops alongside frequency drops would directly point to thermal or power throttling. For GROMACS, which can be highly compute-intensive, sustained high power consumption and temperatures are common. Analyzing frequency, power, and temperature trends can reveal if the ns/day performance is being limited by the CPU's ability to maintain its turbo frequencies due to thermal constraints or if it's hitting a configured power envelope. Discrepancies between expected maximum performance and observed ns/day can often be explained by these dynamic system responses. Tools for Monitoring: Various tools can collect this data, including vendor-specific utilities (e.g., Intel's pcm, AMD's uProf), Linux tools (perf, turbostat, sensors), or IPMI/BMC interfaces for server-level metrics. Correlating these dynamic metrics with the benchmark's reported performance can significantly aid in precise bottleneck identification and system optimization. Conclusion Effective HPC benchmarking goes beyond simply running an application and reporting a single performance number. As demonstrated with miniFE and GROMACS in a CPU-centric environment, a deep understanding of the benchmark's computational characteristics is essential. Identifying whether a workload is memory-bound, compute-bound, or communication-bound is the first step toward optimizing performance. Furthermore, leveraging dynamic monitoring of CPU frequency, power consumption, and temperature provides invaluable diagnostic data. By integrating performance metrics with detailed system telemetry, HPC administrators and researchers can precisely pinpoint bottlenecks, fine-tune system configurations, and ultimately extract the highest possible performance.

LLMs are Getting Better and Smaller Let’s look at Llama as an example. The rapid evolution of these models highlights a key trend in AI: prioritizing efficiency and performance. When Llama 2 70B launched in August 2023, it was considered a top-tier foundational model. However, its massive size demanded powerful hardware like the NVIDIA H100 accelerator. Less than nine months later, Meta introduced Llama 3 8B, shrinking the model by almost 9x. This enabled it to run on smaller AI accelerators and even optimized CPUs, drastically reducing the required hardware costs and power usage. Impressively, Llama 3 8B surpassed its larger predecessor in accuracy benchmarks. Setup details Tested with llama.cpp on Machine: Gv4 r8g.24xlarge OS: ubuntu 2204 kernel: 6.8.AWS Model: Meta-Llama-3.1-8B-Instruct- Q8_0.gguf Test sweep nproc x nthreads x bs [1-32] Graphs with observations highlighting benefits Token generation is done in an auto-regressive manner and is highly sensitive to the length of output needed to be generated. Arm optimizations help here with larger batch sizes, increasing the throughput by more than 2x. Conclusion For Meta-Llama-3.1-8B-Instruct- Q8_0.gguf, Graviton4 can generate 161 tokens per sec which translates to 102,486 tokens per dollar.

Building a great product is only half the battle. Demonstrating its capabilities convincingly — whether in front of customers, at an exhibition, or during a PoC — is often what seals the deal. Yet, for many companies, setting up demos ends up as a side project that falls through the cracks. At Whileone, we understand this challenge. That’s why helping companies build reliable, repeatable demos has been part of our mission since day one. The Problem with Ad-Hoc Demos Most companies start strong when building a product, but creating demos usually gets delegated to engineers as a “when-you-have-time” task. This often results in: Inconsistent setups that don’t reflect the product’s full potential Broken environments due to configuration drift or missing dependencies Missed opportunities at conferences, sales pitches, or proof-of-concept trials The reality is — demos are critical, and they deserve dedicated engineering effort. Our Role in Fixing It Since our inception, WhileOne has been the go-to partner for companies needing production-quality demo setups. Whether it’s for exhibitions, PoC engagements, or internal experimentation frameworks, we build environments that work — every time. We've Supported Demos At: ComputeX Open Compute Project (OCP) Summit CloudFest RISC-V Summits SuperCompute And many more These demos are often used in booths, technical sessions, or partner showcases, and they just work — because we build them with reliability, repeatability, and reproducibility in mind. Kubernetes and Container-Based Environments Our demo environments are often built on Kubernetes or Docker, ensuring they are: Easily reproducible across developer machines and exhibition floors Modular and maintainable, for rapid iteration and updates Cloud-ready and on-prem compatible This allows your team to focus on what matters engaging your audience rather than wrestling with deployment issues. Demo Infrastructure Vision We believe demo infrastructure should be treated like production infrastructure: Version-controlled Testable Portable And every next demo being built up on a previous version/iteration. By working with WhileOne, your demos will never be an afterthought again.

Meta’s Llama 4 Scout, released in April 2025, is a 17-billion parameter general-purpose language model that brings powerful reasoning to a broader range of applications—including those running without GPUs. This blog focuses on benchmarking Llama 4 Scout on CPU-only systems, covering: Tokens per second Latency per token Prompt handling efficiency Quantization techniques Architecture-specific optimization for x86, ARM, and RISC-V (RV64) Converting to GGUF format for efficient deployment Why Benchmark on CPU? While most LLMs are deployed on GPUs, CPU-only inference is often necessary for: Edge devices Cloud VMs with no GPU access Open hardware ecosystems (e.g., RISC-V) Cost-conscious deployments That makes Llama 4 Scout a strong candidate, especially with quantized variants. Key Benchmark Metrics Tokens/sec Overall throughput, critical for long completions Latency/token Time to generate one token; important for chats Prompt size sensitivity How inference speed degrades with longer inputs Memory usage RAM footprint determines if the model can run at all Why Quantization Is Essential Quantization reduces the memory and compute requirements of large models. Llama 4 Scout quantized to int4 or int8 can run comfortably on CPUs with 8–16 GB of RAM. Benefit: Impact on Llama 4 Scout Memory savings: From 34GB → ~5–7GB (int4) Speedup: Up to 3× faster than float16 Hardware fit: Allows ARM & RV64 CPUs to host inference Tools like ggml, llama.cpp, and MLC support quantized Llama 4 models, including CPU backends. Architecture-Specific Performance Considerations 🔹 x86-64 (Intel, AMD) Vector Support: AVX2 or AVX-512 preferred Threading: Mature OpenMP and NUMA support Performance: High; well-optimized in llama models ARM (Graviton, Apple Silicon, Neoverse) Vector ISA: NEON (128-bit) on all, SVE/SVE2 on newer chips Threading: Requires tuning due to core heterogeneity Quantization: NEON handles int8 and int4 efficiently Tip: Use taskset and numactl to pin threads for optimal performance. RISC-V (RV64 with RVV) Vector ISA: RISC-V Vector Extension (RVV), variable width Quantization: Essential; float32 models are impractical on RV64 edge devices Tooling: llama.cpp support is experimental but growing For RV64, memory layout and cache-friendly quantization are critical due to limited bandwidth. Sample Inference Results (Hypothetical) Architecture Model Variant Prompt Size Tokens/sec. RAM Usage x86_64 Llama 4 Scout int4 512 11.2 ~6.5 GB ARM Neoverse Llama 4 Scout int4 512 8.7 ~6.5 GB RISC-V RV64 Llama 4 Scout int4 512 3.2 ~6.5 GB These results assume multi-threaded CPU inference with quantized weights using llama.cpp or similar. From Raw Model to GGUF: Why and How? To run Meta Llama 4 Scout efficiently on CPU-only systems, especially with tools like llama.cpp, the model must be in GGUF format. Why Convert to GGUF? GGUF (Grokking GGML Unified Format) is a compact, memory-optimized model file format designed for CPU and edge inference using: llama.cpp mlc-llm text-generation-webui GGUF Advantage : Benefit Memory Efficient: Packs quantized weights and metadata Fast Load Times: No need to re-tokenize or parse configs Metadata Preserved: Tokenizer, vocab, model type included Simplified Use: Single file usable across many tools How to Convert Llama 4 Scout to GGUF Download the Raw Model (HF Format) Get the original model from Hugging Face (e.g., meta-llama/Meta-Llama-4-Scout-17B). Install transformers and llama-cpp-python tools pip install transformers huggingface_hub git clone https://github.com/ggerganov/llama.cppcd llama.cppmake Run the GGUF Conversion Script From the llama.cpp/scripts directory: python convert.py \ --outfile llama4-scout.gguf \--model meta-llama/Meta-Llama-4-Scout-17B \ --dtype q4_0 3. Load It in Your Inference Tool Once converted, the .gguf file can be run directly:./main -m llama4-scout.gguf -p "Hello, world" GGUF + Quantization = CPU Superpowers Converting to GGUF enables you to quantize during the conversion: q4_0, q4_K, q5_1, and q8_0 supported You reduce size dramatically—from ~34GB → ~5–7GB for q4 It ensures compatibility with CPU SIMD instructions like AVX, SVE, or RVV On RISC-V or ARM boards with limited memory, GGUF + int4 is often the only way to get Llama 4 Scout running at all. Pro Tip: GGUF Conversion Options You can fine-tune conversion settings: --vocab-type to customize tokenizer structure --trust-remote-code if the Hugging Face repo uses custom loading --quantize q4_K for better int4 accuracy Final Thoughts Meta's Llama 4 Scout is one of the most practical open-source LLMs for CPU inference in 2025. With quantization and SIMD-aware deployment, it can serve: Edge applications (IoT, phones) Sovereign compute platforms (RISC-V) Cloud-native environments without GPUs If you’re interested in pushing the limits of open LLMs on CPU architectures, Llama 4 Scout is one of the best starting points.