Whileone Tech Space

Whileone has been making slow incremental changes to XNNPACK that makes the first convolution of common vision models 1.20–1.39x faster on AWS Graviton3, Graviton4 and Graviton5. Every other layer runs at the same speed. XNNPACK is the CPU engine under LiteRT (TensorFlow Lite), ExecuTorch and MediaPipe. A small win there reaches a very large number of devices. The problem: a kernel built for wide inputs On Arm CPUs with the I8MM extension, XNNPACK runs int8 convolutions throu

JetStream 3 deliberately moved away from microbenchmarks, and for browser engineers that was the right call. Small, tight loops once let engines win benchmarks without making real web apps faster. Pre-silicon CPU teams face the opposite problem. They need to know how a chip that doesn't exist yet will run browser workloads, and a full JavaScript suite is nearly impossible to run on a simulator. The answer is not to abandon microbenchmarks, but to build ones that are correlate

Every silicon vendor at every keynote this year has said some version of the same sentence: the next CPU generation will be built for agentic workloads. Intel talked about CPUs as the orchestration layer for agentic systems, Arm argued that agents are driving a fresh wave of data centre CPU demand, and NVIDIA introduced Vera as a CPU designed for agentic workflows rather than general-purpose compute. The direction is settled. The definition is not. Because here is the questio

Agentic AI has done something no previous AI workload managed: it made the host CPU interesting again. In single-shot inference the CPU tokenizes a prompt, hands off to the accelerator, and waits. In an agent loop it runs everything between model calls — orchestration and control flow, tool dispatch, sandbox spawn and teardown, file I/O, test execution, retrieval, JSON parsing, retries. Recent characterization work has put numbers on this. In tool-dominated agentic pipelines,

In the world of enterprise IT, raw computing performance used to be the only metric that mattered. If a server crashed through complex tasks, nobody cared how much electricity it burned or how hot the room got. That mindset shifted dramatically in December 2007 when the Standard Performance Evaluation Corporation (SPEC) released SPECpower_ssj2008, the first industry-standard benchmark designed to evaluate both performance and power consumption across varying workload levels.

Industry-standard benchmarks play a critical role in evaluating and comparing server performance across different processor architectures, operating systems, and hardware platforms. Among these benchmarks, SPEC CPU 2017 is widely recognized for measuring compute-intensive performance using a diverse collection of integer and floating-point workloads that represent real-world applications such as artificial intelligence, simulation, image processing, and data analytics. As org

Most engineering teams find out their system is broken when a customer tweets about it at 2 AM. We find out before the customer ever notices. And when something does go wrong, we know the root cause in under three minutes, not three hours. Here's what we actually do, and why it matters more than most people realize. Site Reliability Engineering Isn't Just Dashboards and On-Call Rotations There's a widespread misconception that Site Reliability Engineering is glorified sysadmi

Managing cloud expenses effectively is no longer just a finance problem; it’s an operational necessity. As cloud adoption scales, tracking and optimizing costs across distributed engineering teams becomes incredibly complex. Selecting the right cloud expense management tool helps organizations gain visibility, curb runaway spend, and maximize their infrastructure investments. But with dozens of platforms on the market, how do you separate high-level dashboards from tools that

using PyTorch on ARM and x86 CPUs CPU Inference Benchmarking · ARM vs x86 · W8A8 Quantization Overview ResNet50 is a convolutional neural network built using bottleneck residual blocks of the form: 1×1 Conv → 3×3 Conv → 1×1 Conv + Skip Connection Among these layers, the 3×3 convolution layers dominate execution time, making Conv2d the primary hotspot during inference. Although ResNet50 performs ~4 GFLOPs per inference, it is not compute-heavy enough to fully utilize moder

For decades, AI benchmarks lived in comfortable isolation. A model answered a question, we checked the answer, we assigned a score. Agentic AI broke that contract. When a model can browse the web, write and execute code, call external APIs, and chain its own decisions across hundreds of steps, a single accuracy number tells you almost nothing about whether the system is actually trustworthy. Evaluating an agent is less like grading an exam and more like auditing a junior empl

Objective Building an AI-powered system that automatically identifies and classifies software errors into meaningful categories, and provides root cause analysis and recommendations. Challenge Solution Developed multiple machine learning models to classify errors within seconds Integrated retrieval-augmented generation (RAG) for Root cause analysis Used Large language Models (LLMs) for providing narratives and recommendations Benefits

Introduction For this project, I have used llama.cpp as the local inference engine and TinyLlama-1.1B-Chat-v1.0 as the language model. llama.cpp is a lightweight C/C++ inference framework designed to run LLMs locally with minimal setup across CPUs and GPUs. It is well suited for embedded and edge-oriented workflows because it supports efficient local execution without depending on cloud APIs. The TinyLlama model used here is the chat-tuned 1.1B parameter variant published on

Introduction In our previous post, How CloudNudge Can Help You Optimize and Manage Your Cloud Expenses, we discussed how visibility is the first step toward financial control. However, for software and hardware engineers, a low cloud bill is a hollow victory if it comes at the cost of system latency. Saving money is great. Saving money without breaking your application is Performance Engineering. The Performance-Cost Paradox The most common mistake in cloud optimization is "B

Software efficiency is more critical than ever. Users expect applications to be fast, reliable, and scalable. Achieving this requires more than just writing clean code; it demands a strategic approach known as software performance optimization. This process ensures that software not only meets functional requirements but also performs optimally under various conditions. By focusing on performance engineering, businesses can deliver superior user experiences, reduce operationa

Chaos Engineering: Enhancing System Resilience Chaos engineering is the discipline of intentionally introducing controlled faults to validate system resilience. In any production ecosystem, spanning silicon validation, system integration, and software stacks, it helps uncover performance, reliability, and scalability risks long before production deployment. Understanding Kubernetes Pods Modern validation and benchmarking workloads increasingly run on Kubernetes. Pods, the sma

Cold starting docker containers is expensive. Before a Dockerized application does anything useful, it pulls images, initializes the runtime, loads classes or modules, allocates memory, opens files and sockets, and slowly warms into a steady operating state. In modern infrastructure, this cost shows up everywhere: pod restarts, scale-outs, rollouts, autoscaling events. Each time, the same warm-up work is paid for again. Capture and restore offers a different idea: instead of

Discover how agentic AI is flipping the performance engineering paradigm. Learn how to evolve from a "loop tuner" to a "constraints governor" with our checklist for AI-proof SLOs.

Engineering R&D in a Changing Landscape Engineering Research and Development has always been at the heart of innovation. But today, its role is evolving rapidly. What was once primarily about pushing technical boundaries is now equally about speed, efficiency, and alignment with business outcomes. As industries grow more complex and interconnected, Engineering R&D teams are being asked to deliver faster, smarter, and with fewer margins for error. From a marketing and sales po

Introduction to RISC-V and Software Readiness As RISC-V expands into accelerator domains, software readiness becomes as critical as hardware innovation. This work focuses on implementing a set of mathematical and BLAS primitives for a custom RISC-V architecture. These primitives form foundational building blocks for numerical computing. The implementation includes vector and matrix operations, with careful attention to numerical correctness and floating-point behavior. Overco

With the evolution of hardware design and development, two tools have become fundamental for those working on Instruction Set Architectures (ISA) QEMU and FPGA boards. Although both serve as key resources for developing, testing, and experimenting with different ISAs (such as RISC-V, ARM, x86, etc.), they operate in significantly different ways. This blog highlights the key distinctions between QEMU and FPGA boards and their use cases across various architectures. Key Featur