Search this site

In today’s software-driven world, ensuring the seamless performance of applications under varying workloads is a necessity. For performance testing, tools like NeoLoad empower testers to simulate real-world conditions and optimize application behavior. In this blog, we’ll delve into the practical use of NeoLoad for WebFocus performance testing, focusing on scenarios like chart rendering, page loads, data uploads, and resource utilization. Understanding the Scope of Performance Testing In the WebFocus performance testing project, the primary focus areas included: Chart Rendering: Time required to render single and multiple charts. Page Performance: Monitoring login/logout and page rendering times. Resource Utilization: Tracking CPU and memory usage during various operations. Data Uploads: Measuring the time, CPU, and memory required for data uploads. These benchmarks were vital to ensuring optimal application performance under increasing workloads. The Process: From Recording to Optimization 1. Recording the Scenario Flow The first step was to record user interactions. For instance, in a chart-rendering scenario, the flow of loading, interacting, and rendering charts was captured using NeoLoad's recording feature. 2. Script Customization Post-recording, scripts were adjusted to ensure reusability: Dynamic Parameters: Tokens (e.g., session IDs) generated during requests were identified and correlated. For hidden tokens, developer tools were utilized to trace their origin and ensure they were passed correctly in subsequent requests. Parameterization: URLs, usernames, passwords, and IPs were parameterized to make scripts adaptable for different environments, releases, or multiple users. Loops: Requests with multiple calls were optimized by implementing loops to handle repetitions effectively. 3. Preliminary Testing Before load testing, the scripts were tested with a single user to verify functionality. This step ensured that all dynamic parameters and correlations were correctly handled. Load Testing with NeoLoad Once the scripts were verified, load testing was performed: Setup: A Load Generator (LG) was configured to simulate user traffic, and the controller executed the recorded scripts. Testing Parameters: CPU and memory usage were monitored. Response times were analyzed for specific workloads. Scaling Observations: Tests assessed how the system scaled with increasing loads, identifying bottlenecks or performance degradation. Performance Analysis and Optimization When performance issues arose during testing, the following measures were taken: Resource Allocation: Increased CPU and memory for the WebFocus environment. Request and Database Optimization: Identified long-processing database queries and collaborated with developers for optimization. Heap Dump Analysis: Collected heap dumps for detailed investigation of memory-related issues. Detailed Reporting: Created detailed reports highlighting response times, resource utilization, and optimization recommendations. NeoLoad Features that Enhanced Testing Dynamic Parameter Handling: Simplified the treatment of session tokens and hidden parameters. Parameterization: Enabled reusability of scripts across environments and user scenarios. Realistic User Simulation: Simulated complex interactions such as rendering multiple charts or uploading large datasets. Resource Monitoring: Provided real-time insights into CPU and memory usage, enabling quicker bottleneck identification. Integration: Collaborated with tools like developer consoles for deeper analysis. Example Scenario: Chart Rendering Benchmarking Objective: Test the rendering performance of 10 simultaneous charts. Process: Recorded the chart rendering workflow in NeoLoad. Customized the script to handle dynamic session tokens and parameterized user inputs. Ran a baseline test with a single user, followed by load testing with 100, 500, and 1,000 concurrent users. Outcome: Identified bottlenecks at 500 users due to CPU exhaustion. Recommended increasing resources and optimizing chart generation queries. Benefits of Using NeoLoad Efficiency: Parameterization and dynamic handling reduced scripting effort. Scalability: Load generators allowed easy simulation of large-scale user traffic. Precision: Real-time monitoring and detailed reporting enabled accurate issue identification. Reusability: Modular scripts streamlined testing across environments and releases. Conclusion NeoLoad proved instrumental in ensuring the WebFocus environment could handle real-world workloads. From recording and customizing scripts to analyzing performance under stress, NeoLoad simplified the performance benchmarking process. Its ability to simulate realistic conditions, monitor resource utilization, and provide actionable insights makes it a vital tool for any performance testing team. Whether you’re testing chart rendering, page loads, or data uploads, NeoLoad offers the tools and flexibility needed for comprehensive performance benchmarking. Start leveraging NeoLoad today to unlock your application’s full potential!

Fuzzing RISC-V compilers like GCC and LLVM is a crucial practice for ensuring the correctness and security of the entire software ecosystem built on this architecture. It's not about finding vulnerabilities in the final compiled code, but rather about discovering bugs within the compiler itself that could lead to incorrect code generation, unexpected behavior, or even exploitable flaws. Why Compiler Fuzzing is a Unique Challenge  Fuzzing compilers is different from fuzzing a typical application. Instead of feeding random data to a program, you're generating random, yet syntactically valid, source code  to feed to the compiler. A dumb fuzzer that just mutates bytes will quickly generate code that can't even be parsed, missing deeper bugs. The primary goal of compiler fuzzing is to detect two main types of bugs: Crashes and Panics:  The fuzzer generates code that causes the compiler to crash, hang, or throw a fatal error during compilation. This indicates a compiler bug that needs to be fixed. Miscompilations:  This is the most dangerous type of bug. The compiler successfully compiles the fuzzed code, but the generated machine code (the RISC-V assembly) is incorrect. This can lead to silent data corruption, security vulnerabilities, or unpredictable program behavior. Finding these requires a technique called differential fuzzing . The Power of Differential Fuzzing for RISC-V Compilers  Differential fuzzing is an exceptionally powerful technique for finding miscompilations in RISC-V compilers. Here's how it works: A fuzzer, often one that generates valid C or C++ code (like csmith), creates a unique program. This program is compiled by at least two different compilers  (e.g., GCC and LLVM) or with different optimization flags  (e.g., -O0 and -O3). The compiled binaries are then executed, and their outputs are compared. If the outputs don't match, it means at least one of the compilers has a miscompilation bug. The fuzzer then saves this specific source code as a test case for a developer to analyze. This method effectively uses a "test oracle" to automatically identify bugs without needing to know the correct output beforehand. It's a key reason why so many compiler bugs have been found in both GCC and LLVM. Key Tools and Repositories for RISC-V Compiler Fuzzing  While many of the general-purpose fuzzers mentioned before (like AFL++) can be used to fuzz a compiler's source code, specialized tools are often needed to effectively generate and test valid RISC-V-specific code. csmith:  This is a well-known, randomized test case generator for C programs. It creates complex, valid C code that is a perfect input for differential testing of C compilers like GCC and LLVM. While not RISC-V-specific, it's an essential part of the workflow for fuzzing any C compiler's RISC-V backend. GitHub Repo:   https://github.com/csmith-project/csmith RISCV-DV:  Maintained by the RISC-V community, this tool is primarily for design verification of RISC-V processors, but it can be used to generate complex instruction sequences for testing compiler backends. It's highly configurable and can target specific ISA extensions. GitHub Repo:   https://github.com/google/riscv-dv IRFuzzer:  A specialized fuzzer for the LLVM backend . Instead of generating C/C++ source code, it generates LLVM's Intermediate Representation (IR), allowing it to directly test the backend code generation without worrying about frontend bugs. This is a very targeted approach for finding issues in LLVM's RISC-V code generator. GitHub:  As a research tool, you can often find resources on arXiv and university websites. Searching for "IRFuzzer" on GitHub will lead to related projects. RISCV-Vector-Intrinsic-Fuzzing (RIF):  A specific fuzzer designed to generate random code using the RISC-V Vector Extension (RVV) intrinsics. This is crucial for verifying that compilers like GCC and LLVM correctly implement this complex and performance-critical part of the RISC-V ISA. GitHub Repo:   https://github.com/sifive/riscv-vector-intrinsic-fuzzing Patrick-rivos/compiler-fuzz-ci:  This GitHub repository provides a great example of a Continuous Integration (CI) setup for fuzzing RISC-V compilers. It demonstrates how to combine tools like csmith with a CI pipeline to automatically fuzz GCC and LLVM and report bugs. GitHub Repo:   https://github.com/patrick-rivos/compiler-fuzz-ci Fuzzing RISC-V compilers is an ongoing and critical effort. It ensures that the software developers are building on top of is reliable, secure, and correctly translated to the underlying hardware, strengthening the entire RISC-V ecosystem.

Network testing tools such as netperf can perform latency tests plus throughput tests and more. In netperf, the TCP_RR and UDP_RR (RR=request-response) tests report round-trip latency. With the -o flag, output metrics can be customized to display the exact information. Here’s an example of using the test-specific -o flag so netperf outputs several latency statistics: Google has lots of practical experience in latency benchmarking and as per blog using-netperf-and-ping-to-measure-network-latency , we tried to create own latency benchmarking before and after migrating workloads to the OCI cloud. Which tools and why All the tools in this area do roughly the same thing: measure the round trip time (RTT) of transactions. Ping does this using ICMP packets. ping -c 100 ping command sends one ICMP packet per second to the specified IP address until it has sent 100 packets. netperf -H -t TCP_RR -- -o min_latency,max_latency,mean_latency -H for remote-host and -t for test-name with a test-specific option -o for output-selectors. When we run latency tests at Google in a cloud environment, our tool of choice is PerfKit Benchmarker (PKB). This open-source tool allows to run benchmarks on various cloud providers while automatically setting up and tearing down the virtual infrastructure required for those benchmarks. After setting up, PerfkitBenchmarker, its simple to run ping and netperf benchmarks ./pkb.py --benchmarks=ping --cloud=OCI --zone=us-ashburn-1 ./pkb.py --benchmarks=netperf --cloud=OCI --zone=us-ashburn-1 --netperf_benchmarks=TCP_RR These commands run intra-zone latency benchmarks between two machines in a single zone in a single region. Intra-zone benchmarks like this are useful for showing very low latencies, in microseconds, between machines that work together closely. Latency discrepancies We've set up two VM.Standard.E4.Flex machines running Ubuntu 22.04 in zone us-ashburn-1, and we'll use Private IP addresses to get the best results. If we run a ping test with default settings and set the packet count to 100, we get the following results: ping -c STDOUT: PING 172.16.60.168 (172.16.60.168) 56(84) bytes of data. 64 bytes from 172.16.60.168: icmp_seq=1 ttl=64 time=0.202 ms 64 bytes from 172.16.60.168: icmp_seq=2 ttl=64 time=0.205 ms … 64 bytes from 172.16.60.168: icmp_seq=99 ttl=64 time=0.329 ms 64 bytes from 172.16.60.168: icmp_seq=100 ttl=64 time=0.365 ms --- 172.16.92.253 ping statistics --- 100 packets transmitted, 100 received, 0% packet loss, time 101353ms rtt min/avg/max/mdev = 0.371/0.450/0.691/0.040 ms By default, ping sends out one request each second. After 100 packets, the summary reports that we observed an average latency of 0.450 milliseconds, or 451 microseconds. For comparison, let’s run netperf TCP_RR with default settings for the same amount of packets. netperf-2.7.0/src/netperf -p {command_port} -j -v2 -t TCP_RR -H 132.145.132.29 -l 60 -- -P ,{data_port} -o THROUGHPUT, THROUGHPUT_UNITS, P50_LATENCY, P90_LATENCY, P99_LATENCY, STDDEV_LATENCY, MIN_LATENCY, MEAN_LATENCY, MAX_LATENCY --num_streams=1 --port_start=20000' --timeout 360 Netperf Results: {'Throughput': '4245.34', 'Throughput Units': 'Trans/s', '50th Percentile Latency Microseconds': '228', '90th Percentile Latency Microseconds': '239', '99th Percentile Latency Microseconds': '372', 'Stddev Latency Microseconds': '92.06', 'Minimum Latency Microseconds': '215', 'Mean Latency Microseconds': '235.08', 'Maximum Latency Microseconds': '21059' Which test can we trust? To explain, this is largely an artefact of the different intervals the two tools used by default. Ping uses an interval of 1 transaction per second while netperf issues the next transaction immediately when the previous transaction is complete. Fortunately, both of these tools allow to manually set the interval time between transactions. For ping, -i flag to set the interval, given in seconds or fractions of a second. On Linux systems, this has a granularity of 1 ms, and rounds down. $ ping -c 100 -i 0.010 For netperf TCP_RR, we can enable some options --enable-spin flag to compile with fine-grained intervals -w flag, to set the interval time, and the -b flag, to set the number of transactions sent per interval. This approach allows to set intervals with much finer granularity, by spinning in a tight loop until the next interval instead of waiting for a timer; this keeps the cpu fully awake. Of course, this precision comes at the cost of much higher CPU utilization as the CPU is spinning while waiting. *Note: Alternatively, setting less fine-grained intervals by compiling with the --enable-intervals flag. Use of the -w and -b options requires building netperf with either the --enable-intervals or --enable-spin flag set. The tests here are performed with the --enable-spin flag set. netperf with an interval of 10 milliseconds using: $ netperf -H -t TCP_RR -w 10ms -b 1 -- -o min_latency,max_latency,mean_latency Now, after aligning the interval time for both ping and netperf to 10 milliseconds, the effects are apparent: Ping result is --- 172.16.92.253 ping statistics --- 1000 packets transmitted, 1000 received, 0% packet loss, time 15981ms rtt min/avg/max/mdev = 0.252/0.306/0.577/0.025 ms Netperf results are Minimum Latency Microseconds,Maximum Latency Microseconds,Mean Latency Microseconds 215,235.08,21059 We have integrated OCI as a provider in Perfkitbenchmarker which we are using to carry out testing. Here are the results of the inter region ping benchmark for A1.Flex2, E4.Flex.1 and S1.Flex vms. Tested netperf for intra region, considered us-ashburn-1 region here. Generally, netperf is recommended over ping for latency tests. This isn't due to any lower reported latency at default settings, though. As a whole, netperf allows greater flexibility with its options and we prefer using TCP over ICMP. TCP is a more common use case and thus tends to be more representative of real-world applications. That being said, the difference between similarly configured runs with these tools is much less across longer path lengths. Also, remember that interval time and other tool settings should be recorded and reported when performing latency tests, especially at lower latencies, because these intervals make a material difference.

Introduction: High-Performance Linpack (HPL) is a widely used benchmark for testing the computational performance of computing systems. In this blog post, we explore an intriguing scenario where we conducted HPL tests on two ARM64 machines. Surprisingly, the Host-2 machine exhibited a 20% lower performance than the Host-1 machine in the HPL test. Intrigued by this result, we embarked on a journey to comprehensively diagnose the underlying cause of this performance discrepancy. Why HPL for Performance Testing? People use the High-Performance Linpack (HPL) benchmark for performance testing because it provides a standardised and demanding workload that measures the peak processing power of computer systems, particularly in terms of floating-point calculations. It helps assess and compare the computational capabilities of different hardware configurations. This benchmark helps in comparing and ranking supercomputers' performance and is often used as a metric for the TOP500 list of the world's most powerful supercomputers. For more information, you can refer to the TOP500 article here: TOP500 List​ Objective: The primary objective of this investigation was to identify the reason behind the 20% performance difference observed in the HPL test between the Host-1 and Host-2 machines. To comprehensively diagnose the performance discrepancy, we conducted additional benchmark tests, including Stream, Lmbench, and bandwidth tests. 1. System Details: We conducted a fair and controlled experiment using two ARM64 machines, referred to as Host-1 and Host-2. 1.1 Machine Specifications ( Host-1 and Host-2 ): CPU(s): 96 Architecture: aarch64 Total memory: 96 GB Memory speed: 3200 MHz 2. Running HPL Benchmark: To run the HPL benchmark on an arm64 machine, you can refer to the GitHub repository provided: https://github.com/AmpereComputing/HPL-on-Ampere-Altra.​ This repository likely contains instructions, scripts, and configurations specific to running HPL on Ampere Altra ARM64-based machines. It's important to follow the guidelines provided in the repository to ensure accurate and meaningful benchmarking results. 2.1 HPL Scores: Upon completing the HPL benchmark on both machines, we computed and compared the achieved HPL scores. The Host-1 machine garnered a higher HPL score, signifying better computational performance. Machine​ Time ( sec ) Score Host-1 619.91 1245 Host-2 784 985 This result raised a critical question: why was there such a substantial performance gap? To delve into the root causes behind this discrepancy, we decided to conduct a series of additional tests to comprehensively investigate the issue. 3. Exploring Additional Tests: We conducted several other benchmark tests to comprehensively investigate the performance discrepancy between the Host-1 and Host-2 ARM64 machines. These tests aimed to shed light on various aspects of the systems' hardware and memory subsystems, providing a holistic understanding of the observed difference. Below, we detail the tests and their findings: 3.1 Stream Benchmark: The Stream benchmark assesses memory bandwidth and measures the system's capability to read from and write to memory. The benchmark consists of four fundamental tests: Copy, Scale, Add, and Triad. Copy: Measures the speed of copying one array to another. Scale: Evaluates the performance of multiplying an array by a constant. Add: Tests the speed of adding two arrays together. Triad: Measures the performance of a combination of operations involving three arrays. The Stream benchmark helps uncover memory bandwidth limitations and assess memory subsystem efficiency. Host-1 machine result : Function Best Rate MB/s Avg time Min time Max time​ Copy 103837.8 0.367897 0.36192 0.373494 Scale 102739.4 0.369191 0.365789 0.372439 Add 106782.7 0.536131 0.527908 0.542759 Triad 106559.1 0.533549 0.529016 0.537881 Host-2 machine result : Function Best Rate MB/s Avg time Min time Max time Copy 66071.3 0.572721 0.568794 0.575953 Scale 65708.8 0.575758 0.571932 0.580686 Add 67215.5 0.843995 0.838667 0.848371 Triad 67668.1 0.837109 0.833058 0.84079 Best Rate MB/s vs Function Graph In the Stream Benchmark results, Host-1 outperformed Host-2 across all functions (Copy, Scale, Add, Triad). Host-1 demonstrated higher memory bandwidth in each function, achieving significantly faster data transfer rates. This suggests a stronger memory subsystem performance in Host-1 compared to Host-2. 3.2 Lmbench for Memory Latency: Lmbench is a suite of micro-benchmarks designed to provide insights into various aspects of system performance. The suite includes latency tests for system calls, memory accesses, and various operations to quantify the system's responsiveness. Memory access tests include random read/write latency and bandwidth, helping to identify memory subsystem performance. File I/O tests evaluate file system performance, providing insights into storage subsystem capabilities Result Memory Latency: Memory latency refers to the time it takes for the CPU to access a specific memory location. Lower latency values indicate better performance, as data can be fetched more quickly. size (MB) latency (ns)-HOST-1 latency (ns)-HOST-2 0.00049 1.43 1.429 —----- —----- ​—----- —----- —----- —----- 2 32.355 32.786 3 34.503 36.012 4 37.403 37.932 6 39.982 52.922 8 41.007 54.001 12 44.315 55.466 16 65.52 73.016 24 95.131 117.278 32 115.081 138.945 48 126.796 151.945 64 129.558 159.225 96 134.413 166.359 128 136.239 167.788 192 136.245 168.689 256 136.366 170.464 384 137.732 170.461 —----- —----- —----- —----- —----- —----- 2048 135.61 149.809 4. Analysis and Findings: After conducting these benchmark tests, we observed the Host-2 machine consistently exhibited lower performance across different tests compared to the Host-1 machine. The most significant finding came from the Lmbench test, which revealed that the Host-2 machine's RAM had notably higher latency compared to the Host-1 machine. Notably, an additional factor was identified—the RAM rank. The Host-1 machine is equipped with Dual-Rank RAM, while the Host-1 machine has Single-Rank RAM. This RAM rank difference could contribute to the performance discrepancy. The observation is in line with findings from various other studies that have examined the influence of RAM rank on system performance. To gain a more comprehensive understanding of this subject, the following articles could be of interest: Single vs. Dual-Rank RAM: Which Memory Type Will Boost Performance? - This article provides a thorough comparison between single and dual-rank RAM, aiding in comprehending the disparities between these two RAM types, methods to distinguish them, and guidance on selecting the most suitable option for your needs. (LINK) Single Rank vs Dual Rank RAM: Differences & Performance Impact : This article delves into the differences between Single Rank and Dual Rank RAM modules, investigating their structural dissimilarities and assessing the respective impacts on performance. (LINK) 5. Conclusion: After conducting an extensive series of benchmark tests, we have pinpointed certain factors that contribute to the performance disparity observed in the HPL test between the two ARM64 machines. In the Stream Benchmark results, Host-1 outperformed Host-2 across all functions (Copy, Scale, Add, Triad). Host-1 demonstrated higher memory bandwidth in each function, achieving significantly faster data transfer rates. Additionally, the higher memory latency in the Host-2 machine's RAM was identified as a key contributor to the performance gap. This latency impacted the efficiency of memory operations and had a cascading effect on overall performance. Another significant factor was the difference in RAM rank configurations — Host-1 had Dual-Rank RAM, while Host-2 had Single-Rank RAM. This divergence likely contributed to the varying memory access speeds between the two machines. 6. Future Scope: In the context of further exploration, it is recommended to extend the investigation by including additional benchmark tests, specifically focusing on the Lmbench memory bandwidth test. This test would provide deeper insights into the memory subsystem's performance on both the Host-1 and Host-2 machines. Additionally, an interesting avenue for investigation could involve modifying the RAM configuration in one of the machines and assessing its impact on performance. This would provide valuable information about the role of memory specifications in influencing the overall system performance.

We recently migrated one of our websites to Azure Arm64 VMs. However, as soon as we pushed the infrastructure change in production, we started to observe our server process being restarted infrequently. These restarts may happen within a few seconds sometimes while not occurring for hours at other times. While the redundancy in our setup ensured minimal end-user impact, we wanted to quickly address the issue at hand. Looking at the logs A quick look at the logs showed the following error before process restarts: ​malloc(): corrupted top sizeAborted (core dumped) This is a Node.js based Next.js website with nothing memory intensive being performed. So, we were surprised to see a memory related issue. A quick look at the top also suggested we had adequate memory available for our running processes. So, this definitely looked like a memory corruption. Our next challenge was to identify what caused the memory corruption. On analyzing the logs further, it did not appear that there was a single website url causing this issue. Reproducing the issue With this information at hand, we went back to our test environment (which was also running on Azure Arm64 VM) and setup a more detailed logging. We then visited a large number of our website urls to see if we could reproduce the restart. Eventually, we did find a couple of urls where the Node.js process would exit with the corrupted memory error message. Identifying the root cause Once we could reproduce the issue, we narrowed it down the to images loading on these pages. Our images were being served by Next.js next/image library. This library internally leverages the `sharp` package to optimize the images being served. So, it appeared that for some images (not all), the sharp image optimization logic was resulting in memory corruption causing our Node.js process to exit. Looking at the current & past issues for lovell/sharp on github took us to this issue, which summarized our experience. Issue details & Fix On probing further, we understood that the libspng library being used by lovell/sharp had a memory corruption issue when trying to decode a paletted PNG on Arm64. libspng addressed this issue with v0.7.2 which was picked by lovell/sharp within v0.31.0. On pinning our sharp dependency within the package.json to v0.31.0, we were able to force our next/image to pick-up this version of the sharp library (instead of the older one) for image optimiztaion. With this change, the specific images that were causing Node.js process exit earlier were now being optimized as expected. Once the change went into production, we watched our production Node.js processes for any restarts. With no restarts observed for a couple of days, we were able to mark the issue as addressed.

DLRM stands for Deep Learning Recommendation Model. It is a neural network architecture developed by Facebook AI (Meta) for large-scale personalized recommendation systems. DLRM is widely used in real-world applications where personalized recommendations or ranking predictions are needed. DLRM designed for click-through rate (CTR) prediction and ranking task. Examples: Online Advertising, E-commerce Recommendations, Social Media Feed Ranking, Streaming Services, Online Marketplace and Classifieds etc. DLRM features: DLRM Installation Options: Install Original Facebook DLRM(PyTorch) using git and python. Install DLRM using TorchRec Install NVIDIA DLRM Install DLRM in Docker (CPU-only or GPU) What Is the Relationship Between DLRM and PyTorch? DLRM is built using PyTorch. PyTorch serves as the foundational deep-learning framework that powers every component inside DLRM. PyTorch Is the Framework; DLRM Is the Model DLRM is not a framework, it is a specific neural-network architecture designed by Meta (Facebook) for large-scale recommendation systems. PyTorch provides: DLRM uses these tools to construct its dense MLPs, embedding tables, and feature-interaction layers. Pytorch Installation Options: PyTorch can be installed in several ways depending on your environment, hardware, and workflow. Install via pip (Most Common & Easiest) Install via Conda (Best for GPU Environments) Install via Docker (Isolated & Production-Friendly) Install from Source (For Developers and Custom Builds) Cloud-Based PyTorch Installation Install via Package Managers (Limited OS Support) Pytorch Installation via Docker: Installing PyTorch through Docker is one of the most reliable and hassle-free ways to set up a deep learning environment. Instead of manually managing Python versions, CUDA toolkits, cuDNN libraries, and system dependencies, Docker provides a pre-configured container where everything already works out of the box. By pulling an official PyTorch image—either CPU-only or with CUDA support—you get an isolated and reproducible environment that runs identically on any machine. Quick steps 1. Pull an image CPU-only: docker pull pytorch/pytorch:latest GPU (CUDA 11.8 example): docker pull pytorch/pytorch:latest-cuda11.8-cudnn8-runtime 2. Run the container CPU: docker run -it pytorch/pytorch:latest bash GPU (with NVIDIA container toolkit): docker run -it --gpus all pytorch/pytorch:latest-cuda11.8-cudnn8-runtime bash 3. Verify inside the container python3 -c "import torch; print(torch.__version__); print('cuda:', torch.cuda.is_available())" How to Run DLRM Inside a PyTorch Docker Container? Pull a PyTorch Docker Image Start the Container Install Dependencies (Inside the Container) Clone DLRM Repository Run DLRM DLRM Command: Running DLRM effectively requires understanding the key command-line options that control data loading, model architecture, training configuration, and performance tuning. DLRM accepts a rich set of flags that allow you to configure everything from batch sizes to embedding dimensions. These options fall into four major categories: Data Options Training Options Model Architecture Options System / Performance Options Frequently Used DLRM Command: python dlrm_s_pytorch.py \ --data-generation=synthetic \ --mini-batch-size=2048 \ --learning-rate=0.01 \ --arch-sparse-feature-size=16 \ --arch-mlp-bot="13-512-256-64-16" \ --arch-mlp-top="512-256-1" \ --print-freq=10 Conclusion Using PyTorch Docker containers to run DLRM (Deep Learning Recommendation Model) provides a streamlined, consistent, and reproducible environment across different hardware platforms. Docker eliminates dependency conflicts, simplifies setup, and ensures that the exact software stack—PyTorch version, libraries, and optimizations—can be deployed seamlessly. In short, PyTorch Docker + DLRM offers a reliable, flexible, and efficient path to train, evaluate, and deploy recommendation models with minimal friction.

Goal of this project: This project aims to determine RISC-V's readiness for running YOLOX for the latest edge requirements. Target Application: Running YOLOX on RISC-V QEMU involves setting up a RISC-V virtual machine and then configuring the necessary environment to compile and run YOLOX. Please note that this is a complex process, and it's essential to have prior experience with virtualization and RISC-V development. From the RISCV website, this is a blog (https://riscv.org/blog/2023/07/yolox-for-object-detection/) which describes the steps to build and run YOLOX for a development board. These steps did not work as is when running on QEMU. This blog assumes the readers of this blog are comfortable with a Linux-based host system (this guide is based on Ubuntu 22.04). Step 1: Install QEMU and Set Up a RISC-V Virtual Machine First, you need to install QEMU and the RISC-V toolchain. You can do this by running: ​sudo apt-get install qemu-system-riscv In this step, you'll create a RISC-V virtual machine using QEMU. You'll need a RISC-V disk image for this. You can find pre-built RISC-V images for various Linux distributions online. You can also build your own RISC-V image if you prefer. wget https://cdimage.ubuntu.com/releases/22.04/release/ubuntu-22.04.3-preinstalled-server-riscv64+unmatched.img.xz tar xf ubuntu-22.04.3-preinstalled-server-riscv64+unmatched.img.xz #Rename the qemu_image mv ubuntu-22.04.3-preinstalled-server-riscv64+unmatched.img riscv-ubuntu2204.img qemu-img resize ubuntu-22.04.3-preinstalled-server-riscv64+unmatched.img +16G Launch the Qemu VM as follows: qemu-system-riscv64 -nographic -machine virt -m 16G -append "root=/dev/vda rw" -drive file=riscv-ubuntu2204.img,if=none,format=raw,id=hd0 -device virtio-blk-device,drive=hd0 -device virtio-net-device,netdev=net0 -netdev user,id=net0 This will boot the RISC-V VM with 16GB of RAM. Step 2: Configure the Python Environment Once the VM is up and running, log in, and set up your RISC-V development environment. You may need to install the necessary dependencies, which may vary depending on the distribution and the version. Most of the software packages that Python program software depends on can be installed by pip. You can run the following command to install pip. apt install python3-pip Before installing other Python packages, install the venv package that can be used to create a Python virtual environment. apt install python3.11-venv Create a Python virtual environment and activate it. cd /root python3 -m venv yolox source /root/yolox/bin/activate Step 3: Install necessary whl packages The Python ecology of the RISC-V architecture is still lacking. We have created build packages to be able to install directly on python3.11. Step 4: Build and Run YOLOX Next, clone the YOLOX repository into your RISC-V qemu git clone https://github.com/Megvii-BaseDetection/YOLOX Navigate to the YOLOX directory and build the YOLOX code. This step may involve installing additional dependencies and configuring the build for RISC-V architecture. cd YOLOX make With YOLOX successfully built, you can now run it on your RISC-V system. You'll need to adapt the YOLOX commands to work with your specific use case and input data. ​Standard models https://github.com/Megvii-BaseDetection/YOLOX#standard-models In this example, yolox_s is downloaded. wget wttps://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_s.pth -P /home/ubuntu/ python3 tools/demo.py image -n yolox-s -c /home/ubuntu/yolox_s.pth --path assets/demo.png --conf 0.25 --nms 0.45 --tsize 640 --save_result --device cpu #Output Logs 2023-09-15 17:05:49.803 | INFO | __main__:main:269 - Model Summary: Params: 8.97M, Gflops: 26.93 2023-09-15 17:05:49.860 | INFO | __main__:main:282 - loading checkpoint 2023-09-15 17:05:53.884 | INFO | __main__:main:286 - loaded checkpoint done. 2023-09-15 17:06:24.598 | INFO | __main__:inference:165 - Infer time: 30.0775s 2023-09-15 17:06:24.708 | INFO | __main__:image_demo:202 - Saving detection result in ./YOLOX_outputs/yolox_s/vis_res/2023_09_15_17_05_53/demo.png We would like to hear from you if this blog was useful to you. Please contact us at info@whileone.in. We would be happy to understand and discuss your requirements and showcase our expertise in a variety of cloud and edge technologies.

We at Whileone Techsoft pvt ltd understood the requirements of our customer who wanted to have a basic Yocto based RiscV deployment for their custom SoC chip. The customer intended to share this basic deployment with their clients who wished to make use of our customer’s SoC in their products.  Our customer was unaware of Yocto and what was needed to ensure a favorable deployment. They had their own custom patched Linux kernel, Root file system, Toolchain and custom Bootloader and a custom simulator as well to boot the final image. Their client insisted on Yocto instead of their default BuildRoot deployment. As Yocto has its own tools, compiler and dependencies, the challenge was to ensure the final Image generated by Yocto was compatible enough to be run by their custom simulator. Introduction to Yocto: With the Opensource Yocto Project, we can create custom Linux based systems for embedded products. It is quite possible to tailor the Linux images as per requirements with a set of flexible tools and friendly customizable scripts. Yocto provides a reference embedded distribution called ‘Poky’ that was used for this project The customer’s custom patched Linux Kernel was of a much smaller version than the current that was available in the kernel.org website. So, initially when we went with the latest Yocto version (Mickledore, v4.2) which featured GCC compiler version 12.x, we got errors during Kernel build. The errors pointed to some unknown assembly instructions. The reason was that our custom kernel version was old and it wasn’t updated. As the Customer was already using GCC 11.x in their build infrastructure, we did a search for a match of Yocto version that provided the nearest GCC 11.x and that was found to be Yocto Honister (v3.4). Initial test build was successful with Honister and so we finalized this version before moving ahead. Yocto uses Bitbake as its build tool. So, whenever we plan to create recipes in Yocto, we should create a separate folder inside poky that starts with “meta-” as per the Yocto manual. Also, referring to similar meta folders like meta, meta-yocto-bsp, meta-poky; we came up with our own “meta-riscv-custom”. To add a new meta layer, make use of bitbake commands, such as the one given below, $> bitbake-layers add-layer meta-riscv-custom Yocto Configuration options: As we were using the sample Poky distribution of Yocto and to let Poky know that we intend to use our custom “meta-riscv-custom” folder in the build process, we have to update a file “bblayers.conf” in the build/conf directory. This build/conf directory is generated after we initialize the environment by executing “source oe-init-build-env” in the Poky root folder. Also, we have to modify the variable “MACHINE” among others in the location “build/conf/local.conf” to “qemuriscv64” and comment out the default value. There are other options in the file “local.conf” that we can modify to get image output in a desired format. The variable IMAGE_FSTYPES = “tar cpio” will generate the image in both tar and cpio formats. This is especially useful when we want to generate a root file system in this format. Creating recipes: Recipes are like script files that are created under the meta- folders. Files like “riscv-linux.bb” which is a recipe for building Linux kernel, “riscv-boot.bb” for building bootloader and so on. Custom changes: The customer was also interested to know how one could add a custom directory and files and make custom changes to existing files in the file system. Yocto has its own package group recipe file “packagegroup-core-boot.bb” that can be modified. For example, 1. We can disable UDEV by commenting it out 2. Similarly, we can also comment out HWCLOCK in the same file To create a custom folder “custom-riscv” inside root (“/”) and a file named “custom.conf” with some configuration options and comments, we had to modify a recipe file “base-files_3.0.14.bb”. Build: Yocto uses Bitbake as its build tool. To build, we make use of the following commands. $> bitbake -cclean riscv-linux $> bitbake riscv-linux The above command skips the extension “.bb”. Also, if we had not added the folder path of “meta-riscv-custom” to bblayers.conf, then we would get an error here after running the above command. Build artifacts: The artifacts are generated in the work directory under the path, Poky/build/tmp/work/riscv64-poky-linux/riscv-linux/1.0-r0/custom-linux/*

In May 2022, one of our customers asked us to tune Elasticsearch with Esrally for cloud providers. We started with trying multiple combinations of manual runs on all cloud providers. We were collecting scaling runs with 2/4/8/16 cores. In the above data collection, we could not see the proportionate scores. Hence, we decided to experiment with running the Elasticsearch ESRally benchmark throughout the day. As Esrally doesn’t run for a particular duration, we carried out the runs 50 times so that it will span a whole day. And here is what we saw! Used configurations are: Altra - t2a-standard-16 Intel Icelake - n2d-standard-16 Milan - c2d-standard-16 Elasticsearch 8.4.1 Esrally 2.6.0 Server     Altra.    Intel Icelake.    Milan.  Client   Altra   Altra   Altra Variation is observed according to time of day. AMD is the best where SD is lowest. But Intel and Altra show large standard deviations. NGINX-wrk benchmark also shows such behaviour on GCP. NGINX- wrk runs are carried out 1440 times keeping each run ‘s duration 60 seconds. Variation in p95 latency is observed through the time of day. Both Intel and Altra show 10% standard deviation in p95 latency numbers.   Do consider Time-based Score Variations before running network applications: Time of day does affect latency since neighboring VMs might be busy or idle depending on the time of day.  Run-to-run variation is a function of time of day. Eventually, we were able to help the customer figure out where the performance difference was coming from. To ensure a specific output through the day the scaling of VM has been suggested.

At Whileone we take pride in customer's success. We help customers achieve goals and execute out of the box ideas that are necessary for success. One such project was to get IPCs for cloud applications on different architectures, completely omitting network stack. This would give the RISC-V chip designing customer a good picture whether architecture IPC ( Instructions per Cycle ) is inline with competition like Intel or ARM.  To achieve this, we modified cloud applications to profile and benchmark the performance with no network or socket calls. The idea here was to see performance of different architectures with vanilla versions and lite ( modified ) versions. This would help the customer run these applications on their simulator. This would help them to get the IPC number of that architecture for that application and compare it with the competition. To give you an example, one of the applications we picked was Redis -a cached server application. Redis takes SET/GET requests from clients and processes those internally to keep cached copy for quick response. To get away with the network part, we simulated the client and to look like Redis has N SET/GET requests and processed those. Now the performance numbers we have are solely for that application processing on that architecture. This helped eliminate network noise and get a good picture of what the IPC is for core application processing. Table below shows the IPC Redis vs RedisLite. Drop in IPC can be attributed to networking sockets being removed. SET Redis Redis-Lite Graviton2 Intel 8275 Cascade Lake Graviton2 IPC 0.94 0.69 1.76 Icount / packet ~39000 ~30000 ~20200 In doing so, we made sure that we do not modify program logic and core behavior of the application in any way. We could see a similar call stack in case of Redis and Redis Lite. Below are the snapshots. REDIS Flamegraph REDIS-LITE Flamegraph As evident from the flame graphs- the call stack of the core application is not altered. In the Redis-lite flamegraph, the network component is absent.  Redis is a single threaded application. We helped the customer port various multi-threaded / multi-process applications. The customer was able to cross-compile these application and run it on the it’s RISC-V simulator. This was an interesting experiment from the performance numbers point of view and useful for the customer in the early phase of chip development. This helped the customer to understand where they are placed with respect to the competition.

Basic Linear Algebra Subprograms(BLAS) and BLAS Like Interface Software(BLIS) are libraries that can accelerate mathematical operations on current CPU microarchitectures. As a part of the FLAME project, BLIS was introduced to handle the dense linear algebra software stack. The framework was designed to isolate essential kernels of computation that, when optimized, immediately enable optimized implementations of most of its commonly used and computationally intensive operations. BLIS offers enhanced performance for cases of matrix multiplications where the operands are small. BLIS supports both, single and multi-threaded modes of operations. Oracle in its efforts has optimized the BLIS libraries for exceptional performance on Ampere Altra Family of processors. Let us look at how we can leverage this to our benefit and what sort of performance boost can be expected. Step 1: Getting BLIS Sources git clone https://github.com/flame/blis.git cd blis git checkout ampere​ Step 2: Building BLIS ./QuickStart.sh altramax #Change ./QuickStart.sh altramax to ./QuickStart.sh altra if building for Ampere Altra #processors source ./blis_build_altramax.sh source blis_setenv.sh export LD_LIBRARY_PATH=/lib/altramax Note: BLIS can be built for OpenMP(default) or pthreads. Details can be found in documentation/tutorial. Step 3: Performance Experiments ​ For our test, we will be using HPL- 2.3, a High Performance Linpack benchmark that is commonly used to test systems. We will be comparing performance of Oracle BLIS library with OpenBLAS and Arm-PL System Config: OS: Ubuntu 22.04 Kernel: 5.19.0-46-generic Toolchain: gcc (GCC) 12.3.0 Memory: 16x32GB Results: For HPL, the Oracle optimized BLIS libraries provide 1.2 times boost in performance.

We had a requirement to build a charting facility that could provide several charts. The requirement implied that we needed to support several different charts, but those charts weren’t really defined, and flexibility was required. Our setup was a headless CMS ( Strapi.io ) and NextJS for server-side-rendered and statically generated pages. We found react-google-charts to be an interesting library. For any chart, this library requires inputs as chart-type, data, width and height. Our workaround to deliver this overnight was to add a JSON field in CMS for accepting these parameters and on the frontend, pass these to the react-google-charts. Implementing it in this way implied we could support 100% of what the react-google-charts supports. Given that most content managing stakeholders weren’t from a web development background, we documented and trained them on how they could leverage this feature. This again was made extremely simple by react-google-charts live examples . Using any one example which suited the requirements, the content managers could view the chart live, then open the sandbox, tweak data as required, then create a JSON from the data and chart-type. Initial few charts did take development time to figure out things like tweaking the colors, having two y-axes with values in different units and modifying the content of the hover bubble. However, after initial 3-4 charts, such additional things were repetitive and the content managers could easily figure things out on their own by referring to these initial 3-4 charts and using the sandboxes that react-google-charts provide.