Embedded Systems Design Europe - August/September 2008 - (Page 21)

benchmarks VMs in turn can host Java VMs or other operating systems such as Linux, a real-time operating systems from commercial vendors, or even a home-grown scheduler or thin executive. The key for the embedded systems market is that the hypervisor allows different operating systems to run simultaneously and in isolation from one another on a single common device. While a powerful enabler of a wide variety of technologies in single-core applications, the technique has particular strength when applied to multicore devices as it allows a simple mapping of system resources and permits existing legacy code to run on one processor and operating system while new applications can be written for the new, more complex, environment. This ﬂexibility also enables developers to make use of new, more power-efﬁcient hardware and still reuse their existing tried and tested code. BENCHMARKING HYPERVISORS To ensure a level playing ﬁeld for the benchmarking process, all the benchmark suites and kernels must be executed in the CPU’s unprivileged execution environment so that any required system calls must be made through the hypervisor’s API. This is required to meet the basic security deﬁnition of a hypervisor. All the benchmark kernels that are used as the workloads must be executed in isolated virtual machines with the scheduler alternating execution among the virtual machines at least every 10ms. This is required to measure the impact of the VM switching time. But there are no restrictions on the CPU implementation. It can be singlecore, multicore, multithreaded, or any combination. What is necessary is that the CPU must have at least two privileged modes of operation to provide isolation of virtual machines from the hypervisor. This also means the CPU must have a MMU or MPU to enable VM-to-VM isolation, which limits the range of single processors that can be assessed. This is not an assessment of the processor, the multicore implementation, or the communications links themselves, all of which can have an impact on the performance of the software. Rather, the benchmarks are designed to test the implementation and performance of a particular hypervisor on a particular processor or multiprocessor device. The aim of the benchmarks is to provide comparisons on the same processor and in the same software environment and to conﬁne themselves to demonstrating the overhead associated with the hypervisor implementation. EEMBC’s testing environment uses three different scenarios to test the systems in different ways and ensure that the assessment is reasonable across a wide range of applications, from the basic task running without the hypervisor to several virtual machines running. These results are then combined to provide the overall score. Scenario #1 runs each of the workloads sequentially, to be measured N times without the hypervisor and stores total execution time as RESULT#1 and the power consumption as ENERGY#1. In scenario #2, each of the workloads runs sequentially N times in one virtual machine with the hypervisor, giving a total execution time as RESULT#2 and power consumption as ENERGY#2. Scenario #3 sees three separate virtual machines each running the workloads N times with all other parameters for the workloads the same as in scenarios #1 and #2, providing RESULT#3 and ENERGY#3. The benchmark is calculated from the difference in execution time between Scenario 1 and 2 as the percent overhead for a single virtual machine, and the difference in execution time between Scenario 1 and 3 as the percent overhead for multiple virtual machines. It’s vital to stress that the EEMBC hypervisor benchmark scores can only be compared for hypervisors running on identical systems, from exactly the same CPU and board with the same clock frequencies, cache/memory conﬁguration and speeds, to the same compilers, linkers and libraries, and, of course, the identical versions of the software. These scores can also only be compared with the same number of iterations, types, and order of workloads, as initial testing has shown that variations can make signiﬁcant differences.Some of the early evaluations have revealed surprising results. The implementation of the hypervisor makes a signiﬁcant difference to the workload’s operation and the system’s efﬁciency. With some hypervisors, the number of workloads that a hypervisor can support can plateau quite rapidly, indicating that adding more processes is not cost or power effective. MOVING FORWARD Several key issues exist for hypervisors running on multicore devices, including load balancing (or rather dynamic resource allocation, shifting the resources between cores while in operation), debugging the operation across multiple cores with synchronized break points, and the way the cores themselves and the processes communicate. This set of EEMBC’s benchmarks does not speciﬁcally address these issues, and the operation of multiple work loads across multiple cores doesn’t allow for dynamic changes in resources or look at the communications, nor does it show how easy these systems are to work with. Eventually benchmarking will cover the interactions between multiple cores on a single chip and document how easy a hypervisor is to implement and use; the need for this data is inspiring systems developers to push for standards at the software level that allow the same hypervisor implementation to be used across different processor architectures. Once achieved, this ﬂexibility will further drive the reuse of applications across new devices and provide higher performance and lower power consumption for the next generations of embedded systems. Markus Levy is founder and president of EEMBC. He is also president of The Multicore Association and chairman of Multicore Expo. You may reach him through www.eembc.org. 21 www.embedded.com/europe | embedded systems design europe | AUGUST – SEPTEMBER 2008 020-021_EETE.indd 21 28/08/08 11:49:48 http://www.eembc.org http://www.embedded.com/europe

Table of Contents Feed for the Digital Edition of Embedded Systems Design Europe - August/September 2008

Embedded Systems Design Europe - August/September 2008 Contents TI Overhauls DSP Lineup, Adds 15 Processors QNX Publishes Source Code for File System Congatec to Take on Proprietary Market Swiss Multicore Project Wins Microsoft Grant OpenCores Bundles Development Tool ARM Compiler Boosts Freescale i.MX31 LabVIEW Updated for Multicore and Wireless Cover Feature: Interactive Tool Supports Multiprocessor SoC Design Wanted: Benchmaking for Embedded VMM Hypervisors Graphical Design Empowers Spider Robots Building a Power Supply for Discontinuous Transmission Wireless Networks RTOS Selection & Best Practices Achieving Cache Coherence in a MIPS32 Multicore Design New Products Advertising Contacts

Embedded Systems Design Europe - August/September 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.