What makes a benchmark suite

On Distributed Verification and Verified Distribution. PhD thesis, defended on Nov. Springer Verlag, April Springer Verlag, Mar. Jan Friso Groote and Frank van Ham. Interactive Visualization of Large State Spaces.

Electronic Notes in Theoretical Computer Science 1. ACM Press. Francesco Ranzato and Francesco Tapparo. Information and Computation, vol. Katia Hristova , Yanhong A. Liu , and K. Tuncay Tekle. Radu Mateescu and Anton Wijs. Weisong Li. Algorithms for Computing Weak Bisimulation Equivalence.

Journal version: Theoretical Computer Science, vol. Johan Oudinet. Jakub Chaloupka. Algorithms for Mean-Payoff and Energy Games. Distributed algorithms for SCC Decomposition. Journal of Logic and Computation, vol. Oxford University Press, Marie-Claude Gaudel. Counting for Random Testing. Science of Computer Programming, vol. Software Tools for Technology Transfer, vol. Interactive specification and verification of behavioral adaptation contracts.

Information and Software Technology, vol. Rafal Somla. Logics and Algorithms for Verification of Concurrent Systems. Current benchmark suites cover only a small range of mobile applications, and many cannot run directly in simulators due to their user interaction requirements.

In this paper, we introduce and characterize Moby, a benchmark suite designed to make it easier to use full-system architectural simulators to evaluate microarchitectures for mobile processors. Moby contains popular Android applications, including a web browser, a social networking application, an email client, a music player, a video player, a document processing application, and a map program.

To facilitate microarchitectural exploration, we port the Moby benchmark suite to the popular gem5 simulator. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful. Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.

If you disable this cookie, we will not be able to save your preferences. In some cases, the obvious approach works pretty well: for example, many layout and rendering optimizations can be driven by measuring page load time on representative web pages. But measuring the performance of a programming language implementation requires more subtlety. We want to minimize the opportunities for system noise to throw off our measurements, but anytime a workload is inherently prone to noise, we want a benchmark to show this.

We want our benchmarks to represent a high-fidelity approximation of the workloads that WebKit users are likely to care about. JetStream combines a variety of JavaScript benchmarks, covering a variety of advanced workloads and programming techniques, and reports a single score that balances them using a geometric mean. Each benchmark measures a distinct workload, and no single optimization technique is sufficient to speed up all benchmarks. Some benchmarks demonstrate tradeoffs, and aggressive or specialized optimization for one benchmark might make another benchmark slower.

Demonstrating trade-offs is crucial for our work.