이번에, 알고 있던, 다양한 벤치마크 툴 들을 연구실에 소개하려 자료를 만들다 보니
문득, 블로그 에도 올려야겠다는 생각이 들었다.
우선, 내가 써본건 SPECcpu2006 과 PARSEC 이고,
사용해 보진 않았지만, 논문에서 자주 본 것은 lmbench 이다.
그리고, SPLASH-2 도 써봤는데, 이건 이젠 PARSEC에 내장되어 있다.
BBench 는 좀 성격이 다른 친군데, 좀 더 실제 스마트폰 사용환경에 유사하다고 주장하니 써봐야겠다...
-논문에 따르면 SPEC 과 비교했을 때, 좀 더 실제 스마트폰 환경과 유사한 경향성을 보여줌을 보였다. 이를 실제 환경에서는 VM이슈 등으로 Cache 등 Hit ratio 가 더 나쁘다 말함.
각 벤치마크 도구마다 특징이 있는데 간략하게 말하자면
(이하 내용은 작성하던 PPT 복붙이다. 나는 PPT 작성 시, 영어 공부도 할 겸 주로 영어로 작성한다. 물론 영어는 죽도록 못하지만...)
•PARSEC Benchmark Suite
‐The Princeton
Application
Repository
for Shared-MEmory Computers
(PARSEC)
‐Benchmark
for
parallel shared memory systems
‐System
researchers(In many cases memory) use
it
•Key Features
‐Multithreaded
‐Emerging
Workloads
‐Diverse
‐Not
HPC-Focused
•Used paper
‐About
353 papers. (2016 ~ Current)
>Search
for the number of articles citing "The PARSEC benchmark suite:
Characterization and architectural implications"
•SPLASH-2
‐Includes
applications
and kernels mostly in the area of high performance computing (HPC)
‐It
has been widely used to evaluate multiprocessors and their designs
‐Benchmark
for
parallel shared memory systems
‐Recent
studies show that SPLASH-2 and PARSEC benchmark suites complement each other
‐SPLASH-2
into the PARSEC environment in current release
‐System
researchers(In
many cases HPC)
use
it
•Key Features
‐Multithreaded
‐Memory
overhead
•Used paper
‐About
246
papers.
(2016 ~ Current)
>Search
for the number of articles citing "The SPLASH-2 programs:
Characterization and methodological considerations"
•SPEC CPU Benchmark Suites
‐Standard
Performance
Evaluation
Corporation
CPU
(Current: SPEC CPU 2006)
‐In particular
their SPECint and SPECfp
>SPECint : Benchmark
specification
for CPU integer processing power
>SPECfp : Benchmark
specification for CPU floating
point processing
power
‐As
siblings, SPECcloud,
SPECapc,
SPECpower
and ETC...
‐Least
version(2006)
is
to old, but still valid
‐800$
(Only SPEC CPU not allows it’s siblings)
‐System
researchers(In many cases HPC) use it
•Key Features
‐CPU
‐RAM
‐Compiler
‐Does
not
test I/O,
networking, or graphics
•Used paper
‐About
186 papers.
(2016 ~ Current)
>Search for the number of articles
citing "SPEC CPU2006 benchmark descriptions"
•lmbench
‐micro-benchmark
suite designed to focus attention on the basic building blocks
of many
common system applicationsAs
siblings, SPECcloud,
SPECapc,
SPECpower
and ETC...
‐Least
version(2.5,
2006) is
to old, but still valid
‐System
researchers(In
many cases Small
or Specify systems) use it
•Key Features
‐Bandwidth
benchmarks
>Cached
file read,
Memory copy
(bcopy),
Memory read, Memory write, Pipe, TCP
‐Latency
benchmarks
>Context
switching
>Networking:
connection establishment, pipe, TCP, UDP, and RPC hot potato
>File system
creates and deletes.
>Process
creation.
>Signal
handling, System call
overhead
>Memory
read latency
‐Miscellanious
>Processor
clock rate calculation
•Used paper
‐About
40 papers.
(2016 ~ Current)
>Search for the number of articles
citing "lmbench:
Portable Tools for Performance Analysis"
•BBench
‐Targeting
smartphone, so made by web(HTML5 + JS)
‐Measure
a suite of widely available mobile applications for audio, video, and
interactive gaming
>benchmark
to
assess a web-browser's performance when rendering some of the most popular and
complex sites
‐suffer
performance penalties incurred due to
the use of high level software
abstractions on current mobile CPUs
•Key Features
‐Similar
to the actual smartphone environment
‐Display
rendering
•Used paper
‐About
32
papers.
(2016 ~ Current)
>Search
for the number of articles citing "Full-system analysis and
characterization of interactive smartphone applications"