이번에, 알고 있던, 다양한 벤치마크 툴 들을 연구실에 소개하려 자료를 만들다 보니

문득, 블로그 에도 올려야겠다는 생각이 들었다.

우선, 내가 써본건 SPECcpu2006 과 PARSEC 이고, 

사용해 보진 않았지만, 논문에서 자주 본 것은 lmbench 이다.

그리고, SPLASH-2 도 써봤는데, 이건 이젠 PARSEC에 내장되어 있다.

BBench 는 좀 성격이 다른 친군데, 좀 더 실제 스마트폰 사용환경에 유사하다고 주장하니 써봐야겠다...

-논문에 따르면 SPEC 과 비교했을 때, 좀 더 실제 스마트폰 환경과 유사한 경향성을 보여줌을 보였다. 이를 실제 환경에서는 VM이슈 등으로 Cache 등 Hit ratio 가 더 나쁘다 말함.

각 벤치마크 도구마다 특징이 있는데 간략하게 말하자면

(이하 내용은 작성하던 PPT 복붙이다. 나는 PPT 작성 시, 영어 공부도 할 겸 주로 영어로 작성한다. 물론 영어는 죽도록 못하지만...)

•PARSEC Benchmark Suite

‐The Princeton Application Repository for Shared-MEmory Computers (PARSEC) 

‐Benchmark for parallel shared memory systems

‐System researchers(In many cases memory) use it

•Key Features

‐Multithreaded

‐Emerging Workloads

‐Diverse

‐Not HPC-Focused

•Used paper

‐About 353 papers. (2016 ~ Current)

>Search for the number of articles citing "The PARSEC benchmark suite: Characterization and architectural implications"

•SPLASH-2

‐Includes applications and kernels mostly in the area of high performance computing (HPC)

‐It has been widely used to evaluate multiprocessors and their designs

‐Benchmark for parallel shared memory systems

‐Recent studies show that SPLASH-2 and PARSEC benchmark suites complement each other

‐SPLASH-2 into the PARSEC environment in current release

‐System researchers(In many cases HPC) use it

•Key Features

‐Multithreaded

‐Memory overhead

•Used paper

‐About 246 papers. (2016 ~ Current)

>Search for the number of articles citing "The SPLASH-2 programs: Characterization and methodological considerations"

•SPEC CPU Benchmark Suites

‐Standard Performance Evaluation Corporation CPU
(Current: SPEC CPU 2006)

‐In particular their SPECint and SPECfp

>SPECint : Benchmark specification for CPU integer processing power

>SPECfp : Benchmark specification for CPU floating point processing power

‐As siblings, SPECcloud, SPECapc, SPECpower and ETC...

‐Least version(2006) is to old, but still valid

‐800$ (Only SPEC CPU not allows it’s siblings)

‐System researchers(In many cases HPC) use it

•Key Features

‐CPU

‐RAM

‐Compiler

‐Does not test I/O, networking, or graphics

•Used paper

‐About 186 papers. (2016 ~ Current)

>Search for the number of articles citing "SPEC CPU2006 benchmark descriptions"

•lmbench

‐micro-benchmark suite designed to focus attention on the basic building blocks
of many common system applicationsAs siblings, SPECcloud, SPECapc, SPECpower and ETC...

‐Least version(2.5, 2006) is to old, but still valid

‐System researchers(In many cases Small or Specify systems) use it

•Key Features

‐Bandwidth benchmarks

>Cached file read, Memory copy (bcopy), Memory read, Memory write, Pipe, TCP

‐Latency benchmarks

>Context switching

>Networking: connection establishment, pipe, TCP, UDP, and RPC hot potato

>File system creates and deletes.

>Process creation.

>Signal handling, System call overhead

>Memory read latency

‐Miscellanious

>Processor clock rate calculation

•Used paper

‐About 40 papers. (2016 ~ Current)

>Search for the number of articles citing "lmbench: Portable Tools for Performance Analysis"