An evaluation of open-source software microbenchmark suites for continuous performance assessment
作者: Christoph LaaberPhilipp Leitner
作者单位: University of Zurich, Zurich, Switzerland;;University of Gothenburg, Gothenburg, Sweden
论文集英文名称: Mining Software Repositories
来源数据库: Association for Computing Machinery
DOI: 10.1145/3196398.3196407
关键词: Large scale compilationRunnable software repositoriesSoftware mining
原始语种摘要: Continuous integration (CI) emphasizes quick feedback to developers. This is at odds with current practice of performance testing, which predominantely focuses on long-running tests against entire systems in production-like environments. Alternatively, software microbenchmarking attempts to establish a performance baseline for small code fragments in short time. This paper investigates the quality of microbenchmark suites with a focus on suitability to deliver quick performance feedback and CI integration. We study ten open-source libraries written in Java and Go with benchmark suite sizes ranging from 16 to 983 tests, and runtimes between 11 minutes and 8.75 hours. We show that our study subjects include benchmarks with result variability of 50% or higher, indicating that not all...
全文获取路径: ACM 

  • software 软件
  • benchmark 基准点
  • performance 性能
  • integration 集成
  • compilation 编绘
  • discovery 发现
  • API Accurate Position Indicator
  • mining 矿业
  • generate 
  • source