Cloud service benchmarking can provide important sometimes surprising insights into the
quality of services and leads to a more quality-driven design and engineering of complex
software architectures that use such services. Starting with a broad introduction to the field
this book guides readers step-by-step through the process of designing implementing and
executing a cloud service benchmark as well as understanding and dealing with its results. It
covers all aspects of cloud service benchmarking i.e. both benchmarking the cloud and
benchmarking in the cloud at a basic level.The book is divided into five parts: Part I
discusses what cloud benchmarking is provides an overview of cloud services and their key
properties and describes the notion of a cloud system and cloud-service quality. It also
addresses the benchmarking lifecycle and the motivations behind running benchmarks in
particular phases of an application lifecycle. Part II then focuses on benchmark design by
discussing key objectives (e.g. repeatability fairness or understandability) and defining
metrics and measurement methods and by giving advice on developing own measurement methods and
metrics. Next Part III explores benchmark execution and implementation challenges and
objectives as well as aspects like runtime monitoring and result collection. Subsequently Part
IV addresses benchmark results covering topics such as an abstract process for turning data
into insights data preprocessing and basic data analysis methods. Lastly Part V concludes
the book with a summary suggestions for further reading and pointers to benchmarking tools
available on the Web.The book is intended for researchers and graduate students of computer
science and related subjects looking for an introduction to benchmarking cloud services but
also for industry practitioners who are interested in evaluating the quality of cloud services
or who want to assess key qualities of their own implementations through cloud-based
experiments.
