PhD course

Software Development Measurement Programs: from theory to practice: PhD Course, 10 HP/ECTS

Introduction

One part of software engineering research and practice is based on using quantitative data for decision making and assessments. Software engineers work with metrics to understand the complexity of their software products, which they can use to predict various quality attributes (e.g. maintainability, portability). Software engineering researchers use metrics to assess the outcome of their research activities, checking whether the introduced change in a software process leads to an improvement or deterioration of process efficiency, software quality and other aspects.

In this course, you will learn how to work with software metrics and metrology in the context of software development measurement programs. We will start with the theories needed for correct software measurement. Then we will gradually move towards the practice of software measurement (measurement systems), organization of measurement (measurement programs) and the ability to use metrics for decision making (machine learning).

Content of the course and assignments

The course is organized in modules, where we combine reading, discussions and small assignments. In order to pass the course, you need to pass at least 60% of all assignments. The grade for the course is either pass or fail, there are no partial grades and no partial points.

Module 1: Introduction to measurement and measurement systems

In this module we go through the theory of measurement and the ISO 15939 standard for measurement systems. The goal is to introduce the principles of measurement and understand the “why” behind the measurement systems and metrology in software engineering. We learn the foundations needed to design measures and data collection mechanisms for such applications like machine learning, mining software repositories, software analytics and data-driven development.

Assignment: design a measurement system for defect backlog based on a chosen open source software.

Module 2: Data collection and quantification

In this module we discuss methods and tools for data collection – metric tools, scripts for mining software repositories and repositories of software data. The goal of this module is to learn how to take the design of a measure/metric from the design to the implementation domain. We learn both about the technical aspects of data collection and about the ethical aspects of it.

Assignment: design and implement a system for collecting data from a chosen open source repository.

Module 3: Data consistency and quality

In this module we learn how to assess quality of data sets. We start with simple methods for assessing data consistency and then we move to other quality attributes, e.g. timeliness. We use AIMQ and other models of data quality, including ISO 25024, as the theoretical background. The goal is to learn how to check whether our data is good-enough for using it in making decisions.

Assignment: design a script for checking the quality of data extracted in module 1 or 2.

Module 4: Machine learning techniques

In this module we move from discussing data to discussing applications of it. We start with using our data to work with machine learning. Here, we learn some basic ML algorithms – decision trees and k-Nearest Neighbours. We introduce these techniques from the perspective of data analysis, not mathematical foundations.

Assignment: design and develop a machine learning recommender system based on the data from a chosen open source repository.

Module 5: Dashboards

Another application of data and measurement is to support decision-making. Therefore, in this module we introduce techniques used to design actionable dashboards. We learn about different kinds of dashboards and how they communicate message from the data in an effective way. We also learn how the presentation of the data influences the message. The goal of this module is to learn what is important when turning data into decision support.

Assignment: design and develop a dashboard based on the data from the recommender system.

Module 6: Introducing measurement systems into organizations – measurement programs

In the last module, we learn about the organizational aspects of measurement – measurement programs. We learn about how a measurement program is constructed and which elements of the measurement programs exist. The goal is to is learn how a measurement program is organized and which roles are important for the program.

Assignment: present a selected paper about a measurement program.