DiffDetective

KO | EN

gitlite — search

#hacktoberfest #ai #typescript #go #android #machine-learning #deep-learning #linux #macos #windows #kubernetes #vue #mcp #open-source

(★ 9)

Library for Variability-Aware Differencing and the Analysis of Edits to Preprocessor-Based Software Product Lines

DiffDetective Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

![Maven](https://github.com/VariantSync/DiffDetective/actions/workflows/maven.yml/badge.svg)
[![Documentation](https://img.shields.io/badge/Documentation-Read-purple)][documentation]
[![GitHubPages](https://img.shields.io/badge/GitHub%20Pages-online-blue.svg?style=flat)][website]
[![License](https://img.shields.io/badge/License-GNU%20LGPLv3-blue)](https://raw.githubusercontent.com/VariantSync/DiffDetective/main/LICENSE.LGPL3)
[![Demo Repository](https://img.shields.io/badge/Demo-Try-blue)][demo]

# DiffDetective - Variability-Aware Source Code Differencing

DiffDetective is an open-source Java library for variability-aware source code differencing and the **analysis of version histories of software product lines**. This means that DiffDetective can **turn a generic differencer into a variability-aware differencer** by means of a pre- or post-processing. DiffDetective is centered around **formally verified** data structures for variability (variation trees) and variability-aware diffs (variation diffs). These data structures are **generic**, and DiffDetective currently implements **C preprocessor support** to parse respective annotations when used to implement variability. The picture below depicts the process of variability-aware differencing.

Given two states of a C-preprocessor annotated source code file (left), for example before and after a commit, DiffDetective constructs a variability-aware diff (right) that distinguishes changes to source code from changes to variability annotations. DiffDetective can construct such a variation diff either, by first using a generic differencer, and separating the information (center path), or by first parsing both input versions to an abstract representation, a variation tree (center top and bottom), and constructing a variation diff using a tree differencing algorithm in a second step.

Additionally, DiffDetective offers a **flexible framework for large-scale empirical analyses of git version histories** of statically configurable software. In multiple studies, DiffDetective was successfully employed to study the commit histories of up to 44 open-source git repositories, including the **Linux Kernel, GCC, Vim, Emacs, or the Godot game engine**.

## Setup

DiffDetective is a Java Maven library. While DiffDetective depends on some custom libraries ([FeatureIDE library](https://featureide.github.io/), [Sat4j](https://sat4j.org/), [Functjonal](https://github.com/VariantSync/Functjonal)) these are prepackaged with DiffDetective. So **all you need is Java ≥16 and [Maven](https://maven.apache.org/) or alternatively [Nix](https://nixos.org/)**. In the following, we explain the setup with Java and Maven, as well as via Nix.

### Cloning the Repository

Clone this repository and navigate inside it:
```shell
git clone https://github.com/VariantSync/DiffDetective
cd DiffDetective
```

> In case you are using Nix Flakes, you may skip cloning the repository.

### Building and Installing

You can build and install DiffDetective with Maven such that it can be used in your own project. Alternatively, you can use a jar which includes all necessary dependencies. Such a jar can either be built manually using Maven or using Nix.

#### Building and Installing With Maven

First, Maven needs to be installed. Either provide it yourself (e.g., using a system package manager or on Windows, download [from their website](https://maven.apache.org/guides/getting-started/windows-prerequisites.html)) or, if you have Nix installed, run `nix-shell` (stable Nix) or `nix develop` (Nix Flakes) to provide all necessary build tools.

Next, build DiffDetective and install it on your system so that you can access it from your own projects:
```shell
mvn install
```

To add DiffDetective as a dependency to your own project, add the following snippet to the pom.xml of your Maven project, but make sure to pick the right version number. The current version number can be obtained by running `scripts/version.sh`

```xml
<dependency>
    <groupId>org.variantsync</groupId>
    <artifactId>DiffDetective</artifactId>
    <version>2.3.0</version>
</dependency>
```

If you prefer to just use a jar file, you can find a jar file with all dependencies at `DiffDetective/target/diffdetective-2.3.0-jar-with-dependencies.jar` (again, the version number might be different).
You can (re-)produce this jar file by either running `mvn package` or `mvn install` within you local clone of DiffDetective.

> Disclaimer: Setup tested with maven version 3.6.3.

#### Building with Nix

Alternatively to manually building using Maven, Nix can be used. Both a [flake.nix](flake.nix) and a [default.nix](default.nix) are provided. Hence, you can build DiffDetective using
```shell
nix-build # stable version
# or
nix build # Flake version
```
In case you are using Nix Flakes, you can skip cloning the repository as usual: `nix build github:VariantSync/DiffDetective#.`

Afterward, the [result](result) symlink points to the [Javadoc](result/share/github-pages/DiffDetective/docs/javadoc/index.html), a [DiffDetective jar for use as a library](result/share/java/DiffDetective.jar), a [self contained DiffDetective jar with all dependencies included](result/share/java/DiffDetective/DiffDetective-jar-with-dependencies.jar) and a simple [script](result/bin/DiffDetective) for executing a DiffDetective main class provided as argument (e.g., evaluations used in previous research, see below under 'Publications').

## How to Get Started

For a demonstration on how to get started using the library, we have prepared a demo repository [here][demo].
You may clone it as a template and example for including the library into your own projects.
Additionally, there is a screencast available on YouTube, guiding you through the demo's setup and source code:

[![DiffDetective Demonstration](https://raw.githubusercontent.com/VariantSync/DiffDetective/main/docs/yt_thumbnail.png)](https://www.youtube.com/watch?v=q6ight5EDQY)

## Supported Differencing Algorithms

In principle, any generic differencing algorithm (i.e, any algorithm that may operate on text or trees) can be made variability-aware with DiffDetective, as explained in our demo paper (see below). Some algorithms are integrated directly in the DiffDetective library, while others come as additional Maven projects.

### Shipped with DiffDetective
- Git Diff as implemented by [JGit](https://github.com/eclipse-jgit/jgit)
- [GumTree](https://github.com/GumTreeDiff/gumtree), and all algorithms and matching engines supported by the GumTree library

### Extra Modules
- [TrueDiff](https://gitlab.rlp.net/plmz/truediff): Support for TrueDiff comes as [a separate Maven project](https://github.com/VariantSync/TrueDiffDetective).

## Publications

### Variability-Aware Differencing with DiffDetective (FSE 2024, ⭐ [Best Demo Paper](https://2024.esec-fse.org/info/awards) ⭐)

[![Preprint](https://img.shields.io/badge/Preprint-Read-purple)](https://github.com/SoftVarE-Group/Papers/raw/main/2024/2024-FSE-Bittner.pdf)
[![Paper](https://img.shields.io/badge/Paper-Read-purple)](https://dl.acm.org/doi/10.1145/3663529.3663813)
[![Slides](https://img.shields.io/badge/Slides-Here-purple)](https://github.com/SoftVarE-Group/Slides/raw/main/2024/2024-07-17-FSE-DiffDetective.pdf)
[![Screencast](https://img.shields.io/badge/Screencast-Watch-purple)][screencast]
[![Demo Repository](https://img.shields.io/badge/Demo-Try-blue)][demo]

> P. M. Bittner, A. Schultheiß, B. Moosherr, T. Kehrer, T. Thüm. _Variability-Aware Differencing with DiffDetective_. Demonstrations at International Conference on the Foundations of Software Engineering 2024, ACM, New York, NY, July 2024

This paper gives an overview of DiffDetective, its design, features, use-cases, and past case studies. We recommend reading this paper if you are interested in the design of DiffDetective or if you consider using it for your own projects or research. The paper is accompanied by a [demo project][demo] as well as a [screencast][screencast] (also see `How to Get Started` above).

### Classifying Edits to Variability in Source Code (ESEC/FSE 2022)

[![Preprint](https://img.shields.io/badge/Preprint-Read-purple)](https://github.com/SoftVarE-Group/Papers/raw/main/2022/2022-ESECFSE-Bittner.pdf)
[![Paper](https://img.shields.io/badge/Paper-Read-purple)](https://dl.acm.org/doi/10.1145/3540250.3549108)
[![Appendix](https://img.shields.io/badge/Appendix-Read-purple)](https://github.com/VariantSync/DiffDetective/raw/main/appendix/appendix-esecfse22.pdf)
[![Talk](https://img.shields.io/badge/Talk-Watch-purple)](https://www.youtube.com/watch?v=EnDx1AWxD24)
[![Slides](https://img.shields.io/badge/Slides-Here-purple)](https://github.com/SoftVarE-Group/Slides/raw/main/2022/2022-11-14-ESECFSE-ClassifyingEdits.pdf)
[![Original Replication Package](https://img.shields.io/badge/Replication_Package-Original-blue)](https://github.com/VariantSync/DiffDetective/tree/esecfse22)
[![Updated Replication Package](https://img.shields.io/badge/Replication_Package-Updated-blue)](https://raw.githubusercontent.com/VariantSync/DiffDetective/main/replication/esecfse22/)
[![Artifact DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7110095.svg)](https://doi.org/10.5281/zenodo.7110095)

> P. M. Bittner, C.Tinnes, A. Schultheiß, S. Viegener, T. Kehrer, T. Thüm. _Classifying Edits to Variability in Source Code_. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2022), ACM, New York, NY, November 2022

This was the initial work, introducing DiffDetective as a means to conduct an empirical evaluation of a classification of edits.
In particular, we used DiffDetective to classify the effect of edits on the variability of the edited source code in the change histories of 44 open-source C-preprocessor-based software projects.

The classification is implemented within the [org.variantsync.diffdetective.editclass](src/main/java/org/variantsync/diffdetective/editclass/) package.
The empirical evaluation of the classification, including a respective `main` method, is implemented in the [org.variantsync.diffdetective.experiments.esecfse22](src/main/java/org/variantsync/diffdetective/experiments/esecfse22) package.

The original replication package can be found on the [esecfse](https://github.com/VariantSync/DiffDetective/tree/esecfse22) branch or via the DOI [10.5281/zenodo.7110095](https://doi.org/10.5281/zenodo.7110095). The replication is also available for the most recent version of DiffDetective with various improvements, which will likely yield to slightly different results than the initial study. The updated replication package can be found in the [replication/esecfse22](replication/esecfse22) subdirectory with its own [README](replication/esecfse22/README.md).

### Views on Edits to Variational Software (SPLC 2023)

[![Preprint](https://img.shields.io/badge/Preprint-Read-purple)](https://github.com/SoftVarE-Group/Papers/raw/main/2023/2023-SPLC-Bittner.pdf)
[![Paper](https://img.shields.io/badge/Paper-Read-purple)](https://dl.acm.org/doi/10.1145/3579027.3608985)
[![Appendix](https://img.shields.io/badge/Appendix-Read-purple)](https://github.com/VariantSync/DiffDetective/raw/main/appendix/appendix-splc23-views.pdf)
[![Slides](https://img.shields.io/badge/Slides-Here-purple)](https://github.com/SoftVarE-Group/Slides/raw/main/2023/2023-08-30-SPLC-Views.pdf)
[![Original Replication Package](https://img.shields.io/badge/Replication_Package-Original-blue)](https://github.com/VariantSync/DiffDetective/tree/splc23-views/replication/splc23-views)
[![Updated Replication Package](https://img.shields.io/badge/Replication_Package-Updated-blue)](https://raw.githubusercontent.com/VariantSync/DiffDetective/main/replication/splc23-views/)
[![Artifact DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8027920.svg)](https://doi.org/10.5281/zenodo.8027920)

> P. M. Bittner, A. Schultheiß, S. Greiner, B. Moosherr, S. Krieter, C. Tinnes, T. Kehrer, T. Thüm. _Views on Edits to Variational Software_. In Proceedings of the 27th ACM International Systems and Software Product Line Conference (SPLC 2023), ACM, New York, NY, August 2023

In this work, we used DiffDetective for a feasibility study of creating views on edits to C-preprocessor based software.
The idea of a view is to act as a filter on relevant parts of a system.
For instance, a piece of source code may be deemed relevant if it implements a certain feature.
A view on an edit thus is a simplified form of an edit that, for example, contains only changes to a certain feature.
From a mathematical perspective, creating such views is in fact a lifting of operations on single revisions of variational systems to operations on diffs of variational systems.

Views are implemented within the [org.variantsync.diffdetective.variation.tree.view](src/main/java/org/variantsync/diffdetective/variation/tree/view/) and [org.variantsync.diffdetective.variation.diff.view](src/main/java/org/variantsync/diffdetective/variation/diff/view/) packages for variaton trees and diffs, respectively.
The empirical evaluation of the view algorithms, including a respective `main` method, is implemented in the [org.variantsync.diffdetective.experiments.views](src/main/java/org/variantsync/diffdetective/experiments/views) package.

The original replication package can be found on the `splc23-views` branch within the directory [replication/splc23-views](https://github.com/VariantSync/DiffDetective/tree/splc23-views/replication/splc23-views) or via the DOI [10.5281/zenodo.8027920](https://doi.org/10.5281/zenodo.8027920). The replication is also available for the most recent version of DiffDetective with an up-to-date version of DiffDetective, which will likely yield to slightly different results than the initial study. The updated replication package can be found in the [replication/splc23-views](replication/splc23-views) subdirectory with it's own [README](replication/splc23-views/README.md).

### Explaining Edits to Variability Annotations in Evolving Software Product Lines (VaMoS 2024)

[![Preprint](https://img.shields.io/badge/Preprint-Read-purple)](https://github.com/SoftVarE-Group/Papers/raw/main/2024/2024-VaMoS-Guething.pdf)
[![Paper](https://img.shields.io/badge/Paper-Read-purple)](https://doi.org/10.1145/3634713.3634725)
[![Slides](https://img.shields.io/badge/Slides-Here-purple)](https://github.com/SoftVarE-Group/Slides/raw/main/2024/2024-02-07-VaMoS-ExplainingEdits.pdf)
[![Replication Package](https://img.shields.io/badge/Replication_Package-at_Fork-blue)][forklg]
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10286851.svg)](https://doi.org/10.5281/zenodo.10286851)

> L. Güthing, P. M. Bittner, I. Schaefer, T. Thüm. _Explaining Edits to Variability Annotations in Evolving Software Product Lines_. In Proceedings of the 18th International Working Conference on Variability Modelling of Software-Intensive Systems (VaMoS 2024), ACM, New York, NY, February 2024

In this work, we formalized an extension of variation diffs, with a typing for edges and pair-wise relations for variability annotations (i.e., mapping nodes in variation diffs). Such edge-typed variation diffs show for example that two annotations exclude or imply each other. Such edge-typed diffs might help better explaining or analyzing edits in the future.

Edge-typed variation diffs and the replication package are implemented in a fork of DiffDetective ([https://github.com/guethilu/DiffDetective][forklg]). The replication package is archived under the DOI [10.5281/zenodo.10286851](https://doi.org/10.5281/zenodo.10286851).

### Student Theses

DiffDetective was extended and used within bachelor's and master's theses:

- _Unparsing von Datenstrukturen zur Analyse von C-Präprozessor-Variabilität_, Eugen Shulimov, Bachelor's Thesis, 2025, [DOI 10.17619/UNIPB/1-2385](http://doi.org/10.17619/UNIPB/1-2385), (german): Eugen added an unparser for variation trees, essentially inverting the horizontal arrows in our commuting diagram at the top of this README file. The unparser for variation diffs reuses the unparser for variation trees by projecting a variation diff to its two variation trees (before and after the change), unparsing the trees, and then diffing the obtained text files to eventually compute a text-based diff.
- _Constructing Variation Diffs Using Tree Diffing Algorithms_, Benjamin Moosherr, Bachelor's Thesis, 2023, [DOI 10.18725/OPARU-50108](https://dx.doi.org/10.18725/OPARU-50108): Benjamin added support for tree-differencing and integrated the GumTree differencer ([Github](https://github.com/GumTreeDiff/gumtree), [Paper](https://doi.org/10.1145/2642937.2642982)). In his thesis, Benjamin also reviewed a range of quality metrics for tree-diffs with focus on their applicability for rating variability-aware diffs. The [org.variantsync.diffdetective.experiments.thesis_bm](src/main/java/org/variantsync/diffdetective/experiments/thesis_bm) package implements the corresponding empirical study and may serve as an example on how to use the tree-differencing.
- _Reverse Engineering Feature-Aware Commits From Software Product-Line Repositories_, Lukas Bormann, Bachelor's Thesis, 2023, [10.18725/OPARU-47892](https://dx.doi.org/10.18725/OPARU-47892): Lukas implemented an algorithm for feature-based commit-untangling, which turns variation diff into a series of smaller diffs, each of which contains an edit to a single feature or feature formula. This work was later refined in our publication _Views on Edits to Variational Software_ illustrated above.
- _Inspecting the Evolution of Feature Annotations in Configurable Software_, Lukas Güthing, Master's Thesis, 2023: Lukas implemented different edge-types for associating variability annotations within variation diffs. He published his work later at VaMoS 2024 under the title _Explaining Edits to Variability Annotations in Evolving Software Product Lines_, illustrated above. His work can be found in a [fork][forklg] of DiffDetective.
- _Empirical Evaluation of Feature Trace Recording on the Edit History of Marlin_, Sören Viegener, Bachelor's Thesis, 2021, [DOI 10.18725/OPARU-38603](http://dx.doi.org/10.18725/OPARU-38603): In his thesis, Sören started the DiffDetective project and implemented the first version of an algorithm, which parses text-based diffs to C-preprocessor files to variation diffs. He also came up with an initial classification of edits, which we wanted to reuse to evaluate [Feature Trace Recording](https://variantsync.github.io/FeatureTraceRecording/), a method for deriving variability annotations from annotated patches.

[documentation]: https://variantsync.github.io/DiffDetective/docs/javadoc
[website]: https://variantsync.github.io/DiffDetective/
[forklg]: https://github.com/guethilu/DiffDetective
[demo]: https://github.com/VariantSync/DiffDetective-Demo
[screencast]: https://www.youtube.com/watch?v=q6ight5EDQY

DiffDetective

DiffDetective Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

DiffDetective

DiffDetective Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README