Knockoff-ML

KO | EN

gitlite — search

#hacktoberfest #react #typescript #go #machine-learning #deep-learning #linux #macos #nodejs #windows #cli #vue #ios #openai

(★ 6)

A knockoff-based machine learning framework for controlled variable selection and risk prediction in electronic health record (EHR) data

Knockoff-ML Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Knockoff-ML: A knockoff machine learning framework for controlled variable selection and risk stratification in EHR data 
## Overview
* Knockoff-ML first generates multiple knockoffs using the sequential conditional independent tuples (SCIT) algorithm. Next, both the original and knockoff datasets are fed into ML models of choice. Knockoff-ML then computes feature importance (FI) for each feature in all models using SHapley Additive exPlanation (SHAP) values. Knockoff-ML then calculates knockoff statistics and identify key features with false discovery rate (FDR) control. Finally, Knockoff-ML trains risk prediction models with identified key features. We illustrate the workflow for Knockoff-ML as follows.
![flowchart2](https://github.com/user-attachments/assets/7c8373e5-4ee0-49d2-bacc-0a539304528d)
---
## Repo Contents
* `Knockoff-ML.R`: R scripts for generating multiple knockoffs and calculating knockoff statistics for controlled variable selection.
* `Knockoff-ML_FI.ipynb`: Python notebook for computing feature importance, leveraging SHAP values.
* `Knockoff-ML_Prediction.ipynb`: Python notebook for risk prediction with features identified by Knockoff-ML.
* `Data.zip`: An example dataset which contains the original data, knockoff data, outcome, subsampling indices, feature importance, selection information, and prediction results. This dataset can be used to test the workflow code.

---

## Workflow
**Step1:** Generate multiple knockoffs using `generate_knockoff` function from `Knockoff-ML.R` file.

The function (`generate_knockoff`) will return a list containing your original data, the generated knockoff data, and subsampling indices. Please ensure that each knockoff dataset will be saved as a .csv file following the naming convention X_k`i`.csv, where `i` denotes the i-th knockoff. 
```bash
#load data
X <- read.csv('/Data/X.csv')

#generate knockoffs
M <- 5
X_mk <- generate_knockoff(X=X, M=M, scaled=TRUE, subsample=TRUE)

#write knockoffs and index
for(i in 1:M){
 write.csv(X_mk$X_MK[,,i],paste0('/Data/X_k',i,'.csv'),row.names = F))
 }
write.table(X_mk$Index,paste0('/Data/Index.csv'),row.names = F, col.names = F)
```
- **X:** Original dataset. 
- **M:** A positive integer for the number of knockoffs. The default is 5. 
- **scaled:** Logical indicating whether continuous variables have been normalized. The default is TRUE. 
- **subsample:** Logical indicating whether subsampling should be performed for SHAP value calculation. The default is TRUE.

**Step2:** Compute feature imporatnce using `calculate_fi` function in the `Knockoff-ML_FI.ipynb` file.

The function (`calculate_fi`) will return an (M+1)*p data frame which contains feature importance for each specific ML model. M is the number of knockoffs, and p is the number of features. The first row of the data frame will contain the feature importance for the original data, and rows 2 through M+1 will contain the feature importance for the knockoff data. These data frames will be saved as .csv files following the naming convention `model_name`_fi.csv, where `model_name` denotes the ML model used within Knockoff-ML. 
```bash
#dichotomous trait
#load data 
X = pd.read_csv('/Data/X.csv')
y = pd.read_csv('/Data/y_dict.csv')
index = pd.read_csv('/Data/Index.csv', header=None)

#calculate feature importance
calculate_fi(X, y, index, models=['catb', 'ligb', 'xgb', 'gbdt', 'rf'], binary=True, M=5, kopath='/Data', outpath='/Data')

#quantitative trait
#load data 
X = pd.read_csv('/Data/X.csv')
y = pd.read_csv('/Data/y_quan.csv')
index = pd.read_csv('/Data/Index.csv', header=None)

#calculate feature importance
calculate_fi(X, y, index, models=['catb', 'ligb', 'xgb', 'gbdt', 'rf'], binary=False, M=5, kopath='/Data', outpath='/Data')
```
- **X**: Original dataset. 
- **y**: Outcome of interest. 
- **index**: Index for subsampling in SHAP calculation. 
- **models**: A list of ML models used within Knockoff-ML. You can choose one or more of the following models: 'catb', categorical boosting; 'ligb', light gradient boosting machine; 'xgb', extreme gradient boosting; 'gbdt', gradient boosting decision trees; 'rf', random forest; 'svm', support vector machine; 'nn', neural networks; 'knn', k-nearest neighbors; 'linear', linear regression only for quantitative trait; 'logistic', logistic regression only for dichotomous trait. 
- **binary**: Logical indicating whether the outcome y is binary or continuous. The default value is True. 
- **M**: A positive integer for the number of knockoffs. 
- **kopath:** Path to the knockoff data. Please ensure that each knockoff dataset is saved as a .csv file following the naming convention `X_ki.csv`, where i denotes the i-th knockoff. 
- **outpath:** Path to save feature importance.

**Note:** Knockoff-ML is a flexible framework that can incorporate various type of machine learning models, you can choose any machine learning models suitable for your work, such as logistic/linear regression, support vector machine, neural networks, and k-nearest neighbors.

**Step3:** Identify features with FDR control using `Get_select_info` function in the `Knockoff_ML.R` file.

The function (`Get_select_info`) will return a data frame including information about knockoff statistics and selection results. The `Select_info` will be saved as .csv files following the naming convention `model_name`_select.csv, where `model_name` denotes the ML model used within Knockoff-ML.

The function (`Get_select_info`) will return a p*4 data frame.
- **column1 'Feature':** the feature names of the original data.
- **column2 'w':** knockoff w statistics.
- **column3 'q':** knockoff q values.
- **column4 'select':** logical whether the features should be selected or not.
```bash
#load data
X <- read.csv('/Data/X.csv')
M <- 5

#feature selection with FDR control
for(model in c("catb","ligb","xgb","gbdt","rf")){
 FI <- read.csv(paste0('/Data/',model,'_fi.csv'))
 Select_info <- Get_select_info(Feature_name=colnames(X),T_0=t(FI[1,]),T_K=FI[2:M+1,],M=M,fdr=0.1),
 write.csv(Select_info,paste0('/Data/',model,'_select.csv'),row.names = F)
 }
```
- **FI:** An (M+1)*p data frame, where M is the number of knockoffs, and p is the number of features. The first row of FI should contain the feature importance for the original data, and rows 2 through M+1 should contain the feature importance for the knockoff data. 
- **Feature name:** The feature names of the original dataset. 
- **X:** Original dataset. 
- **T_0:** A numeric vector of length p for the feature importance of p features in the original dataset.
- **T_K:** An M*p data frame for the feature importance of p features in each knockoff dataset.
- **M:** A positive integer for the number of knockoffs. 
- **fdr:** A real number in a range of (0,1) indicating the target FDR level. The default is 0.1.

**Step4:** Train prediction models and return prediction results using `koml_prediction` function in the `Knockoff-ML_Prediction.ipynb` file with identified features by Knockoff-ML.

The function (`koml_prediction`) will return a data frame containing prediction results. The prediction results will be saved as .csv files following the naming convention `model_name`_prediction.csv, where `model_name` denotes the ML model used within Knockoff-ML.

The function (`koml_prediction`) will return an n*2 data frame, where n is the number of rows in original data X.
- **column1 'Actual':** the actual outcome.
- **column2 'Predicted':** the predicted probabilities for binary classification or the predicted values for regression.
```bash
#dichotomous trait
#load data
X = pd.read_csv('/Data/X.csv')
y = pd.read_csv('/Data/y_dict.csv')

#Train prediction models and return prediction results
koml_prediction(X, y, models=['catb', 'ligb', 'xgb', 'gbdt', 'rf'], binary=True, colpath='/Data',outpath='/Data')

#quantitative trait
#load data
X = pd.read_csv('/Data/X.csv')
y = pd.read_csv('/Data/y_quan.csv')

#Train prediction models and return prediction results
koml_prediction(X, y, models=['catb', 'ligb', 'xgb', 'gbdt', 'rf'], binary=False, colpath='/Data',outpath='/Data')
```
- **X:** Original dataset. 
- **y:** Outcome of interest. 
- **models:** A list of ML models used within Knockoff-ML. You can choose one or more of the following models: 'catb', categorical boosting; 'ligb', light gradient boosting machine; 'xgb', extreme gradient boosting; 'gbdt', gradient boosting decision trees; 'rf', random forest; 'svm', support vector machine; 'nn', neural networks; 'knn', k-nearest neighbors; 'linear', linear regression only for quantitative trait; 'logistic', logistic regression only for dichotomous trait. 
- **binary**: Logical indicating whether the outcome y is binary or continuous. The default value is True. 
- **colpath:** Path to feature selection information obtained in **Step3**. 
- **outpath:** Path to save prediction results.

## Example data
Note that data access to the MIMIC IV need Data Use Agreement with PhysioNet. We are unable to provide the datasets directly in this repository. However, for your convenience, we've included an example dataset in the `Data.zip` file. You can use this example data to reproduce and test the workflow demonstrated in our code. The results files provided in the `Data.zip` were generated specifically from a dichotomous classification task.

## Dependencies
The code should run with the following environment.

"R version 4.4.1", "Matrix 1.7.0".

"Python version 3.9.13", "numpy 1.21.6", "pandas 1.4.4", "shap 0.46.0", "scikit-learn 1.5.2", "catboost 1.1.1", "lightgbm 3.3.5", "xgboost 1.7.5".

Knockoff-ML

Knockoff-ML Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Knockoff-ML

Knockoff-ML Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README