KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#developer-tools #codex #typescript #ai-agents #ai-agent #cli #claude-code #open-source #deepseek-harness #dsh-plugin #python #mcp

FutureOfAIviaAI

★ 84 Open GitHub ↗

Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge network

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Forecasting the future of artificial intelligence

**[Forecasting the future of artificial intelligence with machine learning-based link prediction in an exponentially growing knowledge network]([https://arxiv.org/abs/2210.00881](https://www.nature.com/articles/s42256-023-00735-0))**\
_Mario Krenn, Lorenzo Buffoni, Bruno Coutinho, Sagi Eppel, Jacob Gates Foster, Andrew Gritsevskiy, Harlin Lee, Yichao Lu, Joao P. Moutinho, Nima Sanjabi, Rishi Sonthalia, Ngoc Mai Tran, Francisco Valente, Yangxinyu Xie, Rose Yu, Michael Kopp_

This page collects results for an AI benchmark for link prediction in exponentially growing knowledge networks. It is a follow-up of the [IEEE BigData Science4Cast competition 2021](https://github.com/iarai/science4cast), described in our [Nature Machine Intelligence paper](https://www.nature.com/articles/s42256-023-00735-0).

## Files

- **create_data.py**: A simple python file for creating the datasets SemanticGraph_delta_N_cutoff_M_minedge_P.pkl from the full semantic network all_edges.pkl.
- **evaluate_model.py**: Runs my simple baseline model on all datasets
- **simple_model.py**: My baseline, containing 15 predefined properties that are computed for each unconnected pair. Same model as in the competition.
- **utils.py**: Contains useful functions, such as the creation of datasets from the full semantic network (unbiased for test-set, and biased [i.e. same number of positive and negative solutions] for training if desired), and AUC computation.

Datasets can be downloaded via [zenodo.org](https://zenodo.org/record/7882892#.ZE-Egx9BwuU) (file names: SemanticGraph_delta_N_cutoff_M_minedge_P.pkl).

## Running the code
1) Create the required environment (see `environmen.yaml` and `pyproject.toml`). Requirements: pytorch, numpy, networkx, scipy, matplotlib.
2) Download the dataset files from [zenodo.org](https://zenodo.org/record/7882892#.ZE-Egx9BwuU) and store them in the same directory as the python files.
3) Run `python evaluate_model.py`. This runs the baseline model M6 on all 18 datasets. Each dataset takes roughly 2h on a standard notebook (i.e. a total of 36 hours).
4) The python code generates a log file called `logs_SemanticGraph_delta_M_cutoff_N_minedge_K.pkl.txt` (M,N,K are numbers). One example (for M=1, N=25, K=1) of such an expected output file is stored [here](https://github.com/artificial-scientist-lab/FutureOfAIviaAI/blob/main/expected_output/logs_SemanticGraph_delta_1_cutoff_25_minedge_1.pkl.txt). Similar files will be generated for all other 17 datasets.

## Degree Evolution
![alt text](https://raw.githubusercontent.com/artificial-scientist-lab/FutureOfAIviaAI/main/miscellaneous/node_degree_loglog.gif)

## Result of [Yichao's Model (M1)](https://github.com/artificial-scientist-lab/FutureOfAIviaAI/tree/main/all_models/M1)

*Area under the Curve (AUC) for prediction of new edge_weights of 1*

|           | *cutoff=0*  | *cutoff=5* | *cutoff=25* |
|-----------|-------------|------------|-------------|
| *delta=1* | 0.9330      | 0.9252     | 0.9248      |
| *delta=3* | 0.9172      | 0.9191     | 0.9096      |
| *delta=5* | 0.8960      | 0.8987     | 0.8935      |