error-consistency

KO | EN

gitlite — search

#python #hacktoberfest #typescript #android #machine-learning #rust #macos #nodejs #java #cli #kubernetes #vue #mcp #openai

error-consistency

(★ 9)

Error consistency: a black-box analysis for comparing errors between decision makers (NeurIPS 2020)

error-consistency 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Data and analysis code from "Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency"

Error consistency is a quantitative analysis for measuring whether two decision making systems systematically make errors on the same inputs. The paper is available on [arXiv](https://arxiv.org/abs/2006.16736).

## dependencies
The R analysis scripts have the following dependencies which can be installed via ``install.packages("package-name")``. Data analysis was performed using R version `3.5.1`.

library(lattice)
	library(jpeg)
	library(R.matlab)
	library(graphics)
	library(pROC)
	library(psych)
	library(grid)
	library(gridExtra)
	library(stats)
	library(png)
	library(pBrackets)
	library(PET)
	library(TeachingDemos)
	library(binom)
	library(RColorBrewer)
	library(ggplot2)
	library(scales)
	library(xtable)
	library(viridis)
	library(binom)

The Brain-Score parsing script has the following dependencies:

pip3 install jupyterlab
	pip3 install urllib
	pip3 install bs4
	pip3 install numpy

## data-analysis
Scripts to analyse the data from `raw-data/` and plot figures to `figures/`.

The main analysis script is `data-analysis.R`. Confidence intervals are simulated via `simulate-confidence-intervals.R`. Brain-Score metrics are parsed from the [Brain-Score website](http://www.brain-score.org/) with `get_data_from_Brain_Score.ipynb`.

## documentation
Project-related documentation.

## figures
Figures generated by scripts from `data-analysis/` using data from `raw-data/`.

## raw-data
Experiments with the prefix `noise-generalisation` are from this [paper](http://papers.nips.cc/paper/7982-generalisation-in-humans-and-deep-neural-networks.pdf); the raw data is copied from the corresponding [github repository](https://github.com/rgeirhos/generalisation-humans-DNNs). 
Experiments with the prefix `texture-shape` are from this [paper](https://openreview.net/forum?id=Bygh9j09KX); the raw data is copied from the corresponding [github repository](https://github.com/rgeirhos/texture-vs-shape).

error-consistency

error-consistency 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

error-consistency

error-consistency 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README