benfords-law

KO | EN

gitlite — search

#python #ai #typescript #llm #rust #linux #macos #nodejs #windows #cli #awesome-list #security #mcp #open-source

(★ 140)

Experimenting with Benford's Law

benfords-law Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Testing Benford's Law

This is a simple experiment by [Jason Long](http://twitter.com/jasonlong) and [Bryce Thornton](http://twitter.com/brycethornton) to test how many real-life, publicly available datasets satisfy [Benford's Law](http://en.wikipedia.org/wiki/Benfords_law).

## Contributing Datasets

If you find this to be an interesting idea, we'd encourage you to help add more datasets to the site. We've intentionally kept the site as simple and lightweight as possible. There is no real backend - the data has been crunched in advance and the results are simply entered into JSON files.

To contribute a new dataset, you'll need to do two things:

### Add the dataset name to the JSON index file

The format of `js/datasets/index.json` is simply a key/value pair:

{
      "twitter-users-by-followers-count": "Twitter users by followers count",
      "distance-of-stars-from-earth-in-light-years": "Distance of stars from Earth in light years",
      "loan-amounts-on-kiva-org": "Loan amounts on kiva.org",
      "total-number-of-print-materials-in-us-libraries": "Total number of print materials in US libraries",
      "population-of-spanish-cities": "Population of Spanish cities"
    }

### Create a dataset JSON file

Add your new file in the `/js/datasets/` directory with a name that matches the key used in step 1. The format looks like this:

{
  		"values": {
  			"1": 32.62,
  			"2": 16.66,
  			"3": 11.80,
  			"4": 9.26,
	  		"5": 7.63,
			  "6": 6.55,
  			"7": 5.76,
  			"8": 5.14,
  			"9": 4.56
  		},
  		"num_records": "38,670,514",
  		"min_value": "1",
  		"max_value": "4,706,631",
  		"source": "http://www.infochimps.com/datasets/twitter-census-twitter-users-by-friends-count"
  	}

It's important to include the source of the data used so that others can verify and reproduce the results.

## Crunching the data

Generating Benford stats is a fairly straightforward process. We've made a simple ruby class for you to use if you'd like.

First, grab a copy of the class from here: https://gist.github.com/1044174

Second, include the class in your script like so:

```ruby
require 'benford_counter'
require 'rubygems'
require 'csv'

counter = BenfordCounter.new

CSV.foreach("spain.txt") do |row|
  counter.count(row[9])
end

counter.results
```

### Additional Tools

[fweez](https://github.com/fweez) contributed the Linux filesize dataset and [created a Python script for tallying filesizes in a directory](https://gist.github.com/1049438).

## Updating Javascript and CSS

We're using CoffeeScript for the Javascript and Sass/Compass for the CSS.

Once CoffeeScript is installed ([see the CoffeeScript docs](http://jashkenas.github.com/coffee-script/)), run this command from the project root to observe and compile changes:

coffee --watch -o js/ --compile js/coffee/*.coffee

Note that the only file that should be edited is `/js/coffee/app.coffee`. The `/js/coffee/app.js` file is generated by CoffeeScript.

To make changes to the CSS, you need to install Sass and Compass ([see the Compass docs](http://compass-style.org/install/). Then edit `/css/sass/screen.scss`. You observe and compile changes by running this command from the project root:

compass watch

To compile a production-ready compressed version:

compass compile --output-style compressed --force

benfords-law

benfords-law Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

benfords-law

benfords-law Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README