sefr_multiclass_classifier

KO | EN

gitlite — search

#react #ai #typescript #go #deep-learning #docker #macos #nodejs #java #windows #awesome #kubernetes #mcp #open-source

sefr_multiclass_classifier

(★ 28)

Implementations of multiclass version of SEFR linear-time fast classifier (TinyML)

sefr_multiclass_classifier 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# SEFR Multiclass Classifier

### A simple and fast linear-time TinyML algorithm for low-powered microcontrollers

![Figure_1](https://github.com/alankrantas/sefr_multiclass_classifier/raw/master/sefr_binary.png)

This is based on [SEFR: A Fast Linear-Time Classifier for Ultra-Low Power Devices](https://arxiv.org/abs/2006.04620) and its implementation [sefr-classifier/sefr](https://github.com/sefr-classifier/sefr), which was originally a binary classifier. I use one-vs-rest (ovr) strategy to expand it into a working multi-class (multi-label) version.

The idea is to quickly calculate weighted averages and find hyperplanes between classes, so it needs far less computing resources/time and can actually run on-board training on low-end microcontrollers, including 16 MHz AVR processors that have as much as 2 KB ram. It would also be possible to directly re-train models on-device whenever there are new data avaliable.

> Note that I made a minor change from the authors' paper: the weights and bias are calculated based on "not 0", "not 1", "not 2"...instead of 0, 1, 2. In other words, I treat "not N" as the positive label and "N" as negative label. For prediction the model would find the least possible label of "not N" (so it would most possibly to be N). I've found this way generates more accurate results in many datasets.

On the other hand, SEFR would not be super accurate when the difference between classes are not so clear. So it is more suitable for structured data or simple image patterns.

|File|Usage|
|---|---|
|[sefr_binary_visiualization.py](https://github.com/alankrantas/sefr_multiclass_classifier/blob/master/sefr_binary_visiualization.py)|The original binary SEFR classifier with demostration results (the graphs you see above)|
|[sefr.py](https://github.com/alankrantas/sefr_multiclass_classifier/blob/master/sefr.py)|A scikit-learn like Python classifier class|
|[sefr.ino](https://github.com/alankrantas/sefr_multiclass_classifier/blob/master/sefr.ino)|Arduino C++ implementation (can be run on AVRs)|
|[sefr.go](https://github.com/alankrantas/sefr_multiclass_classifier/blob/master/sefr.go)|Golang/TinyGo implementation (can be run on any 32-bit boards supported by TinyGo)|
|[sefr_micrpoython.py](https://github.com/alankrantas/sefr_multiclass_classifier/blob/master/sefr_micrpoython.py)|MicroPython implementation (ESP8266, ESP32 & Raspberry Pico)|
|[sefr_circuitpython_ulab.py](https://github.com/alankrantas/sefr_multiclass_classifier/blob/master/sefr_circuitpython_ulab.py)|CircuitPython 7.0.0 implementation (for CP 7.0.0+ firmwares that has the ulab module)|

## SEFR Example (on computer)

### Iris dataset

This is an example of using the famout Iris dataset (3 classes, 4 features x 150 instances):

```python
from sefr import SEFR  # import from sefr.py
import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_predict
from sklearn.preprocessing import LabelEncoder
from sklearn.metrics import accuracy_score, classification_report

# load Iris dataset
# source: https://archive.ics.uci.edu/ml/datasets/iris
df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data',
                     header=None, names=('sepal length', 'sepal width', 'petal length', 'petal width', 'class'))
    
# extract data and target and convert to ndarray
X = df.drop(['class'], axis=1).to_numpy()
y = df['class'].to_numpy()
    
# encode labels to intergers
le = LabelEncoder()
y = le.fit_transform(y)
class_names = le.classes_  # save class names

# prepare training and test dataset
X_train, X_test, y_train, y_test = \
    train_test_split(X, y, test_size=0.2, random_state=0)

# train model and predict labels
clf = SEFR()
clf.fit(X_train, y_train)
predicted = clf.predict(X_test)
cv_predicted = cross_val_predict(clf, X_train, y_train, cv=5)

# view prediction results
print('Training time:', clf.training_time, 'ns')
print('Training CV score:', accuracy_score(y_train, cv_predicted).round(3))
print('Test accuracy:', accuracy_score(y_test, predicted).round(3))
print('')
print('Test classification report:')
print(classification_report(y_test, predicted, target_names=class_names))
```

Which generates the result below:

```
Training time: 0 ns
Training CV score: 0.942
Test accuracy: 0.967

Test classification report:
                 precision    recall  f1-score   support

Iris-setosa       1.00      1.00      1.00        11
Iris-versicolor       1.00      0.92      0.96        13
 Iris-virginica       0.86      1.00      0.92         6

accuracy                           0.97        30
      macro avg       0.95      0.97      0.96        30
   weighted avg       0.97      0.97      0.97        30
```

### MNIST dataset

Here is another example of using the MNIST dataset (10 classes, 28x28 images, 70,000 instances):

```python
from sefr import SEFR  # import from sefr.py
from tensorflow.keras.datasets import mnist
from sklearn.model_selection import train_test_split, cross_val_predict
from sklearn.metrics import accuracy_score, classification_report
    
# load mnist dataset
(X_train, y_train), (X_test, y_test) = mnist.load_data()
    
# flatten images to one-dimensional
img_size = X_train.shape[1]
X_train = X_train.reshape(-1, img_size ** 2)
X_test = X_test.reshape(-1, img_size ** 2)

# train model and predict labels
clf = SEFR()
clf.fit(X_train, y_train)
predicted = clf.predict(X_test)
cv_predicted = cross_val_predict(clf, X_train, y_train, cv=5)

# view prediction results
print('Training time:', clf.training_time, 'ns')
print('Training CV score:', accuracy_score(y_train, cv_predicted).round(3))
print('Test accuracy:', accuracy_score(y_test, predicted).round(3))
print('')
print('Test classification report:')
print(classification_report(y_test, predicted))
```

Which gets you

```
Training time: 1141000000 ns
Training CV score: 0.797
Test accuracy: 0.809

Test classification report:
              precision    recall  f1-score   support

0       0.96      0.76      0.85       980
           1       0.94      0.88      0.91      1135
           2       0.91      0.78      0.84      1032
           3       0.86      0.77      0.81      1010
           4       0.81      0.77      0.79       982
           5       0.66      0.83      0.74       892
           6       0.91      0.87      0.89       958
           7       0.97      0.72      0.82      1028
           8       0.60      0.86      0.70       974
           9       0.69      0.85      0.76      1009

accuracy                           0.81     10000
   macro avg       0.83      0.81      0.81     10000
weighted avg       0.83      0.81      0.81     10000
```

It takes only 1.141 seconds (on my machine) to train a image recognition model with about 80% accuracy.

## On Microcontrollers

All the microcontroller versions have a built-in Iris dataset (some are quantized into integers to speed up calculation). They will perform training (using the whole dataset) on startup, then predict existing data added with 0~30% random noises. Below is some serial output of an Arduino Uno:

```
Test data: 6.10 2.70 5.20 3.00 
Predicted label: 2 / actual label: 2 / (SEFR training time: 68 ms)

Test data: 6.20 3.30 1.40 0.30 
Predicted label: 0 / actual label: 0 / (SEFR training time: 68 ms)

Test data: 8.30 3.30 4.00 2.30 
Predicted label: 2 / actual label: 2 / (SEFR training time: 68 ms)

Test data: 3.50 2.40 2.40 1.30 
Predicted label: 1 / actual label: 1 / (SEFR training time: 68 ms)

Test data: 3.60 3.80 1.60 0.20 
Predicted label: 0 / actual label: 0 / (SEFR training time: 68 ms)
```

For the Iris dataset, it only takes 0.068 seconds to train on a 16 MHz AVR microcontroller.

The MicroPython version, with minor modifications, can also be used as a pure Python 3.4 implementation.

## Experiment Projecet

I've demostrated SEFR in a color recognition experiment project, which solely use a Arduino Nano and some cheap sensors to train and run the classifier. See my [Hackster.io page](https://www.hackster.io/alankrantas/use-sefr-ml-on-arduino-nano-for-color-recognition-b59e24) fore details.

sefr_multiclass_classifier

sefr_multiclass_classifier 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

sefr_multiclass_classifier

sefr_multiclass_classifier 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README