AdaOpt
Want to share your content on python-bloggers? click here.
AdaOpt
is a probabilistic classifier based on a mix of multivariable optimization and a nearest neighbors algorithm. More details about it are found in this paper. When reading the paper, keep in mind that the algorithm is still very new; only time will allow to fully appreciate all its features. Plus, its performance on this dataset is not an indicator of its future performance, on other datasets.
Currently, the package containing AdaOpt
, mlsauce
, can be installed from the command line as:
pip install git+https://github.com/thierrymoudiki/mlsauce.git
In this post, we’ll use mlsauce
’s AdaOpt
on a handwritten digits dataset from UCI Machine Learning repository.
The model is firstly trained on a set of digits – to distinguish between a “3”, or a”6”, etc.:
from time import time from tqdm import tqdm import mlsauce as ms import numpy as np from sklearn.metrics import classification_report from sklearn.model_selection import train_test_split from sklearn.datasets import load_digits # Load datasets digits = load_digits() Z = digits.data t = digits.target # Split data in training and testing sets np.random.seed(2395) X_train, X_test, y_train, y_test = train_test_split(Z, t, test_size=0.2) obj = ms.AdaOpt(n_iterations=50, learning_rate=0.3, reg_lambda=0.1, reg_alpha=0.5, eta=0.01, gamma=0.01, tolerance=1e-4, row_sample=1, k=3) # Teaching AdaOpt to recognize digits start = time() obj.fit(X_train, y_train) print(time()-start)
0.03549695014953613
Then, AdaOpt
is tasked to recognize new, unseen digits (X_test, y_test)
, based on what it has seen on the training set (X_train, y_train)
:
start = time() print(obj.score(X_test, y_test)) print(time()-start)
0.9944444444444445 0.19525575637817383
The accuracy is high on this dataset. Additional error metrics are presented in the following table:
preds = obj.predict(X_test) print(classification_report(preds, y_test))
precision recall f1-score support 0 1.00 1.00 1.00 31 1 1.00 0.97 0.99 40 2 1.00 1.00 1.00 36 3 1.00 1.00 1.00 45 4 1.00 1.00 1.00 37 5 0.97 1.00 0.98 29 6 1.00 0.98 0.99 42 7 1.00 1.00 1.00 35 8 0.97 1.00 0.99 33 9 1.00 1.00 1.00 32 accuracy 0.99 360 macro avg 0.99 1.00 0.99 360 weighted avg 0.99 0.99 0.99 360
Ad here is a confusion matrix:
At test time, AdaOpt
uses a nearest neighbors algorithm. Which means, a task with quadratic complexity (a large number of operations). But there are a few tricks implemented in mlsauce
’s AdaOpt
to alleviate the potential burden on very large datasets, such as: instead of comparing the testing set to the whole training set, comparing it to a stratified subsample of the training set.
row_sample == 0.1
for example in the next figure, means that 1/10 of the training set is used in the nearest neighbors procedure at test time. The figure represents a distribution of test set accuracy:
We also have the following timings in seconds (current, could be faster in the future), as a function of row_sample
:
The paper contains a more detailed discussion of how these figures are obtained, and a description of AdaOpt
.
Note: I am currently looking for a gig. You can hire me on Malt or send me an email: thierry dot moudiki at pm dot me. I can do descriptive statistics, data preparation, feature engineering, model calibration, training and validation, and model outputs’ interpretation. I am fluent in Python, R, SQL, Microsoft Excel, Visual Basic (among others) and French. My résumé? Here!
Want to share your content on python-bloggers? click here.