Skip to content
OliverHennhoeferPublic

Repository files navigation

aberrant

Real-time anomaly detection for data streams.

PyPI version Supported Python versions CI status Code coverage OpenSSF Scorecard Documentation status MIT license

Documentation · Quickstart · Model guide · Evaluation · API reference · Changelog

Running it in production? The integration guide shows how to build detectors from configurations.

Quickstart

# pip install "aberrant[eval]"
from itertools import islice

from aberrant.drift import ADWIN
from aberrant.evaluate import PrequentialEvaluator
from aberrant.model import QuantileThreshold
from aberrant.model.distance import SDOStream
from aberrant.stream.dataset import Dataset, load
from aberrant.transform import IncrementalPCA, StandardScaler

stream = load(Dataset.SHUTTLE).stream()
preprocess = StandardScaler() | IncrementalPCA(n_components=5, n0=100)
for event, _ in islice(stream, 512):
    preprocess.learn_one(event)

detector = SDOStream(k=128, seed=42)
evaluator = PrequentialEvaluator(
    detector,
    warmup=256,
    metrics=True,
    metric_window_size=None,  # Retain all labeled scores for full-stream metrics.
)
threshold = QuantileThreshold(quantile=0.99, window_size=256)
drift = ADWIN(max_window_size=2048)

projected_stream = ((preprocess.transform_one(event), label) for event, label in stream)
alerts = 0
for record in evaluator.iter_evaluate(projected_stream):
    score = record.score
    if score is None:  # The detector is still warming up.
        continue

    score_event = {"score": score}
    if threshold.score_one(score_event) == 1.0:
        alerts += 1
    threshold.learn_one(score_event)
    drift.update(score)

result = evaluator.result()
print(f"Average precision = {result.average_precision:.3f}")
print(f"{alerts} alerts, {drift.n_detections} score drift signals")

Why aberrant

  • Broad toolbox. Forest, density, sketch, graph, matrix-profile, SVM and autoencoder detectors.
  • Score to alert. Drift detectors and adaptive thresholds turn scores into decisions.
  • Built for services. | pipelines, bounded state, detectors built from plain configuration.
  • Measured on the stream. Prequential evaluation, benchmark streams, separate timings.
  • Typed and extensible. Ships py.typed and any protocol-conforming object joins a pipeline.
aberrant river PySAD
Isolation forests and half-space trees ✓ ✓ ✓
Projection sketches (LODA, RS-Hash) ✓ ✓ ✓
Graph-edge streams (MIDAS, AnoEdge, ISCONNA) ✓
Time-series discords (matrix profile, DAMP) ✓
Drift detectors (ADWIN, KSWIN, Page-Hinkley) ✓ ✓
Score thresholds or calibration ✓ ✓ ✓
Detectors built from declarative configuration ✓
General online ML (classification, regression) ✓

Non-trivial detection

Many anomalies have no outlying value at all. In this ECG from the UCR Time Series Anomaly Archive, one malformed heartbeat sits among roughly 180 normal ones, under sensor noise and baseline wander, and every one of its values lies inside the normal range. XLagDAMP, a matrix-profile discord detector, scores the stream in a single pass from the first sample, with no training split and bounded memory, and its highest score lands on that beat. Because the beat differs in shape rather than in value, a detector that scores single values has no outlier to find.

A 30,000-sample noisy ECG with 181 heartbeats, one of them anomalous. The XLagDAMP score stream has a single dominant peak on the anomalous beat. An overlay of all beats shows the anomalous beat stays within the normal value range but has a different shape.

Series 013_UCR_Anomaly_DISTORTEDECG3 (Wu and Keogh, 2021), with XLagDAMP(subsequence_length=167). Reproduce with docs/scripts/discord_figure.py.


Installation

aberrant requires Python 3.11 or newer and is continuously tested on CPython 3.11 to 3.14 on Linux and Windows.

pip install aberrant
Optional extras
Extra Adds
eval scikit-learn metrics for model evaluation
dl the PyTorch-backed Autoencoder
faiss the FAISS similarity-search engine used by models such as KNN
benchmark River and pytest-benchmark
docs the documentation build toolchain
dev linting, typing, testing, and development dependencies
all all optional and development dependencies

For example:

pip install "aberrant[eval,faiss]"

Choose a starting point

Goal Start with
General multivariate detection OnlineIsolationForest or another isolation-forest variant
Local-neighborhood or density anomalies LocalOutlierFactor, KNN, SDOStream, or a cell-based detector
Compact projection or frequency sketches StreamingLODA, MStream, or StreamingRSHash
Anomalous edges and graph evolution AnoEdgeL, ISCONNA, MIDAS, or SignedGraphSketchDetector
Discords in a scalar time series RollingMatrixProfile or XLagDAMP
Departures from a known seasonal forecast SeasonalResidualDetector
Changes in relationships between time-series channels MultivariateRollingMatrixProfile
Interpretable rolling statistics Univariate and multivariate moving statistics
Adaptive margin-based detection Online SVM models
Learned reconstruction error OnlineAutoencoderEnsemble or the optional PyTorch Autoencoder
Detecting distribution drift ADWIN, KSWIN, or PageHinkley
Turning a score into an alert signal QuantileThreshold or ThresholdModel
Misses of an online forecast or regression, at any lead ResidualDetector with HoltWinters, SeasonalNaive or RLSRegressor
An alarm threshold that adapts online, with delayed feedback ScoreCalibrator with ACI, conformal PID, SAOCP and twelve more

See the model guide for inputs, score interpretation, warm-up behavior, and memory characteristics.

Included public model families
Family Implementations
Isolation forest ASDIsolationForest, HalfSpaceTrees, MondrianIsolationForest, OnlineIsolationForest, RandomCutForest, StreamRandomHistogramForest, XStream
Distance CellNeighborhoodDetector, KNN, LocalOutlierFactor, SDOStream, StationaryRegionNeighborDetector
Sketch MStream, StreamingLODA, StreamingRSHash
Graph AnoEdgeL, ISCONNA, MIDAS, SignedGraphSketchDetector
Time series MultivariateRollingMatrixProfile, RollingMatrixProfile, SeasonalResidualDetector, XLagDAMP
SVM GraphGatedOneClassSVM, IncrementalOneClassSVMAdaptiveKernel
Statistical MovingAverage, MovingAverageAbsoluteDeviation, MovingGeometricAverage, MovingHarmonicAverage, MovingInterquartileRange, MovingKurtosis, MovingMedian, MovingQuantile, MovingSkewness, MovingVariance, MovingCorrelationCoefficient, MovingCovariance, MovingMahalanobisDistance, OnlineECOD, OnlineCOPOD
Reconstruction OnlineAutoencoderEnsemble, optional Autoencoder
Score policy QuantileThreshold, ThresholdModel
Baselines NullModel, RandomModel
Drift detection ADWIN, KSWIN, PageHinkley

Pipelines and custom components

Transformers compose left to right, with at most one terminal model:

from aberrant.model.iforest import OnlineIsolationForest
from aberrant.transform import IncrementalPCA, StandardScaler

detector = (
    StandardScaler()  # PCA needs features on comparable scales
    | IncrementalPCA(n_components=3, n0=100)
    | OnlineIsolationForest(window_size=512, seed=42)
)

Read the pipeline guide for lifecycle and composition rules.


Funding acknowledgement