Link Search Menu Expand Document
dnn-tip

DNN-TIP: Common Test Input Prioritizers

A collection of dnn test input prioritizers often used as benchmarks in recent literature.

Implemented Approaches

  • Surprise Adequacies
    • Distance-based Surprise Adequacy (DSA)
    • Likelihood-based Surprise Adequacy (LSA)
    • MultiModal-Likelihood-based Surprise Adequacy (MLSA)
    • Mahalanobis-based Surprise Adequacy (MDSA)
    • abstract MultiModal Surprise Adequacy
  • Surprise Coverage
    • Neuron-Activation Coverage (NAC)
    • K-Multisection Neuron Coverage (KMNC)
    • Neuron Boundary Coverage (NBC)
    • Strong Neuron Activation Coverage (SNAC)
    • Top-k Neuron Coverage (TKNC)
  • Utilities
    • APFD calculation
    • Coverage-Added and Coverage-Total Prioritization Methods (CAM and CTM)

If you are looking for the uncertainty metrics we also tested (including DeepGini), head over to the sister repository uncertainty-wizard.