Detailed Evaluation

This page hosts the detailed evaluation figures (confusion matrices and positioning error densities) for the WavesFM release. Classification tasks list the class labels on the page, while the figures use numeric indices. Last updated 2026-01-23.

Regime

Dataset

Task

Type

33 entries

5G NR Positioning (POS) (FT2)

5G NR Positioning

FT2positionall_seeds

Dataset protocol

Mean error: 1.24

Median error: 0.82

P90 error: 2.74

Test samples: 2325 (20% split)

DeepMIMO (FT2)

Beam Prediction

FT2classificationall_seeds

Dataset protocol

Acc@1: 80.50

Acc@3: 94.45

PCA (mean per-class accuracy): 78.38

Test samples: 2968 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

EfficientFi (HAS) (FT2)

Human Activity Sensing

FT2classificationall_seeds

Dataset protocol

Acc@1: 99.31

Acc@3: 100.00

PCA (mean per-class accuracy): 99.31

Test samples: 240 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

ICARUS (INTD/INTC) (FT2)

Interference Classification

FT2classificationall_seeds

Dataset protocol

Acc@1: 75.00

Acc@3: 100.00

PCA (mean per-class accuracy): 78.94

Test samples: 500 (20% split)

Class labels: 0: No Interference, 1: Interference Type 1, 2: Interference Type 2

DeepMIMO (FT2)

LOS/NLOS Classification

FT2classificationall_seeds

Dataset protocol

Acc@1: 95.66

Acc@3: 100.00

PCA (mean per-class accuracy): 95.46

Test samples: 2968 (20% split)

Class labels: 0: LOS, 1: NLOS

RML2022 (FT2)

Modulation Classification

FT2classificationall_seeds

Dataset protocol

Acc@1: 55.25

Acc@3: 77.45

PCA (mean per-class accuracy): 55.16

Test samples: 92400 (20% split)

Confusion matrices at selected SNRs

SNR 0

SNR 20

Class labels: see dataset protocol for the mapping from indices to labels.

RADCOM (FT2)

RADCOM Signal Classification

FT2classificationall_seeds

Dataset protocol

Acc@1: 94.53

Acc@3: 99.96

PCA (mean per-class accuracy): 94.53

Test samples: 113400 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

POWDER (RFP) (FT2)

RF Fingerprinting

FT2classificationall_seeds

Dataset protocol

Acc@1: 99.70

Acc@3: 99.99

PCA (mean per-class accuracy): 99.70

Test samples: 208894 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

CommRad RF (RFS) (FT2)

RF Signal Classification

FT2classificationall_seeds

Dataset protocol

Acc@1: 84.53

Acc@3: 94.29

PCA (mean per-class accuracy): 83.41

Test samples: 724 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

UWB Indoor Positioning and Tracking (UWB-Indoor) (FT2)

UWB Indoor Positioning

FT2positionall_seeds

Dataset protocol

Mean error: 0.83

Median error: 0.60

P90 error: 1.71

Test samples: 3160 (20% split)

UWB Industrial Localization (UWB-Industrial) (FT2)

UWB Industrial Localization

FT2positionall_seeds

Dataset protocol

Mean error: 0.88

Median error: 0.72

P90 error: 1.62

Test samples: 8052 (20% split)

5G NR Positioning (POS) (LoRA)

5G NR Positioning

LoRApositionall_seeds

Dataset protocol

Mean error: 1.45

Median error: 0.99

P90 error: 3.09

Test samples: 2325 (20% split)

DeepMIMO (LoRA)

Beam Prediction

LoRAclassificationall_seeds

Dataset protocol

Acc@1: 79.68

Acc@3: 93.82

PCA (mean per-class accuracy): 77.38

Test samples: 2968 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

EfficientFi (HAS) (LoRA)

Human Activity Sensing

LoRAclassificationall_seeds

Dataset protocol

Acc@1: 98.47

Acc@3: 100.00

PCA (mean per-class accuracy): 98.47

Test samples: 240 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

ICARUS (INTD/INTC) (LoRA)

Interference Classification

LoRAclassificationall_seeds

Dataset protocol

Acc@1: 74.87

Acc@3: 100.00

PCA (mean per-class accuracy): 78.94

Test samples: 500 (20% split)

Class labels: 0: No Interference, 1: Interference Type 1, 2: Interference Type 2

DeepMIMO (LoRA)

LOS/NLOS Classification

LoRAclassificationall_seeds

Dataset protocol

Acc@1: 95.60

Acc@3: 100.00

PCA (mean per-class accuracy): 95.24

Test samples: 2968 (20% split)

Class labels: 0: LOS, 1: NLOS

RML2022 (LoRA)

Modulation Classification

LoRAclassificationall_seeds

Dataset protocol

Acc@1: 56.59

Acc@3: 78.59

PCA (mean per-class accuracy): 56.49

Test samples: 92400 (20% split)

Confusion matrices at selected SNRs

SNR 0

SNR 20

Class labels: see dataset protocol for the mapping from indices to labels.

RADCOM (LoRA)

RADCOM Signal Classification

LoRAclassificationall_seeds

Dataset protocol

Acc@1: 93.78

Acc@3: 99.99

PCA (mean per-class accuracy): 93.78

Test samples: 113400 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

POWDER (RFP) (LoRA)

RF Fingerprinting

LoRAclassificationall_seeds

Dataset protocol

Acc@1: 99.82

Acc@3: 100.00

PCA (mean per-class accuracy): 99.82

Test samples: 208894 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

CommRad RF (RFS) (LoRA)

RF Signal Classification

LoRAclassificationall_seeds

Dataset protocol

Acc@1: 86.46

Acc@3: 93.51

PCA (mean per-class accuracy): 84.49

Test samples: 724 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

UWB Indoor Positioning and Tracking (UWB-Indoor) (LoRA)

UWB Indoor Positioning

LoRApositionall_seeds

Dataset protocol

Mean error: 0.65

Median error: 0.50

P90 error: 1.30

Test samples: 3160 (20% split)

UWB Industrial Localization (UWB-Industrial) (LoRA)

UWB Industrial Localization

LoRApositionall_seeds

Dataset protocol

Mean error: 0.74

Median error: 0.63

P90 error: 1.33

Test samples: 8052 (20% split)

5G NR Positioning (POS) (LP)

5G NR Positioning

LPpositionall_seeds

Dataset protocol

Mean error: 3.17

Median error: 2.77

P90 error: 5.92

Test samples: 2325 (20% split)

DeepMIMO (LP)

Beam Prediction

LPclassificationall_seeds

Dataset protocol

Acc@1: 70.66

Acc@3: 90.96

PCA (mean per-class accuracy): 67.67

Test samples: 2968 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

EfficientFi (HAS) (LP)

Human Activity Sensing

LPclassificationall_seeds

Dataset protocol

Acc@1: 94.44

Acc@3: 100.00

PCA (mean per-class accuracy): 94.42

Test samples: 240 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

ICARUS (INTD/INTC) (LP)

Interference Classification

LPclassificationall_seeds

Dataset protocol

Acc@1: 67.47

Acc@3: 100.00

PCA (mean per-class accuracy): 71.50

Test samples: 500 (20% split)

Class labels: 0: No Interference, 1: Interference Type 1, 2: Interference Type 2

DeepMIMO (LP)

LOS/NLOS Classification

LPclassificationall_seeds

Dataset protocol

Acc@1: 95.35

Acc@3: 100.00

PCA (mean per-class accuracy): 95.02

Test samples: 2968 (20% split)

Class labels: 0: LOS, 1: NLOS

RML2022 (LP)

Modulation Classification

LPclassificationall_seeds

Dataset protocol

Acc@1: 50.49

Acc@3: 74.34

PCA (mean per-class accuracy): 50.39

Test samples: 92400 (20% split)

Confusion matrices at selected SNRs

SNR 0

SNR 20

Class labels: see dataset protocol for the mapping from indices to labels.

RADCOM (LP)

RADCOM Signal Classification

LPclassificationall_seeds

Dataset protocol

Acc@1: 90.12

Acc@3: 99.97

PCA (mean per-class accuracy): 90.10

Test samples: 113400 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

POWDER (RFP) (LP)

RF Fingerprinting

LPclassificationall_seeds

Dataset protocol

Acc@1: 98.96

Acc@3: 100.00

PCA (mean per-class accuracy): 98.97

Test samples: 208894 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

CommRad RF (RFS) (LP)

RF Signal Classification

LPclassificationall_seeds

Dataset protocol

Acc@1: 55.52

Acc@3: 83.33

PCA (mean per-class accuracy): 42.85

Test samples: 724 (20% split)

Class labels: see dataset protocol for the mapping from indices to labels.

UWB Indoor Positioning and Tracking (UWB-Indoor) (LP)

UWB Indoor Positioning

LPpositionall_seeds

Dataset protocol

Mean error: 1.43

Median error: 1.24

P90 error: 2.65

Test samples: 3160 (20% split)

UWB Industrial Localization (UWB-Industrial) (LP)

UWB Industrial Localization

LPpositionall_seeds

Dataset protocol

Mean error: 2.71

Median error: 2.50

P90 error: 4.69

Test samples: 8052 (20% split)