API Reference¶

Matching¶

Matching algorithms: greedy, Hungarian, and structure-aware.

setjoin.matchers.greedy_match(scores, show_progress=False)[source]¶

Match records greedily by selecting highest-scoring pairs first.

Parameters:

scores (NDArray[float64]) – Pairwise score matrix (n_source x n_target)
show_progress (bool) – Whether to show progress bar

Returns:

Greedy matches

Return type:

MatchResult

setjoin.matchers.hungarian_match(scores)[source]¶

Match records using Hungarian algorithm to maximize total score.

Parameters:: scores (NDArray[float64]) – Pairwise score matrix (n_source x n_target)
Returns:: Optimal global assignment
Return type:: MatchResult

setjoin.matchers.structure_aware_match(scores, hierarchy, show_progress=False)[source]¶

Match records while preserving group structure.

Two-level assignment: first assign groups optimally, then assign records within matched groups. Unequal group counts or sizes can leave records unmatched; their positions are returned in metadata.

Parameters:

scores (NDArray[float64]) – Pairwise record score matrix (n_source x n_target)
hierarchy (HierarchySpec) – HierarchySpec defining group structure
show_progress (bool) – Whether to show progress bar

Returns:

Structure-preserving matches

Return type:

MatchResult

Raises:

ValueError – If the groups do not partition the score-matrix positions

setjoin.matchers.match(scores, method='hungarian', hierarchy=None, show_progress=False)[source]¶

Match records using the specified method.

Parameters:

scores (NDArray[float64]) – Pairwise score matrix (n_source x n_target)
method (str) – Matching method - “greedy”, “hungarian”, or “structure_aware”
hierarchy (HierarchySpec | None) – Required for structure_aware method
show_progress (bool) – Whether to show progress bar

Returns:

Matches using specified method

Return type:

MatchResult

Raises:

ValueError – If method is unknown or hierarchy missing for structure_aware

setjoin.matchers.compare_methods(scores, hierarchy=None, methods=None)[source]¶

Run multiple matching methods and return results for comparison.

Parameters:

scores (NDArray[float64]) – Pairwise score matrix (n_source x n_target)
hierarchy (HierarchySpec | None) – Required if “structure_aware” is in methods
methods (list[str] | None) – List of methods to run (default: all available)

Returns:

Method name to result mapping

Return type:

dict[str, MatchResult]

Scoring¶

Configurable scoring functions for record comparison.

setjoin.scorers.exact_match(a, b)[source]¶

Return 1.0 where values match exactly, 0.0 otherwise.

Parameters:

a (NDArray[object_])
b (NDArray[object_])

Return type:

NDArray[float64]

setjoin.scorers.abs_diff(a, b)[source]¶

Return negative absolute difference (larger = more similar).

Parameters:

a (NDArray[float64])
b (NDArray[float64])

Return type:

NDArray[float64]

setjoin.scorers.levenshtein(a, b)[source]¶

Compute Levenshtein similarity ratio for each pair.

Parameters:

a (NDArray[object_])
b (NDArray[object_])

Return type:

NDArray[float64]

setjoin.scorers.jaro_winkler(a, b)[source]¶

Compute Jaro-Winkler similarity for each pair.

Parameters:

a (NDArray[object_])
b (NDArray[object_])

Return type:

NDArray[float64]

class setjoin.scorers.Scorer(config)[source]¶

Bases: object

Configurable scorer for computing pairwise similarity between records.

Parameters:: config (dict[str, dict[str, float | str] | FieldConfig])

score(source, target, source_suffix='', target_suffix='')[source]¶

Compute pairwise score matrix between source and target records.

Parameters:

source (DataFrame) – DataFrame with source records
target (DataFrame) – DataFrame with target records
source_suffix (str) – Suffix to append to field names for source columns
target_suffix (str) – Suffix to append to field names for target columns

Returns:

Score matrix of shape (len(source), len(target))

Return type:

ScoreMatrix

Raises:

ValueError – If field not found or unknown comparator

setjoin.scorers.score_matrix(source, target, weights, comparators=None)[source]¶

Convenience function to compute a score matrix with simple configuration.

Parameters:

source (DataFrame) – Source DataFrame
target (DataFrame) – Target DataFrame
weights (dict[str, float]) – Mapping from field name to weight
comparators (dict[str, str] | None) – Optional mapping from field name to comparator name

Returns:

Score matrix of shape (len(source), len(target))

Return type:

ScoreMatrix

Hierarchy¶

Hierarchy specification and decomposition for structure-aware matching.

type setjoin.hierarchy.GroupMatchMap = dict[tuple[Hashable, Hashable], list[tuple[int, int]]]¶

class setjoin.hierarchy.HierarchySpec(source_groups, target_groups)[source]¶

Bases: object

Specification of hierarchical structure for record groups.

This defines how records are grouped (e.g., persons within households, students within schools, items within orders) for structure-preserving matching.

Parameters:

source_groups (Mapping[Hashable, Sequence[int]])
target_groups (Mapping[Hashable, Sequence[int]])

source_groups: Mapping[Hashable, Sequence[int]]¶: Mapping from source group ID to record indices.

target_groups: Mapping[Hashable, Sequence[int]]¶: Mapping from target group ID to record indices.

classmethod from_dataframe(source, target, source_group_col, target_group_col)[source]¶

Create HierarchySpec from DataFrames with group columns.

Parameters:

source (DataFrame) – Source DataFrame
target (DataFrame) – Target DataFrame
source_group_col (str) – Column name containing group IDs in source
target_group_col (str) – Column name containing group IDs in target

Returns:

Group mappings

Return type:

HierarchySpec

classmethod from_groupby(source_groupby, target_groupby)[source]¶

Create HierarchySpec from pandas GroupBy objects.

Parameters:

source_groupby (Any) – GroupBy object for source records
target_groupby (Any) – GroupBy object for target records

Returns:

Group mappings

Return type:

HierarchySpec

property n_source_groups: int¶: Number of groups in source.

property n_target_groups: int¶: Number of groups in target.

property source_group_ids: list[Hashable]¶: List of source group IDs.

property target_group_ids: list[Hashable]¶: List of target group IDs.

get_source_group(group_id)[source]¶

Get specification for a source group.

Parameters:: group_id (Hashable)
Return type:: GroupSpec

get_target_group(group_id)[source]¶

Get specification for a target group.

Parameters:: group_id (Hashable)
Return type:: GroupSpec

source_group_sizes()[source]¶

Get sizes of all source groups.

Return type:: dict[Hashable, int]

target_group_sizes()[source]¶

Get sizes of all target groups.

Return type:: dict[Hashable, int]

setjoin.hierarchy.compute_group_score_matrix(hierarchy, record_scores)[source]¶

Compute score matrix at the group level with optimal within-group assignments.

For each pair of (source_group, target_group), computes the optimal assignment of records within those groups and returns the total score.

Parameters:

hierarchy (HierarchySpec) – HierarchySpec defining group structure
record_scores (NDArray[float64]) – Pairwise record score matrix (shape: n_source_records x n_target_records)

Returns:

Group score matrix and within-group match assignments

Return type:

tuple[NDArray[np.float64], GroupMatchMap]

setjoin.hierarchy.decompose_by_size(hierarchy)[source]¶

Decompose groups by size for efficient matching.

Groups source and target groups by their sizes to enable size-aware matching strategies.

Parameters:: hierarchy (HierarchySpec) – HierarchySpec defining group structure
Returns:: Size pairs to group ID lists
Return type:: dict[tuple[int, int], tuple[list[Hashable], list[Hashable]]]

Diagnostics¶

Diagnostic metrics and report generation for match evaluation.

class setjoin.diagnostics.MatchReport(result, scores, ground_truth=None, source_groups=None, target_groups=None, source_df=None, target_df=None, _cache=<factory>)[source]¶

Bases: object

Comprehensive diagnostics for evaluating match quality.

Parameters:

result (MatchResult)
scores (NDArray[float64])
ground_truth (list[tuple[int, int]] | None)
source_groups (dict[Hashable, list[int]] | None)
target_groups (dict[Hashable, list[int]] | None)
source_df (DataFrame | None)
target_df (DataFrame | None)
_cache (dict[str, dict[int, Hashable]])

result: MatchResult¶: The match result being evaluated.

scores: NDArray[float64]¶: The score matrix used for matching.

ground_truth: list[tuple[int, int]] | None = None¶: Optional ground truth matches for accuracy computation.

source_groups: dict[Hashable, list[int]] | None = None¶: Optional source group structure for coherence metrics.

target_groups: dict[Hashable, list[int]] | None = None¶: Optional target group structure for coherence metrics.

source_df: DataFrame | None = None¶: Optional source DataFrame for detailed analysis.

target_df: DataFrame | None = None¶: Optional target DataFrame for detailed analysis.

property n_matches: int¶: Number of matches.

property record_accuracy: float | None¶: Fraction of ground-truth links recovered (record recall).

property group_accuracy: float | None¶

Fraction of records matched to the correct group.

Returns None unless both group maps and the ground truth are set.

property group_exact_match_rate: float | None¶

Fraction of complete source groups mapped entirely into one target group.

This measures group coherence, not whether the target group is the true one.

property score_sacrifice: float | None¶

Total score lost compared to unconstrained Hungarian matching.

Positive value means structure-aware matching sacrificed score for coherence.

match_confidence()[source]¶

Compute per-match confidence metrics.

Returns DataFrame with columns: - source_idx, target_idx: the match - score: score of this match - rank: rank of this match among alternatives for source - margin: difference between this score and best other target

Return type:: DataFrame

group_margins()[source]¶

Compute per-group score margins.

Returns DataFrame with columns: - source_group: source group ID - target_group: matched target group ID - within_score: total score of within-group matches - best_alternative_score: best score with a different target group - margin: difference between within_score and best_alternative

Return type:: DataFrame | None

error_anatomy()[source]¶

Analyze errors by comparing to ground truth.

Returns DataFrame with columns: - source_idx: source record - matched_target_idx: where we matched it - true_target_idx: where it should have matched - score_diff: how much score we lost - error_type: ‘swap’ or ‘mismatch’

Return type:: DataFrame | None

method_comparison(other)[source]¶

Compare this result with another method’s result.

Returns DataFrame showing where the methods differ.

Parameters:: other (MatchReport)
Return type:: DataFrame

to_csv(output_dir)[source]¶

Write all diagnostic artifacts to CSV files.

Creates: - match_confidence.csv - group_margins.csv (if groups provided) - error_anatomy.csv (if ground truth provided)

Parameters:: output_dir (str | Path)
Return type:: None

summary()[source]¶

Return a dictionary of summary metrics.

Return type:: dict[str, float | int | None]

setjoin.diagnostics.evaluate_matches(source, target, matches, source_id_col='latent_person_id', target_id_col='latent_person_id', source_group_col=None, target_group_col=None)[source]¶

Evaluate linkage quality with standard metrics.

Parameters:

source (DataFrame) – Source DataFrame
target (DataFrame) – Target DataFrame
matches (list[tuple[int, int]]) – List of (source_idx, target_idx) pairs
source_id_col (str) – Column in source containing true record ID
target_id_col (str) – Column in target containing true record ID
source_group_col (str | None) – Optional column for group coherence metrics
target_group_col (str | None) – Optional column for group coherence metrics

Returns:

Evaluation metrics

Return type:

dict[str, float]

Types¶

Type definitions and protocols for setjoin.

class setjoin.types.Comparator(*args, **kwargs)[source]¶

Bases: Protocol

Protocol for string/value comparison functions.

class setjoin.types.MatchResult(matches, total_score, method, group_assignments=None, metadata=<factory>)[source]¶

Bases: object

Result of a matching operation.

Parameters:

matches (list[tuple[int, int]])
total_score (float)
method (str)
group_assignments (dict[Hashable, Hashable] | None)
metadata (dict[str, object])

matches: list[tuple[int, int]]¶: List of (source_idx, target_idx) pairs.

total_score: float¶: Sum of scores for all matched pairs.

method: str¶: Name of the matching method used.

group_assignments: dict[Hashable, Hashable] | None¶: Mapping from source group ID to target group ID (structure-aware).

metadata: dict[str, object]¶: Additional metadata about the matching process.

to_dataframe()[source]¶

Convert matches to a DataFrame.

Return type:: DataFrame

class setjoin.types.FieldConfig(weight=1.0, comparator='exact', missing_value=0.0)[source]¶

Bases: object

Configuration for a single field in scoring.

Parameters:

weight (float)
comparator (str)
missing_value (float)

weight: float¶: Weight applied to this field’s similarity score.

comparator: str¶

exact, abs_diff, levenshtein, jaro_winkler.

Type:: Name of comparison function

missing_value: float¶: Score to use when either value is missing.

class setjoin.types.GroupSpec(group_id, indices)[source]¶

Bases: object

Specification of a group of records.

Parameters:

group_id (Hashable)
indices (Sequence[int])

group_id: Hashable¶: Identifier for this group.

indices: Sequence[int]¶: Record indices belonging to this group.

setjoin.types.ScoreMatrix¶

Type alias for score matrices (n_source x n_target).

alias of NDArray[float64]

Visualization¶

Visualization functions for match diagnostics (requires matplotlib).

setjoin.plots.plot_score_heatmap(scores, matches=None, ax=None, cmap='viridis', mark_matches=True)[source]¶

Plot score matrix as a heatmap with optional match overlay.

Parameters:

scores (NDArray[float64]) – Score matrix (n_source x n_target)
matches (list[tuple[int, int]] | None) – Optional list of matches to highlight
ax (Axes | None) – Optional matplotlib Axes to plot on
cmap (str) – Colormap name
mark_matches (bool) – Whether to mark matched pairs

Returns:

Matplotlib figure with heatmap

Return type:

Figure

setjoin.plots.plot_match_comparison(report1, report2, label1='Method 1', label2='Method 2', ax=None)[source]¶

Compare scores of matches between two methods.

Parameters:

report1 (MatchReport) – First method’s report
report2 (MatchReport) – Second method’s report
label1 (str) – Label for first method
label2 (str) – Label for second method
ax (Axes | None) – Optional matplotlib Axes

Returns:

Matplotlib figure with comparison

Return type:

Figure

setjoin.plots.plot_method_comparison_bar(results, metric='total_score', ax=None)[source]¶

Bar chart comparing a metric across methods.

Parameters:

results (dict[str, MatchResult]) – Dictionary of method name to MatchResult
metric (str) – Which metric to plot (“total_score” or “n_matches”)
ax (Axes | None) – Optional matplotlib Axes

Returns:

Matplotlib figure with bar chart

Return type:

Figure

Raises:

ValueError – If metric is unknown

setjoin.plots.plot_confidence_distribution(report, ax=None)[source]¶

Plot distribution of match confidence (margin to second-best).

Parameters:

report (MatchReport) – MatchReport to analyze
ax (Axes | None) – Optional matplotlib Axes

Returns:

Matplotlib figure with distribution

Return type:

Figure

setjoin.plots.plot_accuracy_by_ambiguity(data, methods, ambiguity_values, metric='record_accuracy', ax=None)[source]¶

Plot accuracy metric vs ambiguity level for multiple methods.

Parameters:

data (DataFrame | list[dict[str, float]]) – DataFrame or list of dicts with columns [method, ambiguity, <metric>]
methods (list[str]) – List of method names to plot
ambiguity_values (list[float]) – List of ambiguity values
metric (str) – Which metric to plot
ax (Axes | None) – Optional matplotlib Axes

Returns:

Matplotlib figure with accuracy plot

Return type:

Figure