Feature Parity Ledger Comparing Two Derivations — compare_feature

Quantifies standardized mean differences (SMD), correlation, missingness gaps, and variance ratios to validate a new OMOP feature pipeline against an older one.

Usage

compare_feature_tables(
  left,
  right,
  keys = NULL,
  by = NULL,
  tolerances = NULL,
  smd_threshold = 0.1,
  missingness_gap_threshold = 0.05,
  scale_ratio_threshold = 5,
  min_cell_count = 11L,
  left_manifest = NULL,
  right_manifest = NULL,
  stream_to_parquet = NULL,
  partition_by = NULL,
  to_arrow_dataset = FALSE
)

Arguments

left: First feature matrix data frame.
right: Second feature matrix data frame.
keys: Key column or vector of column names to join on.
by: Optional site column to stratify parity ledger by (default NULL).
tolerances: Optional named list of custom per-feature SMD thresholds.
smd_threshold: Standardized mean difference threshold flagging discrepancy (default 0.10).
missingness_gap_threshold: Maximum allowable missingness gap (default 0.05 = 5 percent).
scale_ratio_threshold: Maximum allowable ratio of standard deviations (default 5.0).
min_cell_count: Cell count threshold for privacy suppression (default 11).
left_manifest: Optional companion manifest for left matrix.
right_manifest: Optional companion manifest for right matrix.
stream_to_parquet: Optional path to stream results to Parquet file(s).
partition_by: Optional column name to partition Parquet outputs by.
to_arrow_dataset: Logical; if TRUE, returns an Arrow Dataset object.

Value

A data frame with columns: site, feature, matched_n, left_mean, right_mean, left_sd, right_sd, smd, correlation, left_missing_rate, right_missing_rate, missingness_gap, flagged, reason.