Skip to contents

Sparse concept matrix as tidymodels-ready data

Source: R/tidymodels_bridge.R
as_tidymodels_data.Rd

Turns an extract_sparse_concept_matrix() result into a tibble that works with tidymodels (rsample, recipes, parsnip, workflows, tune, yardstick):

Usage

as_tidymodels_data(
  sparse_result,
  event = "yes",
  non_event = "no",
  sparse = TRUE
)

Arguments

sparse_result

A result of extract_sparse_concept_matrix().

event, non_event

Labels for the outcome factor (event first).

sparse

Use sparse predictor columns when sparsevctrs is available (default TRUE).

Value

A tibble. The attribute "concepts" maps each predictor name (feature) to its token, domain, concept id and concept name.

Details

  • y: the outcome as a factor with the event as the first level, which is what yardstick and tidymodels assume (event_level = "first"). Omitted if the cohort has no outcome (e.g. when preparing new patients for scoring).

  • person_id, index_date: identifiers. Give them the "id" role in a recipe, and resample with rsample::group_vfold_cv(data, group = person_id) / group_initial_split(): a patient can have several index events, and ordinary row-wise resampling would put one patient in both training and assessment data (leakage).

  • one numeric predictor per concept, named <domain>_<concept_id> (for example condition_201826): syntactically valid, stable across cohorts, and the same names materialize_concept_features() creates in DuckDB, so a model trained here can be scored there.

With the sparsevctrs package installed (and sparse = TRUE) the predictors are sparse columns, so high-dimensional concept matrices stay memory-light and tidymodels can hand them to sparse-aware engines such as glmnet. Otherwise they are dense.

Examples

if (FALSE) { # \dontrun{
res <- extract_sparse_concept_matrix(con, "cohort.parquet", min_patient_freq = 50, value = "binary")
dat <- as_tidymodels_data(res)
split <- rsample::group_initial_split(dat, group = person_id)
rec <- recipes::recipe(y ~ ., data = rsample::training(split)) |>
  recipes::update_role(person_id, index_date, new_role = "id")
spec <- parsnip::logistic_reg(penalty = 0.01, mixture = 1) |> parsnip::set_engine("glmnet")
fit <- workflows::workflow(rec, spec) |> parsnip::fit(rsample::training(split))
} # }