Build many relatedness matrices from one data set

Computes registered and user-supplied relatedness measures on one numeric data set. Completed measures are returned as typed matrices, and every request is recorded in the "precise_dist_run" ledger attribute.

Usage

precise_dist(
  data,
  dists = NULL,
  dist_funcs = NULL,
  suffix = "",
  file = NULL,
  overwrite = FALSE,
  parallel = FALSE,
  plan_only = FALSE,
  batch_size = NULL,
  max_memory = NULL,
  max_time_per_metric = Inf,
  max_time_total = Inf,
  seed = NULL,
  verbose = TRUE,
  coerce = "none",
  local_timeout = NULL
)

Arguments

data: A numeric matrix, or a data frame or tibble with at least one numeric column. Non-numeric columns are dropped with a message.
dists: NULL or a character vector of registered measure names, aliases, or family keywords. See precise_dist_list().
dist_funcs: NULL or a list of functions taking the data matrix and returning a square matrix. Functions from precise_func_fact() carry a type. Other functions are returned with type = NA. Unnamed elements are keyed user_func_<i>.
suffix: A string prepended to every output key, from both dists and dist_funcs. Duplicate resolved keys are an error.
file: NULL or a path for a versioned RDS envelope. The master process writes a partial checkpoint after each batch and a complete result on return. An interrupt checkpoints and re-signals; a worker crash checkpoints and raises "precise_dist_crash". Read either envelope state with precise_read().
overwrite: TRUE or FALSE. Allow file to replace an existing path.
parallel: TRUE or FALSE. Run each batch on the active future::plan() through future.apply::future_lapply(). Requires the future.apply package. Not a core count.
plan_only: TRUE or FALSE. Validate and return the plan ledger without computing matrices or writing file. The ledger carries n_tasks, est_result_gb, est_peak_gb, batch_size, n_batches, and would_block_memory attributes.
batch_size: NULL for automatic, or a positive whole number of measures per batch. Automatic uses the worker count when parallel with file or a total time limit, 1 when sequential with file, and one batch otherwise.
max_memory: NULL or a positive number of gigabytes. The estimate covers the input and dense results, but not worker transients or captured function environments. The run stops before computing when the estimated peak exceeds this value. With NULL, an estimated result footprint above getOption("PreciseDist.large_run_gb", 8) warns and proceeds.
max_time_per_metric: Seconds allowed for one measure. The default is Inf. Enforcement uses setTimeLimit() and cannot reliably interrupt long compiled calls or blocking waits. A timeout is an errored ledger row with reason timeout.
max_time_total: Seconds allowed for the whole run, checked between batches. Remaining requests become time_budget skips.
seed: NULL or a whole number from 0 through .Machine$integer.max. NULL uses the session stream sequentially and future.seed = TRUE in parallel. A number gives each measure a key-derived stream, produces the same result sequentially and in parallel, and restores the caller's random-number state.
verbose: TRUE or FALSE. Report progress with message(). Use suppressMessages() to silence these messages.
coerce: "none", the default, returns each measure's native output tagged by the type column. "distance" hands the result to precise_transform(to = "distance"), which requires every row to have a defined conversion. Any other value is an error.
local_timeout: Deprecated name for max_time_per_metric. Supplying both is an error.

Value

A tibble with one row per completed measure, in request order, and the columns distance (the display key, carrying any suffix), metric (the canonical registry name, NA for custom functions), matrix (a list-column of matrices), type ("distance", "similarity", "correlation", "affinity", or NA for an untyped custom function), and time_taken_seconds. With coerce = "distance" the result also carries the history list-column from precise_transform(). The "precise_dist_run" attribute holds the full accounting for every requested measure. If nothing completed, the tibble has no rows but the ledger is still complete. With plan_only = TRUE the return value is the plan ledger described under plan_only.

Details

dists accepts canonical registry names, aliases, and family keywords, all listed by precise_dist_list(). Family keywords work with or without their _dists ending. Unknown names are errors. Unavailable backends are recorded without stopping compatible requests, and duplicate expansions are collapsed with a message.

Measures whose registry input class is nonnegative, probability, unit_interval, or binary are validated against the data before dispatch. Incompatible measures receive an input_domain error in the ledger while compatible measures continue. Input data are not transformed to satisfy a measure. Output type comes from the registry or a function created by precise_func_fact() and is never inferred from matrix values; other custom functions have type = NA.

The ledger records completed, errored, skipped, unavailable, planned, and pending requests. After an execution run, incomplete requests produce one consolidated warning naming those requests and their statuses. See precise_run_report() for the ledger schema and vignette("A-Parallel-Future", package = "PreciseDist") for parallel setup, resource planning, checkpoint recovery, and progress handling.

Author

Brian Muchmore

Examples

data(data_cell_cycle, package = "PreciseDist")

# The bundled cells are stored in label order, so take four from each block.
cells <- c(1:4, 60:63, 118:121)
cell_cycle <- as.matrix(data_cell_cycle[cells, 2:41])
rownames(cell_cycle) <- paste0(data_cell_cycle$Cell_cycle[cells], "_", cells)

measures <- precise_dist(
  cell_cycle,
  dists = c("euclidean", "manhattan", "canberra", "cosine"),
  verbose = FALSE
)
measures[, c("distance", "metric", "type")]
#> # A tibble: 4 × 3
#>   distance  metric    type      
#>   <chr>     <chr>     <chr>     
#> 1 euclidean euclidean distance  
#> 2 manhattan manhattan distance  
#> 3 canberra  canberra  distance  
#> 4 cosine    cosine    similarity

precise_run_report(measures)[, c("distance", "status", "reason")]
#> # A tibble: 4 × 3
#>   distance  status    reason
#>   <chr>     <chr>     <chr> 
#> 1 euclidean completed NA    
#> 2 manhattan completed NA    
#> 3 canberra  completed NA    
#> 4 cosine    completed NA