Create typed distance and similarity matrices

precise_dist() is the breadth engine of PreciseDist. It takes one numeric data set, evaluates one or more registered relatedness measures, and returns a typed tibble: each row contains the matrix, the canonical metric name, and the matrix type. Every requested metric is accounted for in a run ledger attached as the "precise_dist_run" attribute (see precise_run_report): metrics that error, time out, are skipped by a time budget, or are unavailable never vanish silently — they are absent from the returned rows but present in the ledger with a status, reason, and error message, and a consolidated warning summarizes them.

Usage

precise_dist(
  data,
  dists = NULL,
  dist_funcs = NULL,
  suffix = "",
  file = NULL,
  overwrite = FALSE,
  parallel = FALSE,
  plan_only = FALSE,
  batch_size = NULL,
  max_memory = NULL,
  max_time_per_metric = Inf,
  max_time_total = Inf,
  seed = NULL,
  verbose = TRUE,
  coerce = "none",
  local_timeout = NULL
)

Arguments

data: A numeric matrix, or a dataframe/tibble with at least one numeric column (non-numeric columns are dropped with a message).
dists: NULL or a character vector of registered metric names, aliases, or family keywords. Use precise_dist_list("all_dists") to inspect choices. Family keywords ending in "_dists" and their short aliases are accepted; for example, "static" is equivalent to "static_dists". Requested metrics whose backend package is not installed are recorded as unavailable in the run ledger (unknown names are still an error).
dist_funcs: NULL or a list of custom distance functions. Functions produced by precise_func_fact are typed by the factory; arbitrary user functions are returned with type = NA. Unnamed elements are keyed user_func_<i>.
suffix: A string prepended to every output distance key (both channels).
file: NULL or the path for the RDS checkpoint/result envelope; read it back with precise_read. See Details.
overwrite: TRUE or FALSE. Allow file to replace an existing path.
parallel: TRUE or FALSE. If TRUE, metric batches run via future.apply::future_lapply() on the user's future::plan(); requires the future.apply package.
plan_only: TRUE or FALSE. Return the plan ledger without computing. See Details.
batch_size: NULL (auto) or a positive integer: metrics per master-side batch. See Details.
max_memory: NULL or a positive number: your memory budget in GB for the estimated peak result footprint; the run stops before computing if the estimate exceeds it.
max_time_per_metric: Seconds allowed per metric (best-effort; default Inf).
max_time_total: Seconds allowed for the whole run, checked between batches (default Inf).
seed: NULL or a whole number from 0 through .Machine$integer.max. See Details (Reproducibility).
verbose: TRUE or FALSE. Narrate progress via message() (suppressable with suppressMessages()).
coerce: A string controlling optional coercion of the returned matrices. "none" (the default) returns each measure's native output, tagged by the type column (tag-and-leave). "distance" delegates to precise_transform(to = "distance") to coerce every row to a distance (requires an all-convertible set; see precise_transform). Any other value errors.
local_timeout: Deprecated alias for max_time_per_metric; supplying both is an error.

Value

A tibble with columns: distance = the display name of the measure (the requested name, carrying any suffix), metric = the canonical registry key the row resolves to (NA for custom dist_funcs), matrix = the nested matrix corresponding to the row, type = the measure's output type ("distance", "similarity", "correlation" or "affinity"; NA for unknown custom dist_funcs) and time_taken_seconds = The amount of time in seconds it took to calculate each matrix. Rows appear in request order and contain only completed metrics; the full per-metric accounting (completed / errored / skipped / unavailable, with reasons and messages) is the tibble's "precise_dist_run" attribute, accessible via precise_run_report. If nothing completed, the tibble has zero rows but the ledger is complete. With coerce = "distance", the returned object also carries the history list-column produced by precise_transform.

Details

Depending on dists or dist_funcs, the native output may be a distance, similarity, correlation, affinity, or a custom row with unknown type. Factory functions created by precise_func_fact carry their native type; arbitrary user functions remain unknown unless the user handles them explicitly. The function tags the native result and leaves conversion to precise_transform. It does not infer type from matrix values.

Parallelism. parallel = TRUE dispatches each batch of metrics via future.apply::future_lapply() on whatever future::plan() the user has set; parallel = FALSE (the default) is a plain lapply() and touches no future machinery. Worker count, backend, and exported-memory limits are user decisions, made outside PreciseDist.

Reproducibility (seed). With seed = NULL (default), sequential runs use the ordinary session RNG stream. Parallel runs use future.seed = TRUE (statistically sound parallel streams, no reproducibility promise across plans) and advance ordinary session RNG state even when the requested metrics are deterministic. With a whole-number seed, every metric draws from its own RNG stream derived from (seed, metric key) — results are then identical across sequential and parallel runs and stable when other metrics are added to or removed from the request; the session RNG state is saved and restored around the call. Seeded results are reproducible for a given R version and platform endianness. Stochastic metrics include the random_forest_* family and the kernel metrics (rbf_*, laplace_*, whose sigma estimation consumes randomness).

Input domains. Registry input classes with objective value-domain rules (nonnegative, probability, unit_interval, and binary) are validated before dispatch. An incompatible metric receives an errored/input_domain ledger row while compatible metrics continue; input data are never normalized or otherwise changed silently. numeric, nominal, and time-series remain descriptive because they have no general value-only validation rule.

Time limits. max_time_per_metric applies setTimeLimit() around each metric; this is best-effort — it cannot interrupt long compiled calls (randomForest, kernlab, philentropy) or blocking waits. A metric over the limit is recorded as errored with reason "timeout". The package clears its limit transiently, preserving still-live caller limits after metric execution; interactions between nested limits while a metric is running remain best-effort. max_time_total is checked between batches; once exceeded, remaining metrics are recorded as skipped with reason "time_budget" and the completed rows are returned.

Memory (max_memory, in GB). Before dispatch the run computes a resource estimate: each dense result is exactly 8 * nrow(data)^2 bytes. The estimate covers the input/result footprint only — it is not a guarantee about worker-side transients or the environments carried by user-supplied dist_funcs closures. If the estimated peak exceeds max_memory, the run stops before computing anything (your own budget). If no max_memory is set and the estimated result footprint exceeds getOption("PreciseDist.large_run_gb", 8) GB, a warning is emitted and the run proceeds — the package never refuses on its own.

Planning (plan_only). plan_only = TRUE performs validation, metric resolution, and the resource estimate, then returns the run ledger (statuses "planned"/"unavailable", plus errored/input_domain for incompatible requested metrics) without computing anything or touching the filesystem. Attributes: n_tasks, est_result_gb, est_peak_gb, batch_size, n_batches, would_block_memory.

Checkpointing (file). file= writes a versioned RDS envelope, rewritten atomically by the master process after every batch (workers never touch it, so file= combines freely with parallel = TRUE). While running the envelope holds state "partial" (completed native matrices + the ledger); on normal return it is rewritten once as state "complete", holding the exact returned object. Read either with precise_read. An existing path errors unless overwrite = TRUE. Each checkpoint rewrites the accumulated matrices; increasing batch_size reduces write amplification at the cost of a larger recovery window. On interrupt (Ctrl-C) the run checkpoints what it has, then re-signals the interrupt — with no file=, interrupted work is lost. If a parallel worker dies, the batch is recorded as worker_crash, the envelope is checkpointed, and the run stops with an error carrying the ledger (class "precise_dist_crash").

Progress. If the progressr package is installed, each metric signals one progress step; enable display with progressr::with_progress() (progressr is inactive in non-interactive sessions unless you enable it).

Batching (batch_size). Metrics are dispatched in master-side batches; batch boundaries are where checkpoints are written and the total time budget is checked. NULL picks this automatically (worker count when parallel with a file/time budget; 1 when sequential with a file; otherwise one batch). Within a batch, scheduling across workers belongs to future.

Other details:

Use precise_dist_list to inspect available registered measures.
Use pd_convertible_to_distance and precise_transform before downstream methods that require a single coherent type.
suffix is useful when combining relatedness matrices from multiple feature sets that share the same observations. It is applied to both dists and dist_funcs display keys.
Duplicate metrics after alias/family expansion are collapsed with a message. A dists/dist_funcs name collision (identical display key) is an error: rename the custom function or use suffix.
The run ledger attribute does not survive most data-frame operations (dplyr verbs, subsetting); the file= envelope and the consolidated warning are the durable records.

References

Muchmore, B., Muchmore P. and Alarcón-Riquelme ME. (2018). Optimal Distance Matrix Construction with PreciseDist and PreciseGraph.

Author

Brian Muchmore

Examples

x <- matrix(
  c(1, 0, 2, 0, 1, 2, 1, 1, 0, 2, 1, 1, 1, 2, 3, 3, 1, 2),
  nrow = 6,
  byrow = TRUE
)
rownames(x) <- paste0("item_", seq_len(nrow(x)))

result <- precise_dist(
  x,
  dists = c("euclidean", "manhattan"),
  verbose = FALSE
)
result[c("distance", "metric", "type")]
#> # A tibble: 2 × 3
#>   distance  metric    type    
#>   <chr>     <chr>     <chr>   
#> 1 euclidean euclidean distance
#> 2 manhattan manhattan distance
precise_run_report(result)
#> # A tibble: 2 × 7
#>   distance  metric    type     status    reason message time_taken_seconds
#>   <chr>     <chr>     <chr>    <chr>     <chr>  <chr>                <dbl>
#> 1 euclidean euclidean distance completed NA     NA                0.001000
#> 2 manhattan manhattan distance completed NA     NA                0