Skip to contents

Deduplicate proposed ontology terms

Source: R/term-deduplication.R
deduplicate_proposed_terms.Rd

Applies I-ADOPT compositional deduplication to a proposed-terms dataframe. This prevents term proliferation by:

  1. Removing duplicates across tables (same term_label)

  2. Collapsing age-stratified variants (X Age 1..7) into one base term

  3. Collapsing phase-stratified variants (Ocean/Terminal/Mainstem X) into one base term

  4. Identifying terms that should use constraint_iri instead of new term_iri

Usage

deduplicate_proposed_terms(proposed_terms, warn_threshold = 30L)

Arguments

proposed_terms

A data frame with columns: term_label, term_definition, term_type, suggested_parent_iri. Typically loaded from a proposed-terms CSV.

warn_threshold

Integer. If the input has more than this many rows, issue a warning about potential over-engineering. Default is 30.

Value

A tibble with deduplicated terms and additional columns:

  • is_base_term: TRUE if this is the canonical base term for a pattern

  • needs_age_facet: TRUE if age variants should use constraint_iri

  • needs_phase_facet: TRUE if phase variants should use constraint_iri

  • collapsed_from: Count of how many variants were collapsed into this term

  • dedup_notes: Explanation of deduplication applied

Details

The function returns a deduplicated dataframe with added columns for facet handling.

Target ratio: For a dictionary with N measurement columns, expect ~N/10 to N/5 distinct base terms, NOT N terms. If the output still has >30 rows, consider further manual review.

Anti-patterns detected:

  • "Spawners Age 1", "Spawners Age 2", ... patterns -> collapsed to "SpawnerCount"

  • Duplicate term_labels across different tables -> deduplicated

  • Phase-stratified variants (Ocean X, Terminal X) -> collapsed to base term

Examples

proposed <- tibble::tibble(
  term_label = c("Spawner Age 1", "Spawner Age 2", "Spawner Age 3"),
  term_definition = rep("Spawner count by age", 3),
  term_type = rep("owl_class", 3),
  suggested_parent_iri = rep("", 3)
)
deduped <- deduplicate_proposed_terms(proposed)
#> ✔ Deduplicated 3 -> 1 terms (2 collapsed/removed).
#> ℹ Review 'dedup_notes' column for facet handling guidance.
deduped |> dplyr::filter(collapsed_from > 1)
#> # A tibble: 1 × 10
#>   term_label    term_definition      term_type suggested_parent_iri is_base_term
#>   <chr>         <chr>                <chr>     <chr>                <lgl>       
#> 1 Spawner Age 1 Spawner count by age owl_class ""                   TRUE        
#> # ℹ 5 more variables: needs_age_facet <lgl>, needs_phase_facet <lgl>,
#> #   collapsed_from <int>, dedup_notes <chr>, label_pattern <chr>