Shape · Matching and dedupe

Blocking

Finds candidate pairs of records that share a key, such as the same postcode, within one table or across two. It narrows down the pairs before fuzzy matching.

Also known as: Record linkage, candidate pairs

How it works

Id column must be unique per input; with a second table wired into right you get a Left id column and a Right id column (same name as the left by default). Each blocking rule is either a Column, compared on both sides, or an Expression, a SQL expression evaluated on both sides, such as lower(substr(last_name, 1, 3)); two records pair when the key is equal and not NULL. With one input, each unordered pair appears once with id_a < id_b. The output has id_a, id_b, blocking_rule (the first rule that matched, by its name) and every input column prefixed a_ and b_. Each rule is a hash join, so the cost is the size of the blocks, never every row against every other.

Inputs and outputs

Inputs

  • left — Table
  • right — Table

Outputs

  • data — Table

Settings

The Attributes panel of Blocking, as in the app: the controls work, but nothing runs — there is no data behind it.

The same settings, as the Assistant writes them
{
  idColumn: string
  rightIdColumn?: string
  rules: [{ name: string, kind: "column"|"expression", leftColumn?: string, rightColumn?: string, expression?: string }]
}

id columns must be unique per input (NULL ids are allowed but never pair); one input = pairs within the table (id_a < id_b); wire `right` for pairs across two tables; a rule matches rows whose key (a column, or a DuckDB SQL expression evaluated on both sides, e.g. "lower(substr(last_name, 1, 3))") is equal and non-NULL; emits id_a, id_b, blocking_rule (first matching rule) + every input column prefixed a_ / b_; hash-joined per rule, never all-pairs

Example

Candidate pairs in a customer table: id column customer_id, rule postcode as a column and rule name3 as the expression lower(substr(last_name, 1, 3)). Score the pairs with Fuzzy Match, then cluster them with Match Grouping.