What is deduplication in an Address Management System and what approach would you use?

What is deduplication in an Address Management System and what approach would you use?

• A. Deduplication removes duplicate address records using exact-match checks and fuzzy similarity scoring (Levenshtein/Jaccard), phonetic matching, and rule-based checks; combine results with a human review workflow.
• B. Deduplication means merging all addresses into a single canonical address automatically with no human intervention.
• C. Deduplication only uses exact-match checks.
• D. Deduplication is unnecessary if the data store enforces uniqueness on address_id.

Answer: (A)

Deduplication in an Address Management System aims to identify when different records actually point to the same address, even if there are variations in spelling, formatting, or data quality. The strongest approach combines several signals: exact-match checks for precise duplicates, fuzzy similarity scoring to catch near-duplicates, phonetic matching to recognize variants that sound alike, and rule-based checks to enforce standardization and business rules (like common abbreviations and postal formats). Because each signal has strengths and blind spots, mixing them provides a more robust detection method than any single technique alone. After the automated matching, a human review workflow is used to verify ambiguous cases and decide on the canonical record, ensuring that merges don’t accidentally fuse distinct addresses or overlook subtle differences.

This approach is preferable because it handles real-world data messiness—typos, abbreviations, swapped order of address components, and varying validation rules—while maintaining accuracy through human oversight. Automatic merging without review can lead to incorrect consolidations, and relying only on exact matches misses many duplicates. Merely enforcing a unique address_id doesn’t prevent duplicates at the content level, since the same physical address can exist with different IDs or formatting.