12.14 Common Algorithms

Remove Duplicates

The single most common use of a set — convert to a set and back to a list. If insertion order must be preserved instead, dict.fromkeys() is the standard alternative, since dicts have preserved insertion order since Python 3.7.

def remove_duplicates(lst):
    return list(set(lst))

>>> sorted(remove_duplicates([1, 2, 2, 3, 1]))
[1, 2, 3]

Unique Elements

def unique_elements(lst):
    return set(lst)

>>> unique_elements([1, 2, 2, 3])
{1, 2, 3}

Common Items

Finding overlap between two lists — convert both to sets and intersect.

>>> a, b = [1, 2, 3, 4], [3, 4, 5, 6]
>>> set(a) & set(b)
{3, 4}

Difference Between Lists

>>> set(a) - set(b)     # in a but not b
{1, 2}

Quick Interview Answer

“Four problems reduce to a one-liner once a list is converted to a set: deduplicating (list(set(lst)), or dict.fromkeys(lst) if order needs to be kept), finding unique elements (set(lst) directly), finding what two lists have in common (set(a) & set(b)), and finding what’s only in one (set(a) - set(b)). The pattern behind all four is the same — the moment ‘does this exist across both collections’ becomes the question, converting to sets and using the operators from 12.7 Set Operators is both shorter and asymptotically faster than nested loops.”

Common Mistakes

• Deduplicating with list(set(lst)) when the original order matters — a set discards order entirely; use dict.fromkeys(lst) (then wrap in list(...) if a list is needed) to deduplicate while preserving first-seen order.
• Solving “common items between two lists” with a nested loop (for x in a: for y in b: ...) — that’s O(n × m); set(a) & set(b) is O(n + m) and far more readable.
• Forgetting set(a) - set(b) is directional — it answers “what’s in a but not b,” not the reverse; swap the operands for the other direction.