SQL NOT IN with NULL: Why It Returns No Rows and How to Fix It
NOT IN returns no rows once its subquery yields a NULL, due to three-valued logic. Why, the NOT EXISTS fix, and the tree node interview question.
- Course: SQL study plan
- Module: Subqueries
- Kind: Lesson
- Reading time: 15 min
- Runtime: MySQL 8 syntax on SQLite 3.49
Why does NOT IN return no rows when the subquery contains NULL?
x NOT IN (a, b, NULL) means x <> a AND x <> b AND x <> NULL. Any comparison with NULL is unknown rather than true, and an AND containing an unknown can never be true, so WHERE keeps no row. A single NULL in the subquery's column therefore empties the result. NOT EXISTS does not have this problem.
Lesson
NOT IN reads like English — "customers whose id is not in the list of ordering customers" — and on clean test data it gives the right answer. Then one row with a NULL arrives in the column the subquery reads, and the query returns nothing at all, with no error. This is one of the most common SQL bugs in production and one of the most popular interview questions, because it tests whether you really understand NULL. This lesson works through why it happens, the related case of a NULL on the outer side, the three reliable alternatives, and a classic problem where the trap is built in.
The symptom
CREATE TABLE customers (id INT PRIMARY KEY, name VARCHAR(40));
CREATE TABLE orders (id INT PRIMARY KEY, customer_id INT);
INSERT INTO customers VALUES (1, 'Asha'), (2, 'Ben'), (3, 'Chen');
INSERT INTO orders VALUES (10, 1), (11, 2);
SELECT name FROM customers
WHERE id NOT IN (SELECT customer_id FROM orders);
| name |
|---|
| Chen |
Correct: Chen never ordered. Now a guest checkout is stored with no customer:
INSERT INTO orders VALUES (12, NULL);
-- the same query now returns:
| name |
|---|
| (no rows) |
Why: three-valued logic
SQL comparisons have three outcomes: true, false and unknown. Any comparison with NULL — 3 = NULL, 3 <> NULL, even NULL = NULL — is unknown, because NULL means "no value known", and nobody can say whether an unknown value differs from 3.
IN and NOT IN against a list expand into comparisons:
orders.customer_id = 1, 2, NULL — and Chen's id is 3- x IN (list) is shorthand for one equality test per value, joined by OR. For Chen, 3 IN (1, 2, NULL) becomes three comparisons.
- 3 = 1 and 3 = 2 are plainly FALSE. So far nothing matched.
- 3 = NULL is UNKNOWN: nobody can say whether a missing value equals 3. FALSE OR FALSE OR UNKNOWN is UNKNOWN — not FALSE.
- NOT IN negates it: an AND of <> tests. 3 <> 1 and 3 <> 2 are TRUE, but 3 <> NULL is UNKNOWN, and TRUE AND TRUE AND UNKNOWN is UNKNOWN.
- Run for every customer: Asha and Ben are FALSE (their id is in the list), Chen is UNKNOWN. WHERE keeps only TRUE, so the query returns no rows at all.
AND is true only if every part is true; one unknown part makes the whole unknown (unless another part is false, which makes it false). WHERE keeps only rows where the condition is true, so it discards unknown. For Chen, every row: discarded. For Asha, 1 <> 1 is false, so false — discarded as before. Nobody survives.
IN with a NULL in the list is mostly harmless: a match makes the OR true no matter what else is there. The non-matching rows are unknown rather than false, which you only notice once something negates them — and NOT IN is the negation.
NULL on the outer side
The trap has a mirror image. If the outer column is NULL:
-- products.category_id is NULL for unclassified products
SELECT name FROM products
WHERE category_id NOT IN (SELECT id FROM discontinued_categories);
For an unclassified product the test is NULL NOT IN (…), unknown whenever the list is non-empty — so the product is never returned, even though it is certainly not in a discontinued category. NOT EXISTS (… WHERE d.id = p.category_id) returns it, because no row can equal NULL and so no matching row exists. Neither behaviour is wrong in itself; they answer subtly different questions, and you should choose deliberately.
Three reliable anti joins
NOT EXISTS — the default choice:
SELECT c.name
FROM customers c
WHERE NOT EXISTS (SELECT 1 FROM orders o WHERE o.customer_id = c.id)
ORDER BY c.name;
It asks a yes-or-no question per customer — is there a matching order? — and a NULL customer_id simply never matches. No unknowns leak out.
NOT IN with the NULLs filtered out:
SELECT c.name
FROM customers c
WHERE c.id NOT IN (SELECT o.customer_id FROM orders o WHERE o.customer_id IS NOT NULL)
ORDER BY c.name;
Correct, but it relies on everyone remembering the filter every time. Declaring the column NOT NULL in the schema removes the risk at the source — when the data model allows it.
LEFT JOIN … IS NULL:
SELECT c.name
FROM customers c
LEFT JOIN orders o ON o.customer_id = c.id
WHERE o.id IS NULL
ORDER BY c.name;
Correct as long as the tested column is never NULL in a real match (the primary key is ideal).
All three agree on the outer-side NULL case too, for the same reason — they test for a matching row rather than compare against a list. MySQL 8.0.17+ plans NOT EXISTS and NOT IN as antijoins; PostgreSQL often plans NOT IN worse precisely because it must honour the NULL semantics. That is the second reason to default to NOT EXISTS.
The empty subquery
x NOT IN (empty) is true for every x, including NULL — an AND of zero comparisons is true. So "products not in any discontinued category" correctly returns every product when nothing is discontinued. The trouble starts only when the list is non-empty and contains a NULL (or the outer value is NULL).
A classic: classifying tree nodes
A table stores a tree as (id, p_id), with p_id NULL for the root. Label each node Root, Inner (has a parent and children) or Leaf (has a parent, no children). The tempting solution:
SELECT id,
CASE
WHEN p_id IS NULL THEN 'Root'
WHEN id IN (SELECT p_id FROM tree) THEN 'Inner'
ELSE 'Leaf'
END AS type
FROM tree
ORDER BY id;
This one is correct, and it is instructive to see why: IN with the root's NULL p_id in the list still finds the matches, and for leaves the unknown result just falls through to ELSE. Now flip the test, as many people do:
WHEN id NOT IN (SELECT p_id FROM tree) THEN 'Leaf'
ELSE 'Inner'
The root's NULL p_id is in the list, so NOT IN is never true, and every leaf is labelled Inner. Same data, same intent, one negation — wrong answer. Writing the child test as EXISTS (SELECT 1 FROM tree c WHERE c.p_id = t.id) makes both versions correct.
Rules to remember
NOT IN (subquery)is correct only if the subquery's column cannot be NULL and the outer value cannot be NULL.NOT EXISTSis correct regardless; use it by default for "has no match".<> ALL (subquery)is the same asNOT INand has the same trap.- The same three-valued logic explains why
WHERE col <> 'x'skips rows wherecolis NULL.
Pitfalls
NOT INon a nullable column — the query empties the day the first NULL arrives.- Forgetting the outer side: a NULL outer value is never returned by
NOT IN. - Assuming
NOT (x IN …)behaves differently — it is the same expression. - Negating a correct
INtest into an incorrectNOT INtest while refactoring. - Testing with data that has no NULLs and concluding the query is right.
Key takeaways
- A comparison with NULL is unknown;
WHEREkeeps only true. x NOT IN (…, NULL, …)is never true, so one NULL empties the result.NOT EXISTS(orLEFT JOIN … IS NULL) is the safe anti join;NOT INneedsWHERE col IS NOT NULLinside.INsurvives NULLs for matching; its negation does not.NOT INof an empty subquery is true for every row.
Common questions
How do I fix NOT IN with NULL values?
Prefer NOT EXISTS (SELECT 1 FROM t WHERE t.col = outer.col), which asks only whether a matching row exists. If you keep NOT IN, filter the NULLs out inside it: NOT IN (SELECT col FROM t WHERE col IS NOT NULL). A LEFT JOIN … WHERE t.col IS NULL anti join also works.
Does IN have the same problem with NULL?
Not for finding matches. x IN (1, NULL) is true when x is 1, because one true comparison makes an OR true. It is unknown, not false, when x matches nothing, which only matters once the result is negated, and negation is exactly what NOT IN does.
What happens when the outer value itself is NULL?
NULL NOT IN (1, 2) is unknown, so a row whose own column is NULL is never returned by NOT IN, while NOT EXISTS returns it, because no row can equal NULL. The two forms therefore disagree on NULLs in either column; decide which behaviour the question wants.
What does NOT IN return for an empty subquery?
True for every row, including a row whose value is NULL, because there is nothing to differ from: an AND of zero comparisons is true. NOT EXISTS also keeps every row when the subquery finds nothing, so for an empty subquery the two agree.
Exercises
Employees who manage nobody
List the employees who are nobody's manager. The top of the hierarchy has a NULL manager_id — so id NOT IN (SELECT manager_id FROM employees) returns nothing here.
Table: employees(id, name, manager_id).
| id | name | manager_id |
|---|---|---|
| 1 | Meera | NULL |
| 2 | Arjun | 1 |
| 3 | Kiran | 2 |
| 4 | Leela | 2 |
| 5 | Omar | 1 |
Output: columns id, name, ordered by id.
id | name
3 | Kiran
4 | Leela
5 | OmarClassify the tree nodes
A tree is stored as tree(id, p_id), where p_id is the parent's id and NULL for the root. Label every node: Root if it has no parent, Inner if it has a parent and at least one child, Leaf otherwise.
Table: tree(id, p_id).
| id | p_id |
|---|---|
| 1 | NULL |
| 2 | 1 |
| 3 | 1 |
| 4 | 2 |
| 5 | 2 |
Output: columns id, type, ordered by id.
id | type
1 | Root
2 | Inner
3 | Leaf
4 | Leaf
5 | LeafIn this module: Subqueries
← Derived tables — a subquery in FROM · Checkpoint — Subqueries →