AI InterviewSearch…Search problems, topics, or pages…Ctrl KGet started
Skip to content

09 Set operations and CTEs / Lesson 4 of 4

Recursive CTEs — hierarchies, paths and number series

16 min · +10 XP · 2 exercises · 5 questions

Read the text16 min
Exercises0/2
Quiz5 q

Quick answer

How does a recursive CTE work in SQL?

A recursive CTE, written WITH RECURSIVE, has an anchor query and a recursive query joined by UNION ALL. The anchor produces the starting rows; the recursive query reads only the rows produced in the previous round and produces the next ones; the engine repeats until a round returns no rows. The union of every round is the result. It is how SQL walks trees such as org charts and generates series.

Some data refers to itself. An employee row has a manager_id that is another employee's id; a category has a parent_id that is another category; a flight connects to more flights. Ordinary SQL can follow such a link a fixed number of times — one self-join per level — but not "as many times as the data needs". A recursive CTE can. It repeats a query over its own output until nothing new appears, which is how SQL walks an org chart of unknown depth, builds the full path of a category, finds every part inside an assembly, or simply generates the numbers 1 to 100 or every date in a month. This lesson covers how the repetition works, the classic hierarchy queries, building paths, generating series, and how to keep recursion bounded.

The shape

WITH RECURSIVE nums AS (
  SELECT 1 AS n                 -- anchor member
  UNION ALL
  SELECT n + 1 FROM nums        -- recursive member
  WHERE n < 5
)
SELECT n FROM nums ORDER BY n;
n
1
2
3
4
5

Evaluation goes in rounds:

  1. The anchor runs once. Its rows (n = 1) are the first round, and they go into the result.
  2. The recursive member runs with nums meaning only the previous round's rows. From n = 1 it makes n = 2 — round two.
  3. Step 2 repeats on each new round: 2 makes 3, 3 makes 4, 4 makes 5. From 5, WHERE n < 5 filters everything out — the round is empty and recursion stops.
  4. The result is the union of all rounds.

In MySQL the word RECURSIVE is required if any CTE in the WITH list refers to itself (PostgreSQL requires it too; SQL Server and Oracle have no keyword and detect self-reference). The two members are joined with UNION ALL; plain UNION is allowed and removes rows already produced, which can stop a cycle but costs a duplicate check every round.

The recursive member may not use aggregates, window functions, GROUP BY, ORDER BY, DISTINCT or LIMIT in MySQL (LIMIT is allowed from 8.0.19), and must refer to the CTE exactly once, in its FROM clause — not in a subquery. All of these exist to keep the "one round at a time" model well defined. Aggregate in the final SELECT instead.

Walking down an org chart

employees

idnamemanager_id
1MeeraNULL
2Arjun1
3Divya1
4Kabir2
5Leela4

Everyone under the CEO, with their level:

WITH RECURSIVE org AS (
  SELECT id, name, 0 AS depth
  FROM employees
  WHERE manager_id IS NULL                  -- the top
  UNION ALL
  SELECT e.id, e.name, org.depth + 1
  FROM employees e
  JOIN org ON e.manager_id = org.id         -- the previous level's people are managers
)
SELECT name, depth FROM org ORDER BY depth, name;
WITH RECURSIVE, one round at a time
anchor: manager_id IS NULL · recursive: JOIN org ON e.manager_id = org.id

The anchor runs once: the employee whose manager_id IS NULL. Meera is round one, at depth 0, and goes into the result.

0:01 / 0:16Step 1 of 5
Transcript · 5 steps

Round one is Meera. Round two joins employees whose manager_id is Meera's id: Arjun and Divya. Round three finds Kabir under Arjun; round four Leela under Kabir; round five finds no one under Leela and stops. The join is the stopping condition: leaves have no children.

To start lower — everyone under Arjun — change only the anchor: WHERE id = 2. To include only the subordinates and not Arjun himself, anchor on WHERE manager_id = 2 with depth 1.

Walking up: a person's chain of command

Reverse the join to climb from one employee to the top:

WITH RECURSIVE chain AS (
  SELECT id, name, manager_id, 0 AS steps FROM employees WHERE id = 5
  UNION ALL
  SELECT m.id, m.name, m.manager_id, chain.steps + 1
  FROM employees m
  JOIN chain ON m.id = chain.manager_id
)
SELECT name, steps FROM chain ORDER BY steps;

This returns Leela, Kabir, Arjun, Meera — and stops when Meera's manager_id is NULL, since m.id = NULL matches nothing.

Building a path

To show where a node sits — Electronics > Phones > Android — carry a string column and extend it each round:

WITH RECURSIVE tree AS (
  SELECT id, name, CAST(name AS CHAR(200)) AS path
  FROM categories
  WHERE parent_id IS NULL
  UNION ALL
  SELECT c.id, c.name, CONCAT(t.path, ' > ', c.name)
  FROM categories c
  JOIN tree t ON c.parent_id = t.id
)
SELECT path FROM tree ORDER BY path;

The CAST matters in MySQL. The anchor decides each column's type: without it, path would be typed as wide as the longest root name, and a longer path built later is truncated (or rejected with "Data too long for column" in strict mode). Casting to CHAR(200) — or any width that fits the deepest path — avoids it. PostgreSQL has a similar rule and wants the types of both members to match exactly. Sorting by the path puts every child right after its parent, which displays a tree as an indented list.

Generating series

A recursive CTE with no table at all is a generator. Numbers:

WITH RECURSIVE seq AS (
  SELECT 1 AS n UNION ALL SELECT n + 1 FROM seq WHERE n < 100
)
SELECT n FROM seq;

Dates — the standard fix for reports with missing days:

WITH RECURSIVE days AS (
  SELECT '2024-03-01' AS day
  UNION ALL
  SELECT DATE_ADD(day, INTERVAL 1 DAY) FROM days WHERE day < '2024-03-07'
)
SELECT d.day, COALESCE(SUM(s.amount), 0) AS revenue
FROM days d
LEFT JOIN sales s ON s.sale_date = d.day
GROUP BY d.day
ORDER BY d.day;

A plain GROUP BY sale_date over sales skips a day with no sales; the date series provides every day, and the LEFT JOIN with COALESCE turns an empty day into 0. PostgreSQL has generate_series() for this; MySQL has no such function, so the recursive CTE is the idiom. Bound the series by reading the range from the data — SELECT MIN(sale_date) … in the anchor, WHERE day < (SELECT MAX(sale_date) FROM sales) in the recursive member.

Keeping recursion bounded

A recursive member that never runs out of rows would run forever. MySQL protects you with cte_max_recursion_depth (default 1000 rounds) and max_execution_time, but the query should stop on its own:

  • Counters stop at a limit: WHERE n < 100.
  • Joins down a tree stop at the leaves — but only if the data really is a tree. A cycle (A manages B, B manages A, by a data-entry mistake) loops forever.
  • Guard against cycles by carrying a depth and stopping at a sane maximum (WHERE org.depth < 20), or by carrying the path of ids and refusing to revisit: WHERE FIND_IN_SET(e.id, org.id_path) = 0. PostgreSQL 14 adds a CYCLE clause for this.
  • Use UNION instead of UNION ALL when rows can be regenerated — such as walking a graph that has several routes to one node — so already-seen rows are dropped.

Recursion for aggregates

Because aggregates are not allowed inside, total-over-a-subtree questions — "how many people work under each manager, at any depth" — are answered in two steps: the recursive CTE produces one row per (manager, subordinate) pair, and the outer query groups:

WITH RECURSIVE reports AS (
  SELECT manager_id AS boss, id AS person FROM employees WHERE manager_id IS NOT NULL
  UNION ALL
  SELECT r.boss, e.id
  FROM reports r
  JOIN employees e ON e.manager_id = r.person
)
SELECT boss, COUNT(*) AS team_size FROM reports GROUP BY boss ORDER BY boss;

The anchor pairs each manager with each direct report; each round extends every pair one level further down. Grouping by boss counts all of them.

In other databases

PostgreSQL and SQLite use WITH RECURSIVE exactly as MySQL does. SQL Server writes plain WITH and limits depth with OPTION (MAXRECURSION n) (default 100). Oracle accepts recursive WITH and also has its older CONNECT BY PRIOR syntax, which you may still meet in legacy code.

Pitfalls

  • Forgetting RECURSIVE, or the stopping condition.
  • An anchor column too narrow for the values built in later rounds.
  • Cycles in data that "should" be a tree.
  • Aggregating inside the recursive member instead of in the final query.
  • Relying on the CTE's row order instead of an outer ORDER BY.

Key takeaways

  • WITH RECURSIVE = anchor UNION ALL recursive member; each round reads only the previous round's rows; recursion stops at an empty round.
  • Join on child.parent_id = cte.id to walk down a tree, the other way round to walk up.
  • Carry a depth column for levels and a CAST(… AS CHAR(n)) path column for readable paths.
  • Generate number and date series to fill gaps before a LEFT JOIN.
  • Keep recursion bounded: counters, depth limits and cycle checks; aggregate outside the CTE.

Common questions

What is the anchor member of a recursive CTE?

The anchor member is the SELECT before UNION ALL that does not refer to the CTE itself. It runs once and supplies the starting rows, such as the employee with no manager or the first date of a range. Its column types also fix the types of the CTE's columns in MySQL.

How do you stop infinite recursion in a recursive CTE?

Give the recursive member a condition that eventually returns no rows, such as WHERE n < 100 or a join that runs out of children. For data that may contain cycles, carry a depth or path column and stop when it grows too large or when the path already contains the next id. MySQL also stops at cte_max_recursion_depth, 1000 by default.

Why is my recursive CTE path truncated in MySQL?

In MySQL the anchor member decides each column's type, so a path built from a short name in the anchor gets a short string type, and longer values from later rounds are cut or raise "Data too long". Cast the anchor's column wide enough, for example CAST(name AS CHAR(200)).

How do you find all subordinates of a manager in SQL?

Use a recursive CTE whose anchor selects the manager's direct reports and whose recursive member joins employees to the CTE on e.manager_id = cte.id. Each round adds the next level down, and the union of all rounds is every direct and indirect subordinate. A depth column counts the levels.

Can a recursive CTE use GROUP BY or aggregates?

Not in the recursive member. MySQL, like the SQL standard, forbids aggregate functions, window functions, GROUP BY, ORDER BY and DISTINCT in the recursive SELECT, because each round must see only the previous round's rows. Aggregate in the final query that reads the CTE instead.

Mark as readSign in to keep your place.

Recursive CTE in MySQL: Org Charts, Trees and Date Series

WITH RECURSIVE runs an anchor query, then repeats a recursive step until it returns no rows. Walk org charts and category trees, build paths and generate dates.

  • Course: SQL study plan
  • Module: Set operations and CTEs
  • Kind: Lesson
  • Reading time: 16 min
  • Runtime: MySQL 8 syntax on SQLite 3.49

How does a recursive CTE work in SQL?

A recursive CTE, written WITH RECURSIVE, has an anchor query and a recursive query joined by UNION ALL. The anchor produces the starting rows; the recursive query reads only the rows produced in the previous round and produces the next ones; the engine repeats until a round returns no rows. The union of every round is the result. It is how SQL walks trees such as org charts and generates series.

Lesson

Some data refers to itself. An employee row has a manager_id that is another employee's id; a category has a parent_id that is another category; a flight connects to more flights. Ordinary SQL can follow such a link a fixed number of times — one self-join per level — but not "as many times as the data needs". A recursive CTE can. It repeats a query over its own output until nothing new appears, which is how SQL walks an org chart of unknown depth, builds the full path of a category, finds every part inside an assembly, or simply generates the numbers 1 to 100 or every date in a month. This lesson covers how the repetition works, the classic hierarchy queries, building paths, generating series, and how to keep recursion bounded.

The shape

WITH RECURSIVE nums AS (
  SELECT 1 AS n                 -- anchor member
  UNION ALL
  SELECT n + 1 FROM nums        -- recursive member
  WHERE n < 5
)
SELECT n FROM nums ORDER BY n;
n
1
2
3
4
5

Evaluation goes in rounds:

  1. The anchor runs once. Its rows (n = 1) are the first round, and they go into the result.
  2. The recursive member runs with nums meaning only the previous round's rows. From n = 1 it makes n = 2 — round two.
  3. Step 2 repeats on each new round: 2 makes 3, 3 makes 4, 4 makes 5. From 5, WHERE n < 5 filters everything out — the round is empty and recursion stops.
  4. The result is the union of all rounds.

In MySQL the word RECURSIVE is required if any CTE in the WITH list refers to itself (PostgreSQL requires it too; SQL Server and Oracle have no keyword and detect self-reference). The two members are joined with UNION ALL; plain UNION is allowed and removes rows already produced, which can stop a cycle but costs a duplicate check every round.

The recursive member may not use aggregates, window functions, GROUP BY, ORDER BY, DISTINCT or LIMIT in MySQL (LIMIT is allowed from 8.0.19), and must refer to the CTE exactly once, in its FROM clause — not in a subquery. All of these exist to keep the "one round at a time" model well defined. Aggregate in the final SELECT instead.

Walking down an org chart

employees

idnamemanager_id
1MeeraNULL
2Arjun1
3Divya1
4Kabir2
5Leela4

Everyone under the CEO, with their level:

WITH RECURSIVE org AS (
  SELECT id, name, 0 AS depth
  FROM employees
  WHERE manager_id IS NULL                  -- the top
  UNION ALL
  SELECT e.id, e.name, org.depth + 1
  FROM employees e
  JOIN org ON e.manager_id = org.id         -- the previous level's people are managers
)
SELECT name, depth FROM org ORDER BY depth, name;
employeesidnamemanager_id1MeeraNULL2Arjun13Divya14Kabir25Leela4org (the result so far)namedepthMeera0Arjun1Divya1Kabir2Leela3round 1round 2round 3round 4round 5: no one → stop
WITH RECURSIVE, one round at a time. Example: anchor: manager_id IS NULL · recursive: JOIN org ON e.manager_id = org.id
  1. The anchor runs once: the employee whose manager_id IS NULL. Meera is round one, at depth 0, and goes into the result.
  2. The recursive member joins employees to the previous round only — Meera. Whoever reports to that person is round 2: Arjun and Divya, at depth 1.
  3. The recursive member joins employees to the previous round only — Arjun and Divya. Whoever reports to them is round 3: Kabir, at depth 2.
  4. The recursive member joins employees to the previous round only — Kabir. Whoever reports to that person is round 4: Leela, at depth 3.
  5. The recursive member runs again with org meaning only Leela. Nobody reports to Leela, so the round is empty and recursion stops. The result is the union of all 4 rounds.

Round one is Meera. Round two joins employees whose manager_id is Meera's id: Arjun and Divya. Round three finds Kabir under Arjun; round four Leela under Kabir; round five finds no one under Leela and stops. The join is the stopping condition: leaves have no children.

To start lower — everyone under Arjun — change only the anchor: WHERE id = 2. To include only the subordinates and not Arjun himself, anchor on WHERE manager_id = 2 with depth 1.

Walking up: a person's chain of command

Reverse the join to climb from one employee to the top:

WITH RECURSIVE chain AS (
  SELECT id, name, manager_id, 0 AS steps FROM employees WHERE id = 5
  UNION ALL
  SELECT m.id, m.name, m.manager_id, chain.steps + 1
  FROM employees m
  JOIN chain ON m.id = chain.manager_id
)
SELECT name, steps FROM chain ORDER BY steps;

This returns Leela, Kabir, Arjun, Meera — and stops when Meera's manager_id is NULL, since m.id = NULL matches nothing.

Building a path

To show where a node sits — Electronics > Phones > Android — carry a string column and extend it each round:

WITH RECURSIVE tree AS (
  SELECT id, name, CAST(name AS CHAR(200)) AS path
  FROM categories
  WHERE parent_id IS NULL
  UNION ALL
  SELECT c.id, c.name, CONCAT(t.path, ' > ', c.name)
  FROM categories c
  JOIN tree t ON c.parent_id = t.id
)
SELECT path FROM tree ORDER BY path;

The CAST matters in MySQL. The anchor decides each column's type: without it, path would be typed as wide as the longest root name, and a longer path built later is truncated (or rejected with "Data too long for column" in strict mode). Casting to CHAR(200) — or any width that fits the deepest path — avoids it. PostgreSQL has a similar rule and wants the types of both members to match exactly. Sorting by the path puts every child right after its parent, which displays a tree as an indented list.

Generating series

A recursive CTE with no table at all is a generator. Numbers:

WITH RECURSIVE seq AS (
  SELECT 1 AS n UNION ALL SELECT n + 1 FROM seq WHERE n < 100
)
SELECT n FROM seq;

Dates — the standard fix for reports with missing days:

WITH RECURSIVE days AS (
  SELECT '2024-03-01' AS day
  UNION ALL
  SELECT DATE_ADD(day, INTERVAL 1 DAY) FROM days WHERE day < '2024-03-07'
)
SELECT d.day, COALESCE(SUM(s.amount), 0) AS revenue
FROM days d
LEFT JOIN sales s ON s.sale_date = d.day
GROUP BY d.day
ORDER BY d.day;

A plain GROUP BY sale_date over sales skips a day with no sales; the date series provides every day, and the LEFT JOIN with COALESCE turns an empty day into 0. PostgreSQL has generate_series() for this; MySQL has no such function, so the recursive CTE is the idiom. Bound the series by reading the range from the data — SELECT MIN(sale_date) … in the anchor, WHERE day < (SELECT MAX(sale_date) FROM sales) in the recursive member.

Keeping recursion bounded

A recursive member that never runs out of rows would run forever. MySQL protects you with cte_max_recursion_depth (default 1000 rounds) and max_execution_time, but the query should stop on its own:

  • Counters stop at a limit: WHERE n < 100.
  • Joins down a tree stop at the leaves — but only if the data really is a tree. A cycle (A manages B, B manages A, by a data-entry mistake) loops forever.
  • Guard against cycles by carrying a depth and stopping at a sane maximum (WHERE org.depth < 20), or by carrying the path of ids and refusing to revisit: WHERE FIND_IN_SET(e.id, org.id_path) = 0. PostgreSQL 14 adds a CYCLE clause for this.
  • Use UNION instead of UNION ALL when rows can be regenerated — such as walking a graph that has several routes to one node — so already-seen rows are dropped.

Recursion for aggregates

Because aggregates are not allowed inside, total-over-a-subtree questions — "how many people work under each manager, at any depth" — are answered in two steps: the recursive CTE produces one row per (manager, subordinate) pair, and the outer query groups:

WITH RECURSIVE reports AS (
  SELECT manager_id AS boss, id AS person FROM employees WHERE manager_id IS NOT NULL
  UNION ALL
  SELECT r.boss, e.id
  FROM reports r
  JOIN employees e ON e.manager_id = r.person
)
SELECT boss, COUNT(*) AS team_size FROM reports GROUP BY boss ORDER BY boss;

The anchor pairs each manager with each direct report; each round extends every pair one level further down. Grouping by boss counts all of them.

In other databases

PostgreSQL and SQLite use WITH RECURSIVE exactly as MySQL does. SQL Server writes plain WITH and limits depth with OPTION (MAXRECURSION n) (default 100). Oracle accepts recursive WITH and also has its older CONNECT BY PRIOR syntax, which you may still meet in legacy code.

Pitfalls

  • Forgetting RECURSIVE, or the stopping condition.
  • An anchor column too narrow for the values built in later rounds.
  • Cycles in data that "should" be a tree.
  • Aggregating inside the recursive member instead of in the final query.
  • Relying on the CTE's row order instead of an outer ORDER BY.

Key takeaways

  • WITH RECURSIVE = anchor UNION ALL recursive member; each round reads only the previous round's rows; recursion stops at an empty round.
  • Join on child.parent_id = cte.id to walk down a tree, the other way round to walk up.
  • Carry a depth column for levels and a CAST(… AS CHAR(n)) path column for readable paths.
  • Generate number and date series to fill gaps before a LEFT JOIN.
  • Keep recursion bounded: counters, depth limits and cycle checks; aggregate outside the CTE.

Common questions

What is the anchor member of a recursive CTE?

The anchor member is the SELECT before UNION ALL that does not refer to the CTE itself. It runs once and supplies the starting rows, such as the employee with no manager or the first date of a range. Its column types also fix the types of the CTE's columns in MySQL.

How do you stop infinite recursion in a recursive CTE?

Give the recursive member a condition that eventually returns no rows, such as WHERE n < 100 or a join that runs out of children. For data that may contain cycles, carry a depth or path column and stop when it grows too large or when the path already contains the next id. MySQL also stops at cte_max_recursion_depth, 1000 by default.

Why is my recursive CTE path truncated in MySQL?

In MySQL the anchor member decides each column's type, so a path built from a short name in the anchor gets a short string type, and longer values from later rounds are cut or raise "Data too long". Cast the anchor's column wide enough, for example CAST(name AS CHAR(200)).

How do you find all subordinates of a manager in SQL?

Use a recursive CTE whose anchor selects the manager's direct reports and whose recursive member joins employees to the CTE on e.manager_id = cte.id. Each round adds the next level down, and the union of all rounds is every direct and indirect subordinate. A depth column counts the levels.

Can a recursive CTE use GROUP BY or aggregates?

Not in the recursive member. MySQL, like the SQL standard, forbids aggregate functions, window functions, GROUP BY, ORDER BY and DISTINCT in the recursive SELECT, because each round must see only the previous round's rows. Aggregate in the final query that reads the CTE instead.

Exercises

The org chart by level

List everyone in the company with their level: 0 for anyone with no manager, 1 for their direct reports, and so on down.

employees: idnamemanager_id
1MeeraNULL
2Arjun1
3Divya1
4Kabir2
5Leela4
6Anil3
7Zara3

Return name and depth, ordered by depth, then name.

name | depth
Meera | 0
Arjun | 1
Divya | 1
Anil | 2
Kabir | 2
Zara | 2
Leela | 3

A company may have more than one person without a manager; each starts a tree at depth 0. The depth of the chart is not fixed — use a recursive CTE.

Full category paths

Show every category with its full path from the top, the names joined by ' > '.

categories: idnameparent_id
1ElectronicsNULL
2Phones1
3Laptops1
4Android2
5iPhone2
6BooksNULL
7Fiction6

Return id and path, ordered by path.

id | path
6 | Books
7 | Books > Fiction
1 | Electronics
3 | Electronics > Laptops
2 | Electronics > Phones
4 | Electronics > Phones > Android
5 | Electronics > Phones > iPhone

In MySQL, cast the anchor's path wide enough (CAST(name AS CHAR(200))) so longer paths built later are not cut off.

In this module: Set operations and CTEs

← Common table expressions — naming the steps of a query · Checkpoint — Set operations and CTEs →