AI InterviewSearch…Search problems, topics, or pages…Ctrl KGet started
Skip to content

19 Performance and undefined behaviour / Lesson 2 of 5

The cost model — counting copies, allocations and calls

14 min · +10 XP · 2 exercises · 7 questions

Read the text14 min
Exercises0/2
Quiz7 q

Quick answer

What does copying a std::vector cost in C++?

Copying a std::vector in C++ costs one heap allocation plus a copy of every element — for a million ints, four megabytes — and each element that owns memory, such as a long std::string, adds its own allocation. Copying the three-pointer handle is cheap; copying the elements is the cost. Pass by const& to read, move when handing over ownership, and return by value.

Performance reasoning in C++ starts long before a stopwatch: it starts by naming the operations a line of code performs and counting them. A std::vector<int> passed by value is an allocation plus a copy of every element; a std::string longer than fifteen characters is an allocation; s = s + piece in a loop copies the whole prefix every time. None of those costs is visible in the source, and all of them are predictable from the value semantics the earlier modules taught. This lesson builds the model — what a copy, an allocation and a call cost relative to each other — and applies it to the decisions that recur in every program: how to pass, how to return, when to reserve, how to build a string, and when a view beats a copy.

What a copy is made of

Copying a value means copying its bytes and, for anything that owns memory, allocating and copying what it owns:

ValueThe copy is
int, double, a small struct of thema register move; effectively free
std::string of ≤ 15 characters (libstdc++'s small-string buffer)a copy of the handle's bytes; no allocation
std::string of 100 charactersone allocation plus 100 bytes copied
std::vector<int> of a millionone allocation plus four megabytes copied
std::vector<std::string>one allocation, then one copy per element — each with its own rule above

A std::vector handle is three pointers; copying the handle is not the cost, copying the elements is. The std::string boundary — fifteen characters on this platform's libstdc++, twenty-two on libc++ — is why "small strings are cheap" is true and "strings are cheap" is not.

Orders of magnitude

The relative costs below are from published measurements of ordinary hardware; the absolute numbers vary by machine, the ratios do not:

OperationRoughly
Integer add, compare, correctly predicted branchunder 1 ns
Read from L1 cache~1 ns
Read from main memory (a cache miss)~100 ns
Small allocation and free (new/delete pair)~20–50 ns
Mispredicted branch~5–7 ns
A system call~200 ns and up

An allocation costs about as much as fifty arithmetic operations, so a loop that allocates per iteration is an allocation loop whatever else it does; a cache miss costs about a hundred, which lesson 3 is about.

Passing parameters

Module 4 gave the rule; the model explains it. Passing by value copies; passing by const& binds a name to the caller's object for the price of one pointer:

long long total(const std::vector<int>& v);   // no copy, cannot modify
long long total(std::vector<int> v);          // allocation + n copies per call

By value is right for small trivially copyable types (int, double, a Point of two doubles, std::string_view, std::span), where a copy is a register move and the indirection of a reference would cost more. It is also right for a sink — a parameter the function will store — because the caller can then move into it:

void Log::add(std::string line) { lines_.push_back(std::move(line)); }

log.add(text);              // one copy (into the parameter), one move (into the vector)
log.add(std::move(text));   // one move, one move
log.add("literal");         // constructs in place: zero copies, one move

The instrumented type in this lesson's exercise counts exactly these: an lvalue argument to a by-value parameter is one copy, std::move of it is one move, a const& parameter is neither, and a temporary passed by value is nothing — C++17 guarantees the parameter is constructed directly from the prvalue (Module 9, lesson 5).

Returning

Return by value. A prvalue return (return std::vector<int>{…}) is elided by rule; a named local (return result;) is elided by every compiler through NRVO, and moved when it cannot be. return std::move(result); disables NRVO and forces the move — a pessimisation. Output parameters (void fill(std::vector<int>& out)) save nothing over returning and cost the caller a declaration.

Growth and reserve

A std::vector that outgrows its capacity allocates a bigger block, moves every element across and frees the old one. With a growth factor f the total number of element moves over n pushes is bounded by n · f / (f − 1) — twice n for doubling — so the amortised cost per push_back is constant. For a doubling vector filled from empty:

PushesAllocationsElements movedFinal capacity
1051516
1008127128
1 000111 0231 024

reserve(n) first turns that into one allocation and zero moves. It is the single most common cheap win in C++ code: whenever the final size is known or bounded, reserve. The factor and the exact capacities are implementation choices — libstdc++ doubles, MSVC grows by 1.5 — which is why the exercise models its own Vec with a stated factor rather than reading capacity().

emplace_back(args…) constructs the element in place from its arguments; push_back(x) copies an lvalue and moves an rvalue. For a std::vector<std::string>, v.push_back(s) is an allocation and a copy of the characters, v.push_back(std::move(s)) is a pointer swap, v.emplace_back("text") constructs once.

Building strings

std::string out;
for (const auto& piece : pieces) out += piece;          // amortised linear
for (const auto& piece : pieces) out = out + piece;     // quadratic

out += piece appends into spare capacity, reallocating geometrically like a vector, so k pieces of m characters copy about k·m bytes in total. out = out + piece builds a fresh temporary containing everything so far plus the piece, then moves it into out: the *i*-th iteration copies i·m bytes, and the total is m · k(k + 1) / 2. For a thousand ten-character pieces that is ten kilobytes copied against five megabytes — the same output, five hundred times the work. std::ostringstream and std::format are also linear; reserve on the string removes the growth copies too.

Views instead of copies

std::string::substr allocates a new string for anything past the small buffer; std::string_view::substr adjusts a pointer and a length. A parser that slices its input with views does zero allocations for the slicing itself, which is the whole case for std::string_view (Module 5, lesson 6) — with the standing rule that the view must not outlive the string it looks into.

The method

For any line you suspect: name the operation (copy, allocation, comparison, call), count how often it runs per element, multiply by the elements, compare with the alternative. The model is coarse — it ignores caches, branch prediction and what the optimiser removes — but it ranks alternatives correctly far more often than intuition does, and it is the form every exercise in this module takes: a counter that reports what the code did, never a clock.

Pitfalls

WrittenCost you did not see
void f(std::vector<int> v) for read-only vAn allocation and n copies per call
for (auto s : names) over stringsOne copy per element (Module 3, lesson 4)
push_back a million times without reserve~20 reallocations and ~a million element moves
out = out + piece in a loopQuadratic copying
return std::move(local);Forces a move where elision was free
std::string key = line.substr(a, b) in a hot loopOne allocation per slice; a string_view is none

Key takeaways

  • A copy of an owning type is an allocation plus a copy of what it owns; small strings and small structs are the cheap exceptions.
  • An allocation costs about fifty arithmetic operations; a cache miss about a hundred. Loops are judged by what they allocate and touch.
  • Pass by const& to read, by value to sink or for small types, and move into sinks; return by value and never return std::move(local).
  • reserve when the size is known; geometric growth keeps push_back amortised constant, but the moves are real.
  • += builds a string in linear time; s = s + piece is quadratic; views slice without allocating.

Common questions

When should I pass by value in C++?

Pass small trivially copyable types such as int, double, std::string_view and std::span by value, because the copy is a register move. Pass a sink parameter — one the function will store — by value too, so a caller can std::move into it. Pass anything else you only read by const&.

Why is return std::move(local); bad?

It disables named return value optimisation and forces a move where the compiler would otherwise have constructed the result directly in the caller. Plain return result; is elided by every compiler, and moved automatically when elision is impossible.

What does vector reserve do in C++?

reserve(n) allocates capacity for n elements up front, so the following push_back calls neither reallocate nor move elements. Without it, a doubling vector filled with 1,000 elements makes 11 allocations and moves 1,023 elements along the way. Reserve whenever the final size is known or bounded.

Why is string concatenation in a loop slow in C++?

out = out + piece builds a new temporary holding everything so far on every iteration, so the total copying grows quadratically. out += piece appends into spare capacity and is amortised linear — for a thousand ten-character pieces, about ten kilobytes copied instead of five megabytes.

Is copying a std::string expensive?

It depends on the length. libstdc++ keeps strings of up to 15 characters inside the object itself (libc++ up to 22), so copying one needs no allocation. A longer string costs an allocation plus a copy of its characters, which is why slicing with std::string_view beats substr in a hot loop.

Mark as readSign in to keep your place.

C++ Performance Cost Model: Copies, Allocations and reserve

What a copy, an allocation and a call cost in C++: pass by value vs const reference, vector reserve and growth, quadratic string building, return by value.

  • Course: C++ study plan
  • Module: Performance and undefined behaviour
  • Kind: Lesson
  • Reading time: 14 min
  • Runtime: Clang 18 · C++20

What does copying a std::vector cost in C++?

Copying a std::vector in C++ costs one heap allocation plus a copy of every element — for a million ints, four megabytes — and each element that owns memory, such as a long std::string, adds its own allocation. Copying the three-pointer handle is cheap; copying the elements is the cost. Pass by const& to read, move when handing over ownership, and return by value.

Lesson

Performance reasoning in C++ starts long before a stopwatch: it starts by naming the operations a line of code performs and counting them. A std::vector<int> passed by value is an allocation plus a copy of every element; a std::string longer than fifteen characters is an allocation; s = s + piece in a loop copies the whole prefix every time. None of those costs is visible in the source, and all of them are predictable from the value semantics the earlier modules taught. This lesson builds the model — what a copy, an allocation and a call cost relative to each other — and applies it to the decisions that recur in every program: how to pass, how to return, when to reserve, how to build a string, and when a view beats a copy.

What a copy is made of

Copying a value means copying its bytes and, for anything that owns memory, allocating and copying what it owns:

ValueThe copy is
int, double, a small struct of thema register move; effectively free
std::string of ≤ 15 characters (libstdc++'s small-string buffer)a copy of the handle's bytes; no allocation
std::string of 100 charactersone allocation plus 100 bytes copied
std::vector<int> of a millionone allocation plus four megabytes copied
std::vector<std::string>one allocation, then one copy per element — each with its own rule above

A std::vector handle is three pointers; copying the handle is not the cost, copying the elements is. The std::string boundary — fifteen characters on this platform's libstdc++, twenty-two on libc++ — is why "small strings are cheap" is true and "strings are cheap" is not.

Orders of magnitude

The relative costs below are from published measurements of ordinary hardware; the absolute numbers vary by machine, the ratios do not:

OperationRoughly
Integer add, compare, correctly predicted branchunder 1 ns
Read from L1 cache~1 ns
Read from main memory (a cache miss)~100 ns
Small allocation and free (new/delete pair)~20–50 ns
Mispredicted branch~5–7 ns
A system call~200 ns and up

An allocation costs about as much as fifty arithmetic operations, so a loop that allocates per iteration is an allocation loop whatever else it does; a cache miss costs about a hundred, which lesson 3 is about.

Passing parameters

Module 4 gave the rule; the model explains it. Passing by value copies; passing by const& binds a name to the caller's object for the price of one pointer:

long long total(const std::vector<int>& v);   // no copy, cannot modify
long long total(std::vector<int> v);          // allocation + n copies per call

By value is right for small trivially copyable types (int, double, a Point of two doubles, std::string_view, std::span), where a copy is a register move and the indirection of a reference would cost more. It is also right for a sink — a parameter the function will store — because the caller can then move into it:

void Log::add(std::string line) { lines_.push_back(std::move(line)); }

log.add(text);              // one copy (into the parameter), one move (into the vector)
log.add(std::move(text));   // one move, one move
log.add("literal");         // constructs in place: zero copies, one move

The instrumented type in this lesson's exercise counts exactly these: an lvalue argument to a by-value parameter is one copy, std::move of it is one move, a const& parameter is neither, and a temporary passed by value is nothing — C++17 guarantees the parameter is constructed directly from the prvalue (Module 9, lesson 5).

Returning

Return by value. A prvalue return (return std::vector<int>{…}) is elided by rule; a named local (return result;) is elided by every compiler through NRVO, and moved when it cannot be. return std::move(result); disables NRVO and forces the move — a pessimisation. Output parameters (void fill(std::vector<int>& out)) save nothing over returning and cost the caller a declaration.

Growth and reserve

A std::vector that outgrows its capacity allocates a bigger block, moves every element across and frees the old one. With a growth factor f the total number of element moves over n pushes is bounded by n · f / (f − 1) — twice n for doubling — so the amortised cost per push_back is constant. For a doubling vector filled from empty:

PushesAllocationsElements movedFinal capacity
1051516
1008127128
1 000111 0231 024

reserve(n) first turns that into one allocation and zero moves. It is the single most common cheap win in C++ code: whenever the final size is known or bounded, reserve. The factor and the exact capacities are implementation choices — libstdc++ doubles, MSVC grows by 1.5 — which is why the exercise models its own Vec with a stated factor rather than reading capacity().

emplace_back(args…) constructs the element in place from its arguments; push_back(x) copies an lvalue and moves an rvalue. For a std::vector<std::string>, v.push_back(s) is an allocation and a copy of the characters, v.push_back(std::move(s)) is a pointer swap, v.emplace_back("text") constructs once.

Building strings

std::string out;
for (const auto& piece : pieces) out += piece;          // amortised linear
for (const auto& piece : pieces) out = out + piece;     // quadratic

out += piece appends into spare capacity, reallocating geometrically like a vector, so k pieces of m characters copy about k·m bytes in total. out = out + piece builds a fresh temporary containing everything so far plus the piece, then moves it into out: the *i*-th iteration copies i·m bytes, and the total is m · k(k + 1) / 2. For a thousand ten-character pieces that is ten kilobytes copied against five megabytes — the same output, five hundred times the work. std::ostringstream and std::format are also linear; reserve on the string removes the growth copies too.

Views instead of copies

std::string::substr allocates a new string for anything past the small buffer; std::string_view::substr adjusts a pointer and a length. A parser that slices its input with views does zero allocations for the slicing itself, which is the whole case for std::string_view (Module 5, lesson 6) — with the standing rule that the view must not outlive the string it looks into.

The method

For any line you suspect: name the operation (copy, allocation, comparison, call), count how often it runs per element, multiply by the elements, compare with the alternative. The model is coarse — it ignores caches, branch prediction and what the optimiser removes — but it ranks alternatives correctly far more often than intuition does, and it is the form every exercise in this module takes: a counter that reports what the code did, never a clock.

Pitfalls

WrittenCost you did not see
void f(std::vector<int> v) for read-only vAn allocation and n copies per call
for (auto s : names) over stringsOne copy per element (Module 3, lesson 4)
push_back a million times without reserve~20 reallocations and ~a million element moves
out = out + piece in a loopQuadratic copying
return std::move(local);Forces a move where elision was free
std::string key = line.substr(a, b) in a hot loopOne allocation per slice; a string_view is none

Key takeaways

  • A copy of an owning type is an allocation plus a copy of what it owns; small strings and small structs are the cheap exceptions.
  • An allocation costs about fifty arithmetic operations; a cache miss about a hundred. Loops are judged by what they allocate and touch.
  • Pass by const& to read, by value to sink or for small types, and move into sinks; return by value and never return std::move(local).
  • reserve when the size is known; geometric growth keeps push_back amortised constant, but the moves are real.
  • += builds a string in linear time; s = s + piece is quadratic; views slice without allocating.

Common questions

When should I pass by value in C++?

Pass small trivially copyable types such as int, double, std::string_view and std::span by value, because the copy is a register move. Pass a sink parameter — one the function will store — by value too, so a caller can std::move into it. Pass anything else you only read by const&.

Why is return std::move(local); bad?

It disables named return value optimisation and forces a move where the compiler would otherwise have constructed the result directly in the caller. Plain return result; is elided by every compiler, and moved automatically when elision is impossible.

What does vector reserve do in C++?

reserve(n) allocates capacity for n elements up front, so the following push_back calls neither reallocate nor move elements. Without it, a doubling vector filled with 1,000 elements makes 11 allocations and moves 1,023 elements along the way. Reserve whenever the final size is known or bounded.

Why is string concatenation in a loop slow in C++?

out = out + piece builds a new temporary holding everything so far on every iteration, so the total copying grows quadratically. out += piece appends into spare capacity and is amortised linear — for a thousand ten-character pieces, about ten kilobytes copied instead of five megabytes.

Is copying a std::string expensive?

It depends on the length. libstdc++ keeps strings of up to 15 characters inside the object itself (libc++ up to 22), so copying one needs no allocation. A longer string costs an allocation plus a copy of its characters, which is why slicing with std::string_view beats substr in a hot loop.

Exercises

Counting copies and moves

Instrument a type so that every way of passing it reports what it cost. Payload owns a std::vector<int> and keeps two static inline long long counters, copies and moves; write its copy constructor and its noexcept move constructor so each increments its counter. Three functions take it: takeByValue(Payload), takeByRef(const Payload&), and a Store whose add(Payload p) is a sink — it push_backs std::move(p) into its vector. makeTemp() returns Payload{}.

main reads n commands and applies each to one long-lived item (a moved-from item stays valid; only the counts matter). Before the loop, reserve(n) the store's vector so that growth never moves an element. After each command print <command> copies=<c> moves=<m> with the counts that command alone added, and at the end total copies=<C> moves=<M>.

CommandDoes
valuetakeByValue(item)
reftakeByRef(item)
temptakeByValue(Payload{})
sink-copystore.add(item)
sink-movestore.add(std::move(item))
sink-tempstore.add(Payload{})
makePayload made = makeTemp();

Input: n, then n command words. Output: n lines, then the total line.

5
value
ref
temp
sink-copy
sink-move

prints

value copies=1 moves=0
ref copies=0 moves=0
temp copies=0 moves=0
sink-copy copies=1 moves=1
sink-move copies=0 moves=2
total copies=2 moves=3

A Vec that reports its growth

Model geometric growth with your own container, never by reading std::vector::capacity(). Vec holds long long elements in a std::unique_ptr<long long[]> with size, capacity and an integer growth factor given at construction. reserve(want) does nothing when want <= capacity; otherwise it allocates a block of exactly want, copies the size existing elements across (count each as one move), counts one allocation, and adopts the block. push_back(x) calls reserve(capacity == 0 ? 1 : capacity * factor) when full, then stores.

Read k scenarios, each factor pushes reserve: construct a Vec with the factor, call reserve(reserve) first (a reserve of 0 does nothing), then push 1, 2, …, pushes. Print one line per scenario: factor=F pushes=N reserve=R: allocations A moves M capacity C sum S, where S is the sum of the stored elements read back through operator[].

Input: k, then k lines factor pushes reserve (factor ≥ 2). Output: k lines.

3
2 10 0
2 100 0
2 1000 0

prints

factor=2 pushes=10 reserve=0: allocations 5 moves 15 capacity 16 sum 55
factor=2 pushes=100 reserve=0: allocations 8 moves 127 capacity 128 sum 5050
factor=2 pushes=1000 reserve=0: allocations 11 moves 1023 capacity 1024 sum 500500

In this module: Performance and undefined behaviour

← The undefined behaviour catalogue — what the compiler is allowed to assume · Memory layout and the cache — contiguity, padding and traversal order →