Python list comprehension vs generator: when to use which

Use a generator expression when you only need to iterate once and memory matters. Use a list comprehension when you need to access the result more than once, check its length, or index into it. The performance difference on a single pass is small — the real difference is memory and re-use.

The Syntax Difference

The two forms look almost identical. Square brackets produce a list; parentheses (or no brackets inside a function call) produce a generator:

# list comprehension — builds the whole list in memory immediately
squares_list = [x * x for x in range(1_000_000)]

# generator expression — produces values one at a time, on demand
squares_gen = (x * x for x in range(1_000_000))

# inside a function call, the outer parens double as the generator's parens
total = sum(x * x for x in range(1_000_000))

Memory: The Clearest Difference

A list holds every element at once. A generator holds almost nothing — just enough state to produce the next value when asked.

import sys

n = 1_000_000

lst = [x * 2 for x in range(n)]
gen = (x * 2 for x in range(n))

print(sys.getsizeof(lst))  # 8,448,728 bytes  (~8 MB)
print(sys.getsizeof(gen))  #       200 bytes

Output on this machine:

8448728
200

The list consumes ~8 MB. The generator object is 200 bytes regardless of how large the range is. If you're processing millions of rows from a file or database, this difference is the whole point.

Benchmarks: Four Scenarios

We ran every benchmark with timeit (5–500 repetitions) on Python 3.12.3.

Scenario 1: Single-pass sum

If you only iterate once, a generator is slightly faster because it avoids building the list first:

import timeit

n = 1_000_000

t_list = timeit.timeit("sum([x * 2 for x in range(n)])", globals={"n": n}, number=5) / 5
t_gen  = timeit.timeit("sum(x * 2 for x in range(n))",   globals={"n": n}, number=5) / 5

print(f"list: {t_list*1000:.1f} ms")
print(f"gen:  {t_gen*1000:.1f}  ms")

list: 200.3 ms
gen:  183.8 ms

The generator wins by ~8% here, but both times are dominated by the arithmetic — the difference is minor for a single pass.

Scenario 2: Accessing the result multiple times

This is where the list wins decisively. A generator is exhausted after one pass; you have to recreate it each time:

# list: build once, sum three times
t_list = timeit.timeit(
    "data = [x*2 for x in range(10_000)]; s = sum(data) + sum(data) + sum(data)",
    number=500) / 500

# generator: must recreate for each sum
t_gen = timeit.timeit(
    "s = sum(x*2 for x in range(10_000)) + sum(x*2 for x in range(10_000)) + sum(x*2 for x in range(10_000))",
    number=500) / 500

print(f"list: {t_list*1000:.2f} ms")
print(f"gen:  {t_gen*1000:.2f}  ms")

list: 1.96 ms
gen:  5.83 ms

The list is 3x faster here because it does one pass and stores the result. The generator must recompute every element from scratch each time.

Scenario 3: Early exit

When you only need the first few values, a generator is dramatically faster because it never computes the rest:

n = 1_000_000

# list: builds ALL 1 million elements, then takes [0]
t_list = timeit.timeit(
    "data = [x*2 for x in range(n)]; result = data[0]",
    globals={"n": n}, number=50) / 50

# generator: stops after the very first value
t_gen = timeit.timeit(
    "result = next(x*2 for x in range(n))",
    globals={"n": n}, number=50) / 50

print(f"list: {t_list*1000:.1f} ms")
print(f"gen:  {t_gen*1000:.2f}  ms")

list: 186.5 ms
gen:    0.00 ms

The generator is effectively instantaneous here. The list wastes time building 999,999 values you never use.

Scenario 4: any() / all() short-circuit

The built-ins any() and all() stop as soon as they have an answer. With a list comprehension they get no benefit — the whole list is already built before they see a single element:

n = 1_000_000

# Check if any element > 1 (true at index 2, so any() can short-circuit early)
t_list = timeit.timeit("any([x > 1 for x in range(n)])", globals={"n": n}, number=50) / 50
t_gen  = timeit.timeit("any(x > 1 for x in range(n))",   globals={"n": n}, number=50) / 50

print(f"list: {t_list*1000:.2f} ms")
print(f"gen:  {t_gen*1000:.2f}  ms")

list: 100.06 ms
gen:    0.00 ms

The generator short-circuits at the third element. The list comprehension builds all one million elements before any() gets to look at them. This is the most important practical rule: always pass a generator to any(), all(), and next().

The Exhaustion Gotcha

The most common generator bug: a generator object can only be iterated once. After that it is empty and silently returns nothing:

nums_list = [x * 2 for x in range(5)]
print(list(nums_list))   # [0, 2, 4, 6, 8]
print(list(nums_list))   # [0, 2, 4, 6, 8]  ← works fine

nums_gen = (x * 2 for x in range(5))
print(list(nums_gen))    # [0, 2, 4, 6, 8]
print(list(nums_gen))    # []  ← exhausted, silently empty

There's also no len() on a generator — it raises TypeError: object of type 'generator' has no len(). If you need to check length or index by position, you need a list.

The Decision Rule

When in doubt and the sequence is small (under a few thousand elements), a list comprehension is fine — the extra memory is negligible and you get the flexibility of re-use.

Verify It Worked

Run this to confirm the behavior on your machine:

python3 -c "
import sys, timeit
n = 1_000_000
lst = [x*2 for x in range(n)]
gen = (x*2 for x in range(n))
print('list size:', sys.getsizeof(lst), 'bytes')
print('gen size: ', sys.getsizeof(gen), 'bytes')
print('gen sum:', sum(gen))          # exhausts the generator
print('gen sum again:', sum(gen))    # 0 — generator is exhausted
"

Tested on Ubuntu 24.04 with Python 3.12.3.

References

• Python docs — Generator expressions
• Python docs — timeit
• Python docs — tracemalloc