Enumerable#lazy — Processing Large Collections Without Loading Them All
Ruby’s Enumerable is one of the language’s best features — map, select, reject, reduce, and dozens of other methods that make working with collections feel natural. But every one of those methods is eager: it processes the entire collection and returns a new collection before the next method touches anything. For small arrays that’s fine. For large datasets, database result sets, infinite sequences, or file streams, it’s a problem. Enumerable#lazy solves it by deferring evaluation until you actually need the results.
The Eager Problem
Example:
# Eager — processes all 10 million numbers, builds three arrays
result = (1..10_000_000)
.map { |n| n * 2 }
.select { |n| n % 3 == 0 }
.first(5)
# This works but allocates and iterates three massive intermediate collections
Each method in this chain processes all 10 million elements before passing to the next. map builds an array of 10 million items. select filters that into another array. first(5) finally takes five. All that work for five results.
Example:
# Lazy — evaluates only as many elements as needed
result = (1..10_000_000)
.lazy
.map { |n| n * 2 }
.select { |n| n % 3 == 0 }
.first(5)
With .lazy, Ruby processes elements one at a time through the entire chain. When first(5) has collected five results, evaluation stops — no further elements are processed. For this example, it evaluates far fewer than 10 million numbers.
How Lazy Evaluation Works
Calling .lazy on any Enumerable returns a Enumerator::Lazy object. Methods chained after .lazy build up a pipeline but don’t execute it. Evaluation happens when you call a terminal method — first, to_a, force, each, take, or reduce.
Example:
pipeline = (1..Float::INFINITY)
.lazy
.map { |n| n ** 2 }
.select { |n| n.odd? }
.reject { |n| n % 5 == 0 }
puts pipeline.first(4).inspect
# => [1, 9, 49, 81]
This works on an infinite range — something impossible with eager evaluation. The pipeline runs until first(4) is satisfied, then stops. No infinite loop, no memory explosion.
Real-World Use: Processing Large Files
Reading a large file line by line and finding the first match is a common pattern where lazy evaluation shines:
Example:
# Eager — reads entire file into memory
results = File.readlines("large_log.txt")
.select { |line| line.include?("ERROR") }
.map { |line| line.strip }
.first(10)
# Lazy — stops reading after finding 10 errors
results = File.foreach("large_log.txt")
.lazy
.select { |line| line.include?("ERROR") }
.map { |line| line.strip }
.first(10)
File.foreach returns an enumerator that yields one line at a time. Combined with .lazy, it reads only as far into the file as necessary. For a 10GB log file where errors appear early, this is the difference between seconds and minutes.
Generating Infinite Sequences
Lazy evaluation makes infinite sequences practical:
Example:
# Fibonacci sequence — infinite
fibs = Enumerator.new do |yielder|
a, b = 0, 1
loop do
yielder << a
a, b = b, a + b
end
end
puts fibs.lazy.select { |n| n.even? }.first(5).inspect
# => [0, 2, 8, 34, 144]
puts fibs.lazy.find { |n| n > 1000 }
# => 1597
The Enumerator yields Fibonacci numbers indefinitely. .lazy lets you consume as many as you need without pre-generating them. This pattern is useful for generating test data, sequences for simulations, or any situation where you need “as many as needed” rather than “exactly N.”
Custom lazy enumerators
Example:
def prime_numbers
Enumerator.new do |y|
n = 2
primes = []
loop do
if primes.none? { |p| n % p == 0 }
primes << n
y << n
end
n += 1
end
end
end
puts prime_numbers.lazy.first(10).inspect
# => [2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
Pro-Tip: Lazy enumerators don’t memoize. Every time you call
firstor iterate, the pipeline runs from the beginning. If you need to iterate the same lazy sequence multiple times, call.to_aor.forceto materialize it once and reuse the array. Lazy evaluation is for “I need some of this” — once you’ve decided you need all of it, materialize it.
When Not to Use Lazy
Lazy evaluation adds overhead per element — a small scheduler cost for each value as it moves through the pipeline. For small, in-memory arrays, eager Enumerable methods are faster. The crossover point varies, but if you’re working with collections under a few thousand elements that are already in memory, eager is fine and often faster.
Use lazy when:
- The source is large (files, database cursors, network streams)
- The source is infinite or unbounded
- You need only the first N results from an expensive chain
- Memory is constrained
Use eager when:
- You need all results anyway
- The collection is small and already in memory
- The transformation is simple (eager avoids the lazy scheduler overhead)
Conclusion
Enumerable#lazy makes Ruby’s elegant collection methods practical at scale. The API is the same — map, select, reject — so there’s almost no learning curve if you already know Enumerable. You add .lazy, chain your operations, and call a terminal method when you’re ready for results. For files, database result sets, and infinite sequences, this is the difference between code that works and code that works without melting your server.
FAQs
Q1: Does lazy evaluation work with ActiveRecord relations?
ActiveRecord relations are already lazy — they don’t hit the database until you call to_a, iterate with each, or access the collection. Adding .lazy to an AR relation adds Ruby-level lazy evaluation on top of the SQL result, which is useful for post-query processing but doesn’t reduce the SQL result set size. Use where, limit, and select at the SQL level first.
Q2: Is there a performance cost to using lazy?
Yes, for small collections — lazy adds per-element scheduling overhead. For large collections or sequences where you take fewer results than the total, lazy wins significantly on memory and often on time. Benchmark your specific case if performance is critical.
Q3: Can I chain lazy with non-Enumerable methods?
The lazy pipeline ends when you call a method that isn’t lazy-aware. Methods like first, to_a, take, and each terminate the pipeline. After calling one of these, you’re working with a regular Array again.
Q4: How does Enumerator::Lazy differ from Enumerator?
A regular Enumerator is also lazy in the sense that it yields values one at a time, but chaining Enumerable methods on it is eager. Enumerator::Lazy (returned by .lazy) makes the entire chained pipeline evaluate element-by-element until a terminal condition is met.
Q5: Can I use lazy with zip or flat_map?
flat_map is supported in lazy mode. zip has partial lazy support — check the Ruby documentation for your specific version, as lazy behavior of some Enumerable methods has expanded across Ruby versions.
Check viewARU - Brand Newsletter!
Newsletter to DEVs by DEVs - boost your Personal Brand & career! 🚀