python / intermediate
Snippet
Chunked QuerySet Processing using Iterator Chunk Size
When handling large datasets in Django, evaluating a QuerySet loads all model instances into memory. Calling iterator(chunk_size=N) streams results in batches to dramatically lower RAM utilization.
snippet.py
python
1
2
3
4
5
6
7
8
9
10
11
12
from .models import UserProfiledef archive_inactive_profiles(days_threshold=365):inactive_queryset = UserProfile.objects.filter(is_active=False)count = 0for profile in inactive_queryset.iterator(chunk_size=500):profile.archived = Trueprofile.save(update_fields=['archived'])count += 1return count
django
Breakdown
1
inactive_queryset = UserProfile.objects.filter(is_active=False)
Construct a lazy QuerySet filter without immediately executing SQL against the database.
2
for profile in inactive_queryset.iterator(chunk_size=500):
Fetch database records in stream chunks of 500 rows to prevent high RAM memory usage.
3
profile.save(update_fields=['archived'])
Persist model changes while explicitly updating only specified fields to minimize write overhead.