Snippet

Processing List Datasets Efficiently using Django ORM bulk_create with Batching

Inserting large lists of items individually creates extreme database overhead. Django's bulk_create method takes a list of model instances and inserts them using batched multi-row INSERT queries.

snippet.py

python

from myapp.models import MetricEntry
 
def insert_metrics_in_batches(metric_values: list[float], batch_size: int = 500):
    instances = [MetricEntry(value=val) for val in metric_values]
    created = MetricEntry.objects.bulk_create(
        instances,
        batch_size=batch_size,
        ignore_conflicts=True
    )
    return len(created)

django

Breakdown

instances = [MetricEntry(value=val) for val in metric_values]

Uses a list comprehension to create model objects in memory from an input array of values.

created = MetricEntry.objects.bulk_create(

Calls bulk_create on the model manager to insert the list of instances in database operations.

batch_size=batch_size,

Restricts each generated INSERT query to a maximum number of records to avoid database memory limits.

Previous snippet Next snippet

More Intermediate Python exercises →

Processing List Datasets Efficiently using Django ORM bulk_create with Batching

Related