How to Split List Into Even Chunks

Python Lists are powerful data structures that allow you to easily work with different data types using a single data structure. It is one of the most common data exchange formats in python, supported by most functions. Sometimes you may have a very large python list and may need to split it into even-sized smaller lists. In this article, we will learn how to split list into even chunks in Python.

How to Split List Into Even Chunks

There are several ways to split list into even chunks in Python. We will look at some of them.

Using Generators

Generators are often used to work with large data such as big files and large lists. This is because generators have a small memory footprint and does not require you to load the entire data into memory. Here is a generator function that yields successive n-sized lists from a large list.

def chunks(lst, n):
    """Yield successive n-sized chunks from lst."""
    for i in range(0, len(lst), n):
        yield lst[i:i + n]

In the above example, we use range function to return a sequence of numbers 0 to len(list) with step-size n. Here is a simple example to call the above function to split a given list into smaller lists.

import pprint
pprint.pprint(list(chunks(range(10, 75), 10)))
[[10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
 [20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
 [30, 31, 32, 33, 34, 35, 36, 37, 38, 39],
 [40, 41, 42, 43, 44, 45, 46, 47, 48, 49],
 [50, 51, 52, 53, 54, 55, 56, 57, 58, 59],
 [60, 61, 62, 63, 64, 65, 66, 67, 68, 69],
 [70, 71, 72, 73, 74]]

If you are using Python 2.x you may need to use xrange() function instead of range() function.

def chunks(lst, n):
    """Yield successive n-sized chunks from lst."""
    for i in xrange(0, len(lst), n):
        yield lst[i:i + n]

In the above code, the last chunk may not have exactly n elements but only the remainder ones that are left after all other chunks have been created.

Using List Comprehension

You can also use list comprehensions to split a list as shown below.

[lst[i:i + n] for i in range(0, len(lst), n)]

In the above code, we use range() function to once again list of lists, each having n elements. We use it to obtain index value of starting element of each smaller list, and use i+n to get the last element.

If you are using python 2.x, you may need to use xrange() function.

[lst[i:i + n] for i in xrange(0, len(lst), n)]

Here also, the last chunk may not have exactly n elements but only the remainder ones that are left after all other chunks have been created.

Using Numpy

You can also use Numpy library’s array_split() function to split a list into smaller lists.

import numpy as np

lst = range(50)
np.array_split(lst, 5)

The above approach is fast but it only allows you to set the total number of chunks in your list, and not the number of elements in each chunk.

In this article, we have learnt how to split list into even chunks in different ways. Often developers need to split large lists into smaller lists of equal size in their application or website. You can include the above code in your script as per your requirement.

How to Split List Into Even Chunks