Sometimes, we want to read a big file in Python.

In this article, we’ll look at how to read a big file in Python.

How to read a big file in Python?

To read a big file in Python, we can call the readlines method on the opened file object with the buffer size.

For instance, we write:

BUF_SIZE = 100

bigfile = open('bar.txt', 'r')
tmp_lines = bigfile.readlines(BUF_SIZE)
while tmp_lines:
    print(tmp_lines)
    tmp_lines = bigfile.readlines(BUF_SIZE)

to call open with the file path of the file and 'r' to open the file with read permission.

Then we call bigfile.readlines with the BUF_SIZE buffer size.

BUF_SIZE is in bytes.

Next, we have a while loop that runs while tmp_lines isn’t None.

In the while loop, we call bigfile_readlines to read the next chunk in the buffer.

Conclusion

To read a big file in Python, we can call the readlines method on the opened file object with the buffer size.