How can I tail a log file in Python?

Question

I'd like to make the output of tail -F or something similar available to me in Python without blocking or locking. I've found some really old code to do that here, but I'm thinking there must be a better way or a library to do the same thing by now. Anyone know of one?

Ideally, I'd have something like tail.getNewData() that I could call every time I wanted more data.

Answer 1

Non Blocking

If you are on linux (as windows does not support calling select on files) you can use the subprocess module along with the select module.

import time
import subprocess
import select

f = subprocess.Popen(['tail','-F',filename],
        stdout=subprocess.PIPE,stderr=subprocess.PIPE)
p = select.poll()
p.register(f.stdout)

while True:
    if p.poll(1):
        print f.stdout.readline()
    time.sleep(1)

This polls the output pipe for new data and prints it when it is available. Normally the time.sleep(1) and print f.stdout.readline() would be replaced with useful code.

Blocking

You can use the subprocess module without the extra select module calls.

import subprocess
f = subprocess.Popen(['tail','-F',filename],
        stdout=subprocess.PIPE,stderr=subprocess.PIPE)
while True:
    line = f.stdout.readline()
    print line

This will also print new lines as they are added, but it will block until the tail program is closed, probably with f.kill().