python - link several Popen commands with pipes

Question

I know how to run a command using cmd = subprocess.Popen and then subprocess.communicate. Most of the time I use a string tokenized with shlex.split as 'argv' argument for Popen. Example with "ls -l":

import subprocess
import shlex
print subprocess.Popen(shlex.split(r'ls -l'), stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE).communicate()[0]

However, pipes seem not to work... For instance, the following example returns noting:

import subprocess
import shlex
print subprocess.Popen(shlex.split(r'ls -l | sed "s/a/b/g"'), stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.PIPE).communicate()[0]

Can you tell me what I am doing wrong please?

Thx

Answer 1

I think you want to instantiate two separate Popen objects here, one for 'ls' and the other for 'sed'. You'll want to pass the first Popen object's stdout attribute as the stdin argument to the 2nd Popen object.

Example:

p1 = subprocess.Popen('ls ...', stdout=subprocess.PIPE)
p2 = subprocess.Popen('sed ...', stdin=p1.stdout, stdout=subprocess.PIPE)
print p2.communicate()

You can keep chaining this way if you have more commands:

p3 = subprocess.Popen('prog', stdin=p2.stdout, ...)

See the subprocess documentation for more info on how to work with subprocesses.