Python subprocess.run vs Popen: Which One to Use

Use subprocess.run() for almost everything. It starts a process, waits for it to finish, and gives you a CompletedProcess object with the return code, stdout, and stderr. Use subprocess.Popen() only when you need something run() cannot do: streaming output line-by-line as the process produces it, writing to the process's stdin while it runs, or polling whether a long-running process has finished without blocking.

Why the two APIs exist

subprocess.Popen has been in Python since 2.4. It is the low-level primitive: it starts the process and gives you direct access to the file-like .stdout, .stdin, and .stderr handles. You are responsible for calling .communicate() or .wait() and for avoiding deadlocks.

subprocess.run() was added in Python 3.5 as a convenience wrapper. It calls Popen internally, calls .communicate(), and returns when the process exits. That one call handles the common case and avoids the easiest deadlock (forgetting to drain pipes).

subprocess.run() — the everyday tool

The two most important keyword arguments are capture_output=True (collect stdout and stderr into strings instead of letting them print to the terminal) and text=True (decode bytes to strings automatically).

import subprocess

result = subprocess.run(
    ["echo", "hello from run"],
    capture_output=True,
    text=True,
)
print(result.returncode)  # 0
print(result.stdout)      # 'hello from run\n'
print(result.stderr)      # ''

Add check=True to raise subprocess.CalledProcessError automatically when the command returns a non-zero exit code. Without it you have to check result.returncode yourself and it is easy to silently swallow failures.

# Succeeds silently
result = subprocess.run(["ls", "/tmp"], capture_output=True, text=True, check=True)

# Raises CalledProcessError
try:
    subprocess.run(["ls", "/nonexistent_dir"], capture_output=True, text=True, check=True)
except subprocess.CalledProcessError as e:
    print(e.returncode)  # 2
    print(e.stderr)      # "ls: cannot access '/nonexistent_dir': No such file or directory\n"

Real output from running these on Python 3.12.3:

returncode: 0
stderr: "ls: cannot access '/nonexistent_dir': No such file or directory\n"

You can also add a timeout in seconds. If the process does not exit in time, subprocess.TimeoutExpired is raised and the process is killed.

try:
    subprocess.run(["sleep", "10"], timeout=0.1)
except subprocess.TimeoutExpired:
    print("process timed out")  # prints this

subprocess.Popen() — when you need more control

Streaming output line by line

If the command produces output gradually and you want to process each line as it arrives (for example, tailing a log or showing progress from a build), run() is the wrong tool — it buffers everything and only returns after the process exits. Use Popen and iterate over .stdout:

proc = subprocess.Popen(
    ["bash", "-c", "for i in A B C; do echo $i; done"],
    stdout=subprocess.PIPE,
    text=True,
)
for line in proc.stdout:
    print(f"got: {line!r}")   # prints each line as it arrives
proc.wait()  # reap the process
print(proc.returncode)  # 0

Real output:

got: 'A\n'
got: 'B\n'
got: 'C\n'
returncode: 0

Writing to the process's stdin

When you need to send data to a process that reads from stdin (a password prompt, an interactive interpreter, gpg --decrypt), use Popen with stdin=subprocess.PIPE and then call .communicate(input=...):

proc = subprocess.Popen(
    ["python3", "-c", "import sys; data = sys.stdin.read(); print(f'got: {data!r}')"],
    stdin=subprocess.PIPE,
    stdout=subprocess.PIPE,
    text=True,
)
stdout, _ = proc.communicate(input="hello from stdin")
print(stdout)  # got: 'hello from stdin'

Note: even here, .communicate() buffers everything. If you need truly interactive back-and-forth (send a line, read a line, send another), you need pexpect or asyncio.create_subprocess_exec to avoid deadlocking on full pipe buffers.

Non-blocking poll

Popen.poll() checks whether the process has finished without blocking. It returns None if still running, or the exit code if done. This is useful when you want to do other work in your Python process while waiting.

proc = subprocess.Popen(["sleep", "2"])
while proc.poll() is None:
    print("still running...")
    time.sleep(0.5)
print(f"done, exit code: {proc.returncode}")

The shell=True pitfall

Both run() and Popen accept shell=True, which passes the command to /bin/sh -c. Avoid it when any part of the command comes from user input — it opens a shell injection vulnerability.

# DANGEROUS: if user_input is "; rm -rf /", the shell runs that
subprocess.run(f"echo {user_input}", shell=True)

# SAFE: pass arguments as a list, never as a concatenated string
subprocess.run(["echo", user_input], capture_output=True, text=True)

shell=True is occasionally useful for quick shell pipelines in scripts where all inputs are hard-coded, but it should never appear in web server code or anywhere that handles external data.

Quick decision guide

• Run a command and get its output → subprocess.run([...], capture_output=True, text=True, check=True)
• Run a command and raise on failure → add check=True
• Limit how long a command can run → add timeout=N
• Stream output as it is produced → Popen + iterate proc.stdout
• Write data to stdin → Popen(stdin=PIPE) + communicate(input=...)
• Check if a process finished without blocking → Popen + poll()

Confirming it works

The easiest sanity check: run your command with capture_output=True, text=True, check=True and print result.stdout. If the command fails, CalledProcessError.stderr will tell you exactly what went wrong. For Popen workflows, always call .wait() or .communicate() before exiting your script — leaving a child process running is a resource leak.

Tested on Ubuntu 24.04 with Python 3.12.3.

References

• Python docs — subprocess