Python for DevOps Engineers: The Scripting You Actually Need

Most people who want to move into DevOps hear two pieces of advice that seem to clash. One says "you must learn to code". The other says "DevOps is not software engineering, you just need tools". Both are half right. You do not need to build web applications, but you do need to write small, reliable scripts that save time and remove manual work. For most teams, that language is Python, with Bash for the very short jobs.

This guide covers the Python a DevOps engineer actually uses: the parts of the language that matter, the standard library modules you will reach for every week, a handful of real automation scripts you can run today, the mistakes beginners make, and a realistic six-week practice plan.

Why Python, and why not just Bash?

Bash is excellent for gluing commands together on a Linux box. If you have not got comfortable with the shell yet, start with our guide to Linux skills for cloud jobs, because Python does not replace it. But Bash becomes painful once a script needs to handle JSON, call an API, retry on failure, or run on Windows as well as Linux.

Python sits in the space between a one-line shell command and a full application. It is readable, it ships on most Linux distributions, and nearly every cloud provider and DevOps tool has a Python library or API you can call.

A sensible rule: if a shell script grows beyond a screen, needs error handling, or touches structured data, rewrite it in Python.

The Python you actually need

You do not need decorators, metaclasses or async programming to start. You need these, roughly in this order:

1. Variables, strings and f-strings for building messages and paths.
2. Lists and dictionaries, because almost everything a cloud API returns is a dictionary containing lists of dictionaries.
3. Loops and conditionals to walk through servers, files or resources.
4. Functions so your script is made of small named pieces you can test.
5. Exceptions (try / except) so a single failed call does not crash the whole job.
6. Modules and virtual environments so you can install libraries without breaking the system Python.
7. Reading and writing files, including JSON and YAML.
8. Command-line arguments so the same script works for staging and production.

If you can do those eight things confidently, you can write most of the automation a junior DevOps engineer is asked for.

Setting up properly

Install Python 3 (any currently supported 3.x release is fine; check python.org for which versions are still receiving updates). On Ubuntu or Debian:

sudo apt update
sudo apt install -y python3 python3-venv python3-pip
python3 --version

On Windows, install from python.org or the Microsoft Store and tick "Add python.exe to PATH". Better still, use WSL so your scripts run in the same Linux environment as your servers.

Always work inside a virtual environment. Newer Ubuntu and Debian releases will refuse a system-wide pip install for exactly this reason.

mkdir ops-scripts && cd ops-scripts
python3 -m venv .venv
source .venv/bin/activate        # on Windows: .venv\Scripts\activate
pip install requests boto3 pyyaml
pip freeze > requirements.txt

Commit requirements.txt to Git and add .venv/ to .gitignore. Anyone (or any CI runner) can then rebuild the same environment with pip install -r requirements.txt.

The standard library modules that matter

Before installing anything, learn what Python already ships with. These are the modules you will use constantly:

• pathlib for file paths that work on Linux and Windows.
• subprocess for running shell commands and capturing their output.
• json for reading and writing JSON.
• os for environment variables (os.environ).
• argparse for command-line flags.
• logging for proper log output instead of scattered print calls.
• shutil for copying, moving and checking disk usage.
• datetime for timestamps and working out ages of files or backups.

And three third-party libraries worth knowing early: requests (HTTP), boto3 (AWS SDK) and PyYAML (YAML files such as Kubernetes manifests and CI config).

Example 1: a disk space checker

A classic first script. It checks free space and exits with a non-zero code if any path is above a threshold, which means cron, a CI job or a monitoring agent can react to it.

#!/usr/bin/env python3
"""Warn when disk usage crosses a threshold."""
import argparse
import shutil
import sys


def usage_percent(path: str) -> float:
    total, used, _free = shutil.disk_usage(path)
    return used / total * 100


def main() -> int:
    parser = argparse.ArgumentParser(description="Check disk usage")
    parser.add_argument("paths", nargs="+", help="Mount points to check")
    parser.add_argument("--threshold", type=float, default=80.0)
    args = parser.parse_args()

    status = 0
    for path in args.paths:
        pct = usage_percent(path)
        flag = "OK"
        if pct >= args.threshold:
            flag = "HIGH"
            status = 1
        print(f"{flag:<5} {path:<15} {pct:5.1f}%")
    return status


if __name__ == "__main__":
    sys.exit(main())

Run it:

python3 check_disk.py / /home --threshold 75
echo $?

Notice the structure: a small function that does one thing, a main() that handles arguments, and an exit code. Every script in this guide follows that pattern, and it is the habit that separates a useful script from a throwaway one.

Example 2: running shell commands safely

You will often wrap existing tools such as git, docker or kubectl. Use subprocess.run with a list of arguments, not a single string with shell=True. The list form avoids quoting bugs and shell injection.

import subprocess


def run(cmd: list[str]) -> str:
    result = subprocess.run(cmd, capture_output=True, text=True, check=False)
    if result.returncode != 0:
        raise RuntimeError(f"{' '.join(cmd)} failed: {result.stderr.strip()}")
    return result.stdout.strip()


branch = run(["git", "rev-parse", "--abbrev-ref", "HEAD"])
commit = run(["git", "rev-parse", "--short", "HEAD"])
print(f"Building {branch} at {commit}")

This is the kind of helper you will find in many release and build scripts: get the current commit, tag a Docker image with it, push it. If you want to practise the Git side of this, the Git and GitHub for DevOps guide covers the commands you would be wrapping.

Example 3: a health check with retries

Checking that a service is up after a deployment is a daily job. This script polls a list of URLs, retries with a short back-off, and fails loudly if anything is still down.

#!/usr/bin/env python3
import sys
import time

import requests

URLS = [
    "https://example.com",
    "https://example.com/health",
]


def check(url: str, attempts: int = 3, wait: int = 5) -> bool:
    for attempt in range(1, attempts + 1):
        try:
            resp = requests.get(url, timeout=10)
            if resp.status_code == 200:
                print(f"UP    {url} ({resp.elapsed.total_seconds():.2f}s)")
                return True
            print(f"try {attempt}: {url} returned {resp.status_code}")
        except requests.RequestException as exc:
            print(f"try {attempt}: {url} error: {exc}")
        time.sleep(wait)
    print(f"DOWN  {url}")
    return False


if __name__ == "__main__":
    results = [check(u) for u in URLS]
    sys.exit(0 if all(results) else 1)

Two details matter here. The timeout=10 stops the script hanging forever on a dead endpoint (by default requests waits indefinitely). And the retries account for flaky networks, which is useful when you are testing from a home connection that drops now and again.

Example 4: finding forgotten AWS resources with boto3

Cloud bills are charged in dollars, so a forgotten instance hurts. This script lists running EC2 instances in a region and flags any without an Owner tag, which is a common first step in cost clean-up.

First configure credentials with the AWS CLI (aws configure) using an IAM user or, better, IAM Identity Center. Never paste access keys into the script itself.

#!/usr/bin/env python3
import argparse

import boto3


def untagged_instances(region: str) -> list[dict]:
    ec2 = boto3.client("ec2", region_name=region)
    paginator = ec2.get_paginator("describe_instances")
    found = []
    filters = [{"Name": "instance-state-name", "Values": ["running"]}]
    for page in paginator.paginate(Filters=filters):
        for reservation in page["Reservations"]:
            for inst in reservation["Instances"]:
                tags = {t["Key"]: t["Value"] for t in inst.get("Tags", [])}
                if "Owner" not in tags:
                    found.append({
                        "id": inst["InstanceId"],
                        "type": inst["InstanceType"],
                        "launched": inst["LaunchTime"].strftime("%Y-%m-%d"),
                    })
    return found


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--region", default="eu-west-1")
    args = parser.parse_args()
    for item in untagged_instances(args.region):
        print(f"{item['id']}  {item['type']:<12} launched {item['launched']}")

Points to notice:

• Paginators. AWS returns results in pages. Beginners often call describe_instances() once and miss resources. Use paginators wherever boto3 offers them.
• Read-only first. This script only lists. Write the "stop" or "terminate" version only once the listing is correct, and add a --dry-run flag that is on by default.
• Free tier and budgets. Practise in a personal account with a budget alert set to a small amount. If your card is a naira card with dollar limits, a budget alert is the cheapest insurance you can buy. Check the AWS site for current free tier terms, as they have changed over time.

The same idea works on Azure with the azure-identity and azure-mgmt-compute packages, and on Google Cloud with its client libraries. The pattern (authenticate, list, filter, report) does not change.

Example 5: reading and validating YAML config

DevOps lives in YAML: Kubernetes manifests, GitHub Actions workflows, Docker Compose files. A small validator catches mistakes before they reach a cluster.

#!/usr/bin/env python3
import sys
from pathlib import Path

import yaml

REQUIRED_LABELS = {"app", "team"}


def check_file(path: Path) -> list[str]:
    problems = []
    with path.open() as fh:
        for doc in yaml.safe_load_all(fh):
            if not doc or doc.get("kind") != "Deployment":
                continue
            name = doc["metadata"]["name"]
            labels = set(doc["metadata"].get("labels", {}))
            missing = REQUIRED_LABELS - labels
            if missing:
                problems.append(f"{path}: {name} missing labels {sorted(missing)}")
            for c in doc["spec"]["template"]["spec"]["containers"]:
                if ":" not in c["image"] or c["image"].endswith(":latest"):
                    problems.append(f"{path}: {name}/{c['name']} uses an untagged or latest image")
                if "resources" not in c:
                    problems.append(f"{path}: {name}/{c['name']} has no resource limits")
    return problems


if __name__ == "__main__":
    all_problems = []
    for p in Path(sys.argv[1] if len(sys.argv) > 1 else ".").rglob("*.yaml"):
        all_problems.extend(check_file(p))
    print("\n".join(all_problems) or "All manifests look fine")
    sys.exit(1 if all_problems else 0)

Always use yaml.safe_load or safe_load_all, never plain yaml.load on files you did not write. If Kubernetes is new to you, read a Kubernetes beginners guide first so the fields make sense.

Example 6: cleaning up old backups

A script that deletes files older than a set number of days, with a dry-run mode on by default and proper logging.

#!/usr/bin/env python3
import argparse
import logging
import time
from pathlib import Path

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger("cleanup")


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("directory", type=Path)
    parser.add_argument("--days", type=int, default=14)
    parser.add_argument("--delete", action="store_true", help="Actually delete files")
    args = parser.parse_args()

    cutoff = time.time() - args.days * 86400
    for f in args.directory.glob("*.tar.gz"):
        if f.stat().st_mtime < cutoff:
            if args.delete:
                f.unlink()
                log.info("deleted %s", f)
            else:
                log.info("would delete %s", f)


if __name__ == "__main__":
    main()

Schedule it with cron once you trust it:

crontab -e
# run at 02:30 every day
30 2 * * * /home/ubuntu/ops-scripts/.venv/bin/python /home/ubuntu/ops-scripts/cleanup.py /var/backups/app --days 14 --delete >> /var/log/cleanup.log 2>&1

Note the full path to the virtual environment's Python. Cron does not activate your venv, and "it works in my terminal but not in cron" is one of the most common support questions.

Running your scripts in CI

Scripts become much more valuable when they run automatically. A minimal GitHub Actions job that runs the YAML validator on every pull request:

name: validate-manifests
on: [pull_request]

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - run: python check_manifests.py k8s/

Because the script exits with code 1 on problems, the pipeline fails and the pull request is blocked. Check the GitHub Marketplace for the latest major versions of these actions. Our CI/CD with GitHub Actions guide goes further.

Testing your scripts

Even a short script deserves a test or two, especially once it deletes things. Install pytest and test the pure functions:

# test_check_disk.py
from check_disk import usage_percent


def test_usage_is_a_percentage():
    pct = usage_percent("/")
    assert 0 <= pct <= 100

pip install pytest
pytest -q

This is why the examples keep logic in small functions: functions can be tested, a 200-line block at the top level of a file cannot.

Common mistakes

• Hard-coding secrets. Read them from environment variables (os.environ["API_TOKEN"]), a secrets manager or your CI's secret store. Run git log -p before pushing if you are unsure what you committed.
• Installing packages system-wide. Use a virtual environment every time.
• No timeouts on network calls. One dead endpoint can hang a pipeline for hours.
• Using `shell=True` with user input. Pass a list to subprocess.run instead.
• Ignoring pagination in cloud SDKs, so the script silently misses resources.
• Destructive by default. Make deletion an explicit flag, and log what would happen first.
• Printing instead of exiting. A script that prints "ERROR" but exits with 0 tells CI and cron that everything is fine.
• Learning only from video. Watching someone write a script is not the same as debugging your own. Type everything.
• Trying to learn all of Python first. Learn enough to automate something real, then go deeper as problems demand it.

A realistic six-week practice plan

This assumes around an hour a day on weekdays and a longer session at the weekend. If power or data is unreliable, do the reading and writing offline and save the cloud steps for when you have a stable connection. Most of the work below runs on a laptop or a free Linux environment.

Week 1: foundations. Install Python, create a virtual environment, and practise strings, lists, dictionaries, loops and functions. Write a script that reads a text file of server names and prints them sorted and de-duplicated. The Python Arena is a good place for short daily drills.

Week 2: files and the shell. Use pathlib, json and subprocess. Build the disk checker and a script that prints the current Git branch and last commit. Add argparse to both.

Week 3: HTTP and APIs. Learn requests. Build the health checker. Then call a public API (the GitHub REST API works without a key for low request rates) and print your repositories as a table.

Week 4: the cloud SDK. Create a free-tier account with a budget alert. Use boto3 to list S3 buckets and EC2 instances, then build the untagged-instance report. Delete anything you create at the end of each session.

Week 5: YAML, logging and tests. Build the manifest validator, switch your scripts from print to logging, and add a few pytest tests.

Week 6: automate it. Put your scripts in a GitHub repository with a README, requirements.txt and a GitHub Actions workflow that runs tests and the validator. This repository is now a portfolio piece; see cloud portfolio projects for how to present it.

By the end you will not be a software engineer, but you will be able to read other people's scripts, write your own, and talk about them confidently in an interview.

FAQ

Do I need to learn Python before Docker, Terraform or Kubernetes?

No. Linux and Git come first. Learn Python alongside your other tools, using it to automate what you are already doing. Terraform and Kubernetes use their own configuration languages, so you can learn them without much Python.

Python or Go for DevOps?

Start with Python. It is easier to learn and more widely used for day-to-day scripting. Go is common for building DevOps tools themselves (Docker, Kubernetes and Terraform are written in Go), so it is a reasonable second language once you are working.

Which Python version should I use?

Any version still receiving updates from the Python core team; python.org lists the current supported releases. Avoid Python 2 entirely, and match the version your team or CI uses.

Is there a Python certification worth getting for DevOps?

Cloud and Kubernetes certifications usually carry more weight for DevOps roles than Python ones. A GitHub repository of working, tested scripts is often more convincing to a hiring manager than a Python certificate. If you want a structured path, compare options on the official certification sites and check current prices there.

Can I practise without paying for cloud?

Yes. Everything except Example 4 runs locally. For cloud work, use a free tier with a budget alert, or practise against local tools such as Docker. Browser-based labs also let you practise in the browser.

Getting help with your learning path

Python is one piece of the DevOps toolkit, and it is most useful when it is tied to Linux, Git, containers and the cloud. Our Python for DevOps track and wider courses take you through all of these with hands-on projects and mentor feedback. If you are not sure where to start, or how Python fits your own career switch, book a free consultation and we will help you plan it.