Jupyter Notebook Bash: Tips and Tricks for DevOps

You’ve figured out how to run bash in Jupyter Notebook. Now let’s make it actually useful for DevOps work.

These tips will help you build better infrastructure notebooks—and show you where Jupyter’s limits are.

Tip 1: Set Up Your Environment First

Create a setup cell at the top of every DevOps notebook:

import os

# Ensure tools are in PATH
os.environ["PATH"] += ":/usr/local/bin:/opt/homebrew/bin"

# Set Kubernetes context
os.environ["KUBECONFIG"] = os.path.expanduser("~/.kube/config")

# Set default namespace
os.environ["KUBECTL_NAMESPACE"] = "production"

# AWS/GCP credentials
os.environ["AWS_PROFILE"] = "production"

print("Environment configured")
!kubectl config current-context

Run this first. All subsequent cells inherit these settings.

Tip 2: Create Reusable Shell Functions

Define bash functions you can reuse:

%%bash
# Save to a temp file that persists
cat > /tmp/k8s-helpers.sh << 'EOF'
kpods() {
    kubectl get pods -n "${1:-default}" -o wide
}

klogs() {
    kubectl logs -f "$1" -n "${2:-default}" --tail=100
}

kexec() {
    kubectl exec -it "$1" -n "${2:-default}" -- /bin/sh
}

kdesc() {
    kubectl describe pod "$1" -n "${2:-default}"
}
EOF
echo "Helper functions created"

Then source them in subsequent cells:

%%bash
source /tmp/k8s-helpers.sh
kpods production

Tip 3: Handle Long-Running Commands

For commands that take time, add progress indicators:

%%bash
echo "Starting rollout..."
kubectl rollout restart deployment/api -n production

echo "Waiting for rollout to complete..."
kubectl rollout status deployment/api -n production --timeout=300s

echo "✓ Rollout complete"
kubectl get pods -n production -l app=api

Or use Python for more control:

import subprocess
import sys

def run_with_output(cmd):
    process = subprocess.Popen(
        cmd,
        shell=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
        text=True
    )
    for line in process.stdout:
        print(line, end="")
        sys.stdout.flush()
    return process.wait()

run_with_output("kubectl rollout status deployment/api -n production")

Tip 4: Build Interactive Dashboards

Combine bash output with IPython widgets:

import ipywidgets as widgets
from IPython.display import display, clear_output

namespace = widgets.Dropdown(
    options=["production", "staging", "development"],
    value="production",
    description="Namespace:"
)

def get_pods(change):
    clear_output(wait=True)
    display(namespace)
    !kubectl get pods -n {namespace.value}

namespace.observe(get_pods, names="value")
display(namespace)
get_pods(None)

Now you can switch namespaces with a dropdown.

Tip 5: Structured Output Parsing

Parse JSON output for better displays:

import json
import pandas as pd

# Get pods as JSON
pods_json = !kubectl get pods -n production -o json
pods_data = json.loads("\n".join(pods_json))

# Convert to DataFrame
pods_list = []
for pod in pods_data["items"]:
    pods_list.append({
        "name": pod["metadata"]["name"],
        "status": pod["status"]["phase"],
        "restarts": pod["status"]["containerStatuses"][0]["restartCount"],
        "age": pod["metadata"]["creationTimestamp"]
    })

df = pd.DataFrame(pods_list)
df

This gives you a nice table instead of raw output.

Tip 6: Error Handling

Wrap commands with proper error handling:

def run_kubectl(cmd):
    """Run kubectl command with error handling"""
    import subprocess
    
    result = subprocess.run(
        f"kubectl {cmd}",
        shell=True,
        capture_output=True,
        text=True
    )
    
    if result.returncode != 0:
        print(f"❌ Error: {result.stderr}")
        return None
    
    print(result.stdout)
    return result.stdout

# Usage
run_kubectl("get pods -n production")
run_kubectl("get pods -n nonexistent")  # Will show error nicely

Tip 7: Secrets Management

Never hardcode secrets. Use environment variables or secret managers:

import os
from getpass import getpass

# Prompt for sensitive values
if "API_TOKEN" not in os.environ:
    os.environ["API_TOKEN"] = getpass("Enter API token: ")

# Now use in bash
!curl -H "Authorization: Bearer $API_TOKEN" https://api.internal/health

Or use a secret manager:

# AWS Secrets Manager example
import boto3
import json

def get_secret(name):
    client = boto3.client("secretsmanager")
    response = client.get_secret_value(SecretId=name)
    return json.loads(response["SecretString"])

secrets = get_secret("prod/api-credentials")
os.environ["DB_PASSWORD"] = secrets["password"]

Tip 8: SSH Through Bastion

Running bash in Jupyter Notebook over SSH is tricky but possible:

# Set up SSH config first
%%bash
cat > ~/.ssh/config << 'EOF'
Host bastion
    HostName bastion.example.com
    User admin
    IdentityFile ~/.ssh/id_rsa

Host prod-*
    ProxyJump bastion
    User admin
EOF

Then run remote commands:

!ssh -o BatchMode=yes prod-web-1 "hostname && uptime"

Limitation

Interactive SSH sessions don’t work. You can only run one-off commands.

Tip 9: Organize Notebooks by Workflow

Structure your notebook like a runbook:

# API Deployment Notebook

## 1. Pre-flight Checks
[Cell: Check cluster connection]
[Cell: Verify current deployment status]
[Cell: Check resource availability]

## 2. Deploy
[Cell: Update image tag]
[Cell: Apply deployment]
[Cell: Wait for rollout]

## 3. Verify
[Cell: Check pod status]
[Cell: Run health checks]
[Cell: Verify metrics]

## 4. Rollback (if needed)
[Cell: Rollback command]

Use markdown cells liberally to explain each step.

Tip 10: Version Control Notebooks

Jupyter’s .ipynb format is JSON, which creates noisy diffs. Clean it up:

# Install nbstripout
pip install nbstripout

# Configure git to strip output
nbstripout --install

# Now commits won't include cell outputs

Or use Jupytext to sync notebooks with plain Python/Markdown:

pip install jupytext
jupytext --set-formats ipynb,md notebook.ipynb

The Limits of Jupyter for DevOps

Even with these tips, running bash in Jupyter Notebook for DevOps has fundamental issues:

Infrastructure Overhead

You need:

Python environment
Jupyter server running
Browser open
Kernel connected

That’s a lot of moving parts for “run a kubectl command.”

Poor Portability

Jupyter notebooks are JSON files that:

Don’t render well on GitHub
Require Jupyter to run
Have ugly diffs
Don’t work in standard editors

No Native SSH

Every SSH workaround is a hack. Jupyter wasn’t built for remote server operations.

Not Incident-Friendly

During an incident, you need speed. Starting Jupyter, waiting for kernels, navigating cells—it’s too slow.

When to Use Something Else

If you’re primarily running bash, consider tools built for that purpose:

Markdown files that can be executed step-by-step
Native terminal support without browser overhead
SSH built-in for remote servers
Git-native format that diffs cleanly

Stew provides exactly this. Write your DevOps procedures in Markdown, execute bash commands directly, run locally or over SSH.

Join the waitlist and try a notebook experience built for ops.