A bot that tells you your own site is broken

Why this one

You can look at your own site every day and never notice that the analytics tag stopped firing three deploys ago. The page renders. Nothing is red. The failure is invisible precisely because the page looks fine.

This is the class of bug worth automating: silent, cheap to check, expensive to miss. Nobody can hold twenty sites' worth of invisible invariants in their head. A twenty-line loop can.

It also teaches the thing that separates a check from a guess — every assertion has to be falsifiable, and you have to prove it can fail before you trust it passing. That turns out to be the hard part, and it is most of the lesson.

Before you start

You need Python 3.10 or newer and a list of your own domains. No API keys and no accounts — every check here reads publicly served HTML.

⚠ Only point this at sites you own. It sends real requests, and one check below submits a real form. On someone else's site that is abuse; on your own it is a smoke test.

Step 1 — Install

pip install requests beautifulsoup4

requests fetches, beautifulsoup4 parses. No framework, no headless browser. If a check needs a real browser you have usually picked the wrong check.

Step 2 — The site list

Keep it in a file, not in the script. You will add sites faster than you edit code, and a data file is something you can hand to someone else.

{
  "expected_ga4": "G-XXXXXXXXXX",
  "sites": [
    {"slug": "example",  "url": "https://example.com/"},
    {"slug": "internal", "url": "https://internal.example.com/", "expect_status": 401}
  ]
}

expect_status matters more than it looks. A password-gated staging site returning 401 is correct, and a checker that cannot express “this is supposed to be locked” will cry wolf until you stop reading it.

Step 3 — The checks

Save as check.py. Each check takes the parsed page and appends findings with a severity. Keeping them as separate small functions is what lets you add the twentieth check without touching the first nineteen.

import re, sys, json
import requests
from bs4 import BeautifulSoup

CRITICAL, WARN = "critical", "warn"


def check_analytics(page, findings, expected):
    ids = set(re.findall(r"G-[A-Z0-9]{8,}", page.text))
    if expected not in ids:
        findings.append((WARN, f"analytics tag {expected} missing"))


def check_head(soup, findings):
    if not soup.find("link", rel="canonical"):
        findings.append((WARN, "no canonical tag"))
    if not soup.find("script", attrs={"type": "application/ld+json"}):
        findings.append((WARN, "no structured data"))
    if not soup.find("meta", attrs={"name": "viewport"}):
        findings.append((CRITICAL, "no viewport meta - not responsive on mobile"))
    og = soup.find("meta", property="og:image")
    if not og or not og.get("content"):
        findings.append((WARN, "no og:image - link previews will be blank"))


PLACEHOLDERS = [
    (r"\(?234\)?[\s.-]*567[\s.-]*890", "fake phone number"),
    (r"lorem ipsum", "lorem ipsum"),
]


def check_placeholders(soup, findings):
    clone = BeautifulSoup(str(soup), "html.parser")
    # Labels and buttons legitimately say things like "Your name".
    for t in clone(["script", "style", "label", "legend", "option", "button"]):
        t.decompose()
    text = clone.get_text(" ", strip=True)
    for pattern, label in PLACEHOLDERS:
        if re.search(pattern, text, re.I):
            findings.append((CRITICAL, f"placeholder text live: {label}"))


def check_site(cfg, expected):
    findings = []
    try:
        page = requests.get(cfg["url"], timeout=25)
    except Exception as e:
        return [(CRITICAL, f"unreachable: {type(e).__name__}")]

    want = cfg.get("expect_status", 200)
    if page.status_code != want:
        return [(CRITICAL, f"HTTP {page.status_code} (expected {want})")]
    if want != 200:
        return []

    soup = BeautifulSoup(page.text, "html.parser")
    check_analytics(page, findings, expected)
    check_head(soup, findings)
    check_placeholders(soup, findings)
    return findings


if __name__ == "__main__":
    cfg = json.load(open("sites.json"))
    worst = 0
    for site in cfg["sites"]:
        found = check_site(site, cfg["expected_ga4"])
        worst = max(worst, sum(1 for sev, _ in found if sev == CRITICAL))
        print(f"{site['slug']:<20} {len(found)} finding(s)")
        for sev, msg in found:
            print(f"    [{sev}] {msg}")
    sys.exit(1 if worst else 0)

Note what check_placeholders throws away before it searches. A contact form legitimately contains the words “Your name” — in a <label>. Scanning the whole page for that string reports every working contact form as broken. Strip labels, legends, options and buttons first, and search only the body copy.

Step 4 — Run it

$ python check.py
example              2 finding(s)
    [warn] no og:image - link previews will be blank
    [warn] no structured data
internal             0 finding(s)
shop                 1 finding(s)
    [critical] placeholder text live: fake phone number

The exit code is the point. sys.exit(1) on any critical finding is what lets a scheduler tell you something is wrong without you reading anything.

Step 5 — Put it on a schedule

A checker you run manually is a checker you run twice and forget. GitHub Actions runs it nightly for free:

name: site health
on:
  schedule:
    - cron: "0 9 * * *"
  workflow_dispatch:

jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install requests beautifulsoup4
      - run: python check.py

A failed run emails you. That is the entire notification system, and it is enough.

When it goes wrong

Every single site fails the same check. Suspect the check, not the sites. This is the big one, and it is worth more than the rest of this page.

A real example. Netlify registers a form only if the HTML carries data-netlify="true", so scanning deployed pages for that attribute looks like an obvious way to catch forms that silently discard submissions. Run it across a portfolio and it reports that every form is broken.

They were not. Netlify strips that attribute once it has detected the form, and injects a hidden form-name input in its place. The attribute being gone is the signature of detection having worked. The check was reading the evidence exactly backwards, and a probe that appeared to confirm it rested on the same wrong assumption.

The tell was in the output the whole time: zero out of twenty-one forms carried the attribute. A real defect is never that evenly distributed. A check that fails 100% of a population is almost always the thing that is broken.

The fix is a test that can distinguish the two cases rather than one that merely agrees with you. Static file serving ignores the request body; a form handler does not. So vary the body and watch whether the response changes:

import requests

NONSENSE = "definitely-not-a-real-form-zzq9"


def forms_are_processed(url):
    """
    Safe to schedule. Netlify rejects an unknown form with 404 and stores
    nothing; plain static hosting ignores the body and answers 200 or 405.
    """
    r = requests.post(url, data={"form-name": NONSENSE},
                      allow_redirects=False, timeout=25)
    return r.status_code == 404


def form_is_registered(url, form_name):
    """
    The full two-sided test. It files ONE real submission, so run it by hand
    after a deploy -- never on a schedule.
    """
    known = requests.post(url, data={"form-name": form_name},
                          allow_redirects=False, timeout=25)
    control = requests.post(url, data={"form-name": NONSENSE},
                            allow_redirects=False, timeout=25)
    if known.status_code == control.status_code:
        return False                       # body ignored - nothing processes this
    return control.status_code == 404      # unknown rejected, known accepted

Then prove it can fail. Point it at a host with no form handling at all — GitHub Pages answers the same request with 405, not 404. A check you have never seen fail is not a check, it is a decoration.

Use a fake form name for routine runs. An unknown form is rejected and nothing is stored, so the check costs nothing. Probing with the *real* name files a real submission every run — which is how you end up explaining eighty entries reading healthcheck-ignore@ in a live inbox.

Certificates always look nearly expired. Let's Encrypt issues 90-day certificates and most hosts renew around the 30-day mark, so warning below 30 days fires on every healthy site you own. Warn under 14, go critical under 5.

A same-origin POST is not evidence of anything. On PHP or WordPress it is normal and works. Check what serves the page before judging what the form does.

Your own network counts as a finding. A proxy timeout is not an outage. Retry, and report a site you could not reach differently from a site that is down.

What this costs

Nothing. No API keys, no model calls, no server. GitHub Actions gives public repositories unlimited scheduled minutes and private ones a monthly allowance this will never approach — a nightly run over thirty sites takes about ninety seconds.

Make it yours

Check the pages that matter, not just the homepage. Conversion forms are rarely on /. Pull your sitemap and pick the URLs whose paths contain contact, quote, signup or pricing.

Diff against yesterday. Store each run's findings as JSON and report only what changed. A report that lists the same forty known warnings every morning stops being read by Thursday.

Assert your own invariants. The generic checks here are the boring half. The valuable ones are specific to you: every product page has a price, every article has an author, no page references the staging domain.

Render it. A JSON file is for the scheduler; an HTML page with the criticals expanded is for you. Both come off the same run.

Watch response times. Record how long each homepage takes and flag anything that doubles. It is the cheapest performance regression alarm you will ever write.