← All projects Scrapers & crawlers · Intermediate

A bot that logs in and files the same report every week

Log in, pick the week, type the same numbers, submit, download the receipt, file it. A browser can do that in a second. The real work is making sure it happens exactly once: not zero times because the session expired, and never twice because a run died halfway through.

You'll have at the end: a browser bot that signs in somewhere, fills the same tedious form, downloads the result and files it — on a schedule, without you.

Why this one

Browser automation usually gets taught as scraping: open a page, pull the data out. Filling in a form is the other half, and it has a property scraping doesn't. Reading is safe to repeat; submitting isn't. A scraper that runs twice gets the same data twice. A bot that submits twice has filed two reports, and someone has to notice and clean up after it.

So the clicking is the easy bit — Playwright will fill in anything that has a label. This sheet is mostly about the bookkeeping around the click: what the bot writes down before it presses Submit, what it writes down after, and when it should stop and ask you instead of trying again.

Before you start

Python 3.10 or newer, and a portal you file something into on a schedule — a timesheet, a weekly status report, a compliance return. If you haven't driven a browser from Python before, do scraping a site that builds itself with JavaScript first; this one assumes you know what a locator is.

Built and tested with Playwright 1.56 against a test portal made to fail in every way described below: sessions that expire, a CSRF token, a week picker, a renamed field, a value it refuses, a receipt download that errors, and a server that saves the report and then shows an error page. The finished bot passes 50 checks across 13 scenarios.

⚠ Automate your own account, on a portal whose terms allow it, filing numbers you would file by hand anyway. If the portal has an API or a CSV import, use that instead — it will outlast any form. And if the report goes to an employer or a client, tell them a script files it.

Step 1 — Install

pip install playwright python-dotenv
playwright install chromium

The second command downloads the browser the bot drives, a few hundred megabytes. On Linux, add --with-deps if Chromium complains about missing system libraries. python-dotenv reads the settings file you'll write in Step 5.

Step 2 — Find the labels

The bot finds fields the way you do: by the words next to them. Before writing any code, let Playwright show you what those words are:

playwright codegen https://portal.example.com/login

A browser opens beside a recorder. Log in and fill in the form by hand, and the recorder writes a line of Python for each thing you do, with a locator for each field — usually its role and label. Those names go into the bot. Close the window before you press Submit: it's a real browser, and the portal will take whatever you send it.

Labels beat CSS selectors here for a specific reason. An ID like #hrs can survive a change to what the field means, so a selector-driven bot keeps typing into it. A label usually changes when the meaning does, and get_by_label("Billable hours") then stops working — which, for a form that files things in your name, is exactly what you want.

Step 3 — The numbers

The bot doesn't decide what to report; you do. Save the numbers as numbers.csv, one row per ISO week:

week,hours,tickets,notes
2026-W37,31.5,18,Two new clients
2026-W38,34,22,Q3 audit prep
2026-W39,29.5,15,Conference week
2026-W40,36,24,

They could come from anywhere — a time-tracker export, a spreadsheet, a database query. Keeping them in a file you fill in means the bot has nothing to guess, and a missing row stops the run instead of filing zeros.

Step 4 — The bot

Save as file_report.py, next to numbers.csv. This is the complete file. Everything specific to the test portal is a label, a button name, a URL path, or the element that shows the confirmation number (get_by_test_id("ref")) — change those to what the recorder showed you.

"""
Log in to a web portal, file the same weekly report, and keep the receipt.

    python file_report.py              # files last week
    python file_report.py 2026-W38     # files a specific week

Settings come from the environment, or from a .env file next to this script:
    REPORT_URL=https://portal.example.com
    REPORT_USER=you
    REPORT_PASS=...
"""
import csv, datetime as dt, os, sqlite3, sys
from pathlib import Path
from dotenv import load_dotenv
from playwright.sync_api import sync_playwright

HERE = Path(__file__).resolve().parent     # a scheduler won't start you in this folder...
load_dotenv(HERE / ".env")                 # ...or hand you your environment variables
BASE = os.environ["REPORT_URL"].rstrip("/")
USER, PASSWORD = os.environ["REPORT_USER"], os.environ["REPORT_PASS"]
STATE = HERE / "state"; STATE.mkdir(exist_ok=True)
SESSION = STATE / "session.json"           # the saved login: treat it like the password
FILED = HERE / "filed"                     # receipts, one folder per week


def log(msg):
    """Print it and keep it. At 9am on a Monday nobody is watching the console."""
    print(msg)
    stamp = f"{dt.datetime.now():%Y-%m-%d %H:%M}  "
    with open(HERE / "run.log", "a", encoding="utf-8") as f:
        f.write(stamp + msg.replace("\n", "\n" + " " * len(stamp)) + "\n")


def why(e):
    """Playwright errors run to many lines. Keep the ones that say what it was waiting for."""
    lines = [l.strip(" -=") for l in str(e).splitlines()]
    return "\n".join(l for l in lines if l and l not in ("Call log:", "logs"))


def stop(msg):
    log(msg)
    sys.exit(1)


def last_week():
    y, w, _ = (dt.date.today() - dt.timedelta(days=7)).isocalendar()
    return f"{y}-W{w:02d}"                 # ISO year + week: what <input type="week"> takes


def the_numbers(week):
    """The report's content: a CSV you keep, one row per week."""
    with open(HERE / "numbers.csv", newline="", encoding="utf-8") as f:
        for row in csv.DictReader(f):
            if row["week"] == week:
                return row
    stop(f"No row for {week} in numbers.csv - nothing filed.")


def ledger():
    db = sqlite3.connect(STATE / "filed.db")
    db.execute("CREATE TABLE IF NOT EXISTS filed (week TEXT PRIMARY KEY, ref TEXT, receipt TEXT, at TEXT)")
    return db


def log_in(page):
    page.goto(f"{BASE}/login")
    page.get_by_label("Username").fill(USER)
    page.get_by_label("Password").fill(PASSWORD)
    page.get_by_role("button", name="Sign in").click()
    page.wait_for_load_state()
    if "/login" in page.url:
        # One attempt only. A bot that retries a wrong password locks the account.
        stop("Login failed - check REPORT_USER / REPORT_PASS. Not retrying.")


def main():
    week = sys.argv[1] if len(sys.argv) > 1 else last_week()
    db = ledger()
    done = db.execute("SELECT ref, receipt FROM filed WHERE week = ?", (week,)).fetchone()
    if done:
        log(f"{week} is already filed ({done[0]}). Nothing to do." + ("" if done[1] else " No receipt was saved."))
        return
    sending = STATE / f"submitting-{week}"     # exists only while a submit may be in flight
    if sending.exists():
        stop(f"{week} may already be on the portal: the last run stopped after Submit.\n"
             f"Check the portal. If it isn't there, delete this file and run again:\n{sending}")
    numbers = the_numbers(week)

    with sync_playwright() as p:
        browser = p.chromium.launch()
        ctx = browser.new_context(accept_downloads=True,
                                  storage_state=SESSION if SESSION.exists() else None)
        page = ctx.new_page()
        try:
            page.goto(f"{BASE}/report")
            if "/login" in page.url:                 # no session yet, or it expired
                log_in(page)
                page.goto(f"{BASE}/report")
            # Every field first: if a label has changed, this raises
            # before the submit button is ever touched.
            page.get_by_label("Reporting week").fill(week)
            page.get_by_label("Billable hours").fill(numbers["hours"])
            page.get_by_label("Tickets closed").fill(numbers["tickets"])
            page.get_by_label("Notes").fill(numbers.get("notes") or "")
            sending.touch()                          # from here on, it may have been sent
            page.get_by_role("button", name="Submit report").click()
            page.get_by_test_id("ref").or_(page.get_by_role("alert")).wait_for()
            alert = page.get_by_role("alert")
            if alert.is_visible() and "already been filed" not in alert.inner_text():
                sending.unlink()                     # refused outright: nothing was filed
                raise RuntimeError(f"Portal refused the report: {alert.inner_text()}")
            ref = "filed-elsewhere" if alert.is_visible() else page.get_by_test_id("ref").inner_text()
            now = dt.datetime.now().isoformat(timespec="seconds")
            db.execute("INSERT INTO filed VALUES (?, ?, '', ?)", (week, ref, now))
            db.commit()                              # written down before anything else can fail
            sending.unlink()
            if ref == "filed-elsewhere":             # someone filed it by hand
                log(f"{week} was already filed on the portal. Recorded, not resubmitted.")
                return
            with page.expect_download() as download:
                page.get_by_role("link", name="Download receipt").click()
            receipt = FILED / week / download.value.suggested_filename
            receipt.parent.mkdir(parents=True, exist_ok=True)
            download.value.save_as(receipt)
            db.execute("UPDATE filed SET receipt = ? WHERE week = ?", (str(receipt.relative_to(HERE)), week))
            db.commit()
            log(f"Filed {week}: {ref} -> {receipt.relative_to(HERE)}")
        except Exception as e:
            shot = STATE / f"failed-{week}.png"
            page.screenshot(path=shot, full_page=True)
            log(f"Could not file {week}: {why(e)}\nScreenshot of where it stopped: {shot.relative_to(HERE)}")
            raise
        finally:
            ctx.storage_state(path=SESSION)          # keep whichever session we ended with
            browser.close()


if __name__ == "__main__":
    main()

The ledger (state/filed.db) is checked before a browser is launched. A week that's in it is done: the run costs nothing and touches nothing.

Every field is filled before Submit is pressed. If a label has changed, fill raises before the button is touched, so a half-understood form is never sent.

The marker file state/submitting-<week> is created a moment before Submit and deleted once the outcome is written down. If a run dies in between, the marker is still there next time, and the bot refuses to guess.

The reference number is recorded the moment the portal shows it — before the receipt download, which is the step most likely to fail afterwards.

One login attempt. A wrong password stops the run. Retrying locks accounts.

The session is saved to state/session.json after every run and reused, so most weeks it doesn't log in at all. When the session has expired, the portal sends it to the login page; the bot notices and signs in again.

Step 5 — Run it

Put your settings in a file called .env, next to the script:

REPORT_URL=https://portal.example.com
REPORT_USER=you@example.com
REPORT_PASS=your-password

Then file a week, and file it again:

$ python file_report.py 2026-W37
Filed 2026-W37: RPT-FFAD0A -> filed/2026-W37/receipt-2026-W37-RPT-FFAD0A.csv

$ python file_report.py 2026-W37
2026-W37 is already filed (RPT-FFAD0A). Nothing to do.

$ python file_report.py 2026-W36
2026-W36 was already filed on the portal. Recorded, not resubmitted.

The first run logged in, filed, and saved the portal's receipt in about a second. The second never opened a browser — the ledger answered first, and the portal's request log shows nothing arrived. Week 36 had already been filed by hand; the portal said so, and the bot wrote that down instead of treating it as an error. Every line also goes into run.log with the time, because a scheduled job has nobody watching its console.

With no argument, it files last week — the ISO week before the current one. That's what the schedule in Step 7 relies on.

Step 6 — Break it on purpose

Before trusting it with a real portal, watch it fail. Two failures you will eventually meet: the portal renames a field, and your password changes. Here the test portal's "Billable hours" became "Hours billed (client)", and then .env got a wrong password:

$ python file_report.py 2026-W41
Could not file 2026-W41: Locator.fill: Timeout 30000ms exceeded.
waiting for get_by_label("Billable hours")
Screenshot of where it stopped: state/failed-2026-W41.png

$ python file_report.py 2026-W41
Login failed - check REPORT_USER / REPORT_PASS. Not retrying.

The first names the label it couldn't find (Playwright's traceback follows), and the screenshot shows the form exactly as the bot saw it: the week filled in, the renamed field empty, Submit never pressed. Nothing reached the portal and no marker was left, so once you've updated the label the same command files the week. The second made exactly one login attempt.

Step 7 — Put it on a schedule

Run it every day, not once a week. A laptop that's asleep at 9 on a Monday misses a weekly job; a daily one catches up the next morning, and the ledger makes the other six runs free — each one checks, finds the week filed, and exits without opening a browser.

The scheduler needs the full path to the Python that has Playwright installed. This prints it:

python -c "import sys; print(sys.executable)"

macOS and Linux: run crontab -e and add a line with that path and the full path to the script:

# 09:00 every day: file last week's report if it isn't filed yet
0 9 * * *  /usr/bin/python3 /home/you/weekly-report/file_report.py

On a Mac, cron isn't allowed into Desktop, Documents or Downloads unless you give it Full Disk Access, so keep the folder somewhere else — ~/weekly-report is fine.

Windows: open Task Scheduler and choose Create Basic Task. Trigger: Daily, 09:00. Action: Start a program, with the Python path as the program and the full path to file_report.py, in quotes, as the argument. Leave Start in empty. Then open the task's Properties and, on the Settings tab, tick "Run task as soon as possible after a scheduled start is missed".

Neither scheduler starts your script in its own folder or hands it the variables you set in your terminal — cron starts in your home folder, Task Scheduler in C:\Windows\System32. That's why the bot finds everything relative to itself and reads .env on its own. After the first scheduled run, read run.log.

⚠ Your password now lives in .env, and state/session.json works as a login for as long as the session lasts. Keep this folder out of anything that syncs or gets committed — git, Dropbox, OneDrive.

When it goes wrong

It filed the same week twice. The first version of this bot wrote to its ledger after downloading the receipt. Then the test portal's receipt download started failing: the report went in, the download errored, the ledger stayed empty, and the next run filed the week again. On a portal that rejects duplicates you'd never notice; on one that doesn't — and plenty don't — that's two reports. The fix is ordering. The reference is recorded the moment it appears, and a rerun after a failed download says 2026-W43 is already filed (RPT-E8CBFA). Nothing to do. No receipt was saved.

The portal saved it, then showed an error. The worst case, and not a rare one: a timeout or an error page after the server has already stored the submission. There's no reference to record, and the first version filed again on the next run. That's what the marker file is for. It exists from just before Submit until the outcome is written down, so a run that dies in between leaves it behind, and the next run stops:

$ python file_report.py 2026-W44
2026-W44 may already be on the portal: the last run stopped after Submit.
Check the portal. If it isn't there, delete this file and run again:
/home/you/weekly-report/state/submitting-2026-W44

A bot that isn't sure whether it submitted must not submit again. Asking you costs a minute; guessing wrong costs a duplicate that someone else has to find.

The "more robust" version filed a blank report, and said it had filed. The tempting fix for a renamed label is to skip fields that can't be found. With that one-line change the test portal accepted a report with the hours empty, and the log read Filed 2026-W41 like any other week. A portal that validates on the server would have refused it; the test portal, like plenty of real ones, took what it was given. Let it fail.

Malformed value on the week field. <input type="week"> only takes the ISO format: 2026-W38, a four-digit ISO year, a capital W, a two-digit week. 2026-W9, a date, and weeks that don't exist (2025-W53) are all refused. The trap is strftime("%Y-W%W"), which looks right and isn't ISO. In 2026 it runs a week behind all year — it calls 17 September week 37, not 38 — so a Monday run quietly files the week before last. In the first days of January it produces week 00, which the form refuses. isocalendar() gets the ISO year and week together.

Four login attempts in half a second. A variant that retried a failed login made them against the test portal before anyone could have noticed. Many real portals lock the account after three to five failures, and then someone has to phone IT. One attempt, then stop.

KeyError: 'REPORT_URL', but only on the schedule. The first version read its settings from environment variables and its files from the current folder. Run by hand, both were there. Started the way a scheduler starts it — from another folder, with a bare environment — it died before doing anything. Hence HERE and .env.

A failing run takes 30 seconds. That's Playwright waiting for a label that never appears: every action waits up to 30 seconds by default. Leave it. The same patience is what gets a slow portal through on a bad day, and a weekly job can spare the time.

The tests passed, and the bot was still wrong. The first version passed all 26 checks written for it and still had both double-filing bugs, because no test had tried a portal that fails after Submit. So the suite was checked the other way round: the bot was broken on purpose, eight different ways, and each broken copy had to fail at least one check. Every one did:

fails  broken copy
  4    no ledger check
  4    skips fields it can't find
  1    retries a failed login
  2    doesn't save the session
  3    ledger written after receipt
  3    no marker before Submit
  3    paths from the current folder
  1    week from strftime("%Y-W%W")

If a deliberately broken bot passes your tests, the tests aren't testing that. Every check here counts what reached the portal — requests in its log, reports in its records — not what the bot said about itself.

What this costs

Nothing to run: no API keys, no model calls. Chromium takes a few hundred megabytes of disk, once. A run takes about a second when it files, a fraction of a second when the week is already done, and about thirty seconds when something is wrong.

Make it yours

Hear about failures. Every failure exits with status 1 and leaves a line in run.log, but nobody reads logs. Send yourself a message from stop() and the except block — the send() function from the inbox digest bot drops straight in.

Two-factor login. Don't automate the second factor. Sign in by hand once with playwright codegen --save-storage=state/session.json https://portal.example.com/login, close the window, and the bot uses that session until it expires. When it does, the run fails with a screenshot of the login page — your cue to sign in again.

Watch it, or replay it. p.chromium.launch(headless=False, slow_mo=500) lets you watch a run. For the failures you didn't see, record a trace: ctx.tracing.start(screenshots=True, snapshots=True) after creating the context, ctx.tracing.stop(path=STATE / "trace.zip") in the finally, and playwright show-trace state/trace.zip replays every step with the page as it was.

Take the numbers from where they already are. the_numbers() is the only function that knows about the CSV. Point it at a time-tracker export or a spreadsheet's CSV download and nothing else changes.

More than one form. Give each form its own filling function and key the ledger on form and week together. The marker file and the ordering carry over unchanged — they're the hard part, and they don't care what the form is.

What you just learned: a form submission is a one-way door. Write down what you're about to do before you do it, write down what happened the moment it happens, and when the bot can't tell whether it went through, make it stop and ask.
You got to the end

Did you build it?

Ticking it off keeps your pathway honest and moves the next build to the top of every page.

One new build sheet a week

A new build sheet every Thursday, written by a person who built it. No hype, no roundups.