Site Changes

18.2. Site Changes — Catching What Changes on a Website

The standard pattern: script diff + agent evaluation

script is the “secret weapon” for this — a Python script runs before each turn, handling the mechanical part (fetching, diffing), while the agent handles the reasoning part (is this change interesting). This is exactly the “boring stack” architecture described at the start of the chapter: normalized page snapshot → SQLite stores state → diff → agent classifies the importance level → alert.

A real example

/cron add "every 1h" "Compare the new snapshot of page X with the previous one. If there is a significant content change, summarize and alert. If not, reply [SILENT]." --script watch_site.py

Notes

• Always store the previous snapshot (SQLite or a file) to compare against — without persistent state, each run has no way of knowing “changed compared to what”.
• Start deterministic: use the LLM only to evaluate pre-filtered diffs, do not let the agent crawl on its own and “feel out” changes from scratch every time — it is both slow and wastes tokens for nothing.
• For pages that change frequently but not importantly (ad banners, view counters), filter at the script layer first — do not let the agent have to learn to ignore this noise through repeated runs.