What this automation does
This automation scrapes your competitors' pricing pages, product pages, and blog/news sections on a scheduled basis. AI compares the current version against the previous snapshot, identifies meaningful changes, and sends you a weekly digest summarizing what changed and why it matters.
Instead of manually checking competitor websites or setting up dozens of Google Alerts that flood your inbox with noise, you get a single, curated summary. The AI filters out cosmetic changes and highlights pricing shifts, new feature launches, positioning changes, and strategic moves.
Tools you need
- Apify or Browseless: Web scraping service to capture competitor page content ($10-20/mo)
- OpenAI API: GPT-4 compares snapshots and generates analysis ($0.10-0.20 per competitor per week)
- Make or n8n: Orchestrates the scraping schedule, comparison, and email delivery
How to set it up
Step 1: List the URLs you want to track for each competitor — pricing page, features page, homepage, and blog index. Start with 3-5 competitors and 2-3 pages each. Store the URLs in a Google Sheet.
Step 2: Set up Apify actors (or Browseless jobs) to scrape each URL weekly. Store the scraped text content in Google Sheets — one row per URL per scrape date. This creates a historical record you can reference.
Step 3: Create a Make scenario that runs weekly. For each URL, pull the latest scrape and the previous week's scrape. Send both to OpenAI and ask it to identify meaningful changes — ignore layout or styling differences, focus on pricing, features, messaging, and new announcements.
Step 4: Compile the AI's analysis for all competitors into a single digest email. Include a summary at the top with the most important changes, followed by detailed per-competitor breakdowns. Send to your team's strategy channel or email list.
Cost breakdown
| Item | Cost | Notes |
|---|---|---|
| Apify or Browseless | $10-$20/mo | For scraping 15-20 pages weekly |
| OpenAI API | $5-$15/mo | ~$0.15 per comparison, 15-20 comparisons/week |
| Make or n8n | $10-$20/mo | Weekly scheduled scenario |
| Setup time | 40-60 min | One-time |
| Total monthly | $25-$55/mo | Replaces 3+ hours/week of manual monitoring |
Frequently asked questions
Scraping publicly available information is generally legal in most jurisdictions, but always check the website's terms of service. Stick to public-facing pages (pricing, blog, features) and avoid scraping behind login walls. Use reasonable scrape rates — once a week is well within normal browsing behavior.
Some sites block automated requests. Use a scraping service like Apify that rotates IP addresses and uses headless browsers. If a site still blocks you, consider monitoring their RSS feed, social media, or newsletter instead — AI can analyze those sources too.