ImageScraper (IFSTA Tools Suite)

An upgraded IFSTA shop scanning, image discovery, and product page audit toolkit.

Tools in this Suite

1. ImageScraper (index.html): Product image lookup by 5-digit SKU, size exploration, valid SKU discovery, and ZIP batch downloading.
2. Sitemap Navigator (map.html): Interactive browser for IFSTA sitemap URLs with content image verification.
3. Product Page Audit (audit.html): Automated, read-only public catalog auditor for product display metadata, Latest Edition banners, and IFSTA Validated markers.

---

Product Page Audit (/audit.html)

Purpose & Scope

The /audit.html tool inspects public IFSTA product display pages to identify:

• Product SKU & SKU Consistency: Compares structured metadata SKU, primary image URL SKU, and page URL tail segment.
• Latest Edition Status (latestBannerRendered): Detects whether the public page renders the "Newest Edition" graphical banner (latest-edition.png).
• IFSTA Validated Status (validatedMarkerRendered): Checks for the .product-image-wrapper.validated DOM marker.
• Node ID & Public URL: Extracts the Drupal node ID from shortlinks for catalog cross-referencing.

Architecture & Proxy

• Client Frontend (audit.html): Vanilla JavaScript single-page application with dark/light themes, instant multi-filter search, progress tracking, live KPI counters, and direct RFC-4180 CSV export.
• Serverless Backend (functions/fetch-image.js): Netlify function running Node 18, node-fetch, and cheerio.
  • action=getSitemapUrls: Parses https://www.ifsta.org/sitemap.xml.
  • action=auditProductPage&url={URL}: Fetches a single public page, validates approved IFSTA hostname, parses DOM structures in one round-trip, and returns normalized audit JSON.

Status Definitions & Evidence Model

• latestBannerRendered:
  • yes: Banner graphic (latest-edition.png / Newest Edition alt text) detected in confirmed product fields.
  • no: Confirmed product display structure present without any Latest Edition banner.
  • unknown: Incomplete or unrecognized page structure where product fields could not be confirmed.
• validatedMarkerRendered:
  • yes: .product-image-wrapper.validated class present.
  • no: .product-image-wrapper present without .validated class.
  • unknown: Image wrapper element not found.
• skuStatus:
  • match: All available SKU sources (metadata, page URL, primary image URL) agree.
  • partial: SKU confirmed from a single source without conflicting data.
  • mismatch: Disagreement between metadata, URL, or image SKU values.
  • unknown: No valid 5-digit SKU could be extracted.
• scanStatus:
  • ok: Page audited successfully.
  • not_product: URL does not represent an IFSTA product display.
  • error: HTTP error, 404, network failure, or access restriction.
  • unknown: Structural ambiguity prevented definitive classification.

Known Control Products

The audit engine is validated against three reference controls:

• SKU 37888 (Essentials 8th Edition: Firefighter 1): Latest Edition = Yes, Validated = Yes
• SKU 36777 (Essentials 7th Edition): Latest Edition = No, Validated = Yes
• SKU 36329 (Leadership for the Wildland Fire Officer, 2nd Edition): Latest Edition = Yes, Validated = No

Persistence & Reset

Audit sessions are persisted in browser localStorage under key ifstaProductAudit:v1. To reset state, click the Clear Results button in the audit control bar.

---

Limitations & Disclaimers

• Public Output vs. Drupal Database State: This tool audits what public web pages render (latestBannerRendered). It does not directly query the PostgreSQL Drupal table field_data_field_latest_edition. Cached CDN/Varnish pages may lag behind internal database updates until caches clear.
• Unpublished Products: Unpublished or staging content requiring Drupal login authentication is out of scope.
• Read-Only: The tool performs no write operations against Drupal, xTuple, or the image asset server.

---

Hosted Deployments

• Frontend: https://aaronfpp.github.io/image-scraper/
• Sitemap Navigator: https://aaronfpp.github.io/image-scraper/map.html
• Product Page Audit: https://aaronfpp.github.io/image-scraper/audit.html