SpunkArt
Websites, APIs and search

How to run a basic website audit by hand

A site audit is a structured look at everything that can stop a website from working well for visitors and search engines. For a small site you can do a useful one in an afternoon with free, local tools.

Before you start: make an inventory

You cannot audit what you have not listed. Build a spreadsheet with one row per URL. Sources for the list: your sitemap file, your CMS export, the navigation menu and any search-engine webmaster tool you have access to. Add columns for status code, title, meta description, word count, canonical and notes. The gaps between these sources are your first findings: pages in the sitemap but missing from the menu, or live pages that appear nowhere.

Phase 1: crawl

A crawler follows links from the home page the way a search engine does. Desktop crawlers exist for every platform; for a tiny site even wget --spider -r -nd -o crawl.log example.com produces a log of every URL it reached and its status. Look for:

  • 4xx responses: broken internal links. Fix the link, or redirect the old URL to its closest replacement.
  • Redirect chains: A goes to B goes to C. Point A straight at C.
  • 5xx responses: server errors, often only on certain templates.
  • Orphan pages: URLs in your inventory the crawler never found because nothing links to them.

Phase 2: indexability

For each important page, confirm that nothing tells search engines to skip it. Check four places: the robots.txt file, a noindex robots meta tag, an X-Robots-Tag header (curl -sI shows it), and the canonical tag, which should point at the page itself unless it is a deliberate duplicate.

Phase 3: content signals

Sort your spreadsheet by title. Identical titles mean pages competing with each other or a template that never fills in the page name. Do the same for descriptions. Then flag thin pages, those with very little unique text, and decide whether each should be expanded, merged into a stronger page, or removed with a redirect.

Phase 4: performance and mobile

Run Lighthouse in DevTools on one page of each template type (home, article, product, listing) rather than every URL. Templates share problems, so one fix often repairs hundreds of pages. Test the same pages at a narrow window width for text that overflows or buttons that are too small to tap.

Phase 5: rank the fixes

PriorityTypical finding
Fix nowKey pages blocked or set to noindex; site-wide 5xx errors; HTTPS errors
Fix soonBroken internal links; redirect chains; duplicate titles on important pages
PlanThin content; slow templates; missing structured data
Nice to haveMinor wording, image alt text on decorative images

Audit mistakes

  • Exporting a 400-row error report and fixing it top to bottom. Group by cause instead.
  • Crawling a large live site at full speed. Throttle the crawler so you do not slow the site for real visitors.
  • Auditing once and never again. Save the spreadsheet and compare next time.

Questions

How often should I audit?

After any redesign or migration, and on a regular schedule that matches how often the site changes.

Do automated audit scores matter?

They are a prompt to look, not a goal. A page can score well and still be unhelpful to the people reading it.