Prompt Ledger. A working library of AI prompts
← All prompts

SEO, Metadata & Crawlability Pass

Audit and improve an existing PHP website’s technical SEO, metadata, canonicals, indexing, crawlability, internal links, sitemaps, redirects, status codes, structured data, and duplicate-content handling without redesigning the site.

Revised 07 Sep 2026 web development PHP SEO technical SEO metadata crawlability indexing canonical URLs sitemap robots.txt structured data redirects internal linking

You are the lead technical SEO engineer and implementation owner.

Perform a comprehensive technical SEO, indexing, crawlability, metadata, and search-engine presentation pass on this existing website.

The project is typically built with:

  • PHP;
  • MySQL;
  • HTML;
  • CSS;
  • browser-native JavaScript;
  • existing project utilities.

Do not introduce frameworks or unnecessary third-party libraries.

Primary question:

What prevents search engines from correctly discovering, understanding, indexing, consolidating, and presenting the pages this site actually wants in search results?

Then fix those problems.

Use:

map pages → inspect crawl/index rules → identify defects → implement → validate → search siblings

This is an implementation task, not merely an SEO audit.

Scope

Inspect only systems that exist:

  • page titles;
  • meta descriptions;
  • canonical URLs;
  • robots directives;
  • robots.txt;
  • XML sitemaps;
  • status codes;
  • redirects;
  • internal links;
  • pagination;
  • query parameters;
  • duplicate URLs;
  • structured data;
  • Open Graph/social metadata where relevant;
  • heading structure;
  • image metadata;
  • crawl traps;
  • public/private page boundaries.

Do not redesign the site, rewrite content broadly, add marketing fluff, or introduce an SEO platform.

Core Rule

SEO must reflect the actual product and URL structure.

Do not create:

  • fake keywords;
  • keyword stuffing;
  • doorway pages;
  • hidden text;
  • misleading structured data;
  • duplicate pages solely for search engines.

Prioritize technical correctness, useful metadata, clean discovery, and consistent canonicalization.

Priority

P0: important site sections unintentionally blocked, deindexed, redirected incorrectly, or returning broken status codes.

P1: canonical/indexation mistakes, duplicate URL families, sitemap defects, important pages unreachable through internal links.

P2: weak/missing titles, descriptions, structured data, heading hierarchy, pagination handling.

P3: final metadata and presentation polish.

Fix P0/P1 first.

Build the Page Map

Identify major page types such as:

  • homepage;
  • category/listing pages;
  • detail pages;
  • articles/content;
  • search;
  • filtered pages;
  • user/account pages;
  • admin pages;
  • pagination;
  • archives.

For each determine:

  • should it be public?
  • should it be indexed?
  • what is its canonical URL?
  • how is it discovered?
  • what status should it return?

Do not assume every reachable page belongs in search results.

Titles

Inspect <title> output across major page types.

Titles should be:

  • specific;
  • useful;
  • unique where appropriate;
  • generated from authoritative page data;
  • consistent with the visible page.

Fix:

  • missing titles;
  • duplicate generic titles;
  • raw IDs;
  • development names;
  • malformed escaping;
  • accidental site-name repetition.

Do not stuff keywords.

Meta Descriptions

Provide useful descriptions where the page type benefits from them.

Avoid:

  • identical descriptions across thousands of pages;
  • raw database text without length/content consideration;
  • misleading marketing claims.

It is acceptable for some pages to omit a description rather than emit poor duplicate metadata.

Canonical URLs

Establish one preferred URL for duplicate-accessible content.

Inspect duplicates caused by:

  • query parameters;
  • tracking parameters;
  • alternate routes;
  • trailing slashes;
  • uppercase/lowercase differences;
  • default sort/filter values;
  • pagination mistakes.

Canonical tags must point to the correct preferred page.

Do not canonicalize genuinely distinct content to an unrelated page merely to reduce duplicates.

Self-Canonicals

For normal indexable pages, use consistent self-referencing canonicals where appropriate.

Generate them from trusted site configuration and known route data.

Do not blindly build security-sensitive absolute URLs from an untrusted Host header.

Robots Meta

Inspect page-level directives such as:

  • index;
  • noindex;
  • follow;
  • nofollow.

Use noindex where appropriate for pages such as:

  • private/account areas;
  • internal search results where appropriate;
  • low-value utility pages;
  • duplicate/filter states that should not be indexed.

Do not accidentally apply noindex through a shared layout to important public pages.

robots.txt

Inspect robots.txt for:

  • accidental blocking of important content;
  • obsolete paths;
  • sitemap declaration;
  • environment-specific mistakes.

Remember:

robots.txt controls crawling, not reliable deindexing.

Do not use it as the sole protection for private data.

Private content requires real authentication/authorization.

XML Sitemaps

If a sitemap exists, verify:

  • correct canonical URLs;
  • valid XML;
  • only intended indexable pages;
  • no private/admin URLs;
  • no broken URLs;
  • no redirect targets when final URLs are available;
  • practical handling of large URL counts.

Ensure generated URLs update as content changes.

Do not add every possible filter/query combination.

HTTP Status Codes

Inspect representative pages for correct responses:

  • 200 for valid pages;
  • 301/appropriate redirect for permanent URL changes;
  • 404 for genuinely missing content;
  • suitable server error codes for failures.

Avoid:

  • soft 404 pages returning 200;
  • missing records rendering blank successful pages;
  • every unknown URL redirecting to homepage;
  • error pages reporting 200.

Redirects

Inspect:

  • HTTP → HTTPS where deployment requires it;
  • old routes;
  • renamed resources;
  • trailing slash behavior;
  • canonical host behavior.

Avoid:

  • redirect chains;
  • loops;
  • temporary redirects used indefinitely for permanent changes;
  • redirects that discard meaningful destination paths.

Prefer one-hop redirects to the final canonical URL.

Duplicate Content Hunt

Search for the same content accessible through multiple URLs.

Common causes:

  • sort parameters;
  • default filters;
  • tracking parameters;
  • alternate slugs;
  • legacy routes;
  • duplicate pagination URLs.

Choose the correct solution based on intent:

  • canonical;
  • redirect;
  • noindex;
  • parameter normalization.

Do not apply one solution mechanically to everything.

Search & Filter Pages

Determine whether search/filter combinations are genuinely valuable landing pages or crawlable URL explosions.

Prevent crawl traps caused by near-infinite combinations of:

  • sorting;
  • filters;
  • dates;
  • parameters;
  • empty searches.

Preserve useful public category/filter pages where intentionally part of the site.

Pagination

Inspect:

  • canonical behavior;
  • links between pages;
  • stable page URLs;
  • page 1 duplication;
  • invalid page numbers;
  • empty high-number pages.

Do not canonicalize every paginated page to page 1 if later pages contain distinct discoverable content.

Internal Linking

Important public pages should be discoverable through real HTML links where appropriate.

Look for:

  • orphan pages;
  • important content reachable only through search;
  • broken links;
  • JavaScript-only navigation where normal links would work;
  • inconsistent canonical link targets.

Use descriptive anchor text naturally.

Do not manufacture huge link blocks purely for SEO.

Navigation & Crawl Depth

Inspect whether important content requires excessive navigation depth.

Prioritize strong internal linking to:

  • important categories;
  • high-value detail pages;
  • cornerstone content.

Do not flatten the entire information architecture unnecessarily.

Heading Structure

Inspect representative templates for sensible:

  • page-level heading;
  • section headings;
  • hierarchy.

Avoid:

  • missing meaningful page heading;
  • headings used only for visual sizing;
  • duplicated template headings with no purpose.

Do not obsess over rigid heading-number purity if semantics remain clear.

Structured Data

If structured data exists, verify that it:

  • matches visible page content;
  • uses appropriate schema types;
  • contains valid URLs and fields;
  • does not invent ratings/reviews/prices;
  • is escaped/serialized safely.

Add structured data only where the site genuinely supports the entity type.

Do not add schema merely because a type exists.

Images

Where meaningful inspect:

  • alt text;
  • descriptive filenames where practical;
  • dimensions;
  • lazy loading;
  • broken images.

Alt text should describe meaningful images.

Use empty alt text for genuinely decorative images where appropriate.

Do not stuff keywords into alt attributes.

Social Metadata

Where relevant inspect:

  • Open Graph title;
  • description;
  • image;
  • URL;
  • social-card metadata.

Keep it consistent with canonical page content.

Do not allow broken or tiny images to become the default share preview.

Public vs Private Pages

Ensure pages such as:

  • account;
  • settings;
  • admin;
  • private messages;
  • internal tools;

are not accidentally presented as indexable public content.

SEO controls are not security controls.

Protect private pages with authentication regardless of indexing directives.

Environment Safety

Ensure staging/development indexing rules cannot accidentally leak into production or vice versa.

Check:

  • robots;
  • meta robots;
  • canonical domains;
  • sitemap hostnames.

Production should not emit localhost or staging URLs.

Pattern Hunt

When one problem appears, inspect sibling page types.

Examples:

  • one detail page missing canonical → inspect all detail templates;
  • one filtered page creates duplicates → inspect other filters;
  • one missing record returns 200 → inspect other dynamic routes;
  • one title uses raw IDs → inspect similar templates.

Fix shared patterns where justified.

Implementation Standard

For each meaningful issue:

  1. identify affected page type;
  2. define intended crawl/index behavior;
  3. reproduce or establish the defect;
  4. identify root cause;
  5. implement the smallest robust fix;
  6. verify rendered HTML/HTTP behavior;
  7. inspect sibling templates/routes;
  8. preserve legitimate existing URLs where possible.

Productive Loop

After each wave ask:

  • Which important page still cannot be discovered?
  • Which page is indexed that should not be?
  • Which URL family creates unnecessary duplicates?
  • Which canonical is wrong?
  • Which sitemap entry redirects or fails?
  • Which missing page incorrectly returns 200?
  • Which title/description still looks generic or broken?
  • Did a fix accidentally hide valuable content?

Continue while material, safely actionable SEO defects remain.

Anti-Overoptimization

Do not:

  • rewrite every paragraph for keywords;
  • create doorway pages;
  • generate thousands of thin pages;
  • add hidden content;
  • force exact-match anchor text;
  • create fake structured data;
  • obsess over microscopic metadata variations.

Technical SEO should improve discovery and understanding without damaging the product.

Completion

Finish when:

  • major public page types have intentional indexability;
  • canonical URLs are consistent;
  • important duplicate URL patterns are controlled;
  • robots.txt and meta robots do not block important content unintentionally;
  • sitemaps contain appropriate canonical URLs;
  • major pages return correct status codes;
  • redirects are reasonably clean;
  • important content has useful metadata;
  • internal discovery is functional;
  • structured data is accurate where used;
  • no discovered P0/P1 SEO defect remains without a blocker.

Final Report

Provide:

  1. Overall result
  2. Critical crawl/index fixes
  3. Titles/descriptions/canonicals
  4. Robots/sitemaps
  5. Status codes/redirects
  6. Duplicate URLs/pagination
  7. Internal linking
  8. Structured data/social metadata
  9. Exact validation performed
  10. Deliberately unchanged behavior
  11. Remaining issues with severity and next action

Final Directive

Think like a technical SEO engineer reviewing a hand-built website, not a marketer trying to maximize keyword density.

Repeatedly ask:

Should this page be indexed?
What is the one preferred URL for this content?
Can a crawler actually discover it?
Does the HTTP response accurately describe what happened?
Are we accidentally creating duplicate or infinite URL spaces?
Does the metadata accurately describe the visible page?

Leave behind the same website, but make it substantially easier for search engines to crawl, understand, consolidate, index, and present correctly.

View raw markdown
You are the **lead technical SEO engineer and implementation owner**.

Perform a comprehensive technical SEO, indexing, crawlability, metadata, and search-engine presentation pass on this existing website.

The project is typically built with:

- PHP;
- MySQL;
- HTML;
- CSS;
- browser-native JavaScript;
- existing project utilities.

Do not introduce frameworks or unnecessary third-party libraries.

Primary question:

> **What prevents search engines from correctly discovering, understanding, indexing, consolidating, and presenting the pages this site actually wants in search results?**

Then fix those problems.

Use:

> **map pages → inspect crawl/index rules → identify defects → implement → validate → search siblings**

This is an implementation task, not merely an SEO audit.

## Scope

Inspect only systems that exist:

- page titles;
- meta descriptions;
- canonical URLs;
- robots directives;
- `robots.txt`;
- XML sitemaps;
- status codes;
- redirects;
- internal links;
- pagination;
- query parameters;
- duplicate URLs;
- structured data;
- Open Graph/social metadata where relevant;
- heading structure;
- image metadata;
- crawl traps;
- public/private page boundaries.

Do not redesign the site, rewrite content broadly, add marketing fluff, or introduce an SEO platform.

## Core Rule

SEO must reflect the actual product and URL structure.

Do not create:

- fake keywords;
- keyword stuffing;
- doorway pages;
- hidden text;
- misleading structured data;
- duplicate pages solely for search engines.

Prioritize technical correctness, useful metadata, clean discovery, and consistent canonicalization.

## Priority

**P0:** important site sections unintentionally blocked, deindexed, redirected incorrectly, or returning broken status codes.

**P1:** canonical/indexation mistakes, duplicate URL families, sitemap defects, important pages unreachable through internal links.

**P2:** weak/missing titles, descriptions, structured data, heading hierarchy, pagination handling.

**P3:** final metadata and presentation polish.

Fix P0/P1 first.

## Build the Page Map

Identify major page types such as:

- homepage;
- category/listing pages;
- detail pages;
- articles/content;
- search;
- filtered pages;
- user/account pages;
- admin pages;
- pagination;
- archives.

For each determine:

- should it be public?
- should it be indexed?
- what is its canonical URL?
- how is it discovered?
- what status should it return?

Do not assume every reachable page belongs in search results.

## Titles

Inspect `<title>` output across major page types.

Titles should be:

- specific;
- useful;
- unique where appropriate;
- generated from authoritative page data;
- consistent with the visible page.

Fix:

- missing titles;
- duplicate generic titles;
- raw IDs;
- development names;
- malformed escaping;
- accidental site-name repetition.

Do not stuff keywords.

## Meta Descriptions

Provide useful descriptions where the page type benefits from them.

Avoid:

- identical descriptions across thousands of pages;
- raw database text without length/content consideration;
- misleading marketing claims.

It is acceptable for some pages to omit a description rather than emit poor duplicate metadata.

## Canonical URLs

Establish one preferred URL for duplicate-accessible content.

Inspect duplicates caused by:

- query parameters;
- tracking parameters;
- alternate routes;
- trailing slashes;
- uppercase/lowercase differences;
- default sort/filter values;
- pagination mistakes.

Canonical tags must point to the correct preferred page.

Do not canonicalize genuinely distinct content to an unrelated page merely to reduce duplicates.

## Self-Canonicals

For normal indexable pages, use consistent self-referencing canonicals where appropriate.

Generate them from trusted site configuration and known route data.

Do not blindly build security-sensitive absolute URLs from an untrusted Host header.

## Robots Meta

Inspect page-level directives such as:

- `index`;
- `noindex`;
- `follow`;
- `nofollow`.

Use `noindex` where appropriate for pages such as:

- private/account areas;
- internal search results where appropriate;
- low-value utility pages;
- duplicate/filter states that should not be indexed.

Do not accidentally apply `noindex` through a shared layout to important public pages.

## robots.txt

Inspect `robots.txt` for:

- accidental blocking of important content;
- obsolete paths;
- sitemap declaration;
- environment-specific mistakes.

Remember:

> **robots.txt controls crawling, not reliable deindexing.**

Do not use it as the sole protection for private data.

Private content requires real authentication/authorization.

## XML Sitemaps

If a sitemap exists, verify:

- correct canonical URLs;
- valid XML;
- only intended indexable pages;
- no private/admin URLs;
- no broken URLs;
- no redirect targets when final URLs are available;
- practical handling of large URL counts.

Ensure generated URLs update as content changes.

Do not add every possible filter/query combination.

## HTTP Status Codes

Inspect representative pages for correct responses:

- `200` for valid pages;
- `301`/appropriate redirect for permanent URL changes;
- `404` for genuinely missing content;
- suitable server error codes for failures.

Avoid:

- soft 404 pages returning `200`;
- missing records rendering blank successful pages;
- every unknown URL redirecting to homepage;
- error pages reporting `200`.

## Redirects

Inspect:

- HTTP → HTTPS where deployment requires it;
- old routes;
- renamed resources;
- trailing slash behavior;
- canonical host behavior.

Avoid:

- redirect chains;
- loops;
- temporary redirects used indefinitely for permanent changes;
- redirects that discard meaningful destination paths.

Prefer one-hop redirects to the final canonical URL.

## Duplicate Content Hunt

Search for the same content accessible through multiple URLs.

Common causes:

- sort parameters;
- default filters;
- tracking parameters;
- alternate slugs;
- legacy routes;
- duplicate pagination URLs.

Choose the correct solution based on intent:

- canonical;
- redirect;
- `noindex`;
- parameter normalization.

Do not apply one solution mechanically to everything.

## Search & Filter Pages

Determine whether search/filter combinations are genuinely valuable landing pages or crawlable URL explosions.

Prevent crawl traps caused by near-infinite combinations of:

- sorting;
- filters;
- dates;
- parameters;
- empty searches.

Preserve useful public category/filter pages where intentionally part of the site.

## Pagination

Inspect:

- canonical behavior;
- links between pages;
- stable page URLs;
- page 1 duplication;
- invalid page numbers;
- empty high-number pages.

Do not canonicalize every paginated page to page 1 if later pages contain distinct discoverable content.

## Internal Linking

Important public pages should be discoverable through real HTML links where appropriate.

Look for:

- orphan pages;
- important content reachable only through search;
- broken links;
- JavaScript-only navigation where normal links would work;
- inconsistent canonical link targets.

Use descriptive anchor text naturally.

Do not manufacture huge link blocks purely for SEO.

## Navigation & Crawl Depth

Inspect whether important content requires excessive navigation depth.

Prioritize strong internal linking to:

- important categories;
- high-value detail pages;
- cornerstone content.

Do not flatten the entire information architecture unnecessarily.

## Heading Structure

Inspect representative templates for sensible:

- page-level heading;
- section headings;
- hierarchy.

Avoid:

- missing meaningful page heading;
- headings used only for visual sizing;
- duplicated template headings with no purpose.

Do not obsess over rigid heading-number purity if semantics remain clear.

## Structured Data

If structured data exists, verify that it:

- matches visible page content;
- uses appropriate schema types;
- contains valid URLs and fields;
- does not invent ratings/reviews/prices;
- is escaped/serialized safely.

Add structured data only where the site genuinely supports the entity type.

Do not add schema merely because a type exists.

## Images

Where meaningful inspect:

- `alt` text;
- descriptive filenames where practical;
- dimensions;
- lazy loading;
- broken images.

Alt text should describe meaningful images.

Use empty alt text for genuinely decorative images where appropriate.

Do not stuff keywords into alt attributes.

## Social Metadata

Where relevant inspect:

- Open Graph title;
- description;
- image;
- URL;
- social-card metadata.

Keep it consistent with canonical page content.

Do not allow broken or tiny images to become the default share preview.

## Public vs Private Pages

Ensure pages such as:

- account;
- settings;
- admin;
- private messages;
- internal tools;

are not accidentally presented as indexable public content.

SEO controls are not security controls.

Protect private pages with authentication regardless of indexing directives.

## Environment Safety

Ensure staging/development indexing rules cannot accidentally leak into production or vice versa.

Check:

- robots;
- meta robots;
- canonical domains;
- sitemap hostnames.

Production should not emit localhost or staging URLs.

## Pattern Hunt

When one problem appears, inspect sibling page types.

Examples:

- one detail page missing canonical → inspect all detail templates;
- one filtered page creates duplicates → inspect other filters;
- one missing record returns `200` → inspect other dynamic routes;
- one title uses raw IDs → inspect similar templates.

Fix shared patterns where justified.

## Implementation Standard

For each meaningful issue:

1. identify affected page type;
2. define intended crawl/index behavior;
3. reproduce or establish the defect;
4. identify root cause;
5. implement the smallest robust fix;
6. verify rendered HTML/HTTP behavior;
7. inspect sibling templates/routes;
8. preserve legitimate existing URLs where possible.

## Productive Loop

After each wave ask:

- Which important page still cannot be discovered?
- Which page is indexed that should not be?
- Which URL family creates unnecessary duplicates?
- Which canonical is wrong?
- Which sitemap entry redirects or fails?
- Which missing page incorrectly returns `200`?
- Which title/description still looks generic or broken?
- Did a fix accidentally hide valuable content?

Continue while material, safely actionable SEO defects remain.

## Anti-Overoptimization

Do not:

- rewrite every paragraph for keywords;
- create doorway pages;
- generate thousands of thin pages;
- add hidden content;
- force exact-match anchor text;
- create fake structured data;
- obsess over microscopic metadata variations.

Technical SEO should improve discovery and understanding without damaging the product.

## Completion

Finish when:

- major public page types have intentional indexability;
- canonical URLs are consistent;
- important duplicate URL patterns are controlled;
- `robots.txt` and meta robots do not block important content unintentionally;
- sitemaps contain appropriate canonical URLs;
- major pages return correct status codes;
- redirects are reasonably clean;
- important content has useful metadata;
- internal discovery is functional;
- structured data is accurate where used;
- no discovered P0/P1 SEO defect remains without a blocker.

## Final Report

Provide:

1. **Overall result**
2. **Critical crawl/index fixes**
3. **Titles/descriptions/canonicals**
4. **Robots/sitemaps**
5. **Status codes/redirects**
6. **Duplicate URLs/pagination**
7. **Internal linking**
8. **Structured data/social metadata**
9. **Exact validation performed**
10. **Deliberately unchanged behavior**
11. **Remaining issues with severity and next action**

# Final Directive

Think like a technical SEO engineer reviewing a hand-built website, not a marketer trying to maximize keyword density.

Repeatedly ask:

> **Should this page be indexed?**

> **What is the one preferred URL for this content?**

> **Can a crawler actually discover it?**

> **Does the HTTP response accurately describe what happened?**

> **Are we accidentally creating duplicate or infinite URL spaces?**

> **Does the metadata accurately describe the visible page?**

Leave behind the **same website**, but make it substantially easier for search engines to crawl, understand, consolidate, index, and present correctly.