Technical SEO Audit & Infrastructure Case Study
Technical SEO Audit · Site Architecture · Core Web Vitals

Technical SEO Case Study: Fixing Crawl Waste and Indexation for a Growing Fashion Catalog

Clothing & fashion (D2C) · India · 3 months

Project Overview

Industry:

Clothing & fashion (D2C)

Service:

Technical SEO Audit & Infrastructure

Objective:

Identify and fix the specific technical issues causing crawl budget waste and slow indexation across a growing product catalog, so important pages got crawled, indexed, and updated reliably as the catalog expanded.

As the brand's catalog grew (new arrivals, seasonal collections, and the filter/sort options customers expect), the underlying technical structure hadn't been revisited, and search engines were spending an increasing share of their crawl effort on pages that didn't need it.

The Challenge

The brand had a large, genuinely valuable product catalog, but its technical structure was working against search visibility, not for it:

Filter and sort combinations (color, size, price range) generated thousands of unique, crawlable URLs that added no real search value but consumed crawl budget

New product pages and updated stock information were taking days to get re-crawled, since crawlers were spending most of their time on low-value filter URLs instead

No canonical tag strategy existed for near-duplicate pages created by filter combinations, so search engines sometimes indexed the wrong version of a page

Core Web Vitals scores were inconsistent across product pages, with no clear understanding of which specific elements were causing the slowdown

No structured data (product schema) was implemented, so listings weren't eligible for rich results like price, availability, and review stars in search

The challenge wasn't a lack of valuable pages — it was that search engines were spending their limited attention on the wrong ones.

Research & Discovery

We started with a full crawl and real server data, rather than guessing at the cause from Search Console alone:

  • • Screaming Frog full-site crawl, to map every URL pattern being generated, including filter and sort combinations
  • • Server log file analysis, to see exactly how Googlebot was actually spending its crawl budget — which URL patterns were being crawled most frequently, regardless of their value
  • • Google Search Console Coverage and Crawl Stats review, to identify indexation errors and crawl frequency trends over time
  • • PageSpeed Insights and Core Web Vitals review across a sample of product pages, to isolate which specific elements (images, scripts, fonts) were causing the slowdown
  • • Review of existing structured data (or lack of it) against what competitor fashion sites were already implementing and benefiting from in search results

This stage showed the core issue clearly in the log files: a large share of total crawl activity was going to filter and sort URLs that should never have been crawled at all, directly at the expense of new and updated product pages.

Technical SEO & Measurement Setup

Before making changes, we recorded a clean baseline so the impact of each fix could be measured on its own:

  • ✓ Baseline crawl stats from Search Console and the log file analysis, recording the proportion of crawl activity going to low-value versus genuinely important URLs
  • ✓ Baseline indexation count for valid product and category pages
  • ✓ Baseline Core Web Vitals scores recorded page-type by page-type (homepage, category, product)
  • ✓ Organic traffic tracked separately for pages directly affected by the technical fixes, so improvement could be attributed to this work specifically rather than general site growth

This made it possible to report the direct impact of fixing crawl waste and indexation, not just overall traffic trends that could have other causes.

Proven Case Study Impact
Technical SEO: -58% Crawled Low-Value URLs

Redirecting existing crawl budget to genuinely valuable pages — not adding new content or pages.

Technical SEO Strategy

1. Crawl Budget Recovery Through Indexation Control

Implemented canonical tags and robots rules to stop filter and sort URL combinations from being crawled and indexed as separate pages, redirecting that freed-up crawl budget toward genuinely valuable, unique pages.

2. Log File-Informed Crawl Prioritization

Used the log file analysis to identify which important pages were being under-crawled, and adjusted internal linking and sitemap structure specifically to improve discovery of those pages.

3. Core Web Vitals Fixes by Page Type

Addressed the specific causes of slow Core Web Vitals scores identified in research — primarily unoptimized product images and render-blocking scripts — fixing each page type (category, product) based on what was actually slowing it down.

4. Structured Data Implementation

Implemented product schema markup (price, availability, review rating) across the catalog, making listings eligible for rich results in search and improving click-through from search results pages.

What We Did & When

Month 1 — Audit & Baseline
Ran the full Screaming Frog crawl and server log file analysis, reviewed Search Console coverage and crawl stats, and recorded baseline Core Web Vitals scores.
Month 1–2 — Indexation & Crawl Control
Implemented canonical tags and robots rules for filter/sort URLs, and restructured internal linking and the XML sitemap to prioritize important pages.
Month 2 — Core Web Vitals Fixes
Addressed image optimization and render-blocking script issues identified in the audit, page type by page type.
Month 2–3 — Structured Data Rollout
Implemented product schema markup across the catalog, validated against Google's structured data testing tools.
Month 3 — Monitoring & Refinement
Re-ran the Screaming Frog crawl and reviewed updated log file data to confirm crawl budget was being redirected as intended, and reviewed indexation and traffic changes against baseline.
Before After
Crawl budget dominated by filter/sort URLs Crawl budget redirected to genuinely valuable pages
New/updated pages slow to get re-crawled Faster discovery and re-crawling of important pages
No canonical strategy for near-duplicate pages Canonical tags and indexation rules in place
Inconsistent, unexplained Core Web Vitals scores Core Web Vitals fixes targeted by specific page-type cause
No structured data, no rich result eligibility Product schema implemented across the catalog
Results

Improvements came from redirecting existing crawl budget and fixing specific, log file-identified issues — not from adding new content or pages.

Measured over 3 months (pre-audit baseline vs. month 3). Source: Screaming Frog, Google Search Console, server log files. Illustrative figures pending real campaign data.

-58%
Crawled Low-Value URLs — filter, sort, and duplicate URL crawl activity, pre-audit vs. month 3, from log file analysis.
+44%
Valid Indexed Pages — genuinely valuable product and category pages indexed, via Search Console.
+26%
Organic Traffic from Technical Fixes Alone — isolated to pages directly affected by the indexation and Core Web Vitals work, month 3 vs. baseline.
3 Months
Measurement window — pre-audit baseline vs. month 3, with crawl, indexation, and Core Web Vitals tracked separately.
Business Impact
The catalog's genuinely valuable pages became easier for search engines to find, crawl, and index quickly as new products and collections were added — turning technical infrastructure from a hidden constraint into something that actively supported the catalog's growth instead of working against it.
Get Started