Chapter 5: Technical SEO
5.1 Crawl control, indexing and status codes
20 entries- 206 robots.txt A plain text file at the root of the domain telling crawlers which paths they may and may not request.
- 207 XML Sitemap A file listing the URLs you want indexed, with optional metadata about when each last changed.
- 208 noindex A meta instruction telling the engine not to include a page in the index, while still allowing it to be crawled and its links followed.
- 209 nofollow A link attribute telling the engine not to follow the link or count it as an endorsement.
- 210 301 Redirect A permanent redirect, telling browsers and crawlers that a page has moved for good.
- 211 302 Redirect A temporary redirect, telling the engine the original URL will return.
- 212 404 Not Found The server's answer that the requested resource does not exist.
- 213 Soft 404 A page returning success while showing nothing useful: no results, product removed, content unavailable.
- 214 5xx Server Error A server-side failure: internal error, service unavailable, gateway timeout.
- 215 Redirect Chain A sequence of redirects where each URL points at the next: A to B to C to D.
- 216 Redirect Loop A cycle where A redirects to B and B redirects back to A, or a longer ring that never resolves.
- 217 Crawl Error A failure the crawler hit while requesting a page: DNS failure, connection refused, timeout, 404, 5xx.
- 218 Index Status Whether a page has been accepted into the index and is therefore eligible to appear in results.
- 219 Index Bloat A large number of low-value, duplicated or empty pages sitting in the index and diluting the site's standing.
- 220 Duplicate Content The same or nearly the same content reachable at more than one URL.
- 221 Canonicalization Choosing and declaring the official URL among several showing the same content.
- 222 URL Parameters The key-value pairs after the question mark, used for filtering, sorting, tracking and pagination.
- 223 Pagination Splitting a long list of items across multiple pages.
- 224 Infinite Scroll Content that loads continuously as the visitor scrolls, with no page boundaries.
- 225 JavaScript SEO Making sure content a browser assembles with JavaScript is also available to a crawler.
5.2 Speed, rendering and modern front-ends
15 entries- 226 Page Speed How long a page takes from request to being usable.
- 227 TTFB (Time to First Byte) The time between the request leaving the browser and the first byte arriving from the server.
- 228 CDN A network of servers distributed geographically, serving cached copies from the location nearest the visitor.
- 229 Caching Storing computed or fetched data closer to where it is needed so it does not have to be produced again.
- 230 Compression Reducing file size in transit using an algorithm both the browser and the server understand.
- 231 Image Optimization Reducing the bytes an image costs without reducing what the visitor sees.
- 232 Lazy Loading Deferring the load of images or content until they are close to the viewport.
- 233 Minification Removing whitespace, comments and redundant syntax from code without changing behaviour.
- 234 Critical Rendering Path The sequence of steps the browser takes to turn HTML, CSS and JavaScript into pixels.
- 235 Render Blocking Resources, chiefly stylesheets and synchronous scripts, that stop the browser painting until they are dealt with.
- 236 Async Loading Downloading a resource in parallel without letting it hold up the rest of the page.
- 237 HTTP/2 and HTTP/3 Later versions of the protocol that carries pages, adding multiplexing, header compression and a lower connection cost.
- 238 Mobile-First Indexing The mobile version of a page is the one indexed and ranked; the desktop version is secondary.
- 239 Core Web Vitals Pass All three metrics inside the good thresholds, across enough of the site's traffic to count.
- 240 Page Experience Signals The combined set of experience signals: the three vitals, HTTPS, mobile usability and the absence of intrusive interstitials.
5.3 Security, HTTPS and accessibility
8 entries- 241 HTTPS HTTP carried over an encrypted connection, authenticated by a certificate.
- 242 SSL Certificate The document proving a site's identity and enabling the encrypted connection.
- 243 Mixed Content A secure page loading some of its resources over plain HTTP.
- 244 Website Security Keeping a site from being compromised, injected, or used to attack its visitors.
- 245 Malware Code injected into a site to harm or deceive the people who visit it.
- 246 Blacklist A list of sites flagged as dangerous by a search engine or a security vendor.
- 247 Accessibility (a11y) Whether people with visual, hearing, motor or cognitive impairments can use the site.
- 248 ARIA Attributes that describe roles, states and labels to assistive technology when HTML alone cannot.
5.4 Logs, SSR vs CSR, migrations and i18n
17 entries- 249 Log File Analysis Reading the server's record of requests to see what crawlers actually did.
- 250 Edge SEO Applying changes at the CDN layer, before the request reaches the origin server.
- 251 Prerendering Generating a static HTML snapshot ahead of time for crawlers, while visitors receive the interactive version.
- 252 Dynamic Rendering Serving rendered HTML to crawlers and the JavaScript application to browsers, from the same URL.
- 253 Server-Side Rendering (SSR) Building complete HTML on the server so the browser can display it without executing application code first.
- 254 Client-Side Rendering (CSR) Shipping a minimal HTML shell, then building the page in the browser with JavaScript.
- 255 Static Site Generation (SSG) Producing every page as static HTML at build time, with no rendering at request time.
- 256 PWA (Progressive Web App) A site that can be installed, work offline and receive notifications, using a service worker and a manifest.
- 257 AMP A restricted HTML framework for very fast mobile pages, once required for some search features.
- 258 Structured Data Testing Tool Google's tool for checking whether a page's structured data is valid and eligible.
- 259 Site Migration Moving a site to a new domain, server, platform or URL structure.
- 260 Multilingual SEO Optimising a site for audiences in several languages.
- 261 hreflang A tag declaring the language and region a page targets, together with its counterparts in other languages.
- 262 Geotargeting Tuning content and search presence for one specific country or region.
- 263 Sitemap Index A file listing other sitemap files, used when a site exceeds the limits of a single one.
- 264 Crawl Stats The console's record of crawler activity: requests, bytes downloaded, response times and file types.
- 265 Index Coverage Report The console's breakdown of what is indexed, what is excluded, and why.
Chapter SOP and interview questions
Technical audit SOP for a site you have just inherited
Twelve checks, in this order. Do not open a ticket until you have run all twelve — half the “issues” found in step three disappear once you know the answer to step one.
- Crawl the site. Screaming Frog or an equivalent, full site if it fits, otherwise a stratified sample that includes each template. Export status codes, titles, meta, canonicals, H1s, word counts, indexability.
- Read robots.txt from the outside. Fetch it as a URL, not as a file on your laptop. Look for a leftover
Disallow: /in a staging block, and for rules that block CSS or JS the renderer needs. - Validate the XML sitemap. Fetch it, check it returns 200, check for non-canonical and 4xx URLs inside it, and confirm it is submitted and parsed in Search Console.
- Audit indexation state. In the Page Indexing report, read the reasons, not just the totals. A large “crawled — currently not indexed” bucket is a quality signal, a large “discovered — currently not indexed” bucket is a crawl-priority signal.
- Hunt duplicates. Duplicate titles, duplicate H1s, parameter permutations, trailing-slash and case variants, and every URL pattern that resolves to the same content without a canonical pointing at one of them.
- Map the redirect graph. Chains, loops, 302s used where 301s belong, and internal links still pointing at redirected URLs — every internal redirect is a small tax on crawl budget.
- Test rendering. Compare raw HTML with the rendered DOM on the main templates. Anything important that exists only after JavaScript is a risk you should be able to name and quantify.
- Measure Core Web Vitals. Field data first (CrUX or Search Console), lab data second. Record LCP, INP and CLS per template, not per site — an average hides the template that matters.
- Check transport and security. HTTPS everywhere, no mixed content, no certificate chain problems, HSTS in place if the team supports it.
- Validate structured data. Sample each template through a validator, not a single page you happen to like. Watch for required fields going empty because a CMS field was renamed.
- Read the logs if the site is large. Bot hits by directory, status code distribution, and which sections consume crawl budget without earning traffic.
- Write the report for the person who will do the work. Severity, evidence, affected URLs, expected impact, effort, owner. An audit that lists 400 issues without a priority order is a research paper, not a plan.
Interview questions and answer frameworks
Q1: A large number of pages are “crawled — currently not indexed”. What do you do?
Framework: Start with the interpretation, because it drives the fix: the engine fetched the page and decided it was not worth storing. That is a quality judgement, not a technical error — telling the client “we’ll resubmit the sitemap” is not an answer. Then investigate in three directions. Content: sample the pages and look for thin pages, near-duplicates generated by a template, or AI text with no unique information. Site-level trust: on a young or low-authority domain, the engine is simply stricter about what it stores. Demand: pages nobody searches for and nobody links to have no reason to be indexed. Fixes follow from which of the three dominates: merge or improve the thin pages, add internal links from pages that are already indexed and performing, and use
noindexfor pages that exist only for site functionality. Be honest that recovery is slow and measured in months.
Q2: What is render blocking and how do you fix it?
Framework: Any resource the browser must fetch and process before it can paint the first meaningful screen — typically blocking CSS and synchronous JavaScript in the head. Fix in order of impact: inline the critical CSS for above-the-fold content and load the rest asynchronously; mark non-critical scripts
deferorasync; preload the one or two assets the first screen genuinely needs; audit third-party scripts and remove or delay everything that is not earning its cost. Then measure again — the goal is LCP under 2.5 seconds on the templates that receive traffic, not a perfect lab score on a page nobody visits.
Q3: What do you check before, during and after a site migration?
Framework: Before — crawl and archive the old site completely, export every URL, build a one-to-one redirect map (not a blanket rule), freeze content changes, and stage the new site with robots blocked so it cannot be indexed early. During — switch DNS, verify redirects resolve in one hop, spot-check status codes per template, update internal links and the sitemap, and confirm analytics still fires. After — file a Change of Address for the old domain if the domain itself moved, watch 404s hourly for the first days, compare crawl stats and index coverage weekly, and track rank and traffic against the pre-migration baseline for at least eight weeks. Name the two classic failures explicitly: redirecting everything to the homepage, and launching with the new URLs accidentally blocked. Both cost more than the entire migration.
Q4: How do you prioritise a 400-item technical backlog?
Framework: Score each item on three axes: does it affect pages that carry traffic or revenue, does it affect crawl or indexation of those pages, and how expensive is it to fix. Anything that blocks indexing of high-value URLs outranks everything cosmetic, even if the cosmetic list is longer. Then group by owner: what only engineering can do, what content can do, what you can do this week. Publish the top ten with expected impact, and re-score monthly — a backlog that never shrinks is usually a prioritisation failure, not a capacity failure.