How to Build a Keyword Research Process from Scratch
Last updated
Keyword research produces one output: a keyword map. A spreadsheet, one row per page, that tells you which query each page is built around, what supporting queries it should also satisfy, what the searcher intent is, and whether the page exists yet. Every content creation and optimisation decision flows from that map.
This guide walks through the five-stage process for building one from scratch: discovery, organisation, filtering, mapping, and cannibalisation checks. Each stage has its own section. The keyword-research pillar articles cover each concept in depth; this guide covers the workflow that connects them.
What a “keyword” represents here
One caveat before the process, because it shapes how to read every stage. Google matches meaning, not strings. Since Hummingbird, BERT, and MUM it interprets synonyms, related concepts, and the intent behind a query, so a single well-built page ranks for hundreds of related queries it never states word for word. Optimising a page around one exact phrase, repeating it, or forcing in a checklist of related terms is the antiquated model, and semantic search made it obsolete.
So treat each “primary keyword” in the map as a shorthand label for a topic and the intent behind it, not a phrase to place on the page a set number of times. The map exists to assign demand and intent to pages, not to license writing for a string. Everything below is about understanding what people want and which page should satisfy it; the exact wording is the least important part.
What you need before starting
A spreadsheet. Google Sheets or Excel. The keyword map lives here from stage one.
Access to Google Search Console. For an existing site, GSC query data is one of your most reliable discovery sources: real queries where your pages already appear. If you don’t have GSC access, request it before starting.
A keyword tool. Paid tools (Ahrefs, Semrush) accelerate stages one and three significantly. Free alternatives exist: Google Keyword Planner for volume estimates, Google Autocomplete and People Also Ask for discovery, and Ahrefs Webmaster Tools (free tier) for some site-level data. This guide notes where paid tools make a meaningful difference and where free methods work fine.
Clarity on your domain’s current authority. You’ll use this to set a realistic difficulty ceiling in stage three. If you’re on Ahrefs, note your Domain Rating. On Semrush, your Authority Score. If you have neither, check which queries your site currently ranks for in positions 1–10 via GSC: the difficulty of those queries gives you a calibration point.
Stage 1: Build the keyword universe
The keyword universe is the complete set of potentially relevant queries, assembled before any filtering takes place. You build it wide on purpose. Filtering comes later; discovery comes first.
Most keyword research workflows skip this separation. They take a handful of seed terms, run them through one tool, and filter immediately. The result is a filtered output from a single source, structurally blind to demand it never surfaced. Building the universe separately fixes this.
Pull from seven sources. Each surfaces different kinds of demand.
Seed term expansion
Start with two to four head terms that describe your topic, product, or service at its broadest: “SEO”, “keyword research”, “running shoes”. Enter each into your keyword tool and export the full set of related queries it generates. These are not your targets; they are inputs to the discovery process.
Keyword tool suggestions
Run each seed term through your keyword tool and export the full suggestions list, including question variants and long-tail clusters. If you have access to two tools, run the same seeds through both: tool databases differ, and some queries surface in one but not the other.
Google Autocomplete and related searches
Enter each seed term in Google (in a private browsing window to avoid personalisation) and collect the autocomplete suggestions. Append letters and modifiers to extend coverage: “keyword research a…”, “keyword research for…”, “keyword research tool…”. Also check “People also search for” and “Searches related to” at the bottom of results pages.
These suggestions reflect real query behaviour. They often surface phrasing variants and modifier patterns that tools undercount.
People Also Ask
For each seed term, open the PAA box and expand it two levels. Record all questions. PAA questions map the informational layer of your topic: the specific sub-questions your audience is asking. They tend to be useful for guides, FAQ sections, and long-tail cluster pages.
Google Search Console
For an existing site, this is often the most valuable source. Open the Performance report, set the date range to 12 months, and export the full query list. Filter for queries in positions 8–20 where you already have impressions: these are near-miss queries where your content has some relevance signal but is not yet earning clicks. They are the easiest wins in the entire dataset.
Include your full GSC export in the universe, not just the near-miss segment. Queries you rank for in positions 1–7 confirm that demand exists and that your domain is relevant for the topic. Low-impression queries in the long tail often reveal question variants and niche angles that tools miss entirely.
Competitor rankings
Identify two to four competitors that rank well for your core topics. Run a keyword gap analysis in Ahrefs or Semrush to export queries where at least two competitors rank but you do not. Queries where multiple competitors rank are higher-confidence targets: the demand is confirmed, and you know content exists to satisfy it.
If you don’t have paid access, manually check a competitor’s top pages in Ahrefs’ free website analysis tool, or use the “site:competitor.com” operator in Google to browse their indexed pages and identify topics you’re not covering.
Customer language
The vocabulary in reviews, forums, and support tickets often differs from the language in keyword tools. Skim relevant Reddit threads, G2 or Trustpilot reviews for products in your category, and industry communities. Note the phrases people use to describe their problems. These tend to match long-tail commercial and comparison queries that tools undercount because they’re spread across many phrasing variants.
This step matters more for commercial and transactional content than for informational. If you’re building an editorial site, prioritise the other six sources.
Stage 2: Organise the raw list
The raw output from seven sources will have duplicates, near-duplicates, and clearly irrelevant terms. Organise before filtering.
Set up your spreadsheet columns. At minimum: query, monthly search volume, keyword difficulty, source (which channel surfaced it), raw intent type, and notes. Don’t fill all columns now; add volume and KD in stage three.
Deduplicate. Merge exact duplicates. Consolidate near-duplicates (singular/plural, word-order variants, common abbreviations) into single entries. Keep the most commonly searched phrasing where variants exist.
Remove obvious exclusions, but conservatively. Drop queries that are clearly out of scope: competitor brand terms you cannot realistically target, navigational queries pointing to a specific external product, or queries with intent that cannot match any content you could produce. If you’re unsure, keep the term. The cost of keeping an irrelevant query is one extra row. The cost of discarding a relevant query is a gap in your content plan.
Do not filter by volume or difficulty at this stage. A query with 30 monthly searches and clear commercial intent can outperform a query with 5,000 monthly searches that has no conversion path for your business. Filtering before you can see the full picture removes queries before you have evaluated them.
Stage 3: Filter by difficulty
Now you apply the coarse difficulty filter to remove queries that are unrealistic at your current domain strength.
Set a realistic ceiling
KD scores estimate the link authority required to rank in the top 10, calculated from the backlink profiles of current ranking pages. They do not measure content quality, topical relevance, or intent match, so a high KD score does not mean a query is impossible; only that the current ranking pages are well-linked.
A practical starting point: set your ceiling at a KD score 15–20 points above what you currently rank for in positions 1–5. If your current top-ranking pages sit in the KD 30–40 range, set your ceiling at 50–55. Adjust as your domain authority grows.
Scores are not comparable across tools. A KD of 40 in Ahrefs is not the same as a KD of 40 in Semrush. Pick one tool and use it consistently throughout the process. The keyword difficulty article covers what these scores actually measure and where they mislead.
Check high-KD queries before discarding them
Before removing a query for high difficulty, open the SERP and look at what’s ranking. High KD scores sometimes reflect a saturated SERP; they also sometimes reflect a SERP filled with high-authority sites ranking with thin, outdated, or poorly matched content. If ranking pages have weak content and your domain has clear topical relevance, high KD is less of a barrier than the score suggests.
Keep low-volume queries
Do not set a minimum volume threshold and discard everything below it. A query with 30 searches per month, specific intent, and strong conversion alignment can drive more value than a 3,000-volume query where traffic is broadly distributed. Zero-volume queries in tools are not zero-volume in reality. Tool databases have coverage limits, especially for long-tail and niche queries.
Stage 4: Validate intent
Intent is the most important filter. A page targeting the wrong intent type will not rank consistently regardless of how well it’s optimised, because Google interprets it as a mismatch for what searchers want.
Read the SERP, not the keyword
The SERP is the reliable intent signal. Enter each query in Google and note the format of the top results:
- Articles and guides: informational
- Product pages, category pages, review roundups: commercial or transactional
- Specific brand URLs dominating: navigational
- Mixed formats: mixed or ambiguous intent
Do not infer intent from the keyword text alone. “Best SEO tools” reads commercial; so does the SERP. “SEO audit” could be informational (how to do one) or commercial (tools to do it): only the SERP shows which is dominant right now. The search intent article covers the four intent types and how to read the SERP for each.
Match your content type to the SERP format
For each query, record whether your existing or planned content matches what’s ranking. If the top results are comparison roundups and you’re planning a how-to guide, the intent is mismatched. Either reposition the content or deprioritise the query.
Remove intent mismatches
Queries where your content type cannot match what the SERP shows should be removed or deprioritised. You can note them for future content types you don’t currently produce, but they don’t belong in the active mapping stage.
Stage 5: Map queries to pages
Mapping assigns each surviving query to a specific URL. Every query gets a home: either an existing page that already covers it, a page that needs updating to cover it properly, or a new page.
Group by topic first
Before assigning to URLs, group related queries by topic. Queries that share the same searcher intent and would be satisfied by the same content belong together. This grouping stage reveals your content clusters: related pages that should link to each other and to a shared pillar page.
For each cluster, identify the primary query (the one that best represents the topic and carries the highest combination of volume and intent match) and the supporting queries that a well-built page should also satisfy.
Build the keyword map
The map is a spreadsheet with one row per page. Each row needs:
| Column | Content |
|---|---|
| URL | The page’s canonical address (existing or planned) |
| Primary keyword | The one query the page is built around |
| Supporting keywords | Related queries the page naturally satisfies |
| Intent type | Informational / commercial / transactional / navigational |
| Monthly volume | For the primary keyword |
| KD | From your chosen tool |
| Content status | Existing / needs update / needs creation |
Pages in “needs creation” status become your content plan. Pages in “needs update” status are your optimisation backlog. The map makes both visible in one place.
One primary keyword per page
Each page has one primary keyword. That query is not a string to hit a target number of times; it is the label for the single intent the page is built around, and every structural decision (title tag, H1, content angle) follows from that intent. Supporting keywords are related queries the page satisfies naturally; they don’t require separate sections or forced inclusions. A page that answers its intent thoroughly will rank for a long tail of related phrasings it never states verbatim, which is the whole point of the semantic model: you build for the topic and the intent, and the wording follows.
A page cannot effectively target two queries with conflicting intent. A page cannot be both “what is keyword mapping” (informational) and “keyword mapping tool” (commercial). If a valuable query has a different intent to your existing content, it needs its own page.
The keyword mapping article covers the map structure and common mistakes in detail.
Stage 6: Check for cannibalisation
Before acting on the map, check whether existing pages are already competing for the same primary keywords. Cannibalisation is common on established sites and invisible until you look for it systematically.
The GSC method
Open Search Console Performance, filter by a specific query, and switch the breakdown to Pages. If two or more of your URLs have impressions for the same query, you have cannibalisation to resolve.
Work through your highest-volume primary keywords this way. It doesn’t take long, and the findings often change prioritisation: pages you were planning to create may already exist, and optimisation may outperform creation.
The site: search method
For a quick check on any query: run site:yourdomain.com "keyword phrase" in Google. Multiple results suggest Google sees content overlap.
How to resolve cannibalisation
Once you’ve identified competing pages, the resolution is usually one of three options:
Consolidate. Merge the weaker page into the stronger one via a 301 redirect. The surviving page absorbs both sets of content and any links pointing to the redirected URL. This is the right call when both pages cover the same topic and intent.
Differentiate. If the pages serve genuinely different intents or audiences, sharpen the targeting so they’re no longer competing. A page on “keyword research for e-commerce” and a page on “keyword research for local SEO” can coexist if both are specific enough that Google doesn’t see them as competing.
Noindex or remove. For thin or low-value pages with no links worth preserving, removal is cleaner than a redirect. The keyword cannibalisation article covers detection and resolution in full.
What to do with the completed map
The keyword map is the input to every downstream content decision:
- Content plan. Rows with “needs creation” status, ordered by volume and difficulty, become your content backlog.
- Optimisation backlog. Rows with “needs update” status, particularly near-miss queries from GSC, become your first wave of improvements.
- Internal linking. Pages grouped into the same cluster should link to each other. The map makes those relationships explicit.
- Content briefs. For each new page, the primary keyword, intent type, supporting keywords, and SERP format noted during stage four provide the brief outline.
Keeping the map current
A keyword map becomes stale as new queries surface, competitors expand their coverage, and the business changes. The full five-stage process doesn’t need repeating from scratch; a maintenance cadence handles ongoing updates.
Monthly: pull new GSC queries (filter for the past 28 days), add any new terms to the universe, flag near-miss queries that have appeared since the last review.
Quarterly: run a competitor gap analysis against two or three competitors. Export new queries they’ve added rankings for. Add any relevant terms to the universe and filter them through stages three to five.
When the business changes: new products, new audiences, or new market expansion often require a partial rebuild of the discovery stage for the new topic area. Treat it as a focused run of stage one for the new scope, not a full restart.
The keyword universe article covers the persistent-asset model in more detail, including how to structure the universe for long-term maintenance.