Posts

How to Create and Optimize Your Robots.txt File

/0 Comments/in /by

Key Takeaways

  • Robots.txt is a plain text file in your root directory that tells search engine and AI crawlers which pages on your site to crawl and which to skip.  
  • By guiding bots away from technical clutter and low-value pages, you make sure they spend their time on the important, high-value content that drives results. 
  • The four AI crawlers most worth knowing (GPTBot, ClaudeBot, Google-Extended, and CCBot) respect robots.txt directives and can be blocked individually with their user-agent strings.  
  • Common robots.txt mistakes include using disallow: / on a live site, blocking CSS or JavaScript files (which hurts rendering), and confusing disallow with noindex, since a disallowed page can still be indexed if linked externally.  

Think of your robots.txt file as your site’s GPS.  

It tells web crawlers for search engines like Google or Bing (and now AI) where to look and what to index. That’s significant in today’s search world. Yet, it’s often an overlooked part of technical SEO. 

Many treat robots.txt with a set-it-and-forget-it mentality, not realizing the toll that can take on search visibility.  

With AI now claiming top positions on the search engine results pages (SERPs), the right robots.txt configuration is more important than ever.  

To help you stay ahead, I’ve put together this refresher on how to create a robots.txt file that promotes modern-day visibility and delivers real business results.  

What Is a Robots.txt File?

The robots.txt file, also known as the robots exclusion protocol or standard, is a text file that tells web robots (often search engine crawlers and AI scrapers) which pages on your site to crawl. 

It also tells web robots which pages not to crawl. 

Let’s say a search engine is about to visit a site. Before it visits the target page, it will check the robots.txt for instructions. 

There are different types of robots.txt files, so let’s look at a few different examples of what they look like. 

Let’s say the search engine finds  this example robots.txt file: 

An image displaying the correct basic structure of a robots.txt file 

This is the basic skeleton of a robots.txt file. 

The asterisk after “user-agent” indicates that the robots.txt file applies to all web robots visiting the site. 

The slash after “Disallow” tells the robot not to visit any pages on the site. However, it’s important to note that disallowing a page won’t prevent it from being indexed if external links are pointing to that page.  

Why Robots.txt Matters for SEO

You might wonder why anyone would want to stop web robots from visiting their site. 

After all, one of the major goals of traditional and AI SEO is to get search engine or AI bots to crawl your site easily, thereby increasing your visibility. 

That’s where the secret to this SEO hack comes in. 

You probably have a lot of pages on your site, right? Even if you don’t think you do, check. You might be surprised. 

If a search engine crawls your site, it’ll crawl every single page. 

And if you have a lot of pages, it’ll take the search engine bot a while to crawl them. That can negatively affect your ranking. 

That’s because Googlebot (Google’s search engine bot) has a “crawl budget.” This breaks down into two parts.  

The first is the crawl capacity limit, which is the maximum number of connections Google can use to crawl a site at any given time. Google goes into more detail here: 

A screenshot of Google Developer resources explaining how Googlebot’s crawl capacity limit works 

The second part is crawl demand, which is essentially Google’s appetite for your content. It comes down to how popular your pages are and how often you update them. Here’s a deeper explanation from Google: 

Google resources explaining how Googlebot’s crawl demand works 

Basically, crawl budget is “the number of URLs Googlebot can and wants to crawl.” 

You want to help Googlebot spend its crawl budget for your site as efficiently as possible. That means you want it crawling your most valuable pages. 

To make sure you’re leading bots to the right places, Google advises minimizing these common drains on your crawling resources: 

  • Faceted navigation: URL parameters for sorting and filtering can create an “infinite space” that traps bots in a maze of redundant pages. 
  • Duplicate content: When the same information exists across multiple URLs, consolidate them so crawlers can focus on your unique content. 
  • Hurdles and dead ends: Soft 404 errors and long redirect chains waste crawl demand, forcing bots to work harder for no reward. 
  • Server performance: If your site responds slowly, Google may not be able to read as much content from your site. 

OK, let’s come back to robots.txt. 

A well-structured robots.txt page tells search engine bots (and especially Googlebot) to avoid certain pages. 

Think about the implications. By curating your robots.txt file, you’re highlighting your best work. You’re effectively steering the bots away from technical clutter and toward your most valuable content. 

In other words, your robots.txt helps make sure that every second a bot spends on your domain is a worthwhile one. It’s the difference between a bot wandering aimlessly through your digital storage and one that heads straight for the pages that drive results. 

Intrigued by the power of robots.txt? Let’s talk about how to create a robots.txt file and use it properly. 

How to Create a Robots.txt File 

Using robots.txt effectively starts with getting the basics right. Follow these steps to create a robots.txt file that gets your “website GPS” off to the right start.  

Step 1: Open a Plain Text Editor

You can create a new robots.txt file by using a plain text editor, like Notepad on PC and TextEdit on Mac. Whatever you use, make sure it’s a plain text editor. 

If you already have a robots.txt file, make sure you delete the text (but not the file) to give yourself a fresh start. 

how to create a robots.txt file 006

Step 2: Locate and Format Your File Properly

To start, you must name your file “robots.txt.” That may seem obvious, but it’s so important that it’s worth stating. If you get your naming wrong, nothing else that you do will matter. 
 
Also note that each site can have only one robots.txt file. That file must also be placed at the root domain of the site it applies to. 
 
Google provides more context here (we also summarize the key takeaways below): 

Google documentation explaining the correct location and formatting for a robots.txt file  

 Think of it as the technical fine print. Here are the three biggest things to keep in mind from Google’s guidance: 

  • Location is everything: Your file must live at the root of your host (e.g., yoursite.com/robots.txt). If you tuck it away in a subfolder, crawlers simply won’t look for it. 
  • Stay in your lane: A robots.txt file only has authority over its specific protocol (HTTP vs. HTTPS), subdomain, and port. If you have a mobile site (m.yoursite.com), it needs its own dedicated file. 
  • Stick to UTF-8: The file must be a plain text file with UTF-8 encoding. If you use non-standard characters, Google might find your rules invalid and ignore them entirely. 

Step 3: Write Your Robots.txt Rules

I’m going to show you how to set up a simple robot.txt file, putting the rules we mentioned above into action. 

Every robots.txt file starts with the user-agent directive. This defines which crawlbot is subject to the rule. This example from Google’s robots.txt documentation sets Googlebot as the user. 

An example robots.txt rule allowing Googlebot to crawl any webpage on www.example.com that doesn’t have the /nogooglebot/ URL slug 

The example also defines two rules: allow and disallow. They enable the robots.txt file to guide Googlebot toward any page under the root domain www.example.com, except for those with the /nogooglebot/ URL path. All other crawlbots are free to crawl any page within the site.  

I know it looks super simple, but these two lines are already doing a lot. 

This rule also links to an XML sitemap, but that’s not strictly necessary. It serves as a universal map for all crawlers, including AI. It’s especially important for larger sites, as it gives bots a direct path to your most valuable pages without them having to hunt for links. 

Voila, you now have a basic robots.txt file with simple (but effective) rules in place.  

As you get more familiar with using robots.txt, there are more rules you can use to your advantage. Google lists them all, along with what they do, here.   

Step 4: Save and Upload to Your Root Directory

To do its job, your robots.txt file needs to be uploaded to your site’s root directory. How you do this depends on your hosting platform and your site architecture. 

A common exception to this is WordPress, which can generate its own virtual robots.txt file when you launch a site. To change it, you may need a plugin or manual upload to override it.  

When in doubt, though, contact your hosting platform or search through their support documentation for upload methods. You can usually do this by navigating to their help articles or knowledge base and searching “upload files [hosting company name].”  

How to Block AI Crawlers with Robots.txt

Blocking AI crawlers gives you more control over how your content is used.  

Some site owners do it to limit AI training use. Others do it to reduce crawler load, protect gated-style content that accidentally became public, or keep competitors from repackaging their work through AI tools. 

The trade-off is visibility. If you block everything, you may protect more of your content, but you can also reduce your chances of showing up in AI-generated results. 

The major AI crawlers worth knowing are GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended (Google), and CCBot (Common Crawl). All four support robots.txt controls, and each publishes a specific user-agent string you can target.  

CCBot is one that many people overlook, even though its public dataset powers dozens of open-source models, making it too impactful to leave out. 

To block each crawler individually, list each user-agent with its own disallow rule: 

User-agent: GPTBot 

Disallow: / 

User-agent: ClaudeBot 

Disallow: / 

User-agent: Google-Extended 

Disallow: / 

User-agent: CCBot 

Disallow: / 
 
The major AI crawlers worth knowing span both training and search functions. OpenAI runs GPTBot for training and OAI-SearchBot for search. Anthropic runs ClaudeBot for training and Claude-SearchBot for search. Google uses Google-Extended for training. CCBot, run by Common Crawl, powers dozens of open-source models, so it’s worth including even though many people overlook it. 
 
That distinction matters in practice. Blocking GPTBot does not block OAI-SearchBot, and blocking ClaudeBot does not block Claude-SearchBot. If you want to stop both training and search crawling, you need separate rules for each bot. 
 
All of these crawlers support robots.txt controls, and each publishes a specific user-agent string you can target. To block them individually, list each user-agent with its own disallow rule: 

User-agent: GPTBot  
Disallow: / 

User-agent: OAI-SearchBot  
Disallow: / 

User-agent: ClaudeBot  
Disallow: / 

User-agent: Claude-SearchBot  
Disallow: / 

User-agent: Google-Extended Disallow: 
User-agent: CCBot Disallow: / 

If you’d rather block every non-search bot at once, flip the logic. Disallow everything by default, then explicitly allow the search engines you want to keep. 

User-agent: * 
Disallow: / 

User-agent: Googlebot 
|Allow: / 

User-agent: Bingbot 
Allow: / 

Note that Google-Extended is a separate token from Googlebot. Blocking it opts you out of Google’s AI training data and has zero effect on how you rank in regular Google Search. 

Keep in mind that while blocking AI crawlers stops your content from feeding model training, it also reduces your chances of getting cited in AI answers. It’s important to proceed with caution if you want to implement these rules.  

If AI visibility is part of your strategy, use an llms.txt file for SEO to guide AI systems toward your best content rather than locking them out entirely, as you would with your robots.txt file. 

How to Test Your Robots.txt File

After your robots.txt file goes live, confirm Google can read it correctly. Google retired the old standalone robots.txt Tester in late 2023 and replaced it with the robots.txt report inside Google Search Console. 

To find it, open Search Console, pick your property, and click Settings in the left sidebar. The report shows which robots.txt files Google has fetched for your site, when each was last crawled, and any syntax errors or warnings it hit during parsing. If you’ve just pushed an update, you can request a recrawl right from that screen. 

A screenshot displaying the location of the robots.txt report within Google Search Console 

Source 

To test how a specific URL behaves under your current rules, switch to Search Console’s URL Inspection tool. It tells you whether Googlebot can access the page or whether a directive is blocking it.  

This move is useful for catching a misplaced disallow rule before it tanks an important page. Make this part of your regular technical SEO site audit. 

A screenshot of Google Search Console’s URL inspection tool 

Another pro tip: Type the root domain followed by /robots.txt in your browser to view that site’s robots.txt file. It’s a quick way to see how competitors structure their rules, which directories they protect, and which AI crawlers they’re blocking.  

Pair it with a full SEO audit for a complete picture of where you can improve and overtake your competition. 

Common Robots.txt Mistakes to Avoid

Robots.txt mistakes are easy to make and hard to spot until traffic drops. Even small errors can have site-wide consequences.  

Here are the most common missteps to watch for: 

  • Using disallow: / on a live site. This single line blocks every URL on your site from every crawler, including your homepage. It usually slips into production when a staging file gets pushed live without being updated, so be sure to review your robots.txt after every migration. 
  • Blocking CSS and JavaScript. Googlebot renders your pages the same way a browser does, so it needs access to your CSS, JavaScript, and image files to evaluate them properly. Blocking these resources can force Google to crawl your site “blind,” resulting in demoted rankings. 
  • Confusing disallow with noindex. A disallow rule stops crawling but doesn’t prevent indexing. A blocked URL can still appear in Google Search if it’s linked from another site. To keep a page out of search results, use a noindex meta tag or password-protect the page instead. 
  • Leaving the file empty or missing. A missing robots.txt won’t break your site. Google will assume everything is crawlable, but you lose the ability to point crawlers to your sitemap, manage crawl budget, or opt out of AI crawlers. Build it into your standing SEO checklist so it’s not an afterthought. 

FAQs

How does robots.txt work? 

Crawlers check yoursite.com/robots.txt before crawling your pages. The file uses user-agent and disallow directives to tell them which paths to skip. Compliance is voluntary, but major crawlers respect it. 

 Do I need a robots.txt file? 

Not necessarily. Google can crawl your site without one, but the file lets you control crawl budget and block AI training crawlers, which is worth doing even for small sites. 

What should a robots.txt file look like?

A minimal file that allows all crawlers and points to your sitemap looks like this: 

User-agent: * 

Disallow: 

Sitemap: https://yoursite.com/sitemap.xml 

Add disallow rules for any directories you don’t want crawled, like /wp-admin/ or /checkout/. Use a separate user-agent block per crawler you want to give different rules to. 

How do I edit robots.txt in WordPress? 

The easiest path is an SEO plugin like Yoast, which includes a robots.txt editor in its settings. Otherwise, edit the file via FTP or your hosting file manager and upload it to your site’s root directory. 

How do I fix “Indexed, though blocked by robots.txt?”

This warning means Google indexed a URL it couldn’t crawl. Either remove the disallow rule so Google can read your page’s noindex tag, or password-protect (or remove) the page entirely. 

Conclusion

Robots.txt is a small file with a big impact on how your site shows up across the web. A few well-placed directives can keep low-value pages out of search results and decide whether AI systems get to train on your content. 

Already have a robots.txt file? Audit it against the mistakes covered above.  

Starting from scratch? Build it using the steps in this guide and test it in Search Console before calling it done. 

The conversation around robots.txt has shifted. What started as a tool for managing Googlebot and the SERPs now extends to handling AI’s rise in search and emerging standards like llms.txt.  

Whatever comes next, robots.txt remains a foundational part of staying in control of your content. 

Read more at Read More

Sundar Pichai: Google Search, AI agents, and tools will become one

/0 Comments/in , /by
Google agent future

Google CEO Sundar Pichai said Google’s AI search box, app-building tools, and agent products will eventually converge.

Pichai also rejected publisher fears that Google will stop sending traffic to the web in an interview with Nilay Patel, editor-in-chief of The Verge.

  • “Through it all, we are very committed to both meeting user expectations and also connecting them to what’s out on the web,” Pichai said.

However, Pichai’s comments showed why those fears are growing: Google is moving Search toward conversations, agents, and AI tools that can answer questions or complete tasks for users without requiring a click.

Why we care. Many people worried AI Mode would become the default Search experience at I/O. It didn’t, and many people were relieved. AI Mode still isn’t the default Search experience, but Pichai outlined a future where Search, Gemini, and agents merge into a single AI layer for finding information, creating content, and completing tasks.

Agents are the future. AI agents are the next major change for Search and the web, Pichai said:

  • “I look at agents, and that is the next evolution of the web. I think it will evolve the web pretty profoundly.”

Google is building agentic tools across Search, Gemini, Spark, and Antigravity. Pichai said those products should eventually come together for users. Pichai has previously said that he sees Google Search evolving into an ‘agent manager’.

One product. Asked whether Google’s AI search box, app-building tools and agent products should become one product, Pichai said: “It will.”

Pichai said agents should work in the background when users plan trips, build things or complete tasks. Pichai said Google is still building the pieces needed for that shift.

  • “We are laying a lot of the primitives of what we need for agents to work end to end, and more importantly, for AI to work,” he said.

Dig deeper. Google’s Liz Reid: Search and Gemini may converge, or diverge further

Google rejects Google Zero. Patel asked Pichai about Condé Nast CEO Roger Lynch’s recent comment that the publisher was planning as if search traffic would fall to zero.

  • “That is Google Zero” Patel said. “Condé Nast is saying, ‘We’re assuming that search will go to zero.’ How would you respond to that?”

The information market has changed far beyond Google, Pichai said:

  • “The information ecosystem is so much broader beyond Google, by far. We see it in the data, you see it everywhere,” Pichai said.

Pichai said publishers have spent years adjusting to new formats, platforms, and user habits.

  • “It’s exceptionally dynamic, and so it makes sense to me every publisher is adapting to this new world.”

Google says some clicks are going away. Patel pressed Pichai on whether publishers should plan for a future where search traffic falls to zero, citing Lynch’s comment that his teams were told to “assume there is no search.”

Pichai declined to tell publishers how to run their businesses.

  • “I’m not in a position to tell such an iconic publisher what they should think about their business or plan,” Pichai said. “If they are building content that is high-quality and people like it, I expect us to reflect that in our products. That much I can commit to them.”

He also said Search is already filtering out some visits.

  • “As the technology improves, low-quality clicks get filtered out,” he said. “That’s a natural evolution we see. We see it in our metrics. Bounce clicks are going down.”

Google points to subscriptions. Subscriptions are one way Google is adapting to publisher business models, Pichai said.

  • “One of the small features we have done, but very important I think, is if you’ve subscribed to something, we reflect that as a preferred source for you as a user.”
  • “We are adapting to the fact that publishers are increasingly turning to subscription offerings, too.”

The interview didn’t address why many publishers moved harder into subscriptions: Because they could no longer rely on search traffic the way they once did.

Search had to move faster. Pichai said Google reorganized Search because the product needed to move faster in the AI era.

  • “Search needed to move faster, and Search was split across many leaders,” he said.

Google put Search under Elizabeth Reid, with Nick Fox leading the broader area. Josh Woodward also helped lead Labs and later Gemini work. The goal was to make Google “set up well for this moment where we need to move faster as a company, which means we need to make faster decisions,” Pichai said.

The interview. Sundar Pichai on AI, the future of search, and what’s happening to the web

Read more at Read More

Google folds Display Ads into Demand Gen campaigns

/0 Comments/in , /by

Google is moving Display Ads management into Demand Gen campaigns as it pushes advertisers toward more unified, AI-driven campaign structures.

What’s happening. Advertisers can now manage Google Display Network (GDN) placements directly through Demand Gen campaigns whilst retaining the option to run ads exclusively on GDN if preferred.

Demand Gen campaigns will continue serving ads across YouTube, Discover, Gmail, Maps and the broader Display Network, bringing Display inventory into a more centralized campaign environment.

Why we care. Google is steadily consolidating more inventory, automation and AI optimization into Demand Gen campaigns, making it increasingly important for performance and discovery advertising strategies.

The update gives Display advertisers access to newer AI-powered features, broader cross-surface reach and potentially stronger efficiency, while also signaling that traditional standalone Display campaign management may become less central over time.

The bigger picture. Google is increasingly positioning Demand Gen as a central campaign type for visual discovery advertising, combining social-style creative distribution with Google’s AI targeting systems.

The company says advertisers adding GDN inventory into Demand Gen campaigns are seeing, on average, a 9.5% increase in ROI.

Between the lines. The move also gives Display advertisers access to newer Demand Gen features announced at Google Marketing Live, including expanded channel controls and future AI-powered campaign capabilities.

What to watch. As Google continues consolidating campaign management into fewer AI-led products, advertisers may need to rethink how they separate upper-funnel discovery, Display and performance-focused media buying.

Read more at Read More

Google’s latest AI ad push shows ads are becoming conversations, not clicks

/0 Comments/in , /by

Google Ads Liaison Ginny Marvin recently published an extensive piece outlining more than 40 new innovations across Google Ads, Analytics, creative tooling, AI, lead generation, and measurement. While the updates span everything from conversational AI to predictive attribution, the bigger story underneath the announcements is much more significant.

Google is steadily reshaping advertising around intent prediction, AI-assisted decision-making, and automation systems designed to qualify users long before they become customers.

The article itself positions these launches as solutions to a problem every lead generation marketer understands well: the gap between generating leads and generating good leads.

Google wants ads to become conversations

One of the clearest examples of this shift is Business Agent for leads. Instead of relying solely on traditional click-through experiences, Google is introducing conversational AI interactions directly within Search Ads.

According to Marvin’s piece, prospective customers will be able to ask detailed questions about services, expertise, availability, or pricing and receive responses grounded in a business’s website content.

That fundamentally changes the role of the ad itself.

Historically, lead generation followed a relatively simple path: click the ad, visit the landing page, fill in the form.

Now Google is attempting to insert AI-powered qualification and reassurance directly into the ad experience.

For businesses operating in sectors where trust matters — such as finance, legal, healthcare, or home services — this could significantly alter lead quality dynamics.

The lead arriving after an interactive conversation is very different from someone who clicked impulsively on a headline.

Intent is becoming more important than volume

Many of the launches outlined by Marvin point toward the same strategic direction: Google increasingly wants advertisers to optimise toward predicted business outcomes rather than raw conversion volume.

Features like lead intent scores, journey-aware bidding, qualified future conversions, and enhanced spam filtering are all designed to reduce the number of low-quality leads entering pipelines.

In theory, this solves a genuine industry frustration.

Too many campaigns optimise toward cheap conversions that never turn into customers.

But there’s another side to this evolution.

As Google handles more of the qualification, forecasting, attribution, and optimisation process, advertisers lose more visibility into how decisions are being made.

And that becomes even more important as AI-driven campaign systems continue expanding.

AI Max feels like the next evolution of Performance Max

Another major takeaway from Marvin’s article is how aggressively Google is extending AI-driven optimisation into Search itself.

AI Max applies broader algorithmic exploration logic to Search campaigns, allowing Google’s systems to expand targeting and discover additional query opportunities beyond traditional keyword intent.

For ecommerce advertisers with strong revenue tracking and reliable first-party data, this could unlock meaningful scale.

For lead generation advertisers without robust offline conversion data, however, the risks are much higher.

This is where many advertisers may repeat the same mistakes seen during the early rollout of Performance Max: over-trusting automation without feeding back enough business-quality signals into the system.

AI systems optimise based on the data they receive.

If a campaign only tracks form fills, Google will optimise toward more form fills — regardless of whether those leads ever become customers.

That’s why so many of Google’s launches now focus heavily on offline conversion imports, first-party data integration, unified enhanced conversions, and CRM connectivity.

The advertisers who can feed richer revenue and sales-quality signals back into Google Ads will likely gain the biggest advantage in this new AI-led environment.

Measurement is becoming predictive

One of the most important shifts hidden within these announcements is Google’s move toward predictive measurement models.

Features like Attributed Branded Searches and qualified future conversions aim to connect ad exposure with downstream behaviours that may happen months later.

Instead of simply measuring what happened historically, Google increasingly wants to estimate what will happen next.

That could help advertisers better understand long buying journeys where awareness campaigns influence conversions far outside traditional attribution windows.

But it also creates growing dependence on AI-generated forecasting systems advertisers cannot independently audit in full.

This may become one of the biggest strategic conversations in PPC over the next few years:
how much visibility are advertisers willing to trade for automation and efficiency?

Creative production is becoming infrastructure

Another notable theme throughout Marvin’s piece is how Asset Studio is evolving into a full-scale AI creative production ecosystem.

Google is no longer treating creative generation as separate from media buying. Instead, the platform increasingly wants to generate assets, analyse them, optimise them, and test them automatically at scale.

For lean marketing teams, this could dramatically reduce production bottlenecks and lower creative costs.

But if AI-generated creative becomes widely accessible to everyone, differentiation becomes even more dependent on brand strategy, audience understanding, and first-party insights rather than production capability alone.

The bigger picture behind the announcements

Individually, many of these launches may feel incremental.

Taken together, however, they reveal a much larger shift happening across Google Ads.

Google is steadily positioning itself as the infrastructure layer behind modern advertising decision-making. The platform increasingly wants to:

  • facilitate customer conversations,
  • qualify leads,
  • generate creative,
  • optimise budgets,
  • predict future outcomes,
  • and unify measurement across channels.

For advertisers, the challenge now is balancing automation with visibility.

AI systems can absolutely improve performance. Predictive models can uncover opportunities humans miss. Automation can unlock efficiency at enormous scale.

But the marketers who succeed long term will likely still be the ones who understand which signals actually matter, what drives genuine business outcomes, and when human judgement needs to override the machine.

Dig deeper.

Read more at Read More

Google’s Nick Fox: AI search rewards content that goes deeper

/0 Comments/in , /by
Google go deeper

Content must go beyond surface-level answers to stand out as AI summaries take over more basic search queries. That’s according to Nick Fox, Google’s senior vice president of Knowledge & Information, who was interviewed at Google Marketing Live 2026 by Semafor editor-in-chief Ben Smith.

What hasn’t changed. Fox said the way to rank in AI search is still the same as traditional search.

  • “The way to optimize for AI search is the same way to optimize for search. Create great content.”

But you need to go further than basic summaries, he added:

  • “The additional piece of advice we give is go beyond the surface level.”

He said Google’s AI summaries may provide the first layer of information for many queries. The content most likely to perform well will answer the next layer of questions, he said:

  • “If you assume that the AI will provide sort of a first-level response, high-level framing, the best content that will do the best within AI is one that goes one level deeper, two levels deeper, and is really helpful there.”

Fox didn’t explain how Google measures “deeper” content or how it separates useful depth from longer, more detailed pages.

Google wants content AI can’t easily copy. The comments echoed Google’s new AI search guidance, which warns against “commodity” content that repeats what others have already published or what generative AI models can easily produce.

Google said content built around common knowledge and generic summaries adds “little unique insight.” It described stronger content as work that provides expert or experienced takes that go beyond ordinary information.

Fox reinforced that idea during the interview when discussing the future role of the web in AI search.

  • “If you’re looking to buy something, you don’t just want to hear what the AI says. You want to hear someone that’s used it. What did they think? What went wrong with it? What was amazing about it? How did they what accessories did they get? You know, all of that kind of rich human content.”
  • “As humans we want to hear from humans. We want to hear human perspectives. We want to hear human experiences.”

Traffic concerns unaddressed. Fox’s comments made clear that Google sees human experience as a key part of the web’s value as AI answers expand.

  • The interview didn’t address publisher concerns about AI summaries reducing organic search traffic. While Google says it wants original, experience-driven content, AI answers reduce the clicks that help support that work.

Search queries are getting longer. Search behavior has already changed as people become more familiar with conversational AI tools, Fox said:

  • “The questions that people are asking now are these two-, three-, four-sentence queries.”

He said users are increasingly searching with natural-language prompts that include more context, problems, and constraints, rather than short keyword phrases. Google didn’t share any supporting data during the interview.

Why we care. AI-generated answers are already giving searchers basic informational summaries. So your content needs original reporting, firsthand experience, or useful analysis that gives people something they can’t get from a generic AI response.

The interview. Google’s Nick Fox on the Future of Search and AI

Watch this video on YouTube

Dig deeper. Google’s AI search guidance is naive and self-serving

Read more at Read More

Interrupting buyer journeys: The SEO strategy hiding in plain sight

/0 Comments/in , /by
Interrupting buyer journeys- The SEO strategy hiding in plain sight

Most content meets users exactly where they are. Someone searches “best MBA programs” and gets a roundup of MBA programs. But sometimes the highest-value content challenges the query’s premise. This introduces the concept of surfacing alternatives that users didn’t know to ask about.

Intentionally expanding a user’s awareness beyond their assumed path doesn’t always take center stage in SEO and content marketing strategies. However, when done correctly, it can help your services and products appear for more keywords while educating your audience about more solutions to their problems.

For example, when someone searches for a specific degree, medication, certification, or product, they’ve often locked in on a solution before fully evaluating the problem. Content that respectfully introduces alternatives (“apprenticeships vs. four-year degrees,” “herbal supplements vs. prescription options,” or “business bootcamps vs. MBA programs”) can capture high-intent traffic while delivering more value than a straight intent match.

Here’s a roadmap for making this strategy part of your ongoing editorial production.

LLMs are already doing this

LLMs and AI Overviews are already doing a version of this. After answering your query, they often ask a follow-up question, such as whether you’d like to learn about alternatives or explore the topic more deeply. Following an LLM down this path can lead users toward alternatives they didn’t know about.

For instance, in the supplements query below, I was looking for supplements to help with mood and stress. (Note: LLMs and AI aren’t a replacement for medical advice. Always speak with your medical professional before making changes to your diet, medications, supplements, or other health-related routines.)

I gave ChatGPT the stack of supplements I was already taking and asked whether I should remove any. Unprompted, it also asked the following question:

ChatGPT - search query on food supplements

After we went back and forth with suggestions and questions, it gave me additional modifications I hadn’t asked about, including timing recommendations and suggestions tied to other details I’d mentioned previously, such as caffeine use.

ChatGPT - search query on food supplements additional suggestions

In this case, ChatGPT went beyond telling me which supplements might help with stress, which is usually what happens in SERPs for a query like “mood supplements.” It helped me build a better supplement protocol.

This is what you can do for audiences searching for solutions. 

Your customers search everywhere. Make sure your brand shows up.

The SEO toolkit you know, plus the AI visibility data you need.

Start Free Trial
Get started with

Semrush One Logo

How to identify queries where users may benefit 

Let’s say you’re optimizing for “mood and stress supplements” for products designed for that purpose. 

To expand your keyword research beyond obvious queries, think about why someone may be searching for mood and stress supplements in the first place. They probably feel overwhelmed by work or personal life. They may be going through a temporarily stressful period and looking for ways to feel better.

With that line of thinking, you can expand your keyword research into related areas, uncover keywords about stress relief, and create articles and content that introduce other ways someone might relieve stress.

Often, this works the other way as well. A user may start their journey thinking they just need meditation, sound baths, or forest walks to calm stress and improve mood. While those things can help, they may not even be aware that mood supplements exist.

So while it’s a good idea for a supplement company to create content about mood and stress products, it’s also in its best interest to expand its content into other solutions for the problems users are facing. Then, in those articles, the company can include its products as another solution that users may not have considered.

For instance, in this article about sleep and stress, after including non-supplement solutions to help with stress, a product suggestion is included:

ChatGPT - sleep and stress non-supplement solutions

Structuring content around alternative solutions

When creating this type of content, focus on quality and valuable information above all else. When you provide high-value information, users stay on the page longer, click more internal links, and see your content as a resource they can trust.

Content should be structured so it ranks for the original intent while responsibly pivoting to the solutions you provide. Beyond written content, other ways to help users expand their horizons include:

  • Free spreadsheet or PDF templates, even if you offer database or document software (like Smartsheet).
  • User stories and testimonials about experiences with the problem, even if the solution wasn’t solely your offering.
  • Webinars, online courses, or in-person workshops related to your offerings. For example, a stationery store offering junk journal nights, or a bag charm retailer hosting a bag charm styling class at a winery.

Your offering shouldn’t be front and center, or it’ll quickly be labeled promotional content and won’t be taken seriously. Include product mentions organically in an article, webinar, or video through on-screen mentions, links within paragraphs, or examples that illustrate how something works.

These types of mentions may shift a user’s one-track mindset and introduce solutions they hadn’t considered before.

Get the newsletter search marketers rely on.

See terms.

Keyword and SERP signals that signify openness

When might a user be open to these types of journey-disruption options? It’s important to identify keywords and signals that indicate a user is in the research and consideration stage, rather than fully committed to purchasing a specific solution.

Branded terms

For instance, a user searching [“brand name” buy] is more likely to purchase that specific brand than someone searching terms that signal ongoing research, such as [“brand name” pricing], [“brand name” competitors], or [“brand name” reviews].

Industry ‘widetail’ queries

A “widetail” query is a term I’m using to describe a wider net of queries that all fall within the same user journey. For instance, a user struggling to keep their lawn mowed may search terms like these within the same period, even though they represent different angles of the same problem:

  • “Robot lawnmower price”
  • “Lawn service near me”
  • “How often to cut grass?”
  • “Sprinkler watering schedule”
  • “Price to pay teenager for cutting grass”
  • “Grass cutting schedule”

Instead of solely optimizing for your landscaping company offering with terms like “lawn care in Kansas City,” interrupt earlier buyer journeys by creating content around terms your users are also searching for.

When ethical guardrails are needed

After using supplements as an example, it’s important to note that you have a responsibility to use this content strategy responsibly.

For industries that can negatively affect users, such as healthcare, careers, finance, or other YMYL verticals, exercise discretion to ensure you aren’t positioning your product as the solution to a serious problem that could affect users’ well-being.

It’s one thing to mention a supplement that may support stress response. It’s another to promise a “cure to stress.” FDA and FTC guidelines exist for a reason: to protect customers from misleading and potentially dangerous claims.

Interrupting buyer journeys at the right time

In the lawn care example above, we see several consideration funnels that all point to the same goal: making lawn care easier for someone who can’t keep up with it.

These queries represent the user’s attempts to figure out how to keep the grass mowed. Looking at each query as a standalone journey fails to account for the user as a whole customer.

Many customers don’t use varied queries. They may only search [“brand name” pricing] because they’re overwhelmed, their boss suggested that brand, or they don’t have time to explore other solutions.

By proactively expanding your content, you can appear during basic comparison searches and when tangential searches lead users to your site.

Getting in front of customers when they aren’t expecting you can be a powerful way to capture more search traffic, leads, and loyalty from an audience that’s glad to have found you.

Read more at Read More

Web Design and Development San Diego

SEO changelogs: The missing layer of enterprise site governance

/0 Comments/in , /by
SEO changelogs- The missing layer of enterprise site governance

Across large enterprise websites, dozens of stakeholders can push live changes at any time: SEO teams, developers, content editors, product managers, PR teams, UX designers, and more. One of the biggest frustrations is discovering those changes after they’ve already impacted performance.

Maybe a CMS template update quietly removes a core content component from hundreds of pages. Maybe a new product page rollout creates canonical mismatches at scale. By the time you notice the issue, rankings, traffic, reporting KPIs, and stakeholder conversations are already under pressure.

That’s where SEO changelogs come in. More than a simple record of deployments, a strong changelog process creates visibility, accountability, and cross-team awareness around website changes that can affect search performance.

Why enterprise SEO teams need changelogs

Enterprise SEO teams are often the last to know when impactful website changes go live. Even with strong workflows and deployment processes, changes can still happen across large websites without SEO visibility.

An SEO changelog helps close that gap by creating a documented, shared record of website changes that could impact SEO or wider digital marketing performance. That could include anything from metadata edits and schema updates to internal linking changes, template deployments, analytics implementations, or robots.txt updates.

A strong changelog process helps teams identify risks faster, understand the downstream impact of deployments, and reduce the likelihood of costly SEO surprises. It should clearly document what changed, where it happened, when it went live, and the intended outcome.

Large businesses already have deployment records through tickets, Git commit histories, or CMS audit logs. The problem is that these systems often exist in silos and rarely frame changes through an SEO lens. That leaves SEO teams reacting to issues or performance shifts after the fact instead of proactively monitoring them.

About 53% of enterprise teams struggled with SEO misalignment across departments, a 2023 Lumar study found. With Google SERPs more volatile than ever, enterprise SEO teams need stronger operational visibility into how websites evolve over time. A robust changelog process can help create that visibility.

Your customers search everywhere. Make sure your brand shows up.

The SEO toolkit you know, plus the AI visibility data you need.

Start Free Trial
Get started with

Semrush One Logo

The anatomy of an enterprise SEO changelog

A solid SEO changelog framework should strive to provide clear data on:

  • What was changed, exactly, and where.
  • The context.
  • The stakeholder. 
  • Expected impact.
  • Observed impact.

What was changed, exactly, and where

Include a clear definition and scope of the change made. For example:

  • Schema markup was updated on all product pages to include AggregateRating.
  • Hreflang tags were modified on URLs across 10 European markets.
  • The robots.txt file was updated to disallow a particular path.

The context 

Why was this change made, and what was the intended aim? This can be one of the most valuable inputs for retrospective analysis. For example:

  • Schema markup was implemented to improve the potential for rich snippet results.
  • Hreflang tags were updated to help search engines serve the correct regional version of the page to users in the respective market.
  • The robots.txt file was updated to prevent the path in question from being crawled following suboptimal crawl behavior patterns identified in Google Search Console. 

The stakeholder 

Who made the change, and what team are they on? This helps you make sure there’s a clear and efficient path to the person responsible for the change if action needs to be taken. Transparency and accountability are two core components of maintaining a strong culture of SEO awareness as part of the changelog process. 

Expected impact

While it may not be feasible or even necessary to detail the expected impact or the full rationale behind every deployed change, it should be encouraged where possible.

A larger, more ambitious deployment might have a forecast or broader business case attached to it. For example, there might be a site speed rationale behind optimizing a heavy component. 

Other changes might be straightforward tests tied to specific metrics without a clearly defined outcome, and that’s fine too. The idea is to get teams thinking about SEO-adjacent and broader business outcomes, rather than simply deploying changes to a site or webpage.

Observed impact

This is added retrospectively to the relevant changelog environment once sufficient data has been collected. It could include a report on clicks or impressions following a change, notes on the visibility of a keyword cluster, or even AI Overview citations. 

The goal is to build a culture of testing and learning alongside accountability and visibility.

The tools behind enterprise SEO changelogs

You want to eventually automate much of what’s currently logged, and several tools and approaches can help. Here are a few.

GitHub/GitLab webhooks

These webhooks can be configured to post deployment summaries to a centralized SEO changelog channel, such as Slack or email, or to a database whenever a production push occurs.

Jira/Linear automation

With either of these tools, you can set up a rule so that when any ticket with an SEO-impact label is moved to “Done” (i.e., deployed live in production), an entry is automatically created in the changelog with the ticket title, assignee, and completion date.

CMS change logs

Most enterprise CMS platforms, including Contentful, Sitecore, and Adobe Experience Manager, maintain internal audit logs. Consider surfacing these into your central changelog via an API or scheduled export.

Third-party SEO tool alerts

Tools like Botify, Lumar, and ContentKing have scheduling and alerting capabilities. When a change or crawl anomaly is detected, such as a spike in broken links, 3xx or 4xx response codes, or even a simple metadata change, users can be alerted quickly by email or via integrations with platforms such as Slack and act accordingly. 

Get the newsletter search marketers rely on.

See terms.

Building a changelog workflow

With the core tenets of the changelog defined, the next step is to create a workflow that functions smoothly at scale. A practical way to approach this is in three phases.

Start with a pilot

Start with one team and one simple logging method as your proof of concept. Development might be a particularly impactful place to start. Your changelog could initially live in a Slack channel or Google Sheet.

Expand and standardize the workflow

Once the value of the changelog becomes clear, especially when it captures a potentially harmful change that may have caused an issue, you can begin bringing in other teams and standardizing the format across departments.

From there, you can scale the process further by introducing some of the automation tools outlined above.

Add SEO context to the changes

Once the changelog is in place, the next step is having your SEO team provide context behind the changes. This is where SEO teams need to bring their proactivity and institutional knowledge into the process.

That means asking a series of questions and ensuring you have answers to them, including:

  • Are we aware of and aligned with the changes that have been deployed according to the changelog?
  • If a content block optimization led by the SEO team was deployed, was it implemented correctly according to our recommendations?
  • Has that complicated redirect chain been updated correctly to ensure a straightforward crawl path?
  • Are these new breadcrumb components something we recommended, or did they originate elsewhere in the business?

These are the types of questions a robust SEO changelog should help answer.

The SEO changelog as a buy-in tool

Enterprise SEO teams often struggle because of gaps in stakeholder management and organizational alignment.

Buy-in sits at the core of enterprise SEO. A robust SEO changelog process can help overcome some of the challenges of securing buy-in from non-SEO stakeholders within large organizations. Here are a few things to consider.

Think ‘business risk mitigation tool’ rather than solely ‘SEO changelog’

SEO changelogs can help reinforce the importance of SEO across a business. Position them as business risk mitigation tools rather than straightforward SEO monitoring systems. That framing speaks the language other teams already understand.

There are plenty of examples of site changes leading to major revenue losses across organic search and other channels. SEO changelogs should be positioned as a way to prevent those issues from going unnoticed. After all, something as simple as a faulty bulk canonical URL update across a series of product pages could cost thousands of dollars if left unchecked.

For large ecommerce brands with global website footprints, this challenge is especially common. Changes are regularly made across hundreds of product pages through template updates, content edits, and metadata adjustments without centralized visibility for SEO teams. Implementing a changelog system can help surface those changes automatically.

The bigger shift, however, is cultural. Once teams can see the downstream SEO impact of their changes, contributing to the changelog becomes a natural part of the workflow rather than something that needs to be enforced. 

Identify internal changelog champions

SEO affects multiple departments across a business. Is there someone in development, content, or product management who would benefit from this type of visibility? Identify those people early and work with them to embed changelog contributions into existing workflows.

  • For development teams, that might mean adding changelog updates to sprint definition-of-done checklists. 
  • For content teams, it could become part of the publishing signoff process. 
  • For QA teams, it may become a mandatory step before any production push.

A large-scale canonical URL mismatch isn’t just an SEO problem. It’s a business problem. When the right stakeholders understand that, changelog participation starts to feel less like an extra task and more like professional due diligence.

This level of governance should also extend to leadership, aligning SEO changelog processes with broader business OKRs and KPIs.

Communicate your changelog wins

When an SEO changelog identifies a potentially harmful issue before it impacts search visibility, traffic, or conversions, make sure the outcome is shared across relevant teams.

Be prepared to explain:

  • What issue did the changelog identify?
  • How quickly was it addressed?
  • What was the outcome?

Averted problems are often more persuasive than any presentation deck.

The same applies to positive outcomes. If changelog-tracked deployments led to measurable SEO wins, those insights should also be communicated upward across the organization.

Further ways to measure changelog success

SEO changelog processes should continue evolving over time. There are several metrics you can use to measure effectiveness and identify areas for improvement.

  • Coverage rate: What percentage of significant site changes are being logged? Were any important changes missed and only discovered later by the SEO team? 
  • Time to detection: How quickly can the SEO team identify issues after deployment? Can detection happen faster next time?
  • Issue interception rate: How many potentially harmful changes were caught and addressed before they impacted traffic or visibility?
  • Cross-team contribution: Is the SEO team the only group contributing to the changelog, or are other departments actively participating as well?
  • Correlation insights: Are meaningful patterns emerging between changelog entries and SEO performance? Are certain SEO-led optimizations consistently driving stronger outcomes on specific page types? Insights like these can be extremely valuable for refining SEO strategy and strengthening stakeholder buy-in.

See the complete picture of your search visibility.

Track, optimize, and win in Google and AI search from one platform.

Start Free Trial
Get started with

Semrush One Logo

SEO as part of brand culture

The broader goal of an SEO changelog extends beyond documentation. It’s about improving organizational awareness of how website changes impact SEO and other digital channels.

Large brands that build this kind of culture don’t just improve monitoring capabilities. They also strengthen institutional knowledge and make SEO more resilient over time.

The goal should be to make SEO visibility part of standard business operations rather than something SEO teams uncover retrospectively. Brands that succeed in organic search in 2026 will be the ones that treat SEO as a shared responsibility across teams, and SEO changelogs can play an important role in making that happen.

The SEO changelog is no longer just an operational safeguard. It’s also a strategic asset for navigating what comes next.

Read more at Read More

Web Design and Development San Diego

The new playbook for localized AI search optimization

/0 Comments/in , /by
The new playbook for localized AI search optimization

AI has become part of nearly every industry, integrated into apps, company processes, and everyday life. As someone who’s been doing local SEO since it became a thing, I’m seeing a major shift in how people search and the answers they get. 

In the good old days, the average local business could rank well by optimizing its website, optimizing its Google Business Profile, building about 50 citations, and asking for reviews. In an AI search world, those activities are table stakes.

To perform well in AI-powered local search, you also need to shape what the broader web says about your business, or, in other words, how well-known your brand is.

Think of local search as a digital “word-of-mouth” system.

  • What are people saying about your brand?
  • Are you mentioned in publications, blogs, or industry sites?
  • Do people talk about you on social media?
  • What sentiment exists around your business beyond your website and GBP?

These are the questions AI systems ask when users request local business recommendations. Here’s how to shape the reputation signals AI search engines rely on.

How to do competitor research for AI visibility

One of the first steps in an AI search strategy is identifying which brands LLMs recommend most often and finding out what they’re doing.

Identify which businesses get mentioned most in AI responses

AI responses change constantly, so you need to run the same query multiple times to study patterns.

Run your most common brand searches at least 20 times in your preferred LLM. You can do this manually or use software like Gumshoe or Waikay. These tools run synthetic prompts based on your business details and show how often you appear.

Brand visibility and competitive leaderboard

Identify the sites that AI most often cites

After identifying your competitors, look at the sources LLMs use. You can dig through the results manually or use one of the tools mentioned above.

Get your brand mentioned on those sites

Once you have that list of sites, try to get your brand mentioned on them.

If AI systems cite blogs, offer to contribute expert content. If they mention podcasts or YouTube channels, ask to be a guest. The goal is to amplify your brand.

Your customers search everywhere. Make sure your brand shows up.

The SEO toolkit you know, plus the AI visibility data you need.

Start Free Trial
Get started with

Semrush One Logo

How to build reviews for AI

Since Google has been the primary discovery channel for the past decade, most businesses have focused only on getting reviews on Google. To perform well in AI results, you also need reviews on other sites.

Diversify your review strategy

Ask for reviews on a wide range of sites: Yelp, BBB, Facebook, and other review sites prominent in your industry. Frequent reviews across diverse platforms increase your brand’s visibility and can also help rankings in traditional search results.

Optimize the way you ask for reviews

Don’t ask for generic reviews. Give customers direction. Guide them toward experiences or product qualities AI searchers may ask about.

For example, if you have a plumbing company, your review request might sound like this:

Hi [Name],

Thank you for trusting us with your hot water tank repair. If you have a moment, could you please leave us a review on [Link to Platform] and tell us how we did? Some things you could mention in your reviews:

— What plumbing issue did we help you with?
— Are you happy with the quality of our service?
— Did your plumber arrive on time and have a professional attitude?
— Do you think the cost matches the quality of the service?

Your review is a big help to us and to others looking for a quality plumber.

Thank you!
[Name]

AI systems directly cite review content, so you want to make sure you’re getting detailed reviews.

Respond to all reviews

If you aren’t responding to reviews, start now. AI systems read and consider the content in review responses.

Be everywhere

AI systems often scour the web for even obscure mentions of your business and use them to build responses. Your business should be present and active across platforms, including:

  • YouTube.
  • Reddit.
  • Industry forums.
  • Social media, especially LinkedIn.
  • Industry publications.
  • Local and hyperlocal blogs.
  • Local news sites.
  • Local and industry podcasts and video channels.
  • Best-of lists in your city or industry.
  • Press releases.

Be active on the platforms your peers and customers use. A tool like Sparktoro can show where your audience is active so you can focus your efforts there.

audience research

Get the newsletter search marketers rely on.

See terms.

How to write content that AI models love

You’re no longer writing only for humans. You’re also writing for machines, so your content structure has to change.

Dan Petrovic researched Google’s “grounding snippets,” or the sentences it selects from your page to build answers.

One of Petrovic’s key takeaways is that Google prefers sentences that are semantically close to the query and early on the page.

Get straight to the point

While humans might appreciate a well-written introduction that provides context, LLMs scan pages for answers to specific questions.

Because AI systems often scan content higher up on the page, present your key points in the first paragraph. Then make sure the rest of the page supports them.

Understand what questions to answer

This goes back to keyword research and query fan-out. Identify what people type into the search bar, or AI bar, to find businesses like yours. Your website needs to become an answer engine for those prompts.

For local businesses, these are the must-answer questions:

  • What do you do?
    • What products or services do you offer?
    • Who are your products or services for?
    • What problems do you solve?
  • Where are you located?
    • What neighborhoods or cities do you serve?
    • Do you offer on-site services, or do customers need to visit your location?
  • What are your business hours?
    • Do you offer emergency or same-day services?
    • Do you work weekends or holidays?
  • How can customers contact you?
    • What’s the booking process?
    • Do you offer quotes or consultations?
    • Is your business appointment-only, or do you accept walk-ins?
  • Why should someone choose your business?
    • What sets you apart from competitors?
    • Do you have awards or certifications?
    • Are you best known for a specific product or service?
  • How much do your products or services cost?
    • Do you offer discounts or packages?
  • What do customers say about you?
    • Can you display reviews and testimonials?
    • Can you show case studies or before-and-after examples?
  • What are the answers to your most frequently asked questions?
  • How do you demonstrate authority and expertise?
    • What does your work process look like?
    • Do you educate people in your field through tips, guides, or blog articles?

AlsoAsked is a great tool for expanding this question-generation process.

content research

Once you answer these questions, you can use a free tool like Qforia to do query fan-out and generate additional questions AI systems may ask in relation to users’ initial searches.

Answer these questions on your website. Then make sure your answers stay consistent across brand mentions on the web, including citations, guest articles, and press releases.

Structure your content in a machine-friendly way

Most local businesses describe their services like this: “Services we provide: plumbing, drain cleaning, pipe replacement, etc.”

You should do a better job of helping machines understand your business in a clear and concise way by using semantic triples.

A semantic triple consists of:

  • [Subject] + [predicate] + [object]

The subject is what you’re defining. The predicate describes the subject’s relationship to the object. The object is what defines the subject.

For example:

  • [Rescue Plumbing] [is] [a plumbing company in Denver].
  • [Rescue Plumbing] [provides] [drain cleaning services].

Drop the “we” and replace it with your brand name. Machines still need clear signals, so you need to explain what your business is and what it does as clearly as possible.

Have something new to say

Information gain is essential for AI search. Your content shouldn’t reiterate existing information. It should contribute something new.

LLMs want content that enriches their knowledge about your brand, your industry, and your location.

Draw on your personal and professional experience. Answer questions that haven’t been addressed in your industry. Describe on-the-job experiences only you can speak to. This is your opportunity to surface for AI searches your competitors don’t appear in.

See the complete picture of your search visibility.

Track, optimize, and win in Google and AI search from one platform.

Start Free Trial
Get started with

Semrush One Logo

Your AI visibility to-do list

AI visibility depends on more than your website and Google Business Profile. Use this checklist to strengthen the reviews, citations, content, and brand signals AI systems rely on.

  • Shift your local SEO strategy. Optimize and maintain your website and Google Business Profile while cultivating broader brand visibility across the web.
  • Identify your competitors and study their content and citation strategies.
  • Identify the sources LLMs cite in relation to your industry and location, and get your brand mentioned in them.
  • Diversify the sites where you collect reviews, optimize your review requests, and respond to all reviews.
  • Build your presence across blogs, social media, forums, YouTube channels, podcasts, and the press.
  • Write unique, informative, and comprehensive content on your website, citations, and brand mentions across the web. Structure key information using semantic triples.

There’s much more I could write about optimizing for localized AI search, but I’ve probably already exhausted your attention span, so stay tuned for the next article.

Read more at Read More

Security patch: Yoast SEO Premium 27.6.1

/0 Comments/in , /by

Yoast SEO Premium 27.6.1 is out now. This release contains a security fix affecting the Redirect Manager in Yoast SEO Premium. The good news: the vast majority of users are not impacted. If you’re a customer of Yoast SEO Premium, Yoast WooCommerce SEO, or Yoast SEO AI+, please read on. 

Are you affected? 

The vast majority of customers are not impacted. Your site is only potentially at risk if all three of the following are true: 

  • You are using a plan that includes the Yoast SEO Premium plugin. This includes Yoast SEO Premium, Yoast WooCommerce SEO, and Yoast SEO AI+ 
  • Your server runs Apache and you have manually changed your redirect method to write to .htaccess. If you’re using the default PHP-based redirects, you are not affected 
  • A user who has access to your site with edit_posts capability. Without this, the vulnerability cannot be exploited even if the other conditions are met 

What was the issue? 

An authenticated user could inject unexpected configuration into a site’s .htaccess file by including special characters in a redirect. Depending on what was injected, this could range from a site crash to, in the most serious cases, remote code execution.  

We have reviewed a sample of sites using the affected configuration and found no evidence of exploitation. There are no known cases of abuse. 

What’s fixed 

The patch includes three layers of protection: 

  • Input sanitization: control characters are now stripped from redirect fields before they’re saved 
  • Removed unused code: the specific endpoint involved in the vulnerability has been removed, as it was no longer used by the plugin anyway 
  • In-plugin warning: we’ve added a proactive notification that will alert you if anything unusual is detected in your redirects or .htaccess file, so you can review and act quickly without the need to go looking for it 

What you should do 

Please update to 27.6.1 from the WordPress plugins screen, your Admin can do this in under two minutes. 

If you meet all three conditions above, we recommend updating as soon as possible. Should you not, the security fix doesn’t apply to your setup, but keeping your plugins current is always good practice, and 27.6.1 is the version we recommend for everyone. 

If you’re unsure whether you’re affected, check your redirect settings directly at [www.yoursite.com]/wp-admin/admin.php?page=wpseo_redirects#/redirect-method,  if you don’t see .htaccess mode enabled, you’re not at risk. 

Security method in app UI

A full security advisory will be published soon. If you have any questions or concerns in the meantime, our support team is here to help you. 

Thank you for your continued trust in Yoast. 

The post Security patch: Yoast SEO Premium 27.6.1 appeared first on Yoast.

Read more at Read More

AI-Powered Lead Gen: The New Way Multi-Location, Franchises, and Global Companies Scale

/0 Comments/in /by

Key Takeaways

  • AI lead generation works best as a system, not a collection of separate tools. The three core layers are data, activation, and optimization.
  • Traditional lead gen breaks at scale because teams fragment strategy across locations, operate in silos, and rely on manual budget decisions.
  • Local search carries the highest purchase intent in digital marketing. Most multi-location brands are losing those searches due to inconsistent listings and weak profiles.
  • AI improves lead quality, not just volume. Lead-to-close rate by location is the metric that actually matters.
  • You don’t need a full overhaul to start. A focused 30-day rollout can produce measurable pipeline impact.

Multi-location brands are generating more leads than ever. And yet, many are still struggling to turn that activity into consistent revenue across every market they serve.

Here’s the real problem: traditional lead gen was never built for scale. It was built for one team, one market, one campaign at a time. The moment you’re managing dozens or hundreds of locations, that model cracks. Fragmentation sets in. Quality drops. And the manual work required to hold it all together eats your team alive.

AI lead generation changes the equation entirely, but only if you use it the right way. This isn’t about automating what you’re already doing. It’s about building a system that gets smarter across every location, every market, every campaign, at the same time.

This article lays out how to actually do that.

Why Traditional Lead Gen Breaks at Scale

Multi-location lead gen has three structural failure points. Once you can see them clearly, the solution becomes obvious.

Fragmentation. Different teams run different playbooks in different markets. There’s no shared learning system, no central source of truth, and no way to know why your top location outperforms your worst one. According to NP Digital survey data, only 16 percent of multi-location businesses report “very consistent” lead quality across their locations. The majority fall somewhere between “significant variation” and “highly inconsistent.”

A bar graph comparing Lead Quality consistency across locations.

Inconsistent quality. High lead volume in one region doesn’t translate to high revenue. The locations that look like top performers by lead count often rank near the bottom by close rate. Without visibility into lead quality at the location level, you’re optimizing for the wrong thing.

Manual optimization that can’t keep pace. Most teams still allocate budget manually, review performance monthly, and build campaigns market by market. That cadence worked when the scale was manageable. At 50 or 100 locations, it’s a liability. Budget decisions made quarterly can’t respond to demand signals that shift weekly.

Buyers make it harder, too. By the time someone contacts your business, they’ve already researched you using search, reviews, and word of mouth. 98 percent of consumers verify an AI-recommended brand before buying, and about 65 percent of Google searches now end without a click to any website. Your presence has to be consistent, accurate, and compelling long before a lead form ever gets filled out.

The old model is broken. The fix isn’t more campaigns. It’s a better system.

The AI-Powered Lead Gen Framework

The brands scaling successfully with AI for lead generation aren’t just using more tools. They’re using tools that connect.

Most companies have pieces of the puzzle. The problem is those pieces don’t talk to each other. Paid media AI can’t access your lead scoring data, so you optimize for clicks that don’t convert. Local listing data lives in a separate system, so top-performing locations can’t surface insights to underperformers. Performance data stays siloed in individual markets and never informs the broader strategy.

A graphic breaking down AI-powered lead gen frameworks.

The AI-powered lead gen framework has three layers:

Data Layer: Location data, CRM signals, and customer behavior. This is the foundation. If your data is fragmented or inconsistent, everything built on top of it will be, too.

Activation Layer: Ads, SEO, social, and local listings. These are your channels. The goal is to run them from a centralized playbook while adapting execution to each market’s demand signals.

Optimization Layer: AI testing, budget allocation, and personalization. This is where the system learns. It improves not just individual campaigns, but the entire operation simultaneously.

A graphic that breaks down the 3 layers that make AI work at scale.

The key distinction is centralized strategy with localized execution. Brand messaging, campaign frameworks, and budget guardrails are set at the top. Creative, offers, and targeting adapt to each market’s specific signals. AI models are trained on the full dataset, not just one region, so outputs are informed by what’s actually working across your entire footprint.

This is how you stop duplicating the same campaign across 50 markets and start building something that compounds. Scale doesn’t come from more campaigns. It comes from smarter systems,

AI and Local Search: Capturing High-Intent Demand at Scale

Your next customer isn’t searching for your brand name. They’re searching “near me.” And that intent matters enormously.

“Near me” searches carry some of the highest purchase intent in all of digital marketing. The problem is that most multi-location brands lose those searches before they ever have a chance to convert. The culprits are predictable: inconsistent Google Business Profiles, weak local SEO signals, and no coherent review strategy.

NP Digital’s research found that 59 percent of multi-location businesses are not tracking their Map Pack visibility at all. You can’t optimize what you don’t measure, and you can’t win local search if you’re not paying attention to it.

A graphic showing how often map pack visibility is tracked.

AI addresses each of these gaps directly.

Automated listing optimization keeps your business information accurate and consistent across every platform and every location simultaneously. Name, address, and phone number (NAP) inconsistency is one of the most common reasons brands lose local rankings. AI can audit and sync that data at a scale no manual process can match.

AI-generated localized content means each location gets landing pages, service descriptions, and posts that reflect its specific market, without requiring a dedicated content team for every region. Add schema markup so search engines and AI tools can surface your location data in map features and AI-generated answers.

Review sentiment analysis lets you monitor feedback across every location and flag negative trends early, before they compound into a visibility or reputation problem.

A breakdown of AI opportunities in listing, localized content, and review sentiment.

The metrics that matter at the location level: local visibility share, calls and direction requests, and location-level conversion rates. Track these per location, not just in aggregate, and the gaps in your strategy become obvious fast.

Scaling Paid Media Across Locations Without Wasting Budget

Manually managing paid ads across 100+ locations is where growth breaks.

Budget gets spread evenly across markets regardless of demand. Creative runs until someone manually pulls it. Performance gets reviewed monthly, by which point underperforming campaigns have already wasted weeks of spend. No one is learning what actually works in each market, because the data stays local.

AI fixes all three. Here’s how it works in practice:

Performance Max runs across Search, Display, YouTube, Maps, and Discovery from a single campaign structure. Rather than building separate campaigns for each location, you set the inputs and let AI distribute across channels based on where demand is showing up.

Dynamic creative optimization means AI is testing headline, image, and call-to-action combinations by market automatically. Creative adapts to what resonates locally, rather than running a single approved version everywhere.

Demand-based budget reallocation is the biggest unlock. NP Digital’s research shows that only seven percent of multi-location businesses use AI or automation to guide budget allocation. The majority allocate manually or based on historical performance. That means most brands are treating their best markets the same as their worst ones.

AI shifts spend toward the locations showing real-time opportunity signals. Same total budget, redistributed by what’s actually working right now. The result: the same dollar goes further because it’s going where it’s most likely to convert.

A graphic showing changes in budgeting before and after AI.

For more on building a paid strategy that generates more leads without inflating spend, this post breaks down the fundamentals.

Personalization Across Markets: Why One Message Doesn’t Fit All

Customers in Phoenix don’t behave like customers in New York. Generic messaging across locations produces low engagement and lower conversion rates.

NP Digital’s Personalization Maturity by Location data tells the story: 62 percent of multi-location brands are still “mostly standardized” in how they reach customers across markets. Only three percent are fully customized per location. The gap between standardized and partially customized is where most of the conversion lift is hiding.

A bar graph showing the local personalization maturity gap.

AI enables three things that manual personalization can’t deliver at scale:

Location-based messaging adjusts the content, offers, and tone of your campaigns based on where a user is and what that market’s demand signals look like. A promotion that converts in one region might be irrelevant in another. AI can surface those distinctions without a marketer manually monitoring every market.

Behavioral personalization goes further. Rather than one-size-fits-all follow-up sequences, AI can trigger personalized responses based on how a specific lead has interacted with your content. The follow-up feels timely and relevant because it is.

Localized ad creative adapts headlines, images, and calls-to-action by market automatically. What works in a competitive urban market is often different from what converts in a suburban or rural one.

Each location also needs its own landing page with unique copy, local reviews, and the specific services offered there. Region-specific pages aren’t just an SEO play. They’re what closes the gap between click and conversion.

Relevance drives conversion. AI delivers relevance at scale.

Lead Quality Over Lead Volume: What AI Actually Optimizes For

More leads does not mean more revenue, especially across locations where quality varies wildly by region.

The metric most multi-location teams are missing is lead-to-close rate by location. It tells you which markets actually convert customers, not just which ones fill the top of the funnel. Without it, you’re optimizing for activity, not revenue.

NP Digital’s data shows that only 22 percent of companies can accurately track lead-to-close by location. Another 32 percent say they can’t do it at all. That means two-thirds of multi-location brands are flying blind on the metric that matters most for growth.

A pie chart showing the accuracy gap in lead-to-close reporting.

Three metrics separate volume from value:

Lead-to-close rate by location. Which markets are actually converting? This is the signal that tells you where to invest more and where to pull back.

Cost per qualified lead. Not cost per lead. Cost per lead that had a real chance of closing. The difference often reveals which channels are generating noise and which are generating pipeline.

Pipeline contribution. Which locations, channels, and campaigns are directly tied to revenue? This is the number that justifies more investment, and the one most teams can’t answer accurately.

AI addresses each of these through lead scoring models that evaluate more variables per lead than any human team can process manually, smart routing that gets the right lead to the right team within minutes based on location, service type, and availability, and predictive conversion optimization that improves over time as the system learns which signals actually predict a close.

For teams looking to build better systems for nurturing leads once they enter the funnel, that post covers the mechanics in detail.

The 30-Day AI Lead Gen Rollout Plan

You don’t need a full transformation to start seeing results. A focused, four-week rollout can produce measurable pipeline impact, and it gives your team a framework to build on.

Week 1: Audit location data and identify top performers. Pull all location data into a single view: listings, lead volume, close rates, and ad performance. Flag any locations with inconsistent or outdated NAP data. Rank locations by revenue contribution, and identify your top 10 percent and bottom 10 percent. The gap between them is your opportunity map.

Specifically: go into your Google Business Profile dashboard and note which locations are incomplete, missing photos, or haven’t had a review responded to in more than 30 days. That list becomes your Week 2 priority.

A graphic showing key steps of Week 1 of an AI-lead gen transformation.

Week 2: Launch AI-driven campaigns and optimize listings. Launch Performance Max campaigns targeting your highest-opportunity locations first. At the same time, fully optimize Google Business Profiles across all locations, including photos, services, FAQs, and hours. Set up dynamic creative testing so ad variations can start adapting by market automatically. Fix the listing inconsistencies flagged in Week 1.

A graphic showing key steps of Week 2 of an AI-lead gen transformation.

Week 3: Implement personalization and start lead scoring. Deploy location-based messaging on your top landing pages. Set up AI lead scoring to prioritize high-intent leads over raw form fills. Build region-specific landing pages for your highest-traffic markets. Automate lead routing so every inbound lead reaches the right team within minutes, not hours.

A graphic showing key steps of Week 3 of an AI-lead gen transformation.

Week 4: Measure pipeline impact and reallocate budget. Pull lead-to-close rates by location and compare against your Week 1 baseline. Identify which campaigns and channels are driving qualified leads. Shift budget toward the markets and formats showing real pipeline contribution. Cut what isn’t working.

Small AI implementations compound quickly. The goal of this rollout isn’t to solve everything at once. It’s to build a feedback loop that makes your system smarter every week.

For teams that want to layer in automation across the nurturing side of the funnel, lead nurture automation is worth reading before you get into Week 3.

A graphic showing key steps of Week 4 of an AI-lead gen transformation.

FAQs

How to use AI for lead generation?

Start with the data layer: consolidate your location data, CRM signals, and customer behavior into a unified view. From there, activate AI across your paid campaigns, local listings, and content. Use the optimization layer, AI testing, budget reallocation, and personalization, to improve performance across all channels simultaneously rather than one at a time.

How does AI lead generation work?

AI lead generation uses machine learning to identify high-intent prospects, score and route leads based on conversion likelihood, personalize outreach by market, and reallocate budget toward the channels and locations showing the best performance in real time. The key is building a system where these tools share data, rather than operating in separate silos.

How can AI agents boost lead generation and sales?

AI agents can handle the repetitive, data-intensive work that slows human teams down: monitoring listing consistency, running creative tests across hundreds of markets, scoring inbound leads, and routing them to the right sales rep within minutes. That speed and precision at scale is what produces conversion lift.

Conclusion

The brands that win won’t just generate more leads. They’ll generate better ones, faster, and across every market they serve.

Multi-location complexity is only going to grow. New locations, new markets, more channels, more data. The gap between brands that build AI systems now and those that wait will widen quickly. The difference between a system that scales and one that fragments under pressure isn’t budget; it’s infrastructure.

Start with the audit. Build the connective tissue between your data, activation, and optimization layers. And measure at the location level, because that’s where the real signal lives.

If you want support building out that system, NP Digital’s consulting team works with multi-location brands on exactly this. If you want deeper insights on this topic, check out the full webinar as well.

Read more at Read More