profile

Smart SEO Newsletter

Technical SEO for LLMs?

Published 11 days ago • 1 min read

Someone has recently asked me about the most important technical SEO considerations for AI visibility.

So I thought I'd share them with you.

Here you go 👇🏻

1. Allow AI Crawlers in your robots.txt

Make sure your robots.txt does not block major AI bots:

  • GPTBot (ChatGPT / OpenAI)
  • ClaudeBot (Claude AI / Anthropic)
  • PerplexityBot (Perplexity AI)
  • Bingbot (used by Microsoft Copilot)
  • Googlebot + Google-Extended (used by Google Gemini and AI Overviews)

💡 But if you want to block AI crawlers, robots.txt might not be enough, as some AI bots just straightly ignore it and crawl your website anyway.

That's why in this case it would be better to use a CDN like Cloudflare to block AI bots from visiting your website.

2. Use HTML over JavaScript

Yes, Google has learned to understand JavaScript much better than it used to, but AI bots have not.

So even though correctly implemented JavaScript doesn't negatively impact your rankings on Google, it can hurt your visibility in LLMs.

See, AI bots don't really render your pages. So your best approach is to rely on HTML and use JavaScript for functionality rather than content.

💡 If you want to learn JavaScript SEO, there's a whole module dedicated to it in the Tech SEO Pro course.

3. Fast websites win

Remember when speed became a ranking factor in Google?

Good news:

Well, all our (hard-earned) SEO efforts in making websites much faster will also translate into more visibility in LLMs.

4. Structured data (-ish)

Structured data indirectly helps LLMs understand relations between entities.

Which means that it is great to have it, but it's not as important as many people think.

(This can change).

We talked a lot about it in one of our sessions of the SEO for LLMs course.

The best approach right now is 👇🏻

💡 Prioritize the schema that earns you rich results in Google.

Skip the deep-dive 1,000-types-of-schema rabbit hole unless you have unlimited time.

Ok, my friend, I hope it was helpful for you.

Till next time.

Smart SEO Newsletter

by Kristina Azarenko. Tech SEO Pro

I'll help you trade an imposter syndrome for a technical SEO superpower. My mission is to break down and simplify complex technical SEO things so that you can understand and use them to advance your SEO career.

Read more from Smart SEO Newsletter

Heads up, I'm doing a series of super valuable tips to help you improve SEO <> Dev communication (and grow your paycheck). If you're not interested, click here to opt out but stay on the list. In the previous email, we agreed on this: The results of your work depend on your SEO recommendations being implemented by devs. That’s why SEO + Devs should 🤝 These 4 steps will help you collaborate with developers much more effectively by speaking the same language with them. Let’s go 🙌 Step 1: Create...

about 1 month ago • 2 min read

As an SEO, you are always doing your best, even when it feels hard. That’s why it’s so painful when developers just ignore your SEO recommendations. It’s like they don’t even care. 😭 And it hurts even more when you realize that the results of your work depend on your SEO recommendations being implemented by devs. (So your performance is basically judged based on the things that are out of your control.) I’m sorry, my friend. It does feel hard to speak the same language with developers. But...

about 1 month ago • 1 min read

Story time 🥁 (plus practical tips on how you can recover YOUR traffic from LLMs too) First, why was my traffic 'lost'?Well, I noticed that ChatGPT would sometimes link to my Tech SEO Pro course with the wrong URL:It was sending traffic to techseopro(.)cominstead of techseo(.)proThe wrong domain just returned an error, as it was never set up. BUT! I actually owned it from the beginning for marketing purposes (you gotta be proactive when building big things 😉)So I set up a redirect from...

2 months ago • 1 min read