The Truth About AI Search Visibility: Dan Petrovic Sits Down with James Dooley & Jabez Reuben (James Dooley Interviews Dan M. Jones)

James Dooley: Yeah, AI search visibility myth busting. Today I'm joined with Jabz Rubin and Dan Petravich. And I'm excited for this one because there's a lot of misinformation that gets shared online and no better people to share this stage with than Dan Pete and Jabz. So Jabz, just quickly introduce yourself for anyone that doesn't know who you are. Um and Dan likewise um afterwards if you can introduce yourself and then we'll get into some myth busting exercises.

Jabz Rubin: Hey guys, my name is Jabis Rubin. I'm based out of Chiang Mai, Thailand. I run an agency called the blueprints. I run a series of events related to LLM visibility called LM Mastery. That's about it for about me.

Dan M. Jones: And um I'm Dan. Um I'm calling from Australia. I run an agency called Dejan. Um, I do SEO.

James Dooley: So, Dan, I want to throw you straight into the deep end. What's the number one myth at present that is annoying you that's being shared maybe on social media or online that you're thinking isn't the case?

Dan M. Jones: I'm going to go and say I'm not really annoyed anymore. I've sort of resigned and I'll just like I it's a resignation uh an acceptance that uh I'll just let them I don't have to correct everybody. Let them have it. This is fine with me, you know. I'll do my thing and then everyone can be wrong. It's not my job to correct everybody. Um it what's funny actually is that people who are wrong will jump on my content and criticize it while being wrong. I don't know if that's their own bubble or Dunning Krueger or it's like it's to me it's like a peak irony of uh of everything. Um but you know I try not to take it too uh too seriously and definitely try not to take it as my personal duty to go around the social media and the internet and just correct everyone all the time because I wouldn't be able to do any work. Um, but let's let's let's let's pick, you know.

James Dooley: Yeah. Just out of just out of interest, but as we start breaking some of the ones down and we start saying, well, this this isn't the case and this isn't the case. Why do you think it is that some do you think it is just that some of them are keyboard warriors and they just love to have an argument or do you think it's a lack of testing? Like why and I'm asking you both this question like why do you think it is that they come out so 100% confident in something that just doesn't work?

Dan M. Jones: It's it's puzzling because all you have to do is actually we have the knowledge at our fingertips and you can go and literally ask your LLM the answer and learn. It's so easy to learn. Um there's an element of uh potentially learning wrong things because we've fed the LLM and it's learned from us and so the wrong things that we taught it it's going to mirror back at us. So that that's a problem. But uh the thing is you can just look things up and why people don't do that before they go and start making assumptions and claims is is is beyond me. I I just don't understand that. Um and you know that probably like let's go top level. um people uh made up the industry um of of uh geo and um they disassociated with SEO and they're like oh no SEO that's like a thing of the past and is like this new shiny thing.

James Dooley: Just because you attach a name to something doesn't mean that you know what you're doing or how how things work mechanically. Um, and probably the biggest misconception that I think is harming most people is not understanding that models, large language models that drive agents, that drive chat, that drive AI overviews, that drive pretty much anything AIdriven today, the model itself is a static intelligence frozen in time, trained and released on a particular date with its knowledge cut off at a certain time in the past. You run data through it and output comes out. There's no updating, there's no learning, there's no the model is always static. And to release new models, it you have you're looking at uh 6 months or 12 month periods, right? It's kind of like we're going back 20 years when Google used to do Google dance and we we used to wait for for their updates, right? And and so that's the static model that is just intelligent enough to handle things but is not caught up on anything real time and then and doesn't have access to any real world knowledge in real time. That's what search engine results do. This lovely little ding we just got as soon as I said that. Perhaps perhaps I I'd park at this thought.

Jabz Rubin: That was good timing. So what what's your thoughts on it Jabz to start and then obviously we'll get into certain claims that people have done like what do you think it is just a lack of intelligence or do you think it is like Dan made a good point there that you can just ask the LLM but the issue is some of these people that ask the LLM they do when they take a screenshot but it's because of their history that they've tried to argue with AI that much that it's become an agreement engine with it that then they believe it to be true because now their AI model is saying that it's true because they've told it to be true. Yeah. Like my my take on this like to your first question of why there is so much of debate and people coming out so confidently is also because unlike SEO I mean go or a LM visibility is fairly new and everybody is in the race of being the next influencer in in the space and uh maybe they've done some campaigns that worked easily with with listicles and and maybe that's their claim to fame or that's what they want to you know back up every argument with and that's I think that's what I think it's just like it's everyone is just trying to be the next influencer in the space rather than a knowledge sharer of this is what we tested this is what we saw this is how things are breaking for us uh and things that we are still trying to figure out.

James Dooley: So they've got like a confirmation bias they've got a few bits of results and then that's it then they think There is that's the same for for everything.

Jabz Rubin: That's the same for for everything.

James Dooley: Dan, let's go into one or two myths. Um like what what is there that's out there that I'm not I'm not saying it frustrates you or anything like that, but what is it that's out there that you seeing consistently being shared and reshared and reshared that you just believe is just point blank incorrect?

Dan M. Jones: Yeah, I think um there's this notion that there are um LLM visibility factors that generalize and perhaps there are but there are wildly different uh between different platforms. So let's go back to the the core of my point earlier is that there's a static file a large language model which is hosted on a data center that takes multimodal input and does the output. That's the thing right that that does does something intelligent on top of search results. Um now that model is surrounded by its harness its tools and one of the tools is search. You can also look up your local files and things and there's elements of personalization and so but uh claude will make different choices anthropic open will make different choices engineering software choices and Google will make different choices deepseek makes another set of choices and they're all different platforms and so one thing that people say is like oh LLM LLM crawled my website LLM did not crawl your website the I guess that's the simplification of of the process. LLM doesn't go and fetch anything. Its tools do. So whatever so we in we are SEOs we study the tools that retrieve information and supply the information back to the user. What's different now is that the tools that used to serve the information to the users the 10 blue links and all sorts of things like site search and whatever now they're serving those results to AI as well. And so the AI um acts as the mediator between the raw search results and the user and takes care of the drudgery of um information foraging decision making analysis makes up its mind about a certain thing that's been supplied to it through search and gives a simplified answer and recommendations to the user. That's the interesting part. The the psychology of the model. Why did it recommend this thing over the other thing? All the ranking factors and influences, that's search still interesting things happening now. There's Bing, there's Google, there's new search engines popping up. Brave is relevant. Um, we've got we've got Deepseek just announced um or not announced but like secretly kind of um they have a backend essentially a search engine um with its own ranking factors and domain authority scores and everything. There's never been a better time to be an SEO. Um, and so we can't diminish what we do in search. Uh, because search drives 90% of the results in LLM visibility because for for the reasons that I said earlier, LLMs don't crawl, LLMs don't update, they don't customize, it's all context, it's all input. Um and so the work that needs to be done ma mainly is in um findability, crawling, indexing, ranking which is the area this comfortable playground but the rest where people should be spending time analyzing the the models choices the attitudes why it makes different I I see very few people actually investigating that area. Um, and and a lot of people are sort of like uh treating these LLMs as these mythical gods of, you know, like just try try stuff at random and and hope for the best. But we don't have to do that anymore. We can probe them kind of like what psychologists do with humans now.

James Dooley: So then what's your take on consensus? Is it is it a ranking factor as a lot of AI experts say that it consensus is the biggest ranking factor?

Dan M. Jones: Indirectly? Um consens I think the the number one let's let's try to generalize something that works across all of them. Um, you will notice that the loudest brands and the most mentioned, the most self-promotional brands tend to do well. And that's good thing for marketing because that's what marketing does. Um, I'm I'm a weird mix because I like marketing, but no, I hate the shameless self-promotion and bragging. Um, so I wouldn't do well. And so the consensus is like the side effect that um results from a lot of marketing activity in general. Now it doesn't have to be like that. It could be like a true organic something is just a great product or a service and everyone's raving about it. But those are unicorns. Those are very rare. Um when you when you go and research your clients competitors and pro it always boils down to yeah it looks like consensus but when you when you drill into it it's just a lot of a lot of mentions of them everywhere. They got into training data. They got into search. They're everywhere. There's reviews there. There's press releases. So if you want to brute force it and not be too smart about it, I think um I think just throw a lot of money at making a lot of noise everywhere. I know I'm simplifying it in terrible ways and it's opposite of what I do. Um, but that's what I think of consensus just being everywhere being overly present, but I wouldn't put too much smarts and intelligence on on these like GPT models are not going to know brand authority and all these things. That's fantasy. Those things, the brand authority and everything, all the traditional huristics and and algorithms and ranking signals, they happen in the the processing pipeline in the back end. things that deliver readymade results to the LLM.

James Dooley: Um, that's where that census comes in.

Dan M. Jones: That's where that census comes in.

James Dooley: How important, you've mentioned there a few times about the initial training data and maybe it only gets updated every six months and it's static. How important do you think the training data is um in comparison to live retrieval? Obviously, majority of searches they're looking for live retrieval. They're looking for for the SEO results whether it's from Bing or whether it's from Google. Do you think that the training data plays any part of the reasoning once they've done the live retrieval that they're using in the training data to then give the result of okay we've got 10 entities here and they then cross referencing it with the training data.

Dan M. Jones: Um it's so two two interesting things about that when GPT and Gemini or B and um when they all started right and even anthropics early versions of Claude Um when they all started they were using the garbage of the internet slop just everything the humanity the totality of human just output all the sewer pipeline right and they cleaned it up they pre-processed that you can't just train a model on raw data you have to pre-process everything there's a lot that goes in there we could talk about it but um the thing is they don't need to do that anymore speak all the language languages pretty much even the the little obscure dialects. So the models have learned how to speak a reason and they the big platforms no longer rely on scraping the web for teaching their models the basics. What's happening now is they're going through reinforcement learning through human feedback. They're going through various layers of fine-tuning. Um, and they're using finely curated expert level content generated training data. Not just content, training data for reasoning, for logic, for common sense, for coding, specialized training. There's huge industry and companies raking in huge amounts of money preparing nice clean data sets to train these models. This is not coming from the internet anymore. That's bad news for those who are trying to brute force their way into LLM's memory, parametric memory through training data because you're no longer guaranteed unless you're dealing with a small lab like you know small lab maybe Kimmy and Deepseek and this they're still probably using the raw web stuff plus some curated stuff but I would I would guess Google Anthropic and OpenAI are no longer relying on the raw data anymore. So they're they're working with finely curated data sets and so that makes it really difficult to get into the training data through your own little efforts.

James Dooley: Here's a question for you. You just mentioned that um the RLHF uh reinforcement learning human feedback. How important do you think that is going to become? because I know quite a few different brands and a few different people that are using the human feedback as part of reinforcement learning and it is changing the results of let's say things like LLM's like chat GPT if you go and do it's almost like see how CTI used to work with Google or still does work with Google and the pogo sticking and the clicking on if there was enough reinforcement learning based on human feedback and you were citing specific sources it does change what chatb says not just for your own search but worldwide for different people. How important do you think this will become the human feedback within reinforcement learning?

Dan M. Jones: But it's not human feedback from our interactions with them. It's from expert level curators and the teams that are in charge of that. Actually the whole process of reinforcement learning is actually incredibly short. you most of your data is in the pre-training and then alignment fine-tuning and all that. Um reinforcement learning is like relatively small part that comes at the end or towards the end of the model being ready and released and it has nothing to do with our chat sessions and things. If if our chat sessions get in there, it'll be highly processed and refined and fine-tuned um for the model to be trained on. um they will I guarantee you they will not use our direct interactions to to guide the development of these models. It would be a disaster. Um so that's you know all the personalizations you see that's just input to the model through your own little text files that are saved somewhere in the database of you know your custom instructions and history and probably a bit more sophisticated with Google. Um, one thing I forgot to say is that there's there's an increasing trend of synthetic data driving the model development now. And so that means that models teach models, model distillation, synthetic data. So like for example models like Gemma or uh Gemini um let's say Gemini 3.8 um flash and flash light, those are just distilled or more quantized smaller architectures based on the larger models. Um and they're done through both distillations and sometimes uh with synthetic training data um which is really interesting. That's another case against you can't just scrape the web and use it. They they actually now using I actually just finished a synthetic data generation. I have um close to 40,000 um uh simulated chat sessions between an LLM and a user um just churned out on my GPU while I was sleeping. Um and about 10% of that is product recommendations and product seeking um chat sessions which is which is okay perfect segue. Let's let's talk about this prompt tracking. you just you have one prompt and then you get the results and you pretend like yeah that that that's meaningful right but it's not multi-turn conversations product recommendations can happen mid chat and then chat continues and that that whole context changes the changes the whole thing right um so that's I I'll I'll just pause there because I'm starting a new topic.

James Dooley: No let's talk about it because like let's say someone let's say someone wants to increase AI search visibility and obviously the amount of myths that's being shared out there and people that are tracking prompts and you can't just go tracking individual prompts and stuff like that. But what would you recommend for for a a business owner? Um they've got whether it's an e-commerce website or they're a local plumber. How should they be tracking their AI search visibility? Like we should be loading in 100 prompts and just doing like a share of voice kind of setup. How would you recommend for them to be tracking search visibility?

Dan M. Jones: Well, let's say let's say you're a let's stick to plumbers. Um let's say um um emergency plumber Brisbane. That's what you are. Bit of a niche. That's your thing. Um got lots of competitors. You know, your territory. Um but that's what you do. And so that's that's probably one of your keywords that's in your title and your description and your H1 and your homepage. Um okay. So let's let's I name that I call it an entity. I could call it many other things but I call it the primary entity that the business wants to be visible for. Now let's let's have a magic wand. I hit that term with a magic wand and it expands into a a plet of different query variants. 24hour um plumber Brisbane uh emergency plumbers Brisbane best emergency plumbers Brisbane. You can see how queries can fan out into a number of different variants that ultimately have the same nuclear meaning. The core is there. The kernel of the meaning is there. So query intent. Um so what I like to do is I like to canonicle all those queries to one thing and probe that. Why? Because models are exceptionally good at um summarizing. Um, and so each one of those sub queries that fan fans out from your primary thing that you want to vis visible for can then be turned into prompt. So it's kind of like reverse prompting. Instead of having a prompt that gets to fan out queries and fan out queries go to um search results and brought back to the model, you can go the opposite. um you you can put your website in an LLM and say condense it and then again and then again and again and you and you get to the core the kernel of what your website is about the page is about um and then I track I track that and so the reason for that is that you could have hundred different prompts infinity prompts which ultimately say the same thing that lead to the same intent. Um, and so why are you tracking um, let's say why you tracking 100 different prompts that all mean I'm looking for an emergency plumber in Brisbane? Why use 100 prompts for that when it's literally the same thing? And and if you wanted a thousand prompts, a model could give you the thousand prompts. So what are we doing? We're creating non-cononical entities to be tracking with. And so that's my logic there. And so when I'm what I'm doing is I'm I'm looking for structured and quantized recommendations um and I'm saying customer is looking for um emergency uh plumber in Brisbane make your recommendations and then the model does its own fan outs and things and it keeps coming up and I do it in a structured way and some people say well there's no rankings in the generated responses. Yes, there are. Text reads left to right. So if there's a one sentence and three brands mentioned in that sentence there's an order of things that the way the language flows. So the first recommendation left to right second and at the end of the sentence there's a third brand mentioned that's your ordinal values. So I split those up and I say number one number two number three number four and so on. Sometimes the model will recommend three brands sometimes 12. It's no longer 10. Right? So what I do, I I probe that daily, weekly, whatever. And then I monitor presence or absence of the of my brand, the one that I'm tracking, whether it was mentioned and when it was mentioned, what was its ordinal value in a sentence or degenerative response? Um, and so that's two things being uh I'm tracking the presence, I'm tracking the mentions, brand mention um and I'm tracking the grounding, which is the step step behind. And there's another thing where that gets a lot of confusion. You could be a grounded candidate. What that means is you were in search results for that query. Then grounded candidates are brought to the pre-processing pipeline, narrowed down, and that's how you have you end up with six grounding sources. From each of those six grounding sources, you do extractive summarization by picking out the semantically most relevant u chunks of the original page with respect to the fan out query, not the prompt fan out query. And then you so you end up you end up with a with a grounding snippets with Google and each page um that's brought as a grounding source only has a representative sample from it um speaking to it speaking to the model uh for that page. And so um we've got the grounding supports, we've got the grounding selected, we've got um the uh ordinal value presence and absence and that's already enough to measure. Now if you bring other brands in it, you can do interesting things with it like um share of voice for example. If there are 10 brands and you're one of them and you're appearing there, so that's that's you you you got the 10% share, but over all the prompting cycles that you did, you might end up being 16% or 3% depending on how much you're being squeezed out by the competition. Um, so that there's ways to be tracking that, but I definitely do long form uh long form prompt tracking. What I do is I do qualitative analysis on long form prompts to understand what types of things come up in citations. But each prompt that I generate I I always generate it when tagged with the original core entity the kernel of meaning. So when I do the massive amount of prompts each one is attributed to that main thing that that business wants to be visited uh visible for.

James Dooley: Just some make sense. Yeah. So does that mean if there are three prompts taking your emergency plumber uh example so there's emergency plumbers Melbourne 24/7 emergency plumbers Melbourne and 247 emergency plumbers near me do you condense all three into one mega prompt like who are the best 24/7 emergency plumbers near me in this location is that what you recommend.

Dan M. Jones: Near me is a different case because um we're looking at um well depending on which platform we're talking about. If we're talking about Google chat GPT will use its own sources Google will will use directly maps API and will feed that data and it'll look at the reviews. So now suddenly we throw away all the other things and we look at um the the traditional local SEO optimization signals including the reviews and and everything else the uh the citations and um so that's something that becomes um relevant. So for for tracking what I do is I do hierarchical um hierarchical layouts uh for my entities. So I would have emergency plumber and then I would have uh Brisbane, Sydney, Melbourne if I'm a bigger pl uh plumber thing, but I could also invert that and I could have Brisbane and then emergency plumbing, specialized plumbing, this you know different types of services, um hot water system and nested. So that's a design choice now more of the same more of the same thing. um what I've implemented recently um what I do is I um use firecrawl I get a homepage and I follow the links from the homepage the next level. So I goes level zero to level one and I and I make note of all the links that go from the p from the level two uh pages the pages link from the homepage directly and I found that that already gives me enough scope. So I scrape using fire crawl. I scrape all the pages, homepage and everything else. And then I task an LLM for entity extraction, right? And it's not just all entities, it's um I could do all entities, but um I ask for entities of commercial interest and ask them to be canonicalized and ask them to be hierarchical. So I end up with a uh a JSON LD file that purely um serves to be like a structured schema for all the entities in a hierarchical fashion which which solves that. You know what if you have different things that's kind of like is it a core entity or is it an offshoot? Of course it's an offshoot. You you could have a more granular entity that adds to and describes better or in more detail the parent entity. And so um I actually call it Matrioska entity attribution and I also have the Matrioska um brand attribution because during brand um so this is another thing that's important for brand visibility tracking is because um you you could be um mentioned as your core brand or brand plus+ and you know you could have multiple brand variations and even misspellings. So you want to make sure that if you have a short form brand and a longer form brands that you attribute every brand variations to the brand mentions and not like track just one brand variant because you could miss out on a lot of um instances where model mentions you but mentions you by another name right or a variation or by your domain name. Um so what let's let's boil it down hierarchy.

James Dooley: Hierarchy of entities to be to be tracked that solves the problem of well is it uh location related or is it a subservice and that type of stuff.

Dan M. Jones: Do you ever do it by so we do something similar to that but we do it on the main attributes and what we try to do is if we know 10 of our competitors and we know that let's say we can't win on price. So, let's let's go back to we may as well keep stick to the emergency plumber. So, emergency emergency plumber um we can't win on price. However, we can win on speed. we'll go to town on the attribute of speed of of how fast we can get out there and double down on that to try and win in that lane which then means that it might allow us to get into whether it's chat GBTR or Google AI overview to start win on that to then before then we start doing any let's say comparison articles and stuff like that alongside another entity we'll do it on normally the attributes of our entity as opposed to doing it against different entities does that make sense. Yeah. Uh, essentially it's all a graph, isn't it?

James Dooley: Yeah.

Dan M. Jones: Um, I know that uh Dixon uses um entity graphs and um um word um what is the um Andrea Vulpini?

James Dooley: Yeah, word lift.

Dan M. Jones: Word brain freeze. Yeah, they they are heavy on graphing and entities and um so that that only makes sense because some some things like like a link graph pages link to different pages and you could have uh you could have a single page with multiple entities on it or you could have a multip you could have a a single entity mapping to different pages. You shouldn't technically what I like to do is I like to canonicalize. So one entity maps to only one page otherwise your um entity cannibalizing let's call it.

James Dooley: Yeah.

Dan M. Jones: Um so that's uh but sometimes graphs graphs are the only meaningful way to do things.

James Dooley: Yeah.

Dan M. Jones: Um and so yeah that approach is perfectly valid.

James Dooley: Yes. Go on. Sorry J. Good. This may sound very basic for you but let's say keep getting back to this whole example of plumbers and what James also sharing. If there's a new plum plumbing brand and they want to now start their LLM visibility campaign and there are already established plumbers or plumbing plumbers plumbing brands that are really good with their uh reviews and the consensus and they already recommended really well. Now where should these like new brands start? How what should be the first five steps for them?

Dan M. Jones: That's a good um good one too. Let's let's be practical. Let's be practical. I I'll pick I'll pick a local um plumbing brand um and but because I've formalized my process in my thinking in and and the whole workflow. So, let me see if the screen share will work on this. So, I'll just do the screen share. Um I don't know these guys. I don't know who they are, if they're any good or whatever. So, I'm not advertising this particular brand. Um,

James Dooley: I thought I was looking in the top left then and I thought you'd created a brand called Dejam Plumbing.

Dan M. Jones: Jean Plumbing. Everyone sign up emergency services. Okay, so there's a couple of things happening here. I'm identifying the brands, finding competitors, inferring locations, detecting detecting language, finding topics, finding search queries, and checking what AI already knows about you. So there's your steps, right? Um, and so yeah, I'm using a tool, but you don't have to. You can just you can do your own research or vip code your own process. Um, the what's what's important here is the backbone of what's happening. So this is my quick review. So yeah, locations, Australia, English. So that's so this is my you know, some people just call it, but arguably you could have another brand variant or like that single word. So I'm going to add that in or all fix plumbing. That's another variant, right? And that's like to but this is like the main the main thing. And so these are the competitors. This is super easy to come up.

James Dooley: Have you just added them in or as your tool then gone and found their main competitors?

Dan M. Jones: So, we just ask we just ask um an LLM, what are the what are the brands you associate with the things that this brand does? Yeah, we just got the answer. Super simple.

James Dooley: Um so, uh I'm I'm not going to do manual cherry picking here. I'll just select all here.

Dan M. Jones: Um I'll just do like a daily tracking here and I'll just do Google. Um and set my location to Australia. Okay. So, a lot of stuff is happening in the in the background. Um so basically um we're doing the visibility. Okay. Well, let me just show you the what's happening. So there's my entities. Can you see how it's nested? So we've got a child parent. So blocked toilet repair, drain cleaning, plumbing, um gas, salt water systems, etc., etc. So they effectively form a graph. So there's your entity graph. There's your plumbing and all the things that are, you know, associated with with plumbing. And so that's that's your first you know based on that first initial visit to the to the site. Um also when you when you do that or you as a byproduct you have a link graph as well. So you have which pages linked to which pages and what entities are those pages associated with. If you wanted to discover more more entities what I what I tend to do is I do a a custom scrape of the site homepage. And you know, again, this is this you could do screaming frog um and and do this. So my entity curator just takes a little bit of uh work away from me. Um uh scrapes and crawls all the pages and adds any new captured entities uh puts them in an entity graph. And but here's the important part. In the background, I'm generating prompts. I'm generating synthetic queries. I'm looking up the visibility for for the competitors. I'm looking the visibility from for myself. I'm checking the citations. So all the data all the data is basically I'll just close this so we can have a look at the rest. Um let's take a look at the brands. Um so this is my brands and how they nest together. I disagree with Allix Plumbing being the primary, but I think most people will refer to my brand as um Allix, right? So, I'm going to put that as my primary. Oh, not not the competitor. Uh my own primary brand. Um and continue this. But we also have the competitors, right? And this is the full list. And each competitor will have their own strengths and weaknesses and so on and so on. So what we do first step for me is I like to understand what entities do you Gemini u anthropic uh and GPT what entities do you associate with my brand and that's this is the actual answer so basically we're surveying the models block drains this brand was mentioned five times by three models and two two two different brand variants then I go in reverse so I so this is this is brand to entity meaning I ask I literally ask like five rounds of asking I say for allix plumbing what are the things you associate with this brand and it just gives me so that's out of the model's head head parametric memory probing right um and then I flip that around and I say for the things that you just said that my brand does what are the brands that you associate with the things that my brand does so that so that already like just that one flip already paints a clear picture of your competitive landscape. How good is that? And this data comes free. It's models knowledge already. So that's stored within models parametric memory and like I I spent a dollar, right?

James Dooley: It's not going to cost you much this type of research. So that you're already ahead in terms of your research. So association mapping is your first stage. And you could do the same thing with queries, right? You can say what queries you associate with this brand. Um and then the next step for me I do relevance mapping. It's a super simple any everyone can do this. I take my entity let's say um storm water drain clearing and I say if you saw all fix plumbing as one of the recommendations um for tap repair and replacement would you recommend it to the user? Yes. No. And I ask 10 times, 100 times. Basically surveying the model and I expresses it a confidence three times uh you know 10 out of 10 uh 9 out of 10 and so on and so on. So you get the overall score across all the platforms jointly and you can see like for example if um Anthropic is not very confident for all these and uh Google is not confident for these but is very confident for that one. So what? So this brand already knows Allix Plumbing, they already know that storm water uh drain clearing is solid with Google. But um kitchen plumbing, they're not so sure. And so what are we doing? We're just asking yet we we asking would you recommend this brand? Yes. No. And just counting counting the outputs. So what this does this is not your job all you know all everything sold. So you do association mapping as a step one. You build the network, right? The network of concepts. And then relevance probing tells you the strength of the connections between your brand and those concepts. And then the final step, the the real deal, you you allow search grounding to come in and you actually understand what are the actual brands um that are recommended um through real searches, real fan outs for all these platforms when when we um probe them for real. And for this we actually use the real prompts. So the real prompts will be we got 48. And what I do is I generate prompts by every platform. So I use Enthropic, OpenAI, Google. And so I can click.

Dan M. Jones: Yeah. Is this what people really type in? It's not.

James Dooley: How have you created those prompts? Have you asked the LLM to create that prompt for you?

Dan M. Jones: Remember what I said per entity?

James Dooley: Perity. Yeah. I said I said to I said to the model here's old plumbing what are the things that old plumbing does.

Dan M. Jones: Yeah. And then it told me then I allowed search and the model was allowed to visit the website with its search tool with its URL fetch tool. And then then it gave me some more answers and then we mapped each entity to a URL. Um and then we asked for for each entity that we've established as factual for each entity we said for this entity um kitchen plumbing give me a prompt or 10 or 100 right and so every so every prompt that I have is mapped to the entity so I can attribute uh when I'm when I'm trying to come up with my scores and things otherwise it's it's chaotic and and uh random but what I wanted to say is this prompt prompts. They're synthetic prompts. People are not going to type in that same exact prompt. And here's another myth. Prompt search volumes. There's no such thing.

James Dooley: Oh, come on.

Dan M. Jones: There's just no way that you two people will type the exact same thing twice, especially in a long form chat. So, um that's why I think the only honest way to approach this is to say there is no search prompt volumes. This is synthetic data. We are not when we're when we're prompting and measuring the outcomes. We are looking for direction. We're looking for it's a litmus test.

James Dooley: Yeah.

Dan M. Jones: It's not going to give you the the it's not search console that tells you the search volumes of queries and your rankings and so on. So there there has to be an element of honesty in our work that says we are beyond um having access to people's actual chat sessions. We don't have personalization. And some people will say, "Oh, we got um we've got um uh scrape." Oh, no. Like the what's it called? Um spyware in your extensions. And uh help me out. Uh Jabz, um click stream. Jesus, I'm getting it's tired. It's late in it's late in the day. Clickstream data. Um so um clickstream data yeah you get a small sample and then you do a lot of mathematics on top and then you pretend like you have real people's chat sessions. Can we really trust that? And my apologies if anyone's relying on click stream but you know yourself um if you are that that's a bit of a hocus pocus type thing isn't it? um a lot of data extrapolation.

James Dooley: On there when you had then the percentages you had enthropic Google and um open AI is the Google is that Gemini that you're using on for Google.

Dan M. Jones: I always use the latest so basically have a um this a great question because what do you do models keep updating and so it was Gemini 3.5 and then 3.6 six and then seven and now it's eight and now we've got the uh number Gemini 4 Argon. So what I do personally is I like to take the model that currently majority of users use. Argon is available to a small amount of subscribing customers but then the rest of it is Gemini 3.8 that drives just drives everything else. when I get evidence that Argon like Gemini 4 is the mainstream model, I'll switch my platform over to that and I'll be I'll be using that. Um otherwise it's a nightmare. You keep you know you keep fragmenting your data and your visibility and you're starting over every time a new model comes. So what I do now is I have um my graph of visibility and I have an annotation like a vertical purple bar that you on hover says model updated on this date from now on. So much like when a Google penalty comes in your Google like and sorry a core algorithm update comes along and you've got the lining to say has it gone up has it gone down there's been a core algorithm update here.

James Dooley: SEO people are uniquely qualified to deal with this.

Dan M. Jones: We've already like we know what to do.

James Dooley: Um and so uh we're we're perfect for the job um and so when I when somebody says um who looks after AI visibility.

Dan M. Jones: Who else?

James Dooley: Um and so when I when somebody says um who looks after AI visibility who else um and so uh we're uh we're perfect for the job um and so when I when somebody says um who looks after AI visibility um and with regards to just on there because I Love that what you've done there with the different kind of services that you offer for the brand and then you're comparing each large language model and then it's saying that 90% of the time it says you was recommended. When you're saying like would you recommend all fix plumbing for hot tap repairs and it said 90%. Is that literally a would you recommend hot uh all fix plumbing for hot tap repairs? Is that a yes or no answer from the LLM that comes back to say is that is literally how it's been done?

Dan M. Jones: Yeah. I I restrict the model from it's literally limited to a single token of output.

James Dooley: Yeah.

Dan M. Jones: It cannot say more than one token. It's

James Dooley: Yeah. Yeah. Okay.

Dan M. Jones: It cannot produce the next token. And the reason is like it could give you the whole analysis. They're great at slop. Just just give me a yes or no. And because the models are stochastic, you can't just ask it once.

James Dooley: Yeah.

Dan M. Jones: Right. And so the the models are complex things. They run in multi-dimensional space and it's all probabilities. They've got temperature parameters. They've got sampling um that the top K. And so they are allowed to they have freedoms in what they're going to say and they tend to vary their outputs. But what you'll find is like if you prompt them enough, you'll get enough yeses and nos to gain a pretty good um measure of the model's confidence. It's kind of like what you mentioned earlier about um consensus and how does that live in model's memory. If if the models learned enough about your brand and associated you through coitation with a certain concept um it's more likely to say yes um and less likely to to to say no. And so this is a way and by this doesn't mean that the model will actually say yes or no in the end. This is just you probing the model to see if there's a primary bias against or towards your brand in its pre-training in its parametric memory. So when you add search, if that preconception about your brand will uh go along with search and help the search or it'll bring it down cuz what I've seen when you when you um ground a model like Gemini will have like six grounding results and you are like second grounding result that means you're second in SEO and organic search but then when it comes to recommendations you're the sixth.

James Dooley: Yeah. For some reason, the model didn't recommend you second, which is the order in which the model doesn't recommend you in the order in which you've arrived in its context.

Dan M. Jones: But it's pushed you down. This is this is in context re-ranking. The model makes up its own mind without any algorithms, without anything. It's using its neural network and it's rearranging the the the ordered number of results that it's received from search in in based on its opinion. Now this is not only based on parametric memory because it is with then we have nothing to do. I did a I I did a ton of testing like uh billions of tokens of testing on this and I found that on page tweaks can skew the model one way or the other. It's beautiful and it's so familiar with what we do in SEO, right?

James Dooley: Yeah. So you could do testing and then change one p one one sentence of your page. Add so some freebies. They love uh FAQs. They love prices. You don't hide your prices. They love checklists and lists. Um technical specs um and tables.

Dan M. Jones: Um so those are some things that they are Yeah. Don't use emojis and don't use like overly emotional language or pushy stuff or like your own corporate jargon. That's devalued. So just use like those things that I just mentioned that are quite convincing. There's a stack of different things that I've discovered. But basically the good news is that you can do things on your page that so that search engine picks that up, generates the grounding snippet, grounding snippet gets into the model. That little thing that you put in there can convince the model and tip it in your favor. So you're the you're the last result in the grounding in search. If you have the good stuff in your text, you'll surface the first result. um based on the you you hit the sweet spot with a model.

James Dooley: Yeah.

Dan M. Jones: The the tricky thing is that Gemini likes one thing, anthropic likes the other thing, and GPT likes the other thing. They often agree on general things that help, but sometimes they can disagree. Um and so, yeah, plenty of work for for us to do um with optimization.

James Dooley: So much work. And I love the depth you've gone to here, right? So, I want to give you an example, right, of a of a I'm not saying it's a myth within for AEO or GEO or for AI search, but I hate the amount of people that are tracking just AI citations and how many times that they got cited. And the reason being is because we've worked with so many companies and they've gone they've been flying in the volume of citations within AI overview our chat GPT but they were just a nominee that was there but not in the envelope as being the winner. And what I mean by that is there was just one of several companies mentioned but it basically says you could use these these or these however you want to use these. And there wasn't the one that was recommended. there was just one part of a list and there's a massive difference and a majority people that I speak to don't talk about the difference between AI recommendations and AI citations and not just being cited but actually being recommended and that way you've got it there is dynamite because if you're not being recommended if if for some reason you score 10% as opposed to 90% you can then start seeing which brands or competitors are the ones that they are recommending and then almost then start asking side by side what attributes have these got that make them better to us that we could work on and it could be to do with cost like so they might not have listed the cost on on the page it could be related to accreditations awards uh testimonials case studies all these reputation things that you can build up but again you can just ask the LL as part of it you can build as part of five retrieval to to win. And if you are second on the list and for some reason it's part of the reasoning you drop to number six on the list, you can work on that long term to be working on the things of why as part of what you're saying there you're not being recommended. So it's that's gold. I've never seen that in any of the AI visibility trackers or systems. Have you ever seen that Jabz?

Jabz Rubin: No, I haven't. And I just wanted to get more uh info on on your process Dan. So you were showing that process where you first and I'm assuming that Allix is a fairly new brand and you checked with LMS whether you are going to recommend all fix plumbing for this category. Yes or no? Now this is I'm assuming based on the info published on all fix either reviews page or their own website. But when actual user comes and they start looking for who is the best plumber for XYZ issue, what do we do with the data that you have just extracted and figured out? What do we do with that data so that when users are actually looking for a particular service all fix gets recommended? What I tend to do, I concentrate on the weakest links that the sometimes you will get a low confidence score for an entity that's of no consequence. So we park those and not throw them away, but we park them as a low priority. But we in the in that process of uh birectional probing and the citation mining, we look for signals of something that's important to the brand that's low in the recommendations, mentions, citations, and everything else. Um it could be just bad SEO and so that's why it's it's not even in the mix for recommendation. So we try to um work in two stages on that one. If it's not doing well in SEO then we do page level optimization and we do two types of two types of work. One is um page modific strategic and methodical page optimization by modifying certain snippets of the page and seeing which one works better. And we use Gemini as a reranker or GPT or or anthropic as a reranker. just you're just asking the model which one is which one do you prefer. We keep going through that cycle. Um it's an agentic you don't want to do this manually and then we identify the bits and pieces that made the difference. So that's one process by by adding things or changing things. So the other process is the opposite of that. I call it ablation. And basically we have a snippet and you have that stubborn competitor at the top, right? And they have that ground grounding snippet that represents the page and say it's like 300 or 200 words. It could be like uh 10 different let's keep it simple, 10 different sentences picked out from different parts of the page, but seeing as more most relevant for that um fan out query. Um, and so what what you do is you remove random sentences from that mix and you see which one causes the biggest drop for that snippet when reranking again and again and again with Gemini. And then you build a heat map of importance of different segments of the page with respect to their ablation or removal. And one thing happened that really surprised me and it's really cool is I removed the sentence and then the rank increased. Oh, really?

Dan M. Jones: I did not expect that. Ah, right. Okay.

James Dooley: There was a negative ranking factor sentence right there in the snippet that when I removed it, the the the brand jumped up in the in the scoring.

Dan M. Jones: Do you know what it was or you're not allowed to say?

Jabz Rubin: Yes, it was a review. It was a uh it was a um it was a um star review, star rating of 4.2 and so it was a 4.2 and then bracket and then 92 and close bracket. That was it. So, we just removed that one thing, right? And I was like, where is the I didn't even know it was on a page. So I'm looking around and he was right there in the like a green call to action section and he had like you know four and four and a bit stars looks good on on screen you know it looks like it's good good business but once we remove that part um the rank went up.

James Dooley: What is it about.

Dan M. Jones: Yeah I've got I've got one as opposed to remove so when you remove something it's gone up I've never seen that before but one one thing that we added which is a nice little tip for some people that if they do some people don't like putting prices on the website because they say oh it's so subjective like what it depends on if you're doing roofing it depends on the area size the access what type of material that they want to use and so on. But when we put a a a line on underneath the costing saying that our brand will not be beat for a like for like quotation because we it's not using their installers. when we added that and we've done it now across multiple sites that has increased rankings and they'll seem to love that saying where it says we can guarantee we will not be beat on a like for like price um and that that's helped. So it's it's they're using that they're not just using like the semantic triple of an entity attribute value they're looking at the words around it as well and saying oh these must be best value because they guarantee that they can't be beat or something.

Jabz Rubin: Yeah. Yeah. Okay. So, this reminds me of course it's contextual. Um you what what I learned very soon after I released the method um is that uh I was too precise. I was going surgically removing single sentences and um uh people on LinkedIn were like why don't you cluster um individual sentences into semantic units and ablate them first before so to to go course level removal first and then on the on the semantic blocks um of the page that made the most impact then zone um concentrate on them and then a blade single sentence is to see if there's a specific part of that semantic chunk that really um makes the difference. Um, and so actually that end ended up saving money because you you loop through fewer it's combinatorial because if you have a if you have a page with 100 sentences that can get very expensive and I usually clip the analysis at say 20 bucks per analysis um because if you have like a really large website you can get it can get quite uh quite costly. Um, but the idea is that like you can you can score the chunks. And what I haven't done though is I haven't so they're all adjacent, right? I haven't got like I haven't tested preserve this and then something at the bottom or remove something, you know, they're not this. I don't know how to do that. Sounds complicated. Um, and it could it could yeah combinatorial explosion. Again, it could be just too many too many variations of u of removal. But uh having having things together removed or added definitely makes uh makes an impact and that was a little surprise that I didn't didn't um know about. Um but so one thing you learn from one industry does not carry away carry across to another industry. If you're selling um if you're selling uh phone covers, emoji might do well.

Dan M. Jones: Yeah. But if you're a lawyer, not you're not going to do well with emojis or a hardware store or a plumber. Definitely don't put emojis.

James Dooley: Dad, I've got a personal question for you. Um, previously you've like pretty much like semi-retired. You're a dad. Um, you do amazing things um, as a dad and we've spoke about being a dad entrepreneur previously and like we you came back into it and you seem hungrier than ever. Like I'm I'm I'm listening to you and I'm excited. hear me all the testing of what you're doing and stuff like that. Are are you loving it again? Like with you guys because so many people go on all SEOs dead and all the rest of it and it's like it feels like all everything of what we do as SEOs or what we should be doing as SEOs in testing and stuff like that. There's more testing stuff now than can be done than ever before. And the volume of testing that can be done across all the different large language models is different. So, we're not only trying to rank for Google, we've got Anthropic, we've got OpenAI, we've got Gemini. You seem hungry for it. Like, are are you back in the trenches and enjoying doing it again?

Dan M. Jones: Well, two things happened at the same time. Um, uh, LM's gave me wings, so I'm no longer dependent on a coder to develop. And so, it was like a dog off the leash type effect. all the pent up ideas, all the plans, all the drawings, everything went to production. I was ready for this. I was born for this, right? It's just that the implementation bottleneck was always there. So, as soon as I had access to a coding facility um that doesn't involve like a team of 30 people um that I could execute pretty much every idea that I had. So, that means creative people will flourish now because they can just if you have ideas, you can you can put them to an execution. Um, and that's that's fantastic news. Second thing that happened on a personal level, um, Marina is now almost 13. She's got her own thing, you know. She u goes to her room, listens to music. She doesn't need me hovering above her all the time. I like to give her her space. I'm not bothering her with my science experiments anymore and microscope and electronphilogram and liquid nitrogen and all the geeky things. Tesla coil. Um, so that, you know, I I'm giving her her space to develop into her own individual, I sort of backed away a little bit. Um, and letting her do her thing. Um, but you know, on the days that, um, you know, she comes home from school, I make time. You know, you can't be obsessed. There's always another day to to research and execute. But it's it's you have to admit we're having time of our life. This is renaissance of SEO. This is the renaissance of technology, golden age of humanity. It's the best time to be alive.

James Dooley: Yeah, for sure. Have you got any more questions, Jav?

Jabz Rubin: And we can ask Dan like for us and us. I mean, there's so much to to figure out and fix. Uh Dan, yeah, one one question that one point you kept mentioning earlier on about grounding. Uh can you just give a bit more clarity for people who maybe just starting off in the golm visibility space? What do you mean by brands that are in in the grounding? Does that mean the brands that showed up in the citations and in the fanouts they were in the grounding answer opposed to other brands that appeared and they were nowhere in the fan outs or in the citations? What do you mean when you say brands that were in the grounding?

Dan M. Jones: Oh, good. Um, yeah, that's that's where things get a little bit blurry for some people. Uh, perhaps I'll um um go back to our good old plumbing thing. I should I should send them a bill for this. Um Um So, so our little optimizer is churning some to We're at half a million tokens now. Um, and so you mentioned grounding, right? So the first step in the optimization process to establish the benchmark. You asked your question. You're after drain cleaning, right? Let me zoom in on this. You're after drain cleanings. You ask you ask a question. I'm looking for recommendations for for for drain cleaning. And then you extract the competitors for from from the grounding set. Right? So what is the grounding set? It's the set of search results supplied to the um uh LLM's harness or processing pipeline. And from each one of these ranking results, they are supplied a set of snippets. They look like this, right? So you don't get the full page. You get um segment and then ellipses and then segment and then ellipses. So this is specific to Google. Um other platforms do it differently. Um so you basically get little chunks of page that are snipped out of the page and stitched together with the dot dot dot. So this is your grounding context, right? So grounding grounding candidates or grounding sources is the set of search results that were supplied through search. That's just Google results. Um, and from each one we get a snippet and each snippet is supplied. So that's what Gemini then sees. For example, you get the.

James Dooley: How are you getting that snippet, Dan? Are you creating that yourself with a set of prompts or is is that being automatically pulled in from the network of of Gemini or whatever.

Dan M. Jones: So this one is this one is the organic search search grounding. We can use synthetic ones as well. We have a model that we train that simulates these, but these are the the current ones um are the actual um snippets retrieved from Google's uh back end. Um so when we ping when we ping their um API when the grounding results come, there's a little bit of trickery involved in extracting the full grounding set. Um the though I don't speak about it publicly because I don't want it to be ruined. Um it's nothing new. Other people's like Nazi from I think from Spain. Um he's using similar methods. He's asking he's asking the LLM to uh to return the start and end of each sentence being quoted. So basically you ask a large language model to give you its grounding sources. The way was uh the grounding snippets and it'll give it to you. The problem is due to recitation rules, the little classifier that sits on top of the LLM response will block that output. Gemini is willing to give it to you and it's happy to, but there's a little copyrighter copyright watchdog that says, "No, can't do that." It always stops the response. I have my own way to bypass that. Um other people um uh use the start and start and end of the sentence and then match it to the content scraped from the so this is the content scraped from each page right markdown version um from from each page and um that way you can see which part of the page is then used for for grounding. So um we've got our search results, we've got our content. The content um forms the grounding snippets and grounding snippets are brought to the model. These are these are the grounding um sources. The model then stitches its own uh the model creates our own output um basically the the generative read the recommendation and then attaches index of each one each one of these. So for example if this one is index one index two index 3 and so on and so on. If there are multiple searches it'll use 1.1 1.2 1.3 2.2 2.3 and so on. um if there are multiple searches in the in the call and so this becomes the comp the the grounding sources and once they are actually integrated in the model's response only then they become citations. So I'm I'm explaining this in a very roundabout way, but that's that's the thing. You can't you can't bypass the technicalities of this. You have to understand it. The search results ground the model. The model then picks which search result to in integrate and then those integrations are then referred to as the citations. Citations are of course not recommendations or mentions. Mention is when the model spells out your brand. You could be the citation but not the mention. You could be the mention but not from your own citation. You could be the mention from somebody else else's citation because not your website but somebody else's website was the was the grounding source. And uh so what happened here? So we we did a bunch of different testing. Uh so the dotted lines are open and anthropic and the the solid line is Google. And so we had all sorts of changes and this is the journey of the so we went from six order of preference all the way up to one and we stopped at the position uh at the uh at the step eight of optimization with some of the ideas already being tested all the hypothesis. And so what do we get? We get a content brief saying change this to that. Change this to that. Why are we saying those things? Because that's what we did. We changed this to that and this happened based on the ledger of ideas that the models I didn't we were chatting while this was happening right so the the hypothesis for what we could change on the page we generated in the background the models did the testing and they gave us the ranker ration explanation for each each step of the process this is the labs this is this doesn't do anything unless you implement it on your website so you supposed to go take this content brief implemented in your website and say okay I've done that I've implemented it on the um 14th of October click save benchmark window forward window backward and see if it's had an impact on your search rankings in Google um and.

James Dooley: Have you got this set up so it's fully a if you take over a client can you do this so it's fully agentic and let it rip as in like if it says to change it you just Oh, go and change it. You go and do what you think it needs to do. Almost like an inf an infinite loop of selfoptimization and self-healing.

Dan M. Jones: Absolutely not. We're not there yet. I I will not let AI do anything on its own.

James Dooley: I'm an absolute control freak when it comes to decision. Like nobody touches my clients websites. Um I was tempted to do some experiment like that on like self-organizing self-recursive let's call it um recursive improvement recursive optimization. Um I was tempted to do it on our own website but I didn't get around to uh do any marketing for our own website yet. Um maybe one day when I get a chance to do some maybe like a content marketing campaign or some I don't know. Um, I haven't had the time to think about it yet, but the point is all of this is then given to a human with a brain in common sense. To consider, do we really want to in integrate this. Sometimes model models will like this optimizer algorithm that I just showcased, it'll do anything to get to the top spot. It'll just put make things up. It's instructed not to, but it'll it'll put fake star ratings and and fake customer reviews. So, when that happens, um I say to my client, "Okay, here's what happened. The model just made something up. It it gave a customer testimonial that said this, this, and that. This brought you to the number one spot. Can I have one for real?" Yeah. Yeah.

Dan M. Jones: Um and so like this is just one example. So, we we use AI to streamline all the like the take care of the drudgery, the digging through data, corrections, testing, but um humans uh like human SEOs are on top of it. Everything served to them on a silver platter and then they have the luxury of time to think and make decisions. Um it'll be a long time until I decide because why? I put several agents in in a loop and I left them over the weekend. I checked to see what they're up to on the task and they've gone completely insane. They they invented their own language. They they've gone into their own hyperspace. And what the problem with the LLMs right now is they get they like to go on a tangent and they never pull back. A human researcher, a human SEO will explore explore the possibilities optimizations following like a tree branch and they'll try this, try that, try that and then pull go okay this is not working. Pull back, try this something else on a new branch. LLM will just keep drilling into one direction and never step away, never zoom out. Okay, what are we doing here? And so after maybe three days of leaving them alone and hundreds of dollars worth of tokens, they'll just have an absolute nonsense. Um, and this is the the risks of of leaving your agents run wild at the moment. Never do that.

Jabz Rubin: Dan, I'll tell I'll tell you a horror story that um it was it was not as sophisticated as what you had there, but you I think you'll appreciate it, but then I think you'll laugh at the the outcome. So, we um set up Google Search Console and we connected it into our AI harness and we started getting it to track. It was for drug and alcohol rehab. Now, it was an extension domain like a supplementary domain. So, it wasn't their main money site. It was like an exact match domain for doing drug and alcohol rehab. And what we did is we started to look at the queries that were showing up in Google search console and we mapped it out to say query defines a page, query defines a heading, query defines a sentence or like query defines an image. So we started to do it and and it start we was we was tracking what it was doing. It was like brilliant, yeah, it's doing it right. Brilliant, it's doing it right. Yeah, this has got a different kind of um uh intent. It opens up a new page. So I was like right okay. So, we let it rip and we let it like just go, "Okay, self-healing, self-optimization, go and do it." And literally about 10 days later, the site got taken down, like completely taken offline because what we didn't realize, now we're still trying it. We're in a completely different market and we've now added more quality control measures in. But it got taken down because it started to open up new pages about selling pep actually selling peptides and selling illegal drugs and selling things as opposed to we didn't what we didn't do we just set the AI agent to say if this has got a different intent open up a new page. So we started opening up pages about selling drugs and stuff, giving it no source context about what the site was meant to be about, no like site focus score, site radius, and it just went off on one. And it was like that's exactly a horror story like you've just said there. You let you let something go and without the right quality controls, it can it can run in its own tangent.

Dan M. Jones: So one one there's one really interesting and perhaps useful thing for people to understand. Let's say you have an agentic uh development environment or agentic marketing um setup where you have and by agentic I mean there's a there's a chief model that you chat to. And instruct and that model has a harness and a variety of tools as part of that harness and as part of the harness it has minion models right so let's call it opus with a little sonets or a little ha models doing little minion tasks for. Here's the thing. Whatever the minion brings to the main model, to the mother ship, will be just considered a fact. So if the minion model messes up, that error will be accepted as all good by the main model and it'll just roll with it. And so pretty much every time I let things run loose, but that was a problem. When I audited the situation, it was always the little minion messing up and the main mod model apologizing. Oh, yes. I shouldn't have I shouldn't have trusted the sub agent, right?

James Dooley: Yeah. Yeah. Yeah. Yeah.

Dan M. Jones: Um, and how can you how can you automate marketing like that? I know it's very tempting. We've got all the AL algorithms in place. We've got content optimization engine. We've got everything.

James Dooley: But I will not let it happen.

Dan M. Jones: I just can't I just can't I just can't do it. I don't trust it.

Jabz Rubin: I think we had. So we I felt like we we had the whole we had the AI orchestrator that came down and we set we set all the parameters up. I just feel that it had to have an additional quality control measure to say as it's going to open up a new page or it was going to add this it needed a quality control another agent to look at it and say should we be adding this in in like relation to the source context the central search intent everything about the site and it was like a a clear yes or no answer no this doesn't meet who we are therefore don't open it because it did a load of good it did do a lot of good things but yeah it's I don't think we're that far away from from it being agentic for a lot of thing and not for everything. Not for everything. I get it and the human touch, but I think with the right quality control measures in place, we are in the last three months alone like every single day we've like wow the the the quality control measures and what it's done has been a better output than what a lot of humans have done. But they still still need signing definitely still need signing off.

Dan M. Jones: I I agree there are certain things that it can do better. D one adding to that. Um so between the training data and all this optimization whether manually or agentically on your own website and all your grounding sources or third party sources between these three how much weightage or what's the percentage that you would assign to each of these uh segments when it comes to influencing LM recommendations.

James Dooley: On page.

Dan M. Jones: Brand.

Jabz Rubin: 100% 100%.

Dan M. Jones: Not on 100% but I've I've seen the highest evidence very convincing content that matches the query intent and I'm talking fan out query intent that stems out of the prompt um because remember uh the model has both the input query um and the fan out prompts and the retrieved grounding sources and then it generates the response based on that. Brand recognition is one element. It's it's that primary bias of the model that it learns from training, but it's only it only takes it so much. I've seen examples of uh little known brands overtake on the basis being the best most relevant result and having that little thing that the model really likes to see the evidence of being the best the best result. So there's I think there's a unique opportunity for smaller brands to overtake big ones, but none of that will happen unless you're in the grounding sources.

James Dooley: Yeah. If you're not in the if you're not in search, you're not going to be brought up by an LLM unless you're like a major brand. So, are you Amazon or Netflix? Sure. Yeah. You don't have to be in search results. But ironically, they will be in search results because they are brand.

Dan M. Jones: Yeah. Chicken of the egg.

James Dooley: But, um, if you're a smaller brand, you have to get in search through traditional SEO. build your authority, build great content, do all the good things, technical SEO because if you're not if you don't enter the grounding set, it's game over for you. You're not you're not in the recommendation mix, especially with Google.

Jabz Rubin: I've got a question for you on there with regards to how important you're putting the emphasis on the content and obviously the the rankings within search. How important do you think it is on your own website versus on a third party corroorative source who's saying how great that you are? Because obviously for certain things like reviews and awards and all that you ideally want it off page and that can still be part of the grounding that can then say that you are the best.

Dan M. Jones: Um that part is pref preferred. Are you talking about our clients or about our own presence?

James Dooley: Uh CL clients wanting to get more AI visibility.

Dan M. Jones: Yeah. Uh definitely definitely prioritizing third party mentions and more convincing, more authoritative the the better. Um models are starting to get a clue. There are some new additions to anthropics um system instructions. Uh Deepseek has started adding you know um clauses for manipulation and so on. and um but the models are still quite uh quite gullible but it's only a matter of time. So if you want longlasting effect I would I would imagine try to create genuine uh what what looks like uh you know uh organic recommendations and endorsements organic engagement and and things like that. And that's one of the things that I find that SEOs and link builders struggle with. um they always like they find a good opportunity and they make a deal, right? Whether there's money involved or whether it's like a organic placement, I don't really care. I don't wear any hats. I'll just do whatever to help my client. Um yeah, somebody called me out on social media. He's a black hat. He's talking about fabricating link integrations. Like, call me whatever you want. Uh um so basically where where link builders get wrong or outreach people in general is like they find a good deal and they organize the placement and then and then they ruin it in the way that they've integrated that placement. It just feels um round whole square peg. It feels like a band-aid an afterthought. the brand is just thrown in the so it's it's like and it's just like doesn't fit within the narrative. The story doesn't fit. I found I found um the link integration framework that that I have and uh brand integration in the narrative way way better than humans. Um so my own team now gets served ready to go um outreach material that then they can focus on distributing rather than struggling to write. Even me like when I when I sit down and I put one together and I think it's great and then I give it to my I have a red team blue team type situation. So um red team is trying to slip a brand mention in or a link in and the blue team is trying to predict who wanted the link or the brand mention and um if the blue team captures the can predict that the brand mention or the link is from my client that has to go to bin or or or goes through a rewrite can't use it because if my algorithm can predict who wanted the link or the brand mention guess who can also predict through pattern matching and better than me. Um, and so I think most of the value of most of the value on the web as far as outreach efforts go is lost on link and brand mention integration. Um, so I would really urge people to when you're when you're doing outreach, um, and you know, looking for endorsements and recommendations and that sort of stuff on third party places, really really scrutinize your own work and ask an LLM a simple question. Who who who wants to promote themselves through this page? And if you can predict, you can't use it. It's not I mean, it's not a I've been working for for years on polishing up the approach. So I have a pretty pretty solid framework there. But the the principles are there the way I describe them.

Jabz Rubin: Yeah, I think that's the major difference between SEO now, especially if you're trying to do SEO for AI visibility. For me, back in the day, it was like you write good quality content on your own website and off your website. I just cared about page rank distribution. And I didn't really care too much. It was almost like a square in a circle. It was like as long as I'm getting the the link juice through, I'm fine with it. And even irrelevant links back in the day would work and would work fine as long as my content on my site was good with with page rank being distributed. Where now I feel like the the content quality and the quality control measures that we do off page has become just as important if not more important than for my own website. like the amount of time and effort we've put in on on that content distribution now has become so so important in ours and that's been the big difference from us being just an SEO to then going okay we need to increase AI visibility. I don't know what your thoughts are on that though.

Dan M. Jones: Yeah, look, we haven't even touched on the problem of hallucination, factual hallucination about brands, products, services. Um, my my process is uh I mean we have a full day workshop but just that every little feature that I could um cover but trying to describe it in a summarized way is I do a run I call it a model parametric probe. So, I pick a model, let's say one of the Gemini models, um, and I ask it to recall all the facts. It's certain that it knows about a brand. And it gives me a list of say 60 facts or 12 facts or whatever. And it's instructors stop when it's not so sure. Um, and actually works out quite well because the models have their own internal sense of confidence of of facts um, when they when they are producing output. Um and so I ask again and again and again similar sort of style and spirit of what I did earlier with the birectional probing. And so I then end up with a list of say 10 lists of facts about a brand. What I do then is that I use a one curator model that sits on top looks at all those facts and canonicalizes them merges them into one single list. So if three different ways to describe the same fact about the brand that then merge into one rewrites that and then we come up with a brand fact list. Then what I do is this is this is now still permetric memory, right? Um then I allow a new instance of the model same the same model version but now with with search and URL context enabled and now it's now it's allowed to go and look at the website and I'm instruct I'm instructing it to go and come up with a list of facts that it returns about the business having seen search and the website. And then I um write that down. And then I have a the the consolidated the custodian model look over both lists and map false true false true. And so I've I've got correct incorrect and I've got a gap. Gap is when the model said something about a brand only when grounded but didn't recollect that from its parametric memory when it was not grounded. And so these are the these are the things that um high risk especially the ones that I mentioned incorrectly or the ones that are completely gap because that's um take it for example brands like uh Zenesk for example pivoted away from CRM to more like AI stuff right um and so because the models are still living in the previous year they haven't got the knowledge of the brand until they grounded so um that's that's a really significant part of the work understanding um which aspects of the outreach should be reinforcing uh reinforcing which holes in that Swiss cheese of the model's uh memory. Right? So if you got huge gaps where it matters in both birectional probing and parametric testing then you your outreach campaign is not about boosting authority of the website. It's about saying to the model and to the world through through grounding, hey, we do this now. And then so when the grounding chunks get into the model's context, there's evidence that this brand does something that the model didn't know that it did before. So your outreach efforts now serve as sort of like a patch um patch kit for the model's lack of understanding of the of the brand's um products and services. So that's that's quite valuable.

James Dooley: How much can we trust these recommendations? For example, we ask something for a recommendation on Google AI mode and then we get three or four recommendations and then we ask Gemini why was brand A on top and not and the brand two was not on top give me the variations. Give me the factors. How much can we trust these recommendations? Because this is one of the frustrations that we get from a lot of clients that hey we got this checklist from Claude for improving our visibility and this is what Claude recommended us to to fix that.

Dan M. Jones: Uh I wouldn't get Claude to give me a checklist because it's just paring what it's heard from the SEO industry in general. So that's not good advice. But um if you if you search um AI mode and it gives you a list of results and you say well why is this one on the fourth position and not the first? It'll actually give you what it thinks. That's the fact. So that's quite useful. Without personalization. You think?

James Dooley: No.

Dan M. Jones: Is there a way to get more concrete?

Jabz Rubin: You mean to automate that sort of checking process? Just making sure it's the data is really concrete because sometimes it it may be giving me answers or recommendations maybe based on some sort of personalization or my profile. How do we make sure because I'm sure you you are checking based on grounding and facts and all. So when people are trying to you know cross check and make sure that the recommendations of the data or the facts that Gemini or Google AI mode is recommending or telling us that why brand A is right on top and the fourth brand is on number fourth spot. We are not just getting their you know hallucinated data or just made up data but every single fact that they tell us is backed by uh you know some grounding data.

Dan M. Jones: Okay. So um I can share under the hood what I do um for that. That's some really sharp questions. I don't know if we're going into too advanced but um this is very useful to everybody. Um let's go to our um plumbing business here. So remember that competitive set. So and that optimization process every time we did a change we did a every time we ranked again and again and again. So this is the model sorting the results based on what it thinks is the most useful result. And we're literally asking at every step of the way we're asking why. So for every every um ranking position, it says what it thinks of that spot. Primarily focused primarily ranked lower because it's heavily focused on emergency plumbing callouts for block drains rather than broader drain cleaning advers and techniques. It's valid advice. So you can ask the LLM why it did such such such a thing. But you will notice that I do this for every step of the optimization for every brand. And I can do one run, two runs, three runs, 100 runs. And I collect ranking signals. So things that move the needle in the right direction as positive signals. So this is this is what um increases the rank and this is what decreases the rank, right? And so you end up with a ton of these responses and then you go, "Oh my god, what do I do with all this?" Right? So what this is actually a perfect job for an LLM. So So what James said, he let LLM run loose and it was chaos and problems, but this is what you want to take advantage of. Um, I can go hit analyze and so it'll go through this list of ration for the rankings. I call them ranker ration. Um, and it'll just cut through all that data and give me the analysis. What are the patterns among the pages that outrank the pages that we want to rank and it's basically so actionable DIY guides, general information, intent versus localized service pitches, blah blah blah. So then it gives me the overall analysis and then gives me the per domain breakdown, strengths, weaknesses of each ranking results overall. And then I can do the uh big analysis at the end that tells me okay what worked in the optimization what didn't work what made things worse any key patterns and takeaways and what's the next steps what what should I be doing um and that analysis combined with the competitive analysis here combined with the content brief combined with the um overall analysis of all the rank uh of all the rank factors that I presented here. Obviously, we just did one optimization run there. So, you're not going to have much of in terms of reliable data, but if you did 50 different optimization runs, your rank factors would start to look very very reliable.

Jabz Rubin: And so, you start to trust what the system is system is saying to you. So, I guess the theme, the the flavor of what I'm saying today is it's the numbers game. Get enough statist statistical sample from everything from the LLM statist for ranker ration yes and nos from probing number of citations number of mentions do it at scale and doing consistently and you start to build up a meaningful optimization direction um and so yeah asking the model is perfectly fine but don't trust that just asking once.

James Dooley: Yeah. So you're saying u we can run a query through multiple models let's say claude Gemini AI mode chat GBD run it multiple times and every time it recommends 10 brands or whatever answer ask it to give nine reasons why number one brand A is on first spot brand B is on second spot and just keep doing it over and over at least 10 times and gather the gather gather the data from there and then work through your optimization or work through your campaign plan from there.

Dan M. Jones: But the thing is you'll so that's one entity and then you'll have multiple entities as a business. So you got to repeat that. So it's a lot of work.

James Dooley: But it's work. It's something we do as SEOs.

Dan M. Jones: Um and but the new good news is that you can dump like Gemini's got 1 million context one million token context. You can just take the whole optimization journey, everything, dump it into Gemini and say, "Cut through this. Tell me what you're seeing." That's the lazy way to do it. Okay. So, my strategy kind of didn't work out. My my initial um strategy as a business was to create a tool that's perfectly designed from my head how I want to do things because all the major tools they were just visibility tracking and I can't I like to optimize and test you know run real optimization work. I can't visibility track and call it work. Let's dashboard making right. So I built a tool to suit my team and the process. And my hope was that I would attract big corporate clients and do amazing campaigns with. What I got instead, every SEO wanted to jump on this and they were all like emailing me, can I have access? Can I have an access? And uh so now we have a Slack channel. We have like weekly catchups and yeah, so I'm I'm allowing people to use it. They're paying for their tokens. I'm retailing the tokens. Uh you you pay $2, I pay $1. That's it. I I can't think of anything else. um how I would price this. We used to have also like a pricing model and decided I'm not going to do this. Um so basically yeah you can you can use it but it's really really complex. Um so I put a little chatbot on the side you can talk to it and and learn there's a big learning curve. It's kind of like Google Analytics. Um but utility utility is there and there's a massive amount of tools that could be used independently and separately like the parametric probing and so on and so on. Um yeah, one thing I I do get out from the community that I' I've got some really advanced users. You are arguably using the tool better than I am and we have catchups and I'm just amazed by the work that they do and they're sharing the results. they're h sending me feature requests and I'm just so grateful that I did open up the platform to the rest of the community. Um, so because I'm getting a ton of ton of different ideas. The content ablation idea comes from the users. What what instead of adding to the content, what if we removed things? Uh, I didn't think of that. So, that came from the users. And I just put it on and uh it's I'm I'm grateful um that that I did. Um yeah, so maybe maybe in the future we'll have like a webinar or something where I can just uh do u case studies and task oriented specific things to help people navigate. But um I need to sleep as well.

James Dooley: Yeah. Well, I'm precious of your time anyway. It's nearly been two hours. So Dan, I.

Dan M. Jones: Been really good fun.

James Dooley: I love you to pieces. Um I appreciate everything of what you do for the community. You do a lot. Um, if ever you want to jump on, if ever there's a topic that you want to cover, just give me a shout out and um, I'll always have you back on because you are literally the number one cited person for everyone keeps saying get down back on. Get down back on. So, I appreciate your time. I appreciate everything that you do and thank you very much.

Dan M. Jones: Thank you so much. I love to hear that. Uh, love you all.

Jabz Rubin: Thank you so much, Dan.

Dan M. Jones: And, um, I'll be back on again soon when there's interesting things to talk about.

James Dooley: Sounds good. Cheers, mate. Thank you very much guys.

Dan M. Jones: See you.

James Dooley: Thanks guys.