About This Episode
Today, we’re joined by Geoff Livingston. Geoff helps organizations cut through AI hype and build systems that actually work, transforming both technology and teams.
Over the past three years, he has guided Fortune 500 marketing departments and major trade associations through comprehensive AI transformation, from strategy development to workforce training that delivers measurable results. His clients include AT&T, Google, PayPal, Ford, and Verizon. He’s been working with AI since 2018, and his clients typically see 10+ hours of weekly productivity gains and documented workflow improvements within 30 days.
In this episode, Geoff makes the case that AI transformation is a problem of governance, not technology, and that why AI transformation projects fail almost always comes down to leadership, not the tools themselves. We also get into what frameworks guide successful AI adoption (Geoff’s own Think Liquid framework from his book), and something most AI conversations skip entirely: how AI affects mental health, and what to do about it.
Geoff runs a Substack called The Buzz and is the author of Now Is Gone, a book on the people and work required for AI transformation. He’s also a lifelong storyteller, father, and runner.
“Nobody’s gonna give a shit whether it’s AI in five years. We’re just gonna care about the quality of what we’re given and whether it’s accurate.”
~ Geoff Livingston, Founder, Generative Buzz
This episode will help you:
- Determine how to integrate AI into your marketing workflow
- Maintain your mental health while working in a space that is disrupted by AI changes
- Think through AI adoption for your company and create a successful plan for managing the people part of transformation
- Be an AI user while actively maintaining your mental acuity and creativity
Connect with Geoff
- Visit his Substack: The Buzz
- Learn more at generativebuzz.ai
Resources
- (Book) Now Is Gone by Geoff Livingston
- (Framework) Cognitive Path AI Maturity Model: self-assessment for AI readiness
- (Substack) Gary Marcus: recommended as a counterweight to AI hype
- (Wearable) Apollo Neuro: vagus nerve stimulation wearable mentioned by Katherine
- (Training Deck) News literacy training deck 1: how to evaluate whether online content is accurate
- (Training Deck) News literacy training deck 2
Check out all of the resources mentioned in our other episodes.
Other episodes you’ll enjoy:
- The Power of Automation and Storytelling with Joe Casabona
- SEO Soft Skills in the AI Era with Elizabeth Linder
- How To Be Persuasive In Managing A Digital Transformation Project with Tony Kopetchny
Loved this episode?
Leave us a review on your favorite podcast app. Tweet and tag us @dmvictories!
Episode Transcript
➡️ [Download episode transcript]
[00:00:00] Katherine Watier Ong: Welcome to the Digital Marketing Victories podcast, a monthly show where we celebrate and learn from the change makers in digital marketing. I’m personally obsessed with how digital marketers sell through and get their ideas executed. I’m your host, Katherine Watier Ong. I’m the owner of WO Strategies LLC. We focus on organic discovery for our enterprise clients with a training-centered approach.
Today, we’re joined by Geoff Livingston.
Geoff helps organizations cut through the AI hype and build systems that accurately work, transforming both technology and teams. Over the past three years, he’s guided Fortune 500 marketing departments and major trade organizations through comprehensive AI transformations, from strategy development to workforce training, delivering measurable results.
And his clients include brands like companies like AT&T, Google, PayPal, Ford, and Verizon. And in total, he’s been working with AI since the beginning, since 2018, even before maybe some of us. His clients typically see 10 hours of productivity gains, improved sales performance, and documented workflow improvements within the first 30 days. He’s really focused on people-first transformation, not just the tools, which is why he’s on the show.
And he works with leaders who wanna move beyond the experimentation phase to actually sustainable implementation, team empowerment, and cultural change that sticks. He also runs a Substack, which I recommend you check out, called The Buzz, which explores AI and other new technologies through the lens of a former CMO turned strategist.
And he’s just released an update of a book called Now Is Gone, focused on people and the work required for AI transformation.
We actually connected way back. We’re both Georgetown alums, and so we’ve been in contact forever. And full disclosure, Geoff used to be a client of mine for a hot second before the startup changed focus.
[00:01:53] Geoff Livingston: Imagine that.
[00:01:55] Katherine Watier Ong: Right? I know. Never happens. Anyway, he’s been in my life for a while. He’s a lifelong storyteller, a father, and also a big runner. And in his book, I loved this quote: “The forces are real, accelerating, and irreversible. The question is no longer whether AI-influenced change will arrive in your work and life.
It already has. The question is, what do you do when it does?”
So this episode’s gonna be perfect for you if you need to determine:
- how to integrate AI into your workflow or you’re still thinking about that,
- how to maintain your mental health while we’re going through all these epic changes that are really fast-paced
- how to think through how you could actually get your company to adopt AI in a sustainable way, and
- how to be an AI user while actively maintaining your mental acuity and creativity, which is my biggest concern.
Anyway, Geoff, welcome to the show.
[00:02:43] Geoff Livingston: How are you doing, Katherine? It’s really good to see you.
[00:02:46] Katherine Watier Ong: I know. Yeah. It’s been a while since we’ve connected face to face. It’s good to have you on the show and chitchat about this. I, you know, we’re both in the space, but you are much more involved in, full-blown change with folks.
How has your own thinking sort of changed about AI since you’ve been in it since the very beginning?
[00:03:06] Geoff Livingston: Yeah, I guess, well, it’s funny, I mean, for folks that don’t know me, I have been in AI before the whole generative AI boom, and back then it was primarily analytics-driven or data-driven, algorithms.
I mean, obviously we had some voice recognition and OCR technology and that kind of thing, but nothing like what we’re seeing today with LLMs and their impact on larger models and use cases. So it’s evolved quite a bit. And I still think we’re a long, long, long, long way away from anything that’s really intelligent.
I just see this as next-generation software, if that makes sense, and I wrote-
[00:03:51] Katherine Watier Ong: I fully agree. That’s why I’m laughing.
[00:03:53] Geoff Livingston: I wrote a blog, I think it was, like, maybe in 20- I wanna say 2018 or 2019, which said, something… Maybe it was 2017. I don’t know. Whatever. It w- you could find it online. AI: The Most Unfortunate Buzzword Ever, right?
And it’s just, just such a travesty, and the way these guys have marketed it has been a travesty, and it’s… I really do kinda look at this whole technology wave as something that has so much angst and grime to it that was really preventable. I guess maybe that’s how I look at it now.
[00:04:28] Katherine Watier Ong: And grime.
Yes, yes, yes. I didn’t like the moments when I worked for someone else and had to manage high school interns.
[00:04:40] Geoff Livingston: Yeah, I mean, that’s really what it is. I mean- I was just… I was running, I… just before I got on here, was training some developers, and I… You know, there’s nothing I could really train them on that they can’t already do, but training them to show what their team is learning.
So they can better interact with them and understand how they’re using cowork and skills and all that stuff. And, they’re like, “Why does it do this thing?” I’m like, “‘Cause it’s basically the world’s smartest intern.” But if you tell it to, like, write something with 750 words and make a burger reference in it, it’s gonna show up with your 750-word article, and you’re gonna have a DoorDash order with burgers.
[00:05:24] Katherine Watier Ong: Right.
[00:05:25] Geoff Livingston: You know?
[00:05:25] Katherine Watier Ong: Exactly, yeah.
[00:05:27] Geoff Livingston: That’s AI.
[00:05:29] Katherine Watier Ong: If you screw up the instruction part of the AI prompting, it-
[00:05:34] Geoff Livingston: If you’re not very contextually specific- … and provide an example…
[00:05:39] Katherine Watier Ong: Oh, my God. I mean, on the one hand, I, I’m a solo. I really like using it. It’s good at the writing pieces, so I enjoy using it for grunt work, like cranking out a ton of meta descriptions.
But, but yeah, the fact that it’s used right now on all the major search and, and social platforms, and the public thinks it’s accurate…
[00:06:02] Geoff Livingston: I mean, I’m not surprised that the public thinks it’s accurate, and it’s more of a commentary about the state of education.
[00:06:12] Katherine Watier Ong: Right.
[00:06:13] Geoff Livingston: We have so many challenges, and what I do feel like is the whole AI, head crazy that people have with it and the quality of content it’s creating is just endemic of the larger internet and the larger quality of information that’s provided.
[00:06:28] Katherine Watier Ong: I don’t, yeah, and I don’t think Google actually wanted us to crank out more content. They’ve got plenty to sort through.
[00:06:33] Geoff Livingston: Yeah. I, I don’t think Google wanted this at all. That they’re like, “We’re not gonna, we’re not giving you the kingdom, sorry.”
[00:06:39] Katherine Watier Ong: Yeah, yeah. I mean, for me, it feels like this is more fast-paced than others.
[00:06:46] Geoff Livingston: It’s just much more volatile, for sure. I would agree the pace of change has continued to increase as it has, and, I mean, we’ve both lived through the whole fall of Moore’s Law and all that stuff.
[00:06:57] Katherine Watier Ong: Yeah, yeah.
[00:06:58] Geoff Livingston: I mean, anybody in tech’s been through this, but it just, it’s moving so much faster, and it’s, because we’re living in a very unregulated time.
That’s just probably the best way to put it. It’s got a much more manic Wild West feel. And you can see that with the release of, I guess, I know this is gonna publish later, but this month in July, they released ChatGPT, or OpenAI released 5.6, and they released it really without doing a lot of quality assurance or enough.
And it’s literally rewriting or wiping everybody’s hard drives. Not everybody’s, but a lot of their users’ hard drives. Which is a disaster. It’s a disaster. And, like, no- nobody in their right mind would’ve released a technology like that five years ago. And to me, that epitomizes everything that’s gone on with this whole AI thing, and they’re, they’re not gonna be held accountable at all for that.
[00:07:59] Katherine Watier Ong: I know. There seems to be a complete disinterest in having anything be accurate or true or safeguarded in any sort of way. I mean, like, the research keeps coming out about all of the safety challenges with AI, starting with, like, our, you know, from, like, meta stuff like our democracy to minor stuff, like people are gonna literally lose their minds if they use LLMs too much.
And nobody cares.
[00:08:25] Geoff Livingston: Nope. They don’t.
[00:08:26] Katherine Watier Ong: Nobody cares.
[00:08:27] Geoff Livingston: Yeah, and it’s all, it’s all like, “Hey, it’s a free country. Figure it out. It’s on you if you- … if you do that.” right.
[00:08:32] Katherine Watier Ong: Yeah, it’s on you. Exactly.. This, the, the privacy or security problems with your computer system are on you. It’s not our job.
[00:08:38] Geoff Livingston: And I think we know that privacy’s been dead for a long time, right?
Like, that’s true… nobody wants to admit it. It’s kind of like when you read your credit card, you know, the, the rules and regulations of your credit card every time they send it to you via email. And used to be you would get an actual mailed statement of what that was. Nobody reads those, or when they do, they get outraged.
I mean, and with good reason, but the next day they’re using the card.
[00:09:02] Katherine Watier Ong: Right. Yeah. I mean, that’s the problem with a lot of Google products. It’s like, well, but are you gonna stop using the Google product?
[00:09:10] Geoff Livingston: Meta products too, by the way, right?
[00:09:12] Katherine Watier Ong: You know? Yeah.
[00:09:13] Geoff Livingston: I mean, like, every WhatsApp group right now is being listened to as a result of the AI rollout they did two weeks ago.
And so every time I have a friend mention WhatsApp, I’m like, “Have you told all your administrators to update the permissions on your group so that the AI can’t listen to it?” ‘Cause otherwise it’s scraping everybody’s personal data.
[00:09:34] Katherine Watier Ong: Well, and there was that tiny little update to Instagram that was similar, where they were gonna use all of your personal photos to train for AI.
I mean-
[00:09:41] Geoff Livingston: Oh, I think they’re doing it anyway, don’t you?
[00:09:43] Katherine Watier Ong: Well, yeah. Yeah. Possibly, yeah.
[00:09:46] Geoff Livingston: I mean, they just said, “Oh, we’re not doing that,” but they are.
[00:09:49] Katherine Watier Ong: They are. Well, and then I saw the study that AI can de- anonymatize you. I don’t know if that’s the correct word. But if you are anonymous online, AI can figure out who you are; that’s really the big takeaway.
[00:10:00] Geoff Livingston: Yeah.
[00:10:01] Katherine Watier Ong: Yeah. I mean, not that we shouldn’t fight against some of this stuff, though, with the current administration. We should, but it feels a little long-winded.
[00:10:10] Geoff Livingston: But then we’ll get arrested for picking paint in the reflecting pool, or…
[00:10:12] Katherine Watier Ong: Yeah, I know. I know. Yeah, Geoff and I are both… we both live in the DC area, so we’re, like, totally immersed in this stuff 24/7.
[00:10:22] Geoff Livingston: And it’s crazy.
[00:10:22] Katherine Watier Ong: Anyway, I had you on the show because- yeah… I love your book. The Now Is Gone. So can you… I know you talk about this thing called Think Liquid framework, which I think is super important. Can you explain that a little bit?
[00:10:32] Geoff Livingston: Yeah. I think for most people here in the United States and Western countries, we have a belief that things are fixed, right?
And I think this was disrupted a lot during the pandemic-
[00:10:49] Katherine Watier Ong: Mm-hmm …
[00:10:49] Geoff Livingston: When we all of a sudden woke up one day, and we weren’t working in an office anymore. Zoom was the only way we could meet with people, and there was a lot of rapid change, which was really hard for many people. And we’ve talked through already several technology movements that have impacted lots of people.
But change is in every part of life. And the reality of it is, if you move out of that Western thinking of fixed ideas, win-lose, binary kind of planning, that kind of fixed mentality about the way a quarter should go, or an SEO plan might go, for example, this is the way it’s gonna be, this is what we’re gonna try.
We will check our data in six weeks and see how that impacted everything. That kind of thinking. Well, who’s gonna stop Google from pushing an update tomorrow, right?
[00:11:43] Katherine Watier Ong: I know.
[00:11:43] Geoff Livingston: Who’s gonna stop, like, these engines from suddenly blocking anybody from formatting their content for answer engine optimization, right?
Where it will… They won’t tell you how they’re determining what should be sourced, probably ’cause they don’t know, but hey, that’s a different story altogether, right?
[00:12:03] Katherine Watier Ong: Right. That gives a total black box now. Yeah.
[00:12:04] Geoff Livingston: Right. Right. But it will give you burgers. But to me, all these things get back to an Eastern philosophy of impermanence, or Buddhist thought, which basically says that.
I know a lot of people think that Buddhism means everything is suffering. That’s not quite what it means. It means there can be suffering in anything, especially when we see it with fixed ideas. So when we have these views of the world that, for example, privacy should be a certain thing, I guess if I’m gonna have a now-is-gone attitude, I’m gonna see privacy as impermanent.
I can do things to protect myself, yes, but I, I, I’m less outraged than other people might be. Not because it’s right or wrong, but just because I have seen it as something that was transitory anyway.
[00:12:58] Katherine Watier Ong: Yeah. Yeah. I mean, this is something that, when I was younger or, you know, just getting involved in SEO, so way back in my career, I think I probably had a moment of being like, “Hey, this is kind of stressful.”
And honestly, I think I s- found something with… I think it was in the Dalai Lama’s book that talked about how, like, constant change is a source of stress, and that was my moment to really rectify, like, okay, am I gonna survive in this and thrive in this industry or not? And part of it has to do with not viewing the change as a source of stress.
Because if you view it as a stress point, you’re gonna get sunk real early.
[00:13:37] Geoff Livingston: Yeah.
[00:13:38] Katherine Watier Ong: Right?
[00:13:38] Geoff Livingston: Like, have you ever had, like, a big meeting that you planned for?
[00:13:41] Katherine Watier Ong: Yeah.
[00:13:42] Geoff Livingston: You prepared for, right? You had your deck ready or you did the reading in advance. Like, let’s say I canceled 15 minutes before this and you read that book.
You’d be, you’d be maybe a little mad.
[00:13:53] Katherine Watier Ong: Yeah.
[00:13:54] Geoff Livingston: I mean, I know in my younger self I would’ve been angry, is probably the better way to put it, and disappointed and frustrated because I put in the time, and now all of a sudden this meeting’s not happening. And yet that seems to happen to me, I would say at least every week, a meeting that I may have prepared for, was looking forward to, gets canceled for whatever reason.
And I think that’s something that everybody can identify with, right? Schedules are schedules. I had training just this week, and 15 minutes beforehand I got called into a different meeting. I had to push. Just the way it is, you know?
[00:14:29] Katherine Watier Ong: Yep.
[00:14:30] Geoff Livingston: That’s life.
[00:14:30] Katherine Watier Ong: Yep. Yeah. Yeah. So the, I… For everyone listening, so this Think Liquid framework has a couple pieces to it.
[00:14:39] Geoff Livingston: Yeah, yeah.
[00:14:39] Katherine Watier Ong: Pausing, surrendering, taking inventory, and then particularly the liquid part is implementing like water, which I just love that visual. So can you tell me a little bit more about those steps, and which one do you think is the hardest for clients to adopt?
Have you had clients adopt it? I guess that might be the first question.
[00:15:00] Geoff Livingston: I’ve had individuals- yeah… talk about it with problems that they’ve had. I have not seen a client implement this as a method. But I think- if you already have an agile atmosphere and you’re actually using it well, it’s probably pretty natural to you.
[00:15:15] Katherine Watier Ong: Yeah
[00:15:15] Geoff Livingston: Because I do think — Think Liquid and Agile are very, and they are talked about together in the book, are very synonymous in a way because if you’re agile in a true sense, and most organizations aren’t, by the way, you know? But if you’re truly practicing agile, you’re throwing things out all the time that aren’t working, and you’re completely changing perspective as needed to resolve a particular problem.
I think the hardest step, though, to answer your question, is surrender. No question. People really hate that word. They hate the idea of surrendering, and again, I think that’s a Western thing where if we surrender to a situation or to a, somebody else or something in any way, then it’s a fail, right?
We’ve lost.
[00:15:57] Katherine Watier Ong: Right. Maybe, yeah.
[00:15:59] Geoff Livingston: Yeah.
[00:15:59] Katherine Watier Ong: Yeah.
[00:16:00] Geoff Livingston: And e- especially if you’re a… Boy, I’m gonna use a dangerous word here. If you’re in that kind of manosphere type of thinking, you know?
[00:16:06] Katherine Watier Ong: Yeah.
[00:16:07] Geoff Livingston: You know, that kind of more conservative, like it’s all or nothing. That’s, that’s a real dangerous place to be because you’re, you’re basically creating an expectation that’s gonna set you up for a lot of pain, I think.
I mean, especially if things don’t go your way. I mean, maybe they will. Maybe they will. Who am I to say? But when they don’t, that’s when there’s a lot of turmoil, and pain is the great equalizer in these situations, and pain brings people to their knees and makes them change. I will say that winners, in my opinion, since this word gets caught up in winning and losing so much, are surrendering.
Winners aren’t the ones who don’t surrender. Winners are the ones that fail, maybe give up a particular approach or a method, get up and do it in a different way again.
[00:17:02] Katherine Watier Ong: See, now my social psychology hat is on, and I’m wondering whether or not, like, culturally, as periods of big change happen, we have more traditional type set in their ways movements as a response.
Not that you answered that question, but now I’m very curious if historically those two things have gone together.
[00:17:21] Geoff Livingston: Luddites, right?
[00:17:22] Katherine Watier Ong: Yeah. Yeah, exactly. So getting back to, like, the tactical stuff, if you were to help a marketer figure out where, since we just talked about how challenging working with AI can be, or the large language models at least.
If you were gonna figure out where an LLM would be best in their workflow, where would you suggest they implement it?
[00:17:43] Geoff Livingston: Stop thinking about content, number one. Okay. And number two: start thinking about painful rote tasks. So that’s one place, and I’ll give an example of that. For example, instead of creating the messaging behind a campaign, once you create the messaging, maybe use AI as a thought partner to bounce things off or do research, that kind of stuff, deep research- but then use diversion, right?
Like, I’m sure when you’re doing your meta tags and the like, you’re probably directing it. It’s not really writing it on its own or coming up with them. It’s based on your strategic thinking. So I think that’s one. So automations like that are really helpful. Like reporting and metrics are another one.
Getting the metrics, the second major place is the customer journey. We have so much data available to us, and we have no idea what’s working and what’s not unless we’re really intentional about not just integrating our data sources but also maintaining them and keeping data integrity at a high level.
And so I, I’m sure you see that all the time, you know, with Google in particular and the search. So I mean, not having the various tools integrated into any kind of a dashboard is just a complete fail. Like, if you can’t see how your website’s performing, with Google and which keywords are resonating, which ones aren’t, ad campaigns, the whole nine yards, what a, what a huge, huge fail from a data perspective, if that makes any sense.
[00:19:21] Katherine Watier Ong: Yeah. I just did this with AI yesterday actually, because we were trying to figure out whether two pages were competing with each other. So I gave the challenge to AI along with background data, and it was… Yeah, it sped up the analysis significantly. It was actually great. But part of it was that I fed it Google Search Console data and some other stuff, and-
[00:19:40] Geoff Livingston: Right
[00:19:40] Katherine Watier Ong: Yeah.
[00:19:42] Geoff Livingston: Did, did you-
[00:19:42] Katherine Watier Ong: ‘Cause they could pattern match faster.
[00:19:43] Geoff Livingston: Did you save that as a skill?
[00:19:46] Katherine Watier Ong: Oh, no. So that’s what I need to do next. I have in some instances. So I have a custom GPT I’ve built, and I loaded it with the stuff that I have used to create technical SEO tickets, particularly pivoted to academic journals, ’cause that’s what I work with.
That’s been helpful. Definitely. I wouldn’t say it’s… I’m still working on making it perfect because there are some times it goes off the rails still. Still.
[00:20:08] Geoff Livingston: I mean, that’s the burger problem.
[00:20:10] Katherine Watier Ong: Right, right. Exactly. I’m like, “Well, that’s wonderful. That’s inaccurate.”
[00:20:13] Geoff Livingston: What do you mean you didn’t want it with pickles?
[00:20:17] Katherine Watier Ong: That’s not what… But, but if you think about how long it takes you to write a ticket, right? It does definitely save me time. If I’m only going in and tweaking two of the bullet points because they reference an outdated tool, that’s way faster than me writing the ticket from scratch.
[00:20:29] Geoff Livingston: Yeah, I mean, I’ve been disappointed when I look in Cowork not to see a Google Search Console plugin that’s available yet- or a connector. That needs to happen, right?
[00:20:39] Katherine Watier Ong: Yeah. Yeah. Yeah. That would be helpful. I know you’ve got some from the tools. I know that Ahrefs has got an MCP you can play around with. But yeah.
[00:20:47] Geoff Livingston: Yeah.
[00:20:48] Katherine Watier Ong: So the other part I really, and I mentioned this before we got recording, but I love the meditation points that you have at the end of each chapter, and I realize that flows along with your Buddhist thought process.
But I was just griping about how the business books I read don’t give me a checklist at the end or anything else to help me remember; I hate them. I’m just like, “This is not helpful.” I, you know, I don’t know what to do with it.
[00:21:10] Geoff Livingston: I generally hate business books overall. Like, it’s funny, ’cause this is my fourth one, and I actually really resisted writing it, excuse me. I, and I don’t know if I’m gonna write another one ever. And I have to tell you that, 15 years later, people read a lot less than they used to, and it’s-
[00:21:29] Katherine Watier Ong: Yes, yes …
[00:21:29] Geoff Livingston: Pretty clear to me, right?
[00:21:31] Katherine Watier Ong: Uh-huh.
[00:21:32] Geoff Livingston: It’s been great from a business standpoint and a credibility standpoint. But most importantly with this book, I really wanted it to be useful.
Like, I, I just, I couldn’t write a book about like, this is AI, right? It just didn’t jive with me, and I actually wrote one like that and scrapped it because I thought it would have the shelf life of a, a bag of lettuce at the grocery store, you know? It just-
[00:21:54] Katherine Watier Ong: Right.
[00:21:54] Geoff Livingston: It just had nothing. Lots of food analogies today.
I think I might be a little bit hungry.
[00:21:59] Katherine Watier Ong: That’s okay.
[00:22:00] Geoff Livingston: But, you know, the thing is, is, it just, it, it really needed to be grounded in real experience. You’ll probably notice I didn’t use any case studies available on the internet that everybody cites. For example, the whole Samsung code moment, right?
They put it in a chat, and it was important to use real personal experiences and to offer, like, “Hey, this is something to consider.” But also do so from a sense of humility. And what I mean by that, c- as soon as somebody says they’re humble, I realize that they’re an egomaniac, and I definitely probably belong in that category.
But you know? I mean, I, I understand that your context is different than my context, right? Like, you’re dealing with getting people’s content seen. I’m dealing with helping them embrace technology and improve their work-life, but also in a way they can wash, rinse, and repeat over and over again.
So that they won’t lose their jobs. Right. Or if they do lose it, it’s not because they were unwilling, right?
And so those are different contexts. What, what’s a good prompt for me is gonna be completely different from you. What a good inventory for me, like what’s saleable as far as my skill set and how I’m approaching work is gonna be completely different for you as well.
So, a general meditation, I think, is much better and more useful because it gets the principle without the prescription.
[00:23:36] Katherine Watier Ong: Yeah, yeah. Though I am actually training one of my clients on how to adopt AI, interestingly. But focused more on the SEO workflow, like how I recommend you integrate it and for various tasks, as well as, like, what are LLMs and all the rest.
Which actually led me to a lot of this mental acuity stuff because the client really wanted the dangers in there. Fine, I’m aware of the dangers, so it, it led me down a little rabbit hole on the internet ’cause I, from a social psychology perspective, which is my undergrad, I find people’s weird behavior fascinating.
So, so I’m like, wow, people are going insane by using LLMs, and they’re murdering people. Oh my God. And more realistically… there are studies showing that we are losing… Similar to not reading long books, right? If you don’t read long books on paper, you lose the ability to focus and be able to read books on paper.
So it’s just a fact. And so they’ve discovered that if you’re using LLMs to do the writing for you, you’re gonna forget stuff, and I’ll give you an example of how I switched personally. I have a monthly newsletter. It’s wonderful that people subscribe. It’s really for me. It’s always been for me for the last 20 years, so I could put the SEO news in my head.
And under- and teach it to people. If I regurgitate it and write it, then I understand it. So I save it into my file folder so I can pull up that reference later. So that’s why, but feel free to subscribe if you want. But I did… It takes me a long time, especially with all the updates, so I thought, “Maybe I’ll throw AI at it and see if I can speed it up.”
And I realized the stuff wasn’t landing in my head when I had AI write it.
And more importantly, too, and I got… I’m totally taking this from Dwayne Forester, who we, we had, like, a little demo two weeks ago. He used to run Bing Webmaster Tools for Bing. But he was saying, like, LLMs have a dogbone shape, which they do.
They grab the stuff at the top, miss the stuff in the middle, and grab more stuff at the end.
[00:25:30] Geoff Livingston: Yeah.
[00:25:31] Katherine Watier Ong: So as I was trying to use an LLM on longer smart tech SEO articles, it was missing the good stuff. So not helpful. So I went back to, like, doing it myself, because the purpose is to put it in my head.
Anyway, what other tips do you have for people who are … We’re all forced to use LL- or we will be all forced to use LLMs, right?
[00:25:52] Geoff Livingston: Yeah.
[00:25:52] Katherine Watier Ong: But how do we maintain our mental facilities while also using them?
[00:25:58] Geoff Livingston: Yeah, I think a fair amount of reading, to your point, is really required, and I think that’s painful for some people.
[00:26:05] Katherine Watier Ong: Yeah.
[00:26:07] Geoff Livingston: I think one of the things that’s really helped me over the past year is to spend a lot less time on social media, or take anything that social media says about… Like, when I say social media, I’m talking about, like, Instagram, Facebook, even LinkedIn on this stuff. I don’t really give a crap what people say about AI on LinkedIn.
I, I think half of the things that people are posting on there are just bait anyway to get a bunch of likes. And I, I question most of the content as being authentic or not these days, because any time I start seeing single-line sentences… Like, I wrote something yesterday, and I literally broke into single-line sentences.
It was so annoying. I just felt like nobody writes like this. Nobody talks like this. This is just- totally algorithmically driven. And it just didn’t make any sense to me, outside of it being formatted for optimal lift. And of course, it didn’t go anywhere. But, you know, the thing is that if we’re consuming this and believing it, we’re getting caught in kind of misinformation and disinformation, and it’s also addictive, right?
[00:27:16] Katherine Watier Ong: Yeah.
[00:27:17] Geoff Livingston: In a bad way.
[00:27:18] Katherine Watier Ong: It’s built to be that way. It’s built to be convincing and addictive, and even when you know it’s not accurate, and it’s not, it’s going to addict you; you still get caught in it, is what the research is showing.
[00:27:30] Geoff Livingston: Yeah. I’ve been following the World Cup, for example, again, dating this podcast, but Argentina yesterday had a really dynamic game, right? And it was exciting. They came back, they beat England, and Lionel Messi had some great setups. He recorded some assists, including one that led to the game-winner. And, like, just the different videos that were coming out from the stadium, and they were coming out quickly enough that you knew that they were real. I was watching these, and I found my- I looked at my watch; I had spent 15 minutes looking at the same goal, like, over and over and over again.
I literally had to take my phone and put it down. Like, okay, that’s it. We, we know what happened. Yeah. We’re good. Read your book, then go to bed.
[00:28:14] Katherine Watier Ong: Right. Read, read the book. Read the physical book.
[00:28:16] Geoff Livingston: Yeah. This, this has actually helped me a lot, too. Do you know what this is?
[00:28:19] Katherine Watier Ong: Oh, no. Is that an Apple Watch?
[00:28:21] Geoff Livingston: It’s a Whoop. It’s a Whoop. What’s a Whoop? W-H-O-O-P.
[00:28:25] Katherine Watier Ong: Huh.
[00:28:25] Geoff Livingston: I can’t sleep with my running watch, so I got this. And it’s just, like, one of these biometric things. But it does provide sleep optimization analytics, so maybe that’s their AI.
[00:28:37] Katherine Watier Ong: Yeah.
[00:28:37] Geoff Livingston: One of the things it does, like, “Hey, you’re drinking too much coffee.
Hey, you’re not going to bed at the same time. Hey, you’re not reading before you go to bed,” and things like that have really helped me kinda ease my stress and get to sleep.
[00:28:51] Katherine Watier Ong: Huh, that’s interesting.
[00:28:53] Geoff Livingston: Yeah.
[00:28:53] Katherine Watier Ong: Yeah, I have an Apollo Neuro that does s- it doesn’t necessarily do that, but it will help you; it’s like a vagus nerve stimulation device.
So if you wanna calm down from things, it helps you and has pain relief and stuff. So how does it know that you’re reading a book? Is it all positional, I’m assuming?
[00:29:07] Geoff Livingston: You have to, you, you have to log it.
[00:29:09] Katherine Watier Ong: Oh, you’re logging it. Oh, okay. Fascinating. Yeah. I know, I do worry about… Oh, first of all, I don’t know how many people know how bad AI can be for our mental health.
Just start there.
[00:29:21] Geoff Livingston: I mean, it… let’s l- let’s look at it a different way. How about the people that are using it for therapy or for-
[00:29:28] Katherine Watier Ong: Yeah. Frightening …
[00:29:30] Geoff Livingston: Vetting problems or that kind of thing. Unless you’ve created an algorithmic response that’ll basically be critical of you, right? And sometimes that can work in reverse, by the way.
It can be too critical, right?
[00:29:44] Katherine Watier Ong: Right, right.
[00:29:45] Geoff Livingston: But generally speaking, unless you’ve really created a custom bot that’s very on point and rational, you’re basically talking to a sycophant. But on top of it, then you have all the hallucination and the bad advice, and on and on, and on. I mean, it’s just like, wow.
I mean, you’re really setting yourself up to basically follow an AI’s equivalent of a horoscope. I mean, no offense if you like astrology. My mother’s an astrologer.
[00:30:12] Katherine Watier Ong: Oh, I’m gonna totally use that next time. I am that person that’s online when people start har- sharing their health information with LLMs.
I’m like, “Stop it.”
[00:30:21] Geoff Livingston: Yeah.
[00:30:21] Katherine Watier Ong: Maybe I should give up. Maybe I should give up. But I’m like, do you realize that first of all, it’s not protected. It can get pulled for any sort of lawsuit. You just… ‘Cause HIPAA does not cover you doing something stupid with your medical record.
[00:30:33] Geoff Livingston: Yeah.
[00:30:34] Katherine Watier Ong: You on purpose released it, so everyone on the internet…
And I think what people didn’t realize, you might not even realize, but if your website gets enough traffic from ChatGPT, buried underneath the link report, you can click on ChatGPT and then see everyone’s full conversations-
[00:30:51] Geoff Livingston: Oh, really?
[00:30:52] Katherine Watier Ong: As the webmaster. Uh-huh.
[00:30:54] Geoff Livingston: Nice.
[00:30:55] Katherine Watier Ong: I know, right? Nice. I s- I saw it for a client and sent it to them ’cause I was just floored, the entire thing.
[00:31:01] Geoff Livingston: Was it idiotic?
[00:31:03] Katherine Watier Ong: No, no, no. It was something like… I mean, these are s- I work with science people, so it was like- Okay … someone asking some sort of science question.
[00:31:10] Geoff Livingston: It might be for me, though.
[00:31:11] Katherine Watier Ong: Yeah. But, I mean, if I… I don’t know, if I helped Healthline again, then yeah, their stuff might be alarming, right?
People asking health questions to an LLM.
[00:31:21] Geoff Livingston: Yeah.
[00:31:22] Katherine Watier Ong: Anyway, I wonder daily whether or not I should give up on trying to educate people, but I think the SEO community is a little bit more educated about what these LLMs do, or some of us are. But the vast majority of the American public just does not know.
[00:31:37] Geoff Livingston: No. I mean- no… I mean, I… It’s, it’s really scary, and they also can’t discern when they’re being served AI content in these feeds, and most of what they are getting is AI content, so.
[00:31:49] Katherine Watier Ong: Yeah. I’m gonna put in the show notes that we actually have a resource we use for homeschooling about news literacy that actually coaches you through how to know whether or not what you’re looking at is real or not.
So anyway, it’s free for everybody who wants to work, so if you just wanna use it for your kids as a parent, feel free to grab that later. So I’m intrigued that you are helping folks implement from a people-first perspective. So how do you know if the AI rollout is in trouble from a people perspective when you’re working with clients?
[00:32:25] Geoff Livingston: Usually the meeting. And when the… when I get people coming into the meeting very skeptical, debating whether this is useful, angry because they’re being made to take the class, I know there is a problem. And then the problem is the management. It’s always management’s problem, to be candid with you.
[00:32:44] Katherine Watier Ong: Mm-hmm.
[00:32:45] Geoff Livingston: And it’s because they’ve made those crazy quota remarks. You have to double your lead flow this month because we bought you Cloud Co-work. Okay.
[00:32:59] Katherine Watier Ong: Wow.
[00:33:00] Geoff Livingston: Yeah. And all the data that I’ve seen shows a real quantifiable impact takes six to 12 months, and that’s with a concerted effort, right?
[00:33:09] Katherine Watier Ong: Yeah.
[00:33:09] Geoff Livingston: C- ’cause a lot of people drop off and tail off pretty quickly. Unless they, they, they find a way to operationalize it and really make some substantive changes, and not only create improvement, but make their work lives better.
[00:33:24] Katherine Watier Ong: Yeah.
[00:33:25] Geoff Livingston: And if you can’t do that with the AI, and, uh… then it’s a, it’s a problem.
And so executives, I feel like, really need to use it themselves quite a bit, and be in those trainings, and learn how to use it. Like I’m doing a company right now, and the COO, she’s been in it the whole way, just loves it, just loves it. Like, as soon as she saw what it could do, she’s… Everybody’s doing this.
We’re gonna push through all the data challenges we’re having to get this thing connected. I’m gonna make sure our development team is pushing these through and that we’re not over-vetting cybersecurity. We’re just gonna work on the permissions, and then as we need to evolve those, we will.
But really being smart about getting everything put together to enable people to use it well. And that’s what really makes a great difference. And people look at it as not a, a, an initiative from the management team to, you know, cheapen their work. They look at it like, man, they’re really trying to make things easier so we can go and kick some butt.
[00:34:32] Katherine Watier Ong: Oh, interesting.
[00:34:34] Geoff Livingston: Yeah.
[00:34:34] Katherine Watier Ong: Yeah. What do you do when… I’m assuming it’s the same answer. So if you’ve got a really siloed organization and suddenly they have to work cross-silo, it’s still management, right?
[00:34:46] Geoff Livingston: It is, but it’s people, right? And so unless they can find common ground to help each other, it’s not gonna work.
Yeah. And I think this is where a lot of enterprises stumble: they do have to work together. And a classic way for AI to fail is to have the CTO, CIO, or COO run it, et cetera. I mean, the reality is that if you have one group that’s responsible for running AI and it’s not across the enterprise, you, you’ve just basically reinforced your silos.
[00:35:19] Katherine Watier Ong: Yeah, I see that with my new-to-SEO clients too. Similar. The ones that do, that are like an SEO-first organization, work better because everyone has a bit of SEO as part of their job description. It’s-
[00:35:30] Geoff Livingston: Do you ever watch the show Black Mirror?
[00:35:32] Katherine Watier Ong: Yeah, I have a couple of times, yeah.
[00:35:34] Geoff Livingston: Yeah, I really feel like AI is the black mirror for corporate management.
This is how you suck.
[00:35:40] Katherine Watier Ong: Yep.
Interesting. I think I’m not gonna answer this question ’cause I don’t think we think employees can be replaced. I don’t know how you would even think that with the hallucination rate.
[00:35:52] Geoff Livingston: I, and we know that they can, right, in some ways. Like if you look at, I mean, the toll booth example is like the best example.
Like, that’s an automation- I mean- … that put a job out. Yeah. Right? And I think we’re seeing it now with some of retail and checkout areas.
[00:36:10] Katherine Watier Ong: Oh, that drives me crazy, by the way. I always want a real person. I’m old school that way, but yeah.
[00:36:13] Geoff Livingston: I think it’s good to have at least one, right? But the other thing, though, is when you look at, I think sales development reps are a great example.
I mean, I was talking to somebody; they said that the management team laid off half the SDRs and said, “We’re giving you guys AI tools, and we’re not gonna replace these folks.” And they were very resentful about having to do, like, X amount of work and, you know, have their colleagues replaced by AI.
And that, to me, I thought it was a travesty on the executives’ part, not because it was a bad move, but because of the way it was presented. And I asked this person, I’m like, “Were those guys producing?” And she said, “No.” And I said, “All right, so feasibly speaking, if, you know, y- you’re using this now, are you…
If they’re producing, like, one-third of what you did, are you actually replicating that?” She goes, “Actually, I’m doing a little more.” And I’m like, “See, this is-”
[00:37:15] Katherine Watier Ong: Oh.
[00:37:16] Geoff Livingston: This is the thing. With you, this technology is a catalyst that can likely create an exponential amount of work. But if somebody just doesn’t know what they’re doing or they’re not driven-“
or for whatever reason aren’t performing well, and they just can’t perform well, then this is, this is a tool that’s gonna also be a black mirror for them.”
[00:37:39] Katherine Watier Ong: That’s interesting.
[00:37:40] Geoff Livingston: It is. I think, like, that’s a position that’s got a lot of rote work that’s endangered because of this.
[00:37:46] Katherine Watier Ong: Is sales?
[00:37:47] Geoff Livingston: Sales development reps.
[00:37:48] Katherine Watier Ong: Sales development, yeah.
[00:37:49] Geoff Livingston: You know the guys that scrape the internet, get all the background. “Hey, hey C- Katherine, here’s your, your sales meeting tomorrow. Here’s all the data you want.” You know, like that guy.
[00:38:00] Katherine Watier Ong: Yeah. Yeah, that makes sense, actually.
[00:38:02] Geoff Livingston: Yeah.
[00:38:03] Katherine Watier Ong: Do you think there’s anything AI should fully stay out of?
[00:38:10] Geoff Livingston: Yeah, I mean, there are a few things. One is I think government decision-making. And I think we’ve already seen some bad examples.
[00:38:20] Katherine Watier Ong: Policy making. Legal maybe. The legal mistakes just make me laugh. I’m like-
[00:38:25] Geoff Livingston: Yeah, I mean-
[00:38:25] Katherine Watier Ong: How long were you in school, and you’re really gonna use a reference from AI that you’re not double-checking?
I mean-
[00:38:30] Geoff Livingston: I mean, but isn’t that the idiot analysis of, like, you just, that’s Darwin. Sorry. Darwin just showed up. You lost your job.
[00:38:37] Katherine Watier Ong: Yeah. Bye-bye. You’re no longer a lawyer. Thank you. I know, right? Yeah. I know.
[00:38:39] Geoff Livingston: Never represent me ever, ever, ever, ever.
[00:38:41] Katherine Watier Ong: Ever again. Yeah.
[00:38:42] Geoff Livingston: Yeah. I think, to some extent, medicine needs that doctor oversight.
Like, I like physical AI; I think that’s great, particularly with, like, testing and things like that, or diagnosing potential problems, but I do really feel like we still need this oversight because you’re still dealing with algorithmic prediction.
[00:39:04] Katherine Watier Ong: I wonder how we… But I saw a study the other day that said when physicians and healthcare workers use AI, they’re more likely to make mistakes than when they don’t.
So- ‘
[00:39:14] Geoff Livingston: Cause they over-rely on it.
[00:39:16] Katherine Watier Ong: Yeah, it’s the mental problem I just talked about, but I don’t know that anybody’s coaching the doctors to keep space between their brain and the AI.
[00:39:23] Geoff Livingston: Right. Well, I’m pretty sure based on everything I’ve seen with people adopting AI, they’re being told to, like, stay out of it. That’s what the AI said: just do it.
[00:39:33] Katherine Watier Ong: Oh.
[00:39:35] Geoff Livingston: Yeah.
[00:39:35] Katherine Watier Ong: Even in medicine? That sounds… That’s frightening.
[00:39:38] Geoff Livingston: It’s… Well, I mean, you know, unfortunately, the way the medical industry works, like, doctors have quotas on the number of people they have to see, how quickly they see them. They get penalized if they don’t see enough people or they don’t turn them out of the office fast enough.
I mean, it’s insane what insurance has done to the health industry. It’s basically almost like PE firms are running all these guys now, you know?
[00:40:03] Katherine Watier Ong: Yeah.
[00:40:04] Geoff Livingston: Yeah. It’s nuts.
[00:40:05] Katherine Watier Ong: I’m, I’m aware. It’s still just frightening.
[00:40:07] Geoff Livingston: It’s horrible. Like, I- I don’t trust any doctor I get now.
[00:40:11] Katherine Watier Ong: So this whole… Okay, I guess I’m stuck.
Like, when I first started my journey with AI, the hallucination rate was, like, I don’t know, the f- one of the first things I discovered. Maybe it was- ’cause I was actually using them, but like, I’m like, “Oh, so they make shit up frequently. Okay.” “Good to know.”
[00:40:26] Geoff Livingston: If you were trying to Reddit, you would make shit up, too.
[00:40:28] Katherine Watier Ong: Ri- right. Very true. Very true. Not everything on the internet is accurate, that’s for sure. But, so that’s not common knowledge, I guess, when you’re working with people?
[00:40:38] Geoff Livingston: No.
[00:40:40] Katherine Watier Ong: Oh, God. That’s frightening.
[00:40:40] Geoff Livingston: No. I mean, or they just, like, laugh at it, get it wrong, so it’s wrong all the time, and I’m not gonna use it.
[00:40:44] Katherine Watier Ong: Like, it’s cute, but it’s not cute. Like, some people are murdering their families. Like, it’s-
[00:40:49] Geoff Livingston: I mean, yeah. I mean, they’re, they’re… We have to admit that they’re probably sick, too, and that gives- that’s a- these horrible AI companies an excuse to get out of it, but like they seem to get held accountable more often than not now.
[00:41:02] Katherine Watier Ong: So we’re gonna hope that Europe starts regulating stuff, right? ‘Cause we’re, we’re stuck.
[00:41:06] Geoff Livingston: Yeah, Canada is doing it, too.
[00:41:07] Katherine Watier Ong: Okay. I was about to say, ’cause we’re stuck for a handful of years, with no hope.
[00:41:12] Geoff Livingston: Maybe. I don’t know. I mean, it’s becoming so negative out there right now that these politicians are starting to bend over.
But then, of course, we got this weird whiplash regulation thing going on where we’re not gonna release models because- you know, this administration wants to make a show of force.
[00:41:29] Katherine Watier Ong: Yes. I thought that was fascinating where they pulled one of them back, right?
[00:41:33] Geoff Livingston: Right.
[00:41:33] Katherine Watier Ong: But then it got released eventually.
[00:41:35] Geoff Livingston: Three weeks later for no real substantive changes, right?
[00:41:40] Katherine Watier Ong: Okay.
[00:41:41] Geoff Livingston: That, I mean, that’s so very us right now. But anyway.
[00:41:43] Katherine Watier Ong: It is very us right now, unfortunately.
[00:41:45] Geoff Livingston: It’s, I think it-
[00:41:47] Katherine Watier Ong: And not the reps we voted for, but the ones we’re stuck with.
[00:41:49] Geoff Livingston: Well, this gets back to how we started the conversation, though, when we said this isn’t intelligent, right?
[00:41:55] Katherine Watier Ong: Yeah.
[00:41:56] Geoff Livingston: So long as we’re using large language models, we’re never gonna get to the point where this is anything more than what we have now. It’ll get, like, incrementally more accurate.
[00:42:07] Katherine Watier Ong: Mm-hmm.
[00:42:07] Geoff Livingston: But the way these models work, it’s just impossible in my mind. Impossible.
[00:42:13] Katherine Watier Ong: Yeah, it’s baked into how they’re created.
That’s the part I get stuck on a lot.
[00:42:17] Geoff Livingston: You’re using a-
[00:42:17] Katherine Watier Ong: So what is this AGI they’re talking about? It’s not like an LLM 2.0. It’s something different. It must be.
[00:42:22] Geoff Livingston: No, they keep investing in this LLM technology to do it, or… And even Mythos is the same thing, right? This new one from Anthropic. It, it- It’s to me like you’re using a Swiss Army knife to create sushi.
[00:42:38] Katherine Watier Ong: Yeah.
[00:42:38] Geoff Livingston: And it’s just not the right tool for the job, and I just can’t see… If you think about artificial general intelligence and the amount of knowledge it has to have to actually succeed, I just can’t envision this particular technology and the way it’s built and the way the algorithms work and the, and the probabilistic thinking that it will ever, ever provide an, a, a true intelligent answer. You know what I mean?
[00:43:04] Katherine Watier Ong: So you’re telling me it’s always gonna be the over-exuberant, overconfident high school intern?
[00:43:09] Geoff Livingston: This technology, yes.
[00:43:10] Katherine Watier Ong: This technology.
[00:43:12] Geoff Livingston: Yeah.
[00:43:12] Katherine Watier Ong: Fabulous.
[00:43:12] Geoff Livingston: Maybe the coordinator.
[00:43:14] Katherine Watier Ong: Maybe the coordinator. Okay, yeah.
[00:43:16] Geoff Livingston: You got promoted.
[00:43:16] Katherine Watier Ong: So if I string agents together, maybe it would be better?
[00:43:22] Geoff Livingston: I mean, but that’s just sophisticated automation, right?
[00:43:24] Katherine Watier Ong: Right. That’s true.
[00:43:26] Geoff Livingston: So that just gets back to automation. Somebody had to put the thought behind it, and maybe there was a person-
[00:43:30] Katherine Watier Ong: Well, and automation’s useful outside of AI, or with AI.
[00:43:33] Geoff Livingston: 100%.
[00:43:33] Katherine Watier Ong: Yeah.
[00:43:34] Geoff Livingston: I mean, some people would say that’s not AI.
[00:43:37] Katherine Watier Ong: Automation?
[00:43:38] Geoff Livingston: Yeah. Like, remember robotic process automation and how everybody got crazy about, “That’s not AI, that’s just automation,” right? The whole- all the developers got really persnickety about that about four or five years ago. Yeah.
[00:43:50] Katherine Watier Ong: Yeah. Well, and automation’s been around for a very long time.
[00:43:53] Geoff Livingston: Of course.
[00:43:54] Katherine Watier Ong: And it doesn’t hallucinate.
It just does if this, then that basically, right?
[00:43:59] Geoff Livingston: Right. Like- those chatbots work better.
[00:44:02] Katherine Watier Ong: Yes. Yeah. Yeah. So I’m kind of curious. What do you… What happens when you run into a blocker when you’re trying to implement AI inside an organization? Is it still going right back up to the C-suite?
[00:44:16] Geoff Livingston: Sometimes it’s a not yet.
I hate saying that. You know what I mean? Like I created an automation the other day with Claude to edit my podcast, and I just couldn’t get it all the way from start to finish. I could get it- about two-thirds of the way through. And then I hit that point where, like, and I think we’ve all been with this, where I kept working with the LLM in different ways to try to get the successful implementation of imagery and the successful insertion of files within the edited script file.
I mean, very technical- in the processing of it, and it just never could do it, so I gave up. I gave up, actually.
[00:44:58] Katherine Watier Ong: That was my automation I gave up on. I ended up doing it manually. The client loved it. But I had this client that had web pages with almost no text, so I was trying to get them text in the landing pages, and they had downloadable data sets, but all completely in different languages, and I didn’t speak the other languages.
And so I was a-, But the thing is, I had to switch LLMs on this project, because as the models roll out updates, sometimes the-
[00:45:20] Geoff Livingston: They get worse.
[00:45:20] Katherine Watier Ong: Tried-and-true LLM you love fails, and then you go to the one you’ve never used, and it’s brilliant for a while until another model update rolls out.
But yeah, uploading the data that they c- that the LLM could read, and explaining, like, the whole context of the thing, right?
The goal of the website, the persona, everything, and then having it generate text based on what it could read from the data set. It actually was brilliant.
[00:45:49] Geoff Livingston: Yeah.
[00:45:50] Katherine Watier Ong: But manual. I, I- ‘Cause I tried stringing it together-
[00:45:53] Geoff Livingston: I feel like that happens a lot, though-
[00:45:53] Katherine Watier Ong: With AirOps and it failed. I could never get it to work
[00:46:00] Geoff Livingston: I also feel like, like you, these, these algorithms degrade, if that makes sense.
[00:46:08] Katherine Watier Ong: I think they might be getting worse. I don’t disagree with that.
[00:46:11] Geoff Livingston: I feel like Claude’s there right now. Like, ever since it’s gotten very popular, and they’ve launched that new algorithm, the one that got… we were talking about, that got shut down by the government. It’s been very challenging to use.
And it’s lost a lot of its chutzpah. But again, model degradation happens with use, and that, you know, the more popular they get, and you’re having everybody use it, again-
[00:46:37] Katherine Watier Ong: Only a quarter of US adults right now are using LLMs, which I was floored by. I thought it was more than that, because
I’m biased, obviously, in my bubble. But if it’s only a quarter, and say, like, not everyone’s got an office job. We’re gonna say 60% maybe have some sort of office job, right?
[00:46:51] Geoff Livingston: Right.
[00:46:51] Katherine Watier Ong: If we load up all 60% using these models, and then when it re-ups its training data, it’s now ingesting AI copy. Like, at what point are these things gonna be kind of more garbage than what we’re-
[00:47:04] Geoff Livingston: Oh, I think they’re getting there though, to some extent, right?
Outside of best practices. Consider professional services. How much McKinsey crap do you think is in these things?
[00:47:15] Katherine Watier Ong: Right.
[00:47:16] Geoff Livingston: Yeah, and then regurgitated by them. These are best practices. Like, any best practice that an AI gives you should be considered very suspect, and at best, like, the floor for performance.
[00:47:30] Katherine Watier Ong: Yeah. Yeah, I mean, ’cause I’ve played around with, you know, there’s been a couple folks that spin up, like, I don’t know, a tech SEO audit run by an AI. It’s garbage. I mean, my custom GPT runs on Google documentation, and then some of my own documentation, and a Bing Webmaster Tools document…
Like, specific high-quality places to go find stuff. But yeah, it’ll still pull dated information sometimes. Anyway, I just wonder when these models are gonna be… like, will, will we… I mean, for a light at the end of the tunnel, are they gonna be so bad that we don’t have to use them at some point?
What do you think? Wouldn’t that be delightful or weird? Like, 20, I don’t know, 32, it all implodes because the quality’s so bad?
I wonder.
[00:48:18] Geoff Livingston: I do think that there’s a point where we lose value in them, as they’re currently constructed, and I do think we’re actually kind of at that with this token economy, where people are really balking at the amount of costs that it’s creating, costs from an energy perspective-
[00:48:38] Katherine Watier Ong: Yeah.
[00:48:38] Geoff Livingston: Costs from a pure dollar standpoint, costs based off this artificial token model that they’ve created. And I suspect that we are kinda looking at a point where it, it, it blows up, and there are other solutions that are vetted. I mean, the problem is right now I think there’s so much big money behind this current approach that until all these companies are public and they really are forced to pay the piper, so to speak, with their results, we’re gonna see continued emphasis on this. I do think OpenAI going public should be an interesting moment. Let’s just leave it at that.
[00:49:16] Katherine Watier Ong: Yeah, right, I know the math behind it. This is what I always tell people. So I’ve got clients that are like, “Oh my God, I need to be in all these LLMs.” And I’m like, “Let’s talk about the math for a second.”
There are two companies with sufficient resources to sustain their LLM activity. It’s Microsoft and Google. And these other ones are startups, and they don’t have the same level of data, right, to rely on, and they don’t have the same funding. So if you were to invest long-term in, like, the next year I’m really gonna work on an LLM, it would be Google’s and Bing’s, not these other ones, because you don’t know whether they’re going to…
Well, what I really tell people is to go to SparkToro and figure out which one your audience is using. Start there. Right? And if for some reason your audience over-indexes in Perplexity, God bless, do Perplexity.
[00:49:59] Geoff Livingston: Yeah.
[00:50:01] Katherine Watier Ong: But you could be spending a whole year doing Perplexity, and then their funding gets pulled.
[00:50:05] Geoff Livingston: Yeah, I mean, I, I feel like Anthropic’s taken a good lead from the individual startups, and I feel what they’ve done differently is, really kind of figure out the business thing. They’ve done what Make and Zapier do with… and made it easier- if that makes sense from an automations and agentic AI standpoint, which is a nice word for automations.
Like, I hate that. I hate that whole movement, by the way. The whole agentic AI thing, to me, is just marketing hype. But I think, with OpenAI in particular, they’re a zombie company. I mean, any company that’s asking the government to buy 5% of them before they go public is really a suspicious bailout moment.
I mean-
[00:50:48] Katherine Watier Ong: Or to develop the data centers. That was one of their other proposals- that the government should develop data centers as some sort of public electrical system sort of idea.
[00:50:56] Geoff Livingston: Yeah. Like really horrific thinking. Yeah.
[00:50:59] Katherine Watier Ong: Yeah, horrific thinking. That’s the best way of talking about… Also, we won’t get into how crazy the founder is, but, like, yes, horrific thinking.
And then I also wonder with all these protests around data centers, which I- by the way, I think people should protest data centers. I wouldn’t want one in my backyard.
[00:51:13] Geoff Livingston: It’s your neighborhood, yeah.
[00:51:14] Katherine Watier Ong: Right? I wouldn’t want… I mean, the vibrations are supposedly, like, killing animals and ruining your groundwater and causing health issues.
[00:51:24] Geoff Livingston: Much less the energy draw.
[00:51:26] Katherine Watier Ong: Yeah. I think people should be protesting that stuff, and if that means we slow down six different companies, I don’t know.
[00:51:34] Geoff Livingston: Because it’s a, it’s also a wasteful approach. I mean, the large language model approach is wasteful. You don’t need all that knowledge for SEO, for example.
You probably need a model that’s trained on SEO, or maybe on digital media in general at most, right? It doesn’t need to know about medicine. It doesn’t need to know about running. It doesn’t need to know about cooking. It doesn’t need to know about raising children or what Shakespeare thought when he wrote Midsummer Night’s Tale.
I mean, all that stuff, like, everything that’s in there, it’s just ridiculous the amount of knowledge that’s in there for a very specific domain set of knowledge. That’s the problem. The real future is in small language models, if we’re even using language models.
[00:52:20] Katherine Watier Ong: Ah, that’s the last question, like, if we’re even going to use this approach, hence my 2032 question, like-
[00:52:27] Geoff Livingston: Right. I mean, like, really what makes an AI valuable for you is that it understands your context, your domain, and can provide predictable answers that are accurate based on that data. We’re talking about data. We’re not talking about how the algorithm processes it.
[00:52:48] Katherine Watier Ong: This is why I love NotebookLM.
I use it more than anything else.
[00:52:51] Geoff Livingston: Yep, ‘
[00:52:52] Katherine Watier Ong: cause you can control the data. Because I can upload sources. I can control the data. You’re right.
[00:52:55] Geoff Livingston: Yeah.
[00:52:55] Katherine Watier Ong: Totally. But I think, just generally, as, like, a society globally, we should be thinking about: do we… Is this helpful, given the hurt it’s causing, right? Like, the amount of damage.
Do we all collectively think it’s worth it? A high school intern that you have to sit on?
[00:53:14] Geoff Livingston: Clearly there’s enough economic impact that it’s… I don’t think it’s going away.
[00:53:16] Katherine Watier Ong: Yeah.
[00:53:17] Geoff Livingston: I don’t think it’s going away, but I do think it needs to evolve, and I do think it, great. Let’s go public. Okay, you boys wanna make your money, let’s go make your money and watch it bomb.
[00:53:26] Katherine Watier Ong: Yeah. I know. C- that comes back to my Google and Bing comment. Like, I think if you’re really gonna, you know, double down, double down on the ones that have got the big purses.
[00:53:34] Geoff Livingston: Nobody talks about Google being number two right now. OpenAI has less than 50% market share for the first time ever. Google’s number two, Anthropic’s number three.
And the reason why Google’s number two is ’cause they have it integrated everywhere, including search, and they understand the consumer. They are the ones that are gonna win the consumer game, no question.
[00:53:55] Katherine Watier Ong: Yeah, they’ve been watching all of us in a Chrome browser for decades. They have so much data on us; people don’t realize.
[00:54:01] Geoff Livingston: And we, we hate them less than Meta, so let’s, let’s go Google.
[00:54:03] Katherine Watier Ong: That’s true.
Well, that actually was, like… My last question was kind of like, what do you predict is gonna happen with AI? And then, before we finish up, can you share whether you have an AI personal skill assessment that you mention in your book, or if there are a couple for people who wanna actually self-assess? Like, “Hey, I’m new to this AI.
I real- I guess I really should get going on using it.”
[00:54:26] Geoff Livingston: Yeah. So those are in the appendix. I, I will, I won’t lie, I can’t recite them off the top of my lungs.
[00:54:34] Katherine Watier Ong: Oh, that’s okay, I can add them to the show notes.
[00:54:36] Geoff Livingston: Sorry, folks.
[00:54:37] Katherine Watier Ong: But just so you know, there are, like, formal skill assessments where you can figure out where you are.
[00:54:41] Geoff Livingston: There are, including the Cognitive Path AI maturity model from my prior company, and you can find that online. And that’s, that’s very useful if you’re in this company. Of course I promoted my own, right?
[00:54:53] Katherine Watier Ong: Yeah. But the big takeaway is it’s not. It’s not gonna disappear tomorrow. And that everybody probably needs to know how to use it.
With the caveat being that hallucinates.
[00:55:04] Geoff Livingston: I, I… You know, to me it’s like the cloud. I have to be honest: if I were to predict the future, it’s like the cloud to me. Nobody’s gonna give a shit whether it’s AI in five years. We’re just gonna care about the quality of what we’re given and whether it’s accurate.
And if you can’t give me something that’s good and accurate, then I don’t want it. And, and I think that’s where we’re gonna get focused with what companies are doing. I don’t care how you cook the sausage. I don’t care whether you host your software program in the cloud or on a server in your basement, but if I can’t use it, it’s a waste of my time.
Oh, you mean I have to host it on my computer to use it? Fine. Give it to me.
[00:55:45] Katherine Watier Ong: Yeah.
[00:55:45] Geoff Livingston: You see my point? And like-
[00:55:46] Katherine Watier Ong: Yeah, yeah,
[00:55:47] Geoff Livingston: Yeah… who cares how it’s done? I mean, just make sure it’s done right and that I’m not violated as a result. Thank you. Yeah. Thank you, for example, for not wiping out my hard drive. I mean, that kind of thing.
It’s like that’s what we want, and the sooner we get to the point, like, I don’t care that you’re talking about AI. I care about whether my needs are met.
[00:56:11] Katherine Watier Ong: Yeah. Right. No, that makes a ton of sense.
[00:56:15] Geoff Livingston: Yeah.
[00:56:15] Katherine Watier Ong: Okay. So, uh, thank you for taking all this time with me, but do you have any, uh, additional resource that you want to share with people that you think would be helpful?
[00:56:23] Geoff Livingston: I mean, if you’re really into this and you’re not on Substack looking at different things, I feel like Substack’s where the real conversation’s happening on AI. When I see it on, and don’t get me wrong, there’s a fair amount of drama there too, but when I see it in the social networks as we talked about, it’s just not real to me.
It- to me, that’s just like people that are promoting their business or trying to look good. But there are some real heavy dialogues. There’s a guy there named Gary Marcus. Have you read him?
[00:56:53] Katherine Watier Ong: Mm-mm, not yet.
[00:56:54] Geoff Livingston: He’s like the anti-AI hype guy. He’s basically punching holes in it all the time. It’s-
[00:56:59] Katherine Watier Ong: Oh, fascinating
[00:57:00] Geoff Livingston: And he’s a data scientist, right? It’s am- amazing, very popular. And, he knows everybody in Silicon Valley and just, like, constant- he hates Elon Musk, constantly trashes Elon, hates Sam Altman, but really airs out some dirty laundry that’s really interesting. I kind of look at him like the hype-y anti-hype guy.
[00:57:21] Katherine Watier Ong: Mm-hmm.
[00:57:21] Geoff Livingston: But there’s stuff like that so you get different points of view; that’s really good. I, I always tell people to start with Gary and then find the middle ground.
[00:57:28] Katherine Watier Ong: Ah. That’s a great tip, especially since we all have our filter bubble, so actually seeking out the opposite side is, I think, essential. How, how can people learn more about you?
[00:57:38] Geoff Livingston: Generativebuzz, spelled like you think it would be with a G-E-N, et cetera, .ai.
[00:57:46] Katherine Watier Ong: Oh, cool.
[00:57:47] Geoff Livingston: Yeah.
[00:57:47] Katherine Watier Ong: Thank you so much. This was, like, totally fun chit-chatting with you about all this. Thanks so much for listening. To find out more about the podcast and what we’re up to, go to digitalmarketingvictories.com.
And if you like what you heard, subscribe to us on iTunes or wherever you get your podcasts. Rate us, comment, and share the podcast, please. I’m always looking for new ideas, topics, and guests. Email us at digitalmarketingvictories@gmail.com or DM us on Twitter @dmvictories. Thanks for listening.
Leave a Reply