A History of Marketing / Episode 52
Gian Fulgoni has spent 50 years as a pioneer in market research and audience measurement. From his work on scanner data at IRI in the 1970s to co-founding Comscore in 1999, Gian helped invent how marketing gets measured, first in supermarkets and then on the internet.
His career sits at the center of two transformations that reshaped the field. At IRI, he helped pioneer the use of supermarket scanner data and built one of the earliest controlled experiments in television advertising, a system that could send different ads to different households in real time, in 1979. Two decades later, he co-founded Comscore to bring that same measurement rigor to the chaos of the early internet, building the panels and tools that defined how digital audiences and e-commerce got counted.
Gian has lived through every major shift in modern marketing measurement, and he’s candid about what went wrong along the way. He has watched the industry get seduced by metrics that are easy to capture but don’t actually measure whether advertising works.
In this conversation, we cover:
Why digital marketing metrics like click-through rates and ROAS are misleading, and why the industry keeps using them anyway
How scanner data accidentally flipped CPG spending from advertising to promotion and handed power to retailers
Why data shows that creative is the biggest driver of advertising effectiveness, and why the industry keeps ignoring that lesson
What the dot-com era might tell us about today’s AI revolution
Listen to the podcast: Spotify / Apple Podcasts
Special thanks to Xiaoying Feng, a Marketing Ph.D. Candidate at Syracuse, for reviewing and editing transcripts for accuracy and clarity. And to Tod Johnson, whom you may remember from episode 51 of this podcast, for introducing me to Gian.
Andrew Mitrak: Gian Fulgoni, welcome to A History of Marketing.
Gian Fulgoni: Well thank you. Thanks for the invitation to be here today.
Andrew Mitrak: I want to start right at the beginning. You studied marketing in London and then moved to Pittsburgh to work in marketing. How was the marketing scene different between the UK and the US?
Gian Fulgoni: Well, you know, marketing was kind of viewed as having originated in the US, but that’s really not the issue that I was focused on. So my undergraduate degree is in physics. Right? And while I might have been good at it in high school, it was like going from the minor leagues to major league baseball when I got to university. I had no competitive advantage in physics. I was trying to figure out what to do next, and it was the beginning of marketing, actually, in the US and certainly in the UK. I did some research and realized that marketing might be a good place for me to be.
I did, I think, anticipate correctly that data and computers and the like, analytics, would become more important in marketing as time went by, which kind of reinforced my decision to major in marketing. I got a master’s degree in it. Then I got offered out of the blue. I got a job while I was still at school that took me to Pittsburgh, and it was a company named Management Science Associates that was started by a professor out of Carnegie Mellon who wanted to do research on things he was interested in. He started a company that was focused on analyzing data, basically. Processing and analyzing data. And that’s where I ended up.
Is Marketing a Uniquely American Discipline?
Andrew Mitrak: I want to follow up on, you said that it seemed like marketing had originated as more of an American field. It’s something that on this podcast I’ve actually encountered. Like I’ve talked to Phil Kotler, who is often called the father of modern marketing, and he kind of says that marketing is uniquely American or comes from an American tradition. And I’ve talked to folks though from abroad who reject that or they push back on that, and it’s just sort of like a North American bias. So it’s interesting as somebody who was in the UK, you kind of perceived it that way. Can you speak to that?
Gian Fulgoni: Yeah, I mean there’s no question in my mind. There’s no question in my mind. For example, where I got my master’s was the only university in the UK that had a master’s degree in marketing. That was in 1969. I mean, you could get a master’s degree in marketing in a bunch of universities at that point in time in the US. There were only two MBA programs in the UK at Manchester and London. You know, you had dozens of them. So, if you look at all of the people who pioneered marketing, they’re really from the United States. So I don’t think there’s any question that the US was ahead at that point in time and maybe to this day is still ahead.
Andrew Mitrak: So did you go into marketing knowing you wanted to go to the US eventually?
Gian Fulgoni: No. No, it was, I had done some research, talked to some other people who were going on to MBA programs when I was in my undergraduate final year. And that’s what I decided that marketing looked really interesting. As I said, I think I anticipated the data and analytics, computers, would become more important there. But I had no idea, no intention of coming to the US. It was when the job offer came along that I suddenly thought, man, this is the opportunity of a lifetime. I gotta do this.
The Early Adoption of Computers and Data in Marketing
Andrew Mitrak: You were really early to computers and data in marketing. Marketing as a field in the UK was early, and then attaching computers and data onto it. How did you make that connection initially?
Gian Fulgoni: I think in large part it was because the company I worked for, Management Science Associates, their business was helping companies use whatever marketing data they had. And that would involve taking raw data, if you will, and processing it, analyzing it, whatever data it was. It could have been panels of consumers, back in those days it was diary panels. Or it could have been shipment data that companies had, or it could have been Nielsen audit data, or another database was SAMI warehouse withdrawal data, or whatever data they had. And so I was able to learn the basics of what was available as data, how to process it, analyze it, how to improve it, and I think started to get a feel for what was not available that maybe could be a home run if it became available.
Riding the Technology Wave in Market Research
Andrew Mitrak: It strikes me, this is a little bit of an odd question, but have you seen the show Mad Men?
Gian Fulgoni: Yes. Yes.
Andrew Mitrak: It strikes me the analogy I was thinking of like people like you who adopted computers early. In that show, there is a character, Harry Crane, who adopted TV, and he became the head of television and sort of rode the wave of TV. And people like you were very early on to computers and data and sort of rode that wave. I feel like marketers who can identify the right technology ride a wave, it can propel you in your career. Do you think of it that way at all, like part of it is timing and finding the right technology and positioning yourself as the expert in it?
Gian Fulgoni: Absolutely. Oh, absolutely. I mean, I have often said I didn’t create any particular technology. I just took advantage of breakthroughs in technology that allowed for the creation of new applications and new products. But I think I did see early on that it just had to evolve, right? Computers, it was pretty clear, were getting faster, at that point bigger by the way, we hadn’t reached the trend when things were getting smaller. But you could see that the data that was becoming available, that was changing. The way that data was being analyzed, things that could be done with data that wasn’t available at the time. I mean, the emergence of scanner data was a great example, because that changed everything in how consumer packaged goods marketers operated. One truth at least that’s evident to me is that data, the availability of data, can change markets fundamentally. And I think there are numerous examples of that in history, if you will, certainly over the past 40 years or so.
The Founding of IRI and the Emergence of Scanner Data
Andrew Mitrak: Can you tell me the story of what led to you founding IRI?
Gian Fulgoni: Yes. Well, I wasn’t the founder. Let me say I was hired shortly after they had started the business. And as it happened, the person that started it, John Malec, had worked at Management Science Associates where I was at, so I knew him. And what they did was pretty amazing, even looking at it today. So basically, scanning was beginning to be installed in supermarkets, but it was nowhere near pervasive. And so what IRI did is they bought the scanners for the retailers in two small cities, Pittsfield, Massachusetts and Marion, Indiana. And they gave the scanners to the retailers with the understanding that the retailers would stock new products as they came along that IRI would bring to them, that they would only supply the data to IRI, and that they would accept an ID card from households who became members of the IRI panel. Alright? So that was the scanning part of it, and that was a breakthrough because then we were able to cover the entire city. And there was no other city in the US that you could measure.
Inventing the “Black Box” for Targeted TV Advertising
Gian Fulgoni: And then they did a second thing that maybe was even more dramatic. They invented a black box that sat on the television set of these households. In these towns, you couldn’t watch over-the-air television, the broadcast signals weren’t strong enough, so you had to have cable. They gave the panelists, with permission, a box that allowed IRI to change the television advertising in real-time without you knowing when that was occurring. Okay, so it was targeted advertising in 1979. Crazy to think about it today, right? And so what we could do is we could send one advertising campaign to one half of the panel and a different campaign to the other half then read the impact by looking at the scanner data and seeing what they were buying. It was a home run! … In 1979.
Andrew Mitrak: I have to ask. So this black box, how did that work? Did they detect there was a commercial break and change advertising was it just like almost any broadcasting?
Gian Fulgoni: Technically they way it worked is that we had an agreement with the cable company and so we had a studio at the head end of the cable system and so we knew when the ad was coming down that we had to change. So if we wanted to reduce ad spending, we would substitute a commercial for anti-smoking or something like that, or otherwise we would be able to ride the ad over another ad that the client owned. Okay? And then the box, we would direct the boxes that we wanted to see advertising A to get A, and the other ones would get B. But it all happened in real time. So if you were a panelist, you gave them permission for this to occur, but you were never aware of when it was actually happening. And as I said, it was a home run in terms of its ability to measure advertising effectiveness. It was just a beautiful A/B test design. Very profitable business. And we took the company public in 1983. So that was way before scanners had become pervasive. And then there’s a separate story around that as we built out the capability nationally.
The Tension Between Data Measurement and Creative Quality
Andrew Mitrak: It sounds just, that tech sounds way ahead of its time. It’s so ingenious. As you got all this data and you ran these AB tests, were there any myths about consumer behavior that IRI helped debunk or what were the key learnings that you found?
Gian Fulgoni: Yeah, that’s a really good question. So I’ll give you one example. I won’t mention the client, but it was a client that was spending a lot of money on television advertising. And they wanted to see if they could move the needle in sales. So they came to us and they said, we’re going to spend six times what we’re spending, and we want to do that as an experiment. And that was putting the equivalent of a ton of money. And so off we went. We ran the test for six months. I’m looking at the results and there’s no increment in sales. There’s nothing. Right? I’m thinking, oh, this is going to be trouble. I gotta go and present these results. And so before going to the meeting, I was talking to the head of research who had commissioned the study and I said, this is going to be really a problem, isn’t it? And he goes, no, no. And I go, why is that? He goes, well, six times nothing is still nothing. And his point was that the creative had never tested well. And so you put six times the money behind poor creative, and you get nothing. And that was a real learning point for me. Because it pointed to the importance of creative. And then we did other work that confirmed it. That basically, and others have done the same thing, it basically points to the fact that typically two-thirds to 70% of an advertising campaign’s impact is because of the creative, not the money that you’re spending or the media plan. And I think that lesson is kind of being forgotten today. It’s all become a question of how many impressions you shove at people.
Andrew Mitrak: I feel like sometimes, and I’m generalizing here, the creative people are almost at odds with the data people, or that they feel like they’re in different camps of marketing. But in this case, your data was supporting the importance of creative, and it’s sort of a better together type story.
Gian Fulgoni: Well, yes, except for the creatives who wrote the creative that we just tested. They probably didn’t like the results. Yeah, I mean, there’s always been that tension, and I suspect it’s probably still there. I mean, there’s just this feeling that they don’t like being measured. I don’t know whether it’s fear of failure, if you will, or just the ego saying, “Well, look, I came up with this great piece of creative. Why do we need to measure it?” I don’t know what it is, but I do know that it’s always been an issue. But as I said, I think one of the really troubling aspects of advertising today in the digital world is that we’ve gone to a point where the attention paid to creative is minimal compared to all of this automation that’s going on that I think is just trouble personally, but that’s my view.
The Pitfalls of Modern Digital Advertising and AI Automation
Andrew Mitrak: Yeah, for sure. That people just think, oh, just automate away your creative. It’s like, well okay, then your creative is going to look like everybody else’s.
Gian Fulgoni: Right. Exactly. And now we’re going to use AI to create the commercials. Right? And AI is going to decide where it’s going to run. And then we’re going to use synthetic panels that are AI-built to figure out the response. I mean, it just seems idiotic to me. But I think I know the drivers of this. I think the drivers are the technologies available, and there’s a need for speed and low cost. And that overwhelms sensibility sometimes.
Andrew Mitrak: Have you seen that story play out before at all? Or does it seem uniquely different? Like, you’ve gone through a handful of technology shifts and a lot of excitement around a thing where “this will change everything.” And of course, it does change a lot, or things change a lot, but maybe the changes might be misunderstood. Do you feel like this feels like a familiar moment to you, or does it feel like something different?
Gian Fulgoni: No, it does. It feels like an exaggeration of the availability of technology and maybe a lack of focus on, it seems to be technology’s ability to drive the metrics and the system, but nobody seems to be paying enough attention to whether those metrics are really meaningful or not, right? Or how to deal with them. I can talk a lot about that in a second, but I’ll give you one vivid example.
The Unintended Consequences of Scanner Data on Market Share
Gian Fulgoni: And it’s scanners, right? So at the time that scanners became available broadly in the US, if you were in the consumer packaged goods world, the only way to know what was selling at retail was Nielsen’s manual audits. So Nielsen would have 10,000 people or something like that around the country who would manually audit stores. They’d go in, they’d look at what was stored in the back room in the store, they’d get the invoices for what was ordered from the warehouse, and derive what was sold over a two-month period. And so marketers were using two-month data points. Alright? Now, if you use two-month data points, everything seems to smooth out, and it becomes, the lines of market share seem to look flat. There’s not a lot of variability in it. Well, imagine when one day scanning data became available, and it was measuring weekly sales movement. All of a sudden you saw those weekly sales responses that were hidden in bimonthly data. Right? And it was caused by the promotions that were running. Promotions in packaged goods had been running week to week. A manufacturer would pay a retailer to lower the price for a week, they’d get an in-store display, they’d run an ad in the newspaper, and then next week it would be another different manufacturer, probably in the same category doing it, right? Well, if the only database you had to measure the impact was two months of data, and these events were running week to week, you could never figure out what was really happening. And then scanning came along, and it was dramatic because in a flat world where change doesn’t seem to be occurring, advertising thrived. And typically, two-thirds of marketing spending in packaged goods went to advertising, television predominantly, and about a third went to promotion. When scanning came along, it reversed itself really, really quickly. Because all of a sudden the retailers could see what was happening, and the retailers had such power that they were able to demand more promotions. And the manufacturers had to go along with it. And suddenly all of their spending started shifting to promotion. And it went from 70-30 in favor of advertising to 70-30 in favor of promotion. And I’m not sure that enough thought was given to the impact on a company’s financials. And the retailers became much more powerful to this day.
The Short-Term vs. Long-Term Debate
Andrew Mitrak: As you do promotions, there’s sort of this race to the bottom, and then that led to this whole brand equity thing too, right, where companies started to say, “Well, you gotta, you can’t just measure sales week to week, you gotta measure your overall brand equity, and if these promotions are hurting it...”
Gian Fulgoni: Yeah, I mean it’s a real issue. I mean, race to the bottom, you know, is maybe a little bit of an exaggeration, but if price becomes the driver of people’s buying behavior, that’s not particularly healthy, I would think, you know, speaking as a marketer. So, speed was a big driver of it, and the data, the ability to understand what was going on. And you know, if you then fast forward through the years, things get faster and faster, and the ability to run promotions becomes faster and faster. And if you were to say to a marketer or a CMO or a CEO today, “Hey, I want you to run a campaign, an advertising campaign, but it’s going to be a branding campaign, I’m not going to be able to give you the results for six months.” They’d laugh you out of the room. Right? I mean, I can’t wait that long. I gotta know within a week or within a day what’s happening. And the moment you have that perspective, you know, I think you fall into the trap of fast digital marketing with metrics that are available but may not mean anything at all. And so I kind of trace it, you know, the evolution, at least in my lifetime, from the availability of scanner data through to today when digital marketing is pervasive, and the next step is going to be what does AI do to all of this.
How Scanner Data Favored Promotions over Creative Advertising
Andrew Mitrak: That’s really interesting. I don’t know if I’ve heard it articulated that way, that the scanner, of course I was aware that scanners were related to promotions, but that scanners were almost the causality of accelerating promotions and amplifying promotions, and that because this is something that’s measurable, you kind of over-index on this channel because that is what’s being measured. Right. And it’s just interesting, you’re somebody who’s made your career in this form of analytics, and just kind of hearing that you’re able to reflect on some of the unintended consequences or it kind of being taken the wrong way or going too far, is just kind of an interesting reflection.
Gian Fulgoni: Well, I mean, maybe I had too negative of a view of it. I mean, when all of this reality hit home, a new area of analytics emerged, which was trade deal evaluation, price promotion strategies. And the big consulting companies took it over, right? It wasn’t the market research companies that owned that. I mean, the charge for those big studies was way, way more than we would ever charge. And you had to have the credibility of McKinsey or Bain to be able to get away with it. But it did drive a lot of thinking about, you know, how do you run these promotions, you know, with these retailers, and, you know, what should your price strategy be, etc., etc.
The Early Days of E-commerce Measurement
Andrew Mitrak: So, you were early to adopting computers and data with marketing. When did the internet come on your radar? And what was your initial encounter with the internet? And as somebody who’s an early adopter of technology, what was your initial impression of, “oh, this is going to be big, this is going to be important for marketing”?
Gian Fulgoni: Yeah, I was probably not as early as I should have been in the impact of the internet. I was invited to join the board of a company called Yesmail that was doing email marketing, basically. Around 1999, it became clear that the internet was accelerating, I mean, we didn’t anticipate the dot-com bubble at that point, for sure, and that there was the opportunity to kind of replicate what we had done at IRI on the internet. Alright? And my business partner and I, who had been president of IRI, and you know, when I was CEO there, a gentleman named Magid Abraham and I started Comscore. And the idea was, there were two aspects to it. One was e-commerce was emerging, why don’t we build a system to measure e-commerce trends across all categories, not just consumer packaged goods? And as a byproduct of that, we’ll be able to measure the effectiveness of digital advertising on online sales. Easy to say, not too easy to build. There’s no UPC code. So we had to develop all these screen-scraping technologies to just pull the data off from the computer screen. And then you had to have massive panels of people. And we had to figure out how do you recruit 2 million people who will let you, who give you permission to put our measurement software on their machines. So we figured that out and started measuring e-commerce, and we were the only independent company doing it. So we’d get invited onto Squawk Box or Squawk on the Street, and during the holidays, Black Friday, we would always be on predicting what was or reporting on, you know, what had happened. Um, so you know, it worked out well, and we started measuring advertising effectiveness around the same time.
The Dot-Com Frenzy and 24-Hour Financial News Networks
Andrew Mitrak: So Squawk Box, was that CNBC or like Fox Business?
Gian Fulgoni: It was CNBC.
Andrew Mitrak: And this is like the rise of CNBC as well, as another part of the internet dot-com era, is that 24-hour news is a thing, and then all of a sudden 24-hour financial news and stock markets. And this is when people kind of, I feel like, an early wave of consumers sort of getting into the market and trading and all that. So you found Comscore, and you’re kind of a thought leader who’d appear on those types of shows?
Gian Fulgoni: Yeah, we always looked for that. I think one of the things we did pretty well was marketing ourselves. And you know, we knew that if we could get onto Squawk Box first thing in the morning when a lot of stock analysts and traders were watching it, and you know, eventually we were public, that’s going to be good for the business. But this was in the early days, this was like 2001, 2002. We went public in 2007. So this was before we went public, but we got the publicity, and you know, that just helped the business, that helped the customer base, if you will. We started getting a lot of inquiries from companies wanting to buy our information and so forth.
Bringing Market Research Rigor to the Wild West of Digital
Andrew Mitrak: Let’s go back to founding Comscore. Something that I’ve observed is that when a new technology paradigm emerges, often people kind of bring things in from the previous paradigm. Like, if you look at the very early movies, for instance, like the silent movies, they almost film it like it’s a play. A lot of movies just sort of point a camera at the stage. And then sometimes you see this with tech too, where early internet advertising kind of looks like newspaper advertising. It doesn’t sort of look like internet native, per se. Were there any things as you were figuring out and building Comscore that you’re bringing from IRI that did or didn’t translate into this sort of new paradigm?
Gian Fulgoni: Probably more in the area of the underlying technology than anything else. What I mean is, as opposed to us figuring out how marketing was going to evolve. I don’t think it was that so much, but we certainly did understand the importance of scale in our data collection platform. That it had to be scalable in a massive way because it was clear that we would need to measure more and more and more things. And remember, at the beginning there was no social media, there was no search, there was no video. All of those things emerged later, but we had to make sure that the basic technological infrastructure was able to handle it. We did also though, um, bring over analytical methods that had worked for us. I mean, we knew the importance of measuring individual consumer behavior at the individual consumer level and then aggregating it up. We also knew the importance of A/B testing. We understood clearly the difference between causality and correlation, which I still think haunts digital marketing to this day. So, you know, we built out a company that used a lot of what worked for us at IRI in terms of technology and analytical approaches. And you know, how do we market the company? And what’s the best way to build it and grow it, and what kind of people do we need and where we’re going to get them.
The Comscore Competitor Landscape
Andrew Mitrak: It’s an interesting story of a dot-com era company, that’s a startup being founded by veterans of an industry. You’re well into your career at this point, you have this long track record at IRI. Were there any competitors that were young, fresh out of college startups who didn’t really, maybe they knew the tech but they didn’t understand the fundamentals of marketing and running an analytics business?
Gian Fulgoni: Uh, that’s a really good question. I don’t think so. I mean, the two competitors, the two competitors I can remember, you know, was NPD and Media Metrix [See Tod Johnson episode to learn more about NPD and Media Metrix], right, who were the first to measure audiences. And then Nielsen was doing it. And neither of those companies were being headed by young executives. So I don’t think I, I can’t maybe I’m not remembering accurately, but if you were to say to me, so who were the competitors you were worried about at that point in time? It was Nielsen and Media Metrix, you know, no question. So that’s an interesting question. I’m wondering what the implication of that is.
Andrew Mitrak: It’s an interesting example because usually, very often when a new technology emerges, it paves the way for some new disruptor or somebody who’s kind of native to that category to have an opportunity. But it sounds like sort of the bigger, more established players, I guess Comscore was a new company, it wasn’t part of IRI, right? But it was founded by people who are veterans.
Gian Fulgoni: Yeah, Comscore was really new. You know what it could be, Andrew, as I think about it, is that where the young kids were going was into the, they were forming the companies that were executing things online, whether it was e-commerce or advertising or whatever. And the whole industry was still young and relatively small. And so the need to measure some of these things maybe wasn’t as obvious as it was to us, where we came from legacy businesses that were pretty big. Right? And maybe we were anticipating that as e-commerce got bigger and as digital advertising got bigger, the need for these measurements would just become more and more important.
How Cookies Inflated Metrics
Gian Fulgoni: Now, that’s not to say that there wasn’t a need for certain data elements at that point in time. And the one I will point to probably more than anything else is audience measurement. Alright? That was there because to get the advertising dollars, you had to show what, you know, how many people you were reaching. And that led to a big issue. A big, big issue. Because most of the website operators, the publishers let me call them, right, whether it was a newspaper or an e-commerce site or whatever, but you know, they wanted to get advertising dollars so they needed to show the size of how many people were visiting their website.
And so they would turn to their analytics, their website analytics. Could have been, I don’t know if Google Analytics was around at that point, but there were a bunch of different, I think Adobe had some. And the problem that we highlighted is cookie deletion. And so all of these analytical tools running on the website were basically counting unique cookies. Well, if the cookie, if Jack Smith visits the website, goes off, deletes his cookies, and comes back, he’s going to be counted as a new visitor. And the cookie deletion was prevalent. And so the audience counts were grossly inflated. And we came along with much, much lower numbers. And so the arrows were being fired at us at that point, but you know, I think we did a pretty good job of clarifying that issue and pointing to what the real numbers were.
Andrew Mitrak: Interesting. So the publishers have an incentive, because they want to charge more money to advertisers, to sort of raise their profile and say they have more numbers than they do. They kind of have an incentive to sort of overlook these cookie issues or other things. And you’re kind of coming in as a more neutral third party who’s calling balls and strikes and saying, “Ah, this is a different approach.”
Gian Fulgoni: I’m not saying that they were doing this on purpose knowingly, that their counts were inflated. I mean, a lot of them didn’t. Nobody realized it. Nobody understood the degree of cookie deletion. I mean, we at Comscore produced reports that showed how often these cookies were getting deleted. Right? And then what did that mean to the overstatement of the audience. But you know, not everybody liked to believe it, and so it became a kind of a rough time, if you will, because the advertisers wanted to know what the true numbers were. The publishers wanted the highest number possible. I mean, no surprise there, right?
Andrew Mitrak: Yeah, and at the time also, this is, I was probably a teenager or something like that at the time, using the internet, and if I recall on your browser, it was sort of just a hygiene thing. Like, “Oh, it’s running slow,” “Oh, whatever, delete your cookies, refresh.” And it was sort of just like a thing that you’d do. A listener who’s younger might not delete their cookies every day now, or cookies don’t really even exist anymore, right? So in some ways you’re vindicated about that. But it was just a behavior that you’d do, almost like defragging your computer, which you used to do or something like that. You just sort of like delete your cookies more often.
Gian Fulgoni: Yeah, but you were also, what you also needed to wipe off was malware that could have been on your machine. And so the systems you were using would do both. I mean, it would just clean, you know, very few people would say, “Well, delete the malware but leave the cookies on.” Right? And so everything got wiped off, and you could, as I recall, you could delete it as frequently as you wanted. And so we had these charts developed that showed according to the frequency of deletion how much the overstatement was of the visitor counts.
The Flawed Fixation on Click-Through Rates
Andrew Mitrak: Interesting. So something within this is that you kind of pivoted from doing more e-commerce tracking to doing audience measurement, right? Was this sort of a deliberate pivot for you?
Gian Fulgoni: Well, we always, so let me separate advertising measurement into two buckets, if you will. There’s the audience measurement part, and then there’s the effectiveness measurement part. We always were after the effectiveness measurement part. Because we had the sales numbers so we could do it. So we were doing that from the beginning. In terms of the market for the straight, you know, counts of how many people were visiting a website, that, the importance of that, I think emerged for us a little later. And you know, that’s when we went, you know, full tilt and introduced our product, which was built to scale and it was very successful. But yeah, I mean, we did know that there were multiple market segments that would emerge if we could capture the data, the variables for all of the elements that we were interested in. What we didn’t anticipate is the degree to which these computers were spitting out metrics that were irrelevant. But if you wanted to ignore that and leverage them, you could do that, and I’m heading straight down the path of identifying the click.
Andrew Mitrak: Yeah, click-through rates.
Gian Fulgoni: Pioneering work on that. I personally did a lot of it, where we basically showed that in a controlled test, so we’d look at click rates and then we’d look at the advertising effectiveness measured by an A/B test. There was no correlation. No correlation.
The “Punch the Monkey” Era and Misaligned Metrics
Andrew Mitrak: And it’s funny from this era as well, the advertisement that I have sort of a vivid memory of like on the internet was this “Punch the Monkey” thing where like it was a banner ad where like a monkey would dance around and it’d say “Punch the monkey,” and if you move your click over it or mouse over it, it says you just want to like click the monkey with something. And the trick of the ad is that anywhere you click on it, it works, whether you hit the monkey or not. But it’s just like, I don’t even remember what it was advertising. It was probably some scam or something like that, but it’s like it was everywhere at the time. And it was just purely designed to click on it. And it’s like, and they’re just like amping up clicks.
And I think a lot of ads that were display ads or banner ads that we now almost think of as like a type of like a brand advertising and not necessarily optimized for a click in the same way, at the time, you’re sort of forcing those into clicks because that’s what got measured. And so it’s just sort of like a lousy ad that was sort of hacky. And so people kind of played games like that.
Gian Fulgoni: Yeah, well if you were getting paid per click, you know, you’d want more clicks. Right? The problem was if you looked across the internet, the click rates were one in a thousand on average, right? And yet they were being used to evaluate the effectiveness of ad campaigns. I mean, the numbers were so small they were ridiculous, but then we provided more than enough evidence that it was irrelevant. Right? And here’s the problem, right? We could never get the entire industry to stop using clicks. Because they’re fast, and they don’t cost anything to produce. And I remember doing surveys of the industry, and I asked advertisers, publishers, and agencies, “How often do you use click rates to evaluate the effectiveness of ad campaigns?” A third of them said always. And this was after all this research had been published.
The Overstatement of Digital Advertising’s Impact
Gian Fulgoni: And I think if you fast forward now, you know, years later to look at what’s going on with digital marketing, there’s something really wrong with it, in my opinion and others, that the metrics that are being used are completely wrong, and they’re not measuring advertising effectiveness. It’s been said, you know, if you were to take and believe all of the studies that have been published about the effectiveness of digital advertising, the size of the US GDP would be double what it is today. And I think it’s a valid point. Not everything that can be measured matters. I forget who said that, it might have been Einstein, but it’s true. And unfortunately, in a digital world, these computers are throwing out digital markers all the time of likes and interest and clicks. And then the... I might be old enough to know the way that one should evaluate advertising effectiveness and the difference between causality and correlation, but I think the industry has gone crazy for these metrics that just don’t seem to be doing what they think they do.
So I’ll give you an example. There’s a metric called ROAS, return on ad spend, that you might be familiar with. Right? Well, when you look at “return” on ad spend, it suggests it’s measuring the effectiveness, right? And the incremental lift you’re getting, right? Well, there are so many stories circulating where a CMO goes into the CEO and says, “Oh, great news, you know, the ROAS on our campaign is fantastic, here it is, it’s 4.6.” And the CEO goes, “Well then you explain to me why our sales aren’t growing.” We are in my opinion, in a phase where the effectiveness of digital marketing has been grossly overstated. I’ll go on record as saying that. And I think the drivers are you got so many metrics that are available and it doesn’t cost anything to use them. And you need the results quickly. So speed is another driver. And you end up using approaches that don’t necessarily do what you think they do.
Andrew Mitrak: Do you think on net it’s overstated more because of bad practices and waste dragging it down or do you think it’s overstated because there are fundamental limits to how effective it can be?
Gian Fulgoni: That’s a really good question. I would say both. I’d say both, and I would add a comment on the second one, which is creative doesn’t seem to be getting a lot of attention in digital marketing these days, right? And if one believes that an AI system can create the kind of creative you want, good luck with that one. I mean, as you said, I mean, if one believes that then all the advertising that’s coming out is going to be the same, everybody’s going to be using AI applications. An interesting thought is one AI application going to be better than another one at creative things.
Andrew Mitrak: Yeah, for sure. It’s silly. There’s this Disney Pixar movie called The Incredibles. And there’s this villain in it, is this guy named Syndrome. And one of the things he says is he wants to give everybody superpowers. And he says, “If everybody has superpowers, nobody has superpowers.” And I kind of think of that with AI sometimes, of like if everybody has an equally capable AI to do all their stuff, well then how do you stand out in that world? Right? And play that. And I do think that the answer is actually that there is sort of a human element plus AI has to be part of the equation, that some “what are the things that I could uniquely make with AI, or AI could not make uniquely without me,” and then having some human touch. I just, maybe it’s just me being too sentimental for the value of humans, but I think that human touch has to be a part of it.
Gian Fulgoni: I think you’re right. I hope you’re right. You know, and maybe, maybe it’s the prompting that’s going to be the solution there. Or maybe some of the other data that you need to collect on your consumers. I think that’s the other thing that seems to be slipping is, you know, “Well, how much do you know about your buyers as human beings?” Right? You know, and that kind of leads to this whole thing of “are we going to get AI doing automatic buying for consumers?” There’s a big split on that today, right? Some people say it’s crazy. I’m probably in that bucket. You know, that maybe it’s because I’ve just been around buying measurement for so long that I know that it’s not as rational as we’d like to think it is. It’s not. I mean, people don’t, you know, a lot of things affect the decisions people make about what they’re going to buy. You know, so I find it hard to believe that an AI, you know, ultra-logical system is somehow going to emerge as the way that the average consumer buys. That’s just me.
Andrew Mitrak: Yeah, it’s funny. I am sort of in both worlds. I definitely am, I think it’s actually really useful to look at history and look at big trends and see what happened before. And then I am also like, I am very bullish on AI and a lot of things. But also, I don’t want to get over my skis, and I want to learn from things. Like you went through the dot-com crash, and it’s not to say that the internet wasn’t important, the internet was really important, but maybe it just got a little ahead over its skis. And I’m wondering, like, if you think of that era, are there lessons you draw from that? As somebody who navigated that, still had a successful business after that, probably went through some pains through it. Do you think of lessons from that era that you kind of have in mind today?
The E-Commerce Bottleneck
Gian Fulgoni: Yeah, no, I do, and I think it’s a good, very good analogy. Because if you look back at the internet and say, “Well, what happened with the bubble?” Well, it got way over its skis, right? The pricing and the valuations were crazy. I mean, you had companies that had no revenues, let alone no profits, were getting these crazy valuations, and then it was all hyped up, maybe inadvertently, but you know, and then it crashed, right? And a whole bunch of companies went out of business. Right? But what was left was a real foundation of technology and infrastructure, I think, that was then built on by the companies that either survived it or that then came along. And it became stronger than ever. And you know, if you look at, if you had bought a bunch of these stocks after they had crashed, you’d make a fortune given what happened to the right ones. I mean, a lot of the companies went out of business. But it is, it’s fascinating to me to see.
And the one example I’ll give you that people point to is Pets.com. “What a stupid idea that was. I don’t know what thought of that.” Well, all you gotta do is look at Chewy today to realize that somebody made a lot of money, you know, on that basic idea. So, I think a lot of the ideas maybe failed because the capital infrastructure needed for them evaporated and so they were gone. But they came back in a different form later. And I think that might well happen with AI. I mean, the amount of money that’s being pumped into it is such that I personally believe it’s got to lead to a whole bunch of companies failing. Right? The question is which are the ones that, you know... some of these applications just seem to me to be way out there.
Advice for Marketers Navigating Tech Revolutions
Andrew Mitrak: We’re looking at it kind of like on the company level or the sector level. Do you have any insights on like at the level of an individual marketer, or somebody who works in marketing and is thinking of like their career ahead? Are there any takeaways from that besides, I guess, looking at which companies will navigate it well and wanting to align yourselves with them, sure, but as a marketer are there any thoughts you have for an individual?
Gian Fulgoni: It’s a great question, Andrew. The one thing I will say that I think has been true in my time on this planet is that if you are running a business, it’s a lot, lot easier to reduce your costs than it is to build your sales. And I would say that my feeling is that I am no doubt that AI is going to reduce costs dramatically for a whole bunch of companies. Now, is it at the same time going to help them sell more? I don’t know about that. Because if everybody has the same capability, I mean, you go back to what we were talking about earlier, right?
What’s the competitive advantage that allows you to sell more than your competitor? I don’t know.
I’ve heard one theory say that, and I think this was referring to companies in the service world, whether it’s consulting companies or auditing companies or whatever, right, where their assets are people. And they could reduce their operating costs pretty significantly using AI, I have no doubt about that. But once that’s done, is everybody then at parity? And what are clients going to say? A client’s going to say, “Well wait a minute, I’m not going to pay you the same amount of money I was paying you before, if you’re now making three times what you were making before.” So, I’m not clear in my mind on the impact of AI on the sales end of industries. I’m very, very bullish on its ability to reduce costs, but I also think there’s been hype about it. You know, maybe I come from the analytical world, but man, we’ve been talking about AI and its analytical contributions for years now. This isn’t, you know, like the past two years, and I’m still waiting to see more evidence of these breakthrough analytics that weren’t possible before AI. I mean, I’ve heard of a few, but nowhere near what the hype would suggest. I am worried that the purity of the analytical methods that I kind of grew up with has been kind of laid by the wayside today. And we are just dealing with a lot of hype and a lot of exaggeration.
By the way, one other thing that I will go back to, just go back to the internet, that I think is important to realize that I didn’t comment on. If you look at the financial results of internet companies, you have to separate them into e-commerce companies and then advertising-related companies. The advertising ones are making a fortune, okay, the ones that survived. Maybe the newspaper industry was decimated by it, but if you look at the amount of money that’s being made through advertising, it’s mind-boggling, right? If you look at e-commerce, there aren’t many companies who can point to profitable operations if they have to ship products that are in any way, shape, or form heavy. I spent about 20 years on the board of a public e-commerce company that got around that issue called PetMed Express, and we sold pet medications. You had to have a prescription from your vet. But we were shipping products that didn’t weigh much. And so as Amazon forced retailers to go with two-day free shipping, and then, you know, the pressure’s on to do one-day free shipping, we were able to, you know, just survive and actually thrive in it, because the cost of shipping these light prescription products was not a constraint. And you know, maybe that’s another thing that will play out in some way with AI, is that depending upon the type of application of AI, you’re going to be making a lot of money versus not. I’m just kind of throwing that out as a hypothesis.
Anticipating the Next Bottleneck
Andrew Mitrak: That’s right, that you optimize up to the bottleneck, and with the internet and e-commerce, the bottleneck is heavy freight shipping, right? And that’s just a problem that the ones and zeros and digits of the internet can’t kind of solve the atoms and weight and things of the physical world. And that I’m wondering what there are blind spots of the optimization of where are the next bottlenecks, where it gets up to this point and then there’s something, and then how long do these bottlenecks last? It’s kind of an interesting one.
Gian Fulgoni: Yeah. No, that’s really interesting to think about. And I think it is also really valuable to look back and see what one can maybe decipher from the technological shifts that have occurred in the past.
Andrew Mitrak: Gian Fulgoni, I really enjoyed this conversation, and I think listeners will too, so I just want to thank you so much for your time and for your insights. I really had a good time.
Gian Fulgoni: You’re welcome. You’re welcome. Thanks Andrew. I appreciate the invitation.
