Archive for the 'Metrics' Category

Aug 29 2013

Cargo Cult Analytics (best thinking on big data I’ve read in a long time)

Published by under Journalism,Metrics

Former IDG colleague Matt McAllister tweeted a link to a wonderful post by Stijn Debrouwere, a Knight-Mozilla OpenNews fellow who is “loosely affiliated with the Guardian’s data science team in London.”

Debrouwere tackles the futility of newsroom analytics and measurement, something I lived at Forbes.com and IDG and then Lenovo when I ran web analytics, mainly Omniture (now Adobe) and Google Analytics. As Debrouwere sarcastically notes, putting dashboards on big flat panel screens and making them really big makes them really important. He compares media executives who cling to their dashboards to New Guinea primitives waving at the sky and waiting for the cargo to come to them the way it used to come during WW II when the US Army was fighting the war.

Some zingers from his post:

  • “If you’re like most people, you don’t stray very far from the dashboard you get when you log in. You stare and squint and hope insight will magically manifest itself.”
  • “There’s nothing like a dashboard full of data and graphs and trend lines to make us feel like grown ups. Like people who know what they’re doing. So even though we’re not getting any real use out of it, it’s addictive and we can’t stop doing it.”
  • “There’s enough social media analytics tools to merit listicles that helpfully introduce you to the top 8.”
  • “You’re supposed to put these dashboards up on a wall, on a huge plasma screen. Because of course numbers are twice as persuasive if you make them twice as big.”
  • “Metrics are for doing, not staring.”
  • “I honestly can’t recall the last time I’ve looked at our pageviews. I know it wouldn’t get me anywhere.”

http://stdout.be/2013/08/26/cargo-cult-analytics/

This Big Data thing has a lot of people confused, myself included. And for good reason. We think of this big database in the cloud, doing something so big and difficult that it requires lots and lots of processing power and a thing called “Hadoop,” watching individuals interacting like so many ants with companies and stuff in real-time like some scary NSA spooky datacenter in Utah listening to all our phone calls  (but respectful of our privacy of course), figuring out patterns and trends and opportunities and MAGICAL INSIGHTS in REAL-TIME.  So let’s get ourselves some of that there Big Data and save the company, be like Google, A/B test the shit out  of stuff, and get rid of the Highest Paid Person’s opinion, blah, blah blah…..

First, a lot of people, including me, suck at math and statistics and so we overcompensate by regarding any numbers and the word “quantitative” as mystical.  If it has a number attached to it (not an adjective) it must be important.

Second, the legends around Big Data and the magical insights they deliver to retailers are kind of cool to consider and become mythical. Target can tell when a woman is pregnant based on her shopping history. Wal-Mart figured out 80% of its store visitors turn right when they enter the store. The promise of finding one of those awesome insights is just too compelling to miss. The problem is staring at a dashboard doesn’t equate to discovering an insight. Hence a lot of us are like the Cargo Cultists.

Third, modern management is obsessed with measurement — former colleague Lew McCreary called it the Tyranny of Metrics — and the Pokemon Model of Got To Get ‘Em All applies to equating Big Data with Big Data Collection, which yields the ugly phenomenon coined by Google metrics guru Avinash Kaushik: “Data Puking.” The admonition that, “You can’t manage what you don’t measure,” has built a corporate culture more concerned with looking buttoned-up, on the ball, and obsessively accurate than being intuitive, empathetic and innovative.

I was the guy who built these dashboards, peered at them for magical insights, puked them at my bosses, and over time I started to get really cynical and put  tired old quotes pissing on measurement   into my PowerPoint presentations:

Einstein: “Not everything that can be counted counts, and not everything that counts can be counted.”

Warren Buffett:”They studied what was measurable, rather than what was meaningful” – 

I know Debrouwere’s post appealed to me because he was specifically addressing metrics in the newsroom — a place I spent most of my career. But it also struck a current chord with me because of  my work for clients, all of whom cite Big Data incessantly as a force for disruption and transformation, yet haven’t the faintest clue of how to harness it or whom the Oracle will be in their organization who will study the digital tea leaves and come up with the single “AHA!” that will make them Measurement Legends.

 

One response so far

Apr 24 2013

3 Ways to Write an Annoying “ListLine”™

The recently departed Al Neuharth — the man who gave the world McJournalism when he created USA Today in 1982 — was famous in my mind for two things (no, make that three, because this is a post in part about the magic powers of “three”):

  1. Always publish the tits above the fold
  2. Bulleted lists are better than paragraphs
  3. Infographics that twist statistics and invoke the Royal We into cartoons are engaging

People love lists. Decades ago there was a bestseller entitled “The Book of Lists,” a classic toilet-side tome in many a household. There are  management books about the power of to-do lists.  I must have at least three or four list apps on my phones and tablets and PC. Most horrible is the tendency of the lower life forms in online journalism and especially digital marketing/SEO/Content marketing bloggers to use lists as linkbait. There are so many headlines about “Three Ways to Increase ROI” and “Four Ways Content Marketing Can Engage and Delight Your Customers” that I have to wonder what’s driving this obsession with numerical sequence.  I know that if I click through to actually read the stuff I’m going to read some airhead social media/digital marketing “guru’s” rehashed airheaded jargon twisted bloviations.

Working off off my feeds this morning  I found this actual set of … oh hell, let’s just call them “ListLines™“, e.g. headlines promoting lists:

  • 13 Smart Podcasts That Will Feed Your Hunger for Knowledge and Ideas
  • The 45 Best Restaurants in America (BusinessInsider is a huge fan of  ListLines™, generally cutting up the content into slideshows to pump up the pageviews). They have a daily list which is semi-useful called …..
  • Ten Things You Need To Know
  • 10 Habits of Remarkably Charismatic People
  • We Try 4 New Electric Hot Water Kettles for Coffee and Tea

The king of the numbered ListLine has to be the Content Marketing Institute, which on its home page has the following headlines, and all save one has a numeral in it:

  • 4 Truths About Content Marketing Clients
  • 6 Tips to Start Creating Content on Tumblr
  • 3 Tips for More Effective Content Marketing Visuals
  • 9 Questions to Help You Prioritize Content Creation
  • 12 Roles Essential to the Future of Content Marketing
  • Thought Leadership Strategy: 3 Ways to Leverage Live Event Content
  • 3 Tips for Keeping Your Buyer Personas Fresh and Alive
  • How Enterprises Handle B2B Content: 6 Key Insights From Our Research

McKinsey, the organization that lives on PowerPoint, had an unofficial Rule of Threes during my short stint– as in no slide should have more than three bullet points on it because that was all the typical audience member could hold in their head during the time it took the expensive consultant to present the slide. McKinsey was into numerology in general and the place should have had the Pareto Principle inscribed over the door as its motto (the “80/20″ rule). I admit I stick to the Rule of Threes to this day.

My theory about the abuse of the numbered list in online headlines is the corruption of editorial good sense by the scuzzy underworld of Search Engine Optimization and the Tyranny of Metrics. Let’s turn to the experts at the Content Marketing Institute, enter in the search term “lists” and what do you know? In a post entitled “Content Strategy: 9 Secrets for Awesome Blog Post Titles“, Tracy Gold writes in item number 5:

“We all groan about numbered lists in blog posts. But the truth is, they work. In our research, titles that began with a number performed 45 percent better than the average.

“Another approach is to start with a keyword and include a number later in the title. Take “Content Marketing Checklist: 22 To-dos for SlideShare Success,” for example. We tested both title types, and when the headline started with a keyword, it actually performed slightly better.

“While one approach to this method is to work more numbered lists into your blog content strategy up front, you can also use a numbered list in a post after it’s written. Is the post split up into sections? Can those sections be numbered? Boom. But again, don’t mislead your readers — make sure a numbered list format actually fits the content of your post.”

Now we know the secrets of the masters. My theory is by announcing ahead of time how many pieces of b.s. the reader will have to digest, they figure they aren’t in for a reading of Procopius History of the Early Church and can snack on the info before their Adderall buzzing brain clicks them away.

Before closing, let me digress back to USA Today and my indoctrination into the art of the list.

I worked at a newspaper — The Lawrence Eagle-Tribune — that rented its color presses to print the New England edition USA Today at night, receiving the pages via satellite and then churning out the colorful McPaper so familiar to residents of the Marriott Courtyard Suites. This close relationship unfortunately colored the judgment of Eagle-Tribune editor-in-chief Dan Warner, who decided that Al Neuharth was a visionary genius and that the Tribune’s staff  would learn to write lists instead of stories and develop “infographics” about Why We Love Ice Cream,” complete with a cartoon of a melting ice cream cone, a gushing thermometer and some made up statistic about what flavors “We” preferred.

This was strictly enforced to the point that every story opened with a classic lead (my favorite lead of all time, courtesy of Edna Buchanan, the legendary police reporter of the Miami Herald is cited below*), a standard second paragraph, and then an inevitable list of bulleted items before the jump to an inside page.  I would pile into the newsroom after a scintillating evening covering the Salem, New Hampshire board of selectmen and pound out some lifeless copy (“This ain’t a short story about your dead grandma bub, so get over it” my editor, Al White, told me after taking a machete to my first story about a sewer bond hearing) that always had a bullet list up high where Dan Warner would be sure to see it. Hence:

“In other actions, the board voted to:

  • Ban pit bulls from playgrounds
  • Postpone a hearing on bingo licenses
  • Authorize door-to-door cigarette sales by Brownie Troop 5
  • Commend Police Chief Nickerson for Sunday’s arrest of undercover Massachusetts State Policemen harassing Bay State liquor and fireworks customers

At first the mandate to use bullet lists offended my delicate Strunk & White sensibilities about prose composition.  One of the joys of great writing is a well-written list, contained in a single flowing sentence, ordered just so to delight the ear and paint a picture in the mind’s eye, but alas the world has become addicted to the staccato stack of one-liners preceded by the bold typographical dot and so I have given up all hope of resistance.

But I know in my heart of hearts that William Faulkner never wrote a bullet list in his life or worried about SEO.

 

*: Calvin Trillin, profiling Buchanan in the New Yorker: “In the newsroom of the Miami Herald, there is some disagreement about which of Edna Buchanan’s first paragraphs stands as the classic Edna lead. I line up with the fried-chicken faction. The fried-chicken story was about a rowdy ex-con named Gary Robinson, who late one Sunday night lurched drunkenly into a Church’s outlet, shoved his way to the front of the line, and ordered a three-piece box of fried chicken. Persuaded to wait his turn, he reached the counter again five or ten minutes later, only to be told that Church’s had run out of fried chicken. The young woman at the counter suggested that he might like chicken nuggets instead. Robinson responded to the suggestion by slugging her in the head. That set off a chain of events that ended with Robinson’s being shot dead by a security guard. Edna Buchanan covered the murder for the Herald—there are policemen in Miami who say that it wouldn’t be a murder without her—and her story began with what the fried-chicken faction still regards as the classic Edna lead: “Gary Robinson died hungry.”

 

3 responses so far

Apr 04 2012

Big data visualization beauty

I marvel at the art of visually representing quantitative data. There have been some excellent examples over the time. I used to be particularly obsessed with Smartmoney’s heat map of the stock market which blew a lot of minds in the late 1990s, and went out of my way to try to recruit the genius who came up with it into Forbes.com (with no success). Today it seems so static and Web 1.0, but still, cavemen used to be freaked out by fire, imagine what they would do with a Bic lighter?

Uncle Fester, the collector of all that is interesting, sent me a link to a very cool wind map.  Meteorological maps are generally fairly dull and impenetrable, with their own symbolic language of isobars, beaufort scales, and occluded fronts. Indeed, weather has long been considered one of the greatest data challenges. Consider that for decades the standard was something like this:

 Not very friendly to the layman, more the sort of thing a pilot or professional could read and derive some sense of the future from. Wind is personally the single most interesting element of a weather forecast. As a former sailboat racer, I’d obsess over the probability of a wind shift occurring during a race, or, plan ahead on whether or not to take a crew to help hold the boat down if the breeze increased in velocity. Too much weight and I’d lose. Too little weight and I’d be screwed trying to keep the boat flat in the gusts.

Here’s what wind maps used to look like:

And here is what they look like today. This is beautiful and very addictive to play with. I highly recommend clicking through to see this in all of its animated glory.

 

 

 

 

 

 

And sorry, but I can’t forget this classic:

No responses yet

Mar 17 2012

Personal Analytics: Track Thy Self

I’ve been logging my physical activities and diet for a while, moving from spreadsheets to programs to web-apps to device apps in search of the best way to keep consistent track of my progress in the belief that if I don’t measure it, I won’t stick with it.

One of my former rowing coaches, Tom Bohrer, an amazing oarsman and former Olympic-level athlete, told me the first step towards success in losing weight is to log every bite. The discipline of noting what one puts in one’s mouth forces an awareness of what is on the plate and the high number of random, thoughtless calories that can creep onto the plate during the day. Tom had me write it down in a simple $1.00 spiral notebook and not bother with calories counts, ounces and grams, or totals. Just have the honesty to admit to the bag of Swedish Fish and the courage to show that transgression to him every week.

In this Moneyball era, some sports are very number/goal based and others are getting more so. Any of the racing sports — swimming, rowing, running are stark time-over-distance efforts that can be timed, charted, and plotted over time.  Team sports — football or lacrosse for example — are subjective and don’t lend themselves to improvement-metrics the way baseball does.

I’m most interested in the trend of personal tracking and the rise of technology that allows a person to track every step taken during the day, every session completed on the machine, every moment spent in deep sleep, down to blood glucose levels. Tim Ferris’ bestselling The Four Hour Body exemplifies the degree to which a person with enough motivation and money can obsessively test one’s self. This is a guy who flies to Central America where he can gets a lot of expensive tests performed cheaply. A guy who is open to any device or toy that will help him plot performance and levels over time.

I learned the discipline of logging early on thanks to the early efforts of Concept2 — the Vermont maker of the Concept 2 rowing ergometer, the standard indoor rowing machine adopted by most teams because of its high quality and very capable digital monitor, a device called the PM4 which was developed for Concept2 by the Pennsylvania company Nielsen & Kellerman who also make monitors for on-the-water rowing and portable meteorological instruments. Concept2 was smart in opening up the code interface to the PM monitor and equipping it with a USB and ethernet jack. Third party software such as RowPro followed, giving devoted rowers and coaches even more data about their performance. Concept2’s smartest move, in my opinion, was serving up an online logbook that allows a rower to enter their workouts and compare themselves on public leaderboards against other rowers of the same weight, gender, age over set benchmark times and distances. The online logbook at Concept2.com sees billions of meters logged every year, and gives a disciplined rower a clear sense of progress and goals.

For more than a year I have been logging my diet through a free tool offered by the Livestrong Foundation called MyPlate.  The web-service is designed and managed by Demand Media and is buried in a content site that delivers nutrition and health stories and social network functions which I pretty much ignore.

The calorie tracker combines the functions of a log book with a deep database of calorie counts and nutritional levels for essentially any food one could imagine, including branded food such as a quarter cup of Trader Joe’s organic dried white peaches to a Five Guys Bacon Double Cheeseburger. I can combine ingredients into standard meals to ease the logging of frequently eaten combinations, set nutritional targets ranging from the amount of sodium to the number of net calories consumed per day, and log and plot my weight, body mass index, and specific physical measurements such as the diameter of my neck, check and abdomen over time. MyPlate will calculate calorie levels to achieve specific weight loss or gain goals and does a good job of plotting progress on X,Y charts. A subscription version offers richer functionality.

To log my exercise progress — I could and do use MyPlate as it calculates calories expended and deducts those from my gross calorie count. Hence I can log a two mile run at 13 minutes, 43 seconds, and it will cough up a calorie expense of 438 and subtract that from the inputs.

Since I am spending most of my workout time in Crossfit — I also need to track my performance and progress against a lot of benchmarks ranging from my personal records for weight lifting such as deadlifts, back squats, snatches, presses and cleans, as well as specific Crossfit workouts such with names like Fran and Kelly. I had been logging that work in a paper notebook I leave at the gym, but a fellow crossfitter introduced me to a site called Beyondthewhiteboard.com which does an excellent job of letting me log my progress against my gym’s prescribed daily workouts. There is a food logging capability on the site, but it isn’t driven by a crowd-sourced calorie database, so I tend to ignore it. I do throw my weight in there though to keep a record of progress there as well.

The Four Hour Body piqued my curiosity about the role of supplements in physical well being and improvement. Ferris prescribes some fairly outre tips ranging from his so-called PAGG Stack (policasonol, alpha-lipoic acid, garlic extract and green tea extract) to induce a state of fat-burning thermogenesis , to eating three brazil nuts in the morning and at night to improve selenium levels and testosterone production. I personally agree with the man who said people who take vitamin supplements have the most expensive pee in the world, but I also spend a lot of cash on stuff ranging from Omega-3 fish oil to all sorts of pills, protein powders and vitamins. Since I don’t have the free cash to spend on a lot of blood tests to see exactly what is going on in my metabolism I take this stuff as an article of faith.

A good source of deep and usually impenetrable advice about supplements comes from the forums at Longecity.com which is where I learned about the online log service, CRON-O-Meter. This service is essentially MyPlate taken to another level of specificity for total nutrition geeks with automated tracking of very specific vitamin and protein information for those who believe food is essentially culinary pharmaceuticals and who like to geek out by reading every word of Dr. Barry Sears, the Zone diet founder or Gary Taubes, the au courant dispeller of the why we get fat myth. I tried CRON-O-Meter for a while, but I’m just not that anal retentive or well-heeled to figure out if I need more lysine or niacin or vitamin D in my life and then buy it.

Rising in popularity are sleep monitors as the fitness-measurers are pushing the idea that sleep quality and duration has a big effect on health, recovery from exercise, and general well-being.  The owner of my Crossfit gym, Mark Lee has been using a sleep monitor, and there are some that track the time it takes you to fall asleep, how many times a night you wake up, when you go into deep sleep, etc..  One brand I’m aware of is Zeo with a $150 bedside setup.

Then there are the new breed of pedometer like devices that track every step, capture all the data, and can be uploaded and tracked online. Fitbit is probably the best know of these, and at a $100 seems reasonable enough as it also purports to track sleep but I’m not compelled to wear one on my belt.

One can obviously go overboard on the personal tracking obsession and I know I am coming close to being too geeky about the whole thing, but you can expect to see and hear about more of it, not less, as awareness over dietary and supplement chemistry rises thanks to people like Tim Ferris; the paleo diet craze expands because of Reebok’s commercial embrace of Crossfit “the Sport of Fitness (Crossfit, aka “Cultfit” to its detractors, embraces paleo principles as part of the program); and the device makers push their meters, gauges, wireless scales and pedometers at you more and more.

My personal testimony to whether any of the tracking works is this: I’ve dropped 50 pounds in 18 months, cholesterol levels have plummeted (I took myself off prescribed statins and have yet to see if I can manage my HDL/LDL levels through diet and exercise alone), and I eat a fairly strict paleo diet that restricts calories to around the 2,000 per day level. My rowing times are as good, if not better than they were ten years ago, and my running times have improved from a sluggish ten-minute mile pace to a 7 minute mile in a matter of months. Yes, this is insanely narcissistic, but it is efficient, it beats the old method of carrots and cottage cheese, little paper calorie counter books, and endless jogs around the block with a daily visit to the bathroom scale.

 

5 responses so far

Jun 28 2009

Spikes in stats

Published by under Metrics,Personal

Feedburner displays my feed subscribers in the left column. I keep an eye on as a casual reference to growth in readership and declare little victories everytime the odometer clocks another 100 readers.

Typically it hovers around 600 subscribers but in the last few days it has spiked to 900 plus. Why? No clue. The number fluctuates up and down, but a 30% spike means either Feedburner has burped or … (update, Nathan Gilliatt said FriendFeed subs are added)

Some undetected thing spiked inbound traffic.

Look at the green bar just go nuts in the last week.

I’m not a collector of stat counts — I have to worry about followers and ranks too much in the real world of Lenovo — but it is an ego-stroke to know someone reads this stuff.

Then again some don’t ….. Stefan Constantinescu, a great commentor on all things related to ThinkPads, had to unsub when he realized that my professional title doesn’t mean this blog follows in my career’s footsteps. (No hard feelings Stefan, just citing your decision as example of blog identity crisis).

  1. Churbuck: David Churbuck works for Lenovo (OTCPK: LNVGY), a company that makes a line of laptops known as the ThinkPad. Why do I know this? I’m a huge ThinkPad nerd. Practically every laptop I’ve ever owned has been a ThinkPad. I love the design, the dependability, the battery life, but do I love David? This is his personal blog more or less. He constantly writes about getting back into shape and fishing. I’m sorry, but I just don’t care. Decision: Unsubscribe.

5 responses so far

Sep 08 2008

Web metrics quote of the century

Published by under Metrics

This came to me via colleague Gary Milner, who forwarded a column by Joe Marchese,president of socialvibe, who quotes Rishad Tobaccowala who in turn channels the ghost of Albert Einstein:

“Not everything that counts can be counted; and not everything that can be counted, counts”

One response so far

Mar 07 2008

The stupidity of metrics

Published by under Metrics

I have been in solid meetings the past two days and yesterday watched a presentation that reminded me of the story of the crash of Eastern Airlines Flight 401 in 1972.
The pilots were coming in for a landing but the “gear down” light didn’t illuminate in the cockpit. They tapped the light. They flipped switches. The co-pilot opened a hatch and climbed down to see what the problem was. They continued to obsess about the light but they didn’t notice when one of them bumped into the steering column and turned off the autopilot, putting the jet into a slow descent.

No one looked out the windshield. They were looking at the dead nose gear light.

Splat. 99 dead. 77 survived.

Metrics — the act of collecting data about systems and processes — and then reporting them in dashboards can lead to the type of tunnel vision those pilots displayed 36 years ago. The obsession with gathering status reports for the sake of gathering status reports can divert the organization and its people from the task at hand. If you’re trying to smelt gold but you spend so much time tracking ingot development that you fail to notice that you’re in fact smelting lead — then you’re going to be really good at ingot development, but oblivious to the quality of the final output. This is why formerly good things get ruined when big companies acquire them and start to obsess about the efficiencies. “We’ll just swap out the good stuff for the okay stuff and no one will notice.”
Subjectivity — the measurement of quality — is it good? is it bad? Do we suck or do we rock? Those unmeasurable intangibles are dismissed by technocrats as “feeling” behavior prized by people to sloppy to appreciate precision. Or, they attempt to quantify the subjective with surveys and stupid metrics like “sentiment.”
Objectivity — the measurement of facts — has become de rigeur ever since Neutron Jack Welch of GE set forth the commandment that you have to measure it to manage it. And so commenced the age of the tyranny of metrics. The Excel tyrants are really really good at demanding status reports and updates, but the reality is no one looks at their work and is terrified to say: “Go away. Here’s a beach. Start counting grains of sand and give me a TPS report by tomorrow.”

Metrics people — turn yourself into analysts by looking out the window and telling your boss the swamp is getting really close.

19 responses so far

Feb 12 2008

Google’s Data Asset

Google’s Data Asset | Union Square Ventures: A New York Venture Capital Fund Focused on Early Stage & Startup Investing

Brad Burnham, partner at Union Square with Fred Wilson, blogs about the impact of Google’s data driven services. In discussions over dinner last week with Dan Gertsacov from GoogleTV, we got onto the topic of data as differentiator in advertising. Brad nails the same point in this post (which is about the impact of marginal data collection on services)

“Each incremental point of data adds value to the ones you all ready have. It is easy to see this in the context of an advertising network. If the ad network knows that a user is female it can show more relevant ads. But, If the ad network knows that female’s age, it can do even better, and data about location, household income, and recent web sites visited all add value to the existing data points, making it possible to show more and more relevant ads. Google’s services all benefit from additional data albeit in different ways.

“So what does all this mean about the market for web services. It means that we all need to begin to think about the degree to which Google’s enormous data asset will allow it to dominate this important sector.” [emphasis mine]

The hot buttons in the last paragraph are “think” and “dominate.” Data driven ad services (or cloud computing) are squarely in the sights of the privacy stewards and the federal government. Anyone who had to answer reader mail in 1995 when a web user freaked out about a cookie being placed on their harddrive knows never to underestimate the public and government’s paranoia about data.

As an advertiser, Google’s data store and processing capability is extremely valuable in transforming next-marketing-dollar allocation. The other networks don’t possess the same analytical and data driven rigor. It could, if I take Brad’s post to the paranoid extreme, be a harbinger of Google’s future focus on persuading the world that a) their data is anonymous and b) safe with Google.

No responses yet

Feb 12 2008

The Lost Weekend with Feedburner

Published by under Metrics

I’m a Feedburner fan and run this blog’s RSS through the service so I can gain some sense of traffic trends. Per Peter Kim’s M20 measurement criteria, I made my RSS subs visible with this badge that tallies my feed subscribers.

Every weekend about ten percent of my subscribers seem to unsubscribe or vanish. By Tuesday, they return.

So what happened to the 43 mystery subs? And why do they go away every Saturday?

5 responses so far

Sep 21 2007

Online advertising not measurable enough?

Compared to what! Television? Print? Mark Cahill calls my attention to this insanity.
Scott Karp rips apart a recent McKinsey survey/report on online advertising (which I meant to do, but got sidetracked). Here’s the upshot as reported in Adweek:

McKinsey polled 410 marketing executives in five sectors, and among those already advertising online, 52 percent said “insufficient metrics to measure impact” [emphasis mine, ed] was the biggest barrier, followed by insufficient in-house capabilities (41 percent), the difficulty of convincing management (33 percent), limited reach of digital tools (24 percent) and insufficient capabilities at agency (18 percent).”

Scott gets right to it:

“Does that mean advertisers really believe metrics like cost per lead, cost per sale, or even cost per visit are inferior to traditional “bottom line” metrics like reach and frequency, gross rating points, and rate base? Does that mean advertisers believe mass media have better “capabilities” than online advertising platforms like keyword-targeted search advertising, behavioral targeting”

Client-side as I am, let me agree with the survey panel that the primary barrier is indeed in-house capabilities. Budgets, staff and mindset are still geared towards television, print and out of home. Convincing management? Not an issue for me. Limited reach? Sure, online is growing more expensive and yet it is harder to put money in market as strong opportunities begin to vanish under high demand. Agency capabilities? Agencies are scrambling to staff up, I’d put the onus on the client to deal with the in-house capabilities, we’re moving to a multi-agency, specialist network model with the client providing the management and the glue, the metrics and the execution.

But insuffiicient metrics? That is doubtlessly the most ignorant thing I have heard all year. McKinsey needs to either change its survey methodology or find marketers who have a clue, because the one’s they surveyed are probably relying on their agency to give them click-through reports and therefore, deserve what they get.

Metrics in online advertising are the responsibility of the client, not the agency, not the publisher. If the client is incapable of attributing revenue to a dollar placed in the market, then the client is wasting its money. Only the client can detect the action on the client site. Period. If an advertiser is spending online (I suspect McKinsey’s survey panel is obsessed with CPM and CTR on banners) and not measuring the impact end-to-end then they need to fire their agency, hire an interactive marketing manager, and invest in a decent metrics package. If not, well, stick to your TV, radio and print and have fun with such wonderful measurements as “pass-along” and “drive-time impressions”

Note, McKinsey loves its registration wall, in the belief that its content is super precious (it is good, but sorry Stuart and Jeff, it needs to be easy to get to), so I shall not waste your time with a link to the survey which I can’t find anyway.

6 responses so far

Aug 24 2007

Web Analytics for Dummies

Published by under Metrics

Spot the day when Valleywag came after me.

One response so far

Jul 14 2007

Kaushik’s book is out

Published by under Metrics

Avinash Kaushilk, something of a hero to web analytics people, writing incisive stuff at Occam’s Razor has published Web Analytics: An Hour a Day and I returned home to find the package from Amazon, having pre-ordered it about six months ago.

Now to read it. Knowing Avinash, and appreciating the expertise he’s been sharing at his blog, it is sure to become a bible for the art.

No responses yet

Jul 11 2007

Ding dong, the page view is dead

Rob O’Regan at Magnosticism reports Nielsen’s decision to drop pageviews as the primary metric for site measurement. Good riddance say I.

Ranking top sites by total minutes instead of page views gives Time Warner Inc.’s AOL a boost, largely because time spent on its popular instant-messaging software now gets counted. AOL ranks first in the United States with 25 billion minutes based on May data, ahead of Yahoo’s 20 billion. By page views, AOL would have been sixth.

Google, meanwhile, drops to fifth in time spent, primarily because its search engine is focused on giving visitors quick answers and links for going elsewhere. By page views, Google ranks third.

I posted on this a while back, arguing that Ajax and other page-cache models were making the Web 1.0 model of page-by-page sessions irrelevant. Expect to see some sites rail against this decision, and don’t expect the IAB (Internet Advertising Bureau) to make a move anytime soon.  Here’s the pdf of the Nielsen-Netratings release.

One response so far

Jun 06 2007

Less is more

There are some great resources and wiser minds than me on the topic of dashboard design, but let me go back to the oft-quoted Kenneth McGee at Gartner, and his 2003 book, Head’s Up, for some fundamental guidance on how a community/engagement marketer can best act as a listening post for an organization.
In a meeting with Andy Beal of Marketing Pilgrim last month, I cited McGee and his case example of the Galveston Hurricane of 1900 which decimated the Texas coastline and killed over 6000 people because the local weather expert, Issac Cline, ignored the obvious warning signs. McGee’s thesis comes down to this: in business intelligence the issue is not measuring enough things, it’s knowing which things to look at. Pilots crash planes when they take their eyes off the job at hand and worry about fetching the pencil that rolled under their seat. CEOs can crash companies if they fret over the wrong bullshit.

McGee cites as one of the book’s case studies the CEO of GM, Rick Waggoner, and what he looks at every day. Comes down to three things: daily dealer sales (for a geographical view of demand), factory floor rejection rates (for a view of quality), and a third I can’t remember but will when I return to my home office and my bookshelf.

So what do dashboards have to do with my lot in life? Lots. We run the business on dashboards. We spend a lot of time peering at red, yellow and green spreadsheet cells to determine if the business is on track for the quarter, if our advertising is delivering the ROI we expected, and if we need to develop a promotion or coupon to make our goals.

Now I need to develop a single page, to be produced once a week, for the company’s leadership that encapsulates, at a glance, what they need to know about Lenovo and the online world known as Blogistan. This isn’t a KPI achievement dashboard. This isn’t a green-yellow-red series of gauges. This is a simple thing that permits a CEO or SVP to see, at a glance, that last week the Blogs were chattering about us more or less than the week before and the prevailing themes were A, B, and C. Maybe a chart scraped off of Technorati showing number of gross mentions of “Lenovo” and “ThinkPad” trended over the last three months with a Plus/Minus indicator. This basically is blunt buzz. Then, underneath two sets of bullets. One titled Highlights with the three good things that were blogged, posted, or commented in the previous week. Then Lowlights with three bad thing. And, at the end, a sentence or two from me saying something like: “Buzz spiked on Thursday with the announcement of the R61 and T61. Commenters hate the placement of the audio jacks on the front panel. Competitor X stepped into a mess when it told the Old Lady Association to go to hell and Blogger X called the CEO of Competitor Y a pinhead.”

This word of mouth dashboard is dependent on the verbatim quotes of the forum posters, bloggers, and blog commenters. What I lack is an overall positive-negative sentiment rating — a leading indicator of customer satisfaction — and in the ideal world would have one benchmarked against my competition. Yes, there are services that provide such ratings, but we don’t subscribe to them. I sense there may be a way to kludge something together with some degree of dependability.

What is the purpose of the measurement and reporting (the battle cry of our metrics and analytics leader, Jim Hazen)? Primarily as a communications mechanism to alert our CEO and his direct reports of commentary and customer satisfaction over issues which may not be understood, or detected internally as soon as me and my team see them. As McGee emphasized, the earlier something is detected the number of options and the amount of time to react to increases. Waiting for the hurricane to arrive is too late to evacuate or bring in the patio furniture.

2 responses so far

May 18 2007

When the elephants dance …

… the grass gets rich. Microsoft takes on Aquantive — owners of the Atlas ad server and interactive agency Ave.A/Razorfish. Obvious that Billg is not conceding nothing to Google in the war for advertising. This, on the heels of 24/7 going, means the ad server industry just went away and has been consumed by the mammoths. I’ll predict MSFT next buys Yahoo — by December.


Then comes the Valleywag rumor that Google is looking at spending $100 million for Feedburner. Interesting move on their part — makes sense, and checks off the RSS advertising box in one neat swoop. Any blogger with half a brain launders their feeds through Feedburner — their stats are solid and they provide the strongest insight into RSS traffic of anyone. Add that to Google Analytics and … (I’m getting tired of being a Google Gusher)
 

One response so far

Apr 05 2007

Management and measurement

Published by under Metrics

Tis KPI season — “key performance indicators” — and I need to pick the three or four measures that will drive a web marketing operation.

Any suggestions? If you could track everything with utter precision, what would you track? How would you measure brand strength? Awareness?

I have my theories. I need more.

22 responses so far

Mar 29 2007

DoubleClick Sale Could Risk Publisher Exodus

DoubleClick Sale Could Risk Publisher Exodus

News from ClickZ that DoubleClick is in play and could be sold — something we knew was coming after they were snapped up by a private equity outfit. MSN and Google are mentioned as contenders. Ad serving is the central nervous system of display advertising and integrating the data stream that comes out of an ad server’s cookie logs is crucial to an advertiser like me trying to correctly assign revenue recognition to specific campaigns. Thinking back to the old days of NetGravity at Forbes.com (which was acquired by DoubleClick), to the hell-on-earth known as ad ops at CIO.com (ad ops is the unwanted step-child of online publishing), to the potential of behavioral optimization, dynamic creative, and mid-campaign retrafficking …

The story is right, it’s a “Stale World,” but one I get worked up about after a couple beers.

“The stale world of online ad serving just got interesting again, as a possible acquisition of ad management firm DoubleClick was floated yesterday. According to the Wall Street Journal, Microsoft or another buyer may grab the ad serving colossus soon. If a deal with Microsoft does become reality, it would boost the firm’s online ad capabilities and make for readymade relationships with advertisers and agencies. However, it could put DoubleClick in hot water with its publisher clients, including AOL, which would be loathe to let the company access user data flowing through DoubleClick’s DART ad serving system, and which compete directly with Microsoft’s MSN for ad dollars. Indeed, AOL could be a potential buyer, some believe.”

One response so far

Feb 17 2007

Ratios and Leading Indicators – what drives me

Every industry has a specific, almost canonical metric it pays attention to and benchmarks its participants against one another. I wanted to take a few seconds to think out loud about the basic ratios that guide me on a daily basis.
Some business cultures emphasize one financial and operational ratio over another. In Switzerland and Asia, inventory turnover is a big deal. In NASDAQ-listed tech stocks its price-earnings as a measure of value and expectations (P/E). In other industries it’s revenue per employee, while others focus on average revenue per user (ARPU), and then there are operational, specific ratios such cost-per-click.

I live in a world of expense-to-revenue, or “E2R“, expressed variously as “e:r,” “e/r,” and “e|r.” As a relatively green marketer, I assume this is the prevailing canon for measuring marketing effectiveness and efficiency: for every dollar spent in market, what revenue is received, and what budget should be fenced off based on the top line to support a marketing expense. In dummy terms, “How much did I spend to make $X?”

A 1:1 e-to-r means I spend a dollar to make a dollar; that comes out as a 100 percent E/R. We can call that a wash. A negative E:R is when I spend two dollars to make one dollar. This is a 200% E/R and means I’m looking for a new job. A good E:R is when I spend one dollar to make five, a 20% E/R, or as Warren Buffett would say, buying dollars for twenty cents.

This applies in interactive marketing on ad tactics and assumes perfect insights into the purchase of an impression (based on cost-per-thousand impressions, or CPM), yield click-throughs (measured by click-through-rates or CTR), and ultimately ARPU, or average revenue per user (or unique visitor). If I am able to track the user from the first purchased impression to the final checkout, then I can credit the tactic with the sale.

Still with me? This is predicated on that user accepting the tracking beacon, or cookie, from my metric system, a cookie that the user gets when clicking on the search term or the banner ad I’ve purchased for N. If the user permits cookies — in my case one deposited by a script on every page of our website that will permit our metrics tool, Omniture SiteCatalyst to follow that user across multiple visits — then, if our commerce guys have successfully done their job, the user will buy something and I will be able to credit the original marketing tactic with the sale.

Sounds hard and imprecise? It is, but compare it to a newspaper ad. If I buy a full page ad in the Daily Planet and it is seen by a theoretical 100,000 readers, and sales from the zip codes where those 100,000 readers live goes up by 10 percent can I declare the ad a success?

This is the old handgrenade-and-horseshoes school of marketing — the famous I-know-half-of-my-advertising-works-but-which-half” school of marketing.

So back to what I worry about, which is interactive advertising: the interesting thing in assigning expense-to-revenue ratios to a particular tactic is knowing what the intent of the original message was, and second, knowing what it yielded.

Those tactics are pretty simple. Search engine marketing, which is very precise in that I am purchasing clicks through an auction model, but which is usually promotion driven by non-brand tactics like “cheap notebook” or “fast PC.” Then there is display advertising — banners, buttons, IAB standard graphical units — and that is arguably either pure branding or can be pure promotion, but generally are a hybrid of the two. Success in assigning an E/R ratio depends on accuracy in following a customer through multiple visits to my store, because in my market, where I am selling products for $500 to $3000, the customer is researching, comparing, and considering before hitting the purchase button.

We can do this, the rest is pure optimization, arbitrage, and discipline. It’s actually kind of cool and very fun, and a nice feeling at the end of the week to say, “I bought a dollar for twenty cents.”

8 responses so far

Feb 04 2007

The Sponsorship Model for Bloggers

I have been advising a blogger on her negotiations with three publishers for the sponsorship of her blog which has, in a relatively short period of time, accrued about 250,000 page views a month. This blog has a lot of buzz and attention paid to it, and she finds herself in the enviable position of considering multiple offers.
She has not filtered her feeds through Feedburner yet, so she has no idea what her subscription stream looks like. All she has is a SiteMeter bug which gives her a rolling 30-day view of the traffic. Based on that traffic, the publishers are coming back with offers more suited to the existing page-view/CPM model, looking for fixed placement of their message, a subscribe-link for their print products, and promotions for their own RSS products.

She has tried AdSense and a CafePress store, but essentially isn’t making a dime from her efforts. The content of this blog is unique, and is not comparable to any other source. It also has a significant barrier to entry for a competitor to move in.

My theory on what places downward pressure on CPMs is the concept of “fungible” — a twenty-five cent word for anything that is easily replaced by something else. In media terms this could apply to stock quotes, wire news, anything that is commodity content which the consumer would be hard pressed to identify its origin based on its tone and substance. In other words, if I covered up the source of a story on Reuter’s website about Apple’s earnings and showed it to a reader next to a similarly disquised story from the Wall Street Journal on Apple’s earnings, would the reader be able to identify the source?

Assuming the answer to the rhetorical answer is “no,” then one can see how a CPM on such a site will always trend lower as more venues begin to produce and publish the content. In the end, the most unique content that a publisher can claim is probably the opinion side — hence the movement of the columnists behind a cost wall at the New York Times with the TimeSelect program.
The question is whether a page view of a blog should be treated on an apple-to-apples basis with a page view of a magazine’s website. One offer values my friend’s traffic at an $8 CPM — which translates to $2000 a month. Chump change in my opinion.

I believe she can, and should, do two things to get maximum value for her efforts. First, avoid any ad network like Federated Media like the plague. Federated acts as a rep for traffic, selling it and managing the insertion of ads based on a demographic algorithm, and then reporting the results on the back end to the advertiser. They take a significant chunk of the revenue in the process. Anyone who has participated in an online affiliate program — Amazon’s for instance — knows what thin beer such programs can be. Second, she may want to consider an auction system where she lets the potential sponsors bid for the placement. How she can accomplish this, well, it’s beyond me.
So, what’s the recommendation? I felt in 1995, and I still feel today, that some online content will have such high value, and be so unique, that their creators will be able to eschew the tyranny of metrics and pageview economics and offer their stuff on a flat sponsorship basis sold purely on the basis of time, selling a patronage model that essentially lets the sponsor bask in the glory of the content and perhaps receive some business on the side. This is the PBS Nova model. Is the General Motors sponsorship of a PBS show an ad or a statement of corporate social responsibility or both? Expecting a low-traffic, high value content site to incorporate the kind of metrics and accountability and ad ops that a traffic behemoth has is ridiculous.
The hard part for my blogger friend who is on the verge of making some serious money, is how to negotiate these deals. She has published several books and has a longstanding relationship with a great book agent … who hasn’t the faintest idea how to represent her online efforts. There must be some agents out there who are emerging as strong negotiators on the behalf of online artists and talent, the question I pose is who are they and what are their terms?

Am I being naive in believing some web content stars can and should expect a sponsorship model, and that their efforts and pageviews are vastly different than a standard site’s?

7 responses so far

Dec 28 2006

The End Of The Page View

Fred Wilson: 2007: The End Of The Page View

Om weighs in this morning with the resignation that the page-view model is what we have for a benchmark of web audiences and therefore it is the benchmark we have to live with. Knowing that Om’s GigaOm network is dependent on advertising revenue, he has to conform the prevailing metric in the market, which is the 1990’s Web 1.0 measurement of audience as expressed by page views and unique visitors.

Fred sparked Om’s defense of the PV with this:

“…there are changes afoot in the Internet measurement business. Everyone is recognizing that pageviews matter less now. Ajax and other more modern web technologies allow for new ads to be diisplayed without a page reload. Ad views can grow even as page views decline. I know that there have been a number of discussions about this at the highest levels of the leading Internet measurement firms and the leading Internet businesses. And we’ll be seeing the outcomes of those discussions at some point in 2007.”But it doesn’t even stop there. Web pages themselves are changing, moving from pages controlled by publishers to pages controlled by users.”

I’ll reiterate my position from the point of view of a buyer of traffic, which is ultimately the onus on the site I buy from is not gross tonnage of views, nor even clicks, but the end-of-funnel conversion that occurs on my site after the publisher delivers the traffic into it. This of course is further complicated by the reality that the “deal is everything”, no publisher can control the creative run on the page — AJAX or static — but in the end, when I operate the campaign, optimize the creative, retraffick the placements and optimize the backend landing page for A/B and multivariate possibilities, I will look at those sites or networks which sent in the best traffic conditioned to respond to me.

I don’t buy large numbers. I don’t buy CPMs. I look at publishers as providing me blunt approximations of an audience, and then it is up to me and my agency to put the right offer in front of that audience at the right time. If I believe a publisher’s pitch on gross tonnage, then I’d be buying into the fact that they are pushing forced page refreshes to hit their campaign guarantees (which I did in former lives) are needlessly chunking long stories into multi-jumps to force up their pages per session, and otherwise playing the games with the logs that I played myself.

It takes one to know one and I know that ad impression numbers are wholly unreliable and again, reiterate, that the burden is on the buyer — aka, let the buyer beware — and take responsibility for the user experience once they wind up on the destination site.

2 responses so far

Next »