WriterAccess Webinar Archive
Critical Competitive Intelligence Tools and Tricks
Wednesday, August 31, 2011 – 1:00 PM ET
To compete in today's fierce marketplace, you need a competitive edge with insightful information, meaningful metrics and dollar-producing data that helps you find the fastest path to growing your business organically–the content marketing way. Let's face it–the right data can save you a lot of time and money.
Join SpyFu founder Mike Roberts and ideaLaunch founder Byron White for this month's Content Marketing Webinar focused on killer competitive intelligence tools, tricks and resources you can use to discover important insights about your customers, competitors and even your friends (and enemies).
From PPC patterns to social media behaviors to organic market share analysis, get the competitive information you want and need to make critical decisions at your fingertips.
In this webinar, you'll learn:
- The top 10+ competitive intelligence tools and resources
- A competitive intelligence methodology for content plans
- A competitive intelligence methodology for SEO plans
- PPC keyword and campaign tools for competitive intelligence research
The slidedeck from this webinar is available for download.
Byron White: Welcome everyone, it’s Byron White here; welcome to the monthly content marketing webinar. I’m here today with Mike Roberts. Mike, welcome.
Mike Roberts: Thanks, Byron.
Byron White: We’re very excited to talk about a topic that is certainly one of great importance for what we’re all doing on the web out there; namely, competitive intelligence and as a matter of fact, Mike and I have been talking and barking for some time together about running a competitive intelligence/content marketing-centric webinar. Well, we may be getting close to that possibility in October of watching this SpyFu University extension for interested folks out there who want to hear some deeper insight into what possibilities there are, in learning about your competitors and how to leverage that information, and we’re going to deeply dive into it.
The hope throughout 2012 is that Mike and I just take this to another level and put together two incredible assets for you that we begin to trickle out throughout the course of the year, which I'm excited about, as is Mike, to get this done and what would be a 101 competitive intelligence tips on how to use data, and we’re going to publish that in a book, and then another 101 tips on tools to use for competitive intelligence, which I think is super valuable for everyone as well. So we’re excited about that, we’re going to work on that and get back to everyone both in our world here at IdeaLaunch and then Mike’s world at SpyFu with some more data on that.
Today, we’re going to dive right in, and I’m going to talk for about 10 or 15 minutes about some competitive intelligence tools and tricks that I think you'll find extremely valuable and I’m going to actually talk about the tricks first, strangely enough, because I want you to see as quickly as you possibly can how this data can be used, and how it fits in with overall content marketing strategy. Then, I’m going to do just a quick review and give you a nice punch list of about 20 interesting tools, other than the famous and great SpyFu that you can use to find great information and other critical elements that you need to know.
So without further ado, I always begin these content marketing webinars with a quick overview and just a couple of house rules I forgot to mention. Number one, please ask questions for both Mike and I throughout the presentation by using your toolbar on the right side and number two, feel free to Tweet both Mike and I, @ByronWhite and I think it’s @MikeRoberts. Mike, your Twitter account is @MikeRoberts right?
Mike Roberts: Mine is actually @mrspy, M R spy. You can also Tweet us at @SpyFu; that will make its way to me.
Byron White: Cool. Well, we’ll be checking our accounts as well, and I look for any great feedback from anybody, so feel free again to ask for any questions you have throughout the presentation. So, without further ado, let's dive into to what is content marketing and I think it’s really important to understand what's happening with content marketing. Now, there is a revolution going on, and I want to make sure that everyone on this presentation is well aware of what's happening and the power that you can put to work.
What is content marketing? It’s really the art of listening to the wants and needs of your customers and that is probably the hardest thing to do. I’m certain that we’re going to see some major advancement on that, both from SpyFu and some of the other leaders in the industry that are helping us understand the wants and needs of customers. We can look internally in our own search box and certainly look out in the social media sphere to see what conversations are happening. We can look at analytics, but how you really know what a customer wants and how they react to different things you throw at them, and it's hard to figure that out and it’s something we need to make a lot more advancement on than old questionnaires or surveys we used to send out to customers,
Of course, if the science of delivering compelling content to them, and you need to do that in a variety of ways, and you need a new content asset portfolio if you will to read to your customers, and those assets are really difficult to produce. I like explaining to people that forward-thinking companies are starting to think like old-school publishers; gathering ideas, developing stories and publishing a steady stream of content now to enforce a wide channel of distribution. You need to be thinking about all these assets if you’re going to be successful online and, of course, you have to catch readers who are orbiting at high speeds and that means off your website with apps and other online events and informational portals and feeds that you can send out to people.
You need to create information that they want and need that’s not the features and benefits of your products, but it's probably more informational assets that help educate and entertain. You can start looking at the metrics and the success of those assets by measuring downloads and the time on your site BP visitations and all kinds of other fun stuff, but content marketing by definition should be testing campaigns with multivariate testing. I like telling people, “If you’re not testing your website, you’re living in the dark ages, so do take advantage of that opportunity.” Finding the most efficient path to engagement is probably the name of the game and that means understanding your pipeline.
When do people need certain types of content assets that you create? How are you scoring the engagement and qualifying leads and motivating them to make a purchase? Are you in tune with your conversion path and are you providing different assets along the way? Finally, a new slide for those of you who are gritting your teeth and bearing with me through this introduction, thank you for that. There really is a process and a workflow to content marketing and it’s particularly appropriate today to understand where competitive intelligence fits in.
Essentially, the answer to that is in the planning stage and we’re going to talk about that, but the net of it is, the planning stage is where you want to be doing your research really before you even write one article or one content asset. You’ve got to understand where you stand versus the competition. You’ve got to do your keyword research and your customer research and develop style guides. You’ve got to put together personas; you need a thorough understanding of how you stack up versus the competition if you're going to be successful in driving more traffic to your websites, improving your listing positions, decreasing your user acquisition and all that fun stuff.
There are, as you can see, five other steps to the process, and you can read them through there and learn about them. Of course, content performance is not even possible without doing the right planning and the right strategic work up front. So let's just dive in real quickly and offer a few tricks on how to use competitive intelligence tools to arrive at those performance goals you’re looking for. The first take on this is to really understand deeply that planning process and really, what's in it so we’re going to look at some of that in just a second, and some actual charts and graphs that we put together with these massive reports that we put together that cost anywhere between $15,000 and $50,000 until Mike and his crew figure out a way to trim our research time down to a very low level.
We’ll be working on that throughout 2012 with Mike and his team to try and really figure out how to arrive at these critical things that we need to learn to come up with the purpose of a content plan, which is very simply to figure out how much content do I need to capture organic market share, how frequently do I need to publish it, how good does it need to be and what distribution channels do I need to really follow down? So these are the questions that that we’re trying to learn from; all this competitive research we’re doing and this customer analysis. All of this research points at those four primary questions, so let’s try to go take a look now at some tips with how we roll this data together.
The first thing you can do is take a top-level view of how your website is performing versus your competitors for just some really simple things like internal links, external links, your Alexa rating, the number of pages that you have indexed versus your competition totally unique monthly views, Google home page rank; you could argue that many of these things right here are meaningless. You could have a wonderful debate with me about whether page rank means anything or not and I might be on the side of the fence that says page rank is useless and you might say, “What are you talking about? It’s competitive information.” So if the sum is greater than the individual parts, you do need to look at all of these variables to try to get a better feel for where you stand and this single chart here gives you some pretty good data.
You can dive deeper into the data and look at traffic performance, and graphs, and charts, and data pulled up and look at each of the properties you’re trying to look at, and get a feel for how big they are, and how big their content universe is, and pages indexed, and how powerful they are. You can also look at the fundamentals of content quality and frequency and quality, or quantity it should be. And you can do that for all the competitors and literally total how many articles do they have, how many blog posts do they have, how many ebooks have they published, FAQs, how many questions do they have in that section? Job descriptions; this happened to be for a staffing company where we needed a particular column in there for how many jobs were posted on their website, how much copy was on their website as far as copyrighting pages were there? Were there sell or offers on those pages, how many tips did they have, how many news articles did they have?
This stuff is very difficult to manually calculate, and it’s even more difficult because sometimes, you have to come up with a custom comparison depending upon the industry or the website or all kinds of other data, which is why we have to do this stuff by hand. The other challenge is, how frequently are they publishing this content? With blog posts you can go in and take a peek and see if they’re publishing daily or weekly or what the challenge is, but with other pages on their website, you really don't have a feel for how often they're necessarily updating their pages or how frequently they're pushing out white papers or ebooks, and by the way, there's often not a lot of consistency with that.
They may bring in a new director of marketing and everything may change quickly, so all of this data really needs to be live. It needs to be relevant, it needs to be driving your strategy to answer those four questions that I mentioned and, of course, quality is another subjective grade, although with Google Panda, who knows what Google has done? I have a feeling that they’re looking close to get the linguistics, the elements of a particular copy and being able to pick out crappy copy that’s created in Indonesia or the Philippines or India. There are subtleties that are perhaps transcribable into a penalty if you're producing crappy content. We go in and subjectively view the content and then make some great scoring to help our clients understand how they stack up.
Additional content assets, photo galleries, podcasts, webinars, videos, workbooks; there’s just lots of things you need to compare and contrast as you look at your website versus a competitor. And of course, social media; how many posts, how many people are linked in, how many tweets have you produced? These are all things again, that help you sum up how you stack up against the competition. How content widgets and applications become relevant; they can also often connote the time on site and repeat visitation and all kinds of things. If you've got cool widgets and applications and free tools, chances are likely that you're going to be beating your competition with traffic and all kinds of other variables.
Mike Roberts: Can I interrupt you for a second?
Byron White: Sure.
Mike Roberts: I think that there's something really interesting you said about detecting low-quality content because it’s a classic computer science trick to be able to figure out whether something’s written by Shakespeare programmatically by looking at the mix of words. If they’re really serious about quality, you can totally figure out the accent that this is written in. Who's doing that? I think it's the Mycept Ad Center that’s basically calculating the educational level of a site, like how quality is the content? That’s a really interesting thing in terms of this content, because I know that with your new service you’ve got the ability to slide the scale to see how to determine the quality of the writer, eh? Yeah, that’s actually probably a much bigger signal than I’ll bet anyone’s letting on.
Byron White: How can you justify knocking out traffic at levels that were catastrophic for a lot of websites? They were just blown off the map and believe me; Google took a hit on that. I mean, they were running Google ads on a lot of sites that Google purged. So this was a statement of quality as relevant. Everyone tends to discuss and point themselves at duplicate content and those lousy product descriptions that you copied over from the manufacturer’s site and I need to get rid of my duplicate content. Wait a second; it might be something more than that. You might need better content on your website; so I think we’re seeing some very interesting things happen, and you can certainly bet that Google has more than two to five dozen linguistic PhDs on their staff working on the algorithm and understanding it.
Mike Roberts: Yeah, it’s really just a training thing; it’s not really all that difficult to do. I’m no linguistics PhD but I could totally bust that out. Yeah, that’s a really interesting point; I don’t want to totally sidetrack your thing.
Byron White: No, no. That’s a good point, and so we will look for SpyFu to look for some potential research.
Mike Roberts: I don’t know what the productization of that would be from our side.
Byron White: It’s a grader just like you said. I’m going to do some research because we use Copyscape, for example, over at WriterAccess and we’re scouting the web to find any duplication. Actually, I could pull that up as you’re talking; I’ll try to find some graphs for that. Yeah, you look at a Copyscape report and it’s highlighting individual words or entire sentences that were found elsewhere on the web and with every asset that runs through WriterAccess. We run every single one of those assets through Copyscape originality verification and we show the reports. They can look at it and see if they're worried about a possible full-blown duplicated content issue before they approve any content.
Even if it’s scattered, we actually show the URL so you can click on it and take a quick look at where they peeled this copy from. If you’ve bought 10 or 20 assets and they keep popping it from that same website, that's an indication to just in one swoop reject it all and say, “Dude, I was looking for original content here, not copying a website’s entire copy and lifting it over to my website.” That's another case in point; where Googled algorithms are probably picking up too many common elements between one site and another would indicate a slight modification. And on that same subject of duplications, keep the conversation going because it’s interesting.
There used to be this rule that we used to talk about and think about with transferring copy from one site to another like, “Oh, you just need to change one word in once sentence and then it won’t be a duplicate.” That is insane and if you look around, you can see how easily the Google algorithm would pick up on that right away. You look at one Copyscape report and it’s like, “Okay, this entire sequence is shot. Oh look, that word is different but look, the very next sequence is copied.” On the duplicate content issue, I have always thought all along that one of the main reasons that Life Tips, which has been around for like 13 years now, is so damn popular with 1.2 million, is because everyone’s just copying it. Our link popularity is pathetic; it’s pathetic, it’s like 300 links, 500 links. I mean, that’s not a lot of links to justify 1.2 million visitors, but now you take into consideration all the people who have scraped those 120,000 tips and try to launch them on other sites. It's almost like a compliment to life; it’s an interesting point.
Okay, so competitive comparison. You look at a way to summarize this and that’s what you kind of really need to do. It’s hard to do that; you could start weighting some of this comparison but what you’re trying to find out is, “I know I suck, how bad do I suck?” Or I know I’m dominating, but who is the closest competitor that I need to worry about? What’s their grading score on publishing frequency and content? Some of these things are more relevant than the others but it is important to see it summarized, particularly when you're trying to drive strategy from it.
This is where the pedal hits the metal for all this great keyword research, so we actually use SpyFu to create this first column here of total keywords from every competitor for every client we’re working with at Content Six. Remember that IdeaLaunch is complicated; we’ve got Content Six, our high-end content marketing agency that only works with 30 to 50 clients, and those are the folks who ate pay a lot of money to have even test plans done. Then we’ve got this marketplace where we create content just on a per-article basis and have access to 4,000 writers, and then we’ve got this technology in the middle that’s this Word Vision, that’s powered by SpyFu that helps us track the ROI and the impact of all this content we create.
If we get our act together in 2012, I'd really like to give Word Vision away for free so people at WriterAccess could use it in addition to the people at Content Six, our agency. But anyway, we do all this work and SEO really boils down to this crazy little chart; search engine optimization, search engine, competitive intelligence, all this research you’re doing it comes down to, “Let me quickly and easily find all the keywords that all of my competitors are driving organic traffic from, right?”
That would be a good piece of information; well, guess what? You can get that in 20 minutes at SpyFu; you can download the organic keywords at all the competitors that you have are driving from the search engine. It's magic; it's amazing and that's this first column that says 50,000 keywords, okay? But then we need to pull out the duplicates naturally and in this case we pulled out 15,000 and it's leaving us with like 35,000 uniques, but then we need to filter out a bunch of garbage. Honestly, there are duplicates, there are irrelevant products, there are keywords in there that we would never write content about for a particular client. Products that they may not sell that a competitor might be selling, so the filtering needs to be a manual process. You start with a really simple fast way to find all the keywords. That's the key and the relevance, so this is really how it works now.
When you find that keyword universe of 3,000 keywords, you’re dividing it up into all of these keyword silos and the keyword silos are driven by keyword research, they’re also driven by the input for customers on the types of topics they want to write about, so you’re reverse engineering these silos by looking at all the competitive keywords that are driving traffic. It does end up being a fairly time-consuming process to leave yourself with silos, but once you have those silos and once you’ve done your competitive research and broken it all down, SEO becomes really easy and that’s what Word Vision is helping you do. It’s helping you score content for SEO strength for a particular keyword silo.
We’re not here to talk about Word Vision, but I wanted you to really look at how simple and easy it is to gather competitive intelligence, and make decisions, and answer those four questions, and I just wanted to drill down to show you this. Okay, let’s talk about tools. This is a typical slide that I’ve put together over the years and here’s some of the major tools that I think are relevant out there. Some are free, some are paid; great tools out there. But what I decided to do is to introduce a couple of new tools that I think are worthy of getting on your radar screen, and just kicking the tires up to see how they could help you with this competitive intelligence that we all know we need to do. So, the first one is Search Monitor, which crunches a lot of data. Take a look at it; this deck will be available and we can send it to anybody.
You’re actually going to be able to view this whole show, but I’m just going to spin through these quickly because I want to give Mike a little bit more time to chat; I’m reaching my budget of time. I want Mike, the guru of competitive intelligence, to talk here but another great one is Yahoo! Site Explorer; clearly a nerve center for lots of competitive intelligence, particularly because they will actually report unlike Google, inbound link popularity; those are the actual links.
Compete.com we also know; it’s a Boston-based company. What’s interesting about compete.com is I think you can pretty much get the same information if you pay them like $10,000 a month, or if you pay them like $30 a month; I haven’t been able to figure out their business model quite yet, but they do certainly scrape and have a lot of data. It’s worth taking a look at, particularly their Compete Pro tool to see how it stacks up for the information that you may need, This is a really interesting tool; I’ve been told that it will no longer be serviced, but the concept is actually really, really interesting. Basically, you set this tool up and it spiders any competitive sites or your site, and it will find new pages that are published on that website, and it will throttle that search query as frequently as you’d like.
And at the end of the day, is kind of a really important thing that's happening out there from our perspective and wanting to measure; not just to do our reports initially, but to start tracking and monitoring content being published on competitive sites after you do your initial contact plan. So, it's worth digging into this tool a little bit more; there’s another tool called Update Patrol, which has gotten better reviews with the exception of a disclosure on the top of it that says no technical support.
Again, this is critical stuff if you're in the content creation world and you’re trying to figure out how to win the war of words on the web. You need to understand and know what your competitors are producing from a content perspective, and that data needs to actually make its way to writers creating content for your website. So, we hope to have some advance and some breakthroughs in there within Word Vision because this is critical data and we’re going to talk with Mike about that in more detail, but not getting that data and monitoring fresh content on competitive sites is a key, key thing. I don’t think, frankly, either of these two models that I mentioned are doing the kind of job we’d like to.
Mike Roberts: I just want to throw our hat in the ring on that. Discovering new pages that weren’t ranking on before is actually a section in the SEO recon report. It doesn’t necessarily get pages that don’t rank organically.
Byron White: Yup.
Mike Roberts: But the ones that really rank organically are kind of more important, and nobody does that better than we do. That’s a new thing.
Byron White: Cool.
Mike Roberts: That’s not new, that’s recon files SEO. That’s almost six months old now.
Byron White: Right, right, right. Yeah, what's interesting is the correlation, if we could ever get it, from when they published it and when it got ranked, right? That’s the horsepower right there. You pop a page up, but when does it rank? How long does it take and what does it rank for? And that’s cool stuff.
Mike Roberts: Actually, we can figure that out.
Byron White: Yeah; so, Post Rank. What I really love about Post Rank, and Mike, I’m going to beat you up with this as well. The e-mails they send out daily are both heart throbbers and painkillers; I don’t know what they are. Engagement is a big deal, and by the way, very few people have 4,482 engagement points in the course of a day; just so you know. I've been getting these e-mails since I actually had the founder of the company on one of these webinars and he’s like, “Oh, you got zero today, zero out of ten, twenty” but it’s still good to know when you get that engagement because it tells you that you did something right in the content world.
There's very few monitoring tools other than Post Rank that are telling you how people are reacting to your content, so it’s a very, very great tool to take a look at. Pete compares another tool if you’re running a feed burner, a blog to give you some comparative data. Google trends, another interesting data point. I hate the fact that they don't tell you the actual traffic; you’ve got to go over to Compete to find the actual traffic; it’s a pain in the neck but it may be more accurate. Who knows where Compete is actually getting their data? Here’s how Life Tips compares to About.com. It’s fairly pathetic; like a flat, blue line on the bottom but we still like Life Tips.
And in the way back machine, love looking at this, is an old 2004 Life Tips design that we had up eons ago. It seems like dog years or something; it seems like it was 40 years ago. But, it’s really, really interesting to look at how your competitors have changed their site design and migrated into new offers and to also look at the correlation, for example, of how their traffic might affected by that design. You can start piecing together some interesting data when you look at visual information and not just stats and data. Readability.info is an interesting website. I did a little quick test on here and this, Mike, is diving into some of the actual content analysis that you had done, that I’m referring to. Literally diagnosing sentences, and average lengths, and short sentences versus long sentences and it's only doing it on a particular page, but imagine if you spidered into our website, brought back all this data, and compared performance in the search engines with some of these standards. A lot of people are constantly asking us, “Hey, do we need a 300-word blog post, do we need a 500-word blogging? What’s the right length of a blog post?”
Mike Reynolds: Yeah, it’s funny because it’s very arcane.
Byron White: Should my Coleman-Liau be 18? It’s like, “Wait a second; Google is not judging you on word count, people.” You don’t tell a writer to create a long article for the sake of a long article describing how to walk up stairs. Does someone need to hear how to walk up stairs in 1200 words? I don't think so. It's all interesting, but there are diagnostics, and research, and competitive intelligence that is very revealing, so it’s worth noting. The problem with readability.info is it’s really only doing that for one page. It’s good for perhaps looking at conversion on a page and analyzing their sentence structure versus yours, but it’s not telling us things that we really need to know from a linguistic perspective probably with quality content but it’s worth noting. So, Google alerts are just so easy to set up and so simple to use. It’s harder to do something with all of this data so that’s what we’re going to be taking a look at, but it certainly is a great tool to use.
Twitter search is also fantastic for reverse engineering what people are saying about competitors’ products and/or services. Social Mansion is another interesting that doesn't look specifically at Twitter, although I did a search on IdeaLaunch and all I found were tweets; I didn’t find anything else or any other social media platforms. CM Rush is a great tool; I would classify it as an advanced competitor research tool. In looking at some of this data, like putting in your competitors’ common keywords; that's interesting. How does About.com stack up with Life Tips? That would be interesting to learn, and to quickly find those keywords.
Of course, SpyFu has that overstepping keyword and you can actually pull them out and start doing something with them which is a little more interesting, but it's good to know there’s overlapping information and it can help to drive content marketing strategy. Quadcast is doing some neat stuff; I’ve still got to figure out how these guys are going to make money, I hope they stay in business. I think they’re funded, so they may be okay for a while. Demographic information is particularly interesting to compare. I've always had a hard time trying to understand how you could pull demographic information of an IP address, but maybe there's more to it than I understand. Maybe there's some more science to it in looking at whether it’s a business IP or a home IP, but somehow, miraculously, they pull a rabbit out of the hat with telling you the type of people visiting your website and the demographic profiles. We haven’t been able to do a lot with that honestly, from a content creation perspective and a persona development prospective; clients just do not want to put too much weight on information like that.
SEO Moz; some fabulous new tools including Open Site Explorer, which will literally tell you the backlinks and the sites linking over to a website and the page authority web site. Of course, Word Vision; I can’t leave that out. This is IdeaLaunch and how it’s stacking up against Gent42. We have a little market share report in there that shows a nice little triangle and little circles overlapping; circles much like SpyFu creates, and allows you to actually see those keywords, and take some action on those keywords as far as content creation is concerned. And lots of SEO Moz stuff that I’ll skip over for now, but let me end my presentation and turn it over to Mike. And without further ado, let me pass you the baton Mike, and over to you.
Mike Roberts: Cool; let me see if I need to show my screen; there you go, there you go.
Byron White: I’m going to hit “Okay” and now you’re good to go.
Mike Roberts: Awesome. Again, I’m Mike Roberts; I’m the president and founder of SpyFu. You can hit me up at mrspy. I don’t know if this is a great title for my set of slides here. Really, I may end up hijacking my slides because I have a fun and interesting scenario that happened ten minutes before we went on the air, so it ended up being kind of funny, and I’ll probably hijack a little bit of my own slides, or hijack a little bit of my own examples to take us down that path.
First of all, we’re a competitive intelligence company and I just read on the internet yesterday about some kind of weird things that you can do, and I just want to differentiate what competitive intelligence is versus really creepy shit that you can do, and how you can create a system of values that doesn’t make you do weird shit. So, anyway, then I want to try to take us through some tricks and tools. I want to show you exactly how I use them and there’ll be a “To Be Continued” because I can only go through a couple of different cases. There are so many different things we can do with just trying to figure out what people are up. Finally, I’m going to give you a peek behind the curtain on what we’re doing in terms of research and development on competitive intelligence. We’re actually coming out with a new PPC competitive intelligence recon report, so I’ll show you where that’s at and let you get a peek before anyone else; probably it will go into beta in a couple of a couple weeks.
All right, so without further ado, here's what I found out: 80 percent of the US population is uniquely identified by date of birth, gender and zip alone. You can check this out by this lady here. To me, that’s creepy, right? You don’t want to be able to take voter records and health records and basically do a little SpyFu combat action to figure out whom in your neighborhood has a particular type of cancer. Companies want to be able to do that really badly. I will never be in that business because I think it's creepy. The thing is, is that business itself is a game; life isn’t. Spying on businesses is just part of that game; I think that it's totally legit to do all kinds of crazy spying on your businesses and competitors and so on. I don’t know if it’s a disclosure that I’m making, I feel like I’m talking in a weird sort of disclosure sort of way. Anyway, that’s just kind of how I feel about it.
Byron White: That's the beauty of you, Mike; keep it going, don’t change a thing.
Mike Roberts: So, the first instruction I want to talk about is the most basic one. Everyone wants to figure out about someone else’s website; you want to figure how much traffic they're getting, how big are they, are they gaining traction, are they losing traction, what’s up with it and so you’ve got really the best-of-breed is Compete and Quantcast. You’ve got Alexa, which most people think of as sort of a pile of crap that died in the 90s. Actually, I did that, but it died in the early 2000s, but it actually has been resurrected; it’s actually not too bad anymore. It's got some things that Quantcast and Compete don't have and thus it creates a triangulation effect that I’ll talk about in a second.
I’ll show you how to use SpyFu to help triangulate that because we do our own traffic estimates only for search traffic, how much search traffic comes from Google and so on. You can you those numbers in conjunction with Alexa numbers to calculate actual traffic numbers so it’ll help you triangulate and then finally, what I really should add here is don’t forget what you know about your own website or your own industry. You need to use that to sort of multiply or subtract or add or whatever you need to do to adjust the numbers that you get when you triangulate from various sources because what we’ll find is that oftentimes, no matter what the state-of-the-art is, people are wrong, including us sometimes. So, that's that one.
I checked out your Post Rank thing. So, here’s what I’m talking about. We go to Compete, we go to SpyFu.com; it looks like we’re getting 200,000, this is the shape of the charts. So, you want to look at it in terms of maybe we had more traffic then and now; it's kind of dipped and now it’s coming up. Anyway, what they’re saying is that we get about 137,000 unique visitors. Take a look at it; that’s about a year’s span there. We want to look at Quantcast; Quantcast says something similar, a little bit of a dip and then going back up. They're saying that we’re in the roughly 300, what is it, 340,000 range?
Okay, Alexa doesn’t give us any real numbers; Alexa doesn’t give any actual traffic numbers. What they do that’s useful is they give you the share of traffic that is searched. Nobody else really does that, and you can also break it down by where that traffic is actually coming from, so you can look at it. This is interesting, because 34.12 percent is actually what I use when I’m using this set of data, so I’ll add up 34+4+3 and so on, to come up with that actual estimate because if you go back here, they’re saying it’s about 20 percent, but interestingly, that doesn’t mesh up with what you see here; 34 percent of the traffic comes from Google and Google search traffic so that’s kind of a weird thing. I actually find this set of numbers to be more accurate than this set of numbers.
So anyway, that gives us a couple of triangulation points; we got 137,000 from these guys, 340,000 from these guys and then 20 percent of the search traffic comes from Alexa and then we look at SpyFu. What does SpyFu say about SpyFu.com? We see that there are 2,500 organic visitors per day and so you have 75,000 per month and then, if we want to take the Alexis number, we multiply by five. We get 375,000, so actually all these things sort of triangulate to roughly a similar set of numbers; Compete’s obviously the lowest and actually,
SpyFu is the highest with Quantcast in the middle, but someplace in the 300,000 range. So, that's what you we would come up with as an estimate, but when we actually look at our analytics site, we actually do about 2.2 million visits, so we would actually have to take everything that we know about our industry if we were estimating another competitor to actually multiply by almost 10 to get the right number. So, that’s kind of what I’m talking about with that triangulation piece that I’m doing here, and then finally, adding in what you know about your company.
So, I guess the rule of thumb there is that triangulation is really important with anything you want to do. You actually want to use multiple data sources. I'd love to say that SpyFu should be your only source for anything you ever do and actually, in that example here, we were the closest to correct with 375,000; closer than Alexa and Compete. Really, you should use as many different sources as you possibly can because everybody's going to be wrong in their own special way.
All right, the next thing I want to talk to you about is certifying on your competitor’s technology and their structure and that type of thing. So, the first place I look, or one place I want to look at is builtwith.com; so let’s look at SpyFu.com here in builtwith.com. You can see what servers we’re using and whether or not we’re using Google analytics and conversion tracking. We can look and see the widget on the side; we’re using User Voice and so you can kind of say, “Oh, I like what they're doing there; I wonder how they’re doing it?” You don’t have to develop or reverse engineer it; you can just go look at it and that you’re like, “Oh, User Voice; let me check out uservoice.com” and so on, right?
If you want to look at SpyFu University, not that SpyFu University is a particularly lovely-looking content management system; it’s WordPress. You can see that we’re using WordPress and PHP, and then you can look in there and see all these plug-ins that we have; the WordPress plug-ins and live writer support and so, right? Sort of interesting, pretty cool stuff actually, if you want to figure out how somebody is building stuff. The next best way, really the best way if you want to really find out what kind of research technology people are using, is to look at LinkedIn and look at their job postings. I think you can get what’s going on there because when people post a job posting, they’re basically saying, “Well, here’s what we’re looking for in terms of engineers and we want them to have these skills,” and so it’s like, “Oh, I see, I see; that's how you’re building that really cool product that I also think is cool.”
In terms of reverse engineering company structures. I think that LinkedIn has some really interesting stuff. If you put “Compete” in here, just searching for it, the first thing you see is the company. You can drill into it and see their new hires; they have 171 employees, right? If you want to drill into those employees, you can see who they are, and you can even see where people have changed job titles. What’s cool is you can see their staff breakdown. Are they engineering driven, are they sales and marketing driven? Here, you can see that Compete is actually general administrative and so on, right? You can also see if they’re hiring people with a lot of experience. Well, generally, yeah; it’s 5 to 10 years, so they’re hiring people with a lot of experience. Largely bachelor’s degrees and master’s degrees; really nobody who doesn’t have any of those things, right? So, three percent of their people come from MIT, which is pretty interesting.
So, this doesn’t really work perfectly well; for a company like SpyFu, we’re small; you can’t see a really awesome trend. I think that it works reasonably well for slightly above midsize companies and up to whatever giant company you want to look at, but then things will start to look more skewed. In any case, that's what I would you there. Let's see; I think I’m going to skip this physical stuff, so we can figure out office size and Google drive-bys and Mozilla stuff. I’m going to do a “to be continued” on that, and I’m going to take you quickly through this scenario that I had earlier today.
So, SpyFu, like I just showed you, gets 2.5 million people that visit the site currently. At some point, we had 15 million and we shut it off using robots.txt. It kind of helped us launch and get our brand out there, but at the end of the day, it ended up being a huge support cost because out of a million people, there's like some really stupid people; really insane and stupid, right? You get that combination out a million, you know what I mean? Like actual murderers and weird crap. So, one of my jobs here is to evaluate whether or not we’re going to take anyone seriously when they threaten to sue us or report us to the Indian Mafia or whatever they do.
So, here’s an email that we just got and we just kind of make fun of it in here: “We are Carnal Monstrosity from carnalmonstrosity.com, and you are in violation of our copyright. We did not give any consent to add links to our website on yours. You must remove any links and references to Carnal Monstrosity from SpyFu.com immediately, or we will be forced to press charges.” So, it takes me 10 minutes to figure out whether not to take seriously at all. Pretty much I would default to “not,” but here's the little process that I go through: I go to carnalmonstrosity.com. I’m like, “Okay, okay; definitely leaning towards not taking this seriously.” I try to click around, can’t get anything to do anything, so then I just go to GoDaddy and I check out who it's registered to, find this guy, and then I just Zilo his house. A lot of these people are poverty-stricken and I know that they’ll never be able to afford a lawyer, no matter what like they do, so I just ignore them based on the value of their trailer and that’s one thing that I do.
In this case, he was actually pretty close to the beach in San Diego, and he has a pretty decent apartment so I’m not going to ignore him based on that. I try to search for him on LinkedIn, can’t find him. I ran a quick thing on Carnal Monstrosity on Compete and they don't have any data on it, which makes sense. Then, I look at SpyFu and I’m like, “What is this guy even complaining about?” and there's really not much there. All that we have is this Monstrosity official site, and so on. So then, I ran a little quick search on them on Facebook and couldn’t find them, but then this link leads to this About Us page, and then I started reading this and it’s pretty funny. “I, Jeffery Madden, designed this website from pure scratch with the web programming language XHTML 1.0 Transitional because I firmly believe that XHTML 1.0 Transitional should be the primary language that web designers use for their website.” He also then talks about, “Our musical formula consists of mainly death metal with a lot of thrash metal and a touch of doom metal. Carnal Monstrosity has been auditioning musicians in San Diego since 2009. Many have applied but none have qualified.”
So anyway, I wasn't too worried about this guy and I've basically have decided that I’m not going to respond to it anyway. Maybe it took me three minutes, four minutes to just bust that out. That sort of business decision is actually really common; I end up evaluating competitors, I end up evaluating business leads, I end up evaluating take down notices; Just really quickly going through this series of sites; it’s part of the last seven years or whatever, running various businesses. It’s just one of the dozens of decisions that you have to make all of the time; ignore or act. Running a competitive intelligence company, I don't necessarily recommend that you spend all of your time focused on your competitors, but you need to be able to just quickly decide: are they doing something good or are they not, am I even going to pay attention, are they stupid, do they have experience? These are more or less quick things; down the road, we’ll show you some more quick tricks.
Real quickly now, I want to show you what's going on behind the curtain at SpyFu. I mentioned earlier that our next report is going to be a pretty big blockbuster, I am fairly certain. It's the PPC competitive intelligence report and in that, we’re doing a lot of those things that I'm talking about; really taking and triangulating the right answer, actionable things based on what your competitors in an aggregate are up to, right? You don't necessarily want to trust one competitor, because like I said, some of them may be stupid; I mean really stupid. You don't necessarily want to trust what they’re doing right now because it may be a bad bet.
If you look at the things that many of your competitors have been doing over time and spending a lot of money on, they’re probably making money doing that stuff, so there's a lot of effort involved in actually crunching those things together yourself. What we’re doing is building these awesome reports that basically distill that knowledge and make it so that you can act on them today. It’s something we haven’t been able to do at SpyFu ever because we can really only spend a couple of seconds per month per domain. We’re doing the same thing for 38 million domains, so with the recon reports we’re able to spend hours and days computing on your domain.
So, here's what we’re up to: this is the PPC competitive intelligence report as it sits. Basically, the first thing we want to do is get you a quick overview; there’s going to be a bunch of words in each section and an additional charge. Here's the basic idea: you're able put in your own editors and that's really important in this report because competitors are people you want to emulate. You want their advice, which is a little bit different than in the SEO site, because you can have a tendency to know more. Anyway, what we’re showing you here is basically how you're overlapping keywords or changing over time, whether you’re outranking them, whether they’re outranking you, what their approximate spend is and so on. This just gives you a basic idea of what your competitors are up. That's not much triangulation, right? Then, we get into identifying the best negative match opportunities; and, of course, this report is for newegg.com. So New Egg would've been the one that bought it or whoever is managing New Egg’s account and what we’re going to do is say, “Here's your best negative match opportunity.”
These are keywords that you’re spending money on, that you probably shouldn't be. You’re showing up on these keywords; they’re probably broad matched terms and we think that we can save you money if you stop doing this. So, this is actually based on real data; in fact, you can see that every piece of information on SpyFu is actually backed up by a screenshot of the actual search engine result page so in this case, we’re leveraging that; we’re saying New Egg is advertising on cheap carpet cleaning and look here; they're advertising on 49ers tickets. Together, those two keywords cost them 9,000 bucks a month. Probably they shouldn’t be spending money on it; I don’t think it’s a good idea. We show the ad, and then we show a screenshot and if you click on that screenshot link and here they are on July 4, 2011; and here they are for 49ers tickets. NFL stadium tickets are what they’re trying to advertise on; you can see that they’re showing up on 49ers tickets on July 3.
Again, everything on SpyFu actually tracks right back to our cache. So, when you're talking to a client, it’s kind of cool to just bait them into disagreeing with something you say, because you can just be like, “Well, here's the screenshot, now I’m the smartest person in the room.” Okay, so there’s another section; that’s emerging keywords. I’m not really that fond of this one in this example; I think we need to tweak this particular algorithm a little bit more. In fact, we have. This is the PPC competitor report; there are only three sections right now. We’re going to release this probably with the three sections alone, for free and open beta in a couple of weeks. So, that's what’s going on behind the scenes here. And that’s pretty much the end of my presentation. Do we have any questions going on?
Byron White: Yeah, we’re going to get hopefully some questions from the attendees; I’ve got several dozen myself and not a ton of time left. There’s just a few minutes left, so we like to trim these questions down for the recording. Here’s one question for you, Mike: “Everyone in the world wants to know where SpyFu pulls all this data from?” This is 30 million domains; this is a lot of data. What can you tell us publicly about the data you’re pulling and also, how hard is it to pull and what you do with it when you get it, which is what the really more remarkable thing is?
Mike Roberts: Sure. Well, we basically pull the data directly from Google, and it’s important for us to try to appear to be an actual web browser so we don’t want to use Google’s API; we don’t want them to know where we’re coming from because we don’t want them to manipulate it. That’s for their benefit and ours, to be totally honest with you. SpyFu is a keyword tool that helps people expand their keyword universe and Google makes money, in essence, off the stuff that we do. But we really would prefer to just not have them know at all, and so we go to pretty significant lengths to make it so our searches come in anonymously.
So, yeah, then we get that data and we actually spend 10 days or something getting the information. We store it forever; we store not just the actual data, but we store the actual screenshot of every page that we pull and we have that going back six years. Then, we spend about 20 days of the month trying to figure out things; the important stuff is not, for example, eBay or New Egg, because we’re using New Egg, right? Not that New Egg has 179,000 keywords, but really, what are the most profitable keywords for them? If you look down here, this is kind of a naked sort. This sort here down below, this is just some random keywords and generally what you get from most keyword tools.
We spend the majority of our time trying to calculate the most profitable keywords, and so you see that their brand keyword is right there at the top and that tends to be the case for these big brands even if they have 179,000 keywords but you can bet that “surround sound systems,” “PC laptop,” “hard drive” and “RAM memory” are blockbusters for those guys. So anyway, my point is we spend a lot of time trying to make sense of the data, which I think is actually a differentiator for us. There's nobody else; it’s what we do best.
Byron White: Mike, do think Google is ever going to open up the data in an open source mentality? And if yes, when? If no, why not?
Mike Roberts: Well Google, I think, opens up certain amounts of data, but yeah, it’s a good point. I don't think they’ll open up the data to the extent that you can type in your competitor’s domain and see what keywords to buy; obviously, that doesn’t make any sense. It would be really nice if they provided the number of searches per month, because I’ll tell you, if you pull all of those keywords like we do, you can see that there are only about 140 discrete values that come out of the traffic estimator. A keyword either has 44 searches per month or it has 170 searches per month or it has 220 searches per month; these are all real values that I can remember seeing. But it never has 219, it never has 210; there are only 140 discrete values that you can get out of the traffic estimator.
Byron White: That’s Google’s traffic estimator, yeah?
Mike Roberts: Yeah, yeah. Of course, our job is to try to improve that stuff and triangulate it better, but it would actually be pretty cool if it came out. Yeah, but I don’t know. I actually don’t know for sure why they wouldn’t do that, why they wouldn’t release the real traffic numbers. It seems like something they could do and not ruin themselves and I don’t know why they won’t do it; I have no idea when.
Byron White: Competitive intelligence has typically been a behind closed doors, smoke-filled room sport, particularly back in the days of the supergiant, elephant companies out there competing with one another sifting through trash to find data points. If you read some of the wonderful intelligence stories that have literally changed business models as a result. Is competitive intelligence coming down to the masses now, thanks to SpyFu? Are we able to get information now that can quickly change our business flow? I guess the answer is probably yes, because we saw a lot of that today. Are people really taking this data? Is this the secret sauce?
Mike Roberts: Here’s the thing: I think people want more tactical information that strategic, generally. If you think about it, Southwest Airlines has been the most successful airline for 20 years, and everyone could copy their business model but none of the major players do, right? They're like, “Guess what we’re going to do?” I mean, you’ve got Jet Blue and so on, but it’s only emerging players that actually do it. Nobody actually copies their model, whereas it's the most successful one and has been for a long time. So really, you don’t have to be that concerned about people ripping off your stuff, because the question is whether not they have the brains to act on things, or the guts to act on things.
I think that historically, or currently, there’s just a lot of information out there, and information is harder to process than opinions. At SpyFu, we’re actually kind of working towards giving people our opinion a little bit more. Like in the recon report, we actually say, “Here's what we think you should do, given what we know,” as opposed to, “Here's a bunch of information, you're the smart one.” Well in some cases, we’re also the smart one, so we’re going to give you our best guess.
Byron White: Well, isn't that really what people want, and isn't that the model that really is the future? If you read Ian Ayer’s book Super Crunching, and I’ve had him on the LifeTips show, actually. Fascinating; we’ll have to listen to that again. He is just all about taking all of this data and making it actionable, and putting forth recommendations based on all the data crunching.
Mike Roberts: Yeah, I agree. Actions are what you ultimately want, because it takes so much time to sift through and build an opinion. Just contrasting two sections to this paid keywords area down below here, versus our top 10 best keywords. This is like the way back machine. Honestly, if I look at this page, this is like what we were saying when we owned SpyFu. This is the best information we had; we were like, “Here are the keywords and at that point, that was the state of the art; nobody else was doing it and obviously, we were therefore doing it best. We released this five years ago, so this is fast forward five years; here's how we improved that. I never really thought of it like that until I look at it, but that's coming up with the best actionability. Then, when you see recon files, the PPC competitor report, will basically be the state-of-the-art in terms of giving you actionable information to do with your PPC campaign based on the behavior of your competitors.
Byron White: Well, we’ve had a great fan base and we really appreciate that. When Mike and I are at shows, people come up and say things like, “Hey, really enjoy that, keep it going.” I think it's because they're seeing on these presentations our ability to think this stuff through and start looking at some of these big problems. Point blank, I talked today about exactly what this $25,000 to $50,000 plan is delivering to somebody. They want to know very specifically, “How much content to I need to create, how much is it going to cost me, where do I need to distribute it, how good does it need to be? It's right there in front of us, and what we need from SpyFu is the ability to say, “Here's my domain name, enter, here's my five competitors, enter.”
Okay, let me answer a couple of questions for you like, “All right, grouping tells me you care about these topic areas, but tell me what you really care about.” Then hit “go” and tell me, “Okay, based upon our research of how many pages they’ve indexed versus you and how many keywords in that group versus what you have, and blabity, blabity, blah, you’re screwed unless you produce at least 1,000 to 5,000 new pages indexed to Google by next year. That’s what everyone wants to know; that’s what they want to know right there; point blank.
Mike Roberts: I like it; you know what? We’re actually moving there and just this week, the second question “Enter your five competitors” and I just now added it.
Byron White: You’re getting there; you’re getting there.
Mike Roberts: I think that we will. I think that there are a couple of ways the people use the SEO recon report that were not the way that I expected them to, and one is to do sales proposals; people use them for sales proposals to functionally change their business. We adapted a little bit there; we made it so you can generate an executive summary, because a 50-page report is way too long to give to a prospective client; I mean a three-page report. I think that the strategy one, like the one where we’re actually breaking things into these keyword silos that you're talking about and basically saying, “Here’s almost a blueprint, almost a roadmap for an actual article.” I think that could be pretty big; people are using it for that. Right now, I think we could actually make a tailor-made report. This is actually a template in SEO recon files called Strategy, so you can do a strategy report, but I think we actually make it even a little bit better and sort of cleanup that process.
Byron White: Well, we’re going to keep pushing each other with data and discovery and competitive intelligence overview with good data there. There are some super cool tools out there; you had a great comment about somebody who loved the tips you were giving about Alexa. I don’t know if you saw that or not; thanks, for whoever mentioned that. Using these competitive intelligence tools is what people want to dig into and that's what we've got to get to. How you take this data and quickly apply it to make a difference for your business? The other big pain points that we’re seeing over and over and over again is, “How do I sell content investment, for example, or even investment in any online marketing to my management team, to my board?
My answer, perpetually, is that you have to show them the pain points. You have to show them that you suck right now online, and that you're getting killed; you’ve got to dive into those difficult areas. Mike and I are partnered with Word Vision, but we’re going to arm wrestle each other at the next show we both speak at and say, “What can we do with Word Vision, because Word Vision may be the overlap between our content world and your technology and database world where we just make this stuff simple and easy for people to take action on, because they just want to know, “How do I sell this stuff; quickly show me my pain points.” Super laser-focused, and this is an audience of writers and SEO people. It’s more of a laser-focused audience of people that just want to create content; that's what they want to do.
Mike Roberts: Yeah.
Byron White: So stay tuned for recon reports; we’re over, as per usual. We did have some questions; I’m going to point out that United and Ted while you were talking about another airline. Great job gentlemen and somebody said, “It still sounds like black magic.” Maybe that's what competitive intelligence is; black magic.
Mike Roberts: The reality is it’s an allocation of resources. Like I was talking about, business is kind of a game; it really doesn’t matter when you die. You can’t spend your entire day or your entire life focused on your competitors; you can only spend a certain amount of time, and allocating that amount is a strategic thing for you. So it's fair, but how much effort do you really want to put into it, versus developing your own products and nurturing your own customers and so on?
Byron White: Well, the topic is there for everyone to dive into when Mike and I formalize our October competitive intelligence webinar. There’s absolutely something here on how to get this data, how to use this data, so let’s keep the conversation going and really have some fun with it. Thanks for tuning in everyone; we’re 15 minutes over, a little bit longer today. Thanks for listening in; a lot of people stayed on to listen to us talk, really appreciate that. Thanks for taking your time throughout the course of the day to tune in. Until next month everybody, I hope you enjoyed the webinar. Mike, thanks for tuning in.
Mike Roberts: Thanks a lot, Byron.
Byron White: Right on; thanks everyone for listening in. Until next month, a double month invitation for everyone on the line. We’ll catch everyone next month; thanks again for listening in, check with you later.