Real-time search optimization

With the recent buzz of real time search, most major search engines have been criticized for displaying results from what has been recently referred to as “yesterday’s news.” Specifically, search engines follow a three stage process to display traditional results:

  1. Crawl the web and place the results in a sandbox
  2. Process the results
  3. Place the processed results in the index and make them available for search

Thanks to sites like Twitter, people want their news in real time; the above process is just not fast enough. Depending on your sites authority ranking, the process above can take days. In the best case scenario, a high authority site like CNN.com can have an article ranking within minutes but that still isn’t fast enough. High authority sites can have articles a ranking in minutes, but how long did it take them to produce that article? Typically, traditional news sites produce well formed content with verified leads from respected sources. This takes time; in today’s world, a lot of time. By the time this content is assembled, edited and approved for publication it’s no longer news. Sites like Twitter literally spread breaking news across the world in seconds.

A good example where real time search is necessary is celebrity news. Most celebrity news it now discovered via social search because it is real time (”Social search” is searching social media sites like Twitter).  People are naturally using social mediums to find out about breaking topics like Michael Jackson’s death for instance because they naturally confronted with the trending topics. However, most people still turn to Google to research a topic because we have come to trust the authenticity and reliability of Google’s results.

To combat this potential loss of market share, search engines are working fast to find a way to implement real time results in its traditional web search. The best recent example is Google and Bing’s decision to surface Twitter results in real time. Bing was the first to implement this and was quickly followed by Google. Both search engines have different implementations and they will likely change drastically over the next few months but it will be important to keep a close eye as this develops.

Video of Google’s implementation of twitter in real time search

So what does this mean today? The biggest takeaways in order to remain competitive in of-the-moment content publication are the following:

  1. Produce content fast, refine later
  2. Seed links to your of-the-moment content is social media sites immediately upon publication
  3. Repeat link seeding occasionally throughout the duration of high search volume period, this will allow you to continually show in real time searches
  4. Surface links to of-the-moment content in high prominence locations on section fronts and home page
  5. If you don’t already, begin publishing Google news feeds and submitting feeds to pinging services upon each new article publication – See my post on “SEO through blogs and feeds”

Finally, pay close attention to major search engines implementation of real time search. There is a lot of skepticism of Google’s current implementation because it is easily spammed and doesn’t adhere to Google’s traditional stringent standards. Bing’s current Twitter implementation is actually not included in the traditional search results but instead has its own section dedicated to Twitter search: bing.com/twitter. Although the current real time search implementations will be fluctuating in the near future, there not going away. This has been a topic that has been on the table for some time but now that major search engines have opened the door to real time search, it can’t be closed.

Tags: ,

Metromix launches a new platform in new markets

Metromix is a nightlife and entertainment guide for things to do in your city. Recently, Metromix has launched in 27 new cities on a new platform with some great new features.

The newly launched cities are focused on helping its users to plan their their night. To accomplish this, the new sites have an advanced recommendation system that helps suggest places to go. Once you know where you want to go you can add those places to your plans and lists and share those amongst your friends.

So without further adieu check out some of the new markets below:

Not  in any of the new markets? Check out Metromix’s full list of Restaurants, Nightlife and Event Guides.

Tags:

Load time to affect Google rankings

Google Caffeine may use page speed in ranking algorithm

For a long time people have discussed the likelihood that one day Google will use page load time in its natural search ranking algorithm. We’ve known that Google has been using webspeed as a quality factor in AdWords but until now Google hasn’t officially announced its intention to use load time as a factor in ranking natural results.

WebProNews had the opportunity to talk about the Google Caffeine release with Matt Cutts at PubCon. In the video below, just after the 3 minute mark you’ll hear Cutts confirm that Google is seriously considering using web page load time as a ranking factor in the natural search algorithm.

Google wont be rolling out the Caffeine update to all of its data centers until after the holiday buying season. This actually is good news for most webmasters because it will give them time to address their site speed performance and devise a plan to speed up.

The page speed tools Cutts mentioned are:

Tags: , , ,

Does the canonical tag work in the body?

I’ve had a couple of people ask if the canonical tag (Note: <link> is actually an element, rel=”canonical” is actually an attribute,  but for our purposes I’ll be referring to it as tag in this post)  will work if it is placed in<body> section of the HTML document rather than placing it in the <head>. Well first of all, if you don’t know how to properly use the canonical tag, please refer to my canonical tag example post. This post will attempt to use the canonical tag incorrectly and I would not recommend using it in this fashion.

Regardless, there was significant enough interest out there to test if placing the canonical tag in the body will work the same as if it were placed in the head.

Canonical Head Test

To conduct this test, I’ve created two sets of pages. The first set of pages uses the canonical tag correctly by placing it in the head. These pages are all canonicalized to  http://stroseo.com/test/canonical-head-test. All pages in this test set are exactly the same except their URL is different. Also, to remove any variables, all test pages will only be linked to once with the same anchor text and are not included in the XML sitemap. Consider these variations:

Canonical Body Test

The second set of pages uses the canonical tag incorrectly by placing it in the body. These pages are all canonicalized to  http://stroseo.com/test/canonical-body-test. All pages in this test set are exactly the same except their URL is different,  Consider these variations:

Canonical Free Test

This set of pages doesn’t use the canonical tag. These pages are necessary to prove that Google will index all of the different variations.  All pages in this test set are exactly the same except their URL is different,  Consider these variations:

Canonical tag usage conclusion

My hypothesis is that the canonical tag will not work when placed in the body section. I’ve used this same advice when asked previously but I didn’t have the proof.

The results of the test depend on the pages included in Google’s index. If the canonical tag works in both the head and the body, when you Google site:stroseo.com/test/ you should see two results, one from each of the canonicalized sets. Also, we should see each of the pages from the canonical free test set.

    This test was launched on October 25, 2009 and will take a couple of weeks for a solid conclusion, so check back soon.  Regardless of the outcome I would still never recommend using the the tag incorrectly. Even if it does work, that doesn’t mean it always will. But for the case of curiosity, this test exists.

    Tags: , , , , , ,

    Swine Flu Search Trends

    Today, (October, 24 2009) President Obama declared a Swine Flu national emergency.  In recent years, the number of people infected with the normal flu peaks in October. The question that many people are wondering is whether or not the Swine Flu, also known as H1N1, will also peak on October or continue its pandemic nature and continue to spread further into the fall.

    As more and more people begin to become infected or know someone who has, they naturally want more information such as flu vaccination locations, symptoms, outbreak locations, deaths and more. To answer these these questions, these information seekers turn to search engines for help. We can use the frequency and locations of these searches to help determine the severity and help predict the outcome or reoccurring events such as the flu.

    The graph below compares search volume trends in the U.S. from January 2004 through October 2009. Specifically, it shows search trends for the following keyword searches:

    • Flu
    • Swine Flu
    • H1N1
    Web Search Trends for Flu, H1N1, and Swine Flu

    Web Search Trends for Flu, H1N1, and Swine Flu

    The chart above shows a very consistent trend for the normal flu. Over the past 5 years we have seen annual flu  search volume peak in October which correlates closely with the CDC’s medical records as the peak of flu infection. It should be noted that this is a relatively small window in the grand scheme of things. The CDC also says “the peak of flu season has occurred anywhere from late November through March.” If you look back at the image above, you can see that the 2007-2008 flu search volume peaked in February of 2008.

    So back to the main question: when will we see the severity peak of the Swine Flu pandemic. The data above only shows us information concerning the “normal” flu and search engines didn’t exist during the last out break of Swine Flu in 1976 -1977.  However,  traditional media sources such as magazines and newspapers did report heavily on the last Swine Flu outbreak. Many of these traditional media companies digitized their archives of old articles including their publish date. With Google being the data beast it is, it has made all of that information searchable and allows us to trend that information. Consider the following charts that display Swine Flu news article frequency:

    Swine Flu News Archive Trend

    Swine Flu News Archive Trend

    Now we have historical data that shows the Swine Flu peaking in October. Also we can see that news articles began to surface in early March, just as we have seen in 2009. So will these Swine Flu outbreaks have comparable end dates? Only time will tell.

    There are countless places to turn today to help us evaluate and track the flu’s spread, consider these resources:

    Tags: , , ,

    SEO is a Scam, SEO is Spam

    Is SEO nothing but a scam? Have you recently been overcharged for an under performing SEO campaign? Before you jump to conclusions about your SEO misfortune, consider this recent article by Danny Sullivan titled Thoughts On Web Developers, SEO & Reputation Problems.

    Frankly, I’m tired of receiving to emails from colleagues labeled “FYI, Interesting view on SEO” which usually turn out to be an anti SEO spit about how it SEO is a Spam industry. Typical titles for these types of articles can be one of the following:

    • SEO is dead
    • SEO is a scam
    • SEO is spam
    • Is SEO worth it?
    • SEO is easy
    • SEO is a joke
    • SEO is useless

    I don’t believe however that most of the people who send me these emails and tweets are really trying to suggest that the SEO profession is a joke. At the same time, I’m not sending notes to our traditional media marketer asking why he can’t show ROI on his advertising spending. I also won’t send an email to our developers asking them why traffic from search engines was 80% lower than it was before we “fixed” their mistakes.

    Rather, my monthly reports tell that story. If someone sends me an article about a “unique view on SEO” I can comfortably say, “open last months SEO Dashboard” and then tell me that SEO is really a scam. A good SEO proves their value in reporting.

    SEO is no different than any other marketing position in that if you want to target a group of people, it’s best to hire a person specialized in doing so. In a traditional corporate environment, you wouldn’t ask a member of your analytic team to create a direct mail marketing campaign. He/she might be able to analyze the data in reference to a change in the marketing campaign but that doesn’t mean he/she is a good candidate for direct mail marketing. Similarly, you shouldn’t ask a developer to be responsible for both developing a site’s core functionality and for to be responsible for targeting millions of potential customers found in search engines.

    Bottom line, SEO is not a scam. If a company is naive enough to hire a consultant that can’t show performance, cut them loose. If it’s too late and you already spent enough to consider yourself scammed, than shame on you. As a company, do your research on who you’re hiring. If you’re in a corporate position and have the responsibility to purchase goods or services, you know competence and quality are essential factors in your decision making. If you had to buy a fleet of cars for your sales staff, are you going to buy cars from a “Bobs Auto Sales?” Of course not. Similarly, you shouldn’t hire Jimmy SEO who works out of his parent’s basement in Montana.

    Thanks again Danny for taking the time to write a response to the critics. The next “FYI, Interesting view on SEO” email I get will get a reply with nothing but a link to your article.

    Tags: , , , , ,

    Enhance Your Google Search Results with Rich Snippets

    Have you seen enhanced Google search results like these lately?

    Yelp Rich Snippet Enhanced Search Result

    Yelp Rich Snippet Enhanced Search Result

    LinkedIn Enhanced Search Result

    LinkedIn Rich Snippet Enhanced Search Result

    Youtube Rich Snippets Enhanced Search Result

    Youtube Rich Snippets Enhanced Search Result

    Are you wondering how your can get your results to be enhanced like this also? Well it’s pretty easy really, and here’s how:

    Google has been quietly testing these enhanced results with a select few companies and you can apply for consideration at Google’s Rich Snippets Feedback Page.

    If your company website is authoritative, contains useful information and you have applied for consideration at the link posted above, a Google representative will reach out to you via email. They will attempt to schedule a call with you and it’s best to move your schedule around to meet theirs because they tend to be pretty busy.

    The first thing they will ask you is whether you’re familiar with rich snippets and how they work. So make sure you are to expedite the process. It’s best to have a member from your product development team, a rich snippets savvy developer and your SEO guru in on the call.

    For those of you not familiar with rich snippets, they basically are supplemental front end HTML tags that tell crawlers what information they are about to crawl. So first of all, you’ll need rich targeted snippets of information displayed on your web pages, after all there needs to be something to tag.

    You’ll also want to decide which format you want to mark you pages up with. You have a couple of options here: Microformats or RDFa.

    Microformats are simple, easy to implement and can mark up a wide range of content types. It is compatible with HTML version 4 and up. Not officially a standard but has surpassed RDFa in popularity.

    RDFa is also a fairly simple mark up but is a little heavier in terms of additional page weight (more lines of code). It is technically announced to be compatible with XHTML 1.1 but the W3C has announced they expect people to use RDFa with older versions of HTML.

    It’s your choice what you use, both are fairly robust but I personally recommend microformats strictly because of its simplicity. It’s also worth noting that both formats are compatible with Yahoo Search Monkey.

    Once you’ve chosen your rich snippet mark up syntax, you’ll need to figure out what you’ll want to mark up. Both Rich snippets and RDFa allow for mark up for the following types of content:

    Rich Snippet Categories

    • People and Organizations
    • Calendars and Events
    • Opinions, ratings and reviews
    • Licenses
    • Tags, keywords, Categories
    • Lists and Outlines

    These categories are not all supported by Google but the representative that contacts you will help you determine what you’ll want to mark up on your pages. However, you may want to mark up everything you can because other utilities such as Yahoo! SearchMonkey do accept all available attributes.

    If you have specific questions about Rich Snippet Markup, you can always contact me or refer your developer to the microformats.org or W3C page on RFDa.

    Tags: , , ,

    Google Place Pages to Compete with Destination Sites

    This post is particularly concerned with Place Pages for Businesses and the destination sites such as Metromix, Citysearch and Yelp that provide Google with the information to populate these pages.

    If you aren’t familiar with Google Place Pages already, here is Google’s definition of Place Pages:

    “A Place Page is a webpage for every place in the world, organizing all the relevant information about it. By every place, we really mean *every* place — there are Place Pages for businesses, points of interest, transit stations, neighborhoods, landmarks and cities all over the world.”

    Mashable.com was one of the first to post about how Place Pages will Rival Yelp. Well, it’s worth noting that that the sites that are supplying Google with the information to are doing so voluntarily.

    Many of these sites were not aware of Google’s intentions to aggregate competitive information into one place. Some are fearful that this aggregation of information will cease a searchers need to visit the content originator’s site. Many of the content originating sites that are providing Google with this information through Google Local Feeds are heavily reliant traffic from Google. If the search giant becomes more “sticky,” it may cause some of these content originators’ ad supported business models to fail. Since Google does not own the information it is displaying, and if many of these businesses fail because of heavy traffic losses, the available content on Place Pages will become much thinner as these failed businesses stop providing Google with the business information feeds.

    Ironically, although these sites are voluntarily providing this information to Google, it’s too late to stop now. Google has already launched its Place Pages project and now these sites must move forward with optimization of their local feeds or they will not appear as sources in Google’s Place Pages.

    Even large sites like Citysearch and Yelp who have a following of repeat visitors cannot deny the revenue importance of traffic received from Google. Not providing Google with local feeds may be a death sentence because of the potential lost traffic. It’s amazing, though not surprising, that Google was able to create such a competitive landscape without any of its own hard work of gathering the nation’s localized data.

    So how will this help/hurt these local destination sites?
    Larger destination sites like Yelp and Citysearch who already have a steady source of traffic from repeat visitors will not see an immediate effect from Google Places. They may have to reformulate their growth projections in a much more modest fashion however. Most of the loyal repeat visitors that come to sites like Yelp and Citysearch originated from major search engines like Google. If new users looking for localized information are sticking around Google rather than moving on to specific destination sites, long term growth will be less optimistic.

    Smaller sites that are supplying Google with optimized local feeds will get more visibility and potentially improve traffic in the short term. Long term growth goals are also less optimistic though. Unless these destination sites can differentiate themselves from being traditional Yellowpage type information providers, long term potential growth is dismal. Users will ask the question, “Why go to example.com when Google provides the same information?”

    Will Google Pages really make that much of a difference?
    Only time will tell, but it’s apparent that Yelp believes that Google Place Pages is a game changer. Why? Well consider that Yelp has, for the time being, decided not to provide Google with extensive localized information.

    On the other hand, most people aren’t using Google maps for local restaurant and business information. Maybe now they will, but the majority of searchers still rely on natural search to find their restaurants and local businesses. To that point, these Place Pages will not be visible in normal web search. Place Pages have individual URLs but have been disallowed in Google’s robot.txt file making Google Place Pages non-indexable.

    Tags: , ,

    Google Place Pages – Disallowed in Googles Robots.txt

    Google's Robots.txt File

    Google's Robots.txt File

    Google Place Pages are Not Crawlable

    Many sites are blogging about how Google Place Pages are posing a major threat to Yelp, Citysearch, Metromix etc. Most blogs state 2 reasons why these new Place Pages are going to “rival yelp.”

    1) All the information you’ll ever is need aggregated and posted within Google Place Pages, therefore you’ll never need to go to the source originators such as Yelp, Citysearch or Metromix etc.

    2) Google’s Place Pages are going to show up in natural search results and compete against Yelp, Citysearch, and Metromix.

    To keep this post short and sweet, I’ll only be discussing the second point. Google’s Place Pages are not indexable because Google has disallowed them in their robots.txt file. If you go to google.com/robots.txt and scroll down to line 102, you’ll find “Disallow: /places/”

    So bottom line here, Google Place Pages will not show up in natural search results.

    Tags: ,

    Geo Targeting with IP Delivery

    SEO Friendly Geo Targeting with IP detection Delivery

    Today, many sites specialize in localized content. To improve user experience, it’s becoming common practice to serve visitors with the most appropriate content based on the user’s physical location. There are multiple ways to determine a user’s location but today, the most reliable way is by sniffing the user’s IP address and cross evaluating that with an IP location database.

    It’s worth noting that IP addresses aren’t perfect at determining a person’s location. An IP is dependent on the user’s internet service provider (ISP) and can be easily manipulated. Until recently, the early 2000’s, geo targeting has only been used to determine a user’s country. However, as the web shifts to content hyper-localization, IP delivery is being put to the test. At this time, late 2009, most IP addresses are accurate enough to geo locate a user down to a city level.

    Generally, IP detection should only be used for new visitors to your website. Once the user has entered your site, their preferred location should be stored in a cookie. It’s important to allow the user to “change location” and if the user does so, their preferred location should change and the cookie should be updated. That said; make sure your main navigation clearly shows your visitor that his/her location can be changed. It is also important to have a “Change Location” landing page that is fully accessible to search engines. This will allow the search engine to access all sections of your site.

    Is IP Delivery SEO Friendly?

    From a SEO’s stand point, initial thoughts on using IP Delivery typically involve a red flag. The main cause for concern is that you don’t typically want a search engine crawler being redirected to a specific portion of your site based on the crawlers IP address, especially if you’re using IP addresses to direct your visitors to a hyper-local portion of your site. This is because search engine crawlers are not hosted with IP addresses from every country and definitely not every city and you run the risk of not having your site indexed as efficiently as possible. So the answer to the question “Is IP Delivery SEO Friendly?” is no, at least not without taking into consideration the side affects mentioned above.

    Redirection of search engines using User Agent Detection

    One technique to prevent search engine crawlers from being redirected to a certain area of your site based is by User Agent Detection. A user agent is a name given to the software that is being used to access the internet. For instance, your internet browser name may be “Mozilla Firefox” or “Internet Explorer.” When a search engine accesses your web site they have names like googlebot, msnbot and slurp. Therefore, it seems that detecting a search engine via user agent and preventing IP delivery is the easy answer to make IP delivery SEO friendly. Unfortunately, it’s not that easy. Search engines have clearly defined this technique as cloaking. Whenever a site shows different content to visitors based on a variable such as location it runs the risk of being penalized by search engines for cloaking. Cloaking is typically a black hat SEO technique and a violation of Google’s webmaster guidelines. Search engines consider a site to be guilty of cloaking when the content presented to the search engine spider is significantly different to that presented to the user. Search engine penalties for cloaking are severe and include removal from the search engine’s index. To avoid such penalties, view Google’s video presentation on IP Delivery.

    Basic Google rules for IP Delivery:

    1. Do not auto translate content’s language based on IP. (Rather use the browser’s preferred language)
    2. Serve Googlebot the same content that a user would see
    3. Create Separate URLs for significantly varied content
    4. If subdomains or subfolders are used in your site structure, use webmaster tools to set preferred geo location. (Note: at this time, late 2009, webmaster tools only allow you to set preferred location down to the country level, not regional or city level.

    Basic Stroseo rules

    I rarely would suggest bending the rules that Google has set forth because you are at risk of being penalized, but this case is an exception. Most sites that use IP Delivery do so on the homepage. This is typically your most authoritative page because good portions of your inbound links from other sites are likely pointing at your homepage. With that, it is essential that you are in control of the distribution of that link equity. You’ll want to be able to evenly pass down that authority to each section of your site and Google’s suggestions for IP Delivery makes that very difficult. With that, follow these rules when implementing IP Delivery.

    1. A landing page at the root level of your site is required i.e. www.example.com. That page can be a simple set of text links that serves as a router for search engines, and users if your IP Delivery system fails, to different areas of your site. Nofollow all unnecessary links on this page to ensure maximum link equity transfer to the most important pages on your site. Use text navigation rather than images.
    2. Only redirect users via IP Delivery if the user has both Cookies and JavaScript enabled. Standard search engine crawlers don’t have Cookies enabled and definitely wont have both cookies and JavaScript enabled. This will have a similar effect on search engines as User Agent Detection but without the potential cloaking penalties involved. You are legitimately preventing IP Direction because of the user’s lack of cookie usage, and you would prefer to have the user pick his/her own location rather than potentially guess incorrectly, especially for repeat visitors.
    3. If the visitors Cookies and JavaScript are enabled, redirect users with 302 redirects rather than 301s. If a search engine somehow does manage to be redirected via IP Delivery, it will only be temporarily redirected. Search engines occasionally crawl the web in disguised user agents to test for cloaking. If you are redirecting with temporary redirects you won’t be penalized since you’re reporting that the visitor is being 302 temporarily redirected and that if the visitor would come back later, the same redirect might not be in place. If you use 301 redirects, then you’re telling the user that your homepage is going to be permanently located at the new location and the user/crawler should no longer attempt to index or go to your homepage. Also, temporary redirects don’t transfer your link equity while 301 redirects do. In this case we don’t want our link equity transferred via IP Delivery; rather we want to distribute it via natural crawling.
    4. Create a “Choose Your Location” landing page and host it at a location similar to the following: www.example.com/choose-your-location. You’ll want to prominently link to this on every page, preferably in the header, across your site to ensure proper link equity transfer to all sections of your site. This is your fail safe, regardless of what happens with your IP Delivery, search engines still need to be able to traditionally crawl all sections of your site.
    5. And that is it, finally! There was a lot of information covered in this section and a lot more could have been discussed. IP Delivery is a technique that will likely be used more and more in the near future until a more reliable approach is available. Also, it will be important to stay updated on recent search engine policies regarding IP Delivery and cloaking.

      As always, if you have any questions or specific implementation questions regarding your site, feel free to Contact Me.

    Tags: , , ,