Sunday, January 22, 2012

Struggling with Crawling

I've tried to implement the approach here:

http://code.google.com/web/ajaxcrawling/index.html

I've done two things:
  1. look for the _escaped_fragment_ query parameter and use it to generate a non-ajax view of the site
  2. added the meta tag in my index.html:
<meta name="fragment" content="!">

If I visit my site with _escaped_fragment_ as a query parameter, a snapshot is correctly delivered to my browser.

However, when I use the Google Webmaster tools to "Fetch as Googlebot", I just get the raw (and content free) index.html text: I never get the snapshot.  Maybe that's what I should expect, I don't know.  In any case, if I do a google search for some of the words on the first page, I don't get any results.

I've just added some counters to detect the Googlebot (by looking for that string in User-Agent) and the _escaped_fragment_ parameter to see if I'm getting either, but the Webmaster tools do say that my app is being crawled.

If I've just implemented it incorrectly somehow, or if there's a major step I'm missing, that would be much appreciated feedback.

Thanks!

--
You received this message because you are subscribed to the Google Groups "Google Web Toolkit" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-web-toolkit/-/govRGQh76wEJ.
To post to this group, send email to google-web-toolkit@googlegroups.com.
To unsubscribe from this group, send email to google-web-toolkit+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-web-toolkit?hl=en.

No comments:

Post a Comment