Encoded

Google announced late Monday evening that is has launched a crawler that can dig through Flash content to expose the text found inside the Flash movie.

For years, sites have been going to great efforts to expose content inside of Flash movies to web crawlers. Most often, websites would create one experience entirely in Flash and a carbon copy of that experience in HTML solely for google bot. 

Other methods of opening up Flash content involved powering your Flash movie with XML. You would use PHP or another server side language to parse the contents of the XML and append that data to a page. Or, if you’re site was mostly static you could just add the text into the HTML by hand.

Googlebots inability to crawl Flash caused much frustration to site designers and developers who needed their sites found. However, for those sites that didn’t want to be found, Flash - like Ajax -provided a benefit. 

One tactic we used at AOL in order to target pages for SEO was hiding non relevant content behind JavaScript / Ajax and Flash. This guaranteed non relevant information wouldn’t distract from the page target. For instance, on this video, the page is attempting to win on the video title ‘Makes Me Wonder’ not on Maroon 5, and not on any other terms.  AOL Music wanted to win on very specific terms while also providing a robust user experience, because of this, items like the ‘Poll’ in the right column and the ‘More on AOL’ at the bottom of the page are pulled in via Ajax in order to keep crawlers out. 

If Googlebot now has the ability to crawl Flash and perhaps (or maybe already) JavaScript, site designers may become unable to control what is indexed on their site. Or worse yet, may be forced to limit user experience in order to win in SEO. Back in September, I raised this concern in my post ‘That Monster Called SEO‘. It’s problematic that in order to win in SEO via the bots algorithm, one must degrade the overall user experience.

Matt Cutts also blogged about the expansion of indexing Flash sites, here. However, he didn’t offer any insight on how Googlebot will see content in Flash or how Google will link directly to content that may be pages / clicks deep into a complex Flash site. Hopefully, these critical details will soon become available.

Update: Lee Brimelow has posted on The Flash Blog, here, that Yahoo! will also be able to crawl and index Flash files in the future. Lee is also saying that no best practices have been created yet. Looks like it will be another long, slow road of trial and error, which *could* drastically effect the way in which Flash SWF files are authored.