Understanding ROBOTS.TXT (which will help search engines understand you)

So search engines are tricky to understand, especially if you want the search engines to find your site and people to find your site via the search engines. One of the things you may want to do is restrict pages that the search engines find, that might seem silly to exclude yourself but I don't want people inadvertently finding my site.

Here's an example, I don't let^* the search engines search my monthly archives. Because on the page of 20-50 unrelated posts the information on that page is pretty much unrelated by the end of the month. In December 2006 the mention of “iPods” on the 3rd, has nothing to do with any of the mentions of “family” or “Christmas” but if some searched for “iPods” & “family” or “iPods” & “Christmas” they may come upon that page and while I want traffic I'm not trying to lure people here under false pretenses. This is why I have categories like “Apple” and “Friends + Family” that will have related posts in one location (and even then it's still kinda a wide range of articles).

So far I've just told you why you want a ROBOTS.TXT file but I haven't told you how and I'm not planning on it, because Google just put together on “Controlling how search engines access and index your website with ROBOTS.TXT”. It's got a lot of links that take you all over the place but there's lots of good info there.

If you're just getting started don't get confused with all the “User-Agent:” rules, just use “User-Agent: *” for now and that will apply to all search engines (that's all I still use although I'm going to start working on getting some of my pages more mobile-ish).

For another example, I don't index my Search Results page, this page lists what search words people have used to find my site. Because it those pages were indexed it would just kind of create a self-fulfilling loop of search results.

While on the other hand I do allow my “Talked About” page to be indexed since it doesn't seem to give a lot of false positives to the search engines. plus once someone gets there they'll quickly see that it's probably not the page they're looking for. But if they did put in some eclectic words that they see I mention often, they might hang around a bit.

* Please note that no search engine is obligated to find your rules, but it they want good results, they'll probably want to follow your suggestions in your ROBOTS.TXT file. People wanting to steal content from your pages (text or images) will still traverse all the links on your site.

Montezuma Waterfalls (the Movie)

So here's the Montezuma Waterfall that we climbed way up into the hills to get to. We swam and (I) jumped into the pool there and then we climbed way farther up into the hills to get way above the waterfall (and there were a few more pools up there too) and then I climbed down pas them so that if you were where I was when I took this movie you'd seem me at the top of the falls.

I know I'm behind in getting stuff up, but I'm getting there. FYI, I rotated this movie, so if it doesn't play, can you please let me know?!?

Technorati Tags: Costa Rica, Montezuma, Vacation, Waterfall

Technorati Tags

So I'm playing with Technorati Tags on my pages (you'll see some attached to some of my posts) and I'm not liking how they are working from a placement standpoint. I am experimenting so we'll see where they get moved to (or if I even keep them).

Tags are supposed to do two things (IMHO):

Tortuga Island

So almost everyone was giving us mediocre reviews of the volcano this trip, so we decided to skip it. Since we skipped it we were staying in Montezuma for another day and then going back to Jaco.

Staying in Montezuma (Costa Rica) resulted in us taking a tour to Tortuga Island. Every travel group in town seemed to offer a similar tour to this. A trip on the water (hopefully spotting some whales/dolphins) some snorkeling and some time on the beach with with lunch cooked there and then some more (optional) snorkeling and back along the coast. I'm glad we went but the day went by really fast.

The photo with the island (big rock) in the middle is not Tortuga, it's nearby and it's where we snorkeled (if you look real close you can see people snorkeling). Now remember where we are, there aren't real lessons. They hand you some flippers and a mask and a snorkel and off you go! I asked some of the other tourists and got the gist of it but was always afraid of somehow getting a big gulp of water somehow (which never happened). The fish were awesome, different colors and if you stayed still you could be in the middle of a big school of them. Different colored rocks and coral, it was all very cool. Now one some of the people who had gone down to the Caribbean (or other exotic snorkeling places) were a little disappointed but the rest of us were happy.

I also saw some kind of Eel and some kind of Sting Ray. The eel was a squiggly but I'd guess it was about 3-4 feet long and the Sting(?) Ray was probably a little less than 3 feet (from wing tip to wing tip). They were 6-10 feet below me so it's hard to judge the size.

The beach was nice, set up as if you were outside a hotel on the ocean. The rest of the island was a “state park” of sorts (and had admission costs, in hindsight probably pretty cheap) but the day went by so fast, there just wasn't time for exploring.

And they made us a pretty good grilled lunch too!

All the photos in this post are clickable for larger versions.