4 Months, Several Revisions, and no listing on Google. Not even using Site:

Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > 4 Months, Several Revisions, and no listing on Google. Not even using Site:

Space Skeleton: Sep 28, 2004

Try adding a robots.txt?

# ¿ Dec 7, 2014 05:10

Adbot: ADBOT LOVES YOU

# ¿ Apr 20, 2024 00:04

Space Skeleton: Sep 28, 2004

sharkytm posted:

No robots.txt means that the spider just crawls every link, which is just the only page there.

That's the default behavior yeah but there are ways for a server to tell crawlers to take a hike so to be safe it's a really good idea to include a robots.txt in the root directory of each site anyway. From minor experimenting I have found that Google seems to go with what "www.sitename.com/robots.txt" says even if it finds conflicting instructions elsewhere.

# ¿ Dec 8, 2014 20:52

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > 4 Months, Several Revisions, and no listing on Google. Not even using Site: