# Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name; "*" is for everyone # Disallow: if this matches first part of requested path, # forget it # Allow: put prior if one file in a directory that will be disallowed # from http://www.wdvl.com/Location/Search/Robots.html Disallow: /old_site Disallow: /mys-new Disallow: /Not_Used