Content deleted Content added
limitations; cleanup |
m Tweak; |
||
Line 54:
== Limitations ==
WebCite honors the [[robots exclusion standard]], as well as no-cache and no-archive tags and will not archive sites that disallow archiving. For example, ''[[The New York Times]]'' has a robots.txt page at http://www.nytimes.com/robots.txt which includes:
:<code>Disallow: /archives/</code>
Thus, archive requests for this site will be rejected.
|