Files
danbooru/config
evazion 74d77c1e23 Update robots.txt.
* Generate /robots.txt dynamically.
* Include link to sitemap.
* Update list of allowed urls.
* Disallow crawling of non-canonical subdomains.
2020-01-31 02:43:07 -06:00
..
2020-01-07 11:49:46 -06:00
2019-11-26 01:58:57 -06:00
2020-01-07 11:49:46 -06:00
2019-08-27 18:13:30 -05:00
2018-04-06 18:09:57 -07:00
2020-01-31 02:43:07 -06:00
2017-01-24 14:32:06 -08:00