jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Files
7b4bab54aff859266cb4c4a6dcbf059f2478da3a
danbooru/app/views/robots
History
evazion 30b7345900 robots.txt: disallow crawling version pages. 
Some of these pages were accidentally crawled because rules like `Allow:
/artist` allowed `/artist_versions` to be crawled (Allow rules are
prefix matches).
2021-02-07 23:28:50 -06:00
..
index.text.erb
robots.txt: disallow crawling version pages.
2021-02-07 23:28:50 -06:00