jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
7,588 Commits 1 Branch 0 Tags
c301c316bfbbf3aa3f64e6c1f3a8f2154cd7ddbb
Commit Graph

1532 Commits

Author SHA1 Message Date
Albert Yi
12b88f7f97 fixes #3960 2018-10-17 15:37:47 -07:00
Albert Yi
2aabaf56d8 Merge pull request #3951 from evazion/fix-maintenance-logging 
Improve daily maintenance error handling
 2018-10-15 14:20:44 -07:00
Albert Yi
a85f3773e3 fix nil commentary case for pixiv strategy #3948 2018-10-12 14:35:44 -07:00
Albert Yi
d15c0b8266 fix delete file delayedjob 2018-10-11 16:43:16 -07:00
Albert Yi
185d303ba1 eliminate tag cache expiry delayed job 2018-10-11 16:36:50 -07:00
Albert Yi
ca7e7e2877 add additional servers to cloudflare expiry service 2018-10-11 11:42:15 -07:00
Albert Yi
6ca9b65716 add dynamic sitemap.xml 2018-10-11 10:13:54 -07:00
evazion
5e57cccd18 maintenance: log errors to rails log and newrelic. 2018-10-09 20:11:42 -05:00
evazion
5201954413 maintenance: consolidate maintenance tasks in logical/maintenance.rb. 2018-10-09 20:11:23 -05:00
evazion
16b1b72da5 tumblr: fix video urls not being recognized. 2018-10-09 12:44:59 -05:00
evazion
184a5ebf3e tumblr: fix _640 images not being recognized (#3944). 
Fixes _640 images not being matched by the IMAGE regex and therefore not
being rewritten to the largest size.
 2018-10-09 12:44:59 -05:00
evazion
d874c68419 tumblr: fix image_urls when api data is unavailable. 2018-10-09 12:44:59 -05:00
evazion
b0d7d90103 tumblr: extract info from url when api data is unavailable. 
Derive the artist name / profile url / page url from the source URLs when
the API response is unavailable because the Tumblr post was deleted.

This fixes the artist finder to work on bad_tumblr_id posts.
 2018-10-09 12:44:59 -05:00
evazion
0c31a5d6a9 tumblr: don't fail when api data is unavailable (#3948). 
The api data is unavailable when the work is deleted (bad_tumblr_id), or
when the source is a direct image url with no page referer.
 2018-10-09 12:44:59 -05:00
evazion
4c55c809b0 tumblr: don't fail when api key isn't configured. 2018-10-09 12:44:59 -05:00
evazion
8aed1e19b9 tumblr: remove obsolete cdn rewrite rule. 
This cdn is no longer used by tumblr.
 2018-10-09 12:44:59 -05:00
Albert Yi
f065186c32 change default server login from albert -> danbooru 2018-10-08 15:51:32 -07:00
Albert Yi
7eab2fc0a7 disable server sync 2018-10-08 15:51:32 -07:00
evazion
5d77f1db08 daily maintenance: remove dead tag pruner code. 2018-10-06 14:27:53 -05:00
evazion
c8d538f618 moebooru: delegate to substrategy based on post source (#3911). 
If the yande.re or konachan.com post has a source from a supported site,
for example Pixiv or Twitter, then delegate the artist and commentary
lookup to that substrategy.

Only do this for sources from recognized sites, not the null strategy.
 2018-10-06 14:27:49 -05:00
evazion
e5a4193dd4 moebooru: support batch bookmarklet previews (#3911). 2018-10-06 00:58:22 -05:00
evazion
864349dc7b moebooru: fetch tags (#3911). 2018-10-06 00:58:22 -05:00
evazion
7eee77d520 moebooru: proxy images to avoid hotlink protection (#3911). 
Fix image previews being broken on the upload page due to yande.re not
allowing hotlinking of images.
 2018-10-06 00:58:22 -05:00
evazion
c78dece411 tag corrections: remove distributed cache logic (#3943). 
Remove logic for displaying the state of the tag category cache on both
servers.
 2018-10-04 13:46:10 -05:00
r888888888
abbe181de6 add cronjob for syncing files 2018-10-04 01:51:42 -07:00
r888888888
a45bd52e89 delete files job should move to default queue 2018-10-04 01:08:30 -07:00
Albert Yi
27072d695d add lcms2 dependency in install 2018-10-03 13:52:04 -07:00
evazion
88a177e1d5 TagSetPresenter: refactor humanized_essential_tag_string. 
Move Post#humanized_essential_tag_string to TagSetPresenter#humanized_essential_tag_string.

This allows humanized_essential_tag_string to reuse the same set of tags
already fetched by the tag set presenter for the sidebar.

This avoids fetching the tag categories from memcache again (via
Post#typed_tags) when we're already fetched the tags once before.

This also means it's no longer necessary to cache humanized_essential_tag_string
itself in memcache, since it can be generated as quickly as the sidebar taglist.
 2018-09-30 21:52:24 -05:00
evazion
984cd0432c Fix #3928: fix case sensitivity in metatags. 2018-09-27 23:42:30 -05:00
Albert Yi
6a375d04e0 fix indentation 2018-09-27 16:58:22 -07:00
evazion
f5c9616f62 tests: fix broken tests. 2018-09-24 17:34:08 -05:00
evazion
d9063a9f2a nijie: support preview urls (#3919). 2018-09-24 17:08:37 -05:00
evazion
52704ef6cd nijie: take artist id from url when possible (#3919) 
Fixes the artist finder to work on Nijie posts that have been deleted.
 2018-09-23 20:08:14 -05:00
evazion
5525bbe1ca nijie: normalize all thumbnail urls (#3919). 2018-09-23 20:08:14 -05:00
evazion
d294514dc0 nijie: don't crash on invalid urls or deleted works (#3919). 2018-09-23 20:08:14 -05:00
evazion
b6228505aa nijie: fix page_url method. 
The id in a bare image url is the member id, not the illust id.
 2018-09-23 20:08:13 -05:00
evazion
3414d27c78 post replacements: fix self-replacements; don't swallow upload errors. 2018-09-23 10:51:16 -05:00
evazion
39fe5273a9 Fix #3917: Uploads: validate file before resizing/distributing. 2018-09-23 10:51:16 -05:00
evazion
39f9e01b13 nicoseiga: fix canonical_url to use the image url. 2018-09-22 11:07:18 -05:00
evazion
09a8198979 /artists: add wildcard, regex search to url field (#3900) 
Allow searching the URL field by regex or by wildcard.

If the query looks like `/twitter/` do a regex search, otherwise if it
looks like `http://www.twitter.com/*` do a wildcard search, otherwise if
it looks like an url do an artist finder search, lastly if it looks like
`twitter` do a `*twitter*` search.
 2018-09-21 21:19:01 -05:00
evazion
03abbd0683 Fix #2894: Use [[:space:]] instead of \s in regexes. 2018-09-20 19:24:38 -05:00
evazion
29cdaddd86 PostSetPresenters::Post#related_posts: clean up metatag parsing (#2894). 
* Fix `#related_tags` to use `Tag.has_metatag?`.
* Fix Tag::SUBQUERY_METATAGS and Tag::METATAGS to be arrays instead of regexes.
 2018-09-20 19:23:47 -05:00
evazion
6fe883c316 posts index: clean up limit:<n> / order:random parsing (#2894) 
* Move the limit:<n> / order:random metatag parsing from the controller
  to the post set.

* Introduce `Tag.has_metatag?` and use it to parse these metatags
  instead of using a regex (#2894).
 2018-09-20 19:23:47 -05:00
evazion
958a9f505b moebooru: rewrite sample urls + support bookmarklet on html page. 
* Fixes #2942: Add Moebooru Rewrite for Sample Images.
* Addresses #3911: Improve Moebooru support.
 2018-09-19 23:32:21 -05:00
evazion
f4e08ef30d Downloads::File: fix SSRF inside is_cloudflare? (#2498). 
Fixes the banned IP check not being applied when sending the HEAD
request for is_cloudflare?.

Also fixes the `#size` method not using the uncached url (which meant
the bookmarklet could report the wrong filesize on artstation uploads).
 2018-09-19 20:11:53 -05:00
Albert Yi
9e1e73ec4c migrate blacklists during mass updates (#2251) 2018-09-19 16:25:06 -07:00
evazion
2f17082e73 Downloads::File: fix SSRF when following redirects (#2498). 
Fixes the banned IP check not being applied when following redirects:

  http://danbooru.donmai.us/uploads/new?url=http://httpbin.org/redirect-to%3Furl=http://127.0.0.1/test.jpg
 2018-09-18 16:00:03 -05:00
evazion
99221e4028 Downloads::File: fix SSRF attack when fetching remote size (#2498). 
Fixes the banned IP check not being applied when fetching the remote
file size. This allowed one to trick Danbooru into sending HEAD requests
to private IPs:

  http://danbooru.donmai.us/uploads/new?url=http://127.0.0.1/test.jpg
 2018-09-18 12:16:27 -05:00
evazion
488b1fd994 Downloads::File: memoize strategy. 
Also remove unused options param from constructor
 2018-09-18 10:22:15 -05:00
evazion
9cdfbba6c2 Fix #3910: Corrupted images during upload. 
Use a fresh tempfile for each download attempt instead of reusing the same
file (and having to rewind/truncate it after each failed attempt).
 2018-09-18 10:01:44 -05:00