Commit Graph

1784 Commits

Author SHA1 Message Date
evazion
faeec18efc twitter: add hashtag normalization test.
Add test for #4243. Also fix warning from bootsnap:

    iseq.rb:18: warning: nested repeat operator '+' and '?' was replaced with '*' in regular expression: /(?<!\A)生誕祭(?:\d+)?\z/
2020-01-05 17:38:30 -06:00
evazion
692e5957a2 Merge pull request #4243 from BrokenEagle/normalized-twitter-hashtags
Add normalization for Twitter hashtags
2020-01-05 17:30:05 -06:00
BrokenEagle
67860a845c Add normalization for Twitter hashtags 2020-01-05 23:01:50 +00:00
BrokenEagle
4b904dff5a Convert classes to use data attributes
- Remote resolved classes on post flags since they were unused
2020-01-04 22:02:45 +00:00
BrokenEagle
223a6df5d8 Add API data to show/index views 2020-01-04 22:02:44 +00:00
BrokenEagle
043944e1dd Convert index tables to using table builder 2020-01-04 22:02:43 +00:00
evazion
917ffa87ed users: remove score confidence intervals from /users index.
These were originally used to find uploaders with high-scoring uploads
for promotion purposes. The weekly reports generated by Reportbooru are
better suited for this.

Also clean up some unused and redundant code.
2020-01-02 01:44:03 -06:00
evazion
c251d145e4 Revert "Add Danbooru Winter Sale."
This reverts commit 963f1f7991.
2020-01-02 01:29:55 -06:00
evazion
c0171f49a6 storage manager: add option for storing files under original/ dir.
Add option to allow for storing original files under an `original/`
subdir. Previously preview, crop, and sample files were stored under
subdirs but original files were stored in the top-level directory. This
is optional for backwards compatibility.
2020-01-01 13:46:05 -06:00
evazion
c95108d37b maintenance: prevent exceptions from halting daily maintenance.
Fix cases where not all daily maintenance tasks run because some earlier
task raised an exception.
2019-12-28 00:21:56 -06:00
evazion
56d787a17f storage manager: make rclone path configurable.
Fix bug where pruning uploads failed because the rclone binary couldn't
be found. The upload pruner runs under cron, which has a fixed default
$PATH of "/bin:/usr/bin", but in production rclone is installed under
/usr/local/bin. This caused the upload pruner to fail, which prevented
the rest of daily maintenance from running.
2019-12-28 00:21:40 -06:00
evazion
963f1f7991 Add Danbooru Winter Sale.
* 25% off Gold and Platinum accounts.
* Free 6 tag searches for Member-level users.
* Runs from midnight Dec. 25th to 28th (UTC time).
2019-12-24 15:39:29 -06:00
evazion
ef2eb9d0f5 user upgrades: factor out gold/platinum prices. 2019-12-24 12:04:15 -06:00
evazion
7694be9cb3 users: remove no_flagging and no_feedback permissions. 2019-12-24 10:24:15 -06:00
evazion
bdebd1a594 approver pruner: change inactivity threshold to 30 approvals in 45 days.
* Change minimum activity threshold to 30 approvals in 45 days.
* Exclude mods from losing privileges, not just admins.
* Fix undeletions of old posts not counting towards the activity threshold.
2019-12-23 00:44:22 -06:00
evazion
ac9d06bf79 maintenance: remove hourly upload error checker. 2019-12-23 00:14:38 -06:00
evazion
cd442d60af tag_category.rb: fix windows line endings. 2019-12-22 21:24:00 -06:00
evazion
309821bf73 rubocop: fix various style issues. 2019-12-22 21:23:37 -06:00
evazion
2320fad139 danbooru logger: fix request parameter filtering. 2019-12-22 17:33:51 -06:00
evazion
eddd43c21f recommender: fix typo in recommend_for_user. 2019-12-17 02:24:44 -06:00
evazion
84ba1d417f Fix #4220: Uploading from Tumblr is broken. 2019-12-15 19:04:52 -06:00
evazion
e0f5103e4a reltags: fix nil exception on fast_count timeouts.
Fixes a nil dereference error when calculating related tags for the
sidebar. Cased by Post.fast_count returning nil when it times out.
2019-12-15 17:06:58 -06:00
evazion
41378bc8e3 sources: replace HttpartyCache with Danbooru::Http. 2019-12-15 17:06:58 -06:00
evazion
0adec60259 recommender: replace HttpartyCache with Danbooru::Http. 2019-12-15 15:48:38 -06:00
evazion
518de2fdec cloudflare: fix purging of cached urls.
* Switch CloudflareService from HttpartyCache to Danbooru::Http.

* Purge cached urls from Cloudflare when a post is replaced and the md5
  doesn't change. This happens when a corrupted image is replaced or
  thumbnails are regenerated. Before we purged urls when a post was
  expunged, which was unneeded because those urls can expire naturally.
  It was also wrong because the subdomains were hardcoded, the urls used
  http:// instead of https://, and we didn't account for tagged urls.
2019-12-15 14:03:15 -06:00
evazion
3cc636e719 Fix uninitialized constant error with Resolv class.
Fix uninitialized constant error (introduced by the switch to zeitwerk?)
2019-12-13 18:19:41 -06:00
evazion
da84e3a2f2 twitter: replace twitter gem with our own API client.
The twitter gem had several problems:

* It's been unmaintained for over a year.
* It pulled in a lot of dependencies, many of which were outdated. In
  particular, it locked the `http` gem to version 3.3, preventing us
  from upgrading to 4.2.
* It raised exceptions on normal error conditions, like for deleted
  tweets or suspended users, which we really don't want.
* We had to wrap it to provide caching.

Changes:

* Fixes #4226 (Exception when creating new artists entries for suspended
  Twitter accounts)
* Drop support for scraping images from summary cards. Summary cards
  are the previews you get when you link to a website in a tweet. These
  preview images aren't always the best image.
2019-12-13 17:27:03 -06:00
evazion
0b556ece1c Add Danbooru::Http client.
This is a wrapper around the http.rb gem that provides caching.
2019-12-13 15:32:21 -06:00
evazion
316690f393 storage manager: remove S3 backend.
Remove in favor of the rclone S3 backend.
2019-12-13 02:14:14 -06:00
evazion
92aca9ed2d storage manager: remove fog backend.
Remove in favor of rclone.
2019-12-13 01:45:43 -06:00
evazion
afea704b55 storage manager: add rclone backend. 2019-12-11 01:44:23 -06:00
evazion
2e6486ab5f BURs: fix exception in category change requests for new tags.
`category blah -> character` failed when `blah` didn't exist yet.
2019-12-10 15:17:10 -06:00
evazion
2f27a4eddd image resizer: fix incompatibility with libvips-8.8.
In libvips-8.8 the `auto_rotate` option was replaced with `no_rotate`.
Using `auto_rotate` in libvips-8.8 fails, so we need a version check
here.
2019-12-10 14:04:40 -06:00
evazion
90f64eb407 image resizer: remove shell fallback for old versions of libvips.
Remove fallback for old versions of Debian. Debian stable now provides
libvips 8.7.
2019-12-10 14:04:40 -06:00
evazion
4b426ec5b9 storage manager: fix b2 storage bugs.
* B2 doesn't allow the path to start with a '/' character.
* When storing the file, we have to rewind the file pointer to make sure
  we get the whole file.
2019-12-10 14:04:40 -06:00
evazion
e4f4326982 storage manager: add backblaze b2 backend. 2019-12-10 02:04:32 -06:00
evazion
6196f7ba67 BURs: fix exception when estimating post counts of category changes.
Fix an error in operator precedence:

> sum + Tag.find_by_name(token[1]).try(:post_count) || 0

This was treated as `(sum + X) || 0` not `sum + (X || 0)` as intended.
This failed when X was nil.
2019-12-10 01:43:05 -06:00
evazion
f24220982a seo: don't noindex /posts?tags=order:rank. 2019-12-07 22:18:58 -06:00
evazion
5919b825d7 seo: don't hide the front page from web crawlers.
Don't mark the front page (empty tag search) as noindex.
2019-12-07 18:35:53 -06:00
evazion
ea718122da Fix #4219: Deviantart broke download urls when not logged in. 2019-12-06 22:54:08 -06:00
evazion
1c3b96dc4e deviantart: fix non-downloadable posts (#4219).
Adapted from e51e1aa7dd.
2019-12-06 22:54:08 -06:00
evazion
2bffad9311 BURs: credit tag edits to DanbooruBot.
Credit tag edits that are performed as part of an alias, implication, or
mass update to DanbooruBot instead of the admin who approved the BUR.
2019-12-06 22:54:08 -06:00
evazion
0c483cad15 config: fix references to Danbooru.config.app_name.
Danbooru.config.app_name is either Safebooru or Danbooru. Fix some
places where it should always be Danbooru.
2019-12-06 01:33:52 -06:00
evazion
06078ef172 danbooru_default_config.rb: remove unused config params. 2019-12-06 01:33:52 -06:00
evazion
cb45aa6fe6 Remove unused automod code.
The DynamoDB update job was disabled a few months ago. Remove it
completely.
2019-12-03 10:45:18 -06:00
evazion
5d1a1cc87e Change github repo url to https://github.com/danbooru/danbooru.
The danbooru repo was moved to https://github.com/danbooru.
2019-12-02 17:33:40 -06:00
evazion
86a09a322f cloudflare: use new api token auth scheme.
Use Cloudflare's new API token authentication method to talk to
Cloudflare. This lets us use API tokens with minimal privileges set.

https://blog.cloudflare.com/api-tokens-general-availability/
2019-12-02 16:45:32 -06:00
evazion
27a1a90cf3 recommendations: allow filtering recs by tags. 2019-12-02 03:08:01 -06:00
evazion
efda9f37e1 recommendations: add search form, allow searching by username. 2019-12-02 02:26:21 -06:00
evazion
ae46f7a665 recommendations: hide recommendations for users with private favorites.
Don't allow users to view recommendations for users who have private
favorites.
2019-12-01 19:07:55 -06:00