Commit Graph

8374 Commits

Author SHA1 Message Date
evazion
6a52216631 newrelic: log additional request headers.
Log the Referer header, as well as the Sec-Fetch-* headers. These are
only sent by recent versions of Chrome; see https://www.w3.org/TR/fetch-metadata.
2020-12-28 01:53:22 -06:00
evazion
59c61f249f posts helper: remove dupe nav_params_for method.
Already defined in PaginationHelper.
2020-12-28 00:57:18 -06:00
evazion
805bbc8a33 users: add config option to disable verification of new accounts.
Fixes #4618.
2020-12-28 00:01:35 -06:00
evazion
9dc788c0ce users: improve sockpuppet detection on signup.
Require new accounts to verify their email address if any of the
following conditions are true:

* Their IP is a proxy.
* Their IP is under a partial IP ban.
* They're creating a new account while logged in to another account.
* Somebody recently created an account from the same IP in the last week.

Changes from before:

* Allow logged in users to view the signup page and create new accounts.
  Creating a new account while logged in to your old account is now
  allowed, but it requires email verification. This is a honeypot.
* Creating multiple accounts from the same IP is now allowed, but they
  require email verification. Previously the same IP check was only for
  the last day (now it's the last week), and only for an exact IP match
  (now it's a subnet match, /24 for IPv4 or /64 for IPv6).
* New account verification is disabled for private IPs (e.g. 127.0.0.1,
  192.168.0.1), to make development or running personal boorus easier
  (fixes #4618).
2020-12-27 23:41:07 -06:00
evazion
7e8f859b24 tags: eliminate Tag.category_for method.
Tag.category_for looked up a tag's category in the Redis cache. This was
only used in a few places (in related tags, and on the popular/missed
search pages). Get rid of this method so we can work towards getting rid
of caching tag categories in Redis.
2020-12-27 21:03:26 -06:00
evazion
1047b1f8af Fix #4427: Opening a post from a profile's favorites brings up a fav: search. 2020-12-27 17:14:55 -06:00
evazion
7f1b798b05 searchable: refactor search_boolean_attribute. 2020-12-27 05:26:21 -06:00
evazion
ddd149e22b seo: mark login links as nofollow.
Mark links to the login page as rel="nofollow" so that search crawlers
don't constantly try to crawl it. Otherwise the fact the login url is
different on every page (/login?url=<current_url>) confuses crawlers.

Also strip the url param from the canonical url (<link rel="canonical">)
on the login page.
2020-12-27 04:41:49 -06:00
evazion
57e70e5b82 /users: fix exception when last_ip_addr is nil. 2020-12-26 11:16:19 -06:00
evazion
fd18291382 Add Danbooru Winter Sale. 2020-12-25 06:07:21 -06:00
evazion
d9db32640a user upgrades: fix checkout form leaking recipient's email.
The checkout form should be prefilled with the purchaser's email
address, not the recipient's.
2020-12-25 02:01:42 -06:00
evazion
e030a07816 user upgrades: add index action. 2020-12-25 01:21:54 -06:00
evazion
96f08b78c5 /contact: update contact page with more contact methods. 2020-12-25 00:47:08 -06:00
evazion
069231a33b user upgrades: update upgrade landing page.
* Add a frequently asked questions section.
* Add nicer looking upgrade buttons.
* Format the page nicer.
* Prevent users from attempting invalid upgrades on users that are
  already Platinum or above.
2020-12-25 00:47:08 -06:00
evazion
058d71aa30 user upgrades: send dmail to purchaser for gifted upgrades.
* Refactor to move upgrade logic from UserPromotion to UserUpgrade.
* Send the recipient and the purchaser of a gifted upgrade separate
  dmail notifications.
2020-12-25 00:43:58 -06:00
evazion
74ed2a8b96 user upgrades: add UserUpgrade model.
Add a model to store the status of user upgrades.

* Store the upgrade purchaser and the upgrade receiver (these are
  different for a gifted upgrade, the same for a self upgrade).
* Store the upgrade type: gold, platinum, or gold-to-platinum upgrades.
* Store the upgrade status:
** pending: User is still on the Stripe checkout page, no payment
   received yet.
** processing: User has completed checkout, but the checkout status in
   Stripe is still 'unpaid'.
** complete: We've received notification from Stripe that the payment
   has gone through and the user has been upgraded.
* Store the Stripe checkout ID, to cross-reference the upgrade record on
  Danbooru with the checkout record on Stripe.

This is the upgrade flow:

* When the user clicks the upgrade button on the upgrade page, we call
  POST /user_upgrades and create a pending UserUpgrade.
* We redirect the user to the checkout page on Stripe.
* When the user completes checkout on Stripe, Stripe sends us a webhook
  notification at POST /webhooks/receive.
* When we receive the webhook, we check the payment status, and if it's
  paid we mark the UserUpgrade as complete and upgrade the user.
* After Stripe sees that we have successfully processed the webhook,
  they redirect the user to the /user_upgrades/:id page, where we show
  the user their upgrade receipt.
2020-12-24 21:15:04 -06:00
evazion
7762489d7d user upgrades: upgrade to new Stripe checkout system.
This upgrades from the legacy version of Stripe's checkout system to the
new version:

> The legacy version of Checkout presented customers with a modal dialog
> that collected card information, and returned a token or a source to
> your website. In contrast, the new version of Checkout is a smart
> payment page hosted by Stripe that creates payments or subscriptions. It
> supports Apple Pay, Dynamic 3D Secure, and many other features.

Basic overview of the new system:

* We send the user to a checkout page on Stripe.
* Stripe collects payment and sends us a webhook notification when the
  order is complete.
* We receive the webhook notification and upgrade the user.

Docs:

* https://stripe.com/docs/payments/checkout
* https://stripe.com/docs/payments/checkout/migration#client-products
* https://stripe.com/docs/payments/handling-payment-events
* https://stripe.com/docs/payments/checkout/fulfill-orders
2020-12-24 19:58:29 -06:00
evazion
c17678d509 routes: add a new 404 page.
* Fix a bug where non-GET 404 requests weren't handled.
* Fix a bug where non-HTML 404 requests weren't handled.
* Show a random image from a specified pool on the 404 page.
2020-12-24 00:17:35 -06:00
evazion
ca742db07a routes: remove legacy /user/index and /artist/index API endpoints.
These endpoints get zero traffic.
2020-12-24 00:17:26 -06:00
evazion
dbb66ace90 routes: replace hardcoded routes in models with route helpers.
Add a Routes module that gives models access to route helpers outside of
views, and use it to replace various hardcoded routes.
2020-12-24 00:17:19 -06:00
evazion
a947a10c53 config: add debug_mode option.
Add a debug mode option. This is useful when debugging failed tests.

Debug mode disables parallel testing so you can set breakpoints in tests
with binding.pry (normally parallel testing makes it hard to set
breakpoints).

Debug mode also disables global exception handling for controllers. This
lets exceptions bubble up to the console during controller tests
(normally exceptions are swallowed by the controller, which prevents you
from seeing backtraces in failed controller tests).
2020-12-24 00:17:19 -06:00
evazion
a084da2dbe artists: hide other names of banned artists on index page.
Don't show other names of banned artists on the /artists page to
anonymous users. Hides potentially sensitive information from Google and
logged out users.
2020-12-22 02:15:21 -06:00
evazion
fbb4cfb807 users: let mods see users' last login time. 2020-12-21 22:42:50 -06:00
evazion
3c4781f6d8 users: update last_logged_in_at hourly.
Update last_logged_in_at on an hourly basis instead of a weekly basis.
2020-12-21 22:42:50 -06:00
evazion
025631ee64 users: show IPs to mods on /users page. 2020-12-21 22:42:50 -06:00
evazion
8221c8dcba users: inline search form on /users index page.
* Add the user search form to the /users page.
* Remove the /users/search page.
2020-12-21 22:42:50 -06:00
evazion
7a2f72ce98 ip bans: fix /ip_bans listing not showing subnet. 2020-12-21 22:42:50 -06:00
evazion
db488c247d ip bans: fix deleted field in /ip_bans search form.
Fix the value not being remembered in the search form because we
accidentally used `input_html` instead of `selected`.
2020-12-21 22:42:50 -06:00
evazion
0be9c8dc8b emails: optimize /emails listing.
Fix a suboptimal query that made the /emails page really slow.
2020-12-21 22:42:50 -06:00
evazion
6c99bbbf47 posts: limit sources to 1200 chars long.
The longest sources on Danbooru are DeviantArt wixmp.com sources, which
max out at ~900 chars.
2020-12-21 22:42:39 -06:00
evazion
6ac9882711 newrelic: log country of each request in newrelic.
Log the country of each HTTP request in NewRelic. Uses the CF-IPCountry
header set by Cloudflare.
2020-12-21 20:47:58 -06:00
evazion
efb836ac02 wikis: normalize Unicode characters in wiki bodies.
* Introduce an abstraction for normalizing attributes. Very loosely
  modeled after https://github.com/fnando/normalize_attributes.
* Normalize wiki bodies to Unicode NFC form.
* Normalize Unicode space characters in wiki bodies (strip zero width
  spaces, normalize line endings to CRLF, normalize Unicode spaces to
  ASCII spaces).
* Trim spaces from the start and end of wiki page bodies. This may cause
  wiki page diffs to show spaces being removed even when the user didn't
  explicitly remove the spaces themselves.
2020-12-21 20:47:50 -06:00
evazion
48ff7c42cd autocomplete: bump opensearch description version.
Fix browsers still using the old autocomplete endpoint for opensearch
searches (searches performed in the browser toolbar).
2020-12-20 01:33:10 -06:00
evazion
3ad4beac02 autocomplete: fix exception when completing unsupported metatags. 2020-12-20 01:27:48 -06:00
evazion
28926c2332 autocomplete: remove old autocomplete endpoints.
Remove /tag/autocomplete.json and /saved_searches/labels.json.
2020-12-20 00:51:29 -06:00
evazion
9de7a07af7 /status: fix blank distro version field.
The `source` command is a bash-ism and doesn't work in a strictly POSIX
shell like dash, which is the /bin/sh on Debian/Ubuntu. Use `.` instead.

https://en.wikipedia.org/wiki/Dot_(command)
2020-12-20 00:51:29 -06:00
evazion
a129eb4251 wikis: force wiki names to follow same rules as tag names.
Don't allow wiki pages to have invalid names.

This incidentally means that you can't create wiki pages for pools. For
example, you can't create a wiki titled "pool:almost_heart-warming".
This is not a valid tag name, so it's not a valid wiki name either. This
was done in a handful of cases to translate Pixiv tags to Danbooru pools
(see: <https://danbooru.donmai.us/wiki_page_versions?search[title_like]=pool:*>)

Also fix it so that titles are normalized before validation, not before save.
2020-12-20 00:51:29 -06:00
evazion
7708e2e08f wikis: don't allow adding other names to artist wikis.
Prevent users from adding other names to artist wikis. These should be
added to the artist entry instead.
2020-12-20 00:51:29 -06:00
evazion
4cb39422b2 post replacements: rename <attr>_was to old_<attr>
Rename the following post replacement attributes:

* file_size_was -> old_file_size
* file_ext_was -> old_file_ext
* image_width_was -> old_image_width
* image_height_was -> old_image_height
* md5_was -> old_md5

In Rails 6.1, having attributes named `file_size` and `file_size_was` on
the same model breaks things because it conflicts with Rails' dirty
attribute tracking.
2020-12-19 14:26:07 -06:00
evazion
09e3146819 artist finder: add blog.livedoor.jp to blacklist. 2020-12-19 00:51:34 -06:00
evazion
c97186abd7 /status: add missing template files.
Add missing templates that were forgotten in 7a87225ac.
2020-12-19 00:27:24 -06:00
evazion
53653372ec notes: include search form on search results page.
Also eliminate /notes/search endpoint.
2020-12-18 01:59:39 -06:00
evazion
2c92794eba wiki: include search form on search results page.
Include the search form on the search results page so you can more
easily refine your search.
2020-12-17 23:57:13 -06:00
evazion
25069865b7 ip bans: add search form.
* Add IP ban search form to /ip_bans page.
* Make some attributes searchable that weren't previously searchable.
2020-12-17 23:57:13 -06:00
evazion
3d1ff9dff9 autocomplete: fix not detecting correct tag in edit box. 2020-12-17 23:57:13 -06:00
evazion
2c1da660fd tags: allow tag abbreviations in searches and during tagging.
Expand the tag abbreviation system introduced in b0be8ae45 so that it
works in searches and when tagging posts, not just in autocomplete.

For example, you can tag a post with /evth and it will add the tag
eyebrows_visible_through_hair. You can search for /evth and it will
search for the tag eyebrows_visible_through_hair.

Some more examples:

* /ops is short for one-piece_swimsuit
* /hooe is short for hair_over_one_eye
* /saol is short for standing_on_one_leg
* /tlozbotw is short for the_legend_of_zelda:_breath_of_the_wild

If two tags have the same abbreviation, then the larger tag takes
precedence. For example, /be is short for blue_eyes, not brown_eyes,
because blue_eyes is the bigger tag.

If there is an existing shortcut alias that conflicts with the
abbreviation, then the alias take precedence. For example, /sh is short
for suzumiya_haruhi, not short_hair, because there's an old alias for
/sh -> suzumiya_haruhi.
2020-12-17 23:57:13 -06:00
evazion
991896c4eb tags: don't allow tags more than 170 chars long.
Limit tag length to 170 chars. 170 chars was chosen because it's
longer than the longest active tag on Danbooru.

Tag length is limited because in some contexts we can't deal with
excessively long tags. Tag autocorrect for example uses the levenshtein
function in Postgres, which can't handle strings more than 255 chars long.
2020-12-17 21:38:24 -06:00
evazion
1809f67b2b tags: don't allow tags to begin with a '/'.
Disallow tags from starting with a '/' character. This is so that tag
abbreviations in autocomplete, which start with a '/', don't conflict
with regular tags.

Also disallow some other punctuation characters: `%{})]. Currently no
tags start with these characters. This is to reserve other special
characters in case we need them for other future syntax extensions.
2020-12-17 21:38:18 -06:00
evazion
7a87225ac8 Add basic server status page at /status.
Lists versions of various dependencies plus some Postgres and Redis
metrics.
2020-12-17 03:13:54 -06:00
evazion
ee4516f5fe searchable: refactor searchable_includes.
Pass searchable associations directly to search_attributes instead of
defining them separately in searchable_includes.
2020-12-16 23:57:07 -06:00