Commit Graph
100 Commits
Author SHA1 Message Date
evazion 060bf1a64b Update .gitignore. 2019-11-19 00:16:40 -06:00
evazion fd29f842fd db: drop keeper_data from posts. 2019-11-19 00:13:24 -06:00
evazion 839b9cf32f tests: fix session loader tests. 2019-11-19 00:06:20 -06:00
evazion 1ce9eff0f5 views: convert /notes to table builder. 2019-11-19 00:06:17 -06:00
evazion 5afac29693 views: convert /tags to table builder. 2019-11-17 22:35:53 -06:00
evazion be5df37328 views: add table builder abstraction. 2019-11-17 22:35:53 -06:00
evazion 43d0955b61 Fix #4065: Sometimes normalization brakes twitter intent links.
Fix incorrectly normalized Twitter intent URLs:

    url: https://twitter.com/intent/user?user_id=881080941443239936
    normalized_url: http://twitter.com/intent/

to

    url: https://twitter.com/intent/user?user_id=881080941443239936
    normalized_url: http://twitter.com/intent/user?user_id=881080941443239936/

Fixes about ~1300 incorrectly normalized urls in the production db.
2019-11-17 19:08:07 -06:00
evazion 91d1817abf artists: don't return domains in /artists/{id}.xml.
Don't return the `domains` field in /artists/{id}.{json,xml}. Fixes a
failure in /artists/{id}.xml:

    https://danbooru.donmai.us/artists/156646.xml

    <result success="false">
        undefined method `domains' for #<ArtistUrl:0x00005566dd340af0> Did you mean? DomainName
    </result>

`to_xml` passes down the `methods` param to all nested models, which
doesn't work.
2019-11-17 18:24:06 -06:00
evazion 320ff01e07 login: remove 'remember' checkbox; make session cookies permanent.
Remove the "Remember" checkbox from the login page. Make session cookies
permanent instead. Phase out legacy `user_name` and `password_hash` cookies.

Previously a user's session cookies would be cleared whenever they
closed their browser window, which would log them out of the site. To
work around this, when the "Remember" box was checked on the login page
(which it was by default), the user's name and password hash (!) would
be stored in separate permanent cookies, which would be used to
automatically log the user back in when their session cookies were
cleared. We can avoid all of this just by making the session cookies
themselves permanent.
2019-11-17 17:50:23 -06:00
evazion 9b893db640 autocomplete: fixup uses of tag_aliases.post_count.
Fixup for 64728c89d.
2019-11-17 16:03:56 -06:00
evazion d1689f895e tests: fixup /artists/banned endpoint test. 2019-11-17 14:41:09 -06:00
evazion 6e403fddb6 alias/implications: fixup uses of creator_ip_addr.
Fixup for a2ea2a65a. Use 127.0.0.1 rather than the creator's original IP
for any edits performed as part of an alias or implication.
2019-11-17 14:39:46 -06:00
evazion e29d7162a4 autocomplete: fixup limit param.
Fixiup for 378559ff4. Fix case when limit param is nil.
2019-11-17 14:39:46 -06:00
evazion 64728c89d8 db: remove tag_aliases.post_count column.
This was only used in autocomplete, but it was unnecessary here because
we could already get the post count from the tags table.
2019-11-17 14:39:41 -06:00
evazion a2ea2a65a1 db: drop IP addresses from certain tables.
Don't track IP addresses for post appeals, post flags, tag aliases, tag
implications, or user feedbacks. These things are already tightly
limited. We don't need IPs from them to detect sockpuppets.
2019-11-17 02:45:28 -06:00
evazion 72f17fd1de Fix #3534: Remove Janitor Trials. 2019-11-17 02:10:24 -06:00
evazion 1ae971269c db: drop tag subscriptions table. 2019-11-17 01:52:36 -06:00
evazion fedb525305 notes: add fallback font for comic sans (#4212). 2019-11-16 22:43:16 -06:00
evazion 364ecfe68f db: drop unused columns from users table.
Note that the password_hash column was replaced by bcrypt_password_hash
in 2013, but the old password_hash column was never dropped.
2019-11-16 19:34:30 -06:00
evazion 90713ef94c db: fix more inconsistencies between structure.sql and production db.
Fix various cases where db/structure.sql either didn't match the
production db or didn't match the db structure produced when running
migrations from a fresh install.
2019-11-16 16:35:04 -06:00
evazion 9248266659 Fix #4213: Empty Last IP link. 2019-11-16 10:39:57 -06:00
evazion 13f26c81d8 Remove unused files. 2019-11-15 23:59:56 -06:00
evazion 3746c9890a db: resync db/structure.sql. 2019-11-15 23:36:31 -06:00
evazion 529fef7224 db: set timestamps to not null. 2019-11-15 22:44:22 -06:00
evazion dad8114c97 db: fix index inconsistencies with production db. 2019-11-15 22:44:13 -06:00
evazion 0726e4346a migrations: fix old migration not removing posts.uploader_ip_addr index.
Bug: running migrations on a fresh database failed when we got to a
migration adding a posts.uploader_ip_addr index on 2019-11-11. It failed
because the index already existed.

The index already existed because we used to have it at one point, until
it was removed by another migration on 2015-01-20. This migration didn't
correctly remove the index though, because it tried to remove a
posts.source index first, which failed because this index didn't exist
(it probably existed only in production). This error was swallowed,
causing the migration to silently skip removing the posts.uploader_ip_addr
index, which eventually blew up when we tried to add the index again
later on.
2019-11-13 02:06:41 -06:00
evazion 378559ff47 autocomplete: make number of results configurable (partly).
Add `Danbooru.Autocomplete.MAX_RESULTS` param that controls the number
of results returned by autocomplete.

This does not work with tag autocomplete for Builders. Builder
autocomplete still needs to be refactored and unified with Member
autocomplete.

Also fix a bug in the /saved_searches/labels endpoint where the limit
param wasn't respected.
2019-11-13 01:47:41 -06:00
evazion 89c5ac73af autocomplete: switch to async/await. 2019-11-13 01:47:41 -06:00
evazion 2d1eabbe5f autocomplete: factor out clearing of menu bindings. 2019-11-13 01:47:37 -06:00
evazion 682aa220b8 /ip_addresses: add to sitemap; add IP info links. 2019-11-12 20:51:48 -06:00
evazion f9881e5414 autocomplete: add saved search label autocomplete in more places. 2019-11-12 20:24:24 -06:00
evazion cc98f3b7cc Update gems. 2019-11-12 19:05:32 -06:00
evazion da8fbef5d6 /ip_addresses: add missing created_at, uploader_ip_addr indexes.
Add some missing indexes used by the /ip_addresses page.
2019-11-11 21:03:07 -06:00
evazion 73c02021a1 /ip_addresses: allow searching multiple IPs at once.
Fixes #4208. Allow searching for multiple IPs or user IDs. Separate
using spaces or commas.
2019-11-11 21:03:07 -06:00
evazion 68e1140b0d /ip_addresses: allow grouping IPs by subnet. 2019-11-11 21:03:07 -06:00
evazion bf6bb94702 Add new IP address search page.
Add a new IP address search page at /ip_addresses. Replaces the old
search page at /moderator/ip_addrs.

On user profile pages, show the user's last known IP to mods. Also add
search links for finding other IPs or accounts associated with the user.

IP address search uses a big UNION ALL statement to merge IP addresses
across various tables into a single view. This makes searching easier,
but is known to timeout in certain cases.

Fixes #4207 (the new IP search page supports searching by subnet).
2019-11-11 21:03:07 -06:00
evazion 2aac42b112 Add show actions for note, artist, artist commentary versions.
Add these endpoints:

* /note_versions/1234
* /artist_versions/1234
* /artist_commentary_versions/1234

This is so the /ip_addresses listing can link to these endpoints.
2019-11-11 12:56:39 -06:00
evazion 9cf5b1f01c dtext: add dtext cheatsheet popup. 2019-11-10 14:48:17 -06:00
evazion 69f005a89c gems: remove cityhash gem.
This gem uses a native extension that requires a C++ compiler to build.
Removing this gem removes the need to have a C++ toolchain to install Danbooru.
2019-11-09 14:42:30 -06:00
evazion 76674597f1 Upgrade to Rails 6.0.1. 2019-11-06 21:57:32 -06:00
evazion a92c24f093 wiki pages: add undelete link in navbar.
Also redirect back to the wiki page after deleting or undeleting it,
rather than redirecting to the index.
2019-11-06 21:57:32 -06:00
evazion 70b6525e69 wiki pages: fix error when force renaming wiki page.
Bug: force renaming a wiki page fails. Fix: use the wiki page id instead
of the title in the update action.
2019-11-06 21:56:21 -06:00
evazion 19549e2c12 css: fix overlong parent/child preview boxes on post show page.
Bug: if a post had a large number of child posts, the child post preview
box would expand beyond the width of the screen.

Setting `min-width: 0` makes the flexbox column shrink instead of
expand.
2019-11-05 20:58:07 -06:00
evazion e031fb0400 users: use inline dtext for ban reason on profile page. 2019-11-05 19:15:44 -06:00
evazion 06d0ceb016 dmails: fix bug with parsing wiki links in email notifications.
Bug: sending a dmail containing a wiki link (ex: [[tagme]]) failed when
the recipient had email notifications turned on.

Cause: wiki links inside email notifications use absolute urls, which
the dtext postprocessor didn't parse correctly.
2019-11-05 19:10:41 -06:00
evazion a653513e0a wiki pages: fix 404s for page titles containing dots.
Bug: links like these returned 404s:

* https://danbooru.donmai.us/wiki_pages/...
* https://danbooru.donmai.us/wiki_pages/.hack//
* https://danbooru.donmai.us/wiki_pages/ssss.gridman

Cause: by default, Rails uses dots in route segments to separate the id
from the format. For example, in /wiki_pages/ssss.gridman, the id is
parsed as "ssss" and the format is "gridman" (as if "gridman" were a
format like "json" or "xml").

We work around this by specifying the regex for the id param manually.
The trick here is to use a non-greedy match-all combined with a positive
lookahead to detect the extension but not include it in the match.
2019-11-05 15:14:08 -06:00
evazion aeb5edaff6 Fix deadlocks while updating tags counts.
Sometimes uploads fail with this error:

    ActiveRecord::QueryCanceled - PG::QueryCanceled: ERROR: canceling
    statement due to statement timeout CONTEXT: while updating tuple
    (2808,110) in relation "tags"

This is caused by two uploads trying to update tag counts for the same
tags at the same time. The UPDATE statement doesn't guarantee the tags
will be updated in any particular order, which can cause deadlocks when
two UPDATEs try to update the same tags in a conflicting order. For
example, if one process tries to increment '1girl' and 'solo' (in that
order), while at the same time another process tries to increment 'solo'
and '1girl' (in that order), the result will be a deadlock.

The solution is to explicitly lock the tags in a consistent order before
the update statement.
2019-11-05 12:53:52 -06:00
evazion a677c3ff1f /posts: fix link to wiki in wiki excerpt. 2019-11-04 18:53:08 -06:00
evazion a418ed7281 post tooltips: use more compact timestamps.
* Use more compact timestamps in post tooltips ("39 minutes ago" -> "39m ago")
* Move timestamps to the right (after favcount + score).
* Switch favorite icon from star to heart.
2019-11-04 18:18:56 -06:00
evazion c1939f78d6 artists: replace /artists/banned endpoint with simple search. 2019-11-04 17:15:02 -06:00
evazion db01696e91 artist: fix /artists?name=<name> search. 2019-11-04 17:11:24 -06:00
evazion d7233b3c39 implications: remove duplicate scopes.
These are already defined in tag_relationship.rb.
2019-11-04 16:50:52 -06:00
evazion 90fef76121 artists: reorganize index page.
* Make other names searchable.
* Add post count and update timestamp.
* Add edit / delete / undelete links.
2019-11-04 16:45:19 -06:00
evazion 75ca53a9a1 autocomplete: fix underscore-to-space conversion in usernames.
Didn't handle names with multiple underscores correctly.
2019-11-04 15:45:07 -06:00
evazion 623de37b09 views: remove current user <meta> elements from <head>.
Previously deprecated. Use the data attributes on the <body> tag
instead.
2019-11-04 15:42:19 -06:00
evazion 02b9acac9e wiki pages: replace show_or_new page with redirects.
* Redirect the show_or_new action to either the show page or the new
  page. Don't use show_or_new to render nonexistent wikis; do that in the
  regular show action instead.

* Make the show action return 404 for nonexistent wikis.
2019-11-04 15:35:47 -06:00
evazion 6e5e07022d Have single result searches go to the show pages for certain controllers #4204 2019-11-04 14:24:10 -06:00
evazion 953de62d70 Update dtext gem. 2019-10-31 22:23:50 -05:00
evazion 9b9afff8c8 wiki pages: remove unused noredirect=1 from /wiki_pages index. 2019-10-31 19:06:39 -05:00
evazion 3a908f84bb wiki pages: use names instead of ids in urls.
Switching to using wiki names in URLs instead of IDs:

* https://danbooru.donami.us/wiki_pages/vocaloid
* https://danbooru.donami.us/wiki_pages/hatsune_miku

ID numbers can still be used, but they redirect to the name instead:

* https://danbooru.donami.us/wiki_pages/11 (redirects to /wiki_pages/touhou).

Numeric tags are prefixed with '~' to distinguish them from IDs:

* https://danbooru.donami.us/wiki_pages/2019 (the wiki with id 2019)
* https://danbooru.donami.us/wiki_pages/~2019 (the wiki for the tag named 2019)

The tag names 'new' and 'search' are disallowed to prevent conflicts
with existing routes:

* https://danbooru.donami.us/wiki_pages/new
* https://danbooru.donami.us/wiki_pages/search
2019-10-31 19:04:18 -05:00
evazion 0ccfb3f5f6 wiki pages: fix error in new controller action. 2019-10-31 19:04:18 -05:00
evazion 9826776e66 Remove unused dtext/help template. 2019-10-31 19:04:18 -05:00
evazion 5422db1c3c Standardize links to wiki help pages.
* Fix inconsistencies in how wiki pages were linked.

* Link directly to the wiki instead of to a title search that is expected
  to redirect to the wiki.
2019-10-31 19:04:18 -05:00
evazion a8fc2ba508 wiki pages: fix redirects to exact matches during search.
Change wiki page search to redirect to exact matches only when using the
quick search bar. Fixes searches sometimes unexpectedly redirecting when
doing a regular (non-quick) search that happens to return a single result.

Also remove the logic that tries to expand the search when no results
are found. This will eventually be replaced with a smarter "did you mean?"
search.
2019-10-31 19:04:18 -05:00
evazion 91fd5e1bff js: add "ctrl+enter to submit form" shortcut.
Press Ctrl+Enter inside any text box to submit the form.
2019-10-31 00:27:43 -05:00
evazion 9cfb753c1b post tooltips: fix null deref when tooltips are disabled.
Fix an exception that is triggered by mousing out of a thumbnail when
tooltips are disabled. Caused by trying to access `qtip.cache` when
`qtip` is null.
2019-10-30 23:43:52 -05:00
evazion bf841dcd44 gems: update gems. 2019-10-30 23:34:52 -05:00
evazion 2821559221 gems: drop term-ansicolor gem. 2019-10-30 23:32:28 -05:00
evazion 55a2c6de2b user feedbacks: reorganize index page.
* Move search form to index page instead of a separate page.
* Merge creator + timestamp into one column.
* Add category column.
* Fix N+1 query issue.
2019-10-30 23:29:24 -05:00
evazion 1f191d2cdf bans: list long bans as 'forever'. 2019-10-30 23:29:24 -05:00
evazion 64881d8840 bans: merge banner name + timestamp into one column. 2019-10-30 23:29:19 -05:00
evazion f4b3d4e075 mod actions: reorganize index page.
* Add category column. Allow filtering by category.
* Merge user + timestamp into one column.
2019-10-30 23:29:13 -05:00
evazion f865cd8aeb modqueue: bump "You haven't moderated any posts..." notice to 72 hours. 2019-10-30 10:49:14 -05:00
evazion 4f858384bb dark mode: make header elements brighter. 2019-10-28 21:57:49 -05:00
evazion b19d2d3934 Add dependabot config. 2019-10-28 21:37:34 -05:00
evazion f075e63573 Merge pull request #4200 from r888888888/dependabot/npm_and_yarn/jquery-3.4.1
build(deps): bump jquery from 3.3.1 to 3.4.1
2019-10-28 21:28:09 -05:00
evazion e4787229fd Merge pull request #4202 from r888888888/dependabot/npm_and_yarn/fstream-1.0.12
build(deps): bump fstream from 1.0.11 to 1.0.12
2019-10-28 21:27:50 -05:00
evazion 18201948ea Merge pull request #4201 from r888888888/dependabot/npm_and_yarn/eslint-utils-1.4.3
build(deps): bump eslint-utils from 1.4.0 to 1.4.3
2019-10-28 21:27:03 -05:00
evazion ee96a696f0 bulk update requests: fix attaching bur to existing topic.
ref: https://danbooru.donmai.us/forum_topics/16411
2019-10-28 20:13:12 -05:00
evazion e3dea024ae dtext: bump version (fix case sensitivity in [expand] tags). 2019-10-28 19:59:15 -05:00
evazion 68c74ed9d8 tests: fixup post votes, comment votes controller tests. 2019-10-28 16:52:02 -05:00
evazion 4bb1bdbe10 dtext: handle [bur:<id>] tags in main parser.
Move the parsing for the [bur:<id>], [ta:<id>], [ti:<id>] pseudo tags to
the main parser in `DText.format_text`. This fixes a bug where wiki
links inside bulk update requests on the forum weren't properly
colorized because the text of the BUR was embedded after we scanned for
wiki links, not before.

This also ensures that tags inside bulk update requests will be recorded
in the dtext_links table, meaning that forum posts can be properly
searched by tags.

This incidentally means that these request pseudo tags can now be used
outside the forum.
2019-10-28 16:40:38 -05:00
evazion 6424a4de74 Do full page counts on small index pages.
Normally we skip doing page counts on index pages when there aren't any
search filters. This is on the assumption that most index pages have
more than 1000 pages (20,000 results), so it's not worth counting them
exactly. This isn't always true, so here we turn on full counts on
certain index pages known to be small.
2019-10-28 15:18:54 -05:00
evazion fd4a9d4d30 post/comment/forum votes: add vote counts to profile pages.
* Add post/comment/forum vote counts to user profiles.
* Show uploaders on post votes index and allow searching by uploader.
* Show forum posters on forum votes index and allow searching by poster.
* Add unvote link to forum votes index.
* Only show unvote links to current user.
2019-10-28 14:46:16 -05:00
evazion d3165f78aa comment votes: add index page. 2019-10-28 14:12:40 -05:00
evazion 93b03c04ad post votes: let gold users use upvote:self, downvote:self metatags.
* Let gold users use upvote:self, downvote:self metatags to search for
  their own votes.
* Don't let mods use upvote:<user>, downvote:<user> metatags to see
  votes by other users. Only let admins see other users' votes.
* Add vote count to profile page.
2019-10-28 03:45:26 -05:00
evazion f4376fc7a3 post votes: add index page. 2019-10-28 03:21:18 -05:00
evazion 754e3a2ee2 tests: fix forum post votes & wiki page updater tests.
* Move forum post vote tests from test/controllers to test/functional.
* Fix forum post vote tests to work with new routes.
* Fix obsolete wiki page tests dealing with updater_id.
2019-10-28 02:13:10 -05:00
evazion dfbf4f3f0a Remove single alias/implication requests.
* Remove the single alias and implication request forms. From now
  on, bulk update requests are the only way to request aliases or
  implications.

* Remove the forum topic ID field from the bulk update request form.
  Instead, to attach a BUR to an existing topic you go to the topic then
  you click "Request alias/implication" at the top of the page.

* Update the bulk update request form to give better examples for the
  script format and to explain the difference between aliases and
  implications.
2019-10-28 01:36:28 -05:00
evazion b5a40aa233 bulk update requests: list upvotes/downvotes on index page. 2019-10-28 01:06:23 -05:00
evazion 0e159960a2 forum post votes: add index page. 2019-10-28 01:06:23 -05:00
evazion d617b20b49 dtext links: allow searching for forum posts linking to specific tag. 2019-10-27 01:05:54 -05:00
evazion d946a84480 iqdb: download files inside danbooru, not inside iqdb.
Instead of sending IQDB the image url and letting it download the file,
download the file on Danbooru's end and send IQDB the downloaded file.

This fixes several issues:

* Some sites need the referer header set to avoid hotlink protection
  when downloading the file. Danbooru knows how to deal with this but
  IQDB doesn't.

* We need to enforce certain restrictions when downloading files,
  including setting max filesize limits, setting max timeouts, and not
  allowing downloads from forbidden IPs (to avoid SSRF attacks).
  Danbooru knows how to handle these things but IQDB doesn't.
2019-10-26 15:02:07 -05:00
evazion a6efaa54a1 Downloads::File: fix following 302 redirects during download.
Fixes downloading yande.re preview images for iqdb. Yande.re previews
return a 302 redirect to the actual file. Before the html in the body of
the 302 response would get prepended to the image file.

https://files.yande.re/data/preview/12/8f/128fb480d8589be26d1dce7e1d841bcb.jpg
=> https://assets.yande.re/data/preview/12/8f/128fb480d8589be26d1dce7e1d841bcb.jpg
2019-10-26 14:04:19 -05:00
evazion a15bbe4264 moebooru: fix preview_urls to fallback to image_urls.
Fix the moebooru strategy to fallback to returning the image url if we
can't find the preview url. Fixes iqdb lookups failing in some cases
because the strategy didn't return a valid url for preview_url.
2019-10-26 13:51:58 -05:00
evazion ecfeaca7d4 iqdb: fix handling of error responses.
Handle JSON error responses returned by IQDB: `{ "error": <message> }`.
2019-10-26 13:01:08 -05:00
evazion 438a7a5a70 wikis: add "tag history", "wiki history", "what links here" sidebar options. 2019-10-26 02:59:46 -05:00
evazion 3a544ba5e0 Fix tag ordering in humanized_essential_tag_string.
* Pick the largest character or copyright tags by post count. Previously
  we picked the tags with the longest names, which was nonsensical.

* Remove tag cateogory logic from config file. We can't avoid hardcoding
  some knowledge about tag categories here, so there's no point in trying.

This affects tab titles on post show pages as well as filenames in
downloaded images.
2019-10-26 02:36:55 -05:00
evazion ac775e2bdd Fix #4199: Aliased tags not included in related wiki tags. 2019-10-24 19:48:30 -05:00
evazion 6dcf0484f0 dtext links: add fix script to populate dtext_links table. 2019-10-24 19:48:30 -05:00