Rewrite related tags implementation.

Rewrite the implementation of related tags to be simpler, faster, and
more accurate:

* The related tags are now calculated by taking a random sample of 1000
  posts, finding the top 250 most frequent tags among those posts, then
  ordering those tags by cosine similarity.

* Related tags can generally be calculated in 50-300ms at these sample
  sizes. Very high sample sizes (25000+ posts) are still relatively fast
  (1-3 seconds), but generally they don't improve accuracy much.

* Related tags are now cached in redis rather than in the tags table.
  The related_tags column in the tags table is no longer used.

* Only the related tags in the search taglist are cached. The related
  tags returned by the 'Related tags' button are not cached.

* The cache lifetime is a fixed 4 hours.

* The 'Related tags' button now works with metatags.

* The /related_tag page now works with metatags and multitag searches.

Fixes #4134, #4146.
This commit is contained in:
evazion
2019-08-30 19:08:56 -05:00
parent 7b8584e3b0
commit 6dd331745a
11 changed files with 99 additions and 256 deletions

View File

@@ -1,27 +1,8 @@
class RelatedTagsController < ApplicationController
respond_to :json, :xml, :js, :html, except: [:update]
before_action :require_reportbooru_key, only: [:update]
skip_forgery_protection only: [:update]
respond_to :json, :xml, :js, :html
def show
@query = RelatedTagQuery.new(query: params[:query], category: params[:category], user: CurrentUser.user)
respond_with(@query)
end
def update
@tag = Tag.find_by_name(params[:name])
@tag.related_tags = params[:related_tags]
@tag.related_tags_updated_at = Time.now
@tag.post_count = params[:post_count] if params[:post_count].present?
@tag.save
head :ok
end
protected
def require_reportbooru_key
unless Danbooru.config.reportbooru_key.present? && params[:key] == Danbooru.config.reportbooru_key
raise User::PrivilegeError
end
end
end