tags: track tag histories.

Track the history of the tag `category` and `is_deprecated` fields in
the `tag_versions` table.

Adds generic Versionable and VersionFor concerns that encapsulate most
of the history tracking logic. These concerns are designed to make it
easy to add history to any model.

There are a couple notable differences between tag versions and other versions:

* There is no 1 hour edit merge window. All changes to the `category`
  and `is_deprecated` fields produce a new version in the tag history.

* New versions aren't created when a tag is created. Versions are only
  created when a tag is edited for the first time. The tag's initial
  version isn't created until *after* the tag is edited for the first time.

For example, if you change the category of a tag that was last updated
10 years ago, that will create an initial version of the tag backdated
to 10 years ago, plus a new version for your edit.

This is for a few reasons:

* So that we don't have to create new tag versions every time a new tag
  is created. This would be wasteful because most tags never have their
  category or deprecation status change.
* So that if you make a typo tag, your name isn't recorded in the tag's
  history forever.
* So that we can create new tags in various places without having to know
  who created the tag (which may be unknown if the current user isn't set).
* Because we don't know the full history of most tags, so we have to
  deal with incomplete histories anyway.

This has a few important consequences:

* Most tags won't have any tag versions. They only gain tag versions if
  they're edited.
* You can't track /tag_versions to see newly created tags. It only
  shows changes to already existing tags.
* Tag version IDs won't be in strict chronological order. Higher IDs may
  have created_at timestamps before lower IDs. For example, if you
  change the category of a tag that is 10 years old, that will create an
  initial version with a high ID, but with a created_at timestamp dated
  to 10 years ago.

Fixes #4402: Track tag category changes

app/controllers/tags_controller.rb

@@ -26,7 +26,7 @@ class TagsController < ApplicationController
 
   def update
     @tag = authorize Tag.find(params[:id])
-    @tag.update(permitted_attributes(@tag))
+    @tag.update(updater: CurrentUser.user, **permitted_attributes(@tag))
     respond_with(@tag)
   end
 end

app/logical/bulk_update_request_processor.rb

@@ -212,17 +212,17 @@ class BulkUpdateRequestProcessor
 
         when :change_category
           tag = Tag.find_or_create_by_name(args[0])
-          tag.update!(category: Tag.categories.value_for(args[1]))
+          tag.update!(category: Tag.categories.value_for(args[1]), updater: User.system)
 
         when :deprecate
           tag = Tag.find_or_create_by_name(args[0])
-          tag.update!(is_deprecated: true)
+          tag.update!(is_deprecated: true, updater: User.system)
           TagImplication.active.where(consequent_name: tag.name).each { |ti| ti.reject!(User.system) }
           TagImplication.active.where(antecedent_name: tag.name).each { |ti| ti.reject!(User.system) }
 
         when :undeprecate
           tag = Tag.find_or_create_by_name(args[0])
-          tag.update!(is_deprecated: false)
+          tag.update!(is_deprecated: false, updater: User.system)
 
         else
           # should never happen

app/logical/concerns/version_for.rb

@@ -0,0 +1,103 @@
+# frozen_string_literal: true
+
+# A concern that adds a `version_for` macro for declaring that a model is the
+# version model (TagVersion, PostVersion, etc) belonging to a versionable model
+# (Tag, Post, etc). The counterpart to Versionable.
+#
+# Defines helper methods like `undo!`, `revert_to!`, `diff`, etc.
+#
+# Assumes the class has `previous_version_id` and `version` columns.
+#
+# @example
+#   class TagVersion
+#     include VersionFor
+#     version_for :tag
+#   end
+#
+module VersionFor
+  extend ActiveSupport::Concern
+
+  class_methods do
+    # Declare a class as the version model belonging to a `versionable` model.
+    def version_for(versioned_model_name)
+      raise "#{name} must have a `previous_version_id` attribute" if !has_attribute?(:previous_version_id)
+      raise "#{name} must have a `version` attribute" if !has_attribute?(:version)
+
+      @versioned_model_name = versioned_model_name                      # "tag"
+      @versioned_model_id_column = "#{versioned_model_name}_id"         # "tag_id"
+      @versioned_class = versioned_model_name.to_s.camelize.constantize # Tag
+
+      self.class.attr_reader :versioned_model_name, :versioned_model_id_column, :versioned_class
+      delegate :versioned_model_name, :versioned_model_id_column, :versioned_class, to: :class
+      delegate :versioned_columns, to: :versioned_class
+
+      belongs_to versioned_model_name
+      belongs_to :updater, class_name: "User", optional: true
+      belongs_to :previous_version, class_name: name, optional: true
+
+      validates :previous_version_id, uniqueness: { scope: versioned_model_id_column } # scope: :tag_id
+
+      before_save :increment_version
+      after_save :validate_previous_version
+
+      scope :first_version, -> { where(previous_version: nil) }
+      scope :last_version, -> { where.not(id: where.not(previous_version: nil).select(:previous_version_id)) }
+
+      alias_method :versioned_model, versioned_model_name
+    end
+  end
+
+  # XXX This is an after_save callback instead of a normal validation so we can refer to the `id`,
+  # `created_at`, and `updated_at` columns (which aren't available until after saving the row).
+  def validate_previous_version
+    if previous_version.present? && previous_version_id >= id
+      raise "The previous version must be before the current version (id=#{id}, previous_version.id=#{previous_version.id})"
+    elsif previous_version.present? && previous_version.version >= version
+      raise "The previous version must be before the current version (version=#{version}, previous_version.version=#{previous_version.version})"
+    elsif previous_version.present? && previous_version.created_at >= updated_at
+      raise "The previous version must be before the current version (updated_at=#{updated_at}, previous_version.created_at=#{previous_version.created_at})"
+    elsif previous_version.present? && previous_version.updated_at >= updated_at
+      raise "The previous version must be before the current version (updated_at=#{updated_at}, previous_version.updated_at=#{previous_version.updated_at})"
+    elsif previous_version.present? && previous_version.versioned_model != versioned_model
+      raise "The previous version must belong to the same #{versioned_model_name} (#{versioned_model_id_column}=#{versioned_model.id}, previous_version.#{versioned_model_id_column}=#{previous_version.versioned_model.id})"
+    end
+  end
+
+  def increment_version
+    # XXX We assume the versioned model is locked so that this is an atomic increment and not subject to a race condition.
+    self.version = previous_version&.version.to_i + 1
+  end
+
+  # Return a hash of the versioned columns with their values.
+  def versioned_attributes
+    attributes.with_indifferent_access.slice(*versioned_columns)
+  end
+
+  # True if this is the first version in the versioned item's edit history.
+  def first_version?
+    previous_version.nil?
+  end
+
+  # Return a hash of changes made by this edit (compared to the previous version, or to another version).
+  #
+  # The hash looks like `{ attr => [old_value, new_value] }`.
+  def diff(version = previous_version)
+    versioned_columns.map { |attr| [attr, [version&.send(attr), send(attr)]] }.to_h
+  end
+
+  # Revert the model back to this version.
+  def revert_to!(updater)
+    versioned_model.update!(updater: updater, **versioned_attributes)
+  end
+
+  # Undo the changes made by this edit (compared to the previous version, or to another version).
+  def undo!(updater, version: previous_version)
+    return if version.nil?
+
+    diff(version).each do |attr, old_value, new_value|
+      versioned_model[attr] = old_value if old_value != new_value
+    end
+
+    versioned_model.update!(updater: updater)
+  end
+end

app/logical/concerns/versionable.rb

@@ -0,0 +1,105 @@
+# frozen_string_literal: true
+
+# A concern used by versioned models (Tag, Post, etc). Adds a `versionable`
+# macro for declaring that a model is versioned.
+#
+# Assumes the class has an `updater` attribute that contains the user who
+# updated the model when the model is saved.
+#
+# @example
+#   class Tag
+#     include Versionable
+#     versionable :name, :category, :is_deprecated
+#   end
+#
+module Versionable
+  extend ActiveSupport::Concern
+
+  class_methods do
+    # Declare a model is versioned. Changes to the given `columns` will be saved in a versions table.
+    #
+    # @param columns [Array<Symbol>] The columns to track as versioned. Changes to these columns will be saved to
+    #   the versions table; changes to other columns will be ignored.
+    # @param merge_window [Duration] Merge multiple edits made by the same user within this time frame into one version.
+    # @param delay_first_version [Boolean] If true, don't create the first version until after the object is edited
+    #   for the first time. If false, create the first version immediately when the object is first created.
+    def versionable(*columns, merge_window: 1.hour, delay_first_version: false)
+      raise "#{name} must have `updater` attribute" if !method_defined?(:updater)
+
+      @versioned_columns = columns
+      @version_merge_window = merge_window
+      @delay_first_version = delay_first_version
+
+      self.class.attr_reader :versioned_columns, :version_merge_window, :delay_first_version
+      delegate :versioned_columns, :version_merge_window, :delay_first_version, to: :class
+
+      has_many :versions, -> { order(id: :asc) }, class_name: "#{name}Version", dependent: :destroy, inverse_of: model_name.singular, after_add: :reset_version_association_cache
+      has_one :first_version, -> { first_version }, class_name: "#{name}Version"
+      has_one :last_version, -> { last_version }, class_name: "#{name}Version"
+
+      after_save :save_version
+    end
+  end
+
+  # Return a hash of the versioned columns with their current values.
+  def versioned_attributes
+    versioned_columns.map { |attr| [attr, send(attr)] }.to_h.with_indifferent_access
+  end
+
+  # Return a hash of the versioned columns with their values before the last save.
+  def versioned_attributes_before_last_save
+    versioned_columns.map { |attr| [attr, attribute_before_last_save(attr)] }.to_h.with_indifferent_access
+  end
+
+  def saved_changes_to_versioned_attributes?
+    saved_changes? && versioned_columns.any? { |attr| saved_change_to_attribute?(attr) }
+  end
+
+  def save_version
+    return unless saved_changes_to_versioned_attributes?
+    raise "Can't save version because updater not set" if updater.nil? && (merge_version? || create_new_version?)
+
+    if create_first_version?
+      create_first_version
+    end
+
+    if merge_version?
+      merge_version
+    elsif create_new_version?
+      create_new_version
+    end
+  end
+
+  # True if this edit should be merged into the previous edit by the same user.
+  def merge_version?
+    version_merge_window.present? && last_version.present? && last_version.updater == updater && last_version.created_at > version_merge_window.ago
+  end
+
+  # True if this edit should create a new version. We don't create a new version if this is a new record and creation of the first version is delayed.
+  def create_new_version?
+    !previously_new_record? || (previously_new_record? && !delay_first_version)
+  end
+
+  # True if this edit should create the first version if the first version was delayed.
+  def create_first_version?
+    delay_first_version && !previously_new_record? && first_version.nil?
+  end
+
+  def merge_version
+    last_version.update!(updater: updater, **versioned_attributes)
+  end
+
+  def create_new_version
+    versions.create!(updater: updater, previous_version: last_version, **versioned_attributes)
+  end
+
+  def create_first_version
+    versions.create!(updater: try(:creator), previous_version: nil, created_at: updated_at_before_last_save, updated_at: updated_at_before_last_save, **versioned_attributes_before_last_save)
+  end
+
+  # After a new version is created, we have to clear the assocation cache manually so it doesn't return stale results.
+  def reset_version_association_cache(record)
+    association(:first_version).reset
+    association(:last_version).reset
+  end
+end

app/logical/tag_mover.rb

@@ -37,9 +37,9 @@ class TagMover
   # Sync the category of both tags, if one is a general tag and the other is non-general.
   def move_tag_category!
     if old_tag.general? && !new_tag.general?
-      old_tag.update!(category: new_tag.category)
+      old_tag.update!(category: new_tag.category, updater: user)
     elsif new_tag.general? && !old_tag.general?
-      new_tag.update!(category: old_tag.category)
+      new_tag.update!(category: old_tag.category, updater: user)
     end
   end
 

app/models/artist.rb

@@ -187,7 +187,7 @@ class Artist < ApplicationRecord
       return unless !is_deleted? && name_changed? && tag.present?
 
       if tag.category_name != "Artist" && tag.empty?
-        tag.update!(category: Tag.categories.artist)
+        tag.update!(category: Tag.categories.artist, updater: CurrentUser.user)
       end
     end
   end

app/models/tag.rb

@@ -1,11 +1,15 @@
 # frozen_string_literal: true
 
 class Tag < ApplicationRecord
+  include Versionable
+
   ABBREVIATION_REGEXP = /([a-z0-9])[a-z0-9']*($|[^a-z0-9']+)/
 
   # Tags that are permitted to have unbalanced parentheses, as a special exception to the normal rule that parentheses in tags must balanced.
   PERMITTED_UNBALANCED_TAGS = %w[:) :( ;) ;( >:) >:(]
 
+  attr_accessor :updater
+
   has_one :wiki_page, :foreign_key => "title", :primary_key => "name"
   has_one :artist, :foreign_key => "name", :primary_key => "name"
   has_one :antecedent_alias, -> {active}, :class_name => "TagAlias", :foreign_key => "antecedent_name", :primary_key => "name"
@@ -25,6 +29,8 @@ class Tag < ApplicationRecord
   after_save :update_category_cache, if: :saved_change_to_category?
   after_save :update_category_post_counts, if: :saved_change_to_category?
 
+  versionable :name, :category, :is_deprecated, merge_window: nil, delay_first_version: true
+
   scope :empty, -> { where("tags.post_count <= 0") }
   scope :nonempty, -> { where("tags.post_count > 0") }
   scope :deprecated, -> { where(is_deprecated: true) }
@@ -194,10 +200,11 @@ class Tag < ApplicationRecord
       end
 
       def find_or_create_by_name(name, category: nil, current_user: nil)
-        tag = find_or_create_by(name: normalize_name(name))
+        cat_id = categories.value_for(category)
+        tag = create_with(category: cat_id).find_or_create_by(name: normalize_name(name))
 
-        if category.present? && current_user.present? && Pundit.policy!(current_user, tag).can_change_category?
-          tag.update(category: categories.value_for(category))
+        if category.present? && current_user.present? && cat_id != tag.category && Pundit.policy!(current_user, tag).can_change_category?
+          tag.update(category: cat_id, updater: current_user)
         end
 
         tag

app/models/tag_version.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+class TagVersion < ApplicationRecord
+  include VersionFor
+
+  version_for :tag
+end