Files
danbooru/script/fixes/097_normalize_post_sources.rb
evazion 61c043c6b1 posts: normalize Unicode to NFC form in post sources.
Fix strings like "pokémon" (NFD form) and "pokémon" (NFC form) being
considered different strings in sources.

Also add a fix script to fix existing sources. There were only 15 posts
with unnormalized sources.
2022-01-31 14:16:49 -06:00

15 lines
398 B
Ruby
Executable File

#!/usr/bin/env ruby
require_relative "base"
with_confirmation do
CurrentUser.scoped(User.system, "127.0.0.1") do
Post.where("source ~ '[^[:ascii:]]'").find_each do |post|
next if post.source.unicode_normalize(:nfc) == post.source
post.update!(source: post.source)
puts({ id: post.id, old_source: post.source_before_last_save, new_source: post.source })
end
end
end