posts: normalize Unicode to NFC form in post sources.
Fix strings like "pokémon" (NFD form) and "pokémon" (NFC form) being considered different strings in sources. Also add a fix script to fix existing sources. There were only 15 posts with unnormalized sources.
This commit is contained in:
14
script/fixes/097_normalize_post_sources.rb
Executable file
14
script/fixes/097_normalize_post_sources.rb
Executable file
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env ruby
|
||||
|
||||
require_relative "base"
|
||||
|
||||
with_confirmation do
|
||||
CurrentUser.scoped(User.system, "127.0.0.1") do
|
||||
Post.where("source ~ '[^[:ascii:]]'").find_each do |post|
|
||||
next if post.source.unicode_normalize(:nfc) == post.source
|
||||
|
||||
post.update!(source: post.source)
|
||||
puts({ id: post.id, old_source: post.source_before_last_save, new_source: post.source })
|
||||
end
|
||||
end
|
||||
end
|
||||
Reference in New Issue
Block a user