jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

storage manager: remove hierarchical option.

Browse Source 
Remove the `hierarchical` file storage option. This means that image
files are always stored in MD5-based subdirectories, like this:

   https://danbooru.donmai.us/data/original/f3/a7/f3a70a89c350b5ed4db22dbb25b934bb.jpg
   https://danbooru.donmai.us/data/sample/f3/a7/sample-f3a70a89c350b5ed4db22dbb25b934bb.jpg
   https://danbooru.donmai.us/data/preview/f3/a7/f3a70a89c350b5ed4db22dbb25b934bb.jpg

instead of in a single flat directory, like this:

   https://danbooru.donmai.us/data/original/f3a70a89c350b5ed4db22dbb25b934bb.jpg

This option is removed because storing files in a single directory is a
bad idea for large installations, and migrating from a single directory
to subdirectories later is a pain.

Downstream boorus who still have files in the old layout can migrate by
running this script:

   `./script/fixes/077_symlink_subdirectories.rb`

This will create symlinks that redirect the 00-ff subdirectories back to
the current directory, so that you can still store files in a single
directory, but use URLs containing subdirectories.

You should also make sure to remove the `hierarchical` option from
`storage_manager` in `config/danbooru_local_config.rb` if you set it
there.
This commit is contained in:
evazion
2021-03-17 00:01:01 -05:00
parent a620a71b59
commit 29d2e7fed2
7 changed files with 57 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
app/logical/storage_manager.rb
View File

@@ -1,12 +1,11 @@
class StorageManager
class Error < StandardError; end
attr_reader :base_url, :base_dir, :hierarchical, :tagged_filenames
attr_reader :base_url, :base_dir, :tagged_filenames
def initialize(base_url:, base_dir:, hierarchical: false, tagged_filenames: Danbooru.config.enable_seo_post_urls)
def initialize(base_url:, base_dir:, tagged_filenames: Danbooru.config.enable_seo_post_urls)
@base_url = base_url.chomp("/")
@base_dir = base_dir
@hierarchical = hierarchical
@tagged_filenames = tagged_filenames
end
@@ -98,11 +97,7 @@ class StorageManager
end
def subdir_for(md5)
if hierarchical
"#{md5[0..1]}/#{md5[2..3]}/"
else
""
end
"#{md5[0..1]}/#{md5[2..3]}/"
end
def seo_tags(post)

6
app/logical/storage_manager/match.rb
View File

@@ -8,15 +8,15 @@
#
# StorageManager::Match.new do |matcher|
# matcher.add_manager(type: :crop) do
# StorageManager::SFTP.new("raikou3.donmai.us", base_url: "https://raikou3.donmai.us", hierarchical: true, base_dir: "/var/www/raikou3")
# StorageManager::SFTP.new("raikou3.donmai.us", base_url: "https://raikou3.donmai.us", base_dir: "/var/www/raikou3")
# end
#
# matcher.add_manager(id: 1..850_000) do
# StorageManager::SFTP.new("raikou1.donmai.us", base_url: "https://raikou1.donmai.us", hierarchical: true, base_dir: "/var/www/raikou1")
# StorageManager::SFTP.new("raikou1.donmai.us", base_url: "https://raikou1.donmai.us", base_dir: "/var/www/raikou1")
# end
#
# matcher.add_manager(id: 850_001..2_000_000) do
# StorageManager::SFTP.new("raikou2.donmai.us", base_url: "https://raikou2.donmai.us", hierarchical: true, base_dir: "/var/www/raikou2")
# StorageManager::SFTP.new("raikou2.donmai.us", base_url: "https://raikou2.donmai.us", base_dir: "/var/www/raikou2")
# end
#
# matcher.add_manager(id: 1..3_000_000, type: [:large, :original]) do