|terribleplan 3d7e58cb45 Final changes once I completed our migration||3 years ago|
|.gitignore||3 years ago|
|README.md||3 years ago|
|cors-config.json||3 years ago|
|move-media.js||3 years ago|
|package.json||3 years ago|
|yarn.lock||3 years ago|
With so many instances either relying on S3 or minio, I figured it was high time to throw some diversity into the mix, for a more resilient federation. You should probably read the whole thing before you start, as the process isn’t exactly linear.
fogto upload files. I have such a patch in my fork., feel free to use it (at your own risk).
There are different storage formats in use by paperclip (what mastodon uses for file uploads/storage) for diferent plugins. By setting up the patch (and some environment variable) you will be switching from the built-in filesystem plugin to the fog plugin.
There is a helper script for migrating from FS to GCS in this repo, the gist is that on the filesystem we have:
And when files are stored in google drive they live in:
You can build that folder structure by:
node move-media ../path/to/live .(Puts the images directory here). You can run the above command with
--dry-runto see the output of what the command will do if you don’t trust the code.
Once that folder has been built you can just use Google’s CLI,
gsutil, to upload that folder and set the ACL to public. (
gsutil -m cp -r images gs://media.nrd.li && gsutil -m acl ch -r -u AllUsers:R gs://media.nrd.li)
You probably want to do this step twice, once before you reconfigure your instance, and once again after so that you can be sure all media has been uploaded to cloud storage. I’m sure there’s a way to sync the directory, but ingress is free so I just uploaded it all twice.
By default there are no CORS headers sent, which needs to be rectified.
gsutil cors set config.json gs://bucket-name
You can use
cors-config.json without editing it, but that is slightly less secure than specifying the origin.