Well, multiple disks hovering close to 99%, it’s definitely time to increase storage for OpenAttribution and AppGoblin. I needed cheap S3 hosting, and I had a pretty strong urge to do it myself.
Parts
This should be up to you, but in the end I ended up grabbing an 8-bay Just-a-Bunch-Of-Disks (JBOD) which is a hard drive rack and 5 8TB HDs to go in it. Unfortunately, this is going to be a big pain in the future as increasing the size of arrays is not possible the way I set mine up, so I’ll be stuck moving everything off if I want to add in those final 3 missing HDs when I get the money.
Software MinIO: Looked great but hard to trust
So, I had long ago come to the conclusion that I would be using MinIO. It was suggested all around and looked very feature rich with dashboards, single node / multi-node and lots of cool ways to manage the object storage.
Strangely, the day my JBOD and HDs arrived MinIO had made massive changes to their open source software. It seems that the first part had happened earlier in the year when they switched to AGPL license, and this week they completely removed ALL features from the open source community edition of MinIO GUI.
Their idea it seems was to remove all features from the GUI dashboard, but keep them available in the CLI. As someone who hasn’t used MinIO, it was pretty unclear what I would be missing out on, I didn’t even plan on using the dashboard, but the sense that this was a signal for how the Community Edition will be treated going forward made me worried to use MinIO.
Specifically that it “will only receive security fixes if any” sounded quite ominous and not like how I’d like to start off a likely multiyear project. Finally, that MinIO was straightforwardly mentioning that it’s common for business-related use cases to require a commercial license, so in the end this just wasn’t going to work for me.
Checking out Garage
The next project people mentioned was Garage. This was interesting comparison because it also is AGPLv3 and it didn’t come with a GUI at all. So at first I was thought this was about the same as situation with MinIO. Digging around though I started to get a sense that Garage was distinctly different and had a few things going for it: developing new features, simple documentation and a strong self-hosted mentality.
Reaching out a bit in the community I got open and constructive feedback. I was in.
Setting up a Single Node S3 in Garage
The Garage docs were refreshingly simple, and I was able to follow the quick start guide here. The overview of steps is:
- Download the garage executable
- Start the garage server
- Create a layout
- Start Using Garage S3
Now, of course I still had some questions. But I think what made this so smooth was how helpful the Garage community was on their Matrix channel. I definitely recommend joining that if you have questions about the quick start.
Setting up the S3 bucket / keys / and connecting with Python Boto3. This part was also so smooth! As is stated on their site, they follow the regular S3 provisions, so I was able to immediately start using Boto3!
Done!
That was it. I just wanted make a little post about this. Hope the team does well.