An update on the situation earlier today where the server was down due to the disk being full. I found the culprits, mostly media_attachments and statuses. Thankfully, mastodon has a cli program to remove the extra cruft.
tootctl media remove
tootctl statuses remove
The first command gets rid of media saved that is not from our local users. The second command removes fediverse toots from the server that have no local interaction.
I'll try to keep an eye on this stuff in the future.
@moth Would make sense. I'll probably setup a systemd timer to take care of this for me.
@derek seems like an oops moment to me.
@derek I recommend something like Wazuh/Ossec to monitor everything. Kibana/Elastic would be big plus on top of that
@derek media and statuses are bloat 😜
@derek is that something that should be in a crontab maybe?
@derek
I had a similar thing happen to my #mastodon #instance yesterday. It was working fine then suddenly it just locked me out.
I checked the drive and it was full, but after I had cleared the problem it still wouldn't let me in.
I read through a few things and figured it may well need updating. It failed. I found out that mimemagic had been yanked at about the same time and is preventing me from even reinstalling a new instance, so I'm migrating it over to #plemora.
@Coopertronics @derek Yeah mimemagic had to be rereleased under GPL2 for license compatibility. Which borked RoR projects and it'll take time to fix since RoR likes MIT/permissive licenses. Fun, right? 🙃
@trippedup @derek
It certainly is. They've pulled mimemagic 0.3.6 as well now, so the fix someone had created using it stopped working after about 3 hours.
@Coopertronics @derek The repos giveth, the repos taketh away. 😜
@trippedup @derek
💀 Ashes to ashes, dust to dust. 🤣
@derek Please keep sharing these things. I'm running an instance too. Thanks for running this one. I enjoy it.
@derek Thank you for your hard work providing us with this platform! #fediverse #mastodon
@derek Sorry to hear about that Dereck.
Why not cron job?