every few days, we gather a huge backlog for a few hours, and there's not a lot that really fixes it or seems to be causing it
@killeveryhetero the only thing i dont know is like, im not sure this is the fediverse either? like witches is pretty busy and all but it mostly hums along, and while sometimes there's just huge spikes of shit idk that's pretty much always either some big instance going down or in this case remember wasabi went down. again. which fucks up a lot of things
it does seem like there's SOMETHING fucky with your server, which i can neither confirm nor deny is a curse
@killeveryhetero I get backlogs sometimes and I only have a 11 person instance. Scaling is hard and there are some issues in github for scaling/performance problems. Mastodon got out on the right foot by implementing redis and elastic cache from the start. That plus the docker containers makes it easy for people to get started. Low barriers to entry + future scaling are difficult problems/concerns. I'm interesting if we'll see JVM (Kotlin/Scala/Java) activitypub implementations in the future.
@killeveryhetero anecdotally we seem.... to be doing that to? ( i have no idea about actual queue status, just observed lag )
but, yeah, mastodon dot mit does the "sudden massive lag, followed by sudden massive catch up" thing every couple days.
can't speak to whether that's a general load thing or a "soooomething is occasionally pumping load onto the network" sort of thing. :\
@gdkar as far as i can tell it's just a general trend as the fediverse gets more and more lively, which is what concerns me longterm
@gdkar the streakiness of queue lag is just kind of how sidekiq works -- it goes in spurts -- that doesn't matter unless the sidekiq workers can't keep up overall
Unstoppable shitposting engine.