despite jorts.horse not really growing much -- we actually shrunk a bit right after our big hardware upgrade -- we've been lagging more and more lately
@killeveryhetero the only thing i dont know is like, im not sure this is the fediverse either? like witches is pretty busy and all but it mostly hums along, and while sometimes there's just huge spikes of shit idk that's pretty much always either some big instance going down or in this case remember wasabi went down. again. which fucks up a lot of things
it does seem like there's SOMETHING fucky with your server, which i can neither confirm nor deny is a curse
@killeveryhetero maybe Mastodon isn't decentralized enough. Right now it's only decentralized at the server level...
What if everytime I browsing their instance did some of the servers work for it in the background?
Instances would then get stronger as more people used it, like a torrent does
@tootbrute every instance stores all of the content of every other instance it communicates with
when you look at my post from your instance... you're being served the content from your instance
this means that even storage is being duplicated across the fediverse exponentially, which is also a longterm concern
@killeveryhetero yes that's good to avoid censorship but bad for efficiency
Does not seem the fediverse in its current firm could scale to 10 million users
@tootbrute well, mastodon can't
pleroma possibly could, it's way more resource efficient at the cost of features
@killeveryhetero I get backlogs sometimes and I only have a 11 person instance. Scaling is hard and there are some issues in github for scaling/performance problems. Mastodon got out on the right foot by implementing redis and elastic cache from the start. That plus the docker containers makes it easy for people to get started. Low barriers to entry + future scaling are difficult problems/concerns. I'm interesting if we'll see JVM (Kotlin/Scala/Java) activitypub implementations in the future.
@killeveryhetero anecdotally we seem.... to be doing that to? ( i have no idea about actual queue status, just observed lag )
but, yeah, mastodon dot mit does the "sudden massive lag, followed by sudden massive catch up" thing every couple days.
can't speak to whether that's a general load thing or a "soooomething is occasionally pumping load onto the network" sort of thing. :\
@gdkar as far as i can tell it's just a general trend as the fediverse gets more and more lively, which is what concerns me longterm
@gdkar the streakiness of queue lag is just kind of how sidekiq works -- it goes in spurts -- that doesn't matter unless the sidekiq workers can't keep up overall
Unstoppable shitposting engine.