despite not really growing much -- we actually shrunk a bit right after our big hardware upgrade -- we've been lagging more and more lately

every few days, we gather a huge backlog for a few hours, and there's not a lot that really fixes it or seems to be causing it

as far as I can tell, it's coming from the general growth of the fediverse as a whole. with increased traffic coming to and from our instance (and any other instances as federated as we are), resource strain is raising across the board

this concerns me as a serious longterm viability problem for the fediverse, or at least for resource-intensive fediverse apps like mastodon. if this trend continues, small instances will eventually stop being economically feasible, if they're getting strained regardless of usercount costs $50 per month, and we're *really not that large*. and we're *lagging*. that's kind of concerning.

i don't have a solution, but, there it is.

@killeveryhetero i would say that patreon has helped a LOT with this but you're right - there's going to come a point at which personal and small instance ownership and maintenance will be a luxury afforded to only those with the money or infrastructure control to keep it up

@mythismathic yeah, is also mostly crowdfunded. we were 100% crowdfunded before, but we lost a large-ish pledge at some point -- i havent resolicited the rest of the funds because i'm holding off until i get my shit together and restore our customizations from when i broke my github, and i can cover it anyway

but yeah

$50/month is a *lot* of money, the server hardware we're on was always supposed to be overkill

i want to just put the messaging out there since i've seen this pattern happen so many times before, going back to the 90s even:

social network loads scale exponentially. not even multiplicatively - exponentially.

the more networks there are and the more users and posts to serve up to each one, the more each one requires more resources for every individual post it pulls;

you see the problem, don't you? plurality will hit a saturation point unless we wildly expand our tech infrastructure and control over it to match.

we're dealing with the problems of success. let's try to outmaneuver them.

cc @killeveryhetero for excellent discussion of the subject!

@killeveryhetero the only thing i dont know is like, im not sure this is the fediverse either? like witches is pretty busy and all but it mostly hums along, and while sometimes there's just huge spikes of shit idk that's pretty much always either some big instance going down or in this case remember wasabi went down. again. which fucks up a lot of things

it does seem like there's SOMETHING fucky with your server, which i can neither confirm nor deny is a curse

@killeveryhetero maybe Mastodon isn't decentralized enough. Right now it's only decentralized at the server level...

What if everytime I browsing their instance did some of the servers work for it in the background?

Instances would then get stronger as more people used it, like a torrent does

@tootbrute every instance stores all of the content of every other instance it communicates with

when you look at my post from your instance... you're being served the content from your instance

this means that even storage is being duplicated across the fediverse exponentially, which is also a longterm concern

@killeveryhetero yes that's good to avoid censorship but bad for efficiency

Does not seem the fediverse in its current firm could scale to 10 million users

@tootbrute well, mastodon can't

pleroma possibly could, it's way more resource efficient at the cost of features

@killeveryhetero I get backlogs sometimes and I only have a 11 person instance. Scaling is hard and there are some issues in github for scaling/performance problems. Mastodon got out on the right foot by implementing redis and elastic cache from the start. That plus the docker containers makes it easy for people to get started. Low barriers to entry + future scaling are difficult problems/concerns. I'm interesting if we'll see JVM (Kotlin/Scala/Java) activitypub implementations in the future.

@killeveryhetero this scares me a lot. as the fediverse increases in size and scale, we're really going to see corporate resistance to the re-personalization of the social internet in the form of net neutrality regulation, ISP throttling, cloud service denials that as often come from service failure as from outside attack;

we need to start creating redundant infrastructure at the high-scale for when the corporations decide to pull their bullshit and i don't know who needs to hear this message but i'm putting it out there all the same. it scares me that we're in this one deep.

@killeveryhetero anecdotally we seem.... to be doing that to? ( i have no idea about actual queue status, just observed lag )

but, yeah, mastodon dot mit does the "sudden massive lag, followed by sudden massive catch up" thing every couple days.

can't speak to whether that's a general load thing or a "soooomething is occasionally pumping load onto the network" sort of thing. :\

@gdkar as far as i can tell it's just a general trend as the fediverse gets more and more lively, which is what concerns me longterm

@gdkar the streakiness of queue lag is just kind of how sidekiq works -- it goes in spurts -- that doesn't matter unless the sidekiq workers can't keep up overall

Sign in to participate in the conversation

Unstoppable shitposting engine.