Just curious if it’s somehow got to do with anything on my end. I’m generally desktop-only, firefox-only.
Yeah, It’s true. Since Sunday, I’ve been noting errors that I’m still working on resolving. It doesn’t make it easier that it’s the post-holiday period, and due to travels and security measures, it’s not the easiest task. I’m working to get everything back to normal as soon as possible.
https://kbin.social/m/kbinDevlog/t/729349/RTR-49-On-site-work
Thanks for putting in all this work, especially over a period that’s traditionally vacation time. Make sure you’re striking a good work/life balance, if you can get the site basically functional (as it appears to be now) don’t sweat the small stuff. :)
Not just me eh, sorry to hear! I had a Jellyfin upgrade go sideways (my fault) once during the holidays and that was bad enough - and all my users live with me! Sorry that you are pulling your hair out, and personally I’m more than content to wait it out until after your vacation.
Username doesn’t lie. :)
Thanks for all the hardwork you are putting in! In the long term, you could consider setting up a kbin.social status page?
Thanks Ernest!
I’m curious, 2hen you say “on-site work”, do you mean you need to travel onsite to do some work for kbin? At a host somewhere? Otherwise, when you say “security measures” for travel, how is that related? Maybe you just mean you are travelling and it is taking up your time…?
It seems a lot more stable right now. I expect @ernest has been occupied with, y’know, actually having a life. Seeing as it’s Christmas and all that.
its gotten worse for me. I used to be able to see my notification but now get 404 for that to.
In addition to the 404 thing that has been posted, nearly every upvote/downvote/click of a page element is getting me the “We are working on resolving issues” error. Took me several tries to get here to type this, and not sure how many it will take to get the comment to actually post.
I’m certainly open to the idea that it’s somehow just me though, it seemed like he was doing a lot of nip and tuck leading up to the holiday, unless I misread his devblog posts.
Nope, on kbin.social, still 100% 404s on the notifications page, timeouts on pages constantly, the “we’re working on it” returns for pages, etc.
For the 404s, try adding ?p=1 to the end of the URL. This lets me use all and subscribed (mostly) as normal.
I think we are all SoL on the error pages though.
In addition to the 404 thing that has been posted, nearly every upvote/downvote/click of a page element is getting me the “We are working on resolving issues” error. Took me several tries to get here to type this, and not sure how many it will take to get the comment to actually post.
I’m certainly open to the idea that it’s somehow just me though, it seemed like he was doing a lot of nip and tuck leading up to the holiday, unless I misread his devblog posts.
It’s the new lemmy update that is the issue. It broke a lot. Lots of people are abandoning lemmy anyhow, which is why the servers are so empty now. I don’t expect many lemmy servers to continue in the next six months.
Kbin is not lemmy
the lemmy changes are causing excessive resource use on my 'bin instance. so yeah, not using lemmy, but being directly affected by the lemmy snafu.
my failed messaging queue is filling, which has its own retry logic… that queue buildup also takes disk space… extra processing, extra disk space… this leads to ‘worker’ slowdown and then system failures and timeouts.
When I looked at Kbin the “caddy” was wrapped around RabbitMQ. You can get RabbitMQ to solve a lot of those issues.
Firstly with Rabbit you can set a Time To Live header in messages.
By default RabbitMQ queues have no limit in size, you can set a limit.
Lastly RabbitMQ allows message prioritisation. So you can drop the priority of things the older/more retries they contain.
Most of this is either RabbitMQ policy or Queue rules based on Headers in the AMQP message. Depending on how KBin is generating messages you might be able to do this as a system admin
What does any of this have to do with kbin? Kbin is not Lemmy. Completely different codebase.
the lemmy changes are causing excessive resource use on my 'bin instance. so yeah, not using lemmy, but being directly affected by the lemmy snafu.
my failed messaging queue is filling, which has its own retry logic… that queue buildup also takes disk space… extra processing, extra disk space… this leads to ‘worker’ slowdown and then system failures and timeouts.
Oh, interesting. My bad then, it’s common for people to be unaware that kbin is a different thing from Lemmy and so I made an incorrect assumption.
I suppose this reveals some room for improvement in kbin, then. Other servers’ problems shouldn’t be impacting kbin as badly as this, likely indicating that kbin needs to add some robustness when it comes to dealing with stuff like this.
your thought process isnt completely off. if my server product was detecting the failures correctly, these resources wouldnt pile up.
i dont think people really understand just how brand new all this stuff is. ‘the fediverse’ is under active development. they call it the ‘bleed edge’ of technology because its painful. most fediverse servers are experiencing growing pains of some sort.
the Lemmy/kbin sides are still wet behind the ears. i just hope people dont give up!
The beginning of reddit was much the same. Things stopped working all the time. Weird bugs popped up. And there were people posting posts like this a lot trying to figure out what was going on.
Just curious, why do you say that lots of people are abandoning Lemmy? I’m still seeing lots of users and posts from Lemmy instances every day.
I don’t expect many lemmy servers to continue in the next six months.
LOL. I am expecting Lemmy to continue far into the future.
And upvotes and boosts aren’t turning green for some reason (in comments only)
Things seem to be starting to resolve for me. I can now get to Notifications and to Newest.
Still getting errors, but now works on a retry.
So it’s a site-wide issue? I’ll not start troubleshooting my account then and can calmly wait for the fix.
