On June 27:th, the https certificate for lemmy.today was supposed to renew itself. Its set up to automatically renew every 3 months, and its been working perfectly since the site was started over 2 years ago.
But this time it failed to do so, due to a syntax error in the web server config that was not noticed until this incident took place.
Its been fixed now but took more than a day unfortunantly. Sincere apologies for the long delay - it was because I was on vacation in Madeira and in the process of travelling home that day.
Life finds a way to kick us in the balls sometimes.
Thanks for fixing it, sorry it came while on vacation
Thanks! Happy to know it got resolved, and I hope you had a nice vacation!
It was very nice and very much needed to get a break from work for a while. Thanks for your understanding!
Yayyy we’re back up! Thanks for the update. And we totally understand that you have a life outside of here lol :p
Enjoy the vacation, appreciate you so much (:
Was really worried for a bit that y’all just shut the instance down. I was thinking “at least give me time to migrate my Furryosa pics!!!”
We will never shut it down just like that… It would be incredibly rude and insensitive towards you guys and we just dont operate that way as people.
This place is supposed to be very different from big tech platforms, and that means we see users as people to start with.
See you guys around!
Chad energy having mfing mod!
May I suggest signing up for hetrixtools.com
Free monitoring service that would have alerted you some weeks before the certificate expired.
Its an auto-renewing cert so it “always” works, but in this case there was a syntax error in the web server config, causing it to fail to actually reload the new cert. Will see what we can do to make that not happen again!
That cert is valid for 90 days at a time and is renewed every 60 days. So having an alert if the current certificate has less than 28 days remaining will give you an early warning that something went wrong with automation.
Ah yeah, thats a good idea. Thanks!
Life finds a way to kick us in the balls sometimes.
BOHICA. :p
old.lemmy.today (and all the other alternative interfaces listed in the sidebar) are still having issues. They all seem to be pointing to the default interface instead of their respective alternatives. They’re also throwing cert errors, probably because the cert is now for “lemmy.today” instead of “*.lemmy.today”
Will be fixed today also, thanks!
You da man! Working great now!
Thanks so much man.
When I got certificate error, I got really surprised. Nowadays no one does it manually. That was the only part made me a little bit concerned. Thanks for having this instance. I really appreciate the effort.
Thanks for looking at it on your vacation! Sorry it came up then!
Understandable. Have a nice day!
I was worried, so I checked the error. And it made me confident something had gone wrong with autorenewal.
Given that my job had a days-long partial outage due to CLOCK SKEW between nodes, a cert renewal issue is positively understandable.
Glad you were able to get it sorted out, but hope you have some more vacation to get back to!
Oh man, clock skew between servers totally screwed me about two years ago because the servers were load balanced traffic and applied their server timestamp to realtime data - which in some cases led to entries being out of order. (never mind all the other issues with this implementation)
Let’s encrypt?
I too thought maybe the instance shut down. I’m glad it didn’t. No worries, I hope your vacation went well. I’ll survive a few days without social media
