| 00:02.10 | znf | honestly, I can't believe how much of a piece of shit is npm and nodejs |
| 00:20.09 | *** join/#wowace purl (ibot@208.53.50.136) |
| 00:20.09 | *** topic/#wowace is https://wowace.com/ | 7.3.x ToC: 70300 | https://wowace.com/paste/ | http://lua.org | This channel is logged, via purl | Official support path https://support.curse.com |
| 01:30.06 | znf | >c 0.00003200 btc to usd |
| 01:30.08 | Catal1na` | 0.000032 BTC == 0.20426464 USD (04-0.18%) https://coinmarketcap.com/currencies/bitcoin |
| 03:06.37 | *** join/#wowace ShadniX_ (dagger@p4FEFCC70.dip0.t-ipconnect.de) |
| 04:09.12 | *** join/#wowace Marla (~textual@209-193-84-82.mammothnetworks.com) |
| 04:42.09 | *** join/#wowace Andols2 (~yaaic@m83-185-244-210.cust.tele2.se) |
| 04:45.55 | *** join/#wowace Marla (~textual@209-193-84-82.mammothnetworks.com) |
| 05:13.24 | *** join/#wowace Megalon (~wig0r@213-157-154-238.adsl.highway.telekom.at) |
| 05:54.57 | *** join/#wowace phyber_ (~phyber@unaffiliated/phyber) |
| 07:01.26 | *** join/#wowace mitch0 (~mitch@94-21-225-5.pool.digikabel.hu) |
| 07:53.55 | *** join/#wowace Megalon (~wig0r@213-157-154-238.adsl.highway.telekom.at) |
| 08:49.08 | *** join/#wowace stolenlegacy (~stolenleg@unaffiliated/stolenlegacy) |
| 09:11.30 | *** join/#wowace Sintacks (~Syntax@75-109-22-174.swbycmtk01.res.dyn.suddenlink.net) |
| 09:43.33 | *** join/#wowace Sintacks (~Syntax@75-109-22-174.swbycmtk01.res.dyn.suddenlink.net) |
| 10:08.31 | *** join/#wowace Ackis (~Ackis@WoWUIDev/WoWAce/ARL/Troll/Ackis) |
| 10:08.32 | *** mode/#wowace [+o Ackis] by ChanServ |
| 14:08.36 | *** join/#wowace Andols (~yaaic@62.119.166.9) |
| 14:54.43 | *** join/#wowace Andols2 (~yaaic@m83-185-244-210.cust.tele2.se) |
| 15:45.15 | *** join/#wowace harl (harl@unaffiliated/harl) |
| 15:54.13 | znf | Gnarfoz, halp |
| 16:04.25 | znf | Gnarfoz, adaptec controller, server rebooted (we had a power surge, ups batteries fried) |
| 16:16.31 | *** join/#wowace Lejving (~Lejving@81-233-148-192-no524.tbcn.telia.com) |
| 17:57.33 | Gnarfoz | so? |
| 17:57.48 | Gnarfoz | znf: guess you figured it out? |
| 17:58.18 | znf | not really :-/ |
| 17:58.47 | znf | What do you get from here? https://i.imgur.com/lAhz5Hj.png |
| 18:00.28 | znf | there's 8 disks in the server |
| 18:00.30 | znf | 0 to 7 |
| 18:00.34 | znf | 0 and 1 are dead |
| 18:00.49 | Gnarfoz | doesn't it have a phyiscal drive view? |
| 18:00.50 | znf | they're numbered CN0/1 : Dev 00 to 04 |
| 18:01.07 | Gnarfoz | somewhere where it complains about what' smissing? |
| 18:01.17 | Gnarfoz | but I assume 0 and 1 are a raid1 group? |
| 18:01.29 | znf | only this at boot: https://i.imgur.com/ajqhQ85.png |
| 18:01.36 | Gnarfoz | (in which case you can say goodbye to the raid10 they're part of) |
| 18:01.42 | znf | I can't tell by how this is numbered |
| 18:01.52 | znf | but I'm also assuming the worst, from what I can deduct |
| 18:02.04 | Gnarfoz | yeah that number is wtf³ |
| 18:02.20 | znf | [00+01] - [02+03] - [04+05] - [06+07] |
| 18:02.22 | znf | that's how they are |
| 18:02.27 | znf | 00+01 are RIP |
| 18:02.39 | Gnarfoz | then whatever 00+01 were part of is also RIP, if it was raid10 |
| 18:02.47 | znf | that's what I feared too |
| 18:02.55 | Gnarfoz | the naming is super strange |
| 18:03.17 | *** join/#wowace stolenlegacy (~stolenleg@unaffiliated/stolenlegacy) |
| 18:03.18 | Gnarfoz | there's only 2 VDs, right? |
| 18:03.23 | znf | yes |
| 18:03.26 | znf | 1 x 50GB |
| 18:03.29 | znf | 1 x 3.5TB |
| 18:03.34 | znf | 8 x 1TB |
| 18:03.40 | Gnarfoz | that's 3? |
| 18:03.40 | Gnarfoz | :D |
| 18:03.46 | znf | no, I mean |
| 18:03.48 | znf | 8 disks x 1 TB |
| 18:03.54 | Gnarfoz | oh, so 8 PDs |
| 18:04.05 | znf | sorry, I'm in full recovery mode and my brain is tired and I have a horrible headache |
| 18:04.11 | Gnarfoz | I know that feel |
| 18:04.25 | Gnarfoz | so I assume the first screenshot looks basicallyt identical for array "01"? |
| 18:04.32 | znf | yes |
| 18:04.34 | Gnarfoz | by which I mean it lists 3 array members, instead of 4 |
| 18:04.39 | znf | mhm |
| 18:04.56 | znf | I was hoping I am reading wrong, and they're 4 x RAID0 |
| 18:05.19 | znf | https://i.imgur.com/KV2rKUl.png |
| 18:05.58 | Gnarfoz | so 01 is missing completely and 00 shows a bad size/speed |
| 18:06.12 | znf | yeah, and it fails media verification |
| 18:06.18 | Gnarfoz | have you tried turning it off and on again |
| 18:06.18 | Gnarfoz | :D |
| 18:06.21 | znf | it wasn't showing up at all, but I had my colleague unplug/replug |
| 18:06.23 | Gnarfoz | (seriously, though) |
| 18:06.36 | znf | I've reset multiple times |
| 18:06.54 | Gnarfoz | yeah... I'd try re-seating them, etc |
| 18:06.55 | znf | I'm having him pull those drives out next and plugging them into a PC to assess their SMART status |
| 18:07.10 | Gnarfoz | that's an option |
| 18:07.17 | znf | he's just grabbing lunch now, he had to cancel date with his GF on her birthday :-| |
| 18:07.22 | Gnarfoz | although I'd be very careful that nothing tries to write to the drives |
| 18:07.44 | Gnarfoz | you could try powering it up with the drives removed |
| 18:07.58 | znf | how's that gonna help? :P |
| 18:07.59 | Gnarfoz | and then plugging them in; maybe it's the controller that's confused |
| 18:08.03 | znf | ah |
| 18:08.13 | znf | doubt it, I had him swap 02/03 with 00/01 |
| 18:08.15 | Gnarfoz | if it sees them as "gone" once, maybe that could clear things up on the controller's end |
| 18:08.26 | znf | the same drives still weren't showing |
| 18:08.31 | Gnarfoz | hm, yeah, that should basically have the same effect |
| 18:08.34 | Gnarfoz | oh, yeah |
| 18:08.35 | znf | (ie: the original 00 and 01) |
| 18:08.50 | Gnarfoz | if the error moved along with the drives, it's unlikely to be anything but the drives |
| 18:08.55 | znf | yup |
| 18:09.17 | znf | kind of weird to have both die during a power outage :-| |
| 18:09.27 | Gnarfoz | pretty common, I'd say |
| 18:09.32 | znf | client didn't report anything wrong with them |
| 18:09.45 | znf | but then again, it's hostgator, and they employ mostly indians |
| 18:10.11 | Gnarfoz | spinning disks just spinning along... then someone gets the idea to move them/turn them off for a while, and bam, they die in droves |
| 18:10.34 | Gnarfoz | (mostly a thing with disks that have been running for long times) |
| 18:10.56 | Gnarfoz | also, you said "surge" earlier, not outage |
| 18:11.04 | Gnarfoz | that might have damaged them directly, theoretically |
| 18:11.21 | znf | yup, I'm assuming it was a surge |
| 18:11.23 | Gnarfoz | either way; I'm still not a 100% on the raid array layout (from the screenshots) |
| 18:11.27 | znf | because our UPS batteries are fried |
| 18:11.37 | znf | both UPSs |
| 18:11.49 | znf | the switchable PDUs have gone in power-protection mode |
| 18:12.01 | znf | (all outlets went to red) |
| 18:12.26 | znf | the fuses in another room not covered by the UPS were all flipped, had to manually unflip them |
| 18:12.34 | Gnarfoz | is it a big RAID10 across all 4 groups of 2 (like you said, 00+01, 02+03, etc), and out of that, there are 2 LUNs (50+3500)? |
| 18:12.53 | Gnarfoz | or is it two RAID10s, one of them across one pair, and the other across 3 pairs |
| 18:12.54 | znf | yup |
| 18:13.03 | znf | uhm |
| 18:13.26 | znf | pretty sure the PDs are partitioned (?!) |
| 18:13.27 | Gnarfoz | (in which case the 3.5 TB one shouldn't be missing any drives?) |
| 18:13.40 | znf | uhm |
| 18:13.44 | Gnarfoz | I don't think any raid controllers do that Intel matrix raid thing |
| 18:14.00 | znf | https://i.imgur.com/eAA4rIa.png |
| 18:14.28 | znf | they are doing that |
| 18:14.37 | Gnarfoz | (where they do, like, a raid1 of partition2 on driveA + partition2 on driveB, and a raid0 of partitions1A and 1B) |
| 18:14.49 | znf | https://i.imgur.com/QzqPgig.png and https://i.imgur.com/QxgkyJ3.png |
| 18:16.01 | Gnarfoz | guess you're right, even though I find that very odd |
| 18:17.06 | Gnarfoz | almost looks like a raid0+1, for some reason |
| 18:17.54 | Gnarfoz | I'm at a loss how to definitely interpret what I'm seeing :D |
| 18:18.32 | Gnarfoz | (I've been working way too many hours for the last couple of weeks as well, though, I'm not in the best of shapes) |
| 18:19.56 | Gnarfoz | I mean, if it was a recoverable situation, with a raid10, it would have just booted |
| 18:21.28 | Gnarfoz | also, if, contrary to all we've just said, it's actually a working situation, then you could force it online and it would work. if not, it's dead regardless of if you try or not. |
| 18:22.41 | znf | ctrl+f says not enough segments |
| 18:23.12 | Gnarfoz | technically correct |
| 18:26.52 | Gnarfoz | apart from putting the drives in a freezer for a bit, or other esoteric remedies, you seem to be out of options (except for restoring from backup) |
| 18:27.11 | znf | yeah, they seem pretty dead https://hastebin.com/anajacanug.vbs |
| 18:27.55 | Gnarfoz | depending on what they're attached to, they might need -d sat or something |
| 18:28.05 | Gnarfoz | but yeah, doesn't look great |
| 18:28.36 | Gnarfoz | also, how does it list it as 137 GB? |
| 18:30.33 | znf | dunno |
| 18:30.39 | znf | it's probably the board of the drives |
| 18:59.27 | znf | client says they're in the process of restoring their data on another server |
| 18:59.28 | znf | which is good |
| 19:48.45 | Andols2 | I wish you all the luck to resolve this :/ |
| 20:30.59 | znf | Gnarfoz, nevcairiel - does hetzner offer multiple IP addresses on any of their server packages? |
| 20:31.14 | Gnarfoz | included in the base package? no |
| 20:31.38 | Gnarfoz | but you can add as many as you like, but ipv4 addresses are not free |
| 20:33.09 | Gnarfoz | https://wiki.hetzner.de/index.php/IP-Adressen/en https://wiki.hetzner.de/index.php/Zusaetzliche_IP-Adressen/en |
| 20:34.00 | Gnarfoz | (not sure how up to date either of those are, but they used to include 4 IPv4 adresses with my server, for example; but they've long been charging 1⬠per extra address, even for people who used to have them included) |
| 20:40.37 | znf | it's fine for that |
| 20:40.45 | znf | a lot of places don't offer extra at all |
| 20:40.47 | znf | see DigitalOcean |
| 20:41.47 | znf | I just need something with ~16GB RAM and 8 IPs |
| 20:44.38 | znf | meh, SoYouStart looks better than hetzner tough :-/ |
| 20:46.38 | Gnarfoz | if you want ovh, get ovh ^^ |
| 20:47.35 | *** join/#wowace Gethe (~Gethe@cpe-70-112-184-154.austin.res.rr.com) |
| 20:47.51 | Gnarfoz | (what for (4? :P) though?) |
| 20:48.07 | Gnarfoz | my current vps hoster also offers extra ipv4, but they're 2⬠each (pfff yeah the shortage is real, but still) |
| 20:48.21 | Gnarfoz | abandoned package is abandoned https://tracker.debian.org/pkg/phpmyadmin |
| 20:57.00 | *** join/#wowace Catal1na (~blu3bot@cloud.linge-ma.ws) |
| 21:02.16 | znf | yes I do need IPv4 |
| 21:02.18 | znf | fuck IPv6 |
| 21:04.25 | Gnarfoz | that wasn't my question, just a homophone joke |
| 21:07.06 | znf | sorry, brain stress |
| 21:07.14 | znf | now here's a question |
| 21:07.22 | znf | why the fuck won't this stupid shit TFTP boot? |
| 21:28.46 | *** join/#wowace Sintacks (~Syntax@75-109-22-174.swbycmtk01.res.dyn.suddenlink.net) |
| 21:40.47 | quiescens | moo |
| 21:48.44 | znf | to answer the above: apparently theres a missconfigured switch |
| 22:03.22 | *** join/#wowace Marla (~textual@72-36-51-35.actaccess.net) |
| 22:20.41 | znf | Gnarfoz, did I tell you about the justice boner from today? |
| 22:20.44 | znf | sticking it up to clients? |
| 22:25.58 | Gnarfoz | I press page up 2x and I'm at wednesday, so... I'd say no :P |
| 22:28.45 | znf | Gnarfoz, I mentioned it yesterday, client complained that 2 weeks ago we had 2 hours downtime (our fiber was cut) |
| 22:29.19 | znf | Asks for discount |
| 22:29.34 | znf | (so that would be 2 hours out of the 2028 - 3 months) |
| 22:29.41 | znf | which would have been something like 0.30 Lei discount |
| 22:29.45 | znf | >xau 0.30 ron to eur |
| 22:29.46 | Catal1na | 0.30 RON == 0.06 EUR |
| 22:30.20 | znf | Then colleague remembers the client always asks for extensions on his payments, always asks to pay later (1, 2 weeks) |
| 22:31.28 | znf | So he replies back: "Sure, we'll make you a discount for the 2 hours the services was unavailable. We would also like to notify you that per contract, we will add late fees to your invoice for the extension you requested" |
| 22:56.34 | *** join/#wowace harl (harl@unaffiliated/harl) |
| 23:19.05 | *** join/#wowace harl (harl@unaffiliated/harl) |
