| 00:30.13 | *** join/#utah braxtron (~braxtron@166.70.128.149) |
| 00:42.56 | *** join/#utah Jantz (~IceChat9@2407:7000:8d04:100:156f:dbf6:fd8a:b28d) |
| 01:29.59 | *** join/#utah thin (~vvvvv@unaffiliated/thin) |
| 01:59.49 | klys | ~slashdot |
| 07:20.15 | *** join/#utah Jantz (~IceChat9@2407:7000:8d04:100:156f:dbf6:fd8a:b28d) |
| 07:38.48 | *** join/#utah Jantz (~IceChat9@2407:7000:8d04:100:156f:dbf6:fd8a:b28d) |
| 08:07.13 | *** join/#utah emcnabb (emcnabb@nat/redhat/x-tuuefossnytpquku) |
| 08:29.57 | *** join/#utah Jantz (~IceChat9@118-93-140-161.dsl.dyn.ihug.co.nz) |
| 11:06.46 | *** join/#utah Jantz (~IceChat9@2407:7000:8d04:100:b540:74a0:bd76:5c7b) |
| 13:40.07 | *** join/#utah tiwula (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 13:45.17 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 13:49.38 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 14:40.26 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 14:44.03 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 14:46.10 | *** join/#utah josephscott (~josephsco@74.114.153.251) |
| 15:07.31 | *** join/#utah Jantz (~IceChat9@2407:7000:8d04:100:1584:1277:f951:b304) |
| 15:15.05 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 15:20.03 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 15:45.20 | *** join/#utah josephscott (~josephsco@74.114.153.251) |
| 16:32.01 | TodPunk | @jfindlay I have a couple people from verisys that describe it as basically a blasted hellscape of swirling doom |
| 16:33.40 | TodPunk | A recruiter friend won't place people there anymore (hasn't for a couple years) and has the position of "I don't care if you snub my job for this as well, but you definitely shouldn't accept the offer from verisys" |
| 16:47.30 | *** join/#utah emcnabb (emcnabb@nat/redhat/x-dhrfyyhoijvaxstn) |
| 16:52.30 | jlp_znc | I was there for a year... it was okay for the first 6-8 months, but then they fired Joe Skeehan and brought in a new VP of Engineering. He took the project we had been working on for nine months and shelved it... "we're just not going to deploy that. I know you have it pretty much finished, but it's not written in Ruby and we think Ruby is the future." (it was Python, as was all of the other code at |
| 16:52.36 | jlp_znc | Verisys that was actually in production.) |
| 16:52.56 | jlp_znc | the "next gen" team had been working on writing all new stuff in Ruby... for three years. |
| 16:54.04 | jlp_znc | then they fired the software team manager, two of the other main programmers (who knew Python) quit (one of them didn't even have a job to go to... he just quit and moved to Phoenix). |
| 16:54.54 | jlp_znc | I bailed shortly after that. then one of the two junior guys on the python team left and the other moved to the next gen team and started writing ruby code. |
| 17:03.20 | *** join/#utah braxtron (~braxtron@216.160.248.178) |
| 17:15.20 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 17:19.51 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 17:28.04 | TodPunk | One of the guys there is supposed to send me his resume |
| 17:29.38 | TodPunk | Daniel something |
| 17:30.16 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 17:35.15 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 17:37.48 | jlp_znc | Daniel Hanson? |
| 17:38.17 | jlp_znc | oh, I guess he was at verisys before me |
| 17:39.43 | jlp_znc | maybe Dan Actor? he started at verisys after I left. |
| 17:43.40 | *** join/#utah keldwud (~keldwud@199.192.164.75) |
| 17:43.40 | *** join/#utah keldwud (~keldwud@unaffiliated/keldwud) |
| 17:43.44 | *** join/#utah Jantz (~IceChat9@2407:7000:8d04:100:1584:1277:f951:b304) |
| 17:45.18 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 17:50.05 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 17:52.09 | jfindlay | I interviewed there twice around the time that jlp_znc got a job there |
| 17:52.48 | jlp_znc | yeah, it was pretty fun when I started there, and for about six months we were doing some cool stuff with python/django |
| 17:53.25 | jfindlay | I'm just glad they never made me an offer because I got hired at saltstack about 3-4 months later |
| 17:53.27 | jlp_znc | there were five of us on the python team... essentially maintaining everythign that was making verisys any money. probably 20 ruby people developing "new" stuff. |
| 17:53.46 | jlp_znc | s/thign/thing/ |
| 17:56.11 | jlp_znc | I did meet/work with some really good people there. |
| 17:58.43 | jfindlay | I worked with some really good people at my job before that, but 'had' to quit because it went downhill really fast (non-amateur tip: don't work for a company run by salespeople) |
| 18:00.23 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 18:04.53 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 18:05.53 | JPaul | "we think Ruby is the future" |
| 18:06.10 | JPaul | anytime someone tells me "X is the future" I can't help but roll my eyes |
| 18:06.38 | JPaul | someone tried to tell me Node is the future and is going to replace all those other languages including PHP... |
| 18:08.11 | JPaul | from a sysops perspective, if you can't be bothered to configure your underlying system to run as a daemon in and of itself, just don't even bother me with it. |
| 18:09.49 | jlp_znc | ~systemd JPaul |
| 18:09.49 | infobot | ACTION renames JPaul's network interfaces |
| 18:10.48 | jfindlay | I personally think python is a better designed language, but technology preferences are appropriate only in limited circumstances |
| 18:11.29 | jlp_znc | I like python, but there are some weird things in it that I don't like (I'm looking at you, regular expression library) |
| 18:11.35 | JPaul | any language can be great if you have someone to support the projects using it |
| 18:12.13 | JPaul | at one point here they had someone that was a ruby fanboi and wrote some utilities in it. |
| 18:12.30 | JPaul | unfortunately he hasn't worked here for years now, and no one since really know Ruby... |
| 18:13.37 | JPaul | Same issue with a custom PHP framework a dev here made and used on all his projects. Now he's not here and the framework is too costly to maintain so we're just going to replace it with each re-write. |
| 18:14.28 | jlp_znc | yeah, that's a problem. I've seen that happen with my favorite write-only language, perl, way too often. |
| 18:14.34 | JPaul | dangers of letting individuals do something fancy or cool that isn't widely used/supported outside the company. |
| 18:14.56 | jlp_znc | (not that you can't write good, portable, readable perl... it's just that it's too easy not to) |
| 18:15.57 | JPaul | node is a thorn in my side since there isn't a standardized way to run it as a daemon. I've seen pmi and systemd stuff, but it's a pain to maintain. |
| 18:16.19 | JPaul | wish they'd just make something that works and have it as part of the packages for it. |
| 18:16.32 | JPaul | I have enough custom packages to deal with as it is. |
| 18:17.24 | JPaul | fortunately there are only two projects here using it right now |
| 18:19.21 | jlp_znc | node is terrible for that. we had a server in node (at medici) that read a bunch of historical data. in order to do that, it spawned a child process for each day's data (something about the library that was used to thread that out). |
| 18:19.58 | jlp_znc | each process only ran for a little while, but it used three FDs (stdout, stdin, and stderr) and it was never waited for. |
| 18:20.11 | JPaul | I view node as something written by developers for developers because it was cool. So no thought given to running it in a production environment |
| 18:20.34 | jlp_znc | so everything worked fine for a few months, then all of a sudden it started crashing because it was hitting some max FD limit or max proc limit or something. |
| 18:20.42 | JPaul | ha, nice |
| 18:21.26 | jlp_znc | of course, no one had any cycles free to fix the code, so we just bumped up the limits on the system... bought us another few months. then it happened again (we were up to, like 4096 FDs at this point) |
| 18:22.39 | jlp_znc | finally they deciced fixing the code was a priority. they didn't actually fix it, but just made it cache the information it was collecting in a sqlite db. so it would start up, go dig up all the historical data and stuff it in sqlite, and then eventually crash. next time it would start, it would pick up where it had left off and continue getting historical data and caching it in sqlite. |
| 18:23.01 | jlp_znc | by the time I left, I think it had to restart four times to fully populate the sqlite db from scratch. |
| 18:23.16 | jlp_znc | it's probably up to eight or ten by now. |
| 18:25.14 | JPaul | that's awesome |
| 18:40.20 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 18:40.45 | jfindlay | JPaul: that's because node represents the courageous, new way of writing software, free from arcane specifications, and constricting design principles: it moves quickly and if things get broken they'll just have to learn how to fly too |
| 18:42.18 | jfindlay | conforming to established engineering patterns is for greybeards who probably don't even know what avocado toast is |
| 18:42.47 | jlp_znc | he he |
| 18:43.54 | *** mode/#utah [+v jlp_znc] by ChanServ |
| 18:46.36 | JPaul | jfindlay, so if it breaks it's up to the end user to fix it? That's a much easier way to program for sure, who needs to be bothered with things like stability. :p |
| 18:49.43 | jfindlay | JPaul: but docker lets you sidestep that harsh reality altogether: it allows a pipeline directly from the developer to the user. Whatever the developer deigns to dump into the container gets delivered to the user's front door, regardless of how many grumpy admins the package has to be handled by |
| 18:50.25 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 18:50.27 | JPaul | ah, docker |
| 18:50.45 | JPaul | yeah, haven't been sold on that being OK for prod yet |
| 18:52.15 | JPaul | i get the idea behind it, but not convinced it's really a good one. |
| 18:53.13 | jfindlay | instead of one or two monoliths, the contents of which are a profound mystery, you have several hundred or thousand pieces of the puzzle |
| 18:57.27 | JPaul | It's the part of relying on the developer to maintain not just good coding practices but other security matters that really trouble me with docker. |
| 18:57.45 | *** join/#utah braxtron (~braxtron@216.160.248.178) |
| 18:57.52 | JPaul | I have as of yet to meet more than a very small percentage of developers that are cognizant of those requirements |
| 18:58.46 | JPaul | I also don't work with large projects either, so there's that too. |
| 18:59.41 | jfindlay | if you really tune the contents of the container it seems to work well enough |
| 19:00.51 | jfindlay | what I don't like is when developers think that because docker provides a git-like audit they can edit the container with a stack of mystery one-offs |
| 19:02.58 | JPaul | I think docker is great for localized development. If i had time I could create docker images that represent the production systems that the devs can use to work from. |
| 19:03.31 | JPaul | then push the code to staging via our bitbucket, test, then move up to prod. |
| 19:04.55 | JPaul | with prod though I have issue when you get people treating containers as disposable, if they misbehave destroy them and spin up a new one automatically |
| 19:05.18 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 19:05.29 | JPaul | don't bother trying to figure out what broke, just get it running again and sweep it under the rug |
| 19:06.12 | JPaul | one guy we had here last year wanted to setup something similar where once something started alerting you just kill the process and restart it automagically |
| 19:10.22 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 19:12.02 | jfindlay | could be problematic once the healthy lifetime drops below a certain threshold :-) |
| 19:12.23 | jfindlay | all you're doing is powercycling containers |
| 19:13.00 | jfindlay | https://www.xkcd.com/986/ |
| 19:16.01 | JPaul | love xkcd |
| 19:18.06 | BigBen212 | xkcd is and Bobby Tables is always the example I use when I talk to somebody who doesn't know what SQL injection is |
| 20:19.49 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 20:29.14 | *** join/#utah keldwud (~keldwud@unaffiliated/keldwud) |
| 20:51.11 | *** join/#utah josephscott (~josephsco@74.114.153.251) |
| 20:57.45 | *** join/#utah emcnabb (~emcnabb@62.209.224.147) |
| 21:10.18 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 21:14.56 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 21:30.24 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 21:33.07 | *** join/#utah lane__ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 22:07.41 | romanovic | today is one of those days. Dev A and Dev B have the same hardware, os, IDE, tool chains, project versions, etc. Except the project won't build for Dev A, but will build for Dev B (me) using the same commands |
| 22:08.20 | romanovic | on the same commit |
| 22:17.07 | jlp_znc | have the user run "env" and diff the output with your own. |
| 22:22.19 | keldwud | ok this is seriously frustrating me. I have $agent which is reaching out over udp 1514 to $server which is a guest vm on $host. confirmed $server is listening on udp 1514. confirmed no firewall on $server. there *is* firewall on $host, confirmed port is open on $host. traffic still isn't getting through. wtf |
| 22:22.52 | keldwud | do I need to tcpdump to figure out what is going on or can I assume it is the virtualization manager that is blocking hte traffic somewhere through some SDN that I haven't found yet? |
| 22:23.00 | keldwud | I'm using ovirt open virtualization manager |
| 22:23.28 | keldwud | all the documentation shows that the firewall is managed by the hosts and not the manager |
| 22:24.13 | jlp_znc | tcpdump is your friend in that kind of situation |
| 22:24.43 | jlp_znc | can the machine $agent is on reach $server via some other protocol, such as ssh? |
| 22:24.59 | keldwud | yeah I was hoping tcpdump wasn't the answer |
| 22:25.12 | keldwud | jlp_znc yeah, the agent was able to register |
| 22:25.19 | *** join/#utah lane_ (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 22:25.22 | keldwud | and they did that over 514 |
| 22:26.55 | jlp_znc | fujitsu-dtcns? (port 1514) |
| 22:27.15 | jlp_znc | or some goofy service that is using that port even though it's not registered? |
| 22:28.10 | jlp_znc | what does netstat -anp | grep 1514 show on $server? |
| 22:30.35 | keldwud | jlp_znc ossec |
| 22:30.49 | keldwud | and I ahve confirmed trhat it is listening on $server |
| 22:32.26 | jlp_znc | so, "tcpdump udp port 1514 and host $agent" on $server and "tcpdump udp port 1514 and host $server" on $agent |
| 22:32.40 | jlp_znc | then watch and see what happens |
| 22:33.22 | keldwud | yeah, I've confirmed with co-worker that I'm gonna have to get down and dirty :) |
| 22:33.33 | jlp_znc | good luck |
| 22:33.37 | jlp_znc | you'll need it |
| 22:35.51 | jlp_znc | if you don't see packets on both sides, I'd look at the ovirt virtual network switch configuration |
| 22:36.56 | jlp_znc | I saw something once where they did something like "brctl ovirtmgmt setageing 0" to turn the virutal switch into essentially a hub and suddenly their thing started working. obviously, not something you would want to do, but might provide a data point. |
| 22:37.19 | jlp_znc | (I assume ovirtmgmt is the name of the virtual network) |
| 22:43.46 | keldwud | jlp_znc that's where I keep getting lost is I can't find any ovirt virtual network switch |
| 22:43.56 | jlp_znc | hmmm |
| 22:43.57 | keldwud | at least not from the management console (web ui) |
| 22:44.05 | keldwud | I haven't checked from the CLI yet |
| 22:44.08 | jlp_znc | check the cli |
| 22:44.15 | keldwud | good call |
| 22:44.25 | jlp_znc | could be some weird rules on that interface |
| 22:51.24 | *** join/#utah tiwula (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 23:44.07 | *** join/#utah Simdol (~Simdol@146.86.225.209) |