| 01:06.54 | levi | TodPunk: I think the 'how do I do this thing?' questioning without learning is because there's way too freakin' much to learn to figure out how most systems work, because they're all 20 poorly-documented layers deep. :P |
| 01:08.35 | levi | Bumbling around and cargo-culting gets you results *now*, while you could spend weeks just trying to figure out the right things to read to figure out how it all works. |
| 05:03.12 | *** join/#utah tiwula (~lane@174-23-49-160.slkc.qwest.net) |
| 06:47.15 | jfindlay | cargo-culting :-) |
| 06:47.48 | jfindlay | one person's shipit is another person's cargo-culting |
| 06:48.31 | jfindlay | another person's old school is one person's insufferable arcana |
| 06:51.23 | jfindlay | I had a colleague once who reminisced about literal bit flipping on the front of a computer with mechanical switches during its boot process |
| 07:55.17 | jlp_znc | ha, the class one year ahead of me in college had to do that. they had these old Honeywell Level 6 machines and they had to toggle in the boot program on the front panel. Ahh, the bad old days. |
| 07:56.17 | jlp_znc | they got rid of the card punch machines the summer before I started, so I never got that "fun", either. |
| 07:57.08 | jlp_znc | (they kept the card readers for another couple of years so people could convert their old card decks into 9-track tape... I guess they never though about bit rot and tape randomization... turns out those cards were a better long term storage format.) |
| 07:57.56 | jlp_znc | I think I still have an old 9-track tape kicking around somewhere, but I'm sure all the bits have fallen off of it. probably have some QIC-2 or QIC-120 cartridges, too. |
| 07:58.48 | jlp_znc | and I know I have a stack of DAT tapes downstairs. I used to do my backups on them (via amanda... just had to remember to shove a new tape in whenever I got the e-mail that the current tape was full). |
| 07:59.20 | jlp_znc | I don't even run a server at home any more... and all my other crap backs up to the network now. |
| 09:23.27 | *** join/#utah fadein (~Erik@c-24-10-167-68.hsd1.ut.comcast.net) |
| 13:37.03 | *** join/#utah aeryndunham (~aeryndunh@ip-64-134-190-30.public.wayport.net) |
| 15:20.44 | *** join/#utah BitBandit (~BitBandit@50-198-187-221-static.hfc.comcastbusiness.net) |
| 15:37.23 | *** join/#utah tiwula (~lane@70.89.246.241-Busname-UT.hfc.comcastbusiness.net) |
| 16:20.16 | *** join/#utah aarobc (uid151482@gateway/web/irccloud.com/x-xlztsdzpzobkosdk) |
| 16:36.17 | spangborn | What's a punch card? ;p |
| 16:45.53 | beelzebob | those things at sandwich places where you get a free sub after buying several subs |
| 17:31.24 | *** join/#utah keldwud (40479a3b@gateway/web/freenode/ip.64.71.154.59) |
| 17:39.49 | keldwud | well that's odd. In AWS when I try to launch an instance from an AMI I created from a snapshot (from an m4.large instance), I cannot select m4.large as an instance type (I can't select dozens of different instance types as they are grayed out). When I launch a new instance using a stock AMI, I can select any instance type I want. So it doesn't appear to be related to https://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_run_in_Amazo |
| 17:40.26 | keldwud | anybody run into this problem before? either I'm googling wrong or it's not that common |
| 17:40.39 | keldwud | or I'm launching instances incorrectly |
| 17:40.56 | keldwud | or creating AMIs incorrectly, but I followed the AWS documentation |
| 17:47.11 | keldwud | has anyone else ever run into the AWS instance limits? if so, what did it look like? were there any notifications or errors of any sort? |
| 17:48.07 | *** join/#utah josephscott (~josephsco@2601:681:8001:c340:fcba:d866:c0c2:7f2a) |
| 17:54.35 | *** join/#utah aeryndunham (~aeryndunh@ip-64-134-190-30.public.wayport.net) |
| 18:06.19 | keldwud | ok I figured out how to check my limits in AWS and it looks like alerts would be sent out. it's all in the Trusted Advisor section of AWS |
| 18:06.34 | keldwud | that lets you see your resource limits compared to your usage |
| 18:07.16 | keldwud | now I just need to figure out why I can't launch most instance types from my newly created AMI |
| 18:22.10 | programmerq | keldwud⸠do you get a more meaningful message if you supply that ami and the instance type you want on the aws cli or other api client? |
| 18:29.17 | keldwud | programmerq: I really should start moving my work to AWS CLI |
| 18:29.31 | keldwud | I've been meaning to :) |
| 18:30.13 | keldwud | but that's another learning curve (even if short) and I wanted to get some stuff done. didn't have the spare cycles to learn the CLI until I get some university courses off my plate. |
| 18:30.54 | programmerq | keldwud⸠it's really not so bad and will help your current situation. |
| 18:31.13 | programmerq | a pain? yes. hard or time consuming? not really |
| 18:31.28 | keldwud | oh I agree it will help me greatly |
| 18:31.58 | programmerq | aws ec2 run-instances help |
| 18:32.15 | programmerq | then just specify your ids for all the things you need-- region, ami, vpc, etc. |
| 18:33.08 | keldwud | on the docker side of things, we ran into this yesterday https://github.com/docker/libnetwork/issues/1765 (we didn't report, it was already known) |
| 18:36.15 | keldwud | what do you call a pirate's chest full of fruit? |
| 18:36.50 | programmerq | his fruity |
| 18:36.55 | keldwud | Berried treasure. |
| 18:40.01 | keldwud | https://imgfave.azureedge.net/image_cache/1316775923926757.gif |
| 18:40.57 | programmerq | so I've never done this before, and it's fun. I set up an iscsi target on my freenas box, and I installed ubuntu in a virtualbox vm (with no local disks). I'm 95% of the way to getting it to be 100% network boot. |
| 18:41.26 | aeryndunham | is drowning in logfiles thousands of lines long and doesn't even know what she's looking for :( |
| 18:41.28 | programmerq | need to get my hands on an ipxe.lkrn that doesn't have an embedded script in it. every time I try to compile it myself, it fails. different error depending on the gcc version I choose. :/ |
| 18:41.40 | programmerq | aeryndunham⸠splunk to the rescue! |
| 18:42.08 | programmerq | I have a local free tier splunk just for those sorts of occasions. I'll toss whatever logs I'm dealing with for the day into that and then search through it |
| 18:42.44 | programmerq | I end up with these big long search queries: * AND NOT "some specific error type" AND NOT "some other specific error string" ... |
| 18:42.56 | programmerq | but it helps reveal the lines I might care about by removing stuff I know I don't |
| 18:42.58 | aeryndunham | I'm pretty sure we have ELK running here somewhere? but then they just throw this stuff at me |
| 18:43.11 | aeryndunham | and I don't even know what errors I'm looking for |
| 18:43.33 | programmerq | of course, a similar approach would be to use grep -v, but splunk has some other things that make it nice ( and I already know splunk, in theory) |
| 18:44.00 | spangborn | Hey it could be worse |
| 18:44.05 | spangborn | We had downtime this morning |
| 18:44.09 | programmerq | splunk's graphing capability is helpful-- you can break things down by type and get a bar graph over time. |
| 18:44.16 | spangborn | one server was throwing 1M errors every 15min |
| 18:44.25 | aeryndunham | ew |
| 18:44.59 | programmerq | The dentist hit a blood vessel when I got numbed this morning. |
| 18:45.05 | programmerq | that was a bit bizarre |
| 18:45.09 | aeryndunham | owwwww |
| 18:45.29 | programmerq | the numbness exploded out from where it hit. faded fairly quickly |
| 18:45.50 | programmerq | but I'm a little extra numb in my nose and eye, but not too significantly different from my normal dentist numbing experience. |
| 18:57.38 | aeryndunham | Ok, so, I've got the numbers I need, but it's a number at the end of a long line of other text/numbers, and I need to get just these numbers so I can work with them. Hmm. |
| 19:02.51 | aeryndunham | ok so |
| 19:02.53 | aeryndunham | https://imgur.com/I0J1jnw |
| 19:03.12 | aeryndunham | I just need the numbers at the end of those lines (the negative value) |
| 19:03.23 | aeryndunham | is this a thing I can extract somehow |
| 19:03.35 | jlp_znc | sed and an appropriate regular expression would probably do the trick |
| 19:03.48 | aeryndunham | sed |
| 19:03.50 | aeryndunham | hmm ok |
| 19:03.51 | jlp_znc | you just need the last field? awk '{print $NF}' |
| 19:04.15 | aeryndunham | Yeah, so I can sort it and then average it |
| 19:04.47 | jlp_znc | you just need the last field? awk '{print $NF}' | sort -n (but you don't need to sort it to average it) |
| 19:05.04 | aeryndunham | Oh, I know, but I want both the highest/lowest and the average |
| 19:05.32 | jlp_znc | awk '{lct++; sum += $NF} END {print $sum/$lct}' should give you the average |
| 19:06.00 | aeryndunham | where do I even tell it what file I'm looking it? dang I'm confused (and should probably just look up a tutorial) |
| 19:06.03 | jlp_znc | and awk '{print $NF}' | sort -n | sed -n -e '1p' -e '$p' ought to give you the lowest and highest |
| 19:06.18 | jlp_znc | just pass it the file as the next arg after the quoted string |
| 19:06.25 | aeryndunham | ah right |
| 19:06.29 | jlp_znc | awk '{do whatever}' my_log_file |
| 19:06.40 | aeryndunham | dang I need to learn sed/awk |
| 19:07.07 | jlp_znc | generally, if I need more than ten commands in a pipeline I'll break out perl to do the heavy lifting... it can do it all for you in one command. |
| 19:07.32 | aeryndunham | but holy crap how does $NF tell it I need that last number? I'm amazed haha |
| 19:07.50 | jlp_znc | NF is the number of fields. $NF is the value in the NF'th field |
| 19:08.27 | jlp_znc | so if NF is, say, 9, then $NF is $9, which is field number nine |
| 19:08.34 | jlp_znc | awk rocks |
| 19:08.44 | aeryndunham | yeah I definitely need to learn this |
| 19:09.24 | jlp_znc | https://www.amazon.com/gp/product/013937681X?ie=UTF8&tag=petersotechno-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=013937681X |
| 19:10.02 | jlp_znc | I read that about 33 years ago... very handy book, although kind of dated now. |
| 19:10.05 | Migs | that reminds me, NF's coming to SLC in March and I need to buy concert tickets |
| 19:10.19 | jlp_znc | that's why I write shell scripts for sh, not for bash. :-) |
| 19:10.57 | jlp_znc | note, if you buy it from that link, I get a (small) kickback from amazon |
| 19:12.49 | aeryndunham | I think I actually have a sed/awk book somewhere |
| 19:14.15 | aeryndunham | I keep getting an error on the average one and I can't tell if I'm typing it wrong |
| 19:16.34 | aeryndunham | highest/lowest worked though |
| 19:18.31 | jlp_znc | range 1 10 | sort -n | sed -n -e '1p' -e '$p' seems to work |
| 19:20.18 | jlp_znc | (range can be found here: https://github.com/jlp78/dotfiles/blob/master/_bash_funs around line 58) |
| 19:21.41 | beelzebob | how is range different than seq? |
| 19:21.55 | jlp_znc | range probably predates seq by a good 15 years :-) |
| 19:23.13 | jlp_znc | looks like I should add seq support to range (it already support jot, dc, and perl) |
| 19:31.57 | aeryndunham | jlp_znc: the line you gave me for average it keeps telling me that I am tying to access an out of range field? |
| 19:32.48 | aeryndunham | awk '{lct++; sum += $NF} END {print $sum/$lct}' l204.txt |
| 19:32.51 | aeryndunham | is what I typed |
| 19:33.10 | jlp_znc | hmmm |
| 19:33.49 | jlp_znc | oh, duh, print sum/lct |
| 19:33.54 | jlp_znc | not print $sum/$lct |
| 19:33.59 | aeryndunham | ok |
| 19:34.11 | jlp_znc | sorry, awk doesn't want the $... my perl brain was getting in the way. |
| 19:34.24 | aeryndunham | Thanks!! |
| 19:34.28 | jlp_znc | np |
| 19:34.38 | aeryndunham | what is lct? |
| 19:35.11 | jlp_znc | lct is the count of the lines... earlier there is lct++, which increments the counter for each line that is read |
| 19:35.23 | aeryndunham | oh of course |
| 19:35.39 | jlp_znc | so, for each line, lct++ happens first, then sum += $NF accumulates the values |
| 19:35.44 | aeryndunham | I don't suppose there's a way I could tell it to round that number :p |
| 19:35.50 | jlp_znc | there probably is |
| 19:35.53 | jlp_znc | :-) |
| 19:36.01 | aeryndunham | looks into it |
| 19:36.17 | jlp_znc | try printf "%d\n", sum/lct |
| 19:36.44 | jlp_znc | or print int(sum/lct) |
| 19:37.03 | aeryndunham | oh wow you're amazing |
| 19:37.06 | jlp_znc | (or, if you really want to round it, print int(sum/lct + 0.5) ) |
| 19:37.27 | jlp_znc | since int and %d will just do floor (i.e. 5.6 will become 5, not 6) |
| 19:37.41 | aeryndunham | floor is more what I want, actually, I think |
| 19:43.51 | jlp_znc | keep in mind that floor(-6.5) is -7 while int(-6.5) will give you -6 |
| 19:44.03 | jlp_znc | negative numbers are weird |
| 19:44.10 | aeryndunham | hmm |
| 19:56.01 | aeryndunham | i'm going to try writing a script |
| 19:56.03 | aeryndunham | wish me luck |
| 19:56.15 | jlp_znc | good luck. |
| 19:56.27 | aeryndunham | thinks she can do it |
| 19:56.48 | jlp_znc | https://www.youtube.com/watch?v=_3jc_3p0YZY |
| 19:57.17 | aeryndunham | can't watch videos =( |
| 19:57.28 | jlp_znc | :-( |
| 19:57.36 | aeryndunham | hmm actually this might be hard |
| 19:58.23 | aeryndunham | only because I just manually got this info for 2 doors, there are hundreds, so I need this to work with... any filename? I should probably stop telling people 'sure I can do that' :p |
| 19:58.43 | jlp_znc | cat * | awk blah blah |
| 19:59.24 | aeryndunham | oh nevermind apparently the info I extracted is useless |
| 19:59.26 | aeryndunham | great |
| 19:59.36 | jlp_znc | oh, well, that's nice. |
| 19:59.42 | aeryndunham | yep |
| 19:59.50 | aeryndunham | thanks for your help though, was very educationa; |
| 19:59.58 | jlp_znc | happy to oblige |
| 20:00.35 | aeryndunham | For context: |
| 20:01.22 | aeryndunham | Basically we have all these door panels onboard now, they use bluetooth to sense guests' medallions and then (theoretically) unlock - these epically long logfiles do list every time access is granted or denied and the rssi values of medallions the panel is picking up |
| 20:02.01 | aeryndunham | so I was asked to find how many times access was granted and denied, and the average rssi value that the panel is sensing (we think some have bad antenna) |
| 20:02.51 | aeryndunham | So I was ~hoping~ to write a little script that'll give you this info for any log you specify |
| 20:03.07 | jlp_znc | that would have been very useful, I would think. |
| 20:03.14 | aeryndunham | One would think |
| 20:03.20 | jlp_znc | but no? |
| 20:03.28 | aeryndunham | but I'm looking at 2 doors right now, one that works well and one that doesn't |
| 20:03.40 | aeryndunham | the average on the good one is -80 and on the bad one -79 so apparently this is not helpful |
| 20:04.14 | aeryndunham | I might see if I can write the thing anyway, for my own bash scripting education |
| 20:04.20 | jlp_znc | hmmm, must be some other factor than signal strength that is causing the problem |
| 20:04.23 | aeryndunham | yeah |
| 20:04.46 | jlp_znc | well, good luck. |
| 20:04.49 | aeryndunham | hah thanks |
| 20:04.58 | aeryndunham | I'd like to write something neat just to give to mr "why aren't you doing this in excel" |
| 20:05.08 | jlp_znc | heh heh |
| 20:05.15 | jlp_znc | do it. |
| 20:05.42 | jlp_znc | sure, you can do it in excel. for one door. |
| 20:05.47 | aeryndunham | Exactly |
| 20:05.49 | jlp_znc | now try to do it for 1000 doors |
| 20:06.08 | aeryndunham | we only have 108 right now but yeah |
| 20:06.10 | aeryndunham | exactly |
| 20:15.26 | aeryndunham | oh my god it worked |
| 20:15.29 | aeryndunham | it's so simple |
| 20:15.31 | aeryndunham | but it worked |
| 20:18.04 | aeryndunham | please behold my first ever bash script that is still not optimal for what I need but it ran and gave me the answer: https://pastebin.com/yty2ZyqD |
| 20:41.25 | spangborn | Doin' some iBeacon strength testing? |
| 20:41.29 | beelzebob | aeryndunham: feels good, doesn't it? |
| 20:42.13 | aeryndunham | spangborn: Trying to see if some doors are reading the levels less than others, yeah |
| 20:42.27 | spangborn | Sounds fun |
| 20:42.34 | spangborn | I have an app idea for beacons |
| 20:42.43 | spangborn | but I have not the energy or time to make it |
| 20:42.53 | aeryndunham | hah |
| 20:43.08 | aeryndunham | but yeah it's super exciting for something to work, as stupid as this is |
| 20:43.28 | spangborn | If it works, it's great |
| 20:43.41 | spangborn | Especially if it's something you'll just use a couple times and never again |
| 20:45.48 | aeryndunham | yeah |
| 20:47.09 | aeryndunham | bash scripting is definitley somerhing I want to learn more of |
| 20:51.31 | beelzebob | aeryndunham: you could leave out the temp file and have it accept the filename as a parameter like: https://pastebin.com/BCFGicpT |
| 20:51.49 | aeryndunham | Oh, cool, I wasn't sure how to deal with that |
| 20:52.23 | aeryndunham | and what does your if statement do? |
| 20:53.32 | beelzebob | if there is an argument ($1) use that, otherwise ask |
| 20:53.57 | beelzebob | so you could do: script syslog1204 |
| 20:54.09 | aeryndunham | oooooh amazing |
| 20:54.11 | aeryndunham | I wondered about that |
| 20:54.16 | beelzebob | if you just ran "script" it would ask for a filename |
| 20:54.30 | aeryndunham | very cool |
| 20:54.41 | beelzebob | if it were me I'd just have it accept stdin though, but it depends how it will be used |
| 20:55.02 | aeryndunham | it probably won't be, hah |
| 20:55.16 | beelzebob | that would just be a one liner: cat /dev/stdin | grep -i 'scanner_scan' | awk '{lct++; sum+= $NF} END {printf "%d\n", sum/lct}' |
| 20:55.36 | beelzebob | and used like: cat syslog120* | script |
| 20:56.57 | beelzebob | hmm I guess you could also dump the grep too... |
| 20:57.29 | beelzebob | awk '/scanner_scan/ {lct++; sum+= $NF} END {printf "%d\n", sum/lct}' |
| 20:57.49 | aeryndunham | laughs and feels very inadequate now :p |
| 20:57.53 | aeryndunham | I TRIED :p |
| 20:58.14 | beelzebob | pfft - nothing inadeequate about learning stuff |
| 20:58.32 | fungus | You did great, we just like helping with micro-optimizations. |
| 20:59.17 | beelzebob | when I was a newbie 15 years ago, I had an irc channel where I was always amazed by wizardry |
| 20:59.59 | spangborn | "We just like helping with micro-optimizations" |
| 21:00.03 | spangborn | Story of the internet right there |
| 21:00.11 | spangborn | "Sorry you're wrong, try this instead" |
| 21:00.45 | aeryndunham | haha |
| 21:00.59 | jfindlay | programmerq: is the aws cli stable, are you supposed to script with it? Use it to build libraries? |
| 21:01.01 | beelzebob | yeah, I actually thought more than I should have about making suggestions |
| 21:01.20 | beelzebob | because making something that works is awesome! and I don't want to take away from that |
| 21:01.54 | TodPunk | jfindlay: yes, except when your language has a lib to do stuff (usually through the cli, to be fair), like boto3 for python |
| 21:02.38 | jfindlay | relatedly, docker's python API has been less than stable from version to version :) |
| 21:03.27 | jfindlay | deping parallel code paths on version minor numbers is not fun |
| 21:04.20 | jfindlay | that's for javascript programmers |
| 21:04.30 | aeryndunham | docker here is broken every other minute, dunno what these devs are doing to it |
| 21:05.32 | TodPunk | jfindlay++ |
| 21:05.38 | TodPunk | ~jfindlay++ |
| 21:06.35 | aeryndunham | and lmao that 'fi' closes an 'if' statement |
| 21:06.43 | aeryndunham | has learned so many things today |
| 21:08.59 | beelzebob | if only that was the strangest thing about bash scripting :/ |
| 21:09.06 | aeryndunham | haha |
| 21:15.08 | spangborn | fi |
| 21:15.14 | spangborn | FTFY, beelzebob |
| 21:15.22 | programmerq | jfindlay⸠it's been basically the same as far as I know. it'll help keldwud with debugging his particular issue. |
| 21:15.23 | spangborn | Please close your conditionals |
| 21:15.42 | programmerq | jfindlay⸠they don't bump the major api version as often as they probably ought to, no. |
| 21:17.26 | beelzebob | it is a sad thing that http://mywiki.wooledge.org/BashPitfalls is such a giant list |
| 21:22.48 | spangborn | https://arstechnica.com/tech-policy/2017/11/comcast-deleted-net-neutrality-pledge-the-same-day-fcc-announced-repeal/ |
| 21:22.57 | spangborn | SURPRISE! |
| 21:23.07 | aeryndunham | is anyone surprised?? |
| 21:24.15 | spangborn | Of course not |
| 21:24.23 | spangborn | It's really par for the course for them |
| 21:27.59 | jfindlay | programmerq: when I was at saltstack, we would routinely get bug reports from users every time docker upgraded because the API we were using changed |
| 21:41.10 | programmerq | yup |
| 21:41.12 | programmerq | :/ |
| 21:42.11 | programmerq | fortunately, you can pin to a specific api version and avoid that. if you specify v1.24 explicitly, then it stays v1.24 across versions of the daemon-- any version of the daemon that supports v1.24, anyway |
| 21:42.33 | programmerq | but if you don't specify, you'll get the most recent API version, so you get that feeling of stuff shifting underneath you. |
| 23:33.14 | frostyfrog | "We just like helping with micro-optimizations" Love it! I'll have to put that somewhere. x3 |
| 23:53.27 | *** join/#utah SunSparc (~jonathan@45.56.57.85) |
| 23:53.48 | SunSparc | All you developers ready to get your game on for the Advent of Code? http://adventofcode.com/ |
| 23:57.43 | programmerq | oh yeah! I forgot about that. |
| 23:57.59 | programmerq | I should try to do it in go this year. |
