| 01:34.23 | *** join/##pxe SJRvanSchaik (~SJRvanSch@82.174.230.73) |
| 07:37.57 | Nirkus | ctugt: what have you tried? which howto did you use and what exactly does not work? |
| 07:38.14 | Nirkus | ach damnit :o) |
| 08:27.41 | *** join/##pxe Norky (~dan@milton.ocsl.co.uk) |
| 12:17.30 | *** join/##pxe ctugt (~ctugt@a237011.upc-a.chello.nl) |
| 12:46.32 | *** join/##pxe Zaphoid (~karabats@karabats.com) |
| 12:47.26 | Zaphoid | I"m having an odd issue when trying to setup a diskless workstation where I can only successfully boot to that PXE from a single IP |
| 12:47.43 | *** join/##pxe xxtjaxx (~xxtjaxx@waltroute.ff02.net) |
| 12:47.43 | *** join/##pxe xxtjaxx (~xxtjaxx@kde/developer/marschke) |
| 12:54.47 | Zaphoid | Running Linux Mint 13, booting to a NFS mount. The boot process gets to the point where the video driver appears to be initalized and the last message on the screen is "about "running script/init-bottom ... done" which also appears during a successful boot. |
| 12:55.18 | Zaphoid | Watching netstat on the pxe/nfs server shows several packets in the send-q |
| 12:59.43 | Zaphoid | The same system will work at that one IP address, but not another. |
| 12:59.51 | Zaphoid | Another IP address, that is. |
| 13:22.34 | *** join/##pxe karabats (~karabats@karabats.com) |
| 13:49.35 | Zaphoid | I'm having an odd issue where the PXE installation will only fully boot to one IP address. Any system at that IP address will boot successfully, but the same system at a different IP address will not complete the boot process. |
| 13:59.41 | *** join/##pxe ctugt (~ctugt@a237011.upc-a.chello.nl) |
| 15:14.28 | Norky | is that IP address hardcoded within the boot environment? |
| 15:15.03 | Zaphoid | Not that I know of, but I could be missing where it might be |
| 15:15.34 | Norky | this is probably going to be better asked/answered in #linuxmint (or wherever they have their IRC channel), PXE is only a very small piece, and that piece appears to be working |
| 15:15.48 | Norky | that said, what are the addresses you're using? |
| 15:16.14 | Zaphoid | They told me to answer here. |
| 15:16.28 | Norky | also, in a failing boot, do you see an NFS mount request from the client in the server logs? |
| 15:16.32 | Norky | right, very helpful |
| 15:16.42 | Zaphoid | Yes, the NFS is successfully mounted |
| 15:16.55 | Zaphoid | There's one thing that concerned me... |
| 15:17.15 | Zaphoid | I get this message in the logs whenever one of these comes online: |
| 15:17.16 | Zaphoid | Aug 21 08:49:39 samba in.tftpd[27795]: tftp: client does not accept options |
| 15:17.24 | Zaphoid | Successfully or otherwise |
| 15:17.30 | Norky | all PXE does is allow one to boot form the network. If you get a kernel (and possibly an initrd) from the network and the system begins booting, that's the whole of PXE. |
| 15:17.37 | Norky | that's fairly normal |
| 15:18.04 | Zaphoid | Considering it happened during a success and a failure, I wasn't THAT worried |
| 15:18.11 | Zaphoid | But, I'm grasping at straws at this point |
| 15:18.16 | Norky | the tftp client in the PXE stack on most (all?) NICs is fairly baisc and does not implement many extra tftp features |
| 15:18.44 | Zaphoid | I last touched PXE about 12 years ago, so I've forgotten most of what I knew then :P |
| 15:18.56 | Norky | there's not much too it |
| 15:18.57 | Norky | to* |
| 15:19.11 | Zaphoid | I got everything else working. Memtest, few Live DVDs booting, things like that |
| 15:19.17 | Zaphoid | This diskless workstation is just acting up. |
| 15:19.28 | Norky | yeah, it does seem distro-sepcific |
| 15:20.08 | Zaphoid | I was going to try the same thing with a CentOS 6 install, something I'm much more familiar with. |
| 15:20.13 | Zaphoid | And see if I ran into the same issue |
| 15:20.24 | Zaphoid | Currently this is Linux Mint 13 |
| 15:20.36 | Norky | what are the IP address/subnet you're using? |
| 15:20.49 | Zaphoid | 192.168.1.253 is the PXE/DHCP/TFTP server |
| 15:21.01 | Zaphoid | 192.168.1.0/24 is the subnet |
| 15:21.03 | Zaphoid | So pretty basic |
| 15:21.23 | Zaphoid | The 192.168.1.83 IP address that I can work with is the IP address of the system when I did the initial install onto a physical disk. |
| 15:21.45 | Norky | right, that might be significant |
| 15:21.56 | Zaphoid | Which is why I'm thinking there has to be something to do with the OS itself, vs pxe or something else |
| 15:22.07 | Norky | does that address resolve to a hostname? |
| 15:22.09 | Zaphoid | Though, oddly enough, I can move that disk to another system at another IP and it boots fine |
| 15:22.12 | Zaphoid | dhcp83 |
| 15:22.13 | Zaphoid | Nothing special |
| 15:22.25 | Norky | what about a non-working address? |
| 15:22.35 | Zaphoid | dhcp80, dhcp89, dhcp170, etc, etc |
| 15:22.54 | Zaphoid | I've manually assigned (via DHCP) other systems to the dhcp83 address, and then they work |
| 15:23.01 | Norky | okay, so it's not the Mint boot environment hanging on an un-answered DNS lookup |
| 15:24.11 | Zaphoid | Not as far as I can tell |
| 15:24.29 | Zaphoid | Also interesting, when it hangs, there are several packets in the send-Q when looking at the NFS connection on the NFS server |
| 15:24.32 | Zaphoid | through netstat |
| 15:26.04 | Norky | what happens if you leave a ping pinging the assigned address? |
| 15:26.29 | Norky | does it stop getting responses when the client 'hangs'? |
| 15:26.42 | Zaphoid | I've not done that during the boot process, I'll try that now |
| 15:27.00 | Norky | I'm wondering if it's switching to a different IP address |
| 15:27.21 | Zaphoid | Not from watching the DHCP server, but it could be |
| 15:27.30 | Norky | let it get a non-working address, keep pinging that, watch the behaviour |
| 15:27.34 | Zaphoid | moment |
| 15:28.16 | Norky | also, is the 'working' IP address (.83?) currently assigned to anything else? |
| 15:28.40 | Zaphoid | No, its in the DHCP range and nothing is there now. |
| 15:30.37 | Zaphoid | Pingable until it gets to that spot |
| 15:30.41 | Zaphoid | Then disappears |
| 15:30.50 | Norky | okay, just for a giggle, try pinging .83 |
| 15:31.40 | Zaphoid | Nothing now, and not seeing the MAC in dhcp, and nothing at dhcp83 |
| 15:31.49 | Norky | okay |
| 15:32.01 | Norky | how 'hung' is 'hung'? |
| 15:32.23 | Norky | does it respond to keyboard lock keys (caps lock, num lock etc)? |
| 15:32.59 | Zaphoid | Responds to those |
| 15:33.11 | Zaphoid | Kepresses are output on the screen |
| 15:33.14 | Norky | okay, so the kernel is alive |
| 15:33.18 | Zaphoid | Yes |
| 15:33.42 | Norky | does ctrl-alt-f1 through to f8 get you a shell on the console? |
| 15:34.07 | Norky | or even debug/informational log out |
| 15:34.09 | Zaphoid | No shell, running task is on f1, all others are blank |
| 15:35.22 | Norky | huh, not terribly useful |
| 15:35.50 | Zaphoid | Thats what I was finding |
| 15:35.52 | Zaphoid | Hrmm |
| 15:35.54 | Zaphoid | Haven't tried single yet |
| 15:35.57 | Zaphoid | Guess I'll try that next |
| 15:36.48 | Norky | I'm afraid I don't know what else to suggest |
| 15:36.59 | Zaphoid | Thanks for the attempts |
| 15:37.08 | Zaphoid | Tried a couple things I hadn't used to troubleshoot, yet |
| 15:37.22 | Norky | not PXE-related, I'm sure of that :) |
| 15:37.52 | Norky | you coudl try running tcpdump or wireshark on the server, and watch what happens on the network at the time of the hang |
| 15:40.07 | Zaphoid | Probably will happen next. |
| 15:45.28 | Zaphoid | Hrmm, interestingly enough, tacking on single doesn't cause it to boot into single user mode, or it doesn't get far enough to. |
| 15:50.58 | Norky | could try turning on NFS debug logging to see what files are being accessed, but I think you can only get filenames from the client |
| 15:51.05 | Norky | https://www.serverstack.com/blog/2012/11/21/debugging-nfs-file-access-on-server-and-client-side/ |
| 15:52.24 | Zaphoid | Never heard of that before, interesting. |
| 15:54.44 | Zaphoid | Well, that shoved a bunch of stuff into the logs that I have no idea what it means :P |
| 15:55.09 | Norky | yeah, it's less useful/more arcane on the server side |
| 15:55.53 | Zaphoid | Looks like its accessing a bunch of the init stuff, as you'd expect |
| 15:56.05 | Norky | like I said, until you know how to interpret it, you want the logging on the client (but of course, the client is broken/inaccessible so that's not easy :p) |
| 15:56.33 | Zaphoid | Yeah, can't do that, except to try and run it on the working IP and try to work backwards from there |
| 15:57.06 | Norky | watching the NFS conversation with wireshark shoudl be more informative |
| 15:57.16 | Norky | it shoudl give you name |
| 15:57.20 | Norky | filenames* |
| 15:59.43 | Zaphoid | Watching the NFS logs, it even gets to pulling the NetworkManager stuff |
| 15:59.48 | Zaphoid | So it gets pretty far before failing |
| 16:00.43 | Norky | is it requesting the NetworkManager files shortly before the hang? |
| 16:00.55 | Zaphoid | That was the last thing it requested, at least at this last attempt |
| 16:01.15 | Norky | that tells us something :) |
| 16:01.23 | Zaphoid | Hrmm |
| 16:01.59 | Norky | try grepping the files that it requests for anything that looks like an IP address |
| 16:02.35 | Zaphoid | Stops at different places each attempt. One time it gat far enough to write out /var/log/udev, but nothing else (not even boot.log) |
| 16:02.39 | Zaphoid | Not this last time, though |
| 16:02.47 | Norky | ahh |
| 16:03.03 | Zaphoid | Have you messed with network manager before? |
| 16:03.07 | Zaphoid | I honestly don't use it much |
| 16:05.25 | Zaphoid | Wow |
| 16:05.31 | Norky | I use it on desktop systems (mostly Fedora) |
| 16:05.33 | Zaphoid | Ok |
| 16:05.34 | Zaphoid | That totally worked |
| 16:05.36 | Zaphoid | Ha! |
| 16:05.57 | Norky | this most recent time it worked where it did not before? |
| 16:05.59 | Zaphoid | Changed NetworkManager.conf |
| 16:06.03 | Zaphoid | From managed=true to managed=false |
| 16:06.21 | Zaphoid | Now have two systems booting off of the same NFS |
| 16:06.23 | Zaphoid | Nice |
| 16:09.06 | Zaphoid | Thanks for some of the pointers. the nfs debugging really helped |
| 16:31.38 | Zaphoid | Woo, now have 4 systems running off of the same PXE/NFS |
| 16:32.18 | Norky | jolly good, best let the Mint people know :) |
| 16:35.31 | Norky | I'm off, cheerio |
| 20:09.11 | *** join/##pxe ctugt (~ctugt@a237011.upc-a.chello.nl) |