| 00:35.50 | *** join/#asterisk-dev otherwiseguy (~otherwise@2605:a601:433:2301:862b:2bff:fead:2cce) |
| 02:34.53 | *** join/#asterisk-dev superscrat (~asanders@173-21-89-217.client.mchsi.com) |
| 06:44.54 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 06:44.54 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 07:11.06 | *** join/#asterisk-dev babak (uid19622@gateway/web/irccloud.com/x-ydtutryruyaczmxh) |
| 07:47.21 | *** join/#asterisk-dev pchero_work (~pchero@109.70.54.56) |
| 08:24.44 | *** join/#asterisk-dev hehol (~hehol@gatekeeper.loca.net) |
| 08:27.54 | *** join/#asterisk-dev oej (~oej@185.76.74.37) |
| 08:27.54 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 08:32.16 | *** join/#asterisk-dev tzafrir (~tzafrir@local.xorcom.com) |
| 08:32.16 | *** mode/#asterisk-dev [+o tzafrir] by ChanServ |
| 08:58.36 | *** join/#asterisk-dev oej (~oej@2.68.227.138.mobile.tre.se) |
| 08:58.36 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 12:01.29 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 12:01.29 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 12:19.03 | roramirez | Hi |
| 12:21.10 | file | hi |
| 12:26.46 | stefan27 | When debugging a deadlock in my experimental code, I'm trying to understand http://doxygen.asterisk.org/trunk/AsteriskArchitecture.html#ArchThreadingModel . In my specific case: SIP peer A calls sip peer B through a simple dialplan with Dial(SIP/B). It seems the sip monitor thread (T_1) receives A's invite and spawns a pbx-thread T_2 which in turn executes Dial(SIP/B). I have interpreted |
| 12:26.46 | stefan27 | the docs this way: T_2 creates both the inbound and the outbound SIP channels and T_2 from that point on processes all sip-message-traffic belonging to these two invite dialogues (?) |
| 12:27.32 | file | stefan27, the sip monitor thread handles all incoming UDP traffic, each TCP connection has its own thread |
| 12:29.24 | stefan27 | Aha, but there are no tcp connections at play in my test case. Anyways: It looks as if the monitor thread receives a "Proxy Authentication Required" from the INVITE to B generated by channelA's pbx-thread's (Dial(...B...)) |
| 12:29.38 | file | yes |
| 12:29.44 | stefan27 | is that how it's supposed to be? I may very well have misinterpreted the flow based on my thread dumps |
| 12:29.57 | file | yes, that is how it works |
| 12:30.05 | file | the monitor thread receives all UDP traffic and it is handled there |
| 12:30.21 | stefan27 | what kind of traffic does the pbx-thread handle? |
| 12:30.41 | file | some outgoing messages, and Asterisk core stuff |
| 12:31.10 | stefan27 | Aha, thanks, I gotta read up some more |
| 12:37.20 | roramirez | could be add a one line for a log DEBUG (https://gist.github.com/roramirez/8726e7587c22c64ec67f)? In case of error on dialplan sometimes it's hard to see where is find a find a file for PlayBack or Background. |
| 12:38.05 | *** join/#asterisk-dev [TK]D-Fender (~chatzilla@216-191-106-163.dedicated.allstream.net) |
| 13:06.10 | file | potentially, but most people don't enable debug when looking at that stuff |
| 13:18.23 | *** join/#asterisk-dev mjordan (mjordan@nat/digium/x-mwsymgirnubzdvtk) |
| 13:18.34 | *** mode/#asterisk-dev [+o mjordan] by ChanServ |
| 13:21.10 | roramirez | file: i know. In the community asterisk-es (http://bit.ly/1S9gO1s) a person have the problem. |
| 14:00.06 | Kobaz | sooo |
| 14:00.12 | Kobaz | help the computer |
| 14:00.14 | Kobaz | http://pastebin.com/7JVAxRp2 |
| 14:00.19 | Kobaz | i'm having a massive problem |
| 14:01.14 | [TK]D-Fender | "No Longer Supported" |
| 14:01.21 | Kobaz | i know it's 1.8 and old fuddy duddy and all that |
| 14:01.23 | Kobaz | hah |
| 14:01.24 | [TK]D-Fender | Upgrade |
| 14:01.24 | Kobaz | yes i know |
| 14:01.27 | Kobaz | we've been through this |
| 14:01.33 | [TK]D-Fender | And will again! |
| 14:08.06 | *** join/#asterisk-dev newtonr (RustyNewto@nat/digium/x-qrtlfnasysnpnyrj) |
| 14:08.06 | *** mode/#asterisk-dev [+o newtonr] by ChanServ |
| 14:11.00 | Kobaz | upgrade = porting a massive ami/agi application |
| 14:11.03 | Kobaz | which is taking some time |
| 14:11.17 | *** join/#asterisk-dev kharwell (kharwell@nat/digium/x-ttcqfxshwifzfiqg) |
| 14:14.25 | wdoekes | Kobaz: looks to me like no one has that lock, right? |
| 14:14.49 | wdoekes | using chanspy? |
| 14:16.43 | wdoekes | (if so, I'd point to ASTERISK-25321) |
| 14:18.02 | *** join/#asterisk-dev gtjoseph (~gtj0@unaffiliated/gtj) |
| 14:19.21 | Kobaz | mm, umm i don't believe chanspy is involved |
| 14:19.27 | Kobaz | ASTERISK-25321 |
| 14:19.39 | Kobaz | didn't the bot used to spit out the url |
| 14:25.44 | Kobaz | i looked through the logs up until the lockup, no chanspy |
| 14:29.20 | Kobaz | === ---> Waiting for Lock #1 (astobj2.c): MUTEX 509 internal_ao2_link c 0xb6fab030 (1) |
| 14:29.26 | Kobaz | so everyone's waiting for that lock |
| 14:29.39 | Kobaz | but where's the lock, yeah that's what you were saying |
| 14:31.02 | Kobaz | maybe the output got truncated |
| 14:31.06 | Kobaz | i have the core file |
| 14:31.25 | Kobaz | my watcher does a kill sigquit of there's a deadlock |
| 14:31.32 | Kobaz | *if |
| 14:40.24 | wdoekes | Kobaz: I have deadlocks like the one in GRID-25321 where the thread holding the lock is gone; it could be what you're seeing. in GRID-25321, the wrong lock is unlocked, so the lock is held for the entire lifetime of the thread (and beyond) |
| 14:42.02 | Kobaz | i'm narrowing down the threads in the backtrace that would have the lock |
| 14:42.06 | wdoekes | you could run https://github.com/ossobv/pycoredump/blob/master/examples/debug-deadlock.py on your core file, but I suspect you'd learn nothing more than you already know from the thread_deubg you have |
| 14:42.12 | Kobaz | like ignoring music on hold and tcptls and all that stuff |
| 14:43.06 | Kobaz | k cool |
| 14:43.48 | Kobaz | what's crazy is this *just* started happening |
| 14:44.23 | Kobaz | my production systems have been 1.8.12 with a massive amount of back patching from reported issues |
| 14:44.32 | Kobaz | and i've had that same version in there since like 2013 |
| 14:44.52 | Kobaz | rock solid up until last week, when i now get two of these lockups per day |
| 14:45.07 | Kobaz | i ported all my stuff to 1.8.32.3 |
| 14:45.16 | Kobaz | and it's still going on, so i'm kind of stuck |
| 14:46.22 | Kobaz | what version of python does this need? |
| 14:46.41 | Kobaz | syntax error: with GdbWithThreads(program=program, corefile=corefile) as dump: |
| 14:47.16 | wdoekes | syntaxerror? python 2.5? |
| 14:47.37 | Kobaz | i've got 2.5 |
| 14:47.40 | Kobaz | 2.5.2 |
| 14:47.46 | wdoekes | ouch, that's .. well .. old |
| 14:48.15 | Kobaz | lemme see if i can instal python without breaking everything |
| 14:48.32 | Kobaz | 2.6.8 |
| 14:48.38 | Kobaz | er 2.6.6-8 |
| 14:48.44 | wdoekes | https://docs.python.org/2/library/__future__.html or use the future import |
| 14:48.53 | wdoekes | from __future__ import with_statement |
| 14:49.41 | Kobaz | i need pycoredump now |
| 14:50.05 | wdoekes | you probably did not clone the entire repo.. |
| 14:50.19 | Kobaz | oh |
| 14:50.20 | Kobaz | no |
| 14:50.20 | Kobaz | hehe |
| 14:50.24 | Kobaz | i just grabbed that script |
| 14:50.28 | Kobaz | whoops |
| 14:51.16 | wdoekes | future music, as we'd say in Dutch. it has no setup.py yet |
| 14:55.03 | Kobaz | http://pastebin.com/wpy78zyw |
| 14:55.07 | Kobaz | it doesn't like me |
| 14:56.10 | wdoekes | blegh, more python2.5 |
| 14:59.46 | wdoekes | well, at least we've learned that it doesn't work with python2.5 :D |
| 15:01.01 | Kobaz | lemme try and make it happy with some ne wpythons |
| 15:01.21 | stefan27 | When my asterisk is about to transmit a sip INVITE, the thread in question is holding both a sip_pvt lock and a channel lock, but if I from within that code, that is chan_sip::transmit_invite, want to read data from other ast channels structs, then I have to call some function which iterates through the ao2_container with channels, for example by using ast_channel_get_by_name(origchanname); |
| 15:01.22 | stefan27 | but I'm not allowed to do that while holding a channel lock, so I tried the following: http://pastebin.com/gpELnmk8 |
| 15:01.36 | Kobaz | same thing actually |
| 15:01.39 | Kobaz | when running python2.6 |
| 15:02.57 | stefan27 | but I had to add a lot of messy code (in an attempt to prevent deadlocks) just to be able to read one value from another channel's data, is there any cleaner way to do this -- keeping track of locks is difficult |
| 15:04.11 | wdoekes | Kobaz: funky. I don't know format() semantics in python2.6. I'll do a quick check if I have an old machine with that somewhere. |
| 15:06.28 | stefan27 | there is no utils function like lock_both(pvt,chan) ? |
| 15:07.20 | Kobaz | what do i need |
| 15:07.21 | Kobaz | 2.7? |
| 15:07.47 | Kobaz | in which asterisk version did masquerades go away? |
| 15:08.02 | wdoekes | I pushed a fix for 2.6 (I think) |
| 15:08.40 | wdoekes | masquerades at least exist in 11, and that's where my issues are |
| 15:11.34 | Kobaz | http://pastebin.com/frta6SBd |
| 15:13.41 | Kobaz | k |
| 15:13.51 | Kobaz | all my agi/ami crap relys on masquerades |
| 15:13.56 | Kobaz | so i can't port to 13 just yet |
| 15:16.54 | wdoekes | so sad. I think that's caused by my fragile gdb parsing. I could use some more core dump input to fix that, but I don't have time to dive into that right now. sorry for wasting your time :) |
| 15:17.48 | stefan27 | does usleep(1) accomplish the same as sched_yield()? |
| 15:18.01 | Kobaz | i wouldn't think so |
| 15:18.08 | Kobaz | usleep is going to be a specific amount of time |
| 15:18.10 | wdoekes | stefan27: practically |
| 15:18.16 | Kobaz | you accomplish a wait |
| 15:18.19 | Kobaz | the end result is a wait |
| 15:18.25 | Kobaz | the internals are different obviously |
| 15:19.25 | stefan27 | I copied that while/lock+unlock-loop with 'usleep' from some other function in chan_sip, but if I'm gonna mimic the logic of ast_channel_lock_both then i should use sched_yield instead |
| 15:26.55 | *** join/#asterisk-dev newtonr (RustyNewto@nat/digium/x-uhkftvdyzknjogwb) |
| 15:26.55 | *** mode/#asterisk-dev [+o newtonr] by ChanServ |
| 15:35.57 | *** join/#asterisk-dev rmudgett (rmudgett@nat/digium/x-pwqztultvfzbevlq) |
| 15:39.46 | *** join/#asterisk-dev Marquis42 (~mrbmw@d192-24-112-5.try.wideopenwest.com) |
| 15:49.15 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 15:49.15 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 16:19.09 | *** join/#asterisk-dev sgriepentrog (sgriepentr@nat/digium/x-fnljdhdndurxovjs) |
| 17:12.31 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 17:12.31 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 17:39.53 | *** part/#asterisk-dev spditner (~simon@23-91-146-202.cpe.pppoe.ca) |
| 17:42.43 | Kobaz | hey umm |
| 17:42.49 | Kobaz | where's the internals to ast_channel these days |
| 17:42.58 | Kobaz | i'm searching all over and can't find it, heh |
| 17:43.39 | file | main/channel_internal_api.c |
| 17:43.56 | Kobaz | yay |
| 18:23.34 | *** join/#asterisk-dev mjordan (mjordan@nat/digium/x-fbbaxfdfpdyzmkvi) |
| 18:23.34 | *** mode/#asterisk-dev [+o mjordan] by ChanServ |
| 18:30.16 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 18:30.16 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 18:37.59 | *** join/#asterisk-dev spditner (~simon@23-91-146-202.cpe.pppoe.ca) |
| 18:42.31 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 18:42.31 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 18:44.40 | *** join/#asterisk-dev jroseAtDigium (jon@nat/digium/x-bznjgncxvxmwknfd) |
| 18:49.09 | *** join/#asterisk-dev putnopvut (putnopvut@asterisk/master-of-queues/mmichelson) |
| 18:49.09 | *** mode/#asterisk-dev [+o putnopvut] by ChanServ |
| 18:51.58 | *** join/#asterisk-dev hexanol (~bibi@modemcable094.94-70-69.static.videotron.ca) |
| 18:53.45 | hexanol | has someone taken a look at https://issues.asterisk.org/jira/browse/ASTERISK-25023 ? if there is, do you have an idea on how complex it would be to have a fix for it ? |
| 18:54.10 | mjordan | I'm sure the issue state is accurate, which is that it is Open and no one is currently working on it |
| 18:55.55 | mjordan | as for how complex the fix is; no, I don't. The issue description is pretty good, but anything involving the scheduler is usually finicky |
| 18:56.05 | hexanol | I understand that, but even if the issue is "Open", someone could have taken a look and have a small idea on the complexity |
| 18:56.23 | hexanol | ok, interesting |
| 18:57.17 | mjordan | but it is definitely a deadlock. You've got circular waiting between the pvt and the scheduler. I'm not sure what the proper locking order for those two is supposed to be; I would suspect that holding the pvt lock while attempting to manipulate the scheduler is the "more wrong", but that should be confirmed. |
| 19:01.21 | hexanol | alright, thank you for the information |
| 19:04.32 | jroseAtDigium | Complexity is a hard thing to estimate in general. No units you see. I'll give the issue 5 question marks. Internally we have an idea of 'story points' given to issues, but that's just a relative estimate related to the complexity of other issues. |
| 19:06.12 | mjordan | jroseAtDigium: So, I'd like to get to the bottom of the exceptions/error codes, because it feels like you either are missing my point, or ignoring it :-) |
| 19:06.22 | mjordan | So: why are you mixing exceptions and error codes? |
| 19:06.26 | *** join/#asterisk-dev superscrat (~asanders@173-21-89-217.client.mchsi.com) |
| 19:06.57 | jroseAtDigium | I fixed it |
| 19:07.50 | superscrat | hello jroseAtDigium |
| 19:07.52 | jroseAtDigium | As for why, I wanted to not have the mailer throw anything at all... I didn't want it to be a cause for more general failure if something used it without trying, but I still wanted it to be able to indicate failure |
| 19:08.27 | jroseAtDigium | But I get why you'd prefer to have the responsibility for error handling on the caller, so I just did away with that. |
| 19:12.20 | jroseAtDigium | As an aside, it's not printing anything anymore, so now it's probably more generally useful. |
| 19:12.20 | jroseAtDigium | Not that I imagine any tests will be writing emails from Python anytime soon. |
| 19:13.41 | mjordan | I wanted to make sure you understood why you don't mix exceptions and error codes though as well. |
| 19:14.01 | mjordan | It isn't just about having the caller deal with things; that principle applies in this case regardless of the error mechanism |
| 19:14.07 | mjordan | That is; don't print that something failed in multiple locations |
| 19:14.18 | *** join/#asterisk-dev newtonr (RustyNewto@nat/digium/x-iiambygmzhledgjt) |
| 19:14.18 | *** mode/#asterisk-dev [+o newtonr] by ChanServ |
| 19:15.46 | mjordan | But if you mix the two idioms, you force the user of the function to handle both |
| 19:15.46 | mjordan | The best that you can hope to do is swallow *all* exceptions or error returns and convert them, which is an enormous amount of work for 0 benefit |
| 19:15.46 | mjordan | in the end, picking one of those paths is going to yield much more maintainable, cleaner code |
| 19:16.01 | Kobaz | whew |
| 19:16.02 | Kobaz | man |
| 19:16.14 | Kobaz | 10k line local branch diff, ported to 11 |
| 19:16.39 | Kobaz | well, kind of |
| 19:16.48 | Kobaz | done diffing and patching, but now i need to see what i broke |
| 19:17.15 | *** join/#asterisk-dev newtonr (RustyNewto@nat/digium/x-snrvdizccyknqwri) |
| 19:17.15 | *** mode/#asterisk-dev [+o newtonr] by ChanServ |
| 19:17.34 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 19:17.34 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 19:18.03 | *** join/#asterisk-dev newtonr (RustyNewto@nat/digium/x-zbvflwhnurbmpuvs) |
| 19:18.03 | *** mode/#asterisk-dev [+o newtonr] by ChanServ |
| 19:18.23 | Marquis42 | Kobaz: Does it compile? That means it's correct, right? :D |
| 19:18.59 | Kobaz | not yet |
| 19:19.17 | *** join/#asterisk-dev jroseAtDigium (jon@nat/digium/x-qvtsaayphoxpbhpc) |
| 19:21.27 | *** join/#asterisk-dev rmudgett (rmudgett@nat/digium/x-acfmkvhatskmpeut) |
| 19:23.54 | Kobaz | i'm breaking... i'm breaking |
| 19:23.58 | *** join/#asterisk-dev malcolmd (malcolmd@pdpc/sponsor/digium/malcolmd) |
| 19:23.58 | *** mode/#asterisk-dev [+o malcolmd] by ChanServ |
| 19:24.56 | *** join/#asterisk-dev tzafrir (~tzafrir@bzq-179-40-172.cust.bezeqint.net) |
| 19:24.56 | *** mode/#asterisk-dev [+o tzafrir] by ChanServ |
| 19:26.52 | *** join/#asterisk-dev newtonr (RustyNewto@nat/digium/x-xmabsypkxhdbmgqe) |
| 19:26.52 | *** mode/#asterisk-dev [+o newtonr] by ChanServ |
| 19:27.42 | mjordan | glares at the interwebs |
| 19:32.42 | *** join/#asterisk-dev rmudgett (rmudgett@nat/digium/x-xoffeazlvopxixmc) |
| 19:34.04 | *** join/#asterisk-dev kharwell (kharwell@nat/digium/x-ndzrgmvxoipfnrxb) |
| 19:40.44 | *** join/#asterisk-dev sgriepentrog (sgriepentr@nat/digium/x-uqdrxremjzmeoodh) |
| 19:53.56 | *** join/#asterisk-dev jrrose (jon@nat/digium/x-gjkyjiltkqxhufkg) |
| 19:58.55 | *** join/#asterisk-dev kharwell (~kharwell@216.186.189.18) |
| 20:05.25 | *** join/#asterisk-dev putnopvut (~putnopvut@asterisk/master-of-queues/mmichelson) |
| 20:05.25 | *** mode/#asterisk-dev [+o putnopvut] by ChanServ |
| 20:05.52 | *** join/#asterisk-dev rmudgett (~rmudgett@216.186.189.18) |
| 20:09.04 | *** join/#asterisk-dev superscrat (~asanders@173-21-89-217.client.mchsi.com) |
| 20:11.07 | putnopvut | I've got an interesting SIP scenario, and I'm trying to figure out the right behavior. I'd look to the SIP subscriptions RFC for guidance, but the network here is funky so I can't pull it up. |
| 20:11.25 | putnopvut | And right as I say that, the RFC loads. |
| 20:11.36 | putnopvut | So hold tight and I'll try to find the answer for myself. |
| 20:12.38 | file | whistles hold music |
| 20:14.24 | putnopvut | Okay, got my answer. If a notifier sends a NOTIFY and receives a non-200 class response, then the notifier is supposed to remove the subscription. Right now, the subscription is staying alive even after receiving an error response. |
| 20:15.00 | file | fixable! |
| 20:15.50 | putnopvut | I was expecting that we already did the right thing. But PJSIP only terminates the subscription based on certain responses (401, 407, 408, 481, and 7XX) |
| 20:16.05 | putnopvut | So we don't get told that, e.g. a 403 has been received. |
| 20:16.38 | file | blargh |
| 20:17.24 | putnopvut | In order to do the right thing, I'll have to add a transaction state callback to subscription code in order to be able to tell that we've received such a response and then send a terminating NOTIFY if needed. |
| 20:19.06 | file | make sure you don't end up double reacting to stuff somehow |
| 20:19.28 | putnopvut | Yeah, that's a concern |
| 20:45.16 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 20:45.16 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 20:55.19 | *** join/#asterisk-dev rmudgett (~rmudgett@216.186.189.18) |
| 20:55.21 | *** join/#asterisk-dev putnopvut (~putnopvut@asterisk/master-of-queues/mmichelson) |
| 20:55.21 | *** mode/#asterisk-dev [+o putnopvut] by ChanServ |
| 20:55.25 | *** join/#asterisk-dev kharwell (~kharwell@216.186.189.18) |
| 20:56.24 | *** join/#asterisk-dev [TK]D-Fender (~joe@64.235.216.2) |
| 20:57.27 | *** join/#asterisk-dev pchero (~pchero@109.70.54.56) |
| 21:18.46 | *** join/#asterisk-dev putnopvut (putnopvut@asterisk/master-of-queues/mmichelson) |
| 21:18.46 | *** mode/#asterisk-dev [+o putnopvut] by ChanServ |
| 21:19.19 | *** join/#asterisk-dev rmudgett (rmudgett@nat/digium/x-xtuwejfvuueczwfb) |
| 21:19.27 | *** join/#asterisk-dev kharwell (kharwell@nat/digium/x-cnuiyceuwkyyifbq) |
| 22:14.19 | *** join/#asterisk-dev sgriepentrog (sgriepentr@nat/digium/x-tabhsbrkymwnweki) |
| 22:49.23 | *** part/#asterisk-dev kharwell (kharwell@nat/digium/x-cnuiyceuwkyyifbq) |
| 23:49.49 | Kobaz | meep, anyone home? |
| 23:50.08 | Kobaz | asterisk 11, i'm trying to get the owner channel from inside ast_rtp_read() res_rtp_asterisk.c |
| 23:57.58 | *** join/#asterisk-dev snuff-work (~blatus@203.87.61.22) |
| 23:57.58 | *** mode/#asterisk-dev [+o snuff-work] by ChanServ |