| 01:10.26 | *** join/#asterisk-dev Corydon76 (~quassel@c-69-137-93-158.hsd1.tn.comcast.net) |
| 01:10.26 | *** mode/#asterisk-dev [+o Corydon76] by ChanServ |
| 01:33.50 | *** join/#asterisk-dev Kobaz (~kobaz@173.234.39.101) |
| 01:57.42 | *** join/#asterisk-dev superscrat (~asanders@173-21-89-217.client.mchsi.com) |
| 02:41.15 | *** join/#asterisk-dev spditner (~simon@206-248-160-15.dsl.teksavvy.com) |
| 02:42.26 | *** join/#asterisk-dev coreyfarrell (~coreyfarr@24-177-3-44.dhcp.nwtn.ct.charter.com) |
| 03:36.04 | *** join/#asterisk-dev TazzNZ (~pieter@office-nat2.ytechops.co.nz) |
| 03:44.40 | *** join/#asterisk-dev TazzNZ (~pieter@office-nat2.ytechops.co.nz) |
| 04:53.36 | *** join/#asterisk-dev tuxd00d (~tuxd00d@ip70-162-146-39.ph.ph.cox.net) |
| 05:14.40 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 05:14.40 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 06:21.01 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 06:21.01 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 06:51.20 | *** join/#asterisk-dev hehol (~hehol@gatekeeper.loca.net) |
| 06:53.27 | *** join/#asterisk-dev pchero_work (~pchero@109.70.54.56) |
| 06:55.15 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 06:55.15 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 07:08.17 | *** join/#asterisk-dev pchero_work (~pchero@109.70.54.56) |
| 07:28.08 | *** join/#asterisk-dev tzafrir (~tzafrir@local.xorcom.com) |
| 07:28.08 | *** mode/#asterisk-dev [+o tzafrir] by ChanServ |
| 09:59.07 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 09:59.07 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 10:53.47 | *** join/#asterisk-dev doome_ (~doome@82.150.48.146) |
| 11:51.48 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 11:51.48 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 12:04.37 | *** join/#asterisk-dev Neuromanc0 (Neuromanc0@62.173.9.248) |
| 12:24.34 | *** join/#asterisk-dev mjordan (~mjordan@75.76.55.191) |
| 12:24.34 | *** mode/#asterisk-dev [+o mjordan] by ChanServ |
| 12:25.36 | *** join/#asterisk-dev [TK]D-Fender (~chatzilla@216-191-106-163.dedicated.allstream.net) |
| 12:38.28 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 12:38.28 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 13:19.30 | *** join/#asterisk-dev Demon_VoIP (~demon@109.60.135.186) |
| 13:21.46 | *** part/#asterisk-dev mjordan (~mjordan@75.76.55.191) |
| 13:24.50 | *** join/#asterisk-dev jameswf (uid27319@gateway/web/irccloud.com/x-sktnvjxcvxzdtajy) |
| 13:41.56 | *** join/#asterisk-dev gtjoseph (~gtj0@unaffiliated/gtj) |
| 13:55.46 | *** join/#asterisk-dev mjordan (mjordan@nat/digium/x-yxkelafwhefoapwt) |
| 13:55.46 | *** mode/#asterisk-dev [+o mjordan] by ChanServ |
| 13:59.34 | *** join/#asterisk-dev sgriepentrog (sgriepentr@nat/digium/x-hjjqkwobbqplaqvp) |
| 14:06.39 | *** join/#asterisk-dev putnopvut (putnopvut@asterisk/master-of-queues/mmichelson) |
| 14:06.39 | *** mode/#asterisk-dev [+o putnopvut] by ChanServ |
| 14:09.42 | Demon_VoIP | 13.4. the last few weeks, periodically (every 8 hours) the server hangs. netstat shows the queue for sending UDP. Very similar to DNS issues resolving, but in the logs there are no errors "getaddr". There are no errors :( How can i debug this? |
| 14:10.22 | mjordan | Demon_VoIP: https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace |
| 14:10.52 | mjordan | Start with a GDB backtrace, as DEBUG_THREADS is a killer. If something looks suspicious, it may be useful getting a 'core show locks' output as well. |
| 14:12.06 | Demon_VoIP | thanks. i'll try :( |
| 14:12.37 | *** join/#asterisk-dev oej (~oej@h87-96-134-129.dynamic.se.alltele.net) |
| 14:12.37 | *** mode/#asterisk-dev [+o oej] by ChanServ |
| 14:12.54 | Demon_VoIP | DEBUG_THREADS uses a lot of CPU. But if no other ways.. |
| 14:13.08 | mjordan | I wouldn't start with that |
| 14:13.20 | mjordan | And you don't need DEBUG_THREADS to get a GDB backtrace |
| 14:13.37 | mjordan | Make sure you have 'DONT_OPTIMIZE' and - preferably - 'BETTER_BACKTRACES' enabled. That shouldn't hurt performance too much. |
| 14:13.52 | mjordan | Right now, you don't know that you have a deadlock; merely that something becomes unresponsive, which can happen for a lot of reasons |
| 14:14.07 | file | the fact it happens with both chan_sip and res_pjsip is odd |
| 14:14.10 | mjordan | So just get a backtrace and pb it. If it looks suspicious, then you can enable DEBUG_THREADS and try to get the full locking picture |
| 14:14.25 | mjordan | I'd suspect it is something else, personally |
| 14:14.45 | Demon_VoIP | file, yep.. i am using both. Is it the reason? |
| 14:14.56 | file | no? probably not? |
| 14:15.03 | mjordan | random guesses? |
| 14:16.06 | *** join/#asterisk-dev kharwell (kharwell@nat/digium/x-kfkuveztiojmudwz) |
| 14:17.08 | Demon_VoIP | it started between 13.3 and 13.4. I updated with 13 branches in git |
| 14:19.56 | *** join/#asterisk-dev superscrat (asanders@nat/digium/x-pdjnaosfxzymzpjj) |
| 14:19.57 | Demon_VoIP | by this time I transfered almost all peers from chan_sip to res_pjsip. And today i rollbacked. But no miracle happened. The server still hangs |
| 14:21.15 | Demon_VoIP | ok. when will backtraces it will be more clear |
| 14:24.40 | *** join/#asterisk-dev DivideBy0 (~DivideBy0@unaffiliated/divideby0x0) |
| 14:35.26 | Demon_VoIP | No such command 'core show locks'. Is th reason 'DONT_OPTIMIZE' off? |
| 14:37.26 | file | core show locks is only enabled when DEBUG_THREADS is compiled in |
| 14:39.00 | Demon_VoIP | ok.. if needed, we will tolerate |
| 14:39.18 | file | no - as mjordan said, start by getting a backtrace as documented on the wiki |
| 14:39.21 | file | that doesn ot require DEBUG_THREADS |
| 14:39.59 | Demon_VoIP | only backtraces withoud locks? |
| 14:40.53 | file | yes, DEBUG_THREADS is not required to get a backtrace |
| 14:42.47 | Demon_VoIP | i wrote watchdog to netstat UDP queue depth... and collect logs and restart. Should I create an issue with backtraces? |
| 14:42.58 | file | you can |
| 14:46.04 | *** join/#asterisk-dev rmudgett (rmudgett@nat/digium/x-bxptptvjydozbcmx) |
| 14:48.59 | *** join/#asterisk-dev leedm777 (leedm777@nat/digium/x-jiqbhbdfezroilrl) |
| 15:08.09 | wanda_ | hello, it's our first tuto about telephone system open source https://twitter.com/v_dagrain/status/609464240307576832 with raspberry pi 2 http://t.co/d4LqAaoMw3 thanks to share because wednesday 17th we organize =) a barcamp WITH mumble about YOUR project in VOIP https://twitter.com/v_dagrain/status/607894402632171520 - message over thank youuuu <3 --- |
| 17:05.55 | wanda_ | o/ :) |
| 17:44.55 | *** join/#asterisk-dev Demon_VoIP (~demon@ip253.net222.n37.ru) |
| 18:00.59 | rmudgett | file: When a module unloads is there a routine to unregister sorcery config objects? The counter part to ast_sorcery_object_register()? |
| 18:01.18 | *** join/#asterisk-dev tuxd00d (~tuxd00d@ip70-162-146-39.ph.ph.cox.net) |
| 18:01.25 | file | not currently, no |
| 18:02.29 | rmudgett | So when a module is unloaded and then the config is reloaded we could have those sections attempt to call functions in the unloaded module? |
| 18:02.51 | file | hrm, yes |
| 18:03.12 | rmudgett | Sounds like a recipe for crashy :) |
| 18:03.30 | file | the other problem potentially is - if anything else has a reference to the object... then you can't unload |
| 18:03.42 | file | because of the destructor |
| 18:05.55 | mjordan | hm. |
| 18:06.11 | mjordan | so, one place I'd prefer we not go to is having every instance of an object bump a module ref count. |
| 18:06.37 | mjordan | At the same time, *something* needs to not allow modules to unload themselves willy-nilly until things that reference functions in that module are destroyed |
| 18:07.28 | mjordan | file: do objects provided by sorcery have their own destructors, or is there some common sorcery destructor that gets called at some point? |
| 18:07.38 | file | they have their own |
| 18:07.58 | file | to everything else they are a normal ao2 object, except for the accessors for sorcery specific info |
| 18:10.35 | mjordan | that's what I thought |
| 18:11.03 | mjordan | the issue there is that if object creation should bump the module reference in some fashion (which sorcery presumably could be made aware of), there's no consistent way to release the reference, and you don't want the consumers of sorcery to be bothered by those notions |
| 18:12.02 | mjordan | One "easy" way to do it would be to require all consumers of sorcery to start the destruction of objects that use their configuration prior to unloading, and then have a call (such as ast_sorcery_object_unregister) that blocks until all 'live' objects are destroyed |
| 18:12.09 | mjordan | but I'm not sure sorcery is even aware of that fact either. |
| 18:12.22 | mjordan | At the end of the day, this feels like something that sorcery consumers should be aware of, and somewhat responsible for. |
| 18:14.38 | file | I don't *think* anything currently uses stuff in this way so it be moot... but we could encounter it further down the line |
| 18:14.46 | mjordan | well |
| 18:15.02 | mjordan | res_pjsip kind of does. |
| 18:15.14 | mjordan | except that it sort of enforces the notion that it unloads after everything else already |
| 18:15.19 | file | yes |
| 18:15.49 | mjordan | rmudgett: what object are you worried about would be using configuration sections provided by res_pjsip_outbound_registration? |
| 18:16.51 | rmudgett | I was looking at the unload function and there was no corresponding function to ast_sorcery_ojbect_register("registration") |
| 18:17.13 | rmudgett | That provides an alloc and an apply callback |
| 18:18.21 | rmudgett | So if a reload happens for res_pjsip.conf we could get those callbacks called for the type=registration sections. |
| 18:18.42 | rmudgett | At that point the module no longer exists. |
| 18:18.46 | file | I think having an object unregister should be fine |
| 18:53.45 | *** part/#asterisk-dev mjordan (mjordan@nat/digium/x-yxkelafwhefoapwt) |
| 19:03.49 | *** join/#asterisk-dev tzafrir (~tzafrir@bzq-179-40-172.cust.bezeqint.net) |
| 19:03.49 | *** mode/#asterisk-dev [+o tzafrir] by ChanServ |
| 19:14.51 | *** join/#asterisk-dev mjordan (mjordan@nat/digium/x-zjvmxzixctqbkiwz) |
| 19:14.51 | *** mode/#asterisk-dev [+o mjordan] by ChanServ |
| 19:45.26 | leifmadsen | so it turns out latency with agents will mess with app_queue something fierce |
| 19:45.28 | leifmadsen | who knew?! |
| 19:47.06 | superscrat | leifmadsen, what was the name of that book you told me to read? i forgot to write it down. |
| 19:47.16 | leifmadsen | Ansible: Up and Running? |
| 19:47.18 | leifmadsen | I assume :") |
| 19:47.21 | superscrat | I think that's it |
| 19:47.29 | leifmadsen | otherwise it was likely Asterisk; The Definitive Guide 4th edition :D |
| 19:47.58 | superscrat | Ha! I have already read that. I used to keep a copy of it under my pillow at night for osmosis (that may or may not be a joke) :p |
| 19:49.00 | superscrat | what's the story with latency? |
| 19:49.04 | leifmadsen | you might need 2 for that |
| 19:49.33 | superscrat | hahahaha, well, i also have the pdf version if that counts |
| 20:13.56 | *** join/#asterisk-dev pchero (~pchero@0x555140b5.adsl.cybercity.dk) |
| 20:29.39 | *** join/#asterisk-dev mjordan (mjordan@nat/digium/x-bbohdozyphtvqckx) |
| 20:29.39 | *** mode/#asterisk-dev [+o mjordan] by ChanServ |
| 20:54.41 | mjordan | leifmadsen: device state woes? |
| 20:55.26 | leifmadsen | mjordan, nah, just latency and the inevitable race conditions when Asterisk sends CANCEL after 12 second ring duration and customer tries to answer, in combination with retry=1 |
| 20:55.39 | leifmadsen | so it just looks like multiple inbound calls at the same time, while answering and getting dead air |
| 20:55.49 | leifmadsen | 400ms connections will do that to you |
| 20:56.02 | file | 400ms of happiness |
| 20:56.06 | leifmadsen | lol |
| 20:56.12 | leifmadsen | longer is better right? |
| 20:56.16 | file | that reminds me |
| 20:56.25 | file | mjordan, my constant pinging has shown nothing out of the ordinary |
| 20:56.47 | mjordan | file: interesting |
| 22:01.16 | *** part/#asterisk-dev mjordan (mjordan@nat/digium/x-bbohdozyphtvqckx) |
| 22:18.46 | *** join/#asterisk-dev [TK]D-Fender (~joe@64.235.216.2) |
| 22:49.40 | *** part/#asterisk-dev kharwell (kharwell@nat/digium/x-kfkuveztiojmudwz) |