| 00:00.20 | *** join/#storm _mup_ (~mup@li37-90.members.linode.com) |
| 00:08.02 | *** join/#storm bigjools (~quassel@60-241-216-21.static.tpgi.com.au) |
| 00:08.02 | *** join/#storm bigjools (~quassel@canonical/launchpad/bigjools) |
| 01:00.56 | *** join/#storm fbru02 (~fbru02@r186-52-188-140.dialup.adsl.anteldata.net.uy) |
| 03:03.31 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 03:15.16 | *** join/#storm fbru02 (~fbru02@r186-52-186-248.dialup.adsl.anteldata.net.uy) |
| 04:15.37 | *** join/#storm lifeless (~robertc@122.58.129.196) |
| 11:41.06 | *** join/#storm fbru02 (~fbru02@r190-135-53-6.dialup.adsl.anteldata.net.uy) |
| 12:32.58 | *** join/#storm niemeyer (~niemeyer@201.11.202.90) |
| 12:32.59 | *** mode/#storm [+o niemeyer] by ChanServ |
| 14:04.40 | *** join/#storm fbru02_ (~fbru02@r190-135-79-100.dialup.adsl.anteldata.net.uy) |
| 14:07.25 | *** join/#storm fbru02 (~fbru02@r190-135-23-201.dialup.adsl.anteldata.net.uy) |
| 14:10.27 | *** join/#storm fbru02_ (~fbru02@r186-54-219-180.dialup.adsl.anteldata.net.uy) |
| 14:20.32 | *** join/#storm fbru02 (~fbru02@r186-54-219-180.dialup.adsl.anteldata.net.uy) |
| 15:12.37 | *** join/#storm fbru02_ (~fbru02@r186-54-243-181.dialup.adsl.anteldata.net.uy) |
| 15:18.42 | magmatt | I'm making a webapp using Twisted and Storm. For now, it's fine that db access blocks the main thread, but I'm concerned about how I would refactor if that every became a problem. |
| 15:18.50 | magmatt | Here's one idea I've been kicking around: http://paste.pound-python.org/show/25495/ |
| 15:19.07 | magmatt | what think ye? |
| 15:31.22 | *** join/#storm fbru02 (~fbru02@r190-135-2-84.dialup.adsl.anteldata.net.uy) |
| 15:56.40 | *** join/#storm esteve (~quassel@h-91-126-88-6.wholesale.rp80.se) |
| 16:00.43 | magmatt | and here's a proof of concept: http://paste.pound-python.org/show/25498/ |
| 16:01.33 | therve | magmatt, that looks interesting |
| 16:01.43 | therve | jkakar may be interested |
| 16:02.11 | magmatt | therve: thanks; do you still run any twisted+storm code? What do you do? |
| 16:02.25 | therve | magmatt, what's in storm.twisted in trunk |
| 16:05.12 | magmatt | oh... we're still stuck on 0.17 here - I didn't know that was in trunk. |
| 16:06.19 | therve | yeah you need 0.19 at least |
| 16:07.00 | magmatt | huh... so that forces you to use a non-Storm-object API, which could be a very nice constraint |
| 16:07.42 | therve | yeah, that's because we didn't want to "hack" it and remove the link to the store like you did |
| 16:07.50 | therve | which may be reasonable |
| 16:08.06 | magmatt | it also avoids the problem of running into database queries at every turn |
| 16:08.15 | magmatt | transact does, not mine |
| 16:09.30 | magmatt | hmm, I really like the transact idea |
| 16:09.42 | magmatt | therve: thanks! I'm going to play with it to see if it suits my needs |
| 16:09.52 | therve | magmatt, no problem |
| 16:53.06 | *** join/#storm fbru02 (~fbru02@r190-135-33-50.dialup.adsl.anteldata.net.uy) |
| 17:01.31 | *** join/#storm fbru02_ (~fbru02@r186-52-165-227.dialup.adsl.anteldata.net.uy) |
| 17:04.36 | *** join/#storm fbru02 (~fbru02@r186-52-155-78.dialup.adsl.anteldata.net.uy) |
| 17:11.32 | *** join/#storm fbru02_ (~fbru02@r186-52-177-56.dialup.adsl.anteldata.net.uy) |
| 17:16.34 | *** join/#storm fbru02 (~fbru02@r190-135-69-97.dialup.adsl.anteldata.net.uy) |
| 17:27.50 | jkakar | magmatt: Heya. We're using transact with Twisted and it works alright. |
| 17:28.46 | jkakar | In our case (Fluidinfo) we chose to run single threaded processes for our API services. So the number of API processes we run defines the maximum concurrency we can have. |
| 17:29.27 | jkakar | We found that 8 single-threaded processes performed better under load than 4 2-threaded processes. It was a bit surprising, but we were able to reproduce the behaviour, so we chose to go that route. |
| 17:29.50 | jkakar | We still use a separate thread for database access, but we don't really need to. Mostly we did that so that, if we want, we could increase the number of threads in a process. So far we haven't done that. |
| 17:30.00 | jkakar | That could you wrote looks pretty nice though. |
| 17:30.31 | magmatt | jkakar: okay, that's good info -- and thanks |
| 17:33.54 | jkakar | np |
| 17:41.00 | *** join/#storm fbru02 (~fbru02@r190-135-3-208.dialup.adsl.anteldata.net.uy) |
| 18:48.32 | *** join/#storm fbru02 (~fbru02@r190-135-3-208.dialup.adsl.anteldata.net.uy) |
| 19:02.35 | *** join/#storm fbru02 (~fbru02@r186-54-245-43.dialup.adsl.anteldata.net.uy) |
| 19:22.14 | *** join/#storm skrite (~piv@pivotrac.texaspivot.com) |
| 19:22.22 | skrite | hey all |
| 19:24.26 | jkakar | skrite: Heya. :) |
| 19:27.43 | skrite | i have been using storm for a few years now, love it. we are about to migrate our server from MyISAM to an NDB cluster. |
| 19:27.53 | skrite | think i will run into trouble? |
| 19:33.13 | jkakar | skrite: Ooh, good question. I have no idea... I rarely use MySQL, other than to run tests... |
| 19:33.33 | jkakar | skrite: What you might do to see what could go wrong is try to run the Storm test suite against the database setup you'll be using. |
| 19:33.40 | jkakar | skrite: If it passes, then you're probably in good shape. |
| 19:41.59 | skrite | jkakar: cool, thanks |
| 19:42.06 | jkakar | skrite: np |
| 19:42.18 | skrite | jkakar: what do you typically use? Postgre? |
| 19:42.28 | jkakar | skrite: The Storm test suite really tries to exercise a lot of crazy cases, so I think you can be fairly confident if everything works. |
| 19:42.40 | jkakar | skrite: Yeah, PostgreSQL most of the time and SQLite in some special cases. |
| 19:42.47 | skrite | i see. |
| 20:42.32 | *** join/#storm fbru02 (~fbru02@r190-135-41-160.dialup.adsl.anteldata.net.uy) |
| 22:17.40 | *** join/#storm fbru02_ (~fbru02@r190-135-34-60.dialup.adsl.anteldata.net.uy) |
| 23:39.14 | *** join/#storm bigjools (~quassel@60-241-216-21.static.tpgi.com.au) |
| 23:39.14 | *** join/#storm bigjools (~quassel@canonical/launchpad/bigjools) |