| 00:08.54 | *** join/#storm artista_frustrad (~artista_f@189-11-98-54.ctaje701.dsl.brasiltelecom.net.br) |
| 01:42.52 | *** join/#storm jkakar_ (~jkakar@245.Red-88-5-215.dynamicIP.rima-tde.net) |
| 01:51.09 | *** join/#storm fcorrea (~fcorrea@189.6.221.60) |
| 02:15.09 | *** join/#storm fcorrea (~fcorrea@189.6.221.60) |
| 02:22.27 | *** join/#storm artista_frustrad (~artista_f@189.11.98.54) |
| 02:42.20 | *** join/#storm fcorrea (~fcorrea@189.6.221.60) |
| 03:00.42 | *** join/#storm fcorrea (~fcorrea@189.6.221.60) |
| 05:58.54 | *** join/#storm artista_frustrad (~artista_f@189-11-98-54.ctaje701.dsl.brasiltelecom.net.br) |
| 06:15.47 | *** join/#storm artista_frustrad (~artista_f@189-11-98-54.ctaje701.dsl.brasiltelecom.net.br) |
| 06:32.30 | *** join/#storm artista_frustrad (~artista_f@189-11-98-54.ctaje701.dsl.brasiltelecom.net.br) |
| 06:49.18 | *** join/#storm artista_frustrad (~artista_f@189-11-98-54.ctaje701.dsl.brasiltelecom.net.br) |
| 07:12.07 | *** join/#storm artista_frustrad (~artista_f@189-11-98-54.ctaje701.dsl.brasiltelecom.net.br) |
| 07:13.50 | *** join/#storm danilos (~danilo@canonical/launchpad/danilos) |
| 08:01.29 | *** join/#storm esteve (~quassel@cli-5b7ee9bd.bcn.adamo.es) |
| 08:02.55 | *** join/#storm danilos (~danilo@canonical/launchpad/danilos) |
| 08:19.55 | *** join/#storm zmijunkie (~sascha@pd95b6fd9.dip0.t-ipconnect.de) |
| 08:28.19 | *** join/#storm niemeyer (~niemeyer@conference/canonical/x-klvszrarcwmhsfes) |
| 08:36.34 | *** join/#storm stub (~stub@conference/canonical/x-xtusfymumjffwzty) |
| 08:36.35 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 08:41.50 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 09:00.45 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 09:35.05 | *** join/#storm stub (~stub@conference/canonical/x-lgliklvfnnmtezhz) |
| 09:35.05 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 09:45.30 | *** join/#storm fcorrea (~fcorrea@189.6.221.60) |
| 10:05.23 | *** join/#storm CrummyGummy (~meh@li185-28.members.linode.com) |
| 10:10.51 | mgedmin | it seems to me that people who use storm in a Zope context ought to catch psycopg2's TransactionRollbackError and convert it into a Retry exception |
| 10:11.00 | mgedmin | why doesn't storm do that by default? |
| 10:23.48 | *** join/#storm jdobrien (~john@50-88-19-25.res.bhn.net) |
| 10:24.40 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 11:22.02 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 11:23.55 | *** join/#storm niemeyer (~niemeyer@conference/canonical/x-xyvvmoipgitxbnml) |
| 11:34.36 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 11:35.40 | *** join/#storm zmijunkie (~sascha@pd95b6fd9.dip0.t-ipconnect.de) |
| 12:50.44 | jamesh | mgedmin: isn't that usually something that you would usually do from within your BrowserPublication's handleException() method? |
| 12:50.53 | mgedmin | I suppose |
| 12:51.01 | mgedmin | z3 doesn't make it easy to replace the default BrowserPublication |
| 12:51.03 | *** join/#storm stub (~stub@conference/canonical/x-igcerbcbemjbnybh) |
| 12:51.03 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 12:51.24 | mgedmin | is currently following the maze of all those little methods delegating responsibility to each other like a hot potato |
| 12:51.53 | jamesh | mgedmin: one thing that would have been nice would be if Zope had an IRetryable interface or something that you could mark exceptions with and have the default handleException() do the right thing with them. |
| 12:52.07 | mgedmin | yeah |
| 12:52.27 | mgedmin | but |
| 12:53.19 | mgedmin | is thinking whether it would be appropriate for storm to depend on the zope-specific package that defines the notional IRetryable |
| 12:54.16 | jamesh | while you can raise Retry deep within the call stack, doing so loses most of the traceback if the request isn't retried |
| 12:54.50 | mgedmin | hm, isn't it so that the deeper you do that the more traceback you have? |
| 12:55.33 | jamesh | no. The publisher unwraps the Retry exception restoring the original exception + traceback |
| 12:55.51 | mgedmin | ok, I'll see how I can replace a publication class used by zope.paste.factory.zope_app_factory() with the least amount of pain |
| 12:55.52 | jamesh | if you caught the exception early, you only get a few frames in the traceback and no idea of what the remaining call stack was |
| 12:56.23 | jamesh | [at least that is how things were when I was working on Launchpad a few years back] |
| 12:57.04 | mgedmin | oh, right, exception tracebacks stop at the level where you catch the exception |
| 12:57.32 | mgedmin | I've been mostly staring at the equivalent of traceback.print_stack() in my Dozer logview middleware; those contain the entire stack |
| 13:06.18 | jamesh | for the Storm side, I think it would make sense to add a database-neutral class to represent TransactionRollbackError though. That way if we wanted to add any retry related stuff to any of the framework integration packages, it wouldn't need to know about psycopg2 in particular |
| 13:06.47 | mgedmin | that would be nice, I think |
| 13:07.22 | mgedmin | I don't need to explicitly rollback a transaction when I catch TransactionRollbackError, right? |
| 13:07.27 | mgedmin | it's already been done? |
| 13:08.10 | jamesh | IIRC you do |
| 13:08.20 | mgedmin | hmm |
| 13:08.39 | jamesh | but doing a transaction.begin() or transaction.abort() with Zope's transaction module will handle that for you. |
| 13:08.55 | jamesh | presumably that is baked into your publication already |
| 13:09.13 | jamesh | or the publisher: it has been a while since I've poked around in that code |
| 13:11.28 | mgedmin | I looked at http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/canonical/launchpad/webapp/publication.py#L596 |
| 13:11.41 | mgedmin | didn't saw an .abort() or .rollback() call before it raises Retry(exc_info) |
| 13:11.44 | mgedmin | s/saw/see/ |
| 13:12.04 | mgedmin | well, a transaction.abort() can't hurt |
| 13:12.18 | jamesh | it has transaction.begin() in beforeTraversal() |
| 13:12.21 | mgedmin | especially since I want to abort any pending ZODB changes as well |
| 13:12.44 | mgedmin | checks |
| 13:12.54 | mgedmin | yep, ZopePublication also has a transaction.begin() in beforeTraversal() |
| 13:13.05 | jamesh | and an abort() or commit() call in afterCall() |
| 13:13.51 | mgedmin | and an abort() call at the very beginning of handleException() |
| 13:14.20 | mgedmin | which made me wonder if I needed that if I do it if I raise Retry before calling the superclass handleException() |
| 13:14.52 | mgedmin | thanks for all the help! |
| 13:15.48 | jamesh | begin() will automatically abort any current transaction |
| 13:16.02 | mgedmin | nods |
| 13:16.07 | jamesh | so if the store is still registered in an open transaction by that point it will roll back |
| 13:30.51 | *** join/#storm danilos (~danilo@canonical/launchpad/danilos) |
| 13:34.14 | *** join/#storm danilo_ (~danilo@conference/canonical/x-dospvearklfcvtkk) |
| 13:43.52 | *** join/#storm niemeyer (~niemeyer@conference/canonical/x-bgsmqjzxfdlklkds) |
| 13:53.40 | *** join/#storm stub (~stub@conference/canonical/x-rdbysfgghtrqiggh) |
| 13:53.40 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 13:57.03 | mgedmin | in the end it wasn't as painful as I feared |
| 14:12.03 | jkakar | :) |
| 14:17.38 | mgedmin | just got permission to share the code: http://pastie.org/private/pt5ol2mf9grt2pnsgzdrug |
| 14:18.07 | mgedmin | might be something worth including in the forthcoming Storm documentation, hint hint ;) |
| 14:19.08 | mgedmin | (with the caveats about <publisher priority=50> overriding zope's default XMLRPC and SOAP publishers, which may be unexpected... perhaps priority=15 would be a better value) |
| 14:21.07 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 14:21.13 | jkakar | mgedmin: Ooh, cool, thanks! That'd be a great thing to include in the manual. |
| 14:25.52 | *** join/#storm magmatt (~Matt@74-92-245-181-Utah.hfc.comcastbusiness.net) |
| 14:32.11 | mgedmin | (also, hi) |
| 14:32.16 | mgedmin | sucks at irc etiquette |
| 14:35.56 | jkakar | Heh |
| 14:38.27 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 14:55.18 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 15:02.37 | *** join/#storm danilos (~danilo@canonical/launchpad/danilos) |
| 15:05.47 | *** join/#storm niemeyer (~niemeyer@conference/canonical/x-gvubcygguhmeorte) |
| 15:13.00 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 15:30.43 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 15:45.28 | *** join/#storm jkakar (~jkakar@245.Red-88-5-215.dynamicIP.rima-tde.net) |
| 15:46.31 | *** part/#storm jkakar (~jkakar@245.Red-88-5-215.dynamicIP.rima-tde.net) |
| 15:46.35 | *** join/#storm jkakar (~jkakar@245.Red-88-5-215.dynamicIP.rima-tde.net) |
| 15:47.49 | *** join/#storm artista_frustrad (~artista_f@201.41.168.191) |
| 15:54.44 | *** join/#storm vokoda (~user@unaffiliated/vokoda) |
| 15:59.57 | *** join/#storm niemeyer (~niemeyer@conference/canonical/x-cerjsylepthzovrx) |
| 16:04.03 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 16:21.00 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 16:37.52 | *** join/#storm artista_frustrad (~artista_f@201.41.168.191) |
| 16:43.29 | *** join/#storm jdobrien (~john@50-88-19-25.res.bhn.net) |
| 16:48.20 | *** join/#storm keppla (~keppla@p4FD475B6.dip.t-dialin.net) |
| 17:13.29 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 17:24.36 | *** join/#storm stub (~stub@conference/canonical/x-cnvqaannujixelwk) |
| 17:24.39 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 17:30.41 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 17:48.31 | *** join/#storm john__ (~john@50-88-19-25.res.bhn.net) |
| 17:49.49 | *** join/#storm artista_frustrad (~artista_f@201.41.168.191) |
| 18:07.05 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 18:25.09 | *** join/#storm artista_frustrad (~artista_f@201.41.168.191) |
| 18:42.27 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 18:58.48 | *** join/#storm niemeyer (~niemeyer@89.127.177.74) |
| 19:03.26 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 19:21.11 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 19:24.59 | *** join/#storm stub (~stub@conference/canonical/x-opfxbxtgwlvoekhj) |
| 19:24.59 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 19:30.30 | *** join/#storm stub (~stub@canonical/launchpad/stub) |
| 19:38.32 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 19:56.50 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 20:13.35 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |
| 20:25.07 | magmatt | I'm getting a buffer back from sqlite when a Decimal was inserted by storm. https://gist.github.com/1054853 |
| 20:25.10 | magmatt | Why is that? |
| 20:28.00 | marienz | magmatt: sqlite itself doesn't know what "numeric" means |
| 20:28.44 | magmatt | right. . . so is it because storm is inserting a buffer? |
| 20:29.10 | magmatt | (easy enough to test) |
| 20:29.17 | magmatt | yup it is |
| 20:29.48 | magmatt | marienz: why was a buffer chosen over a string? |
| 20:29.56 | magmatt | for inserting decimals? |
| 20:29.58 | marienz | let me try to refresh my memory on what's going on here types-wise first |
| 20:30.37 | magmatt | grep buffer leads me only to databases/sqlite.py and variables.py |
| 20:30.59 | marienz | magmatt: actually, using the storm I have here I get (1, 0), (2, 0) |
| 20:31.21 | marienz | magmatt: are you running the code you pastebinned, and which storm and python are you running it with? |
| 20:38.44 | marienz | oh, this is actually fun |
| 20:39.46 | marienz | magmatt: if I interpret the sqlite documentation (http://www.sqlite.org/datatype3.html) and a quick experiment correctly, then for storm to actually store decimals reliably in an sqlite database you should declare that column as being text, not numeric, in your "create table" |
| 20:41.07 | marienz | magmatt: storm is converting decimals to text, because that can be done without losing precision. But "When text data is inserted into a NUMERIC column, the storage class of the text is converted to INTEGER or REAL if such conversion is lossless and reversible." |
| 20:41.27 | marienz | magmatt: unfortunately "SQLite considers the conversion to be lossless and reversible if the first 15 significant decimal digits of the number are preserved", which doesn't really match my own definition of "lossless" |
| 20:42.07 | marienz | magmatt: so if you just define the sqlite column to be "text" storm should do the right thing (convert python decimals to their string representation when storing and back to decimals when retrieving them) |
| 20:42.26 | marienz | obviously this assumes you actually want your decimals to be precise, which I'm assuming you do or you'd not be using them :) |
| 22:54.22 | *** join/#storm vokoda (~user@unaffiliated/vokoda) |
| 23:09.34 | lifeless | win 79 |
| 23:21.01 | magmatt | marienz: sorry, I got pulled in to meetings at 2:30 |
| 23:21.29 | marienz | shrug, irc |
| 23:21.42 | magmatt | I'm reading your research |
| 23:22.19 | marienz | basically I think your table definition should be different, but that's unrelated to what you're seeing, and I can't reproduce your actual problem |
| 23:22.36 | magmatt | I'm using storm 0.17 |
| 23:22.41 | magmatt | I can try with 18 |
| 23:23.09 | magmatt | well I can just change my schema definition â I use sqlite for testing and postgres for live stuff |
| 23:23.19 | magmatt | so if the tests are good enough, they're good enough |
| 23:27.25 | magmatt | I still get the buffer even when I use text in my definition |
| 23:27.28 | magmatt | let me try .18 |
| 23:28.12 | marienz | python version? |
| 23:28.20 | magmatt | 2.6.6 |
| 23:28.41 | magmatt | I updated the gist as well: https://gist.github.com/1054853 |
| 23:29.21 | magmatt | works as designed in .18 |
| 23:29.27 | magmatt | with the text attribute |
| 23:29.46 | marienz | fun |
| 23:30.18 | magmatt | marienz: thanks as always for your help |
| 23:31.02 | marienz | you should notice that with the column set as text it ends up as u'0.00' when you look at it using sqlite3 directly, and the precision of the decimals is actually transferred correctly |
| 23:31.43 | magmatt | I see |
| 23:32.55 | magmatt | changes numerics to text throughout |
| 23:32.59 | magmatt | k, chao |
| 23:43.42 | *** join/#storm artista_frustrad (~artista_f@201-41-168-191.ctaje701.dsl.brasiltelecom.net.br) |