| 00:47.14 | *** join/#storm oubiwann (n=oubiwann@c-24-8-208-199.hsd1.co.comcast.net) |
| 00:53.01 | *** join/#storm oubiwann (n=oubiwann@c-24-8-208-199.hsd1.co.comcast.net) |
| 01:56.56 | *** join/#storm oubiwann (n=oubiwann@c-24-8-208-199.hsd1.co.comcast.net) |
| 02:36.04 | *** join/#storm stub (n=stub@ppp-58-8-16-190.revip2.asianet.co.th) |
| 03:21.22 | jamesh | jrydberg: I'd do that one as: result = store.find((Host.data, Sum(Fact.bytes)), Host.id == Fact.host_id); result.group_by(Host.data) |
| 06:09.17 | *** join/#storm stub1 (n=stub@ppp-58-8-211-17.revip2.asianet.co.th) |
| 06:21.16 | *** part/#storm stub (n=stub@canonical/launchpad/stub) |
| 07:04.47 | *** join/#storm srichter (n=quassel@c-76-19-135-180.hsd1.ma.comcast.net) |
| 07:57.45 | *** join/#storm stub (n=stub@ppp-58-8-211-17.revip2.asianet.co.th) |
| 08:14.06 | jrydberg | jamesh: Mmm. Figured it out. |
| 08:26.09 | jrydberg | sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id -1210058560 and this is thread id -1223816304 |
| 08:26.21 | *** join/#storm jukart (i=lovely@81.189.156.94) |
| 08:26.29 | jrydberg | objects == the store? |
| 08:26.45 | jamesh | that error is from pysqlite; not storm |
| 08:26.53 | jamesh | objects == connections and cursors |
| 08:26.54 | jrydberg | yes |
| 08:27.15 | jrydberg | So I need to create the Store somewhere else than the main-thread? |
| 08:27.22 | jamesh | so yeah, Storm objects will use the relevant connection for certain operations |
| 08:27.40 | jamesh | (access to Reference attributes, resolving invalidated attributes, etc |
| 08:28.11 | jrydberg | i'm doing an insert that is deferred to a thread |
| 08:28.38 | jrydberg | so the Store is the only object created and accessed in the main thread |
| 08:28.39 | jamesh | defer your entire transaction to the thread |
| 08:28.57 | jamesh | and create the store in that thread |
| 08:29.09 | jrydberg | won't that make it slow? :) |
| 08:29.40 | jamesh | well, if you're using a thread pool and you manage one store per thread, not really |
| 08:30.22 | jrydberg | Not sure how to do that. I'm using twisted.internet.threads.deferToThread(transaction_function) |
| 08:30.57 | jamesh | so, for the code I showed you yesterday, we use zstorm to manage the connections |
| 08:31.26 | jamesh | from storm.zope.zstorm import global_zstorm as zstorm |
| 08:31.44 | jamesh | zstorm.set_default_uri("whatever", "postgres:dbname") |
| 08:32.01 | jamesh | then within the transaction function, do zstorm.get("whatever") to retrieve the store |
| 08:32.29 | jamesh | then use transaction.commit() or transaction.abort() to finish the transaction |
| 08:32.38 | jamesh | (zstorm takes care of registering the store with the transaction manager) |
| 08:33.15 | jrydberg | will try that, thansk |
| 08:38.51 | jamesh | if you weren't using sqlite, using the store in multiple threads would work, but I still wouldn't recommend it |
| 08:39.30 | jamesh | statements in a transaction are serialised, and many of the "one connection in multiple threads" designs I've seen seem to ignore that |
| 09:15.02 | *** join/#storm mikex2 (n=mikael@193.11.22.34) |
| 09:55.31 | *** join/#storm akm1 (n=akm1@122.166.46.113) |
| 10:29.29 | jrydberg | jamesh: what about the situation where I have multiple threads doing zstorm.get('same-name') ? |
| 10:29.39 | jrydberg | that will fuck up things too, i suppose :) |
| 10:29.46 | jamesh | jrydberg: they get different stores |
| 10:30.11 | jamesh | managing per-thread stores is the main purpose of the zstorm code |
| 10:30.51 | jrydberg | I see. cool. |
| 10:35.27 | jrydberg | jamesh: The model objects (something that has a __storm_table__ attribute) can be shared between threads? |
| 10:35.55 | jamesh | jrydberg: correct. instances are tied to stores rather than classes. |
| 10:36.23 | jamesh | this is also useful if you're building an app that makes use of sharding |
| 10:36.30 | jamesh | (same schema in multiple databases) |
| 10:56.04 | *** join/#storm niemeyer (n=niemeyer@200-203-57-249.pltce701.dsl.brasiltelecom.net.br) |
| 12:14.12 | *** join/#storm fcorrea (n=fcorrea@187.3.137.148) |
| 12:23.00 | *** join/#storm zmijunkie (n=chatzill@i59F5CEFA.versanet.de) |
| 12:31.14 | jrydberg | jamesh: http://bazaar.launchpad.net/%7Elaunchpad-pqm/launchpad/db-devel/annotate/head%3A/lib/canonical/librarian/db.py#L19 i'm a bit interested in that decorator. where can it be found? :) |
| 12:55.50 | jrydberg | i'll start without it |
| 13:18.33 | *** join/#storm sidnei (n=sidnei@plone/dreamcatcher) |
| 13:41.09 | *** join/#storm vvinet (n=vince@smtpin.revolutionlinux.com) |
| 13:43.56 | *** join/#storm andrea-bs (n=andrea@ubuntu/member/beeseek.developer.andrea-bs) |
| 14:20.13 | *** join/#storm bigdog (n=scmikes@72-197-8-8-arpa.cust.cinci.current.net) |
| 14:24.08 | jamesh | jrydberg: there shouldn't be any code missing from that branch, but I'm not sure why that reset_store thing is in there |
| 14:25.05 | jrydberg | found it a few levels up, but couldn't make sense of it really |
| 15:44.54 | *** join/#storm mikex2 (n=mikael@193.11.22.34) |
| 15:49.00 | *** join/#storm keppla (n=keppla@i577B081C.versanet.de) |
| 15:52.50 | *** join/#storm niemeyer (n=niemeyer@201-66-168-212.pltce701.dsl.brasiltelecom.net.br) |
| 16:04.18 | *** join/#storm keppla1 (n=keppla@i577B02A3.versanet.de) |
| 16:05.52 | *** join/#storm darkhorse` (n=darkhors@adsl-76-254-13-6.dsl.pltn13.sbcglobal.net) |
| 16:18.14 | *** join/#storm drudi (n=drudi@189.117.60.198) |
| 17:17.00 | *** join/#storm jukart (i=lovely@81.189.156.94) |
| 19:25.23 | *** join/#storm niemeyer (n=niemeyer@201-89-133-78.pltce701.dsl.brasiltelecom.net.br) |
| 19:50.16 | *** join/#storm seiflotfy2 (n=seif@P5118.pallas.wh.tu-darmstadt.de) |
| 20:20.15 | *** join/#storm jukart (n=jukart@d86-32-163-51.cust.tele2.at) |
| 20:39.33 | *** join/#storm jukart_ (i=lovely@81.189.156.94) |
| 21:01.40 | *** join/#storm jukart_ (n=jukart@d86-32-163-51.cust.tele2.at) |
| 21:08.48 | *** join/#storm vvinet (n=vince@modemcable041.205-203-24.mc.videotron.ca) |
| 22:55.11 | *** join/#storm shaunm (n=shaunm@c-98-212-133-244.hsd1.il.comcast.net) |
| 22:55.17 | *** join/#storm artista_frustrad (n=artista_@201-15-204-64.ctame704.dsl.brasiltelecom.net.br) |