| 00:00.44 | *** join/#storm wallflower (n=wallflow@ip-205-246-113-216.pool.grokthis.net) |
| 01:58.15 | *** join/#storm oubiwann (n=oubiwann@97-119-4-118.omah.qwest.net) |
| 04:39.15 | *** join/#storm nullie (n=ilya@87.254.139.66) |
| 04:44.07 | *** join/#storm jukart (n=jukart@85-124-221-45.static.xdsl-line.inode.at) |
| 05:28.37 | nullie | humm... do canonical recommend any web framework? |
| 05:29.01 | *** join/#storm jukart (n=jukart@lsfw01.lovelysystems.com) |
| 07:15.14 | thumper | haha |
| 07:15.24 | thumper | nullie: there is a mythical framework |
| 07:18.42 | *** join/#storm andrea-bs (n=andrea@ubuntu/member/beeseek.developer.andrea-bs) |
| 07:24.36 | *** join/#storm goschtl (n=goschtl@p5B0BD5B2.dip.t-dialin.net) |
| 07:43.06 | nullie | thumper, mythical? |
| 07:43.39 | *** join/#storm ubot3 (n=ubot3@unaffiliated/nalioth/bot/ubot3) |
| 09:02.50 | *** join/#storm andrea-bs (n=andrea@ubuntu/member/beeseek.developer.andrea-bs) |
| 09:39.31 | jkakar | nullie: Launchpad and Landscape are Zope3-based applications, if that helps. |
| 09:40.05 | nullie | oh |
| 09:53.57 | jamesh | hi jkakar |
| 09:56.32 | jkakar | Hi jamesh! |
| 09:57.28 | jamesh | I fixed my ResultSet.__contains__() impl to handle a few more cases, but it still doesn't handle set ops well |
| 09:59.38 | jkakar | Cool. Do you see a path to handling set ops well? |
| 10:00.25 | jkakar | I haven't looked at the branch, so I'm not entirely sure what the issues are. |
| 10:10.06 | jamesh | It'll probably require rewriting the query |
| 10:10.40 | jamesh | while PostgreSQL can handle the single-table set ops in my current branch, it can't handle the other cases |
| 10:10.56 | jamesh | and the fix for mysql/sqlite will probably be better for PostgreSQL too |
| 10:24.09 | *** join/#storm andrea-bs (n=andrea@ubuntu/member/beeseek.developer.andrea-bs) |
| 10:32.14 | *** join/#storm olejorgenb (n=olejorge@ip-165-23-149-91.dialup.ice.no) |
| 11:47.16 | jamesh | weird: http://paste.ubuntu.com/23286/ |
| 11:47.28 | jamesh | mysql seems to like the UNION but not the INTERSECT |
| 11:49.35 | jamesh | ah. other tests exit early before testing that with MySQL |
| 12:15.15 | *** join/#storm niemeyer (n=niemeyer@200-138-40-171.ctame705.dsl.brasiltelecom.net.br) |
| 12:31.49 | jamesh | hi niemeyer |
| 12:39.20 | niemeyer | jamesh: Hey, good evening! |
| 12:43.40 | jamesh | niemeyer: I came up with an implementation of __contains__ for set expressions that works on the other DBs: http://paste.ubuntu.com/23297/ |
| 12:44.22 | jamesh | the one case it doesn't handle is different-but-compatible expressions in finds |
| 12:44.43 | jamesh | e.g. x in store.find(Foo.title).union(store.find(Bar.title)) |
| 12:49.06 | jamesh | the above code would treat it the same as store.find(Foo.title).union(store.find(Foo.title)) -- using the first find spec |
| 12:53.48 | niemeyer | jamesh: Hmm |
| 12:53.56 | niemeyer | jamesh: Is there a way we can detect it, and raise an error? |
| 12:54.26 | jamesh | niemeyer: well, if we changed is_compatible() to flag cases like that as incompatible the problem would go away |
| 12:54.44 | jamesh | the problem is that checking if two expressions are equal is difficult |
| 12:54.47 | niemeyer | jamesh: This isn't a backwards compatibility issue, since we've never supported expressions in previous releases |
| 12:54.52 | jamesh | right |
| 12:54.52 | niemeyer | jamesh: I see |
| 12:55.06 | jamesh | "expr == expr" always returns a truth value |
| 12:55.25 | niemeyer | jamesh: We just test with "is" in this case |
| 12:55.59 | niemeyer | jamesh: It isn't bullet proof, arguably, since it is possible that two columns are built independently, but it's usually what's intended |
| 12:57.25 | jamesh | I'll need to check if that covers LP uses (it is probably okay) |
| 12:59.19 | *** join/#storm y_sheidia (n=galih@125.164.224.144) |
| 12:59.23 | jamesh | looks like they are all safe |
| 13:00.07 | jamesh | niemeyer: so, I believe this now covers all __contains__() cases. |
| 13:01.08 | niemeyer | jamesh: That's awesome! Thanks for taking the time to check these out |
| 13:01.25 | jamesh | actually, the "order by" with limit/offset problem probably applies to selects as well as set expressions: should probably handle that too |
| 13:02.13 | jamesh | actually, it'll probably get handled by the DB as a bung query |
| 13:02.25 | *** join/#storm bigdo1 (n=scmikes@72-197-8-8-arpa.cust.cinci.current.net) |
| 13:02.47 | jamesh | it'd only be an issue for ORDER BY using an aliased column that we aren't selecting |
| 13:12.35 | jamesh | niemeyer: I guess this codepath will also be needed for result sets using therve's grou-by support |
| 13:13.28 | niemeyer | jamesh: Hmm, what's the limit/offset/order by problem |
| 13:13.30 | niemeyer | ? |
| 13:14.29 | jamesh | niemeyer: if you have an ORDER BY on a union, it will be referencing some columns from the subexpressions |
| 13:15.17 | jamesh | if we are replacing the columns in those subexpressions then the order by expression will probably break |
| 13:15.45 | jamesh | the only case where this really matters though is if you also have a limit or offset clause for the union |
| 13:16.02 | jamesh | in other cases we don't care what order the union is in for __contains__() |
| 13:16.48 | niemeyer | jamesh: Right |
| 13:16.59 | niemeyer | jamesh: I still missing some context.. are we doing expression replacement now? |
| 13:17.42 | niemeyer | jamesh: I mean, are not subexpressions mangled somehow to get in the contains path? |
| 13:17.44 | jamesh | niemeyer: for __contains__() where self._select is not Undef, yes. |
| 13:17.56 | niemeyer | checks the diff |
| 13:22.40 | niemeyer | jamesh: Wow, it's a bit sad how much work it has to do |
| 13:25.05 | jamesh | niemeyer: do you think it'd be cleaner to use the copy module to duplicate the Select()? |
| 13:26.51 | niemeyer | jamesh: Might be less error prone (considering future versions of Select) |
| 13:27.18 | niemeyer | jamesh: So what's the aliasing about? |
| 13:27.59 | jamesh | niemeyer: a way to reference the column names from outside the set expression |
| 13:28.39 | jamesh | for single table find(), I could do it in Postgres with "SELECT ... FROM (set expr) tablename WHERE tablename.id = ..." |
| 13:28.57 | jamesh | but that doesn't work on sqlite and mysql, and doesn't work if you are selecting multiple tables or expressions |
| 13:30.05 | jamesh | so the new code adjusts the subquery to select just the columns we want, and aliases that can be used in the outer query |
| 13:33.31 | niemeyer | jamesh: Hmmm, ok |
| 13:36.59 | niemeyer | jamesh: I guess I understand the logic now, after your explanations and staring at the code for some time |
| 13:37.19 | *** join/#storm kov (n=kov@189.12.118.205) |
| 13:38.26 | niemeyer | jamesh: I'm trying to imagine a way to prevent the column replacement, but keeping the same logic overall |
| 13:38.59 | jamesh | niemeyer: some form of replacement is necessary to add aliases |
| 13:41.14 | niemeyer | jamesh: Do you know if EXISTS is present in MySQL/SQLite? |
| 13:41.48 | niemeyer | Hmm, I guess they might not be able to optimize the query appropriately |
| 13:42.55 | jamesh | niemeyer: are you thinking of pushing the where clause down into the subqueries? |
| 13:43.36 | jamesh | that won't cover __contains__ on a GROUP BY query (once we support that) |
| 13:44.10 | niemeyer | jamesh: Nope, just trying to come up with a way to prevent knowing the names |
| 13:44.18 | niemeyer | jamesh: a, b, c IN (...) is one way |
| 13:44.43 | niemeyer | jamesh: I'm afraid it might not be optimized properly |
| 13:45.38 | *** part/#storm y_sheidia (n=galih@125.164.224.144) |
| 13:46.12 | jamesh | "select (1, 'name') in (select id, name from person)" is a syntax error in sqlite |
| 13:47.13 | jamesh | but okay in mysql and postgresql |
| 13:47.18 | niemeyer | :-( |
| 13:47.53 | niemeyer | jamesh: Do you know if the two that work optimize it appropriately? |
| 13:48.02 | niemeyer | jamesh: (or if it's still a O(n) operation) |
| 13:48.52 | jamesh | niemeyer: note that you still need to replace the columns of the subselects to use IN |
| 13:48.58 | *** join/#storm mcella (n=mcella@ip-176-239.sn2.eutelia.it) |
| 13:49.17 | niemeyer | jamesh: Really? |
| 13:49.17 | jamesh | since you only want to compare primary keys and expressions |
| 13:49.34 | niemeyer | jamesh: Ah, of course.. :-( |
| 13:49.49 | niemeyer | Well, ok, looks great then |
| 13:50.12 | niemeyer | It's sad that we have to do that, but such is life.. |
| 13:51.07 | jamesh | I noticed Kevin's storm-variable-repr branch too. It looks like a useful debugging aid, but interacts badly with lazy values |
| 13:51.14 | niemeyer | jamesh: Are you using preset aliases to stabilize the query? |
| 13:51.38 | jamesh | niemeyer: yeah. It'd work without them, but stable queries make QA a lot easier |
| 13:51.45 | niemeyer | jamesh: Sweet, ok |
| 13:52.11 | niemeyer | jamesh: Would you mind to use _tmp for the table alias? |
| 13:52.17 | jamesh | sure. |
| 13:52.43 | niemeyer | jamesh: Just so that we know where the alias is coming from when we see a _<whatever> coming by in debugs |
| 13:53.40 | niemeyer | jamesh: +1 on the changes |
| 13:53.48 | *** join/#storm bigdo2 (n=scmikes@72-197-8-8-arpa.cust.cinci.current.net) |
| 13:54.01 | niemeyer | jamesh: It'd be good to have more comments somewhere explaining the tricks we're using and why |
| 13:54.16 | niemeyer | jamesh: Feel free to leave them out for now, as you're probably tired from it |
| 13:54.23 | niemeyer | jamesh: I can add them after the branch is in if you want |
| 13:54.33 | jamesh | okay |
| 14:51.11 | *** part/#storm goschtl (n=goschtl@p5B0BD5B2.dip.t-dialin.net) |
| 15:26.59 | *** join/#storm olejorgenb (n=olejorge@ip-165-23-149-91.dialup.ice.no) |
| 15:36.36 | *** join/#storm jkakar (n=jkakar@nsabfw1.nsab.se) |
| 15:53.40 | *** join/#storm nullie (n=ilya@217.118.83.1) |
| 16:36.23 | weatherman | storm/resultset-groupby-2 r296 committed by thomas@canonical.com |
| 16:36.23 | weatherman | Merge from trunk. |
| 16:44.53 | weatherman | storm/resultset-groupby-2 r297 committed by thomas@canonical.com |
| 16:44.53 | weatherman | Fix failures related to merge: missing imports, missing slot. |
| 16:51.14 | *** join/#storm nullie (n=ilya@217.118.83.1) |
| 16:51.52 | weatherman | storm/resultset-groupby-2 r298 committed by thomas@canonical.com |
| 16:51.52 | weatherman | Add a note in NEWS file. |
| 16:53.52 | weatherman | storm/trunk r250 committed by thomas@canonical.com |
| 16:53.52 | weatherman | Merge resultset-groupby-2 [f=233946] [r=jamesh,radix,niemeyer] |
| 16:53.57 | weatherman | Add group_by and having methods on the ResultSet object, giving access to the |
| 16:54.01 | weatherman | GROUP BY and HAVING features after calling store.find(). |
| 16:54.07 | therve | it's in! |
| 16:55.31 | niemeyer | Woohay! |
| 17:18.09 | *** join/#storm nullguid (n=ilya@217.118.83.1) |
| 18:39.33 | *** join/#storm andrea-bs (n=andrea@ubuntu/member/beeseek.developer.andrea-bs) |
| 21:29.37 | *** join/#storm pyCube_ (n=jmayfiel@12.87.213.22) |
| 21:31.19 | pyCube_ | hmm.. playing with storm and a db that does funky stuff db side.. when you insert into a table, its pri key column gets updated by some stored proc or something.. the point is, i can insert to table with storm.. when i try a flush or commit, it gives me "storm.exceptions.LostObjectError: Can't obtain values from the database (object got removed?)" |
| 21:32.19 | pyCube_ | the select it generates (debug is on) says its trying to query out the entry with a 0 for the prikey/id col |
| 21:32.30 | radix | pyCube_: right, because storm tries to figure out the identity of the object it just inserted |
| 21:32.35 | radix | pyCube_: which database is it, btw? |
| 21:32.40 | pyCube_ | mysql |
| 21:32.43 | radix | ok |
| 21:33.18 | radix | pyCube_: does the database use the value you try to insert as the primary key as input in the procedure to generate the new primary key? |
| 21:33.24 | radix | or does it just generate one completely on its own? |
| 21:34.14 | pyCube_ | its keeps a global toc table..pri key value comes from there |
| 21:34.18 | radix | pyCube_: ok |
| 21:34.34 | radix | pyCube_: so, if you create the object without setting any value as the primary key, it might work |
| 21:34.39 | radix | pyCube_: like, don't assign to the attribute at all |
| 21:34.54 | pyCube_ | thats what i am doing |
| 21:35.50 | radix | pyCube_: hmm, ok, maybe it's worse then |
| 21:35.59 | radix | pyCube_: it's probably even confusing the mysql client library |
| 21:36.12 | radix | pyCube_: storm relies on cursor.lastrowid |
| 21:39.57 | *** join/#storm jbot (i=ibot@pdpc/supporter/active/TimRiker/bot/apt) |
| 21:39.57 | *** topic/#storm is The Storm Python ORM - http://storm.canonical.com/ - 0.12 released! || Review branches: https://code.launchpad.net/~storm/storm/trunk/+landing-candidates |
| 21:50.19 | pyCube_ | hm |
| 22:08.20 | niemeyer | pyCube_: What is the trigger doing? I bet we can find a way to make Storm do the right thing, at least in recent-enough MySQL versions |
| 22:15.33 | pyCube_ | well, before the insert it inserts into a toc table, then passes the id from that toc entry into the insert of the thing you actually meant to insert.. heh |
| 22:28.48 | pyCube_ | it uses LAST_INSERT_ID() after the toc insert to set the original inserts value for the prikey col |
| 22:43.28 | niemeyer | pyCube_: Somehow the logic seems to be breaking the lastrowid assumption Storm makes (that lastrowid points to the last inserted element) |
| 22:43.45 | niemeyer | pyCube_: If you can provide a small self-contained test case, I can probably come up with a fix for it |
| 22:53.16 | niemeyer | => dinner |