| *** dixond has quit IRC | 00:28 | |
| *** dixond has joined #zope | 00:28 | |
| *** J1m_ has quit IRC | 00:46 | |
| *** J1m_ has joined #zope | 00:47 | |
| *** m8 has quit IRC | 00:51 | |
| *** Arfrever has quit IRC | 00:58 | |
| *** Pumukel has quit IRC | 01:06 | |
| *** benji has quit IRC | 01:13 | |
| *** kosh has joined #zope | 02:19 | |
| *** jhammons has quit IRC | 02:46 | |
| *** agroszer has quit IRC | 03:01 | |
| *** do3cc has quit IRC | 03:12 | |
| *** do3cc has joined #zope | 03:24 | |
| *** agroszer has joined #zope | 03:31 | |
| *** benji has joined #zope | 04:05 | |
| *** agroszer has quit IRC | 04:45 | |
| *** tiwula has joined #zope | 04:47 | |
| *** dixond has quit IRC | 05:15 | |
| *** dixond has joined #zope | 05:16 | |
| *** alecm has quit IRC | 05:21 | |
| *** alecm has joined #zope | 05:22 | |
| *** alecm has joined #zope | 05:22 | |
| *** menesis has joined #zope | 05:34 | |
| *** tiwula has quit IRC | 06:25 | |
| *** menesis has quit IRC | 06:53 | |
| *** alecm has quit IRC | 07:14 | |
| *** alecm has joined #zope | 07:15 | |
| *** alecm has joined #zope | 07:15 | |
| *** menesis has joined #zope | 07:27 | |
| *** tiwula has joined #zope | 07:44 | |
| *** Arfrever has joined #zope | 08:33 | |
| *** kosh has quit IRC | 09:03 | |
| *** avoinea has joined #zope | 09:21 | |
| *** tiwula has quit IRC | 09:27 | |
| *** alecm has quit IRC | 09:33 | |
| *** alecm has joined #zope | 09:34 | |
| *** Spanktar has joined #zope | 09:42 | |
| *** __mac__ has joined #zope | 09:46 | |
| *** Spanktar has quit IRC | 09:48 | |
| *** giacomos has joined #zope | 09:58 | |
| *** giacomos has joined #zope | 09:58 | |
| *** yvl has joined #zope | 09:59 | |
| *** tisto has joined #zope | 10:05 | |
| *** __mac__ has quit IRC | 10:44 | |
| *** __mac__ has joined #zope | 10:47 | |
| *** fredvd has joined #zope | 11:59 | |
| jakke1 | ok, I've been reading and reading and searching but haven't found a way yet | 12:29 |
|---|---|---|
| jakke1 | is there a way to ask to the connection or transaction manager to give you the matching transaction for a given tid? | 12:30 |
| jakke1 | something like: interesting_transaction = transaction.get(interesting_transaction_tid) | 12:30 |
| *** agroszer has joined #zope | 12:47 | |
| *** agroszer has quit IRC | 12:48 | |
| *** agroszer has joined #zope | 12:48 | |
| *** fredvd has quit IRC | 13:29 | |
| *** menesis has quit IRC | 14:07 | |
| *** giacomos has quit IRC | 14:08 | |
| *** giacomos has joined #zope | 14:08 | |
| mgedmin | jakke1, why do you need that? | 14:15 |
| mgedmin | I'm asking because it seems like a strange thing to do | 14:15 |
| mgedmin | the transaction package deals with transactions in progress | 14:16 |
| mgedmin | I'm not sure they have tids before they're committed | 14:16 |
| mgedmin | do you want to inspect previously committed transactions? | 14:17 |
| *** J1m_ has joined #zope | 14:35 | |
| jakke1 | mgedmin, because I'm having trouble with transactions :) | 14:49 |
| jakke1 | we're having for some reason transactions that corrupt data | 14:49 |
| jakke1 | we don't know what the trigger is | 14:49 |
| jakke1 | I can identify the broken object, and the related transaction, but I want to know what else in that transaction is happening | 14:50 |
| mgedmin | we're talking about ZODB, right? | 14:50 |
| mgedmin | FileStorage or something else? | 14:51 |
| jakke1 | of course | 14:51 |
| jakke1 | relstorage, but that doesn't matter that much | 14:51 |
| mgedmin | ah | 14:51 |
| mgedmin | well then I can't suggest fsdump.py :) | 14:51 |
| agroszer | hey mgedmin | 14:51 |
| mgedmin | hey agroszer | 14:51 |
| jakke1 | we've used that, but only on an export | 14:51 |
| mgedmin | I wanted to ping you sometime about the failing winbot task -- it runs something-reportlab-related on python 2.6 but needs to do so on 2.7 | 14:51 |
| mgedmin | jakke1, I wrote zodbbrowser so I could see what's happening in my database; I tried to add some introspection into transaction records to it | 14:52 |
| agroszer | z3c.pdftemplate... | 14:52 |
| mgedmin | but the APIs I found are rather inconvenient | 14:52 |
| mgedmin | basically you can iterate over all transaction records from the start | 14:52 |
| mgedmin | there's no better way to find a record with a given tid | 14:52 |
| mgedmin | but this is all based on filestorage | 14:53 |
| agroszer | pretty darn busy nowadays, you-know-why | 14:53 |
| mgedmin | I don't know how relstorage keeps its data | 14:53 |
| mgedmin | I'd be tempted to look at the SQL tables | 14:53 |
| mgedmin | agroszer, sure | 14:53 |
| mgedmin | agroszer, oh, I also wanted a deployment of my updated build status images; that at least should be trivial (if I knew where to cd and git pull on winbot) | 14:54 |
| agroszer | that's easy, I'll do that now | 14:54 |
| jakke1 | mgedmin: zodbbrowser is too superficial for what I need, I've already looked into it | 14:56 |
| mgedmin | right | 14:56 |
| jakke1 | sqltables do the trick | 14:57 |
| jakke1 | but it would be convenient to do it from the debug console | 14:57 |
| jakke1 | cause I can get tot he broken object, fetch the tid, then fetch oid's for that related tid, and then I have to fetch all objects in the debug console | 14:58 |
| agroszer | mgedmin, done | 14:58 |
| jakke1 | if I could do that directly from the debug console, that would make my life a bit easier | 14:58 |
| mgedmin | if you find a way, I want to know it too! | 14:58 |
| jakke1 | I should do a blog post about this I suppose ;-) | 14:58 |
| mgedmin | well, with relstorage it might be easy | 14:58 |
| jakke1 | don't know, haven't checked the relstorage api yet | 14:59 |
| jakke1 | hoping for a non-storeage-specific solution | 14:59 |
| mgedmin | with filestorage you'd have to build an index of tid -> file position, or tid -> list of oids, because I don't think currently filestorage does that | 14:59 |
| jakke1 | without looping over all transactions | 14:59 |
| jakke1 | there already are some mapping classes, don't know exactly what they do though | 15:00 |
| mgedmin | they slice per object, I believe | 15:01 |
| jakke1 | have to read some more code | 15:02 |
| *** kiorky has quit IRC | 15:11 | |
| *** kiorky has joined #zope | 15:14 | |
| *** menesis has joined #zope | 15:17 | |
| *** fredvd has joined #zope | 15:19 | |
| J1m_ | mgedmin, the file storage iterator lets you specify a start tid. It walks over transactions to get to the start, but that's faster than reading the database records themselves. | 15:23 |
| mgedmin | right; my use case in zodbbrowser was "show me the last N transactions" with links going into the past; which is a completely wrong iteration order for the current APIs | 15:27 |
| mgedmin | and it was a low-priority wishlist item so I never worked to improve the API (or build a transaction index myself) | 15:28 |
| mgedmin | I think the filestorage format actually allows backwards traversal (i.e. record size is repeated at the end of each record)? | 15:28 |
| jakke1 | just thinking of it, I think the Undo functionality should have some | 15:29 |
| jakke1 | cause you can select and undo transactions | 15:29 |
| jakke1 | probably also backwards, but worth a check | 15:29 |
| *** kosh has joined #zope | 15:30 | |
| J1m_ | mgedmin, yes, and in fact the file-storage iterator will search from the back if it things the start tid is closer to the end. | 15:35 |
| J1m_ | I agree that a reverese iterator would be nice and straightfoward. | 15:35 |
| jakke1 | but no such thing as: here is a tid, please give me the transaction J1m_ ? | 15:38 |
| mgedmin | oh, cool! | 15:40 |
| mgedmin | it's just that I don't know (in the "last 10 transactions" use case) what the tids are :) | 15:40 |
| jakke1 | mgedmin, you can get them from the undo log, if it's just the write transactions | 15:41 |
| jakke1 | >>> conndb.undoInfo() | 15:42 |
| jakke1 | if I remember correctly it takes an argument: last=-20 | 15:42 |
| jakke1 | def undoInfo(self, first=0, last=-20, specification=None): | 15:44 |
| jakke1 | but you'd have to convert the undo_tid to "normal" tid's | 15:44 |
| J1m_ | jakke1, you get that more or less from the file-storage iterator. | 15:50 |
| jakke1 | our dataset is quite big, like 10Gb datafs and 40Gb blobs | 15:52 |
| jakke1 | won't that be terribly slow? that's why I haven't tried that path yet | 15:53 |
| J1m_ | 10G isn't very big. | 15:53 |
| jakke1 | and it wouldn't work on relstorage ... | 15:53 |
| jakke1 | which is our production environment | 15:53 |
| J1m_ | <shrug> | 15:54 |
| J1m_ | I haven't studies the relstorage schema, but there is likelt an sql query that would give you what you need. | 15:55 |
| jakke1 | yeah, I know | 15:55 |
| jakke1 | but it's an annoying move back and forth between debug zope and sql | 15:55 |
| jakke1 | fetch a broken object, get the tid, then get the matching oid's for the broken tid in sql, and fetch oids back from the debug instance | 15:56 |
| jakke1 | I have found the procedure, just hoped there was a more straightforward way instead of this pingpong | 15:56 |
| jakke1 | but relstorage also implements an itereator, there might be hope | 15:57 |
| J1m_ | jakke1, you also might be able to use the before option in DB.open | 16:07 |
| J1m_ | It alows time travel. | 16:07 |
| jakke1 | and I do have the timestamp, so that could work as well | 16:08 |
| *** yvl has quit IRC | 16:18 | |
| *** tisto has quit IRC | 16:40 | |
| *** agroszer has quit IRC | 17:09 | |
| *** stevegt_ has quit IRC | 17:10 | |
| *** stevegt_ has joined #zope | 17:15 | |
| *** agroszer has joined #zope | 17:42 | |
| *** kosh_ has joined #zope | 17:47 | |
| *** moo-_- has quit IRC | 17:48 | |
| *** kosh has quit IRC | 17:48 | |
| *** tiwula has joined #zope | 17:50 | |
| *** tiwula has quit IRC | 18:05 | |
| *** Arfrever has quit IRC | 18:05 | |
| *** aclark has quit IRC | 18:05 | |
| *** jakke1 has left #zope | 18:06 | |
| *** tiwula has joined #zope | 18:06 | |
| *** ree_ has joined #zope | 18:30 | |
| *** JT has joined #zope | 18:30 | |
| *** JT has joined #zope | 18:30 | |
| *** ajmitch_ has joined #zope | 18:31 | |
| *** Guest34626 has quit IRC | 18:31 | |
| *** kiorky has quit IRC | 18:31 | |
| *** ree has quit IRC | 18:31 | |
| *** ajmitch has quit IRC | 18:31 | |
| *** kiorky has joined #zope | 18:31 | |
| *** stevegt_ has quit IRC | 18:32 | |
| *** Zairex_ has quit IRC | 18:32 | |
| *** Zairex has joined #zope | 18:32 | |
| *** daMaestro has joined #zope | 18:45 | |
| *** kiorky has quit IRC | 18:46 | |
| mcdonc | how did your zodb python dc meetup presentation go jim | 18:47 |
| *** kiorky has joined #zope | 18:50 | |
| *** kosh_ has quit IRC | 19:06 | |
| *** __mac__ has quit IRC | 19:12 | |
| *** fredvd has quit IRC | 19:25 | |
| *** giacomos has quit IRC | 19:33 | |
| *** moo-_- has joined #zope | 19:45 | |
| *** stevegt_ has joined #zope | 19:49 | |
| *** aclark has joined #zope | 20:04 | |
| *** alecm has quit IRC | 21:09 | |
| *** alecm has joined #zope | 21:10 | |
| *** alecm has joined #zope | 21:10 | |
| *** alecm has quit IRC | 21:25 | |
| *** alecm has joined #zope | 21:25 | |
| *** giacomos has joined #zope | 21:48 | |
| *** alecm has quit IRC | 21:50 | |
| *** alecm has joined #zope | 21:52 | |
| *** avoinea has quit IRC | 22:04 | |
| J1m_ | mcdonc, it went very well. | 22:30 |
| J1m_ | Nice crowd. | 22:31 |
| J1m_ | There was someone from Foundation DB and we has some nice discussion comparing approaches taken. | 22:31 |
| J1m_ | s/has/had | 22:32 |
| *** Arfrever has joined #zope | 22:45 | |
| *** stevegt_ has quit IRC | 23:12 | |
| *** agroszer has quit IRC | 23:36 | |
Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!