IRC log of #zope for Friday, 2014-09-19

*** dixond has quit IRC00:28
*** dixond has joined #zope00:28
*** J1m_ has quit IRC00:46
*** J1m_ has joined #zope00:47
*** m8 has quit IRC00:51
*** Arfrever has quit IRC00:58
*** Pumukel has quit IRC01:06
*** benji has quit IRC01:13
*** kosh has joined #zope02:19
*** jhammons has quit IRC02:46
*** agroszer has quit IRC03:01
*** do3cc has quit IRC03:12
*** do3cc has joined #zope03:24
*** agroszer has joined #zope03:31
*** benji has joined #zope04:05
*** agroszer has quit IRC04:45
*** tiwula has joined #zope04:47
*** dixond has quit IRC05:15
*** dixond has joined #zope05:16
*** alecm has quit IRC05:21
*** alecm has joined #zope05:22
*** alecm has joined #zope05:22
*** menesis has joined #zope05:34
*** tiwula has quit IRC06:25
*** menesis has quit IRC06:53
*** alecm has quit IRC07:14
*** alecm has joined #zope07:15
*** alecm has joined #zope07:15
*** menesis has joined #zope07:27
*** tiwula has joined #zope07:44
*** Arfrever has joined #zope08:33
*** kosh has quit IRC09:03
*** avoinea has joined #zope09:21
*** tiwula has quit IRC09:27
*** alecm has quit IRC09:33
*** alecm has joined #zope09:34
*** Spanktar has joined #zope09:42
*** __mac__ has joined #zope09:46
*** Spanktar has quit IRC09:48
*** giacomos has joined #zope09:58
*** giacomos has joined #zope09:58
*** yvl has joined #zope09:59
*** tisto has joined #zope10:05
*** __mac__ has quit IRC10:44
*** __mac__ has joined #zope10:47
*** fredvd has joined #zope11:59
jakke1ok, I've been reading and reading and searching but haven't found a way yet12:29
jakke1is there a way to ask to the connection or transaction manager to give you the matching transaction for a given tid?12:30
jakke1something like: interesting_transaction = transaction.get(interesting_transaction_tid)12:30
*** agroszer has joined #zope12:47
*** agroszer has quit IRC12:48
*** agroszer has joined #zope12:48
*** fredvd has quit IRC13:29
*** menesis has quit IRC14:07
*** giacomos has quit IRC14:08
*** giacomos has joined #zope14:08
mgedminjakke1, why do you need that?14:15
mgedminI'm asking because it seems like a strange thing to do14:15
mgedminthe transaction package deals with transactions in progress14:16
mgedminI'm not sure they have tids before they're committed14:16
mgedmindo you want to inspect previously committed transactions?14:17
*** J1m_ has joined #zope14:35
jakke1mgedmin, because I'm having trouble with transactions :)14:49
jakke1we're having for some reason transactions that corrupt data14:49
jakke1we don't know what the trigger is14:49
jakke1I can identify the broken object, and the related transaction, but I want to know what else in that transaction is happening14:50
mgedminwe're talking about ZODB, right?14:50
mgedminFileStorage or something else?14:51
jakke1of course14:51
jakke1relstorage, but that doesn't matter that much14:51
mgedminwell then I can't suggest :)14:51
agroszerhey mgedmin14:51
mgedminhey agroszer14:51
jakke1we've used that, but only on an export14:51
mgedminI wanted to ping you sometime about the failing winbot task -- it runs something-reportlab-related on python 2.6 but needs to do so on 2.714:51
mgedminjakke1, I wrote zodbbrowser so I could see what's happening in my database; I tried to add some introspection into transaction records to it14:52
mgedminbut the APIs I found are rather inconvenient14:52
mgedminbasically you can iterate over all transaction records from the start14:52
mgedminthere's no better way to find a record with a given tid14:52
mgedminbut this is all based on filestorage14:53
agroszerpretty darn busy nowadays, you-know-why14:53
mgedminI don't know how relstorage keeps its data14:53
mgedminI'd be tempted to look at the SQL tables14:53
mgedminagroszer, sure14:53
mgedminagroszer, oh, I also wanted a deployment of my updated build status images; that at least should be trivial (if I knew where to cd and git pull on winbot)14:54
agroszerthat's easy, I'll do that now14:54
jakke1mgedmin: zodbbrowser is too superficial for what I need, I've already looked into it14:56
jakke1sqltables do the trick14:57
jakke1but it would be convenient to do it from the debug console14:57
jakke1cause I can get tot he broken object, fetch the tid, then fetch oid's for that related tid, and then I have to fetch all objects in the debug console14:58
agroszermgedmin, done14:58
jakke1if I could do that directly from the debug console, that would make my life a bit easier14:58
mgedminif you find a way, I want to know it too!14:58
jakke1I should do a blog post about this I suppose ;-)14:58
mgedminwell, with relstorage it might be easy14:58
jakke1don't know, haven't checked the relstorage api yet14:59
jakke1hoping for a non-storeage-specific solution14:59
mgedminwith filestorage you'd have to build an index of tid -> file position, or tid -> list of oids, because I don't think currently filestorage does that14:59
jakke1without looping over all transactions14:59
jakke1there already are some mapping classes, don't know exactly what they do though15:00
mgedminthey slice per object, I believe15:01
jakke1have to read some more code15:02
*** kiorky has quit IRC15:11
*** kiorky has joined #zope15:14
*** menesis has joined #zope15:17
*** fredvd has joined #zope15:19
J1m_mgedmin, the file storage iterator lets you specify a start tid.  It walks over transactions to get to the start, but that's faster than reading the database records themselves.15:23
mgedminright; my use case in zodbbrowser was "show me the last N transactions" with links going into the past; which is a completely wrong iteration order for the current APIs15:27
mgedminand it was a low-priority wishlist item so I never worked to improve the API (or build a transaction index myself)15:28
mgedminI think the filestorage format actually allows backwards traversal (i.e. record size is repeated at the end of each record)?15:28
jakke1just thinking of it, I think the Undo functionality should have some15:29
jakke1cause you can select and undo transactions15:29
jakke1probably also backwards, but worth a check15:29
*** kosh has joined #zope15:30
J1m_mgedmin, yes, and in fact the file-storage iterator will search from the back if it things the start tid is closer to the end.15:35
J1m_I agree that a reverese iterator would be nice and straightfoward.15:35
jakke1but no such thing as: here is a tid, please give me the transaction J1m_ ?15:38
mgedminoh, cool!15:40
mgedminit's just that I don't know (in the "last 10 transactions" use case) what the tids are :)15:40
jakke1mgedmin, you can get them from the undo log, if it's just the write transactions15:41
jakke1>>> conndb.undoInfo()15:42
jakke1if I remember correctly it takes an argument: last=-2015:42
jakke1    def undoInfo(self, first=0, last=-20, specification=None):15:44
jakke1but you'd have to convert the undo_tid to "normal" tid's15:44
J1m_jakke1, you get that more or less from the file-storage iterator.15:50
jakke1our dataset is quite big, like 10Gb datafs and 40Gb blobs15:52
jakke1won't that be terribly slow? that's why I haven't tried that path yet15:53
J1m_10G isn't very big.15:53
jakke1and it wouldn't work on relstorage ...15:53
jakke1which is our production environment15:53
J1m_I haven't studies the relstorage schema, but there is likelt an sql query that would give you what you need.15:55
jakke1yeah, I know15:55
jakke1but it's an annoying move back and forth between debug zope and sql15:55
jakke1fetch a broken object, get the tid, then get the matching oid's for the broken tid in sql, and fetch oids back from the debug instance15:56
jakke1I have found the procedure, just hoped there was a more straightforward way instead of this pingpong15:56
jakke1but relstorage also implements an itereator, there might be hope15:57
J1m_jakke1, you also might be able to use the before option in DB.open16:07
J1m_It alows time travel.16:07
jakke1and I do have the timestamp, so that could work as well16:08
*** yvl has quit IRC16:18
*** tisto has quit IRC16:40
*** agroszer has quit IRC17:09
*** stevegt_ has quit IRC17:10
*** stevegt_ has joined #zope17:15
*** agroszer has joined #zope17:42
*** kosh_ has joined #zope17:47
*** moo-_- has quit IRC17:48
*** kosh has quit IRC17:48
*** tiwula has joined #zope17:50
*** tiwula has quit IRC18:05
*** Arfrever has quit IRC18:05
*** aclark has quit IRC18:05
*** jakke1 has left #zope18:06
*** tiwula has joined #zope18:06
*** ree_ has joined #zope18:30
*** JT has joined #zope18:30
*** JT has joined #zope18:30
*** ajmitch_ has joined #zope18:31
*** Guest34626 has quit IRC18:31
*** kiorky has quit IRC18:31
*** ree has quit IRC18:31
*** ajmitch has quit IRC18:31
*** kiorky has joined #zope18:31
*** stevegt_ has quit IRC18:32
*** Zairex_ has quit IRC18:32
*** Zairex has joined #zope18:32
*** daMaestro has joined #zope18:45
*** kiorky has quit IRC18:46
mcdonchow did your zodb python dc meetup presentation go jim18:47
*** kiorky has joined #zope18:50
*** kosh_ has quit IRC19:06
*** __mac__ has quit IRC19:12
*** fredvd has quit IRC19:25
*** giacomos has quit IRC19:33
*** moo-_- has joined #zope19:45
*** stevegt_ has joined #zope19:49
*** aclark has joined #zope20:04
*** alecm has quit IRC21:09
*** alecm has joined #zope21:10
*** alecm has joined #zope21:10
*** alecm has quit IRC21:25
*** alecm has joined #zope21:25
*** giacomos has joined #zope21:48
*** alecm has quit IRC21:50
*** alecm has joined #zope21:52
*** avoinea has quit IRC22:04
J1m_mcdonc, it went very well.22:30
J1m_Nice crowd.22:31
J1m_There was someone from Foundation DB and we has some nice discussion comparing approaches taken.22:31
*** Arfrever has joined #zope22:45
*** stevegt_ has quit IRC23:12
*** agroszer has quit IRC23:36

Generated by 2.15.1 by Marius Gedminas - find it at!