IRC log of #zodb for Monday, 2013-06-17

*** JaRoel|4d has joined #zodb00:13
*** Ilja has joined #zodb00:42
Iljahi, is this chat alive?00:42
IljaI know it's most probably Sunday and people are off - but is it generally a populated chat? I'm looking for help/consultant with good understanding of ZODB to help fix issue with the way we've integrated twisted and ZODB01:10
Iljagetting hit by consistency errors and not sure "whom to blame"01:10
J1m_If you're using ZODB with twisted, then you almost certainly don't want to use the default thread-based tranasction manager.01:11
Iljahm, can you hint me with more keywords? we are using threadpool, actually01:12
J1m_as far as a consultant, you might contact agendaless and see if they offer any ZODB consulting.01:12
J1m_well, I know nothing of your architecture, so I was just guessing.01:13
Iljato put it short:
J1m_if you're only accessing ZODB from workers in your thread pool, then I can't think of any reason you'd run into trouble.01:13
Iljamethods that access zodb are decorated with @transact01:13
Iljait feels that it might be related to MVCC01:14
Iljaand connections not being 'cleaned up' properly01:14
J1m_I don't see any connection management in that code.01:16
J1m_No offence, but I'm not up for a deep dive in your code. :)01:16
J1m_if you can think of any questions ...01:17
Iljayes, sorry01:17
Iljaso, connection is essentially created once:
J1m_That's your problem.01:19
Iljaok, let me rephrase, it's a bit late here and my ordering of thoughts is a bit out of order :)01:19
J1m_Each thread needs its own connection.01:19
J1m_You should open the connection in your transact decorator.01:20
J1m_and close it.01:20
Iljahow expensive is this on average? we have quite a lot of functions yielded, each one in its own transact01:21
J1m_Better yet, use the db.transaction context manager.01:21
J1m_It's not expensive. Connections are pooled.01:22
J1m_Just keep in mind that each connection has it's own partial mirror of the database.01:22
J1m_so if you're accessing a lot of data, you'll want to limit the number of threads/connections.01:22
J1m_This is essentially the same as software transactional memory, except that the memory is also saved persistently.01:23
Iljahm, actually, we already have a single connection per thread, sorry for confusion.01:24
Iljai.e. connection is thread-local01:24
Iljaour internal discussion suggests that the warning in an old post / - might still be relevant01:25
IljaI don't want to pollute this chat with technical details, which might be specific to our design - (though imho a correct twisted + zodb integration example would be very-very useful)01:27
Iljaso will ping agendaless, thanks!01:27
J1m_k, good luck.01:29
*** Ilja has quit IRC01:44
*** J1m_ has quit IRC01:59
RixiMI am using gevent and I get this error: anyone know a fix?02:16
RixiMshould I be closing my connections or something?02:16
*** J1m_ has joined #zodb02:43
*** JaRoel|4d has quit IRC02:47
*** srichter__ is now known as srichter03:13
srichterRixiM: why do you think this is due to the ZODB?03:13
RixiMbecause that's the error in the paste.03:14
RixiMi mean... it's not zodb per se.03:14
RixiMit's how i was forking the workers.03:14
RixiMbut I don't know enough to even make intelligent statements about it.03:14
srichterI am not sure you can fork processes with open ZEO conenctions03:14
RixiMi think that is exactly what the problem was. when i stopped forking, i stopped getting that error.03:15
*** JaRoel|4d has joined #zodb03:18
*** JaRoel|4d has quit IRC03:26
*** J1m_ has quit IRC03:57
*** Ilja has joined #zodb07:45
*** Ilja has quit IRC07:56
*** Ilja has joined #zodb08:08
*** Ilja has quit IRC08:11
*** RixiM has left #zodb08:32
*** agroszer has joined #zodb09:43
*** agroszer has quit IRC10:01
*** agroszer has joined #zodb10:01
*** JaRoel|4d has joined #zodb10:13
*** agroszer has quit IRC10:23
*** agroszer has joined #zodb10:23
*** JaRoel has joined #zodb11:41
*** JaRoel|4d has quit IRC11:44
*** agroszer has quit IRC12:48
*** agroszer has joined #zodb12:49
*** fdrake has quit IRC13:31
*** srichter has quit IRC13:36
*** srichter has joined #zodb13:41
*** srichter has quit IRC14:37
*** J1m_ has joined #zodb14:44
*** J1m_ has quit IRC14:56
*** J1m_ has joined #zodb14:59
*** J1m_ has quit IRC15:04
*** fdrake has joined #zodb15:19
*** Ilja has joined #zodb16:10
*** J1m_ has joined #zodb16:14
*** J1m has joined #zodb16:17
*** Ilja has quit IRC17:22
*** agroszer has quit IRC17:24
*** Ilja has joined #zodb18:01
*** srichter has joined #zodb18:46
*** JaRoel has quit IRC18:51
*** Ilja has quit IRC19:25
*** Ilja has joined #zodb19:30
*** agroszer has joined #zodb20:06
*** Ilja has quit IRC20:49
*** agroszer has quit IRC20:51
*** Ilja has joined #zodb21:08
*** Ilja has quit IRC23:28

Generated by 2.15.1 by Marius Gedminas - find it at!