IRC log of #zope3-dev for Saturday, 2010-03-06

J1mSorry, I don't have context.00:00
run|awaylimiting the picklecache by byte-size00:00
J1mWhat byte-size-gc issue?00:00
run|awaydoes not work00:00
J1mOh, the symptom where all of a sudden it evicts everything?00:01
run|awaythis is fixed (in our tests) by setting new_size to a int00:01
J1mHm, as opposed to what?00:01
* J1m looks00:01
run|awayunsigned int00:02
run|awayjust a simple oversight.. which took us forever to fix ;)00:02
J1mHm, so I guess the question is, why would it go negative.00:03
J1miow, why would it underflow.00:03
run|awayit doesnt00:03
run|awayi need to get someone else online to tell you00:03
J1mI assume that it would go negative if it was signed, but now is underflowing to something ginormous.00:04
*** pcardune has quit IRC00:08
run|awaypyarg_parsetuple the new_size is a signed python integer?00:08
run|awayin cc_update_object_size_estimation00:08
*** r0ver has joined #zope3-dev00:10
*** timte has joined #zope3-dev00:11
Vqtimte: godafton00:17
*** lisppaste6 has quit IRC00:18
timteVq: afton00:19
*** lisppaste6 has joined #zope3-dev00:28
* mgedmin not happy with buildbout-generated bin/python that doesn't know any of the command line options of the real python interpreter -- how do I do -W error?00:29
*** benji has quit IRC00:29
mgedminthere doesn't seem to be an environment variable for that :/00:29
*** allisterb has quit IRC00:34
run|awayJ1m, if you want we could diagnosis it more but its hairy (I know "it works for us" isnt very satisfactory)00:36
*** fcorrea has joined #zope3-dev00:36
*** allisterb has joined #zope3-dev00:37
J1mrun|away, PyArg_ParseTuple is using an unsigned int flag.00:44
J1mrun|away, if you could just tell me a reliable (and sane :) way to reproduce the problem, that would help enormously.00:44
J1mWe've only seen it in high duress production situations and have run away (so to speak :) from that option.00:45
run|awayunfortunately sane isnt really in the equation00:45
J1mI guess though that what you say might give me a clue of things I can try.00:46
*** cbcunc has quit IRC00:47
run|awayrunning plone buildout, sprinkle printf in picklecache, ab - shows the problem. change from unsigned to sign, recompile, repeat fixes it00:47
run|awaybut that isnt useful for you - i know00:47
run|awaywe arent using ab but some complicated load testing rig00:48
run|awaybut you can see it w/ ab00:48
run|awayso what we have seen00:48
J1mI can deal with running a plone buildout. :)00:48
run|awayoh cool00:48
J1mI can even deal with ab.00:48
run|awaywell i thought you would want something more isolated00:48
J1mI would :)00:48
J1mBut I'm willing to deal with less than what I want.00:49
run|awayi appreciate that00:49
J1mespecially if I get good instructions.00:49
run|awayi will sit here with you until you repro and see fix00:49
run|awaysvn co and run buildout00:50
run|awaywrong one00:52
run|awaywe are using patchs.. you wont get zodb 3.9 w/ earlier zope200:52
run|awayplease use 4.000:52
run|awayhanno repro'ed it on python2.6 64bit00:52
*** hannosch has joined #zope3-dev00:53
J1mso 64 bit ubuntu is ok?00:53
run|awayi presume so. we are using 64bit windows with python2.4 (32bit)00:53
hannoschheya, I'm trying to write a damn unit test for this, but having a bit of a hard time with it :)00:54
run|awayso J1m here is how we tested it.. we put printf in cPickleCache.c: scan_gc_items inside the while( here!=&before..) we printf total_estimated_size, target_bytes00:56
run|awayin parts/instance/zope.conf change cache-size to 99999 and cache-byte-size to 20000000000:57
run|awaynon_ghost_count < target & you can see total_estimated_size < target_bytes; since these are set to fairly high numbers you would not expect the while to kick in00:59
run|awaybut it does and the printfs will show you that non_ghost_count & total_estiamted_size are < than the targets01:00
run|awayprintf says that ;)01:00
run|awaybut the condition to enter while succeeds01:00
run|awayso we spent a very long time staring at the ring because.. the ghost_coutn/target_bytes conditions look very obvious01:01
hannoschyeah. I have a unit test that's a bit long but it shows the error :)01:02
hannoschnow trimming it down01:02
hannoschthe trick seems to be to make one object smaller01:03
*** harobed has joined #zope3-dev01:03
* mgedmin thinks this is all very exciting and would like to see even the long version of the test in a pastebin somewhere01:03
hannoschdon't blame me. I was trying to get over / under the size limit :)01:04
hannoschthat's in in ZODB01:04
*** grahal has quit IRC01:04
*** alga has quit IRC01:04
* hannosch cleans it up01:04
run|awayJ1m, i can continue with the buildou instructions01:06
run|awayup to you01:07
hannoschJ1m: this is my test
hannoschwithout the removal of "unsigned" the cache size is suddenly 429496729601:11
hannoschit's set to be max 100001:11
hannoschcache.total_estimated_size that is01:11
* hannosch makes a proper branch out of this01:12
*** fcorrea has quit IRC01:13
mgedmin4294967296 is exactly 2**32, curious01:16
J1mrun|away, I got called away fort a bit. I'm back for a bit. Reading ...01:18
J1mah cool. it looks like hannosch made a unit test!01:19
hannoschJ1m: yep. reporting a LP bug, putting up a patch and branch now01:19
J1mAwesome! I'll definitely look at this this weekend.01:20
*** tarek has joined #zope3-dev01:23
J1mrun|away, I think the unit test is what I want.01:23
J1mI'll save this session just in case.01:23
J1mAh, X-Chat gives me that for free. :)01:25
hannoschthe bug is up at
run|awayJ1m, one of our guys is saying this is behavior can be compiler dependent01:32
run|awayhannosch, test passes on py2.4, gcc4.4.1 on ubuntu01:33
*** timte has quit IRC01:33
*** allisterb has quit IRC01:33
hannoschrun|away: without the patch or with it?01:33
run|awaywith the patch01:33
run|awaywithout hte patch01:33
run|awaywithout the patch the test passes01:34
hannoschright, I'd hope so :)01:34
hannoscheh, damn01:34
run|awayand our c geek is saying that this kinda operation is highly dependent on compiler revisions01:34
run|awaycompiler version01:34
run|awayworks on windows py2.4, gcc 3.4.5 (mingw)01:35
run|awayso it is compiler specific01:35
run|awayhannosch, that test should fail w/o the patch, correct?01:35
hannoschright. I can try with more variants, but I only have 64bit here01:35
run|awayso according to our gimp it will vary based on version of compiler01:36
*** allisterb has joined #zope3-dev01:36
*** ignas has joined #zope3-dev01:38
hannoschrun|away: feel free to add more findings to
run|awayhannosch, test passes on windows01:41
run|awayw/o patch01:41
hannoschrun|away: can you put a pdb into the test and see what the value is?01:41
hannoschor simply print01:41
run|awaytotal_estimate at the end?01:42
*** allisterb has quit IRC01:42
hannoschyep cache.total_estimated_size01:42
hannoschok. test fails without and passes with the patch for both Py2.4 and Py2.6 here01:43
run|awaypls specify your compiler on the ticket01:43
hannoschadded to the ticket01:44
*** greenman has quit IRC01:44
hannosch"[GCC 4.2.1 (Apple Inc. build 5646)] on darwin"01:44
run|awayon windows: before last trnx.savepoint estimate_size 704, cache 2 after trnx.sp 0, 101:46
*** allisterb has joined #zope3-dev01:46
run|awaygcc-3.4.5 py2.4.401:46
run|awayposting info to ticket01:47
J1mcool, thanks guys. I'll keep the compiler info in mind.01:48
mgedminso essentially that code is doing (long long) += (unsigned) - (unsigned)?01:48
*** fcorrea has joined #zope3-dev01:49
run|awayas i understand it this is unspecified behavior (putting anything into a bits field) you have to do any sort of bounds checking youself and suggested to do bitshift/masking manually01:49
J1mmgedmin, It's doing unsigned 24-bit integer += ...01:49
mgedmin          self->total_estimated_size += _estimated_size_in_bytes(01:50
mgedmin               _estimated_size_in_24_bits(new_size) - v->estimated_size01:50
mgedmin                                                                 );01:50
mgedmintotal_estimated_size is long long01:50
mgedminv->estimated_size is a 24-bit unsigned bitfield01:50
J1mOops, you're right01:50
mgedminnew_size is unsigned, and hannosch's patch makes it into a signed01:50
mgedminI'm surprised there are compilers that don't trigger the bug...01:51
*** aaronv has quit IRC01:51
run|awaythere are01:51
J1mwhy should that trigger a bug?01:51
mgedminif new_size < old size, you get total += a very large number01:52
J1mwhy, because the subtraction is done unsized?01:53
mgedmindoes C require that in x += a - b the (a - b) be evaluated first, or is it allowed for a compiler to compute it as x = (x + a) - b?01:53
J1mI assume that's what you mean.01:53
mgedminalthough my C is rusty...01:53
J1mThat's plausible.01:53
mgedminI seem to remember advice in books saying "avoid unsigned, any arithmetic involving mixed signed and unsigned values ends up evaluated as unsigned before the final type promotion" or something to that end01:54
J1mAnd making newsize signed maybe forces the unsigned 24-bit # to be converted to signed.01:54
J1mok, well, I can do explicit conversions easily enough.01:55
mgedminafaiu it all that matters is whether the value resulting from (new_size - old_size) computation is sign-extended into long long or zero-extended01:55
mgedminand that depends on what the compiler thinks the type of the expression is01:55
mgedminwhich seems to be unsigned int, afaics01:55
* mgedmin shuts up now01:56
J1mIt's odd that I did that computation that way.01:56
hannoschluckily I don't enough about the subtle semantics here ... just glad I could put you in the right direction :)01:58
J1mYup. Thanks again folks.01:58
J1mI'm heading out, later.01:58
run|awayhave a good weekend J1m01:58
hannoschthx you :)01:58
*** J1m has quit IRC01:59
run|awayi am going to go as well.. too many people talking about bits, twos compliment and compilers for one day01:59
*** hannosch has left #zope3-dev02:00
*** run|away has quit IRC02:03
*** harobed has quit IRC02:10
*** mgedmin has quit IRC02:10
*** fcorrea has quit IRC02:24
*** fcorrea has joined #zope3-dev02:29
*** yota has joined #zope3-dev02:37
*** allisterb has quit IRC02:39
*** fcorrea has quit IRC02:45
*** allisterb has joined #zope3-dev02:48
*** kaeru has quit IRC02:49
*** kaeru has joined #zope3-dev02:53
*** JaRoel|4_ has quit IRC02:55
*** davisagli has quit IRC02:56
*** baijum has joined #zope3-dev02:56
*** JaRoel|4D has joined #zope3-dev03:01
*** JaRoel|4_ has joined #zope3-dev03:04
*** JaRoel|4D has quit IRC03:04
*** kaeru has quit IRC03:07
*** kaeru has joined #zope3-dev03:07
*** allisterb has quit IRC03:46
*** allisterb_ has joined #zope3-dev03:47
*** fcorrea has joined #zope3-dev03:56
*** ignas has quit IRC04:02
*** romanofski has joined #zope3-dev04:09
*** tarek has left #zope3-dev04:42
*** romanofski has quit IRC04:53
*** fcorrea has quit IRC04:59
*** davisagli has joined #zope3-dev05:02
*** davisagli has quit IRC05:14
*** davisagli has joined #zope3-dev05:27
*** alecm has quit IRC05:33
*** romanofski has joined #zope3-dev05:41
*** avn has quit IRC05:44
*** davisagli has quit IRC05:48
*** romanofski has quit IRC05:49
*** avn has joined #zope3-dev06:00
*** afd_ has joined #zope3-dev07:15
*** avn has quit IRC07:27
*** avn has joined #zope3-dev07:28
*** alecm has joined #zope3-dev07:36
*** jfkw has quit IRC08:04
*** timte has joined #zope3-dev08:58
*** timte has joined #zope3-dev09:04
*** redir has joined #zope3-dev09:06
*** baijum has quit IRC09:27
*** timte has quit IRC09:56
*** r0ver has left #zope3-dev10:01
*** timte has joined #zope3-dev10:07
*** matthal has joined #zope3-dev10:30
*** matthal has quit IRC10:57
*** afd_ has quit IRC11:02
*** afd___ has joined #zope3-dev11:06
*** davisagli has joined #zope3-dev11:07
*** jukart has joined #zope3-dev11:09
*** davisagli has quit IRC11:10
*** romanofski has joined #zope3-dev11:32
*** jpcw has joined #zope3-dev11:35
*** harobed has joined #zope3-dev11:40
*** zagy has joined #zope3-dev11:41
*** jukart has quit IRC11:44
*** alga has joined #zope3-dev11:48
*** romanofski has quit IRC11:51
*** zagy has quit IRC11:58
*** zagy has joined #zope3-dev11:58
*** zagy1 has joined #zope3-dev12:05
*** sweh has joined #zope3-dev12:21
*** sweh has quit IRC12:21
*** mcdonc has quit IRC12:42
*** zagy1 has quit IRC13:00
*** baijum has joined #zope3-dev13:36
*** harobed has quit IRC13:52
*** timte has joined #zope3-dev14:39
*** allisterb_ has quit IRC14:46
*** allisterb has joined #zope3-dev14:46
*** avn has quit IRC14:48
*** avn has joined #zope3-dev14:50
*** avn has quit IRC14:58
*** avn has joined #zope3-dev15:00
*** ignas has joined #zope3-dev15:03
*** aaronv has joined #zope3-dev15:43
*** afd___ has quit IRC15:44
*** afd___ has joined #zope3-dev15:44
*** benji has joined #zope3-dev15:51
*** r0ver has joined #zope3-dev15:56
*** timte has quit IRC16:00
*** afd___ has quit IRC16:00
*** afd___ has joined #zope3-dev16:01
*** Theuni1 has joined #zope3-dev16:06
*** mcdonc has joined #zope3-dev16:10
*** timte has joined #zope3-dev16:17
*** timte has quit IRC16:22
*** JaRoel|4_ has quit IRC17:02
*** JaRoel|4D has joined #zope3-dev17:33
*** JaRoel|4D has quit IRC17:39
*** RUNYAGAs has joined #zope3-dev17:41
*** RUNYAGAs has quit IRC17:43
*** Theuni1 has quit IRC17:53
*** alecm has quit IRC18:06
*** JaRoel|4D has joined #zope3-dev18:17
*** ignas has quit IRC18:28
*** allisterb_ has joined #zope3-dev18:34
*** allisterb has quit IRC18:35
*** allisterb has joined #zope3-dev18:58
*** fcorrea has joined #zope3-dev18:58
*** allisterb_ has quit IRC19:00
*** chaoflow has joined #zope3-dev19:03
*** allisterb_ has joined #zope3-dev19:07
*** allisterb has quit IRC19:09
*** allisterb has joined #zope3-dev19:18
*** allisterb_ has quit IRC19:21
*** tarek has joined #zope3-dev19:24
*** allisterb has quit IRC19:27
*** allisterb has joined #zope3-dev19:31
*** allisterb has quit IRC19:40
*** allisterb has joined #zope3-dev19:43
*** mgedmin has joined #zope3-dev19:48
*** mgedmin has quit IRC19:50
*** mgedmin has joined #zope3-dev19:52
*** aaronv has quit IRC19:54
*** avn has quit IRC19:59
*** avn has joined #zope3-dev20:01
*** allisterb_ has joined #zope3-dev20:05
*** allisterb has quit IRC20:07
*** afd___ has quit IRC20:09
*** baijum1 has joined #zope3-dev20:14
*** baijum has quit IRC20:14
*** baijum1 has quit IRC20:14
*** r0ver has left #zope3-dev20:16
*** allisterb_ has quit IRC20:17
*** allisterb has joined #zope3-dev20:19
*** avn has quit IRC20:35
*** avn has joined #zope3-dev20:37
*** allisterb has quit IRC20:46
*** allisterb has joined #zope3-dev20:48
*** pcardune has joined #zope3-dev20:56
*** r0ver has joined #zope3-dev21:10
*** r0ver has quit IRC21:12
*** tarek has quit IRC21:33
*** ignas has joined #zope3-dev21:40
*** tarek has joined #zope3-dev21:50
*** ccomb has quit IRC21:55
*** ccomb has joined #zope3-dev22:16
*** tarek has quit IRC22:18
*** tarek has joined #zope3-dev22:34
*** davisagli has joined #zope3-dev22:40
*** davisagli has quit IRC22:43
*** r0ver has joined #zope3-dev22:47
r0verhello, do you know where (which component) is implemented zmi's left panel ?22:48
*** matthal has joined #zope3-dev22:49
*** davisagli has joined #zope3-dev22:51
*** tarek has quit IRC22:52
*** tarek has joined #zope3-dev22:58
*** davisagli has quit IRC22:58
Vqr0ver: the one containing the navigation tree and the add-menu?23:02
r0verVq: yes, i'd like to reuse the nav tree23:03
*** tarek has quit IRC23:03
Vqthat would be the skin package23:04
Vqif you are using the standard rotterdam skin this would be
r0verok, i was reading the rotterdam one but not why i thought it was an example and not what uses zope 3.4 by default...23:06
Vqit's the default23:06
Vqi guess it's the navigation_macros* things you're after23:07
r0veryup, thks a lot23:07
*** fcorrea has quit IRC23:19
*** allisterb_ has joined #zope3-dev23:26
*** allisterb has quit IRC23:28
*** avn has quit IRC23:28
*** avn has joined #zope3-dev23:30
*** fcorrea has joined #zope3-dev23:41
*** allisterb has joined #zope3-dev23:56
*** allisterb_ has quit IRC23:59

Generated by 2.15.1 by Marius Gedminas - find it at!