| J1m | Sorry, I don't have context. | 00:00 |
|---|---|---|
| run|away | limiting the picklecache by byte-size | 00:00 |
| J1m | What byte-size-gc issue? | 00:00 |
| J1m | Hm | 00:00 |
| run|away | does not work | 00:00 |
| J1m | Oh, the symptom where all of a sudden it evicts everything? | 00:01 |
| run|away | yes | 00:01 |
| J1m | cool | 00:01 |
| run|away | this is fixed (in our tests) by setting new_size to a int | 00:01 |
| J1m | Hm, as opposed to what? | 00:01 |
| * J1m looks | 00:01 | |
| run|away | unsigned int | 00:02 |
| run|away | just a simple oversight.. which took us forever to fix ;) | 00:02 |
| J1m | Hm, so I guess the question is, why would it go negative. | 00:03 |
| J1m | iow, why would it underflow. | 00:03 |
| run|away | it doesnt | 00:03 |
| run|away | i need to get someone else online to tell you | 00:03 |
| J1m | I assume that it would go negative if it was signed, but now is underflowing to something ginormous. | 00:04 |
| *** pcardune has quit IRC | 00:08 | |
| run|away | pyarg_parsetuple the new_size is a signed python integer? | 00:08 |
| run|away | in cc_update_object_size_estimation | 00:08 |
| *** r0ver has joined #zope3-dev | 00:10 | |
| *** timte has joined #zope3-dev | 00:11 | |
| Vq | timte: godafton | 00:17 |
| *** lisppaste6 has quit IRC | 00:18 | |
| timte | Vq: afton | 00:19 |
| *** lisppaste6 has joined #zope3-dev | 00:28 | |
| * mgedmin not happy with buildbout-generated bin/python that doesn't know any of the command line options of the real python interpreter -- how do I do -W error? | 00:29 | |
| *** benji has quit IRC | 00:29 | |
| mgedmin | there doesn't seem to be an environment variable for that :/ | 00:29 |
| *** allisterb has quit IRC | 00:34 | |
| run|away | J1m, if you want we could diagnosis it more but its hairy (I know "it works for us" isnt very satisfactory) | 00:36 |
| *** fcorrea has joined #zope3-dev | 00:36 | |
| *** allisterb has joined #zope3-dev | 00:37 | |
| J1m | run|away, PyArg_ParseTuple is using an unsigned int flag. | 00:44 |
| J1m | run|away, if you could just tell me a reliable (and sane :) way to reproduce the problem, that would help enormously. | 00:44 |
| J1m | We've only seen it in high duress production situations and have run away (so to speak :) from that option. | 00:45 |
| run|away | unfortunately sane isnt really in the equation | 00:45 |
| J1m | heh | 00:45 |
| J1m | I guess though that what you say might give me a clue of things I can try. | 00:46 |
| *** cbcunc has quit IRC | 00:47 | |
| run|away | running plone buildout, sprinkle printf in picklecache, ab - shows the problem. change from unsigned to sign, recompile, repeat fixes it | 00:47 |
| run|away | but that isnt useful for you - i know | 00:47 |
| run|away | we arent using ab but some complicated load testing rig | 00:48 |
| run|away | but you can see it w/ ab | 00:48 |
| run|away | so what we have seen | 00:48 |
| J1m | I can deal with running a plone buildout. :) | 00:48 |
| run|away | oh cool | 00:48 |
| run|away | ok | 00:48 |
| run|away | hold | 00:48 |
| J1m | I can even deal with ab. | 00:48 |
| run|away | well i thought you would want something more isolated | 00:48 |
| J1m | I would :) | 00:48 |
| J1m | But I'm willing to deal with less than what I want. | 00:49 |
| run|away | i appreciate that | 00:49 |
| J1m | especially if I get good instructions. | 00:49 |
| run|away | i will sit here with you until you repro and see fix | 00:49 |
| run|away | https://svn.plone.org/svn/plone/plonenext/3.3/ | 00:50 |
| run|away | svn co and run buildout | 00:50 |
| J1m | k | 00:51 |
| run|away | wrong one | 00:52 |
| run|away | http://svn.plone.org/svn/plone/buildouts/plone-coredev/branches/4.0/ | 00:52 |
| run|away | we are using patchs.. you wont get zodb 3.9 w/ earlier zope2 | 00:52 |
| run|away | please use 4.0 | 00:52 |
| J1m | k | 00:52 |
| run|away | hanno repro'ed it on python2.6 64bit | 00:52 |
| *** hannosch has joined #zope3-dev | 00:53 | |
| J1m | so 64 bit ubuntu is ok? | 00:53 |
| run|away | i presume so. we are using 64bit windows with python2.4 (32bit) | 00:53 |
| hannosch | heya, I'm trying to write a damn unit test for this, but having a bit of a hard time with it :) | 00:54 |
| run|away | so J1m here is how we tested it.. we put printf in cPickleCache.c: scan_gc_items inside the while( here!=&before..) we printf total_estimated_size, target_bytes | 00:56 |
| run|away | in parts/instance/zope.conf change cache-size to 99999 and cache-byte-size to 200000000 | 00:57 |
| run|away | non_ghost_count < target & you can see total_estimated_size < target_bytes; since these are set to fairly high numbers you would not expect the while to kick in | 00:59 |
| run|away | but it does and the printfs will show you that non_ghost_count & total_estiamted_size are < than the targets | 01:00 |
| run|away | printf says that ;) | 01:00 |
| run|away | but the condition to enter while succeeds | 01:00 |
| run|away | so we spent a very long time staring at the ring because.. the ghost_coutn/target_bytes conditions look very obvious | 01:01 |
| hannosch | yeah. I have a unit test that's a bit long but it shows the error :) | 01:02 |
| run|away | hooray | 01:02 |
| hannosch | now trimming it down | 01:02 |
| hannosch | the trick seems to be to make one object smaller | 01:03 |
| *** harobed has joined #zope3-dev | 01:03 | |
| * mgedmin thinks this is all very exciting and would like to see even the long version of the test in a pastebin somewhere | 01:03 | |
| hannosch | http://pastebin.com/SzW7Hb6R | 01:03 |
| hannosch | don't blame me. I was trying to get over / under the size limit :) | 01:04 |
| hannosch | that's in testConnection.py in ZODB | 01:04 |
| *** grahal has quit IRC | 01:04 | |
| *** alga has quit IRC | 01:04 | |
| * hannosch cleans it up | 01:04 | |
| run|away | J1m, i can continue with the buildou instructions | 01:06 |
| run|away | up to you | 01:07 |
| hannosch | J1m: this is my test http://pastebin.com/J00s0qMJ | 01:10 |
| hannosch | without the removal of "unsigned" the cache size is suddenly 4294967296 | 01:11 |
| hannosch | it's set to be max 1000 | 01:11 |
| hannosch | cache.total_estimated_size that is | 01:11 |
| * hannosch makes a proper branch out of this | 01:12 | |
| *** fcorrea has quit IRC | 01:13 | |
| mgedmin | 4294967296 is exactly 2**32, curious | 01:16 |
| J1m | run|away, I got called away fort a bit. I'm back for a bit. Reading ... | 01:18 |
| J1m | ah cool. it looks like hannosch made a unit test! | 01:19 |
| hannosch | J1m: yep. reporting a LP bug, putting up a patch and branch now | 01:19 |
| J1m | Awesome! I'll definitely look at this this weekend. | 01:20 |
| *** tarek has joined #zope3-dev | 01:23 | |
| J1m | run|away, I think the unit test is what I want. | 01:23 |
| J1m | I'll save this session just in case. | 01:23 |
| J1m | Ah, X-Chat gives me that for free. :) | 01:25 |
| hannosch | the bug is up at https://bugs.launchpad.net/zodb/+bug/533015 | 01:25 |
| J1m | k | 01:28 |
| run|away | also | 01:31 |
| run|away | so | 01:32 |
| run|away | J1m, one of our guys is saying this is behavior can be compiler dependent | 01:32 |
| run|away | hannosch, test passes on py2.4, gcc4.4.1 on ubuntu | 01:33 |
| *** timte has quit IRC | 01:33 | |
| *** allisterb has quit IRC | 01:33 | |
| hannosch | run|away: without the patch or with it? | 01:33 |
| run|away | with the patch | 01:33 |
| run|away | without hte patch | 01:33 |
| run|away | sorry | 01:33 |
| run|away | without the patch the test passes | 01:34 |
| hannosch | right, I'd hope so :) | 01:34 |
| hannosch | eh, damn | 01:34 |
| run|away | and our c geek is saying that this kinda operation is highly dependent on compiler revisions | 01:34 |
| run|away | compiler version | 01:34 |
| run|away | works on windows py2.4, gcc 3.4.5 (mingw) | 01:35 |
| run|away | so it is compiler specific | 01:35 |
| run|away | hannosch, that test should fail w/o the patch, correct? | 01:35 |
| hannosch | right. I can try with more variants, but I only have 64bit here | 01:35 |
| hannosch | yes | 01:35 |
| run|away | so according to our gimp it will vary based on version of compiler | 01:36 |
| *** allisterb has joined #zope3-dev | 01:36 | |
| *** ignas has joined #zope3-dev | 01:38 | |
| hannosch | run|away: feel free to add more findings to https://bugs.launchpad.net/zodb/+bug/533015 | 01:39 |
| run|away | hannosch, test passes on windows | 01:41 |
| run|away | w/o patch | 01:41 |
| hannosch | run|away: can you put a pdb into the test and see what the value is? | 01:41 |
| hannosch | or simply print | 01:41 |
| run|away | total_estimate at the end? | 01:42 |
| *** allisterb has quit IRC | 01:42 | |
| hannosch | yep cache.total_estimated_size | 01:42 |
| hannosch | ok. test fails without and passes with the patch for both Py2.4 and Py2.6 here | 01:43 |
| run|away | pls specify your compiler on the ticket | 01:43 |
| hannosch | added to the ticket | 01:44 |
| *** greenman has quit IRC | 01:44 | |
| hannosch | "[GCC 4.2.1 (Apple Inc. build 5646)] on darwin" | 01:44 |
| run|away | on windows: before last trnx.savepoint estimate_size 704, cache 2 after trnx.sp 0, 1 | 01:46 |
| *** allisterb has joined #zope3-dev | 01:46 | |
| run|away | gcc-3.4.5 py2.4.4 | 01:46 |
| run|away | posting info to ticket | 01:47 |
| J1m | cool, thanks guys. I'll keep the compiler info in mind. | 01:48 |
| mgedmin | so essentially that code is doing (long long) += (unsigned) - (unsigned)? | 01:48 |
| *** fcorrea has joined #zope3-dev | 01:49 | |
| run|away | as i understand it this is unspecified behavior (putting anything into a bits field) you have to do any sort of bounds checking youself and suggested to do bitshift/masking manually | 01:49 |
| J1m | mgedmin, It's doing unsigned 24-bit integer += ... | 01:49 |
| J1m | Hm | 01:50 |
| mgedmin | self->total_estimated_size += _estimated_size_in_bytes( | 01:50 |
| mgedmin | _estimated_size_in_24_bits(new_size) - v->estimated_size | 01:50 |
| mgedmin | ); | 01:50 |
| mgedmin | total_estimated_size is long long | 01:50 |
| mgedmin | v->estimated_size is a 24-bit unsigned bitfield | 01:50 |
| J1m | Oops, you're right | 01:50 |
| mgedmin | new_size is unsigned, and hannosch's patch makes it into a signed | 01:50 |
| mgedmin | I'm surprised there are compilers that don't trigger the bug... | 01:51 |
| *** aaronv has quit IRC | 01:51 | |
| run|away | there are | 01:51 |
| J1m | why should that trigger a bug? | 01:51 |
| mgedmin | if new_size < old size, you get total += a very large number | 01:52 |
| J1m | why, because the subtraction is done unsized? | 01:53 |
| mgedmin | does C require that in x += a - b the (a - b) be evaluated first, or is it allowed for a compiler to compute it as x = (x + a) - b? | 01:53 |
| J1m | I assume that's what you mean. | 01:53 |
| mgedmin | yes | 01:53 |
| mgedmin | although my C is rusty... | 01:53 |
| J1m | That's plausible. | 01:53 |
| mgedmin | I seem to remember advice in books saying "avoid unsigned, any arithmetic involving mixed signed and unsigned values ends up evaluated as unsigned before the final type promotion" or something to that end | 01:54 |
| J1m | And making newsize signed maybe forces the unsigned 24-bit # to be converted to signed. | 01:54 |
| J1m | ok, well, I can do explicit conversions easily enough. | 01:55 |
| mgedmin | afaiu it all that matters is whether the value resulting from (new_size - old_size) computation is sign-extended into long long or zero-extended | 01:55 |
| mgedmin | and that depends on what the compiler thinks the type of the expression is | 01:55 |
| mgedmin | which seems to be unsigned int, afaics | 01:55 |
| * mgedmin shuts up now | 01:56 | |
| J1m | It's odd that I did that computation that way. | 01:56 |
| hannosch | luckily I don't enough about the subtle semantics here ... just glad I could put you in the right direction :) | 01:58 |
| J1m | Yup. Thanks again folks. | 01:58 |
| J1m | I'm heading out, later. | 01:58 |
| run|away | have a good weekend J1m | 01:58 |
| hannosch | thx you :) | 01:58 |
| *** J1m has quit IRC | 01:59 | |
| run|away | i am going to go as well.. too many people talking about bits, twos compliment and compilers for one day | 01:59 |
| *** hannosch has left #zope3-dev | 02:00 | |
| *** run|away has quit IRC | 02:03 | |
| *** harobed has quit IRC | 02:10 | |
| *** mgedmin has quit IRC | 02:10 | |
| *** fcorrea has quit IRC | 02:24 | |
| *** fcorrea has joined #zope3-dev | 02:29 | |
| *** yota has joined #zope3-dev | 02:37 | |
| *** allisterb has quit IRC | 02:39 | |
| *** fcorrea has quit IRC | 02:45 | |
| *** allisterb has joined #zope3-dev | 02:48 | |
| *** kaeru has quit IRC | 02:49 | |
| *** kaeru has joined #zope3-dev | 02:53 | |
| *** JaRoel|4_ has quit IRC | 02:55 | |
| *** davisagli has quit IRC | 02:56 | |
| *** baijum has joined #zope3-dev | 02:56 | |
| *** JaRoel|4D has joined #zope3-dev | 03:01 | |
| *** JaRoel|4_ has joined #zope3-dev | 03:04 | |
| *** JaRoel|4D has quit IRC | 03:04 | |
| *** kaeru has quit IRC | 03:07 | |
| *** kaeru has joined #zope3-dev | 03:07 | |
| *** allisterb has quit IRC | 03:46 | |
| *** allisterb_ has joined #zope3-dev | 03:47 | |
| *** fcorrea has joined #zope3-dev | 03:56 | |
| *** ignas has quit IRC | 04:02 | |
| *** romanofski has joined #zope3-dev | 04:09 | |
| *** tarek has left #zope3-dev | 04:42 | |
| *** romanofski has quit IRC | 04:53 | |
| *** fcorrea has quit IRC | 04:59 | |
| *** davisagli has joined #zope3-dev | 05:02 | |
| *** davisagli has quit IRC | 05:14 | |
| *** davisagli has joined #zope3-dev | 05:27 | |
| *** alecm has quit IRC | 05:33 | |
| *** romanofski has joined #zope3-dev | 05:41 | |
| *** avn has quit IRC | 05:44 | |
| *** davisagli has quit IRC | 05:48 | |
| *** romanofski has quit IRC | 05:49 | |
| *** avn has joined #zope3-dev | 06:00 | |
| *** afd_ has joined #zope3-dev | 07:15 | |
| *** avn has quit IRC | 07:27 | |
| *** avn has joined #zope3-dev | 07:28 | |
| *** alecm has joined #zope3-dev | 07:36 | |
| *** jfkw has quit IRC | 08:04 | |
| *** timte has joined #zope3-dev | 08:58 | |
| *** timte has joined #zope3-dev | 09:04 | |
| *** redir has joined #zope3-dev | 09:06 | |
| *** baijum has quit IRC | 09:27 | |
| *** timte has quit IRC | 09:56 | |
| *** r0ver has left #zope3-dev | 10:01 | |
| *** timte has joined #zope3-dev | 10:07 | |
| *** matthal has joined #zope3-dev | 10:30 | |
| *** matthal has quit IRC | 10:57 | |
| *** afd_ has quit IRC | 11:02 | |
| *** afd___ has joined #zope3-dev | 11:06 | |
| *** davisagli has joined #zope3-dev | 11:07 | |
| *** jukart has joined #zope3-dev | 11:09 | |
| *** davisagli has quit IRC | 11:10 | |
| *** romanofski has joined #zope3-dev | 11:32 | |
| *** jpcw has joined #zope3-dev | 11:35 | |
| *** harobed has joined #zope3-dev | 11:40 | |
| *** zagy has joined #zope3-dev | 11:41 | |
| *** jukart has quit IRC | 11:44 | |
| *** alga has joined #zope3-dev | 11:48 | |
| *** romanofski has quit IRC | 11:51 | |
| *** zagy has quit IRC | 11:58 | |
| *** zagy has joined #zope3-dev | 11:58 | |
| *** zagy1 has joined #zope3-dev | 12:05 | |
| *** sweh has joined #zope3-dev | 12:21 | |
| *** sweh has quit IRC | 12:21 | |
| *** mcdonc has quit IRC | 12:42 | |
| *** zagy1 has quit IRC | 13:00 | |
| *** baijum has joined #zope3-dev | 13:36 | |
| *** harobed has quit IRC | 13:52 | |
| *** timte has joined #zope3-dev | 14:39 | |
| *** allisterb_ has quit IRC | 14:46 | |
| *** allisterb has joined #zope3-dev | 14:46 | |
| *** avn has quit IRC | 14:48 | |
| *** avn has joined #zope3-dev | 14:50 | |
| *** avn has quit IRC | 14:58 | |
| *** avn has joined #zope3-dev | 15:00 | |
| *** ignas has joined #zope3-dev | 15:03 | |
| *** aaronv has joined #zope3-dev | 15:43 | |
| *** afd___ has quit IRC | 15:44 | |
| *** afd___ has joined #zope3-dev | 15:44 | |
| *** benji has joined #zope3-dev | 15:51 | |
| *** r0ver has joined #zope3-dev | 15:56 | |
| *** timte has quit IRC | 16:00 | |
| *** afd___ has quit IRC | 16:00 | |
| *** afd___ has joined #zope3-dev | 16:01 | |
| *** Theuni1 has joined #zope3-dev | 16:06 | |
| *** mcdonc has joined #zope3-dev | 16:10 | |
| *** timte has joined #zope3-dev | 16:17 | |
| *** timte has quit IRC | 16:22 | |
| *** JaRoel|4_ has quit IRC | 17:02 | |
| *** JaRoel|4D has joined #zope3-dev | 17:33 | |
| *** JaRoel|4D has quit IRC | 17:39 | |
| *** RUNYAGAs has joined #zope3-dev | 17:41 | |
| *** RUNYAGAs has quit IRC | 17:43 | |
| *** Theuni1 has quit IRC | 17:53 | |
| *** alecm has quit IRC | 18:06 | |
| *** JaRoel|4D has joined #zope3-dev | 18:17 | |
| *** ignas has quit IRC | 18:28 | |
| *** allisterb_ has joined #zope3-dev | 18:34 | |
| *** allisterb has quit IRC | 18:35 | |
| *** allisterb has joined #zope3-dev | 18:58 | |
| *** fcorrea has joined #zope3-dev | 18:58 | |
| *** allisterb_ has quit IRC | 19:00 | |
| *** chaoflow has joined #zope3-dev | 19:03 | |
| *** allisterb_ has joined #zope3-dev | 19:07 | |
| *** allisterb has quit IRC | 19:09 | |
| *** allisterb has joined #zope3-dev | 19:18 | |
| *** allisterb_ has quit IRC | 19:21 | |
| *** tarek has joined #zope3-dev | 19:24 | |
| *** allisterb has quit IRC | 19:27 | |
| *** allisterb has joined #zope3-dev | 19:31 | |
| *** allisterb has quit IRC | 19:40 | |
| *** allisterb has joined #zope3-dev | 19:43 | |
| *** mgedmin has joined #zope3-dev | 19:48 | |
| *** mgedmin has quit IRC | 19:50 | |
| *** mgedmin has joined #zope3-dev | 19:52 | |
| *** aaronv has quit IRC | 19:54 | |
| *** avn has quit IRC | 19:59 | |
| *** avn has joined #zope3-dev | 20:01 | |
| *** allisterb_ has joined #zope3-dev | 20:05 | |
| *** allisterb has quit IRC | 20:07 | |
| *** afd___ has quit IRC | 20:09 | |
| *** baijum1 has joined #zope3-dev | 20:14 | |
| *** baijum has quit IRC | 20:14 | |
| *** baijum1 has quit IRC | 20:14 | |
| *** r0ver has left #zope3-dev | 20:16 | |
| *** allisterb_ has quit IRC | 20:17 | |
| *** allisterb has joined #zope3-dev | 20:19 | |
| *** avn has quit IRC | 20:35 | |
| *** avn has joined #zope3-dev | 20:37 | |
| *** allisterb has quit IRC | 20:46 | |
| *** allisterb has joined #zope3-dev | 20:48 | |
| *** pcardune has joined #zope3-dev | 20:56 | |
| *** r0ver has joined #zope3-dev | 21:10 | |
| *** r0ver has quit IRC | 21:12 | |
| *** tarek has quit IRC | 21:33 | |
| *** ignas has joined #zope3-dev | 21:40 | |
| *** tarek has joined #zope3-dev | 21:50 | |
| *** ccomb has quit IRC | 21:55 | |
| *** ccomb has joined #zope3-dev | 22:16 | |
| *** tarek has quit IRC | 22:18 | |
| *** tarek has joined #zope3-dev | 22:34 | |
| *** davisagli has joined #zope3-dev | 22:40 | |
| *** davisagli has quit IRC | 22:43 | |
| *** r0ver has joined #zope3-dev | 22:47 | |
| r0ver | hello, do you know where (which component) is implemented zmi's left panel ? | 22:48 |
| *** matthal has joined #zope3-dev | 22:49 | |
| *** davisagli has joined #zope3-dev | 22:51 | |
| *** tarek has quit IRC | 22:52 | |
| *** tarek has joined #zope3-dev | 22:58 | |
| *** davisagli has quit IRC | 22:58 | |
| Vq | r0ver: the one containing the navigation tree and the add-menu? | 23:02 |
| r0ver | Vq: yes, i'd like to reuse the nav tree | 23:03 |
| *** tarek has quit IRC | 23:03 | |
| Vq | that would be the skin package | 23:04 |
| Vq | if you are using the standard rotterdam skin this would be zope.app.rotterdam | 23:04 |
| r0ver | ok, i was reading the rotterdam one but not why i thought it was an example and not what uses zope 3.4 by default... | 23:06 |
| Vq | it's the default | 23:06 |
| Vq | i guess it's the navigation_macros* things you're after | 23:07 |
| r0ver | yup, thks a lot | 23:07 |
| *** fcorrea has quit IRC | 23:19 | |
| *** allisterb_ has joined #zope3-dev | 23:26 | |
| *** allisterb has quit IRC | 23:28 | |
| *** avn has quit IRC | 23:28 | |
| *** avn has joined #zope3-dev | 23:30 | |
| *** fcorrea has joined #zope3-dev | 23:41 | |
| *** allisterb has joined #zope3-dev | 23:56 | |
| *** allisterb_ has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!