IRC log of #zope3-dev for Tuesday, 2007-11-13

*** faassen has quit IRC00:00
*** schwendinger has quit IRC00:01
*** dabu has joined #zope3-dev00:05
*** dobee has quit IRC00:19
*** timte_ has joined #zope3-dev00:22
*** timte has quit IRC00:24
*** regebro has left #zope3-dev00:33
*** rocky has quit IRC00:37
*** timte_ has quit IRC00:41
*** nathany has quit IRC00:46
*** RaFromBRC is now known as RaFromBRC|lunch01:03
*** benji has quit IRC01:03
*** hazmat has quit IRC01:06
*** salfield has joined #zope3-dev01:09
*** alga has quit IRC01:11
*** jsadjohnson has quit IRC01:11
*** lucielejard has quit IRC01:15
*** J1m has quit IRC01:26
*** harobed has quit IRC01:32
*** jodok has quit IRC01:35
*** TresEquis has quit IRC01:36
*** greenman has quit IRC01:37
*** philiKON_ has joined #zope3-dev01:38
*** greenman has joined #zope3-dev01:46
*** philiKON has quit IRC01:52
*** afd_ has quit IRC01:52
*** gstratton has quit IRC01:57
*** tjs has joined #zope3-dev01:59
tjsanyone got any info on zopeproject and packaging it to deploy zeo + instance? also curious about how people manage a project with zopeproject env, and switch between branches rapidly02:00
*** lucielejard has joined #zope3-dev02:11
*** lucielejard has quit IRC02:14
*** rmarianski has quit IRC02:31
*** RaFromBRC|lunch has quit IRC02:31
*** MJ has quit IRC02:33
*** greenman has quit IRC02:37
*** greenman has joined #zope3-dev02:39
*** greenman has quit IRC02:39
*** RaFromBRC has joined #zope3-dev02:45
*** greenman has joined #zope3-dev02:55
*** greenman has quit IRC02:56
*** malthe is now known as malthe|Zzz03:01
*** niemeyer has quit IRC03:12
*** greenman has joined #zope3-dev03:19
*** whit has quit IRC03:27
*** fcorrea has quit IRC03:35
*** dunny has quit IRC03:43
*** dunny has joined #zope3-dev03:50
*** rmarianski has joined #zope3-dev03:51
*** reco has quit IRC03:57
*** jsadjohnson has joined #zope3-dev03:57
*** hazmat has joined #zope3-dev04:01
*** b52lap has quit IRC04:02
*** huajie has joined #zope3-dev04:03
*** greenman has quit IRC04:11
*** dunny has quit IRC04:26
*** TresEquis has joined #zope3-dev05:05
*** lucielejard has joined #zope3-dev05:07
*** hazmat has quit IRC05:15
*** jsadjohnson has quit IRC05:20
*** hazmat has joined #zope3-dev05:23
*** baijum has joined #zope3-dev05:28
*** dabu has quit IRC05:37
*** baijum has quit IRC05:51
*** TresEquis has quit IRC06:03
*** RaFromBRC has quit IRC06:04
*** Ariel_Calzada has joined #zope3-dev06:08
*** greenman has joined #zope3-dev06:23
*** rmarianski has quit IRC06:24
*** dunny has joined #zope3-dev06:35
*** natea|boston has joined #zope3-dev06:37
*** cursor has joined #zope3-dev06:43
*** greenman has quit IRC06:44
*** natea has quit IRC06:45
*** jfroche_ has joined #zope3-dev06:45
*** greenman has joined #zope3-dev06:48
*** Jell-O-Fishi has quit IRC06:50
*** salfield has quit IRC06:54
*** lucielejard has quit IRC06:55
*** jfroche has quit IRC06:59
*** dunny has quit IRC07:02
*** dunny has joined #zope3-dev07:09
*** rcrafton has quit IRC07:14
*** pcardune has joined #zope3-dev07:15
*** pcardune__ has joined #zope3-dev07:16
*** dunny has quit IRC07:26
*** dunny has joined #zope3-dev07:34
*** hazmat has quit IRC07:34
*** hazmat has joined #zope3-dev07:35
*** pcardune__ is now known as pcardune07:47
*** tjs has quit IRC07:49
*** stub has quit IRC07:52
*** stu2 has joined #zope3-dev07:52
*** stu2 is now known as stub07:52
*** dobee has joined #zope3-dev07:53
*** greenman has quit IRC08:21
*** hdima has joined #zope3-dev08:39
*** dobee has quit IRC08:48
*** zagy has joined #zope3-dev08:53
*** dobee has joined #zope3-dev08:53
*** BjornT has joined #zope3-dev08:54
*** zagy has quit IRC08:55
*** zagy has joined #zope3-dev08:55
*** afd_ has joined #zope3-dev08:57
*** zagy has quit IRC09:08
*** quodt has joined #zope3-dev09:08
*** zagy has joined #zope3-dev09:13
*** ghendi has joined #zope3-dev09:13
*** timte has joined #zope3-dev09:20
*** pcardune has quit IRC09:23
*** greenman has joined #zope3-dev09:30
*** __mac__ has joined #zope3-dev09:32
romanofskimoin :)09:40
*** dennis__ has quit IRC09:42
*** dennis__ has joined #zope3-dev09:42
*** schwendinger has joined #zope3-dev09:47
*** BjornT has quit IRC09:54
*** BjornT has joined #zope3-dev09:54
*** jodok has joined #zope3-dev10:10
*** sorin has joined #zope3-dev10:19
*** sorin is now known as sorindregan10:20
*** harobed has joined #zope3-dev10:27
*** gstratton has joined #zope3-dev10:28
*** MJ has joined #zope3-dev10:35
*** bigkevmcd has joined #zope3-dev10:37
*** malthe|Zzz is now known as malthe10:40
*** pcardune has joined #zope3-dev10:57
*** greenman has quit IRC11:09
*** yvl has joined #zope3-dev11:15
*** pelle__ has joined #zope3-dev11:15
*** ccomb has joined #zope3-dev11:19
*** MJ has quit IRC11:19
*** malthe has quit IRC11:24
*** malthe has joined #zope3-dev11:25
*** regebro has joined #zope3-dev11:26
*** malthe has quit IRC11:43
*** pcardune has quit IRC11:52
*** dunny has quit IRC11:52
*** cursor_ has joined #zope3-dev11:56
*** goschtl has joined #zope3-dev12:12
*** malthe has joined #zope3-dev12:12
*** cursor has quit IRC12:14
*** ignas has joined #zope3-dev12:29
*** balor has joined #zope3-dev12:31
balor In what distribution of Zope would I find ZODB >3.8.0b1?  SVN?12:31
*** projekt01 has joined #zope3-dev12:36
*** spython has joined #zope3-dev12:36
spythonhi, any pointers on how to setup and make a z3 instance's lib/python/mypack play well with svn?12:38
spythonsomehow instance/lib/python/{trunk,branches,tags} doesn't feel good :)12:38
ccombbalor : http://wiki.zope.org/zope3/Zope340b2  -> http://download.zope.org/zope3.4/controlled-packages.cfg12:41
balorccomb: much oblidged12:43
*** b52laptop has joined #zope3-dev12:45
*** ignas_ has joined #zope3-dev12:49
*** thruflo has joined #zope3-dev12:55
*** __mac__ has quit IRC13:00
*** ignas has quit IRC13:03
*** jodok has quit IRC13:08
*** jodok has joined #zope3-dev13:08
*** mgedmin has joined #zope3-dev13:09
*** MJ has joined #zope3-dev13:14
*** niemeyer has joined #zope3-dev13:15
*** bigkevmcd has quit IRC13:21
*** mkerrin has joined #zope3-dev13:21
*** bigkevmcd has joined #zope3-dev13:24
*** jodok has quit IRC13:26
*** jodok has joined #zope3-dev13:35
*** jsadjohnson has joined #zope3-dev13:39
*** __mac__ has joined #zope3-dev13:39
*** schwendinger has quit IRC13:43
*** MJ has quit IRC13:45
*** MJ has joined #zope3-dev13:46
*** rocky has joined #zope3-dev13:56
*** afd_ has quit IRC14:09
*** afd_ has joined #zope3-dev14:09
dennis__how do you replace the "You are not authorized" page which you get if you do something for which you don't have permission, but are already signed in as a certain user and have disabled the re-ask for credentials if already-signed-in ?  (if similar to the NotFound, which Interface do I need to look for?)14:13
*** MJ is now known as MJ|lunch14:14
*** rcrafton has joined #zope3-dev14:19
philiKON_dennis__: Unauthorized14:20
philiKON_write an 'index.html' view for Unauthorized14:21
*** pcardune has joined #zope3-dev14:21
dennis__thanks, will start grepping :)14:21
philiKON_zope.exceptions14:22
philiKON_:)14:22
*** philiKON_ is now known as philiKON14:22
pan_hi.14:31
dennis__philiKON_ works like a charm, though have to set self.context of the view to getSite() to be able to render the error message inside the usual site template (otherwise looks funky)14:35
pcardunehmm, i think I found a dependency bug between zope.securitypolicy and zope.i18nmessageid.  The latter should require zope.i18nmessageid>=3.4.314:38
philiKONyup14:40
philiKONsolved14:41
philiKONah, wait, zope.securitypolicy14:41
philiKONyes14:41
philiKONi only fixed zope.app.securitypolicy14:41
philiKONmust've forgotten to fix zope.securitypolicy14:41
philiKONbtw, i think >=3.4.2 is correct14:41
pcardune(I just looked at the version I had on a working setup)14:41
philiKON3.4.2 is the one that gained the ZopeMessageFactory thing14:42
philiKONunnecessarily i might add14:42
pcardunewell, I'm glad that got resolved quickly :)14:42
pcarduneopen source strikes again!14:42
philiKONwell, you should still do a bugfix release for zope.securitypolicy :)14:42
philiKONi only did it for zope.app.securitypolicy14:43
pcarduneI don't think I have the required permissions making releases14:44
*** yvl has left #zope3-dev14:44
*** afd____ has joined #zope3-dev14:53
*** huajie has quit IRC14:54
*** pcardune has quit IRC14:56
*** afd_ has quit IRC14:57
*** agroszer has joined #zope3-dev14:57
*** afd____ is now known as afd_14:58
*** tomus has quit IRC14:59
*** tomus has joined #zope3-dev15:01
philiKONpcardune: that's a cheap excuse :)15:04
*** MJ|lunch is now known as MJ15:06
regebroHow does one in buildout best add a specific directory to the python-path?15:13
regebroOr alternatively, how does one check out a svn folder full of python libraries? :)15:13
*** pan has joined #zope3-dev15:15
philiKONthere's an option15:19
philiKONlook at the buildout docs :)15:19
regebrophiliKON: I did, I couldn't find it... Too much docs, not enough pages. :)15:24
regebroI'll try again then.15:24
philiKONit's recipe specific15:24
philiKONi think15:24
*** malthe has quit IRC15:26
*** schwendinger has joined #zope3-dev15:26
*** ignas_ has quit IRC15:27
regebrophiliKON: Aha.15:29
*** pan_ has quit IRC15:29
*** benji has joined #zope3-dev15:30
dennis__can I touch the request.locale at the beginning of the request or something similar in order to have all date operations inside that transaction done in a certain timezone ? currently all dates show up as GMT, but I don't want to go and add the timezone string to all places where dates are shown or parsed15:34
*** hazmat has quit IRC15:36
philiKONdennis__: request.local represents the user's preferred locale setting15:37
philiKONit's determined using the IUserPreferredLanguage adapter15:37
*** malthe has joined #zope3-dev15:38
dennis__yeah, I am defaulting it to the local language already since it is a local site (actually using example from your book)15:38
dennis__but times show up in GMT15:38
dennis__(i know locale doesnt contain reliable timezone info)15:38
dennis__so i would like to default that too15:39
philiKONthen hardcode the setting to one locale15:39
*** Ariel_Calzada has quit IRC15:40
dennis__so if request.locale it set properly it should already work .. well then i have look for a different reason it doesn't :)15:40
*** dabu has joined #zope3-dev15:45
dennis__request.locale is correct, request.locale.dates.getFormatter is used to format context/zope:modified which is (2007-11-13 13:05:45.021056+00:00). The modification really took place at 14:05+01:00, and I would like to show the time on the page as 14:0515:47
ccombHi,  is there a way to discover what happens during an instance startup?15:50
ccombit takes a minute to start15:50
*** lucielejard has joined #zope3-dev15:50
ccombthe zodb takes 800MB, with a BTree filled with 500k tuples15:52
ccombnot a minute, it takes 3 minutes, exactly15:53
*** J1m has joined #zope3-dev15:57
benjiccomb: three minutes exactly, every time?16:03
ccombbenji yes16:03
benjithats suspicious16:04
ccomb81 secs this time16:04
benjium, that's not three minutes, exactly16:04
ccombfirst time it was 175secs, then 81 now16:05
benjino idea, then16:05
ccombmy app is probably suspicious, but I would like to know what happen16:05
ccombbut I need a hint on how to do16:06
benjiI'd start by profiling the start up; there's no textbook approach to that though, you'll have to hack some16:06
*** acsr has quit IRC16:15
*** hazmat has joined #zope3-dev16:15
*** acsr has joined #zope3-dev16:21
*** romanofski has quit IRC16:22
*** romanofski has joined #zope3-dev16:22
*** ignas has joined #zope3-dev16:25
*** xenru has joined #zope3-dev16:25
*** pyqwer has joined #zope3-dev16:31
*** gidas has joined #zope3-dev16:42
*** alga has joined #zope3-dev16:45
*** ruda_porto has joined #zope3-dev16:46
*** Ariel_Calzada has joined #zope3-dev16:46
*** whit has joined #zope3-dev16:47
*** __mac__ has quit IRC16:47
mgedminccomb: usually what happens is the ZODB reads the whole Data.fs and builds an index16:54
ccombmgedmin the index is rebuilt at each start?16:55
mgedminI have the suspicion that if you stop the server with Ctrl-C, ZODB doesn't mark the index file as up-to-date or something like that16:55
mgedminI might be horribly wrong16:55
* ccomb is just going to try that16:56
J1mmgedmin, you are mostly write.  It is more complicated than that.  On a clean shutdown, the index is written.  I believe that ZEO servers don't do the clean shutdown dance correctly.16:57
J1mAlso, the index is written after some number of changes.16:57
J1mUnfortunately, the number of changes rerquired to trigger an index save increases with the size of the database.16:58
*** balor has quit IRC16:58
J1mFor very large databases, like ours, the index is rarely written.16:58
ccombyou mean "fortunately" :) ?16:58
J1mNo, I mean unfortunately.16:59
J1mwe want indexes to be written much more frequently.16:59
J1mFortunmately, indexes are written after packing.17:00
J1mand I think thet are also written on staup with no index.17:00
ccombI've just checked that my instance still takes 81s to start, and the index has not changed at all17:01
*** gidas21 has joined #zope3-dev17:01
*** hdima has quit IRC17:01
ccombit looks like it reads the whole btree while starting17:01
*** gidas has quit IRC17:01
*** gidas21 is now known as gidas17:01
J1mI doubt your DB is big enough for this to matter.17:02
ccombit's 800MB17:02
J1mI assume your long start up time is due to processing lots of ZCML.17:02
* J1m gets back to work.17:02
*** MJ has quit IRC17:14
*** ghendi has quit IRC17:17
*** rmarianski has joined #zope3-dev17:21
*** ktwilight has quit IRC17:23
*** reco has joined #zope3-dev17:27
*** bung_vijay has joined #zope3-dev17:43
*** MJ has joined #zope3-dev17:48
*** gidas has quit IRC17:50
*** gidas has joined #zope3-dev17:51
*** agroszer has quit IRC18:00
*** projekt01 has left #zope3-dev18:00
*** RaFromBRC has joined #zope3-dev18:01
*** sorindregan has quit IRC18:08
*** pcardune has joined #zope3-dev18:15
*** spython has quit IRC18:15
*** gidas has quit IRC18:17
*** gidas has joined #zope3-dev18:19
*** alecm has joined #zope3-dev18:24
*** goschtl has quit IRC18:26
*** natea_ has joined #zope3-dev18:27
*** elro has joined #zope3-dev18:28
*** gidas has quit IRC18:29
*** jfroche_ has quit IRC18:29
*** pelle___ has joined #zope3-dev18:30
*** jfroche has joined #zope3-dev18:30
*** pelle___ has quit IRC18:30
*** dagreener has joined #zope3-dev18:40
*** bung_vijay has quit IRC18:41
*** natea|boston has quit IRC18:43
*** pcardune has quit IRC18:45
*** pelle__ has quit IRC18:46
*** gstratton_ has joined #zope3-dev18:49
*** natea_ is now known as natea18:50
*** gstratton has quit IRC18:57
*** zagy has quit IRC19:12
*** jodok has quit IRC19:14
*** junkafarian has joined #zope3-dev19:14
* junkafarian waves19:14
junkafarianis it possible for me to reference the parent object within the __init__ function of a persistent object class?19:16
philiKONnope, usually not19:16
junkafarianhmz19:17
junkafarianany way around it?19:17
philiKONobjects only have parents after they've been added to a container19:17
junkafariani want child object to inherit some of the parent attributes19:17
philiKONso you should write an IObjectAddedEvent subscriber19:17
junkafarianok19:17
junkafariana bit strange that i cant write it in the business logic :(19:18
philiKONhuh?19:18
philiKONthis *is* business logic19:18
philiKONalso, how would an object know its parent before it's even added to a container19:18
philiKON?19:18
junkafariansorry, wrong terminology19:18
junkafariantrue19:19
philiKONin fact, this is beautiful separation of the stupid data model and business logic that's grouped around it19:19
junkafarianalso true19:20
junkafariani can see how it would lead to a better structure19:20
philiKONthough i admit, it's an indirection19:21
philiKONbut one that's rather fundamental19:21
philiKON(guessing one's parent during object creation is a bit hard)19:21
* junkafarian agrees19:22
philiKONthe only way to solve this properly is to make __init__ take the (future) parent as an argument19:22
junkafarianwas just about to say that might be a nice alternative19:22
junkafarianbut that would probably require a major refactor of the architecture19:23
* junkafarian guesses 19:23
* elro wanders how to write a FileDownloadWidget19:24
* ccomb wonders how to write a FileUploadWidget with a progress bar19:27
elroccomb: flash / java is necessary for that I think19:28
*** quodt has quit IRC19:28
gstratton_I have seen a FileUpload widget with progress bar entirely in ajax19:29
gstratton_I'm not sure whether it could be wired into Zope, though19:30
gstratton_it might require some WSGI middleware to intercept the uploads19:30
junkafariani am using http://uber-uploader.sourceforge.net/19:31
junkafariandefinitely not a widget though19:31
*** harobed has quit IRC19:31
xenruHi, all19:33
xenruphiliKON: r u here?19:33
ccombelro gstratton_  junkafarian : there is some very interesting things here : http://svn.zope.org/zope3org/trunk/src/zorg/live/demo/upload/19:35
*** quodt has joined #zope3-dev19:36
* junkafarian nods19:37
*** ignas has quit IRC19:47
*** norro has joined #zope3-dev19:50
*** bigkevmcd has quit IRC19:58
*** bung_vijay has joined #zope3-dev19:58
*** nathany has joined #zope3-dev19:58
*** zagy has joined #zope3-dev20:00
*** natea|boston has joined #zope3-dev20:00
*** bigkevmcd has joined #zope3-dev20:04
*** dobee has quit IRC20:05
*** pcardune has joined #zope3-dev20:06
*** elro has quit IRC20:08
*** natea has quit IRC20:08
*** quodt has quit IRC20:11
*** ruda_porto has quit IRC20:13
*** bung_vijay__ has joined #zope3-dev20:19
*** bung_vijay has quit IRC20:24
*** malthe has quit IRC20:24
*** bung_vijay has joined #zope3-dev20:24
*** thruflo has quit IRC20:26
*** MJ is now known as MJ|dinner20:26
*** ChanServ sets mode: +o hazmat20:27
*** b52lap has joined #zope3-dev20:30
rockya bit off-topic i know, but has anyone here ever used easy_install to install an egg from an svn url where the svn repo is preotected by apache basic auth? standard http://someuser:somepass@myhost.com/svn/repo/eggpackage doesn't work20:31
*** jodok has joined #zope3-dev20:35
*** schwendinger has quit IRC20:39
*** bung_vijay__ has quit IRC20:40
*** jodok has quit IRC20:42
*** b52laptop has quit IRC20:45
*** harobed has joined #zope3-dev20:48
J1moh lord20:50
*** natea|boston has quit IRC20:50
*** junkafarian has quit IRC20:50
* benji thinks J1m was just attacked by a troup of ninjas.20:52
* philiKON thought J1m ran out of cheese20:53
* J1m was reacting to rocky's question.20:54
rockylol20:54
rockyi got it working in fact -- the problem was i was using a password bash-unfriendly chars ;)20:55
mgedminyou could *probably* svn checkout and pass the pathname to easy_install20:55
mgedminah20:55
* mgedmin too late20:55
philiKONwhat mgedmin says works, btw20:55
* benji wonders how philiKON knows about the error message on our printer.20:56
*** pelle_ has joined #zope3-dev20:56
*** natea|boston has joined #zope3-dev20:56
*** pcardune has quit IRC20:58
*** pelle_ has quit IRC20:58
srichterrocky: lovely has developed some recipe or so to use http auth20:59
srichterbut I think it is buildout-specific20:59
rockyah20:59
rockygotcha20:59
* srichter needs to figure this out soon too21:00
*** ktwilight has joined #zope3-dev21:02
*** dobee has joined #zope3-dev21:03
*** cursor_ has quit IRC21:05
*** bung_vijay has quit IRC21:06
*** bung_vijay_ has joined #zope3-dev21:06
*** bung_vijay_ is now known as bung_vijay21:08
*** mgedmin has quit IRC21:09
*** baijum has joined #zope3-dev21:16
*** bung_vijay has quit IRC21:19
*** dagreener has left #zope3-dev21:22
*** bung_vijay has joined #zope3-dev21:24
*** rcrafton has quit IRC21:25
*** dunny has joined #zope3-dev21:25
*** zagy has quit IRC21:26
*** salfield has joined #zope3-dev21:28
*** bung_vijay has quit IRC21:31
*** bung_vijay_ has joined #zope3-dev21:32
*** hexsprite has joined #zope3-dev21:33
*** bung_vijay_ is now known as bung_vijay21:34
*** bung_vijay has quit IRC21:36
*** xenru has quit IRC21:36
*** baijum has quit IRC21:37
*** greenman has joined #zope3-dev21:40
mcdoncsrichter, J1m: a policy/intent question... is it a desire that people be able to install zope.* distributions from the cheeseshop selectively using easy_install?21:44
*** hexsprite_ has joined #zope3-dev21:46
srichtermcdonc: for packages that do not have many dependencies, such as zope.interface, zope.event, zoep.component, etc. yes21:47
mcdoncsrichter: so the dependency info in distributions that are not meant to be installed this way isnt being maintained, then?21:48
*** natea_ has joined #zope3-dev21:49
mcdonc(e.g. the zope.pagetemplate distro, each zope.app.* distro)21:49
*** rcrafton has joined #zope3-dev21:49
*** Ariel_Calzada has quit IRC21:51
*** jfroche has quit IRC21:51
*** RaFromBRC has quit IRC21:56
*** RaFromBRC has joined #zope3-dev21:59
philiKONmcdonc: sure is22:00
philiKONi'm not sure i totally agree with srichter's statement22:00
philiKONi think if people want to install zope3 apps using easy_install, it should be possible22:01
philiKONand all the dependencies should be installed propery22:01
philiKONproperly even22:01
*** mkerrin has quit IRC22:02
*** hexsprite has quit IRC22:02
mcdoncphiliKON: ok... fwiw, i wrote a bot that checks out the trunk of each top-level module in svn.zope.org and runs "setup.py install" in the checkout, and there are more things that can't be installed than can be currently (mostly due to a single dependency bug i think...).. these aren't just "unmaintained" things.. for instance, very few of the zope.app checkouts can be installed... i'm writing up a message to zope-dev about it22:03
philiKONok, thanks22:03
philiKONi'm surprised they don't work22:03
philiKONbuildout should be equivalent here in terms of dependencies22:03
*** natea|boston has quit IRC22:05
mcdoncphiliKON: i was just trying to figure out if this is a bug or a feature ;-)22:06
philiKONi consider this a bug22:06
mcdoncwhew ;-)22:06
ccombsorry for disturbing, but I have a big memory problem: an OOBtree in the ZODB, which contains 500k tuples.  By just calculating the size or returning the keys, the memory usage raises a lot and never gets released.  Just like the whole btree remains in memory for ever.22:09
ccomband the machine happily swaps until death22:10
*** regebro is now known as regebro|potateoc22:10
*** regebro|potateoc is now known as re|potatopancake22:11
ccombis there something I'm missing about btrees or zodb ?22:12
hazmatccomb, btree's don't keep length very efficiently afaicr, you should use a separate persistent object to trac that22:13
philiKONyup. there's a Length object, iirc22:14
ccombit's not just with the length, I'm in debugzope and  doing  list(btree.keys)  does the same22:14
philiKONhehe22:14
philiKONof course it does22:14
philiKONyou're waking up 500k tuples into memory22:14
philiKON*all at once*22:14
J1mmcdonc, I agree with philiKON I want people to install Zope 3 packages using easy install.22:15
ccombSo how should I ask them to go to bed again ?22:15
J1mOf course, I want lots of things. :)22:15
philiKONccomb: don't wake them up in the first place22:15
philiKONi doubt you need all 500k of them at once22:15
ccombunfortunately I do22:16
philiKONwhat for?22:16
ccombhow can I compute the length?22:16
ccombit's the same if I loop to get every item separately22:17
philiKONno it's not22:17
hazmatyou should track the length as you modify22:18
philiKONif you loop over the btree (for x in btree), it'll discard objects later22:18
ccombI already do that22:18
hazmattypically a separate container class to encapsulate the data structure and api22:18
mcdoncJ1m: roger...22:18
ccombfor i in btree.keys:  a=btree[i]22:18
philiKONfirst of all, keys() is a method22:18
benjiccomb: you can keep the length seperately, no need to calculate it afresh every time22:18
hazmatafaicr, you need python 2.5 to release memory back to the os22:18
philiKONno, i think py2.5 just does it more efficiently...22:19
hazmat"""- Patch #1123430: Python's small-object allocator now returns an arena to22:20
hazmat  the system ``free()`` when all memory within an arena becomes unused22:20
hazmat  again.  Prior to Python 2.5, arenas (256KB chunks of memory) were never22:20
hazmat  freed.  Some applications will see a drop in virtual memory size now,22:20
hazmat  especially long-running applications that, from time to time, temporarily22:20
hazmat  use a large number of small objects.  Note that when Python returns an22:20
hazmat  arena to the platform C's ``free()``, there's no guarantee that the22:20
philiKONccomb: look at how zope.app.container 3.5.x does bookeping with the length22:20
hazmat  platform C library will in turn return that memory to the operating system.22:20
hazmat  The effect of the patch is to stop making that impossible, and in tests it22:20
hazmat  appears to be effective at least on Microsoft C and gcc-based systems."""22:20
philiKONhazmat: dude...22:20
hazmatyeah.. yeah.. minor flood..22:20
hazmatcouldn't link to it effectively22:20
philiKONanyway, no need to iterate over btree.keys(), you can iterate over mappings directly22:21
philiKONfor key in btree22:21
ccombphiliKON ok I think it's the second time I get into this problem of many small objects22:21
philiKONnow, if you're doing   x = btree[key] in the loop, it's much more efficient to use items(), i believe22:21
philiKONfor key, x in btree.items()22:22
ccombeven if I read every single item of the btree?22:22
philiKONyup22:23
philiKONfor key in btree:   will basically traverse the btree structure22:23
philiKONwhen you're doing  btree[key] inside the loop then, it'll have to traverse it *again*22:23
*** redir has joined #zope3-dev22:23
*** elro has joined #zope3-dev22:24
ccombok thanks a lot, I'll try and improve my loops22:24
*** zagy has joined #zope3-dev22:26
*** xenru has joined #zope3-dev22:30
*** MJ|dinner is now known as MJ22:30
*** gidas has joined #zope3-dev22:34
*** spython has joined #zope3-dev22:36
*** junkafarian has joined #zope3-dev22:39
spythonhi, i checked out a zope package from svn. Is there easy way to know make egg of this? Not a binary egg, but one which has src/ and EGG-INFO/ in it22:40
ccombspython have you read all the PEAK docs?22:42
spythonhmm, not all. but from where i look it says how to make a binary egg (setup.py bdist_egg)22:43
*** alga has quit IRC22:49
*** hexsprite_ has quit IRC22:52
*** Ariel_Calzada has joined #zope3-dev22:57
*** redir has quit IRC23:00
*** RaFromBRC is now known as RaFromBRC|lunch23:13
*** zagy has quit IRC23:15
*** MJ|nickcollision has joined #zope3-dev23:16
*** MJ has quit IRC23:17
*** MJ|nickcollision is now known as MJ23:17
philiKONspython: python setup.py bdist_egg23:18
philiKONRTFM setuptools :)23:18
*** RaFromBRC|lunch is now known as RaFromBRC23:27
*** junkafarian has quit IRC23:29
*** hexsprite has joined #zope3-dev23:32
*** junkafarian has joined #zope3-dev23:32
*** nathany has quit IRC23:41
mcdoncsrichter, J1m, philiKON: see "egg bot install results" message on zope-dev re: easy_install vs. zope eggs23:41
*** nathany has joined #zope3-dev23:41
spythonphiliKON: hehe, I will!23:43
srichtermcdonc: do you want answers here?23:43
*** TresEquis has joined #zope3-dev23:43
mcdoncsrichter: whatever works for ya...23:43
srichtermcdonc: the reason it works for buildout is (a) that some buildout.cfg files specify versions, such as zope.traversing=3.4.023:44
philiKONno they don't23:44
srichter(b) they use the KGS23:44
philiKONah, buildout.fg23:44
philiKONsorry23:44
srichterphiliKON: some packages do23:44
philiKONthe conflict mcdonc is seeing is the result of some premature release hiatus23:44
srichteryep23:44
srichterI just stated the reason why it works in buildout23:45
philiKONhaving spoken to the person who was responsible, we all agree that those releases should never have been made23:45
philiKONit *is* however terribly hard to resolve conflicts with setuptools23:45
philiKONbecause it has no conflict resolution whatsoever23:45
philiKONthe setup is always this:23:45
xenruphiliKON: hi23:46
philiKONA requires B which requires any version of C23:46
philiKONso some version of C gets installed23:46
philiKONA also requires D which requires C >= x.y23:46
srichterright23:46
philiKONand the >= x.y qualifier may be in conflict with that any version of C that was installed earlier on23:46
philiKONbasically, setuptools installs blindly, and doesn't resolve first23:47
srichteryep23:47
philiKONthat said, it's what we have to work with23:48
philiKONand we *should*23:48
philiKONso we should find a way around that23:48
mcdoncwould it be heresy to suggest that whatever base package we're getting that requires zope.traversing say "zope.traversing >= 3.4.0, <= 3.4.99"23:48
mcdoncin install_requires23:48
srichtermcdonc: specifying versions in setup.py has a lot of problems; see recent discussions23:49
philiKONspecifying '==' in setup.py creates problems23:50
srichterphiliKON: even if the release of zope.traversing 3.5.0dev was not a mistake, we woul dhave this problem23:50
philiKONsrichter: yup, but we could be more careful23:50
philiKONwe still need the >= qualifiers in setup.py23:50
mcdoncsrichter: i understand the desire to not put version #'s in reuirements given the KGS stuff, but how's it any worse than specifying them in buildout.cfg?23:50
srichterI totally disagree23:50
philiKONi suppose the <= qualifiers are necessary sometimes too23:50
srichterversions should not be in buildout.cfg either23:51
philiKONsrichter: the bug we had with zope.app.securitypolicy (and zope.securitypolicy btw) demonstrated this nicely23:51
philiKONbut we already had this discussion23:51
philiKONIMO, setup.py needs to contain all intrinsic dependency information23:51
srichterthe problem is that as soon as we are maintaining versions in the packages, we are in the business of releasing23:51
philiKONand if dependency X is needed in at least version x.y, then it should say X >=x.y23:51
srichterand we simply do not have the manpower to constantly release packages23:51
philiKONi don't understand how we would do more releases than we do now23:52
srichterwhat if someone backports a fix, and now x-1.z is working as well?23:52
srichterhe has to go through all packages that list X as a requirement and add that statement23:52
J1mmcdonc, version requirements for library packages should be as loose as possible, so we really want to avoid setting them there.23:53
J1mmcdonc, setting upper limits is especially problematic.23:53
srichterthe nicest part about the KGS is that releasing a new working configuration is a matter of uploading one file23:53
philiKONJ1m: but what about the lower limit in case of a new feature?23:53
mcdoncJ1m: yup... agreed.  But maybe these things shouldn't get uploaded to cheeseshop and should just be part of a kgs then...23:53
J1mwell, that is necessary and shouldn't cause a problem.23:54
philiKONJ1m: thanks, just wanted to know23:54
J1mIt's the upper limit that is problematic.23:54
philiKONright, i agree23:54
J1mwhich is why we really need to avoid backward incompat.23:54
srichterJ1m: have you read Roger's argument? I think he makes a great case on why even a lower limit is a problem for us23:54
gstratton_srichter: Hi, what about people using zope packages outside of zope. Surely they need version requirements?23:55
J1msrichter, lower limits are often unavoidable.23:55
*** dunny has quit IRC23:55
J1mgstratton_, if there is a true requirement, then sure.23:55
srichterJ1m: but that removes a subset of working combinations23:55
philiKONmissing lower limits can actually cause bugs, like the one we had with zope.app.securitypolicy. it was accepting any version of zope.i18nmessageid while it clearly needed at least 3.4.223:56
srichterof course you could claim that the benefits outweigh the downsides23:56
J1mso I guess that some package set an upper limit on zope.traversing.23:56
J1mI wonder why.23:56
J1msrichter, that's unavoidable.  Using a KGS removes a subset of working combinations too.23:56
philiKONsrichter: but zope.app.securitypolicy simply won't work with zope.i18nmessageid <3.4.2. hence we need to make sure that it can't be installed that way23:57
srichterJ1m: yep, I agree, but you then have the freedom to create your own KGS23:57
srichterJ1m: by limiting the set in setup.py you globally remove the posibilities23:57
J1mDid someone make a backward incompatible change to zope.traversing?23:57
srichter(mmh, or could we teach buildout to ignore setup.py version specificatins)23:58
J1msrichter, you really have no choice, sometimes, you just know that certain versions don't work,.23:58
srichterJ1m: I totally understand your argument23:58
mcdoncsrichter: if you do that, please dont upload the resulting eggs to the cheese shop23:58
J1mDid someone make a backward incompatible change to zope.traversing? :)23:58
srichterJ1m: but the problem comes into play if a lower is made to work later23:58
*** dunny has joined #zope3-dev23:58
srichterJ1m: zope.traversing; no23:59
J1mthen why did someone set an upper bound on it?23:59
srichterJ1m: see philiKON's explanation above23:59
srichterI don't think anyone did23:59

Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!