IRC log of #zope3-dev for Sunday, 2009-08-02

*** alecm has quit IRC00:11
*** philiKON has joined #zope3-dev00:40
*** tisto has quit IRC00:48
*** JaRoel|4D has quit IRC01:23
*** allisterb_ has joined #zope3-dev01:26
*** drudi has joined #zope3-dev01:31
*** allisterb has quit IRC01:55
*** aaronv has joined #zope3-dev01:59
*** JaRoel|4D has joined #zope3-dev02:08
*** aaronv has quit IRC02:12
*** sunoano has quit IRC02:23
*** sunoano has joined #zope3-dev02:24
*** pcardune has joined #zope3-dev02:36
*** aaronv has joined #zope3-dev02:37
*** aaronv has quit IRC02:52
*** pcardune has quit IRC02:53
*** redir has quit IRC02:59
*** ignas has quit IRC03:18
*** alga has quit IRC03:50
*** romanofski has joined #zope3-dev05:04
*** jamur2_ has joined #zope3-dev05:06
*** J1m has quit IRC05:07
*** jamur2_ has quit IRC05:09
*** pcardune has joined #zope3-dev06:08
*** afd__ has quit IRC06:16
*** afd__ has joined #zope3-dev06:17
*** philiKON_ has joined #zope3-dev06:28
*** kaeru_ has joined #zope3-dev06:39
*** kaeru has quit IRC06:44
*** philiKON has quit IRC06:45
*** kaeru_ has quit IRC07:20
*** redir has joined #zope3-dev07:22
*** srichter has joined #zope3-dev07:28
*** ChanServ sets mode: +o srichter07:28
*** romanofski has quit IRC07:41
*** pcardune has quit IRC07:50
*** pcardune has joined #zope3-dev08:08
*** drudi has quit IRC08:11
*** pcardune has quit IRC08:12
*** pcardune has joined #zope3-dev08:18
*** pcardune has quit IRC08:24
*** kursor has joined #zope3-dev08:30
*** srichter_ has joined #zope3-dev08:33
*** srichter has quit IRC08:34
*** dunny has joined #zope3-dev08:39
*** kursor has left #zope3-dev08:53
*** srichter_ has quit IRC09:46
*** sunoano has quit IRC10:07
*** afd_ has joined #zope3-dev10:11
*** afd__ has quit IRC10:11
*** malthe|Zzz is now known as malthe10:15
*** romanofski has joined #zope3-dev10:18
*** redir has quit IRC10:23
*** sunoano has joined #zope3-dev10:27
*** Theuni1 has joined #zope3-dev10:31
*** tisto has joined #zope3-dev10:48
*** romanofski has quit IRC10:54
*** kursor has joined #zope3-dev11:03
*** junkafarian_ has joined #zope3-dev11:24
*** kursor has left #zope3-dev11:28
*** malthe is now known as malthe|away11:33
*** afd_ has quit IRC11:34
*** junkafarian_ has quit IRC11:35
*** afd__ has joined #zope3-dev11:47
*** jhauser has joined #zope3-dev12:18
*** pelle_ has joined #zope3-dev12:21
*** yota has joined #zope3-dev13:19
*** afd__ has quit IRC13:39
*** kaeru has joined #zope3-dev13:58
*** afd__ has joined #zope3-dev14:14
*** romanofski has joined #zope3-dev14:50
*** J1m has joined #zope3-dev15:13
*** pelle_ has quit IRC15:25
*** philiKON_ has quit IRC15:36
koboldJ1m: do you have a couple of minutes?15:38
J1msure15:38
koboldJ1m: I was offline for two weeks, and I'm reading the backlog of the IRC channel. I read the wonderful work srichter did on zope.release15:39
J1mYup, although there seems to be more to do there.15:39
koboldI just wonder if it wouldn't be easier to automate something like: for each package which is part of the ZTK, do a svn check out and run the tests15:39
koboldit looks it is quite complex to run all the tests together...15:40
J1mMight be.15:40
koboldI suppose we need two type of checks: if I modify a package, I want to test all the packages that use it to check if I broke something15:40
koboldand we have to ensure that a given set of packages work together15:41
J1mIt turns out that srichter ran them individually.15:41
kobolduh? really? I didn't know it15:41
J1mI tried to reproduce his report and got failures.15:41
koboldme too15:42
J1mwhen I asked him about it here, it turns out that he was running the tests separately.15:42
J1mThen we got into a discussion of how to automate *that*.15:42
koboldI have that automatized with buildbot, but it is not working with the versions from the KGS (yet)15:43
J1mWe either need a facility in zope.kgs (used by zope.release) to run packages separately, or15:43
*** dunny has quit IRC15:43
J1mwe need to update the thing that Theuni1 and others did to make it reproduceable.15:44
* kobold doesn't know anything about "the thing" :)15:44
J1mhm, cool15:44
Theuni1Btw: wolfgang told me that compattests already does the right thing: it runs the tests of each package against buildout's working set15:44
J1mTheuni1, ayt?15:44
Theuni1yes15:44
Theuni1i'm listening15:44
koboldTheuni1: hi!15:44
* Theuni1 waves15:44
J1mWhat's the name of the testing system you did that is kinda like zope.kgs?15:45
Theuni1z3c.compattest15:45
J1mthat you did for the grok sprint?15:45
J1mah yes15:45
J1mkobold, that. :)15:45
J1mSo that is an alternative to zope.kgs.15:45
Theuni1I have the feeling that whatever it is that's blocking you from using it is very minor and I just haven't understood what it is yet.15:46
koboldso, it looks like switching to z3c.compattest and automatizing it is the solution15:46
J1mwhat I think we need is something that:15:46
J1m1. A definition of standard packages and versions that need to be tested when a change is made, and15:46
J1m2, someting that automates running their tests.15:46
koboldIMHO, 2 is more important than 115:47
J1m3. and that runs each package's tests in a separate process.15:47
koboldbecause after you have 2, you can start from the bottom of the dependency tree...15:47
J1mkobold, well z3c.compattest gives you that.15:47
kobold...and add the packages going up, through the dependencies, to find out the best set of core packages15:47
J1mbut without 1, it will rot again.15:48
Theuni12 and 3 are there. 1 is there in some form that might need tweaking15:48
koboldit also gives us 3, it seems15:48
Theuni1But I think 1 is there in a way that helps already.15:48
Theuni1Can we get more specific on how 1 should work?15:48
J1mTheuni1, we need 1 and we need it without making people think.15:48
Theuni1jup15:48
koboldTheuni1: where can I find z3c.compattest?15:48
Theuni1kobold: actually its z3c.recipe.compattest15:48
koboldTheuni1: z3c.recipe.compattest, right?15:48
Theuni1yes15:48
koboldok, found :)15:49
Theuni1You include that in the buildout of the package you're working on15:49
koboldTheuni1: ahh15:49
J1mTheuni1, there should be some instructions at some url that people can go to and quickly figure out how to run the tests.15:49
koboldjust reading the README, I like the idea, it is just wonderful15:49
Theuni1J1m: agreed15:49
J1mThe instructions should be a long the lines of:15:49
J1m- check someting out15:49
J1m- give some canned commands (e.g. bootstrap, buildout, run a test script15:50
J1m- if tests pass, update a specifification file to record a new blessed version15:50
J1malthough blessing a new ztk thing might need more ceremony. Not sure.15:51
J1mA big problem is that people check in tests that pass on one python version and os and fail on another.15:51
koboldfine, I'm working on this, let's see if I can get something decent in a reasonable time :)15:51
J1mI *hate* fixing other people's windows test failures. :)15:52
koboldJ1m: you use buildbot for that, you can't really rely on tests run by the developers15:52
Theuni1J1m: and I hate producing them but then again windows is such an annoyance wrt getting a dev environment ...15:52
J1mTheuni1, I agree :)15:52
J1mkobold, I don't understand what you just said.15:53
J1mwhy can't I rely on tests run by developers?15:53
koboldJ1m: because nobody run tests on all the python versions and on different platforms before doing a checkin, I suppose15:53
koboldtest automatization is the only way to ensure that a set of tests pass on different platforms and different python versions15:54
J1mright, but we could run buildbots or the equivalent on all python versions and platforms.15:54
koboldthat's what I said :)15:54
J1mBut then we need some sort of process that works with that.15:54
J1mbrainstorming...15:55
koboldyep15:55
J1mmaybe there's stage branch.15:55
J1mwhen you get tests passing you merge to a stage branch.15:55
Theuni1J1m: something that compattest needs is a specification of the controlled packages which need to be tested. unfortunately we don't have a reliable format for that yet.15:55
koboldmaybe each checkin should trigger a rebuild/test of all the related packages15:56
J1mthe buildbots run tests on the stage branch and the trunk.15:56
Theuni1I looked at KGS controlled-packages list and versions.cfg but they include things we don't want to test15:56
J1mwhen the bots of run the tests on all of the platforms and python versions opn the stage branch, we merge from that to the trunk.15:56
J1mTheuni1, that's not a format issue.15:57
J1mThat's a scope issue.15:57
J1mI agree that the scope of zope.release is wider than it should be.15:57
Theuni1Well... if the scope would be ok we currently would also have a format issue ;)15:58
Theuni1But I agree, it's a scope issue first15:58
J1mto solve my problem, I just want to tests the foundation packages that are used in a zope process,15:58
* kobold was surprised to find lovely.something packages in zope.release, indeed15:58
J1mTheuni1, I don't understand the format issue.15:58
Theuni1J1m: we tried to get a reasonable scope using heuristics for compattest15:58
J1mall you need is:15:58
J1m1. A list of packages to test15:58
Theuni1J1m: yeah, the kgs currently produces a versions.cfg which people use to extend their buildout15:59
J1m2. a versions.cfg (or equivalent) that specifies the versions of packages and dependencies.15:59
Theuni1but the versions section will be merged with other stuff15:59
Theuni1so using the versions section as a list isn't right15:59
J1mI don't understand.15:59
Theuni1I think we both try to say the same thing.16:00
J1mThat would be cool. :)16:00
Theuni1Lemme try again: point 1 is a problem because kgs' scope is too broad16:00
Theuni1even if the scope would be fine I couldn't use it currently because the only output from KGS that I can "see" is the versions.cfg16:01
J1mLemme try again. :)16:01
Theuni1i could enumerate the versions section but even if the kgs' scope would be fine it would be cluttered from other places.16:01
J1mcluttered from where?16:02
Theuni1e.g. the version pins from dependencies16:02
J1mwhat's wrong with that?16:02
Theuni1Should those be included in the list of packages to tset?16:02
Theuni1test?16:02
J1mno16:02
J1mnot imo16:02
Theuni1that's why i can't enumerate the versions section for determining which packages' tests to run16:02
Theuni1but the versions.cfg is the only output format i currently see16:03
Theuni1(from kgs that is)016:03
J1mI think you missunderstand the kgs16:03
Theuni1Might be16:03
J1mno16:03
*** drudi has joined #zope3-dev16:03
J1mJust a sec,16:03
Theuni1sure16:03
J1msee http://svn.zope.org/zope.release/trunk/releases/controlled-packages.cfg?rev=102349&view=auto16:04
J1mThis is what's used by kgs.16:04
J1mNote that I'm not advocating kgs.16:04
Theuni1ack16:05
J1mHere, you have a list of packages and you specify which ones to test.16:05
J1mAn alternative would be a separate list of packages to test and a versions.cfg.16:05
J1mI'm sure there are many other alternatives.16:06
Theuni1We "just" have to pick one specific one that works ;)16:06
J1mwrt kgs, zope.kgs is just mechanism.  clients of it, like zope.release define a scope.  I just say this for clarity, not because I'm advocating it.16:07
J1mIf I were doing this, I'd probably go with a list of packages to test and a versions.cfg, as that leverages machnery we already have.16:07
Theuni1right16:07
Theuni1that's why i built compattest the way it is16:07
Theuni1i rely on the working set that buildout already has activated + the option to get develop eggs checked out from svn.zope.org svn alternatively16:08
*** ccomb has joined #zope3-dev16:09
Theuni1Now I'm trying to find a place where that list of packages already lives so I can reuse that.16:09
J1mwhich list? the ztk list you created?16:09
J1mor the one that zope.release uses?16:09
Theuni1Rephrase: I'm trying to find a place where the list (with right scope) of the packages  that need to be tested already lives (assuming it already exists). The ZTK list I currently consider work in progress ... maybe that's the list I should be consuming for compattest?16:11
J1mMaybe, or you could start from zope.release and remove things.16:11
Theuni1Question about mechanics: will I be able to modify the version section on the fly from a recipe?16:12
J1mBTW, I think compattest should probably provide general mechanism and then provide a way to "instantiate" it for specific test sets.16:12
Theuni1ack16:13
J1mYou shouldn't modify the versions section. Why would you want to?16:13
Theuni1Uhm ... never mind ... I guess the compattest would need to allow specifying which packages to test and the equivalent of the versions section.16:14
Theuni1So it needs to replicate the versions behaviour for its own working set16:15
J1mright16:15
Theuni1ok, i think i understand now16:15
Theuni1I'm not sure when I'll have time to do that, but at least I understand what's missing.16:16
J1mNote that one approach I was thinking of was to generate a single test script and find a way to tell the test runner to run each package in a separate subprocess.16:16
Theuni1hmm. that's probably hard to get right16:17
J1mNow you generate a unit test layer and assign it to layerless tests.16:17
Theuni1the current benefit is that each test runner script really only gets the sys.path include the exactly specified dependencides16:17
Theuni1Me?16:17
Theuni1I don't do that AFAIK16:17
J1mYes, when you refactored the test runner.16:17
*** kiorky has quit IRC16:18
Theuni1ah on that end16:18
J1mI thought you did,16:18
Theuni1sorry, i thought you were talking about the compattests16:18
Theuni1yes i did16:18
J1mso you could instead generate a unit test layer per package.16:18
Theuni1I could, except that the isolation benefit of independent testing wouldn't be there.16:18
J1mand use the test runner's facility for running each layer in a subprocess.16:18
J1msure it would16:18
Theuni1how?16:19
J1mif each package ran in a separate process.16:19
Theuni1You're missing that the test runner will run with a working set larger than the actual dependencies of the package under testing16:19
J1mwhich it would by giving each it's own layer and using the -j option.16:19
Theuni1so we couldn't find incorrect dependency declarations16:19
J1mThat's probably a good thing.16:19
Theuni1which is one of the benefits16:19
Theuni1I agree on the approach of splitting up the unit test layer much more, though16:20
J1mI guess it depends on what  your goals are.16:20
Theuni1wrt to running in parallel16:20
J1mBut I see your point.16:20
Theuni1We wrote compattest because we were refactoring packages heavily, moving code around and wanted to make sure the consumer packages of that code would still work independently.16:20
J1mI manually split up the zodb tests into layers to be able to run them in parallel.16:21
J1mright, I get that.16:21
Theuni1I think another approach would be for the test runner to just split layers when they grow larger than X tests16:21
J1mperhaps.16:21
Theuni1what we're doing there isn't as much an isolation thing but managing performance tradeoffs16:22
J1mThe ZEO tests tend to get lots of benefit because they are generally not cpu bound. OTOH, they tend to be very time sensitive, so I can only run them in parallel on fairly fast machines.16:23
J1mI need to fix a lot of those tests to be less timing dependent. Those tests are so painful to deal with. But I digress. :)16:23
*** kiorky has joined #zope3-dev16:25
Theuni1I'll try to get that compattest stuff done at some point. Need to go back to what I'm actually doing now :)16:26
J1mTheuni1, kobold wants to help.16:27
Theuni1Cool.16:27
J1mso maybe you can help him help you :)16:27
Theuni1kobold: a pointer would be that the include/exclude-mechanism would need to be replaced by retrieving a list from a file like jim posted (from a zope.release)16:28
J1mAlthough he's been pretty quiet the last few minutes.16:28
Theuni1We'll see :)16:28
*** tarek has joined #zope3-dev16:28
Theuni1the second step would be to not rely on the buildouts working set but build a new one based on the versions in that file16:28
Theuni1after that we should be where we want to be, maybe allowing to specify more specific versions for packages that aren't in the list would be nice16:29
J1mI don't know what you mean by the buildout's working set.16:29
J1mdoesn't compattest generate a separate script for each package?16:29
Theuni1yes16:30
J1mEach script would have it's own working set, even if you use the buildout machinery to generate it.16:30
Theuni1but it uses the currently running buildout to find out which versions of packages to use16:30
Theuni1Here's what happens:16:31
Theuni1    def _working_set(self, package):16:31
Theuni1        eggs = zc.recipe.egg.Egg(self.buildout, self.name, dict(eggs=package))16:31
J1mAll of the tests should use the same versions.  They should each have their own working sets, but when a package is used by 2 different test scripts, the same version should be used.16:31
Theuni1then we extract that working set16:31
Theuni1for each test runner16:31
Theuni1i know16:31
J1mcool, so what's the problem? :)16:32
Theuni1Imagine you set up the compattest part two times, then you want each to use different versions, e.g. the 3.4 versions for one and the 3.5 versions for the other16:32
J1mah16:33
Theuni1the set of versions needs to be specified for the part not using buildout's global mechanism16:33
*** __mac__ has joined #zope3-dev16:33
J1mwell, if you want to do that, you should extend zc.recipe.egg to be able to take a versions option.16:33
Theuni1sounds like a good idea :)16:34
Theuni1kobold: did jim and I produce enough text to get you confused? :)16:34
J1mwhen I started working on buildout, I tended to implement options both locally and globally.16:34
J1mBut that was a lot of work and we rarely needed to override an option locally so I stopped bothering with local options.16:35
J1mBut is is always an option to add a local override when needed.16:35
Theuni1good to know that history16:35
J1mI think we scared kobold away.16:35
Theuni1:/16:35
*** sweh has joined #zope3-dev16:35
J1mactually, my irc client shows him as away.16:36
Theuni1Maybe he realized it's sunday after all16:36
J1mMaybe he'll come back later and catch up.16:36
Theuni1we'll see16:36
J1mLet's both get back to what we were doing before kobold started this discussion.16:36
Theuni1ack16:36
*** __mac__ has quit IRC16:41
*** romanofski has quit IRC17:10
*** drudi has quit IRC17:11
*** sweh has quit IRC17:16
*** afd__ has quit IRC17:27
*** afd__ has joined #zope3-dev17:33
*** redir has joined #zope3-dev17:42
*** projekt01 has joined #zope3-dev18:05
*** afd__ has quit IRC18:26
*** afd__ has joined #zope3-dev18:27
*** sunoano has quit IRC18:27
*** sunoano has joined #zope3-dev18:29
*** ccomb has quit IRC18:36
*** pcardune has joined #zope3-dev18:40
*** kursor has joined #zope3-dev18:51
*** redir has quit IRC18:52
*** redir has joined #zope3-dev18:56
*** redir has quit IRC19:07
*** redir has joined #zope3-dev19:09
*** pcardune has quit IRC19:11
*** kursor has quit IRC19:16
*** JaRoel|4D has quit IRC19:17
*** JaRoel|4D has joined #zope3-dev19:19
*** afd__ has quit IRC19:22
*** projekt01 has quit IRC19:36
*** pcardune has joined #zope3-dev19:49
*** pcardune has quit IRC19:56
*** alecm has joined #zope3-dev19:56
*** alecm has quit IRC19:57
*** aaronv has joined #zope3-dev20:04
*** tarek has quit IRC20:05
*** Theuni1 has quit IRC20:26
*** tarek has joined #zope3-dev20:27
*** drudi has joined #zope3-dev20:40
*** junkafarian_ has joined #zope3-dev21:07
*** sweh has joined #zope3-dev21:11
*** kursor has joined #zope3-dev21:19
*** drudi has quit IRC21:26
*** pelle_ has joined #zope3-dev21:27
*** pelle_ has quit IRC21:30
*** pelle_ has joined #zope3-dev21:31
*** sweh has quit IRC21:31
*** malthe|away is now known as malthe21:51
*** drudi has joined #zope3-dev22:07
*** junkafarian_ has quit IRC22:09
*** J1m has quit IRC22:10
*** yota has quit IRC22:12
*** redir has quit IRC22:14
*** tarek_ has joined #zope3-dev22:14
*** tarek has left #zope3-dev22:14
*** tarek_ has quit IRC22:15
*** tarek has joined #zope3-dev22:15
*** pelle_ has quit IRC22:16
*** kursor has quit IRC22:45
*** alga has joined #zope3-dev23:02
*** greenman has joined #zope3-dev23:53

Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!