IRC log of #zope3-dev for Thursday, 2008-03-27

*** b52laptop has joined #zope3-dev00:00
*** norro_ has quit IRC00:04
*** ktwilight has quit IRC00:05
*** d2m has quit IRC00:27
*** whitmo has joined #zope3-dev00:30
*** whit has quit IRC00:33
*** whitmo is now known as whit00:33
*** lucielejard has quit IRC00:33
malthewhat's required to run z3 on python 2.5?00:35
*** J1m has quit IRC00:39
whitis there a buildout equivalent of "dependency_links"?00:52
* whit realize the answer to his own question00:52
*** reco has quit IRC00:53
*** greenman has joined #zope3-dev00:59
*** benji has quit IRC01:00
*** norro has quit IRC01:01
*** nathany has quit IRC01:10
*** tarek_ has joined #zope3-dev01:11
*** tarek has quit IRC01:18
*** salfield has quit IRC01:25
*** tarek_ has quit IRC01:26
*** tarek_ has joined #zope3-dev01:26
*** rmarianski has quit IRC01:32
*** greenman has quit IRC01:38
*** _tarek_ has joined #zope3-dev01:51
*** tarek_ has quit IRC02:00
hazmatmalthe, as far as i know, nothing special, just worked for me...02:07
*** malthe has quit IRC02:09
*** sm has quit IRC02:15
*** alecm has quit IRC02:35
*** bigkevmcd has quit IRC02:42
*** hazmat has left #zope3-dev02:43
*** rcrafton has quit IRC02:45
*** djohnson has joined #zope3-dev03:10
*** jsadjohnson has quit IRC03:16
*** salfield has joined #zope3-dev03:21
*** niemeyer has quit IRC03:32
*** djohnson has quit IRC03:53
*** salfield has quit IRC04:01
*** pcardune has quit IRC04:07
*** rcrafton has joined #zope3-dev04:33
*** whit has quit IRC05:08
*** redir has joined #zope3-dev05:43
*** sm has joined #zope3-dev06:22
*** sm has quit IRC06:51
*** whit has joined #zope3-dev06:58
*** rcrafton has quit IRC07:02
*** afd_ has joined #zope3-dev07:04
*** whit has quit IRC07:15
*** timte has quit IRC07:33
*** timte has joined #zope3-dev08:20
*** timte has quit IRC08:21
*** timte has joined #zope3-dev08:21
*** jukart has joined #zope3-dev08:27
*** menesis has joined #zope3-dev08:31
*** d2m has joined #zope3-dev08:38
*** afd_ has quit IRC08:39
*** jodok has joined #zope3-dev08:41
*** pelle_ has quit IRC08:53
*** sorin has joined #zope3-dev09:05
*** sorin is now known as sorindregan09:05
*** stub has joined #zope3-dev09:15
*** yvl has joined #zope3-dev09:18
*** goschtl has joined #zope3-dev09:24
*** agroszer has joined #zope3-dev09:34
*** b52laptop has quit IRC09:34
*** afd_ has joined #zope3-dev09:40
*** quodt has joined #zope3-dev09:41
*** flox has joined #zope3-dev09:42
*** dobee has quit IRC09:44
*** stu1 has joined #zope3-dev09:56
*** stub has quit IRC09:56
*** stu1 is now known as stub09:56
*** dobee has joined #zope3-dev09:57
*** toko^ has joined #zope3-dev09:57
*** toko has joined #zope3-dev09:59
*** jpcw2002 has joined #zope3-dev10:03
*** pelle_ has joined #zope3-dev10:07
*** markusleist has joined #zope3-dev10:22
*** harobed has joined #zope3-dev10:33
*** MJ has joined #zope3-dev10:34
*** zagy has joined #zope3-dev10:38
*** _tarek_ has quit IRC10:39
*** tarek has joined #zope3-dev10:50
*** alecm has joined #zope3-dev10:52
*** mkerrin has joined #zope3-dev11:01
*** greenman has joined #zope3-dev11:08
*** thruflo has joined #zope3-dev11:10
*** norro has joined #zope3-dev11:13
*** malthe has joined #zope3-dev11:23
*** thruflo has quit IRC11:26
*** thruflo has joined #zope3-dev11:26
*** b52laptop has joined #zope3-dev11:31
*** maurits has joined #zope3-dev11:32
*** kursor has joined #zope3-dev11:44
*** faassen has joined #zope3-dev11:58
*** dunny has quit IRC12:24
*** projekt01 has joined #zope3-dev12:24
*** pelle_ has quit IRC12:33
*** pelle_ has joined #zope3-dev12:33
*** salfield has joined #zope3-dev12:41
*** pelle_ has quit IRC12:49
*** pelle_ has joined #zope3-dev12:49
*** yvl has left #zope3-dev12:53
*** lisppaste6 has quit IRC13:07
*** malthe is now known as malthe|away13:18
*** lisppaste6 has joined #zope3-dev13:20
*** greenman has quit IRC13:21
*** pelle_ has quit IRC13:21
*** pelle_ has joined #zope3-dev13:22
*** ct_ has joined #zope3-dev13:36
*** MJ is now known as MJ|lunch13:44
*** malthe|away is now known as malthe13:47
*** afd_ has quit IRC13:48
*** ignas has joined #zope3-dev14:02
*** tesdal has joined #zope3-dev14:06
malthetesdal: too much noise14:06
tesdalwhat's up d4wgs14:07
maltheso collective.indexing is a bit Plone-specific14:07
*** jpcw2002 has quit IRC14:07
tesdalmalthe: a bit I guess14:07
malthesomewhat without reason, right?14:07
tesdalmalthe: we'd love to make it completely independent of Plone14:07
maltheright---put the integration stuff in, say, plone.search14:07
tesdalsearch is extraction14:08
tesdalindexing is indexing14:08
tesdalbut yes14:08
maltheok gotcha14:08
tesdalit makes sense to use a plone namespace for plone specifics14:08
maltheat any rate, the indexing abstractions and implementation should probably be zope-only.14:08
tesdalguess that makes sense as well14:09
malthehmm how to proceed14:09
tesdalwe put it in collective.indexing first because it was useful at this stage14:09
tesdalbut it would be great if parts went into Zope itself or a zope namespace, and plone specifics in a plone namespace14:10
maltheso on one hand, you have ore.xapian, which is an abstraction *and* an implementation14:10
tesdaland we could plip and include in plone 414:10
tesdalI think ore.xapian is the same, it solves a problem and it made sense to do it like that when it was implemented14:10
tesdalbut I think the concepts and abstractions are quite similar for all of the indexing solutions14:11
tesdalmalthe: are you at a sprint?14:13
malthetesdal: yes14:13
tesdalmalthe: are you working together with someone on indexing?14:14
*** niemeyer has joined #zope3-dev14:14
malthetesdal: yes, sylvain from infrae14:15
*** benji has joined #zope3-dev14:15
tesdalmalthe: we'd love some feedback on collective.indexing even if you choose not to use it in any way14:16
tesdalmalthe: whether the ideas and components are sane, and if you're able to understand what's going on in there etc14:16
*** goschtl has quit IRC14:18
tesdalmalthe: I've got a blog posting draft available,
*** toutpt has joined #zope3-dev14:22
*** jpcw2002 has joined #zope3-dev14:27
*** alecm has quit IRC14:28
*** sm has joined #zope3-dev14:31
*** djohnson has joined #zope3-dev14:32
*** witsch has joined #zope3-dev14:40
witschhey malthe, tesdal :)14:40
tesdalmalthe: witsch has done most of the coding on collective.indexing14:40
witschmalthe: yeah, i guess after winning our code contest i got to do the rest as well... %)14:43
*** goschtl has joined #zope3-dev14:44
*** djohnson has quit IRC14:48
witschgoschtl: let's discuss things over here, though... malthe and tesdal have already been talking about collective.indexing14:50
*** mgedmin has joined #zope3-dev14:51
goschtlwitsch: ok this channel14:56
tesdalgoschtl: any comments so far?14:57
*** rcrafton has joined #zope3-dev15:00
goschtltesdal: no i think i understand what happens. looks very clean to me.15:01
tesdalgoschtl: excellent :)15:02
goschtlwhat are your plans with other indexes date, path, extendedpath ...15:02
goschtlfor collective.solr?15:02
tesdalgoschtl: we don't have any plans for extendedpath in collective.solr15:03
tesdalgoschtl: but you can replicate a lot of the functionality by /query/by/path/* and /explicity/path/query15:04
tesdalgoschtl: I think there is a date index already?15:04
tesdalit might be possible to make a tokenizer/analyzer in solr that replicates at least pathindex15:09
*** kursor has quit IRC15:09
witschtesdal: uh, java? :)15:10
tesdalif needed15:10
tesdalbut it's just about how to split stuff15:10
tesdalsplit in / and add a level indicator or whatever15:10
* tesdal thinks string and asterisk will work15:11
goschtltesdal: this means you will have a plonesite with indexes in solr and idexes in zcatalog, no replace of zcatalog?15:11
tesdalgoschtl: yes15:12
tesdalgoschtl: but that's not a problem15:12
*** djohnson has joined #zope3-dev15:13
witschgoschtl: you can always unregister the catalog-index-processor, effectively turning off indexing in zcatalog15:15
witschgoschtl: that would be just one call to the ca15:15
witschtesdal: perhaps we're gonna need some ui after all ;)15:16
tesdalthe button15:17
*** alecm has joined #zope3-dev15:19
*** whit has joined #zope3-dev15:19
*** johbo has joined #zope3-dev15:19
tesdalthe exotic navtree builder code in EPI is going to be hard to replicate I think15:19
*** lucielejard has joined #zope3-dev15:24
witschtesdal: the hard part will be to find somebody who's willing to do this... :)15:25
tesdalmaybe it can be replicated with a different data structure, or maybe we can build the navigation differently15:25
witschtesdal: but perhaps plone should adopt the new explorer portlet and dump the concept of using the catalog for navigation all together15:25
tesdalwitsch: something like that15:26
witschtesdal: it looks like plone.folder should actually make it possible to not even need the catalog anymore15:26
tesdalsounds good15:26
witschwe'll see :)15:27
*** alga has joined #zope3-dev15:29
*** pelle_ has quit IRC15:29
*** pelle_ has joined #zope3-dev15:29
*** jpcw2002_ has joined #zope3-dev15:30
*** J1m has joined #zope3-dev15:34
*** chtp_ has joined #zope3-dev15:36
*** jpcw2002 has quit IRC15:38
*** jpcw2002_ has left #zope3-dev15:40
*** jpcw2002 has joined #zope3-dev15:42
tesdalhemul in #lucene suggests storing multiple terms in the path field15:43
tesdallike /folder, /folder/with, folder/with/contents15:43
tesdaland possibly some kind of marker at the end15:43
*** MJ|lunch is now known as MJ15:44
tesdalwitsch: "i would implement a tokenfilter that split up the aoslte path in parts, and store the absolute term untokenized in another field."15:46
tesdalin fact we might be able to replicate EPI with some clever tokenization there...15:48
*** chtp has joined #zope3-dev15:48
goschtltesdal: or is it possible to make plone-navigation plone breadcrumbs without epi.  :)15:49
*** ct_ has quit IRC15:49
tesdalgoschtl: of course it is15:49
tesdalthis means we can easily implement regular pathindex15:50
tesdaland EPI implementation would be an interesting challenge, although not strictly necessary15:50
*** pelle_ has quit IRC15:50
*** pelle_ has joined #zope3-dev15:50
goschtliirc regebro has some ideas for a plone-navigation tool based on btrees .15:52
witschtesdal: yep, sounds doable15:53
tesdalgoschtl: we're interested15:53
tesdalthe current navigation solved a problem at the time, but is not the perfect way of doing things15:53
*** jayaraj has joined #zope3-dev15:55
*** chtp_ has quit IRC15:57
whitanybody have any recipes for dealing with eggs in mercurial?15:57
projekt01goschtl, there is z3c.jsontree a JSON-RPC based IContainer tree builder15:58
*** pelle_ has quit IRC16:00
*** pelle_ has joined #zope3-dev16:00
*** pelle_ has quit IRC16:01
*** pelle_ has joined #zope3-dev16:01
*** hazmat has joined #zope3-dev16:02
*** ChanServ sets mode: +o hazmat16:02
whitis it possible to define dependency links directly in a buildout.cfg?16:05
*** jayaraj has quit IRC16:09
*** witsch has quit IRC16:10
*** witsch_ has joined #zope3-dev16:10
hazmatwhit, you mean find links?16:12
hazmatto look for dependencies16:12
whit dependency_links ala setup.py16:12
whitlike "dependency_links=['http://some.dist.tar.gz#egg=MyEgg-dev']"16:13
hazmatwhit, yeah.. i use find links with the same spec16:14
whithazmat: hmmm... I'll try that. in setuptools they mean something different I think16:14
*** witsch_ is now known as witsch16:14
goschtltesdal:  you can see regebros ideas here:
hazmatwhit, it still works the way you want i afaics.. ie pass a tarball as a findlink, attach an egg spec and use it to grab the spec'd egg16:15
whithey hazmat, do you know if there is a way to run a buildout without being inside a folder with a
* whit is trying to modify an existing buildout and is not sure what is a zc.buildout constraint and what is a constraint of the build he is modifying16:19
*** witsch has quit IRC16:22
*** djohnson has quit IRC16:25
*** stub has quit IRC16:25
*** mweichert has joined #zope3-dev16:28
*** rmarianski has joined #zope3-dev16:35
hazmatwhit, buildout doesn't need a anywhere16:37
hazmatwhit, except for development eggs16:37
hazmatand you can specify the path to those16:37
whithazmat: would that be "develop = . " maybe?16:38
whitok... this makes more sense now16:38
malthetesdal: I'm not sure what the best strategy is; definitely I strongly believe collective.indexing has been implemented at the wrong layer.16:39
hazmatwhit, yup, thats it16:39
tesdalmalthe: wrong layer?16:39
maltheyes, as in Zope 2 and even with Plone dependencies.16:40
tesdalmalthe: right16:40
maltheso this is an acute problem16:40
tesdalmalthe: is it clean enough for you to split it up and refactor?16:40
maltheI'll need to look at it some more; I also think there's a lot of duplicate effort between collective.indexing and ore.xapian16:41
tesdalmalthe: or just reuse parts of it if you want to16:41
hazmatmalthe, there is also enfold.xapian16:41
maltheyeah i'm a bit allergic to enfold.*16:42
maltheseems proprietary16:42
* tesdal doesn't recall how much overlap there is between ore.xapian and collective.indexing16:44
hazmatfair enough.. its probably bitrotted at this point as well.. but its available on a public repo..
*** sm is now known as sm-run16:47
tesdalmalthe: there isn't that much duplicate effort, is there?16:47
maltheno I think you're right16:47
tesdalthere are similar semantics with add, update and delete16:48
tesdalboth have queues, but one is within transaction and one is async16:48
tesdalthere are subscribers of course because that's how you add stuff to the queue16:48
maltheright the patterns are necessarily similar16:48
tesdalI like the resolver as adapter16:48
tesdalto enable resolving of other stuff as well16:48
tesdalif I remember correctly we're making assumptions about the content that goes into the queue in collective.indexing16:49
tesdalthat it has a physicalpath or something16:49
tesdalmalthe: if you have the time to extract the zope level stuff from the existing indexing solutions I'm happy to either ditch collective.indexing and put the remains into plone.indexing or just refactor collective.indexing and making it more lightweight16:50
maltheI should have; we're for a couple of days more.16:52
malthehazmat: what's the reason for ore.xapian over enfold.xapian?16:53
tesdalwould be really neat if this all just worked with grok, vudo, plone etc16:53
maltheseem to have been developed at the same time16:53
maltheditto xappy vs. pyxapian16:54
*** dobee has quit IRC16:55
tesdalI think xappy and flaxcode is the new stuff16:55
*** dobee has joined #zope3-dev16:55
malthe" We recommend use of Xappy instead of PyXapian for all new projects"16:55
hazmatmalthe, xappy is actively developed from the prototype done in pyxapian.. i just decided to do a new implementation against it16:55
hazmatthat would fit the use cases i had for indexing external content.. rdb.. svn.. etc16:56
maltheperhaps ``z3c.indexing.queue`` could be the framework part of the project16:57
malthethen ``z3c.indexing.xapian`` etc.16:57
maltheto keep these efforts together16:57
maltheany other preferences, ideas?16:58
tesdalwhat would the queue do16:58
tesdaland would it be a zope transaction queue?16:59
tesdalor async16:59
* tesdal would like to see both16:59
tesdalmaybe the async as a transaction queue consumer16:59
tesdalif queue is the transaction queue16:59
maltheyes that was the thought17:00
tesdalyou'd have a que with add, update and delete semantics17:00
tesdalinterface for queue reducer17:00
tesdaland processing17:00
maltheand optionally, an async way to process that queue17:00
tesdaland transaction manager or similar there17:00
tesdalthe async queue processor would be a consumer just like xapian or solr could be consumers17:00
tesdalif only async queue processor was registered as transaction queue consumer you'd have queuecatalog17:01
tesdalbut it makes sense to be able to config it17:01
malthebut the async queue consumer should be able to talk to the other consumers as well17:01
tesdalif we can find a way that xapian and solr etc wouldn't care whether they consume from async queue or transaction queue17:01
tesdalasync queue is just an intermediate and optional queue17:02
malthefailing to configure correctly would open up for a nice infinite consumption17:02
tesdalextension of the transaction request queue17:02
malthewhich is like the real world17:02
tesdalif the async queue processes the async queue? :)17:02
maltheat any rate, i'll get to work on this; name?17:03
tesdalwhat does z3c mean?17:03
tesdalzope 3 component?17:04
maltheno zope 3 community17:04
tesdalI think z3c.indexing.queue sounds good then17:04
maltheit's a bit verbose17:04
malthebut at least is somewhat representative17:05
tesdalverbose isn't a problem17:05
tesdalwe'll probably have z3c.indexing.asyncqueue, solr, xapian and possibly fast, autonomy and gsa17:05
*** b52laptop has quit IRC17:05
malthetesdal: exactly17:06
*** b52laptop has joined #zope3-dev17:06
*** witsch has joined #zope3-dev17:07
tesdalwitsch: malte suggested the name z3c.indexing.queue for the basic zope stuff17:08
tesdalthe queue and transaction manager basically17:08
* tesdal thinks it makes sense17:08
witschtesdal: z3c.indexing is taken already, or too generic?17:08
tesdalwitsch: is it?17:09
maltheit's free17:09
witschtesdal: no, i'm asking17:09
tesdalwitsch: we'd have z3c.indexing.solr and z3c.indexing.xapian17:09
malthethis will be quite generic17:09
witschmalthe: hey :)17:09
malthehey andi17:09
witschand z3c.indexing.zcatalog? :)17:10
witschbut yeah, if we're gonna have other stuff under z3c.indexing., then yes17:10
tesdalI think we'll have other stuff there17:11
witschbut imho we should first have a release of collective.indexing and let things settle for a little while17:11
witschrefactoring right now would cause even more confusion17:11
malthewitsch: well it's counter to zope 3 development efforts alas17:11
tesdaldepends on whether they need it right now17:11
tesdalit's not a problem to change collective.indexing17:11
tesdalto import interfaces and queue manager from z3c.indexing for example17:11
malthenaturally, I'll look to all existing efforts17:12
tesdalas long as this is all adapters etc, it's not data structure17:12
witschnope, it's not17:12
witschmalthe: what are the zope3 efforts?17:12
witschmalthe: or did you mean the collective namespace is?17:12
malthewitsch: ore.xapian17:12
maltheand a zodb-counterpart we've developed here at the sprint in sorrento17:13
witschwhat's the ore namespace?17:13
witschi guess the transaction manager stuff would just be zodb-related really, right?17:14
witschso this and the queue could be used with zope2 and/or zope317:15
witschis zope3 using anything from the collective namespace anyway?17:15
malthenot to my knowledge17:15
malthetransactions make sense outside of zodb17:16
tesdalas z3c is zope 3 collective they're basically the same but z3c is more future buzz17:16
witschcause z3c. would just make it the other way round, i.e. sort of imply zope317:16
maltheyes, but let's not be afraid of that :-)17:16
* tesdal is ok with implying zope 317:16
malthepun intended17:16
witschmalthe: well sure, a generic interface and an adapter for zodb would make sense of course17:16
malthewitsch: right and it'll be more transparent I think17:17
tesdalbecause plone is using both zope2 and zope317:17
maltheand collective.indexing monkeypatches things17:17
tesdaland grok and zope3 geared projects would be more reluctant to adopting anything in collective17:17
tesdalthe monkeypatch part would not be part of anything zope17:17
tesdalthat is purely plone/archetypes layer17:17
*** johbo has quit IRC17:18
tesdalcollective.indexing or plone.indexing or something17:18
witschok, but what's the timeframe you're proposing?  asap?17:18
tesdalcollective.indexing can't just be renamed17:18
malthethe ZODB uses transactions, not the other way around17:18
tesdalah, lib/python/transaction17:19
witschand where should this live then?  zope repos?17:19
*** sunew has joined #zope3-dev17:19
* tesdal thought it was in zodb17:19
malthetesdal: no because it's also valid for, say, sqlalchemy17:19
witschtesdal: kind of, transaction comes with zodb17:19
* malthe tries to get an overview17:20
witschtesdal: how about making an alpha nowish and add these plans to the readme, blog entry, todo list etc?17:21
witschi'd kinda like to get it out asap, not refactor everything again beforehand17:21
malthewitsch: the good thing is that there's no persistency involved17:21
witschi mean, i'm +1 for refactoring, but first we should let people try it and all17:21
maltheso migration will be smooth sailing17:21
malthei.e. they can use the alpha, then just move over to whatever comes next17:22
witschmalthe: there are some local utilities17:22
tesdalwitsch: sure17:22
tesdalwitsch: collective.indexing is kind of a testbed17:23
tesdalmaking it available right now17:23
*** witsch_ has joined #zope3-dev17:24
witsch_malthe: shouldn't be a problem, though17:24
witsch_sorry, my connection sucks today17:24
witsch_malthe: the utilities just need to be persistent to be registered, they don't store any data17:25
tesdaland because this isn't about data structure, it's not a problem to move interfaces and classes around for a beta of collective.indexing17:25
witsch_so migration shouldn't be too difficult17:25
witsch_tesdal: right17:25
*** witsch has quit IRC17:25
*** witsch_ is now known as witsch17:25
witschbut again, where do you think this should live then?
witschwell, i guess, tons of z3c stuff there already17:26
* tesdal has to get access sometime17:27
witschtesdal: :)17:27
malthetesdal: send a fax17:27
* tesdal is going home17:27
witschtesdal: you're in tbg?17:27
tesdalwitsch: no17:27
*** tesdal has quit IRC17:27
witschmalthe: well, sounds like a plan to me...  let us add it to the readme and blog entry etc and make a first release in the collective namespace (since it's ready & i was gonna do that today anyway) and then start refactoring things soonish17:29
malthewitsch: sure; I'll work parallel in a local repository17:29
witschmalthe: you're doing the xapian stuff on zope3 only, right?17:29
malthesince it's a sprint17:29
malthewitsch: yes17:29
malthebut I'd like to get to Plone as well17:29
malthebut I can hang on a bit with that17:29
maltheit's more important to get it correctly implemented on zope 317:30
witschbut are you actually replicating the same functionality at the sprint?17:30
maltheyes I'm going to copy over your stuff17:30
maltheand mix it with ore.xapian17:30
witschah, okay17:30
witschsounds good17:30
maltheand take all credit17:30
malthejust kidding17:30
malthewe'll all be famous17:31
witschwe should be careful in how we announce and move things etc, though17:31
*** nathany has joined #zope3-dev17:31
witschto avoid further confusion17:31
*** menesis has quit IRC17:31
witschperhaps it makes sense if we start the packages on, for example...  so that people see it's the same effort and not another person doing the same stuff.  know what i mean?17:32
witschmalthe: ah well, i guess you haven't been following the discussion on the enterprise list, have you?17:32
maltheit's not problem; I'll just add an AUTHORS.txt17:33
malthedo you have a problem with relicensing to ZPL btw?17:34
witschmalthe: right, good idea...  but what i meant is let's coordinate so helge or i actually create the package dirs and then you can take over17:34
witschi don't, but i don't know too much about licenses... better ask helge17:35
malthebasically ZPL is required for svn.zope.org17:35
witschyeah, i know17:36
witschphone, hang on17:36
*** michwill has joined #zope3-dev17:41
witschmalthe: ok, so how about if i try to make that release tonight (incl mention of the plan we just discussed) and then create the z3c packages right afterwards?  that way you could commit your code at the end of sorrento17:42
*** MJ has quit IRC17:43
malthewitsch: sure---in the interim I'll commit the code to the vudo git repository then17:43
malthewe use it as a playground for such things17:43
*** chtp is now known as ctp17:43
michwillHi all.. Does anybody know how to delete a persistent object from memory? Once I have loaded one, it can't be removed (may be because it can't be introspected) and so it occurs something like memory leakage17:44
witschmalthe: right, i'll pull it tomorrow and have a look then17:44
malthebtw: there's no code yet17:45
witschmalthe: that's what i thought :)17:45
witschsounded like you're evaluating atm17:45
malthewitsch: definitely17:45
witschmalthe: but have you started on the xapian integration at all?17:46
malthewitsch: yes, that part's done17:46
witschah, cool17:46
malthebut it integrates with ore.xapian; so there's some working sorting out all the connections17:46
witschbut without the queuing part then, right?17:46
malthebut the API helge has laid down is good17:46
malthei.e. the idea of a chain of consumers17:46
malthethe async consumer being a middle man for asynchronous processing17:47
maltheyes, without the queuing part17:47
witschmalthe: the one from collective.indexing you mean?17:47
witschthe API17:47
malthealso, but also the ideas in this chat17:47
*** timte has quit IRC17:47
witschah okay, i guess i've missed those then17:47
maltheI'll use the collective.indexing stuff as the transaction and queue-foundation17:47
malthe(read: carbon copy)17:48
maltheit's good stuff17:48
mgedminmichwill: ?17:49
*** sm-run is now known as sm17:49
witschthe whole async queueing idea is from enfold.indexing/solr btw, we just extended it with the dispatching part17:49
mgedminmichwill: clear the zodb cache, perhaps17:49
witschmalthe: anyway, i need to get some other stuff done, so i can go back to work on this soon enough :)17:50
witschso ttyl and have fun at the sprint :)17:50
*** witsch has left #zope3-dev17:50
michwillmgedmin, zodb is in remote server..17:51
mgedminthe client keeps a cache17:51
mgedminactually, every thread keeps a cache17:51
mgedmina certain max number of objects17:51
TheuniIt looks like REPORT_NDIFF conflicts with testrunners -1 option for doctests17:52
michwillmgedmin, hm.. and.. where is the size of such a cache and how to clean it? Memory "leaks" in sycle like for i i n self.context.values(): del(i) for a persistent object17:53
mgedminTheuni: doctest's global option flags mechanism is BROKEN horribly17:53
mgedminyou specify any of the REPORT_foo, and ALL global flags WILL be ignored17:54
mgedminwhich breaks testrunner's -117:54
*** alga_ has joined #zope3-dev17:54
mgedminmichwill: but it won't leak any more memory if you repeat that loop more than once17:54
* mgedmin just passes REPORT_ONLY_FIRST_FAILURE wherever he passes REPORT_NDIFF etc.17:55
michwillmgedmin, of course, it won't, I see. So it isn't actually a memory leak17:56
Theunii wish zope.testing's doctest factory would have a better default17:56
mgedminmichwill: nope17:56
Theunifirst failure, ellipsis, ndiff etc. should IMHO be enabled by default17:56
mgedminellipsis is tricky17:56
mgedminit sometimes bites17:56
mgedminbut first failure, yes17:56
mgedminand ndiff for any bit of output longer than, say, 2 lines17:56
Theuninormalize whitespace is pretty useful most of the time too17:57
mgedminalso, one thing I dislike, when you add debugging prints to your code and then get an exception, doctest only shows you the traceback but not what your debugging prints printed to stdout17:57
*** sorindregan has quit IRC17:59
Theunithe test runner probably shouldn't set global options but inspect the tests it has for whether they are doctests and modify their flags accordingly18:01
Theuni(if that is possible)18:01
*** chtp has joined #zope3-dev18:01
*** chtp has quit IRC18:02
*** alga__ has joined #zope3-dev18:05
*** michwill has quit IRC18:05
*** alga has quit IRC18:06
*** goschtl has quit IRC18:06
*** goschtl has joined #zope3-dev18:09
*** goschtl has quit IRC18:10
*** ctp has quit IRC18:13
*** pelle_ has quit IRC18:21
*** pcardune has joined #zope3-dev18:22
*** grahal has joined #zope3-dev18:22
*** grahal has left #zope3-dev18:22
*** jodok has quit IRC18:24
*** alga_ has quit IRC18:25
*** b52laptop has quit IRC18:32
mgedminTheuni: that's an idea!18:53
pcarduneTheuni: gocept.registration is pretty darn cool18:54
pcardune(as long as we're saying things to Theuni, I thought I would throw that in18:54
*** agroszer_ has joined #zope3-dev18:55
Theunipcardune: heh :)18:55
Theunisrichter worked on it too18:55
Theuniit's a pretty good example for framework by extraction, i think18:56
pcarduneTheuni: are you going to make a pypi release by chance?18:56
Theunioh, sure. i haven't used it in a project yet myself, so feel free to tell me if it's actually working and i'll just cut a release18:59
Theuniwhy don't you just make one and give me ownership as well?19:00
*** afd_ has joined #zope3-dev19:00
*** toko has quit IRC19:01
*** projekt01 has quit IRC19:03
*** flox has quit IRC19:04
pcarduneTheuni: ok, that sounds good (and yes, it is working :)19:04
*** markusleist has quit IRC19:07
*** agroszer has quit IRC19:07
*** agroszer_ is now known as agroszer19:08
*** harobed has quit IRC19:08
*** toutpt has quit IRC19:33
*** jukart has quit IRC19:37
*** sunew has quit IRC19:38
*** romanofski has quit IRC19:40
*** sm has quit IRC19:42
rockysrichter: ping19:51
*** dobee has quit IRC19:52
rockyok, let me mention this general ...19:52
*** quodt has quit IRC19:53
rockyis anyone aware of the issue where marking a persistent object with a marker interface and then removing that interface from sys.path so it can't be imported anymore and then trying to load the object... you get a nasty unpickling error19:53
*** zenwryly has joined #zope3-dev19:53
faassenrocky: this sounds like the normal issue of persisting references to things that don't exist anymore.19:55
rockyfaassen: yeah, i think it can be broken down that way, but the thing is it happens *alot* for me in plone setting where people install apps and then uninstall apps19:55
rockyand very specifically to interfaces19:55
rockyso could it be simply that the use case is quite rarer in a zope3 setting where people don't often uninstall py libs?19:56
faassenrocky: apparently references to the interfaces are left in the ZODB then.. shouldn't the uninstaller remove those?19:56
faassenrocky: well, it's just the general case. if you uninstall a library that defines something that got persisted, your ZODB won't be able to load that object.19:56
*** stub has joined #zope3-dev19:57
rockywell yeah, that's sort of my fix... but it's quite messy in a large plone site... where i have 2 options, query the catalog for my objects (speedily) and risk missing some objects ... or crawling the zodb hierarchy for the plone site19:57
faassenrocky: that is, if you remove a class of which instances are persisted, or you remove the marker interface definition while you make some instances providei t.19:57
*** pcardune has quit IRC19:57
*** jodok has joined #zope3-dev19:58
rockyand then of course there's the use case where what if someone reinstalls the app and would like for their persistent objects to retain knowledge of the marker interfaces... in the solution you and i describe, that's not possible19:59
rockyi understand this isn't much of a use case for pure zope 3 apps... but it happens alot in the plone world19:59
faassenwell, a pure zope 3 app would have the same issue.20:00
faassenbut I imagine we don't have people uninstalling things a lot yet.20:00
rockyyeah of course it has the same issue, but that scenario doesn't present itself very often i suspect in the pure zope3 setting20:00
rockythat's what i'm trying to say ;)20:00
faassenright. any system that was like plone and allowed you to install plugins would have similar issues though.20:00
*** ignas has quit IRC20:03
*** norro_ has joined #zope3-dev20:14
*** norro__ has joined #zope3-dev20:15
*** pcardune has joined #zope3-dev20:19
*** jodok has quit IRC20:19
*** chtp_ has joined #zope3-dev20:19
*** jsadjohnson has joined #zope3-dev20:20
*** jodok has joined #zope3-dev20:20
*** jodok has quit IRC20:21
*** thruflo has quit IRC20:26
*** alex_smith_ has quit IRC20:26
*** chtp_ has quit IRC20:26
*** alex_smith has joined #zope3-dev20:27
*** mkerrin has quit IRC20:28
*** dobee has joined #zope3-dev20:31
*** norro_ has quit IRC20:32
*** norro__ has quit IRC20:32
*** reco has joined #zope3-dev20:33
*** malthe has quit IRC20:48
*** maurits has quit IRC20:50
*** b52laptop has joined #zope3-dev20:51
*** quodt has joined #zope3-dev20:59
*** tarek has quit IRC21:06
*** mgedmin has quit IRC21:09
*** harobed has joined #zope3-dev21:14
*** agroszer has quit IRC21:14
*** stub has quit IRC21:14
*** alga__ has quit IRC21:17
*** afd_ has quit IRC21:18
*** reco has quit IRC21:23
*** reco has joined #zope3-dev21:23
*** reco has quit IRC21:29
*** reco has joined #zope3-dev21:30
*** jpcw2002 has left #zope3-dev21:49
*** redir has joined #zope3-dev21:50
*** salfield has quit IRC21:53
*** markusleist has joined #zope3-dev21:56
*** alex_smith has quit IRC22:14
*** alex_smith has joined #zope3-dev22:14
*** malthe has joined #zope3-dev22:16
*** djohnson has joined #zope3-dev22:19
*** faassen has quit IRC22:20
*** MiUlEr_ has joined #zope3-dev22:23
*** tarek has joined #zope3-dev22:24
*** reco has quit IRC22:26
*** reco has joined #zope3-dev22:26
*** MiUlEr_ has quit IRC22:27
*** d2m has quit IRC22:31
*** djohnson has quit IRC22:40
*** sunew has joined #zope3-dev22:43
*** dobee has quit IRC22:46
*** alga has joined #zope3-dev22:57
*** tesdal has joined #zope3-dev22:59
malthetesdal: work is progressing, but I've felt the urge to refactor and restructure some of the components.23:00
tesdalmalthe: sure, is it in svn somewhere?23:00
malthetesdal: no, we're using the vudo repository as a temporary workspace23:00
maltheI expect something interesting to be there tomorrow afternoon23:00
malthethe index queue has been replaced by "transactional dispatcher"23:01
tesdalmalthe: is that a name change or something more?23:01
*** jpcw2002 has joined #zope3-dev23:02
maltheit's an effort to make the indexing flow abstraction more sane23:02
malthethe transactional indexing dispatcher is now the key entry point23:02
maltheyou can't get around it23:02
malthe(which changes nothing)23:03
malthethe motivation will be clearer in the README.txt23:03
* tesdal is looking forward to seeing it :)23:03
malthethe main reasoning is that there's no-longer a queue processor.23:04
maltheit'll all about dispatching indexing operations23:04
malthetransactions beyond the main dispatcher don't make much sense (we feel).23:05
malthealthough both xapian and solr support them23:05
malthewe simply don't have a use for a doubly transactional contract.23:05
tesdalis there a queue?23:06
tesdalwhat do you  mean by queue processor?23:06
malthethat was in the original package23:06
*** zagy has quit IRC23:06
tesdalmalthe: the zcatalog consumer?23:06
malthesolr would be a queue processor in the original package23:06
malthewe've gotten ridden of that concept23:07
tesdalas long as we can reduce duplicates and dispatch to both zcatalog and externals it's all good :)23:07
maltheyes we'll be able to do that23:07
maltheyou can code rewrite it23:08
malthethat would be great23:08
malthereview it hehe23:08
tesdaldid you rename the package from indexing.queue to something else?23:08
*** sunew has quit IRC23:08
maltheyes z3c.indexing.dispatch23:08
maltheit might seem odd, but it'll grow on you23:08
* tesdal likes it23:09
tesdalit still has a queue, but it's clearer that it's about dispatching23:09
tesdalthe queue is an implementation details23:10
maltheand --- transactional dispatching23:10
malthethat's really the focus23:10
tesdalmalthe: is it you and sylvain working on it?23:13
malthemostly me right now; sylvain will join tomorrow on actual implementation23:14
malthewe've mostly discussed things together until now23:14
tesdaldiscussing is important23:15
maltheyeah very much so; sylvain is good23:15
tesdalgetting more input on naming and how to set up components23:15
maltheyeah and just if the overall idea is sane23:15
*** quodt has quit IRC23:31
*** junkafarian has joined #zope3-dev23:34
junkafarianhey, is it possible to create 2 objects (one parent container and a contained object) from a single add form in the create method?23:37
junkafarianthe main problem im running into is the inability to add the child object before the container is placed23:38
junkafarianim not sure what the best approach is though23:39
*** faassen has joined #zope3-dev23:47
*** lucielejard has quit IRC23:52
*** jsadjohnson has quit IRC23:53
malthetesdal: git clone

Generated by 2.15.1 by Marius Gedminas - find it at!