IRC log of #zope3-dev for Thursday, 2005-04-14

FarcePesti saw "simple" and "not evolved a lot" and thought apes, early humans, etc.00:00
srichterI see00:04
FarcePestperhaps it was the idea of territorial pissing that planted that idea00:05
srichterhe he :-)00:06
*** MiUlEr has quit IRC00:11
*** bska|mobile has joined #zope3-dev00:13
*** bskahan has quit IRC00:17
*** Aiste has quit IRC00:24
*** MiUlEr has joined #zope3-dev00:35
*** hazmat has joined #zope3-dev00:44
*** bska|mobile has quit IRC00:44
*** hazmat has left #zope3-dev00:45
*** admp has joined #zope3-dev00:47
*** hazmat has joined #zope3-dev00:53
*** niemeyer has quit IRC01:11
*** bradb has quit IRC01:29
*** RaFromBRC has quit IRC01:30
*** srichter has quit IRC01:43
*** srichter has joined #zope3-dev01:43
*** ChanServ sets mode: +o srichter01:44
*** ccrownhart has joined #zope3-dev01:56
*** __gotcha__ has joined #zope3-dev02:14
*** efge has quit IRC02:15
*** efge has joined #zope3-dev02:15
*** d2m has quit IRC02:16
*** efge has quit IRC02:19
*** efge has joined #zope3-dev02:19
*** hazmat has left #zope3-dev02:23
*** hazmat has joined #zope3-dev02:27
*** bradb has joined #zope3-dev02:28
*** efge has quit IRC02:28
*** __gotcha has quit IRC02:30
*** BjornT has quit IRC02:46
*** RaFromBRC has joined #zope3-dev03:11
*** RaFromBRC has quit IRC03:14
*** RaFromBRC has joined #zope3-dev03:14
*** projekt01 has left #zope3-dev03:28
*** srichter has quit IRC03:41
*** BjornT has joined #zope3-dev03:52
*** stub has joined #zope3-dev04:18
*** srichter has joined #zope3-dev04:26
*** ChanServ sets mode: +o srichter04:26
*** hazmat has quit IRC04:54
*** RaFromBRC has quit IRC05:08
*** RaFromBRC has joined #zope3-dev05:17
*** __gotcha__ has quit IRC06:37
*** BjornT has quit IRC07:00
*** RaFromBRC has quit IRC08:12
*** xenwryly has joined #zope3-dev08:52
xenwrylyany word on zope3 with python 2.4?08:54
*** philiMAC has joined #zope3-dev09:05
*** hdima has joined #zope3-dev09:17
*** mexiKON has quit IRC09:22
philiMACxenwryly, it works09:24
xenwrylyhey man09:27
xenwrylyI think I read that for zope2, it works but there had not been a security audit09:28
xenwrylyis it the same deal for z3?09:28
philiMACnot really09:29
philiMACz3 should work just fine on py2.409:29
xenwrylygreat09:30
*** Theuni has joined #zope3-dev09:32
*** d2m has joined #zope3-dev09:51
*** xenwryly has left #zope3-dev10:01
*** lunati1 has joined #zope3-dev10:02
*** sashav has joined #zope3-dev10:03
*** projekt01 has joined #zope3-dev10:37
*** MiUlEr has quit IRC10:56
*** lunati1 has quit IRC11:07
*** RaFromBRC has joined #zope3-dev11:14
*** Theuni has quit IRC11:30
*** Theuni has joined #zope3-dev11:35
*** __gotcha has joined #zope3-dev11:38
*** lunatik has joined #zope3-dev11:54
*** tarek_ has joined #zope3-dev11:54
*** RaFromBRC has quit IRC12:25
*** projekt01 has left #zope3-dev12:51
*** Aiste has joined #zope3-dev13:04
*** ignas has joined #zope3-dev13:14
*** ignas has quit IRC13:35
*** BjornT has joined #zope3-dev13:35
*** SteveA_ has quit IRC13:38
*** regebro has joined #zope3-dev13:42
*** lunatik has left #zope3-dev13:47
*** ignas has joined #zope3-dev13:51
*** SteveA has joined #zope3-dev14:01
*** bskahan has joined #zope3-dev14:08
*** Aiste has quit IRC14:19
*** niemeyer has joined #zope3-dev14:25
*** tarek_ has quit IRC14:25
*** srichter has quit IRC14:36
*** ignas has quit IRC14:37
*** Aiste has joined #zope3-dev14:40
*** mooded has joined #zope3-dev14:50
*** srichter has joined #zope3-dev15:19
*** Aiste_ has joined #zope3-dev15:19
*** ChanServ sets mode: +o srichter15:21
*** tonico has quit IRC15:21
*** tonico has joined #zope3-dev15:26
*** faassen has joined #zope3-dev15:34
*** tarek_ has joined #zope3-dev15:37
*** Aiste has quit IRC15:38
srichterwho is Vobila here?15:49
bskahansrichter: I think that's ignas at POV15:50
srichterah ok!15:51
srichterthanks15:51
bskahanits someone at POV15:51
bskahannp15:51
*** Aiste_ has quit IRC15:51
srichterdarn, I wish he would be here15:51
bskahanhe probably will be in a few hours15:52
srichterSteveA: Are the POV guys already in the UK?15:52
srichtermmh, I wish I could remember the day when he discussed the body instream problem with Jim15:52
srichterI have to fix the bug now, so that Twisted will be happy15:53
bskahanit was about a week ago15:54
srichteractually two, I just remembered to look at the bug date :-)15:54
bskahanif its the bug I'm thinking of15:54
bskahanheh15:54
srichteryeah, there ist is; March 3115:55
bskahanI ran into http://www.zope.org/Collectors/Zope3-dev/373 with creatorannotator, it can be fixed with the same removeSecurityProxy hack but alga said that solution was unloved when he did it15:58
bskahananyone know a better way to get around it?15:58
srichtermake sure that your user has the required permission :-)16:03
bskahan*sigh*16:06
bskahanok, I think create/edit implies being able to set that metadata16:06
*** Aiste has joined #zope3-dev16:07
srichterI disagree :-)16:07
srichterwhat should not happen is that it should not fail with this operation16:07
srichterbut simply not set the annotation16:07
srichteryou might ask Jim for further advise16:07
* bskahan nods16:08
bskahanthat would mean that the DC info would be out of sync with reality though16:09
bskahanI might not want the user to be able to explicitly change "modified" (which I don't), but I always want "modified" to reflect the last change16:09
*** ignas has joined #zope3-dev16:11
srichterbskahan: write a mail to the list challenging the policy16:13
srichterbskahan: I think something useful will come out of that thread16:13
* bskahan nods16:14
srichterignas: remember you body instream discussion with Jim?16:14
srichterignas: so if I read the logs right, you decided:16:14
srichterto deprecate body and bodyFile; add bodyStream that is effectively the same as bodyFile now16:15
srichterright?16:15
ignastrue16:18
srichterok, I am fixing this bug then16:19
srichterbecause it causes problems with the Twisted integration :-)16:19
ignascan't remember what was the decision about making it throw an error if seek()'ed16:19
srichterit will just fail16:19
srichterbecause it should not be seekable16:20
ignasthe fact that it fails silently disturbs me a little, though i guess it is not a significant problem16:20
srichterno, it will not be silent16:21
ignasoh, i see16:21
srichterit will be sormy!16:21
ignaswas that a mistype or a real word ? :)16:21
srichterstormy16:22
srichterI meant16:22
*** SteveA_ has joined #zope3-dev16:31
*** SteveA has quit IRC16:41
*** projekt01 has joined #zope3-dev16:55
*** gintas has joined #zope3-dev17:04
*** hdima has quit IRC17:06
*** sashav has quit IRC17:08
srichterignas: I think I have the bug under control :-)17:22
srichterexcept for one case17:22
srichterwhen the publisher retries a request17:22
srichterthen it simply resets the instream by saying .seek(0)17:23
srichterI have no clue how to fix that! :-(17:23
srichterany ideas?17:23
ignaslooking17:24
ignaswell i heard one idea was to store the request in a file or an attribute to make it seekable ...17:25
ignasthough i guess it is not allways plausible ...17:25
srichterbut this defeats the whole purpose of reading the data as you need it17:26
ignastrue17:27
srichteron the other hand a request has to be completed before it is sent to the publisher17:27
ignaswell, there is a method supports retry17:27
ignassupportsRetry17:27
ignaswhich could return False if the requeststream is not seekable ...17:27
srichtershrug17:28
srichterthis means that as soon as we are switching to any WSGI-based HTTP server, we cannot retry anymore17:28
srichter(which will include our future default implementation, which is twsited.web2)17:28
*** BjornT has quit IRC17:30
ignaswell, the requirements are pretty steep, we want the ability to handle hyge requests by reading them from a stream only when we need the data, and we want to be capable of "resending" the whole request at the same time ...17:30
ignasthe hacky evil way would be - implementing a specialcacheing stream that would cache everything that was read from the request, so we could access the data even with the real request being non seekable stream17:31
*** philiMAC is now known as philiKON17:31
ignasthe hacky stream would support seek(0) and would return the cached data untill someone would read() beyond the limit ...17:32
ignasyet i think you should consult someone more competent in such issues ;) (because i don't think i know the framework well enough)17:33
srichterthis is not a bad idea17:33
srichterwe could wrap the stream by a fake stream that records the read data17:34
srichterand supports a method called; reset stream or so17:34
srichterresetStream() or so17:34
srichterSteveA_: what do you think?17:36
*** MiUlEr has joined #zope3-dev17:36
*** SteveA_ has quit IRC17:37
srichterhah17:37
srichterignas: well, something like this must be the solution17:37
philiKONsrichter, working on twisted?17:38
srichteryeah, and our little seek(0) the stream problem shot right out17:38
srichterphiliKON: I was able to remove most of the seek code except in retry()17:39
philiKONwhat's the problem with seek()?17:39
srichterCGI input streams do not have to be seekable17:39
srichterthey are not files17:39
ignasthe stream passed to request is not allways seekable17:39
philiKONic17:39
philiKONso, why are zope's input streams seekable?17:40
srichterwhen a request is retried it simply says self._body_instream.seek(0)17:40
srichterbecause ZServer just uses IOStrings17:41
philiKONright... so, using seek() is bad; i get it...17:41
philiKONwhat's request retrying supposed to do?17:41
philiKONretry the very same request?17:41
srichteryes, in case of Conflict errors17:41
philiKONic17:41
philiKONthere must be another way that just dooing seek(0)17:42
philiKONafter all, there's no new information to be parsed17:42
philiKONso it's all in the request already17:42
srichterright, so one suggestion was to wrap the stream17:42
srichterand when it is read keep a copy of the data17:42
srichterthe wrapper would need to be smart, since it needs to track readlines and readline calls as well17:44
srichterBut luckily readline does return its newline character, so we can just append to a stringio17:46
FarcePestyou could write to a real temporary file, too. delete it after you open it and it will go away when you close it, at least with unix semantics. if it's small, it shouldn't actually hit the disk in most cases17:50
srichterFarcePest: but this is the same problem; I either have to read all of the stream before I need it (often I do not need it) or I have to cache what has been read17:51
srichterI think the second is the lesser evilk17:51
srichterI think the second is the lesser evil17:51
FarcePestyeah, cache it into a real file instead of cstringio. doesn't cstringio still concatenate strings? or is it more efficient than that?17:52
srichterno clue17:53
srichterwill a tmp file be more efficient, if it does?17:53
FarcePestit might be, it shouldn't be written to disk unless it's relatively large, or the request takes a long time (more than a few seconds)17:54
srichterso here is what I have:17:55
FarcePestthis is due to buffering17:55
srichterclass HTTPInputStream(object):17:55
srichter    """Special stream that supports caching the read data.17:55
srichter    This is important, so that we can retry requests.17:55
srichter    """17:55
srichter    def __init__(self, stream):17:55
srichter        self.stream = stream17:55
srichter        self.cacheStream = cStringIO.StringIO()17:55
srichter    def read(self, size=None):17:55
srichter        data = self.stream.read(size)17:55
srichter        self.cacheStream.write(data)17:55
srichter        return data17:55
srichter    def readline(self):17:55
srichter        data = self.stream.readline()17:55
srichter        self.cacheStream.write(data)17:55
srichter        return data17:55
srichter17:55
srichter    def readlines(self, hint):17:55
srichter        data = self.stream.readlines(hint)17:55
srichter        self.cacheStream.write(''.join(data))17:55
srichter        return data17:55
srichteractually, I probably move that code to BaseRequest, even though it does not support retry17:56
srichterno, actually it is ok in HTTP :-)17:56
*** BjornT has joined #zope3-dev17:56
FarcePestonly difference would be to make self.cacheStream be a real file object, and use os.remove() immediately after opening17:57
srichteryeah, we can do this later, if we see problems17:57
srichterI am always very skeptical about os calls it frameworks liek Zope17:57
FarcePestwould be easy to have two implementations, or to do as a subclass, at least for testing17:58
FarcePestor have a factory function that returns a file-like object: either a stringio or a real file that goes away when closed17:59
srichteractually, I have to add another method to ensure that all of the original stream has been read17:59
srichterright17:59
FarcePestonce upon a time, i wrote a unix-only SafeIO file-like object in C (this was for python-1.5)18:01
srichterhe he18:01
FarcePestthe point of it was, i needed it to timeout safely, and I also needed to do the line ending translations and byte-stuffing for SMTP and POP18:02
FarcePestso it worked on file descriptors, did it's own buffering, and used select with a timeout18:02
srichtercool18:03
FarcePestimproved throughput a lot; it was primarily for a POP server I wrote18:03
FarcePestfun state machine logic with gotos18:04
FarcePestand switch18:04
*** Theuni has quit IRC18:08
*** bradb has quit IRC18:09
srichter:-)18:13
philiKONsrichter, why do we need to capture the verbatim stream anyway, if we actually are going to parse the very same request data upon retry... why not re-use the already-parsed request data?18:15
*** SteveA has joined #zope3-dev18:15
srichterphiliKON: no we do not parse always the request18:15
srichterthe body_instream is everything after the headers18:15
srichterso in fact, for HTML stuff we are not using it at all18:15
srichtercurrently we only use it in XML-RPC and WebDAV, which send request bodies18:16
philiKONah; so if an xmlrpc/webdav view already did some read(), there's no way for us to retry?18:16
philiKON(currently that is)18:16
srichteryep18:16
philiKONok, so it would make sense to cache it then18:17
srichteryeah, I am almost done with it too18:17
srichterone more test18:17
philiKONcool18:17
philiKONstill, better to avoid read() in the future then18:17
srichterbut you cannot18:18
srichterif you are interested in the request body, you have to read18:18
srichterrequest.body is deprecated based on Jim's suggestions on IRC from March 31, 200518:18
philiKONright18:19
srichterdang, WSGI causes a lot of indirection18:20
philiKONso, what's the replacement?18:20
srichterjust use bodyStream directly18:20
philiKONuh, where's the difference?18:20
srichterbody was a string18:20
srichterbodyFile was a stream that was renamed to bodyStream18:20
philiKONok18:20
srichterFarcePest: so andy, what are you doing with Zope 3 these days?18:24
srichter:-)18:24
FarcePestexperimenting with schoolbell a bit18:26
srichterI see, cool18:26
th1aSeeing how many people he can jam directly into the ZODB.18:26
srichterschoolbell is bringing developers to Zope 3; that is cool!18:27
th1aNothing like actual applications.18:27
srichterth1a: so he is working for you? :-)18:27
th1aNo.18:27
FarcePesthttp://bowleg.neosouth.net/cgi-bin/viewcvs.cgi/ <-- i worked on this at the pycon sprints with michael bernstein18:27
FarcePestwe're going to host it somewhere else eventually. it's a weblog (bowleg is an anagram)18:28
philiKONtalked to michael abou tit18:29
philiKONexciting!18:29
* philiKON wants to use it eventually18:29
srichterFarcePest: ahh, I see; you worked with Michael on the Blog app18:29
philiKONFarcePest, i might want to contribute some day, so i hope you'll host it on some open repo18:29
FarcePestthe general idea is that there won't be comments  in the usual sense; you can comment on someone else's blog entry by making one of your own18:30
FarcePestin theory, you can still create a tree that looks like a comment tree but made up of entries18:32
*** bradb has joined #zope3-dev18:33
FarcePestto compare it against MessageBoard, the entries are not containers, so each weblog is flat18:33
philiKONthat's cool18:33
philiKONhow do you manage associations?18:34
FarcePestwe don't, it's very early yet18:34
philiKONok18:34
srichterFarcePest: so its more like the wiki package18:35
FarcePestin the sense that it is flat but there is a tree you can deduce from links, yes (i am thinking of zwiki)18:36
srichterright, this is exactely what I am doing18:37
srichterkeeping a list of children and parents in wikis18:38
srichterand then having a custom traverser18:38
FarcePestdo you keep your child/parent references as url, uri, objectId?19:01
srichterobjectid19:06
srichterthis way it is much easier for the traverser work19:06
*** tarek_ has quit IRC19:10
*** stub has quit IRC19:12
philiKONa simple association framework using intids would also be cool; that way, comments could occurr anywhere in the site and not necessarily as blog articles19:12
srichterphiliKON: I think we should just embrace schooltool's association framework19:19
srichterit is just way cool19:19
philiKONah, right, i remember19:20
philiKONnever looked at it19:20
philiKONhwo does it work?19:20
* bskahan agrees19:20
srichterI do not know the internals, just the api19:20
bskahanphiliKON: you define the relationship as a triple, roleA, relationship, roleB19:23
srichteryeah, basically subject predicate object19:24
philiKONyes, i'm reading thru README.txt right now19:25
bskahanthe blah use in schooltool is for creating many-to-many groups, the more interesting use is creating calendar subscriptions19:25
philiKONthis should be in z319:26
bskahanphiliKON: poke th1a19:26
th1aYes?19:27
th1aYou rang?19:27
srichterMArius already send a mail to the schooltool list about this19:27
bskahanheh19:27
bskahanth1a: did you see that mail?19:27
srichterCCing MArk to ask him whether it can be dual-licenced in ZPL19:27
*** stub has joined #zope3-dev19:30
*** mooded has quit IRC19:45
srichterdoes anyone here know how we parse data sent via POST?19:51
philiKONdoesn't the request.form populator of HTTPRequest do taht?19:52
srichterI dunno; let me check19:52
*** stub has joined #zope3-dev19:54
srichterphiliKON: where is that code? I cannot find it19:54
srichterok, it is in publisher/browser.py19:55
srichterbut it uses FieldStorage?!?19:56
philiKONbrowser.BrowserRequest.processInputs()19:59
srichteryeah, so far so good19:59
srichterI think something in cgi.FieldStorage might not work correctly20:00
*** stub has quit IRC20:02
*** stub has joined #zope3-dev20:02
*** mgedmin has joined #zope3-dev20:14
*** gintas has quit IRC20:18
srichterokay, so something seems to be different in the environment, so that the values are not read20:25
srichterdefinitely a cgi module thing20:25
*** tvon has joined #zope3-dev20:30
*** stub has quit IRC20:32
*** stub has joined #zope3-dev20:32
*** stub has quit IRC20:45
*** stub has joined #zope3-dev20:56
*** faassen has quit IRC21:04
philiKONsrichter, does zeo work with z3 yet?21:11
*** MiUlEr has quit IRC21:18
* philiKON finds that it doesn't look like it is21:24
*** nisha_cgx has joined #zope3-dev21:27
*** RaFromBRC has joined #zope3-dev21:31
*** RaFromBRC has quit IRC21:36
*** RaFromBRC has joined #zope3-dev21:36
*** tvon has quit IRC21:42
*** hazmat has joined #zope3-dev21:48
*** stub has quit IRC21:48
*** tvon has joined #zope3-dev21:52
*** bskahan has quit IRC21:59
srichterphiliKON: I have never looked at ZEO22:01
srichternot even for Zope 222:01
*** bskahan has joined #zope3-dev22:02
*** ignas has quit IRC22:08
*** mgedmin has quit IRC22:30
*** tvon has quit IRC22:34
*** Aiste has quit IRC22:35
*** bskahan has quit IRC22:42
*** deo has joined #zope3-dev22:46
*** lunati1 has joined #zope3-dev22:47
*** tarek_ has joined #zope3-dev22:48
*** SteveA has quit IRC22:50
*** SteveA has joined #zope3-dev22:51
*** lunati1 has quit IRC22:59
*** hazmat has quit IRC23:00
*** SteveA has quit IRC23:00
*** hazmat has joined #zope3-dev23:05
*** SteveA has joined #zope3-dev23:07
*** _projekt01 has joined #zope3-dev23:16
*** projekt01 has quit IRC23:16
*** efge has joined #zope3-dev23:19
philiKONsrichter, got it working :)23:19
srichtercool23:20
*** __gotcha has quit IRC23:20
srichterphiliKON: did I tell you that I will receive an award for Outstanding Contributions to Undergraduate Education from Tufts?23:22
philiKONno, you didn't23:22
srichterI guess my career should be obvious to everyone now :-)23:22
philiKONcongratulations!23:22
srichterthanks!23:22
*** srichter has quit IRC23:58

Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!