| *** RaAtBRC is now known as RaFromBRC|away | 00:00 | |
| benji_york | srichter, good (is that total time, or the time of the actual test?) | 00:01 |
|---|---|---|
| *** donkey has joined #zope3-dev | 00:02 | |
| *** AJC has quit IRC | 00:03 | |
| srichter | one test with 500 lines :-) | 00:05 |
| srichter | we have many of those, once I convert the tests | 00:05 |
| srichter | benji_york: have you figured out what slows down the testbrowser so much? | 00:09 |
| srichter | maybe the constant regex creation and execution makles the difference | 00:09 |
| srichter | I wonder whether using an XML parser and caching the structure would speed things up | 00:10 |
| benji_york | srichter, so far the biggest speed-ups have come from fixing *very* crazy things in pullparser and ClientForm | 00:12 |
| srichter | ok | 00:13 |
| benji_york | the over-arching slow-down comes from the fact that we're actually processing the html from the server, vs. almost totally ignoring it in the old-style functional tests | 00:13 |
| *** donkey has quit IRC | 00:13 | |
| srichter | mhh | 00:15 |
| srichter | is the HTML processing cached? | 00:15 |
| srichter | benji_york: theoretically we could parse the doc once and be done | 00:16 |
| benji_york | srichter, hmm | 00:20 |
| benji_york | srichter, "the doc" being the html returned from the server? | 00:21 |
| srichter | yeah | 00:21 |
| srichter | and maybe a pure SAX parser will be faster? | 00:22 |
| benji_york | I think it's fairly smart about that already (ClientForm and mechanize are the only things that actually parse the HTML, testbrowser deligates to them) | 00:22 |
| benji_york | one idea would be to delay work until neccesary (i.e. if you don't use "getControl" don't do the ClientForm work) | 00:23 |
| srichter | right | 00:23 |
| srichter | I often do not need getControl | 00:24 |
| benji_york | later all | 00:30 |
| *** benji_york has quit IRC | 00:30 | |
| segoave | I have a quick question about building an app on zope 3 | 00:36 |
| segoave | I am creating a PAU for LDAP authentication for my app | 00:36 |
| segoave | but I thought I should probably create an LDAP utitlity first | 00:36 |
| segoave | I'm new to zope, so I want to make sure that I'm not way off on that one | 00:37 |
| srichter | yeah, I think you will need a LDAP adapter first, but I am not sure | 00:39 |
| srichter | just follow the README.txt in ldapauth | 00:39 |
| segoave | Well, I was building my own because of some things I did not like about the ldapauth plugin | 00:41 |
| segoave | but thanks for your help | 00:41 |
| *** segoave has left #zope3-dev | 00:43 | |
| *** RaFromBRC has joined #zope3-dev | 00:51 | |
| *** RaFromBRC has quit IRC | 00:57 | |
| *** RaFromBRC has joined #zope3-dev | 00:58 | |
| *** J1m has quit IRC | 01:28 | |
| *** loreto has joined #zope3-dev | 01:31 | |
| *** FarcePest has quit IRC | 01:45 | |
| *** RaFromBRC has quit IRC | 01:47 | |
| *** RaFromBRC has joined #zope3-dev | 01:48 | |
| *** RaFromBRC has quit IRC | 01:53 | |
| *** fdrake has quit IRC | 02:41 | |
| *** RaFromBRC has joined #zope3-dev | 02:51 | |
| *** yota has quit IRC | 03:03 | |
| *** bradb is now known as bradb-away | 03:09 | |
| *** bradb-away has quit IRC | 03:17 | |
| *** bradb-away has joined #zope3-dev | 03:19 | |
| *** __gotcha__ has joined #zope3-dev | 03:24 | |
| *** __gotcha__ is now known as __gotcha | 03:24 | |
| *** bradb-aw1y has joined #zope3-dev | 03:25 | |
| *** projekt01 has joined #zope3-dev | 03:26 | |
| projekt01 | srichter, ayt? | 03:26 |
| *** bradb-away has quit IRC | 03:30 | |
| *** bradb-away has joined #zope3-dev | 03:31 | |
| *** bradb-aw1y has quit IRC | 03:32 | |
| *** RaFromBR1 has joined #zope3-dev | 03:32 | |
| *** RaFromBRC has quit IRC | 03:33 | |
| srichter | projekt01: yeah | 03:35 |
| *** bradb-away has quit IRC | 03:37 | |
| projekt01 | Do you know what's happen to the SETUP.cfg files? | 03:37 |
| srichter | Fred changed the way zpkgtools and setup.py works | 03:40 |
| projekt01 | Do we have to add SETUP.cfg into the packages and that's it. The setup.py will find them and put the described *.zcml file into the package-includes? | 03:41 |
| srichter | he was looking for Windows testers | 03:41 |
| srichter | right | 03:41 |
| *** RaFromBRC has joined #zope3-dev | 03:42 | |
| projekt01 | Argh, I didn't follow the mailinglist last couple days, because of "going to vacation and have a lot to finish" | 03:42 |
| *** __gotcha_ has quit IRC | 03:42 | |
| *** RaFromBR1 has quit IRC | 03:43 | |
| *** stub has joined #zope3-dev | 03:45 | |
| projekt01 | Hm, what's that: | 03:49 |
| projekt01 | ConfigurationError: ('Unknown directive', u'http://namespaces.zope.org/zope', u'role') | 03:49 |
| srichter | seems as if role was used before securitypolicy ZCML directives were loaded | 03:51 |
| projekt01 | Ah, Ok I'll check it | 03:51 |
| *** bradb-aw1y has joined #zope3-dev | 03:53 | |
| projekt01 | Uhhh, this was a bigger change. The package-includes at all is gone. | 04:00 |
| projekt01 | This was our configuration base folder. I think I have to move our tiks.*.configure.zcml and *meta.zcml files to the skeleton somewhere. | 04:00 |
| srichter | he he | 04:02 |
| bob2 | hm | 04:03 |
| bob2 | is merging testbrowser-integration into zope 3.1's branch likely to break things? | 04:06 |
| *** loreto has quit IRC | 04:06 | |
| srichter | bob2: not if you use Python 2.4 | 04:09 |
| srichter | i.e. I am using test browser on the 3.1 branch | 04:09 |
| bob2 | awesome | 04:09 |
| bob2 | thanks again :) | 04:09 |
| projekt01 | And what's happen to: | 04:09 |
| projekt01 | ConfigurationError: ('Unknown directive', u'http://namespaces.zope.org/mail', u'smtpMailer') | 04:09 |
| srichter | same thing | 04:10 |
| bob2 | projekt01: didn't jim remove it because of security issues? | 04:10 |
| srichter | the smptMailer directive is called before the mail meta directives are loaded | 04:10 |
| srichter | oh yes, that's right!!! | 04:10 |
| bob2 | btw, is the zope.com (.com, not .org) repository available without using svn+ssh? | 04:12 |
| stub | sendmail was removed for security reasons, not smtpmailer | 04:15 |
| bob2 | oh | 04:17 |
| projekt01 | bob2, srichter thanks | 04:17 |
| projekt01 | srichter, btw. you preference configure are also gone | 04:18 |
| projekt01 | stub, I guess Jim removed the zope.app.mail meta.zcml at all | 04:18 |
| *** srichter is now known as hter | 04:19 | |
| hter | projekt01: ok, we need to let fred know | 04:19 |
| hter | mail meta.zcml should not be gone | 04:19 |
| *** hter is now known as srichter | 04:20 | |
| projekt01 | I'm not sure since I don't know how this meta.zcml get loaded after changes. Where are the initial loader for this files? | 04:21 |
| stub | projekt01: I still see it in zope/app/mail | 04:21 |
| projekt01 | How get they loaded? | 04:22 |
| projekt01 | Before there was a initial configure file in package-includes. And now? | 04:22 |
| stub | site.zcml loads package-includes/* , and there is a file in there that loads the mail stuff as far as I can tell | 04:23 |
| projekt01 | Which package-includes? | 04:24 |
| srichter | projekt01: you have to run a special setup command | 04:24 |
| srichter | it will install package-includes from the setup info | 04:24 |
| srichter | see Makefile | 04:24 |
| bob2 | btw, do "serious" zope3 apps use runzope at all? | 04:27 |
| srichter | I dunno | 04:28 |
| projekt01 | What's about the rule explicit is better then implicit? | 04:28 |
| srichter | projekt01: it is explicit | 04:28 |
| srichter | since the SETUP.cfg specifies it | 04:28 |
| srichter | but it allows us easier distribution | 04:28 |
| projekt01 | Uhhh, I guess I also have to add SETUP.cfg for all packages now, otherwise I get a empty package-includes. Right? | 04:29 |
| projekt01 | I have to think about this concept tomorrow. It' isn't this easy for 3rd party packages | 04:32 |
| srichter | yes | 04:33 |
| srichter | it is intended to make it easier for third party packages | 04:33 |
| srichter | let Fred know if you have issues | 04:33 |
| projekt01 | How do I assable different setups for different tiks sub-package based projects with this implicit setup-configure-run implementation? | 04:34 |
| projekt01 | I guess I ping fred tomorrow | 04:34 |
| srichter | yep | 04:34 |
| projekt01 | It was so easy before, just provide a own package-include for the project and put all configure files into this project package-include. | 04:35 |
| projekt01 | Ok, I give up, see you tomorrow. | 04:36 |
| srichter | bye | 04:37 |
| *** projekt01 has quit IRC | 04:39 | |
| bob2 | hm | 05:00 |
| bob2 | how do I merge it in? | 05:01 |
| bob2 | svn -r<rev it was branched> url | 05:01 |
| srichter | svn cp svn://..../repos/main/Zope3/branches/testbrowser-integration/src/zope/testbrowser ./src/zope | 05:02 |
| *** hazmat is now known as replicant | 05:07 | |
| *** sashav has quit IRC | 05:35 | |
| *** sashav has joined #zope3-dev | 05:36 | |
| *** kakella has joined #zope3-dev | 07:09 | |
| *** Theuni has joined #zope3-dev | 08:40 | |
| *** RaFromBRC has quit IRC | 08:44 | |
| *** sashav has quit IRC | 08:48 | |
| *** tvon has quit IRC | 08:51 | |
| *** hdima has joined #zope3-dev | 08:53 | |
| *** jhauser has joined #zope3-dev | 09:14 | |
| *** tvon has joined #zope3-dev | 09:16 | |
| *** SteveA has joined #zope3-dev | 09:21 | |
| *** zagy has joined #zope3-dev | 09:43 | |
| *** jhauser_ has joined #zope3-dev | 09:46 | |
| *** sashav has joined #zope3-dev | 09:51 | |
| *** jhauser has quit IRC | 09:54 | |
| *** sashav_ has joined #zope3-dev | 09:57 | |
| *** MJ has quit IRC | 10:03 | |
| *** sashav has quit IRC | 10:10 | |
| *** d2m has quit IRC | 10:11 | |
| *** d2m_ has joined #zope3-dev | 10:11 | |
| *** d2m_ is now known as d2m | 10:11 | |
| *** __gotcha__ has joined #zope3-dev | 10:12 | |
| *** __gotcha has quit IRC | 10:13 | |
| *** __gotcha__ is now known as __gotcha | 10:13 | |
| *** wdamn has joined #zope3-dev | 10:47 | |
| wdamn | Morning, I'd like to install zope3 for helping with italian translation. I have debian and it tells me that there isn't the libc6 package: what can I do? | 10:47 |
| *** MJ has joined #zope3-dev | 10:49 | |
| *** MrTopf has joined #zope3-dev | 10:58 | |
| jhauser_ | install libc6-dev | 11:35 |
| jhauser_ | apt-get install libc6-dev | 11:35 |
| *** philiKON has joined #zope3-dev | 11:40 | |
| wdamn | even with libc6-dev I can't install zope3 - still libc6 as a dependencie not sadisfied | 11:59 |
| *** stub has quit IRC | 12:03 | |
| *** MrTopf has quit IRC | 12:16 | |
| *** projekt01 has joined #zope3-dev | 12:33 | |
| *** stub has joined #zope3-dev | 12:38 | |
| projekt01 | Somebody here how can ping fred at ZC? | 12:44 |
| *** lunatik has joined #zope3-dev | 12:55 | |
| *** lunatik has left #zope3-dev | 12:55 | |
| *** yota has joined #zope3-dev | 13:15 | |
| srichter | projekt01: he will not be awake now | 13:23 |
| srichter | projekt01: send him an email to fred_at_zope_dot_com | 13:23 |
| projekt01 | srichter, Ok, I will do it. | 13:23 |
| philiKON | projekt01, it's 6:30 in the morning over there :) | 13:24 |
| projekt01 | Ups ;-) | 13:24 |
| projekt01 | I have to find the right command for build the package-includes folder. | 13:24 |
| projekt01 | What do you use on linux? | 13:25 |
| srichter | make | 13:25 |
| srichter | just look in th eMakefile | 13:25 |
| philiKON | projekt01, on linux it's 'make' | 13:25 |
| philiKON | on windows, try: | 13:25 |
| philiKON | python setup.py \ | 13:25 |
| philiKON | build_ext -i install_data --install-dir . | 13:25 |
| philiKON | running build_ext | 13:25 |
| philiKON | err, without the last line | 13:25 |
| projekt01 | Ok, "build_ext -i install_data" seems to install the configuration files into the py24 location. | 13:29 |
| projekt01 | Is there support for install the package-includes in the z3 root? | 13:30 |
| philiKON | look above | 13:31 |
| philiKON | --install-dir . | 13:31 |
| philiKON | python setup.py build_ext -i install_data --install-dir . | 13:31 |
| projekt01 | Ah, ok I'll try | 13:31 |
| projekt01 | philiKON, srichter, thanks my trunk is running again | 13:33 |
| projekt01 | Now I have to find out how to include additional packages ;-) | 13:34 |
| projekt01 | I guess I have to write SETUP.cfg | 13:34 |
| projekt01 | But if so, how can I support different distributions of a package for different projects? | 13:36 |
| *** Alef has joined #zope3-dev | 13:36 | |
| philiKON | huh? | 13:36 |
| philiKON | yes, write a SETUP.cfg | 13:36 |
| projekt01 | Is there a way to assamble selective packages from zope3 trunk and a 3rd party trunk to a project (distribution) | 13:37 |
| philiKON | sure | 13:37 |
| philiKON | see Zope3/releases/Zope/PACKAGE.cfg | 13:38 |
| projekt01 | What's the right starting point? | 13:38 |
| philiKON | use the resource map in Zope3/releases/Zope.cfg and, of course, don't forget a resource map for your 3rd party stuff | 13:38 |
| *** MrTopf has joined #zope3-dev | 13:40 | |
| projekt01 | philiKON, srichter, thanks you saved a lot of my time. | 13:41 |
| projekt01 | This changes HAS to be documented in the README.txt. It doesn't fit right now. | 13:42 |
| philiKON | yup | 13:43 |
| philiKON | there maybe should be a make.bat for windows users | 13:43 |
| philiKON | so that behaviour is the same there | 13:43 |
| philiKON | we have that for kupu | 13:43 |
| projekt01 | Yes, but this is also complex | 13:44 |
| projekt01 | ;-) | 13:44 |
| projekt01 | But it's nice if you understand it ;-) | 13:45 |
| philiKON | i don't :) | 13:45 |
| philiKON | http://codespeak.net/svn/kupu/trunk/kupu/make.bat | 13:45 |
| philiKON | i didn't write it | 13:45 |
| philiKON | but it doesn't look too complex | 13:45 |
| projekt01 | I don't like this for windows. It is most the time a killer factor for other developer if the build part is to complex | 13:47 |
| philiKON | well, a very very simple make.bat would have that one line | 13:48 |
| philiKON | python setup.py build_ext -i install_data --install-dir . | 13:48 |
| philiKON | :) | 13:48 |
| *** Aiste has joined #zope3-dev | 13:50 | |
| projekt01 | If it's documented it's Ok. If not ... ;-) | 13:51 |
| projekt01 | just a note for other win developers | 13:55 |
| projekt01 | This will work as a generic local build_ext.bat | 13:56 |
| projekt01 | python setup.py build_ext -i install_data --install-dir . | 13:56 |
| projekt01 | philiKON, btw. if you delete a SETUP.cfg in a package the relevant configure.zcml get removed | 14:01 |
| philiKON | yup | 14:09 |
| projekt01 | philiKON, how do I use a different SETUP.cfg like used in release/Zope-win | 14:11 |
| philiKON | i don't think i understand | 14:12 |
| *** mgedmin has joined #zope3-dev | 14:14 | |
| projekt01 | I'm looking for the starting point for assembling a project. I guess I can use a own PACKAGE.cfg for this. | 14:18 |
| *** ignas has joined #zope3-dev | 14:24 | |
| projekt01 | is there a dependecy import path extracter script for write the DEPENDENCIES.cfg? | 14:25 |
| *** MrTopf has quit IRC | 14:26 | |
| philiKON | projekt01, utilities/finddeps.py | 14:26 |
| philiKON | as for the starting point, simply copy releases/Zope and modify accordingly | 14:26 |
| philiKON | :) | 14:26 |
| projekt01 | philiKON, thanks for the finddeps.py. | 14:29 |
| projekt01 | How can force to use such a release/Zope PACKAGE.cfg definition during the build_ext process? | 14:30 |
| philiKON | use a different tree | 14:33 |
| philiKON | srichter, was there a proposal for this change of fred's? | 14:34 |
| srichter | no | 14:34 |
| srichter | but it was on the zpkgtools to do list | 14:34 |
| philiKON | seems like this complicated things for projekt01 | 14:34 |
| srichter | afaik | 14:34 |
| srichter | he needs to get zpkgtools going for Z2 | 14:35 |
| *** kakella has quit IRC | 14:35 | |
| projekt01 | btw, we didn't had a concept for packaging right now. But now we need to add one otherwise nothing works anymore ;-) | 14:36 |
| philiKON | projekt01, btw, why aren't you working in instances? | 14:38 |
| philiKON | you seem to be working from a chekcout tree directly | 14:38 |
| projekt01 | Yes, | 14:38 |
| projekt01 | Can you develop on instances and commit them the subversion? | 14:39 |
| projekt01 | I guess instances don't include the .svn part? right? | 14:39 |
| philiKON | huh? | 14:39 |
| philiKON | develop what on instances? | 14:40 |
| philiKON | you can put your svn checkouts of your packages into $INSTANCE_HOME/lib/python | 14:40 |
| projekt01 | I never used INSTANCE_HOME, is there a documentation for a recommended development setup? | 14:42 |
| philiKON | uuh | 14:42 |
| philiKON | read section 2.3 of my book | 14:42 |
| philiKON | it describes instances | 14:43 |
| projekt01 | Remember we don't have symlink and I support 3 projects right now. Does this mean I have to commit changes on one instance and update the other two workspaces? | 14:43 |
| projekt01 | (via subversion) | 14:44 |
| philiKON | no | 14:44 |
| philiKON | you could put it all in one location and include that in your pythonpath | 14:44 |
| philiKON | all i'm saying is that the Zope 3 checkout tree isn't the ideal environment for doing 3rd party development in | 14:45 |
| projekt01 | Ok, I see | 14:48 |
| *** bradb-aw1y is now known as bradb | 15:00 | |
| *** bskahan has joined #zope3-dev | 15:11 | |
| andrew_m | is there a simple way to get the user title from a user id (e.g. zope.manager -> Admin)? did i overlook something in the ZAPI? | 15:14 |
| andrew_m | (i want to display the user who has locked an object if it's locked) | 15:15 |
| philiKON | principal.title i think | 15:16 |
| andrew_m | hmm ok.. i'll try to turn the principal id into an object then.. | 15:20 |
| philiKON | yup, look up the principal obj | 15:23 |
| *** benji_york has joined #zope3-dev | 15:23 | |
| *** faassen has joined #zope3-dev | 15:23 | |
| *** Theuni has quit IRC | 15:39 | |
| benji_york | srichter, ayt? | 15:41 |
| *** bradb is now known as bradb-bbl | 15:46 | |
| *** ignas has quit IRC | 16:01 | |
| *** stub has quit IRC | 16:06 | |
| *** MrTopf has joined #zope3-dev | 16:19 | |
| *** ignas has joined #zope3-dev | 16:26 | |
| *** __gotcha has quit IRC | 16:36 | |
| *** bradb-bbl is now known as bradb | 16:42 | |
| *** sashav_ has quit IRC | 16:56 | |
| *** hdima has quit IRC | 16:58 | |
| *** faassen has left #zope3-dev | 17:02 | |
| *** fdrake has joined #zope3-dev | 17:07 | |
| philiKON | question for make gurus: how do i repeat a certain command for each file in a wildcard? | 17:08 |
| fdrake | you're on unix/linux? | 17:09 |
| philiKON | yep | 17:10 |
| philiKON | i know how to do it in bash | 17:10 |
| philiKON | but not in make | 17:10 |
| fdrake | "for FN in globspec ; do cmd $FN ; done" | 17:10 |
| fdrake | ah | 17:10 |
| philiKON | that's bash :) | 17:11 |
| fdrake | I think there's something documented in the GNU make manual; not sure if it was a gmake-specific thing or not | 17:11 |
| fdrake | don't remember the syntax offhand | 17:12 |
| projekt01 | fdrake, I hit my head on the wall because of your changes in zpkgsetup ;-) | 17:12 |
| fdrake | ouch! | 17:12 |
| fdrake | because I used it in the Zope3 checkout, or the internals? | 17:13 |
| projekt01 | We didn't had all this SETUP.cfg etc files right now | 17:13 |
| fdrake | ah, using for the Zope 3 checkout build hurt? | 17:14 |
| *** faassen has joined #zope3-dev | 17:14 | |
| projekt01 | Yup, I think there is no way to build_ext without this files. Right now I add all this informations where needed. | 17:15 |
| fdrake | in the Zope 3 checkout, that's right | 17:15 |
| projekt01 | btw, finddeps.py -z doesn't support relative path in ZCML | 17:15 |
| fdrake | are you developing other software in the same tree? | 17:15 |
| projekt01 | Yes | 17:15 |
| fdrake | so you had hacked the original setup.py, right? | 17:16 |
| fdrake | (or used an alternate, I suppose) | 17:16 |
| projekt01 | I don't think so. I guess that I only have to add the SETUP.cfg and run build_ext. Or not? | 17:17 |
| fdrake | that should be all you need now | 17:17 |
| fdrake | before, you needed to hack the setup.py | 17:18 |
| projekt01 | Can you add some comments in the README.txt? | 17:18 |
| projekt01 | There are no information in the trunk for how to build it with 3rd. party packages. | 17:19 |
| fdrake | about how to use SETUP.cfg and the like? | 17:19 |
| fdrake | no, guess not | 17:19 |
| fdrake | We haven't been keeping different packages in the same tree, so that's something we haven't tried | 17:19 |
| fdrake | we keep application code separate from Zope 3 | 17:20 |
| fdrake | but the README.txt is out of date; will fix | 17:20 |
| fdrake | i added a page in the Zope3 wiki about using SETUP.cfg files | 17:20 |
| projekt01 | I just checkout a fresh trunk and put the packages into the src folder of this checkout and build it. | 17:21 |
| projekt01 | packages/ custom packages | 17:21 |
| philiKON | projekt01, i really recommend instances here | 17:21 |
| fdrake | http://www.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/BuildAndPackagingInfo | 17:21 |
| fdrake | yes, instances are goodness incarnate | 17:21 |
| srichter | benji_york: I am here now | 17:22 |
| projekt01 | I think I have to take a look at this INSTANCE_HOME part | 17:22 |
| projekt01 | fdrake, I'll take a look at the docu. | 17:22 |
| *** J1m has joined #zope3-dev | 17:23 | |
| fdrake | cool; let me know if that's missing anything | 17:23 |
| fdrake | i'll be glad to correct/improve as needed | 17:23 |
| benji_york | srichter, hi, I don't need to say what I wanted to say before :) ... but I am looking at the optimizations you sent me we'll see what we get | 17:24 |
| srichter | benji_york: I think the response file is read and parsed far too many times in mech browser | 17:26 |
| benji_york | I wouldn't doubt it, that code could use some TLC | 17:26 |
| srichter | it should really stife to parse the response once (when requested) and use that info for all other features | 17:26 |
| srichter | TLC? | 17:26 |
| benji_york | tender, loving care - idiom | 17:27 |
| srichter | oh, ok | 17:28 |
| srichter | benji_york: I wonder whether writing a SAX XML parser that only gets links would be faster | 17:33 |
| benji_york | srichter, I got a 20% speed-up from your mechanize patch | 17:33 |
| srichter | same for the form parser | 17:33 |
| benji_york | hmm, don't know | 17:33 |
| srichter | benji_york: cool | 17:33 |
| benji_york | the down side is that up until now we haven't had to patch mechanize | 17:33 |
| benji_york | and we (Gary specifically) have some very large outstanding patches to ClientForm | 17:33 |
| srichter | that's expected, since you commonly either get the links and click them or you fill in a form | 17:33 |
| srichter | you usually do not want both | 17:34 |
| benji_york | right | 17:34 |
| benji_york | in your email you said someone thinks 1.5.2's cStringIO "doesn't work", what and whome were you referring to? | 17:35 |
| srichter | are Gary's changes going to make the thing faster? | 17:35 |
| srichter | benji_york: it is a comment in the code | 17:35 |
| benji_york | ahh | 17:35 |
| srichter | they said they did not use cStringIO because of 1.5.2 | 17:35 |
| srichter | I think we should just check for the Python version | 17:35 |
| benji_york | Gary's changes: they were to add features that testbrowser needed, but some of my optimizations have added to his patch | 17:35 |
| SteveA | does anyone still support 1.5.2? | 17:35 |
| benji_york | JJL (the mechanize guy) still wants to support some very old python versions | 17:36 |
| benji_york | I think you're right that a conditional import is the thing to do | 17:36 |
| benji_york | hmm, our ClientForm uses cStringIO, where did you change StringIO to cStringIO? | 17:38 |
| srichter | maybe pullparser | 17:38 |
| benji_york | nope, pullparser doesn't use either | 17:38 |
| srichter | I think I just grepped for StringIO | 17:38 |
| benji_york | hmm, I'll look | 17:39 |
| srichter | maybe somewhere in ClientCookie | 17:39 |
| srichter | yeah it is in ClientCookie/_ClientCookie.py | 17:40 |
| benji_york | yep, just found it | 17:40 |
| benji_york | benchmarking now | 17:40 |
| benji_york | srichter, I don't see any speed difference | 17:43 |
| srichter | yeah, I just thought the same thing when I looked at the code again :-) | 17:44 |
| benji_york | :) | 17:44 |
| srichter | (I do not do scientific benchmarking at all) | 17:44 |
| benji_york | between your and mine optimizations my test case has gone from 20 seconds to 7.6! | 17:45 |
| benji_york | not bad for 48 hours :) | 17:45 |
| srichter | mmh, are all your optimizations checked in? | 17:48 |
| srichter | benji_york: also, I have a patch that caches browser.contents after the first call | 17:48 |
| srichter | I did not get any speedup from it though | 17:48 |
| benji_york | ok | 17:48 |
| srichter | benji_york: are you using browser.getLink().click() a lot? | 17:49 |
| srichter | or do you open URLs directly | 17:49 |
| benji_york | we click as much as possible (to emulate what a user would do) | 17:49 |
| srichter | I do a lot of the first one, because I want to demonstrate how to get to pages | 17:50 |
| srichter | yeah, me too | 17:50 |
| srichter | that's probably the reason for our test time increase, sicne the plain fdoctests only called the URLs without intermediate steps | 17:50 |
| benji_york | right, but I definitely think it's worth it | 17:51 |
| srichter | me too | 17:51 |
| srichter | in fact I think that the converted ftests show several places where the UI is not very efficient | 17:52 |
| benji_york | right! If the tests are awkward, it's likely that the UI is too | 17:52 |
| srichter | J1m: btw, I still have not solved my evolution script problem | 17:53 |
| J1m | All you've done is move classes around, right? | 17:53 |
| srichter | J1m: I took the easiest path of resistence and created fake packages and imported the classes there | 17:54 |
| srichter | yes | 17:54 |
| J1m | so did that work? | 17:54 |
| srichter | yes, but: | 17:54 |
| srichter | then I wrote an evolution script to update the instances | 17:54 |
| srichter | this works | 17:54 |
| srichter | however, if those instances are referenced somewhere, like in the __parent__ attribute, then those references do not get updated | 17:55 |
| srichter | so effectively I need to leave the fake object locations around forever | 17:56 |
| J1m | Yup | 17:56 |
| J1m | This is actually some thing that can and should be fixed in zodb. | 17:57 |
| J1m | hm.... | 17:57 |
| * J1m looks at some code | 17:57 | |
| srichter | ok | 17:57 |
| srichter | (I am willing to do the work; I have looked at the serializer and understand it a bit, but not much of the other code) | 17:57 |
| J1m | OK, this is a straightforward change. | 18:00 |
| srichter | yipee | 18:00 |
| J1m | Let me get Tim | 18:00 |
| srichter | great | 18:00 |
| srichter | can you tell me the file to look at (so I learn more about ZODB)? | 18:01 |
| J1m | serialize.py | 18:02 |
| srichter | ok | 18:03 |
| *** ymmit has joined #zope3-dev | 18:04 | |
| *** regebro has joined #zope3-dev | 18:04 | |
| J1m | ymmit, srichter and I were just talking about the issue of having no good way to update references to reflect class changes. | 18:05 |
| J1m | so, if a class moves, the reference has the old name, | 18:05 |
| ymmit | Not sure what "references" means here. | 18:06 |
| J1m | Object a referes to object b. | 18:06 |
| J1m | a's puckle has a persistent reference to b. | 18:06 |
| srichter | ymmit: hi tim | 18:06 |
| ymmit | So we're talking about how a class gets referenced in pickles. | 18:06 |
| J1m | yes | 18:06 |
| ymmit | k | 18:06 |
| ymmit | This is a long-standing problem ;-) | 18:06 |
| ymmit | Have a new idea? | 18:07 |
| J1m | we can write a scipt to find all instances of a class and rewrite those instances to the database. | 18:07 |
| SteveA | there was an idea early in zodb 4 development | 18:07 |
| SteveA | about refering to classes by an id in a registry | 18:07 |
| ymmit | Shane has code on SourceForge that tries to do that too. | 18:07 |
| SteveA | but, that was not used for various reasons | 18:07 |
| ymmit | (very old code, BTW) | 18:07 |
| J1m | But it's not so easy to find all of the objects that reference an object. | 18:07 |
| J1m | Now, the thing to recognize with references is that the class info is effectively cached. | 18:08 |
| J1m | If we can't find the class with the info in the reference, we have another plave to look, which is the object's database record. | 18:08 |
| ymmit | The object containing the reference? | 18:09 |
| J1m | I suggest that if a class named in a reference can't be found, we should fall back to connection.get(oid) | 18:10 |
| J1m | no, the referenced object's database record, which we can find using the oid. | 18:10 |
| ymmit | Or you mean use the oid of the referenced object? | 18:10 |
| ymmit | Crossed in the mail, ga | 18:11 |
| J1m | I would say that whenever we can't create the object in load_persistent, we should fall back to connection.get (aka self._conn.get). | 18:11 |
| ymmit | Not sure I follow. If the class has indeed moved, how can connection.get() load it? | 18:12 |
| J1m | someone may have updated the object since the class moved. | 18:12 |
| J1m | (The referenced object) | 18:12 |
| ymmit | Ah, so it's a may-not-work fallback. | 18:12 |
| J1m | Yup | 18:13 |
| J1m | Stephan had written an update script that: | 18:13 |
| J1m | - created aliases for a moved class | 18:13 |
| J1m | - rewrote all of the objects with that class. | 18:13 |
| J1m | He wanted to tehn get rid of the aliases, but couldn't, because the old class names were still used in the references. | 18:14 |
| J1m | He wanted to then get rid of the aliases, but couldn't, because the old class names were still used in the references. | 18:14 |
| J1m | The fallback would have worked for him. | 18:14 |
| ymmit | So would a script that went looking for references too, right? | 18:15 |
| J1m | In any case, this is a nothing-to-lose may-not-work fallback. | 18:15 |
| J1m | Yup | 18:15 |
| ymmit | If, "in the meantime", someone created "a new class" with the old path, things would get really confused. | 18:16 |
| J1m | Yes | 18:16 |
| ymmit | That's why I'd feel better about a smarter script that did the whole job. | 18:16 |
| srichter | but such a script is hard for a general object tree | 18:17 |
| srichter | are there low-level ZODB facilities to help here? | 18:17 |
| J1m | Fair enough. We would need to give people tools making such a script writable without deep zodb knowledge.. | 18:17 |
| ymmit | Last time I looked for Shane's code for this, SourceForge was dead ... I'd at least look at that first. | 18:18 |
| ymmit | Let me try to find a reference ... | 18:18 |
| srichter | J1m: is there a way in ZODB to load all objects? I could mark them all as _p_changed and then pack the DB afterwards | 18:19 |
| J1m | We have an iteration protocol, but I'm not sure if it's finished. | 18:20 |
| J1m | ymmit, do you remember? | 18:20 |
| ymmit | I believe only FileStorage supports the iteration protocol at this point. | 18:21 |
| srichter | that's good for me :-) | 18:21 |
| ymmit | Heh -- last time Shane talked about his code, SourceForge was dead too ;-) : | 18:21 |
| ymmit | http://mail.zope.org/pipermail/zodb-dev/2003-January/004237.html | 18:21 |
| ymmit | He patched ZODB there too, I think. | 18:22 |
| J1m | I wonder if he is doing application-level traversal, or database-level traversal. | 18:23 |
| ymmit | A SourceForge link that works today: | 18:23 |
| ymmit | http://cvs.sourceforge.net/viewcvs.py/zodbex/zodbex/ChangeModules/change_modules.py?rev=1.3&view=auto | 18:23 |
| ymmit | It's written as a "Zope external method", but appearts to be databae-level traversal ... not entirely sure. | 18:24 |
| J1m | so he is doing low-level traversal. | 18:25 |
| J1m | In the short term, as a definate hack, stephan could get hold of the file-storage index and use that to iterate over all objects in the database. | 18:26 |
| ymmit | According to Shane, "It works with any storage and any ZODB objects and does not require any database shutdown. | 18:26 |
| srichter | I don't see where it patches ZODB | 18:27 |
| ymmit | Well, using the file-storage index to find all the objects is exactly what the new iteration protocol does. | 18:27 |
| J1m | sure | 18:28 |
| J1m | good point | 18:28 |
| J1m | brb | 18:30 |
| ymmit | srichter: I never looked at this code before -- SourceForge was always down when I tried ;-) | 18:30 |
| srichter | ahh, I see :-) | 18:31 |
| ymmit | The ZODB patching is subtle, and I'm not sure it still works: the call to jar.onCommitAction() is the trick. | 18:31 |
| srichter | Okay, I thought is was a legal event subscription | 18:32 |
| ymmit | Connection definitely does not have onCommitAction() anymore in ZODB 3.3+. | 18:32 |
| ymmit | It has other gimmicks for hooking commits, though. | 18:32 |
| ymmit | Figuring out details would, alas, require actual thought ;-) | 18:33 |
| srichter | I don't want to divert your attention, how could I use the iterator | 18:34 |
| srichter | I basically iterate over txns, then Records | 18:34 |
| srichter | the record has an oid | 18:35 |
| srichter | can I load the object from that? | 18:35 |
| srichter | basically, how can the record help me update the objects? | 18:35 |
| ymmit | Are you talking about the new FileStorage iteration method? | 18:36 |
| srichter | yeah | 18:36 |
| ymmit | A basic example is in ZODB's NEWS.txt. | 18:37 |
| *** MJ has quit IRC | 18:37 | |
| ymmit | A more fleshed-out example is in the test, check_record_iternext(), in testFileStorage.py. | 18:37 |
| ymmit | Connection.get(oid) will load the object corresponding to oid. | 18:38 |
| ymmit | record_iternext() happens to return the object's data pickle too. | 18:39 |
| srichter | and when I change that object and then commit the transaction it will be written, right? | 18:39 |
| ymmit | Yes, right. | 18:39 |
| ymmit | Note a couple cautions: record_iternext() only finds the current reversion of non-version data. | 18:40 |
| ymmit | s/reversion/revision/ | 18:40 |
| ymmit | Older revisions are not found, and neither is data "in a version". | 18:40 |
| srichter | that's ok, I would probably pack anyways | 18:40 |
| ymmit | Packing wouldn't get rid of current data "in a version". | 18:42 |
| philiKON | ARRRRGH. why does zope.org have to be fubared every time i want to work with it | 18:42 |
| philiKON | fdrake, ayt? | 18:42 |
| fdrake | yes | 18:42 |
| srichter | ymmit: what is a version here? | 18:42 |
| philiKON | fdrake, hello :) | 18:42 |
| ymmit | philiKON: zope.org special-cases your account ;-) | 18:42 |
| fdrake | hey! | 18:42 |
| philiKON | fdrake, did you get my (private) follow up on issue 250? | 18:42 |
| philiKON | i had to reply via email in private because the collector (still won't) let me create a follow up | 18:43 |
| fdrake | philiKON: don't see it | 18:43 |
| fdrake | what address did you use? | 18:43 |
| philiKON | fdrake@gmail.com | 18:43 |
| philiKON | should i resent to fdrake@zope.com? | 18:44 |
| ymmit | srichter: a "version" is a string name. It's complicated. Every revision of every object has a version associated with it. Normally that's an empty string, in which case it's customary to say the data is "not in a version". | 18:44 |
| fdrake | what's the subject line? | 18:44 |
| ymmit | Versions were a gimmick added to support some notion of long-running transactions. | 18:44 |
| philiKON | fdrake, Re: [Z3d] 250/ 9 Assign "ListSequenceWidget validates items to be removed" | 18:44 |
| fdrake | i'll search; may have archived too quickly | 18:44 |
| ymmit | Dieter uses them, but I'm not sure anyone else does ;-) | 18:44 |
| srichter | ahh, ok | 18:44 |
| srichter | I have not worry about it then | 18:45 |
| ymmit | The idea seemed to be that you could make changes "in a version" named, say, "experimental", and nobody else would see them. | 18:45 |
| srichter | ah, I see | 18:45 |
| ymmit | Later you could commit or abort "the 'experimental' version". | 18:45 |
| fdrake | philiKON: it automatically becomes pending when no one is assigned | 18:45 |
| philiKON | fdrake, ah, ok. never mind about that, then | 18:46 |
| fdrake | ok, then I won't :-) | 18:46 |
| philiKON | fdrake, so, i included a patch | 18:46 |
| srichter | btw, will record_iternext also work, if a connection is opened? | 18:46 |
| philiKON | havent' come around writing tests yet | 18:46 |
| srichter | ymmit: ybtw, will record_iternext also work, if a connection is opened? | 18:46 |
| ymmit | srichter: Define "work" ;-) | 18:48 |
| fdrake | ah, right, that needs to be attached to the collector issue; i'll see if it'll let me | 18:48 |
| J1m | back | 18:48 |
| philiKON | fdrake, thanks :) | 18:48 |
| J1m | ymmit, I'd like to get rid of IStorage until we're ready to write it. | 18:49 |
| srichter | ymmit: I have a connection object | 18:49 |
| ymmit | srichter: It won't blow up, but, in effect, if a Connection is open and the FileStorage is mutation, then record_iternext() is traversing a mutating index. | 18:49 |
| ymmit | s/mutation/mutating/ | 18:49 |
| srichter | ah, I see | 18:49 |
| ymmit | I think it should work well. | 18:49 |
| J1m | ditto for other interface skeletins, | 18:49 |
| srichter | but if I do not commit the transaction, then I will be fine | 18:50 |
| ymmit | But, e.g., if you've iterated over oid 1000000, and a transaction then creates oid 9999, the iteration will miss that. | 18:50 |
| srichter | that's ok, because that 9999 iteration would have already updated references | 18:51 |
| ymmit | Since oids are normally passed out in increasing order, that shouldn't be a major worry. | 18:51 |
| ymmit | Yes, that too ;-) | 18:51 |
| srichter | and noone is committing transactions during that time either | 18:51 |
| fdrake | philiKON: done | 18:51 |
| srichter | ymmit: btw, I just did connection.get(oid=0) and I got a dictionary back | 18:52 |
| philiKON | fdrake, thanks | 18:52 |
| ymmit | Jim: why bother deleting IStorage? Sounds pretty pointless to me, and, e.g., IStorageUndoable derives from it. | 18:52 |
| srichter | ymmit: never mind | 18:52 |
| srichter | it is all good | 18:52 |
| ymmit | srichter: I'd expect you get back a PersistentMapping from oid 0 (the root object). | 18:53 |
| srichter | yes | 18:53 |
| srichter | I just did not realize that right away | 18:53 |
| fdrake | ymmit, is there any plan of Zope 2.9 using ZODB 3.5? | 18:57 |
| J1m | Zope 2.9 will use either ZODB 3.5, or, I hope, ZODB 3.6. | 18:58 |
| fdrake | it would be nice if we could move to ZODB 3.5 sooner rather than later | 18:59 |
| fdrake | ZODB 3.5 contains build changes to allow it to be happier w/ zpkg for in-place builds | 18:59 |
| ymmit | Zope trunk currently uses 3.4.1 | 18:59 |
| fdrake | (and resolve issues with public headers) | 19:00 |
| ymmit | I'd be happy to switch to using 3.5 | 19:00 |
| fdrake | ymmit: currently, yes | 19:00 |
| ymmit | In theory, this is really the Zope release manager's call, though. | 19:00 |
| *** Soulraven has joined #zope3-dev | 19:00 | |
| *** regebro has quit IRC | 19:00 | |
| ymmit | OTOH, nobody in Zope2 land pays any attention to ZODB, so I do that for them ;-) | 19:00 |
| fdrake | that would be good; I don't want to backport the build changes to ZODB 3.4 since that would affect Zope 2.8 | 19:01 |
| Soulraven | is this Zope 3 talk only ? :) | 19:01 |
| ymmit | Yes, 2.8 is stuck with ZODB 3.4 forever. | 19:01 |
| fdrake | Soulraven, sorry, ymmit was here already. :-) | 19:01 |
| Soulraven | o_O | 19:01 |
| fdrake | ?? | 19:02 |
| ymmit | fdrake: I'll ask on zope-dev whether anyone objects to moving Zope trunk to ZODB 3.5. After nobody responds, I'll just do it. | 19:02 |
| Soulraven | sorry for my boldness but i need some help on zope 2.8 | 19:02 |
| fdrake | ymmit: That's the pattern that seems to be developing :-) | 19:02 |
| tvon | Soulraven: try #zope | 19:03 |
| Soulraven | :) thanks. | 19:03 |
| *** Soulraven has left #zope3-dev | 19:04 | |
| *** MrTopf has quit IRC | 19:10 | |
| *** wdamn has quit IRC | 19:12 | |
| srichter | ymmit: the following does not seem to be efficient enough: | 19:12 |
| srichter | record = ('\x00\x00\x00\x00\x00\x00\x00\x00',) | 19:12 |
| srichter | while record[-1] is not None: | 19:12 |
| srichter | print 'OID:', `record[0]` | 19:12 |
| srichter | record = context.connection._storage.record_iternext(record[-1]) | 19:12 |
| srichter | obj = context.connection.get(record[0]) | 19:12 |
| srichter | obj._p_changed = True | 19:12 |
| srichter | it did not get rid of all old references | 19:12 |
| *** bradb is now known as bradb-lunch | 19:17 | |
| ymmit | srichter: ? I don't understand why that code snippet would get rid of _any_ old references? | 19:26 |
| srichter | :-) | 19:26 |
| srichter | ok, what did I do wrong? | 19:26 |
| srichter | Note that the transaction is committed later | 19:27 |
| ymmit | That is, all it does is load an object, and set _p_changed to True. Where does it _change_ anything? | 19:27 |
| srichter | I basically want to force a write | 19:27 |
| srichter | the change is that the class reference changed | 19:27 |
| ymmit | Where in that code did the class reference get changed? | 19:28 |
| srichter | basically I used to have: x.ObjectX | 19:28 |
| srichter | that class reference is stored in the ZODB | 19:28 |
| srichter | now I moved the object to y.ObjectX | 19:29 |
| srichter | but in x I have: from y import ObejctX | 19:29 |
| ymmit | Sorry, I'm pretty lost here. | 19:29 |
| ymmit | Are you saying, e.g., that the code you showed above is not the only code that's relevant? | 19:29 |
| srichter | yeah | 19:29 |
| srichter | it is part of a generation script in Zope 3 | 19:30 |
| ymmit | So you've run some other code before running the code above? | 19:30 |
| srichter | yes | 19:30 |
| srichter | and afterwards of course as well | 19:30 |
| srichter | so the connection is opened before this code snippet | 19:30 |
| ymmit | I'm not sure IRC works for this ;-) | 19:30 |
| srichter | and the transaction is committed afterwards | 19:31 |
| ymmit | Nothing in the code I've seen actually changes anything. Do you agree with that? | 19:31 |
| srichter | yes | 19:31 |
| srichter | and it should not | 19:31 |
| srichter | I only want to force to write the object to the database again | 19:32 |
| srichter | so that the object's class reference is updated | 19:32 |
| ymmit | The code snippet above should force the object to get written again. | 19:32 |
| srichter | ok, then that's fine | 19:33 |
| srichter | however, I think iterating through the records might not be sufficient then | 19:33 |
| srichter | because not all references get updated | 19:34 |
| ymmit | You said two things about it: "does not seem to be efficient" and "did not get rid of all old references". | 19:34 |
| srichter | s/efficient/sufficient | 19:34 |
| ymmit | Ah, that helps a bit. | 19:34 |
| srichter | sorry :-| | 19:34 |
| ymmit | That's fine. | 19:34 |
| ymmit | My problem is that since I haven't seen any code that changes anything, I'm having a hard time guessing blind why it doesn't make all the changes you want made ;-) | 19:35 |
| *** philiKON has quit IRC | 19:35 | |
| srichter | ok, so let me ask you a question: | 19:35 |
| srichter | let's say I have object a | 19:36 |
| srichter | it is referenced in object b | 19:36 |
| srichter | such that b.a gives me object a | 19:36 |
| srichter | if I now say: b._p_changed = True | 19:36 |
| srichter | and commit the transaction | 19:37 |
| srichter | will the class reference of a be also written? | 19:37 |
| ymmit | Yup. | 19:37 |
| *** gintas has joined #zope3-dev | 19:37 | |
| srichter | mmh, that's strange | 19:38 |
| *** alga has joined #zope3-dev | 19:38 | |
| srichter | what about if I have a third object c, which is referenced in b, such that b.c gives obejct c | 19:38 |
| srichter | will b._p_changed = True | 19:39 |
| srichter | followed by a commit | 19:39 |
| srichter | change the class reference of c in the ZODB as well? | 19:39 |
| ymmit | AFAICT, that's exactly the same question, replacing "a" and "b" with "b" and "c" respectively. | 19:39 |
| ymmit | Or vice versa ;-) | 19:40 |
| srichter | oh, I did it wrong thanks | 19:40 |
| srichter | let's say I have an object d, that is referenced in, such that a.d gives object d | 19:41 |
| ymmit | When an object X is written out, references to all persistent objects directly (1 step) reachable from X are written out. | 19:41 |
| srichter | and b.a.d gives obejct d | 19:41 |
| srichter | if I say b._p_changed = True, will the class reference to d be updated? | 19:41 |
| mgedmin | is a persistent in this question? | 19:41 |
| ymmit | Is b.a persistent or non-persistent? | 19:42 |
| srichter | it is persistent | 19:42 |
| ymmit | If b.a is non-persistent, then b.a's entire state is written out too, which must include the reference to b.a.d. | 19:42 |
| ymmit | If b.a is persistent, then no, writing b writes a reference to a, and that's it. | 19:43 |
| srichter | aha | 19:43 |
| srichter | that's the problem then | 19:43 |
| mgedmin | srichter, but you mark all persistent objects as changed | 19:43 |
| mgedmin | or don't you? | 19:44 |
| srichter | mgedmin: no, the above code does not do that | 19:44 |
| mgedmin | I see | 19:44 |
| srichter | this iterator will actually not return all objects in the DB as far as I can tell | 19:44 |
| ymmit | The code snippet above marks the current revision of all non-version data as changed. | 19:44 |
| ymmit | Non-persistent objects don't have "revisions", so it won't return any of those -- only persistent objects. | 19:45 |
| ymmit | Say P is persistent. | 19:45 |
| ymmit | Say P.d is a Python dictionary. | 19:45 |
| ymmit | P.d will not be returned by the code snippet above. | 19:45 |
| ymmit | If P.d.values contains persistent objects, the snippet above won't find them. | 19:46 |
| srichter | mgedmin: I think our problem is that we have those non-persistent objects | 19:46 |
| ymmit | But it will find P, and when P is written out, the entire state of P.d will get written out at the same time. | 19:46 |
| srichter | ymmit: ok, so we do store some non-persistent objects and they seem to be the culprit here | 19:47 |
| ymmit | Don't know. You can't store a non-persistent object directly in ZODB. It has to be reachable from a persistent object. | 19:48 |
| srichter | yes, I think it is | 19:48 |
| mgedmin | ymmit, why won't the snippet above find persistent objects reachable from P? | 19:48 |
| mgedmin | doesn't a record store the pickle of exactly one persistent object? | 19:49 |
| ymmit | The snippet above finds the current revision of all non-version persistent objects. | 19:49 |
| ymmit | Yes, a record stores the pickle of exactly one persistent object (include the full state of all non-persistent objects "attached" to it). | 19:49 |
| mgedmin | yes | 19:49 |
| mgedmin | <ymmit> If P.d.values contains persistent objects, the snippet above won't find them. | 19:50 |
| mgedmin | so if p.d.values() contains other persistent objects, they will be stored as records somewhere | 19:50 |
| ymmit | The snippet above finds the current revision of all non-version persistent objects. | 19:50 |
| mgedmin | and that code snippet should find the current revision of persistent records stored in p.d.values() | 19:50 |
| ymmit | Yes, the snippet above finds the current revision of all non-version persistent objects ;-) All means all. | 19:51 |
| mgedmin | (there are no versions in sight in SchoolTool, which is where srichter is solving this problem) | 19:51 |
| ymmit | That doesn't excuse me from being accurate ;-) | 19:51 |
| mgedmin | ok, the "won't find" bit confused me a bit, that's all | 19:51 |
| ymmit | But for sanity's sake, I'll leave versions out of this. | 19:52 |
| srichter | yeah, I am sure that our non-persistent objects are the problem | 19:52 |
| ymmit | When P is written, the entire state of the dictionary P.d is written out too. | 19:52 |
| ymmit | If P.d has any persistent objects as values, references to them will get written out as part of writing out P.d's full state. | 19:53 |
| srichter | because we have a non-persistent object N that references a persistent object P, such that N.P gives P | 19:53 |
| ymmit | I don't know which code is actually changing references, though (it's not in the snippet above), so perhaps that code might miss P.d.values(). | 19:53 |
| srichter | the references are automatically changed during import | 19:54 |
| ymmit | N must be reachable _from_ a persistent object, though. | 19:54 |
| srichter | yes, it is | 19:54 |
| ymmit | Call that P. | 19:54 |
| ymmit | The snippet above finds P. | 19:54 |
| ymmit | And when P is written, N's full state is written at the same time. | 19:55 |
| *** sashav has joined #zope3-dev | 19:55 | |
| ymmit | I need to eat ;-) Is this a decent time for a break? | 19:56 |
| srichter | yes | 19:56 |
| srichter | thanks a lot for your help so far | 19:56 |
| * ymmit going out for lunch; back in < an hour | 19:58 | |
| *** RaFromBRC has joined #zope3-dev | 20:08 | |
| *** lunarosity has joined #zope3-dev | 20:18 | |
| *** RaFromBRC is now known as RaFromBRC|away | 20:32 | |
| *** hwo4 has quit IRC | 20:36 | |
| *** nederhoed has joined #zope3-dev | 20:37 | |
| srichter | aha, _p_changed = True does not work; it does not seem to cause a write | 20:39 |
| *** lunatik has joined #zope3-dev | 20:43 | |
| srichter | ymmit: mgedmin: aha, I found it | 20:44 |
| *** lunatik has left #zope3-dev | 20:45 | |
| srichter | let's say I have a module "orig" | 20:45 |
| srichter | orig defines three classes Level1(Persistent), Level2(object), Level3(Persistent) | 20:45 |
| srichter | they are refernced as follows: | 20:45 |
| srichter | level1 = Level1 | 20:45 |
| srichter | level1.level = Level2 | 20:46 |
| srichter | level1 = Level1() | 20:46 |
| srichter | level1.level = Level2() | 20:46 |
| srichter | level1.level.level = Level3() | 20:46 |
| srichter | level1 is stored into the ZODB | 20:46 |
| srichter | now all three classes move to module "new" and are only imported in "orig" as follows: | 20:47 |
| srichter | from new import Level1, Level2, Level3 | 20:48 |
| *** Soulraven has joined #zope3-dev | 20:48 | |
| srichter | I then load the DB and get level1 | 20:48 |
| srichter | like that: level1 = conn.root()['level1'] | 20:49 |
| srichter | then I say: level1._p_changed = True | 20:49 |
| srichter | and commit the transaction | 20:49 |
| mgedmin | I expect level1 and level1.level to refer to the new module, but level1.level.level to refer to the old module | 20:50 |
| srichter | no, that's the point | 20:50 |
| srichter | level1.level and level1.level.level refer to new and level1 to orig | 20:50 |
| mgedmin | eh? | 20:50 |
| srichter | yes! sounds crazy eh? | 20:51 |
| srichter | I have the code to prove it so, it is not a concise, one executable test yet | 20:51 |
| srichter | because if I move the code back to "orig" and remove "new" only level1 will load | 20:53 |
| srichter | both the Level2 and Level3 instance are broken | 20:53 |
| srichter | mgedmin: mmh, though, if I dump the records, I can clearly see that level 3 is still orig | 20:57 |
| srichter | which makes more sense | 20:57 |
| srichter | mgedmin: ah, and I know why | 20:58 |
| srichter | because the root persistent mapping did not update its reference, but the object itself did | 20:59 |
| srichter | so this has nothing to do with non-persistency | 20:59 |
| srichter | references to an object are simply not updated | 20:59 |
| ymmit | Note that your code snippet (way above) skipped the root object. | 21:01 |
| ymmit | Don't know whether that's relevant here. | 21:01 |
| mgedmin | srichter, I can reproduce your results | 21:01 |
| srichter | mgedmin: cool | 21:02 |
| srichter | ymmit: no, it's not relevant; it would be only relevant to our application object | 21:02 |
| srichter | but not in general | 21:02 |
| srichter | ymmit: actually the example way up there does include the root; it is txn 0 | 21:04 |
| mgedmin | it is relevant | 21:04 |
| *** nederhoed has quit IRC | 21:04 | |
| mgedmin | if I change my second script to also do conn.root().p_changed = True | 21:04 |
| mgedmin | the third script can load all three objects, and see that they refer to the new module | 21:04 |
| ymmit | srichter: no, you started by passing oid 0 to record_iternext(). The first oid you get back is then the smallest oid larger than 0. | 21:05 |
| *** AJC has joined #zope3-dev | 21:05 | |
| srichter | ymmit: ahh, ok | 21:05 |
| *** faassen has quit IRC | 21:05 | |
| ymmit | Start by passing None instead (like the example in NEWS.txt). | 21:05 |
| srichter | ok | 21:06 |
| srichter | ymmit: where is NEWS.txt? | 21:06 |
| ymmit | But then your ``while`` loop will exit instantly too ;-) | 21:07 |
| srichter | I fixed it already :-) | 21:07 |
| ymmit | In a ZODB checkout, or on any ZODB download page. | 21:07 |
| ymmit | Ack, but there hasn't been a release of ZODB 3.5 yet, so the only place to find its NEWS.txt is in SVN. | 21:08 |
| ymmit | http://svn.zope.org/ZODB/trunk/NEWS.txt?rev=38077&view=markup | 21:08 |
| *** bradb-lunch is now known as bradb | 21:09 | |
| srichter | I have it thanks | 21:09 |
| mgedmin | my understanding of zodb has increased considerably thanks to this discussion | 21:11 |
| ymmit | mgedmin: You have my apologies for that ;-) | 21:11 |
| *** donkey has joined #zope3-dev | 21:11 | |
| *** AJC has quit IRC | 21:12 | |
| srichter | ok, even if the root object gets included it does not fix the problem | 21:12 |
| srichter | mgedmin: so it seems to be a problem with out non-persistent objects | 21:12 |
| mgedmin | no, no, I enjoyed it ;) | 21:13 |
| Soulraven | does this mean i have to dl "ConfigurationError ("invalid value for" "Factory" Couldent import zope.app.tree.utills. no module named zlib ? | 21:15 |
| Soulraven | even tho i installed a private python 2.3.5 (from source) | 21:15 |
| srichter | Soulraven: one of the requirements of Zope 3 is that you have zlib installed | 21:17 |
| Soulraven | does it have to be private if im using a private python for zope=? | 21:17 |
| fdrake | this probably means the library isn't installed, or more likely, that the headers aren't installed | 21:17 |
| *** MJ has joined #zope3-dev | 21:17 | |
| fdrake | no; it can be a system install | 21:17 |
| fdrake | but you'll need to re-configure and build Python | 21:18 |
| fdrake | that should be quick if you still have your build tree | 21:18 |
| Soulraven | umm.. | 21:18 |
| fdrake | (actually, you can probably skip the re-configuration and see if it gets picked up) | 21:18 |
| Soulraven | allright | 21:19 |
| Soulraven | thx again again..:) | 21:19 |
| fdrake | most RPM-based systems will have a libz package and a libz-dev or libz-devel package | 21:19 |
| fdrake | the later contains things like headers | 21:19 |
| Soulraven | oh, i got libz but not dev... :) | 21:20 |
| *** replicant is now known as hazmat | 21:23 | |
| srichter | mgedmin: ymmit: okay, if I am doing pers_obj._p_changed = True, it seems that the reference of pers_obj is not stored correctly | 21:27 |
| srichter | here is my test scenario | 21:27 |
| srichter | orig.py: | 21:27 |
| mgedmin | the class name of pers_obj is not part of the record of pers_obj | 21:27 |
| mgedmin | you have to find all objects that reference pers_obj and mark them as changed | 21:28 |
| srichter | mmh, here is the record for Level1 | 21:28 |
| mgedmin | suddenly I started doubting my understanding -- the "the class name of pers_obj is not part of the record of pers_obj" bit in particular | 21:28 |
| srichter | ('\x00\x00\x00\x00\x00\x00\x00\x01', '\x03_\x90m\x8c\x03!\xaa', 'corig\nLevel1\nq\x01.}q\x02U\x05levelq\x03ccopy_reg\n_reconstructor\nq\x04(corig\nLevel2\nq\x05c__builtin__\nobject\nq\x06NtRq\x07}q\x08h\x03(U\x08\x00\x00\x00\x00\x00\x00\x00\x02q\tcorig\nLevel3\nq\ntQsbs.', '\x00\x00\x00\x00\x00\x00\x00\x02') | 21:28 |
| mgedmin | yep, that bit is wrong | 21:29 |
| J1m | The data record for an object defininately incluses it's class name. | 21:30 |
| srichter | yes, as we can see above, though it did not get updated | 21:30 |
| *** nederhoed has joined #zope3-dev | 21:30 | |
| J1m | If the __module__ and __name__ of all classes are what you want them to be, then writing all persistent objects in the db should leave you with correct data. | 21:31 |
| nederhoed | hello, I want to add a contact form to my site, generating an e-mail when submit is pressed | 21:31 |
| nederhoed | what is the way to? | 21:31 |
| srichter | J1m: but it clearly does not | 21:31 |
| J1m | Perhaps there's a __module__ or __name__ that's not what you expect. | 21:31 |
| nederhoed | (in zope3 naturally) | 21:31 |
| mgedmin | srichter, are you sure you're looking at te correct record? | 21:31 |
| mgedmin | none of the references were changed -- they all point to orig | 21:32 |
| srichter | positive | 21:32 |
| J1m | You don't change any __module__ or __name__ at run time, do you? | 21:32 |
| srichter | no | 21:32 |
| srichter | I specifically move files around | 21:32 |
| srichter | and run the test script with stuff commented out | 21:33 |
| J1m | well, you have some other problem and I don't have time to find it for you. | 21:33 |
| J1m | I know how this stuff works. | 21:33 |
| srichter | (note that this has nothing to do with SchoolTool) | 21:33 |
| * J1m wonders if stale pycs are causing a problem | 21:33 | |
| srichter | I have written a test from scratch only using ZODB | 21:34 |
| srichter | I removed those too | 21:34 |
| J1m | If you wish, you can send me the test script. | 21:34 |
| mgedmin | srichter, wanna take a look at http://mg.pov.lt/zodb-test.tar.gz | 21:34 |
| mgedmin | I think I replicated your use case | 21:34 |
| ymmit | srichter: Can you send code to zodb-dev? | 21:34 |
| mgedmin | and it works for me | 21:34 |
| mgedmin | although I do not do iteration over ZODB records | 21:35 |
| mgedmin | just touch a couple of concrete objects | 21:35 |
| ymmit | srichter: You'd get more eyeballs that way, and I've been frustrated here from the start at not having concrete code to poke at. | 21:35 |
| srichter | ah ok, I specifically tried the iteration stuff | 21:35 |
| srichter | ok | 21:35 |
| srichter | I am modifying marius' code to what I though should work | 21:35 |
| nederhoed | ehm, can anyone point me to an example of z3 code to process a form? | 21:36 |
| *** donkey has quit IRC | 21:37 | |
| *** BjornT_ has joined #zope3-dev | 21:41 | |
| srichter | I think I found it | 21:41 |
| mgedmin | my code has a mistake | 21:41 |
| srichter | _p_changed = True does not necessarily cause a write | 21:41 |
| mgedmin | how do you force a load? | 21:42 |
| mgedmin | _p_activate()? | 21:42 |
| mgedmin | I can never remember the API... | 21:42 |
| srichter | that is certainly a valid method :-) | 21:43 |
| srichter | YIPEEEE! | 21:49 |
| srichter | with Marius' help we got it | 21:50 |
| srichter | here the scoop: | 21:50 |
| srichter | obj._p_changed = true does *not* cause an object to be written | 21:50 |
| srichter | we had to say obj._p_activate() as well to cause the write | 21:51 |
| srichter | and the thing is, you need both: _p_changed and _p_activate | 21:52 |
| *** BjornT has quit IRC | 21:52 | |
| srichter | J1m: ymmit: thanks a lot for you help; | 21:52 |
| srichter | s/;/!!! :-) | 21:52 |
| srichter | nederhoed: look at zope.app.form | 21:53 |
| J1m | did you have to call _p_activate and then set _p_changed? | 21:53 |
| J1m | srichter, | 21:54 |
| srichter | yes | 21:54 |
| mgedmin | http://paste.plone.org/3492 is what I tried, and found that it worked | 21:55 |
| srichter | (btw, the iteration thing is ultra-cool; now the rest of my evolution script will be a breeze) | 21:56 |
| J1m | This is a bug | 21:56 |
| J1m | yopu said a ghost had changed | 21:56 |
| J1m | you said a ghost had changed | 21:56 |
| J1m | and zodb thought that was silly, but didn't tell you how silly you were. ;) | 21:56 |
| srichter | LOL | 21:57 |
| srichter | what is a ghost? | 21:57 |
| J1m | an object whos state hasn't been loaded from the database. | 21:58 |
| srichter | I see | 21:58 |
| fdrake | "unchagable, but resurrectable" | 21:58 |
| fdrake | "unchangable, but resurrectable" | 21:58 |
| srichter | ok, that makes sense | 21:58 |
| fdrake | resurrection at the cost of disk access... hmm, it's getting cheaper :-) | 21:59 |
| srichter | I wanna be a ghost ;-) | 22:00 |
| srichter | the record iterator makes evolution scripts soooo much easier | 22:01 |
| fdrake | My daughter's guinnea pig died last night; I wonder how much disk access it takes to resurrect a guinnea pig... | 22:02 |
| srichter | I'd say 2.4906 TB, exactely | 22:02 |
| fdrake | ah, drat, my disk controller isn't rated for that. | 22:03 |
| nederhoed | srichter: merci | 22:05 |
| *** RaFromBRC|away has quit IRC | 22:14 | |
| *** Aiste has quit IRC | 22:20 | |
| *** alga has quit IRC | 22:21 | |
| *** mgedmin has quit IRC | 22:22 | |
| *** ignas has quit IRC | 22:28 | |
| *** Soulraven has quit IRC | 22:55 | |
| *** ymmit has left #zope3-dev | 22:57 | |
| SteveA | why don't we use events for parsing zcml ? | 23:09 |
| J1m | Why would we? | 23:11 |
| SteveA | i just had a requirement to hook some application-specific policy checks in the view support classes used for browser:page directives | 23:12 |
| SteveA | i need to override browser:page for this, right now | 23:12 |
| SteveA | but, if they used events, i could just hook into the 'browser:page' event | 23:12 |
| SteveA | and do the check there. | 23:12 |
| fdrake | isn't there a chicken-and-egg problem here? | 23:12 |
| SteveA | events are part of the CA | 23:12 |
| SteveA | they don't depend on zcml | 23:12 |
| fdrake | right | 23:13 |
| SteveA | i'd happily hook these up in python | 23:13 |
| fdrake | but we generally register them in ZCML | 23:13 |
| fdrake | hmm | 23:13 |
| J1m | The configuration machine has it's own adapter registry. | 23:13 |
| SteveA | that | 23:13 |
| SteveA | that's interesting. i'd assumed it used the global one | 23:13 |
| J1m | Once could wire subscribers into that using meta directives | 23:13 |
| SteveA | anyway, just throwing the idea out there | 23:14 |
| SteveA | i'm not going to re-do zcml parsing just for this case | 23:14 |
| SteveA | i've noted it here, anyway: https://wiki.launchpad.canonical.com/Zope3Changes | 23:15 |
| SteveA | by the way, have you seen https://wiki.launchpad.canonical.com/ZopeHttpHeaderEncoding ? | 23:16 |
| SteveA | basically, zope3 (and probably zope2 too) does http header encoding wrongly | 23:17 |
| SteveA | and this causes bugs sometimes | 23:17 |
| J1m | It sounds like this should be a collector entry | 23:17 |
| SteveA | the correct solution is rather more complex | 23:17 |
| fdrake | yes, it should be | 23:17 |
| SteveA | the collector entry was on my todo list ;-) | 23:17 |
| J1m | I assume you plan to fix z3 | 23:17 |
| SteveA | this was discovered / written in brazil, where we'd lost internet, but had a local wiki | 23:18 |
| J1m | ah | 23:18 |
| SteveA | yes, we'll fix zope3, although have no firm date planned | 23:18 |
| fdrake | i started looking at some of the problem of encodings related to the content-type header and the body itself, but not the encoding of the headers | 23:18 |
| * SteveA goes to file the collector entry | 23:19 | |
| J1m | I'm surprised that z3 is encoding headers. | 23:19 |
| SteveA | do you know if zope2 does? | 23:19 |
| J1m | It shouldn't. | 23:19 |
| J1m | I doubt ir=t. | 23:19 |
| J1m | I doubt it. | 23:19 |
| J1m | In any case, I'd expect z2 and z3 to behave very differently in this area. | 23:19 |
| nederhoed | during "make check" I get an error on testUmask | 23:22 |
| nederhoed | 3.1.0c1 | 23:22 |
| nederhoed | not from trunk | 23:22 |
| J1m | are you running the tests as root? | 23:22 |
| nederhoed | ahum | 23:22 |
| nederhoed | yes | 23:22 |
| J1m | don't | 23:22 |
| nederhoed | good point | 23:23 |
| nederhoed | merci | 23:23 |
| SteveA | http://www.zope.org/Collectors/Zope3-dev/446 | 23:23 |
| nederhoed | I do want to install zope3 eventually as root, no? | 23:24 |
| SteveA | if you want zope3 to be accessible on port 80, as a standard web server, then something needs to run as root (at least initially) | 23:25 |
| nederhoed | I use /opt/Zope-3.1.0c1/ and add zopectl as a service | 23:25 |
| *** bskahan has quit IRC | 23:25 | |
| SteveA | the usual way to do this is to have zope running as, say, user 'zope' | 23:25 |
| nederhoed | I have it behind apache (currently using X3.0.0) | 23:25 |
| SteveA | and have it listening on some nondescript port, like 8654 | 23:25 |
| SteveA | and then proxypass it through apache | 23:25 |
| nederhoed | ok, sounds logical | 23:26 |
| SteveA | there is a special URL syntax to use when proxypassing, to tell Zope that it is being proxied | 23:26 |
| SteveA | so that it can generate the correct URLs for the outside world | 23:26 |
| SteveA | when it generates absolute urls for things like Location: HTTP headers | 23:26 |
| SteveA | and for the @@absolute_url view | 23:26 |
| nederhoed | yes, I think I have taken that from the Weitershausen book | 23:26 |
| nederhoed | there are quite some steps to be taken to get from nothing to a simple Z3 app running reliably on a server | 23:27 |
| nederhoed | the reason I like Zope3 is it's architecture. I (want to) use it to develop custom applications, but | 23:28 |
| nederhoed | often end up figuring out how to get things working in the perifiry | 23:29 |
| nederhoed | I mean knowledge related to keeping zope up and running | 23:29 |
| *** fdrake has left #zope3-dev | 23:35 | |
| *** GaryPoster has joined #zope3-dev | 23:37 | |
| *** nederhoed_ has joined #zope3-dev | 23:41 | |
| nederhoed_ | sorry for my disappearance, there was a very short, local outage :S | 23:42 |
| *** nederhoed_ has quit IRC | 23:43 | |
| MJ | Not that short then... | 23:44 |
| *** nederhoed_ has joined #zope3-dev | 23:47 | |
| *** nederhoed has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!