IRC log of #zope for Wednesday, 2011-04-27

*** supton has quit IRC00:04
*** menesis has joined #zope00:05
*** J1m has left #zope00:16
*** J1m has quit IRC00:16
*** supton has joined #zope00:23
*** hever has joined #zope00:28
*** robohak has joined #zope00:38
*** ccomb has joined #zope00:51
*** alga has quit IRC01:15
*** Arfrever has quit IRC01:15
*** benji has quit IRC02:00
*** supton has quit IRC02:00
*** f10w has quit IRC02:05
*** f10w has joined #zope02:06
*** benji has joined #zope02:07
*** TresEquis has quit IRC02:15
*** menesis has quit IRC02:15
*** runyaga has quit IRC02:21
*** menesis has joined #zope02:32
*** mr_jolly has left #zope02:32
*** MrTango has quit IRC02:56
*** qwebirc38763 has quit IRC02:57
*** pjfd has quit IRC02:58
*** ccomb has quit IRC03:02
*** menesis has quit IRC03:04
*** planetzopebot has quit IRC03:11
*** giampaolo has quit IRC03:16
*** daMaestro has quit IRC03:16
*** Spanktar has quit IRC03:23
*** robohak has quit IRC03:28
*** hever has quit IRC03:30
*** goschtl has joined #zope03:32
*** goschtl has quit IRC03:45
*** tiwula has quit IRC03:45
*** sylyc0n has joined #zope03:47
*** varta has left #zope03:52
*** pjfd has joined #zope03:56
*** tiwula has joined #zope03:56
*** pjfd is now known as pjfd404:04
*** planetzopebot has joined #zope04:09
*** tiwula has quit IRC05:56
*** sm has quit IRC06:14
*** gqlewis has quit IRC06:28
*** gqlewis has joined #zope06:35
*** gqlewis has quit IRC06:36
*** redir has quit IRC06:36
*** mr_jolly has joined #zope07:05
*** MrTango has joined #zope07:09
*** tiwula has joined #zope07:34
*** mr_jolly has quit IRC07:52
*** mr_jolly has joined #zope07:53
*** Gershwin has quit IRC08:15
*** slackrunner has joined #zope08:16
*** mr_jolly has left #zope08:31
*** slackrunner has quit IRC08:46
*** wosc has joined #zope08:50
*** kleist has joined #zope08:55
*** __mac__ has joined #zope09:02
*** hever has joined #zope09:16
*** hever has quit IRC09:26
*** alga has joined #zope09:28
*** agroszer has joined #zope09:38
*** redir has joined #zope09:43
*** Wu has joined #zope09:52
*** hever has joined #zope09:56
*** tiwula has quit IRC10:05
*** avoinea has joined #zope10:05
*** planetzopebot has quit IRC10:08
*** planetzopebot has joined #zope10:09
*** hever has quit IRC10:10
*** goschtl has joined #zope10:14
*** vortec has joined #zope10:20
*** sylvain has joined #zope10:35
*** avoinea has quit IRC10:44
*** avoinea has joined #zope10:47
*** eperez has joined #zope11:10
*** espent has joined #zope11:12
espenthey, i got a very troublesome zope-installation here. Its a site we are taking over the hosting for. it runs good for a couple of hours, but then after a couple of hours it seems like its ending up in a endless loop... how can i profile the site to get down to where the problem si?11:14
espentits zope 2.911:14
betabugwhat kind of application is installed on the site?11:14
espenti dont know11:15
betabugis there anything in the log (INSTANCE_HOME/logs/event.log)11:15
espentsure, its logging good11:15
betabugespent: ok, first thing, find where the INSTANCE_HOME is11:15
betabugin there is a folder called "Products"11:15
betabughave a look what's in there11:15
betabugcheck the event.log too11:15
espentit loads of them11:16
betabugput a listing on a pastebin?11:16
espenttheres a ZopeProfiler there i can see11:17
espentmaybe that is causing it to run very slowly?11:17
betabugwell, if profiling is enabled, that might take some toll11:18
betabugbut it shouldn't cause a standstill11:18
espentits not completely frozen when it "stops", there is things going on in event.log and Z2.log, but the varnish cache in front times out on the request11:18
betabugat first glance it looks like you have an old version of this:
betabugare there a lot of requests at those times?11:19
betabugI've never heard about that ZTM stuff really11:19
espenti guess thats where the requests go yes... /topics11:20
betabugthe thing with zope is that depending on the setup of the instance, zope runs in a set number of threads11:20
espentsome stuff go fast at the beginning. LIke now, the front page for example, <-- thats the installation11:20
betabugif there are long running requests they can tie up some of the threads for some time11:21
betabugso at some point there might be no free threads any more11:21
espentyes, i think I changed it from 4 to 10 threads11:21
betabughmmm, that doesn't help much11:21
espentit did help a little, but not in the long run11:21
betabugbecause behind the "number of threads" there is also the "number of DB connections"11:21
espentyes, i adjusted the pool_size as well11:22
CIA-83janwijbrand * r121466 grok/ (src/grok/ CHANGES.txt): global_adapter was never imported from grokcore.component, while it is still quite useful11:22
betabugand if there is a bug somewhere that is causing long running threads to get stuck, they will just fill up whatever threads you have11:22
betabugok, next thing to check:11:22
espentit probably is a bug there, yes11:22
betabuglog into the ZMI and find the Control_Panel11:22
espentiam in11:22
betabugand in the Control_Panel, there is a link called "DebugInfo"11:23
espent"Debug Information"11:23
espentok clicked11:23
betabugon the bottom of that page there is a table "Connections:"11:23
betabugwhich lists your threads11:23
espentah yeah, its like apache server-status11:23
betabugshowing what they are doing, how long they've been running etc11:23
betabugmaybe you can see there what is getting stuck11:24
espentand when the thread is not doing anything it goes back to state 'None', right?11:25
betabugyes, on "None" it's waiting for new incoming requests11:26
betabugyou want to look for stuff being listed with a long time in the left column11:26
betabugand not going away11:26
espenti just restarted, but i'll let it run for a couple of hours, and then visit this page again11:27
betabugand then you can check from the URL what method it's trying to run and maybe find out what takes so long11:27
betabugyes, exactly, that's the method11:27
espentcool, thanks for good help!11:28
betabugno problem, you're welcome11:28
*** menesis has joined #zope11:38
*** Reite has joined #zope11:40
*** cpf_ has joined #zope11:47
*** sunew has joined #zope11:47
*** benji has quit IRC11:56
*** evilbungle has joined #zope11:58
*** digitalmortician has joined #zope12:05
*** benji has joined #zope12:05
*** TomBlockley has joined #zope12:13
*** __mac__1 has joined #zope12:31
*** __mac__ has quit IRC12:31
*** __mac__2 has joined #zope12:31
CIA-83janjaapdriessen * r121467 /Sandbox/janjaapdriessen/buildbot/buildout.cfg: Add more js.* packages.12:32
CIA-83janwijbrand * r121468 grokcore.component/ (4 files in 3 dirs): fix global_adapter directive and grokker to allow for an explicit nameless registration even if the factory has a set12:32
CIA-83janwijbrand * r121469 grokcore.component/ (CHANGES.txt Preparing release 2.412:32
CIA-83janwijbrand * r121470 /grokcore.component/tags/2.4: Tagging 2.412:32
CIA-83janwijbrand * r121471 grokcore.component/ (CHANGES.txt Back to development: 2.512:32
CIA-83janwijbrand * r121472 groktoolkit/grok.cfg: update grokcore.component12:32
*** __mac__1 has quit IRC12:35
*** __mac__2 has quit IRC12:58
espentbetabug: there?13:20
betabugyes, now :-)13:20
betabughad a meeting13:21
betabuggot any stuck threads yet?13:21
espentits in the lock now, but unfortunatly i am unable to access the admin interface. Is it possible to kill/hup one thread or something?13:21
betabughmmm, too bad13:21
betabugnot that I know of13:21
betabugthere is a product called "DeadLock Debugger"13:22
betabugI've no personal experience with it, but maybe it could help13:22
betabugpersonally, I'd restart it now and then run a cron job to write the debug info page to disk every 20-30 minutes13:22
espenthm deadlock debugger looks good if it does what the documentation says it will13:24
espentill give it a try13:24
betabugnever had to use it, but give it a try13:26
betabugand maybe you can write the debug info page to disk anyway :-)13:26
*** __mac__ has joined #zope13:34
espentthe page requires that i am authenticated dont it13:37
betabuganother meeting :-)13:39
espenthave fun :)13:39
betabugwasn't that bad in the end13:51
espentdoesnt look like deadlock debugger will accept my threadframe installation14:04
espentthe tests shipped with threadframe looks ok though14:05
koshbetabug: greetings lifeform14:05
betabugtoo bad14:05
betabughey kosh14:05
koshbetabug: I have been making good progress on a better patch for OFS Image and File blob support by detecting if the current storage supports blobs14:05
betabugkosh: cool!14:05
koshmore threads does not make zope less likely to lockup, it can actually make it more likely14:07
betabugwell, I suspect something is eating up those threads, like a request getting stuck14:08
betabugclassical example is something waiting for an external resource14:08
betabugbut he has to find out somehow what it is14:08
koshyes but more threads does not make that better, just a little longer before failing14:08
betabugyupp, it gives you a tiny bit more time till you have to hit "restart" :-)14:08
betabugI have no experience with DeadlockDebugger, but it doesn't seem to work in this case14:09
koshI wonder how the zope devs would feel about images and files having versioned urls in the base, they are just making such a huge difference for my code that I would like others to use them also14:11
betabugkosh: I think this would have to be handled as an add-on config14:11
betabugother than that, I think they would "feel" about it a total indifference14:12
koshI have an add on product that can do it but not automatically for files and images14:12
koshand in my own product I built it in so that every file and image tht runs through my product does it automatically and it has made a large difference14:12
koshzope does set appropriate headers so that a browser will just do a GET and not have to download anything but if you have 40 images that is still 40 additional requests and you can make that 0 requests14:13
*** evilbungle has quit IRC14:16
betabugwell, at least zope can make a conditional GET14:17
koshit does handle that correctly by default14:17
koshso you don't download the image again14:17
koshbut not even trying to grab the image again is even better14:17
betabugplone already has that in some add-on product14:17
koshah interesting14:18
*** mgedmin has joined #zope14:19
koshwith pyramid using more ztk stuff it might actually be possible to move a zope 2.x app to pyramid in another year or so if you write your app correctly :)14:19
betabugwell, this one app here was written a long time ago14:19
koshwell it can use zeo already, it can load a zodb, the security policy is pretty compatible14:20
koshmy stuff basically depends on my own product and exUserFolder and that is it14:20
*** evilbungle has joined #zope14:23
koshbetabug: well the more recent stuff I have been building is to move scripts out of zope and into a nice library I am building14:27
kosheven simple things like I just made a method that will create a link entry for css by just handing it an object and an optional versioning object that way you don't have to actually write the <link etc part>14:28
koshit actually seems to be working very nicely in making stuff easier to get right14:28
koshso just  com.html.link_css(cssobject)  or com.html.link_css(cssobject, cache_version_object)     that way it is easier to work with and fix14:29
koshdoing that for doctype generation, generating jquery html, js stuff, iterating over a catalog safetly  since getObject raises exceptions that you can't really catch inside ttw code etc14:30
koshwhich is helping to build stuff faster and easier to maintain14:30
koshI want to make as many things common as possible with a nice python interface14:31
*** m8 has joined #zope14:32
koshthat way when you move to something like html5 you can change the stuff in a common library and it would apply retroactively to every system you have14:32
*** yvl has joined #zope14:50
*** TomBlockley_ has joined #zope14:50
*** TomBlockley has quit IRC14:52
*** TomBlockley_ is now known as TomBlockley14:52
*** Wu has quit IRC14:54
koshbetabug: well so far my newer patch is working very nicely, it will enable to disable blob support based on if the current storage support it15:13
koshbetabug: and it will work if you have multiple mounted storages and some support blobs and some don't since it checks if the storage you are currently using support blobs :)15:13
*** wosc has quit IRC15:17
*** m8 has quit IRC15:30
CIA-83hannosch * r121473 Products.ZCatalog/ (src/Products/ZCatalog/ CHANGES.txt): Fix possible TypeError in `sortResults` method if only b_start but not b_size has been provided.15:36
*** ccomb has joined #zope15:43
espentbetabug: looks like I got some clues here now. Several threads are using very long time, and all of the is related to ZTopic -> CMFBTreeFolder -> CMFSite15:56
espenta couple of them have > 600s process time15:56
betabugmight very well be your culprit15:56
espentand that is?15:56
betabugcheck the methods in question15:57
betabugI don't know this CMF stuff, not using it myself15:57
koshwhy would a BTreeFolder take more then 600s ?15:57
betabugand I guess you've got some pretty old versions there15:57
espentyes probably15:57
betabugI guess the method in question is doing some kind of job15:57
*** benji is now known as Guest4143415:58
betabuga BTreeFolder should be happy and quick with 10000s and 100'000s of objects in it15:58
*** benji___ has joined #zope15:58
*** Guest41434 has quit IRC15:59
*** benji___ has quit IRC15:59
*** benji____ has joined #zope15:59
*** __mac__ has quit IRC16:00
*** benji____ is now known as benji16:01
betabugoooold stuff16:01
betabugnow it might be that there was a newer version, who knows16:02
espentcant believe they managed to use this stuff in 2000, since its so slow16:03
betabughmmm, dunno16:03
koshBTreeFolder2 I know handles hundreds of thousands of objects just fine16:03
betabugI guess it wasn't always that slow, but then something happened16:03
koshbecause I have some :)(16:03
koshhmm one too many characters there16:03
betabugalso Zope 2.9 isn't from 2000, it came much later - so there might be some stuff later than that ZTopic 1.016:04
espentyeah sure, ZTopic used to be a Product?16:04
betabugespent: what is the exact method / path that they hang on?16:04
betabugit should be in your instance's "Products" folder16:05
espenti need to reproduce it16:06
espenti didnt take copy of the debug info16:06
betabugespent: does the Database file of this instance get "packed" from time to time?16:06
espentyes, i've done it manually16:06
betabugso, ZTopic basically makes "canned" Catalog searches16:07
betabug(looking at the README file in the code here)16:07
betabugZope Software Archeology :-)16:07
betabughmm, these ZTopics should do caching16:08
*** huajie has joined #zope16:08
betabugin fact, the description from the readme reads like it's something really useful :-)16:09
betabugmaybe something hits the URL that clears the cache?16:10
espentcould be16:10
espentzope internal cache?16:10
betabugwell, ZTopic seems to have/use some cache16:10
betabughaven't looked yet what it is16:11
betabuglooks like it puts it into an "intSet", never seen that before16:17
betabugwell, we'd really need to know the method(s) in question16:19
koshbetabug: well I updated my patch, it is much nicer now :)16:35
kosh  is the bug report it is under16:36
_mup_Bug #739692: Add blob support to OFS.Image.Image and OFS.Image.File <Zope 2:Confirmed for tseaver> < >16:36
koshinteresting the bot but that bug on the channel16:37
* kosh beats the bot with a stick :)16:37
espentbetabug: k, thanks again. Need to look more at it tomorrow. Right now its locked again.16:46
betabugoh my16:46
betabugok, good luck so far, I'll be here tomorrow16:46
*** sm has joined #zope16:55
*** __mac__ has joined #zope16:56
*** hever has joined #zope16:56
*** alga has quit IRC16:58
*** J1m has joined #zope17:03
*** goschtl has quit IRC17:08
*** Wu has joined #zope17:11
*** hever has quit IRC17:12
*** huajie has quit IRC17:13
*** Arfrever has joined #zope17:20
*** TomBlockley has quit IRC17:21
*** TomBlockley has joined #zope17:21
*** zagy has quit IRC17:35
*** hever has joined #zope17:39
*** horsey has joined #zope17:40
horseyHi, has anybody successfully installed zope.interface on a 64 bit Windows machine?17:48
betabughorsey: apparently there are buildbots that play through all the zope install for testing17:48
waldiwhat should be the problem? okay, windows is a problem on its own but can't be _this_ broken17:48
betabugso it should work17:48
*** TomBlockley_ has joined #zope17:50
*** evilbungle has quit IRC17:51
*** evilbungle has joined #zope17:51
horseywaldi:  When I run 'python install' on the source directory, I get an error "raise ValueError(str(list(result.keys())) ValueError: [u'path']17:51
*** evilbungle_ has joined #zope17:51
horseyNot sure which path it is.17:52
*** TomBlockley has quit IRC17:53
*** TomBlockley_ is now known as TomBlockley17:53
horseyI cannot install it with easy_install, as setuptools does not provide a 64 bit installer for Windows.  :(17:54
*** evilbungle has quit IRC17:55
*** evilbungle_ is now known as evilbungle17:55
*** sunew has quit IRC17:56
*** bigkevmcd has quit IRC17:57
*** bigkevmcd has joined #zope17:57
*** digitalmortician has quit IRC18:02
*** alvaro_o has joined #zope18:15
*** alvaro_o has quit IRC18:15
*** __mac__ has quit IRC18:42
*** agroszer has quit IRC18:50
*** vortec has left #zope18:50
*** vortec has joined #zope18:52
*** robohak has joined #zope18:56
*** vortec has left #zope19:03
*** tiwula has joined #zope19:14
*** m8 has joined #zope19:26
*** digitalmortician has joined #zope19:29
*** jshell has joined #zope19:34
*** Spanktar has joined #zope19:42
*** m8 has quit IRC19:48
*** __mac__ has joined #zope19:49
*** __mac__ has quit IRC19:53
*** hever has quit IRC19:59
*** TomBlockley has quit IRC20:04
*** runyaga has joined #zope20:19
*** runyaga has quit IRC20:19
*** runyaga has joined #zope20:19
*** srichter has joined #zope20:20
*** eperez has quit IRC20:22
*** goschtl has joined #zope20:29
*** evilbungle has quit IRC20:34
*** avoinea has left #zope20:40
*** goschtl has quit IRC20:51
*** Wu has quit IRC20:54
*** webmaven has joined #zope20:54
*** zagy has joined #zope20:59
*** digitalmortician has quit IRC21:10
*** zagy has quit IRC21:14
*** zagy has joined #zope21:14
*** zagy has quit IRC21:15
*** goschtl has joined #zope21:15
*** zagy has joined #zope21:18
*** hever has joined #zope21:19
*** goschtl has quit IRC21:31
*** ccomb has quit IRC21:36
*** hever has quit IRC21:39
*** m8 has joined #zope21:44
*** alecm has quit IRC21:58
*** zagy1 has joined #zope22:01
*** zagy has quit IRC22:01
*** kleist has quit IRC22:05
*** zagy has joined #zope22:11
*** zagy1 has quit IRC22:11
*** evilbungle has joined #zope22:12
*** supton has joined #zope22:13
*** m8 has quit IRC22:23
*** goschtl has joined #zope22:38
*** goschtl has quit IRC22:40
*** goschtl has joined #zope22:46
*** goschtl has quit IRC22:47
*** menesis has quit IRC22:50
*** srichter has quit IRC23:01
*** zagy has quit IRC23:11
*** MrTango has quit IRC23:12
*** ccomb has joined #zope23:57

Generated by 2.15.1 by Marius Gedminas - find it at!