*** supton has quit IRC | 00:04 | |
*** menesis has joined #zope | 00:05 | |
*** J1m has left #zope | 00:16 | |
*** J1m has quit IRC | 00:16 | |
*** supton has joined #zope | 00:23 | |
*** hever has joined #zope | 00:28 | |
*** robohak has joined #zope | 00:38 | |
*** ccomb has joined #zope | 00:51 | |
*** alga has quit IRC | 01:15 | |
*** Arfrever has quit IRC | 01:15 | |
*** benji has quit IRC | 02:00 | |
*** supton has quit IRC | 02:00 | |
*** f10w has quit IRC | 02:05 | |
*** f10w has joined #zope | 02:06 | |
*** benji has joined #zope | 02:07 | |
*** TresEquis has quit IRC | 02:15 | |
*** menesis has quit IRC | 02:15 | |
*** runyaga has quit IRC | 02:21 | |
*** menesis has joined #zope | 02:32 | |
*** mr_jolly has left #zope | 02:32 | |
*** MrTango has quit IRC | 02:56 | |
*** qwebirc38763 has quit IRC | 02:57 | |
*** pjfd has quit IRC | 02:58 | |
*** ccomb has quit IRC | 03:02 | |
*** menesis has quit IRC | 03:04 | |
*** planetzopebot has quit IRC | 03:11 | |
*** giampaolo has quit IRC | 03:16 | |
*** daMaestro has quit IRC | 03:16 | |
*** Spanktar has quit IRC | 03:23 | |
*** robohak has quit IRC | 03:28 | |
*** hever has quit IRC | 03:30 | |
*** goschtl has joined #zope | 03:32 | |
*** goschtl has quit IRC | 03:45 | |
*** tiwula has quit IRC | 03:45 | |
*** sylyc0n has joined #zope | 03:47 | |
*** varta has left #zope | 03:52 | |
*** pjfd has joined #zope | 03:56 | |
*** tiwula has joined #zope | 03:56 | |
*** pjfd is now known as pjfd4 | 04:04 | |
*** planetzopebot has joined #zope | 04:09 | |
*** tiwula has quit IRC | 05:56 | |
*** sm has quit IRC | 06:14 | |
*** gqlewis has quit IRC | 06:28 | |
*** gqlewis has joined #zope | 06:35 | |
*** gqlewis has quit IRC | 06:36 | |
*** redir has quit IRC | 06:36 | |
*** mr_jolly has joined #zope | 07:05 | |
*** MrTango has joined #zope | 07:09 | |
*** tiwula has joined #zope | 07:34 | |
*** mr_jolly has quit IRC | 07:52 | |
*** mr_jolly has joined #zope | 07:53 | |
*** Gershwin has quit IRC | 08:15 | |
*** slackrunner has joined #zope | 08:16 | |
*** mr_jolly has left #zope | 08:31 | |
*** slackrunner has quit IRC | 08:46 | |
*** wosc has joined #zope | 08:50 | |
*** kleist has joined #zope | 08:55 | |
*** __mac__ has joined #zope | 09:02 | |
*** hever has joined #zope | 09:16 | |
*** hever has quit IRC | 09:26 | |
*** alga has joined #zope | 09:28 | |
*** agroszer has joined #zope | 09:38 | |
*** redir has joined #zope | 09:43 | |
*** Wu has joined #zope | 09:52 | |
*** hever has joined #zope | 09:56 | |
*** tiwula has quit IRC | 10:05 | |
*** avoinea has joined #zope | 10:05 | |
*** planetzopebot has quit IRC | 10:08 | |
*** planetzopebot has joined #zope | 10:09 | |
*** hever has quit IRC | 10:10 | |
*** goschtl has joined #zope | 10:14 | |
*** vortec has joined #zope | 10:20 | |
*** sylvain has joined #zope | 10:35 | |
*** avoinea has quit IRC | 10:44 | |
*** avoinea has joined #zope | 10:47 | |
*** eperez has joined #zope | 11:10 | |
*** espent has joined #zope | 11:12 | |
espent | hey, i got a very troublesome zope-installation here. Its a site we are taking over the hosting for. it runs good for a couple of hours, but then after a couple of hours it seems like its ending up in a endless loop... how can i profile the site to get down to where the problem si? | 11:14 |
---|---|---|
betabug | hmmm | 11:14 |
espent | si/is | 11:14 |
espent | its zope 2.9 | 11:14 |
betabug | what kind of application is installed on the site? | 11:14 |
espent | i dont know | 11:15 |
betabug | is there anything in the log (INSTANCE_HOME/logs/event.log) | 11:15 |
espent | sure, its logging good | 11:15 |
betabug | espent: ok, first thing, find where the INSTANCE_HOME is | 11:15 |
betabug | in there is a folder called "Products" | 11:15 |
espent | ok | 11:15 |
betabug | have a look what's in there | 11:15 |
betabug | check the event.log too | 11:15 |
espent | it loads of them | 11:16 |
betabug | put a listing on a pastebin? | 11:16 |
espent | http://paste.lisp.org/display/121617 | 11:17 |
espent | theres a ZopeProfiler there i can see | 11:17 |
betabug | yes | 11:17 |
espent | maybe that is causing it to run very slowly? | 11:17 |
betabug | well, if profiling is enabled, that might take some toll | 11:18 |
betabug | but it shouldn't cause a standstill | 11:18 |
espent | its not completely frozen when it "stops", there is things going on in event.log and Z2.log, but the varnish cache in front times out on the request | 11:18 |
espent | s | 11:18 |
betabug | at first glance it looks like you have an old version of this: http://ztmproject.org/ | 11:19 |
betabug | are there a lot of requests at those times? | 11:19 |
betabug | I've never heard about that ZTM stuff really | 11:19 |
espent | hm | 11:19 |
espent | i guess thats where the requests go yes... /topics | 11:20 |
betabug | the thing with zope is that depending on the setup of the instance, zope runs in a set number of threads | 11:20 |
espent | some stuff go fast at the beginning. LIke now, the front page for example, http://www.kildenett.no <-- thats the installation | 11:20 |
betabug | if there are long running requests they can tie up some of the threads for some time | 11:21 |
betabug | so at some point there might be no free threads any more | 11:21 |
espent | yes, i think I changed it from 4 to 10 threads | 11:21 |
betabug | hmmm, that doesn't help much | 11:21 |
espent | it did help a little, but not in the long run | 11:21 |
betabug | because behind the "number of threads" there is also the "number of DB connections" | 11:21 |
espent | yes, i adjusted the pool_size as well | 11:22 |
CIA-83 | janwijbrand * r121466 grok/ (src/grok/__init__.py CHANGES.txt): global_adapter was never imported from grokcore.component, while it is still quite useful | 11:22 |
betabug | and if there is a bug somewhere that is causing long running threads to get stuck, they will just fill up whatever threads you have | 11:22 |
betabug | ok, next thing to check: | 11:22 |
espent | it probably is a bug there, yes | 11:22 |
betabug | log into the ZMI and find the Control_Panel | 11:22 |
espent | iam in | 11:22 |
betabug | and in the Control_Panel, there is a link called "DebugInfo" | 11:23 |
espent | "Debug Information" | 11:23 |
betabug | yes | 11:23 |
espent | ok clicked | 11:23 |
betabug | on the bottom of that page there is a table "Connections:" | 11:23 |
betabug | which lists your threads | 11:23 |
espent | ah yeah, its like apache server-status | 11:23 |
betabug | showing what they are doing, how long they've been running etc | 11:23 |
espent | cool | 11:24 |
espent | ! | 11:24 |
betabug | maybe you can see there what is getting stuck | 11:24 |
espent | and when the thread is not doing anything it goes back to state 'None', right? | 11:25 |
betabug | yes, on "None" it's waiting for new incoming requests | 11:26 |
betabug | you want to look for stuff being listed with a long time in the left column | 11:26 |
betabug | and not going away | 11:26 |
espent | i just restarted, but i'll let it run for a couple of hours, and then visit this page again | 11:27 |
betabug | and then you can check from the URL what method it's trying to run and maybe find out what takes so long | 11:27 |
betabug | yes, exactly, that's the method | 11:27 |
espent | cool, thanks for good help! | 11:28 |
betabug | no problem, you're welcome | 11:28 |
*** menesis has joined #zope | 11:38 | |
*** Reite has joined #zope | 11:40 | |
*** cpf_ has joined #zope | 11:47 | |
*** sunew has joined #zope | 11:47 | |
*** benji has quit IRC | 11:56 | |
*** evilbungle has joined #zope | 11:58 | |
*** digitalmortician has joined #zope | 12:05 | |
*** benji has joined #zope | 12:05 | |
*** TomBlockley has joined #zope | 12:13 | |
*** __mac__1 has joined #zope | 12:31 | |
*** __mac__ has quit IRC | 12:31 | |
*** __mac__2 has joined #zope | 12:31 | |
CIA-83 | janjaapdriessen * r121467 /Sandbox/janjaapdriessen/buildbot/buildout.cfg: Add more js.* packages. | 12:32 |
CIA-83 | janwijbrand * r121468 grokcore.component/ (4 files in 3 dirs): fix global_adapter directive and grokker to allow for an explicit nameless registration even if the factory has a grok.name set | 12:32 |
CIA-83 | janwijbrand * r121469 grokcore.component/ (CHANGES.txt setup.py): Preparing release 2.4 | 12:32 |
CIA-83 | janwijbrand * r121470 /grokcore.component/tags/2.4: Tagging 2.4 | 12:32 |
CIA-83 | janwijbrand * r121471 grokcore.component/ (CHANGES.txt setup.py): Back to development: 2.5 | 12:32 |
CIA-83 | janwijbrand * r121472 groktoolkit/grok.cfg: update grokcore.component | 12:32 |
*** __mac__1 has quit IRC | 12:35 | |
*** __mac__2 has quit IRC | 12:58 | |
espent | betabug: there? | 13:20 |
betabug | yes, now :-) | 13:20 |
betabug | had a meeting | 13:21 |
espent | k | 13:21 |
betabug | got any stuck threads yet? | 13:21 |
espent | its in the lock now, but unfortunatly i am unable to access the admin interface. Is it possible to kill/hup one thread or something? | 13:21 |
betabug | hmmm, too bad | 13:21 |
betabug | not that I know of | 13:21 |
betabug | there is a product called "DeadLock Debugger" | 13:22 |
betabug | I've no personal experience with it, but maybe it could help | 13:22 |
betabug | personally, I'd restart it now and then run a cron job to write the debug info page to disk every 20-30 minutes | 13:22 |
espent | hm deadlock debugger looks good if it does what the documentation says it will | 13:24 |
espent | ill give it a try | 13:24 |
betabug | never had to use it, but give it a try | 13:26 |
betabug | and maybe you can write the debug info page to disk anyway :-) | 13:26 |
*** __mac__ has joined #zope | 13:34 | |
espent | the page requires that i am authenticated dont it | 13:37 |
betabug | yes | 13:39 |
betabug | another meeting :-) | 13:39 |
espent | have fun :) | 13:39 |
betabug | wasn't that bad in the end | 13:51 |
espent | doesnt look like deadlock debugger will accept my threadframe installation | 14:04 |
espent | the tests shipped with threadframe looks ok though | 14:05 |
kosh | betabug: greetings lifeform | 14:05 |
betabug | too bad | 14:05 |
betabug | hey kosh | 14:05 |
kosh | betabug: I have been making good progress on a better patch for OFS Image and File blob support by detecting if the current storage supports blobs | 14:05 |
betabug | kosh: cool! | 14:05 |
kosh | more threads does not make zope less likely to lockup, it can actually make it more likely | 14:07 |
betabug | well, I suspect something is eating up those threads, like a request getting stuck | 14:08 |
betabug | classical example is something waiting for an external resource | 14:08 |
betabug | but he has to find out somehow what it is | 14:08 |
kosh | yes but more threads does not make that better, just a little longer before failing | 14:08 |
betabug | yupp, it gives you a tiny bit more time till you have to hit "restart" :-) | 14:08 |
betabug | I have no experience with DeadlockDebugger, but it doesn't seem to work in this case | 14:09 |
kosh | I wonder how the zope devs would feel about images and files having versioned urls in the base, they are just making such a huge difference for my code that I would like others to use them also | 14:11 |
betabug | kosh: I think this would have to be handled as an add-on config | 14:11 |
betabug | other than that, I think they would "feel" about it a total indifference | 14:12 |
kosh | I have an add on product that can do it but not automatically for files and images | 14:12 |
kosh | and in my own product I built it in so that every file and image tht runs through my product does it automatically and it has made a large difference | 14:12 |
kosh | zope does set appropriate headers so that a browser will just do a GET and not have to download anything but if you have 40 images that is still 40 additional requests and you can make that 0 requests | 14:13 |
*** evilbungle has quit IRC | 14:16 | |
betabug | well, at least zope can make a conditional GET | 14:17 |
kosh | it does handle that correctly by default | 14:17 |
kosh | so you don't download the image again | 14:17 |
kosh | but not even trying to grab the image again is even better | 14:17 |
betabug | yes | 14:17 |
betabug | indeed | 14:17 |
betabug | plone already has that in some add-on product | 14:17 |
kosh | ah interesting | 14:18 |
*** mgedmin has joined #zope | 14:19 | |
kosh | with pyramid using more ztk stuff it might actually be possible to move a zope 2.x app to pyramid in another year or so if you write your app correctly :) | 14:19 |
betabug | haha | 14:19 |
betabug | well, this one app here was written a long time ago | 14:19 |
kosh | well it can use zeo already, it can load a zodb, the security policy is pretty compatible | 14:20 |
kosh | my stuff basically depends on my own product and exUserFolder and that is it | 14:20 |
*** evilbungle has joined #zope | 14:23 | |
kosh | betabug: well the more recent stuff I have been building is to move scripts out of zope and into a nice library I am building | 14:27 |
betabug | ok | 14:27 |
kosh | even simple things like I just made a method that will create a link entry for css by just handing it an object and an optional versioning object that way you don't have to actually write the <link etc part> | 14:28 |
kosh | it actually seems to be working very nicely in making stuff easier to get right | 14:28 |
kosh | so just com.html.link_css(cssobject) or com.html.link_css(cssobject, cache_version_object) that way it is easier to work with and fix | 14:29 |
kosh | doing that for doctype generation, generating jquery html, js stuff, iterating over a catalog safetly since getObject raises exceptions that you can't really catch inside ttw code etc | 14:30 |
kosh | which is helping to build stuff faster and easier to maintain | 14:30 |
kosh | I want to make as many things common as possible with a nice python interface | 14:31 |
*** m8 has joined #zope | 14:32 | |
kosh | that way when you move to something like html5 you can change the stuff in a common library and it would apply retroactively to every system you have | 14:32 |
*** yvl has joined #zope | 14:50 | |
*** TomBlockley_ has joined #zope | 14:50 | |
*** TomBlockley has quit IRC | 14:52 | |
*** TomBlockley_ is now known as TomBlockley | 14:52 | |
*** Wu has quit IRC | 14:54 | |
kosh | betabug: well so far my newer patch is working very nicely, it will enable to disable blob support based on if the current storage support it | 15:13 |
kosh | betabug: and it will work if you have multiple mounted storages and some support blobs and some don't since it checks if the storage you are currently using support blobs :) | 15:13 |
betabug | nice | 15:17 |
*** wosc has quit IRC | 15:17 | |
*** m8 has quit IRC | 15:30 | |
CIA-83 | hannosch * r121473 Products.ZCatalog/ (src/Products/ZCatalog/Catalog.py CHANGES.txt): Fix possible TypeError in `sortResults` method if only b_start but not b_size has been provided. | 15:36 |
*** ccomb has joined #zope | 15:43 | |
espent | betabug: looks like I got some clues here now. Several threads are using very long time, and all of the is related to ZTopic -> CMFBTreeFolder -> CMFSite | 15:56 |
espent | a couple of them have > 600s process time | 15:56 |
betabug | aha | 15:56 |
betabug | might very well be your culprit | 15:56 |
espent | and that is? | 15:56 |
betabug | check the methods in question | 15:57 |
betabug | I don't know this CMF stuff, not using it myself | 15:57 |
kosh | why would a BTreeFolder take more then 600s ? | 15:57 |
betabug | and I guess you've got some pretty old versions there | 15:57 |
espent | yes probably | 15:57 |
betabug | I guess the method in question is doing some kind of job | 15:57 |
*** benji is now known as Guest41434 | 15:58 | |
betabug | a BTreeFolder should be happy and quick with 10000s and 100'000s of objects in it | 15:58 |
*** benji___ has joined #zope | 15:58 | |
espent | yeah | 15:58 |
*** Guest41434 has quit IRC | 15:59 | |
*** benji___ has quit IRC | 15:59 | |
*** benji____ has joined #zope | 15:59 | |
*** __mac__ has quit IRC | 16:00 | |
*** benji____ is now known as benji | 16:01 | |
betabug | http://www.zope.org/Members/MikeP/Products/ZTopic | 16:01 |
betabug | oooold stuff | 16:01 |
espent | oh | 16:02 |
betabug | now it might be that there was a newer version, who knows | 16:02 |
espent | cant believe they managed to use this stuff in 2000, since its so slow | 16:03 |
betabug | hmmm, dunno | 16:03 |
kosh | BTreeFolder2 I know handles hundreds of thousands of objects just fine | 16:03 |
betabug | I guess it wasn't always that slow, but then something happened | 16:03 |
kosh | because I have some :)( | 16:03 |
kosh | hmm one too many characters there | 16:03 |
betabug | also Zope 2.9 isn't from 2000, it came much later - so there might be some stuff later than that ZTopic 1.0 | 16:04 |
espent | yeah sure, ZTopic used to be a Product? | 16:04 |
betabug | espent: what is the exact method / path that they hang on? | 16:04 |
betabug | yeah | 16:04 |
betabug | it should be in your instance's "Products" folder | 16:05 |
espent | i need to reproduce it | 16:06 |
espent | i didnt take copy of the debug info | 16:06 |
betabug | espent: does the Database file of this instance get "packed" from time to time? | 16:06 |
espent | yes, i've done it manually | 16:06 |
betabug | ok | 16:06 |
betabug | so, ZTopic basically makes "canned" Catalog searches | 16:07 |
betabug | (looking at the README file in the code here) | 16:07 |
betabug | Zope Software Archeology :-) | 16:07 |
espent | :) | 16:07 |
betabug | hmm, these ZTopics should do caching | 16:08 |
*** huajie has joined #zope | 16:08 | |
betabug | in fact, the description from the readme reads like it's something really useful :-) | 16:09 |
betabug | maybe something hits the URL that clears the cache? | 16:10 |
espent | could be | 16:10 |
espent | zope internal cache? | 16:10 |
betabug | well, ZTopic seems to have/use some cache | 16:10 |
betabug | haven't looked yet what it is | 16:11 |
betabug | looks like it puts it into an "intSet", never seen that before | 16:17 |
betabug | well, we'd really need to know the method(s) in question | 16:19 |
kosh | betabug: well I updated my patch, it is much nicer now :) | 16:35 |
kosh | https://bugs.launchpad.net/zope2/+bug/739692?comments=all is the bug report it is under | 16:36 |
_mup_ | Bug #739692: Add blob support to OFS.Image.Image and OFS.Image.File <Zope 2:Confirmed for tseaver> < https://launchpad.net/bugs/739692 > | 16:36 |
kosh | interesting the bot but that bug on the channel | 16:37 |
betabug | yah | 16:37 |
* kosh beats the bot with a stick :) | 16:37 | |
betabug | haha | 16:37 |
espent | betabug: k, thanks again. Need to look more at it tomorrow. Right now its locked again. | 16:46 |
betabug | oh my | 16:46 |
betabug | ok, good luck so far, I'll be here tomorrow | 16:46 |
espent | great | 16:46 |
*** sm has joined #zope | 16:55 | |
*** __mac__ has joined #zope | 16:56 | |
*** hever has joined #zope | 16:56 | |
*** alga has quit IRC | 16:58 | |
*** J1m has joined #zope | 17:03 | |
*** goschtl has quit IRC | 17:08 | |
*** Wu has joined #zope | 17:11 | |
*** hever has quit IRC | 17:12 | |
*** huajie has quit IRC | 17:13 | |
*** Arfrever has joined #zope | 17:20 | |
*** TomBlockley has quit IRC | 17:21 | |
*** TomBlockley has joined #zope | 17:21 | |
*** zagy has quit IRC | 17:35 | |
*** hever has joined #zope | 17:39 | |
*** horsey has joined #zope | 17:40 | |
horsey | Hi, has anybody successfully installed zope.interface on a 64 bit Windows machine? | 17:48 |
betabug | horsey: apparently there are buildbots that play through all the zope install for testing | 17:48 |
waldi | what should be the problem? okay, windows is a problem on its own but can't be _this_ broken | 17:48 |
betabug | so it should work | 17:48 |
*** TomBlockley_ has joined #zope | 17:50 | |
*** evilbungle has quit IRC | 17:51 | |
*** evilbungle has joined #zope | 17:51 | |
horsey | waldi: When I run 'python setup.py install' on the source directory, I get an error "raise ValueError(str(list(result.keys())) ValueError: [u'path'] | 17:51 |
*** evilbungle_ has joined #zope | 17:51 | |
horsey | Not sure which path it is. | 17:52 |
*** TomBlockley has quit IRC | 17:53 | |
*** TomBlockley_ is now known as TomBlockley | 17:53 | |
horsey | I cannot install it with easy_install, as setuptools does not provide a 64 bit installer for Windows. :( | 17:54 |
*** evilbungle has quit IRC | 17:55 | |
*** evilbungle_ is now known as evilbungle | 17:55 | |
*** sunew has quit IRC | 17:56 | |
*** bigkevmcd has quit IRC | 17:57 | |
*** bigkevmcd has joined #zope | 17:57 | |
*** digitalmortician has quit IRC | 18:02 | |
*** alvaro_o has joined #zope | 18:15 | |
*** alvaro_o has quit IRC | 18:15 | |
*** __mac__ has quit IRC | 18:42 | |
*** agroszer has quit IRC | 18:50 | |
*** vortec has left #zope | 18:50 | |
*** vortec has joined #zope | 18:52 | |
*** robohak has joined #zope | 18:56 | |
*** vortec has left #zope | 19:03 | |
*** tiwula has joined #zope | 19:14 | |
*** m8 has joined #zope | 19:26 | |
*** digitalmortician has joined #zope | 19:29 | |
*** jshell has joined #zope | 19:34 | |
*** Spanktar has joined #zope | 19:42 | |
*** m8 has quit IRC | 19:48 | |
*** __mac__ has joined #zope | 19:49 | |
*** __mac__ has quit IRC | 19:53 | |
*** hever has quit IRC | 19:59 | |
*** TomBlockley has quit IRC | 20:04 | |
*** runyaga has joined #zope | 20:19 | |
*** runyaga has quit IRC | 20:19 | |
*** runyaga has joined #zope | 20:19 | |
*** srichter has joined #zope | 20:20 | |
*** eperez has quit IRC | 20:22 | |
*** goschtl has joined #zope | 20:29 | |
*** evilbungle has quit IRC | 20:34 | |
*** avoinea has left #zope | 20:40 | |
*** goschtl has quit IRC | 20:51 | |
*** Wu has quit IRC | 20:54 | |
*** webmaven has joined #zope | 20:54 | |
*** zagy has joined #zope | 20:59 | |
*** digitalmortician has quit IRC | 21:10 | |
*** zagy has quit IRC | 21:14 | |
*** zagy has joined #zope | 21:14 | |
*** zagy has quit IRC | 21:15 | |
*** goschtl has joined #zope | 21:15 | |
*** zagy has joined #zope | 21:18 | |
*** hever has joined #zope | 21:19 | |
*** goschtl has quit IRC | 21:31 | |
*** ccomb has quit IRC | 21:36 | |
*** hever has quit IRC | 21:39 | |
*** m8 has joined #zope | 21:44 | |
*** alecm has quit IRC | 21:58 | |
*** zagy1 has joined #zope | 22:01 | |
*** zagy has quit IRC | 22:01 | |
*** kleist has quit IRC | 22:05 | |
*** zagy has joined #zope | 22:11 | |
*** zagy1 has quit IRC | 22:11 | |
*** evilbungle has joined #zope | 22:12 | |
*** supton has joined #zope | 22:13 | |
*** m8 has quit IRC | 22:23 | |
*** goschtl has joined #zope | 22:38 | |
*** goschtl has quit IRC | 22:40 | |
*** goschtl has joined #zope | 22:46 | |
*** goschtl has quit IRC | 22:47 | |
*** menesis has quit IRC | 22:50 | |
*** srichter has quit IRC | 23:01 | |
*** zagy has quit IRC | 23:11 | |
*** MrTango has quit IRC | 23:12 | |
*** ccomb has joined #zope | 23:57 |
Generated by irclog2html.py 2.15.1 by Marius Gedminas - find it at mg.pov.lt!